LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Interpreting values supposedly in fixed-point format



DonRoth wrote:
I had the same problem with the bringing in the right number of data points using your I32 methodology. 
THe second SPC file is corrupt. Can you see if you can get a new copy?
 
My method works fine on the original datafile.
 
For the second datafile, you can read the upper data portion if you assume that the data starts a  543 instead of 544.
(If you do this, the early data makes no sense of course)
This simply means that one byte got lost 2/3 into the data portion. The file is corrupt!
 
See image.





Message Edited by altenbach on 05-27-2008 02:07 PM

LabVIEW Champion. It all comes together in GCentral GCentral
Message 21 of 34
(601 Views)
If you insert a byte (x00) at file position 15640, the data is no longer corrupt and the data is identical to the xls file (well within 0.001) . See attached image:
 
 


Message Edited by altenbach on 05-27-2008 02:50 PM

LabVIEW Champion. It all comes together in GCentral GCentral
Message 22 of 34
(590 Views)
The Thermoscientific GRAMS viewer must be able to figure this all out in some fashion because it brought in the corrupt file just fine without any modification.  When I use Hex Workshop, it appears to me that the data starts at 544.  Let me attach a screen shot (you guys have to tell me again how to insert picture in message if the picture is not on another server - I think there is a trick, right?)....Don


Message Edited by DonRoth on 05-28-2008 06:56 AM
0 Kudos
Message 23 of 34
(579 Views)
Here is a screenshot from the Viewer on the 11SN1.spc file.  This is a little puzzling now to me.  Thermoscientific also has a DLL component that can be integrated into LabVIEW.  I'll have to test it with this also.  I wanted to do everything in straight LabVIEW since it is always easier for future debugging / modification, etc. - but I will look at this DLL component method and provide status.


Message Edited by DonRoth on 05-28-2008 07:05 AM
0 Kudos
Message 24 of 34
(572 Views)


DonRoth wrote:
When I use Hex Workshop, it appears to me that the data starts at 544. 
Of course it starts at 544. Setting it to 543 is just a quick&dirty way to induce a frameshift so we can see from what point on the data looks "normal" if we assume there is a missing bit somewhere. If we now insert a byte right at that point, all the data will look normal when read starting at 544.
 
Is the file corrupt in your case too when you try to read with LabVIEW? Maybe it's only my copy?
 
Since the header has significantly more information than what we currently read out, maybe it contains information on the extra editing needed.


DonRoth wrote:
Let me attach a screen shot (you guys have to tell me again how to insert picture in message if the picture is not on another server - I think there is a trick, right?)....Don

After you atttach the image, you can right-click the link and do a "copy shortcut". Now edit the post (within 10 minutes of posting!) and insert them image by clicking on the "Insert an Image" button and pasting the link to the attached image.

LabVIEW Champion. It all comes together in GCentral GCentral
Message 25 of 34
(550 Views)
My reading of the file in LabVIEW gives the same result as yours.  So the header must provide some information for this that the GRAMS viewer interprets and understands correctly to make the spectra appear properly, or the file is corrupt and the viewer is smart enough to repair it.  Here is another file that the researcher sent me that is brought in fine from the LabVIEW code.  I am going to do a sampling from a whole directory to see if this is a common or rare problem. 

On your code regarding reading the data stream (don't worry about the header), can you explain this small portion to me in greater detail so I understand why you did what you did?  (Was some of this trial and error?). (Don't worry about coercion dot, I fixed this.)






Message Edited by DonRoth on 05-28-2008 11:41 AM
0 Kudos
Message 26 of 34
(534 Views)


DonRoth wrote:
On your code regarding reading the data stream (don't worry about the header), can you explain this small portion to me in greater detail so
I understand why you did what you did? 

Well, we need to read the data as little endian I32, but this will need to be divided by 2^32 and multiplied by 2^exponent. This is the same as multiplying by 2^(exponent-32).
 
Here you need to be careful to make sure that you do this in a signed integer datatype. If you would use unsigned integers, the subtraction would lead to a huge number (because it is actually negative) and would result in a multiplier of Inf.
 
The upper left corner reads the exponent and subtracts 32, then calculates 1 * 2^(results if subtraction). This is the final multiplier for the data.
 
In the lower part we take the string subset starting at position 544. We know how many I32 points we need to read from the header, so we need to read 4x more bytes.
Using the x2^n node is more efficient whenever we need to multiply a blue wire with an integer power of two because computationally it is just a bit shift and no fancy
multiply code is needed. You can equally well just multiply with 4. Same difference. 🙂
Now we need to unflatten the string segment containing the data as as I32 array. The type is an empty I32 array. We select little endian, because that's the byte order
of the datafile, and we need to wire a false because the first element does NOT contain information about the array size, but the first element.
 
Older versions of LabVIEW don't support the byte order input, so you would need to manually swap the bytes. This would require a bit more code but is quite trivial..

LabVIEW Champion. It all comes together in GCentral GCentral
Message 27 of 34
(523 Views)
Thanks for your detailed explanation.  The things I got stuck on myself when interpreting the data stream were:

1) what was the missing piece of information that would allow the conversion to the floatY?  You have uncovered it here with the subtraction of 32.  I still don't know why you did that.  I understand the 2^exponent portion.  I don't understand the 2^32 factor.  Why do we do this?  Is there something in your experience of file decoding that led you to figure this out?

2) I got hung up on the fixed-point format that the developer talked about and tried to figure out a way in LabVIEW using LabVIEW fixed point format (which I think can still be done by configuring the fixed point as I32).  The developer hints at things when he says the fixed point format is similar to integer format so that one would use I32 format for the unflatten operation.  It seems you just said "I am going to bring in as I32" and did not worry about the details of his fixed-point format explanation.



I never would have figured this out based on the developer's explanation (and basically, if you are someone who creates an SDK as this developer did, and you are actively opening things up for conversion, you should do a lot better job of explaining things than this person did.) 

ps. Did anyone ever tell you that you are a brilliant guy?
0 Kudos
Message 28 of 34
(508 Views)


DonRoth wrote:
1) what was the missing piece of information that would allow the conversion to the floatY?  You have uncovered it here with the subtraction of 32.  I still don't know why you did that.  I understand the 2^exponent portion.  I don't understand the 2^32 factor.  Why do we do this?  Is there something in your experience of file decoding that led you to figure this out?

It is in the specifications you posted. If we read it as I32, we cannot get any fractional numbers, but according to your earlier posts, there is an implicit decimal point. Basically, all values are 2^32 too big before we multiply by the factor.
Depending on where the decimal point is, the I32 needs to be scaled accordingly. To be able to scale with factors <1, we need to convert the I32 to DBL in order to support the resulting non-integer values.
The location of the implicit decimal point is given by the exponent in the header, which basically is just the position of the decimal point when counting from the highest bit. Each factor of two shifts the implicit decimal point to the right in the binary data.
 
I don't have much experience with FXP data type, but one problem I see is the fact the the FXP type will depend on the exponent. I am not sure how to handle that. Using DBL seems fine, because the mantissa size is plenty to support the 32bits of the original data and most LabVIEW analysis tools wand DBL anyway.


Message Edited by altenbach on 05-28-2008 10:59 AM

LabVIEW Champion. It all comes together in GCentral GCentral
0 Kudos
Message 29 of 34
(503 Views)
So since the exponent is 20, and 20 - 32 = -12, your fractionY = 2^-12 and this is what you multiplied with the extracted I32 data value to scale it and to give you the FloatY....I think I see.

Thanks for all your time and effort on this.  I don't know where I would be without some of the help I have gotten here (and many times from you).

Sincerely,

Don
0 Kudos
Message 30 of 34
(496 Views)