Format precision apparently ignored when using read from spreadsheet file

[Canucklehead]

I hope there is ridiculously simple solution to this problem but so far I can't find it. Using the Read from Spreadsheet File function in LabVIEW7.1, I can't get the floating point format (%f) to work; the decimal (%d) seems to work fine. For example, the first value in the attached file is 399.5853. When I read it in using %.2f format I would think I should get 399.58 but instead I get 399.58529663... Oddly if I use %d as the format, I get an as expected value of 399. Can anyone see what I am missing?

[alkazaa]

The solution to the problem is the internal representation of floating point numbers: with a limited number of bytes only a limited number of the (infinitely many) real numbers can be represented. The nearest representable number to 399.5853 is 399.58529663 if the so-called single precision representation (abbr. SGL in Labview) is used.

It seems that (ridiculously!) NI has chosen to use SGL as the data format in the Read from Spreadsheet VI which causes the unexpected behaviour you observed.

You can change the representation from SGL (which uses 4 bytes) to DBL (double precision, using 8 bytes) in the Read from Spreadsheet VI and you will observe a better approximation then, but still not 'exact' since the number now reads 399.585300000000018. (With t
he EXT representation, 10 bytes, you could go to even higher precision)

I hope someone from NI reads this and they fix the unnecessary limitation of the precision in the Read from Spreadsheet VI.