Not reading the right data value

[caunchman]

So I am reading data from a txt file that I created,in particular I am trying to pull out the first column which is the time data.  When I read it using the "Read from spreadsheet file.vi" I am getting unexpected values. For example when the first value in the file is 1491.044016 it displays 1491.044067, it does this for every value? I thought it may have been a formating issue but I set the vi %.9f on the formating field I'm not sure why it is handling the values this way? Any insights?

thanks,

Chris

[altenbach]

Are you using LabVIEW 7.1 or older? In this case, "read from spreadsheet file" uses only SGL precision, which is not sufficient for your desired resolution.

See also: http://forums.ni.com/ni/board/message?board.id=170&view=by_date_ascending&message.id=315189#M315189

Solution: Read the file as a plain string and then use "spreadsheet string to array" with DBL as type.

... or upgrade to LabVIEW 8.0 or higher. 🙂