03-07-2007 03:36 PM
03-07-2007 03:53 PM
There is no real inaccuracy. The difference between 0.94999999 and 0.95 is 0.00000001 or about 1 in 100000000! (This is just the diplayed part, the difference in real life is even less. Is your number really more precise that 52bits? (or 23bits for SGL).
@ejoseph wrote:
Are decimal and numeric types not supported by labview, and if so how do I get around the inaccuracy of float datatypes?
03-07-2007 03:58 PM
03-07-2007 04:13 PM
03-07-2007 04:17 PM
03-07-2007 04:31 PM
03-07-2007 04:38 PM
03-16-2007 04:57 PM
"When I say it is "blank", it is not viewable in a variant indicator but does indeed hold valuable data."
So what this means is that you can't leave it as a variant indicator. You need to use the Variant to Data.vi in order to see the data.