LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

300*0.0001=?

Solved!
Go to solution

I have encountered the strangest error in my LV career:

 

in LV 7.1 and 8.0 under XP and Win7 on Intel and AMD chips:

 

300*0.0001 != 0.03

 

this is the case for 3, 6, 9, 18 ... 300, 600, ... but not 30, 60, 3000 or 6000. The numbers in question seem to be most multiples of 3 that are not also multiples of 5. It looks like LV calculated the right number in an indicator, but setting it equal to a constant 0.03 will give you FALSE.

 

this is the case both for the multiplication vi as well as the expression node; however, the error does not reproduce if you divide by 10000 instead of multiplying by 0.0001. The problem is solved if you multiply the result by 1000, round and divide back by 1000.

 

please see the vi (LV 7.1) attached below as well as the jpeg.

 

question: can you reproduce this? on which versions of LV / OS?

Download All
0 Kudos
Message 1 of 11
(6,035 Views)

It's not at all incredible and the problem with the representation of floating points numbers on computers has been discussed an almost endless amount of times. This issue is common to all programming languages. Never, ever do an equals comparison on floats.

Message 2 of 11
(6,025 Views)
Solution
Accepted by topic author Michael Ludwig

It'll happen on any language on any computer that uses IEEE 754 floating point (which is virtually all of them). This is due to the nature of how floating point works. In general you can't trust equality checks between floating point numbers, usually you check that the number is close enough for your problem.

0 Kudos
Message 3 of 11
(6,013 Views)

This isn't surprising at all.

 

It's due to inherent inaccuricies of the IEEE 754 floating-point representation supported in basically every programming language in the world.  It is not a LV problem and it is not a bug.

 

It is expected behaviour.

 

See HERE  or HERE for example.

 

Shane.

0 Kudos
Message 4 of 11
(6,008 Views)

ok, i guess i kinda jumped the gun. it took a lot of work to locate this bug in our code, as you might well imagine, and i got a bit excited. I guess i'll just blow everything up into very large integers before i compare.

0 Kudos
Message 5 of 11
(5,988 Views)

Use In Range function.  Set up reasonable min and max values.  To compare to 0.03, set max to 0.031 and min to 0.029.  As you can see in the picture below, the number 0.03 is not exactly represented as 0.03.  But the In Range will make a good comparison.

 

23550i3087DEA63833300F

- tbob

Inventor of the WORM Global
0 Kudos
Message 6 of 11
(5,964 Views)

the code that was giving me trouble was actually searching an array of floats for a float match, so the "in range" function wouldn't have worked. The solution i found was to multiply all the floats by 1,000,000 and then turn them into integers before searching.

0 Kudos
Message 7 of 11
(5,946 Views)

Hi Michael,

 

you should have learned to never compare floats for (in)equality!

Best regards,
GerdW


using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 8 of 11
(5,912 Views)

He did learn that.

 

We just told him. Smiley Happy

 

+1 for the Forums!

 

Shane

0 Kudos
Message 9 of 11
(5,907 Views)

Thanks Shane,

 

yes, now i am a more learned person. i will never trust a float again. and the only good float is an integer, etc. 🙂

0 Kudos
Message 10 of 11
(5,867 Views)