Showing results for 
Search instead for 
Did you mean: 


Go to solution

I have encountered the strangest error in my LV career:


in LV 7.1 and 8.0 under XP and Win7 on Intel and AMD chips:


300*0.0001 != 0.03


this is the case for 3, 6, 9, 18 ... 300, 600, ... but not 30, 60, 3000 or 6000. The numbers in question seem to be most multiples of 3 that are not also multiples of 5. It looks like LV calculated the right number in an indicator, but setting it equal to a constant 0.03 will give you FALSE.


this is the case both for the multiplication vi as well as the expression node; however, the error does not reproduce if you divide by 10000 instead of multiplying by 0.0001. The problem is solved if you multiply the result by 1000, round and divide back by 1000.


please see the vi (LV 7.1) attached below as well as the jpeg.


question: can you reproduce this? on which versions of LV / OS?

Download All
0 Kudos
Message 1 of 11

It's not at all incredible and the problem with the representation of floating points numbers on computers has been discussed an almost endless amount of times. This issue is common to all programming languages. Never, ever do an equals comparison on floats.

Message 2 of 11
Accepted by topic author Michael Ludwig

It'll happen on any language on any computer that uses IEEE 754 floating point (which is virtually all of them). This is due to the nature of how floating point works. In general you can't trust equality checks between floating point numbers, usually you check that the number is close enough for your problem.

0 Kudos
Message 3 of 11

This isn't surprising at all.


It's due to inherent inaccuricies of the IEEE 754 floating-point representation supported in basically every programming language in the world.  It is not a LV problem and it is not a bug.


It is expected behaviour.


See HERE  or HERE for example.



0 Kudos
Message 4 of 11

ok, i guess i kinda jumped the gun. it took a lot of work to locate this bug in our code, as you might well imagine, and i got a bit excited. I guess i'll just blow everything up into very large integers before i compare.

0 Kudos
Message 5 of 11

Use In Range function.  Set up reasonable min and max values.  To compare to 0.03, set max to 0.031 and min to 0.029.  As you can see in the picture below, the number 0.03 is not exactly represented as 0.03.  But the In Range will make a good comparison.



- tbob

Inventor of the WORM Global
0 Kudos
Message 6 of 11

the code that was giving me trouble was actually searching an array of floats for a float match, so the "in range" function wouldn't have worked. The solution i found was to multiply all the floats by 1,000,000 and then turn them into integers before searching.

0 Kudos
Message 7 of 11

Hi Michael,


you should have learned to never compare floats for (in)equality!

Best regards,

using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
0 Kudos
Message 8 of 11

He did learn that.


We just told him. Smiley Happy


+1 for the Forums!



0 Kudos
Message 9 of 11

Thanks Shane,


yes, now i am a more learned person. i will never trust a float again. and the only good float is an integer, etc. 🙂

0 Kudos
Message 10 of 11