300*0.0001=?

[Michael Ludwig]

I have encountered the strangest error in my LV career:

in LV 7.1 and 8.0 under XP and Win7 on Intel and AMD chips:

300*0.0001 != 0.03

this is the case for 3, 6, 9, 18 ... 300, 600, ... but not 30, 60, 3000 or 6000. The numbers in question seem to be most multiples of 3 that are not also multiples of 5. It looks like LV calculated the right number in an indicator, but setting it equal to a constant 0.03 will give you FALSE.

this is the case both for the multiplication vi as well as the expression node; however, the error does not reproduce if you divide by 10000 instead of multiplying by 0.0001. The problem is solved if you multiply the result by 1000, round and divide back by 1000.

please see the vi (LV 7.1) attached below as well as the jpeg.

question: can you reproduce this? on which versions of LV / OS?

[Dennis_Knutson]

It's not at all incredible and the problem with the representation of floating points numbers on computers has been discussed an almost endless amount of times. This issue is common to all programming languages. Never, ever do an equals comparison on floats.

[Matt_W1 · Accepted solution]

It'll happen on any language on any computer that uses IEEE 754 floating point (which is virtually all of them). This is due to the nature of how floating point works. In general you can't trust equality checks between floating point numbers, usually you check that the number is close enough for your problem.

[Intaris]

This isn't surprising at all.

It's due to inherent inaccuricies of the IEEE 754 floating-point representation supported in basically every programming language in the world.  It is not a LV problem and it is not a bug.

It is expected behaviour.

See HERE  or HERE for example.

Shane.

[Michael Ludwig]

ok, i guess i kinda jumped the gun. it took a lot of work to locate this bug in our code, as you might well imagine, and i got a bit excited. I guess i'll just blow everything up into very large integers before i compare.

[tbob]

Use In Range function.  Set up reasonable min and max values.  To compare to 0.03, set max to 0.031 and min to 0.029.  As you can see in the picture below, the number 0.03 is not exactly represented as 0.03.  But the In Range will make a good comparison.

[Michael Ludwig]

the code that was giving me trouble was actually searching an array of floats for a float match, so the "in range" function wouldn't have worked. The solution i found was to multiply all the floats by 1,000,000 and then turn them into integers before searching.

[GerdW]

Hi Michael,

you should have learned to never compare floats for (in)equality!

[Intaris]

He did learn that.

We just told him.

+1 for the Forums!

Shane

[Michael Ludwig]

Thanks Shane,

yes, now i am a more learned person. i will never trust a float again. and the only good float is an integer, etc. 🙂