# LabVIEW

cancel
Showing results for
Did you mean:

Solved!
Go to solution

## 300*0.0001=?

I have encountered the strangest error in my LV career:

in LV 7.1 and 8.0 under XP and Win7 on Intel and AMD chips:

300*0.0001 != 0.03

this is the case for 3, 6, 9, 18 ... 300, 600, ... but not 30, 60, 3000 or 6000. The numbers in question seem to be most multiples of 3 that are not also multiples of 5. It looks like LV calculated the right number in an indicator, but setting it equal to a constant 0.03 will give you FALSE.

this is the case both for the multiplication vi as well as the expression node; however, the error does not reproduce if you divide by 10000 instead of multiplying by 0.0001. The problem is solved if you multiply the result by 1000, round and divide back by 1000.

please see the vi (LV 7.1) attached below as well as the jpeg.

question: can you reproduce this? on which versions of LV / OS?

Message 1 of 11
(4,942 Views)

## Re: 300*0.0001=?

It's not at all incredible and the problem with the representation of floating points numbers on computers has been discussed an almost endless amount of times. This issue is common to all programming languages. Never, ever do an equals comparison on floats.

Message 2 of 11
(4,932 Views)
Solution
Accepted by Michael Ludwig

## Re: 300*0.0001=?

It'll happen on any language on any computer that uses IEEE 754 floating point (which is virtually all of them). This is due to the nature of how floating point works. In general you can't trust equality checks between floating point numbers, usually you check that the number is close enough for your problem.

Message 3 of 11
(4,920 Views)

## Re: 300*0.0001=?

This isn't surprising at all.

It's due to inherent inaccuricies of the IEEE 754 floating-point representation supported in basically every programming language in the world.  It is not a LV problem and it is not a bug.

It is expected behaviour.

See HERE  or HERE for example.

Shane.

Message 4 of 11
(4,915 Views)

## Re: 300*0.0001=?

ok, i guess i kinda jumped the gun. it took a lot of work to locate this bug in our code, as you might well imagine, and i got a bit excited. I guess i'll just blow everything up into very large integers before i compare.

Message 5 of 11
(4,895 Views)

## Re: 300*0.0001=?

Use In Range function.  Set up reasonable min and max values.  To compare to 0.03, set max to 0.031 and min to 0.029.  As you can see in the picture below, the number 0.03 is not exactly represented as 0.03.  But the In Range will make a good comparison.

- tbob

Inventor of the WORM Global
Message 6 of 11
(4,871 Views)

## Re: 300*0.0001=?

the code that was giving me trouble was actually searching an array of floats for a float match, so the "in range" function wouldn't have worked. The solution i found was to multiply all the floats by 1,000,000 and then turn them into integers before searching.

Message 7 of 11
(4,853 Views)

## Re: 300*0.0001=?

Hi Michael,

you should have learned to never compare floats for (in)equality!

Best regards,
GerdW

using LV2016/2019/2021 on Win10/11+cRIO, TestStand2016/2019
Message 8 of 11
(4,819 Views)

## Re: 300*0.0001=?

He did learn that.

We just told him.

+1 for the Forums!

Shane

Message 9 of 11
(4,814 Views)

## Re: 300*0.0001=?

Thanks Shane,

yes, now i am a more learned person. i will never trust a float again. and the only good float is an integer, etc. 🙂

Message 10 of 11
(4,774 Views)