21 > 21 ?

[MarkJonesATX]

I'm having a program increment until it reaches or passes a value. I noticed it would increment once too many times. Used Highlight Execution and noticed that my Greater? was outputting true with two 21s. Both are doubles (64-bit precision). 

Incrementing by 0.25 leads 21=21, but when incrementing by 0.2, they are not considered equal and it will stop at 21.2. 

When incrementing to a height of 0.4 or lower, it will stop at the correct increment, but with a height of 0.6 and onwards and extra increment is always included. It stops correctly with 0.1 and 0.25 increments leaves me very confused. 

I attached the VI so y'all can try it out. I'm new to LabVIEW so let me know if there's a more efficient way to do something you see in my VI.

[RavensFan · Accepted solution]

Expand your indicators to show 15, 16 decimal places.

I bet you are running into a problem with floating point numbers on computers where a number can not be exactly what you think it is.  A number that is obtained by math on other numbers, may not be exactly equal to what you think.

http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm

[Kyle97330 · Accepted solution]

The short version is that any number with a fractional part that isn't exactly some number divided by a power of two can't be represented exactly by a PC.   So 0.25 works (1/4th) as does 0.5, and some that seem strange will work fine like 0.2890625‬ (37/128), but 0.1 and 0.4 and such won't work.

A common workaround is to either consider numbers equal if they are within some tolerance amount (0.000000001 or less, perhaps).  Another common one is to multiply your value by a few powers of 10 and then express it as integers.  So instead of going to 21, go to 21000, and always increment by whole numbers.

[MarkJonesATX]

Thanks guys! Its working now. Definitely something I need to keep in mind.