LabVIEW Math Bug

[10Things_Rob]

I found a bug report from 2004 that sounds similar. This is in LabVIEW 8.0.1

Add 0.05 to zero 11 times.

Apparantly, it does NOT equal 0.55

See enclosed.

[altenbach]

This is not a math bug, but is caused by the limitations of floating point numbers. Many fractions that are nice and round in decimal, cannot be represented exactly in binary.

Comparing floating point numbers is thus dangerous. The attached image shows the same with more decimal digits shown. As you can see, the result is nearly infinitely close to 0.55. It cannot be accurate.

This is not a bug!

Message Edited by altenbach on 09-12-2006 02:54 PM

[altenbach]

Typically, you could look at the difference and see if it is below a certain threshold to decide if they are "equal enough".

Attached is another possible way as discussed elsewhere (http://forums.ni.com/ni/board/message?board.id=170&message.id=158227#M158227).

( For details, here's a link for some easy bedtime reading: http://www.cygnus-software.com/papers/comparingfloats/comparingfloats.htm)

Good luck! 😄

Message Edited by altenbach on 09-12-2006 03:20 PM

[10Things_Rob]

I thought that might be the reason, but I guess that I did not check enough digits down

thanks