LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Strange Rounding Issue when using built-in reciprocal function

I can't seem to figure out why this is happening. Code attached. In the actual code I use, the 1/25 and the 62 come from global variables, if that helps. The 25 is input via a control as 25, and then the reciprocal is applied before storing in the global. All data types are standard Double Precision Floats.

0 Kudos
Message 1 of 8
(3,071 Views)

Your result is actually 14.879999999999999000 (show sufficient significant digits!)

 

Computers operate in binary and amy nice decimal fractions cannt be expressed as a nice fraction in base 2. This is normal and there are many existing discussions about that.

 

Your comparison with the machine epsilon is also flawed.

0 Kudos
Message 2 of 8
(3,066 Views)

ALso note that the order of operation matters. If you would wire the 1/25 at the bottom of the compond node, the difference is zero.

0 Kudos
Message 3 of 8
(3,053 Views)

Here is a picture of what altenbach said.

 

Lynn

 

funky.png

0 Kudos
Message 4 of 8
(3,051 Views)
Altenbach, I didn't know that about the order of operations! I figured it was an FP error, but I couldn't see where. If I were to round the 1/25 to thousandths or ten thousandths would that allow the order to not matter?
0 Kudos
Message 5 of 8
(3,041 Views)

The result is random and you cannot guarantee that any specific order produces a "better" result than any other order. The computer does not know what result you want. 😉

0 Kudos
Message 6 of 8
(3,033 Views)
So what can I do to fix this? Round after the reciprocal?
0 Kudos
Message 7 of 8
(3,027 Views)

1/25 cannot be represented exactly in binary. So any calculation using that value will give a result different from the exact mathematical value. Attempting to rely on specific round off behavior to get what you want is likely to result in frustration.

 

Look at the precision of the values and the accuracy of the sources (if measured data). Consider noise in any measurements. Then assign an appropriate tolerance to the comparison. That way you control the decison making process to be consistent with your data and what you are doing with it.

 

Lynn

0 Kudos
Message 8 of 8
(3,023 Views)