Expecting a result of zero when subtracting, but getting 138.778E-18

[MiBelli]

I am getting unexpected values in a subtraction routine. I am using LabView 2013.

In my Sample code VI I have several condition blocks to sequentially decrement 7 banks of values. I am getting an unexpected result when executing my code with the following conditions:

Bank 1 = 1.0

Bank 2 to Bank 7 = 100

BankTTL = 601.00

Maneuvering Rate = 1, making the Rate(%FP/s) = 0.10

I execute (Run) the code 10 times expecting to decrement Bank 1 by 0.10 each time.

When I run the code the 11th time, Bank 1 should remain at zero and Bank 2 should start decrementing.

However, this is not the case...  Bank 1 becomes -.01.  This should not happen.

In tracing the code, putting a probe on the input to the Bank 1 indicator, the value returned after 10 executions in the "Probe Watch Window" is 138.778E-18. This is unexpected. While the value shown in the front panel is 0.00. 

When I start my logic with the following values:

Bank 1 = 0.1

Bank 2 to Bank 7 = 100

BankTTL = 600.10

Maneuvering Rate = 1, making the Rate(%FP/s) = 0.10

I get the expected result of zero in both the "Probe Watch Window" and the Front Panel for indicator Bank 1.

Your help in resolving the unexpected result would be greatly appreciated.

Thank you.

Lixuan

[Dennis_Knutson]

It is completely expected. You are working with floating point numbers that don't have an exact representation in binary.

Your VI is a mess with all of those local variables. The code could be rewritten to use none.

[JKSH · Accepted solution]

SGL/DBL types store numbers in binary format. Powers of 2 can be represented precisely (..., 0.125, 0.25, 0.5, 1, 2, 4, 8, ...), but most other numbers cannot.

100, 0.1 and 601 are not powers of 2, so these values cannot be stored precisely and rounding errors occur. See: "What causes floating point rounding errors?"

Note that 138.778E-18 is 0.000000000000000138778, which is very close to zero considering that your starting value is ~600.

[MiBelli]

Dennis:  I am new to the LabView Environment. What type would you suggest for for floating point values? Would you be able to provide alternative examples for replacing the local variables?

Thank you.

[MiBelli]

Since I require floating type precision up to 6 decimal places, I will try multiplying my Bank and decrement values by 10e+6 and using DBL to avoid the rounding errors of floats, then divide by 10e+6 to represent the final value.

[Dennis_Knutson]

Why don't you simply change the display properties of the indicator? Don't make it any more complicated.

[MiBelli]

Changing the display properties does not fix the error.  I need the the precision.  This is only half of the system I am trying to represent.  I also need to increment the Banks. If I do not solve the error, the representation of my model will get worse. 

[Dennis_Knutson]

I don't what model you are referring to. I don't know what these 'Banks' are, either. You have greater than 6 decimal places of precision.

[MiBelli]

Thank you Dennis and JKSH. 

JKSH pointed me in the right directions.  I modified my code by multiplying my Bank and decrement values by 10e+6 and using DBL to avoid the rounding errors of floats, then divide by 10e+6 to represent the final value. It is ugly... 

However, decrementing the values I want to decrement by does get me exactly to zero without any errors.

I have not gotten rid of the local variables as you suggested Dennis.  As I learn, I will figure out how to make my code cleaner. 

FYI Dennis... the Banks represent the position of adjuster rods for a nuclear reactor.

Thank you.

[MiBelli]

Here is my final snippet of code.