From 12:00 AM CDT Sunday, October 17 - 11:30 AM CDT Sunday, October 17, will be undergoing system upgrades that may result in temporary service interruption.

We appreciate your patience as we improve our online experience.


Showing results for 
Search instead for 
Did you mean: 

Why LabVIEW calculates wrong?

Hi everyone,


I have two different data which I'm doing very simple calculation with them; substraction.


First value is          1,640220000000000e+18

Second value is     -1,640217348851742e+18


Obviously, these cannot be estimated as equal, the difference between them is 2651148258304. But LabVIEW calculates it as zero.


Why this error happens?





0 Kudos
Message 1 of 7

Please post a VI that demonstrates this.


What version of LabVIEW are you using?  What is the representation of your numbers (single, double, extended)?


Is the fact you are using a comma instead of a decimal point causing you a problem?


And the difference between them is actually about 3.28e18 not 2651148258304.


Also, remember that many floating point numbers can't be represented exactly due to the binary format that PC's use internally.

0 Kudos
Message 2 of 7

What unit are they in?  Were they at least a I64?  


Kudos and Accepted as Solution are welcome!
0 Kudos
Message 3 of 7

I have LabVIEW 2010, and the values are obtained in a calculation inside MathScript.


So, i don't decide the data type. There will be a way, I don't know that either.


In MATLAB, double precision is enough to represent that number. I really don't get why this happen in LV.


I will send a sample to show you that, as soon as possible.





0 Kudos
Message 4 of 7

You might want to post your code and continue this discussion in the MathScript forum.

LabVIEW Champion. It all comes together in GCentral GCentral
What does "Engineering Redefined" mean??
0 Kudos
Message 5 of 7

Here is a sample.


The sum of those are estimated zero in the t(i)=..../(T11(i)+(T12(i)/(rho*c))+(rho*c*T21(i))+T22(i)); the denominator.


I will post a massage in MathScript forum also.





0 Kudos
Message 6 of 7

The new thread for this topic  on the MathScript forum can be found here.

Lea D.
Applications Engineering
National Instruments
0 Kudos
Message 7 of 7