LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Why LabVIEW calculates wrong?

Hi everyone,

 

I have two different data which I'm doing very simple calculation with them; substraction.

 

First value is          1,640220000000000e+18

Second value is     -1,640217348851742e+18

 

Obviously, these cannot be estimated as equal, the difference between them is 2651148258304. But LabVIEW calculates it as zero.

 

Why this error happens?

 

Regards,

 

Ozgur

0 Kudos
Message 1 of 7
(2,620 Views)

Please post a VI that demonstrates this.

 

What version of LabVIEW are you using?  What is the representation of your numbers (single, double, extended)?

 

Is the fact you are using a comma instead of a decimal point causing you a problem?

 

And the difference between them is actually about 3.28e18 not 2651148258304.

 

Also, remember that many floating point numbers can't be represented exactly due to the binary format that PC's use internally.

0 Kudos
Message 2 of 7
(2,611 Views)

What unit are they in?  Were they at least a I64?  

------------------------------------------------------------------

Kudos and Accepted as Solution are welcome!
0 Kudos
Message 3 of 7
(2,587 Views)

I have LabVIEW 2010, and the values are obtained in a calculation inside MathScript.

 

So, i don't decide the data type. There will be a way, I don't know that either.

 

In MATLAB, double precision is enough to represent that number. I really don't get why this happen in LV.

 

I will send a sample to show you that, as soon as possible.

 

Regards,

 

ozgur

0 Kudos
Message 4 of 7
(2,566 Views)

You might want to post your code and continue this discussion in the MathScript forum.

0 Kudos
Message 5 of 7
(2,564 Views)

Here is a sample.

 

The sum of those are estimated zero in the t(i)=..../(T11(i)+(T12(i)/(rho*c))+(rho*c*T21(i))+T22(i)); the denominator.

 

I will post a massage in MathScript forum also.

 

Regards,

 

ozgur

0 Kudos
Message 6 of 7
(2,513 Views)

The new thread for this topic  on the MathScript forum can be found here.

Lea D.
Applications Engineering
National Instruments
0 Kudos
Message 7 of 7
(2,464 Views)