# LabVIEW

cancel
Showing results for
Did you mean:

## Why LabVIEW calculates wrong?

Hi everyone,

I have two different data which I'm doing very simple calculation with them; substraction.

First value is          1,640220000000000e+18

Second value is     -1,640217348851742e+18

Obviously, these cannot be estimated as equal, the difference between them is 2651148258304. But LabVIEW calculates it as zero.

Why this error happens?

Regards,

Ozgur

Message 1 of 7
(1,316 Views)

## Re: Why LabVIEW calculates wrong?

Please post a VI that demonstrates this.

What version of LabVIEW are you using?  What is the representation of your numbers (single, double, extended)?

Is the fact you are using a comma instead of a decimal point causing you a problem?

And the difference between them is actually about 3.28e18 not 2651148258304.

Also, remember that many floating point numbers can't be represented exactly due to the binary format that PC's use internally.

Message 2 of 7
(1,307 Views)

## Re: Why LabVIEW calculates wrong?

What unit are they in?  Were they at least a I64?

------------------------------------------------------------------

Kudos and Accepted as Solution are welcome!
Message 3 of 7
(1,283 Views)

## Re: Why LabVIEW calculates wrong?

I have LabVIEW 2010, and the values are obtained in a calculation inside MathScript.

So, i don't decide the data type. There will be a way, I don't know that either.

In MATLAB, double precision is enough to represent that number. I really don't get why this happen in LV.

I will send a sample to show you that, as soon as possible.

Regards,

ozgur

Message 4 of 7
(1,262 Views)

## Re: Why LabVIEW calculates wrong?

You might want to post your code and continue this discussion in the MathScript forum.

LabVIEW Champion. It all comes together in GCentral
What does "Engineering Redefined" mean??
Message 5 of 7
(1,260 Views)

## Re: Why LabVIEW calculates wrong?

Here is a sample.

The sum of those are estimated zero in the t(i)=..../(T11(i)+(T12(i)/(rho*c))+(rho*c*T21(i))+T22(i)); the denominator.

I will post a massage in MathScript forum also.

Regards,

ozgur

Message 6 of 7
(1,209 Views)

## Re: Why LabVIEW calculates wrong?

The new thread for this topic  on the MathScript forum can be found here.

Lea D.
Applications Engineering
National Instruments
Message 7 of 7
(1,160 Views)