I have two different data which I'm doing very simple calculation with them; substraction.
First value is 1,640220000000000e+18
Second value is -1,640217348851742e+18
Obviously, these cannot be estimated as equal, the difference between them is 2651148258304. But LabVIEW calculates it as zero.
Why this error happens?
Please post a VI that demonstrates this.
What version of LabVIEW are you using? What is the representation of your numbers (single, double, extended)?
Is the fact you are using a comma instead of a decimal point causing you a problem?
And the difference between them is actually about 3.28e18 not 2651148258304.
Also, remember that many floating point numbers can't be represented exactly due to the binary format that PC's use internally.
What unit are they in? Were they at least a I64?
I have LabVIEW 2010, and the values are obtained in a calculation inside MathScript.
So, i don't decide the data type. There will be a way, I don't know that either.
In MATLAB, double precision is enough to represent that number. I really don't get why this happen in LV.
I will send a sample to show you that, as soon as possible.
Here is a sample.
The sum of those are estimated zero in the t(i)=..../(T11(i)+(T12(i)/(rho*c))+(rho*c*T21(i))+T22(i)); the denominator.
I will post a massage in MathScript forum also.