LabVIEW
'Quotient & Remainder' and 'Rounds toward -Infinity' functions are not performing as intended.
Please explain this behaviour, I'm expecting a TRUE result in both scenarios ('Quotient & Remainder' and 'Rounds toward -Infinity').
Hi moderator,
there are several discussion here in the forum on floating point precision - you really should search and read them…
When you don't believe you should increase the precision (umber of digits after decimal point) for your upper constant: it will show something like 34.0499…97!
Proof:
Yeah, I remember, I've already experienced it before but my memory is failing me and what I don't remember now is what is the work around...!!
Edited:
Increasing precision is one way that I already figured it out, but isn't there a native function/subVI which is immune to this issue with floating-point precision
"Rounding toward nearest" instead of "Rounding down"?
To your edited question:
Nope, there is no vaccine. Rounding issues are inherent when using IEEE754 floating point numbers…
@GerdW wrote:
When you don't believe you should increase the precision (umber of digits after decimal point) for your upper constant: it will show something like 34.0499…97!
Proof:
This is not the case here, as the Actual is a random generated number and Low Resolution is derived from the Actual itself (before it is transmitted, to save the bandwidth) using below method:
SORRY THAT IS INCORRECT PROBLEM STATEMENT...
Perhaps I'm working too much now days..!!! I need a beer.
Hi moderator,
why do you scale the "low resolution" number twice? Multiplying by 10k will never give the same as the original number 😄
And still: you CANNOT compare floats for equality without handling rounding errors properly!
Even your "measurement value" has rounding errors when displayed with just 10 digits precision…
Floats are the same when they are in the same range as is given by "machine epsilon"!
