Floating Point Errors when using "Coerce to nearest"

[pogo]

My students found a strange bug this semester that I never saw before (currently using LabVIEW 2018).

A (double precision) knob is coerced to the nearest 0.01.

If the selected value is lower than -1 or larger than 1, an error light should come on.

With "care", the knob can be rotated by hand to a value of -1.000000000000000222044605, which is exactly one bit smaller than -1.000000. In this case,the nearest value is -1.00 (which is correctly shown on the knob's digital display). However, instead of actually coercing the value to -1.000, the output of the knob remains at -1.000000000000000222044605.

As a check, a comparison node says that the value truly is less than -1.

While the comparison node is obviously working fine, why isn't the value actually being coerced?

Even recognizing that final "coerced" values might not be perfect due to floating point round-off, this output value is not correct, because the correct value of -1.00 is exactly representable as a floating point number. I would expect a problem of this type if the cutoff was -1.01, but never for -1.00 itself.

Obviously, workaround exist... I could multiply the value by 100, then convert to an integer, then do my range checking on the integer. But I think this is an actual bug that needs to be corrected, and I'm hoping for another pair of eyes on it.

[billko]

I can't open the code because I only have LV 2016, but I'll go out on a limb and say it's because the coercion value itself is not whole number; therefore, you actually are dealing with FP issues.

[pogo]

Hmmm... I thought that at first, too.

But -1 is a whole number.

And in fact, when I input a number that is 1 bit away from -1 in the other direction, it does coerce to -1 exactly.

So, -1 is an exactly coercable value.

[RavensFan]

But. .01 is not exact.  So if you go 100 counts of .01, the non-exact .01 becomes a non-exact 1.  You are summing up the floating round-off errors.

[pogo]

When I first started looking in to this, I had the same initial reaction: 0.01 is not exact.

But the more I look at it, the more that seems irrelevant.

If this value was being coerced to -1.01, then I totally agree that the coerced value would not be exactly -1.01, it would be rounded a little bit. However, when you run this vi, with this input value, LabVIEW is clearly claiming to coerce it this value to "-1.00", not "-1.01". "-1.00" is exactly representable in double precision floating point. I think it's exactly representable in every floating point representation since the invention of computers, whether single or double precision!

As a test of your theory, a user can type "-1.0" directly into the digital indicator instead of rotating the knob.

When the data is input this way, there is no rounding error whatsoever.

The problem here is that the "coerce" property of the knob is being inconsistent with itself on a floating point level.

I'm intrigued by your mention of "summing". I don't see that any summing is happening here. Can you elaborate on how the "coerce" property might use summing to reach a "final" value?

[RavensFan]

But you are talking about entering the value by typing it in.

But what if you start at another number such as 0 and click up or down 100 times?  100 x an inexact .01 will leave you with an inexact 1.00

Think about 1/3 x 3 in decimal systems.   1 divided by 3 is .333333 to however much precision you want.  Add .3333333, you get .6666666.  Add another .3333333 and you get .999999999, but you are expecting it to be back at 1.00000000.  1 divided by 3 times 3 is not the same as 1.

Run this code and you will see that 1 is not exactly 1 once you are done the math within an inexact .01.

[Ben]

Try showing about 25 digits in the digital indicator.

Drop an "epsilon" constant from the numeric >>> Constants palette and show 25 digits on it.

I THINK the "increment" button is summing the increment (that is a value that can not be represented in binary) and that is where the rounding get in the way.

Ben

[pogo]

I agree with everything you are saying about using the "increment" button.

Interestingly, I never tested or even touched the increment button until your suggestion, because this question is about coercing a value, not creating it.

So I just tried it.

As you and I both expected, starting at zero and clicking "down" 100 times gave an "incorrect" result.

However, that is not part of my original question.

The question is not about how it got rounded in the first place. The question is about after you enter this value into the control, why it does not get coerced to the nearest increment 0f 0.01.

If the "input" data is -1.000000000000000222044605, then the nearest multiple of 0.10 is -1.000000000000000000.

There is no way this data should be "coerced" to a value of -1.000000000000000222044605, because that is not a possible nearest coercion value for a multiple of 0.01 regardless of the value of the input data.

I completely agree that if the input data is near to -1.01, then the "nearest" possible value for a double precision "coerced" value is actually only -1.01000000000000001000, which isn't quite the same as -1.010000000000000000.

For any input value between -1 and -1.01, the resulting coerced value MUST be either

-1.000000000000000000

OR

-1.010000000000000010

There does not exist an input value for which the resulting coerced value should be -1.000000000000000222044605.

Based on your response, I wonder if the coders for the "coerce" feature are actually summing a past history, instead of just starting fresh with each value as entered by the user. IMHO, having the "coerce" feature be based on the entire history of past data is a terrible idea.

[crossrulz]

---

@pogo wrote:

Based on your response, I wonder if the coders for the "coerce" feature are actually summing a past history, instead of just starting fresh with each value as entered by the user.

---

Well, let's talk about how you mathematically coerce.  If y is your coercion value, then the coerced value should be round(x/y)*y.  So you have 2 operations in there with not precise values.  Having a little offset after that is not unexpected.

[pogo]

I think your formula seems correct on the face of it: round(x/y)*y.

This is a helpful way of thinking about this problem.

There will be only one floating point issue (not two) in executing this formula, because the round will become an exact integer.

I have implemented your suggested formula in LabVIEW 18, Mathematica 11.3.0.0, and Excel "Office Professional Plus 2016", all of which use IEEE754 floating point representations.

When x = -1.00000002 (note that the difference from -1 here is several orders of magnitude different than the examples used in the prior messages), and y = 0.01, I get the following results from the three software packages when applying your "coerce" formula:

1. The result in Mathematica is  - 1.000000000000000000

2. The result in Excel is               -1.000000000000000000

3. The result in LabVIEW is        -1.000000000000000000 when the formula is implemented directly.

4.  But the result in LabVIEW is  -1.00000002000000010  (note both the remaining '2" and the extra added "1" at the end) when implemented by the "coerce" property of the knob.

Clearly, if this is the formula or method being used, LabVIEW's results are inconsistent with LabVIEW's results.

In other words, your suggested formula has made me more sure than ever that the knob's "coerce" property is coded incorrectly!

PS: I have no way of checking it, but I don't think I've had this problem using the same code in earlier versions of LabVIEW... I've used every version since version 6.0.