SubVI gives different result when inline

[naswermer]

Hey Guys.

I have a small SubVI that I use to calculate the number of steps to ramp to a final value from an initial value. It works fine most of the time, but when I inline the SubVI, the output is off by 1, consistently.

For example, when I give Initial:9.9, Final:1 Step Size:0.1, Instead of 89, it gives 88.

This happens only when I call the SubVI from a parent VI, not when I run the SubVI.

Also when I give step size 0.01, it gives 890, the right value, in both cases.

When I remove the round towards -Inf, it gives the right value even when inlined. But, I need the floor function.

It's very annoying and I haven't been able to figure it out for days now. It's killing me. Please Help.

san

[RavensFan]

I see you are using LV 10.  When I use LV 17, it works fine for me.

Could you be having problems with not being able to representing floating point numbers in binary exactly?

[altenbach]

Your code is not safe because you are ignoring basic limitations in floating point calculations. Inlining a subVI causes the subVI code to be recompiled into the toplevel code and some optimizations might cause a reordering of operations that causes what you are seeing. This is not an inlining problem, but a code flaw. It could occur whenever the VI is recompiled (code changes, version upgrade, etc.)

Values such as 0.1 cannot be represented in a floating point format with a limited amount of bits, you would need an infinite number of bits. To illustrate, tell me the value of N when the following code completes (then try it! All clear?):

Why don't you look at the value before rounding to -inf and display it with e.g. 20 decimal digits by changing the display format. The result could be microscopically higher or lower than you expect and rounding down is cutting a portion that is either close to zero or close to one, so you simply need to code around it.

On a side-note, your code could be all orange for simplicity. (you still need to implement the sanity check, though)

If this has to do with hardware (e.g. stepper motor stage), it would really help to do all calculations in units of steps or using a FXP representation appropriate for the accuracy of the device.

[naswermer]

Thank you so much for the detailed reply!

But... I couldn't understand a lot of what you said 😕

So, I looked it up and now I understand the issue.

I did a workaround by replacing the floor with a round to nearest.

It seems to work for now.

Is that the kind of solution you suggested?

Could you please clarify the thing you said in the last line about FXP and steps?

---

@altenbach wrote:

If this has to do with hardware (e.g. stepper motor stage), it would really help to do all calculations in units of steps or using a FXP representation appropriate for the accuracy of the device.

---

Thank you again.

nas

[naswermer]

---

@RavensFan wrote:

Could you be having problems with not being able to representing floating point numbers in binary exactly?

---

Yes. That was the problem. I seem to have fixed it with the workaround of doing round to nearest. 

Thanks for the reply!

nas

[altenbach]

---

@naswermer wrote:

I did a workaround by replacing the floor with a round to nearest.

---

That's not really the same. Now you are just having the same potential problem if the fractional part is very, very close to 0.5.

How do you guarantee that the inputs are sized so that the result of "No of steps" is always close to an integer before rounding?

[JKSH]

---

@naswermer wrote:

For example, when I give Initial:9.9, Final:1 Step Size:0.1, Instead of 89, it gives 88.

...

Also when I give step size 0.01, it gives 890, the right value, in both cases.

...

It's very annoying and I haven't been able to figure it out for days now. It's killing me. Please Help.

---

To understand the reason, see http://floating-point-gui.de/