number manipulation question

[Vr6Fidelity]

Ok so I have a milling machine with some stepper controlled tables.

Im writing a VI to allow the end user to type in a numeric distance, and then have the VI round off that input to the nearest STEP.

I don't want all the loose change to accumulate and lead to positioning errors.

The table has 1,250,000 steps per inch, .0000008" per step.

So if the user types in .000035" thats 43.75 steps. so that needs to change to 44 steps.

What is the best way to accomplish this?

[johnsold]

I would multiply the value entered by the user (in inches) by the number of step per inch and convert the result to an integer. Then work in steps internally in the program after that. This will minimize the round off errors which can occur when working with fractional values which may not have an exact representation in binary. Unless you have a very large table I32 representation should be OK (~ +/-1717 inches at 1250000 steps per inch).

Lynn

[LabViewGuruWannabe]

If you simply want to round the value, convert it to an integer instead of a double. Or use the Round to +Infinity in the Numeric functions palette.

B-)

[Vr6Fidelity]

Johnsold is correct.

and GURU is VERY WRONG.

I divided the user input in unches by the inches value of one step to get the number of steps as a DBL, then I forced it into an interger after that.

GURU, if you did it your way it would just round the number, not make it divisible by .0000008.

And thank you,

[jrpe]

---

LabViewGuruWannabe wrote:
If you simply want to round the value

---

He isnt wrong...he clearly stated what his method would do, he just misunderstood your question.

[LabViewGuruWannabe]

I guess I did misunderstand the question, or at least made an incorrect assumption.

Vr6Fidelity had said,
"So if the user types in .000035" thats 43.75 steps. so that needs to change to 44 steps.

What is the best way to accomplish this?"

I thought you were asking how to show 43.75 as 44, not how to divide .000035 by .0000008. I assumed you already knew how to divide. 😉

No worries, as long as you got the problem solved.

B-)

[altenbach]

---

Vr6Fidelity wrote:

So if the user types in .000035" thats 43.75 steps. so that needs to change to 44 steps.

---

You should just prevent the user from typing an noninteger multiple of the step size. 🙂

Go to the properties page of your control and to the "data range..." tab ("data entry..." in LabVIEW 8.5). Set the increment to .0000008 and set to coerce. Now whatever is entered will be coerced to an integer multiple of the step size and you don't need to do any math.

[Vr6Fidelity]

I do have this figured all out now.

Altenbach: Just because a number ends in .0000008 does not mean it is divisible by .0000008.

[jrpe]

---

altenbach wrote:

Set the increment to .0000008 and set to coerce. Now whatever is entered will be coerced to an integer multiple of the step size and you don't need to do any math.

---

He never said make the input end in .000008, if the increment is .000008 and you coerce to the nearest multiple of the increment...then the input value will always be a multiple of .000008

[altenbach]

---

Vr6Fidelity wrote:
Altenbach: Just because a number ends in .0000008 does not mean it is divisible by .0000008.

---

Why don't you spend 60 seconds and try it out before putting words in my mouth that I never said. 😄