Numeric Limit Test Roundoff Technique seeked

[willy1977]

Hello,

My teststand application has a numeric limit test set up as:

GELE -2.0 <= x <= 2.0 where x is measurement data.

There were times that my measurement data reads 2.002 and failed my numeric test.

My application engineer suggests the following:

1. Temporarily round off the 2.002 to 2.0 before TestStand starts the GELE comparison, so that the numeric limit test can pass.

2. 2.002 is still going to be logged at the final ATML report and the Database Schema which will be stored in the SQL database when the main sequence concludes.

I am looking for ways to do this temporary round off for GELE comparison and make sure the 2.002 will not be rounded off in the report and DB schema. Any suggestions will be appreciated.

Thanks and have a nice day

~willy1977

[santo_13]

I would rather not roundoff, if this is a parametric measurement, any border/marginal pass is bound to fail/pass based on process variations. You would need to adjust your limits based on the Cp and Cpk parameters of your measurements to ensure that majority of your measurements fall at the middle of the limits.

This is a cat on the wall situation and since you like it to fall on your side of the fence you're pushing the cat to your wish. Either your limits are not suited to your process or your process if not repeatable enough for the limits.

[Captain_Jack · Accepted solution]

Not disagreeing with what Santhosh pointed out, but if the limit is good, and the precision is accurate for the requirement and measurement capability, here's a link that may be helpful:

How-to-round-a-result-before-reported

I hope this helps!

-Jack