ni teststand 3.5 numeric limits properties

[ioanandrei.ciurdar@leoni.com]

Hello! I have a test step with the numerical limits -0.3 and 0.5. Sometimes the DMM 4 wires measures 0.50007 for example and the test result is FAIL.I don't need this precision. I want to take into consideration only 2 digits after the comma, in order not to make the test step failed. So if the DMM measures 0.501xx to 0.509xx, the test must be PASSED. Can I make it with the option Units set to hecto(h) 10e2?If so, is there an option to make this change for all test steps at once?

[dug9000]

There is no option to compare to a particular precision. TestStand compares to the precision of a double-precision IEEE floating point number. Why not just set the upper bound to 0.509?

-Doug