Documentation about the result container

[jungledede]

Hello,

I am working on a custom model plugin to link our internals trackabilities (yes plural) system to TestStand.

All of our different trackabilities are packaged into one C#/Dot Net dll. This way we got only one entry point and don't have any difference between iTac or SQL or flat file or other in-house tracea system.
This DLL was developed before we start using TestStand.

So today my job is to make a TestStand Plugin that adapt the TestStand data to the format that our DLL accept.

Sadly, I am facing issue as I don't find the relevant documentation about the Result container.
I ask duckduck go and Bing Copilot to get a starting point and i use the "computer" example with breakpoint to try to get data.
I was able write myself some documentation. But when you think you got it you find again more difference....

My finding:

Result is not not the same between PassFail, Numeric and String... (not easy to code something with that..)
Then i discover that the Limit container exist in different formats, even for numeric test.

Result.RawLimits vs Result.Limits => What is the difference (both are limits CT)?
In numeric (at least for GELE limits) i got the limit CT to have members Low and High.
No i got teststand yelling at me because for numeric with LOG limit it do not find limit.low... Indeed with breakpoint I found that the limit CT is now composed of a string ThresoldType and numeric Nominal. WTF ?

How I am supposed to code something when everything changes and no doc?

NI sell us Teststand as an easy/no code sequencer. But reality is that outside their white paper you have no documentation, you are more than encouraged to use only Labview.
Python and Dot Net are supported with only obsolete version... (Who still use DotNet Framework 4.x ??)
Honestly, NI-Teststand is a cold shower. I don't see the "Apple" feeling we get with Labview/CVI where you plug, you code, it Works.

I feel I am missing one important place to look, but i haven't find it yet.

[Scott_Richardson · Accepted solution]

Romain -

As a summary:

• A step type defines a set of properties, some that are used at edit time and some that are used at runtime. The properties under Step.Result define what is available to result processing plugins. All other properties are used to save edit-time configuration for the step. Depending on purpose of the step type, the edit-time and runtime properties will vary per step type. The sub-properties of each built-in step type is listed in the TestStand Environment Reference Help and as an example here is a topic on Numeric Limit Test Step.
• The structure under the Step.Result container defines what the step will capture to demonstrate why a test passed or failed. All steps include the list of properties listed under Custom Result Properties, but steps can define as or as many custom properties as well.
• An execution collects the results under the Locals.ResultList array for a sequence and builds a tree structure for the entire execution hierarchy.
• Plug-ins typically access this hierarchy of results at the end of the execution, or they can be designed to handle on-the-fly results where the execution periodically pass results to the plug-in.
• Understanding how TestStand collects step results during an execution is important, see the following topic for reference: Result Collection.
• The Engine generates a set of standard results under the TS container, and that is documented here: Standard Result Properties.

The above mentioned flexibility allows step types, built-in and custom, to handle whatever they are designed to do.

You are correct that this flexibility makes it more challenging for result processing plug-ins to save results. The approach for a plugin must be one or more of the following:

• Treat a subset or the full superset of all built-in step type result properties in a hardcoded fashion, which requires the plug-in to expect them and to handle them in a defined way.
• Treat any non-expected properties as generic name value pairs that are logged generically, i.e. log the property path and its value.

Regarding "RawLimits", there are two groups of limit comparisons, one that compares against actual numbers and can contain equality, and EQT that compares against a nominal value and its upper & lower threshold value to define a range. All numbers in TestStand are effectively floating point and each property has a numeric format. The default is %f which has 13 digits of precision and allows to properly compare integral values being equal even though they are represented as floating point internally.

If I remember correctly, there were cases where the EQT comparison that used a 'real' numeric format was not appearing correctly in reports for cases where the measurement was within the limits, the test passed, but the calculated actual high and low limit by the report put the measurement outside the limits. The below image now uses the values in raw limits to properly calculate the lower and high bound when the report is rendered.

This is what the result values look like in results:

There might have been other similar cases where the the test result was correct, but what appeared in the report looked like it should have failed.

Regarding creating result processors, if you did not already know it, there is this help topic: Examples for Customizing Result Processing and their examples here: C:\Users\Public\Documents\National Instruments\<TestStand>\Examples\Customizing Result Processing\

Hope the above helps...

[jungledede]

Hello Scott,

Thanks you, a lot! This is what is wanted to find. All you link goes to my favorite bookmark 😄

Last Friday spent my day to write what i have understand and continue to play with teststand and breakpoint.

I better understand now how everything is working.

As you sugest, i am already playing with the report example to see how NI team have done it. And since I will need to code the on the fly, i will continue to look at them.