"Terminated" Status changed between versions?

[Captain_Jack]

I'm trying to update the process model and reporting sequences that we used with TestStand v2.0.1 to TestStand v3.5. While we've heavily modified the process model and text report sequences, I've noticed that the TestStand engine for v3.1 & v3.5 no longer mark steps as "Terminated" the way that v2.0.1 does. In order to to rule out modifications that we have made to the process model and reporting sequences, I created a simple sequence that terminates itself in version v2.0.1 that uses the default NI process model sequences. I then opened this sequence in TestStand v.3.1 and v3.5 and ran it to see how these versions documented the "Terminated" status. My results are listed in the attachements (v3.1 & v.35 where identical).

As you can see, Versions 3.1 & 3.5 no longer show Terminated from the step that generated it, nor does it show in the calling sequences until we get back to the MainSequence call in the process model (Test Report header).

What I'd like to know is - is this a feature of these versions, or is there some subtle change that we've made that has crept into the NI process models (i.e.; our development environment) that is causing this behavior? if this turns out to be a 'feature' of these versions, what would be the recommended method for getting the report to document as it does in v2.0.1?

I'm including the v2.0.1 sequence for reference to see if anyone can duplicate my results.

[TerryS]

Hi,

This appears to be a change between the versions, because I am able to generate the same reports.  The report is pulling this value from the status expression.  If you would like to implement the version 2.0.1 feature, you will have to create a custom step type.  Below is a link that explains how to create a custom step to modify the status expression.

Modify Status Expression

[Scott_Richardson]

Capt. Jack -
I think I can shed some light on the change of behavior. Specifically, in 3.0 and earlier, if a user suspends on a step that caused an error because on the run-time error dialog box, and the user terminated the execution because of the error, TestStand was incorrectly changing the Error status to Terminated for that step. We feel that the Error status should not be altered, so there was a change made in 3.1 to ensure that if a termination does not affect the status of a step that has already executed and set its status. In addition, if the termination occurs while a step is executing and the status has not yet been set, the termination status is given to the executing step upon completion, independent of whether the step is monitoring for a termination status change.

When you say that "This Step Terminates on Failures", I assume that the step calls termination but the step is also directly setting the status. One option is to modify the step to either not set the status on termination or explicitly set the status to "Terminated". For example, if your step type is using the Status Expression like the NI Pass/Fail step type, you could set the status to Terminated as shown below:

Step.DataSource != "Step.Result.PassFail" ? Step.Result.PassFail = Evaluate(Step.DataSource) : False; Step.Result.PassFail ? "Passed" : "Terminated")

Hope this helps...

[Captain_Jack]

Thanks for confirming that it is not something that we introduced in our reporting and execution modifications.

While modifying the Step types as you describe would solve the problem, it generates multiple versions of the same step types and we'd have to make sure that the original NI types no longer get used. It also would lead to the possibility of a lot of re-writing of thousands of lines of of legacy sequence code that we have compiled over the years. Still, it is an option.

I was thinking of adding the ProcessModelPostStep or ProcessModelPostResultListEntry Callback to the Process Model and have it monitor the Execution State of the current execution using 'Execution.GetStates ( runState, termState)' to edit the Step.Status for the last step that executed and all calling sequences. I just need to figure out how to get it to only update the one step was executing when the termination event was generated. I'll update this post with my results if I can get it working.

[Scott_Richardson]

Capt. Jack -

Yes, you could save the latest state after each invocation of the callback to a temporary variable that you create on each thread (Thread.RunTimeVariables), and when the state transitions from non-terminating to terminating, you manipulate the step status. I am sure you know that the callback will slow down your execution speed.

[Captain_Jack]

Scott,

I was able to implement this using the ProcessModelPostResultListEntry callback, which is already enabled in the Sequential Process Model for use with the Database and OTF functionality. It was a simple matter to add Active X checks for the termination state, and then overwrite the Step.Result.Status with "Terminated" as necessary. The reason why I prefer this is that upper level calling sequences would return Passed when a Termination Event occurred inside them, and the report was incorrectly documenting the calling sequence as Passed rather than incomplete (Terminated). While I realize this could be solved by proper initialization of the Pass/Fail or evaluation return value to a failing value at the beginning of the call, I would prefer a method that guarantees documenting that the step did not complete properly (i.e.; no false Passes).

The overhead is negligible and acceptable during normal execution, and while several additional steps where involved once termination was active, it still should not be an issue for what we are doing. If we where worried about time-critical events, we'd be looking into RTOS functionality for our tests.

Thanks for your help and suggestions!

-Jack

[Scott_Richardson]

Capt. Jack -
I am glad that you were able to get what you wanted with the callback.  Thanks for letting us know.

[Anonymous]

Scott,
I was able to implement this using the ProcessModelPostResultListEntry callback, which is already enabled in the Sequential Process Model for use with the Database and OTF functionality. It was a simple matter to add Active X checks for the termination state, and then overwrite the Step.Result.Status with "Terminated" as necessary. The reason why I prefer this is that upper level calling sequences would return Passed when a Termination Event occurred inside them, and the report was incorrectly documenting the calling sequence as Passed rather than incomplete (Terminated). While I realize this could be solved by proper initialization of the Pass/Fail or evaluation return value to a failing value at the beginning of the call, I would prefer a method that guarantees documenting that the step did not complete properly (i.e.; no false Passes).
The overhead is negligible and acceptable during normal execution, and while several additional steps where involved once termination was active, it still should not be an issue for what we are doing. If we where worried about time-critical events, we'd be looking into RTOS functionality for our tests.
Thanks for your help and suggestions!
-Jack