Teststand Report Parameter UUT Status

[MechUnit]

Hello Community,

I am using TestStand 2017 32-bit.

In my UUT Report, what is the best way to customize the text for the UUT Result parameter from saying "Passed" or "Failed" to something else when the test finishes?

Thank you

[~jiggawax~]

Whenever a sequence gets called in TestStand it creates a sequence context.  Basically a pointer to an object, called RunState, with a whole bunch of information about the execution of that sequence.  For example, things like which step is currently executing, what the next step is, whether or not the sequence failed, etc...  At the end of the sequence execution a lot of this information is passed back to the calling step and put into the result for that step. 

At some point your process model is calling MainSequence from your client file.  When MainSequence returns to the process model the information in RunState for MainSequence is put into a result in the process model and that is used to determine the overall pass/fail of your test.

So if you were to put a step as the last step in Cleanup of MainSequence that called "RunState.SequenceFailed = False" then the process model would think that MainSequence passed.  When it generated the report or displayed the passfail banner then it would reflect that MainSequence passed, which is assumed to be the overall pass/fail status for your test.

Ultimately this is a really bad idea because then you can falsify reports, especially if you are dealing with medical or governmental products.  I would argue that if you need to change this at the end of your test then you aren't performing your tests correctly.  Just my 2 cents though.

Hope this helps,

[MechUnit]

My situation...

If the user doesn't do a task within a certain time limit, then I display a message popup that says "Test Aborted" and Goto the next appropriate step. I don't want to set the runstate to Abort or Terminate, because I don't want to end the test or jump straight to cleanup right away. However, the report at the end says UUT Result Passed, even though I want it to say "Aborted". 

I appreciate the feedback and will work on performing tests more correctly moving forward.

[pulidotech]

You could use the SequenceFilePostResultListEntry callback and change result wording there based on whatever criteria you like, but I generally agree with jigg, that it might not be the best.

Could this be done as a special step type?  If it's within the confines of something like a special step type, you could manipulate the result in Post Step in a controlled manner without affecting other steps.

[MechUnit]

I could set the post step runstate to Terminate, but I really prefer for the UUT result on the report to say "Aborted".

If I set the post step runstate to Abort, then I don't even get a report.

Can you please give an example of a special step type where you are manipulating the result Post Step and the UUT Result in the report says Aborted?

[~jiggawax~ · Accepted solution]

Terminate and Abort mean completely different things in TestStand.  Terminate means that cleanup ran.  Abort means the execution ended abruptly and not a single other thing in the TestStand engine executed.  That's why you will NEVER see Aborted on a TestStand report. 

So if you are using a later version of TestStand there is a model callback called PostMainSequence.  You can put an expression in there with RunState.Caller.Locals.ResultList[0].Status = "Aborted".  This will get the report to say Aborted at the top.

[UnholyPenguin]

---

@~jiggawax~ wrote:

Whenever a sequence gets called in TestStand it creates a sequence context.  Basically a pointer to an object, called RunState, with a whole bunch of information about the execution of that sequence.  For example, things like which step is currently executing, what the next step is, whether or not the sequence failed, etc...  At the end of the sequence execution a lot of this information is passed back to the calling step and put into the result for that step. 

At some point your process model is calling MainSequence from your client file.  When MainSequence returns to the process model the information in RunState for MainSequence is put into a result in the process model and that is used to determine the overall pass/fail of your test.

So if you were to put a step as the last step in Cleanup of MainSequence that called "RunState.SequenceFailed = False" then the process model would think that MainSequence passed.  When it generated the report or displayed the passfail banner then it would reflect that MainSequence passed, which is assumed to be the overall pass/fail status for your test.

Ultimately this is a really bad idea because then you can falsify reports, especially if you are dealing with medical or governmental products.  I would argue that if you need to change this at the end of your test then you aren't performing your tests correctly.  Just my 2 cents though.

Hope this helps,

---

Interesting, this bad idea got put as a requirement for the CTD practice test heh

[~jiggawax~]

I would love to understand why you think this is part of the CTD Practice exam.  From my perspective I don't see any result list manipulation on the part of the test taker.

[UnholyPenguin]

---

@~jiggawax~ wrote:

I would love to understand why you think this is part of the CTD Practice exam.  From my perspective I don't see any result list manipulation on the part of the test taker.

---

The very last step is to suppress all errors if initialization failed, but change the UUT Status to "Error".

On a side note, maybe I did something wrong, but I believe the suggested solution worked in TS 2019, but not 2023.. HMM, I'll try again tonight on 2023. 

[UnholyPenguin]

I should add, no one has ever provided a solution to the CTD practice test (as far as I can find), so maybe I'm misinterpreting it.