How can I get my Teststand report to only show the step data for the last iteration of loop?

[Peter3]

Hi all,

I already asked my question in a very old thread:

http://forums.ni.com/t5/NI-TestStand/How-can-I-get-my-Teststand-report-to-only-show-the-step-data-fo...

But since I guess that nobody whatches this old closed question any more, I want to ask here in a new thread.

The issue is that I want to have in the report only shown the results of the steps from the last iteration of loops, applying to all availiable loop types (while, for, dowhile, foreach,...).

My question is: How to implement that?

My current workaround is using ProcessModelPostStep in ProcessModel to get the information whether the current step was a start or end of loop step. At start of the loop I save the Id of that step in FileGlobals, and in the next iteration, if Ids match, I enhable the DiscardResult flag for that step. At each end of loop step I disable the DiscardResult flag.

By doing this, I am having the only the FIRST iteration results in my report. But my goal is to have only the LAST iterations result in report.

I dont want to modify the ClientFile statically (i.e. by manipulating any Callbacks), I can only modify it temporarily at runtime.

I also dont want to modify the result list at the end of the execution (while creating the report), since I have much iterations which slows down my TestStand.

Since I noticed that the OTF reporting slows down my execution significantly this is no option too.

Do you have any suggestions? I would be very glad to find a solution.

Regards, Peter

[~jiggawax~]

Peter,

Let me explain how the result collection and reporting works and then maybe it will help direct you to a solution that will work.

Basically every sequence has a Locals.ResultList.  This is an array of containers that gets populated whenever as step executes that has recording enabled.  Actually....It gets a new entry every time a step with recording enabled starts to execute (not at the end of the step).  See this site for more on what happens when a step executes: http://digital.ni.com/public.nsf/allkb/D1970FCB4DB2347C86256DDB004FAD9E  You can see that the first action is to allocate the step result in the ResultList variable.

So whenever a step gets executed, regardless of if it is in a loop, it will put an entry in the ResultList.  This ResultList is passed back to the caller and eventually back to MainSequence which then gives the list, as a tree, to the Process Model.  The Process Model passes it to the Plugins which parse through it and generate the reports or log it to the database.   The trick with TestStand is sending the correct ResultList from MainSequence back to the Process Model before the parsing occurs.

Understanding this you can see that there are hundreds of ways to accomplish your goal.

Here is just one idea: Don't record anything in the loop. Instead store it off to a local variable (NOT Locals.ResultList).  Then record the local variable after the loop executes.  This way you only have one step recording results that happens after the loop.

Another idea is to go through Locals.ResultList and remove any results with the stepID from a previous iteration.

Another idea is to parse through it in the Cleanup of MainSequence and remove any duplicate stepIDs out of the ResultList tree.  This would be kinda tough.  I don't recommend this one.

If it were me I'd go with the first option.

Let me know if you need specifics on what to do.

[Peter3]

Hi Jigg,

very thanks for your explanations (btw: the link does not work for me).

I think I understood the result collection now in principle.

For my problem, I would also prefer the first idea, since the others are slowing down the process significant (I already tried them out).

But how to implement this without modifying the ClientFile? I think this can be done modifying at least SequenceFilePostStep or SequenceFilePostResults Callback, with moving the complete new result container of the current step into the local variable. But as far as I know this does affect only the SequenceFile in which the Callback is modified. How is it possible to modify any Callbacks of the ClientFile's context from within the ProcessModel?

Regards, Peter

[~jiggawax~]

http://digital.ni.com/public.nsf/allkb/D1970FCB4DB2347C86256DDB004FAD9E

I'll try the link again.  If not you can just google 'TestStand Step Actions'  It will be the first link.

I guess I didn't realize you couldn't modify the client.  That is going to be a huge game changer.  You will most likely need to use the ProcessModelPostResults callback in the process model and look through the ResultList and delete all instances of the Results with the same step ID.

This will slow down your execution though.

Option 1 is not possible without modifying the client.

There are 3 types of Engine/Reserved Callbacks (The purple callbacks).  Here is a list of all of them and where they can reside: http://zone.ni.com/reference/en-XX/help/370052K-01/tsfundamentals/infotopics/callbacks_engine_list/

For you, you need to understand that ProcessModel engine callbacks can only be placed in the ProcessModel but only apply to steps in the Client file. 

SequenceFile engine callbacks can be placed in any .seq file and apply only to steps in the file within which it resides.

[Peter3]

Hi Jigg,

thanks again for your ideas.

I will try to modify ProcessModelPostResults. Possibly the speed reduction is acceptable.

I can not modify the ClientFile since this is auto-generated by an Excel tool. Probably we could modify this tool to add a template of SequenceFilePostResults callback while generation of the sequence. But this would be high effort, I guess.

Maybe we find a better solution...

Adding this as an option to the loop step types would be a great innovation point - or USP - for next TestStand release, wouldn't it?

Regards, Peter

[~jiggawax~]

I don't see this as a huge need for the looping step types.  Your case seems like an outlier.  Plus, sometimes it is not known that you are on the last loop until you evaluate the expression.  My 2 cents here.  You can always put it on the TestStand Idea Exchange and see where it goes.  Generally if it gets a lot of Kudos or traffic they will definitely consider it.

Not sure if you are aware though, you can loop on a single step and have it record the last result.  This would again cause you to edit the dynamic tool that reads from excel.

Your best bet for now is to either edit your tool or use the ProcessModelPostResults callback.  Let me know if you need help on what to call in the callback.

Regards,