Get number of measurements for executed steps

[Grasshopper]

I'm looking to get the number of measurements for the steps that were executed ( not skipped and possibly not force fail or force pass ).  I know I can get the number of steps from the sequence and each step I could check the result status and get the number of elements from the result array but I'm hoping there's already a variable TestStand has to track this value.  Furthermore, looping thru every step as I previously stated is one thing but with multiple sub-sequences that'll probably be more involved and could be an issue with steps that looped until pass/fail.  Thoughts?

[MattyTE]

At what point do you want this info? In your main sequence, post UUT, result processor?

[Grasshopper]

I'm thinking as the last step in my highest level sequence's main as I'm going to compare the number of values to an expected value and pass/fail.

[MattyTE]

I think you are going to have to loop over your resultsList array. This sounds similar to wanting to make a Results Processing plugin, but do it in the main sequence because you want to run it as a test. Plug-ins use the resultsList array for all of that. 

I’m a little curious why you would want to do what you are doing though. I want to believe there is a better way.

[Grasshopper]

So, the reason for this test is because multiple product flavors can be run using the same sequences with preconditions for some of the steps.  Thus, model A might have 100 results and model B 500.  I want to confirm 100/500 results for the model.  This guarantees no steps were accidently skipped by someone troubleshooting something and then not restoring the sequence to its original state, that all tests that were supposed to be ran were actually ran, and by having the result be a pass/fail then a quick look at the PC to see a red result box or a check in the database shows this run wasn't valid.  Then, someone can determine not all steps or too many steps were ran and can dig into that.  An expected result count can be determined outside of TestStand and this is the sanity check on all the preconditions, model entry ( serial number ), user interaction ( skipping ), etc. 

I'm open to suggestions. 🙂

[MattyTE]

I do understand your concern. Assuming that you are confident and verify that the sequence works properly across product lines prior to release, you can use the user manager to give individuals appropriate rights to how they can modify the test during debug. In my opinion, someone debugging should only be able to put breakpoints in and step through, or jump around the sequence. I don’t think that they should be able to “force skip” a step via the step settings. TestStand is powerful enough to allow them to right click on steps during runtime and jump around. 

You can also programmatically disable breakpoints if an “operator” is signed in, allowing any of the debugging breakpoints to be ignored during production. 

I’m not familiar with your knowledge of TestStand, so feel free to ask more. 

Depending on the difference of your products, you can also use different sequences for your different part numbers. I would only suggest this when the product becomes different enough. 

Long story short, I do not think that counting your results to determine is the best path forward. Programmatically opening different sequence files for different products, controlling the signed in “user” to TestStand, and having better source code control to make sure the sequence itself didn’t change (other than breakpoints which can be programmatically disabled).