In the attached sequence "terminating help.seq", I can recreate the issue. First, select NI's built-in BatchModel as the process model...
In the Main of MainSequence, I press Terminate All while socket 0 in the middle of the synchronized section "a". All of the sockets proceed to Cleanup. So far so good.
But now the problem! Socket 0 finishes sync'd section "b" but Sockets 1 and 2 never seem to notice; they hang on the Enter("b") step, never noticing that Socket 0 has Entered and Exited section "b".
It's not until Socket 0 is done with its execution that Sockets 1 and 2 finish. I assume that this is because Socket 1 becomes the "lowest order" thread after Socket 0 is done executing. But I want Socket 0 to be the lowest order thread and direct the execution of these sync'd sections; if I never hit Terminate All, this Socket-0-driven behavior is achieved.
What's going on here?
Solved! Go to Solution.
I am taking a look at your attachments. What version of Teststand are you using?
I'm using TestStand 2014, 32-bit. Operating system is Windows 8.1 64-bit.
Thanks for your quick response, I'm not exactly sure what the issue is but I am certainly taking a closer look at this.
May I ask how this issue is impacting your project? If possible, I would recommend making a service request with NI and we can determine the best way to proceed. Please let me know!
And thanks for looking into this.
I actually have an existing service request for this that I made in parallel to the forum post. The AE on that request is getting some time to sit down with the R&D team with some, hopefully, pointed answers! I'll try to remember to post their findings here to share some knowledge with the community.
The behavior that you are seeing is explained in Nested Synchronized Sections. When you hit terminate, TestStand "ignores the inner section when the type of the outer section is One Thread Only. For example, when you nest one serial section in another serial section, each thread that enters the outer serial section proceeds only until the Enter step of the inner section and then waits for the other threads to reach the same Enter step of the inner section before completing the inner section." If the outer section is One Thread Only, the inner sections are ignored.
Since the exit for thread "a" is never hit, the other threads are left "waiting" at the start of batch synch for "a," until thread 0 is removed from the batch, because the batch synchronization step is One Thread Only. When thread 0 is removed, thread 1 will enter the One Thread Only section for "a," because for all other types of outer sections the inner sections are still honored as a new rendezvous point of sorts for the threads in the batch. The other threads have been blocked so when thread 0 exits, the other threads continue as if there are only three threads in the batch.
To workaround the issue you are seeing in your current sequence file, add a batch synchronization step at the top of the cleanup that exits "a" and ignores any runtime errors, as such:
This will cause a runtime error in all the other threads because they will not have entered "a," but the follow of execution will proceed as intended when terminating all.
This seems to be a simplified case of what your larger projects looks like. Could you share what your flow execution is and how this problem came about?
Happy pi day. And thanks for the explanation and workaround to the problem! In the simplified example, if I hit Terminate or Terminate All, adding Exit() batch synchronization steps in the Cleanup works to ensure the flow of the threads obey the originally intended OneThreadOnly process I've pre-determined. And of course, the Ignore Errors setting avoids having the error dialog popup, which I'm totally fine with.
Playing a little more with the scenario, if I have any step with a pass-or-fail condition that triggers a post-action to go to cleanup or terminate execution, the above workaround also ensures all other threads follow thread 0's lead to end the test.
OK, so... real-world story time:
The post-action stuff I talk about above relates to a pressure controller in my test system. I always need it be at an expected pressure level to avoid bursting something and avoid potentially running the system with a malfunctioning pressure controller. I also only want to control the pressure once because the process takes a long time; in a process where the pressure is raised and vented multiple times, if hit my pressure target once and distribute it to the Batch of UUTs once, I can avoid the time-sink of doing it serially. I utilize a new Batch Sync section for each pressure setting. In my larger project, the pressure is controlled in Setup, Main, and Cleanup. If the pressure fails to build in an acceptable way in Main, I'd like to go to Cleanup in all threads**. I modified the simplified example, adding more sections to Main. It seems seems like if use the technique again (adding Exit() for each possible Batch Synch section possible), I limit use of the steps inside the OneThreadOnly sections to thread 0. So that's good. Maybe we have a solution!
I've attached an example of the original simplified code with some added stuff. Hopefully it's clear what things I added and how I'm handling that fail condition and Terminates.
** Normally, thread 0 controls all of my test system equipment; that is, all the equipment on there is shared between multiple UUTs receiving the same signal in parallel (like pressure or a voltage). I would actually like my other threads to be able to continue testing even if thread 0 fails, retaining the post action of Go To End Group. I think if I shared my VISA references via TestStand queues and appropriately shared that queue information in my sequence, I could essentially hand off control to the next available thread (thread 1). The idea here is that one UUT can fail while the others resume testing. Hopefully this makes sense.
After some investigation, there is also another method (more recommended way due to maintainability) to handle the nesting issue. Instead of using the batch synchronization steps, you can put all of the steps that would execute within the batch synchronization sections and then use the step settings to set the batch synchronization to "One Thread Only (first thread executes, remaining threads skip)." For example, in the main section of you main sequence, two calls to subsequences are made, one for a1 and one for a2:
where a1 subsequence looks like the following:
Note that I skipped the check power step because on failure it was going to cleanup. The a2 subsequence looks the same. In mainsequence, the step settings for a1 and a2 synch subsequence calls look like the following:
The TestStand engine will handle the exit of the batch synchronization rather than requiring the extra steps in cleanup. I tested this out, and removed the extra exit steps from the cleanup section, and this behaved the same way. Please let me know if you have any troubles with this.
To address your other concern, there are many ways to share data between the executions/threads.Is there a reason that you want to use queues? You can also store the reference in a station global and share it that way, or in a file global with the sequence file property set to share globals between execution, or Engine.TemporaryGlobals.
Thanks for the help!