Synchronizing states across multiple loops

[giulio.foletto]

Hello, newbie here. I am trying to find a way to synchronize states across multiple loops. I have a number of loops that run in parallel at different rates. During each iteration some of them can change state and others have to change what they are doing depending of them. Some loops are both "consumers of states" (meaning they read the state of others and behave consequently) and "producers" (meaning they set states). 

I need a compact and scalable solution that can handle multiple producers and multiple consumers of states and is easy to modify if I need to add a new loop. So far the best solution I have found is storing states into enums and using local variables to read them in consumer loops; however, I read everywhere that local variables should be avoided. 

A Queued State Machine seems similar to what I need but it seems optimized for one consumer, whereas I need many of them. I tried using notifiers but they are only useful when changing states, not for storing them, because every Wait on Notification sees the new state only once. This forced me to add a shift register to store the content of the latest notification, but this seems overly complicated and needs a lot of wires and case structures which can be avoided using local variables.

I attached to this post an example VI. This is very general and the content of each loop is simplified compared to what I need. However it should explain my problem. Loop A changes state during each iteration. Loop B is a pure consumer of Loop A states. Loop C is both a consumer of Loop A states and a producer of its own states that Loop A checks in the second part of its iteration. Imagine that there are many other loops like these, and they communicate states between each other. 

What's the best way to handle this synchronization issue?

[Debrosse]

Not sure how many loops you have, but it seams like the Actor Framework might be helpful for your application. AF can manage Asynchronous communication but can also be Synchronous. Learning curve is a bit steep for AF. Another option is a QMH system. There’s a template from Delacor that’s very advanced. Search DQMH for more help. You’ll be able to understand that framework since you’re familiar with producer consumer loops. 

[crossrulz · Accepted solution]

---

@giulio.foletto wrote:

A Queued State Machine seems similar to what I need but it seems optimized for one consumer...

---

What you are actually referring to here is a Queued Message Handler.  A Queued State Machine is just a normal state machine that manages its states with a queue.  Nothing outside of that loop should touch that queue.  A Queued Message Handler receives messages from other loops via a queue.

---

@giulio.foletto wrote:

I need a compact and scalable solution that can handle multiple producers and multiple consumers of states and is easy to modify if I need to add a new loop. So far the best solution I have found is storing states into enums and using local variables to read them in consumer loops; however, I read everywhere that local variables should be avoided.

---

Avoiding just for the sake of avoiding is wrong.  In this case, however, local variables kill your scalability.  First of all, they limit you to a single VI.  I highly recommend you make each of your loops its own VI.  So this would start to point you to Global Variables.  I am still not a fan of using global variables here, but they could work.  I think the route I would take is to go with something along with the Delacor Queued Message Handler (DQMH) where they use User Events to broadcast messages to everybody.  If module C does not care about the messages from module A, it just does not register for those messages.

So if I were you, I would look closely at the DQMH.  There is a support community here: https://forums.ni.com/t5/Delacor-Toolkits-Discussions/bd-p/7120

[giulio.foletto]

Thank you both for the help. I will take a close look at the DQMH and the AF. I'll let you know if I manage to get it right.

[Kevin_Price]

DQMH is what first came to mind to me too, though to date I've only investigated it without yet using it on a project.

However, I'd also gently suggest you might want to take one or two steps back and think about your overall app architecture from a big picture point of view.   You might solve some of your issues by rethinking the general structure of your solution.

To me, multiple parallel loops that have mutual interdependencies on one others' internal states sounds like a difficult architecture to manage.  Does loop A depend on some combo of the states of loops B, D, and E?  Does D in turn depend on A, B, and C?  When two loops each depend on one another, how do you resolve tiebreakers?   With all the different possible state transition permutations, how do you know that you're done debugging?

Something like DQMH will provide a good *mechanism* to enable the kind of broadcast / subscribe messaging you're asking about, but it may be even better to work at cleaning up the interdependencies. 

   For example, I typically use a QMH approach where I have a "main brain" at the top which does all the coordination for the other loops.  The other loops only talk directly to the "main brain" loop and do no direct communication to one another.  The simple and rigid structure of this approach sacrifices some run-time efficiency, but I've found it to be tremendously helpful for troubleshooting and debugging.  

   This isn't meant as universal advice -- some systems need a more complex representation and some programmers are better equipped than I to conceptualize more complex interactions.    I'm just noting that for me personally, the strict hierarchy of loop interactions has been helpful.

-Kevin P

[giulio.foletto]

Thank you Kevin_Price

I agree with you that this kind of architecture is risky. However my case is not that complicated, but I wanted to keep the question as general as possible because it may be of interest to others or in the future.

In short, my loops control some devices and machines that should work at the same time. There is a "main brain" that sets states (think of Loop A in my example) and others simply have to follow (many Loops B). There are also some "slow" devices that, in some particular situations, must ask the main brain to delay a change of state. I am wondering if rendezvous could be of help but they do not seem to be versatile enough.

[crossrulz]

---

@giulio.foletto wrote:
There are also some "slow" devices that, in some particular situations, must ask the main brain to delay a change of state. I am wondering if rendezvous could be of help but they do not seem to be versatile enough.

---

Queues or User Events is how I pass data to the "main brain" loop.  It would be simple enough to add a "Wait" and "Resume" command to be passed to the main loop.  Or you could just make all of the loops send a "Ready" message when they are ready for the next state.