Semaphores and queue together. Is it necessary?

[dan07]

Ravens

The code is not working as I expected. While running in the Highlight Mode I saw that the State Machine starts working while the For Loop is still running (I put 6 states in the array just to make the For Loop runs longer).

Where is my mistake?

Thanks

Dan07

[battler.]

I have used Notifiers for this, not semaphores.  An advantage is you can also pass data with semaphores.

[nathand]

One problem is that you fork the semaphore reference wire just before locking the semaphore, so the lock may not be set until after you start enqueueing elements (which will then allow the other loop to start dequeueing them).  You need to wire the reference out through the for loop to force the lock to occur first.

[Daklu]

---

dan07 wrote:

The evaluation of the time of day was just an example. What really happens inside the State 1 is a code that will open a File Dialog and allow the user load a ascii file. Nevertheless, if the user closes the File Dialog the "open file operation" is cancelled and is no longer necessary to run State 2. This is the real situation. I would like to use the producer loop just to enqueue elements and not do any kind of evaluation or operations inside it.

---

I'm going to go out on a limb and diverge from conventional wisdom and the collective advice given so far...

The fundamental problem as I see it is that your states have not been properly defined.  The queued state machine pattern can be useful for quick and dirty prototypes; however, in common practice it usually devolves into a "function machine" rather than a state machine.  What's a "function machine?"  It's a state machine that uses states where where it should be using sub vis.  Typical characteristics of a function machine are lots of states (say, more than 10) and queue manipulation logic in the consumer loop.  For many reasons function machines are hard to maintain and can contain nasty bugs.  I suggest avoiding the QSM pattern altogether.

If you must use the QSM, here are a couple rules of thumb I like to follow:

-Other than dequeuing states, don't let the consumer loop manipulate the queue.  Definitely do not add states to the queue in the consumer loop.

-Only create states for the process "entry points."  Entry points are those initial states the producer must invoke.  In your example state 1 is an entry point.  State 2 is not.

To address your specific problem, there are a couple ways to solve it without introducing semiphores at all.

Option 1

Remove state 1 and move the load dialog to the producer loop.  If the user cancels out of the dialog box simply do not add a state to the queue.  If the user does select a file, pass the filename to the consumer loop as message data.

Option 2 

Convert state 1 and state 2 into sub vis and condense them into a single state.  Put the state 2 sub vi in a case statement so it is conditional on the user selecting a file from the dialog box.

(LAVA discussions on QSM can be found here and here.)

Message Edited by Daklu on 04-18-2010 10:38 AM

[RavensFan]

---

nathand wrote:
One problem is that you fork the semaphore reference wire just before locking the semaphore, so the lock may not be set until after you start enqueueing elements (which will then allow the other loop to start dequeueing them).  You need to wire the reference out through the for loop to force the lock to occur first.

---

Also, in the consumer loop, the lock queue is not preventing state 1 from starting.

There is no reason you should need to use a semaphore.  But if you are going to use them, I suggest you look in the example finder so you can learn how to use them properly.

[battler.]

Daklu,

Interesting insight.  If you don't recommend QSM structure, what structure do you recommend and why?

Cheers,

Battler.

[F._Schubert]

I also strongly recommend not to use any enqueue in the consumer. It makes a very 'flexible' code, but can evelove into an unpredictable code without much changes. As long as you don't enqueue inside the consumer, I consider it as a very good architecture.

The very first thing I would do is not call this a state machine. I will use the term 'commands' for what you put into the queue.

Daklu already suggested some ways to code around the 'feedback' inside the consumer loop. Here some more:

* The obvious: You can use a state machine in one of the commands of the consumer. Just don't mix it with the queue that drives the consumer.

* My favorite: Use the event handler as a bottle-neck. For this the consumer reports back to the producer, either by a temporary 'callback' notifier that is enqueued with the command (there is an OpenG package). My personal approach using User Events is documented in this community nugget of mine. You still can get concurrency issues with this, but they are more predictable and easier to find. 

Felix 

[F._Schubert]

One more option: make the consumer a state machine. Do not use the command queue as the state queue (I actually use scalar enums + shift register only). Use a state 'idle' as the finishing state of any sequence. Place the dequeue of the command queue inside this state.

This kind of architecture you can find in several postings of DFGray. 

Felix 

[Daklu]

---

F. Schubert wrote:

I also strongly recommend not to use any enqueue in the consumer.

---

 Well I hope so--I believe it was you who suggested it to me in the first place.  🙂 

---

battler. wrote:
If you don't recommend QSM structure, what structure do you recommend and why?

---

It depends entirely on project requirements and programmer's experience.  For maximum flexibility and scalability my default starting point for applications is an MVC architecture that decouples the UI from the underlying functional code.  Not everyone needs that level of flexibility or is comfortable with the additional complexity.  If you're looking for a single vi to function as the application framework, as the OP appears to be doing here, there are several variations that implement essentially the same principle.

Producer/Consumer Design Pattern (Events)

This is one of the standard templates supplied with Labview.  Structurally the QSM and Producer/Consumer patterns are very similar, the only difference being QSM consumer loops modify the queue directly.  Some might view the difference as a minor semantic point.  I believe the semantics are very important as it directly affects how you *think* about the two loops, which in turn influences how you decompose the problem and code your solution.

State machines, by their nature, contain the code necessary to determine what the next state should be.  They are self-deterministic.  When that state is controlled by a queue, it is natural to alter the contents of the queue.  In a Producer/Consumer pattern one loop is clearly the producer.  It, and only it, is allowed to put messages on the queue.  It is not necessarily restricted to only enqueuing messages.  It can and sometimes should execute its own logic to determine which messages to send.  The other loop is clearly the consumer.  It, and only it, is allowed to dequeue messages from the queue.  If you think of your application as using the Producer/Consumer pattern you are more likely to design solutions that avoid the many pitfalls associated with the QSM.

(I'll note the diagrams posted by Dan are, in fact, Producer/Consumer.  However, his terminology and the restriction he placed on the producer loop strongly suggested a QSM was in the making.)

Standard State Machine (Events)

The Standard State Machine is another template supplied with Labview.  To make it work with events, add an "Idle" case and in that case put an event structure to handle front panel events.  By removing the parallelism you've eliminated the need to worry about synchronization between the message sender and message receiver.  This pattern does have the unfortunate side effect of making your UI unresponsive when the program is not in the Idle state.

Standard State Machine (Producer Loop)

This is what Felix is talking about when he referred to Damien's posts.  Start with a Standard State Machine.  Add a producer loop with a message queue.  In the state machine (consumer) loop, add an Idle case that handles all message dequeuing.  This image illustrates the concept.

The fundamental principle all these implementations are enforcing is controlling when changes to the sequence of states can be made.  Each of them complete all the steps necessary for a given message before checking to see if there are any more messages.  The QSM convolutes the state sequence functionality with the message receiving functionality.  This is why it ends up causing so many problems.

Final note - There is nothing inherently bad about the concept of a Queued State Machine.  The problem is entirely in the implementation that has become commonly known as a QSM.

[James_W]

If you are worried about unresponsive UIs, you can even have 3 loops!!

(Someone's gonna kill me here for suggesting this)

Loop 1 = Master producer, Events, En-Q to Loop 2 (and maybe to Loop 3)

Loop 2 = Main program loop, contains all the core code states called by Loop 1. De-Qs Loop 1, En-Qs to Loop 3

Loop 3 = Display update loop, this loop is dedicated to display updates or maybe file logging, De-Qing from Loop 2 here allows Loop 2 to run faster as display updates can be a large code overhead.

This architechture is also fairly simple to debug.

I would have enqueued state X in loop 1 with a timestamp (variant cluster), and done processing in loop 2 to determine which sub routine to follow, just A or A&B.

UI can still be updated (directly if required) from Loop 1) being nice and responsive.

Edit: Loops 1, 2 & 3 are by default in Idle as Loop 1 is the only loop which is a "true Producer".

James

Message Edited by James W on 04-19-2010 05:52 PM