Synchronous Messaging with User Events

[Gregory]

Problem: I was wondering what is the best way to get data out of an asynchronously running process using user events as the transport mechanism.

Premise: User events can be a great way to transport messages/data between processes (i.e. loops). You don't care who is listening or how many people are listening, you just fire the event. Sometimes there is a need to make asynchronous processes behave synchronously. For instance, a stage needs to move, then a measurement must be taken, and that specific measurement needs to be logged to a file. So how does the file logger code get that data?

Possible Solution: The Delacor QMH solves this problem by creating a notifier. That notifier reference is sent to the process, the measurement is sent back on that notifier, and then the notifier is destroyed. I want to do something similar but with events. The "Wait on Event" box shown below would be implemented as a SubVI, whose only purpose is to wait at the event structure for an event to be fired (or timeout). It feels a little wrong at first to be registering and unregistering for an event many times throughout the life of the application, but besides that I can't find anything that is wrong with it. I'm open to any feedback on why this is an OK or a dumb idea. Thanks!

Note: In the real application I want to use user events to communicate from the top loop to the bottom loop as well (they would be in separate VIs), but in this case I used a queue so it would be obvious that the event I am interested in is the one being sent from the bottom loop to the top loop.

[Intaris]

Personally, Event to the consumer, Notifier for the return.

[Gregory]

Hi Intaris,

I'm fine with that. My thought with using the event to return data is that if I already have some code to publish "New Measurement" every time it is ready, then I could re-use that when it's time to return the data. 

Also, if the message handling loop is a state machine, I thought it might get confusing if it has to run a couple states (like Move Stage -> Take Measurement -> Move Stage) and then data has to be sent back on the notifier... where do I store the notifier reference during that time?

I guess I could send the message handling loop two messages:

1. Get Measurement at Location : Move Stage -> Take Measurement -> Move Stage

2. Return Measurement

[Intaris · Accepted solution]

The re-use argument is exactly why I prefer the one-off limited offer lifetime Notifier.

The response of the consumer to a specific request is easy to maintain.  If your consumer receives requests from multiple sources in parallel, the relationship between the return value (broadcast via Event) at any given time and the specific request the particular module may have sent to that response is a LOT more difficult.  This is why I personally much prefer to instigate a private return channel for any request for data I need.  Standard status changes I still broadcast via Events, but these are not context-specific pieces of information.

[Intaris]

To reply to the other part of your question: Where to store the notifier?

In a shift register is the most practical answer.  But the main problem is in the case where multiple requests can be sent in parallel from multiple sources.  Here we now have the need to run perhaps more than one single event case to generate the result, but the intermediate data (return notifier) cannot be overwritten during this process.

In order to allow easy handling of this, for cases where I actually abuse the Event structure as a QMH, I have separate external and internal events.  Once I receive an external event, I disable the registrations for those events until the private event queue is exhausted.  Then and only then do I re-instate the external queue and start processing the next external Event.  Note that this idea works for Events and queues.  It forces atomic execution of individual external commands.

It gets more complicated if you want SEVERAL events to be executed atomically.  Here, instead of sending individual events to your consumer, you can actually send an event registration refnum where the events required have already been registered and called.  Swapping out the entire refnum in the consumer then ensures that the entirety of that event queue is handled before any further external command can be handled.  The external commands are serialised.

[Gregory]

Hi Intaris,

Those are great points, thank you. Right now I am picturing a JKI-style state machine, where all the states on the string are executed first before going back to the event structure and looking for more events to handle. This should be equivalent to handling a private queue before accepting more outside requests.

As for returning data, I don't think it would be too much effort to create an event specifically for returning data. It could happen inside the event structure, and I would be guaranteed to have the fresh data because all of the privately queued up states have to run before going back to the event structure. Just thinking out loud here... but I think I am getting a better picture of what I want the communication to look like. Thanks!

[Ben]

Two ideas I just want to share, something old and something new.

First the New.

Channel Wire >>> Messenger

Will let you use a "Write Ack" node.

The old...

I call it a Self Addressed Stamped Envelope (SASE) and was inspired by the adds in the back of comic books back when I was a kid.

One queue to transfer a request to SOMEWHERE.

Part of the data that is transmitted via that queue is a queue ref to another queue that requestor monitors. When the request is completed the code that completes the request uses the enclosed queue to send a message back to WHATEVER submitted the original request.

It works great and I have actually used this to queue up requests from one machine to another and send the completion message back to the requesting sub-VI on the machine that made the request. It ends up being a way to establish at run-time connections between separate entities. One example was used to let an arbitrary set of graphs to subscribe to arbitrary signal sources. In another case, I used it to let the code that running a location on an assembly line request central robot to move a widget to a location and detect when the widget was in-place.

Edit:

This later ideas is really not that far off from extending the OSI 7-layer model .

Just sharing ideas...

Ben

[mcduff]

I use the JKI State Machine along with Events to communicate loops to loops, send and receive messages. Sometimes I will have a dialog in the State Machine that launches asynchronously, but it connect to Event Structure, just in case anybody hits exit, the dialog/window will get the message.

Look at https://forums.ni.com/t5/Example-Program-Drafts/Message-Bus-Architecture-An-Intro/ta-p/3512280

I have modified this type of architecture extensively. I have an Event Structure loop that listens to all events, if one of the events is needed for the local loop, it sends that message to the loop. The payload is an enum(tells which loop to address), a string command(command for JKI SM), and a variant if any data is available.

This way one event wire can message any loop anyone in the program, along with any loop calling any other loop. Not sure if this answers your question or helps.

mcduff

[Ben]

---

@mcduff wrote:

I use the JKI State Machine along with Events to communicate loops to loops, send and receive messages. Sometimes I will have a dialog in the State Machine that launches asynchronously, but it connect to Event Structure, just in case anybody hits exit, the dialog/window will get the message.

Look at https://forums.ni.com/t5/Example-Program-Drafts/Message-Bus-Architecture-An-Intro/ta-p/3512280

I have modified this type of architecture extensively. I have an Event Structure loop that listens to all events, if one of the events is needed for the local loop, it sends that message to the loop. The payload is an enum(tells which loop to address), a string command(command for JKI SM), and a variant if any data is available.

This way one event wire can message any loop anyone in the program, along with any loop calling any other loop. Not sure if this answers your question or helps.

mcduff

---

I have seen that type of messaging in action and it acts a lot like a bus where the data goes to everything and everything has to decide to act or not to act. For the loops that are not intended to receive a message type (the enum) a "default" case is used to ignore the message. TO figure which loop is supposed to handle the message we have to go through all of the loop slooking for a case that includes the enum value. Tracking down the connections between where it came from and where it is supposed to go is not straight forward unless there is some clear documentation. Imagine 15 or more parallel loops all talking on the same bus....

So if you use that approach and there is slight chance that Ben may be looking at the code, please document the intended connection and spare me the gray grey hair.

Ben

[Gregory]

Hi Ben,

Your self addressed stamped envelope sounds very much like sending the notifier to retrieve data with.

Are you saying Channel Wires are much better than Messengers? I admit I have not played with channel wires much