Helper Loop and cloneable modules

[FabiolaDelaCueva]

---

@Taggart wrote:

---

Genius!

---

I am just Fab 😉 ... and modest!!! 😄 😄 😄

[AlexElb]

@FabiolaDelaCueva wrote:

DQMH takes advantage of FGVs to pass around the references for the user events without seeing the wires. We went for this approach for clones too because we wanted to have the option of sending the same message exactly at the same time to all the clones. 

Fab, thanks for your input about FGVs and AEs. Based on that and together with Jörg thoughts (I also have an ö) about wiring a refnum around I've now implemented the following new approach, which I've attached:

The uninitialized shiftregisters hold two arrays: One array with the refnums of the local events from all clones and one array with all module IDs of running clones.

AE - create

 When creating new events the new Module ID is added to an array. Both arrays have the same size  - grow and shrink simultaniously so the index from the Module IDs Array can be used to search the Event Array.

AE - get

This is used in the get case to retrieve the events of a specific clone.

AE - destroy

When a clone shuts down it passes it's Module ID in the Destroy Case which removes his entries from both arrays and destroys it's local events.

I've build a wrapper VI for each action (create, get, destroy) although it wasn't necessary, but it looks nicer.

Of course that way I still have to pass one extra information (now the Module ID instead of a refnum) in my start & stop helper VI, but I found myself alwas having the Module ID in the MHL in my hand in any case. BTW: I would always add the module id to the module data cluster and set it in the initialize case. This I could also add in an DQMH Template 🙂  Do you do it the same way? When using a broadcast in a cloneabel module I always would like to pass in the module ID anyway....

Do you see any problems with the way the Action Engine is working? Do I need error cases for not finding a Module ID in the array? I don't have an init case. We could use the First? Boolean from the Module Admin for that, but I don't know what should happen in that case. I expect the AE VI to leave memory when all clones has stopped, so it should be okay, shouldn't it? It would be bad if a clones breaks while running and the destroy wrapper is not used...

Before suggesting a new feature request for this I would like to know, if there would be more situations where local events in the DQMH world would be useful, because you mentioned, that we're here only speaking about a corner case. Also may there be downsides? My few thoughts on that point:

• The Create and Destroy cases of the Action Enginge should be easily scriptable by you genius people.
• May be the described Action Enginge would be placed in the DQMH Init Module VI and either have two outputs of refnums - one for local events and one for clone events or put them together. That way the Helper Loop and the EHL could register for local events. In the EHL it should be visible if you are handling a local or a clone wide event.
• The desctroy case could be called within the close Modul VI to ensure the arrays are correct, or whatsoever magically safety LabView experts could think of 🙂
• When creating events with the script maybe there should be option "include creation of EHL and MHL cases for that event". I read in the forum that someone wished to create Helper Loops automatically and complained about the need of deleting the cases for events which were created with the script but then had been changed to private scope to use them as "clone wide local events". That option would do the trick, for both local events that option would be false by default.
• Naming: "local events" vs. "clone wide local events"

Advantages with local events:

• the waiting period of the timeout case of the helper loop from clone A won't get resetet by an event from clone B
• in the helper loop we won't need the addressed to this module VI which improves readability
• one consistent way for exiting loops in DQMH in the stop module case
• we could use local events to send data from helper loops (through the EHL) to the MHL

May be the last point would be of most interest for the community. Now I have to wire the refnum of my Message Queue in the Helper Loop to send data, then I could just drop a VI which seems nicer to me. That would also open the opportunity to call that local event from within any SubVI in the MHL. Reading the Do's and Don'ts one should not self enqueue from within the MHL, but well - the direct option through the Message Queue already exists so that wouldn't open any new dangers I think.

What do you think? Could you think about other advantages of having scripted local events. In what siutaions would they be usefull?

I am glad that I get such a nice feedback from all of you. It makes me happy, that it seems like I am not such a newbie as I thought. I didn't expect that at all after making my first post in the forum.

[FabiolaDelaCueva]

Alex,

I will have time to go in detail over your post later on the week. In the meantime, I suggest you look at the Action Engine that comes with ever cloneable module, it is called Clone Registration.lvlib, more info here

http://delacor.com/documentation/dqmh-html/DQMHSingletonModulevsCloneableMo.html

This library is copied to your DQMH module, so you can edit it all you want and you will not be modifying the framework, just your instance. There you can see when/how Module IDs are added and managed. You can look at where these different VIs are called and you will see that the first clone waits until all the clones leave the memory to destroy the events, so that is a good place to put your destruction of events. 

You have discovered what some call keyed-value pairs or value-name pairs or lookup tables. If you are planning on having hundreds of clones, you might want to look into Variant Look Up Tables, if you go that route, the variant attributes will already give you the output for "found or not found" and you wouldn't have to implement that part of the error handling if an event is not there. 

I will be giving some thought to your post and will get back to you by the end of the week. In the meantime, check out that library and let us know what you find.

---

@AlexElb wrote:

I am glad that I get such a nice feedback from all of you. It makes me happy, that it seems like I am not such a newbie as I thought. I didn't expect that at all after making my first post in the forum.

---

This is one of the advantages of choosing a framework that is continuously maintained by the team that created it and that has a group of LabVIEW programmers actively using it in their projects. I am grateful for the DQMH Trusted Advisors too, you already met some of them in this thread. There are others proficient on the use of DQMH beside the Delacor team. Regarding your level, you are no newbie! There might be some gaps in your knowledge but you are already covering ground taught at the NI Advanced Architectures in LabVIEW course. I am looking forward to having the time to sit down with your approach and your comments.

Thanks for sharing your findings with us,

Fab

[drjdpowell]

---

@AlexElb wrote: What do you think? 

---

It's important to consider readability of code.  You can script away writing difficulties, but you can't script easy reading.  A big part of reading is about "scope": how wide an area do I have to consider in order to understand a portion of a program.  If I look at Fab's "Local Events" design posted above, it takes me only moments to identify that she is using a this-VI-only communication method, and that her "helper loop" can only receive events from the single loop that is just above it.  I do the the latter by following the communication wire.  Note that Fab has clustered her set of User Events, reducing the noise of too-many wires, but has not hidden the wires entirely.  Organised wires are elegant; hidden wires are not.

Your design, on the other hand, is much more time-expensive to read.  I have to identify that you are using an "action engine", a globally-scoped object that can be used anywhere, then understand the "lookup" system and inspect every instance that uses the action engine in order to observe that, actually, you are using your global-with-lookup system to simulate a local-to-Module design, the same thing that Fab did minus the one (most-useful) wire.

[AlexElb]

Thanks to the pointer to variant lookup tables. I've implemented that method instead of the one with two arrays. When it is a common technique and scales way better with large sets of name/value sets, why not use it 😉

Clone Registration.lvlib was interesting to look at how in DQMH a variant lookup table is build for the clonables module ids. I think you would use two separate AEs: "Clone Registration AE.vi" is responsable for the module IDs and "Local Events AE.vi" is responsable for the events associated with one specific clone.

So in "Init Module.vi" I would call "Create Local Events Requests" and in "Close Module.vi" call "Destroy Local Requests Events". Both use "Local Events AE.vi" with the module ids as an input. In "Init Module" use the "is first" boolean to init the variant lookup table if necessary. While closing the module there's no need to wait for the first, since the local events are separate for each clonable.

It would be nice if "Obtain Request Events.vi" would additionally generate the local Events and somehow bundle them together. But I don't invest more time here right now, since its getting too far and the main problem is already solved.

@drjdpowell: I am with you. I wouldn't build that complex method for a module just to start a helper loop, while such easy solutions like channels exists and are even more easy to read. On the other hand that argument looses a bit its strength, when it would be a build in feature in DQMH. I don't have to understand completly how the framework internally works. "Clone Registration AE.vi" is a good example for that. If I read the documentation and it tells me to use this for that, I'm fine with that, as long as it works stable and as expected.
Requests to modules can be called within everywhere in the application. Calls to Local Request Events would be private - and searchable because a vi is used for that. It seems to me like using Local Events Requests would only be consistent to the way DQMH works. But I see you point, complexity and scope would be getting bigger. And as mentioned before I am new to LV and other can evaluate better the pros and cons.

So lets wait and see whats happens 🙂

[drjdpowell]

Yeah, but what's the point?   Fab's design of a local User Event could be a scripted option added to the DQMH.  It has no disadvantages to your design and several advantages.

[FabiolaDelaCueva]

Hi Alex,

I have not forgotten about this post, it has been a little crazy around here and I still want to review in detail your attached code. 

In the meantime, let's talk about global vs local and why we went with FGVs for the User Events. Come with me down the memory lane trip 😉

Why FGVs?

We wanted to use user events because when a user event fires if there is no code registered to listen to the event, no "event queue" is created. LabVIEW dynamically generates a new "event queue" per each registration for events created. This is great, we can have a module that works as a stand-alone, or is called by another module or even called by multiple modules. With a queue, if messages kept getting enqueued and nobody is there to dequeue them, there is a memory leak.

Great! How do we make user events global? Well, we could have used a global but now we needed to control who could create the user events and who owned them. The FGV relies on the VI remaining in memory to keep the value of the events in the shift register. Back when I was working with Justin Goeres on a project that used his private/public events architecture (this is mentioned in the DQMH documentation), we would use a Launcher or Splash screen to create the user events. We quickly realized that as soon as the Launcher or Splash screen would go out of memory, the event references were no longer valid! Even if the modules were already calling the FGV! Turns out the references get cleaned up when the top-level VI that created them goes away. So we decided on the modules to own their references and an FGV allowed us to select "true" for creating the events inside the module itself. 

Also, we wanted DQMH modules to work for different levels of proficiency and abstract away the inter-module communication. This is why we got rid of the event wire and instead it is hidden inside the Request and Broadcast VIs. This is a double edge sword because we remove a level of "immediacy" from the block diagram. You can no longer at a glance see that there is this global communication. We needed to train everyone on the concept of DQMH Request Events and DQMH Broadcast Events. With the forthcoming DQMH 4.2 release, we will be adding a glyph to the DQMH Broadcast events, this will help some with bringing back some of the removed "immediacy".

Local Queue

Other frameworks use the Queue as their inter-module communication mechanism. I already explained why we prefer events. We kept the Queueu as private to the module and you can see the DQMH Q wire going around the Main.vi. It is clear that the DQMH Q is local and only used within the DQMH Main.vi

Local Events

Local events for the helper loop feels different than the DQMH events. You don't need to access these events globally, only the current module cares about it. You don't want to let anyone access that event, even by mistake. Seems to me that these local events would be closer to the Local Queue than they are to the DQMH events. As such, I am leaning towards preferring the wire to be shown and to not use an FGV. We will continue to discuss these options as we look into implementing some Helper Loop scripting. 

By the way, if you have not done so already, you can vote for DQMH feature requests by adding kudos to the different requests here

https://forums.ni.com/t5/Delacor-Toolkits-Documents/DQMH-Feature-Requests/ta-p/3537845

If you feel that your request is not there, you can add a new comment and point to this post or a new post on the discussion of your feature request.

Regards,

Fab

[Jens_S]

---

@FabiolaDelaCueva wrote:

---

@AlexElb wrote:

first of all thank Delacor for offering the DQMH Framework. I started developing with Labview half a year ago and found your framework two months ago. Until now it works like a charm and I'm very happy with it. From time to time I found myself wondering how things work out, but mostly I found something online, e.g. joergs blogpost about helper loops which helped me a lot. Now to my question.

---

Hi Alex,

Glad to know that DQMH is working out for you. 

---

@AlexElb wrote:

Now what I would do is to add the moduleID in the private requests. In the helper loop  I would then create a case structure and use the Addressed to This Module.vi like it is used in the EHL. But wouldn't it be much nicer, if the event in clone B doesn't get fired at all, when clone A is doing something?

---

If using the Addressed to This Module.vi is not enough for your case, meaning you are waking up and putting to sleep the helper loop events at different times and your code cannot cope with the disruption, then you can create a local event then. Instead of relying on the Private wake-up helper loop and set to sleep helper loop, create a local pair of events. To do this, you can create your events directly in the DQMH Main.vi. The picture below shows the creation of the local helper loop events, the MHL firing the Wake up Helper Loop.vi and the destruction of the local events outside the event handler loop. Note that you have to pass along the user event references because these events only exist in this block diagram. 

The code inside the Create Local Request Events.vi, Destroy Local Request Events.vi and Wake up Helper Loop.vi is in the image below together with how I would structure the code inside the cloneable module library.

I hope this helps.

Regards,

Fab

---

Thanks for this solution! Was running into the same problem with Helper Loops and clonable modules. I didn't realized that creating a User Event by handy would be a "local" user event for each clone, thanks for this.

[doyles]

Our team has been doing more and more cloneable modules with helper loops.  However we aren't 100% positive on what the correct reentrancy setting should be for the local events (create, destroy, and the event VIs themselves) within a cloneable module when each module instance must maintain it's own helper loop timing (i.e., not have the helper loop timeout interrupted from other modules).

[Darren]

As a general rule, if a VI in a cloneable DQMH module doesn't maintain state, it should be reentrant.