Generalized Functional Global (GFG)

[X.]

All right, due some technicalities of the LabVIEW Idea Exchange forum, I am reposting this idea with the attachment I was planning to add in part 2 attached to the first part (why is it possible to insert images in a comment but not an attachment is beyond my understanding). There will still be 3 parts to it (assuming each of them fits in the maximum allocated size per post), plus an appendix.

This post is intended to present a solution to a problem that I was struggling with using LV2 global variables in my applications.

I don't claim that this is a unique or elegant solution or even the first of its kind, but I'd be curious to hear comments and suggestions about it.

Since it might be a long post, I will split it into 3 parts. Part 1 will discuss the intended functionalities. Part 2 will present my current implementation. Finally, I will try to summarize all this into Part 3, opening the discussion.

So you may want to wait until that part is published before posting your comments (if any).

Part 1: What do I mean by Generalized (LV2) Global Variable?

The LV2 global variable (or functional global, FG in short) is a well known structure (see for instance the beginning of this thread). It is a subVI which can store data locally in between calls. Usually it comes with a "Write" (or "Set") action and a "Read" (or "Get") action and a few inputs (and the same number of outputs). The inputs are the variables you want to update and the outputs, their corresponding values. The advantage of this design over the standard LV global is that there is only one copy of your variables in memory. If you are only using scalars or string, etc., using LV globals might not necessarily be a big problem, but if you start storing arrays of large data structure, it could result in a significant performance penalty. Note that there are other ways to pass data from VIs to VIs (queues and notifiers). Here, I am concerned with long term storage available from many parts of an architecture (e.g. subVIs or dynamically launched Vis).

To begin with, there are two major limitations with any design based on FG (at least that I am not happy with):

1) First, as emphasized above, due to the limited connectivity of a VI's connector pane, a FG cannot store a large number of variables. You can use clusters, but that is not always very practical.

2) Second, if you try to cramp as many variables in a single FG, you will probably run into the issue that you don't necessarily want to reset all variables at once, but maybe just one or two. That calls for a more sophisticated set of actions (Write Variable 1, Write Variable 2, etc, and possibly Write All, Clear All, etc).

In practice, if you use a lot of those FG, you will encounter a third problem:

3) In which FG did I store this @$%&! variable?

Some of my applications contain many of these FGs and I figured that this had become impractical to handle when I started duplicating them, having forgotten that I was handling variable "X" in an existing FG.

The obvious solution (apart from NOT using FGs) is to have a single FG that is designed such as to handle an unlimited number of variables of ANY type that can be modified at ANY TIME from ANYWHERE.

 I first looked at the WORM (Write Once Read Many) design of tbob (you’ve got to go to the end of the thread to download the final version of the WORM VI). It is a very clever and compact solution to store different types of variables within a single variant.

Two of the limitations I saw in this design are that you need to know:

1) the NAME of the variable you want to access.

2) the TYPE of the variable that you are WRITING or READING.

Let me clarify those two points.

It seems obvious that you HAVE TO know the name of the variable you are interested in. Well, that’s maybe true when you are designing your application. But after a month or more doing other things, it is not obvious anymore whether you’ve stored the number of components as “# Components” or “Component #” in that other part of the program that…where did I save it, BTW? You get my point…

The second point is apparently taken care of by tbob’s solution of outputting the variable (as a variant) as well as its type. The problem is that this “type” provides a very limited description. For instance, what do you do with a “cluster” type?

Finally, since I want to be able to modify any variable at any time, the “Write Once” feature is not cutting it for me. This is could be easily modified, but considering the previous comments, I decided to change the architecture some more. I will describe my solution in the next part.

[tst]

You haven't continued your posts, but I might as well interject:

> The advantage of this design over the standard LV global is that there is only one copy of your variables in memory.

This is a common misconception. If you only use the FGV to store the value, it's not any better than a global. You have to actually add functionality inside the FGV (hence the F) in order to have a single copy of the data. If you don't, you might as well go with the globals which satisfy your requirement.

Also, in 2009 you have Data Value References which can satisfy both the single-copy condition and the race-condition condition (which you didn't mention), but they work in the same way as queues in the sense that you have to obtain a reference to them.

[AristosQueue (NI)]

Tst is correct.

X-) wrote:

"Usually it comes with a "Write" (or "Set") action and a "Read" (or "Get") action and a few inputs (and the same number of outputs). The inputs are the variables you want to update and the outputs, their corresponding values."

The LV2-style global only saves data copies if there is no Write and no Read operations, only other actions to be performed by the LV2-style global. This is why the term "action engine" is preferred -- well designed instances of this pattern do not behave like variables. They behave like state machines that transition to various other states and do work on behalf of the caller.

[X.]

Part 2: Implementation of a Generalized FG

Based on the discussion of part 1 and on my limited knowledge of LV (I have only been working with it for 15 years as an academic scientist), I figured that the easiest way FOR ME to remember the name and type of my global variable was to create an enumerated type definition containing all my variable NAMEs and their types:

Variable 1 [Type 1]

Variable 2 [Type 2]

…

Variable n [Type n]

Eventually, I figured that it would be clearer and therefore more convenient to have some sort of indentation and hierarchical features in this list, so my typedef enum comprises items such as headings and sub-headings:

--- Section 1 ---

     --- Section 1.1 ---

     Variable 1 [Type 1]

     Variable 2 [Type 2]

…

--- Section m ---

     Variable n [Type n]

…

The advantage of a typedef enum is that I have a handy list of ALL my variables anywhere in my software. The Type information in the name of the variable also lets me easily figure a simple way to cast the variant I receive into the appropriate type, using the “Variant to Data” function, as illustrated here (I have even introduced a unit in this specific example):

 

If the type is a bit complex, I designed a polymorphic FG data access to get the variable directly cast into its type, as illustrated here:

 

Instances for complex types are the only one really useful, but you can of course create instances for all types if you have the patience…

Otherwise, if I have a control/indicator for that type of variable on my panel, I can of course just get the variant representation of this variable and cast it using the “Variant to Data” function connected to my control/indicator (or a local variable), as illustrated here:

Let’s take a look at the architecture (see attached zip file, released for LV 2010).

Basically, it comprises:

- a “Global Enum” typedef enum listing all the variables as described before.

- a “Global Core.vi” to write and read any variant by enum.

- a polymorphic “Global Variable.vi” to read complex types (and not so complex ones if needed). Note that this is not mandatory. An alternative is described in the Disclaimer section.

One thing that the “Global Core.vi” provides, which I haven’t mentioned yet, but makes it so practical in my opinion, is as shown below, a “Source” input and a “Global Event”output:

 

The MANDATORY Source input tells which VI is modifying the global variable. It is ignored for a read, so you can just connect an empty string.

The Global Event output is a refnum to a User Event generated inside Global Core.vi, which each subVI or VI using the GFG should register to when it starts. Each time the GFG is updated, it fires an event comprising:

- the Source (modifying VI)

- the Variable Name (typedef enum discussed before)

- the Data (the modified variable as a variant)

This feature is useful in that it tells every subVI that is interested in this information that one of the stored global variables has been modified (which and by whom). Each such subVI needs to register to the User Event and filter those events as it sees fit. For instance, my Main VI, which displays only 3 of the variables (and allows modifying one), only reacts when one of those 3 variables is modified. It DOES NOT react if the modifier is itself (at least I made sure it does not)…

This, in my opinion, covers the main functionalities required from a GFG. Try running Main VI.vi and launching the “Settings” SubVI (Secondary VI.vi) and play with the controls to see it in action.

Notes:

- The “Global Variable Name [Stripped Type].vi” is used to write and read ini files using names not containing the useless “[Data Type]” part of the string).

- The “Multiple Global Variable Access.vi” is helpful when several variables need to be access at the same location.

Disclaimer:

This architecture is not without defaults or imperfections. Here as the main ones I can see (after some practice):

- updating the list of global variables requires manual fixing the Global Core.vi (see appendix below “Fixing Global Core.vi when updating Global Enum.vi”)

- deleting or inserting a global variable in the typedef enum might break some of the subVIs registering the User Event in a subtle way, by messing up the case orders in the case structure (such as the one shown in the figures above).

- if you want to access a complex variable type, you either have to create an instance for the polymorphic VI or copyi a typedef constant on your diagram to use with the “Variant to Data” function.

The next part will discuss the lessons drawn from this architecture and open the discussion (if anybody cares to comment).

[X.]

Part 3: Discussion on the GFG architecture

Again, I am no LabVIEW expert. In particular, I suspect that more elegant/efficient/you-name-it approaches might be able to provide the same functionalities. I’d be curious to see how LVOOP may help in this respect.

The discussion is open but I would like to comment myself on what I wished I could have incorporated in this design.

1. A simpler way to fix the broken User Event refnum (see the appendix below).

2. An easier way to create enums with tabulation and styles. I would have loved to tab-justify my headings and subheadings and use bold and italic styles rather than using “----“ and multiple spaces.

3. An easier way to handle subtly "broken” case structures. As discussed briefly, if you change the item order in the Global Enum (or insert items), the cases that you may have so carefully designed to handle different variables may not correspond to the correct variables anymore. That’s the intended LV behavior, but I think that LV should be able to handle case structures linked to a typedef enum in a cleverer way (maybe providing an option for those case structures to be “Connect to Typedef”?). In particular, I'd love that, when you reorder items in a typedef enum, the “Move Up” or “Move Down” actions would be tracked by the connected case structures and cases rearranged accordingly. Likewise if you “Insert” or “Delete” an item in the typedef enum.

In the “Insert” case, I would have the case structure “break” and report a “Missing case”. In the “Delete” one, I would disable the case or at least “break” the case structure (turn the case red as currently happening) and let the user decide on what to do.

[X.]

Edit: Post removed by moderator at the request of the idea author. It is no longer relevant after corrections to the original idea.

[X.]

Due to some technical "features" of the LabVIEW Idea Exchange forum, I have to repost this thread elsewhere.

But, tst, I am not sure I understand the nature of the misconception.

I understand that if you access your FG in a subVI, a copy of the data is generated. But if this VI access the FG, say, in a case structure, the copy will be generated if and only if the case is executed. My understanding was that each and every time a global variable is modified, each and every copy of it in each of the VIs in memory is updated (including copies in different mutually exclusive cases).

Using Data Value Reference is helpful (memory wise) only if you use the In Place Element structure to MODIFY the stored variable. However, if you want to use it elsewhere, it seems that a copy of the data is created (check the buffer allocation at the Read node in the In Place Element structure below):

[JonJay]

Just one question... since this is the idea exchange and not the the general forums where is the suggestion to NI in all of this?

[X.]

DukeGriffinm, sorry, that's a good point.

First, there are in fact LabVIEW feature suggestions in the last part, but these are secondary in my opinion.

Second, it's not really a request for help on a specific point, but an attempt to discuss how best to implement a functionality. I thought that this forum was an "Idea Exchange", not just a "Feature Suggestion" forum.

Finally, I was ALSO hoping to raise the interest of some NI developers, so as to simplify our job as programmers. But I may be overlooking some simpler approach, so I did not want to formulate it as a "Feature Suggestion".

[AristosQueue (NI)]

The idea to NI, as I read it, is to "do something to make functional variables easier to build/use." I think we can work within that as this discussion continues and refines itself. This idea itself might never become a feature that NI buys, but others may evolve out of it.

[X.]

Fair enough...

BTW, this thread is some kind of extension of an answer I gave to Joe Jasniewski's "LabVIEW computer science or just self-torture?" post in the info-labview mailing list (in which you cannot post attachment or figures). He was the one to raise this general issue of how to best handle "global variables" in a large application and I just happened to have worked on that idea for a recent rewrite of one of my software. Adam Kemp chipped in with some LVOOP concepts which I still don't quite understand how to use in this particular situation. I hope I will get an answer, after having detailed my own requirements and solution.