Design Pattern using queues and notifiers and problem with syncing

[Gregory]

Hi Karen, there is a lot of cleanup necessary before this code is readable. Please do not take this personally, I, and almost everyone else who has navigated the LabVIEW learning curve has created code like this - spaghetti code. The only thing this code does not have a lot of is comments - which will be very helpful even to yourself if you ever come back to it after more than a couple days break.

What hits me first is the references, there are at least 60 of them and probably are not needed since I see a lot of "value" property nodes. Wire to the actual terminal, or to a local variable is much preferred over using a "value" node. If you do have a good reason for having a lot of references, it would probably be best to store them in a typedef cluster (and possibly inside an action engine), but I think the issue here is that you do not need them.

Next to avoid is the stacked sequence structure (and flat sequence is not much better). A state machine is a much more expandable and robust architecture for things that you need to make occur in a certain sequence. You can fit a lot of information in a state machine, which will help compress your block diagram to the recommended "single screen".

I'm sorry if all this seems like a sidetrack when your code feels so close to working, but when you have so many wires and such a large diagram, it is nearly impossible to debug. If you have the resources, (time, patience) I would recommend starting over with one of the included NI project templates. Then many people here will be happy to answer questions like:

• How do I initialize my controls using the NI state machine?
• How do I process data in parallel with the NI QMH?
• How do I update the user interface in a parallel loop?

[Bob_Schor]

Gregory is much more patient than I am -- I almost had a panic attack when I saw a Block Diagram with almost no sub-VIs, one that conservatively was at least 8000 x 6000 pixels in size, requiring more than 50 standard (1280 x 1024) monitors to view, with (unlabelled) wires running everywhere, Stacked Frame Sequence structures, etc.

The single most influential LabVIEW book I read (which, fortunately, I did during my initial year with LabVIEW) was Peter Blume's "The LabVIEW Style Book".  It doesn't purport to teach you LabVIEW (though it does do some of that), but to teach you good LabVIEW style.

When I started with LabVIEW, I was handed a LabVIEW Real-Time project (lower-case "p", as this was LabVIEW 7, before the LabVIEW Project was introduced).  Many VIs were bigger than a single screen -- my biggest was about 10,000 x 2000 pixels, but there were a few sub-VIs (actually, about 800 of them, not all of them very useful).

After spending a few years "patching" this code without being able to make much of a dent in its performance or "ability to change" to new requirements, I finally decided to "Start Over".  It took about 6 months, but I now have code that (a) runs more than twice as fast, (b) saves a lot more information so that we can completely reconstruct the system being studied, (c) is largely "self-documenting" (as every VI has documentation, every VI has an Icon that shows what it does), and (d) every VI fits on a single standard monitor, so it is easy to view, easy to debug, easy to explain to a colleague what's going on.  I credit Peter and his book with developing my "Style" ability, and now am teaching it to my colleagues and students.

Bob Schor

[billko]

---

@Gregory wrote:

Hi Karen, there is a lot of cleanup necessary before this code is readable. Please do not take this personally, I, and almost everyone else who has navigated the LabVIEW learning curve has created code like this - spaghetti code. The only thing this code does not have a lot of is comments - which will be very helpful even to yourself if you ever come back to it after more than a couple days break.

What hits me first is the references, there are at least 60 of them and probably are not needed since I see a lot of "value" property nodes. Wire to the actual terminal, or to a local variable is much preferred over using a "value" node. If you do have a good reason for having a lot of references, it would probably be best to store them in a typedef cluster (and possibly inside an action engine), but I think the issue here is that you do not need them.

Next to avoid is the stacked sequence structure (and flat sequence is not much better). A state machine is a much more expandable and robust architecture for things that you need to make occur in a certain sequence. You can fit a lot of information in a state machine, which will help compress your block diagram to the recommended "single screen".

I'm sorry if all this seems like a sidetrack when your code feels so close to working, but when you have so many wires and such a large diagram, it is nearly impossible to debug. If you have the resources, (time, patience) I would recommend starting over with one of the included NI project templates. Then many people here will be happy to answer questions like:

• How do I initialize my controls using the NI state machine?
• How do I process data in parallel with the NI QMH?
• How do I update the user interface in a parallel loop?

---

I think of spaghetti code this way:

It's like writing text code with random indents.  The computer doesn't care, but if the next developer wanted to straggle you, I would look the other way.

I think of no subVIs this way:

It's like dumping all your code into main().

And of course:

If you use the cleanup button and your code looks better, you probably need to rethink your coding style.

[KarenM]

Hi,

So I suppose an explanation is in order... and I am hoping rather than debugging my code based on the VI snippet, I can describe the code sufficiently and we can all remove the wires as the distraction and somehow be able to find the architecture out of this frazzled coding. I apologize for my lack of organization of this code -- this code is not mine, and was clearly made for another architecture - nevertheless I tried to refactor the code in the fastest way and this is what I get so far --

I am aware of these pitfalls and will indeed correct them in due time - this is a test version so please ask any questions about hte code and I can answer them.... 

It already took a good amount of time to separate and rewire all the coding into these loops. I apologize for the frazzled coding and please bear in mind that this is a test version of the code.  I'm sure you all will understand the extent of the time it will take to refactor it, and that also requires knowing which architecture to use in order to refactor properly....

Between deciding on encapsulation and the time it would take to directly move the code, this is the result that I have, and I suppose all I have left is to determine how to architecture this...

I do want to clean up the code but I don't think I can do so at this point...sorry..

----------------------------

- so I do not have the consumer loops within a loop

- would placing all consumer loops within a loop be the parallelization I am looking for? I am assuming each program has its own optimal design architecture..

----------------------------

I sincerely appreciate all your help and patience. Thank you. 🙂

[Bob_Schor]

Aha, now I think I hear the question!

When I code LabVIEW, I tend to always use Error In/Error Out on the lower corners, and string VIs together by putting them on an Error Line.  This is serial coding, because VI #2 won't begin until VI #1 finishes and puts data on its Error Out to go to #2's Error In.

If, on the other hand, I have VIs arranged vertically, with the same Error Line branching and going into each of them, then these VI's will run in parallel (providing that they aren't sharing any other resource except for the Error In line).  They will each run at their own speed, "as fast as they can and the Scheduler lets them" unless they have a Wait or some other "blocking" function (like a DeQueue) making them wait.

Typically, you don't have just a single VI "running in parallel", but a series of While Loops, all with the same Error In, but no other dependencies on each other.  They will all run asynchronously, with LabVIEW giving each of them CPU cycles (unless they are "waiting"), but, in effect, they are running "in parallel".  Indeed, on a multi-core machine, they might actually be in different cores and really running in parallel.

Another way to have VIs run in parallel (which I tend to use, as I don't like messy block diagrams) is to "spawn" the sub-VI using Start Asynchronous Call.  This really runs in parallel, since once you start it and give it its inputs, it runs more-or-less "disconnected" from the rest of your code.

So if you want Parallel, make sure that your loops aren't "dependent" on each other -- there can be common input wires, but except for that, you should be able to draw a line between them that no wire crosses.  I would also not put a "parallel loop" inside another loop (unless you gave me a really good reason to do so ...).

Bob Schor

[KarenM]

Thanks for the reply!

Main consumer loop aside, each of consumer loops (the loops corresponding to each 4 channel) is identical and shares 5 input variables, has custom input variables and queues their respective queue, as well as queues a common 2-D data array queue. I wired the error wires preceding each of the loops to a 'merge errors' function and wired them into the inputs for both the command and data queues inside each consumer loop.

[KarenM]

Hi,

I have applied the parallel wiring of error outputs from each of the queues. Please see attached for the screenshot. 

The consumer loop execution duration is still at ~4s, when it should be at 1s. I also wired the errors at only one queue per loop. Also I checked the inputs and each loop shares common inputs which from previous explanations is permissible.

 I am looking into the 'start asynchronous call' but that would require reprogramming which would ideally be avoided if the parallel wiring would work. Your (any) advice would be appreciated! Thank you for the help thus far!

Program description:

Once again there are 6 loops in the program:

Top loop: Producer loop (should run as fast as possible or at 1 cycle/s)

2nd loop: commands consumer loop (should run as fast as possible)

3rd loop onwards: channel 1 to 4 loops (should run at 1 cycle/s)

[crossrulz]

Please attach your actual VI(s).  This screen shot is too large to properly analyze and the built-in snippet tool makes things even worse.

[Bob_Schor]

Was the PNG that you posted supposed to be a LabVIEW Snippet of your code?  When I tried to expand it, it "blew up" to a HUGE Block Diagram that was 12 "monitors" wide and 40 monitors tall, impossible to look at.

Can you please post the VI, itself?  It may also be too big, but then we'll at least know we are looking at "the real item".  If, indeed, you have a VI Monster on your hands, you are going to need to make some difficult (and potentially painful) decisions about how to proceed.

While the Diagram was trying to load, it was calling up the "missing" functions that I, of course, don't have.  I noticed that it seemed to be looking at a very complex Directory Structure on your disk.  Are you using LabVIEW Project to organize this code?  Do you have all of the VIs grouped together into a relatively-compact set of folders (maybe a dozen or so, with a maximum folder depth of maybe 3)?  Are you using a Version Control System to guarantee that you have only the current Working Copy of the code under development on your disk?

Bob (Waiting for the VIs) Schor

[KarenM]

Hi, 

Thanks for the feedback. Is it possible to simply view as a PNG? I had marked the PNG just to show what coding I had modified, and the essentially what I had wanted to show are: 

- wired error outputs from all queues to a 'merge error' operator 

- wired 'merged errors' to each consumer loop

- each consumer loop has two queues (data and user input queue). The same 'merged errors' wire is inputted into the dequeue function within the consumer loops

Thanks! Hopefully if this isn't loaded as a snippet then you can just view the image itself. You can even open it in paint. Thanks again.