queue distribution

[labman04]

Hi all! 

I am building a system that dynamically launches multiple test stations (up to 4).  The sensors all have to be initialized when the main application is launched (3 modules on a SCXI chassis, 2 PCI devices all synchronized on an RTSI line). 

So these sensors are always collecting data even if the data is unused.  I am using lossy enqueue to store the data.  Additionally I am running a "Parse" vi which reads the queue and redistributes the data to 4 "Fixture" queues.

The data in each of the fixture queues needs to be asynchronously available to 4 independant programs: 

1. Display (the user interface)

2. Failure Monitor (checks for any condition that will damage a device)

3. Report Generator (logs data at a prescribed interval)

4. Test controller (Provides automated test sequencing)

I'm noticing some serious lag (likely because I am copying the queue data so many times). 

Ultimately the questions are:

How can I get the data being produced from the sensors to each fixture independantly and still distribute the data to additional programs without creating queues for each program explicitly?

Am I going to have to do some serious research into actor frameworks?

Is there another way to approach this that I'm not thinking about?

I've attached the project as I have developed it so far. 

Notes:

I am well aware that I have some housekeeping to do w/r/t file and VIT paths. 

I created my own version of the Queued Message Handler and created a library that may or may not be recognzed by different computers.

[labman04]

In case you need it to make the program functional, I have attached the queued message handler library I created.

Also, I'm currently developing with Simulated cards:

1 x PCI-6602

1 x 6220 (scxi control module)

1 x SCXI-1112

1 x SCXI-1104C

1 x SCXI-1121

1 x PCI-6230

 The meat and potatoes of the issue are found in:

Parse Fixture Data.vit

Sensors.vit

[Bob_Schor]

We've done something similar, but used a slightly different approach, which should be adaptable to your situation.  Here is how we organized the Project:

1. Start by designing a single "Station" from soup-to-nuts.  Think about how you need to interact with this process, and how the data needs to "flow".
2. I'm assuming that you have a "data producer" that is creating data at a fixed rate, and you need to "do something" with this data.  For the sake of simplicity, I'm going to assume you don't turn data collection on and off, but that it is "always on".
3. Are you hoping to save all of the data?  I'm a little curious why you mentioned using Lossy Queues, as these are ideal for "losing" data.  [Confession -- we also used lossy queues, but we did this for "efficiency" and took pains to make the Queue sizes "large enough".  We also put checks on the Queues and threw warnings and errors if the Queues got too full].
4. Figure out what you want to do with the data.  Note that this may well lead to "data-processing" queues.  For example, if you are saving the data coming in at 1 KHz and also plotting "average data" at, say, 20 Hz (averaging every 50 points), you'd put arrays of 50 points (after they've been written to disk) into another (possibly lossy, size 5) Queue and send to the Plot routine, which would plot the average of each Array as they were received.
5. The Single Station VI would get called with input parameters necessary to start it off appropriately, including the name/number of the sensors it was "watching" and a Station Identifier (for us, an ordinal number, e.g. Station 1, 2, 3, etc.) that could be used to label, for example, the data files being produced.  To stop the Station, we used a VIG (a.k.a. a Functional Global) that the Station could monitor for the Stop signal.

This design meant that we could start the Station running asynchronously and it would "do its thing" until the VIG said "Stop".  We did not use VIT's (they were used in Version 0.1, which controlled 4 stations) -- instead, we used the Start Asynchronous Call VIs (and ended up controlling 24 stations).

Since we were monitoring 24 stations, we really didn't want to have 24 plots (and 24 video feeds).  We ended up creating another VIG that held the "Active Station" -- each Station (which knew its own number because we passed that in as one of the Input Parameters) would see if it was chosen to display and if so, it put data on the Plot and Video Queues.  This means that the Main Front Panel could simply start all 24 stations and then (via a slider, I think we used) allow us to monitor each Station one at a time.

What about errors?  [Need to think about errors ...].  We designed the Single Station in such a way that if an error occurred, it shut itself down after sending a Message (yes, we also used something like a QMH) to the Main Routine which was in charge of logging all of the Errors from all of the Stations.

I've since developed other routines that used multiple parallel processes and find that the Start Asynchronous Calls work really well for me.  I find it much more straight-forward than using VI Server and VITs for asynchrounous programming.

Bob Schor

[labman04]

Bob,

That's pretty much what I did, with a couple of exceptions:

1. Sensor Sampling is done continuously regardless of whether or not a single station is running.  - this is the first reason for the lossy queue.

2. Sensor Parsing is done continuously regardless of whether or not a single station is running.  - this is the first reason for the lossy queue.

3. I'm using VITs and a Queued Message Handler so that I could create a modular and interactive program.

         a. As an aside - I have proper exit routines for each of my VITs and sub VITs, but they don't always seem to catch the Exit commands from upstream unless I seriously slow down the interval between exit commands. (~1s)

4.  I have a front panel that controls launching and removing of the statations - > each station launches its own SubVIT (Report, Failure Monitor, Test) while also processing display data. 

5.  The Main VI uses a subpanel that links to a running Station selected from a listbox by the user,  (this allows independant display), but the user also has an "Undock" option which will open the front panel of all running station displays.

I'm at the point where I need to develop each of the downstream modules (Report, Failure, Test), and I'm realizing I need queues for each of these modules as well, and these queues are getting a bit difficult to manage, and parsing them at the collection point seems to be slowing the process down. 

It doesn't help that I'm also being asked to modify what I already have to allow for sensor re-assignment....  (eg, each station has 2 thermocouples, but instead maintain 8 thermocouples separately so that you can attach all 8 to one UUT and disable the other 3 stations - or better yet leave them enabled but don't allow them to collect thermocouple data since those sensors are in use elsewhere)/

[Bob_Schor]

---

@labman04 wrote:

3. I'm using VITs and a Queued Message Handler so that I could create a modular and interactive program.

         a. As an aside - I have proper exit routines for each of my VITs and sub VITs, but they don't always seem to catch the Exit commands from upstream unless I seriously slow down the interval between exit commands. (~1s)

---

That's the virtue of a VIG/Functional Global.  I just cobbled one up that I called Stop Station that has an Enum with three values:  Get (default), which returns the "Stop" value for a numbered Station (Station numbers are assumed to run from 0 to N-1); Set, which sets the numbered Station to Stop; and Init, which is called by the Main (first) with the number of Stations (N) and generates the "all not stopped" Initial State.

The virtue of this approach is there is a single Stop Station VIG that the Main and all of the Stations use.  It is very fast (a simple Array lookup) and can stop any/all routines in any Station (all you need to know is the Station number).  Because the Get Action is set as default, you can leave it unwired in the "use" case where its output is wired to, say, the Stop indicator of a While loop.  Finally, notice that "the Error Line Runs Through It" -- you'll want to place this on the Error Line just before you exit the While loop you are contemplating stopping so that it is checked at the end of the loop.

Bob Schor

[labman04]

Well, that makes sense for resolving the exit routine.  I have a stop FGV that pretty much does this (although not for independant stations - and I would modify it because the goal is to be able to launch, remove, and re-launch stations).  But I still have the queue issue. 

I think I'm just going to modify what I have - Read sensor data to queue - dequeue and parse to station queues - then have the stations create notifiers, reading the station queue and sending notifiers to the support programs.  Now I just have to figure out sequencing (obviously failure mode is the highest priority so it should get the data first, but won't sending out notifiers to the other programs create the same lag issue?)

Either way, back to the drawing board.