Independent instances of filter

[EMCCi]

Hello,

I have a 2D array where every row is a different measured variable, and every row "is a 1D array" of 100 values. I receive a 2D array every second from variables measured at 100 Hz. So every column is a different "timestamp": xx:xx:xx.000, xx:xx:xx.010, xx:xx:xx.020...

The point is that I need to apply a filter to each variable. It would be very easy to index the 2D array inside a for loop, and apply the filter to the 1D array, and then index it again:But the problem here is that there aren't independent filter internal states for each variable, but each variable takes the internal state of the previous one. What I want would be one independant instance of the filter VI for each variable, in order that in the following global iteration each variable can continue the filter from its own internal states.

So there is some way to do this, without placing a filter VI for each row?

I have found this, but my experience calling by reference is next to nothing, and I don't know if this works, or if this will give me a good performance.

Thanks for your time,

EMCCi

[Bob_Schor]

Well, there's Good News, and there's Bad News.  The Good News is that you can, indeed, do this, and it should be pretty efficient (perhaps running as fast or even faster than doing it in a conventional For Loop), and the Bad News (which could turn into Good News) is that it is "New Stuff" to learn.

What I think would work and be pretty efficient is to create a sub-VI that takes a single channel and does the filtering.  You want to give it two Queues, a Data In Queue and a Filtered Data Out Queue (you'll see why in just a second).  Have you worked with Queues?  If not, look at the Help and Examples.  You can have (maybe) one more input, a Cluster representing the parameters of the Filter (or you can just make it a Constant of the Filter, as shown in your example).

Here's how you'd do this:

• Create this VI and name it "Asynchronous Filter".  [The name is significant!].  You are going to "launch" an instance of this Filter as an Asynchronous Clone, one per channel, and create two Queues per channel to handle the inputs to and outputs from the Filters.  The Filter needs to have certain properties so it recognizes when it is being called for the first time, knows how to "stay alive", and know when/how to "gracefully exit".  I'll describe briefly later.
• In a For Loop where N = number of Channels, create the two Queues, create an Asynchronous Clone of your Asynchronous Filter, and start it with Start Asynchronous Call, wiring in the two Clones.  Output the two Queue Arrays -- these are your Communication Paths to your Filters.
• To use these filters, take your For Loop and pass it in the (2D) Array of Data, and the 1D Input Queue Array.  Enqueue each Channel into its own Queue.  What you've just done is to load your N filters with data (essentially simultaneously) and told them to (simultaneously) filter.  Bring the Error Line out of this For Loop and into the next For Loop that you pass in the 1D Filtered Output Queue Array, and dequeue the Results -- when this loop finishes, you'll have all of your N channels filtered with a Filter dedicated to that channel.

When you build your Asynchronous Filter, build it as a While Loop, which will give you something in which to "mount" a Shift Register.  Wire a Boolean True to the output (right-hand, inside-the-loop) edge of the Shift Register, which turns the input edge into a Boolean, default False, which you wire to the Init/Continue terminal of your Filter (so it initializes once).  Inside the Loop, dequeue your Array of Data (this will "block" until you Enqueue an Array in your main program.  Now here's an important step -- test for an Empty Array, and wire the output of the Test to the While Loop's "Stop" Terminal.  This is how you "kill the Clone" -- in your Main program, when you are ready to exit, you run the Filter one last time, sending all the Clones empty arrays (which cause them to exit their While Loops).  Also wire the "Empty Array" output to a Case Statement, and if the array is not empty (i.e. You Have Data), filter it and enqueue it onto the Filter Out Queue, and you are done until you get more data.  Also, when you do finally exit the While Loop, go ahead and Release both Queues -- it is safe to do this, as the Caller sent the "I'm All Done" (or Sentinel) signal to you, which is a "promise" that it doesn't need the Queue any more.

And here's a picture of how to start this process from a routine of mine.  I only have a single Queue, and I'm (in fact) using a Reference to the Asynchronous Clone (I call routines that are detached to run asynchronously a "clone"), but you probably don't need to do this.

Create Clone (with one Queue)

The initial code is creating a unique name for the (single) Queue (my Queue is of a Cluster called an S-Msg, yours will be a Queue of Waveform, or 1D Array of Dbl).  I get a reference to my Clone (CLONE Station) using Static VI Reference -- note the Star on this function, which means I right-clicked and chose Strictly Typed VI Reference (necessary!).  I get a Path to this Clone which I wire to the Type Specifier of Open VI Reference, along with the Path and a hex 140 (the 40 means I'm using a reentrant Clone, which I forgot to mention, the 100 means "Call and Collect" -- you'll probably want to use 80, "Call and Forget", so you'll probably use C0).  Note that I pass the Queue as an argument to the Clone; you'll be creating (and passing) two Queues, and also saving them so you can use them in the caller.

OK, reentrancy.  You need your Asynchronous Filter to "preserve its state" so that you can have N of them, each "remembering" when it was last called.  Once you build your Filter, right-click and choose VI Properties, go to Execution, and choose Pre-Allocated Clone Reentrant Execution.  If you look closely at the picture showing the Icon for the Clone (in the Static VI Reference with the red Star), you might see a lower-case "r" in red just to the right of CLONE -- that's a reminder to me that this is a Reentrant Clone.

Wow.  That was a long explanation.  But it should work for you.  Give it a try.

Bob Schor

[Kevin_Price · Accepted solution]

I like several things about Bob's suggestion, but recognize that it may take some significant work to rework your app to handle asyncronous processing via queues.

I was involved in the thread you linked (as well as a much more involved one that it, in turn, linked to).  The method shown using VI server references worked just fine for me, but the app in question was not very demanding for performance.  I was only processing packets at around 5 Hz so execution efficiency was never a concern I needed to investigate.

However, I was also doing some custom (albeit pretty straightforward) processing.  Because you plan to use the built-in filtering functions, you can take advantage of the legwork NI has already done to maintain filter states across multiple channels of data.  You do this by using the 1D waveform array version of the built-in filter function as shown below.

Note: no claims or promises about the relative CPU efficiency of this method compared to others.  But I'd try this method first because it'll be so easy to fit into your existing code.  If you need more speed, *then* look into ways to take advantage of parallelism.

-Kevin P

[Bob_Schor]

Kudos to Kevin!  Simpler is (almost always)(well, occasionally) better!  Especially if it involves minimal coding (which my solution definitely does not).  I'm always up for "learning new tricks" (don't believe that old saying ...).

Bob Schor

[mcduff · Accepted solution]

Here's a simple method using references, but you can use all of the built stuff in LabVIEW, no need to make your own subVIs. It will work for a 2D array if you don't want to make waveforms.

mcduff

EDIT: Forgot to put the output

[EMCCi]

Okey! Thanks for your responses.

I will try with the waveforms first for simplicity. If it hasn't enough performance then I will try the reference solution.

Since I do one iteration of the code every second, I'll supose that it would be necessary to open all the references in the initialization of the code, store them in a variable, and then every second execute only the for loop?

What is the overhead of call a vi by reference instead of placing it in the block diagram? And for open the reference?

Best regards!!

[mcduff]

---

@EMCCi wrote:

---

Since I do one iteration of the code every second, I'll supose that it would be necessary to open all the references in the initialization of the code, store them in a variable, and then every second execute only the for loop?

That is what I do.  In my data processing loop, basically a State Machine, I initialize the references and put them on a shift register to use later. One thing to note since you have a 2D array, assume you have 3 rows of data corresponding to Channel 1, 2, and 3, then you want to use the first 3 references for the filter. Say you change your data to Channel 1, 2, and 4, hen you want to use the first 2 references for the filter and the 4th reference. Not sure if I am making sense here, but you always want to use the same reference for a particular data channel, otherwise you will lose state. In your loop, you can always reinitialize the filter if needed. When your loop is finished, then close all the references.

What is the overhead of call a vi by reference instead of placing it in the block diagram? And for open the reference?

I do not know, but I think it is insignificant. Once the references are open then it seems to run like a normal subVI. I have used it to process 8 channels of 1 million points each without noticing a slowdown.

mcduff