Simple Moving Average VI

[MogaRaghu]

Just created a simple moving average VI. See the attached code ( LV2012).

It seems to be working - any comments or improvements ?

Thanks

Raghu

[Bob_Schor]

Usually when people talk about a Moving Average, they mean "Replace Point N with the average of M points surrounding Point N".  Suppose I have 100 points whose values are 1, 2, 3, ..., 100, and I want to do a 5-point Moving Average.  First thing to note is that there is an "{End-point" issue -- there are not 5 points "surrounding" points 1, 2, 99, and 100, so let's just forget about those.

The "moving average" of the third point is the average of 1, 2, 3, 4, 5 = 3.  The average of the fourth point is the average of 2, 3, 4, 5, 6 = 4.  However, this is perhaps too simple an example.  How about the average of a Step Function, 0 from 1 to 10, then 20 thereafter.

Again, throw out points 1 and 2.  The average of points 1-5 (to go into Point 3) = 0 (since all the points are 0).  Similarly with Point 4, 5, 6,7, and 8.  However, Point 9 is the average of 0, 0, 0, 0, 20 = 4.  How about Point 10?  Well, it should be the average of 0, 0, 0, 20, 20 = 8, but did you remember to not overwrite Point 9?  Hmm, seems like we need to keep two copies of the Array (which is, in general, "expensive").

There are several ways you can avoid doing this.  Do you understand where the problem arises in the previous paragraph?  If not, try doing this with pencil and paper (or try coding it in LabVIEW).  I'll give you the answer so you can check -- the moving average of the Step Function is --, --, 0, 0, 0, 0, 0, 0, 0, 4, 8, 12, 16, 20, 20, 20, ..., --, -- (where "--" are the "empty" values at the ends of the array, the points you don't have sufficent "neighbors").

Bob Schor

P.S. -- it would not surprise me if there weren't a LabVIEW function that does this for you.  But if you are learning LabVIEW and want to have a better understanding of how the algorithms you are plugging in work, it never hurts to "play" and try it yourself.  You might even come up with an "improvement" (several of us have done so ...).

[altenbach]

There is "mean ptbypt" which does the same. You can inspect the code if you want.

• A big flaw in your code is the fact that you constantly grow and shrink an existing array. You should try to find a solution that works in-place on a fixed size array. May examples have been posted on the forum over the years (look hee for example). The mean does not care if the elements are out of order, so you can simply replace the oldest element, no matter where it is located.
• You are also prepending the new element to the beginning of an existing array, which is always much more expensive than appending to the end.
• your sample size cannot change once the VI is running.
• Your shift register should be initialized with an empty array, not an array already containing a single element that is zero. (This extra zero will give wrong averages!)
• Your code should be made into a subVI so it can be re-used (similar to the ptbypt version).
• Your VI can never be stopped, just aborted.

[MogaRaghu]

Dear Bob

thanks for sensitizing regarding the finer points of the Moving Average method. This afterall is a Statistical tool that helps to "see" what you want to see abstracting the distractors.  So the method is bound to have some shortfalls in some situations or context.

But i guess its perfectly suited for my kind dof data logging - its a pressure or temperature or flow signal - and i acquire at something like 400 samples / sec and then use a averaged single sample. And the process is fairly slow as my main code runs at not more than 20 Hz. So when i do a 5 sample maving average, my first sample arrives 5 x 50ms later, then for every 50ms i get a valid sample.

Basically i am more concerned on trends and not spot values. In this there is little worry about missed samples or rogue values. Of course i would not dare to use this for a Step Function. That would be cruel.

Raghu 

[MogaRaghu]

---

@altenbach wrote:

There is "mean ptbypt" which does the same. You can inspect the code if you want.

• A big flaw in your code is the fact that you constantly grow and shrink an existing array. You should try to find a solution that works in-place on a fixed size array. May examples have been posted on the forum over the years (look hee for example). The mean does not care if the elements are out of order, so you can simply replace the oldest element, no matter where it is located.
• You are also prepending the new element to the beginning of an existing array, which is always much more expensive than appending to the end.
• your sample size cannot change once the VI is running.
• Your shift register should be initialized with an empty array, not an array already containing a single element that is zero. (This extra zero will give wrong averages!)
• Your code should be made into a subVI so it can be re-used (similar to the ptbypt version).
• Your VI can never be stopped, just aborted.

---

Good optimization tips. The point on initializing with Zero missed me. And yes the user should not change the sample size once it starts running.

Finally i will make a SubVI and handle things like stopping etc..

As to the point of prepending than appending the new value to array, maybe there is a performance penalty but given the size of my array i am sure the CPU does not care anwyay ! But for me it has to be this way as i use the final data for plotting a trend of a physical parameter.

Thanks for your time.

[altenbach]

---

@MogaRaghu wrote:
But for me it has to be this way as i use the final data for plotting a trend of a physical parameter.

---

 Well, you did not say anything about that. Still, there is no reason to prepend instead of append. You can always reverse the array at the end if needed.

Even my in-place solution would just require a split-built array step at the very end. Everything that is done only once is insignificant compared to stuff you do over and over with each iteration.

[MogaRaghu]

---

@altenbach wrote:

.... Still, there is no reason to prepend instead of append. You can always reverse the array at the end if needed.

---

Got your point - yes I am anyway going to use only the Mean value so it does not really matter where exacty the new data element joins teh array. Super.

My challenge now is to convert your VI snippet into one capabale of handling a two dimensional array ( simulatneous readings from about 6 sensors ) and get a Averaged Out array for further use.

[Bob_Schor]

---

@MogaRaghu wrote:

Dear Bob

thanks for sensitizing regarding the finer points of the Moving Average method. This afterall is a Statistical tool that helps to "see" what you want to see abstracting the distractors.  So the method is bound to have some shortfalls in some situations or context.

But i guess its perfectly suited for my kind dof data logging - its a pressure or temperature or flow signal - and i acquire at something like 400 samples / sec and then use a averaged single sample. And the process is fairly slow as my main code runs at not more than 20 Hz. So when i do a 5 sample maving average, my first sample arrives 5 x 50ms later, then for every 50ms i get a valid sample.

---

Aha!  So you do not want a "moving average", but just a "simple average".  That's much easier.  Here's the idea (which works much better with a Producer/Consumer Design) --

• Say you are sampling at 400Hz, want to save the data at 400 Hz (i.e. save all the data to disk), but want to display at 20 Hz (because you want to see trends, a longer time base, etc.).
• Set up your A/D system to collect 20 samples at 400Hz (note you can collect N Channels at the same time, giving you a 2D array of samples.  As you get the data (at 20 Hz) from the A/D (making this the Producer), enqueue it to the Consumer.
• The Consumer starts by writing the data to disk (shouldn't take much time).  Now you have a 2D array -- in a For Loop, on a channel-by-channel basis, average the 20 points.  Now you have a 1D array, with an Averaged Point for each Channel.  Go ahead and plot it.

Note that this scheme (a) uses all of the data, (b) handles multi-channel data with aplomb (and, if you are from the Middle East where they grow, you can also handle your data with a juicy plum), and (c) lets you collect your data from the A/D equipment, save your data to disk keeping all of the points, and show your data on the screen using all of your points but also averaging to improve visual Signal-to-Noise ratio, all without losing any data (I've done exactly this with 24 channels at 1KHz, with the data being taken on a remote system and sent to the PC via TCP/IP, so we also have TCP processing in the loop).

Welcome to the exciting world of Data Acquisition and Processing with LabVIEW.  Trust me, this is a wonderful system for doing this type of work!

Bob Schor 

[MogaRaghu]

Based on the feedback i got on my original VI i have refined the Moving Average code into a subVI.

I then used it to average a simulated 10Channel data - just to keep things simple i made sure all10 Channels had identical data. One would then expect to get the same moving average for all 10 channels.  I am surprised at the small variance i notice between channels - generally they are close but not exact.

And just to explain the process i am attempting i also have enclsoed a XLS.

So where does the variation come in from ? The unitialized shift register inside the Sub VI ??

[altenbach]

Your code still makes no sense. SInce you are calling the subVI one scalar at a time, you are not getting what you want because the shift register only rememebers the last N scalars, no matter what channel it is from. Your code is still very inefficient and convoluted. (e.g. why are you still using insert into array to append (both in the mani nad in the sub). (You could use a reenetrant subVI and a parallel innermost FOR loop, but that seems overly complicated too)

If you want to do a running average on each channel, the subVI needs to keep a 2D array in the subVI. All this has been done before.