filter noise under threshold

[glw36]

Hi NI hivemind,

I am  trying to filter some noise in a data set. I want to keep the peaks of my data, since there are only a few points that make up the peaks. So my desire is to smooth out the noise and keep the peaks. I am new to LabView, so pardon my lack of proper vocab.

In this VI, I have a random number generator representing my noise. When it enters the loops five data points are evaluated using the subarray function. The max-min is taken and the average of the five points is also taken and then divided. If the value of (max-min)/avg is greater than 1, the five points are passed along to the new array unchanged. If the value is less than 1, the avg is passed along to the new array. I am using shift registers to store the data abd then move onto the next five points. I am unsure of why the new array's (array 2) values increase in a stair stepping way. I think that because of the averaging of the noise, that the values should be somewhere around .5. Does this have something to do with the shift registers?

Also, I think that because there is a plus 5 at my shift registers, that the data starts getting processed at 5 rather than zero. How do I tell the values to be processed in groups of 5, starting at zero, using the shift register?

tl;dr: -I want to smooth noise (in this example using a random # generator) and my result is a stair-stepping increase in output values instead of a smoothing somewhere along 0.5.

-How am I using the shift registers incorrectly?

-How do yo get the shift register to start at zero, but each successive take, go up by five?

[Bob_Schor]

I'm assuming the inner For loop is intended to give you the average of your 5-element sub-array.  Before I tell you how to "fix" it, here are two other functions you should know -- on the Numeric Palette, there is an Add Array Elements, which (correctly) does what you want your For loop to do, namely add the arrary elements.  In addition, on the Mathematics Palette, under Probability and Statistics, there is a Mean function (which also does the division for you).

Now, why does your average increase by 5 each time?  The Shift Register "remembers" the sum!  The first time the For loop is called, the Shift register adds up the first five values.  The next time, it adds the next five values.  See where this is going?

You can fix this by wiring a desired "starting value" (I recommend 0) to the input terminal (on the left edge, outside the For loop) of the Shift Register.  See if that fixes things.

Bob Schor

[johnsold]

1. Since you calculate the number of iterations of the lop before starting the loop, use a for loop.

2. The unitialized shift register on the inner for loop will retain the value from the previous while loop iteration. This is likely the cause of your step output. Initialize the shift register just as you did the ones on the while loop.

3. Use the Mean.vi from the Probablity and Statistics palette in place of the for loop and divide by 5. It does not have the memory effect and will automatically adapt if you ever change the number of elements to be averaged.

4. Generally expanding an array with a shift register and Build Array in a while loop is not a good idea. It can cause frequent memory reallocations resulting in poor performance or even crashes. Your VI uses small enough arrays that you get away with it but learn best practices to avoid problems in the future.  Pre-initialize the array to the maximum size needed outside the loop. Use Replace Array Subset inside the loop.

5. Use graphs, not charts when you will only be writing an array to the display one time or when all the data is available at the time of writing.

On vocabulary: A threshold operation on noise is not a filter process. It may be called gating, blanking, or thresholding, depending on the details of what you do.  Smoothing is a low pass filtering process. 

Reducing noise can be complicated. The best approach may depend on the nature of the signal, the type of noise, the relative amplitudes of signal and noise, and other factors. 

Please describe your real signal source, the type of noise present, and what you will be doing with the signal after processing so we can recommend a suitable method.

Lynn