Computing long term statistics

[mwil]

I need overall (long term) mean and standard deviation for each channel sampled. Consider the example vi "Cont Acqt&Chart Samples-Int Clk.vi". I used this example as a starting point because I wanted to display chucks of data on the graph and the mean and std. dev. for each channel over that chuck of data. That is to say, I have two indicator arrays showing the mean and std. dev for each channel that is updated on each loop iteration. Starting with this example, I've correctly computed the mean and std. dev. for each channel within each loop iteration. This works fine. What I need to add is to present the overall mean and std. dev. of all loop iterations at the end of the acquisition process. All I've been able to do is to bring out arrays of waveforms but not to get the statistics across all of the waveforms for a given channel. If the number of points acquired for each loop is the same then the average of the averages of each channel is correct but this will not work for standard deviations - I need to sweep across all waveforms for a given channel in one go in order to get a correct std. dev. value.

It seems like this should be pretty simple, just add an overall mean and std dev array (one mean, one std dev for each of n channels) but I cannot see how to do it.

I'm running LabView 8.2 FDS on a Windows XP system using NI USB-6008 device.

Thanks in advance for your help.

[Joseph_Loo]

If I understand what you want, you will need an array for the entire acquistion period containing all the data for each channel. At any time period, you can then get the standard deviation and the mean.

You never mention your sample rate or the sample length. That will depend on your various solutions. Another way is to keep a running total on the data to minimize the amount of data.

[mwil]

Joe, thanks for the reply. I agree that if I could construct this array of all the data then I could extract the channel I want and do stats on this 1D array of values. I don't know how to get the data into an array like this.

If I take the 1D array of waveforms out of the acquisition loop (with indexing) I get a 2D array of waveforms. I can extract the 1D array of waveforms that corresponds to each channel's signals outside the loop after the loop completes but I don't know how to combine all of these waveforms into one object so I can do stats on the entire channel's data set. I can do a mean and std. dev. on each of the waveforms that corresponds to a given channel but these partial stats don't represent the correct stats for the entire data set for the channel. I need, somehow, to get all the data for a given channel into one object and then do stats over that object.

I'm intrigued by the idea of doing a "running total" on the data. Can you point me to an example of how to do that? I currently have a mean and std. dev. for each set of n points that are acquired during one loop iteration. How can I use these to get an overall mean and std.dev.?
 

[Joseph_Loo]

There are several ways of doing it. You can constantly append the data to an array, create clusters of each sample block and combine them in an array. You will then have to pull the data apart, which should not be too difficult.

The running total is actually pretty easy. You will need to have basically have three variables, yi, y2, n. Let us assume your new piece of data is y. Then you will need to compute the yi and y2 variables in the following manner (we assume the values are initialize to 0).

n = n+1 basicallly your count of elements

yi = yi + y baasically keeping a running total

y2 = y2 + y*y summing of the squares

mean = yi / n

variance = (y2 - ((yi * yi)/n)) / (n -1)

std = mean * sqrt(variance)

You will find this formula in any statisical books. Also note there is a variation on the division by n depending your are using the bias or unbias estimator.

[mwil]

Hi Joe, thanks again for the helpful message. I think the incremental summing approach to my long term statistics issue will work for me. Thanks much for the suggestion.

Now that I've discovered the issue, I can well imagine that the issue of combining sets of waveforms will come up again soon.  Do you have any specific suggestions as how to take a 2D array of waveforms - 1D covers each channel in the channel list and the other 1D covers the loop iterations, and construct a 1D array covering the channel list where each waveform is the concatenated sequence of all the waveforms collected for a given channel?  This seems to me to be a very elementary operation but I cannot see any examples of it. I see examples of appending one waveform to another but the complicating factor appears to be dealing with the 2D array. If you have time, could you show me, using the LV example vi named above pulling the 1D array of waveforms out of the loop which produces a 2D array of waveforms and then combining them back into a 1D array of waveforms, one for each channel sampled? This is what I can't make work. Thanks again for all your help.

[Joseph_Loo]

Try this vi

[mwil]

Joe, thanks much. I'm up and computing stats correctly Thanks for your help!