05-26-2009 07:19 AM
Hi, I have a big problem. I try to do a moving average over 6 minutes, having a sampling time for each channel of 25 us. At total I have 8 channel,s for which I have to do the averaging. My programm communicates through three dma fifo with the FPGA programm. At first I wrote a host programm where the I only read one dma fifo, which contents 3 out of the 8 channel I supervise. Well if I want to do a 6 min average I will have to average over 14 400 000 samples, for each channel. So to make the amonut of data smaller I first want to calculate an average over 1 ms (over 40 samples), save it in the next array for which I calculate the 100 ms average (10 samples of 1ms) and so with the 10 s average and finally 6 min(36 samples of 10 s values). So I have don't have so much data but my 6 min is not accurate anymore, but this is not a big problem. I can live with it. Well I thought doing several parallel loops. The first one which gets the data from the dma Fifo communicating with the FPGA Card and then the other loops just for calculating the 100 ms average and so on. I thought using queues which communicate with the other parallel loops. But using queues I always get a timeout, which means that my average calculation is not fast enough and then I loose data. To make the average calculation faster I first calculate the summ of the array and put it in a shift register and then the last value in array will be subtracted from the summ and then deleted and next first element in the array will be added to the summ. So I don't have to calculate the summ everytime over all of the array elements. I also saw when I use the dma fifo I get a timeout with my dma fpga Fifo, which I find that is very strange. Does the use of queues make a programm very slow?? So then I tried to average directly in the first loop over 1 ms and this works fine. I don't get a timeout with my DMA Fpga Fifo. Does someone has any idea how to calculate a average over such big amount of data?? Or does someone know how to calculated this very efficient, so I don't loose data! Thanks , Irina
Attached find the two examples one with queues and one with the direct calculation for 1 ms average.
05-26-2009 07:38 AM
Warning I Have not had my cofee yet.
A few Ideas:
1. Look at the point by point vis these already do the shift register buffering and analysis
2. Try cascading 2 buffers
Have 1 100ms buffer pt by pt update on each new point, put thye output into the next point into the 6second buffer, and only shift the second buffer every 100 ms, this way you only neex about 1/60th of the data in the buffer but only comprimise a small amount of accuracy (should be insignificant unless there are huge impulses in your data)
05-26-2009 07:51 AM
Sorry, I missed the FPGA section (No cofee), I so skip point by point possibly (I have not used pt-by-pt on the FPGA yet). Cascading the data analysis is still a good method for reducing the amount of data in buffer and keep the sum in a buffer should maximize your speed by only requiring one subtraction and 1 add each itteration for each buffer.
So each call on the first buffer (25us) could update a circular buffer, subtract the oldest point, add the new point to the total shift register. Divide the first total and pass the 100ms average to the 2nd buffer. This buffer is also a circular buffer, but only perform a shift every 400 points (100ms).
This should only need 1 400 pt buffer and 1 60 pt buffer per channel to memory useage is 1000 points/channel (you could rebalange the buffer load if desired for memory and accuracy optomization i used easy numbers to calculate), The loop should only require a few calculations per itteration, essentially
-buffer 1 one memory write, read pointer update, write index pointer update, 1 add, 1 subtract 1 divide and loop 2 has the same but the read and write index are only updated each 400th itteration
05-26-2009 07:55 AM
05-26-2009 08:14 AM
Hi, thanks for your fast response. Each DMA FPGA Fifo has 3 channels, each with a reflected and forward value. So infact I have 6 channels, but the subtraction of the reflected and forward is my signal. And each channel has its own charecterics curve, wich is beeing used using the differents coefficients.
But what is point by point approach?? Is is a certain vi??
05-26-2009 08:17 AM
05-26-2009 08:25 AM
Hi Paul,
this was my idea. To cascade my data. I created an array for my 1 ms average, but I want to calculate each average in a different loop. The connenction between these loop is beeing done using queues. But these seem to be to slow , because I get a timeout with my dma FPGA fifo, so I loose data. Or is a buffer something different than an array??
05-26-2009 08:26 AM
05-26-2009 08:27 AM
05-26-2009 09:22 AM