Elements Remaining FPGA acquisition

[Salvio]

Hi again to all.
I'm acquiring a signal with cRIO with an FPGA DMA and i've noticed that the elements remaining  grow up continuously.
What are exactly these elements remaining? i've supposed that are all the elements that i don't see on my graph,aren't they?
is there a way to decrease this number? and if, as i've said before, these elements are the points of the waveform that i don't see, is it possible to see them in my next iteration?how?
Tnx for the explanation

[Salvio]

Nobody can help me?

[jdigiova]

The main problem is that your host VI is not able to read data out of the DMA stream fast enough.  Since you are not monitoring the Full flag on the FPGA VI you do not see that you are actually overflowing the DMA stream.  If you want to guarantee that your host receives all the data you should treat an overflow situation as fatal and stop the FPGA VI when Full is asserted.  Other things that you can change would to not packign the I16 data into 32 bit packets.  Although this is more efficient in terms of PCI bus data transfer it will mean more processor cycles on the host having to split those numbers back.  In most all applications (especially on cRIO) your bottleneck will not be the PCI bus data transfer, but will be the processor.  Therfore a little inefficiency on the FPGA side is made up for by freeing some CPU cycles on the host.  The last change I would make on the FPGA side would be to make the FPGA VI wait for a signal fromt he host before it actually starts putting data into the stream.  This will allow you to open (start) the receiving end of the stream prior to putting points in on the FPGA.  The changes to the FPGA VI are shown below.

You can also change the host VI a little, although looking at your code it looks like you are performing some very processor intensive calculations and may just not have enough CPU power to keep up with the stream.  Some changes to the code, which may not help you run faster, but are better practice include.  Always reading a multiple of the number of channels in the stream.  For example, since you use 8 channels you can use a front panel control to set the number of scans and multiply this number by 8 to determine the number of elements to read from the DMA stream at a time.  You can also multiply the number of elements to read by 2 and use the result to set the host depth for the stream.  This should be adequate for double buffering.  After you set the host depth you can open (start) the DMA stream then send the signal to the FPGA VI to start pushing data into the stream.  The host should also monitor the full flag from the FPGA to see if an overflow event happened.  If so you should stop the host.  When you read the DMA stream use the number of elements calculated earlier.  Since we are reading a multiple of the number of channels we can use some reshape array functions to get the data back into the original form.  You will use the number of scans and number of channels to perform the reshape. See images below for changes to host.

Even with these changes it is important to remember that the amount of data you can process (overall data rate) is going to be limited by the amount of processing you are trying to perform.  You can usally improve performance by reading larger amounts data out of the DM A stream (increase the number of scans to read).  You can increase the size of the buffer, but as soon as you see the backlog growing chances are increasing the depth will only postpone when the overflow will happen.  You can also try slowing down how fast you put data into the stream, if possible.

Regards,

Joseph D.

Message Edited by jdigiova on 02-09-2007 04:01 PM

[Salvio]

First of all Thanks for the explanations.

At moment i'm not at university (i have to wait monday) so i can't try the change that you have advised me.

So if i've understood you say that is not a good thing to change from I32 to I16 because the system would be slacken, wouldn't it?

The idea to put a start in the FPGA is good i'ven't thought at it

I'ven't understood well what you are doing in the last jpeg when you put the number of scans and the number of channels into a block (i don't understand what kind of block it is) before the transpose. Can you explain it to me, again?

So at the end you are saying that is impossible to keep under controll the elements remaining or  shall I agree not to have understood?

I'v tried to put the elements remaining at the start of the new data (as i've done in the JPG) and i see that it works but after few iterations (from 2 to 5) the cRIO disconnect itself.

Is it possible to put the elements remaining in the new data to visualize on the graph and at same time to increase the buffer so that if it stays down a dimension fixed by me I show elements remaining, if, instead, the dimension fixed by me exceeds the normal i can show the normal data to the display or in any case interrupt the VI?

As i've told before i'm at home at moment so i can't verify the changes you advised to me, so can you send me the correct VIs so that monday i can try them?

Tnx for the old and the new explanations and above all for your kindness.

[Salvio]

plz answer me as soon as possible so that  i can try the changes you adviced me.

tnx

[Christian_L]

Salvio,

You do not need to do anything special in your code to handle the 'Elements Remaining' in the DMA buffer. Element Remaining is the number of data items left in the DMA buffer in the host memory at the end of the current Read operation. This data will not be lost. This data is the first data in the array that is returned from the next Read operation. The DMA buffer in the host is a FIFO so that the first data that is read is always the oldest data in the buffer (that you have not already read). So after your DMA Read operation in RT, all you need to do is to extract the individual channels from the 1D array array returned from the Read.

If you do combine two I16's into one U32 for the DMA buffer, you can simply typecast the array back to an I16 array and then decimate the array into eight channels.

Message Edited by Christian L on 02-12-2007 09:51 AM

[jdigiova]

I like Christian's method if you are reading a fixed number of channels back from the FPGA.  Since the FPGA code is written to always transmit all channel data back then it makes more sense to use Christian's approach.

So if i've understood you say that is not a good thing to change from I32 to I16 because the system would be slacken, wouldn't it?

No.  I am saying you do not need to join 2 I16 channels into a single U32 element on the FPGA side.  You can just put the I16 data directly into the DMA stream without using the join function.  You still need to get the data on the host from the U32 datatype back to the original I16 data.

Just to clarify because I had to read it a few times before it clicked.  When Christian said "If you do combine two I16's into one U32 for the DMA buffer, you can simply typecast the array back to an I16 array and then decimate the array into eight channels."  He means if you combine multiple channels into one stream, not multiple channels into one element.  He is not recommending to join 2 I16s into a U32.  By not joining the numbers this means you are not using the full bandwidth of the PCI bus for DMA transfer because 16 of the 32 bits in each element has no useful data.  This is fine however, because it means you do not have to do anything special on the host to split the data out of each element to get the original data back.  In your application saving CPU time is going to be more important than maximizing DMA throughput.

What is the block it is before the transpose. Can you explain it to me, again?

The block is a reshape array.  It can take a 1-D array input and reshape it into a 2-D array.  However, since you are reading a fixed number of channels the decimate method that Christian explained is easier.

So at the end you are saying that is impossible to keep under controll the elements remaining or  shall I agree not to have understood?

As Christian mentioned the Elements Remaining tell you how many points remain in the DMA stream in host memory.  They will remain there until the next read.  You do not need to be changing the number of elements to read on each read.  Elements remaining just tells you that you have more data in the stream.  If this number increases it means the stream is backing up and if you don't read more data (or faster) than eventually you will most likely overflow the stream.  The best thing you can do is pick a good block size to read and if you see the elements remaining increase then you should pick a bigger block size, or try and read more often.

My main concern with you application is not the DMA portion and the tips in this thread can help you improve the efficiency of the DMA stream.  My concern is that you are trying to perform a lot of processing on a controller with limited processing power.  Most likely the DMA stream is so fast that the CPU can't keep up. Do you have to perform continuous acquisition as you are doing the analysis, or can you acquire a finite sample and do post processing?  Or maybe you can do a combination of both where you acquire N scans then do analysis.  When the analysis is done you tell the FPGA to acquire another N scans.  This retriggerable finite acquisition will allow you to turn of the data stream while the processor analyzes the data.  This of course means there will be gaps in the data, but sometimes this is acceptable.

Regards,

Joseph D.

[Salvio]

mmm the problem is that what i have to do is this:

i acquire a number of elements that can be equal to 30 periods (for example)

but after that i'm syncronizating on 10 periods so in that way i have lost 20 periods that i want to know if it is possible to put in the new array of the Data that cames from the FIFO Read.

I've tried to do this but i have some problem because i've supposed that the first thing that i have to do is to say to the FIFO how many elements it has to read...so i've thought in that way:

if the frequency of the signal is for example 50Hz and if my sampling frequency, Fs, is 100 micro seconds (10kHz), i have that the number of elements in a period is equal to Fs/50. So the elements that i need to have to lost only few points of the waveform are given by:

Fs/50*(Number of period that i want to synchronize+1) in my case this number of period are 21.

I have to do this only the first 2 iterations after that the block that i use to make the syncronization give me also the instant value of the frequency (that is variable)...so in a case structure i put this two condizions:

if iteration is <2 i have Fs/50*(Number of period that i want to synchronize+1) in my case this number of period are 21;

if iteration is >2 i have Fs/variable frequency*(Number of period that i want to synchronize+1) in my case this number of period are 21.

but at this point i've got a problem because the division is a float while the FIFO (if i've understood well) accept only integer...so it doesn't work well...

Another thing that i have to do is in the while of the acquisition, so if my backlog are > than 0 i have to put the elements that i don't have analysed into the next data caming from the new iteration.

So in that case i've taken the array caming from the first channel and i've done an array subset on it from the last point that i've used in the syncronization to the all size of the array (that will be always bigger than the array syncronizated) and than i've used insert into array with index 0 and as new element i've put the exit of array subset...

I think that it is theoretically possible to do but i don't know why I am not able pratically to do it...can you help me?

[Salvio]

i think that the problem is in the number of elements because from Fs/F i have a number that is a float...and i've seen that the FIFO Read don't acquire nothing if i put as number of elements a float...i've also tried to make a cast from float to integer but nothing to do...

can you suggest something to me?

tnx a lot for all you have done for me

[Salvio]

i've also tried to convert the float into an integer of 32 bit but the result is the same...it doesn't acquire...

Message Edited by Salvio on 02-15-2007 02:28 AM