High-Speed Digitizers

cancel
Showing results for 
Search instead for 
Did you mean: 

Data stream to disk - Fetch error

Hello,

 

I have this setup in the rack (the setup was there before my arrival) :

 

- NI PXIe-1075 where are plugged :

                3 cards NI PXIe-5185 12.5G bits/s 8bits Memory : 512Mo per channel

                1 Remote contrôle card NI PXIe-8381 linked to the PC

                2 NI-8262 Expansion Module linked to the disks

- 2 NI disks (5To each)  linked in PXIe to the computer

- 1 PC (windows 7, Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz, 3601 MHz, Quad-Core, 8 logical processors)

 

I have a NI PXI-8110 2.26 GHz Quad-Core PXI Embedded Controller card, but not used.

 

I start a project in C and Scilab(like Matlab), that requires streaming acquisition to a disk with a very high sample rate (300 M/s). I created the driver in C language (visual C 2008).

 

The signal is a pulse radar (Pulse width : 3000 pts, period : 76800 pts) triggered with external trig input. So, I decided to acquire 4096 pts for each record, to record the pulse window. When I acquire 1 record, I can plot it in a graphic, no problem, I can see the pulse.

 

The problem is when I want to record more than 500 000 records for streaming, I have this error message when I fetch the datas in a while : "error -1074115594 : Timestamps have been overwritten. You can no longer read any data".

 

This is a part of the code (Property "ALLOW MORE RECORDS THAN MEMORY" is set to 1 and "configure horizontal timing" is set to 500000 records) :

 

...

*errorCode = niScope_SetAttributeViInt32 (sessions[*VISA_ID-1], VI_NULL, NISCOPE_ATTR_FETCH_NUM_RECORDS, 1);

niTClk_Initiate(sessionCount, sessions);

 

while (IterationsAcq < 500000)

{

                niScope_SetAttributeViInt32 (sessions[*VISA_ID-1], VI_NULL, NISCOPE_ATTR_FETCH_RECORD_NUMBER, IterationsAcq);

 

                if (*errorCode == 0) *errorCode = niScope_FetchBinary8 (sessions[*VISA_ID-1], "0", timeout, actualRecordLength_session1, (ViInt8*)waveformPtr_session1, wfmInfoPtr_session1);

                else niScope_FetchBinary8 (sessions[*VISA_ID-1], "0", timeout, actualRecordLength_session1, (ViInt8*)waveformPtr_session1, wfmInfoPtr_session1);

 

                IterationsAcq++;                                            

}

...

 

I can understand that this error message appears when the difference between the number of records acquired and the number of records fetched is more than 100 000 but I don't understand why the fetch acquisition is so slow ! After the acquisition of the 4096 pts in a period of 76800 pts : 72704 left to fetch the datas...before the next trigger !!! I don't understand why the fetch speed don't follow the record acquired !

 

I can see the same problem in the example of NI-SCOPE in CVI when I launch "Multi Record Fetch More Than Available memory.exe" : when the number of records fetched is about 67, the number of record acquired is about 65000 on my system...! it stops with the error "BFFA4BF9 at NiScope_Fetch"...

 

In the while, before the delete of few code lines, the fetch was lower (error after 200 000 records instead of 500 000) but I can't delete the 4 last lines...!

 

I tried to insert the while in a thread with the property priority highest, but it is the the same error...

 

I found an interesting example in CVI labwindows :

 

https://forums.ni.com/t5/Example-Program-Drafts/Reference-Design-for-Streaming-Data-from-Multiple-Hi...

 

and another in LABview :

 

https://forums.ni.com/t5/Example-Code/NI-Scope-Multi-Record-Fetch-Stream-to-Disk/ta-p/3515897

 

It seems that 300 M/s is possible. So, I don't understand.

 

- Do I need to install the NI PXI-8110 2.26 GHz Quad-Core PXI Embedded Controller card instead using an external computer that seems to be more powerful (windows 7, Intel(R) Xeon(R) CPU E5-1620 0 @ 3.60GHz, 3601 MHz, Quad-Core, 8 logical processors) !?

 

- Or is it impossible to fetch record forever  and try another solution !? :

 

https://forums.ni.com/t5/High-Speed-Digitizers/Fetching-triggered-acquisitions-forever/td-p/3650977?...

 

- The final project should be 3 cards (5185) with 3 disks, so 3 data streams with 2 channels during few hours. So the number of records could be more than 150 000 000....! Enough to fill 10 To of datas.

 

- I don't have the last version of NI-SCOPE, I don't know which version I have. I am not sure that it is necessary to have the last one to resolve my problem....(!?)

 

Thanks for your help !

0 Kudos
Message 1 of 5
(2,470 Views)

I make a mistake the controller not used was NI PXIe-8135, not 8110.

 

I have installed the NI PXIe-8135 controller card to try the example "Multi Record Fetch More Than Available memory"....same problem that the setup with external PC : speed of fetched records is too slow.

 

In the first case with 8192 record length, i have a full memory and in the second case with 4096 record length, max delta samples between "fetched" and "record acquired" is reached (100 000)... so, i don't understand how to do streaming if the speed is too low for fetching... i only have NI PXIe 5185 and NI PXIe-8135 controller in the rack NI PXIe1075.

 

Error2.png

Error1.png

 

0 Kudos
Message 2 of 5
(2,446 Views)

The rack :

 

setup.jpg

 

and the disk

 

Disk.jpg

0 Kudos
Message 3 of 5
(2,435 Views)

When i launch the example "fetchforever", the probleme is the same

 

fetchforever.png

 

5.8e6 total points fetched instead of more than 1e9 (1Gbyte memory on the 5185 card). Speed fetch is too low...for data streaming if i want to record 1 hour of data on a disk...

 

If i change the sample rate by 3Ms/s instead of 300, the total point fetched is about 1.2e9...so the fetch speed is enough for 3M/s...

 

i don't understand how it's possible to do streaming with 600Ms/s like there (they use fetch function too) :

 

https://forums.ni.com/t5/Example-Program-Drafts/Reference-Design-for-Streaming-Data-from-Multiple-Hi...

 

0 Kudos
Message 4 of 5
(2,431 Views)

I thing that i have the explanations...

 

1/ The digitizers can transfer datas to a host controller at rates greater than 700 MB/s only if block size is superior or equal to 16MB. It is not possible to aquiere records (at 300 Mb/s) with 4K or 8K.

cap.png

 

2/ The examples of NI are not optimised to test the max speed. Indeed, they use the fetch function to aquiere raw data. Fetchbinary functions like Fetchbinay8 are faster. 

0 Kudos
Message 5 of 5
(2,406 Views)