Help with 4 Channel Acquisition

[Scopes]

Hello again, I am having a lot of difficulty creating a program that can read voltages on up to 4 channels at once at sampling rates > 100,000 points a second. I created a program with 4 loops in parallel as 2 producer/consumer set ups. The first producer/consumer it basic event handling where it just handles events like when the user presses buttons to start different states of the software such as start recording data, preview data, stop preview, clear displays, exit, ect. My main problem is that when I just set up a durability test and have the program record 1 channel at 50,000 points per second for 20 minutes, it will take ~35 minutes to process half of the recording. I want to know if what I am doing is possible, if I created some memory leaks or inefficienties. I can't just easily change from my MAC system to a Windows, so please don't say I should change to Windows unless if it is truly my only option. 

My program:

          producer (events)

                    -reacts to any events by passing states to the consumer (events) loop

          consumer(events)

                    -takes the states from producer and gives states to the consumer (data) loop

                    -managers display properties

          consumer (data) loop

                    -handles all file writing in the cases (create file, record, and end file)

                    -turns on and off the producer (data loop)

                    -decimates data for display (decimation will take every 20th point)

          producer (data)

                    -sets the sample clock

                    -reads the data in packages of 0.1 of a second

Summary

----------------------

Hardware:

-MAC book Pro from 2008

-NI USB-6211

Software:

-DAQmx Base software package

-LabVIEW 2014 for MAC system

Needs:

-4 channel acquisition

-high sample rates (100,000 points a second)

-run time for recording ~15 minutes

IMPORTANT: The version I am talking about is in Multichannel Record -> Main VIs -> Multichannel Recording 0.5

Thanks for your interest and please, if you need any more information I will do my best to get it to you.

[Bob_Schor]

If your needs are 4 channels at 100KHz, you "need" another input device.  The top sampling rate of the 6211 is 250KHz, or 125KHz for 2 channels, or 62.5KHz for 4.

That said, it's not clear to me why you can't "Producer/Consumer" a single channel at 50KHz without falling behind.  I've not used LabVIEW on a Mac, and cannot open your Zip file to see what you are doing (WinZip says "The following invalid filename was encountered in the archive: "Multichannel Recording Function\Multichannel Record\Consumer Data: Consumer Event Com.ctl".  It may be a Mac/PC problem with the colon ...

Bob Schor

[Jordan_M]

Hi Scopes,

I noticed that you are processing (decimating) your data and writing it to disk in the same loop.  Your write speed might improve if you move your processing out of that loop.  Also your decimation function could be improved.  It uses a build array function to create very large arrays, which causes LabVIEW to allocate memory over and over again, in larger and larger chunks.  This is slow, and it leads to memory fragmentation, which could eventually cause you to run out of memory.  You could improve the speed and memory performatnce of the decimation VI by allocating a large array initially, then filling this array up with data as the data becomes available.  Of course this would require up-front knowledge of the size of the array that you want to store.

[Scopes]

Hi Bob & Jordan,

Bob, thanks for the input about the hardware. I do have another piece of hardware that can have 100 000 sample rate I am just using the current device to even get 50 000 Hz possible. I am sorry that your computer can't open the folder, but I am not sure how to solve it, possiblily a 7 zip folder once I will upload that.

Jordan, where exactly could I moe the decimation outside of the loop? I understand that it could greatly improve the efficiency of the program but how will it continously decimate the data? I am currently working on the premaking the array before the decimation and see if that will cure my problems.

[Bob_Schor]

Well, I persisted, and managed to open your ZIP file.  I'm assuming the file to look at is the one buried two folders down in Multichannel Record\Main VIs, and is the one numbered .5.  [My preference is to invert the order -- I put my top level VIs in the base folder, and bury the sub-VIs and TypeDefs in sub-folders, but to each his own.]

One thing that puzzles me is having two queues governing the Data Producer and Consumer loops.  I can understand a reason to turn the Producer On and Off (to suspend sampling, for instance), but the Consumer can be "left on", as if there is no data to Consume, it safely blocks itself.  Having Too Many Cooks Spoils the VI Broth.  You can also consider letting the Producer run all the time, and having a "globally-readable switch" (a Global or a VIG), perhaps called something like "Acquisition Active", that governs whether the data get sent to the Consumer or discarded.  This seems to me to simplify the P/C logic (One Queue to Rule Them All) and to maximize throughput.

Bob Schor

[Jordan_M]

Hi Scopes,

There are some ways to move the decimation to a separate loop, like adding another queue, but they are not easy, and they could be messy.  I would recommend a simple test before you spend too much time on that: put a diagram disable structure around the decimation function, rerun your code, and see whether the speed increases.  If the program gets faster, you will know that the problem is in the decimation vi.  If not, you can focus your attention elsewhere.