Where to put data processing routine when acquiring data using DAQmx

[pblase]

I have a program that is aquiring data using the DAQmx Acquire N Samples mechanism with automatic reset and a data handler callback routine. DAQmx acquires N samples (usually 1024) from the board, calls the handler to do something with it, and then resets to get the next batch of data. The program acquires a number of lines of data, say 512 lines of N points each, with one callback call per line. Triggering is done by a hardware trigger at the start of each line of data. So far so good.

The issue is that the time that it can spend in the callback is limited, or else the callback is not finished when the next batch of data is ready to be transferd from the DAQmx buffers and processed. There is a substantial amount of analysis to be done after the entire frame has been acquired, and it ends up taking far longer than the time between lines; so where to put the processing? The data acquisition is started from a control callback callback that exits back to the idle loop after it starts the data acquisition process, so there is no code waiting to execute, to return to, when the data acquisition is finished.

I could try to put the data analysis routine into an idle-time routine and trigger it with a semaphore, or I could put it into a timer control callback with, say, a 10 millisecond repetition rate and poll a flag, setting the flag when all of the data has been acquired. Any suggestions would be appreciated.

[D_Biel]

I would recommend using Thread Safe Queues. Your acquisition callback can place items in the TSQ and then you can process the data in a separate thread. TSQs are nice because they allow you to install a callback function to run for certain events. Most importantly, you can install a callback for the Items in Queue or Queue Size Changed event which will run the callback if a certain number of items are in the queue. This lets you take advantage of multithreading in a simple and safe way using a standard Producer/Consumer architecture. However, you may still run into problems with this architecture if your acquisition thread is running much faster than the consumer thread. You could eventually overflow the queue. In that case, your only options are to either get a faster system, slow down the acquisition or do the data handling post process.