Operating on data from buffer in daqmx

[TT_user]

Hello all. I have written a program to collect data from 32 channels at 10khz (simulated as a task with 32 AI). The goal is that once a threshold level is reached, I capture .5 second before and 1 second after the event and write to file. I do this using a producer/consumer and state machine architecture with lossy enqueues. Time periods are controlled by buffer size inputs. 

The issue I am having is with DAQmx N samples. If I set samples to say 1000, it outputs 1 array with 32 entries (as expected) but I need to be able to access those 3200 data points to check against my threshold. Right now I can only access the 32 points each time it outputs. I have tried various 2d array configurations and bundling/unbundling but can not figure out how to access each point individually once in my consumer loop.

Thanks for the help!

P.S. I know I shouldn't be using the dreaded write to measurement file VI and I need to improve error handling.

[Bob_Schor]

Turn off the Computer (so you aren't tempted to run LabVIEW).  Get a piece of paper and a pencil.  Figure out the "givens", which (to me) seem to be:

• 32 channels of data, with a continuous sampling rate of 10 kHz
• An unspecified "Threshold", presumably based on the data sampled from one or more of the 32 channels, but possibly also "something else"
•  A desired Sampling Window of 6 seconds, with 5 seconds being data that occurs before the Threshold event.
• All channels are sampled "in synch", even if only one meets the Threshold condition.

What hasn't been specified?  How about how precisely do you determine the sample that defines the Threshold?  Given that you have 6 seconds of data (or 60,000 samples), can you be a little "sloppy" (i.e. determine where the threshold is to the nearest 0.1, 0.01, or 0.001 sec)?

Your task involves looking "backwards" in time -- with pencil and paper, how would you do this?  Do you see any problems here?

Once you have a better idea of what you want to do, you can start worrying about how you would do it in LabVIEW.  Be sure that the "what" comes before the "how"!

Bob Schor

[TT_user]

Bob, thanks for the input. This is a proof of concept, not the final program. 

A couple of things: 

-yes 32 channels continuous, possibly even more

-the threshold will be well defined

-capture data 1/2 second before (.5s)

-in actuality I will be looking at all 32 channels and comparing them to a threshold. The 1 channel is merely a toy

I can get away with being a little sloppy, but the more accurate the better.  

Backwards in time: continuously log data into one end of buffer, remove data from opposite end, and compare with desired threshold. If true flush buffer. 

I do think I have a clear understanding of what I want to do (and if i only log 1 channel with N=1 at 1khz this program executes correctly), It is when I have to increase N that i am running into trouble on accessing values.

Thanks again. 

[Bob_Schor]

Oops, sorry I missed the "dot" in .5 (it's harder to miss if you write 0.5, but that doesn't excuse my poor eyesight).

You still haven't quite answered the question about timing accuracy.  But let's say you want to be within 5 msec of the threshhold (I'm a little confused on sampling rate -- the original post said 10 kHz, but your reply says 1 kHz -- let's assume 10 kHz).

5 ms of data at 10 kHz = 50 points per channel.  You still haven't defined the "threshold task", but let's say 4 standard deviations above the mean (in other words, an N(0, 1) random process > 4).  Here's the first thing that comes to mind:

• Producer Loop -- take 50 points from 32 channels = 1600 points, send to Consumer (written as a State Machine, with three states, Idle, Active, Empty.  Keep Producer going until you get a Stop signal.
• Have two Consumers in series.  The first one looks for Threshold, and decides if Idle (not yet Threshold), Active (Threshold, now save 1 second more), and Empty (wait for Consumer 2 to finish).  The second one manages the File I/O, using a fixed-length Queue big enough to hold 0.5 seconds of data.  While Idle, it does nothing (except open a File to get ready for saving).  While Active, it takes the data "pushed off the end" and writes it to the file.  While Empty, it writes the rest of the data to the file, closes the file, and sets the State to Idle.  Note both Consumers share the State.

 All of these loops run basically at 200 Hz (5 msec periods).  The Producer needs to be able to decide if Threshold is achieved in this time -- I think this should be easy to attain, but that's why you have LabVIEW, to write a little VI with 1600 N(0,1) points generated at 200 Hz and see if you can determine if any of them are >4 at that rate.  Consumer 1 has the easy task -- it just needs to decide when to send data on to Consumer 2 (during Idle and Active).  Consumer 2 needs to manage opening and closing files, which it should be able to do during the Idle State (when it is otherwise not trying to write to the files).

Bob Schor

[TT_user]

Bob, thanks for the thoughts.

I am not concerned about timing accuracy, since as you said. I will have enough memory to store in my buffer and check against the threshold. The threshold is simply based on my AI, say I am measuring accelerations and i am looking at capturing the signal before and after I hit 1g. 

Maybe I wasn't very clear in my initial post. What i am struggling with is that DAQmx read when using N sample acquisition outputs what i'll call "packets" (I don't know if this is the correct term) of N samples per channel. So if I am sampling 2 channels continuously at 10hz and N=10, then every second I get an array with only 2 values, one for each channel, instead of all 20. If I wire the output of DAQmx to write to measurement file i get all values, but if I convert to array and then write i get 1/N values. Can you shed some insight here?

Thanks

[cbutcher · Accepted solution]

Your measurement is made as N Channels, N Samples -> 1D Array of Waveforms, but you're using a queue with datatype 1D array of doubles.

You're losing your data in that step.

You also use Array Subset in the "Detect Event" case, where perhaps you wanted to use Index Array on a 2D array.

I'd suggest (as a first starting point) making your measurement as a 2D array of doubles, then sending those "packets" into your queue.

At the other end (top loop, consumer - by the way, usually people put Consumer loops on the bottom, this convention helps others understand your diagram), you'll need to process your 2D array packet. Index Array, dragged down, will give you the two channels. No need to wire numeric constants to the indices (by default, they start from 0 and go up).

Indexing a 2D array to get the first two (0 and 1 indexed) rows

You'll need to adjust your other sections of the Consumer loop, but I didn't look into that in detail yet.

[TT_user]

That did it! Thank You!