Finite data storage, retrieval and display within a while loop

[jcannon]

Hi all,

I am building an instrument that I can use to control a transcranial magnetic stimulator and and motor evoked potentials.... All is coming along nicely so far...

I have a question regarding how I can store a finite data stream (say 1000 samples) as it passes through a case structure for 10 iterations of the "true" state only and manually (using a button control or the like) retrieve a previous iterations and have my graph update and display this stored data... The idea is that I need to be able to flick back and fowards between itertaions so can be view and compare the waveform shapes.

I could probably do this my self by writing to file and opening the file to retrive the data... But is there an easier way? The iterated data is for display only (storeage on file is not needed) as the entire data file will be stored and properly analysed offline...

Any help would be greatly appreciated..

Thanks!

[DFGray]

I would suggest breaking your code into multiple loops, each handling a particular task.  To do this efficiently, you will need to learn about state machines and producer/consumer architectures.  Look in the LabVIEW help first, then search these forums for additional information (and there will be a lot).  Given your current code and request, I would suggest the following to start:

1. UI loop containing an event structure.  This loop issues commands to the other loops (via queue, typically).
2. Acquisition loop.  This will be a state machine which takes data and dispatches it somewhere (graph or storage, typically).  It could also include analysis.
3. Display loop. The data could come from either a file or live data.

This could be expanded to include other loops such as an analysis loop.  Doing your code in this fashion will remove one of your current problems - polling to get the current value of variables.  This will be handled in your UI loop using an event structure.

This will be a large change.  You can do it without it, but it will be more difficult and the code more fragile.  If you are planning on using the code much, it will be worth the effort.  If not, you can patch something together using case structures.

[jcannon]

Dear DFGray,

Thanks for the feedback... I will work on using multiple loops; likely a producer/consumer for acquisiton and display (using queues) and a master/slave for sending AOs from a control button... So in this case the main loop acquiring data will be the producer/master, while another loop will be a consumer for display and another loop will be a save for sending AOs.... Does this sound ok?

Also, could you please explain the samples to read setting with DAQ...? I am aware that in continous DAQ is refers to buffer size, but I am clear of the ideal relationship betweb sample rate and samples to read... That is to say, how mwuch should they differ... I normally use a ratio of 10:1; that is sampling rate is 4k then samples to read is 400... What would you do?

Finally, within the while loops I think I should include a wait or wait unless next multiple... These free up CPU power, but again, how do I determine how long the wait should be?

Many thanks heaps.. I really appreciate your feedback.

Best regards,

Jack

[DFGray]

Jack,

In addition to your acquisition loop, display loop, and AO loop, you probably need a UI loop to handle user interactions.  I would write the first three as queue-driven state machines and the UI loop as an event structure in a WHILE loop.

Unfortunately, there is no hard and fast rule for how many data points to take when reading from a DAQ card.  It depends on your computer, what else is happening at the time, the DAQ card, how much memory the card has, what sort of latency you can tolerate between data acquisition and data display/action, number of channels you are acquiring, etc.  In general, for efficiency, you want to acquire a few hundred thousand points at once to keep bus transactions down.  However, at 4kHz on one channel, this could mean waiting a full minute or more between data transfers.  So you need to hit a balance.  This is something that is easy to modify and optimize after you write your code, so don't worry about it too much.  Plan on tweaking it after you get everything working.

If you use queue-driven state machines, you will not need to add waits to the loops, since the loops will only execute when there is something to do.  At other times, they will be waiting on the queue.  The UI loop will always be waiting on the event structure.  Event driven programming is great!  There is a subtlety to the data acquisition and generation loops, however.  When in the run state, they will be looping continuously.  However, the DAQ code itself will effectively throttle the processor usage.  When you wait on a data set to be acquired or generated, the loop will be paused.

Good luck.  You are asking the right questions.  Keep them coming...

[jcannon]

Thanks again, DFGray...

Your advice is really helpful... I will add a UI loop also...

When you say the loop will effectively pause when generating output (such as my AO), does that mean if the data acquisition (AI) and data storage are differents loop they will not be paused and will run as normal regardless of the duration of the AO signal generate (i.e. virtual realtime display on a chart and with no pause)?.. I suppose this would be one reason why using multiple loops provides better efficiency and improves perofrmance...

I will work on improving my design. I will need to read up on few few things, this will likely take a week or so... Once I have done this, would you be willing to take another look at my (hopefully) improved vi? I would really value your feedback...

Regards,

Jack

[DFGray]

Yes, since AO, AI, and UI are in different loops, they can be independently paused, started, etc.  Multi-loop programming in LabVIEW is a lot of fun.

TIP: Don't be afraid to put your AI, AO, etc. loops into subVIs.  The only loops that should be on your main diagram are the UI loop (so you can get the events) and the display loop (so you can efficiently write data to the graphs/indicators).  You can find an example of this here (although I need to rewrite that app, the cluster of object and reference did not really work that well).

I would be happy to take a look at your revised code.  Take your time.  The more you learn, the easier it gets.

[Ben]

Lets see how fast I can dump.

Analyze the data paths first. maning, drwa up a diagram showing where the data starts and its path to its final distination (file display etc).

Shorten the paths.

Create enities (loops, sub-VIs) for each entitiy.

Connect the entities with fast paths (queues generally).

When acquiring I configure the buffer so that an acq can be interupted for an extended period of time (this allows for Winodws to go off and open a large Excel file while the app is running). This is typically 30-45 seconds worth of buffer. That is just the buffer!

When acquiring I limit the number of times I grab data from the hardware to tops 30 Hz, if I want to make it look like it is is flowing, 10Hz is better  and if the customer will toloerate it, 1 Hz is nice since you can still see it is running and lets you batch a lot of data processing.

The number of samples acquired is variable. I use a technique of first querying how many samples are in the buffer and read whatever is waiting. This avoids the DAQ waiting for samples and the code to support allows it to "catch-up" when that Excel sheet finally opened.

As the dat is acquired pump queues to any of the clients waiting for the data.

As to your original question...

learn to use queues. YOu can use single queueus or multiple quues to store data and quickly access it without having to move it around.

Have fun!

Ben

[jcannon]

Hi Ben,

Thanks for the great feedback... I will work on implementing these changes..

Regards,

Jack

[jcannon]

Thanks again...

I will get cracking on this and reply to this posting again once I have made the necessary revisions.

Regards,

Jack

[jcannon]

Hi DFGray,

I have started working on implementing the changes you suggested...Using separate loops for AI and AO has greatly reduced the delay in I was experiencing in the generating the AO when writing multiple pulses AND prevented the AI loop from pausing... Thanks heaps.

However, I notice that the frequency rate for the AO is FASTER than the frequency value I set using a control.. I tested this by connecting the AO to the "trigger in" on my device, and "trigger out" from my device to an AI channel and viewed the interspike interval (ISI) from the in zooming in the waveform chart...

Basically, when I input a freq of 25Hz the ISI is ~34ms (not 40ms), when I input 50hz the ISI is ~16ms (not 20ms), and when I input 100hz the ISI is ~8ms (not 10ms)... It apears the measured freq is about 20% higher than the control value..

Any idea why the ISI is not as expected?... Do I need a wait in one or more loops?

Thanks,

Jack