Capturing data instantaneously after condition is met using 2 DAQ cards

[w_travis]

Hi all,

I am wanting to sample a pulse and a sine wave at a low frequency of 1 kHz point by point sampling and upon a condition being met - the leading edge of the pulse being greater than a certain value - I the want to take a certain number (here 270) of samples of the sine wave at 50 kHz, multiple samples. I am doing this within a case structure and using a separate DAQ card. 

The problem is that when the conditon is met is does not capture the data at the higher frequency straight away. It misses part of the sine wave before commencing capturing the 'window' of the 270 samples of the sine wave at 50 kHz. I want to know if there is any way of speeding this up? I am aware that using an FPGA could help but I don't have access to one. 

The other solution would to advance the condition, i.e advance the pulse train by a certain amount so that I compensate for the delay in activating the high freq capture. I would need to set up this advance of the pulse train so that, after the delay in activation, it captures the data exactly where it is required to. What would be the best way to do this? I could delay the pulse train by a certain number of samples/ or delay the sine wave, or I have read that you can use the sample timer/counter within the DAQ card to change the 'trigger of the pulse train'. How would I implement this?

Attached is my vi. I would be very appreciative if you could help. 

[johnsold]

w_travis,

I canot promise that these suggestions will solve all your problems but they should help:

1. Move the configuration for the Dev2 outside the loop. The time it takes to configure the task is almost certainly part of your problem.

2. Consider using finite samples rather than continuous samples for the high speed task. Then every time you call the Read, you get 270 new samples.

3. Move the data analysis and file write to a parallel loop. You do not want the time it takes the OS to write the file to delay the next data set. See the Producer/Consumer Design Patterns for examples.

4. I do not have Mathscript so I cannot test but I suspect that using LV funcitions is significantly faster.

5. Your loop almost certainly does not run at 1 kHz, especially when taking the high speed data.

Consider reading both tasks continuously and post-processing the data to get the 270 high speed samples you want.  How often do the pulses on the low frequency channel occur? What is the minimum (and maximum) possible time between successive pulses? 

Lynn

[w_travis]

Hi Lynn,

Thank you very much for your thorough response. I have tried the following:

1) Move the configuration for the Dev2 outside the loop.

I have tried this before and would love some pointers on how this could be acheived. This doesn't seem to work, at least I think that the while loop for Dev1 does not stop and hence doesn't address the case structure outside the while loop. How could I get round this? 

2) I haven't thought of this and it seems like a good suggestion - will try when I get to the lab on Monday.

3) As above

4) As above

5) This is concerning. Maybe I should explain a little further. The sine and pulse wave are both at 10 Hz and in phase. The pulse leading edge coincides with the sine wave being greater than 0. So the condition from the point by point Low freq sampling occurs every 1/10 = 0.1 seconds. 

I do not want to post process the data as I want to grab the sine wave at the high freq (Dev2) instantaneously. I want to be able to slightly change the freq of the sine wave and see instanlty the effect of Y here. 

Thanks again Lynn for your input and I would really appreciate further help.

[johnsold]

I do not have DAQmx or any suitable DAQ devices so I have not tested this.

This is a start on cleaning and speeding things up. Note that writing to the front panel indicators at 1 kHz will not work because the screen update rate is on the order of 50-100 Hz. Also charts take a considerable amount of computation beacuse they need to (1) store data in the internal buffer, (2) erase old data if the buffer is full, and (3) (slowest) recalculate all the pixels in the display for the updated data. The cart should be moved to the parallel loop. The condition True boolean will only be true for about 5 ms out of each 100.  Look at the 5 ms boolean.vi  attached. It runs close to 1 ms per iteration. The boolean is true about 5% of the time but I never see it change. Remove that boolean from your VI.

With continuous sampling on the pulse channel and reading 1 sample every millisecond (assuming you get that fast) when the high speed sampling occurs it takes 5.4 ms to acquire the 270 high speed samples. So, ten times per second the loop takes >= 6.4 ms for an iteration. The next sample it reads from the pulse channel is the one which was measured (acquired) at 1 ms after the previous sample. Thus, this data point is read ~5.4 ms after it actually occurred. The next time you detect a pulse, it will be 5.4 ms late. The second one will be 10.8 ms late. Eventually you will get a buffer overflow, but your data will be useless long before the error occurs.

The Dual Sampler Simulator.vi shows a possible approach to the issue. It simulates sampling both channels at high speed.  I generate both a sine wave and a square wave and sample both at 50 kHz. It simulates reading 4000 samples at a time (equivalent to reading 12.5 times per second or every 80 ms). The square samples are then processed for the transition using the boolean Implies function. I do not recall whether the Conditional terminal was available in LV2012. I did not get an error when saving for previous version so I think it will be OK. The same thing can be done with a while loop with some extra logic.  This does not handle the case where the transition occurs at the boundaries of the 4000 sample segment. To handle those cases use a shift register on the outer loop to pass the needed samples to the next iteration. The 800 ms Wait makes it run slower than "real" time but allows you to see what is happening.

Lynn

[w_travis]

Hi Lynn,

Again thank you so much for sharing your knowledge. You've given me some reaslly good tips there which I have been working on with the two different solutions simultaneously today.

Firstly, I looked at your version of 'Using two DAQ cards Low and High Freq Sampling2'. You're right in your explanation that a delay is inevitable. In order to try and get round it, I developed a code whereby the flag is delayed such that, in the time it takes to activate high freq capture (270 samples), it activates where it should start if it activated instaneously. Ie I delayed the flag by around 3/4 of a cycle and then used this as the input to my condition. This solution is not very accurate or elegant.

I then looked at your second solution 'Dual sample simulator'. I came up with something that used multiple samples myself a while ago but I used a Mathscript (slower) and collected N cycles and so it wasn't exactly instantaneous. This is much more elegant. I guess your reason for using 4000 samples per loop is that computationally you don't exceed the time it takes for each cycle to come and you isolate each cycle?

I am worried that this won't be so flexible if the frequency of the wave is increased. But I guess you could change this accordingly. Secondly, if you use the wait function it comes up with an error in that the buffer size has been exceeded. I will also be superimposing a burst of high frequency waves onto the sine wave and will require to filter the 270 samples window. I am hoping that this solution will still hold up given the increased computional demand.

Assuming this solution is OK. How would I configure the shift regiesters such that I don't miss the part where the condition is met on the edge (either at 0 or 4000). I guess I would need a buffer each time of 270 each side? I have tried to do this but failed. Would you be able to point me in the right direction? Thanks again. I post my latest version. Notice a couple changes such as recording the sum. (I have left out the log for convenience) I only record the sum when it is 270 elements long since these are the only ones I am interested in. (Not the ones where it equals zero when there a condition is not met).

Thank you so much again in anticipation.

[ColonelFlanter]

Also in anticipation of your response

thx

[johnsold]

ColonelFlanter, are you w_travis colleague, instructor, boss, or just interested in the topic?

w_travis,

The 4000 samples was rather arbitrary. The selection of the number of samples to acquire is a trade-off among factors such as how quickly you need to respond, the frequencies of the low and high frequency signals, the sampling rate, buffer sizes, and how long it takes to process the data.  If your pulse signal was at a known, constant frequency, and if you could synchronize with it, you could always select a samples to read value which contained the desired data. But those conditions usually cannot be met and become quite messy with variable frequency signals.

You do not need the Wait.  The DAQmx Read will wait until it has acquired the specified samples.

The additional computation is not a problem when you use the parallel loop approach. In fact that is exactly what parallel loops are for! Consider a variation on the two VIs I posted: Sample both the pulse and sine channels at high speed and pass all the data to the parallel loop. In the parallel loop find the trigger point, extract the data subset, and process that data.  The write to file should probably be outside that loop. If you need to write to file multiple times over an extended run (many minutes to hours or more), an additional parallel loop for file I/O might be appropriate.

The attached VI shows one way to handle the segment overflow. It has a minor bug when only on datapoint is in the segment.  I did not take the time to track it down. By "segment" I mean that all 270 samples are in the 4000 sample block.

Lynn

[w_travis]

Hi Lynn,

Your last post was superbly helpful and I feel I am etching closer to a good result. I have changed the condition slightly such that it ackowledges when there is an 'incomplete segment' slightly differently. It seems to work. I wanted to check whether I am passing the data to the parallel loop correctly and using the queue function in the right way? I am also having trouble implementing your suggestion of writing to file in a parallel loop, it always seems to write just one element when I try and do this. Could you perhaps point me in the right direction (assuming I write in a seperate loop again).

Lastly, I want to obtain information on the timings of the while loops. Like how quickly is the main while loop performing, is there a lag? And how can I determine how quickly the tasks are being performed in the outer while loop? I am also sticking with using 4000 samples as the window size each time for now. I am still curious as to how to optimise this. I would know the cycle frequency of my data (to +/- 1% and ideally be able to change it). I have a dilemma. Would increasing 4000 to equal the sample length of my data make it perform better, or would it not matter considering the conditions are set up in the case struture to combat any scenario, ie the 'incomplete segment'? I would appreciate your views on this. 

Thanks again and I hope I won't have many more questions!

[johnsold]

The 2D array in the upper loop typically will start with 1 row and 270 columns. On the next iteration it will have 2 rows and 270 columns. On those rare occasions when zero or two segments occur in the same iteration of the while loop, it will add the appropriate number of rows. The reason I put a 2D array in Dual sampler simulator.2.vi was to accumulate all the segments so that they could be written all at once to the file.

What you have in your VI is that the 2D array continues to grow but you send the whole thing to the lower loop on each iteration.  It would probably be more appropriate to send only the 1D array of 270 samples whenever you get one. This would eliminate the 2D array from the upper array and put it into the lower one.

In your lower array you extract the same first row from the 2D array (which grows at the bottom) every time. Then you sum the 270 elements, getting a single number. You convert that to an array with one element and write that array to the file. In your earlier VI you took the log of the sum and then divided by the number of elements in the array.

So do you want to save one number per pulse or all 270 samples?

1. To save one number per pulse: Get rid of the 2D array. Put the 270 samples array into the queue whenever you get a complete segment. Do not Enqueue in iterations where a complete segment is not available. This means that the Enqueue will be inside appropriate cases of the case structure. If it is possible to get 2 complete segments in one iteration (as in case 0), you would need two Enqueues in that case. Cases 3 and default would not have Enqueue.

In the parallel loop Dequeue, sum (and any other math), then build an array of the results.  The false case would pass that array without appending any data. Outside the loop write the array to the file. If you want to write before the end of the run (say when a button is pressed or when the array reaches a certain size), put the write in a case structure and empty the array after the data is written.

2. To save all 270 samples of every segment: Same as the first paragraph in 1. In the parallel loop Dequeue and build a 2D array rather than the sum and math. Otherwise the same as 1.

Note: Use the File path control rather than a string control for the file name. The user can click on the folder button on the right to get a file dialog. This avoids typing errors and having users to find the complete path.

Timing questions: In the upper loop you have the DAQmx Read, the for loop which finds the pulse edge, comparisons and logic, a case structure with Array Subset, and Enqueue. At 50 kHz and 4000 samples per read the DAQmx Read will wait about 80 ms to get the data.  Everything else probably takes a few microseconds. This loop will run about 12.5 iterations per second.  The other loop will run when it gets data from the queue. If the file write occasionally takes longer than 80 ms, the data will accumulate in the queue.

The DAQ timing dominates. The only issue would be how big the DAQ buffers are. If you get buffer overflows, data could be lost. There are both hardware and software buffers. The details are dependent on the DAQ devices and how things are configured. I cannot help with that as I do not have DAQmx. 

Lynn

[w_travis]

Hi Lynn,

Thanks again for your last input I feel I am extremely close to the final result.

I've been working on this since my last message and have been happy that the condition is working correctly. However, I recently wrote both the 'Raw data' and the computed segment result to file. (Just as a check is there a good way of writing the raw data to file outside the loop (assuming this is quicker?). At the moment I write the raw data to file inside the loop.) I wrote both the raw data and the computed segment result to file so I could check that I am not missing any segments in any cycles. I.e I am checking that no. of cycles = no of computed segments.

These were not exactly equal. (V close). I realised that the error is highly likely to be when the condition switches to 0 where there is a 'previous overflow and whole segment'. This produces two arrays which need to be queued and transferred to the next while loop to compute the output.  These need to be outputed into the next while loop one at a time.

In your last message you told me to ''If it is possible to get 2 complete segments in one iteration (as in case 0), you would need two Enqueues in that case''.  I originally added an enqueue element to the second array and then joined it to the 1st array. However, I now realise that instead of adding it to the queue, if two queues occure simultaneously, the largest is taken. This obviously means I will miss a segment.How can I ensure that if there are 2 complete segments that they are queued and then computed in the second while loop one at a time. 

How can I get round this? I have carried out many different techniques and even looked at your solution here: http://forums.ni.com/t5/LabVIEW/Multiple-Queues/td-p/1667604
but I am not getting anywhere. Do you think you could help me please? Thank you so much.