Problem saving data to file using for loop

[wygonski]

I am trying to write 1-D sampled data to a measurement file every 10 samples for efficiency. I have a producer-consumer architecture so that the acquired data is enqueued in a separate while loop. The consumer loop dequeues the data and builds an array in a for loop, and writes to the lvm file when N=10 samples have been accumulated.

The user interface has a FileSave button (switch when pressed, handled in an event loop) that controls whether the incoming data is Enqueued for writing to the measurement file. Each time the File Save button is pressed, a new file is created with the number appended to the base filename is incremented. (This is handled automatically by the Write to Measurement file Express-VI). Data is written to the file as long as File Save is on. This is all working as desired, except for the following:

In the consumer loop, the FileSave local variable terminates the for loop. However, on the second and subsequent times File Save is started and stopped, the beginning of the new file contains "old" data (always less than N samples), and I suspect that it's because the For loop is terminated with a partially-built array, such that the next File Save continues building on the array with the old data. What I intended is that the for loop terminates before completing N passes, and the next time it starts, the array is built up from scratch and the partial data is lost, which is ok. However, I'm not sure whether my code does this correctly, and I'm looking for some insight on this. I've posted screenshots of the consumer loop showing the array being built and also the saving to the file.  

Thanks in advance for your help.

[RavensFan]

---

wygonski wrote:and I suspect that it's because the For loop is terminated with a partially-built array, such that the next File Save continues building on the array with the old data.

---

I don't see why that would happen.  If your For Loop stops and restarts later, you've got an empty array that initializes the shift register.

I think it is more likely you have stale data in you queue.  But it is hard to know since you posted a screen shot of only a part of the program.  Try attaching the actual VI.

[wygonski]

Thanks for confirming that the array should be empty when for loop restarts, and that the problem is stale data in the queue.  

I suppose a remedy is to flush the queue when FIleSave is turned off.

I'll work to post the VIs now.

Thanks,

John

[wygonski]

I've uploaded the VIs.   I'm enqueueing the data in Mag UI (SubVI).vi and dequeueing in Main.vi.  

I do see that it's likely there's stale data in the queue.   If you could take a look, it would be much appreciated.

[RavensFan]

I do believe it is stale data in the queue as your producer is continually adding data to the queue.

I would let the producer loop determine if data is supposed to be saved and only enqueue data in the event File save is turned on.

That, or I would move the Dequeue out of the For loop and put it the top level of the consumer's while loop.  Something just doens't feel right to me like you have something inside out by moving it into the For Loop.

I'm not 100% sure either of these things are causing you stale data.  I think you'll need to debug this yourself using extra indicators and probes and perhaps highlight execution to watch out the arrays build and the data flow while you run your VI.

[wygonski]

Ok, I'll try your suggestions and let you know how it turns out.

Question about your suggestion "move the Dequeue out of the For loop and put it the top level of the consumer's while loop".    If I do that and the queue is empty, what is the flow of execution?   Does the while loop suspend until there's an element to dequeue?  Then, if there's, say, one element that's dequeued, doesnt the for loop execute N times operating on the same element?  

[RavensFan]

Yes the Dequeue will wait until an element arrives.

Why do you need the For Loop?  Just store the data in a shift register in the while loop.  Add data to the array, if you have 10 items then in the true case write it to a file and pass on an empty array into the shift register.  If you don't, then just pass on the array to the next iteration.

[wygonski]

Ok, I see that I don't need the For Loop, thanks.

In my producer loop I do only Enqueue the data if FIleSave is turned on.    But if I'm waiting for 10 elements to be dequeued and I only have dequeued, say, 4 elements when the FileSave goes inactive, I'm not sure how to discard those elements to prepare for the next time FileSave is turned on.

[wygonski]

Wanted to add to my post:   I suppose a timeout for the dequeue operation would signal to write the remaining data.   Would that be a good way to handle it?

[RavensFan]

A timeout is a good way to signal it.