Missing data when writing to a binary file

[ATrumpour]

Hi all, I'm using a cRIO 9047 to acquire data and write to a file, but noticed that an enormous number of datapoints seem to be getting dropped.

In a recent test which ran for 19 seconds, I was acquiring data at 1000 Hz but the file only contained 710 datapoints for that whole interval.  The data itself looked as it should, but there were about 1/26 as many datapoints as there should have been.  As far as I can tell, the missing data was relatively uniform, as opposed to occurring intermittently.  I'm puzzling over why this is happening, and I'm hoping that some of you might have some insight.

In my VI, I acquire data at 1000 Hz and enqueue it point by point into a "data queue".

A parallel "data processing loop" dequeues it and applies some simple filtering, and adds the filtered data to a file queue.

A third loop dequeues from the file queue and writes to the binary file.

Any thoughts as to what could be going wrong here?

I've attached a shot of each of those three sections of the code - please let me know if there's any additional info that might be helpful in diagnosing this.  Many thanks!

[ConnerP]

1) post code, not Screenshots. And god forbid even if there is a reason to post a screenshot, post an actual screenshot and not a camera picture of a monitor.

2) Your whole acquisition loop is gonna be abysmally slow on account of the serial chart writes. This may cause an overflow in what a assume is a FIFO read in the subvi. You really should separate acquisition and display into two separate loops.

2) You are absolutely killing your performance by writing to queue and processing one data point at a time, especially when it comes time to write to file. The bigger chunks you can write to file, the better. Pass the whole 2D array through the queue and loop through the pieces in the processing code if you need to, but I would imagine you can be much more efficient there too.

3) Why in god's earth are you converting to a string before writing to a binary file? That makes absolutely no sense. Binary or ASCII, pick one.

[JensG69 · Accepted solution]

Your enqueue-for-loop inside the RT-loop has some serious flaw, because of several autoindexing of arrays together with a maximum hardcoded loop-count of 10 !

The comment already states it: "Assumes that the number of datapoints <= number of channels".

You state that you aquire data with 1 kHz, the RT-loop has a timing of 250 ms. So I assume that the while-loop will aquire 250 samples for each of your channels. Now the For-Loop will run at most 10 times (perhaps less, if you aquire less than 10 channels), so it will put only 10 (or even less) samples onside the queue the rest will be lost.

10/250 = 0,04 which is already very close to the ratio you have posted.

Regards, Jens

[ATrumpour]

D'oh! Thank you Jens, that was indeed the problem.  I'd been staring at it so much that I missed the obvious.

[ATrumpour]

Thanks for the feedback, Conner. I'm new to the forum - I'll be sure to post code next time.

I'll definitely look at restructuring things as you suggest to avoid having the display bogging things down, and to get away from writing data point by point. I appreciate the advice.

Cheers,

Adam