How can I write to an array on a specified time interval?

[powerpan4]

Hi all,

My current project requires me to read measurement data from load cells and thermocouples, record the data, and use the load cell data in a control loop to change a motor's output. So far I am successful in gathering all of the data into an array and writing it to a file. The problem is that since the data is being acquired so quickly (continuous samples from a NI 9237 and NI 9210), the file being recorded to has more data than it really needs to have. Data is being acquired on the millisecond level, and I only need to record measurements on the level of once every 60 seconds.

For context, this program is for a setup that will be logging data for exactly 1000 hours, and displayed on a logarithmic time scale, so data logging at this speed is unnecessary. 

I do not want to slow down the acquisition rate, as it would be a headache in it of itself to mess around with sample rates and buffering of my hardware, and the load cell force data is to be used in the PID control of a motor. All I want to do is decrease the rate that data is put into the measurement file's array.

I tried using the "Elapsed Time" function but I couldn't get it to work.

How can I implement a system that only allows for the building of an array (adding elements) on a specified time interval? Is that even the best way to go about it? Is there another way that involves writing to the measurement file in a loop? This one is a real head scratcher for me. Thank you in advance!!

the "build array" with 5 orange inputs on the left side is the one in question

[PalanivelThiruvenkadam]

1.Obtain Queue had different datatype but you are enqueuing different datatype

2.You have to create a logic to build array whenever elapsed timer is On and you have to reinitiate the start time again.

3. The Data you are adding to the queue is not dequeued.

Store data once for every 60 seconds.

[altenbach]

I cannot look at your VI (only LabVIEW 2020 here), but your picture raises many red flags.

• You are building ever-growing arrays in shift registers, making it harder and harder on the memory manager as data is acquired.
• Then you are updating xy graphs on this ever-growing data, taxing the UI thread. None of this belongs in the acquisition loop.
• You are only saving to file after the toplevel loops are completed and if anything goes wrong, the data is irreversibly lost.
• You seem to acquire software timed one sample at a time. Hardware timed would be preferable.
• You can easily average a certain amount of data for decimation and there are several techniques.

[powerpan4]

Thank you for your reply and comments, this is my first large-scale project after learning the basics and there is much I have to learn.

• I was unaware of the possible memory issues involved with the growing shift registers. What other ways could I add the data into an array that isn't so memory intensive? Or is there a more efficient route in general rather than shoving it all into an array?
• You are correct on the graphs not belonging in there. Those were there for the beginning stages when I was doing some old trouble shooting, but I no longer need them anymore and they will be removed.
• Do you suggest that the data should be saved concurrent to the acquisition instead of the way it is now? if so how?
• Unfortunately it appears that NI 9237 I'm using for data acquisition is best suited for continuous sampling,which is what I'm doing now.
• Could you describe one of the decimation techniques, or post a link to a source that goes over it?

I'm sorry if I'm asking too many questions at once, I just figure that specific questions are needed for good answers and I want to learn as much as I can. Thank you again.

[PalanivelThiruvenkadam]

---

@powerpan4 wrote:

Thank you for your reply and comments, this is my first large-scale project after learning the basics and there is much I have to learn.

• I was unaware of the possible memory issues involved with the growing shift registers. What other ways could I add the data into an array that isn't so memory intensive? Or is there a more efficient route in general rather than shoving it all into an array.

---

Even Array will have the same impact (You can use Logging Module, which writes to file every 60 sec . Use Queues to transfer data from Data Acq Module to Logging Module)

[Bob_Schor]

Hello, PowerPan4.

     Is this an accurate restatement of your basic question:  "I am collecting data from multiple channels at 1000 Hz (which I use to do something).  I would like to save a sub-set of these data, namely write one data point every 60 seconds to a files (along with a corresponding TimeStamp)."

     If this is your goal, and the underlying question is "How to I save a "sub-sample" of sampled data to a data file", here is a method that I've successfully used:

• In the DAQmx code that takes the 1 kHz Data Sample.  If your buffer size is 1000 (so the DAQmx Read fires at once/second), you have a 2D 1000xN Dbl array generated at 1/second.  Use a Producer/Consumer pattern to send this Sample to the "Save to Delimited Spreadsheet" (which is a "superset" of a CSV file).
• The Consumer ("Save to Delimited Spreadsheet") does the following:
  1. Save both the Time (as a TimeStamp, or "# of seconds since start of Program", or the sample number, e.g. 1, 2, 3, ...) and the Sample in an Array, counting how many values you have.
  2. When you have 60 samples (which takes 60 seconds), average the Data Arrays and assign the time of the first sample, the last sample, or the "average" time-of-sample.  Now make a 1001-element Array with "Average Sample Time" concatenated with "Averaged Sample Array" and write to the File.  Note there is a Boolean input to this function to "Append to file", which needs to be True to keep appending to the end of the file.

Note that the Consumer gets called at 1 Hz (or once/second), but does trivial processing for 59 of its calls and does file I/O in the final second, which requires opening a file, finding its end, and appending to the end, which probably takes a few milliseconds, at most.

Bob Schor