04-15-2020 01:56 AM - edited 04-15-2020 02:24 AM
Hi,
I am trying to acquire the data from a laser machine using NI usb 6251 for around 15 minutes. The VI continuously measures and stores the data in a .lvm file. The sampling rate is 1 Msps. However, halfway through the VI stops and labview gives: memory full error. I have attached the VI used.
My laptop properties are: 64-bit operating system and 16.0 GB RAM.
The labview edition i use is labview 2015 32-bit.
04-15-2020 02:12 AM
Hi Sara,
@SaraAl wrote:
The sampling rate is 1 Msps. However, halfway through the VI stops and labview gives: memory full error. I have attached the VI used.
Well, your sample rate is 200kHz - atleast that's the way you configured the DAQAssistent…
Generic advice: while the DAQAssistent is "nice" for quick&dirty VIs it should not be used for production system (or "high-speed applications" like yours). Use plain DAQmx functions as explained here or in all those example VIs coming with LabVIEW! (You already use DAQmx functions to read that DI channel: why not use the same for your AI data?
Also hiding a "Continuous samples" DAQ acquisition inside a case structure is calling for trouble as DAQmx will try to buffer new samples even when the case is not executed!
How fast is your consumer writing the data to the file? When it is slower than the producer loop then the queue will need to buffer data - which might blow up the memory consumption over time. Please check the queue status to detect growing memory consumption when running the next test!
04-15-2020 02:25 AM
hi,
thank you for your help. I just noticed I have attached the wrong VI. I edited the post and have the one I used in my experiment attached.
04-15-2020 02:38 AM - edited 04-15-2020 02:47 AM
Hi Sara,
so you deleted the only "senseful" part from your previous VI (those plain DAQmx functions) just to add one more DAQAssistent and a (senseless) TRUE constant before the case structure? Throwing in a sequence structure to enforce dataflow also shows lack of LabVIEW knowledge: why don't you use the error wire to ensure dataflow?
(Well, now you are reading 300k samples at 1MHz.)
Still the very same problem: how fast is your consumer loop? Does the queue grow over time?
Please check this using the QueueStatus function!
(I also recommend to replace all ExpressVIs by "plain" functions in your VI.)
One more recommendation: by using plain DAQmx functions you can also have the DAQmx task save all data directly to a TDMS file - without the need for a producer-consumer structure on your side!
04-15-2020 02:49 AM
the TRUE constant is not related to the data acquisition. it is used to start the laser machine when the VI runs. i used this VI to acquire the data at 200Ksps with N=150k and it worked fine. the problem started when i increased the sampling rate to 1Msps, but i will try it again making it "senseful" without using daq assistant.
thanks.
04-15-2020 02:52 AM
Hi Sara,
@SaraAl wrote:
i used this VI to acquire the data at 200Ksps with N=150k and it worked fine. the problem started when i increased the sampling rate to 1Msps
This fits to my assumption of a (too) slow consumer loop. See comments in my last message…
04-15-2020 03:07 AM
If you write to a binary file instead of a text based one you'll write less data and the loop should keep up easier.
04-15-2020 09:13 AM - edited 04-15-2020 09:18 AM
i replaced the daq assistant parts to read the analog input with daqmx functions, and i removed the case structure. should i also change the write to measurement file function?
04-15-2020 09:27 AM
@Yamaeda wrote:
If you write to a binary file instead of a text based one you'll write less data and the loop should keep up easier.
The real problem with the current logger the OP is showing is the use of the Write Data Express VI. That VI is constantly opening and closing the file, which is SLOW. By opening/creating the file before the loop, writing inside the loop, and closing after the loop, you can GREATLY increase the write speed.
But I would go with GerdW's last suggestion, use the DAQmx Logging to stream the data straight to a TDMS file. That is by far faster than using a Producer/Consumer setup. But a caveat I recently ran into with that is the TDMS data is a little more complicated (highly optimized for write speed) and may not work with all TDMS tools.
04-15-2020 10:03 AM
@crossrulz wrote:
@Yamaeda wrote:
If you write to a binary file instead of a text based one you'll write less data and the loop should keep up easier.
The real problem with the current logger the OP is showing is the use of the Write Data Express VI. That VI is constantly opening and closing the file, which is SLOW. By opening/creating the file before the loop, writing inside the loop, and closing after the loop, you can GREATLY increase the write speed.
But I would go with GerdW's last suggestion, use the DAQmx Logging to stream the data straight to a TDMS file. That is by far faster than using a Producer/Consumer setup. But a caveat I recently ran into with that is the TDMS data is a little more complicated (highly optimized for write speed) and may not work with all TDMS tools.
Oh really? I assumed it was clever enough to keep the reference internally. Yes, constantly opening a file is slow.