Not enough memory to complete operation when writing TDMS file.

[alfredog]

Hello,

   I am new to Labview and having a bit of trouble. I attach some code here. What I want to do is sample from a NI 9220 DAQ from 16 channels at 20kHz, while from a second NI 6009 sample from 4 channels at 1000 kHz. I want to append these together and then write to a TDMS file. 

   I have tried to write this code using NIDAQmx VIs but when I have it has resulted in the two DAQs not having the right timing with each other. The 6009 samples for a longer time. 

   I have now tried instead to use the DAQ assistant to read from the two VIs and it works in that TDMS files produced have the correct timing between the two DAQS. However, if I record for more than 2 minutes, in the end I want to end up recording for a much longer time, I have the "Not enough memory to complete operation" message appearing. This still happens even if I get rid of my charts to display the data, and also if I get rid of the NI 6009 completely and just keep the 9220 sampling at 20kHz. It happens even if I repalde my TDMS write and put a write measurement assistant in which I tell it to write a series of files that are each less than 2 minutes long. 

    I think it is something to do with the amount of data I am reading and is being held in memory. What can I do about this? Also, my charts display very slowly, basically evey second when the 20k are read in. However if I lower the amount of data read the charts don't display all the data points. 

    I attach my code, thanks for your help!

    Alfredo 

[BTC_Admin]

Yes, your code needs help:

Even if you delete those charts from your code (and it does reduce memory usage), those wires still need memory - think of them as "local variable" which are still used. However, I don't believe that's the reason you're getting error because you're not constantly appending to the array.

Your problem is in your writing to TDMS, how big does your file get? (Or maybe you're hitting some sort of file size limitation).  You could disable writing to TDMS file (tie a "FALSE" boolean to the IF case structure) and see if that will allow you to keep running more than 2 minutes. If it does then you know for sure that your problem is in that structure.

(By the way, hopefully you realize that when you have "Click to save data"/"Blank Button" placed outside the WHILE loop, clicking on it while the VI is running won't get new value inside the WHILE loop. The way it is currently setup in your VI, you'd have to click/unclick that button BEFORE you run your VI.)

-BTC

[BTC_Admin]

---

@alfredog wrote:

Hello,

   I am new to Labview and having a bit of trouble. I attach some code here. What I want to do is sample from a NI 9220 DAQ from 16 channels at 20kHz, while from a second NI 6009 sample from 4 channels at 1000 kHz. I want to append these together and then write to a TDMS file. 

   I have tried to write this code using NIDAQmx VIs but when I have it has resulted in the two DAQs not having the right timing with each other. The 6009 samples for a longer time. 

   I have now tried instead to use the DAQ assistant to read from the two VIs and it works in that TDMS files produced have the correct timing between the two DAQS. However, if I record for more than 2 minutes, in the end I want to end up recording for a much longer time, I have the "Not enough memory to complete operation" message appearing. This still happens even if I get rid of my charts to display the data, and also if I get rid of the NI 6009 completely and just keep the 9220 sampling at 20kHz. It happens even if I repalde my TDMS write and put a write measurement assistant in which I tell it to write a series of files that are each less than 2 minutes long. 

    I think it is something to do with the amount of data I am reading and is being held in memory. What can I do about this? Also, my charts display very slowly, basically evey second when the 20k are read in. However if I lower the amount of data read the charts don't display all the data points. 

    I attach my code, thanks for your help!

    Alfredo 

---

As far as your charts updating very slowly - the way your code is designed, your charts only get data when both 20K samples & 1M samples are done collecting.  Have you tried setting up DAQ assistant for continuous sampling instead of 20K samples or 1M samples?

-BTC

[alfredog]

Helllo BTC,

   Thanks for the quick reply. I have tried sampling data without saving the TDMS and the program can keep going for a much longer time wikthout reaching this memory limit.  I am moving the "Click to save data"/"Blank Button" to within the loop to be able to not save during the recording, that's a good idea. Both of the DAQs are indeed set to continous recording unfortunately. 

   I previously had made the same VI but instead of using the  DAQ assistants I had used individual DAQmx Create Channel, Sample Clock, and Read VIs. In that version of the program (which I attach here) I could save very large files (5 minutes of recording at 20kHz from the 16 channels on the 9220 DAQ and 1000 Hz from the 4 channels on the 6008) which meant 750 MB. In that version the charts were also working very quickly. However, when I looked at the files it turned out that the DAQs were not synchronized. Basically the 6008 would sample for about 100 ms extra, it seemed like it was either starting before or stopping afterward. Therefore, I tried to switch to the DAQ assistants and then the files did have the right timing but then I had this memory and display issue. 

   Thanks for the help,

    Alfredo

[deppSu]

Unlike plain binary I/O, TDMS file is a structured file format and it does require memory during logging for fast access during log and sanity check (e.g. consistent data type for the same channel).  So it's expected that you see TDMS logging consumes much more memory than binary I/O.

[natasftw]

http://www.ni.com/white-paper/3727/en/

"The binary TDMS file format is an easily exchangeable, inherently structured, high-speed-streaming-capable file format that, when combined with the other technologies in the NI TDM solution, becomes quickly searchable without the need for complicated and expensive database design, architecture, or maintenance."

A big part of the TDMS benefit is that it is a binary I/O.  Isn't it a bit strange to suggest it's unlike binary I/O?

[deppSu]

Yes, the TDMS file format is a binary format. What I want to suggest is that unlike other File I/O functions like "Write to Binary File"  in LabVIEW. TDMS is structured and holds memory during logging.