Large data file management in producer and consumer programming layout

[Satpreet]

Hello all,

I have made a code using producer and consumer layout. My consumer loop is saving file. Everything is working fine but due to high sampling rate, file becomes too big too fast reaching 4Gb in some seconds. 

For some reason, Labview doesn't allow to save files bigger than 4GB and its becomes harder to transfer this big file from Target PXi8108 to my host PC.

In the normal programming, we can use DAQmx read property to log data into a file with fixed number of samples. However, this doesn't work with producer and consumer layout.

My question is how can we make LabVIEW to save TDMS data files in lets say 500Mb packages. So, what I mean is instead of one 4Gb file, I can have 4 512Gb files. 

Is there any property that I can use to do it. I will really appreciate your help.

Thanks

[RTSLVU]

4 GB file size limit is due to the FAT32 file system the PXI 8106 uses, not LabVIEW.

On a Windows computer with NTFS file system the file size limit is like 16TB. 

But right off the top of my head, I suggest you keep track of the number of samples. Then do the math to determine file size and create a new data file as needed.

[Satpreet]

Thank you for your response.

My data logging is really fast and I can't afford time to stop and start new file. It needs to be automated.

[RTSLVU]

---

@Satpreet wrote:

Thank you for your response.

My data logging is really fast and I can't afford time to stop and start new file. It needs to be automated.

---

So automate it... I should not have to tell you that. 

The whole point of the Producer/Consumer architecture is the Queue holds the data so the Consumer loop can do things like process data, open and close data files, and etc. without losing any data or slowing down the data acquisition.

[Satpreet]

Thank you for the advise. I know I need to automate it but is there any example to do it?

I haven't been able to do it. I am using shared variables I am stuck at point that I am creating file name outside the while loop and how to append or change name inside the loop.

I will really appreciate any example.

Thanks