DAQ, TDMS, and Architecture

[RandyC]

I wrote a LV program a while back, and have been using it with limited success.  I needed to get some tests run at the time, and I learned how to make the program work just well enough to get the results.  Now, I am more familiar LabView, and I need a lot more functionality out of the program than it currently has.

I am using LabView 2009 and cDAQ on Windows XP.  I am using the 9172 cDAQ chassis, with several input and output modules.  These modules will measure temperature, pressure, and flow, and use the information from sensors and a PID loop to a flow control valve.  I am reading most of the sensors at 500 S/s.  My program is written with two main loops, one for input and one for output, and not in a producer-consumer architecture.  The general structure is similar to this article:

https://decibel.ni.com/content/docs/DOC-11300

Now, I want to add data logging capability.  I think TDMS is the best way to do this.  However, the TDMS must be at a high rate to capture a pressure pulse, at around 100-200 kS/s.  Obviously the files get too large and the performance suffers greatly if I try and read all channels at this high rate.  The program will need two different sample rates, and don't need to happen at the same time.  I will call this low-speed and high-speed data.

The low-speed data will need to be read and written at about 4 S/s.  The high-speed data, input only, will need to be read at 100-200 kS/s.  The low-speed data will be the "normal" operating rate and will have user inputs, while the high-speed data will only need to be run for 2 seconds or less and requires no user inputs.  I would like to be able to toggle between the two, either automatically or by a user prompt.  The data for both low-speed and high-speed acquisition needs to be saved to a TDMS file(s).  It can be two separate files.

What is the best way to write this program?  I have been going through producer-consumer styles, state machines, queued state machines, event structures, etc...  I can't see where one will be better than the other, and I'm not sure where to start.  Can someone point me in the write direction?  I'll post code when I have an actual specific question, but since this is an general overall question, I would like to rewrite the code beforehand.  I found these other articles, but neither seem to be exactly what I am looking for.

https://decibel.ni.com/content/docs/DOC-2431

https://decibel.ni.com/content/docs/DOC-3792

Thanks,

-Randy

[deppSu]

I'd like to point out something come to my mind.

well, it seems that you got some worry about the replay (read) speed of TDMS. It does take time for TDMS to open a file and parse all the contained meta information. However, it you keep a TDMS file open during the logging and replaying process, the bottle neck will be the IO speed of the hard disk no matter you use one TDMS file or 2 TDMS files. 

My point is that you can probably try writing while reading the same TDMS file which will simplfy your programming complexity.

[BillMe]

I took your "read and written", to mean read from the hardware and written to disk. Is that correct?

[RandyC]

deppSu: Yes, I was able to read and write raw data at the same time.  It seemed to be OK, but after 30 seconds of reading all of the channels at a high rate, I generated a 16MB TDMS file.  And I would like to see the data as an indicator pressure units.  I need to break the data out, multiply each by a certain constant, and display though an indicator.  I tried doing that, but it slowed the performance down and the program became very choppy.

BillMe: Yes, I currently have been reading directly from the hardware.  My goal is to read from the hardware and write to a file/disk.

[Joshua-B]

Hello RandyC,

One option you could try is to display the data in a separate while loop.  I started with the example VI found in the first link you posted and then moved the graph to a separate while loop.  I used a local variable to then allow the data to be sent from the first while loop to the second while loop.  Using this structure could potentially cause some data to not be displayed, but will log all of the data for future use.

[RandyC]

Josh,

Thank you for that idea.  That does work for me in a test program I wrote.

However, how do you think I could toggle between two sampling rates without stopping the VI? 

[Joshua-B]

Hello RandyC,

You may need to use an event-based state machine to accomplish.  I've attached a link below to a Community example that accomplishes this.

Change DAQmx Sampling Rate While Continuously Acquiring Data

https://decibel.ni.com/content/docs/DOC-5712

[RandyC]

Thank you Josh. I think I have enough information to go with now.  I'll start a new thread if I have another specific question.