write to file every hour

[Xiao@Gemini]

My calibrated data is 2D string with 132 sensors, how can I write the first 66 sensors' data to a .text file every 1 hour and the rest of 66 sensors data to another text file also every 1 hour. Should I create a while loop inside the inner case loop and count the time elapsed, or if you have a better idea. Thank you very much.

[Bob_Schor]

It is so much nicer (for us) to open a VI, which lets us see all the code, the Case Statement elements, lets us edit/correct/test elements, etc.  What is your program supposed to be doing the other 59 minutes it is not writing to disk?  Anything?  How often do you actually gather data (it sounds like you're saying "once an hour")?

Depending on the answers to the above (and a better understanding of just what you are trying to do), several things might be possible.  One thing you should consider is possibly doing an "Open File, Go to the End, Append new data, Close File" sequence so that your data file is "vulnerable" only once/hour (suppose your program crashes after 23.4 hours -- if the file stays open, you potentially lose everything, but if you do an open/end/write/close sequence, you've got the first 23 hours safe.

Hmm -- I just noticed you said "... to another text file" -- are you thinking of writing many different text files, once/hour?  That seems silly, to me, but it's your data and your files ...

Bob Schor

[Xiao@Gemini]

My description might be too vague, I'm acquiring 1 data per second for 132 sensors, I want to write one text file per hour for 66 sensors which would be 3600 data, one text file for another 66 sensors. Two text file with different names since it's corresponding to different test sections. The data acquisition run 24/7. 

[RTSLVU]

WOW that code gave me a headache...

But how about this:

1. When you first start saving data save the time in a shift register.(StartTime)
2. Every time after that when you save data check the time (CurrentTime)
3. IF  CurrentTime >= StartTime+1 hour create a new file

[Bob_Schor]

Well, I cheated and wrote Binary Files, and simulated data from 66 sensors sped up by a factor of 1000 (since I took data once a millisecond instead of once a second, but who wants to wait an hour?).

This is a Producer/Consumer Design.  The top loop acquires 3600 samples once a "tick" from two sets of 66 sensors, then combines the two 2-D Sensor Arrays into an All Sensor 3-D array and puts it on a Queue.  The bottom loop takes data off the Queue and if the array isn't empty (see next paragraph), splits it into the two Sensors and writes each Sensor into a separate file, concatenating the data by "adding to the end" of the files.  If you start with existing files, it will just keep appending.  The files stay closed when the Writes are not taking place.

The Stop Button has been changed to "Switch when Pressed" and placed within the hourly collection loop to ensure you get a full "Last Hour" -- you can change this behavior if you want.

When the Producer Loop ends, it places a "Sentinel", an empty array, on the Queue, but otherwise leaves the Queue alone.  When the Consumer sees this, it knows the Producer is done, so it can safely Release the Queue and stop itself.

This is probably not exactly what you want, but should have elements that work for you.  The Producer/Consumer Design means that the Data Acquisition and Data Storage work independently of (and simultaneously with) each other, so no Data are lost between Acquisition and Writing.  If you also want to plot the data, you could do that by putting your Queue inside the "3600" loop and exporting every data point to the Consumer.  If you want to (a) Plot Every point (once/second) and (b) Save Every 3600 points (once/hour), you just have two P/C loops, Acquisition -> Plot -> Write File.

I encourage you to test code by writing Simulation Routines such as this.  They can be small (see below) and used to test ideas and find bugs (I initially had my "Build Array" function in the top loop in the wrong place, and got very strange File Sizes ...).

 Bob Schor

[Xiao@Gemini]

It's like one-hour interval is controlled by sample size 3600, I'm acquiring data through a cRIO and two expansion chassis with real-time scan engine which outputs one data per second to the second while loop, within this loop, raw data is calibrated and plot or saved. Currently, the file is written at the same rate as the data collection 1 HZ, I would like to modify it when 3600 data is written, create a new file. Any suggestion? Thank you very much.

[RTSLVU]

---

Xiao@Gemini wrote:

It's like one-hour interval is controlled by sample size 3600, I'm acquiring data through a cRIO and two expansion chassis with real-time scan engine which outputs one data per second to the second while loop, within this loop, raw data is calibrated and plot or saved. Currently, the file is written at the same rate as the data collection 1 HZ, I would like to modify it when 3600 data is written, create a new file. Any suggestion? Thank you very much.

---

1. Open a new data file and zero a counter. (Counter)
2. Increment the counter everytime you save data (Counter=Counter+1)
3. IF  Counter >= 3600 THEN close the current file and goto 1
   1. Else goto 2