Log to Disk and Read from Different Vis

[T-S.lab]

Hey Everyone,

I'm a couple days away from starting a couple vi builds and i was wondering if anyone has advice on how to tackle the idea here. I work in an independent lab with about 60 thermal chambers throughout the lab. They are a mix of manufacturers (Thermotron, Espec, Cincinnati, etc.) and a wide variety of ages. We have NI-9213 modules throughout the lab with thermocouple wiring pulled to each chamber. These modules are always reading temps of the connected chamber and logging every 10 seconds (1 sample at 0.1Hz).

The purpose of the system is to be a secondary reading of every chamber, give the techs active plots to review ongoing testing, and to be a backup measurement system in case of power failures or other chamber issues which may stop onboard logging (for those chambers which have it). It has grown to be a crucial part of our lab and i'm looking to update/improve the overall system.

The main thing i want to improve is the accessibility of the data and actively use it to show the techs quick chamber availability and show management our lab usage.
I also want to add redundancy to the logger and add automation to stop logging to a single file and restart logging to a new file periodically. Right now i stop the logging tasks at the end of every month and restart them to keep the files from growing to 100's of MB.

What i do need to maintain is the robustness of the log as right now data is written to disk once logged. I do not want data stored in buffers until they're full as we would run the risk of data loss if a crash occurs. It definitely won't be efficient programing to open/close files constantly - but at 0.1Hz it hasn't been very taxing and i wouldn't mind if maybe a couple minutes of data was stored in the buffer before being written. 

Below is a bit of a brainstorm of the overall layout i am envisioning;

So essentially "logger.vi" would write T/C data from the cDAQ modules to disk every 10 seconds. Other vi's like "Utilization Monitor.vi" or "Active Display.vi" would then parse through this disk data to show the monitored items of that particular vi.

What I will be trying to tackle first is how "logger.vi" should be created to make the data accessible to multiple VI's at a time, some of which are looking for live data (or just slightly delayed). I think i will set this to start a new file every Sunday at midnight and use week #'s to maintain an archive of files. What i'm not sure about is

- what the best way to make the write-to-disk process efficient

- if there is anything in particular to consider if other Vi's will be polling this file for new data to update

- how time should be logged to not invoke y2k-like fear anytime day-light-savings comes around the corner.

If anyone has any advice on this it would be greatly appreciated!

Just for some future reference, below are some of the idea's i had for the other vi's shown in the diagram above. Any insight on those would be appreciated too of course;

Heartbeat.vi

- looks to make sure logging.vi is running

- if not, sends and e-mail or other notification to alert logging is compromised

- no idea if LV cab build something like this and i may need to use a different application for this type of function

Utilization Monitor.vi

- shows which chambers are currently in use

- would like to build a dashboard for techs/management to view

- simple Boolean red/green light to show occupied chambers

- might just use tolerance bands to show usage (ie: if ambient = not in use, if outside of ambient = in use)

Utilization Tracker.vi

- same idea as above but would log total hours where usage was registered

- would be called every Sunday to add hours to an ongoing pool tracked in a separate .csv for maintenance to determine PM needs

Active Display.vi

- Shows a live graph of all or selected chambers

- Simple cursors would be ideal

Sentinel.vi

- Tech can target a particular channel of the log and send warning if a specific condition is met (ie: temp >100°C)

Grapher/Analyzer.vi

- Tech can select a specific chamber for set time frame to view data

- No need to load full csv and cut out the chambers which are irrelevant

Thanks to all for taking the time to read through and send any insight/ideas on how to tackle this project!

[GerdW]

Hi T-S,

some thoughts about your huge, but nicely structured message:

---

@T-S.lab wrote:

how time should be logged to not invoke y2k-like fear anytime day-light-savings comes around the corner.

---

I use ISO8601 timestamp formatting when saving timestamps to text files…

(The other option is to write timestamps formatted to show UTC time.)

---

@T-S.lab wrote:

Utilization Tracker.vi

- same idea as above but would log total hours where usage was registered

- would be called every Sunday to add hours to an ongoing pool tracked in a separate .csv for maintenance to determine PM needs

Sentinel.vi

- Tech can target a particular channel of the log and send warning if a specific condition is met (ie: temp >100°C)

---

"Utilization": In my testbench software I implemented "operating hours" counter, which count up based on a certain condition selectable by the testbench technicians.

Example: COUNT operating hours of pump IF pump speed > 10

No need to count once per week when you can count all the time… 🙂

"Sentinel": In my testbench software I implemented a warn/alarm system based on configurable (fixed) range limits per channel. The technicians can edit those ranges.

On a warning there is just a message shown on screen (like "temperature in boiler gets hot").

On an alarm an automated (but configurable) shutdown procedure is called (like "switch off heater when boiler gets too hot, but keep pump and cooler fan running").

All the messages of the testbench software are logged to a (text) file so anyone can check what happened at which time…

---

@T-S.lab wrote:

if there is anything in particular to consider if other Vi's will be polling this file for new data to update

---

When accessing the same file in parallel running processes you create a race condition. You also need to be careful about using a suitable access mode ("read/write" vs. "read"), and you need to be careful about not blocking the file access due to "exclusive" access mode.

Hint: Avoid to open those CSV files in Excel as that tends to use an "exclusive" access mode…

Have you thought about using a database instead of plain text files?

[HorseBattery_Stapleguy]

---

Heartbeat.vi

- looks to make sure logging.vi is running

- if not, sends and e-mail or other notification to alert logging is compromised

- no idea if LV cab build something like this and i may need to use a different application for this type of function

You can do this with LabVIEW. There's multiple ways of achieving this. (You could use SMTP, or connect it to a mail server with other protocols to actually send the notification, etc.)

If you need concurrency in your data, for multiple writers and readers you  might benefit from setting up a database. It's already meant to be used for that sort of stuff.

You can set up that sort of system as well with files like .csv so long as you don't lock them when reading or writing, there shouldn't be any problems.

Some of your VIs don't need to access the data from that log in the way you describe. You could just transfer that data through any other mechanism in buffers. If power is out, that's okay because your data is still in the .csv.

In particular, it looks like these 3 VIs:

• Utilization Monitor.vi
• Active Display.vi
• Sentinel.vi

Could work with data streams. So you would only need to worry about the analyzer and tracker where the entire history is needed.

Alternatively, you could handle logging of all the data as a protected backup that is not accesible to the other processes, and share the rest of the data through queues, tcp/ip, etc.

[T-S.lab]

Hey GerdW,

Thank you for taking the time to make it through the huge message. I definitely tend to trail off!

---

---

"Utilization": In my testbench software I implemented "operating hours" counter, which count up based on a certain condition selectable by the testbench technicians.

Example: COUNT operating hours of pump IF pump speed > 10

No need to count once per week when you can count all the time… 🙂

---

I think this is a great idea and will probably how i make this work. With each temp point being 10s apart i would just need a continuous count of points outside of a set band and then multiply out the time. What i'm not certain of yet is how to rest this counter once a tech performs maintenance and wants to reset the hours to zero.

"Sentinel": In my testbench software I implemented a warn/alarm system based on configurable (fixed) range limits per channel. The technicians can edit those ranges.

On a warning there is just a message shown on screen (like "temperature in boiler gets hot").

On an alarm an automated (but configurable) shutdown procedure is called (like "switch off heater when boiler gets too hot, but keep pump and cooler fan running").

All the messages of the testbench software are logged to a (text) file so anyone can check what happened at which time…

---

---

This is exactly the vision i'm thinking, minus the shutdown procedure .. but that is just next level!

When accessing the same file in parallel running processes you create a race condition. You also need to be careful about using a suitable access mode ("read/write" vs. "read"), and you need to be careful about not blocking the file access due to "exclusive" access mode.

Hint: Avoid to open those CSV files in Excel as that tends to use an "exclusive" access mode…

Have you thought about using a database instead of plain text files?

---

This has been my biggest worry since first thinking of this project - how to protect the write process while being able to read or view live data. I have not thought of a database until now and that has sent me down the rabbit hole. I think i'm going to take a step back and explore this option heavily because this makes great sense. I'd love to be able to use something like Grafana to have specific dashboards for set equipment and even give techs the ability to make their own dashboards of interest.
Do you have any advice on a database for logging measurements from DAQmx functions? My only experience with logging in LabView is either csv or tdms.

Since i have a couple different machines logging (cDAQ modules are USB, connected to PC which is responsible for logging only a handful of the chambers) i would ideally write a local file and also write to a centralized database file on the server which all the machines contribute to. This centralized database would be what other functions (visualizations, alarms, etc.) can work with, while a local file is left alone untouched. This would give me some redundancy with network or connection errors.

Any thoughts of a schema like this?

[T-S.lab]

Hey HorseBattery,

Thanks for the thoughts on strategy.

---

@HorseBattery_Stapleguy wrote:

You can do this with LabVIEW. There's multiple ways of achieving this. (You could use SMTP, or connect it to a mail server with other protocols to actually send the notification, etc.)

---

I was thinking SMTP would have to be involved but this would be in a separate vi outside of the logger. My thinking was to point it to the current log file and send a warning if it's not growing in size. Do you see a different way of approaching this? My main worry is if someone stops the logger accidentally and no one realizes until later in time.

---

@HorseBattery_Stapleguy wrote:

If you need concurrency in your data, for multiple writers and readers you  might benefit from setting up a database. It's already meant to be used for that sort of stuff.

---

This is a fantastic idea and a DB might be the best option for the application. Do you have any advice on a DB to use or any tutorials for databases in labview? I have no experience with measurements written to a database and think it's worth some genuine time to explore. I'll be digging through the help files and any database examples to figure how to write measurements to a database.