LabVIEW
TDMS Logging w/ Excel: Multiple Sheets?
Hi,
I have a bit of code I'm using to log data from strain gages and string pots (data comes in from DAQ Assistant Express VIs). I recently ran into the issue of buffer size causing my program to crash while trying to log to Excel Files. Instead of messing with sampling rate/buffer size, I figured logging to TDMS would be worth a shot if it can be executed faster/not clog up the code execution.
In the "write to measurement file VI" I made the change, and began logging to TDMS files, which seemed to prevent crashing during periods of extended logging, which is great. However, I am running into one issue, which is the way that the data is logged, and how I access that data when I then open it via excel using the NI provided importer.
The problem is, when I open the file in excel, the data is spread out across many, many sheets. Instead of data being spread down multiple rows, with columns being different channels, I am getting a new sheet for every time-stamped entry of data. This is what gets spit out of my "write to measurement file" subVI.
To compare, I went into the DAQ Assistant and enabled TDMS logging (I figured I'd log a separate file at the source of the data, you can see in the code it's upstream of the "write to meas file VI"). This spit out a file that looked much better, with all of the data on one sheet. However, this gave me data at the sampling rate, which is much higher than I would like to Log. In my code, you can see that I average the data before bundling it and then sending it to the downstream "write to measurement file". I am assuming this is where the hangup occurs, possibly because the data is getting bundled every quarter second...making the file think these are different "groups", perhaps?
When I was logging to excel files, I didn't have a problem collecting the data the way I wanted it, which was at the reduced logging rate relative to the collection rate. Collection rate is 1000Hz, while a buffer size of 250 creates an averaged data point every quarter second. This is what I want to log via TDMS.
My question:
How can I Log data to a TDMS file using the "write to measurement file" subVI, while ensuring that all of my data is grouped together on one (or very few) sheets, while at the same time keeping my reduced logging frequency (one data point, per channel, every quarter second)?
Any help would be much appreciated, thanks!
COD. Notice (1)upstream DAQ Assistant (which produces file "B" shown below)
(2) VI calculating the mean of each 250-piece chunk of data, creating averaged data points every quarter second
(3) downstream "write to measurement VI" (which produces file "A" shown below):
File "A": Contains data collected at the desired frequency, but the layout isn't pretty, as it is
scattered between hundreds and hundreds of sheet, with only one timestamp of data on each sheet (see bottom of image):
File "A": Here's a look at one of the many sheets in "file A". I can tell what I'm looking at,
but I'd have to open every single sheet to get a look at all of the data..:
File "B": Organized the desired way, with only two sheets in the document once imported to Excel:
File "B": Looking at the second sheet in the file, which has the desired layout, but contains way too many points (also doesn't have a time stamp):
I know this can be done, but I think you will have to get away from the Express VIs to do it.
You will want to look into the examples of how to use the DAQmx API and then into the TDMS API. They are both simple, but require a little more work than the Express VIs.
Yeah I also vote avoid that express VI. Generally this statement is true. An express VI works great, until you need some minor tweak to it then it is useless. If it doesn't fit your needs, then you won't want to use it.
So hopefully you know that a TDMS file is made up of three levels. You write to a file, which has groups. Each group contains channels, and each channel contains data points. The three levels are File >> Group >> Channel. When you view that in Excel it makes each file a Workbook, each Group a Worksheet, and each Chanel a column. If you use the low level Write TDMS functions you can choose what data goes in what group, in what channel. You can also have control over properties of each of the three levels which is great for indexing and searching for reports with specific properties.
Thanks a ton all, I was able to get this going pretty quickly, but in looking around the forums and online, I haven't been able to clear up the formatting that goes with the TDMS Set Properties. I understand the hierarchy and the idea behind labeling file/group/channel, but I can't seem to get the following to work out:
- Need to carry over the timestamp that gets spit out of the Arithmetic Mean calculation
- Would prefer to have two channels (1) one channel for strain, that contains time stamp, strain from each gage, and load case; and (2) one channel for string pot voltage, which contains a time stamp, voltage from each string pot, and the load case.
Code, with highlighted portion being the area of importance, red circled areas more critical:
Current excel readout. Main concern is getting the time stamp in there with each dataset. Less important would be splitting into two groups, giving one sheet for strain data, containing all of the strain channels, and a second sheet for string pot data, containing all of the string pot channels:
Awesome, thank you. For now I'm logging all to a single column, if it proves too much I could definitely revert to the key points d0/dt etc..
The only remaining issue now has proved to be getting the correct labeling for each channel. When I leave the DDT input data as one merged signal, I can get all data to display as I desire, but not the column headers. I am not sure how to label the columns with an merged signal. When I leave the data merged, I have three types of data bundled (strain, string pot voltage, and an array of doubles (the current load case)). Each of these types of data has a channel count that may differ with each run.
When I unmerge the data, the code only seems to be able to write one channel, even though I have setup array inputs for each desired "set of channels"
I'd like to set up the code so it somehow recognizes the channel count for each of the three data types, and appropriately names the columns accordingly.
What I have:
- DDT data flow containing 3 types of data, each type containing N channels
What I'd like:
- One TDMS file containing one group, with a time column (column A), N strain columns (columns B..C...D...etc), N string pot voltage columns (columns E...F...G...etc), and a single column for the load case (column H)
The two different approach styles are shown below with code, and with the file that results:
Any thoughts? Thanks, per usual..!
CODE "A": Write time into TDMS, and then add in all data via one, merged DDT signal
File "A": Cells B1 through E1 are the output of the "(strain) gage location" array, while truly, column B and C are strain. Column D is string pot data, and column E is load case data
CODE "B": Split DDT before writing each to a new "Write to TDMS" function. I want to keep all data in one group, but have each channel be labeled correctly, regardless of the # of channels per data type
FILE "B": By splitting the data, I somehow get a mix of column labeling (you can see the strain gage #1 lablel in cell B1, and the Load Case label in C1), and just the strain gage data, which fills column B and C.
