How to search a .csv file for data using its timestamp, then import to labview

[ptait]

Hi, I'm currently obtaining density, viscosity and temperature data from an instrument, adding a timestamp and writing it to a .csv file which I can view in Excel. This works fine (see attached code) but what I need to do now is to search that csv file for data which was obtained at a certain time, import the temperature, density & viscosity values at this time back into Labview to do some calculations with them, while the data acquisition process is still ongoing.

I've found various examples on how to import an entire csv file into labview, but none on how to extract data at a specific time. Also, whenever I try to do anything with the .csv file while my data acquistion VI is running, I receive error messages (presumably because I'm trying to write to and import data from the .csv file simultaneously). Is there some way around this, maybe using case structures?

If you need to know my skill level, I've been using Labview for a few weeks and prior to that have basically no experience of writing code, so any help would be great. Thanks!

[jcarmody]

---

@p.tait@ed.ac.uk wrote:

[...] while the data acquisition process is still ongoing.

---

Take a look at using TDMS files; it's only slightly more difficult than what you're doing now.  You can install an Excel plugin that will read it as easily as a csv file, and you can access it as you describe - writing from one program while reading from another.

[ptait]

Cool thanks, I just downloaded the excel plugin so once I get the thing saving the data as a TDMS file rather than a CSV, that solves one of my problems. Though it seems to have created a new one in that the Write To Measurement File VI won't accept my array as an input.

Could I keep my original code and just save the file with the .tdms extension and a tab constant as delimiter, instead of .csv with a comma?

[jcarmody · Accepted solution]

TDMS is a little more involved than that.  Here's a quickly-thrown-together proof-of-concept:

You should look at a few of the TDMS read/write examples to get a better handle on the tool.  So should I...

[ptait]

I've got the TDMS streaming working now, I initially tried using this code which gave me all the data in a single Temperature column for some reason...

But managed to fix it by using a separate TDMS Write operation for each data type.

I'm sure there must be a smarter/less messy way of achieving this, but it's good enough for the moment. So now I just need to figure out how to import the data values for a specific time...

Thanks!

[ogk.nz]

Im not entirely sure that you needed to bother with all of the working through of the TDMS rather then CSV when you could have just opened the CSV line by line and checked the time was what you wanted.

Your first TDMS was very close, but tdms is used to receiving arrays of data. If you converted your data that you were send in into a 2D array then it should have worked as desired with all the data filling different columns. 

With regard to the searching for a time step are you going to be searching for a very specific time or just seeing if there is any data within a specific window? Because given that you time sampling is non-contiguous then it will be a bit strange searching for specific time.

[JarleEkanger]

In your first try, you just need to use the interleaved data layout option:

[ptait]

So for the time searching thing I need to create a separate VI, I think. Density and viscosity are highly sensitive to temperature and liquid composition, and we need to compare current values against older ones in order to determine the scaling is still correct. What I'd ideally like this VI to do is:

1. Seach for values of Temp, Density & Viscosity from earlier in the current run. Say I start recording data at 9am and the current time is 10am. I may need to find out what the data values are at approx 8:30, so I'd search for data which has a timestamp containing the characters e.g. "08:30:12"

2. Import the values of T, D and V at this specific time into labview

3. Plug those values of T,D and V into a mathematical function, which compares them values against those which are currently being output from the device (the code for this part is already writtten).

Sorry, but what does non contiguous mean in this context?

I'm currently away for the weekend with  no access to Labview, but will come back to this on Monday. Cheers.

[ogk.nz]

It sounds as if you are going about this a little backwards writing to a data file and then extracting from the file but its the weekend so I can't think of an improved way to do it at the moment. 

Searching for a specific time with those specific values is quite easy, or if you wanted to select any time then you could interpolate the values to find any value that you want (This is where the contiguous measurement comes in, as you have readings at discrete times you will have to interpolate the values if you want to get the 'measured value' at a time point that is not exactly one of your measured points).

If you can extract the TDMS time column and the T, D & V then simply thresholding and interpolating each of your array/data sets should allow readings at your desired times.

[nafisnuzrin]

---

@ptait wrote:

I've got the TDMS streaming working now, I initially tried using this code which gave me all the data in a single Temperature column for some reason...

But managed to fix it by using a separate TDMS Write operation for each data type.

I'm sure there must be a smarter/less messy way of achieving this, but it's good enough for the moment. So now I just need to figure out how to import the data values for a specific time...

Thanks!

---

How to make this code, able to save newfile everyday with different name