LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Datalogging with options to retrieve subset of log file based on date/time

Solved!
Go to solution

I would like to thank this forum for useful advice so far in completing my LabVIEW software.

I have a data logging challenge. I am supposed to log about 30 parameters every 5 seconds. Some of these parameters are digital (ON/OFF), some are values of speed (rpm) and others, an expression of a percentage (%). It should be possible in future to do a histogram or bar chart plot of some of the parameters, for a specific period range (say the last 5 minutes of a certain day). So in effect, do an extraction of a segment of the total log file.

My challenge is if I use text file, like the one in the attached VI, can it give functionality of retrieving data (while the VI is running) from the log file, based on a certain time range (i.e. retrieve a section of the log file based on a certain date/time range, on demand)?

The format in the text file is close to what I require, since it lists the time n one column and the other parameters on other columns to enable future histogram generation.

Thanks a lot, friends.

Download All
0 Kudos
Message 1 of 10
(3,126 Views)

Convert your time string to a numeric and you can find rows that are between two values. You should really be saving the date as well. Converting a date/time string to a timestamp can be done with the Scan From String function.

 

You also have a major flaw with the saving of the numerics. You are saving them as integer. Don't use Number to Decimal String. Use Number to Fractional String.

Message 2 of 10
(3,118 Views)

Thanks a lot,

 

I have noted the decimal string error.I thought the date is being saved as well. Are you saying I should convert to timestamp before conctenation? How can I do a search for rows between two values?

 

Thanks a lot for your prompt response.

 

0 Kudos
Message 3 of 10
(3,110 Views)

Hi,

 

I have been able to do the conversion to numeric. I still have this challenge:

 

 

Take this simple scenario for example, this simple VI logs the date/time as well as six other parameters every 2 seconds and lets say there are 10 readings in total (10 rows). While the program is running, how can I retrieve data for the first two rows alone, based on the dates?

 

 I ask this because in real life, the VI is going to be running for months for instance and its logging to a file. It should be possible to retrieve the data for 20th July at 5:00pm - 5:10pm alone and log it to a different log file for analysis.

 

This is my challenge. I hope I have conveyed my challenge properly.

 

Thanks for your help.

Download All
0 Kudos
Message 4 of 10
(3,100 Views)
Where's your reader program where you convert the date/time string that you read back to a timestamp? Once you've done that and as I've already mentioned, you search the array for the first timestamp greater or equal to the start time you specify. You search the array for the last timestamp that is less than or equal to the finish time you specify. The elements give you the starting row and number of elements to extract from the array.
Message Edited by Dennis Knutson on 07-23-2009 08:09 AM
0 Kudos
Message 5 of 10
(3,069 Views)

So I have a similar problem in that I also want to extract a subset of my data.  I have two columns of data that I need to do with, these are potential energy (expressed in terms of kT) and relative separation.  The data typically results in a parabola-type shape with the potential energy on the y-axis and the separation on the x-axis.  The potential energy starts at high kT's at small separation and then decreases to zero as the separation increases and then it starts to increase again once it is past zero kT.  The issue I have is that I need to disregard any data that is greater than 6 kT.  Thus I need to extract the data (both the potential energy data and the corresponding separation data) from the array that is 6kT or below.  So the problem I am having is trying to work out how to approach the extraction of this data.  I would love to get some suggestion as to how I should approach it.

 

I initially thought of using a mathscript node to do this, but I am not sure of how I should code this, if this is indeed possible.

 

Another option I considered was to use one of the array vi's to perform this extraction as I thought I could use them to find the element where the kT is higher than 6 and the corresponding index element which I could then use to extract the relevant portions of the array.  But I don't see an easy way to achieve this.

 

So the vi I will write for this will function will actually be a subvi of another vi I have written (this vi is named ‘non-linear fit to PE data mod.vi' and it also has a couple of other subvis).  I have attached this vi and a relevant data file (as an example of the data I will be getting and needing to analyse) in a zip file.  I haven't included an example of the subvi I need to write because I looking to find some pointers as to how I could do this first.  In the ‘non-linear fit to PE data mod.vi' I have included a text box in the location I will be putting this subvi.

 

Any help in helping me out here would be appreciated.

 

 

0 Kudos
Message 6 of 10
(3,037 Views)

Hey maxidivine,

Iv been playing round with your code and found that to perform the search that you require could be quite demanding to system resources when scaled to the size of your application I shall try and find a way to perform the search using .txt files but the there are some other options available. I recommend the use if TDMS files as the file format is a very efficient, manageable method of data-logging. The TDMS file format is designed to write and read measured data at a very high speed, while maintaining a hierarchical system of descriptive information.
 
Traditionally, TDMS was a National Instruments only file format – you could only read it using our products – LabVIEW/CVI/DIAdem. However, thanks to the popularity of the format, a bolt-on is now available for Excel, which allows you to directly open the .tdms files with Excel (see link).

National Instruments Technical Data Management Overview
http://zone.ni.com/devzone/cda/tut/p/id/3676
 
Introduction to LabVIEW TDM Streaming Vis
http://zone.ni.com/devzone/cda/tut/p/id/3539
 
VI-Based API for Writing TDMS Files
http://zone.ni.com/devzone/cda/tut/p/id/6471
 
TDM Excel Add-In Tool for Microsoft Excel User Guide
http://zone.ni.com/devzone/cda/tut/p/id/4906

TDM Excel Add-In for Microsoft Excel Download
http://zone.ni.com/devzone/cda/epd/p/id/2944

Troubleshooting the TDM Excel Add-In for Microsoft Excel 2000-2003
http://zone.ni.com/devzone/cda/tut/p/id/5874

Examples of the use of the TDMS API ship with LabVIEW. You will find them in HELP > find examples > fundamentals > File Input and Output. For you application, I would recommend the “Cont Acq&Graph Voltage - Write Data to File (TDMS).vi”.

Furthermore, if you require some help with DIAdem, I would recommend clicking "getting started" from the DIAdem splash screen. This opens a manual which discusses everything from data analysis to report generation. Also, if you have DIAdem 11 or above, there are tutorial videos which install with DIAdem. These are useful little tutorials, which discuss all the DIAdem fundamentals. You can access these by selecting a particular palette tab (eg. report, view, analysis...etc) and then clicking the tutorial button (shown as a film strip with a question mark) at the top of the group view.

Here are some more helpful DIAdem related resources for future reference.
 
Report Gen in DIAdem...
http://zone.ni.com/devzone/cda/tut/p/id/7379

DataPlugins: Supported Data Formats (ni.com/dataplugins)
http://zone.ni.com/devzone/cda/tut/p/id/4065

Hope this is helpful

 

Philip


Philip
Applications Engineer
National Instruments
UK Branch

===If this fixes your problem, mark as solution!===
0 Kudos
Message 7 of 10
(2,982 Views)
Just in case someone wants to have a go at my problem, you can find the solution here.
0 Kudos
Message 8 of 10
(2,970 Views)
Solution
Accepted by topic author maxidivine

Hey Maxidivine

I have made a parallel loop in your program that will ask for a file to be searched and ask for a start and end datestamp.
These datestamps must be present in the text file prior to clicking the Get Data From File button.

The data stamps are accurate to 1 second but as I understand you are logging every 5 seconds and the program will
auto increase the search length for finding the next available date stamp.

If you set the acquisition time to 8 seconds the program will search for the next available timestamp in the text file 16 seconds after the one's entered this is required as there is not a date stamp in the file for every second.

 

Write Read.JPG

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

let me know if this helps.

 

Philip


Philip
Applications Engineer
National Instruments
UK Branch

===If this fixes your problem, mark as solution!===
0 Kudos
Message 9 of 10
(2,949 Views)

Thank you so much for taking time to look at the issue. You made it look easy. With help from the forum, a queue was actually implemented for the task because the specifications have changed since then. Basically, data is logged to two different files. A main file and a snapshot file. When triggered there is a save from the main file to the snapshot file. The snapshot file has 10 seconds worth of data before the trigger and 10 seconds worth of date after the trigger.

 

Thanks a lot once again. The software is would be very useful.

 

0 Kudos
Message 10 of 10
(2,935 Views)