LabVIEW
Merging data into a single row
Solved!
Hi Jousepa,
@Jousepa wrote:
I would like to combine them into a single row, that is, to have one row with (date, time, pH, temp, CD, temp). I filter which values go together using the sample ID. How can I do this?
I would do it like this:
- Filter the CSV data line by line and remove unneeded lines (header, empty lines).
- Parse each line into a cluster of [ID, timestamp, pH, Temp, CD, temp).
- Now you have an array of clusters with ID as first element: sort this array!
- Now entries with same IDs form consecutive array elements...
- Filter this array by combining neighboring elements into a single cluster (of the same datatype).
- Finished: an array of clusters, containing the data for each ID...
Recommendation:
Don't put ALL that stuff into one VI. Separate the different steps into subVIs to keep the code clean and readable.
This seems like a trivially simple problem once we know more details.
- A CSV file is not an EXCEL file.
- Does the instrument append to an ever-growing file? Are you sure you can access the file while the instrument has it open?
- The filename seems to be encoded with metadata. How often does the filename change?
- Does the instrument always write two rows at once?
- What's wrong with line #3? Seems corrupted.
- Do you just want to show the data in an ever-growing array (6 columns) on the front panel? Why not use a table indicator?
- You should not have to re-read the entire file every time. Use lowlevel file IO and keep track of the last file position of the previous read, then read the new data from there. Make sure to open the file in read-only mode.
- How often does the instrument append new data?
- Are the sample IDs always incrementing by one or can they be in any order? What should happen if a sample ID only has one row?
Assuming there is a dedicated folder for these output files, It seems it would be easier to let the instrument decide on the filename, then you can use "list folder" to find and parse just the newest file(s). Less work for you! You could even have LabVIEW delete the CSV file once it is processed (or e.g. move it to a backup folder for record keeping if needed). This way only files that are not yet processed will be there.
@Jousepa wrote:
It would be a great help if you could give me some brief guidelines on how to do it. I tried today and it was catastrophic.
This is a long thread with many different suggestions. Without quoting relevant parts of the previous post (as I just did here), it is impossible to tell which post you are replying to and what suggestions you were trying to implement.
