Down-sampling (decimate) huge csv data

[cRIO+FPGA Apps]

Hi all,

I'm trying to down-sample (decimate) the attached csv data, hopefuly I can build a decimator to take values from the csv file (input) every certain number of rows and transfer them into a new excel file (output). Any suggestions?

Thanks.

[Coq_Rouge]

Labview do have such functions as from Labview 8.2

Decimate (single shot) VI or Decimate (continuous) VI
Owning Palette: Signal Operation VIs
Installed With: Full Development System

I would recommend to use the avaraging option as this can smooth out the signal

[cRIO+FPGA Apps]

Is it possible to decimate more than one column from the excel file? Could someone post an example on how to call data from different columns in an excel file?

Thanks.

[Pcorcs · Accepted solution]

If you have your data open in Excel, you may choose to save as 'Tab Delimited Text.' This will allow you to directly interact with the Read From Spreadsheet VI pictured here:

Reading in all rows of data will allow you to manipulate the data to extract individual columns using the array functions in LabVIEW. i would reccomend consulting the LabVIEW Help for instruction on File I/O, as well as Array manipulation. 

Cheers,

[cRIO+FPGA Apps]

Could you please provide with an example or something related on how to call a column from an excel data sheet to be use with the "decimator" vi?

Thank you!

[Pcorcs]

You may try the http://www.ni.com search in the upper right corner, for the term "Parse CSV." You should find something related.

Cheers!

[cRIO+FPGA Apps]

Thanks, I have something going on (see attached) ... it works perfectly, however, there are some features I would like to add,

1. Having predefined column titles for the ouput file (excel).

2. Automatically opening the output file at the end.

Any pointers?

Thanks again.

[Pcorcs]

Looks like a great start! I believe adding a second Write to spreadsheet will allow you to define your Headings. See Blow:

With regard to opening the file after execution, if you were hoping to specifically invoke Microsoft Excel for data viewing, you would now be venturing into Active X territory. You may want to examine the functions of the LabVIEW Report Generation Toolkit. An example of invoking the launch of an Excel report is given here:

Open or Create Excel Report

http://decibel.ni.com/content/docs/DOC-3546

Best,

[cRIO+FPGA Apps]

Thanks a lot PCorcs!

How could I clean the output file? i.e. the output file shows empty cells (with "0") from column K until column S (I'm only using columns A to J), how could I get rid off those?

Thanks!

[Pcorcs]

I believe it has to do with the initialization of your array, as full data set. If you create an indicator on the decimated array, can you verify that the points removed have been replaced with 0, yet the array size remains the same?

You may need to use an 'Array Subset' function on the array palette to trim the 0s off of the decimated data set, before writing to the spreadsheet.

Hope this helps,

Message Edited by Pcorcs on 03-11-2010 09:33 AM