Reading in and shrinking a large csv file

[ChummJPL]

I am attempting to read in a large CSV file that contains a days worth of 1hz data (500 columns, 86400 rows, about 450MB). In my program, I am reading in multiple files and plotting certain columns for multiple weeks worth of data at a time. The data used to come in 1hr files (around 14MB), and I could manage it, but now it will be coming in 24hr files instead.

I only need about 100 columns of this data and I only need 1 out of every 300 rows (I would like to set this as a variable later). When I use the 'Read from Text File' function, I get a memory error: 'Not enough memory to complete this operation'. 

Is there a way to only read a certain line (row) from a text file? Right now, it seems like I have to read the entire text file before performing any additional operations on it. I have tried playing with the 'Read Lines' and 'Convert EOL' settings, I've tried using the 'Read from Spreadsheet File' vi, and I have read through many different postings on this site but I can't figure out a way to do this.

If there are any other ways I might be able to read in this large data file, I am interested in those also. Thanks for your help.

[GerdW]

Hi Chumm,

I only need about 100 columns of this data and I only need 1 out of every 300 rows (I would like to set this as a variable later).

Step 1: Use ReadFromTextFile to read just 300 lines/rows.

Step 2: Pick the 1 line/row you need. Filter your ~100 from this row.

Step 3: Back to step 1 until end of file encountered…