dealing with large spreadsheet file

[faustina]

I need to access a large (over 200 MB) spreadsheet file (actually .csv file) quickly.  I need to select a portion of the table by rows and then display selected columns in a graph.  What is the best way to quickly do this?  Would it be best to covert the .csv file to a .tdms or another file format for faster or easier access?

[RavensFan]

Read from Spreadsheet File.

How quickly do you want to read this file?

[faustina]

I am currently reading through the file using Read From Spreadsheet File VI and it is taking 50 msec to get start and stop time.  I know which column contains the time data is in, so I'm just calling the VI 10000 rows at a time until it reaches EOF.  I then pull out the stop time in the last row.  In 50 msec I've gone through 130168 rows.  I would like to do this 10X faster.

[RavensFan]

So you want to read a 200 MB file in 5 msec?

I don't think that is realistic.  That is 40 GB/sec.  Your hard drive isn't that fast.

[smercurio_fc]

You initially said that you're trying to read only a portion of the data in the file based on a portion of rows. Is the data in the file formatted in such a way that you could directly go to the file position where you want to start reading (based on byte count) and then only read the amount of data that you need? For instance, are the rows a complete block as opposed to needing to fetch the 10th, 14th, 101st, etc..? That would cut down the time, but as Ravens Fan pointed out, you will still have real-world limitations on hard-drive throughput.