How can I read in a selected portion of a large ascii file (~200Mb)?

[DianeS]

Ah, are you quite sure?  I just created a small csv file, formatted it exactly the way you describe, and the following 1D string array came out:

header,header,

1,1,

2,2,

header,header,

...

etc.

Exactly like they're supposed to.

So let's back up.  Can you select a small portion of the data file you're trying to process and post it here?  You can use the attached VI to do so.

[doyles]

Nope, not quite sure, as it turns out.  The data is there after all.  I feel about as smart as my shoe right now.    I needed to extend the size of the string element in order to see the data.  I needed to extend it by more than a 1/2" for it to appear, but still.

When I thought about how the file read vi was working, I didn't understand why it wasn't being read in.  But I didn't think that it was actually there and I just wasn't seeing it.  (this seems to be a theme for this me at this point )

But I have it working now.   It's much faster (33s vs 59s) and the memory hit is usable.  I'm attaching the vi.  There may be a better way to convert the string array to the double array at the end.  I'd be interested in continuing to learn about that as well.  I've certainly been learning a lot on this task.

Thank you for the patience to walk me through.

[DianeS]

Yes, that's what I suspected...that's why I made my string indicator as large as I did.  I figured that if you ran the VI, there was a better-than-even chance you'd solve your problem by yourself.  And sure enough...

I have LV2010 open right now, I'll open 2011 later and look at your VI.

My pleasure to help, and you are much, much smarter than your shoe.    It's always a pleasure to help someone who genuinely wants to learn and is willing to put in some work to do so.

[MichaelSol]

In the depths of programming discussions one valid point related to the read text file deficiency of LabView was dropped.

LabView needs to have the capability to set the read marker at the beginning of a specified line number or, alternatively, allow the capability to select both number of the first line to read and number of the lines to be read. Now, you need to implement the complicated workaround to do that -as described in the above discussion.

Why one can make an internal LV code counting number of characters to skip, but not the number of CR/LF's? Clearly NI does not care to fix that.

[csparksfly]

Hi MichaelSol,

Thanks for that idea! Please feel free to add this idea to the LabVIEW Idea Exchange which is monitored by our R&D team in order to incorporate desired features into new releases of LabVIEW.

Christine S

Applications Engineer