What is the maxium size of data file that read spread sheet.vi can read?

[IgG_Kylin]

let me come back to my original question, I probably need to clarify it

I have a 80MB text file and need to read into a 2D array.

but read spread sheet.vi doesn't work, as it is out of memory.

I tried low-level vis, but seems very complicated. do you know any good way to read it?

thanks

jack

[altenbach]

Read it as a plain string, then use "spreadsheet string to array".

[IgG_Kylin]

Hi, altenbach,

Thank you very much. I tried that. It's very good. But right now I have another problem,how can I read chunk of files or part of lines sequencely?

for example, my text file have unknown number of lines, do u know how to read them in a fixed number of times? how can we get this fixed number?

thanks

jack

[Online_Courses]

Hi lgG_Kylin,

Try taking a look at the Advanced File Functions palette.  The Get File Size VI and Get File Position VI should help you. 

Regards,

Starla T. 

[altenbach]

To read the file in sections is hard because the number of characters Per line is most likely variable., thus the file position of a given line Is impossible to calculate from first principles. You need to sniff through every single character to find all the newline Characters and see where the lines end. If the exact number of lines per section is npt important, yuo ca go to an approximate position and find the next linebreak.

[rex1030]

When trying to read in sections the read text file will let you specify how many rows you want to read (right click on the VI and hit read rows). You use the read text file in a for loop, you wire the file reference wire to a shift register so the file position is fed to the next iteration of the loop. You are now reading in chunks. Now do everything else Altenbach has said with the get file size vi (to figure out how many iterations of for loop operation you need). You should be able to handle your data processing in the for loop. You haven't told us what you are doing with the data so any number of things could be what you are looking for.

Once again, its much easier to get help if you give people a lot of information about your problem.

[IgG_Kylin]

great, this is useful.

thank you very much 🙂

[altenbach]

Reading in sections only makes sense if you also process in sections and never have the entire file in memory. Just reading in sections and then assemble the whole in memory would again be inefficient.

As I explained, unless you use binary files or text files with a fixed number of characters/line a lot of work needs to be done to find where each line ends. (even if you use the built-in tools!)

Reading an 80MB string is not that hard, and the resulting DBL array is only a small fraction of that size in memory.