ni.com is currently undergoing scheduled maintenance.
Some services may be unavailable at this time. Please contact us for help or try again later.
LabVIEW
Editing large text (*.CSV) files
I inadvertently created a 3.5Gb file with data using the Write Characters to File.vi and giving it a *.csv extension. I have seen some info on the forums dealing with files this large but noticed that they point to Open G for the most part. I was wondering if someone knew how to quickly divide this file up possibly using the DOS "TYPE" command with echo turned off? This seems to open the file but echoing to the screen takes a really long time. I tried downloading other text editors with not much luck, EditPad Pro 6 (2GB), and UltraEdit which I forced closed after 10mins.
The wild thing is LabVIEW let me create this large file but an't edit it
The data format is as follows:
1, 21, 2, 9, 16.865280
2, 21, 2, 10, 16.834560
.
2, 21, 2, 10, 16.834560
.
.
.
Regards,
-SS
Hi ShotSimon,
I would make a small loop:
- read a part of the big file of (let's say) 50MB
- write to a uniquely named file using iteration count and format string like "part_%05d.txt"
- continue until end of file reached...
using "read lines" option to read in approx 20MB (=1 million lines) of the big file.
I wonder about the file size, I thought LabView will only create up to 2GB files using standard file functions... This could be a problem when reading that file, too.
Message Edited by GerdW on 08-11-2008 09:26 PM
I would make a small loop:
- read a part of the big file of (let's say) 50MB
- write to a uniquely named file using iteration count and format string like "part_%05d.txt"
- continue until end of file reached...
using "read lines" option to read in approx 20MB (=1 million lines) of the big file.I wonder about the file size, I thought LabView will only create up to 2GB files using standard file functions... This could be a problem when reading that file, too.
Message Edited by GerdW on 08-11-2008 09:26 PM
GerdW,
That solution is working and processing my file correctly. It turns out that I was appending my array incorrectly.
Such as:
Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink
1, 10, 1, 0, 14.089472
Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink
1, 10, 1, 0, 14.089472
2, 10, 1, 1, 14.106368
Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink
1, 10, 1, 0, 14.089472
2, 10, 1, 1, 14.106368
3, 10, 1, 2, 14.078720
1, 10, 1, 0, 14.089472
Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink
1, 10, 1, 0, 14.089472
2, 10, 1, 1, 14.106368
Time (sec), Range(Miles), Down Link Data (%), Slot, Uplink, Downlink
1, 10, 1, 0, 14.089472
2, 10, 1, 1, 14.106368
3, 10, 1, 2, 14.078720
It was a 10 day test so I didn't want to lose my data.
Thanks,
-SS
