02-06-2012 03:12 PM
Hey all,
I'm curious as to the most efficient method of writing data to a file.
A) Open the file, point to the end and append the file
B) Add to an initialized array each loop, write to file at the end.
What do you all think?
Thanks!
Patrick
02-06-2012 03:17 PM
Writing everything at once will be the most efficient, but writing inside a loop is the safest - your computer could crash between the time the data is collected and when it gets written. You need to balance the two constraints.
Mike...
02-06-2012 03:21 PM
Good point, thank you!
02-06-2012 03:29 PM
I think if you're really concerned about efficiency, you should write to a binary file.
I'm not sure you've covered all the possibilities here. One reason it's efficient to write a large block at the same time is that you force the operating system to look for a location on the disk where it can put all the data. If you write one small chunk at a time, the operating system needs to find space for that data on each write, which can be inefficient and leads to fragmented files. However, if you know roughly how large your file will be (making it slightly too large is fine, you can shrink it later), then you can preallocate that space by setting the file size before you start writing. In this case, writing in the loop may be more efficient, especially if you would be building the array inefficiently. It also allows you to write one chunk while you process the next (the operating system will normally handle this for you by queueing the disk operations; the actual writes to the disk are asynchronous to your code).