Does "Write to Binary File" automatically buffer?

[thutch79]

Simple question, but I can't find a simple answer. Does the "write to binary file" function buffer data and write periodically? Or does it write data to disk every time it's called? Should I create a nice buffer to collect a certain amount before committing it to disk? Or would that just be redundant? I know TDMS write does buffer (or at least have the option to), but it's unclear if "write to binary file" does.

For the record, I plan to collect 4000 data points of I16 every 100ms. Not breakneck speed, but I want this to be scalable without too much pain.

Thanks!

[Ben]

I would venture that the answer is OS dependent.

I recall years ago that I had to do a "sync" twice on a Unix system before shutting down the OS. failure to do that would lead to disk corruption occasionally.

The predecessor to WindowsNT (VMS) also handle the scheduling of the writes to disk.

Since disks have to be written a full sector at a time, witting smaller chunks is a waste of time. It used to be optimal to write a full sector worth of data at once rather than smaller parts. What your disk sectors are on modern machines??? Don't ask me! I have not been down in that world since sectors were 512 bytes.

So as I began I will end by speculating it is OS dependent.

Ben

[thutch79]

Well, that helps me understand why I haven't found a straightforward answer. This application would be run on Windows 7 or 10. I would love it if the optimal approach is to just call write to binary file every time data was ready and let the OS handle things. I'd much rather that than maintain a buffer for writing. However, I will be happy to create a buffer of whatever size makes send (I've seen 65000 bytes thrown around out there) if that's the right way to do it.

Thanks!

[Ben]

If you REALLY wanted to figure out what the optimal write size is you could watch...

Windows task manager >>> Resource Monitor >>> Disk

While running some test code that gradually writes larger and larger number of bytes to disk. I would (again speculating) suspect you will see the disk I/O rate flatten off when you get to the size of the disk sectors.

Just venturing initial guess!

Discalimer:

Spoiler

I used to be a large disk special in the early days of computers when RAID were called clusters and a large disk was anything over 54Mbytes. my knowledge may be obsolete now.

Ben

[crossrulz]

Just to add to the confusion, there is an option for the Open/Create/Replace File function:

"disable buffering specifies if the file opens without buffering. The default is FALSE. If you want to read or write a data file to a Redundant Array of Independent Disks (RAID), consider opening the file without buffering to speed up data transfers. To disable buffering, wire a TRUE value to the disable buffering input.
Note  If you have a small amount of data to transfer, you might not notice a difference if you disable buffering."

[billko]

Just to make it even more confusing, even the choice of hardware - such as a controller card or hard drive - can affect the answer.