cRIO Hard Drive Fragmentation

[maxwellb]

I'm working on a data logging application for cRIO and I started wondering about hard drive fragmentation. This is something that we've had to worry about in the past on Windows PCs. Our Windows file logging routines preallocate files on disk for the express purpose of avoiding fragmentation when writing to them. Does one ever have to worry about defragmenting a cRIO drive?

[JKMM]

My understanding is that any datalogging done from a real-time cRIO should be to an external drive by exporting the data over TCP or shared variables to a PC hard drive, or to an SD card in a 9802: http://sine.ni.com/nips/cds/view/p/lang/en/nid/204328  Some of the newer cRIO controllers have USB ports which support external data storage as well so you don't have to go through a PC.

The cRIO hard drive would ideally only be used to store the real-time exe, and the drivers and support files required to run your exe.

If your cRIO hard drive ever gets corrupted, just format it, install the drivers again (usually takes less than a minute or two), and deploy your exe (which can take a while longer if you are compiling FPGA code).

One exception would be the Windows cRIO, which currently comes with 16 or 32 GB of onboard memory.  I imagine that those do have a Windows defrag utility.

[rolfk]

Since the "harddrive" in the cRIO is really flash memory I would not do continous data logging to it. That certainly will have an impact on the lifetime of your cRIO "harddrive". It's a little more expensive to replace a cRIO controller after a few years of operation, than a standard PC and for real world installation it is more likely that your cRIO application will still be expected to work 10 years from now, than with a standard PC.

[maxwellb]

That 9802 module looks great. Unfortunately, I don't think it's an option for me to reduce channel count at all in any of these chassis. Looks like USB would be the way to go for me. 

In that department, I'd like to have something that is at least as rugged and reliable as the cRIO itself. Do you have any recommendations for a product that would fit that?

[nathand]

As Rolf said, there's no need to defragment a USB flash drive, and in fact doing so could shorten the life of the drive.

On a standard hard drive, the data is arrange in concentric rings. There is a head flying over the disc platter that reads data. Defragmenting puts the entire file into neighboring locations on the disc. This speeds up access to the file because minimal motion of the read head is required to access the entire file, saving the time required to seek to a new location.

Conversely, a flash drive is RAM - Random Access Memory. It is equally efficient to access any location in memory, thus the random access. There's no need to read in sequence, so no need to defragment. It's just as fast to read a file spread across the disk as it is to read one that's in consecutive locations.

Also importantly, each sector of a flash drive has a limited number of write cycles (from about 100,000 to 1 million for newer drives). Most flash drive controllers, including the one in the cRIO, include a wear leveling algorithm that spreads files across the disk to avoid wearing out some sectors faster than others. The controller deliberately fragments files, so defragmenting could increase the rate at which some sectors fail.

I don't see any reason not to use the internal storage of the cRIO, although as with anything else it's a good idea to have a backup if the data is critical. You need to take some care in how you write to the file - it's better to write large amounts at long intervals than small amounts rapidly to avoid excessive writes to the disk. The internal flash is designed to be as reliable as the rest of the unit, and unlike an external USB device it can't get accidentally bumped and damaged or disconnected.

[maxwellb]

Thanks for the helpful responses everyone! I've not worked with flash memory much as a data storage location, and I've learned some useful things about it here.