cRIO-9068 stream to USB mass storage device

[Deborah_B]

Until this Stream to Disk Benchmarks for cRIO controllers whitepaper is updated for the cRIO-9068, any idea on the change in throughput we can expect compared to VxWorks and Pharlap cRIO targets, understanding there are variables such as frequency of writing data, block sizes, etc.?

FAT16 was a requirement on the VXWorks targets, limiting the size and selection of USB mass storage devices that could be connected. Will NI Linux Real-Time remedy this?

Thanks!

[Deborah_B]

Nevermind, I found the answer here, LabVIEW Real-Time and USB FAQ

Does my external USB drive have to be formatted for any particular file system?

For PXI, external USB drives can be FAT32 or Reliance.

On VxWorks, the only supported structure is FAT32.

For LinuxRT, external USB drives can be FAT32 (for raw NAND devices UBIFS is accepted).

Thanks!

[DAQjr]

Deborah_Y wrote:

For LinuxRT, external USB drives can be FAT32 (for raw NAND devices UBIFS is accepted).

I don't believe that is entirely accurate.  My understanding is that ext2, ext3, and fat32 (vfat) are possible on external drives.

The comment about UBIFS and raw NAND is irrelevant since raw NAND will never be external or a USB drive.  The internal drive is raw NAND and uses UBIFS.

[Zach J · Accepted solution]

You can also use ext2 and ext3.  I made a KnowledgeBase article about this that you can locate here: http://digital.ni.com/public.nsf/allkb/4819E4118F382D4586257C1A00592CF3.  Additionally, there are many virtual filesystems that you can use for mounting non-block memory devices as particular types.  If you SSH into a LinuxRT device, you can issue the command:  cat /proc/filesystems to show what filesystem types are supported on the device.  All filesystems listed that do not state "nodev" are supported for block memory devices such as external drives.  The others that do state "nodev" can be used for shared folders or other types of storage media. 

[rtollert]

In general, most "cheap" flash devices (USB/SD) are optimized for FAT32. This might sound a little crazy on its face, but there is a surprisingly large body of supporting evidence:

• https://wiki.linaro.org/WorkingGroups/KernelArchived/Projects/FlashCardSurvey
  
• https://lwn.net/Articles/428584/

Obviously the underlying technology here changes constantly, and the situation may vary greatly from device to device. But at a minimum, ext2/ext3/ext4 is probably no faster than FAT32 for streaming writes performance, and possibly much worse.

[QFang]

Really? -It's not possible to mount real-time safe file-systems on the new Linux targets?  Please say it isn't so!

Are there different options for SD mounted storage on the LinuxRT targets (vs USB mounted)??

--Though with the dual core and higher frequency CPU, we could hack out a solution by using the local/internal  cRIO storage be a temporary scratch pad, complete with MD5 check-sum, then copy the file (after it is finalized) to the USB FAT32 stick/hdd/sdd, generate MD5 from the "external" location, compare the two MD5's, then if all is good, delete the local file... on our current platform (cRIO-9014) this isn't really feasible due to the (relatively insane) overhead of any and all disk operations, but with the 2GB onboard storage, it's not something we need to do either.

[Michael.Phillips]

Can you define what you mean by "real-time safe"? Is your concern about the determinism of file operations? Throughput? Fault tolerance?

If the concern is primarily about robust file operations, then I would recommend a journaled file system (like ext3). You can also force an fsync by calling the Flush File function in LabVIEW.

[QFang]

Hi Michael, thank you for responding.

We make use of fsync where it makes sense, but that just reduces the risks of file or file-system damage in the event of a power outage or similar not scheduled event.

I should have worded "real-time safe" differently.  What I meant is "unscheduled service interruption tolerant, even if such interruption occurs in the middle of a file (write) operation or in the middle of a file-system (table) update".

I am not familiar with e.g. ext3, but I am familiar (through exposure) to the Reliance file-system used in the cRIO's which was designed to safe, or tolerant, of such "unscheduled service interruptions".

I'll go look into ext3, but if you know or a report or paper that compares e.g. ext3 vs e.g. Reliance or Reliance NITRO or otherwise gives me an indication that it will perform equally well or better, then that would be great!

[Edit]

It seems ext3 and ext4 are superceded by Reliance NITRO, though the info is thin at best and the source of that info is Datalight, maker of Reliance NITRO, so salt accordingly:

http://www.datalight.com/solutions/operating-systems/linux/ext3-ext4-alternatives

[/edit]

Thanks,

QFang

[BradM]

BluesKnight wrote:

You can also use ext2

...

Do not use ext2 if you are looking for fault-tollerance. It does not have journaling and you can end up with an inconsistent filesystem if you interrupt the updating of underlying filesystem structures.

[QFang]

The potentially HUGE time to mount an ext3 volume after a e.g. power interruption is staggering (hundreds of seconds, all depending of course on the system and system state at the time of the power outage).. This (non-deterministic) time-delay on mounting is less than desireable.  We would prefer a transactional system (such as Reliance Nitro) over a journaling system (such as ext3).

http://www.datalight.com/tier2/13453345/Journaling-vs-Transactional-File-Systems.pdf