cRIO Close USB Flash Drive

[Matthew_Kelton]

I was able to reproduce the problem  with a fresh system with no user code on it.  In other words, no direct read/write to the usb drive.  Just insert it, wait, and pull it.

All we need is NI to give us a proper utility to shut the drive down.  I have looked at some third party stuff, but with NI stripping so much out, I didn't have enough time to go down that rabbit hole just to see if the utility would work.

[rolfk]

Did you try:

sudo umount /media/usb

with the /media/usb path of course adapted to whatever your USB stick got mounted to.

And I wonder what you mean with your remark that NI is stripping out all kinds of stuff. Are you comparing it to Ubuntu? If so they are not stripping out stuff, but Ubuntu is adding tons of packages to the core Linux distribution for various reasons, including a very extensive GUI experience (which makes no sense on a headless system).

[Matthew_Kelton]

Rolf,

When I was testing it, umount would make the drive letter inaccessible, but the USB drive still did not completely get disconnected from the system, so you would still get the error message in Windows that it was not shutdown properly.

While I was looking at this back then, I tried nearly everything I could find on Google.  Some were for Ubuntu, but most did not specify and were command lines.  Looking at this page, for example, danorton's response seems to explain exactly the behavior we are seeing and why unmounting the drive is not enough.  And if I recall correctly, the NI distribution doesn't have the eject command.  I also believe I tried Totor's commands and they were not supported.

What I don't remember with udisks - the last thing I was trying was trying to get an add-on to work, and I kept getting countless errors trying to get all the necessary support packages to install successfully.  I don't remember if this was udisks or something else.

[rolfk]

A well I see. I'll try to look into the umount issue later this week. As to why tools like udisk won't just be installable on an NI Linux RT system, that has not much to do with NI taking out things from Linux (aside that they did go with a Linux distribution that was explicitly setup for embedded applications in the first place and hence is far from a desktop distibution like Ubuntu/Debian, RedHat or Suse).

The real problem lies in the fact that different Linux systems use different kernel versions and come with different GCC versions and according libc version support. And you can't easily change just one of them at will as they are pretty tightly coupled together. In addition you can not simply download binaries from some Linux distribution and hope to install them seemlessly on another one, as in the case of the cRIO 906x series you also have an entirely different CPU (ARM) and in case of the other Linux RT systems it is a x86/64 system but there are different ways to support 64 bit x86 libraries in different Linux distributions.

Your best bet for such tools is to check the according package repository (opkg) for the NI Linux RT systems if the tool is present there and if it isn't you'll have to get the sources and compile it directly on your system. This will make sure that it will at least be compiled with the same or a compatible version of GCC as your system itself and libc will be compatible too. There still of course is the problem that your tool may try to use system calls that the kernel version used in the target system does not support or supports differently. Many Linux developers are not known for their stringent strive for backwards compatibility, to the dismal of even Linus.  

[Hooovahh]

So I did actually find a solution related to NTFS drives.  In a separate (but semi-related) thread I asked about NTFS support and what I should format my USB drive with.  NI's recommendation is FAT32 if the drive is to be used on Windows as well, and EXT3 if the drive is intended to be a part of the system and not plugged into Windows.  They also recommend a tool for reading EXT3 on Windows.

Anyway so I was asking about NTFS support, since if you plug in an NTFS drive it just works, until you have a file larger than 4GB.  The file is larger than 4GB but it becomes partially corrupt somehow.  This makes me think that NI for some reason is treating this drive as FAT32.  There was a suggestion in that thread to install an opkg package for adding full NTFS support.  It did and now an NTFS drive works and I've seen no issue with >4GB files just yet.  I noticed this had an odd side effect of no longer saying the drive is corrupt when moving it to Windows.

So if you install the ntfs-3g package, and use an NTFS drive, the corruption dialog isn't seen.  Not sure what the other consequences of this are yet if any.

[rolfk]

I don't think that you can get away with treating an NTFS volume as a FAT32. Those two filesystems are not even intended to be compatible in basic format! So for a FAT32 filesystem driver an NTFS volume looks as unintelligible as an EXT3 or similar.

What is most likely the problem is that the NI Linux RT kernel contains an older NTFS driver. Officially the NTFS driver in the Linux kernel used to be in Beta state for more than 10 years and writing from Linux to NTFS was highly discouraged. It seems that at some stage Microsoft itself stepped in to patch the NTFS driver in the Linux kernel to make it safe to use. You still need to watch out about Windows fast start however. If that is enabled all NTFS partitions mounted in Windows are marked as fast start enabled and then the Linux driver will prohibit write access to such a partition to avoid corrupting the possible hibernation file.

ntfs-3g is not really an NTFS driver but it adds tools to the system to manage, mount and unmount, and repair NTFS volumes. So it could be that without this package the mounting of NTFS volumes is not done properly for large file support, which is in itself a somewhat complicated feature in the Linux libC runtime.

[euio76]

I would bet that the corruption issues are from not properly shutting down the FAT32 USB drive (often called "ejecting").  FAT32 drives will often have corruption problems if not ejected properly. I believe the 'udisks' command was replaced by the 'udisksctl' command.  You might be able to get away with just running a 'sync' command before pulling the drive out, but I recommend trying the following command (from an terminal as admin) to properly close the USB flash drive:

udisksctl unmount -b /dev/sdc1 && udisksctl power-off -b /dev/sdc

(Be sure to change sdc to your device name.  Use 'lsblk' to check if you need.)

As Hooovahh pointed out, NTFS would be a better option, but requires you to install the ntfs-3g package.

[rolfk]

I did a check on the cRIO I have here for testing purposes. And the NI Linux RT does have udisks2 installed, the udisks deamon that handles disk drives of just about any sort. The cli interface to it is the udisksctl utility and the two commands in that order as pointed out by Jonathan Reynolds are exactly what is needed to get a USB flash drive properly "ejected".

As to NTFS support, there is a ntfs kernel module installed in the cRIO. The NTFS kernel support has been long ago depreciated. The driver only supports read access and is incomplete. But NI Linux RT being based on the 4.6 kernel version seems to still contain it. As such it seems to get loaded when the udisks deamon detects an ntfs partition being plugged in, unless you have ntfs-3g installed which is a fuse user space filesystem driver and its according utilities. ntfs-3g supports read/write access and contains Microsoft patches to make it work safely with NTFS partitions.

[Hooovahh]

Thanks for this information, but to be clear without the ntfs-3g installed I could read and write NTFS drives just fine, as long as the file I was writing was < 4GB.  Without that installed a file greater than that is messed up somehow.  With it installed the file is created successfully.  I intend on downloading the packages, and dependencies to be installed offline, so that I can go around and update all my controllers with no internet access to add this support, in case one is plugged in and logged to.

[rolfk]

Well the kernel driver did have experimental write support but that should be disabled by default. It was experimental as it would only work if you wrote to a file that already existed. If you tried to create a new file, things went very quickly very south of Antarctica. It could also be that it only worked for files up to 4GB. ntfs-3g being available since about 2006, this driver wasn't really worked on for a long time now as the immediate need was not urgent anymore. And the required knowledge is very specialistic, kernel drivers and especially kernel filesystem drivers are pretty hard to debug and 99/99% correct functionality is generally not good enough as a filesystem driver is expected to never ever corrupt any data itself. If a network driver misbehaves that is pretty unfortunate and inconvinient, but a misbehaving filesystem driver is pretty much a disaster.