How do I use a USB hard drive with ext3 starting from scratch

[labviewman]

All I am trying to to is take a 2.5" HDD or SDD (via USB-SATA adapter), format it to ext3, partition it, set permissions so my application can use it (cerate/write/delete folders/files), but I am having NO luck.  I have been at this for 8 hours.  I have an active support ticket with NI but so far no luck.

I started with a Linux guru (Red Hat) who format it, partitioned it, setup permissions. Worked fine until I tried to delete files. I started form scratch by using a utility on Windows to format/partition. Tried to write to the drive in LV...error 8 permission error.  I used the crummy web interface to the cRIO to create a folder...when it (the drive) shows-up, which is seldom, I get a forbidden error (I don't remember exactly and can't get it now since I can't get the cRIO to map a drive).  I used (via ssh) mkfs.ext3 /dev/sdb1 to format the drive.  fdsik to partition, chown -R lvuser /dev/sdc1, chown 777 /dev/sdc1...same result, if LV can see drive U (which is seldom), I get error 8.

I have searched the web off and on all day...I'm obviously not searching the right words because this must have been done maybe, I don't know, a million times before.

Can some other smart person give me a link on how to do this on a cRIO (Linux), or the steps to format a drive, partition, set permissions so my LV code can create/delete folders and files?

Thanks in advance!

Todd

[labviewman]

Attached is a snapshot of a few fdisk commands (while in fdisk)

[Sam_Sharp]

This might be obvious, but you didn't mention it in your original post - did you mount the drive / check the drive was mounted? Plugging in the USB stick will show the device/partitions (e.g. /dev/sdc1p1), but you might need to create a mount point for it to be accessible (e.g. as /media/u etc.). It does say they should be automatically mounted though. You write to /media/u (I think Linux RT uses names like /media/u for its automatic mounting) which is the drive in the filesystem rather than /dev/sdc1p1 which is the linux device handle.

I also found this note on some FAQs about external USB drives:

• For LinuxRT, external USB drives should be formatted as be FAT32. For raw NAND devices UBIFS is accepted.

That should seem to suggest that a USB HDD doesn't support ext3 although there is another article here and here that suggests that it does. Good one NI!

There is also more information about supported USB devices here.

[labviewman]

I was successful in doing this 6 months ago where I was logging data about 10MBytes/sec to an external drive formatted to ext3, which the cRIO automatically mounted...the problem is I don't remember how I did it, but I know I used a Windows program  Very frustrating to say the least...this is not rocket science.

I am able to run "Get Volume Info" for /dev/sdc1 which does return the drive space, so maybe the drive is mounted?  However, I can't write to the disk.  I did command "chmod 777 /dev/sdc1 -R" w/o any error, but I still can't write to it.

[labviewman · Accepted solution]

I fumbled around and found a procedure that works.  Here it is so no one else has to go thru the pain (and 12 hrs of work!):

1) Using Minitool-Partition on Windows

-delete the partition

-format drive as ext3

*these steps don't result in the drive as usable on the cRIO, but it completely deletes anything left-over left-over from the cRIO; formatting to NTFS didn't work with the following steps.

2) power-off the cRIO

3) power-on the cRIO

4) login with Putty

5) plug-in the USB drive

6) "fdisk -l" until the USB drive is seen (/dev/sdc1 for me)

7) delete partitions "fdisk /dev/sdc"

-delete all partitions (cmd d)

-write changes to disk (cmd w)

😎 format the drive

-"mke2fs -j /dev/sdc"

9) partition disk

-"fdisk /dev/sdc"

-create partition "cmd n", select "p" for primary

-write changes to disk (cmd w)

10) change permissions

-chmod 777 -R /dev/sdc

-chmod 777 -R /dev/sdc1

11) power off cRIO

12) unplug the USB drive

13 ) power-on the cRIO

14) once logged-in with Putty, plug-in the USB drive

15) "fdisk -l" until the drive shows-up (/dev/sdc1)

16) change permissions

-chmod 777 -R /u

This procedure probably could be optimized, but it works for me as-is and I need to move-on and focus on my code.

Creating a partition AFTER the drive is formatted makes no sense to me. Seems like it should be the opposite (create partition then format), but I am a Windows guy...maybe this is how Linux works.

LabVIEW can now (using vis on the File I/O palette) create/delete files/folders on the drive. The cRIO Webpage can also create/write/delete it's own files/directories, but it can't modify or delete anything LabVIEW created but I'm not sure why (must be different permissions in the web browser). Same issue with using WebDAV vis (LV on the PC) to delete files that LabVIEW created using via from the File I/O palette.

I ran thru this procedure 4x...worked perfectly every-time, with a mechanical drive and a solid state drive.

So, the only thing left, which isn't a big priority, is to figure out why I can't manage files/folders created in LabVIEW using Webdav (LabVIEW on the PC) or the webpage.

To be sure your USB to SATA adapter is working, in Windows create a small partition and format as FAT32.  Plug it into the cRIO...if it's recognized and you can read-write, then the USB-adapter is very likely OK to use.

I hope this makes sense and helps at least one person.

Todd

[mjochinsen]

Thanks labviewman! 

It works. 

It writes and reads fine. 

But the USB drive is not detected when I plug into Windows. 

[rolfk]

Windows does not have any ext3 filesystem driver so it of course won't see a driver formatted with the described procedure. There are ways to read ext3 disks also on Windows. But most involve a special Windows GUI application that knows how to read ext2/3 partitions such as the Explore2fs application.

Another approach is to install a real filesystem kernel device driver that supports ext2/3 partititions such as Ext2 Installable File System for Windows. 

This is one of the reasons that the RT documentation recommends FAT32 instead. While just about any system can read and write FAT32 nowadays, all other filesystems have different kind of support on different systems.

[mjochinsen]

Thanks Rolf,

Windows usually sees my ext3 drive. Although, I does not read the files.

It seems to work for me by modifying (removing) some the steps:

1) Using Windows

-format drive as ext3

2) power-off the cRIO

3) power-on the cRIO

4) login with Putty

5) plug-in the USB drive

6) "fdisk -l" until the USB drive is seen (/dev/sdb for me)

😎 format the drive

-"mke2fs -j /dev/sdb"

9) partition disk

-"fdisk /dev/sdb"

10) change permissions

-chmod 777 -R /dev/sdb

11) power off cRIO

12) unplug the USB drive

13 ) power-on the cRIO

14) once logged-in with Putty, plug-in the USB drive

15) "fdisk -l" until the drive shows-up (/dev/sdb)

16) change permissions

-chmod 777 -R /u

I don't want credit for Todd's (labviewman) solution, I am posting what I adapted it to my case and, so far, seems to work.

[rolfk]

It is unclear what you exactly try to do. If you want to connect the USB drive directly to a Windows computer, you will not be able to read (and write) to it if you format it in ext3 format unless you install one of the solutions that I mentioned.

If you are saying that you can't read or write the drive through Putty from a Windows computer when you connect it to the cRIO then that sounds bogus. Putty is just a program that connects the shell of the Linux system to your computer and there should be no reason other than the normal Unix access rights that could prevent you to see the files. Is the group and owner set right for the mount option? What does "ls -la" for the mounted drive entry show (not /dev/sdc but the mount point where the drive gets mounted, usually /media/u on NI Linux systems)?

And if the - signs indicate steps that you skip on the NI Linux system, you are basically not formating the drive in any way but just tickling Linux to get some feedback until the drive has been properly detected and installed in the device list. So it will keep whatever format it had before, which is likely FAT32 as even Windows still defaults to FAT32 as recommended format for most USB drives. If you format it with NTFS, then you can expect some trouble to fully use it on the NI Linux RT system.

The command that does the formating is mk2fs. Everything else except the fdisk command right after mk2fs in Todds list is simply just tickling the Linux OS until it shows that the drive was successfully installed into the device list.

[rolfk]

Basically what I'm missing in both Todds and your list is the umount command before attempting to format the drive. When you plugin the drive the automount scripts in NI Linux RT will mont the drive to /media/u or something similar. But as long as the drive is mounted you can not format it. So those format commands are most likely not doing anything.

What should work:

Wait for the drive to appear, do repeatedly (lsblk may work too but not sure if that is available on NI Linux RT)

fdisk -l

Make sure the device is not mounted (XY should be something like b1 or c1 seen in above list)

umount /dev/sdXY

You can skip the next two steps if you do not want to erase the complete disk first before formating it:

If you want to make sure everything is erased first (replace X with whatever your device has)

dd status=progress if=/dev/zero of=/dev/sdX bs=4k && sync  

Partition the device

fdisk /dev/sdX
n    (creates new partition)
p    (primary)
w    (write changes to disk)

Format the partition

mk2fs -j /dev/sdX1

That should be all

You need of course to be logged in as administrator for these commands to work or append su in front of each command, not sudo that is ubuntu 🙂

If you find that you need the last  chmod 777 -R /u then it is probably better to look in the usb automount scripts and change the access rights there in how the device should be mounted.