NI-9063 can't interface with USB device

[User002]

I am trying to control a USB device "Kvaser Leaf Light V2" (For reference: Kvaser). I have verified control of the Kvaser from my PC. The problem occurs when I try to control the Kvaser from my 9063 cRIO. NI-MAX shows me an icon of the USB peripheral, but the icon has a red X on it.  When I click the USB, the info tab says "Not Present" for the status of the Kvaser.  Additionally, I cannot open the NI-VISA test panel for the USB. I used SSH to grab the dmesg from the cRIO and have attached it below. I attempted to use the rmmod command to see if I could unload a mod driver for the USB, but the cRIO couldn't find the rmmod command. The cRIO is brand new, but I upgraded the firmware to 2018 because the LV version we are using was newer than the cRIO's firmware.

---

PC Specs

OS: Windows 7 (I think we are using 32 bit)

LV: 2018

[brettski]

Hey The_Pratt!

Sounds like the most likely cause of this issue is that the device drivers aren't included in NI RT Linux (I have not confirmed, it's this just a hunch).

The good news is that the device manufacturer has a download link for their Linux drivers!

https://www.kvaser.com/product/kvaser-leaf-light-hs-v2/#downloads

Looks like you'll need to have at least gcc installed. 

Here is a tutorial on how to install packages and work with the package manager on NI Linux RT:

https://forums.ni.com/t5/NI-Linux-Real-Time-Documents/Step-by-step-Install-Configure-a-Package-using...

[User002]

Thank you for the reply.  I will try making/installing the linux driver.

[User002]

I seem to be having issues making the software on the cRIO.  I transferred the linux driver tar.gz file to the cRIO and unzipped it.  I then proceeded to the folder containing the driver I need (./linuxcan/leaf/) and ran the command of make.  The makefile appears to be looking for a file in /lib/modules/build in order to know how to make the driver.  I checked and the folder didn't exist.  Is there a way to get a hold of the contents that should be in the directory /lib/modules/build?  Attached is few screen-shots of my process.

[euio76 · Accepted solution]

@The_Pratt

When building software for Linux you sometimes (especially for drivers) need the kernel source (header files) to build (make) the software.

From the README file inside your tar ball, you will see Kvaser states that you do need these.

Most Linux distributions will include a linux-headers package which includes the headers used to compile the kernel.  The package will have a name similar to linux-headers-$(uname -r) and is included in the package repositories/feeds.  (The command 'uname -r' returns your kernel version).  As far as I could tell, NI doesn't provide such a package.  I found that going to http://download.ni.com/ni-linux-rt/feeds/2018/ was a good (not great) way to find packages that are available.  In order to get the header files, I rebuilt the kernel.

NI's instructions can be found here: https://github.com/ni/nilrt/blob/nilrt/18.0/KERNEL_SOURCE.txt

All of my comments are in regards to my specific hardware and software configuration.  I am using a cRIO-9039 which has a 64-bit processor running firmware 6.0.0f1 and NIRIO 18.0 for LabVIEW 2017.

I see that you are using a cRIO-9063 which, I believe, has an ARM processor instead of the x64 processor I have.

Building a kernel isn't trivial, but hopefully my steps will help fill in some of the gaps.  You will have to change a couple steps for your processor.

The headers that I ended up with are not an exact match for the kernel I am running, but they were close enough for my software to build and install properly.  At least one difference was my version of GCC.  If you run 'cat /proc/version' you can see what version of GCC the kernel was built with.  In my case it was 4.7.2, which is older.  My feeds contain version 6.3.0.  I found 4.7.2 in one of the old feeds (http://download.ni.com/ni-linux-rt/feeds/2014/x64/x86_64/); however, with this version installed, I could not get squashfs-tools or the kernel to build.  I ended up using the 6.3.0 version without a problem.  Since I only needed the headers, I chose not install the newly built kernel at the end of the process, but I would have done so if needed.  If you do want to use the new kernel, I would recommend that you skip the scmversion naming steps so that the new kernel will have a different name.

Keep in mind, that any changes to the kernel, including 3rd party drivers can change the performance of the device.

Using MAX:

Updated firmware 6.0.0f1 (cRIO-9039_6.0.0.cfg)

Installed NIRIO 18.0 for LabVIEW 2017

[optional] Installed EmbeddedUI and enable SSH

Using Filezilla (SFTP):

Create directory /home/admin/packages

Copy oecore-x86_64-core2-64-toolchain-5.0.sh from http://www.ni.com/download/labview-real-time-module-2017/6763/en/ to /home/admin/packages

[This may not be necessary for compiling on the target, I'm not sure]

[This is the link for my cRIO.  You can get a link for yours from http://www.ni.com/white-paper/14623/en/]

Copy squashfs4.3.tar.gz from https://sourceforge.net/projects/squashfs/files/ to /home/admin/packages

Using terminal in XFCE (the EmbeddedUI desktop) or PuTTY:

su admin

opkg update

opkg install zlib gzip libz-dev coreutils packagegroup-core-buildessential binutils-symlinks libc6-dev cpp cpp-symlinks bc gcc perl perl-modules fuse binutils make git bzip2 findutils python

cd ~/packages

chmod +x oecore-x86_64-core2-64-toolchain-5.0.sh

./oecore-x86_64-core2-64-toolchain-5.0.sh

tar -xzf squashfs4.3.tar.gz

cd squashfs4.3/squashfs-tools/

make

cp mksquashfs /usr/sbin

cd

git clone https://github.com/ni/linux.git

cd linux

git checkout nilrt/18.0/4.9

touch .scmversion && echo -n '-' > .scmversion && echo $(uname -r) | cut -d '-' -f3 >> .scmversion

echo "-6.0.0f1" > .scmversion

export ARCH=x86_64

make nati_x86_64_defconfig

ulimit -s 1024

make ni-pkg

mv /lib/modules/4.9.47-rt37-6.0.0f1 4.9.47-rt37-6.0.0f1.old

cp ni-install/x86_64/lib/modules/4.9.47-rt37-6.0.0f1/ /lib/modules/4.9.47-rt37-6.0.0f1 -r

cp /usr/local/natinst/tools tools.old -r

cp ni-install/x86_64/headers/module-versioning-image.squashfs /usr/local/natinst/tools

From this point, I was able install my software successfully.

[optional: use the new kernel]

cp /boot/runmode/bzImage bzImage.old
cp ni-install/x86_64/boot/bzImage /boot/runmode/bzImage
rm /usr/local/natinst/tools/kernel_versionsource /usr/local/natinst/tools/versioning_utils.sh
setup_versioning_env
versioning_call /usr/local/natinst/nikal/bin/updatedNIDrivers $(kernel_version)

References:

Videos from BradM: https://forums.ni.com/t5/NI-Linux-Real-Time-Documents/Working-with-the-Linux-Kernel-on-NI-LabVIEW-RT...

Official kernel build instructions: https://github.com/ni/nilrt/blob/nilrt/18.0/KERNEL_SOURCE.txt

https://forums.ni.com/t5/NI-Linux-Real-Time-Documents/Tutorial-Adding-Kernel-Modules-on-NI-Linux-Rea...

Post from BradM about building: https://forums.ni.com/t5/NI-Linux-Real-Time-Discussions/Unable-to-compile-uinput-module-on-cRIO-9030...

I hope this helps.

[metux · Accepted solution]

---

@brettski

The good news is that the device manufacturer has a download link for their Linux drivers!

https://www.kvaser.com/product/kvaser-leaf-light-hs-v2/#downloads 

Better forget it, their code is complete bs ... just reading their code causes huge pain.

They're trying to roll their completely own can stack, which is INCOMPATIBLE with standard CANSocket. Over 100.000 lines of code (!) for nothing that the kernel doesn't already provide since aeons. Really horrible to maintain - not suited for production in any ways.

If you're not a kernel hacker, it's not worth touching. 

Perhaps your device is already supported by the mainline kernel. Check the kernel docs.

Otherwise you could hire a kernel hacker to implement a driver (fortunately, we've got their source code, so it shouldn't be a big deal to find out how the device works) and bring it to mainline. If I had the machine here, it would propably take me few weeks of work (done that many times).

The next problem, of course, is updating the kernel on the crio. I've got some unsused one laying around somewhere (a little bit bigger model - probably w/ different CPU/mainboard, so I'd need a 9063 for reference testing). Porting the kernel itself shouldn't be a big deal, but I can't tell whether that's still compatible w/ the labview-generated code ... nobody knows (likely not even NI folks :p).

Anyways, the work to be done will be magnitudes more expensive than the crio itself.

(expect somewhat around 10..15k).

Oh, btw, that's yet another example for why suppliers of those devices should stay very close to mainline and do regular updates.

[metux]

When building software for Linux you sometimes (especially for drivers) need the kernel source (header files) to build (make) the software. 

Correct. Unless you're a kernel hacker (or wanna become one), better don't burn your time with OOT-modules. There're lots of things that could go wrong. If you wanna learn these things, better start off with the basics, eg. installing a Gentoo or LFS from scratch, take a deeper look at their build scripts, build/install kernels manually, write some tiny dummy kernel modules, etc ... you'll first have to learn to swim before trying to cross the ocean 😉

The headers that I ended up with are not an exact match for the kernel I am running, but they were close enough for my software to build and install properly. 

Dangerous ! Ther're lots of subtle things in there that affect binary compatiblity (even highly arch specific stuff). And these kernel-internals change frequently. That's why modules always should be built for the exact kernel version (and configuration!) they're supposed to run in - they're piece of the kernel (that just happens to live in extra files).

At least one difference was my version of GCC.  If you run 'cat /proc/version' you can see what version of GCC the kernel was built with.  In my case it was 4.7.2, which is older.  My feeds contain version 6.3.0. 

Phuh, but then don't play w/ high optimizations here (eg. -O3), the binary interfaces could change in subtle ways - it's a russian roulette game.

[User002]

Thank you for the information and advice.  I see that you are a kernel hacker.  My boss has decided (like you mentioned) that hacking the kernel would incur too much cost for this project.  As such, we have decided to use the NI CAN module (NI-9862).  It will cost about $500 more but will require no hacking of the cRIO and be more integrated.  This first unit will then be replicated for 7 other cells which will be much easier with a built in module instead of hacking all 8 cRIOs.  Granted though, once one is hacked it is easy enough to repeat.

[metux]

This first unit will then be replicated for 7 other cells which will be much easier with a built in module instead of hacking all 8 cRIOs.  Granted though, once one is hacked it is easy enough to repeat.

---

Okay, for such small quantity, it seems to be a proper economic decision.

OTOH, if quanty should scale up, the whole scenario quickly becomes entirely different.