unable to transfer rt data to host using FTP

[falcon98]

Hi,

We save our experimental data in a folder in RT (pxi 1042 , pxi 8108 RT controller). Later we retrieve the data to our host for analysis. Now we used to regularly used to make new folders in RT to reduce the llop time to read the files in host but somehow the data in particular folder increased due to longer experimental campaigns.

Now the particular folder has become unaccessible , we tried to obtain the data using FTp methods like from browser, msdos commands but nothing has worked till now

We tried plugging n a usb drive in RT controller and tried to transfer files on it but on executing the VI the RT gets disconnected.(though we are bale to access other folders)

This folder has our important experimental data so it is crucial to get hold of it.

Please suggest what possible options exist.

Thanks a lot

[Sam_Sharp]

Disclaimer: Some of the suggestions I've made here are potentially warranty voiding and/or could cause a loss of data (e.g. if you damaged the drive trying to remove it) - you have been warned!.

If the RT is disconnecting, that's a sign that you're maxing out the CPU of the RT target - probably because it's trying to perform lots of file operations etc.

The RT controller does have a (standard?) 2.5" hard-drive, you might be able to take it out of your PXI, put it in a caddy and copy the files directly onto your PC - but I would definitely check with NI first to make sure its not going to void your warranty - I'd see it as a last resort option!

You could also use a USB DVD Drive to boot a 'live CD' (a non-installed bootable OS which has some utilities etc. - usually using Linux) and then use that to copy the files onto a memory stick - providing it uses a fairly standard file-system, you should be able to mount the drive to access the files.

For future reference: 

1) Having too many files in a folder will cause things like 'get directory listing' to take longer and use more resources

2) Rather than get a directory listing, have your software create a log of files created, and use this log to FTP the files you need to transfer

3) Have your software push the files after they are closed, rather than transferring them in bulk - or find a way to rate-limit the number of files transferred

4) You have lots of options for transferring the files - FTP, HTTP Post, TCP/IP, USB Stick

How many files are in the folder? Do you have any idea? If you have hundreds/thousands of files in the folder, you might just have to be patient for any file system operations (e.g. directory listing).

[Bob_Schor]

 We run LabVIEW Real-Time with a PXI 8106 (I think -- I'm at a conference far from my lab, so I can't check).  When the PXI boots, it starts up an FTP Server that always runs in the background.  Among other things, this means that from any Windows machine connected to it via Ethernet (for example, its Host machine that runs the Host part of the RT code), we can "see" it as an external FTP site (it shows up in My Network Places, complete with its own icon and name that I believe I gave it when I first connected).  Using this link, we can freely open files and folders on the PXI and exchange data at will (even deleting the files from the PXI after we've copied them over).

In Version 1 of our main data acquisition project (written by my predecessors in LabVIEW 7), we had similar hardware, were able to "see" the PXI as a Network Place (using FTP), and could do the manual File Transfer.  However, we could also transfer the files programmatically -- we had places in the code where we could request "Transfer all the files to the PC and delete them here on the PXI".  This could sometimes take a few minutes ...

With Version 2, completely rewritten from scratch in LabVIEW 2012 (and recently ported to LabVIEW 2014), we use four Network Streams to communicate with the PXI.  A pair are used to sent Messages back and forth to Queued Message Handlers running on both Host and Remote that manage the top-level code on both platforms.  The other two streams are data streams from the PXI -- instead of saving any data in the PXI, we stream it to the PC (typically we collect 16-24 channels of I16 data at 1KHz), where it is immediately written to disk and simultaneously 50-point averages are used for on-line monitoring (i.e. plotting the data or outputting selected numerical values).  One of the data streams is the 1KHz sampled data I just described, the other is for asynchronous "Event" data (for example, every time a DIO line goes high or low, we create an "Event" record that records the type of Event, the time of the Event, and the "value" of the Event.  We also stream this to a (separate) file on the PC, as well as using it to update selected Front Panel indicators (for example, those DIO lines tied to "Flags" on the PXI hardware are brought to Flag Indicators on the Host to allow us to see the Flags go up and down).

We're pretty confident that we are capturing all of the data (something that was not true in Version 1, where the data were saved in memory and periodically streamed to the PXI's hard drive).  Of course, if your data rates are much higher than TCP/IP transfer rates, this might not work for you ...

Bob Schor

[falcon98 · Accepted solution]

Thanks a lot for your answers. We gave a lst try by inserting a USB pend drive in the RT controller.

We then programmatically transferrred the data from RT drive to USB drive. We were able to achieve this and could get the data.