LabVIEW built-in file I/O VIs have poor performance on network file I/O operation?

[Chienwei]

Hi,

I use LabVIEW built-in VIs to save a big 2-D array data into a ascii file (with the "write character to file" vi). When I write it on the local hard drive, it takes about 5 seconds to complete the task. But when I write the file on the network, it takes more than 5 minutes. A typical output file is 5MB big.

If I write the file on the hard drive, and use the built-in File MOVE vi, it takes about 30 seconds to complete the MOVE operation. But if I use Windows File Explorer to move the same file, it takes less than 1 second.

Both the PC and server are connected on a full-duplex 100BaseT network. I tried different file I/O operations (including MOVE, create new file and LIST)
with LabVIEW VIs and compared the speed with windows file explorer. It turned out LabVIEW VIs performance is always A LOT slower than Windows File Explorer. I tried the same test on many different PCs, with windows 98, NT, 2000, and the results are the same.

Is there anyway to improve the file IO performance or is there any other workaround on this issue, such as calling Windows File Explorer to perform the same task via ActiveX maybe?

Your imputs will be very much appreciated. Thanks!

Dan

[wiebe@CARYA]

Hi,

Do the ASCII files have a .txt extention? I've noticed the same behaviour
once, with .txt files. Renaming them (to e.g. .txq) fixed the problem.

I'm clueless why this happend. If you extentions are not .txt, I don't have
a clue...

What if you copy the file, and then delete the original?

Regards,

Wiebe.



"dikrn" wrote in message
news:506500000008000000EA7C0000-1042324653000@exchange.ni.com...
> Hi,
>
> I use LabVIEW built-in VIs to save a big 2-D array data into a ascii
> file (with the "write character to file" vi). When I write it on the
> local hard drive, it takes about 5 seconds to complete the task. But
> when I write the file on the network, it takes more than 5 minutes. A
> typical output file is 5MB big.
>
> If I write the file
on the hard drive, and use the built-in File MOVE
> vi, it takes about 30 seconds to complete the MOVE operation. But if I
> use Windows File Explorer to move the same file, it takes less than 1
> second.
>
> Both the PC and server are connected on a full-duplex 100BaseT
> network. I tried different file I/O operations (including MOVE,
> create new file and LIST) with LabVIEW VIs and compared the speed with
> windows file explorer. It turned out LabVIEW VIs performance is
> always A LOT slower than Windows File Explorer. I tried the same test
> on many different PCs, with windows 98, NT, 2000, and the results are
> the same.
>
> Is there anyway to improve the file IO performance or is there any
> other workaround on this issue, such as calling Windows File Explorer
> to perform the same task via ActiveX maybe?
>
> Your imputs will be very much appreciated. Thanks!
>
> Dan

[Mads]

Just tested writing a 9MB 1D array and a 30 MB 2D array to a text file across the network (100BaseT) with "write characters to file",none of the operations are noticably slower than doing the same thing in Explorer (WinXP). Writing the files takes 1,6 and 6 seconds...

I have also tested the move operation, the performance is identical to that of Explorer.

Perhaps there is something in the way you time the operations and/or the way you have set up LabVIEW or the network that cause the problem...Have you tried it on a different network / writing to a different location?

[shoneill]

Quick comment,

if you are writing the file incrementally from within a loop, the performance will be far slower due to longer access times over the network. I had the same problems before, and re-wrote my software to buffer the files within LabVIEW itself (when reading a file).

Shane

[Chienwei]

Hi Wiebe,

Thanks for the suggestion. my files have many different file extensions (most of my files don't have .txt extension actually).

Copying then deleting the original file IS a faster way to do it. But actually the COPY VI runs a lot slower on network compared to Windows File Explorer too, especially when there are a lot of files stored in one folder. And if you put, say, 100 1kb files in a single folder and try to copy files from that folder to a network drive, it'll take a lot longer than you'd ever expect.

From what I experienced, all the LabVIEW File I/O VIs run very slow when doing I/O op on network drives. The performance is especially poor if the operation is performed between two network drives. I also tried this on LV6.1 and I see no im
provement at all.

Cheers,
Dan

[Chienwei]

Hi Mads,

If I just make a VI with a COPY VI in it it still runs slower compared to Windows File Explorer. I use all default settings in LabVIEW.

Dan

[Chienwei]

Hi Shane,

Thanks for your reply. Yes I noticed if a file is written incrementally within a loop using the file refnum, then it's even worse. It's a lot slower than writing all data at once.

Dan

[Chienwei]

Hi,

Here I'll attach a VI that creates some files, and move them around. It includes 5 steps

1. create 100 files on a local drive 1 (each file contains only a random number)
2. create same files with different filenames on a network drive
3. Move files within local drive to another local drive (on the same hard drive, but different folder)
4. Move files from local drive to a network drive
5. Move files from a network drive to another network drive on the same server

Now on one of my slower PC running win2k&LabVIEW 6.0.2, I have the following time on each steps in seconds:
1. 0.24
2. 5.45
3. 5.87
4. 22.41
5. 70.53

Then I tried the move operations with Windows Explorer. I did not time it, but the results were significantly faster.

Dan

[Mads]

Tried the test, the results on my computer and network was:

1. 0,06
2. 0,58
3. 0,09
4. 1,96
5. 2,73

Doing operation 5 takes more than two minutes(!) using Explorer...so at my end things look the opposite of what you experience. (2,4 GHz P4 WinXP Pro workstation, Dell PowerEdge 2500 1GHz P4 Win2K server, 100Mbs 3COM Superstack switched network of 40 workstations.)

It's a pity NI has not commented on your question, they might have some real comparison tests and info on the inner workings that might explain the performance differences...

[Chienwei]

Interesting results.

That may suggest LabVIEW and Windows File Explorer use different modules for network I/O operations, or they have completely different ways of implementing the I/O op. Also the performance comparison of the two is probably not consistant across different platforms/systems.

So far I haven't come up with a solution that's satisfying.

Thanks for the feedback!

Dan