LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Best Practices for writing data to a network drive?

Historically I've just written data to the local hard drive on the test PC to prevent LabVIEW from locking up if the PC happens to lose network connectivity for whatever reason. However, for multiple reasons (redundancy, remote monitoring, etc.) it would be ideal for our test program to be able to write this info to a network drive location.

 

My question is, what are some Best Practices for accomplishing this? How have you overcome the inevitable loss of network connectivity? Is there a way to have LabVIEW buffer the data while continuing to test and then write it to the network once it has come available again...without freezing up? Are there any existing VIs that can handle this? I don't want to have to re-invent the wheel!

 

Additionally, it would be really nice if LabVIEW could figure out how to continue writing to a CSV or XLS file even though someone has the file open in another program. I can't even count the number of times I've had a test program crash because I forgot to close out of the datalog file in Excel before re-starting a test...nevermind that I was only LOOKING at the file and made no changes to it.

0 Kudos
Message 1 of 16
(5,221 Views)

@R.Gibson wrote:

Additionally, it would be really nice if LabVIEW could figure out how to continue writing to a CSV or XLS file even though someone has the file open in another program. I can't even count the number of times I've had a test program crash because I forgot to close out of the datalog file in Excel before re-starting a test...nevermind that I was only LOOKING at the file and made no changes to it.


That is an issue with Excel.  Excel is locking the file, preventing any other programs from being able to touch it.  For CSV files, you could try an application like Ultra Edit that does not lock the file.


GCentral
There are only two ways to tell somebody thanks: Kudos and Marked Solutions
Unofficial Forum Rules and Guidelines
"Not that we are sufficient in ourselves to claim anything as coming from us, but our sufficiency is from God" - 2 Corinthians 3:5
0 Kudos
Message 2 of 16
(5,200 Views)

You could also implement your own buffer.

 

Assuming you have enough RAM available to ride thru a network outage:

 

Write your data to a queue.

 

In another place (separate VI, maybe):

repeat

    Wait on DeQueue Element (timeout = -1)

    if Error

         Stopped = TRUE (kill the queue to terminate this loop)

    else

        try to write element to Network file

        if error

            push element back in FRONT of queue (Opposite end).

        end if

until Stopped

 

 

Or you could use a local file as a buffer.

 

Steve Bird
Culverson Software - Elegant software that is a pleasure to use.
Culverson.com


Blog for (mostly LabVIEW) programmers: Tips And Tricks

0 Kudos
Message 3 of 16
(5,157 Views)

Since we have no control over network downtime, I don't really like the RAM buffer concept. However, I like your idea of writing the file locally and then mirroring it (backing it up) to the network at some interval when the network is available. This could be run in a relatively simple independent state machine:

 

1) Wait some interval

2) Check network status (target file status?)

3) Copy the file (or go to error handler)

4) Return to a wait state

 

For the local locked file condition could use a similar state machine for error handling as well. For example:

 

IF file is locked

     THEN copy and rename file(add increment number to filename)

     THEN append new data

ELSE

     Append new data to existing file

0 Kudos
Message 4 of 16
(5,108 Views)

I save locally and when the file is closed (done writing data to it) then attempt a copy to the network.

If it fails a local copy of the file is made to another 'pending' folder.

When the application is started again it looks for any files in 'pending' and attempts to copy to the network again.

If network copy is successfull then it is deleted from the 'pending' folder.

 

-AK2DM

~~~~~~~~~~~~~~~~~~~~~~~~~~
"It’s the questions that drive us.”
~~~~~~~~~~~~~~~~~~~~~~~~~~
0 Kudos
Message 5 of 16
(5,095 Views)

@R.Gibson wrote:

 

Additionally, it would be really nice if LabVIEW could figure out how to continue writing to a CSV or XLS file even though someone has the file open in another program. I can't even count the number of times I've had a test program crash because I forgot to close out of the datalog file in Excel before re-starting a test...nevermind that I was only LOOKING at the file and made no changes to it.


Yer doin' it wrong...

 

If you open a file then pass the file reference in your program.

 

The file will be locked by LabVIEW and any other program that tries to open it will have to open a copy or open it read only.

 

Now LabVIEW can always write to the file even if you have a copy open in Excel.

 

========================
=== Engineer Ambiguously ===
========================
0 Kudos
Message 6 of 16
(5,079 Views)

...good point! I've gotten in the habbit of opening and closing resources on an as-needed basis to PREVENT locking them up, but in this case, it makes more sense to open the file at the beginning of the test and close it at the end to INTENTIONALLY lock the file.

 

...FWIW, I just found a "Deny Access" function in the [File I/O]>[Advanced File Functions] menu that looks promising.

0 Kudos
Message 7 of 16
(5,073 Views)

@R.Gibson wrote:

...good point! I've gotten in the habbit of opening and closing resources on an as-needed basis to PREVENT locking them up, but in this case, it makes more sense to open the file at the beginning of the test and close it at the end to INTENTIONALLY lock the file.


As you should do with all "limited resources" (Files, VISA sessions, Com ports, etc.) to prevent outside forces from crashing your test.

========================
=== Engineer Ambiguously ===
========================
0 Kudos
Message 8 of 16
(5,070 Views)

Will the DENY ACCESS status or the BUSY IN ANOTHER APP status survive a network outage?

 

You should test before committing...

 

Steve Bird
Culverson Software - Elegant software that is a pleasure to use.
Culverson.com


Blog for (mostly LabVIEW) programmers: Tips And Tricks

0 Kudos
Message 9 of 16
(5,065 Views)

@R.Gibson wrote:

 

 

1) Wait some interval

2) Check network status (target file status?)

3) Copy the file (or go to error handler)

4) Return to a wait state

 

 


That might or might not be sensible, depending on your partcular requirements.

 

If your file is a few seconds long, no problem.

But if your file is Megabytes, you do NOT want to implement the "synch" process you describe, simply because you would be copying the same bytes over and over and over.

 

The first time your synch routine runs, your file is 1k bytes, so it copies 1k.

The next time your synch routine runs, your file is 2k bytes, so it copies 2k.

The next time your synch routine runs, your file is 3k bytes, so it copies 3k.

The next time your synch routine runs, your file is 4k bytes, so it copies 4k.

 

You have copied 10 k bytes, and have only 4k to show for it.

 

Not a good plan, especially as you get larger.

 

You definitely should consider opening the remote file for appending. But that brings up the same lock question.

Steve Bird
Culverson Software - Elegant software that is a pleasure to use.
Culverson.com


Blog for (mostly LabVIEW) programmers: Tips And Tricks

0 Kudos
Message 10 of 16
(5,061 Views)