Asynchronous custom device fails to shutdown

[Dbryant]

While I was developing an asynchronous custom device, I ran into an issue with undeploying the .nivsproj I created to test the custom device.

The custom device would send out the Error: -307730 Error Message: NI Veristand: One or more asynchronous custom devices failed to shut down properly and were aborted by the Veristand Engine.

I decided to do some troubleshooting for what could be wrong with the custom device by creating a seperate custom device using the Custom Device Template Tool. I left the template completely untouched and deployed it to our pharlap machine. When I undeployed the untouched asynchronous custom device it gave me the same error as above.

I'm using Labview 2010 Professional Development System version 10.0f2, NI Veristand 2010 licensed for a full development system, and deploying the .nivsproj onto a Pharlap machine.

[Zach-H]

Do you get the same error when running one of the custom devices that ships with VeriStand? 

This Help Document lists the included devices. 

I'd like to determine if its any customer device that returning the error, or if its only devices you make yourself. 

Have you restarted  VeriStand since the issue happened? (just looking to make sure gateway isn't stuck in a bad state)

Let me know,

[Dbryant]

I used the Embedded Data Logger that ships with Veristand. It shutdown without giving any errors.

I have restarted Veristand since the issue originally occurred. It still gives the same error.

[Zach-H]

I believe you've already heard this, but the example was created as a starting point of an application, not as a complete customer device. Its not expected to run without error if functionality isn't added to it. 

Have you considered, or are you using the Easy Custom Device Tool? It might help with development.

Have you implemented any debug strings in your custom device to print to the console? They may help us catch what's going wrong. 

[timtamslam]

Hey Dbryant,

I also tested the 2010 Custom Device Template Tool for building a default Asynchronous custom devices on a PXI target. I was able to generate the same error on the RT console upon undeployment.

I then tested the 2011 Custom Device Template Tool for building a default Asynchronous custom device. This did not reproduce the problem.

Between 2010 and 2011 there is a major change in how the Asynchronous custom device is handled in VeriStand. I've attached screenshots of the RT driver VI for the Asynchronous custom device for 2010 and 2011 for anyone that is curious. The primary difference is that the newer versions use event driven commands from the VeriStand Engine in order to handle proper shutdown of the Custom Device. In the 2010 version of the Template Tool, it creates a Timed Loop that relies on Wakeup Reason node information in order to shutdown the loop. The 2011 version uses a Shutdown event that's sent from the VeriStand Engine.

I tested another type of Asynchronous Device (code attached below) which is very similar to what is produced from the template tool. This Custom Device produces a message that says, "Waiting for Custom Device loops to shut down..." So it also doesn't clear up the error generation on undeployment.

Without a proper way to shutdown the loop on the Custom Device we will have this problem.

Even with this message on undeployment though, I would not expect anything to run incorrectly on your next deployment. So we may be able to ignore this error.

Does the custom device you're developing require an asynchronous format? Or can you use a different custom device? Another approach we may be able to take is using the Inline custom device and incorporating a while loop in it. We'd basically be building a unique type of Asynchronous functionality this way as well.

Also, Is it possible to upgrade to VeriStand 2011? This may be a possible way to approach the problems you're seeing. Newer versions also include more functionality and reliable custom devices.

[timtamslam]

Hey Dbryant,

Looking at this more specifically, The likely reason why the Async Custom Device does not properly shutdown in this case is that we have no configured channels in our system. Under normal circumstances, the VeriStand Engine closes the RT FIFO on its end which then causes the RT FIFO Write or Read in the Custom Device to produce an error, and then stop the Asynchronous Loop. However, with how this Custom device is built, we aren't using any RT FIFO functions because when the VI calls the NI VeriStand - Get Custom Device Channel List.vi twice and returns a zero for both, our Custom device will be constantly running in this configuration.

However, if we were to add channels, the Custom device should shutdown properly.

[Dbryant · Accepted solution]

Sorry about the delayed reponse.

Zach-H using the Easy Custom Device Tool you provided I was able to create an asynchronous custom device that was able to shutdown. This allowed me to use print statements in both devices to troubleshoot what was causing the device to throw an error on shutdown.

The culprit seemed to be the NI VeriStand - Get Loop Type.vi. The loop type is never set in the initialization.vi or main.vi of the Custom Device Template Tool. So, when the RT Driver VI.vi ran the Get Loop Type.vi it would always output false for Use Device clock and never use the correct Device Clock name. Without the Device Clock name it would never get a Wakeup Reason.

I forced the vi to use the Device Clock given in the Custom Device Template tool and I was able to shut down properly.

TimothyA, you are correct about the functionality of the Custom device. Even with the improper shutdown error, the Custom device continues to run normally after each deployment.

Thanks for pointing out the problem with no channels being added to the Custom device. I'm going to make sure the custom device I develop handles the case in which no channels, input or output, are added.

Thanks for the help.

[timtamslam]

Hey Dbryant,

Well done in finding the problem with the Get Loop Type.vi. Kudos on your investigative work. I should do a bit more investigative work on my end and get these complications reported so that we can more quickly solve problems like this in the future.

I think your plan moving forward sounds pretty solid. As always, if you ever need any further assistance don't hesitate to reach out to us!

[GSinMN]

Hello All,

  I see that this post has been listed as resolved, but I have experienced similar behavior to what has been posted here, and have a few follow-up questions related to the details. 

  Related to the Custom Device shutdown being aborted:  Is there a certain amount of time that the Veristand Engine waits before it will abort, and if so, is the time configurable?  The reason I ask is that I have a time delay built into one of my sub VIs in the custom device, and I would like to know if that delay could be causing the shutdown to abort.

   Related to the Real-Time FIFO:  I have been experiencing the error -2206 (RT FIFO does not exist) consistently, every time I undeploy my custom device.  I originally thought this was due to the fact that the reference for the FIFO was shutdown to early, but then realized that the Veristand Engine controls the reference.  Is it possible to take control of the FIFO reference to make sure it does not close before the last write?

FYI: I am working in Veristand 2012 and Labview 2012. 

Thanks in advance,

GSinMN  

[StephenB]

The error -2206 is because NI VeriStand deletes the FIFO when its shutting down. Even if you could write to the FIFO... NI VeriStand wouldn't be listening because it is shutting down. There is no reason to be updating channel values when shutting down that I can think of... can you explain your use case?

I'm not sure how long NI VeriStand waits before aborting the custom device. it probably is 5-10 seconds from what I've seen. If you need to run something at shut down and not be aborted... I suggest launching a VI asynchronously.