Configure Watchdog in Veristand

[GSinMN]

Hello All,

  Trying to incorporate watchdog functions into my project, and would like some clarification on how it is implemented.  I've been studying the watchdog example project in Labview, and I think I understand how it works from the host, but if I run it with a RT target that is running a Veristand project, does it work with Veristand, or does it work outside of Veristand?  Also, if I wanted to deploy the watchdog configuration functions to run independently on the target, would it be part of a Custom Device, or just a model?  Please advise.  Thanks.

GSinMN 

[Che_T.]

Hi GSinMN

VeriStand implements Watchdog capabilities, but the implementation is very different from how it's implemented in LabVIEW Real-Time.

Essentially how it works is that the watchdog global time is updated every 500ms by a loop that is lower priority than the HP and LP loops.  The HP loop checks when it runs to see if it has been more than 1s since the watchdog time was updated. If it has it throws an error and stops execution. This code only gets run when the engine is on an RT target but it doesn't use any of the traditional watchdog VIs.  If you check the filter watchdog errors the global channel still gets updated but the error never gets thrown (so filtering errors disables this watchdog).  You can customize the behavior of the watchdog by monitoring the channel yourself.

So, to customize the behavior, enable the Filter Watchdog Errors checkbox on the Controller Configuration page of the System Explorer window. You can configure watchdog functionality by monitoring the Watchdog Timer system channel using alarms and procedures.

What kind of watchdog characteristics do you need?  Do you need FPGA/DAQ outputs to go to a safe level? Or do you just want the whole system to stop executing?

[GSinMN]

Hello Che. T,

  Thanks for the response.  The underlying question was indeed how to draw the line between Veristand and Labview RT.  I think I understand it now.  

Right now, we are just investigating all options related to watchdogs and remote power.  Eventually, we will want some way to reboot the entire controller chassis, which will likely involve and external module, and the power control connector (9-pin Dsub on back), and we will also, as you guessed, want the I/O to revert to safe states.  I'm working with a 8133 controller, 6514 I/O card, and a 7854R FPGA card.  I've already been reading about the built-in watchdog characteristics of the industrial line of I/O cards, and it looks they will be useful.

So just one question to clarify:  You mentioned that if the Veristand watchdog is left to operate in it's default configuration, timer expiration will trigger a fault, and execution will stop.  What exactly happens after that?  Does the entire controller reboot, or does the control loop in Veristand just stop?  

Thanks again,

GSinMN  

[Che_T.]

Hi GsinMN,

If the timer expires, it will throw an exception and VeriStand engine will stop.  The controller doesn't reboot.  To get the engine running again, you can redeploy the SDF.

I hope this helps!

[GSinMN]

Good Morning Che T,

  Thanks, that does indeed help.  I think I am getting a good picture of how I need to architect my application.  Any suggestions how I could consistently "force" the timer to expire, so that I can test the watchdog functionality?  I'd like to stay as close to objective functionality as possible, and don't want any simulated variables to interfere.

Thanks again,

GSinMN 

[Che_T.]

Hi GSinMN,

I totally understand.  Currently, the easiest way to force the timer to expire is to use an inline custom device that takes longer than a second to execute.  The easiest way to create this custom device would be to use the Easy Custom Device Tool.  Please let me know if you have any questions on this and I can provide some guidance.

[dcarvalho86]

Hi Che_T,

You said that: "The controller doesn't reboot.  To get the engine running again, you can redeploy the SDF."

I have one question: When deploy a SDF the PXI is reboot too?

Thanks!

[Inertia]

If you are concerned about putting your system into a safe state, then you should consider an external watchdog that is clocked/pulsed by VeriStand.  Depending on the failure mode, VeriStand may or may not be able to control its outputs so the best safety is to have a dedicted device.   Something like this can do the trick or at least shut off stuff and prevent power from being reapplied until VeriStand is back up and running.  I've also used a software watchdog coming from the PC side (have a VeriStand service that toggles a user variable that is alarmed) to VeriStand/LabVIEW RT when I wanted the RT side to gracefully shutdown if the operator interface went down, e.g. if you are logging data on the PC during a long term test and the PC crashes.

[jeepers52004]

Hello Inertia,

I'm pretty new to VeriStand and I'm the only person at our company with any experience with it.  Could you please explain in a little more implementation level detail how we might be able to implement the pc based watchdog you mentioned "...(have a VeriStand service that toggles a user variable that is alarmed)..."? 

We've experienced our first crash of Veristand on the PC while the cRIO kept executing the control logic.  I need to investigate the failure mode to know if the stimulus file safety logic would still stop shut the system down as intended but having something watching for a Veristand HMI crash would be an immense improvement.

Thanks

Michael

[Inertia]

This should work - I don't think support for this is native yet so this is the old school way:

1. Create a user variable called Watchdog (in System Explorer)

2. Create two alarms: Watchdog_High and Watchdog_Low ,  Watchdog_High has limits from 0.5 to 2, a 5 second delay, enabled by default.  Watchdog_Low has limits from -2 to -0.5 with similar delay, enable, etc.

3. Link both alarms to the same procedure.  The procedure performs your shutdown script (if any) and does a reset alarm and exit.

A rough example is shown in the image below.

Part 2 requires a custom display template (or service) that is going to toggle the user variable Watchdog between 1 and -1 faster than every 5 seconds (or whatever delay you set in the alarms).  A quick and dirty way to do this is just to modify the Numeric Control - Medium.vi display template - please copy as a new name! (located in ..\Documents and Settings\...\National Instruments\NI VeriStand 20xxx\Display Templates).  I included an image of what I did.  Note this is hacked, the cleaner way is to create service to handle this more robustly but this should give you the idea.  If the PC crashes or locks up for any period of time longer than the delays on the alarms, the "real-time" side will fire an alarm and do whatever you've told it to do!