NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

Implementing a Guard Timer per test step

Hi,

I'm a tad new to TestStand and windows et al (previously I've been a Linux/HPUX & embedded developer) so please be patient.

We're currently using the sequence editor as the operator interface (! not my choice) to execute suites of test sequences. The test steps use the C/C++ flexible adapter to call the test cases, the thing being tested is a 3G user equipment i.e. a mobile phone, in conjunction with some other test equipment.

The problem we have is that the guard timers in the test cases can be unreliable and can run continuously e.g. for days if unchecked. So my mission has been to implement a kind of watchdog timer that kicks in after a configurable amount of time, which then terminates the executing step & forces clean
up and continues from the next step.

To do this I updated the SequentialModel.seq (single pass), to which I added an action step prior to the main sequence callback which creates a timer. I also added a step after the MainSequence callback to delete the timer i.e. in the event that it doesn't fire.

If the first instance I tried terminating the execution from the timer handler. This stopped the testsuite running, which is not what is required - I need to stop the currently executing step,goto cleanup and then continue processing from the next step in the
suite .

In the next instance I tried setting the RTEOption for the current execution to continue - I then tried to post a TS_UIMsg_BreakOnRunTimeError to the UI to
attempt to stop the test step and force it to go to cleanup, I also set the error occured on the step and set the status to "Failed" etc ( the test steps are of the string value test type).

Sometimes this works and sometimes it don't - i.e. status is set to "??
?" - but using watch I can see that the error code and strings are set.


Any helpful comments on how I can sort this out would be gratefully accepted.

Cheers,
Justin.

PS : I don't have labwindows or labview only teststand
0 Kudos
Message 1 of 11
(4,882 Views)
JRP -
I assume that what you are asking is that if a single step runs too long, you want another thread to be able to instruct that step to stop, return, and let the thread in the execution execute the next step as if the previous step completed correctly.

How does the "watchdog timer" stop or terminate the already runing step? I assume that the step that you want to stop is monitoring for execution termination?

Scott Richardson (NI)
Scott Richardson
https://testeract.com
0 Kudos
Message 2 of 11
(4,882 Views)
Hi Scott,

Thanks for your reply; In essence your correct - I need to shut down a running step at some given time i.e. a timeout.

Except that timer handler will at a guess be called by the NT kernel (?) - I've not created a seperate thread for that - only for the timer itself - because it uses waitable timers.

As for stopping the step - theres the rub. As per the original query - I've tried "engineering" a runtime
error and setting the RTE option for the execution to Continue. Sometimes it works, sometimes it don't.

Best Regards,
Justin
0 Kudos
Message 3 of 11
(4,882 Views)
Justin -
I am trying to better understand better your requirements...

Questions:
1) What is the step doing at the time it is busy and you want to stop it.
2) Why is it that the step does not complete when expected: timing issue, bad code, hardware, just hung?
3) How will the monitor tell the step to stop, or is this not defined yet?
4) Does the step that does not stop have access to the TestStand sequence context it is running in?
5) Does the step already monitor for execution termination?
6) When you say that your "engineering" sometimes does not work, does the step just not stop or does the step stop but the execution fully terminate when you did not want it to?

Scott Richardson (NI)
Scott Richardson
https://testeract.com
0 Kudos
Message 4 of 11
(4,882 Views)
Hi Scott,

Thanks for your continued patience The answers to you questions are as follows - well I hope they are answers !.

1) The Step is C code thats controlling our User Equipment (UE - 3G mobile phone) and a signalling tester.

2) Don't know; But as we're testing the UE code and hardware we may hit problems like this, so we need a method of rescuing the situation when it goes pear shaped.

3) I've tried the following i.e. my "engineered" approach.
- The sequence context is passed to the timer.
- when the timer fires we get the execution from the
from context and navigate our way around the
containment tree to obtain the current sequence
context i.e. stack frame zero, from the
foreground thread.
- P
ost a TS_UIMsg_BreakOnRunTimeError message to
the UI
- which is in our case TestStand
- Set the Step.Result.Status to
TS_ResultStatus_Failed
- Set the RunState.Step.Result.Error.Occurred to
TRUE.
- Set the RunState.Step.Result.Error.Code to
TS_Err_OperationTimedOut
- Set the RunState.Step.Result.Status to
TS_ResultStatus_Failed

- The Post a TS_UIMsg_ResumeFromBreak message to
the UI

4) get the execution from the context, get the foreground thread index, then call the GetThread method
to obtain the thread and then call GetSequenceContext
with a frame id of zero. Then get the SequenceFile,
Get the Sequence and then get the Step.

5) No it doesn't.
I didn't know you could do that.

6)No it doesn't stop. Sometimes the Step.Result.Status
is visibly set to Failed, but most of the time its
just set as "???" - I've also tried reading the
status back after I've set it and it reads as "???".
But if I
watch it in the sequence editor its set.
Bizarre.

Best Regards,
Justin
0 Kudos
Message 5 of 11
(4,882 Views)
Justin -
I would assume that with all the work that you outlined in item (3), the step still will not return properly. Keep in mind that if the step is executing code, i.e. waiting, looping or just blocking, their is no way for the step to know to return to its caller, TestStand, unless the code is written specifically to do this.

Typically termination occurs only when the Execution.Terminate method is called. Using execution termination to instruct a step to stop and just fail is not the way to go, because once an execution initiates a termination, you typically cannot easily stop it. What you are trying to do is just tell a step to stop and fail.

To illustrate existing code that steps can use to monitor some state while executing, I will show you an example for when a step monitors for termination of the execution. For example in the file \Components\NI\StepTypes\SyncSteps\Wait.cpp has code, the DoWaitForTimeInterval function monitors whether the execution has been terminated while it is waiting for the sunchronization operation to occur. If the execution is terminated while it is waiting, it leaves. In addition, the code also processes messages so that it can allow TestStand to do other messaging based things while it waits. This makes the TestStand application GUI thread more responsive. Another thing the code does it tells the execution that it can be externally suspended. This is nice because you can break in TestStand while the step is still running.

Here is a simplified version of the C++ code:

static void DoWork(struct IDispatch *seqContextDisp)
{
MSG msg;
SequenceContextPtr seqContext;
ExecutionPtr execution;
PropertyObjectPtr termMonitorData;
IEnginePtr engine;
ExecutionRunStates runState;
ExecutionTerminationStates termState_NotUsed;

// initialize execution state variables
seqContext.Attach((SequenceContext *)seqContextDisp, true);
execution = seqContext->GetExecution();
engine = seqContext->GetEngineAsDispatch();
termMonitorData = execution->InitTerminationMonitor();

// place your code here to start task
...

termMonitorData = execution->InitTerminationMonitor();
while (true)
{
execution->GetStates(&runState, &termState_NotUsed);

// place your code here to monitor or continue task
...

If (workDone)
break;

// process messages if any are pending
// must peek because we don't want to block
while (::PeekMessage(&msg, 0, 0, 0, PM_NOREMOVE))
{
::GetMessage(&msg, NULL, 0, 0);
::TranslateMessage(&msg);
::DispatchMessage(&msg);
}

if(execution->GetTerminationMonitorStatus(termMonitorData, _variant_t(seqContext.GetInterfacePtr())))
break; // break out of loop for term or abort.
}
}


Steps can/should do this whenever they will be busy doing something for a long time and the task is iterative or waits without blocking. Clearly if you are making calls into a driver that do not return, then the above code is of no use. But if you make multiple long blocking calls, the above code could be used between each long blocking call.

Now, using execution termination to instruct a step to stop and just fail is not the way to go. Instead you could use the above idea but instead have the step just monitor its own status. While a step is running the Step.Result.Status is set to "Running". If the step "sees" that its status is set to something different than what it was when it started, it could just exit or even set its status to "Failed" before exiting.

Scott Richardson (NI)
Scott Richardson
https://testeract.com
0 Kudos
Message 6 of 11
(4,882 Views)
Hi Scott

Thanks for your detailed reply and continued patience.
I'll go thru the details today by myself and with the
suppliers of the (test) driver software. I'll let you know what happens.

Cheers,
Justin
0 Kudos
Message 7 of 11
(4,882 Views)
Hi Scott,

Our software vendors have provided us with information regarding a function we can call to cancel the test step. This works ;-}.

The next problem (wait for it ...) is that the guard
timers are currently created prior to the main sequence
call back - its driven from an action in the sequential model.

This means (you've most probably worked this out) that
it only gets called once i.e. before the first step in the sequence. So is/are there any pre/post step
callbacks I can use (also override) to provide hooks
to create/delete the timers.

Cheers,
Justin
0 Kudos
Message 8 of 11
(4,881 Views)
Justin -
It is not clear what you mean. Clearly the actions in the sequencial model initiate the running of these timers. Do you want to reinitiate the test step after it has been cancelled? If this is the case, how to do this all depends on whether the guard timers can be "restarted"? This is clearly an issue to bring up with the software vendor of the guard timers.

Scott Richardson (NI)
Scott Richardson
https://testeract.com
0 Kudos
Message 9 of 11
(4,881 Views)
Hi Scott,

I wrote the guard timer code but I needed something
to trigger them, hence I did the following :-

CreateGuardTimer
.
MainSequenceCallback
.
DeleteGuardTimer

The create and delete being c/c++ flexible adapter
calls to a DLL.

Most of the top level test cases are structured as follows :-

Setup - do some stuff

step 1 - is a sequence call
.
step 2 - is a sequence call
.
step 3 - is a sequence call

cleanup - cleanup some stuff.

The model being used is the sequential model, single
pass. With regard to your question, I don't need to
reinitiate them (they may be required later). I need to
stop/delete the timer after the step has finished and
start/create another before
the next step commences.
Hence a guard timer per test step.

Cheers and again many thanks for your patience.
Justin
0 Kudos
Message 10 of 11
(4,881 Views)