Fault Tolerant Programming in RealTime

[DieselPower]

The Windows version of my DAQ program is rich with Try/Catch statements to avoid crashing on errors that are not worth crashing over. 

In C, however, I am struggling with handling faults.  I'll give one example from last night.  I wish I would have taken a picture because I don't know exactly where in the code it happened.

It was something like this:

token = strtok(buffer, "\t");
																
if (strstr(token, "ENGINE CONTROL"))
{
    token = strtok(NULL, "\t");
    CLControls[0]->FinalSetpoint = atoi(token);  			
    token = strtok(NULL, "\t");
    CLControls[1]->FinalSetpoint = atoi(token);  
    token = strtok(NULL, "\t");
    CLControls[0]->RampLength = CLControls[1]->RampLength = atoi(token);
    CLControls[0]->ThisSetpoint = EngineSpeed; 					
    CLControls[0]->LastSetpoint = EngineSpeed; //CLControls[1]->LastSetpoint = CLControls[1]->ThisSetpoint;
    CLControls[1]->ThisSetpoint = DL5.Torque; 					 
    CLControls[1]->cumError = 0;
    CLControls[0]->Active = CLControls[1]->Active = 1;
}

token was null which means that the message didn't get through for some reason.  While running in debugging mode (which we always do), it froze execution and threw a NULL argument to library function error. 

Freezing the program is not acceptable because we are running engines and we have safeties and coolant control systems tied to the software.

The obvious answer is that I need to study up on error handling in C, however, It seems as though when I am in debug mode, even if I try to handle an error it will always throw a message box and break out of execution.

I have three questions.

1.What is the best reference for error handling in Labwindows/CVI for real time applications?

2. Can I run through the development environment and still be fault tolerant?

3. Is it possible to set up the debugger to not break/crash on some kinds of errors?

Thank you!

[Dafrizzy]

Have you seen this knowledgebase article on how to disable those dialog boxes without disabling debugging features? https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z000000P6ArSAK

[DieselPower]

I checked my settings and this is how they are.  If no Break On options are set why would it break on a null value library error?

I checked here, but it doesn't give any example code or best practices.  Is there a better reference for handling errors for real time applications?   Understanding errChk in LabWindows™/CVI™

[DieselPower]

Here is another error that happened this weekend and threw a text box which stopped execution.  Additionally, some errors don't trace properly.  As you can see from the second picture, CVI does not tell me where the error actually occurred.  Is this because the error is in another .c file?

[Dafrizzy]

It looks like that specific error wasn't caught with the debugging options you set because it was a fatal run-time error. I don't believe it's possible to edit any settings to bypass a fatal run-time error. There are a lot of forum posts about that exact error you are seeing though. I've linked to some of them below so hopefully those discussions will have a solution.

https://forums.ni.com/t5/LabWindows-CVI/Missing-terminating-null-in-string-argument/td-p/246312

https://forums.ni.com/t5/LabWindows-CVI/Missing-terminating-Null/td-p/1644046

[DieselPower]

Daniel,

If I change my debugging settings will CVI be able to tell me where the error is?  It's bouncing out of execution anyway, so I'd at least like to know where.  I can't fix a null terminating string error if I don't know what line is causing the error.

[SaileshK]

Hi,

There's not a debugging setting that will tell you exactly where the error is. You can try using Breakpoints or something of that manner to narrow down where the issue might be occurring.

[DieselPower]

I'm not sure I fully understand.  On most of the errors I encounter the debugging environment takes me right to the line of code and highlights it.  I can then mouse over variables to check their values.  Are you trying to say that this should never happen or that not all errors are caught this way?

Additionally, using breakpoints is nearly impossible.  For one, the program is operating on a 100 Hz loop and there are continuous DAQ measurements occurring.  If I break for more than a few seconds I will get another error.  Secondly, if the error has to do with a glitch in communicated data that is completely random I can't really catch it happening.  Now if I use the ErrChk message I can trap it.  I still need to learn how to use that properly and then wrap ErrChk around every line of code in the program to ensure that every possible error is handled.  In the vast majority of cases the error can be ignored as the process will be correct on the next iteration unless a connection is totally broken.

I'm hoping that execution will not stop if the error happens inside of an ErrChk block.

Thank you.

[Dafrizzy]

From the screenshot you provided, it does look like the error handling isn't pointing to the correct line or really providing useful information. That specific error is typically a result of not properly sizing the buffer for the string to the right size. As a result, the string is left without a terminating character '/0'. I would guess there is a different part of your code where a situation like this is occurring and as a result, this error is thrown once the memory locations where this string was stored are used again for a different variable definition.

[DieselPower]

Is there any way to help find this error?  It is completely random.  It can happen once a week or once a day.  I'm not sure how I can put diagnostic messages everywhere that I process a string.  I wish the compiler would remember the offending line of code like it does for every other error!