LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

cRIO-9067 Real-time unexpected error restart (Hex 0x661)

Hi all,

 

I recently moved to LV2014 SP1 and NI-RIO 14.5 to support development of a cRIO-9067 Linux RT controller. The past two days of development have seen LabVIEW crash numerous times (eight at last count), along with numerous VIs being in an undeployable state due to some unseen error (there's no broken run arrows anywhere, and a LV restart seems to cure what was broken).

 

The above issues are tolerable, if somewhat annoying. What's concerning is the most recent error I received this morning. I had just run the top level VI from source (and verified it was running), went to make myself a coffee and came back to this error.

 

LabVIEW: (Hex 0x661) The LabVIEW Real-Time process encountered an unexpected error and restarted automatically.

 

cRIO-9067_crash.png

 

The VI wouldn't have been running for more than 10 minutes. I've since tried to reproduce the error without luck.

 

This is somewhat concerning as the application for this cRIO will be 24/7 process control. Is this a known issue with the newer Linux RT controllers? Is there anything that can be done to detect the error in LabVIEW?

 

Below is a screenshot of the software installed on the controller:

 

cRIO-9067_software.PNG

 




Certified LabVIEW Architect
Unless otherwise stated, all code snippets and examples provided
by me are "as is", and are free to use and modify without attribution.
0 Kudos
Message 1 of 32
(10,443 Views)

After running into the original error this morning, I formatted and redeployed all the software to the 9067, and had several hours of uninterrupted development time.

 

After about six hours this issue has cropped up again. This time I was just about to run my top level VI from source, when the project suddenly lost connection to the cRIO. Attempting to run from source again threw the previous error (0x661). Attempting to run from source again then gave me the below issue (note the run arrow in the background isn't broken):

 

LV2014SP1_deploy_error.PNG

 

A restart of LabVIEW fixes the above error, but this whole experience isn't exactly confidence inspiring.

 

Are there any log files or something I can grab from the controller when this issue happens to aid with debugging?

 

 

EDIT: I've switched over to a 9024 controller and 9116 chassis to try and rule out the 9067/Linux RT. I'll report back here if the error(s) appear with this configuration.




Certified LabVIEW Architect
Unless otherwise stated, all code snippets and examples provided
by me are "as is", and are free to use and modify without attribution.
0 Kudos
Message 2 of 32
(10,386 Views)

I have the same problem over here. I'm also using LV 2014 and a Linux based controller. I have a real time application deployed through the project. Now it is communicating using the STM TCP library. When the library stops communicating and goes back to port listning: a project pop up apears that connection has been lost. If you try to reconnect through the project, the exact same message apears.

Now I closed everything and restarted LabVIEW: the first few times went without any problem (so start / stop communication: and stop go to listen mode) but after a while exact the same problem.

This looks realy like a project related problem (so a problem of the development environment), so hopefully this wil not be a problem when deploying a real time executable. I log the time the controller starts, and I also use the user led to show the controller state, so I hope I can be sure about this when trying the same with a RT exe.

0 Kudos
Message 3 of 32
(10,251 Views)

I'm glad I'm not the only one who's seen this error.

 

After switching to the 9024, I didn't once see the issue. The project I'm working on was very LVOOP heavy (hundreds of classes with nested dynamic dispatch VIs being called hundreds of times per second), but due to poor dynamic dispatching performance was replaced with clusters and VIs. I've since switched back to the 9067 controller to see if the change in design had any effect on this issue, and coincidentally enough I haven't been able to reproduce it.

 

msmeulers, if you've tried and can't reproduce the error in an RT exe, perhaps try enabling debugging in the build. That may point toward it being a development environment issue.

 

Has anyone from NI seen this problem?




Certified LabVIEW Architect
Unless otherwise stated, all code snippets and examples provided
by me are "as is", and are free to use and modify without attribution.
0 Kudos
Message 4 of 32
(10,231 Views)

well a rt exe shows the exact same problem (this is realy a bummer): status led goes in flashing mode. I also use the user led which shows me what the state is of the RT application, it looks like this hangs. Attached is the code, using the STM library.

If i deploy the RT exe and try to connect with the GUI (called PC hoofdscherm) and I close the GUI, do this a few times, a crash of the crio apears.

 

Best regards,

0 Kudos
Message 5 of 32
(10,213 Views)

one short addition: LED is flashing 4 timers indicating that controller is out of memory?!

0 Kudos
Message 6 of 32
(10,204 Views)

 

You should be able to access the error logs on your controller through MAX.

 

Is There an Error Log File for My Real-Time Controller?-http://digital.ni.com/public.nsf/allkb/E734886E027D0B6586256F2400761E30?OpenDocument

 

How Do I Locate the Error Logs on My cRIO if I Don't Have MAX? - http://digital.ni.com/public.nsf/allkb/9D2F9D4F8C834D678625766D00633837

 

Could you post the error log files for the Compact RIO targets that reproduce the error?

 

It’s possible that a memory leak is occurring someplace in your program that causes the crash.

 

Also, are you using the System Config Set Time.VI anywhere in your application? There has been a reported issue with this VI in relation to this error, but the hex code error is not common. We can try to cross-compare the issue internally to see if there are simlarities to the reported case. 

 

Additionally, is this crash repeatable with other code, say a simple shipping example? I would imagine that the crash is related to a routine called in your program, but it’s possible there is a corruption in the software installation on your target.

 

Also, what is the ProcessCommandMessage.vi responsible for/doing in your application?

Will M.
Applications Engineer
National Instruments
0 Kudos
Message 7 of 32
(10,178 Views)

I just ran into this issue again, unfortunately. The RT error log from MAX is attached.

 

I'm not using System Config Set Time.vi, and I'm pretty sure there's no memory leak (at least according to the cRIO System State in the Distributed System Manager).




Certified LabVIEW Architect
Unless otherwise stated, all code snippets and examples provided
by me are "as is", and are free to use and modify without attribution.
0 Kudos
Message 8 of 32
(10,143 Views)

I just ran into the issue again. Error log is attached.




Certified LabVIEW Architect
Unless otherwise stated, all code snippets and examples provided
by me are "as is", and are free to use and modify without attribution.
0 Kudos
Message 9 of 32
(10,131 Views)

hi Will,

 

attached is our error log. We are usting the System Config Set Time.vi : but this one is called when creating a connection. We see this problem when the tcp connection closes. I excluded this code and had another try: resulted in the same behaviour. 

 

0 Kudos
Message 10 of 32
(10,127 Views)