LabVIEW

cancel
Showing results for 
Search instead for 
Did you mean: 

Configuration loss on Real-Time?

THERE WAS A SUDDEN LOSS OF ALL CONFIGURATION DATA ON OUR REAL-TIME DEVICE (PXI-8145RT). Max would no longer see it. I could no longer connect to the Real-Time device via LabVIEW. I could not Ping the device from our network.

What would cause such a thing? As far as I can tell, there was no sudden power loss or surge, no user intervention that caused it, etc.

This is at least the 2nd time something like this has happened with our Real-Time system. We had to clear the Flash memory and reload all the software and reset the IP address.

I'm not getting warm and fuzzies about this when our system suddenly requires reconfiguration from scratch.

- Con
0 Kudos
Message 1 of 8
(4,827 Views)
Hi, Con.

I can think of one thing that would cause a similar behavior. I'm thinking that another computer on the network is trying to get the same IP address that the 8145 is already using, and that kicks the controller out of the network and possibly halts it. If that was the case, you might want to check with your IT department and make sure that they are reserving the IP address, so no other computer can try and use it.
Now, if this was the actual cause of this behavior, I wouldn't think that the only way of getting the controller back is by reformatting the Compact Flash card. You should just be able to reset the IP using the DIP switches on the front. That will set the IP address to 0.0.0.0 and you should be able to reassign an IP to it. Have you t
ried doing this? Another thing you could try is, prior to format the Flash, make a backup of it; that way, after you format you can just copy the files and save time downloading the software back to the controller. Finally, make sure you are formatting the Flash to be FAT file system.
What kind of VI are you running in RT? Do you have any data acquisition boards? I wonder if the VI is running out of memory or disk space.

If you can think of any other piece of info that could be useful, please, let me know.

Gustavo Valdes
Message 2 of 8
(4,827 Views)
Thanks for your suggestions Gustavo!

I will check with our IT dept about having another computer trying to obtain that IP address - but still I would think I would be able to reassign an IP address through Max and keep going. But this froze everything. No network communication signal light flashing on the Ethernet connection (it was a solid green and thats all). The User1 light was a solid amber.

As far as what type of VIs - for now they are mostly straightforward vi's that read information from an FPGA 7831R card and perform some calculations.

I guess the thing that concerns me (ALOT) is that once the Real-Time system is deployed, its supposed to be standalone and running! Not needing to be taken apart and the Flash memory cleared with so
me 3rd party device! You know what I mean?

Thanks!

- Con

As far as running out of space - not that I know of (do you know of a way to check). The VI's I was developing at the time were not writing to files or creating huge arrays or anything like that.
0 Kudos
Message 3 of 8
(4,827 Views)
Con,

I can recommend a couple of things to look for if you run into this issue again. There are a few scenarios that might cause the symptoms you are seeing, and, if there is really something wrong with the system, we�d like to know about it anyway:

Obvious things to look for:

- You could have a startup application that uses up 100% of the CPU.
- Your target might have been configured for a different sub-net while it was running.
- Your target might have crashed repeatedly, falling back into safe mode (although it should still be visible)

Not so obvious things:

- File corruption, for whatever reason, affecting files that get accessed during start-up of the system. This could cause the system to fall back into an un-configured, null-IP state.
- Reboot due to a crash and then fail to obtain an IP through DHCP, if you are using DHCP.

Next time the target falls into such a state I would suggest the following:

1) Reboot it and try to find it in MAX
2) Disable any start-up applications using the hardware switches on the controller.
3) Since this is a target that does not have a video output, use a Serial null-modem cable and a terminal utility (hyperterminal would do) to check the output console. For this you would need to enable Serial Redirect. The User Manual for the controller tells you what switch to use to enable Serial Redirect:
�PXI-8140RT Series User Manual�

I use these settings:

Bps: 9600
Data bits: 8
Parity: none
Stop bits: 1
Flow Control: Hardware

4) The serial redirect output should at least give you an indication of what�s going on, whether it�s failing to get an IP, using the wrong IP, hanging, running out of memory or even if it�s just continually rebooting at startup.
5) Force the target to boot into Safe-Mode. The User Manual mentions how to do this. This should make the target show up again, assuming it is within the same subnet or that it has the valid IP that you are looking for.
6) If the controller comes back to life, try opening an FTP session to it (you could use the FTP client in MAX or Internet Explorer if necessary) and get the following files from the system:

- C:\ni-rt.ini
- C:\persist.pal (if it exists)
- C:\ni-rt\system\rtlog.txt (if it exists)
- C:\ni-rt\system\CONFIG3.MXC (if it exists)
- C:\ni-rt\system\config3.mxs (if it exists)
- C:\ni-rt\system\mxsjar.ini (if it exists)

We might be able to tell what happened from these files. Additionally, it would be good to add more information like the version of LabVIEW Real-Time you are using, boards and drivers, whether you are using a start-up application or not, if using DHCP, and any other details regarding the setup.

7) Worst case scenario would require you to take the Compact flash card out of the controller and get the files (before deleting them) by using a Compact Flash reader on a PC.

Booting the controller into safe-mode should be a no-excuse setting in which you should be able to see the controller from MAX, or ping it, assuming the basic networking conditions are met.

While your application is running you could check for available disk space by using the Volume Info VI (File-IO palette). You could check memory usage by running the Real-Time System Manager (RT 7.1) or using the following utility:

�Programmatically Tracking Memory in LabVIEW Real-Time�

Let us know,

Alejandro

LabVIEW Real-Time R&D
0 Kudos
Message 4 of 8
(4,827 Views)
Thanks for all your information as well. I will keep it nearby and include the information you requested if this happens again.

For now, maybe this will help, here's what happened: This may be repeating some things, but I'm trying to mention everything that I recall, in case it helps you imagine the scenario.

LabVIEW 7.1 with Real-Time
(1) 8145RT controller card
(2) 7831R FPGA cards (mainly used as IO)
(1) RS232 card (not used at the moment)

I was connecting to the RT system over our network and was developing VI's that are being loaded on the Real-Time controller and that access some IO on the FPGA cards. I have been developing similar vi's and loading them and running them just fine. I was not developing Startup vi's and turning the PXI
chassis on/off or anything like that. I would just run the vi's, they would download, run, rinse and repeat. I will check our IT dept to see if another comp was trying to obtain that IP address, but other than that, no one touched the system that I know of.

Friday, while I was switching Operating Environments from RT to LabVIEW for Windows, I noticed the options for the FPGA emulator and FPGA device were no longer available from the drop down menu on LabVIEW. Came back Tuesday to the same. The box was sitting there, on but frozen and not responding. Ping did not see it. FTP did not see it. MAX did not see it. Rebooting did not work. Turning it off/on did not work. Trying to reconfigure through MAX did not work.

This has happened before and one of our engineer placed a call and the option of clearing the flash memory was presented. So thats what we did this time.

Should this happen again, I will try to get the files you mentioned.

Thanks for your assistance.

- Con
0 Kudos
Message 5 of 8
(4,827 Views)
Con,

As Alejandro recommended on another thread, feel free to send those files if you get the same type of crash again.

Gustavo
0 Kudos
Message 6 of 8
(4,827 Views)
I apologize for starting another thread, it took a while before I got to posting my comments.

Thanks for linking to the other thread Gustavo.

Alejandro
0 Kudos
Message 7 of 8
(4,826 Views)
Con,

It just occurred to me that the FPGA code will still be running even after you reset the system. RIO boards will run the code for as long as they have power, so if the code was misbehaving or overloading the system (constantly generating interrupts or something similar) then you could get into this situation.

Since rebooting the system doesn't really cut power to the FPGA board, you might want to try unplugging the system, trying to turn it on (while umplugged) and then plugging it back in. Taking the RIO board out is probably another temporary solution, if this is the case.

I hope this helps,

Alejandro
0 Kudos
Message 8 of 8
(4,826 Views)