Lookout acting strange (No alarms logged and later 2 General Protection Faults)

[RobMiller2002]

Hi

   I have a lookout system running on Windows XP Pro it has been working fine for over a year. All of the sudden we have had 2 instances when none of the alarms were being logged. There were no error codes from lookout or in the computer event logs. We had to reboot and then it started working again for about a month and then did it again.

  A couple of  weeks later the system appeared to be woking but was not doing anything when any of the inputs changed as it was supposed to. I checked the logs and found it had a General Protection Fault :

Event Source:\tab Application Error\par
Event Category:\tab None\par
Event ID:\tab 1000\par
Date:\tab\tab 8/1/2005\par
Time:\tab\tab 11:59:01 PM\par
User:\tab\tab N/A\par
Computer:\tab AWD-SCADA\par
Description:\par
Faulting application lookout.exe, version 5.0.0.7, faulting module gdi32.dll, version 5.1.2600.1106, fault address 0x0000e300.\par
\par
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.\par
Data:\par
0000: 41 70 70 6c 69 63 61 74   Applicat\par
0008: 69 6f 6e 20 46 61 69 6c   ion Fail\par
0010: 75 72 65 20 20 6c 6f 6f   ure  loo\par
0018: 6b 6f 75 74 2e 65 78 65   kout.exe\par
0020: 20 35 2e 30 2e 30 2e 37    5.0.0.7\par
0028: 20 69 6e 20 67 64 69 33    in gdi3\par
0030: 32 2e 64 6c 6c 20 35 2e   2.dll 5.\par
0038: 31 2e 32 36 30 30 2e 31   1.2600.1\par
0040: 31 30 36 20 61 74 20 6f   106 at o\par
0048: 66 66 73 65 74 20 30 30   ffset 00\par
0050: 30 30 65 33 30 30 0d 0a   00e300..\par
}
 

2nd Error :

Event Type: Information
Event Source: DrWatson
Event Category: None
Event ID: 4097
Date:  8/2/2005
Time:  6:44:16 AM
User:  N/A
Computer: AWD-SCADA
Description:
The application, C:\Program Files\National Instruments\Lookout 5.0\lookout.exe, generated an application error The error occurred on 08/02/2005 @ 06:44:16.484 The exception generated was c0000005 at address 77C7E300 (GDI32!GdiConvertToDevmodeW)

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
00000: 0d 00 0a 00 0d 00 0a 00   ........
00008: 41 00 70 00 70 00 6c 00   A.p.p.l.
00010: 69 00 63 00 61 00 74 00   i.c.a.t.
00018: 69 00 6f 00 6e 00 20 00   i.o.n. .
00020: 65 00 78 00 63 00 65 00   e.x.c.e.
00028: 70 00 74 00 69 00 6f 00   p.t.i.o.
00030: 6e 00 20 00 6f 00 63 00   n. .o.c.
00038: 63 00 75 00 72 00 72 00   c.u.r.r.
00040: 65 00 64 00 3a 00 0d 00   e.d.:...
00048: 0a 00 20 00 20 00 20 00   .. . . .
00050: 20 00 20 00 20 00 20 00    . . . .
00058: 20 00 41 00 70 00 70 00    .A.p.p.
00060: 3a 00 20 00 43 00 3a 00   :. .C.:.
00068: 5c 00 50 00 72 00 6f 00   \.P.r.o.
00070: 67 00 72 00 61 00 6d 00   g.r.a.m.
00078: 20 00 46 00 69 00 6c 00    .F.i.l.
00080: 65 00 73 00 5c 00 4e 00   e.s.\.N.
00088: 61 00 74 00 69 00 6f 00   a.t.i.o.
00090: 6e 00 61 00 6c 00 20 00   n.a.l. .
00098: 49 00 6e 00 73 00 74 00   I.n.s.t.
000a0: 72 00 75 00 6d 00 65 00   r.u.m.e.

I have the rest of this Dr. Watson error dump if you need it to help but the size of the file is over  1MB so I did not attach it here.

I have run many diagnostics on the computer and have found no errors. It runs on a very good UPS system 24/7 so power should not be an issue. The computer does not have any network or internet access so nothing new has been loaded onto the system. Lookout is the only application loaded on the system other than 1 Adaptech application that monitors the mirrored raid array.

Any help in resolving these problems would be greatly appreciated.

Thank You

Rob

[erblock]

The first thing I would do is backup all process files, lookout.sec, lookout.ini and then perform a repair of lookout.

Next perform a chkdsk /R on all disks.  If errors are discovered....likely the source.

Depending on types of disk errors (and which disks the errors are found on), you need to consider replacing hard drives or raid controller or both.  At a minimum, If serious disk errors are encountered or if the problem repeats itself after errors are fixed... as long as new disk errors don't keep coming up...reload the OS from scratch and reinstall everything.Then repatch Lookout with fixes from ftp://ftp.ni.com/support/lookout/fixes/5.1.  If disk errors are present and they persist....time for some new disks, raid controller, or both.

I have found even Microsofts detection of needed files in XP Pro, will not always tell you when files are missing (corrupted on disk).  Furthermore, even during a fresh install of XP Pro, if the disk is not in perfect shape (chkdsk made all necessary repairs) before you begin the install, XP Pro can miss the fact that things are messed and you can end up with a bad install!

Good Luck!

Ed

Of course it helps to have a spare machine to load and test this scenario.  A lot less work that way...just copy your lookut stuff (process, .sec and .ini) over and run.

[Khalid]

Rob,

Both errors seem to have occured in GDI32.dll -- this is a Windows' dll responsible for display, graphics, printing, etc.  So you may want to look closely at such operations around the time the problem occurs. 

The Dr. Watson log should give us more info'.  You do not have to attach the entire log -- open the file in Notepad, go to the end of the file and search for the word 'Fault' backwards.  Once you find it, copy and paste that portion of the log, including a few lines above it. 

Regards,

-Khalid

[RobMiller2002]

Hi

    I am fairly certain that the hard drives are ok. I have nothing in the logs for the raid controller indicating any problems. I have surface scanned them for any problems and none were found. I have also looked at the windows logs and found no drive related problems. The system has been running ok for about a week and a half now but I am a little leary of it. I had thought about a total reinstall of XP and lookout but that is very time consuming and I would have the system down for approx 6 hours. If I have to go this route I will but I would rather not due to the downtime. This system runs a city fresh water system and due to the heat we have been having recently they are having trouble keeping up with the demand. There is a backup system but it does not work very well in a high demand situation.

   The log I have said it was from Dr. Watson in the windows error event I have attached it in a zip file. I could not find any fault text in it so maybe theis is not the log you were asking about. If there is another place Dr. Watson logs get saved please let me know and I will find the information you requested.

  Thank you for any help you can give me.

Rob Miller

[Khalid]

For some reason the text on the left in the log file is not what I usually see -- a dot seems to have been inserted after each character.  I was thus able to locate the "F.A.U.L.T" in the log, but there isn't much info' to figure out which function in Lookout is causing this crash in the GDI32.dll, unfortunately. 

Do you use lot of animations on your panels?

-Khalid

[RobMiller2002]

Hi

I do have quite a few animations in use. There are approx 9 on the main panel and 5 or so each on 3 other panels. Is this a problem?

Thanks again for your help

Rob

[Richard P.]

I understand this might be difficult since the system is in use, but it soudns like reducing the number of animations would be worth a try. Do they animate more frequently under periods of heavy water use?

Also, rather than starting an uninstallation from Windows XP upwards, it may be possible to do a very clean uninstall which would save you some time. There are 3rd-party uninstaller programs out there and I could point you in the direction of one over email. This is probably not necessary just yet as Khalid has done a great job identifying this Windows dll and is on to something there.

[Mike@DTSI]

This might also lead us to some answers

Resolution? 
Color Depth?
Video Card/Drivers?
Describe said animations(spinners?)
Any extra bitmaps on the screen?
Hypertrends? How many, big, etc?

Our systems can have as many as 400 animate items per screen (spinners, multistates, animators, etc.)  .
Also trends with a lot of traces, that take a good part of the screen can cause "video" issues.

Make sure your drivers are all up to date as well as Windows patches.

Good luck
Mike

[Richard P.]

Also, how fast is your processor, how much video RAM and cpu RAM do you have?