NI Home > Community > NI Discussion Forums

Lookout

Showing results for 
Search instead for 
Do you mean 
Reply
Active Participant
Mike@Forshock
Posts: 325
0 Kudos

Lookout 6.2, Citadel Crash

[ Edited ]

Ok, beating a dead horse... I think we are definitely getting to the end of our NI lifespan :smileysad:

 

Background

System installed with Lookout 6.1 in 2008, ran well for about 1.5 years.  Random crashes, database corruption, etc. Initially believed to have been the machine (Software RAID 5 sucks).  New machine built in Oct. 2010; no RAID, Intel Core2 machine, XP SP3.  Upgraded to Lookout 6.2.  Could not import the 6.1 database, performance slowed to a crawl, believed to be from a corrupt database.  Created a new database.  

 

Today

Machine is experiencing system failures on Citadel and Lookout. We have nothing but AlarmDialer 4 (custom notification software), RealVNC Enterprise, Lookout 6.2, HP Printer and standard system related tools (TrueImage, AVG, Adobe Reader (Latest version, for manuals)) installed on this machine.  All updates installed.  

 

Latest Windows Mini-dumps: http://support.forshock.com/files/MRCWD/Minidump.zip

Latest (related) error messages:


Event Type: Error

Event Source: SQLBrowser
Event Category: None
Event ID: 1
Date: 12/21/2010
Time: 5:27:46 PM
User: N/A
Computer: SCADA
Description:
The SQL configuration for SQL is inaccessible or invalid.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 4d 00 53 00 53 00 51 00   M.S.S.Q.
0008: 4c 00 53 00 65 00 72 00   L.S.e.r.
0010: 76 00 65 00 72 00 5c 00   v.e.r.\.
0018: 43 00 75 00 72 00 72 00   C.u.r.r.
0020: 65 00 6e 00 74 00 56 00   e.n.t.V.
0028: 65 00 72 00 73 00 69 00   e.r.s.i.
0030: 6f 00 6e 00 00 00         o.n...  
Event Type: ErrorEvent Source: SQLBrowserEvent Category: NoneEvent ID: 1Date: 12/21/2010Time: 5:27:46 PMUser: N/AComputer: SCADADescription:The SQL configuration for SQL is inaccessible or invalid.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.Data:0000: 4d 00 53 00 53 00 51 00   M.S.S.Q.0008: 4c 00 53 00 65 00 72 00   L.S.e.r.0010: 76 00 65 00 72 00 5c 00   v.e.r.\.0018: 43 00 75 00 72 00 72 00   C.u.r.r.0020: 65 00 6e 00 74 00 56 00   e.n.t.V.0028: 65 00 72 00 73 00 69 00   e.r.s.i.0030: 6f 00 6e 00 00 00         o.n...  
Event Type: Information
Event Source: MSSQL$CITADEL
Event Category: (2)
Event ID: 26037
Date: 12/21/2010
Time: 10:01:26 AM
User: N/A
Computer: SCADA
Description:
The SQL Network Interface library could not register the Service Principal Name (SPN) for the SQL Server service. Error: 0x54b, state: 3. Failure to register an SPN may cause integrated authentication to fall back to NTLM instead of Kerberos. This is an informational message. Further action is only required if Kerberos authentication is required by authentication policies.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: b5 65 00 00 0a 00 00 00   µe......
0008: 0e 00 00 00 53 00 43 00   ....S.C.
0010: 41 00 44 00 41 00 5c 00   A.D.A.\.
0018: 43 00 49 00 54 00 41 00   C.I.T.A.
0020: 44 00 45 00 4c 00 00 00   D.E.L...
0028: 00 00 00 00               ....    

We get lots of these entries:
Event Type: Information
Event Source: MSSQL$CITADEL
Event Category: (2)
Event ID: 17401
Date: 12/21/2010
Time: 1:44:58 PM
User: N/A
Computer: SCADA
Description:
Server resumed execution after being idle 10079 seconds: user activity awakened the server. This is an informational message only. No user action is required.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: f9 43 00 00 0a 00 00 00   ùC......
0008: 0e 00 00 00 53 00 43 00   ....S.C.
0010: 41 00 44 00 41 00 5c 00   A.D.A.\.
0018: 43 00 49 00 54 00 41 00   C.I.T.A.
0020: 44 00 45 00 4c 00 00 00   D.E.L...
0028: 00 00 00 00               ....    

Event Type: Error
Event Source: Application Error
Event Category: None
Event ID: 1000
Date: 12/22/2010
Time: 7:29:18 AM
User: N/A
Computer: SCADA
Description:
Faulting application lookout.exe, version 6.2.0.49153, faulting module nicitmid.dll, version 5.6.0.49153, fault address 0x00069ffb.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 41 70 70 6c 69 63 61 74   Applicat
0008: 69 6f 6e 20 46 61 69 6c   ion Fail
0010: 75 72 65 20 20 6c 6f 6f   ure  loo
0018: 6b 6f 75 74 2e 65 78 65   kout.exe
0020: 20 36 2e 32 2e 30 2e 34    6.2.0.4
0028: 39 31 35 33 20 69 6e 20   9153 in 
0030: 6e 69 63 69 74 6d 69 64   nicitmid
0038: 2e 64 6c 6c 20 35 2e 36   .dll 5.6
0040: 2e 30 2e 34 39 31 35 33   .0.49153
0048: 20 61 74 20 6f 66 66 73    at offs
0050: 65 74 20 30 30 30 36 39   et 00069
0058: 66 66 62 0d 0a            ffb..   
Event Type: Error
Event Source: System Error
Event Category: (102)
Event ID: 1003
Date: 12/21/2010
Time: 5:28:15 PM
User: N/A
Computer: SCADA
Description:
Error code 00000024, parameter1 001902fe, parameter2 b9cebbbc, parameter3 b9ceb8b8, parameter4 b965a667.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
Data:
0000: 53 79 73 74 65 6d 20 45   System E
0008: 72 72 6f 72 20 20 45 72   rror  Er
0010: 72 6f 72 20 63 6f 64 65   ror code
0018: 20 30 30 30 30 30 30 32    0000002
0020: 34 20 20 50 61 72 61 6d   4  Param
0028: 65 74 65 72 73 20 30 30   eters 00
0030: 31 39 30 32 66 65 2c 20   1902fe, 
0038: 62 39 63 65 62 62 62 63   b9cebbbc
0040: 2c 20 62 39 63 65 62 38   , b9ceb8
0048: 62 38 2c 20 62 39 36 35   b8, b965
0050: 61 36 36 37               a667    
-----------------------------------------------------------------------
Forshock - Consult.Develop.Solve.
Active Participant
Mike@Forshock
Posts: 325
0 Kudos

Re: Lookout 6.2, Citadel Crash

Just the answer I was looking for! :smileyindifferent:

 

Mike

-----------------------------------------------------------------------
Forshock - Consult.Develop.Solve.
Member
skibum019
Posts: 41
0 Kudos

Re: Lookout 6.2, Citadel Crash

Mike,

You are not alone.  We've run Lookout for close to a decade and have had various problems with it over time.  We frequently have Citadel database crashes which take quite a bit of time to recover from.  We are on 6.1 currently, but have had the issue now for a few years.  I have not been able to identify any one thing that is causing the problem.  we have a second instance of Lookout at a separate plant that runs without incident, so to be honest, I'm at a bit of a loss as to what is causing the problem.  The only thing I can possibly point to in our situation is that the database(s) on the system that crashes are extremely large (up to 20GB) and have been upgrade from 4.x versions of Lookout over time. 

Active Participant
Mike@Forshock
Posts: 325
0 Kudos

Re: Lookout 6.2, Citadel Crash

Thanks for responding.  

 

We have decided to stop actively developing in Lookout.  We are currently transitioning to another SCADA/HMI package, developed by a company not too far from NI headquarters...

 

We have this happening in different impacts on multiple systems.  We have one system logging about 25 traces, very small process.  It exhibits the same problems.  

 

Have a good day,

Mike

-----------------------------------------------------------------------
Forshock - Consult.Develop.Solve.
Active Participant
Mike@Forshock
Posts: 325
0 Kudos

Re: Lookout 6.2, Citadel Crash

[ Edited ]

Year later.  We have a system with approximately 8.05GB in citadel.  System shutdown, restart.  Database is now taking over 1 hour to "rebuild" and is not complete.  Secondary has 7.99GB, same issue.

 

Processes will not function, system is sluggish, basically useless.

 

Long live Citadel... :smileymad:

-----------------------------------------------------------------------
Forshock - Consult.Develop.Solve.
Active Participant
Mike@Forshock
Posts: 325
0 Kudos

Re: Lookout 6.2, Citadel Crash

Forgot to add:  In this time, the processes will not function!

-----------------------------------------------------------------------
Forshock - Consult.Develop.Solve.