Time change killed TestStand?

[mfitzsimons]

I have two test systems that run 24 hours a day and they both died within 1 minute of each other at 12:08 on 4/3/05. Both exited TestStand completely with no error log. Is it possible for TestStand to kill itself on a time change?

Matt

[RayFarmer]

Hi Matt,

What version of software are you running, Teststand, LabView, CVI...?
Is it in the Operator Interface or SeqEditor?

How long had the test sequences been running?
Were the two system started together?

Is there any link between the two system, eg a network?

Are you using Report Logging On the Fly?

Regards
Ray Farmer

[mfitzsimons]

Ray,

What version of software are you running, TestStand, LabView, CVI...?
Is it in the Operator Interface or SeqEditor?
* I am running TS 3.1 using the recompiled operator interface and LV 7.1 on XP Professional SP2. The operator interface is out of the box with no modifications.

How long had the test sequences been running?
* Systems typically run two weeks without any issues but have run a month straight
* Here is a portion of the log from both systems with start and end times:

Date Time Test Number
3/30/2005 5:11:23 PM 1
4/3/2005 12:08:42 AM 26389


Date Time Test Number
3/28/2005 4:30:50 PM 1
4/3/2005 12:09:06 AM 37660

Were the two system started together?
* No

Is there any link between the two system, eg a network?
* They are networked to the same server but not together and share a hub

Are you using Report Logging On the Fly?
* On the fly reporting is disabled. I write my own logs in .csv format

I have had the systems crash (complete exit of TestStand with no log) on infrequent occasions but this time they crashed within 1 minute of each other, very strange. Search for thread, "TS deployment shutdown no log" posted by mFitzsimons for more detail. I have also attached a graph of the system memory used by the system (using PerfMonControl.ocx) and it does not seem to be the issue. I am very aware what a memory leak can do to a system. I checked Windows Event viewer but there are no events at the time of the crash. Frustrated by TestStand exiting but not leaving a log message.

Matt

[RayFarmer]

Matt,

Do you have timestamp enable on the logged results, to see if the systems were starting to run slow. (Thinking out aloud here). Its probably not running out memory because you probably would have seen some sort of windows warning. Its more likely, that a call has been made either in the test code or the OI and its decided not to return.

Do you know what the last logged result is, so as to try identify where in the test cycle the systems may be prior to the crash and hence what bit of hardware may have been commumicated with?

Have you tried running your test sequences using the SeqEditor rather than the OI. (Just trying to limit whether its the OI or the test code.)

(You could run with the default OI which is at LV6.1 but set the LabVIEW Adapter to use the LV RTE7.1.)

Regards
Ray Farmer

[dug9000]

Are you by any chance using the Volume Licensing Manager for your TestStand license? There was an issue with that that could cause teststand to shutdown if it couldn't connect with the licensing manager. It was fixed in 3.1f1. See the first item in the following link:

http://digital.ni.com/softlib.nsf/websearch/5E355B100180BBA186256F6900703A77?opendocument&node=132070_US

Doug

[mfitzsimons]

Do you have timestamp enable on the logged results, to see if the systems were starting to run slow. (Thinking out aloud here). Its probably not running out memory because you probably would have seen some sort of windows warning. Its more likely, that a call has been made either in the test code or the OI and its decided not to return.

Do you know what the last logged result is, so as to try identify where in the test cycle the systems may be prior to the crash and hence what bit of hardware may have been communicated with?

* Test was running consistent all the way to the end (see attached log). The attached file is a .csv file but changed extenson to .txt because of error message, "The file does not have a valid extension for an attachment. jpg,gif,txt,xls,vi,zip,doc,bmp,llb,pdf,png,seq,c,h,cpp,cs,vb,ini,gz,ctl,scr,ico,tar,z,uir are the valid extensions.
" Average test time was 12 seconds. This is a game playing application so absolute time per test doesn't have as much meaning as a manufacturing test. Test system was running fine at time of the shutdown with nothing abnormal.


Have you tried running your test sequences using the SeqEditor rather than the OI. (Just trying to limit whether its the OI or the test code.)

* My development system has never crashed and completely exited. In the same note, it is my development system and typically only run overnight.


(You could run with the default OI which is at LV6.1 but set the LabVIEW Adapter to use the LV RTE7.1.)
* Tried to run the OI out of the box and caused much pain! System was locking up. Solved by NI tech support as needed to recompile operator interface after a week of debug.

Are you by any chance using the Volume Licensing Manager for your TestStand license? There was an issue with that could cause teststand to shutdown if it couldn't connect with the licensing manager. It was fixed in 3.1f1.
* Not using Volume License Manager. I am running 3.1 and not 3.1f1

[Caroline Bright]

Hi Matt,

This is a tough issue to determine the source. Since we have not seen this behavior with other applications, I believe that the problem may be in one of the code modules you are callng. Are there any time dependencies in these modules? Are there any dependencies between the two computers? Also, there may be some sort of problem where your memory filled up in one of the external modules. Because there has not been any other reports of this problem we need to figure out what makes your application unique.

Thanks,
Caroline
National Instruments