NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

Feedback on TestStand Performance

Hey Justin,

 

My two cents, here we use TS4.2.1+LV2009RTE+winxp

1. TS launch time and module load/unload time, especially the latter

I have a small application, 4 seq files, all step modules in one llb(~4.5MB), other vis in SupportVIs.llb(~13MB). It takes >15 sec to pop up the preuut window for the first time. Plus the TS launch time(~10sec), it makes the application looks really slow

 

2. Datalogging is good, parallel executions are fine.

 

3. Deployment is the major issue. I am more concerned about inaccurate error messages and undetectable vi prototype mismatches. Say I have an application about 100MB vis, the deployment takes about 1hour. the time is long but acceptable, normally we can let it run after work or write a script to automate it. But if the deployment failed, the error in DeploymentUtility.log often doesn't point to the actual reason or doesn't make sense at all, especially for large, complex system needs time-consuming deployment. Also if I change the vi input/output connectors and forget to update one of vi instances in seq, the TS deployment can't detect it. So after 1hour deployment, I am so exciting to install and try the new software on test station, then get a prototype mismatch errorSmiley Indifferent

 

Report generation time is another issue, HTML works fine for small or intermediate size report, but it can't handle large report, TS will hang there forever and I have to kill the thread. Right now I switch to customized pdf report and don't have this problem any longer, but somebody else may still have the same issue I experienced

 

Chenny Zhenyu Wang
Certified LabVIEW Architect
Certified TestStand Architect
0 Kudos
Message 11 of 22
(1,249 Views)

Hi Jervin,

 

I know this is an old thread, but since I am dealing with a performance issue, I may as well chime in...  I will provide feedback in the same order as you have asked questions:

 

1.  For the present project, the major issue is the extremely long time it takes for TestSTand to Load modules (>1 min).  Obviously loading modules is very important.  All the modules are LabVIEW.  There is a LabVIEW GUI using the Invoke Node "SequenceFile: LoadModules".  The timestamp shows that this call takes >1 min.  It prevents parallel activities from running in LabVIEW.  As a matter of fact, it gives the operators the impression that the application hangs..

 

2.  From a performance point of view, I can't say that TEstSTand exceeds any needs.  I've always had the impression that TestStand was rather slow.

 

3.  Obviously loading modules...

 

4.  I have not done any official benchmarks for TestStand nor compared improvements / changes with the various releases over the years..  Plus every customer has different requirements, so it is difficult to compare. 

 

Sorry for the extremely late reply (2 years).  I was searching for ways to improve / reduce loading time and saw this thread.

______________________________________________________________________
0 Kudos
Message 12 of 22
(1,165 Views)

@dug9000 wrote:

For module loading speed issues, please provide more details such as the following:

 

1) What adapter(s) do you primarily use?

2) If LabVIEW is one of you primary adapters, are your VIs typically already mass compiled (i.e. up to date with the version of LabVIEW you are using)?

3) Approximately how many code modules are you loading and how much time is it taking?

 

The above information will help us better diagnose the issue and target the area with the greatest problem.

Thanks for any details you are able to provide us.

-Doug


LabVIEW modules.

Yes... they're all compiled. 

There are many calls (100s), but most are calling the same VIs, so probably calling <50 VIs.  I suspect that this is what takes >1 min.  That's on the development machine.  On the target machine, it is much longer.  ~ 3 mins.  The LabVIEW code is compiled into an executable for the target machine.

 

 

______________________________________________________________________
0 Kudos
Message 13 of 22
(1,162 Views)

1) Are you running the Vis in the labview runtime, or the labview development environment?

2) What verison of LabVIEW are you using? What version of TestStand?

3) how many vis in your system? Are any particularly large?

4) Does the exe you compile for the target machine take 3 minutes to load, or does it only take 3 minutes if you don't use an executeable?

 

-Doug

0 Kudos
Message 14 of 22
(1,142 Views)

1)  On the target machines which is where the client runs the tests, it runs the executable.  I think the machine is a cRIO dual core..  I have the specs somewhere and can dig it up.  The development machine probably has more "horsepower", hence able to load the modules faster.  I don't remember if there was significant difference to load the modules between development and runtime.  I will do some benchmark tests to confirm.

 

2)  LabVIEW 2010SP1 and TestStand 4.1.  They do have TS2012, but the decision was to stay with 4.1.

 

3)  There are 108 VIs.  None are particularly large.  I don't know how many calls to LabVIEW modules in the test sequence..  Anywhere from 500 to 1000.  However, there are probably around 20 different VIs being called by TestStand.

 

4)  Yes.  On the target machines, only the executables are used.

 

______________________________________________________________________
0 Kudos
Message 15 of 22
(1,138 Views)

Correction:

 

The target machine is a PXI-8108 Core 2 Duo 2.53GHz Ctrlr with WIn XP.

 

NI 780446-01

______________________________________________________________________
0 Kudos
Message 16 of 22
(1,128 Views)

I'm pretty sure there have been some significant improvements in VI load time in the more recent versions of TestStand. If you are interested and able to do so, you might want to try running your system with TestStand 2012 or newer and see if it is much faster now.

 

-Doug

0 Kudos
Message 17 of 22
(1,115 Views)

Thanks,

 

I will make thatrecommendation and provide feedback.

 

🙂

______________________________________________________________________
0 Kudos
Message 18 of 22
(1,107 Views)

I recommend verifying that you are seeing the performance improvement before spending a lot of time trying to migrate everything if this is the only reason for doing so. Again, I'm pretty sure there have been some improvements which should help with this, but you should verify with your particular use case that you see such an improvement.

 

-Doug

0 Kudos
Message 19 of 22
(1,077 Views)

Chenny, 

 

In recent versions of TestStand (2011 and later) we have made a very big effort to improve the quality of the error messages displayed by the Deployment Utility based on feedback from customers like you who have reported errors. While we have not fixed every issue with deployments, we are definitely making an effor to improve the deployment experience. 

 

We appreciate your feedback on the errors you found and I have written a ticket (ticket number #418983 if you want to follow up) to fix the probelm with mismatched prototypes during deployment. 

 

Please note that in TestStand 2010 we also introduced the sequence analyzer which is a static analysis tool that will help you find problems like mismatched prototypes or invalid paths. We recommend that you run the analyzer before you create a deployment.

 

Thanks,

Francisco

0 Kudos
Message 20 of 22
(981 Views)