NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

Deployment methods for distributing test stations - discussion

Hi,

 

I'm wondering what's the most proper method of code deployment in my case and I hope we can have a good discussion about it here.

 

Quick introduction:

 

I have one development test station + few production ones. They're identical in terms of the hardware.

Regarding the software, there are:

  • TestStand as the test executive with my custom process model, plugins and step types
  • GUI (LabVIEW) using TestStand Engine
  • Test procedures calling :
    • Hardware drivers API (LabVIEW)
    • Custom test-specific Vis

 

Most of the work is done on the development station, creating test procedures, hardware drivers, testing them, etc.

There are production stations in factories where we run actual production tests.

 

Now, the point of the discussion is: how should the deployment process and project structure look like!

 

My [R]equirements:

  • [R01] I have only one set of development licenses (LabVIEW + TestStand), so production stations should normally run with TestStand Base Deployment license and LabVIEW adapter set to RTE
  • [R02] The deployment process should be easy and traceable
  • [R03] There should be the possibility to activate the development license on the production station to debug/modify test procedure or hardware drivers and commit changes as a bugfix to the git production branch, so other stations can be updated
  • [R04] There should not be much overhead to the project that makes it too complex. Let's KISS!

 

There are a few possibilities of how you can deploy software. These are two setups I've tried:

 

  solution description / preparation pros cons
1 TestStand Deployment Utility
  • compile GUI to EXE
  • change LV adapter in TS to RTE
  • use built-in deployment utility
  • probably recommended by NI
  • all dependencies included (in theory)
  • fully automated process
  • the deployed app is not linked to the repository any more [!R03]
  • compilation takes a lot of time in case of my project (~30min)
  • the deployed app has to be copied manually to target [!R02]
  • actually, there are sometimes problems with dependencies
  • you can specify a Workspace to include, but it does not handle auto-populating directories, so adding each sequence file has to be updated manually [!R04]
  • sequence files in the deployed app are a bit different than the original ones (e.g. paths are updated). You can't modify the sequence and then copy it to the source project. [!R04][!R03]
  • any change introduces a lot of potential issues. To modify, one needs to: modify the VI, test it, once it works, reproduce on the development machine, test in there, which is not always possible due to external factors, once it works, compile using the deployment utility, update the deployed app manually to all machines...

2

Clone git repository on each production station

  • compile GUI to EXE.
  • change LV adapter in TS to RTE.
  • don't separate compiled code from the block diagram for hardware drivers and test-specific VIs.
  • use PPL for modules called in the process model
  • make sure that password is required each time somebody wants to do a git action
  • the app is linked to the repository, can pull/push/commit easily [R02][R03]
  • code update is very quick and easy [R02]
  • actually, there is no much preparation work
  • you have to be careful what you pull/push/commit
  • full source code is present at the production site
  • PPL required for some modules
  • probably it's necessary to include compiled code in Vis, at least for hardware drivers and test-specific Vis
  • sequence loads slower, because it takes LabVIEW more time to find and load dependencies from vi.lib, instr.lib and few other place

 

After experiments with various configurations, now I'm in favor of option 2). It's not tested 100% yet, but I'm open for feedback and I'm curious how you do it in your projects.

 

Mateusz Owczarek

Certified LabVIEW Architect
Certified TestStand Architect
0 Kudos
Message 1 of 9
(1,494 Views)

 

Hi Mateuscz_owczarek, 

 

good summary of the options. I'm in a similar position: 1 development station, multiple deployment stations. 

 

Option 1: TestStand Deployment Utility

I gave this a cursory look but found in my situation where I have a decent mix of products and changes happening somewhat frequently that having a monolithic deployment where the entire thing has to be re-compiled not a satisfactory answer. 

 

Advantage: It can simplify the path from development to deployment BUT no escaping the black box. Good luck if you have multiple small fixes along the way. 

 Major CON: Have to recomplile the entire package to incorporate a small change. 

          2nd CON: The compiling process is error prone and hard to debug. 

          3rd CON: As you are re-compiling the ENTIRE solution, in my opinion, you should do a regression test of the ENTIRE solution to confidently say nothing broken. 

   

 

Option 2: "Modular deployment" 

  All code is modularized and bundled into PPLs. 

  Sequences call PPLs. 

  Simple copy /verify for deployment. I use BeyondCompare2 for "deployment". 

  

  Major Advantage: Can do update single Modules at will without having to re-compile & re-deploy the entire package.

 

  Disadvantage: PPL's are not great. Many potential pitfalls to workaround. TestStand2019 has painful PPL bugs, see my post history. 

 

  This is the path I've chosen as I value the ability to do small fixes over whatever advantages the Teststand Deployment Utility offers. It is by no means perfect, 

I might try the TDU again someday. 

 

Sr Test Engineer at American Innovations - LabVIEW CLA - Kudo's are appreciated!!
0 Kudos
Message 2 of 9
(1,444 Views)

Agree with you.

 

PPL does not work well with classes, unfortunately. I have an OOP-based drivers hierarchy. Building one big PPL with all drivers is not a good option. It takes a lot of time to build and you have to do that if you change anything.

On the other side, building separate PPL per driver is problematic in this case. Let's assume I have class: Driver, Power Supply (inherits from Driver), Keysight Power Supply (inherits from Power Supply), NI Power Supply (inherits from Power Supply). I cannot easily build PPL for a child class, because it won't include parent methods, which are of course necessary. To do that, I would need to add all parent classes manually to the PPL, which brings a lot of complexity.

 

Also, how do you deal with hiding the source code from the customer/production workers?

Mateusz Owczarek

Certified LabVIEW Architect
Certified TestStand Architect
0 Kudos
Message 3 of 9
(1,435 Views)

You may also want to look at NI Package Builder.  You could build each component as a package and then upload all of them to a feed hosted on a network share or SystemLink Server.  SystemLink would also give you the option of pushing the packages to the testers.

 

It is kind of a hybrid between your two options since it gives you the modularity to build and deploy individual components as packages, but you don't have to worry about giving access to your source code to testers.

 

It also provides some additional isolation so that you can continuously make changes to your source and then choose when to publish packages to the package repository for consumption.

0 Kudos
Message 4 of 9
(1,422 Views)

Thank for suggestion, it seems to be a sensible solution for pure LabVIEW project, but with TestStand things get more complex 😞 

To generate the package, one need to "pack" the whole TestStand environment, while preserving dependencies. I'm afraid it's not that easy. And, as far as I understand, it wouldn't be too easy to let me debug and modify the code on the production if necessary.

Mateusz Owczarek

Certified LabVIEW Architect
Certified TestStand Architect
0 Kudos
Message 5 of 9
(1,414 Views)

We have the following setup:

- Custom app written in LV that calls TestStand API and executes test sequences (not sure what type of TestStand license this requires, hope only basic one ?? ) TS process model and some custom steps are managed by this app.

- project repo that has TestStand sequences and LabVIEW code

- powershell deployement scripts that create 'package' (simple zip file) that runs unit tests, builds all into PPLs, password protects sequence files so they can not be modified, generates pdfs etc.

-If something breaks we try to find issues based on logs and data from technicians. If all that fails we must go to the station, run package from teststand or checkout code and run it from teststand - here full TS license for sure is required

 

We looked at NI package manager but it installs package globally, not on project level - it makes it useless for our setup.

My coworkers tried to use TS deployement utility but it never worked as expected (i do not know more details)

 

We have NI Volume License Server so license should be always avaliable. In long run we plan to get rid of LV entirely and replace it with .net code to cut unnecesary licensing costs.

0 Kudos
Message 6 of 9
(1,391 Views)

Do I understand correctly that your setup also creates a deployed package that is in fact totally disconnected from the source?

What do you do when you need to change something on the production? What if the introduced bugfix is applicable to the source application - do you just copy changed VIs or sequences?

Do you have a way to compare your production package to the source application? 

0 Kudos
Message 7 of 9
(1,366 Views)

Yes it is disconnected and contains binaries, randomly password protected sequences, some documentation etc.

 

You can not and should not modify this package and that is huge benefit compared to raw code checkout in our  we had sscenario.When we had source checked out people modified production code without controll, left uncommited changes, you were newer sure what is running on production.

 

I try to create package that matches tag in our SVN repo.

 

When there is bug or an issue after it is fixed we create new package and distribute it on production. In severe situations when production stops someone has to go to test station check out code, fix issue, test it and create new package after. I do not know how to approach integration tests to actually run sequences with simulated hardware in automated way.

 

Running tests, build scripts and distribution is unfortunately manualy triggered but still faster than clicking build for all lv projects. We are in process of splitting 2 companies and there is chance to do drastic changes, hope in time SVN will be replaced with bitbucket, we add CI and some distribution mechanism.

 

I must warn You that there is huge pain when trying to automate LV build process. It happens that build throws unknown error and after retry works fine, LV hangs, not to mention that build destination in LV is absolute and had to be fine tuned when checking out code... 

0 Kudos
Message 8 of 9
(1,353 Views)

@pawhan11, thanks for answer, I hope you are doing well 🙂 

I was also considering a similar approach, but what I miss there is that you actually cannot perform a debug&fix actions on production units easily and that's a use-case that I predict in our case. We have production stations in a few factories in different countries and sometimes it happens that there is a specific event that occurs at one station and cannot be reproduced on the development station to work on that. It has to be debugged and fixed on the production one and then all others should be updated as well. Of course, there is always a possibility to manually do a lot of copy&compare&paste&adjust actions, but that's not something I dream of.

Mateusz Owczarek

Certified LabVIEW Architect
Certified TestStand Architect
0 Kudos
Message 9 of 9
(1,335 Views)