TestStand executable test sequence deployment

[j_dodek]

Hi Jigg,

what a great clarification! wihch I absolutely agree, especially with the costs!

Thanks for sharing and spending your time!

Juergen

Maybe you have some time to spent your thoughts on my upper discussion.....

[Kochise]

---

~jiggawax~ a écrit :

Hey Kochise,

I have heard some of the same gripes regarding TestStand.  Let me try and give my 2 cents for your questions:

Labview is "essentially a programming" language yet you can compile it into one executable that doesn't need a run-time engine. Not True.  Just like .NET requires the CLI, LabVIEW requires the LV-RTE that is somewhat large.

---

Yeah, I forgot about that LV-RTE. I'm from the embedded world when we 'often' link these things statically. And yeap, it's pretty large.

---

~jiggawax~ a écrit :
Why couldn't Teststand compile a sequence (and needed dependencies) into one executable that doesn't need a run-time engine ? TestStand is flexible enough to call many different languages as well as DLLs.  I don't think it would be possible to cram assemblies, VIs, dlls, etc.. into one exe.  In other words, cramming different, diverse, languages into the same executable is impossible. 

---

Well, I used to make Windows application installations, first using InstallShield, then InnoSetup. So I now it is possible to "cram" files into a nicely crafted installer or an auto-extracting application that will remove the temporary files on exit. But OK, that's just cosmetic things. It's just easier to transport and/or send by mail to install on a production machine.

---

~jiggawax~ a écrit :
Plus the whole run-time matching the VI's version is a management mess left to the developpers while it could have been solved on NI side. Usually a quick mass-compile can fix this.  In the newer versions of TestStand you can make VIs run in their current version.  You do have to have all of the different RTEs for each version however.

---

Why the latest LV-RTE revision cannot open older VI versioned files ? That's the main point. Why changing/locking down everything at each iteration ?

---

~jiggawax~ a écrit :
I just need a deployement tool to select the main sequence file, calculate the depencies, compile the VIs and pack everything into one executable. This exists.  You can point the deployment utility to your folder containing the sequence file.  The TS Deployment Utilty will then collect all the dependencies for you.  Refer to my comments above regarding the exe.

---

Yeah, trying this currently, hence my 728 MB comment below for 4 MB test sequence/VI files. But under my previous experience with Windows installations :

https://community.flexerasoftware.com/showthread.php?131159-Minor-Upgrade-not-replacing-files

---

~jiggawax~ a écrit :
Right now I have a 4 MB worth of sequence/VI files bundled with 728 MB worth of Teststand/Labview run-times/editors and stuffs. 4MB isn't that large.  Disk space is cheap today.  One thing we do to mitigate the larger disk space with the run times and UIs is to create a separate installer.  We call it a baseline. It contains all the engines, UIs, native codes, etc...  Then our automations are much smaller and can just be instlaled on the top of the baseline.  Usually they only install the baseline once and then when we fix the automation it will be small.  There is a white paper out there on this if you google around.

---

I found http://www.ni.com/white-paper/9923/en/ and http://zone.ni.com/reference/en-XX/help/370052M-01/tsdeploysystem/infotopics/identifyingcomponents/ link, yet it looks little automated. I'm looking for a way to install the needed sequence files with the suitable DLLs (scan VIs) with a link to the targeted MainSequence. Just like you install an EXE that will run the 'void main(void)' method.

---

~jiggawax~ a écrit :
Is that really the way it should work ? That doesn't look quite professionnal, considering the obligatory intrusively expensive licensing scheme. $500 bones for a deployment license is really cheap in the manufacturing world.  Considering that the deployment license is a one time purchase that is upgradable for life is nice as well.  I did a study on the cost of creating your own test executive in LabVIEW vs all of the costs of TestStand.  The difference was astronomical.  I think it was around $800k vs $30k or something like this.  This includes development time at $100/hr.  Granted you need to take into consideration the project size and scope.  Sometimes a large framework is overkill as Dennis pointed out.

---

Again I understand that. But again, regarding my previous experience with InstallShield, the product was supposed to bring simplicity yet it proved to be bug ridden, step learning curve and a money pit. You had to pay 1700$ for each upgrade (that doesn't always solve bugs) and 800$ for each language pack that was basically a unlock key just to add new columns in the resource string editor. When you upgrade, you have to buy again the suitable language pack (buy again the key) without bargain.

I spent a lots of time to get the product running and had patched it from start to end. Read the topic linked above. I assure you, I pushed this product to its limits. I tried InnoSetup and within a week I got the same result I had obtained in a year of InstallShield tweaking. I don't want to spend a year tweaking Teststand only to find out it's an half backed expensive product. I'm already quite unsatisfied with Labview that pretend developers don't need a tabbed interface or a zoom functionality, they just have to do some other way. Come on...

---

~jiggawax~ a écrit :
Where is the buy/install-once, compile/deploy-many option ? It really looks like a non-ergonomic non-developer friendly software.  Not even sure how to respond here.  Any language is non-developer friendly if you are ignorant to how it works and how to use it effectively.  Sometimes I think LabVIEW developers that are making the transition to TestStand get frustrated because of the learning curve.  LabVIEW has a low barrier to enter to do mediocre things.  To do great things in LabVIEW you need to really learn it.  With TestStand the low barrier to enter only allows you to do simple things.  To do mediocre and advanced things in TestStand requires that you really learn it.  So basically the learning curve with TestStand is a bit more steep once you go beyond the basics.

---

Why not providing a progressive interface too, not throwing at the user's face all these panes, tabs, sub menus, whatever. Let's start softly an "unlock achievements" when more functionalities are required ?

---

~jiggawax~ a écrit :
Like I said.  I've heard your same frustrations.  Just keep in mind that TestStand is a scripting language that can call about anything out there: ActiveX, Assemblies, DLLs, Raw Code, Java, Python, Etc...  In order to make it so flexible there is a cost in making a nice little neat package.  One thing that might help is knowing that if you are only using LabVIEW there is an option to put all your LabVIEW code in the packed project library.  So then you just have your Sequence file and packed project lib.  There are documents out there on how to do this in TestStand.

---

You speak about the $100/h price for a dev. Mine is $80/day. But beside creating a software "from scratch" that just work and testing a software, learning it, trying to get things done nicely, etc, don't you think that also must be taken into consideration as well in the final cost ? I mean, imagine the cost spent for more than a month trying/testing/tweaking things to make Teststand and Labview communicate containers/clusters elegantly :

http://forums.ni.com/t5/NI-TestStand/Enqueue-Teststand-cluster-to-Labview-return-a-cluster-by/td-p/3...

These kind of frustrating things are putting me on my nerves. But perhaps I already reached Teststand/Labview technical limits.

---

~jiggawax~ a écrit :
Anyhow, don't give up yet.  I've seen companies become really successful with TestStand when it is the right tool to use.  Sometimes it isn't always the right tool.  I know that NI is always willing to sit down and work with customers to help them become successful.

Hope this helps some,

---

I never give up, but that don't gives me hope.

David Koch

[Dennis_Knutson]

Per your question about a runtime for each version - the runtime engine does not include the compiler. Each older VI would, of course, need to be recompiled before it can be run. A deployed LabVIEW exe also doesn't a block diagram so even if the runtime did include the compiler, there is no source code to compile.

[Kochise]

What I do not understand is that the runtime engine is -to me- like a language interpreter, so I see no reason why it couldn't open a non compiled source file and interpret it.

It is not compiled in x86 machine code, hence it should be able to interpret files from older version, just like QWBasic 5 was able to open and run QWBasic 2 files and interpret them without the need to compile them.

Listening, well, reading to you, it's like you say you need GCC 2 to compile CPP files produced with GCC 2, as GCC 4 cannot and is only suitable to interpret CPP files produced with GCC 4.

That sounds so weird I hope I missed something in the process.

David Koch

[~jiggawax~]

A couple thoughts:

An installer and executable are NOT the same thing.  It is true that an installer can include any type of file.  It is not true that an exe can.  Even when LabVIEW compiles exes it puts 3rd party dependencies (i.e. dlls) into a sub folder and not into the exe.  I think you keep jumping back and forth between the 2 in your verbage.

I do like the idea of having a simpler sequence editor for beginners and then turning more on as devs become familiar.  It is a lot to take in for newbs and underskilled engineers right away.  As you are experiencing.

Like Dennis pointed out in his first post in the thread- It is important to choose the right tool.  In my line of work TestStand has saved us millions of dollars.  When I first came to this company we had siloed engineers all over the place re-inventing the wheel.  We created a centralized software team with about 35 engineers.  All of them obtained, at the very minimum, a CLAD and CTD.  Once we got our framework in to place and their skillset increased we whittled that number down to 14.  When I first got here engineers were spending about 6 months to a year to create an automated test.  Now it takes our guys about 3 weeks.  Let me clear, these tests are not simple.  We are talking multiple Spectrum Analyzers, Signal Generators, Power meters, etc... taking thousands of measurements.  I guarantee that we wouldn't have been able to do this without TestStand.  In our case it was the right tool.  Especially because we are in a high mix low volume manufacturing facility.  The plug-n-play capabilities are nice for this.

Generally I find that if it becomes hard to do in TestStand then I am doing it the wrong way.  🙂  Like I always say- There are a million ways to arrive at the same goal in TestStand.  It is so flexible and open it I love it.  It is so flexible and open I hate it.

Cheers,

[Kochise]

---

~jiggawax~ a écrit :

It is so flexible and open it I love it.  It is so flexible and open I hate it.

---

I thinks I'm gonna borrow it from you 🙂

I never said Teststand is not the right tool, I bet if large companies use it for years there must be a reasons.

But as a coder and user, I think there are easier ways to get the same job done.

BTW I know the difference between installer and EXE/DLL that can be bundled in an installer.

Just that I also made auto-unpacker EXE that contains an application (EXE/DLL) that unpack, runs and delete the unpacked EXE/DLL after exit.

David Koch

[Dennis_Knutson]

A VI is ALWAYS compiled. That is what happens when you click the run button. There is no interpreter!

[Dennis_Knutson]

Let me add a few more comments about how I TestStand at a couple of locations. At one place, we had about a dozen developers and a hundred or so test systems with a couple of hundred of test sequences for nearly that many products. At another, a couple of dozen products but about a thousand test stations. In both places, the developers were only trained in TestStand. I was the only LabVIEW developer. I created custom test steps that were built into a dll. The custom test steps were used exactly like the built-in step types, especially the IVI steps. This made training much simpler as the developer did not need to learn LabVIEW. The test stations only had the operator interface and the engine (do you know the OI can also be an editor). Pulling the steps out of source code control kept all developers up-to-date. The validated step types and sequences were kept on a release server that all test stations linked to. Updating a sequence or step type meant updating a single location and each sequence or step type was a relatively small file so updates were quick.

I'm not sure if there any benefits to TestStand when you are creating a single, specific test sequence. You have two large learning curves that have to be overcome.

p.s. My record for a new sequence from start to finish was 4 hours! A