Deciding whether TestStand is the right solution for my company

[Kyle97330]

I'm working at a growing company and we recently started a push in our Test group to try and standardize our future development as much as possible to allow as much re-use of code as possible.  I don't want to say exactly what it is we make for confidentiality reasons, but let's just say that 90% of what we make is the same basic type of product, but it comes in literally hundreds of models ranging in size from about the size of a dime to the size of a small washing machine.  We're considering switching to TestStand on all new stations and on all updates of previous stations to assist in rapid development with increased efficiency.

Over the years we've made about 20 or so test stations as new versions of our product came out, each able to fit products of a different size and with different test requirements, but because our main products are all so similar to each other, there's only a pool of about 6 different types of tests we need to run, with some products needing only one of those tests and others needing all 6, and plenty in the 2-5 range.  The size differences among different products also mean we have a large amount of different power supplies, and for various other reasons the measurement devices for the test stations aren't terribly standardized as well.

Step 1 was that we standardized the database we used to store all of the test data.  We now have one database that can store data from all of our test stations in the same way.  A few of the old stations had to have some creative workarounds done (i.e. instead of rewriting their current test program, write something that takes their old output format and converts it to the new database format, then upload it), but this step is 100% complete.

Step 2 was to abstract out the most common pieces of hardware, so we have started basically putting the IVI types in a LVOOP wrapper.   Now instead of being limited to interchangability among IVI devices we can also swap in any of the other assorted devices we've accumulated over the years, many of which don't have IVI drivers and even a few that don't follow SCPI standards, as this is a great use of inheritance and overrides.  We're also implementing a few device types that don't have IVI classes.  This effort is already well underway and we're OK here.

Step 3 is where we're at now.  As we standardized on hardware interfacing, we also figured it would be a good idea to attempt to effectively write each of the 6 or so tests one last time each with very flexible setup parameters as inputs, using the abstracted hardware classes so that it wouldn't matter which hardware we had, so that all the tests would run the same way.  So we started looking at solutions to some form of sequence management, and we came up with a couple of possibilities for homegrown versions but after debating those for a while we started to ask ourselves, "Are we just re-inventing TestStand here?"

We use TestStand on a few stations here already so we had licences for development, but the stations we use it on at the moment fall into the 10% of outliers that we have and are somewhat non-standard.  So none of the 6-ish usual tests were used on them and they're for a very different product type, so we never tried to use them with our standardized database.  They also were all made before I came on board here, so I don't have much experience with TestStand myself, and I've run into some roadblocks when trying to come up with a framework for how to integrate our standard database and our standard instrument LVOOP classes into a TestStand environment that meets our current and future test requirements.

The first roadblock is that our database standardization and TestStand's built-in reporting don't mesh at all.  Right now, all of our test stations use the same "mega-cluster" to store their data in, which is a type-def cluster of all the things our products could ever be tested for on any of our stations.  We pass it to a common database write function that writes everything that has changed from the default values (NaN and blank strings\arrays) and ignores the rest, so we don't fill the database with zeroes for the tests we don't do.  I do want to emphasize how big this cluster got in order to have all of the potential save values we might ever need in it.  It is a cluster of 13 other clusters, the largest of which has 10 items in it, and one of those is a 1-D array of another cluster with 19 elements in it, and one of those elements is another 1-d array with a cluster of 24 units in it.  If all of the controls in the mega-cluster are shown on the front panel at once, I can't fit them on 2 full-size 1080p screens at the same time.  The context help for the main cluster is 5 screen heights tall just for the listing of all of the elements.  

So I really can't see this being integrated with the built-in TestStand reporting system.  Additionally, the "Type" system in TestStand appears to be unhelpful when attempting to pass this cluster between steps, and even worse when trying to pass it between sequences.  When I add the mega-cluster to it, it also creates over 30 other types, one for each of the other clusters contained inside of it.  Within LabVIEW alone this is pretty easy to manage, as we just have a centralized repository of typedefs in source control that gets checked out to a common directory on each computer, as we do minor changes to acommodate new products a few times a year.  However, on TestStand, I can't find a method that's the equivalent for just storing a bunch of .ctl files in the same shared directory like we do with LabVIEW.  I see that they can be stored in certain INI files or in sequence files.  The INI files appear to all be station-specific, and since all of our 20ish test stations have different hardware and capabilities, we can't just synchronize those files.  And if I want to pass the mega-cluster type between sequence files that call sub-sequence files, every one of those files will need to be manually updated to get their types to match... and we'lll be needing at least 200 sequence files when we're done, at a guess.  Unless there's some method I'm missing here, I don't see how to use TestStand to move this data around.  We could use something like just passing it by reference, but then if we do that I don't actually see much of a benefit here of using TestStand instead of just pure LabVIEW as two of the things that TestStand is supposed to handle (reporting steps as pass/fail against limits and reporting and logging the results afterwards).  Plus then we have the additional burder of making sure the module holding the reference in LabVIEW doesn't unload before we're done using it.

Is there some better way of handling massively complex types in TestStand that I am not seeing?

There may be more that I'm not seeing yet, but if there isn't a way to handle these massively complex types in TestStand, all we'd be getting out of it is a slightly easier method of managing sequencing than we have now, which I don't see as being worth the investment in time and licenses it would take.

I'd be grateful for any feedback.  Thanks!

[dpnsw]

Considering you already seem to be leveraging LabVIEW heavily and before jumping into TestStand I would look at the PTP Sequencer. This seems to be a cutdown version of TesStand but inside LabVIEW and is a simple install from the LabVIEW Tools network using VIPM.

If you are happy to look at alternative representations then maybe check out NI's StateChart Module or the Open StateChart by Vogel (available as an install from VIPM).

The advantages of any of the above is they will be alot cheaper than TestStand and the required run time licences if you don't require all the power of TestStand (extensible everything).