NI TestStand Idea Exchange

Post an idea

I would like to ask to add named types support for TestStand array literals. The current behavior is described as follows:

Declares a one-dimensional array of numbers, strings, Boolean values, object references, or containers. If all elements are of the same type, the result array is an array of elements of that type. If all elements are not of the same type, the result array is an array of containers.

I would like it to be more or less like this:

Declares a one-dimensional array of numbers, strings, Boolean values, object references, containers, or named type. If all elements are of the same type, the result array is an array of elements of that type. If all elements are not of the same type, the result array is an array of containers.

The problem with the current implementation can be seen on the screen.

It is not a bug (BUG 1828580 to be more precise), it is a feature.

Currently, there is no unambiguous solution for passing empty Array of X (where X is a different type than 64-bit Floating Point) as a sequence parameter. If we use {} it is assumed to be of type 64-bit Floating Point. To have an empty Array of Strings we can use ambigues Split("",""). I have no idea how to pass e.g. empty Array of Signed 64-bit Integer without creating empty variables.

It would useful to have Empty Array Literals of a particular type e.g.:

{} - default

{}i64 - for Signed 64-bit Integer

{}ui64 - for Unsigned 64-bit Integer

{}s - for String

{}b - for Boolean

{}r - for Object Reference

{}c - for Container

{}TYPE_NAME - for type definition where TYPE_NAME is type definition name; e.g. Path.

The concept of TestStand, a sequence. Sequences of groups. Sequences of Sequences.

Pre Sequence, Post Sequence, Pre Test, Post Test. Run as a process, iterates on product instance. Single Up, Multi Up. Is incredibly flexible.

The paradigm is for test. No reason to constrain it to test processes. The paradigm cannot be adapted for an process that runs sequences, and iterates.

Test Stand can be used as a vehicle to build and manage pipelines for Machine Learning Processing. For both training and inference.

For example - I want to do some ML training with hyperparameter tuning. Queue up the variants and run the sequences, compiling the results.

Better Yet, Andrew NG is pushing data centric AI. Use test stand to queue up data variants, train and eval a model. Auto generate the results.

The BIG Tech platforms all support data pipelines.

TestStand has it built in.

Looking at any step result, there's a TS.StartTime property, but it's referenced against the "the number of seconds since the application initialized the TestStand Engine"

What this means is that each step result is referenced against a somewhat arbitrary and changing epoch that isn't recognizable outside any one instance of the TestStand application, let alone by external applications.

I'd like to propose adding TS.StartTimeUnix, which contains the absolute time the step started.

I've added this in our model customization plugins, but to convert the epoch to UNIX I have to perform this calculation:

Seconds(False)-Seconds(True) + Parameters.Result->TS.StartTime

Why I'm asking for this:

In our step result database table, we've added a column for STEP_START_DATE_TIME which is absolute UTC time. To populate this I have to go through the gymnastics above. I revisited this code today and thought, "What silliness. This should be something natively supported." (Other than that I have no strong opinion on the matter)

View All (7)

(This idea was created as response to this discussion board topic: https://forums.ni.com/t5/NI-TestStand/ExpandPathMacros-FindFile/m-p/3767542#M56811)

The commands "Engine.FindFile", "Engine.FindPath" and "Engine.ExpandPathMacros" only support one macro:

$(Platform)

but there are other generic paths that could be helpful e.g. when trying to call a third-party .dll (that's my main usecase) or write a logfile:

-------

$(commonprogramfiles) - The actual folder name varies depending on Windows language settings and bitness. No absolute paths possible but a lot third-party .dlls are here.

$(temp)

$(localappdata)

$(appdata)

$(user)

$(public)

$(MyData) - The actual folder name varies depending on current user and Windows language settings

It hopefully shouldn't be too hard to access the windows functions which get the actual names via macros.

The concept of TestStand Environments was introduced in TestStand 2016 and allows you to define multiple configurations on a single test station or development PC. It would be nice if there was an option to link a sequence file to a specific environment, similar to how you can set a sequence file to "Require Specific Model" In the Advanced tab of the Sequence File Properties. When a sequence file is loaded, the engine could check to see if the required environment matches the current environment and:

Automatically relaunch the app in the correct environment
Generate an error
Prompt the user for an action
Other??

It would be nice to choose the behavior when a required environment doesn't match.

If anyone has other thoughts or use cases, please share below.

-Trent

View All (3)

The Search Directories.Insert method should only insert the directory if it is not already there.

The Method includes an index argument, if the directory is already there, then it should move the existing directory to the requested index.

While we were working on the shipping examples for DQMH, we discovered that the insert method was creating duplicates every time it was called. We implemented a work around that includes a for loop to check each of the items in the search directories list to see if it is the directory we are trying to insert, if it is, we delete it. Once the for loop ends, then we insert the directory where we want it.

You can see a video of the issue and how we worked around it here: DQMH 3.1 Only inserts the Delacor examples directory into Search Directories once

Let's say I'm building a fairly long compound expression that has some repeated parameters - I would like a way to specify (without creating additional locals) a variable/macro for use just within that expression.

For example, instead of:

Locals.CommandLine = "cmd /c C:\some path\that\is\reused"
Locals.CommandLine += " /path C:\some path\that\is\reused"

I could specify:

#pathmacro "C:\some path\that\is\reused"
Locals.CommandLine = "cmd /c " + pathmacro
Locals.CommandLine += " /path " + pathmacro

The idea being that with this I only need to update the macros in one place in a long expression.

This is a simple example, but hopefully you can see why this would be useful and why I wouldn't want to create lots of locals when the values are required only within the expression.

View All (2)

There should be an Engine Callback that is executed as the very last step in the order of step execution that will execute regardless of any and all settings or step results: the PostStep "No-Matter-What" Engine Callback. I have a requirement to perform certain actions at the end of every client sequence step regardless of step, station, report settings as well as step results (Status, Error). Right now, the callback that gives the best coverage is ProcessModelPostResultListEntry, but this does not fire when client sequence developers set Result Recording Option for a step to False. My requirements call for my actions to execute whether or not the developer of the client sequence desires the results to land in the TestStand report or not. As with other callbacks, if it is blank, the engine can skip it.

The PostStepNoMatterWhat callback would execute regardless of all these, but as an aid to Framework developers, NI should provide a matrix for each of the default Process Models that shows which of the engine callbacks will execute given the following data&colon;

Step.Result.Status {Done, Skipped, Passed, Failed}
Result Recording Option {Enabled, Disabled}
Step.Result.Error.Occurred {True, False}
Run Mode {Normal, Skipped}
Ignore Run-Time Errors {Enabled, Disabled}
On-The-Fly Reporting {Enabled, Disabled}
Error Dialog Selection {Ignore, Run Clean-up, Abort}

TestStand API SearchDirectory objects in TestStand have the following writeable properties:

Disabled
ExcludeFileExtension
ExcludeHiddenSubdirectories
FileExtensionRestrictions
Path
SearchSubdirectories

When you use the SearchDirectories.Insert method to add a new search directory, you can only specify the following for the new search directory:

Disabled

ExcludeFileExtension

FileExtensionRestrictions

Path

SearchSubdirectories

It would be really nice if the SearchDirectories.Insert method also had an additional input parameter for setting the ExcludeHiddenSubdirectories property for the new search directory. This would result in a more consistent interface and fewer API calls when working with new search directories.

Because of the way .NET applications and assemblies are invoked in TestStand they are a child process of TestStand. This means that they share TestStand's resources. For most applications this is not an issue but if the application or library being instrumented by TestStand is resource intensive this creates a significant problem. In the scenario that served as the impetus for this suggestion we saw performance 1/10 that when running the target application outside of TestStand.

To correct this I recommend the .NET adapter architecture be changed or be able to be configured such that instead of directly instantiating target applications a call to create an object with a .NET adapter would create a separate process that consisted of a TestStand WCF client wrapper process that would host the target .NET process and communicate with the parent TestStand instance via WCF.

Here is a simple block diagram of the intended architecture:

Let consider I have an array of containers. The container is defined as a user type, (array of the same elements) and consist, let say, from three string variables A, B and C. And, let say the array size is for, example 4.

Now, let say, the developer needs to access the array of elements A. Let say he/she has to create an array od strings called StrArr So what he has to do is for example like below:

FileGlobals.StrArr={FileGlobals.MyContainerArray[0].A, FileGlobals.MyContainerArray[1].A,FileGlobals.MyContainerArray[2].A, FileGlobals.MyContainerArray[3].A}

It is a lot of writing isn't it?

Wouldn't be worth considering to implement implicit conversion which allow just to write

FileGlobals.StrArr=FileGlobals.MyContainerArray.A

FileGlobals.StrArr=FileGlobals.MyContainerArray[].A

View All (4)

When setting arrays, even multi-dimensional arrays, it is possible to initialize them in single assignment expression. For example, "Locals.Array = {1, 2, 3}" will re-define Locals.Array as a 1D array with elements "1", "2", and "3". This is essentially the same as C-style initialization syntax, which also supports structs.

It would be helpful if containers could be assigned in a similar manner. For example, the illustrated container:

could be assigned completely using "Locals.Container = {True, 1, "foo"}".

Currently, that syntax generates a run-time error, "Expected Container, found Array of Containers".

The only scenario I could think of where assignment gets a little weird is with Object types, but in that case you'll have to be assigning Nothing, the return of a function call, or an existing object from another property - there's no way to define a literal value to assign there, but that's already the inherent nature of Object types.

My use case is often container initialization. There are several kludges around this - keeping an empty copy of the container and assigning it to the working copy to clear the working copy, individually listing out each parameter, and a few others. Another case is when it's useful to assign a constant to module parameter - it's debatable that may be bad form, but would still dramatically improves the ease of skimming parameters if it were implemented. It would be a slight bonus to Sequence adapter in particular, which cannot expand containers in the parameter list, as other adapter types can (go kudos Allow Sequence Adapter to expand containers in the module tab to fix that!).

Since our projects involve several developers working in tandem, we manage all project relevant data under SCM (Source Code Management). This way sharing and exchange of data between developers is much easier. To ensure that all developers can access and use the same TestStand Components – especially the language directory – the TestStand configuration directory is also under SCM control.

During the development process if a developer checks out the project folder, for example to „C:\ProjectX“, then the language folder is available under „C:\ProjectX\TestStand\Components\Language“.

In Teststand there is an option to point the configuration directory to a custom directory. To do this, go under the Station Options – Preferences – Configuration Directory, and change the default path to our custom checkout path.

However, even changing this setting, the language files are still get loaded from the default configuration path. In other words, assigning new custom path (which is „C:\ProjectX\TestStand\Components\Language“) seems to have no effect and the older default location (which is „C:\Users\Public\Documents\National Instruments\TestStand“) keeps getting selected instead.

This means, every developer has to copy the ProjectX TestStand folder into the default location manually a SCM update.

Suggested solution: make the whole default location directory changeable, to point to a custom directory. Thus during development of different projects, it is much easier to switch between the projects and to have the possibility to check out the code from a SCM to only one directory (and continue working with them).

View All (3)

I just had an issue (SRQ#1672094 - for NI employees who want to look it up) where I tried to open a termination monitor in a sub VI which was called in a VI that already had an termination monitor running. The whole thing seemed to run properly until the second termination monitor was about to be closed. It threw an error telling me that I was playing around with an invalid reference. I was a bit confused...

The issue was solved by NI hotline in a short time - as usual: It's not possible to have several termination monitors running in this constellation and I should forward the open monitor into the subVI and use it there.

I wonder whether it is possible that an error message is thrown when one tries to open a redundant termination monitor. Even if LabView can't know, TestStand should be able to and respond accordingly if the function is called. It would make debugging easier.

View All (1)

What do you need our team of experts to assist you with?

Request a quote Find the right product Place an order Get support on a product

How can we help?

Please enter your information below and we'll be intouch soon.

This field is required

Preferred communication method

Email Phone call

What do you need our team of experts to assist you with?

Request a quote Find the right product Place an order Get support on a product

NI TestStand Idea Exchange

Review all idea Exchange and Update status

Array Literal Support for Named Types

Empty Array Literal

TestStand Sequences for Machine Learning Pipelines

For each StepResult, add StartTime with UNIX time offset

Smart rename into/out of container

Additional macros for "Engine.FindFile", "Engine.FindPath" and "Engine.ExpandPathMacros"

Provide ability to link a Sequence File to a specific Environment File

Allow model plugins access to Engine callbacks

Search Directories Insert method should not duplicate a search directory

Programatically allow to select LV Adapter Development System version

TestStand Expression Macros

Modify sequence during excution

Add New PostStepNoMatterWhat Engine Callback

SearchDirectories.Insert method should have additional input parameter for excluding hidden subdirectories

Host .NET processes outside of TestStand's application space

Implicit and fast container array elements access.

Allow containers to be assigned/initialized completely in a single expression

Change the path to localization language directory in TestStand

Termination Monitor should throw error in LV when created too often