NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

TestStand Report Generator Needs to be More User Friendly

I was working on another project and decided to use the TestStand Report Generator to record data from all my testing.

 

I had some fairly simple sounding requests that turned out to be not so easy to implement the way Report Generator is designed. Report Generator saves the programmer time by providing a test report that can be run from the process model. However, customizing the test report to display "what you want" or "how you want" requires special knowledge and understanding of the report generator itself that's not easy to learn without a lot of support. You'll spend days.

 

1. I'd wanted to just change the file name that the report is saved under programmatically based upon the subsequence that was called within a main sequence. My main sequence called "Performance Testing" had about 10 subsequences which were "individual tests" that were run as "single pass." I don't just run them all at once, but individually.

 

You can create ReportOptions callbacks to change the Base File Name, but this must be done immediately and if you place the callback in a subsequence it won't work. Spent a lot of time running it and seeing the file name still using the main sequence name and getting frustrated. You must use "Run as New Execution" for the callback to work from the subsequence. Then your subsequence call step from main needs to be changed significanly if you want to pass parameters to it from main. You have to play tricks with the prototype used for the subsequence call step. This proved to be a big change to how my subsequences were already setup.. and all this to just programmatically change the file name of the test report based upon a subsequence called from main!

 

 

2. I wanted to have the report routed to a specific directory programmatically based upon the test that was run (subsequence) from the main sequence.

 

Same difficulty in having to using ReportOptions callback.

 

Suggestion: Report Generator should be easy to modify the File Name, File Path and contents of the report at "any time" before the test is over. Create some "hooks" in report generator that make it "user friendly" within TestStand code. Spend a little bit of time on this and the product will be easier to use and folks will be a lot happier to talk about using it.

 

Thanks.

 

BTW- Many folks on the forums discussed this difficulty with Report Generator and report customizaiton. A few of the folks were sarcastically suggesting to create an executable that runs in cleanup that modifies the report, name, path, and contents outside of the Report Generator because it's not easy to work with it.

 

 

 

 

0 Kudos
Message 1 of 24
(4,742 Views)

Hello tears, 

you ended up posting to the Feedback board,  but since you question relates to a specific product you should post it to the appropriate board,  in you case the Teststand one. 

 

FYI the Feedback board is intended as a place where to discuss questions related to the forum in general. 



Proud to use LW/CVI from 3.1 on.

My contributions to the Developer Community
________________________________________
If I have helped you, why not giving me a kudos?
0 Kudos
Message 2 of 24
(4,727 Views)

Both the name and directory of your report can be changed in the ReportOptions callback. However, ReportOptions is called during initiallization (this is prior to your test being run). 

 

You can use NameOf(RunState.Caller.RunState.Sequence) to get the calling sequence name, however, that is probably not the sequence name you're looking for because the process model calls ReportOptions. 

 

I suggest you use this method:

Use a FileGlobal to store the information you want (like your sequence name) using something like:

FileGlobal.TestName = NameOf(RunState.Sequence)

inside of your subsequence.

 

Use the TestReport Callback, and prior to the Call TestReport step inside TestReport, modify:

Parameters.ReportOptions.Directory

and 

Parameters.ReportOptions.BaseName

using your previously defined FileGlobal.TestName

 

TestReport is called at the end of each run on SinglePass and allows you access to both the directory and name of the report.

0 Kudos
Message 3 of 24
(4,660 Views)

I tried the method that you described, but have not been successful. The report generator is fairly stubborn.

I haven't been able to change the file name or path programmatically from a SUBSEQUENCE.

 

Maybe you can provide more details.

 

0 Kudos
Message 4 of 24
(4,614 Views)

It seems TestReport is too late as the report location has already been determined (normally).

 

Using <UUTStatus> in Specify Report File Path by Expression will force Teststand to generate the path post-test and you can then modify the expression in TestReport in order to adjust the target path. 

 

I suggest when you're setting it up to put a break point in the GetReportFilePath callback to see the result of your changes (Parameters.ReportFilePath).

 

You could also rename/move the report file in the GetReportFilePath callback.

 

Let me know if that helps.

0 Kudos
Message 5 of 24
(4,590 Views)

Where is the step "Specify Report File Path by Expression" that you are referring to?

 

Do you have something generated that works? I've been playing with these call backs for weeks and nothing seems to work.. they never use a paramter that you try and pass into them.

 

 

0 Kudos
Message 6 of 24
(4,576 Views)

I followed your suggestion, but it did not work. I placed a breakpoint in the code and I can see the change to the Parameters.ReportOptions.BaseName, but this does not affect the final name of the test report.

 

I added <UUTStatus> in the Specify Report File Path by Expression on the Generate Report File Path pane. You were predicting that this would force the NI Report Generator to create the file after the test ran.

 

Have you actually gotten this to work? If so, maybe you can provide a screen shot on how you did this.

 

 

0 Kudos
Message 7 of 24
(4,543 Views)

https://www.ni.com/en/support/documentation/supplemental/08/teststand-report-generation-and-customiz...

https://www.ni.com/docs/en-US/bundle/teststand/page/specify-report-file-path-by-expression.html

 

You can further control the report file name and path by selecting Specify Report File Path By Expression option in the File/Directory Options control.  Using this setting, you can create an expression which TestStand evaluates to determine where the report is stored. This expression can include predefined macros, accessible in the Menu of the Report File Path control. 

Note: 

When configuring the report expression, use the Evaluated Report File Path field and the preview image to see the behavior of the current expression.

 

Sorting report files based on UUT results

You can create separate file locations for passing and failing reports by using the UUTStatus macro in the report file path expression.  For example, you can use this macro in your file path to create different locations for passing and failing reports. When this macro is present, TestStand re-evaluates the report file path expression after the test execution completes, since the UUT status is unknown before this point.

 

I have messed with it but I do not have an example. You can also use the GetReportFilePath callback to manually move the file to the location you want. 

0 Kudos
Message 8 of 24
(4,529 Views)

None of the suggestions appear to work. The NI Report Generator "hooks" for the file name and path always occur prior to a subsequence executions. Even if you use the call backs in a subsequence it ignores whatever you try and pass in because it's already beyond the point where it can change anything.

 

All the decisions are made when Main Sequence is called, unless you set the Subsequence to "Use New Execution" which is basically starting Report Generator fresh when that Subsequence starts. If you do this, which is what NI Applications Experts recommended to me then you lose the ability to PASS parameters from Main into your Subsequence call. You have to play around with the Subsequence call prototype and use the root variables instead to pass parameters from Main because of the entry point.

Setting <UUT STATUS> as part of the file name under Report Options does not change when the file name and path are generated.

 

None of that works.

 

It's pretty poor that you can't just RENAME or SET A NEW PATH programmatically within TestStand. You shoud be able to do this ANYWHERE in the code before the sequence ends.

 

It's a deficiency in the NI Report Generator VI design. NI needs to FIX IT.

0 Kudos
Message 9 of 24
(4,394 Views)

Here is a quick working example: 

 

1) I set a pop-up dialog to select what test you want to run.

2) In each test - I set 2 FileGlobals: 1 is the calling seq name, the other is the test name.

3) In GetReportFilePath I set a FileGlobal equal to the ReportFilePath so that I can reference it from ProcessCleanup

4) In ProcessCleanup I call on a Labview VI to move the report file to where I want it, making use of the TestName FileGlobal (you could use the calling sequence in your report path generation too). 

 

I'm not sure why we can't just change the target path in any of the report callbacks. It's not ideal. 

 

It might also be possible to modify the process model to call on ReportOptions every run instead of just once. I just havn't messed with modifying process models enough to know the answer to that. I make use of the ReportOptions in my case because we have a seperate Seq file for each test we run and when we run a test it's repeated all day. 

0 Kudos
Message 10 of 24
(4,386 Views)