TestStand Report Generator Needs to be More User Friendly

[tcaros]

I was working on another project and decided to use the TestStand Report Generator to record data from all my testing.

I had some fairly simple sounding requests that turned out to be not so easy to implement the way Report Generator is designed. Report Generator saves the programmer time by providing a test report that can be run from the process model. However, customizing the test report to display "what you want" or "how you want" requires special knowledge and understanding of the report generator itself that's not easy to learn without a lot of support. You'll spend days.

1. I'd wanted to just change the file name that the report is saved under programmatically based upon the subsequence that was called within a main sequence. My main sequence called "Performance Testing" had about 10 subsequences which were "individual tests" that were run as "single pass." I don't just run them all at once, but individually.

You can create ReportOptions callbacks to change the Base File Name, but this must be done immediately and if you place the callback in a subsequence it won't work. Spent a lot of time running it and seeing the file name still using the main sequence name and getting frustrated. You must use "Run as New Execution" for the callback to work from the subsequence. Then your subsequence call step from main needs to be changed significanly if you want to pass parameters to it from main. You have to play tricks with the prototype used for the subsequence call step. This proved to be a big change to how my subsequences were already setup.. and all this to just programmatically change the file name of the test report based upon a subsequence called from main!

2. I wanted to have the report routed to a specific directory programmatically based upon the test that was run (subsequence) from the main sequence.

Same difficulty in having to using ReportOptions callback.

Suggestion: Report Generator should be easy to modify the File Name, File Path and contents of the report at "any time" before the test is over. Create some "hooks" in report generator that make it "user friendly" within TestStand code. Spend a little bit of time on this and the product will be easier to use and folks will be a lot happier to talk about using it.

Thanks.

  Tony

BTW- Many folks on the forums discussed this difficulty with Report Generator and report customizaiton. A few of the folks were sarcastically suggesting to create an executable that runs in cleanup that modifies the report, name, path, and contents outside of the Report Generator because it's not easy to work with it.

[QueenSelene]

Hi tcaros,

Thanks for your very detailed feedback, this is very valuable to us and we really appreciate it. I think your post would actually be best recieved on the NI TestStand idea exchange, https://forums.ni.com/t5/NI-TestStand-Idea-Exchange/idb-p/teststandideas which is monitored by our R&D team amongst others and is designed for valuable thoughts and feedback such as this, if you want to copy it over to there. Thanks!