NI TestStand

cancel
Showing results for 
Search instead for 
Did you mean: 

Questions modifying a report. Skipped steps and Failures.

Using ATML format, is there a way to exclude "Skipped" steps from the report? I see you can in HTML, using result options, and also XML by modifying the stylesheet.

 

Also, using ATML, with multiple failures that occurred during the test, the report failure summary, at the top, only displays the first failure. I am not sure where the critical failure stack is generated in the atml model. In the xml, I see that it is in the AddReportBody sequence and I tried using xml format, and see that it only sees one failure when multiple occurred. Not sure why that is. Would like to know where the critical failure stack is located in the atml model.

 

If I can figure out the issue with the failure stack using xml, I would not mind changing the format to xml as I can then exclude the "Skipped" steps and also have a failure summary at the top of the report.

0 Kudos
Message 1 of 9
(4,385 Views)

First of all, i am not familiar with the ATML Test Result (TR) specification. Therefore, i am not sure if TR defines that all executed steps must be listed in the TR. That being said, i find it possible that removing step results from the TR is against the ATML specification.

 

However, you can alter the report generation by customizing the appropriate code. Please do create copies before doing this!

The ATML report is mainly created in the ATMLSupport.dll which is provided with source code written in LabWindows/CVI.

 

You find a method called "UpdateCriticalResultStack" there which i would expect to generate the failure chain.

Adding to this, there is the philosophy behind the failure chain to only display the first failure. The reason is that for this is that each failure is equal in its severity. So if a failure occurs, the whole UUT is failed. The failure stack shall only display "it is failed because of the first issue found here". If you require the whole bunch of failures, you should review the report.

However, you can modify the failure chain to add ALL failures.

 

Norbert

Norbert
----------------------------------------------------------------------------------------------------
CEO: What exactly is stopping us from doing this?
Expert: Geometry
Marketing Manager: Just ignore it.
0 Kudos
Message 2 of 9
(4,371 Views)

Thanks, Norbert.

 

I was able to disable "Skipped" step recording using XML and also add all failures to the Failure Chain, using XML.

 

One other question though. I have a sequence file that calls 5-6 different sequence files. Is there a way to add a report section, similar to the Failure Chain, that shows each sequence Pass/Fail? I have started looking into the models, and modiying some, to add something similar to the CriticalFailureStack but with the "SequenceCall" step types, these are the steps in the ResultList that are the calls. Now, I am unsure of adding this information to the report, if possible.

 

Edit to add, a possible approach, if possible, was to get the data for that specific "SequenceCall" type step and store into an array. Have this inside the UUT.AdditionalData. Reason for this, I see where I can modify the XML stylesheet to add this array into a report section similar to the Failure Chain. I noticed that maybe I can take a similar approach as to how the xml stylesheet uses the CriticalFailureStack in the html code: "Prop[@Name='UUT']/Prop[@Name='CriticalFailureStack']/Value"> for the Failure Chain.

0 Kudos
Message 3 of 9
(4,351 Views)

Guess my main questions is right now. Is there a way to add another table to the report, at the top, similar to "Failure Chain"?

0 Kudos
Message 4 of 9
(4,332 Views)

You can modify the style sheet to collect data and display accordingly. This would be the simplest solution.

 

Norbert

Norbert
----------------------------------------------------------------------------------------------------
CEO: What exactly is stopping us from doing this?
Expert: Geometry
Marketing Manager: Just ignore it.
0 Kudos
Message 5 of 9
(4,325 Views)

Correct. But, I don't have the time currently to be able to learn what I need too. I know a little about it but not enough to create a table, at the top, and add the array of data I want. Was wondering if there was anything out there on this. I have found the NI "Customizing XML Style Sheets" page but seems it is basic information.

0 Kudos
Message 6 of 9
(4,307 Views)

The "problem" is that having data in the XML (ATML) file is not sufficient. The style sheet instructs the viewer application (e.g. Microsoft Edge) on HOW to display the data. So if the style sheet contains NO instructions for specific data, they WILL NOT SHOW UP AT ALL.

 

That being said, it is very likely that without customization of the style sheet, you will not get the requested additional header content (table).

What you CAN do is to use the "ModifyReportHeader" callback override in your client sequence file to append or prepend (or with more effort even "inclusion") text to the header. However, for XML(ATML) it is additional text in the XML (based) format which is "data". Again, if that data is not in a way that the default style sheet "accepts" it for display, you will simply not see it when looking in the report except looking into the bare XML file.

 

Norbert

Norbert
----------------------------------------------------------------------------------------------------
CEO: What exactly is stopping us from doing this?
Expert: Geometry
Marketing Manager: Just ignore it.
0 Kudos
Message 7 of 9
(4,303 Views)

One other thing. I have modified the style sheet to not report 'Skipped' results. This causes the indentions in the report to be incorrect which causes steps to be cut off on the pages of the report. I followed Excluding Results but this results in the report indentions being incorrect. Anything else I would need to do?

0 Kudos
Message 8 of 9
(4,281 Views)

If not dug into indenting with XML. That being said, i can only guess.

 

Indenting is a way to identify sub-sequence execution in the report. Each sub-sequence call shall increase indenting (up to a maximum). In HTML and TXT reports, the indenting is integral part of the report file itself and is handled by the report generator by managing a variable called "Level".

 

My guess for your indenting-issue is that you removed too much from the style sheet or you have incorrectly changed things for steps of type "Sequence Call" when these are Skipped....

 

Norbert

Norbert
----------------------------------------------------------------------------------------------------
CEO: What exactly is stopping us from doing this?
Expert: Geometry
Marketing Manager: Just ignore it.
0 Kudos
Message 9 of 9
(4,274 Views)