UTF to JUnit report

[MattP]

Hi all,

I'm building out some continuous integration tooling for DCAF, and to my dismay I see there is no native option for UTF to output test results in Junit format. 

I haven't seen any community code out there to do this, but before I jump in and build it, I want to check with this fine group.  Do any of you have UTF JUnit report generation code already implemented that can be shared publicly? 

If not, my plan is to build a small bit of code that will take UTF results and spit out a JUnit compatible report.  It would be supported in 2014 or later, and would depend on the JUnit Results API published on the tools network.  If you have comments, I'd love to hear them.  I expect to get this going in the next day or two.

(why UTF for DCAF?  As far as I am aware, JKI's VI Tester does not execute code on remote RT targets, which is rather important for what we're doing.  I'd love to be proven wrong.)

[crelf]

I'd like to see a utility that converts UTF reports to junit - let me know if you need a hand.

[MattP]

Also, to be clear, I'm looking at getting the minimum subsbet of data into the JUnit report to allow Jenkins to keep track of unit test status for automated builds.  I'm sure there are lots of fancy things that can be done here, but I'm planning on keeping it as simple as possible to get the job done.

I am not planninng to make a ATML to Junit converter, rather I'm looking at taking the raw result data that would normally go into the UTF Create Report VI and use that data to generate a new JUnit report type.

Of course, I'll post the source code when done.  Not sure where it will live (might be on the https://github.com/LabVIEW-DCAF repository, might be on the SE repo).

[Oli_Wachno]

Hi Matt,

during niACS' presentation about CI during NIWeek, he was talking about his plans to intergrate JUnit reports into the CI project. Don't know how far this is. I guess you know about each others plans?

Oli

[MattP]

Yep.  He has the JUnit Results API I mentioned.  The mapping from UTF to JUnit is what is missing.  My work will hopefully be folded into his.

[MattP]

Ok, looks like I have something usable.  Takes UTF report data and makes a JUnit compatible report that Jenkins can correctly consume, including correct listings of passed, failed, error, and skipped.  Errors from UTF are passed to the test report.  Failure details are not passed to JUnit report, other than which test case(s) failed.

Next up is getting the calling infrastructure in place so it can be invoked automatically from Jenkins on new build.

[MattP]

Only tested in 2013 - I have not attempted it with later versions yet.

[MattP]

Source code is here:

https://github.com/LabVIEW-DCAF/UTF-Test

Still building some things, so please forgive the mess.  Will be a VIP once complete.

Note: this depends on an as-yet unpublished version of the JUnit API package, attached here for convenience.

[MattP]

And here is a build (still depends on ni_lib_junit_results_api-1.0.0.2.vip posted above).  This build is currently in use in the DCAF autobuild system, and works quite well as far as we can tell.

I'd appreciate any feedback you have.

[chinghwayu]

Hi Matt,

I tried this and for the most part, it seems to work pretty well. However, when looking at the execution times for the test cases, it's off by 1000 as in it's reporting in milliseconds when it's expecting seconds.

In the "create junit report.vi" default case, there's a VI that calculates the exec time named "get run time total.vi". This calculates the test case execution time in milliseconds. However when it's written to XML using "Add Test Case.vi", the time expected is in seconds.