Unit Test Tools in LabVIEW

[jon_mcbee]

I am working on a new blog post about Unit Testing in LabVIEW and am going through the pros and cons of the existing tools for unit testing in LabVIEW.  I figured I could try crowd sourcing the ideas to increase the variety of feedback.  So if you are using VI Tester, Caraya, or the UTF (any others?) I would love to hear what you like, what could be improved to increase your efficiency, or any other thoughts you have on features of your tool of choice. 

I am also interested to hear from people who aren't unit testing, although I don't expect to find them on this community page.

Feel free to reply with feedback here, or if you prefer you can post thoughts here:  https://unittesting.ideas.aha.io

I appreciate any feedback.

Thanks,

Jon

[kjeld]

Hi Jon

I have a couple of ideas

• Integrate UT as a native part of LabVIEW project explorer
• One button for executing all Unit Test (UT) in LabVIEW project explorer
• VI’s that are Unit Tested should have an overlay icon for passed or failed
  
• Right click option for open UT for at specific VI. Link VI and UT together
  
• When creating a UT, the UT framework should auto generate a set of test vectors for the VI under UT
  
• Test speed is crucial for executing all UT’s
  
• Test coverage report for VI’s under UT, also for Reentrant VI!
  
• Test results as a Junit / XML file(s)
  
• API for continuous integration test, executing and controlling all UT
  

/christian

[kjeld]

I forgot that it is also importent that it can test class without making a wrapper first

[jon_mcbee]

Hi kjeld,

I appreciate the feedback, this is great!  I have a couple questions for you, or anyone else following this thread.  What do you consider a "unit" in your unit testing?  Do you work in a regulated industry, and if so how does that impact your unit testing?  Do you use test doubles in your unit testing practice, and if so how do you implement them?  How do you use the test results, do you save them or do you mine the data from the files for use in a tracking tool?

Thanks,

Jon

[Oli_Wachno]

Hi Jon,

not very much to add to kjelds' points, they would really leverage unit testing  under LabVIEW.

I do particualarly like the point about CI integration

Currently I am considering a VI to be the UUT. I like the idea of reagarding a class as a unit, but this can probably increase complexity tremendeously... So I'd be happy to work with VIs for now.

[jon_mcbee]

Hi Oli,

Thanks for the follow up.  With regard to CI, would you prefer to have an API that gives you the hooks you need to integrate into whatever CI tool you like, or a complete Jenkins integration that you can just use, or both?

Would you (or anyone else reading this thread) find use in an integration of a tool like JIRA that allowed you to trace your testing back to your user stories/requirements/bugs?

Would you (or anyone else reading this thread) be interested in coded UI test at the system test level (moving out of scope of a unit test tool, but why not)?

How important is code coverage analysis?  Would you be interested in a tool for mutation testing that can provide information on the quality of your unit tests?

Thanks,

Jon

[kjeld]

Answer to your questions:

A unit is typically a single VI tht you can do static test on.

I work with software test at Siemens Wind Power. And we are using UT on som parts of our software for our simulators, LabVIEW related.

We only perform UT on code that can run HW independent.

The test results are used in TeamCity or Jenkins, for the LabVIEW project it is Jenkins.

[Oli_Wachno]

Hi Jon,

I love the idea of really pushing this thing forward   Still I would be very happy to have only a basic intergration for the first step.

I am using Jenkins, so integration into Jenkins would be my favourite.

Traceability features are always favourable to have, so.... yes!

Cheers

Oli

[jon_mcbee]

Hi Oli,

That is good feedback, basic integration that is simple and efficient is more important than shooting for the moon, missing, and dying slowly in space like George Clooney (Spoiler Alert: That is how Ocean's Fourteen ends).

Thanks,

Jon

[kjeld]

...sorry the ideas just keep on coming

It would also be nice if we could execute UT that is related to a RT target or FPGA on Windows. This would of course set som restriction on not having HW in the VI

An other solution would be to asign a RT target with a FPGA(s) but this is a expensive approach.