Which Unit Testing Tool are you using?

[A.E.M]

Hi All.

I have just setup a Jenkins server and a couple of workers with LabVIEW Command Line Interface from NI. This is working very well with Github.

Now the time has come to Unit Testing, so I have been spending some time on watching the videos and discussions in this group. Based on that, it seems like most people are using JKI VI Tester.

What I value the most about a Unit Testing framework is the time it takes to create a new test: It must be easy, intuitive and fast.

Before spending too much time on any of these tools, I wanted to ask if JKI VI Tester is still the go-to tool or if NI or any other 3. party has improved their tool to work even better here in 2019?

Thanks. 

[bokorpet]

Hi Alex,

you may find this comparison table with the different available unit testing tools interesting: https://instacoverage.io/details/

In general, when it comes to usability, it really depends on your use cases. You may want to check each tool for a quick try-out as they offer different GUI, API and configurability.

Peter

[James_McN]

Hi Alex,

Peter's table shows the various players well and I don't think there is a clear leader in the pack at the minute.

I currently use VI Tester myself as I don't need code coverage and like the programming model - I can make it work for real-time and FPGA as well with a little pain (and running the logic on Windows).

Unit Test Framework has traditionally been the go-to if you need coverage and certification but people tend to have a bad experience with it. InstaCoverage covers the same ground now but I've not tried it in anger.

Caraya has a similar feature set to VI Tester but a very different approach so as Peter said it may come down to trying a couple out and seeing what feels natural for what you want to achieve.

[schddc]

• JKI VI Tester for LabVIEW unit testing.
  • With JKI VI Tester Junit XML Test Results Plugin
• TeamCity for a CI server. The TC Build is configured with a XML Report Processing build feature which is able to understand and display the tests results directly in TeamCity.

The main problem I have with JKI VI Tester is that it's not maintained. I have open issues on the github repo for over a year.

[A.E.M]

Thanks for your replies!

This is my main concern:

---

The main problem I have with JKI VI Tester is that it's not maintained. I have open issues on the github repo for over a year.

---

I do not want to initiate a new process and suggest this tool to my colleges if it is no longer maintained... 😞

Is InstaCoverage the new go-to? It's not free as opposed to VI Tester, but if it works and is maintained it might be worth it. 

However, I did found this post on InstraCoverage and it sounds like it is getting some hits for usability:

https://forums.ni.com/t5/Unit-Testing-Group/Unit-tests-for-RT-target-InstaCoverage-2-1-is-now-availa...

Any comments on this?

[bokorpet]

Quick comment: InstaCoverage Core is a free tool (not open-source yet but it is planned to be one). InstaCoverage Pro requires a paid license and it adds coverage, API and real-time execution features. New features will be added continuously (advanced test template generation, project coverage measurement, etc.). 

[ryan_sheppard]

---

@A.E.M wrote:

Hi All.

I have just setup a Jenkins server and a couple of workers with LabVIEW Command Line Interface from NI. This is working very well with Github.

Now the time has come to Unit Testing, so I have been spending some time on watching the videos and discussions in this group. Based on that, it seems like most people are using JKI VI Tester.

What I value the most about a Unit Testing framework is the time it takes to create a new test: It must be easy, intuitive and fast.

Before spending too much time on any of these tools, I wanted to ask if JKI VI Tester is still the go-to tool or if NI or any other 3. party has improved their tool to work even better here in 2019?

Thanks. 

---

Also using VI Tester and the JKI VI Tester JUnit XML Test Results plugin with success. Highly recommend outputting your results in JUnit format, Jenkins has a plugin that allows for it to be published.

Also using James McNally's VITAC to extend VI Tester.

[Dhakkan]

Hi Alex,

We use NI's Unit Test Framework (UTF), but with a User-Defined Test (UDT) template. (If you've never used it, a simple example is provided in LabVIEW's examples once you install the UTF.)

Our template is based off the JKI state machine (string message handler for the purists). Therefore, it has been very easy to perform setup and teardown within the same UDT VI. Additionally, we use JSON files as source for test vectors. Therefore, a single run of the UDT exercises several input-output test conditions, and rather quickly. One caveat is to account for the statelessness or statefulness of the VI under test.

A problem we've had with the UTF approach is that the executable build process ends up saving ALL the *.lvtest and UDTs of the project in the support folder. (This has to do with how our project is set up; we haven't figured out how to avoid this problem apart from a brute force post-build action.)

Apart from the problem, it has been quite straightforward to use LabVIEW CLI to execute the unit tests through the Continuous Integration tool and get the UTF's familiar report.

Hope this helps.

Vishak

[James_McN]

@bokorpet on this thread is the developer/owner for Instacoverage so he might be able to give some idea on where they are at addressing those issues.

[FabiolaDelaCueva]

---

@Dhakkan wrote:

A problem we've had with the UTF approach is that the executable build process ends up saving ALL the *.lvtest and UDTs of the project in the support folder. (This has to do with how our project is set up; we haven't figured out how to avoid this problem apart from a brute force post-build action.)

Speaking of UTF not having a good DX (developer experience).

Check out the properties for your project, there is a section for UTF and you can select there the target for your unit tests. I always put them in a folder at the top level of my project directory called Unit Tests. This way they are outside of the folder for source distributions and exe building.