Building Good Habits

[Penar]

• We found generating the tests time consuming using the IDE. We didn't put any effort into 'automating' the framework though (either editing the files manually or generataing them automatically).
• Our unit test suite for one small project took almost an hour to run (around 400 tests). We found this to be out of sync with our TDD practices (and also quite frustrating). Moving to VI Tester has reduced the same test time down to less than 20 seconds. I have seen similar comments on the forums (both 'dark' sides).

We have heard the feedback of editing and running UTF being slow. Here are a few considerations behind when designing UTF.

• Openning lvtest editor UTF will go through all the inputs/outputs VI under test to make sure any API changes of VI under test is refreshed in UTF test editor.
• There are special cases where UTF creates temp wrapper VI for VI under test. For example, global variables/local variables are not on the front panel but UTF needs to provide them for configuration.
• All the effort above is to allow a programatically decoupled test data (lvtest files) from VI under test. UTF wants the test data to be easy to maintain for regression test.
• Running a VI does not run directly on the VI under test. UTF creates wrapper VI to pass in data from setup VI and pass out data to teardown VI. Another reason for UTF to run on wrapper VI is to make sure VI under test is decoupled from unit test status. For example, you have input value and output value of VI under test. And you run a test case with some other values from lvtest file. After running test, you may find the input value and output value of VI under test does not 'disappear' because of running a lvtest file.

[crelf]

Penar wrote:

We have heard the feedback of editing and running UTF being slow. Here are a few considerations behind when designing UTF.

So what I think I'm hearing is that it's the decoupling of the VI under test from the definition of the test is what takes a long time? I'm all for the decoupling, but it seems the significant time penalty (and, in a lot of cases, it's awfully significant) is driving people away from the NI UTF. Are there any plans on the roadmap to take a good look at seriously optimizing these items? Because, although I'm a fan of NI UTF, making it a pain to run it means that people just won't run it, defeating the purpose of unit testing altogether. We're tyring to help people walk into unit testing in an easy and painless manor (there's a lot of misnomers out there about unit testing being hard), and the responsiveness is a pretty big roadblock for adoption IMO.

[FabiolaDelaCueva]

Penar wrote:

• Running a VI does not run directly on the VI under test. UTF creates wrapper VI to pass in data from setup VI and pass out data to teardown VI. Another reason for UTF to run on wrapper VI is to make sure VI under test is decoupled from unit test status. For example, you have input value and output value of VI under test. And you run a test case with some other values from lvtest file. After running test, you may find the input value and output value of VI under test does not 'disappear' because of running a lvtest file.

what I have seen is that if the Unit under test is open, I do see the last values from the last test case that was executed in the UTF Demo I did during NI week at minute 1:14  , you can see how the values are originally set to default and then after the test runs, the values from running the test stay there (minute 1:18). Perhaps I am missunderstanding what you mean by "Running a VI does not run directly on the VI under test". Would you please clarify?

Thanks,

Fab

[tyk007]

Christopher Relf wrote:

So what I think I'm hearing is that it's the decoupling of the VI under test from the definition of the test is what takes a long time? I'm all for the decoupling, but it seems the significant time penalty (and, in a lot of cases, it's awfully significant) is driving people away from the NI UTF. Are there any plans on the roadmap to take a good look at seriously optimizing these items? Because, although I'm a fan of NI UTF, making it a pain to run it means that people just won't run it, defeating the purpose of unit testing altogether. We're tyring to help people walk into unit testing in an easy and painless manor (there's a lot of misnomers out there about unit testing being hard), and the responsiveness is a pretty big roadblock for adoption IMO.

Agreed. This is our number 1 complaint and the single biggest reason we moved away from UTF for anything but the simplest projects.