Building Good Habits

[tyk007]

Hi Fab,

I happen to work for a regulated industry - we are happy in-house to accept other Unit Testing Frameworks if we can validate (get confidence) in their ability to generate the correct results. The only thing we miss is automated code coverage - but if you use custom tests in UTF then you don't get this either.

What is it in particular that the regulated industries you do work for require? I am wondering whether we are missing something or if there are additional features in UTF we have over-looked.

[crelf]

tyk007 wrote:

...we are happy in-house to accept other Unit Testing Frameworks if we can validate (get confidence) in their ability to generate the correct results.

You can work with pretty much anything in most regulated industries, as long as, as you said, you can validate it - as well as show how you validated it, that you followed a validation process, that the validation was valid (love that one), and the results to whichever authority oversees the industry. I think what Fab was getting at (correct me if I'm wrong here Fab) is that the UTF was released by NI, NI developed it under ISO 9001 guidelines, so it's one thing less to burn hours on.

tl;dr: using the UTF over other frameworks has the benefit of being one less thing to validate. That means you spend more time on what you're getting paid to do.

[FabiolaDelaCueva]

Christopher Relf wrote:

tyk007 wrote:

...we are happy in-house to accept other Unit Testing Frameworks if we can validate (get confidence) in their ability to generate the correct results.

You can work with pretty much anything in most regulated industries, as long as, as you said, you can validate it - as well as show how you validated it, that you followed a validation process, that the validation was valid (love that one), and the results to whichever authority oversees the industry. I think what Fab was getting at (correct me if I'm wrong here Fab) is that the UTF was released by NI, NI developed it under ISO 9001 guidelines, so it's one thing less to burn hours on.

tl;dr: using the UTF over other frameworks has the benefit of being one less thing to validate. That means you spend more time on what you're getting paid to do.

Chris Relf is right, the other reason some of my customers in regulated industries like UTF is for the code coverage reporting, requirements tracking with Requirements Gateway or Doors and the reports it generates.

There are ways through scripting to get code coverage, but then again this requires to do extra code that is not related with the main project at hand.

Now, if the code has lots of dynamic dispatch VIs and we need to create wrapper tester VIs for them, UTF starts loosing some points.

[crelf]

FabiolaDelaCueva wrote:

Now, if the code has lots of dynamic dispatch VIs and we need to create wrapper tester VIs for them, UTF starts loosing some points.

Exactly.

[tyk007]

It would be fantastic to have a few "in-the-right-position" NI employees watching this dicussion now....

I don't mean to sound like I'm anti-UTF. I started with it and thought it was great. My frustration just seemed to grow over time with little things (as I mentioned). Improvements in these areas certainly makes the native solution more attractive.

It is interesting to hear feedback from other people working in regulated industries. Here we often use open-source frameworks; not because open source is "cooler" (debatable) but often because there is no legitimate alternative. Funnily enough we don't automate requirements tracking through to testing and coverage; but I expect this will change over time.

Great discussion!

[crelf]

tyk007 wrote:

It would be fantastic to have a few "in-the-right-position" NI employees watching this dicussion now....

Looking at some of the members of this group, I'd say there are some pretty good NI eyes on this

tyk007 wrote:

Funnily enough we don't automate requirements tracking through to testing and coverage; but I expect this will change over time.

We use NI Requirements Gateway on every project - even the unregulated ones. It took a little time to set it up to dovetail into our document set (then again, what doesn't these days?), but I'd never go back. Even little things like numbering two requirements the same, NIRG catches minor stuff like that, and that makes me happy. But when I can generate a full-path traceability matrix, right from customer spec, through requirements, design, code, acceptance test proceedures, test artifacts.  Ohhhhh I'm getting excited just *thinking* about it.

tyk007 wrote:

Great discussion!

Absolutely! This is fun

[James_McN]

Great suggestion Scott,

I actually started working on a Jenkins plugin for LabVIEW a while back but other work meant it never got finished. Would be a good excuse to dust it off and see how far I got as well as hopefully get more our of my existing tests.

If it works. I shall get it shared somewhere.

[crelf]

James_McN wrote:

I shall get it shared somewhere.

Now you're talkin'!

[tyk007]

FabiolaDelaCueva wrote:

Tony,

Same request as for Scott, would you mind starting a discussion or document pointing us on how to get started with TeamCity?

I'll have a look and see what I can get away with. There is always that nasty "IP" tiger lurking around!

[scrook]

Hi Fab,

I have actually had a paper on what we do at TriQuint in the approval pipeline for pulication. The wheels sometimes turn slowly however but I will post something on this as soon as I get the OK.

Scott