Unit test framewok code coverage with Conditional Disable Structure

[ViltBalint]

Hi everyone!

I have to use unit test framework becuase of code coverage check, but as far as i see, unit test framework ignors the content of any conditional disable structure and calculates false positive code coverage result. Have somebody information about that, is it a feature or a bug? 🙂

Thank you for help.

[RavensFan]

I would consider it to be expected.

A disable structure is the equivalent to commenting out sections of code.  If this was a text based language, I wouldn't want a test framework to be trying to figure out if my comments are valid syntax.  Disabled code has been removed from a path of execution.  How would a test framework even be able to execute it?

[baalint]

Hi!

Thanx for your answer, but maybe my problem was not clear. Conditional disable strucure (CDS) is more similar to a specific switch case, because one of case will be active during code execution always. And UTF doesnt check sub coverage of _active_ branch, which is a problem, if CDS contains more branch (that means, you can reach 100%, without cover all of sub branch in CDS). I know, this solution makes UTF implementation easier, but that is not inpossible, to calculate subcoverage of active CDS branch. What do you think?

[RavensFan]

No it is not similar to a specific switch case.  A regular case structure has any case possible to execute and all code is compiled as such.  A conditional disable structure is fundamentally changing the underlying compiled code.  That is why you can wrap up broken snippets of code in a VI and have the run arrow go from broken to runnable.

If you want code in every case to be compiled and studied by the UTF, then you need to make sure it all works and it is placed in a regular case structure.  Use an enum control to drive what cases would actually execute when you run the code.

[bokorpet]

I see it different. The active "case" of a conditional disable strucuture is actual code, which needs to be tested as well. It's basically a "macro" in preprocessing step. I would fully expect the UTF to cover that code too.

[Olivier_L]

Hi,

I started (and erased) this message differently because I misunderstood the discussion, but here is what I'm seeing with code coverage, and I realized an issue with "Diagram Disable Structure"

I'm actually trying UTF, and I'm particularly interested by code coverage, and I'm also facing some strange behaviors in results:

With this simple VI, I have a test, with 2 test cases, 1 with Boolean = True, and the second with Boolean = False.

Code coverage gives

Where I expect 66,7 both times.

If I add a Diagram Disable structure, but let case enabled

Then, Code voverage gives

There is probably a bug somewhere.

Thanks

Olivier

[ViltBalint]

Thanks for your answer, and it is what i tried to explain 🙂

The second case is clear for me, and i think so, it is a LV bug, because UTF does not calculate subcoverage in "Diagram Disable Structure" (in "enabled" case).

But the first case is not clear, because, i dont know how may reach on that vi 66% and 100% code coverage. So there is a case structure with TRUE and FALSE cases, and i would expect only 50% coverage is reachable within one test run.

[Olivier_L]

Actually, UTF uses the number of subdiagrams as the percentage of coverage.

- There is always 1 diagram (principal)

- Then, each structure may have 1 or sevral diagram. For a case structure with only False/True, there is 2 subdiagrams.

So, in my example, there is 3 subdiagrams (Principal + case structure). And each test case runs 2 diagrams on 3 (the principal + 1 case of case structure) -> 66%

[ViltBalint]

Hi,

thanks, i did not know about existence of "principal" diagram, and i dont know why is it necessary to measure (branch) coverage. Could you explain for me? Im not expert in code coverage topics, but as far as i know LV calculates branch coverage (or maybe not) and if im right, the mentioned example contains only two branch.

[Olivier_L]

I'm not an expert on code coverage. What do you mean by "branch coverage" ?

UTF only regards the part of diagrams being executed during a test to realize a code coverage measurement.

What I called the principal diagram is the block diagram of you VI under test, there is no possiblity that this block diagram is not executed when your VI execute.

Then, depending on your code, and particularly on case structure where there can be several sub-diagrams (1 for each case), and depending on the combinatory of your complete code, there will be several "branches", and UTF will measure a percentage of subdiagrams executed as a result of code coverage, which may not fulfill with what you say "branch coverage" if I well understand.

In my example, there is only two branchs (as you pictured), but there is 3 diagrams. Thus, when you expect to have 50% branch coverage, UTF will return 66% code coverage.

And as far as I could read, UTF will not follow branches into subVIs:

- If you case structure is in a subVI, UTF consider that there is only one diagram in your VI under test, and that 100% of code (for VI under test) as been covered.

- You have to create a second unit test , testing you subVI, to test your 2 branches. Then, you could have a 100% code coverage of the code in project, combining several test covering all VIs.

I don't know if I'm clear...