Testing when the expected output is an array of NaNs reports a false positive

[FabiolaDelaCueva]

Hi fellow Unit Testers,

I remember having this issue with the older version of the Unit Test Framework, but alas, it is still present in the new version.

A customer contacted me asking what to do, because the expected output was array of NaNs and the VI under test was returning an array of numbers and yet, the test was passing!!!

My suggestion to use as a workaround was to create an User Defined Test that would test for the array of NaNs. I was wondering if anyone here had found this issue and if you had any suggestions other than the User Defined Test.

Before someone suggests that we move to JKI VI Tester for this particular project, I can tell you that we are using UTF and not JKI VI tester because the customer is in a regulated industry.

[Edit] Cross-posted at the forums: http://forums.ni.com/t5/LabVIEW/Unit-Testing-with-Unit-Test-Framework-when-the-expected-output/td-p/...

[Edit] Here is a video describing the problem and how we are addressing it via a User Defined Test.

Thanks,

Fab

[tyk007]

Wow. As you know I don't use the UTF that much but that's a little scary. I would have expected that any NaN comparison would have simply failed no matter what (if they were simply testing using the Equals? primitive for any native type). But successful comparison between a number and NaN? Presumably you reported this.

The only solution I can think of is a user Defined Test with specific IsNan comparison for the one test; there are no other obvious solutions.

I wonder if the UTF has it's own set of unit tests?

[FabiolaDelaCueva]

tyk007 wrote:

Wow. As you know I don't use the UTF that much but that's a little scary. I would have expected that any NaN comparison would have simply failed no matter what (if they were simply testing using the Equals? primitive for any native type). But successful comparison between a number and NaN?

To be fair, if you have a scalar it does work, it is when you have an array of NaNs that it breaks.

tyk007 wrote:

Presumably you reported this.

Other than the tweet and the post on the forums: http://forums.ni.com/t5/LabVIEW/Unit-Testing-with-Unit-Test-Framework-when-the-expected-output/td-p/...

No. I was hoping that it would make someone at NI scare, but I guess since I present a workaround on the video, it wasn't scary enough... or maybe they thought it was a Halloween trick... anyway, I will give it a couple of days, if I don't see any movement on the forums, I will call an AE.

tyk007 wrote:

I wonder if the UTF has it's own set of unit tests?

I do wonder as well. If it does, hopefully they will fix that unit test or add a unit test to check for this.

Thanks for validating that the user defined test would be the only way to go.

Fab

[FabiolaDelaCueva]

update from the forum post. AE was able to reproduce the problem and has filed a CAR (Corrective Action Request) with NI R&D.

CAR # 554457

[FabiolaDelaCueva · Accepted solution]

Update on this issue, I just tried creating a unit test that had an array of NaNs as expected output and the issue has been solved in LabVIEW 2017 and LabVIEW 2018. So I did a search and it turns out this was fixed for UTF for LabVIEW 2016. You can find the CAR #554457 listed as fixed in the bug fixes section of the LabVIEW 2016 Unit Test Framework Readme at http://www.ni.com/pdf/manuals/375338b.html