Mock Objects - Any Success?

[James_McN]

I am realising that hardware is causing a significant testing headache. Admittidly often at a slight higher level than unit testing but it still seemed like an appropriate place to ask:

Has anyone had any experience trying to use mock objects in LabVIEW?

I'm considering attempting a mockable DAQmx library (hopefully with a reusable mock framework for other systems). There are different levels of mock objects in many languages from the simple to the complicated: (using the terminology I have seen in a JS library I have used, sinon.js)

1. Spys: Original function is called but we can see with what parameters.
2. Stubs: Function is replaced and returns a value configured earlier in the test.
3. Mocks: The above for a full API.

I can only see though, to achieve this in LabVIEW will require heavy use of:

• Dynamic dispatch - meaning existing APIs need to be wrapped in objects.
• Scripting - To enable easy generation of mock APIs as everything is strictly typed. This should make spys really easy.

As always the question is when will I find the time! But in case I do any tips/ideas/experiences with this type of thing?

[Thoric]

You learn something new every day on this Community.

James, mock objects (and fake and stub objects) sound immensely useful in a high demand software testing environment, I can immediately see their applicability in a number of scenarios (past and present).

They also sound potentially like a lot of work to implement in LabVIEW, especially if you're mimicking a class to which you don't have full access, like the protected DAQmx libraries.

I'm keen to hear if other wiresmiths have implemented mock objects before and how useful they found them.

[tyk007]

We implement mocks all the time in order to prove testability. Generally we lean heavily on dynamic dispatch and dependency inversion but for simple APIs we can often bake the functionality directly in.

But as Thoric metions they are a pain to develop in LabVIEW. Given the very time-consuming refactoring capability of the IDE we have gone into projects with the mind-set of the necessary abstraction rather than refactoring as we go (which I would have preferred, Agile-style). This means we have focused on APIs that are either safety-critical, would greatly reduce testing away from the development PC or are fundamental to performing any kind of unit-testing (such as controlling volatile dependencies such as date/time, file system etc.).

My preference would be to follow this philosophy (http://artofunittesting.com/definition-of-a-unit-test/) as much as practical as this is more in line with my background in .NET. If you haven't read the book by this author then I strongly recommend it; it is quite readable even if you are not familiar with the language.

We made an attempt some time ago (LabVIEW 2012) of generating a scripting library that would effectively re-create a given class but with mocking capability. We managed to get something functional at compile-time via the IDE but had difficulty with dynamic creation at run-time (my preference); as far as I know there was some issue with the way the class was loaded into memory that we never quite sorted.

If you are thinking of a mockable-DAQmx library - there is an alternative to this but would require using the DAQmx .NET libraries and using one the .NET mocking frameworks. The interface would then be wrapped up in LabVIEW. Hard to tell if this would really be much less work - it depends on how much of the current DAQmx API you want to mock.

EDIT: I have posted about mocking Date/Time previously: https://decibel.ni.com/content/message/90542#90542

[FabiolaDelaCueva]

James_McN wrote:

I am realising that hardware is causing a significant testing headache. Admittidly often at a slight higher level than unit testing but it still seemed like an appropriate place to ask:

Has anyone had any experience trying to use mock objects in LabVIEW?

When working with hardware, I tend to implement the simulation version of the hardware first. It helps me verify that the API that I had envisioned is the correct one. Also, for unit testing, I can check that the  message formatting, protocol or whatever other details needed are correct. This often is a lot of work at the beginning, but once the simulation is solid, doing the actual hardware is straight forward and the same unit tests can be used. If you are using JKI VI Tester, you can even make the hierarchy of JKI VI Tests match the hierarchy of your hardware objects.

The nice thing about these unit tests is that when hardware is implemented, the unit tests can be used for the integration test and make sure that the basic communication with the instrument works.

Either case, I agree with you:

James_McN wrote:

I can only see though, to achieve this in LabVIEW will require heavy use of:

• Dynamic dispatch - meaning existing APIs need to be wrapped in objects.
• Scripting - To enable easy generation of mock APIs as everything is strictly typed. This should make spys really easy.

I guess I have gone the dynamic dispatch route to implement:

James_McN wrote:

1. Stubs: Function is replaced and returns a value configured earlier in the test.
2. Mocks: The above for a full API.

I don't think I have used spys.

The more I think about this, I guess there are two types of simulation in this case. The one that you use while your application is running and where one can simulate failures, change booleans states and the one that would be used for unit testing with no interaction from the developer needed.

[tyk007]

There is a lot of terminology (mocks, stubs, spys, doubles, fakes  etc.) floating around. It can get quite confusing given that there seems to be a lack of consensus over the terminology.

I have used what James is referring to as "spys" previously when testing that calls were made appropriately to an interface but only when I expected some variation those calls (eg. a common interfacemethod but a large variation in parameter values expected for different functionality). Manually mocking (whoops, terminology again) this is possible but a pain to repeat on other APIs.

EDIT: I like your approach of implementing the simulation / stub functionality first and covering this with unit tests prior to the real implementation. Makes those tests much more useful!

[JeffreyH]

I tend to always build a hardware abstraction layer tailored to the appilcation between the higher level code and the actual HW. This is always an OO-layer and most of the time I use an abstract class with (at least) two childs, one that accesses the actual HW and one that is the mock-object.

In some smaller straight-forward projects I might omit the abstract class and just implement the class that accesses the HW rightaway. Only if the HW is also available at the time of coding ofcourse. If at a later time I do need to simulate HW for some reason, I'll often inherit from the HW class to facilitate this.

[Thoric]

Jeffrey Habets wrote:

I tend to always build a hardware abstraction layer tailored to the appilcation between the higher level code and the actual HW. This is always an OO-layer and most of the time I use an abstract class with (at least) two childs, one that accesses the actual HW and one that is the mock-object.

I hadn't appreciated that a simulated HW class is a mock-object. In my HW abstraction layers I do the same, and it's definitely true that the simulation class proves invaluable for system-level testing.

[JeffreyH]

It's funny how most of us probably use a lot of standard techniques and design patterns without actually realising it.

Often, when talking to people about the applicability of design patterns I find myself referring back to a presentation I did in 2009 (http://www.vi-tech.nl/public/files/downloads/NIDays2009%20OOD_Design%20patterns.pdf, sorry for the big logo's, still trying to establish the brand back then I guess.. ).

While preparing for that presentation I kept on falling back to the same example which was from an actual recent (pretty straight forward) daq-project I did. See slides: 27, 31, 33 and 34.

In this single (to me) everyday example we have: the mock object, oberver pattern, template method and active object.

[James_McN]

Couldn't agree more! terminology so often gets in the way! That I why I decided to emulate the terminology of a particular library so we can all get on the same page

Hardware simulation is important, what I would like to do is take that a step forward where we have some of these additional features that help with automating tests like confirming call parameters or having a programmatic way to change the response of the mock.

I have some ideas for the "spy" abilities as we can use variants and scripting to quite easily add this to VIs I think. Basically something that on a given run it can scrape all of the controls and store them as variant attributes with their label which would allow us to get the given run-time parameters back later.

[FabiolaDelaCueva]

We could use control tags, Darren exposed some VIs to work with them in his Hidden Gems package.

As long as the values kept are small (not large arrays for example). This might work.