Unit Tests with Actor Framework

[rik_aspinall]

So I've been thinking about unit tests for the AF. I can get some traction just by using the usual unit test utility as provided by NI, which seems very basic, but at least can check that my methods

- Respond as expected in incoming errors

- Can handle corner case values of inputs

- Can compute things correctly if they use mathematical functions

- I might catch an error if I were to change the class private data without modifying any accessors

- Not much else

It's a good start. But now I want to check more actor-like behaviour, such as:

- no memory leak when launching and destroying actors, or failing to destroy nested actors before themselves

- e-stop messages take priority, even with a bunch of other messages on the queue

- even if e-stop message arrives, shut down is tidy

- put a bunch of messages on the queue in different orders and check that order produces expected result

Perhaps I could pick your brains for any methodologies you use, or what other contingencies and common gotchas you look out for

I also have 2 more specific concerns:

1) I would like to know if I could somehow script the "putting messages on queue in an order" to allow for easy variations on the test. The only way I have right now is to launch the actor under test, have a case selection of many different message combinations, and then shut down the actor. I then unit test by selecting the enum and using a message box to visually display the result. One option would be to use a synchronous method to ask for the actor class data and get a reply, but then I don't want to release that into the wider world!

2) I would like to know if there's any way to automate a user interface button press on a front panel (probably not just AF, but still relevant to AF folks). I could make a "register for events" and handle that in the same event case as the button press, but it seems excessive and can still be broken if the button handling code is not kept tight with the event.

Thanks,

Rik

[AristosQueue (NI)]

> One option would be to use a synchronous method to ask for the actor class

> data and get a reply, but then I don't want to release that into the wider world!

Create that method, mark it as Community scope, and friend your test harness VI. A variation is to create the Community scoped method and then create a reply message to call it, marking the reply message as a friend. Never ship the reply msg.

One theory of mine is that once you get past the point of testing the state of the actor matches what you expect after each method call, you don't care about the internals. Your actor is correct if it is correct from point of view of all external sources. So send it some messages and see if it replies with the right messages. If what you really care about is some hardware or system resource whose state your actor controls, then call the methods on the actor and then interrogate that resource.

You may need to mock the underlying resource. With the Evaporative Cooler sample project that ships with LabVIEW, when we launch, we launch with either a simulated hardware or real hardare. The simulator would let us test the actor that controls the cooler by directly checking the global VI that is the simulator's state after each call.

[CaseyLamers1]

I use the JKI VI Tester to unit test my actors.

I test the public interface. I use the init methods in the setup.vi to get valid enqueuers and then put the actor in my test class private data.

Then I create a new test for each message. I just take the actor and wire up a Do.vi with the message going to the actor.

I then do one of two things.

1. I check that the actors internal state has changed as I expect. (this might use #2 below as a state change is generally sent in a message)

2. I check that the actor sends the appropriate messages.

For the case of sending messages to other actors I create unique queues (obtain queue and get both the enqueuer and dequeuer see the Setup.vi above)

Casey

[FabiolaDelaCueva]

rik_aspinall wrote:

2) I would like to know if there's any way to automate a user interface button press on a front panel (probably not just AF, but still relevant to AF folks). I could make a "register for events" and handle that in the same event case as the button press, but it seems excessive and can still be broken if the button handling code is not kept tight with the event.

Thanks,

Rik

A relevant discussion on UI testing can be found at the Unit Testing group: https://decibel.ni.com/content/message/85604#85604

Regarding the testing of private messages, tyk007 in the Unit Testing Group has made reference to http://artofunittesting.com/definition-of-a-unit-test/ there the author defines a unit more as a unit of work. In our world a unit can be a single VI, a group of VIs that execute a function, a class or even a group of classes.

I think that view is in agreement with AQ's view of : "One theory of mine is that once you get past the point of testing the state of the actor matches what you expect after each method call, you don't care about the internals. Your actor is correct if it is correct from point of view of all external sources. So send it some messages and see if it replies with the right messages."

I agree with Casey that the JKI VI Tester might work better for testing actors. For one thing the JKI VI Tests are also classes, you can create a VI Testing hierarchy that matches your classes hierarchy. Having the parent VI-Test testing the parent of your class hierarchy and the children tests testing the children in your hierarchy. Two happy families working together at every generation

Hope this helps.

Regards,

Fab

[AristosQueue (NI)]

At this point, I need to throw in my usual comment: The NI Unit Test Framework and the JKI VI Tester are not mutually exclusive tools. They are two different testing philosophies and, as such, provide different types of testing. Many people look at them and say, "Which one should I use?" The answer for some software may be "both." Personally, I prefer the philosophy of the VI Tester, which tends to treat the depths of your software as a black box and focus more on testing the VIs in a context.

Here at work, I don't use either -- we have a custom test suite that was created long before either tool was formalized, and the tests I write sometimes look like the "call this subVI with a bunch of inputs and test that you get the expected outputs" and sometimes they look more like "set up an environment, execute a process in that environment, see if the environment responded as expected." The former is more the UTF system, the latter is more the VI Tester system. There are pros/cons to both styles.

[rik_aspinall]

Thanks for your comments everyone. I was suspecting that the approach requires more than one type of testing, and so I can see use for all of your suggestions. I must admit to being a little intimidated by the JKI tester - I tried it a little a while ago and was taking a long time to get up and running with it (I wasn't in love with the amount of documentation either). I'd really appreciate it if someone could point to a simple AF project (perhaps one root Actor and one nested actor) to show me if you can spare the code.

As far as events and UI's are concerned, I can see the point that the actor should be an autonomous entity and so any UI action should ultimately be "put this message on the underlying actor's queue". I think by using batch messages I should be able to create such sequences, which goes a long way to helping test use cases for each actor (user sends message X followed by Y).

In the meantime I will plug away and let you all know shortly. I could probably share the example project when I'm done.

[FabiolaDelaCueva]

rik_aspinall wrote:

Thanks for your comments everyone. I was suspecting that the approach requires more than one type of testing, and so I can see use for all of your suggestions. I must admit to being a little intimidated by the JKI tester - I tried it a little a while ago and was taking a long time to get up and running with it (I wasn't in love with the amount of documentation either). I'd really appreciate it if someone could point to a simple AF project (perhaps one root Actor and one nested actor) to show me if you can spare the code.

I don't have an example with an AF project, but I do have some videos showing how to get started with voth the JKI VI tester and Unit Test Framework. Hope they help, they can be found here:

https://decibel.ni.com/content/docs/DOC-39109

Regards,

Fab

[TailOfGon]

AristosQueue wrote:

> One option would be to use a synchronous method to ask for the actor class

> data and get a reply, but then I don't want to release that into the wider world!

Create that method, mark it as Community scope, and friend your test harness VI. A variation is to create the Community scoped method and then create a reply message to call it, marking the reply message as a friend. Never ship the reply msg.

AristosQ: Is there an example doing this approach? Do we need Init Actor Queues FOR TESTING ONLY.vi to accomplish this? Do we need to create an actor just for unit testing? 

I am not planning to buy JKI VI tester nor NI Unit Test Framework at this point. What I would like to do is to simply create a text file listing up name of unit tests which is read by my simple framework. The framework will just execute unit test VIs one by one and create a report at the end.

[tyk007]

Just a note that VI Tester is not a commerical product; it is free.

[Suda苏打！]

-----