SLUGs Meeting on December 6th

[jlokanis]

We have a meeting coming up next week.  We did not have a topic lined up in advance so I am going to put together a presentation I have been thinking about doing for a future NI Week.  Here is a brief desciption:

Decoupling LVOOP Classes via Abstraction

Presented by John Lokanis (CLA, LabVIEW Champion)

Description:

One issue with LVOOP based applications is the static linking that can occur when passing information using objects between separate programs or dynamically loaded components.  The end result of this linking is each component will load the other’s entire codebase into memory as a dependency.  One way to alleviate this issue is to create abstract classes to decouple the class hierarchies from each other.

We will walk through an example of how this static linking happens and then demonstrate how to implement a decoupling solution.  This is directly applicable to designs that leverage some of the more powerful architectures like the Actor framework but also can apply to simpler applications that use components developed in isolation from each other.

If time permits, we can also have an open discussion some of the other advantages and disadvantages of LVOOP based designs.

Hope everyone can make time to attend the meeting.  I am looking forward to the the discussion portion of this talk and learning how others are solving this type of challenge in their LVOOP projects. 

[jlokanis]

Attached are the presentation and some of the example code from the meeting last night.

[styrum]

Going to all these lengths just to keep messages implemented as classes while the root of the coupling problem in the NI's current implementation of AF is that very implementation (of messages as classes). Namely, bundling the message data with the (re)action code that we want to run when the message is received (into the same class when it is exactly these two "things" that must be decoupled!). Such implementation is mandated by the command pattern used to implement actors in AF. A command pattern establishes a one-to-one correspondence between a message received and the handling action. And that is its main limitation. However, in reality, a reaction of a receiving actor, i.e. the action(s) run when a message is received (which constitutes an event for the receiver) can and does depend not only on which (type of a) message is received but also on the state of the receiving actor at that moment. That reaction to any external message is the sole business of the receiving actor. And the receiver's state at that moment is none of the business for the sender. Therefore, to achieve decoupling a message itself sent by some sender actor should never carry any methods to process it. Guess what? Not everything has be classes in LabVIEW even when you do use LVOOP. Notice how eventually you arrive to making messages fully abstract classes ("interfaces"), i.e. having a definition only for the data of a message and not really having any methods, and how these abstract classes are totally separate from the actors themselves (are not associated with neither receiver nor sender actor classes in any way as opposed to the "fully coupled" and "one way coupled" "ways"). So, what exactly was wrong with (much simpler way of) implementing messages as data only, sent via whatever asynchronous interthread/interprocess  communication channels (queues, TCP connections, Network streams, etc.) with those messages having a partially standardized format? Say, a cluster of a string which always carries the name(type) of the message plus a variant or a binary string which can carry any data. Or a variant that can carry any data and an always present attribute which carries the message name. Or (fully serialized) a binary string the format of the beginning of which is always the same and again always contains the name(type) of message. Oh, yeah, we can't use the beloved command pattern and dynamic dispatching then. But who says that it is the only or at least the best pattern that can be used to implement receiving and processing of the messages? Which, again, in any non-trivial case, varies based not only on the type of the received message but also on the state of the receiving actor at the moment the message is received.

[jlokanis]

As with everything, there are always tradeoffs.  The code that implements the actions taken when a message is received must exist somewhere and must be statically coupled to the actor/handler/loop that executes it to allow for manipulation the state data of the actor.

So, if your messages contain only data, the your actor must contain a case statement or similar construct with a case for each possible message action.

In command pattern (ex: AF) the message contains the execution and the actor itself can be just an instance of the parent message hander.  This means that you do not even need to write any code at all for the actor other than defining its state data and accessors.  The actor behavioral code exists in each massage.  This lets you add new messages and thus new behaviors without editing the actor's class.  You can even load these messages at runtime to extend the behavior of the actor.  But again, the messages are statically linked to the actor in order to have access to the state data. In a monolithic application, this is not an issue.  But when sending messages between applications this causes the issue we were trying to solve with abstract classes.

In the process of decoupling these messages, it is true that we are removing the execution of the message from the message class and moving it to the concrete implementation of the class of execution methods that the messages now call instead.  This is now similar to the large case statement that is maintained in the data-only message architectures, but now you are maintaining a set of methods in a class instead.  Keep in mind that you only need to decouple the messages that cross the application boundary.  The rest of you messages can remain coupled to their actors without penalty, so the extra effort required is limited to a subset of message classes.

So, what are the advantages of command pattern?  Well, the ability to have strictly type def'd data that is unique to each message is one of the main attractions.  This allows you to avoid runtime errors when the data is not constructed in the proper format by the sender.  I suppose you could always use a variant to send your data instead but then you again need a type def to construct and then cast the data on both ends.  In command pattern, the message class simply takes the place of that type def.

Another advantage is the ability to contain the data definition and execution within a single class.  This can make maintenance easier since you do not need to locate the code that defines the data and then the code that operates on it before making a change.
Additionally, extension is easy by simply created additional message classes.  You can even clone similar message classes from other actors and reuse them with small modifications.  Also, you can create messages for parent actors that the child actor can use for similar groups of actor to extend reuse.  One example of this is an exit message that I include in my architecture that every actor is able to execute.  It even allows for portions of the exit message to be overridden to include custom shutdown processes for an actor if needed.

So, in the end there are many solutions to messaging between actors, each with its own tradeoffs and benefits.  You really need to look at the type of application, the complexity and the future expansion and maintenance requirements to make a good selection of architectures.  One solution does not fit all nor is any solution better than others in most circumstances.

The goal here was to understand and solve the decoupling issue that arises from command pattern messaging architectures, not to state that command pattern is always the best option when choosing an architecture.

[styrum]

"the ability to have strictly type def'd data that is unique to each message is one of the main attractions." And how exactly is this different from having a data typedef, external with respect to but used by both sender (to flatten/encode) and receiver (to unflatten/interpret/decode)? How can the programmer possibly send a wrong data with a message? Well, if that message even carries any data in addition to just the name/type of the message. In any case the plumbing code in the receiver can and should process only the messages it knows and ignore (or return an error to the sender or directly to user/UI) for all the others. Naturally it should do the same when decoding/interpreting/casting message data causes an error.

[styrum]

"Another advantage is the ability to contain the data definition and execution within a single class". My main point is that is NOT an advantage! It is the very source of the problem. It is a business/responsibility of the receiver to locate the code to run based not only on the (type of) the message received but also on its current state (which is none of the business for the sender!) Sorry for repeating myself but this is the essence of the problem.

[styrum]

A solution I arrived at is to not have state and/or message dependent branching in the code of the actions themselves whatsoever. How? If you think about it, the entire information about which action(s) to run when which messages are received while in which state plus which state should we go to next, is just that, "information". Which means it does not necessarily have to be code. It can be a data structure, the contents of which varies but the processing code for which can remain static, the same in any actor. It just has to be a little bit more complicated that the command pattern. But that doesn't matter if it is always the same! That "behavior data" structure and the plumbing code to process it can support not only flat but also hierarchical state machines (statecharts). Yes, I am hinting at my LabHSM toolkit. I guess it is time to update it with LVOOP.

[jlokanis]

---

@styrum wrote:

"Another advantage is the ability to contain the data definition and execution within a single class". My main point is that is NOT an advantage! It is the very source of the problem. It is a business/responsibility of the receiver to locate the code to run based not only on the (type of) the message received but also on its current state (which is none of the business for the sender!) Sorry for repeating myself but this is the essence of the problem.

---

What you call a problem is actually a feature that was intended to work this way.  Since this design is the heart of Actor Framework and that was created by one of the primary developers on the LabVIEW team who has much more software engineering experience than both of us combined, I am going to accept it as a valid architecture.

You may not prefer it in your solutions but stating that it is a problem is not a fact, just an opinion.  Every architecture has its advantages and disadvantages.  If you perceive this as a disadvantage, then don't use it.

[jlokanis]

---

@styrum wrote:

"the ability to have strictly type def'd data that is unique to each message is one of the main attractions." And how exactly is this different from having a data typedef, external with respect to but used by both sender (to flatten/encode) and receiver (to unflatten/interpret/decode)? How can the programmer possibly send a wrong data with a message? Well, if that message even carries any data in addition to just the name/type of the message. In any case the plumbing code in the receiver can and should process only the messages it knows and ignore (or return an error to the sender or directly to user/UI) for all the others. Naturally it should do the same when decoding/interpreting/casting message data causes an error.

---

I stated exactly that in my response.  Having a typedef to validate data on both ends has the same effect as using a message class.  It leaves you with code that must be included on both ends of the message transmission.  Your statement above seems to imply that you would prefer runtime errors when attempting to translate misconstructed message data.  That is certainly one way to go with an architecture.  Others prefer to minimize those and instead detect the issue at dev time, thus requiring a typedef or message class.  It all comes down to personal preference and your environment.  Larger teams would more likely benefit from the strict type def while lone developers could deal with looser restrictions on message format since they are coding both ends of the transaction themselves.

[jlokanis]

---

@styrum wrote:

A solution I arrived at is to not have state and/or message dependent branching in the code of the actions themselves whatsoever. How? If you think about it, the entire information about which action(s) to run when which messages are received while in which state plus which state should we go to next, is just that, "information". Which means it does not necessarily have to be code. It can be a data structure, the contents of which varies but the processing code for which can remain static, the same in any actor. It just has to be a little bit more complicated that the command pattern. But that doesn't matter if it is always the same! That "behavior data" structure and the plumbing code to process it can support not only flat but also hierarchical state machines (statecharts). Yes, I am hinting at my LabHSM toolkit. I guess it is time to update it with LVOOP.

---

I think some of the confusion may be arising for the fact that you are treating actors like state machines.  While it may be possible to implement traditional state machines as actors, that is only one possible type of behavior they can take on.  In most of my code they are simply parallel processes that either execute the business logic of the application or act as an interface to the UI portions of the application.  They do not have defined states or defined transitions between states like a true state machine would, even if they sometimes do have persistent data that they can store and recall when acting on an incoming message.