Need some guidance on AF project

[Taggart]

Sequencing - thanks for helping me to clarify that.

My situation falls more into

2.  If you want the actor to step through a series of steps, and you don't want those steps interrupted, offload the sequence into a helper loop.

My problem is I would like to able to abort the process if necessary.  I guess I'd just have to implement some sort of communication mechanism between the MHL and the helper loop to interrupt the helper loop. 

As far 2 things needing the pump.  There are 2 processes that are independent except for the fact that both need to pump some chemicals and there is only 1 pump in the system that handles both pumping tasks.  To do one or the other you just have to switch some valves,but you can't pump both chemicals at the same time.  Since they are otherwise independant it seemed like making them seperate actors would be a good choice.  The two tasks are otherwise completely unrelated.

So another option I was thinking was to have a seperate pump actor and up the tree there would be a controller actor that called both processes and the pump.  Then each process could send a message up to the controller requesting use of the pump and the controller arbitrates who has access to the pump when.  Does that make more sense?

[justACS]

cirrusio wrote:
An FGV would be a bad choice here, because it does exactly the same thing (creating a peer-to-peer connection between actors), but it does so in a very non-AF manner, which will complicate debugging.
And now you have built an error logger that is usable only in the context of the Actor Framework.  Error loggers are used in many applications; unless you are intending to only create AF projects, you have just increased your work load (which is good I guess if you bill by the hour).

The alternative is to have actors that *must* use *only* your special error logger.  AF and ESF are different metaphors for communication.  One of the strengths of AF is compatibility - since actors use the same communications scheme, it is easy to use them together (though you'll want to use low- or zero-coupling for maximum benefit).  If you need to mix the two, then wrap the ESF item in an actor.

(which is good I guess if you bill by the hour).

I despise writing code to maximize one's billable hours.  Even when I was consulting, and chasing those billable hours, I avoided the practice.

And, I am not clear on how this makes a peer-to-peer connection - your actor depends on the FGV, not the other way around, so the use of the FGV does not effect coupling between actors.  

If you create an FGV to contain an actor's enqueuer, and then call that FGV in multiple actors, you have coupled them.

But that's beside the point.  The connection is peer-to-peer, because Actor A can now send messages to Actor B, where B is neither the caller nor the nested of A.  That would, in fact, be the point of creating the FGV, would it not?  To bypass the tree hierarchy?

A little peer-to-peer is OK, btw, but it adds to overall complexity in the application, because now Actor B can get messages from three actors (its caller, its nested, and Actor A).  It can make the message traffic harder to follow.

And with regards to complicating debugging - how exactly do FGVs effect the complexity?  In this case (a logger), you have a fairly simple execution instance that has to do one thing and one thing only - maintain a list of messages generated by the system.  

An FGV holds shared data, and thus creates a communications back channel in your AF system.  That is one more potential for race conditions.  Just getting a fixed piece of data isn't too bad, at least until someone starts to think they can use the concept to store an enqueuer that they expect will change,or that they can piggyback other data with it.  It's an unnecessary short cut that can lead to real trouble later.

I'm actually not very fond of ESF, except possibly for its original intended use case - an extensible smart reference for instrumentation.  Anything that you can do with ESF you can do with an actor, and actors are data flow safe.  ESF, like most reference-controlled resources, simply is not.
I will disagree with this - my opinion is that, regardless of whether you use the ESF or some other method of implementing IOC, the AF is just the place where you would want to use this.  Anywhere you share a reference amongst multiple application instances (actors) I think that this principle excels.  If you choose to go with an actor, you will have to be very careful about how you launch and close your system otherwise you will end up with a case where the system is trying access a resource that is no longer is available.  

Shared references like ESF are a real problem in systems with high concurrency, because they guarantee either performance issues (when you lock the resource to modify its data) or race conditions (when you copy the data, modify it, and then check it back into the resource - usually to avoid performance issues).  They also breed deadlocks (when two concurrent processes both need the same two resources).  AF reduces these issues because the messaging system protects the data in an actor, and the actor arbitrates how that data is modified.  Wrap your resources in actors, and share actor enqueuers instead.

And if you want to talk about difficult to debug, race conditions in the AF are the definition given the rentrancy of all core AF VIs. 

Debugging can be harder in AF.  It's worth your time to unit test your actors individually before adding them to your system.  Also, have you checked out the DETT support in AF?

This is the root of the problem with implementing an AF structure that is not tree-like in nature - who is going to be in charge of the actor that is responsible for logging?  The launching actor will be.  But why?

Who else?  The logger isn't functionally owned by any of the other subsystems in the application.

Why is the logger not waiting for everyone to exit (including the controller) so that nothing is left dangling?

It's a straightforward task to code an AF system to guarantee that a particular actor is the last to shut down, though it gets harder the farther down the tree it is.

[justACS]

Taggart wrote:

Sequencing - thanks for helping me to clarify that.

My situation falls more into

2.  If you want the actor to step through a series of steps, and you don't want those steps interrupted, offload the sequence into a helper loop.

My problem is I would like to able to abort the process if necessary.  I guess I'd just have to implement some sort of communication mechanism between the MHL and the helper loop to interrupt the helper loop. 

Offload the sequencer, and have one of its steps be "Abort".  You can (and should) store the sequencer's queue in your actor's private data.  You can send an "abort" message from any actor method, and you can send a "stop" or kill the queue to stop the sequencer.

As far 2 things needing the pump.  There are 2 processes that are independent except for the fact that both need to pump some chemicals and there is only 1 pump in the system that handles both pumping tasks.  To do one or the other you just have to switch some valves,but you can't pump both chemicals at the same time.  Since they are otherwise independant it seemed like making them seperate actors would be a good choice.  The two tasks are otherwise completely unrelated.

So another option I was thinking was to have a seperate pump actor and up the tree there would be a controller actor that called both processes and the pump.  Then each process could send a message up to the controller requesting use of the pump and the controller arbitrates who has access to the pump when.  Does that make more sense?

Based on this, I'd say you'd have a "chemical pump" actor that drives your pump and the associated valves.  Then you'd have messages like "switch to Shiner Bock", "switch to Guinness", and "fill pint glass".  The two processes wouldn't know anything about the pump or valves, just that they need their respective brands of beer.

[Taggart]

This is exactly what i was thinking.  I think we are both on the same page (at least the same book).

[cirrusio]

The alternative is to have actors that *must* use *only* your special error logger.  AF and ESF are different metaphors for communication.  One of the strengths of AF is compatibility - since actors use the same communications scheme, it is easy to use them together (though you'll want to use low- or zero-coupling for maximum benefit).  If you need to mix the two, then wrap the ESF item in an actor.

Not following this.  Seems that actors *must* use *only* one logger anyway, so what's the point?  But that is besides the point - I am using  ESF here as an example of a design principle.  That principle is about more than just communication - it has to do with who controls programatic flow.  And that is what I was getting at with who's in charge...

If you create an FGV to contain an actor's enqueuer, and then call that FGV in multiple actors, you have coupled them.

But that's beside the point.  The connection is peer-to-peer, because Actor A can now send messages to Actor B, where B is neither the caller nor the nested of A.  That would, in fact, be the point of creating the FGV, would it not?  To bypass the tree hierarchy?

A little peer-to-peer is OK, btw, but it adds to overall complexity in the application, because now Actor B can get messages from three actors (its caller, its nested, and Actor A).  It can make the message traffic harder to follow.

OK, I misunderstood your use of FGV here.  By no means am I suggesting that you create an FGV containing an enqueuer.

An FGV holds shared data, and thus creates a communications back channel in your AF system.  That is one more potential for race conditions.  Just getting a fixed piece of data isn't too bad, at least until someone starts to think they can use the concept to store an enqueuer that they expect will change,or that they can piggyback other data with it.  It's an unnecessary short cut that can lead to real trouble later.

Once again, I think that I misunderstood what you were suggesting with an FGV.  Not suggesting storage of enqueuers.

Shared references like ESF are a real problem in systems with high concurrency, because they guarantee either performance issues (when you lock the resource to modify its data) or race conditions (when you copy the data, modify it, and then check it back into the resource - usually to avoid performance issues).  They also breed deadlocks (when two concurrent processes both need the same two resources).  AF reduces these issues because the messaging system protects the data in an actor, and the actor arbitrates how that data is modified.  Wrap your resources in actors, and share actor enqueuers instead.

Meh. Remember, we are talking about logging - and I don't see any of these as an issue when you are referring to logging functionality.  If one piece of code decides to close a file reference in your ESF and open a new one, why care?  I think in this context you are overselling deadlock and performance (if you want performance, don't call a logger in time critical loops).  But, I do see your point.

Who else?  The logger isn't functionally owned by any of the other subsystems in the application.

Why is the logger not waiting for everyone to exit (including the controller) so that nothing is left dangling?

It's a straightforward task to code an AF system to guarantee that a particular actor is the last to shut down, though it gets harder the farther down the tree it is.

But, the logger is owned by the actor that spawns it.  And that is the point.  Yes, you can guarantee that actors will close in a particular order, but as you grow your system, possibly adding actors as requirements change, you have to continue to maintain that order.  Why not just go ahead and let the logger decide when to quit?

(which is good I guess if you bill by the hour).

I despise writing code to maximize one's billable hours.  Even when I was consulting, and chasing those billable hours, I avoided the practice.

Lol. I am a consultant and personally I have enough work so that I don't have any desire to do that.  And that was the point of that ridiculous statement - I never enjoy increasing my workload. 

[justACS]

cirrusio wrote:

The alternative is to have actors that *must* use *only* your special error logger.  AF and ESF are different metaphors for communication.  One of the strengths of AF is compatibility - since actors use the same communications scheme, it is easy to use them together (though you'll want to use low- or zero-coupling for maximum benefit).  If you need to mix the two, then wrap the ESF item in an actor.

Not following this.  Seems that actors *must* use *only* one logger anyway, so what's the point?

You were claiming that writing the logger as an actor was somehow limiting, because now that logger can only be used in AF systems (not actually true, btw).  I was countering with the observation that requiring your actors to use a logger written in ESF binds those actors to *that* interface style.  I maintain that you will get more interoperability by staying within the framework, and writing shim code for those times when you really want to use something written in a different framework.

But that is besides the point - I am using  ESF here as an example of a design principle.  That principle is about more than just communication - it has to do with who controls programatic flow.  And that is what I was getting at with who's in charge...

And now I start to see the delta in our positions.  It is fair to say, in an actor system, that *no one* controls progamatic flow.  Actors are self-deterministic, concurrent processes.  We impose a certain order with the tree hierarchy, but even there, the actors do their own things pretty much when they want to.  (The hierarchy gives you control over which actors can exchange messages, but the actor has final say over what it does with any message it receives.)  So, in a very real sense, communication and messaging relationships *define* an actor system.  I see by-reference objects as working against that core concept, because they introduce a communications back channel that is problematic in a concurrent, dataflow environment (for the reasons I've mentioned).

OK, I misunderstood your use of FGV here.  By no means am I suggesting that you create an FGV containing an enqueuer.

So we're in violent agreement here.  Good. 

Shared references like ESF are a real problem in systems with high concurrency, because they guarantee either performance issues (when you lock the resource to modify its data) or race conditions (when you copy the data, modify it, and then check it back into the resource - usually to avoid performance issues).  They also breed deadlocks (when two concurrent processes both need the same two resources).  AF reduces these issues because the messaging system protects the data in an actor, and the actor arbitrates how that data is modified.  Wrap your resources in actors, and share actor enqueuers instead.

Meh. Remember, we are talking about logging - and I don't see any of these as an issue when you are referring to logging functionality.  If one piece of code decides to close a file reference in your ESF and open a new one, why care?  I think in this context you are overselling deadlock and performance (if you want performance, don't call a logger in time critical loops).  But, I do see your point.

Sometimes we seem to be talking about "just a logger", and sometimes about broader implications for programmatic control.

If we are just talking about a logger, then I agree that the issues I raise are not likely to become a problem.  But "not likely to become a problem" and "not the best way to do that" are not mutually exclusive.  I'd prefer to see the OP stick to the framework where he can, because that will serve him best in the long run.

But, the logger is owned by the actor that spawns it.  And that is the point.  Yes, you can guarantee that actors will close in a particular order, but as you grow your system, possibly adding actors as requirements change, you have to continue to maintain that order.  Why not just go ahead and let the logger decide when to quit?

Because the cost of allowing it to do so in the manner you describe (i.e. some form of by-reference singleton) is potentially very high in an AF system.  Probably not for a simple logger, but it doesn't cost more to do it right.  The logger, by its nature, is going to be somewhat offline from the main thrust of development in your application.  Realistically, is it going to change all that much over the life of your project?

[justACS]

Taggart wrote:

This is exactly what i was thinking.  I think we are both on the same page (at least the same book).

Awesome!

[cirrusio]

Because the cost of allowing it to do so in the manner you describe (i.e. some form of by-reference singleton) is potentially very high in an AF system.  Probably not for a simple logger, but it doesn't cost more to do it right.  The logger, by its nature, is going to be somewhat offline from the main thrust of development in your application.  Realistically, is it going to change all that much over the life of your project?

Having done this over and over now I have found that the cost of trying to find why something did not shutdown properly in the AF to be quite high in and of itself, so in this instance I will maintain that allowing an object to have programatic control of itself to be a good solution to this problem.  Despite functional independence, actors other than the top-level actor generally depend on other actors to let them know when it is appropriate to shut down. 

You were claiming that writing the logger as an actor was somehow limiting, because now that logger can only be used in AF systems (not actually true, btw).  I was countering with the observation that requiring your actors to use a logger written in ESF binds those actors to *that* interface style.  I maintain that you will get more interoperability by staying within the framework, and writing shim code for those times when you really want to use something written in a different framework.

Shim code seems to imply limited use and I would file this under code smell.  My experience is that the AF is great in many applications and complete overkill in a lot of others.  Despite this I still log errors and messages and don't want to shim everytime I do that (really just want to drop it in).  But, I will go ahead and counter for you and then just agree to disagree - you can wrap the reusable functionality in an actor.

Sometimes we seem to be talking about "just a logger", and sometimes about broader implications for programmatic control.

If we are just talking about a logger, then I agree that the issues I raise are not likely to become a problem.  But "not likely to become a problem" and "not the best way to do that" are not mutually exclusive.  I'd prefer to see the OP stick to the framework where he can, because that will serve him best in the long run.

I am not certain what the first statement is refering to.  IOC is all about control of programatic flow. It is a solid design principle (many frameworks take advantage of this - Spring in Java comes immediately to mind), but I am not suggesting that it is appropriate all of the time.  However, when you are referring to a shared reference (which I am not clear on why you make a distinction between a file reference and a DAQmx reference and a network connection reference etc etc) I think that IOC is a perfectly reasonable way of handling that reference. 

Personally, I will leave the "not the best way to do that" judgement up to the developer themselves - best practices seem to change with a shift in the wind and it is likely we will all soon find ourselves do something that is not a best practice (Darren spent a whole talk at NI Week that seemed to grind against NI best practices).  I am merely putting forth a suggestion for taggart on what has worked well for me...

[Taggart]

NiACS,

          Just looked at your state pattern implementation.  I like it.  There's a lot there to digest and wrap my head around, but I think I have the basics.  You mentioned in your post to be careful about state pattern and sequencing.  What if my actor kind of has a bit of both? For example:  I have a process which when it runs, follows a sequence.  However when it is not running (ie. idle), they would like to be able to manually manipulate the pumps, valves, and motors associated with the process.  So I was thinking that I would have an actor, say process context (inherited from the state actor) and use the state pattern with an idle and running states (as children of the process context).  The sequence would run in a helper loop. In the entry into running state it would start the sequencer.  And then while in the running state, the only message it would process would be abort, which would somehow abort the helper loop.  When the sequence is over, the last step would be to send a sequence overmessage to the parent Actor core, which would then reset the state to idle.  Then in idle, it would accept all the manual commands.

Just wanted to make sure that all makes sense.  What do you think?

[justACS]

That would certainly work.  Put the actual hardware calls in protected methods that you can call from either public methods of the actor or from your sequence.

Here's another thought, though.

Put the hardware interactions (pumps, valves, motors, etc.) into an actor that *only* serves as an interface to these items.  Make a sequencer actor that launches your interface actor as a nested actor, and then *only* runs a sequence, making calls to the interface actor as needed.  Then, make a stateful actor with "manual" and "running" states.  When you enter the manual state, this actor launches the interface actor and invokes its methods as needed.  When you exit manual, it shuts down the interface.  When you enter the running state, the stateful actor launches the sequencer, which launches the interface actor.  When the sequence is done, the sequencer actor shuts down and sends a Last Ack to the stateful actor, which is its cue to exit the running state.

Starting and stopping actors involves minimal overhead, but there is some.  (I haven't benchmarked it, but it's several hundred ms.)  If you don't want to shut down the interface actor, you can keep it alive to call directly from the stateful actor, and then just pass its queue to the sequence actor when you launch it.

Bonus:  Testing gets easier, because it's easier to test your actors individually than as a set.