Actor Framework limited by not having an Actor Arbiter/Administrator

[PrimaryKey]

Hi, I had an idea that it would make actors more powerful if there was some Administrator that keeps track of all the actors in current process and has the enqueuer references to those. With a construct like this we could:

1. Obtain an actor enqueuer reference by name from an actor system.

2. Deploy actors on a remote system that runs the Actor Administrator.

3. Keep track of the messages sent between actors for debugging purposes - this could be turned off when performance needs to be optimized.

4. Have no hard boundary between where an actor is located. Local and remote messages would work exactly the same.

5. Be able to provide global settings for an actor system.

6. Make Actor Framework easier to integrate with other languages - tcp/udp strings communication.

7. To many benefits to list here...

All of these are natural consequences and would follow very closely to what is possible with any other (non-LV) actor framework out there ex. Akka. In Akka a local address to an actor can be /MyBaseActor/MyActor1 and remote address can be akka.tcp://MyActorSystem@localhost:9001/MyBaseActor/MyActor1 and these two would work exactly the same way. No more wasting time with network endpoints etc. 

Why yes? Why no?

[AristosQueue (NI)]

Almost all of the things you are asking for are things that are explicitly on the "AF was designed NOT to allow that" list. Most notably:

>Obtain an actor enqueuer reference by name from an actor system.

The ability to arbitrarily connect actors is something I have prevented in every possible way. In the design philosophy of the NI Actor Framework, it is a bug, not a feature, to allow an actor to be looked up by name. That violates the actor tree and -- worse -- violates the actor tree in a way that is not under the control of the section of the tree where the actor resides. That sort of violation is the major causes of failure for most high parallel applications that I studied when creating the AF -- it leads directly to race conditions, deadlocks, and other "cannot regularly reproduce" bugs.

The AF goes out of its way to prevent that kind of contamination of message paths. All of this is discussed at length in the shipping documentation -- on the Getting Started Window, choose "Create Project" and pick the "Actor Framework" sample project. In your newly created project, you'll find a .html file with lots of details about the value of the actor tree and why this kind of communication is disallowed.

You are welcome to build such a thing on your own, but it will not be part of the official AF ever.

> Be able to provide global settings for an actor system.

That is undesirable in the extreme. No actor should have any knowledge of any actors other than its direct caller and its direct nested actors. Any settings that a nested actor needs should be supplied by its caller because only the caller knows the needs of that particular actor. Just because there is a setting for the system as a whole doesn't mean that every actor needs that setting -- making them all aware of the setting creates easy-to-trigger race conditions if the setting ever changes, creates project-wide code churn if the setting is re-coded, and may make it impossible to create actors that bridge from one system to another (where the remote system is operating under a different setting). From every use case I have ever looked at, it's a bad idea to ever create that kind of global broadcast.

> Make Actor Framework easier to integrate with other languages - tcp/udp strings communication.

Such integration is easy and straightforward. There is lots of documentation in this forum and other places about how to handle network communication within an actor hierarchy.

> 7. To many benefits to list here...

I know of none except "simpler for a programmer to write code that he/she THINKS works (but secretly doesn't and will fail when least expected)." 🙂

You're welcome to argue with me about this -- many users do. So far, with every user who has proposed this sort of system to me, for any non-trivial system (meaning the sorts of systems where name lookup is even useful, because when there's just a few actors, both systems are equally simple), I've been able to find rare-but-significant race conditions in their code caused by arbitrarily connected actors, so I'm pretty entrenched in my position at this time. 🙂

[jlokanis]

I would agree with AQ on most of his points with one exception: Allowing actors to direct message each other outside of the tree will not lead to race conditions as long as every actor behaves in a completely asynchronous fashion. This means no 'wait for reply' type messages.  Ever.  In fact, if an actor needs to make a request and get a response, it should be designed to act on the response in the exact same manner regardless of whether it originated the request in the first place or not.

This is how I architect my AF-ish framework and it works well for me.  But I am sure AQ will let me know if I am forgetting some other corner case that will prove me wrong.

I do think that the real issue the OP is trying to solve is how to manage and debug any AF-like architecture in LabVIEW.  This is an area where we do need better solutions than what we have now.  Personally, I find that I manage to debug complex interactions by sheer will alone and mostly in my head.

I like to think I do actor programming the way Hawking does physics. 🙂

(But I wish there was a better way...)

[PrimaryKey]

Making things more difficult does not prevent programmers from doing them. They will still break your approach, whenever they feel like it. I have seen many Actor Framework based projects where the Enqueuers are shared all over the place or even globally. I agree that tree structure is desirable but you cannot prevent people from breaking it, especially because sometimes you have to. This means that not having an arbiter makes this more difficult when things do break, and you try to debug them. One of the first extremely useful tools for AF was the debugged actor that creates a global list of actors. Why do you think that is?

I find your arguments valid only if we live in a perfect world where everyone programs AF projects like you envisioned. There are many AFs for other languages (Akka, CAF) which all realize that its better not to fight with programmers, but instead allow them to make mistakes, and give them good tools to understand and fix those mistakes. An actor arbiter or an actor system tracker would be a good tool. You still wouldn't have to use it, in the same way like you don't have to use debugged actor, but yet it would be available if needed.

Another thing is the artificial complexity introduced by forcing people to follow current way of doing things. Don't you think it is a significant barrier in learning and adopting AF? 

There are two options:

1. Follow AF philosophy immediately - completely change the way you think about programming big apps - very difficult - results in people turning away from AF to use DQMH or Messanger Library (if they continue to use LV)

2. Slowly learn from your mistakes, thanks to using great debugging tools like an arbiter - stay with AF

Add to this networking it only gets more complicated.

This philosophical argument does not convince me. I would like many more people using actor framework, and this is limiting adoption. The more restrictive you make a tool the less people will use it.

If you had the control over a tool like an arbiter, you could also enforce some protection mechanisms, keep track of Reply Messages to prevent race conditions, keep track of actors that get lost in memory, graph the activity and a 100 more things that are currently either not possible or require custom solutions. Wouldn't you like that? Do you not see value in that? 
You could still write your actor projects in the same way as you do right now. We could discourage people from using direct communication, but we could make it available in a controlled manner, like we teach right now.

[PrimaryKey]

You should not have to be a genius to use AF.

[Oli_Wachno]

Hi Piotr,

my thoughts on that:

---

@PrimaryKey wrote:

Making things more difficult does not prevent programmers from doing them. They will still break your approach, whenever they feel like it. I have seen many Actor Framework based projects where the Enqueuers are shared all over the place or even globally. I agree that tree structure is desirable but you cannot prevent people from breaking it, especially because sometimes you have to. This means that not having an arbiter makes this more difficult when things do break, and you try to debug them. One of the first extremely useful tools for AF was the debugged actor that creates a global list of actors. Why do you think that is?

Point given, programmers (humans) learn my making and (hopefully) fixing errors. Yet, not all of us have to make the same mistakes, this is why there are Best Practices. Like not using local variables in unappropriate places. Nevertheless, everybody is free to ignore those. So if anyone feels the need to implement different things, feel free to do so.

But you they have to be aware, there is a good chance of having to eat dirt as a consequence.

Another thing is the artificial complexity you introduce by forcing people to follow your way of doing things. Don't you think it is a significant barrier in learning and adopting AF? 

There are two options:

1. Follow AF philosophy immediately - completely change the way you think about programming big apps - very difficult - results in people turning away from AF to use DQMH or Messanger Library (if they continue to use LV)

2. Slowly learn from your mistakes, thanks to using great debugging tools like an arbiter - stay with AF

Nothing wrong about using DQMH instead, if developers don't get along with the AF, it is ok to use DQMH, which offers a really convenient way of programming.

Yet comparison of frameworks is off topic here.

I am aware that debugging can be is a challenge. Maybe an arbiter would make it easier. Yet the possibility of misusing it (just like local variables) is a severe thread.

That's why I go with AQs arguments. If the restrictions imposed by the design decisions would eventually lead to not using AF anymore, it's called evolution.

Now. From Philosophy back to work

[Oli_Wachno]

It works without being one 😉

[PrimaryKey]

Looks like I will have to write it myself 😄

[AristosQueue (NI)]

> Another thing is the artificial complexity you introduce by forcing people to follow your

> way of doing things. 

Ah. "Artificial complexity" and "forcing people to follow your way of doing things" -- those are word-for-word quotes I heard from multiple C programmers in 1994 about private scope and the entirety of object-oriented programming in C++... and also what many G programmers said about OOP when it came out in 2006. You are wrong in the same way they were wrong: The complexity is not artificial. The complexity is real in the applications being written!!! Hyperparallel, massively communicating modules is complex. The introduction of scope just organizes that complexity. It does not create it.  The actor tree is basically just another form of access scope.

> Don't you think it is a significant barrier in learning and adopting AF?

Nope. It is actually the major reason why AF is being so widely adopted: AF is usable. Seriously -- I cannot count how many people come to me grateful for those limits. They finally got a major system working because the AF structured their code well enough that they could encode their project requirements without injecting a wealth of bugs they had no idea how to contain. Users can choose to not use OOP, but the principles of encapsulation apply even when they just use procedural programming, but they are responsible for tracking it (see all the old LLBs that had not-top-level VIs with red icons and context help that said, "Do not call from outside this LLB!"). Same situation occurs when messaging between actors -- users don't have to segregate the messages, but in my observation, everyone ends up tracking on paper all the message routes that are possible in order do code maintenance. Better to let the framework enforce the routes. Saves a lot of work!

Prior to AF, I went on customer visit after customer visit to companies. I saw both novice teams and teams full of skilled CLAs and full CS degrees. All ends of the spectrum were frustrated with massive parallel systems that developed the weirdest bugs that didn't reproduce regularly. Several of them e-mailed me their frameworks, and I found the same issues in all of them, and, at the root of all of them, was the listener pattern, creating violations in how they *thought* their system operated.

I worked directly with many of them to create the AF, operating under the hypothesis that the listener pattern is a bad idea because it violates the encapsulation of message scope without any core benefit. That hypothesis turns out to be correct... we can eliminate the source of entire classifications of bugs without loss of either performance or correctness, and increase the maintainability of code while doing so.

AF was the core of four of the keynote state applications at NIWeek this past year. Even those frameworks that allow for non-tree structure are discouraging the use of arbitrary by-name listener patterns in documentation and describing the very specific message routing patterns that are allowed, like a looped pipeline (in a tree, A behaves like a leaf with respect to the rest of the tree, but it uses a pipeline to do its work where A sends to B sends to C sends to D and results are sent back to A).  Often, such patterns can be set up without resorting to a name or global lookup.

AF allows construction of such patterns. It does not encourage them. Such patterns turn out to be necessary only rarely. That is actually a surprising result to me. I did not expect the actor tree to be so robust when I started on the project. I thought it would be the core connection between major modules, but we would have listener meshes at the tips. Turns out we can eliminate the meshes entirely without downside.

> Looks like I will have to write it myself 😄

My job is to facilitate users writing good code, not prevent users writing bad code. What bugs you choose to code in your own projects are your own choices. 🙂

[AristosQueue (NI)]

---

@PrimaryKey wrote:

You should not have to be a genius to use AF.

---

AF is designed with novices in mind. It has tested quite successfully on relatively junior programmers who have been tasked with writing massive test systems that seem so simple on paper but have this problem of needing many tasks to run in parallel and all the subtle gotchas that come with that.

You are infected with the previous paradigm. Exactly the same as procedural programmers learning OO for the first time. Exactly like polling loop authors exposed to the Event structure. You must unlearn what you have learned if you want to be more successful. That's not to say you will be unsuccessful if you do not -- clearly, you can write listener code successfully. All I'm saying is that we have found a better way. Even if you never use the AF and go write your own messaging system (as many have done), I hope you recognize why the AF works the way it does and try to maintain its principles in your own code.