Actor framework for data acquisition and data process pattern

[champion2019]

Good to know. My project needs 5kHz message rate. So I think AF message is not competent for that.

[Taggart]

---

@wrkcrw00 wrote:

Mixing and matching frameworks is ok, you just need to understand the complexities and hopefully know what you are doing .

---

Exactly!

[paul.r]

The extra overhead of the Actor Framework queue system is pretty minimal. I work on large actor based applications where a large percentage of messages/inter actor communications are piped through a mediator - actors publish data (send a mediator an AF message), mediator sends the data to a dedicated topic actor for that particular data topic, then the topic actor sends the data to its list of subscribers. Many of these applications have hundreds of messages being sent between actors every second, and I have never seen a performance issue that was related to the AF queues. 

[champion2019]

---

@paul.r wrote:

The extra overhead of the Actor Framework queue system is pretty minimal. I work on large actor based applications where a large percentage of messages/inter actor communications are piped through a mediator - actors publish data (send a mediator an AF message), mediator sends the data to a dedicated topic actor for that particular data topic, then the topic actor sends the data to its list of subscribers. Many of these applications have hundreds of messages being sent between actors every second, and I have never seen a performance issue that was related to the AF queues. 

---

Really? Then how about 5 kHz messages per second? 

[wrkcrw00]

For some systems higher message rates work. As for my Q creation on the fly, my entire actor system is defined from a set of config files. The system is highly configurable because the same SW runs dozens of unique test systems. Within the config file, I define a set of processes (actors) and the connections between them (Q's). My application creates these Q's on the fly and hands them to the appropriate process. Note that in my pipelined data stream the data sets are all transferred in the same format, so I don't have to worry about message classes and formats. I needed the transfer to be as fast as possible. 

[paul.r]

@champion2019 wrote:

---

---

Really? Then how about 5 kHz messages per second? 

---

In this really simple example, I am able to send 50,000 messages to an actor in about 200ms. The time from when I send the last message, to when the actor receives and processes the last message, is about 1 second. 

The send rate in a real world use case will depend on what is in the payload - sending messages with a lot of data will be slower. Additionally, if the actor is slow to process the messages, the queue buffer will need to be resized periodically, which will add time to the send.

So, yes, the actor framework is more than capable of handling 5000 messages/S.