Pipelining data though Actors

[OneOfTheDans]

One cool use case for the AF that I've imagined but not implemented are standalone data processing Actors. You'd feed in a waveform (or whatever); the Actor performs one action (e.g. smooth the waveform with an LPF); then forward it to the next data processor. As I've imagined this, the data processors would be chainable and standalone, so you could easily reuse this across projects by stringing together new data processing paths depending on your application.

BUT, I can't figure the best way to connect all the Actors. I think there are only 2 options...

1. Processor Actors send their results back to the Caller. A Master Actor owns all the Processor Actors and traffic-directs the entire data flow map. This respects the AF hierarchy and keeps each Processor simple, but the Master becomes a giant synchronous bottleneck negating the performance benefits of pipelining.
2. Processor Actors send their results forward to a "Next" enqueuer rather than back to Caller. This cuts across the AF hierarchy and makes the Processor Actors more complicated (e.g. What if the Next enqueuer goes invalid? Could Next be >1 destination? Could the Next list change dynamically?)

As I'm writing down this daydream for the first time, I realize #2 is the way to go for performance alone. All the concerns about Next should be encapsulated in an inheritable Processor Actor.

But what do you think? Have you done something similar? Any novel insights? I'm always looking to make better use of the AF.

[IlluminatedG]

This sounds like a sledgehammer being used when a ball-peen would suffice. AF has a whole bunch of startup and state management and I don't see how that contributes to the data processing without adding unnecessary management code exactly as you're questioning.

If there is a good answer for that I would definitely take the route of #2, the extra handling by a caller actor in #1 sounds extremely detrimental to any kind of DSP. There's not much wrong with a caller telling a child actor "here's where to send your data". That's not really a violation of the hierarchy as the data paths are still managed in a top-down manner, it's just that the caller can tell the children that they should just play with their siblings and that they don't have time to join in on the fun.

[OneOfTheDans]

Thanks for the vote of confidence on #2, I like the view that the Master still owns & manages all the Callees, even if the siblings are playing together.

Regarding AF being a sledgehammer, a standalone signal processor (or its caller) will need to:

1. Create/Dequeue/Destroy the incoming queue
2. Create/Enqueue/Destroy the outgoing queue(s?)
3. A while loop with some reliable stop signal (likely the queue getting destroyed)
4. Correct launching & cleanup of a background reentrant VI
5. Occasional setup/teardown code depending on the signal processing complexity

After building all that boilerplate, you've re-created at least 60% of the AF 😄.

[IlluminatedG]

I've been working on a framework (very slowly with many changes and delays, silly jobs getting in the way of my side projects and stomping on a room full of rusty nails) specifically for processing pipelines. Think GNU Radio Companion for LV BDs. Perhaps I should keep track of a list of people to ping about it.

But it uses XNodes and Channel Wires and... Oh is that DNatt coming? *Hides*

I will throw out there that Channel Wires are great for this kind of processing. The initialization for the communication is automated within the Channel Wire itself, they have some side-channel capabilities for closing down, and once the initial hit of scripting the type specialized instance is done (which has gotten faster), they're perfect for this kind of reconfigurable collection of processing nodes that run in parallel. Then you don't need async VIs and 1-4 of your list nearly completely evaporate. Run everything within the same hierarchy, MAGA (Make Abort Great Again) is achieved, and you're down to a while loop with a handful of cases to init, run, and cleanup.

[OneOfTheDans]

Do you have a favorite intro to Channel Wires? When they were first introduced, I thought they were just syntax candy around queues so didn't bother with them...

Fast forward a few years I decided to try them out. The popup wizard, large set of OOP dependencies pulled in, and code complexity under the hood turned me off and I never really gave them a fair shot. And do you know if they work on cRIOs?

[drjdpowell]

#2 is a standard way of doing things in my "Messenger Library", with the Caller defining which subactor messages which other subactor.  It is quite straightforward and subactors are simple.  Code defining the "pipeline network" is all in one place in the caller.

[IlluminatedG]

They definitely are syntactic sugar around queues (or DVRs or "Notifiers" or Events or something else) but they are especially convenient (IMO) when you want to have reconfigurability (changing order of nodes, adding nodes, etc) without having to explicitly change the communication boilerplate. Just change which terminals are connected by Channel Wires.

Can't say I've seen much about Channel Wires outside of the shipping examples, which are certainly on the smaller size. There is an entire forum for them https://forums.ni.com/t5/LabVIEW-Channel-Wires/gh-p/5080 and there are a bunch of videos on youtube. SLM's original demo from 2016 is actually pretty good for an overview with some more background https://www.youtube.com/watch?v=Czdf1Q7ubF4 though it doesn't go into anything "behind the scenes" about Channel Wire implementations.