How to approach the integration of asynchronous communication framework? (+ concurrent access issue )

[Athaj]

Dear LV community,

for the sake of reusability and improved cleanliness, I would like to design a new "integration" layer, that would encapsulate our custom communication framework (based on CANopen) into Labview.

Long story short, up to 127 communicating nodes (identified with a unique identifier) send and receive packets (with a fixed-size payload) using an addressing scheme that contains 2048 of so-called object channels. There is a (mostly static) assignment between the object channels and the nodes. The payload is fixed-size (64bits), but a device-specific semantics (e.g. weather station sends four 16bit integers with T/T2/P/RH; a displacement gauge sends one 32bit integer with position etc.)

In our scenario, the framework is interfaced via a DLL library, that uses a good old polling approach:  

• the function read_chan() returns a numeric identifier of an object channel ("objid") where new data is available (or -1 when there is none)
• the function read_chan_data(objid) then retrieve the most recent payload on the particular object channel.
• then the node itself parses/interprets the payload

So far, we used several variants of the brown-coal-era-style straightforward approach - single loop calling the read_chan(), then comparing the object numbers in a case statements that exploded the payload into vast amounts of local variables. These are used in the other part of the program: UI displays, control state machines, servo loops (all of those with different timing) 

And now to the core of my inquiry: I would like to implement an "application layer", that would :

• relieve my fellows of using the low-level calls (do the encapsulation easily usable by not-that-experienced programmers)
• manage to dispatch ~5000 packets/second (i.e. minimize excess overhead)
• won't need the flood of local variables so that, e.g. the decomposition into a hierarchy of sub-VIs would be easier

The approach I have been wondering around is shown in the attached figure (top_level_vi.png) :

An object descriptor cluster, ObjectDescriptor, is defined and contains the identification of channel, and a boolean New Data flag that indicates the presence of new data.

An encapsulating class, CanMaster.lvclass, a notifier-based Singleton, is defined and used. It contains an (initially empty) array of Object Descriptors. 

  * The INIT function instantiates the connection to the DLL and returns a reference to the CanMaster class.

  * Then a REGISTER_DEV function internally marks what object channels are of interest (subscribed) according to the node-object channel mapping, i.e. it adds corresponding ObjectDescriptors to the array.

  * Then the MASTER_SERVER VI (acting as producer, schematic attached) carries the polling with the read_chan() function and set the New Data flags in the corresponding Object Descriptor. A Test-and-set approach, encapsulated into the PingObjectDescriptor method uses a semaphore to prevent concurrent access. 

  * The consumer loops use the device-specific READER (actually a object-channel specific) methods (e.g. Edlen_rx1, Edlen_rx2, Adda3g_rx1 ), that hold the last payload seen on the particular object channel (with a shift-register one-pass loop FGV), updates it once the New Data flag is raised, and explodes the payload into the (object channel-specific) meaningful data. The reader methods dereference the reference to the CanMaster class, retrieve the corresponding object descriptor with a GetObjectDescriptor method (attached), that uses the PingObjectDescriptor to test-and-clear the New Data approach.

  * The CLOSE function just let the MasterServer quit and close the connection to the DLL.

----

The open questions are:

• Can I avoid the Singleton pattern?
• Can I further simplify the interface?
• Is there any feasible design pattern/approach I just do not see?
• So far, each object descriptor contains a semaphore, to prevent race conditions, but I am not sure the approach is valid? 

Regarding the last issue:

The point I am missing is I am not sure how atomic/isolated are the DVR operations with the in-place-element construction. I actually do not know, whether I do a coarse-grain locking because the in-place-elements used both in the MASTER SERVER and the READERS or whether the PingObjectDescriptor will only update the local copy and a race condition could happen here.

Any thoughts and suggestion would be deeply appreciated!

-----

Background: I work at a research institute, where we use the CANopen to tame the very heterogeneous ecosystem of both custom and third-party hardware and software modules (nodes) used in the experimentation (generally revolving around lasers and related applications). A typical experiment requires ~10 nodes, while the most extensive experimental testbeds use 50+ nodes. We use traditional Labview versions between 2014 and 2019.