Multiple readers of Single Network Stream Writer

[JENIK182]

Hello, I wanted to ask you a question about streaming the data between multiple computers. I understand the basics about sharing data between two computers/targets that are in the same network, but I don't have any experience with networks constisting of more computers/clients. I want to use (or I was thinking about using it) passing data with Network Streams. 

The thing is that there is only ONE writer (measured electrical data from cDAQ9185 with NI9205 module) and MULTIPLE readers (Laboratory PC #1, Laboratory PC#2, Notebook ... etc.). I understand that single network stream works like a Queue, but how to accomplish that all computers gets the same data? One solution would be to create a Network Stream Writer for every computer, but I think that is a little bit off. Another thing is that, when computer is not connected there is no need for buffering data for that computer at all.

So basically I want to ask you, if there is a way to dynamically create network stream writers and how to pass data from ONE source to MULTIPLE readers. Also there is a need to ensure that all readers gets the same data.

Thank you for your ideas and answers! 🙂

Ing. Jan Baroš

Technical University of Ostrava

Department of Cybernetics, Internal PhD. student

[crossrulz]

First question would be what kind of data are you sending?  Do you need to stream the data (need every data point starting from the connection until disconnected) or just tag data (only the latest value matters)?  If you need a stream, then you need to spawn a new clone of your Network Stream VI and use User Events to tell each close to send the data.  If tag data, you could use UDP or even Network Published Shared Variables (*shuddering*), which broadcast the latest values and no actual connections need to be made.

[Bob_Schor]

If you really want to stream data to multiple Remotes, then Network Streams work quite well, but I believe they are point-to-point, which means you need a Reader/Writer "pair" for every Stream.  A similar problem is communicating with multiple detached Asynchronous Clones -- if you are using Queues, you need a Queue for each Clone (trust me, I've done this ...).

Bob Schor

[JENIK182]

Well, I am sending 1D Waveform Array consisting of 6 Waveforms which are electrical data (Double, 41.6 kS/s for each channel).

The point of the application is to make data available for each client, because in the laboratory we are working on some different tasks about measuring electrical power and power grid quality. But we have only one measurement chassis and we are not able to connect simultaneously. So the idea with sharing data was proposed.

And in order to have applications working properly we need to have every TimeWindow properly delivered to each client as long as the client is connected. It is not required to have it in memory for long (only until it is read by the connected client, so basically FIFO for each client with short timeout).

So from what you have written to me, it looks like I need to STREAM the data. Could you make me some diagram/VI/example about spawning a new clone of Network Streams and use User Events? Does it mean that for each client I have to create a new copy of Network Stream VI or just unique name of Writer? And how do I say to WRITER that I have a new READER which wants a data? (for example a new client was connected). If I would know that a new client was connected I would just store that information in an Array and would create programatticaly a new WRITER for that specific READER but I don't how to do that 😞

Thank you

Ing. Jan Baroš

[crossrulz]

Use the Asynchronous Call By Reference to spawn each clone.  There are a couple of good examples in the LabVIEW Example Finder (Help->Find Examples).  Thinking this through, I would probably have a thread that does nothing but listen for a TCP connection.  Establish the connection to share what the endpoint names should be, close the connection, and launch the clone with inputs to state what the endpoint names it should connect to in order to create the network stream.

[Bob_Schor · Accepted solution]

Let's consider a Host and a single Remote.  I gather that the Host has the "measurement chassis", and the Remote does the "analysis" (with multiple Remotes possibly doing different analyses) on the (common) data.  [This might not be the right model, but let me flesh out my perhaps-wrong idea ...].

We set up a Network Stream with a local Writer and a remote Reader, using the IP of the chosen Remote.  The Remote "knows" it is the Reader, and starts listening for a connection to be made.  Once the Connection has been established, the Remote starts a "Listener Loop", trying to read from the Stream with, say, a 1-second TimeOut before trying again.  This could be configured as the Producer of a Producer/Consumer Design Pattern, running as an asynchronous Loop so that it doesn't materially impact the functioning of the Remote.  When data are received, they are put on the P/C's Queue, so that the Consumer can analyze them at leisure, with any new data being received and placed in the Queue for analysis.  

The Host goes ahead and generates data to be sent to the Remote, then simply sends it to the Remote via its Network Stream.  It's not clear if the Host needs to get any data back from the Remote, nor whether the Host ever tells the Remote "OK, we're all done", but that could be embedded in the data stream from Host to Remote.

So now you want to do this for N Remotes, each of which has its own IP.  Each Remote has the same code as the above Remote, and will only get the data sent to it over its (host-unique) Network Stream.  What do you do on the Host?

You isolate the code that deals with the Remote, including establishing the Network Stream, sending Data to the Remote, and possibly telling the Remote to exit.  You might write this as a Queued Message Handler, i.e. a loop that takes a Queue that says "Do This with the following Data", which means you can write it as a self-enclosed VI with inputs and outputs.  Now configure this VI as a "Clone", a pre-allocated Reentrant VI that is called using Start Asynchronous Call.  An input to this Clone is the Message Queue.  Here are some Messages you'll need:

• Define Stream, with data being the IP of the chosen Remote.
• Broadcast Data, with data being the reading being sent to the Remotes.
• Quit, a message to tell the Clone to alert the Remote that it is exiting, then stopping the Clone.

Now you'll want to spin up N Queues (save as an Array of Queues) to send Messages to each Clone, and an array of Clone References (just in case).  Once they've all been "Start Asynchronous Call"ed, you can (with a For Loop) send each its appropriate Define Stream message to get them to connect to the Host.

If you do choose to go this way, Start Simple.  Write a simple Host routine that defines some communication routine (like Queued Messages) for Host-to-Remote commands, and write a simple Remote that processes simple Messages, all without using Clones.  Once a single Host-Remote path works, and you can "Start", run, and stop the Remote, make the Host part a Clone and debug again.  Once that works, you can simply change "Clone" to "Array of Clones".

Bob Schor

[JENIK182]

So after a few days of programming, I have been sucessfull! 🙂 My solution is basically what you have told me:

1. I have a TCP server which is listening for new clients that wants to connect. With each connected client I am dynamically creating a new "TCP handler" for handling TCP communication with that specific client.
2. TCP Handler asks client what is his "VI CLONE ID" and also sends his "VI CLONE ID" to the client. Then the address for Network Streams are parsed (//"IP"/"CLONE ID Number"/"reader x writer")
3. Network Stream Connection is Made
4. Server starts to fill Handlers FIFO with measured data which are send to the clients via Network Streams

DONE!

Thank you very much for all your ideas and tips! 🙂

Best regards

Ing. Jan Baroš