tcp write/read and flattened data

[mrbean]

I'm looking at the TCP Client/Server examples in the LabVIEW 2009 folders, and I have a couple of questions.  In those examples, they employ the two-write, two-read method, where the message length is written/read first then the rest of the data.  There, they use the Type Cast to convert the data out of the first read to get the length of the message.  I would like to use the one-write, one-read method if possible.  Here is a little more detail.  The data I want to send is in a bit of flux because I don't know if I need to send a length field, or if I might have to Pad the message so that all messages are the same length.  If I can make the messages all one length, I could tell the TCP Read VI how many bytes to read each time without having to pass that along, and I would avoid the second read/write scenario.

The other problem I have is that the main portion of my data contains a variant.  Specifically, the msg packet contains a Msg_ID (U32) and a Msg_Data (variant), where the ID tells the receiving end what type to convert the variant to.  I have a successful messaging system now, but it uses shared variables, and if a non-labview client wants to use this tool, they're out of luck.  That's why I want to switch to a TCP system.  It's all on localhost.  What I'm not sure about is the best way to get my message packet into the TCP Write. 

Do I Flatten To String, then send it.  Do I then Unflatten From String on the Read end.  For this to work, do I need the message packets to all be the same length (include a pad).  Otherwise, how will the ONE read know how many bytes to read. 

I noticed the Flatten To String contains a prepend array or string size attribute.  If True (default), do I HAVE to use the two-read method?  As I mentioned, I ultimately want to read the data, and based on the msgId, convert the variant to a known type, then process appropriately.  Any thoughts on the multiple issues I've brought up?

[Mark_Yedinak]

If you want other applications to be able to read the data I would not use either a variant or a flatten to string. I would define a packet format that will be readable and easily understood by any application. This means you would have to manually build your packet or data. Ultimately you would be sending an array of U8 data. I have found it is easier to explain a simple packet description to someone who doesn't know LabVIEW than to try an explain the format of flattened data or variants. You are effectively creating a binary stream. While the flattened to string will simply turn your data into binary stream you may run into issues with complex data types. An explicit definition of the data will help you to make it portable. Besides, you would need to provide the data format if you used flatten to string.

If you want a generalized solution I would include the lenght in the data. This is the safest approach since you minimize modifications to the read and write subVIs when your data changes. To make it even better for future changes you can include a version number to the data. This could be the first field after the length. Applications can use the version number to determine how the data should be parsed.

Fixed data sizes are easy to implement but they can be trouble to maintain.

[nathand]

Why do you dislike the two-write, two-read method?  If you're concerned that you're duplicating the number of packets sent, don't worry, you're not - your operating system's TCP/IP stack is smart enough to see two packets with the same destination being sent one immediately after the other, and will combine them.  Of course you can also force the same effect by prepending the data length to your string and then writing that modified string in a single write.

[mrbean]

This may take some considerable work to retool my messages.  So, as to minimize re-work, I'm wondering if you can clarify a few things.  You mentioned "Ultimately you would be sending an array of U8 data".  As I mentioned, all of my messages are of varying complexity and length.  Because of that, (since I was using the Shared Variable LabVIEW specific approach) I went with the ID/Variant approach so the receiving end could just convert from variant to the labview control type.  Now, should all my message types be of type String so that I can use the String to Byte Array and feed that in to the TCP Write?  Do I then (at the receiving end) look at the ID and basically convert all the substrings (after a Byte Array to String conversion) to their respective types/values?  I just want to take the approach that will require the fewest type conversions and type conflicts.