Issue with TCP IP data client using message headers with the message size included in header

[bradhoward]

I would be grateful for anyone that can help me out with this issue.   I am using labview as a data client.   Basically, I have a computer that sends out a TCP IP message to me that is on the same LAN as the labview computer.  The TCP IP messages from the source computer are supposed to have a specific message format that is in the following form;

Byte 1) Message Number

Byte 2) Format Number

Byte 3) Format Version Number

Byte 4+5) Packet Length of Body    (All multi byte data transmitted BigEndian)

Body

I wrote a TCP IP program that establishes a connection to the data server.  It then reads the first 5 bytes of the message.   In particular, I am looking for a message ID of 248 with a Format Number of 1 (Byte fields 1 and 2).  It is sent 1 a second and I want to write this to a Byte Array, which I convert into another message and send to a third system.  If the message doesn't match the desired message, I read the rest of it and through it into a byte array that isn't processed. 

The problem I am getting is that sometimes, the sender's send queue builds up until it maxes out.  The problem is quite random and has made it hard to debug.   In addition, I noticed that when I change certain Bytes in the Sender message.  Sometimes the change is imediately received by labview, but other times there can be a 20 second to 40 second delay in Labview seeing the change in the message.   I looked into this quite a bit and can't say for sure what is causing this.   The ICD is supposed to be well defined where each of the messages corresponds to the format described above (see TCP Reader File).  

To try and make sure this wasn't the case, I built another data client that continously reads TCP IP data packets looking for 248 ID and the format number 1 in sequential order and then takes the bytes after this as an input to the number to read.   However, this didn't fix the problem either.    I am at a loss and would be grateful for the help.

[nathand]

I've taken a look at your code, and re-read your post several times, and I still don't understand it. Could you attempt to explain either the problem or the error that you're seeing, and then make sure there's a clear question that you want answered? Also make sure that you explain any terminology specific to your application - for example, what is "ICD"?

You started the last paragraph with "To try and make sure this wasn't the case" - to make sure WHAT wasn't the case??

[bradhoward]

Nathand, I appreciate you taking the time to help me out with this.   I apologize, I had spent a good bit of time trying to word the question but I guess I didn't do a good job.   A couple of quick items by ICD, I meant to say Interface Control Document.  And when I discussed the 2nd vi I wrote, I was trying to take into account there might be additional messages being sent over the 5750 port from the data server, although I was told there was not supposed to be.

    The code I have seems to lag in reading from the buffer occasionally, causing the send queue of the data server to max out.    As part of this, I tried to create a TCP Reader that reads 1 byte at a time and looks for a byte with a 248.  Then it checks for a one (signifying the message I am looking for over the TCP IP interface).  I would have thought this would have eliminated the case where a different format or different ID comes through and messes up my TCP Reader.

The best way I can explain this is that I am receiving output messages from a moving car, say a value like gear position.   I have this TCP IP thing sitting there reading the gear position of the car (a field in the message).   However when I change gears, there can be a 20 second lag before I register that the car's gear has changed.   In reality, I should see it change 1-2 seconds later as I recieve the message every second.  I have followed the bug all the way up to the 248 byte array that I have in the class data.   So I can see that my case structure that bundles the 248 Byte Array is running, because I can watch the counter go up.   However, the values in the byte array are not changing even though they are changing at the source.

[nathand]

Can you explain "causing the send queue of the data server to max out"? That sounds like a connectivity issue, in that maybe your connection gets interrupted momentarily. While I'm hesitant to say it's impossible, I don't know how your LabVIEW code would cause the data server to fail to send data, since the TCP acknowledgements are handled at the operating system level and it should be able to buffer a large amount of incoming data. Do you have to send some sort of reply to the data server? I don't see that in your code.

Are you sure this is an issue in your LabVIEW code? What else is your LabVIEW code doing in parallel? If you have a time-consuming operation elsewhere that never yields time to the TCP Read, then you could buffer a bunch of data points which could take time to process. What's determining the rate of reading data? If you're limiting to something other than the rate at which data arrives (ie, you could be receiving data faster than you process it) then of course you'll introduce a delay.

Have you looked at what the sending side is doing, for example by installing Wireshark and logging the network packets?