TCP/IP large data packets sending problem

[nathand]

---

AndyN wrote:

Since LV does not do anything with it and TCPIP is data lossless how is it happening it losses packets in my case?

---

I don't know; at best I can make some suggestions. Did you try a wired connection yet? Why are you using a Timed Loop for the read? This is a bad idea, since the timing will be limited by the TCP/IP reads. If the timed loop is configured to discard missed periods or maintain phase, you might end up missing loop cycles, which could possibly lead to dropped data.

Do you check if the queue fills on the sending side? If the queue filled and you couldn't add another element to it, you might lose a packet. Also, I don't see much error handling, so I'm not sure how you are determining what error, if any, occurs. On the receiving side for example, if there's an error on the read it appears that you drop it (and also clear it, which is useless - why is Clear Error just floating there?).

[AndyN]

Wired connection is being tested as we speak. I will know tomorrow. I left it working for the night.

I used a regular while loop with exactly the same results. The only reason to use a timed loop with zero period was to force (hope) higher priority. It is set not to interfere.

The sending queue does not fill up on the sending side  (checked). It has a buffer good for 5s and the timeout was way shorter. I monitored all errors and timings for this loop and found no issues. I tested it well and it never gave me an error of any kind or a timeout. A new frame shows up exactly every 1/60s. 

There is not much I can do with the errors except see what they are, log them and reset the connection. 

If you look at the diagram ( it is a snipped so you can get the whole code) I do check what the errors are and, log them, stop the transfer and reset the connection (not it the diagram).

This is the receiver side (Windows) so the pop up error windows are quite annoying so I clear it. I have the error logged anyway.

The only errors I get is the sending side  -write timeout and read timeout- on the receiver side relating to exactly the same frame. The receiver gets only  the remainder of the 65kB/11kB(5kB). That seems like a problem with servicing the buffer. That is why I tried to use the times loop with high priority for receiving as well as trying to raise the VI execution priority. It did absolutely nothing. Not sure if it does anyway on Windows.

[AndyN]

The wired test came with the same results as the wireless. It crashed after like 20 min.

Then it is not the wireless network issue as I expected.

[Mark_Yedinak]

---

@AndyN wrote:

I know the details of the header and such.

I know the 1460 number and well as that TCP returns only the data portion.

I though it was clear... I know wikipedia.

 My question was in a case of 11kB of single data portion is the 11kB passed straight to the TCPIP layer or LV cuts is into 1460B pices on its own and the passes to TCPIP?

No, my code is NOT doing any arbitrary data reads. This is rude. If you really looked at it, it actually does have the 4bytes at the front followed by the real frame.

I also mantioned it in the first post. Did you read it?

---

My apologizes if you thought I was being rude. You need to understand though that most of the people here are volunteers and have other responsibilities. That means we may need to quickly review the posts and code. In addition I cannot actually open your code because I do not have LV 2012 installed at the moment.

For the record I am not simply quoting Wikipedia for my information. I have been working with low level networking for 26 years. I was involved with the early definition of the 802.11 protocol. I can ask you the same question about reading my posts. It has been stated several times that LabVIEW is not doing anything with the data. Everything in regards to fragmentation is being done by the stack.

If you are getting a timeout on your write it could be a buffering issue on that side. When you call the TCP write it will hand the data off to the stack and return immediately. The data may not get sent immediately depending on the stack and the state of the connection. Try installing Wireshark on your receiver PC and capture the traffic. That will help to identify where you are having issues. Are there any other types of transmitters in the area that could be causing interference?

[Mark_Yedinak]

Definitely do a network capture with Wireshark on your receiver. It will help to identify where the issue is happening.

[AndyN]

I installed the wire shark way before starting this post.

As I mentioned, the problem seems not have anything to do with wireless network as it happens also on the wired network and there is no other traffic involved.

I have 100MB of Wireshark data to look at waiting for me for a  few days now.

In general, it looks very clean to me. Except the three broken connection moments. 

Any suggestions what specifically I should pay attention to?

[nathand]

The first thing you want to do is determine if the "lost" packet was sent but not received, or was never sent at all. You'll need the actual packet contents and might need to do some work to parse it out, but I believe you said there's a sequence number in the data and you record which packet number goes missing. If it's never sent then it's probably a sender problem; if it's sent but not received, then the problem is in the network or the receiver.

I would also check if you see one end of the connection specifically send a message to close the connection. I'm still not completely clear about the behavior you're seeing. You should be able to handle a timeout without closing the connection, and I don't remember seeing anything in your code that specifically closes the connection on a timeout. Is the entire connection dropping, or are you just losing one set of data?

[rolfk]

I had in the past some similar issues although with smaller data. The connection would sometimes simply get abandoned. My first solution and what I have done since always was to create a reconnectable TCP/IP stream, that would act on errors other than timeout being returned by the network functions and then close the connection and reestablish it.

Later we found out it was caused by the dynamic network addressing. Whenever the adress was refreshed the connections would be dropped. Changing to static network addresses would solve that, but it is much more usefull to create the communication link in such a way that you accept errors to happen and gracefully reestablish a connection after an error. Assuming that your network connection never will drop is way to naive.

[AndyN]

Rolfk, Nathand ,

The labview RT (sending) computer has a static address. The other computer has a preassigned IP that does not change.

The broken connections happen every 10-30 min. I do don't think it can be related to renewing the IP but I will check it with full static addresses on both sides. Thanks for the hint.

If the connection is broken ( timeout on any side), I reset it and automatically start a new one and the transmission keeps going like nothing happened.  This feature was built in since the very beginning.

A user that does not pay too much attention does not even see it happening but the data during the 1s of handshaking is lost.

This is a different loop that handles all the TCP/IP connection/reconnections. It is not shown here but work flawlessly for a long time.

The datagram (the missing one)  is getting sent and partially recevied. I.e. one side sent 11kB and the other side was expecting 11kB but got only 5kB. The sender complains about the lack of confirmation, the receiver about missing the remaining part and both close the connection. As I mentioned before, the missed part looks like the remainder of the division of the TCPIP buffer size 65kB/11kB (or rather the part that did not fit in the buffer). Wire shark reports maximum size of the buffer. - >Nathan

I did not work on making a more intelligent read/write timeout connection I am not sure if it is even worth it. I would have to handle the data pilling up on one side and figure out a balance control not too overflow the network  with  the data from the buffer causing more connection problems. Essentially I would have to put a sort of small layer over the TCPIP with exactly the same functionality. If TCPIP fails for some reason with data transmission why would I assume I can do it better?

Also the timeout is used to diagnose a broken connection here. It makes it more complicated when a  timeout can be actually a sign of other problems. Sending 11kB at 60Hz for 10 min takes less than 17ms.  It goes like that for lets say 15 min. Then suddenly after 10 000 cycles 1500ms is not enough...

This is where I am a bit fuzzy with understanding. I assume that LV Write TCP is the blocking type- it stops the execution until it receives confirmation of sending the datagram? 

[AndyN]

Just to be sure I set everything to a static IP. It did not help.