file transfer using UDP protocol

[omar21]

Hi everybody

I hope someone can help me. I'm trying to build a client-server file transfer application. The client reads a file stored on a given working station(PC) and sends it to a server installed on another working station for further usage. I am using UDP as the transport protocol. The problem is that my application works perfectly with files whose sizes are in the range of kilobytes, however when I try to send files whose seizes are in the range of megabytes, the application crashes down and the attached error message shows up. I understood that I must increase the size of the sending and receiver UDP buffers but i don't know how to do so. please if you have any hints or ideas, just let me know.

Best regards

[altenbach]

UDP is connectionless and, unlike TCP does not offer any kind of delivery guarantee on the protocol level. There is no handshake or ack so the sender has no way of telling if the data has arrived successfully. If you use UDP, you need to implement all the connection management yourself if you want a reliable file transfer.

It is not clear how you are actually transfering the data, but you probably need to send it in chunks. Can you show us some code?

How are you increasing the buffer sizes? Is this external to LabVIEW? Did you read the help on "UDP write", especially for the data terminal?

I would definitely recommend TCP.

[omar21]

well, i am using a simple UDP configuration with UDP open connection, UDP write, UDP close connection VIs at the side of the sender, and again UDP open connection, UDP read, UDP close connection VIs at the side of the receiver. i know that the maximum size of datagrams you can send or read using this VIs can't exceed 2mb due to the limited size of buffers, which implies that we have to increase the size of those buffers, and to do so, i tried the following solution

http://digital.ni.com/public.nsf/allkb/D5AC7E8AE545322D8625730100604F2D

which is not sufficient, or may be it is me who didn't understand how to use it.

i'm really interested in what you said about "chunts", could please provide me with more details about that

[altenbach]

That very old link talks about a 2MB/s transmission rate, not a buffer size.

Also 2mb (millibits) is extremely little (fractional bits are not possible!), I assume you are talking about 2MB (megabytes). Is this correct?

You need to set up the transmission in a loop and send smaller portions at once, while reassembling the data on the other end. I sitll would recommend TCP. Why did you decide on UDP?

[omar21]

Well, here is the scenario i need to go through, i make measurments from a power substation simulator, i write them to a TDMS file, then i need to transfer this file to another workstation via ethernet. My TDMS file size is 600Mb and it must be sent as fast as possible, this is why i wanna use UDP. the fact i don't understand is that when using TCP VIs, the file is sent sccessfully and no problem occurs whatever the size is, even thought it takes a considerable time. But when using UDP, many problems raise about the buffers size. Another point i want you to help me about is: how can i devide the file into packets to be sent via UDP, i mean is there any method to precise the size of each packet, or i just need to use some loop with UDP modules and the packets will be created automatically??.

Best regards

[altenbach]

You simply need to use TCP. Even if your network has only 0.01% packet loss, trasnferring 600MB via UDP will give you a corrupt file on the other end virtually every time unless you do a lot of verification and handshaking. The additional overhead for TCP is insignificant. You will not be able to implement a reliable transfer via UDP that is faster than TCP.

UDP is not meant for reliable transfer. It is more sutable for e.g. music streaming, where a lost packet does not need to be recovered.

I am sure there are much better ways to do all this if you think about if a bit more. For example you could use datasocket, or even simply write to a shared drive directly on the other computer.

---

@omar21 wrote:

...  i mean is there any method to precise the size of each packet, or i just need to use some loop with UDP modules and the packets will be created automatically??.

---

I have no idea what you have in mind here, but from the wording alone most of the question makes no sense.

[omar21]

i know that TCP protocol takes care of creating data packets and sending them such that the user don't have to worry about that, whereas this is not the case with UDP. i have just heard many people here talking about controling the packets and enqueuing them using loops with UDP, and from that point my question raises. i hope this makes the picture clear for you 🙂

[RavensFan]

As Altenbach has said, and he clearly knows his facts on this, UDP is NOT going to do what you are trying to make it to.  Just forget it about it.

You need to use TCP.

[omar21]

what about TFTP, isn't it UDP based??, and it is considered as a reliable file transfer protocol

[smercurio_fc]

TFTP is its own protocol, and you cannot generalize on the reliability of UDP based on how a protocol that uses UDP may or may not be reliable. It works by the sender sending a "packet", and then waiting for an ACK from the receiver. The packets are numbered so the receiver knows which packet number it received, and the ACK includes the packet number, so the sender knows the packet number that was received. That's pretty much about it. It's used primarily for devices to download any new code as part of their boot process since it's extremely small, requiring very small amounts of memory to work. In that respect you could consider it "reliable", but this has nothing to do with UDP, as the reliability is due to the the way TFTP is implented (i.e., it specifically waits for an ACK, and the packets include numbering information), and not with the way UDP fundamentally works (which has no requirement for an ACK, or anything regarding packet numbering).