NI Home > Community > NI Discussion Forums

LabVIEW

Showing results for 
Search instead for 
Do you mean 
Reply
Knight of NI
altenbach
Posts: 27,605

Re: file transfer using UDP protocol

[ Edited ]

Ethernet makes no guarantees on the fate of a packet. All the sender can do is place them on the wire and all the receiver can do is listen for them. A series of packets might arrive at the destination in a different order, some packets might be missing, some might be corrupt, and some might even arrive in duplicate.

 

TCP adds additional information to the packet header that guarantees a reliable two way communication. It is connection based, meaning that before data is ever exchanged, a connection is established by a threeway handshake of the two participants. The client initiates a connection with a SYN packet and only if the server agrees to communicate, it responds with a SYN-ACK packet which is acnowledged by an ACK. At this point a "connection" is established: both sides have agreed to talk to each other. You cannot have a one-way TCP connection, even if all data flows in one direction there will at least be connection management related packets in the other direction. The TCP header contains enough information that the original datastream can be correctly assembled at the receiving end. duplicate packets are discarded, missing packets retransmitted, and out of order packets correctly sorted. At the end, the connection is broken down by mutual agreement with another exchange of packets containing special TCP flags in the header.

 

With UDP, the packet is placed on the wire with a certain destination IP and destination port (or even with a broadcast or multicast address) and that's it. The sender cannot know if the packet ever made it to the destination. Each UDP transmission is stricly one-way and unrelated to any other transmission. Of course you can construct a more reliable connection in software, for example if the sever sends back an acknowledment with another UDP packet. Again there is no guarantee that this packet ever makes it back, but the client could add information to the next packet that it actually received it. and so on. As Saverio said, you could add a sequential number to successive packets so the server can arrange them in order and know if one is missing, at which time it could request a retransmission with another specially formed UDP packet. You can add a checksum, so the other side knows if the data arrived intact, you can add a connection ID so the other side knows where the data belongs if multiple conversations occur at the same time, for example if several clients talk to the server at the same time. Implemented this way, there are many UDP based protocols (such as tftp) that have a similar reliability like TCP, except that everything needs to be done in software and you have to build your own house of cards, defining all connection management commands, data structures, timeouts etc. such that both sides are aware of the state of the current virtual connection. For very simple needs, this can be relatively primitive.

Many very smart people have done that, but it will be much more complicated than you think. Why reinvent the wheel? All this has been already implemented on the protocol level for exactly this purpose with TCP. I don't understand why you insist on UDP. Do a google search for "TCP state diagram" to get an idea what would need to be done if it should be done right.

 

As I said, UDP is used for example streaming protocols where lost packets are ignored and the connection is mostly one-way. If you listen to internet radio, the sender does not expect acknowledment from all the clients listening and if the first word of the weather report got lost, it does not make sense to retransmit it later. Other uses for UDP are stricly one-way, for example a syslog sever passively sits on the network and waits for UDP packets on port 514. no incoming packet is ever acknowledged and they are simply appended to an existing log. If a packet got lost, nobody will ever know about it. (Over 10 years ago, I once wrote a simple syslog server in LabVIEW, all you need is follow the RFCs :smileywink:)

 

There is a reason we have both TCP and UDP, they have a very fundamentally different purpose so use the right tool for the job.


LabVIEW Champion . Do more with less code and in less time .

Member
omar21
Posts: 22
0 Kudos

Re: file transfer using UDP protocol

I am with you i the choice of TCP/IP instead of UDP, but unfortunately, my teacher doesn't do so, he insists on using UDP!!, so here is my new question: I have a 600Mb file and i want to devide into packets of 65Kb each, then send them using UDP. Some people told me to wire the file directly to the UDP write VI, but it didn't work, and the same error message appears. so shall i use a loop with this configuration? or there are other VIs that can be used in creating packets

Knight of NI
Posts: 18,064
0 Kudos

Re: file transfer using UDP protocol

So this is a homework problem? Perhaps your teacher is trying to test your capability to deal with this issue when it comes to big files, and to come up with a system that can handle it. In that case, we would be remiss in providing you the answer.

 

There are no subVIs to create these packets. You have to program it yourself by reading the file in smaller chunks and transmitting those smaller chunks. That's your homework assignment. Now, how do you intend to solve the problem?

Knight of NI
altenbach
Posts: 27,605
0 Kudos

Re: file transfer using UDP protocol


omar21 wrote:

I have a 600Mb file and i want to devide into packets of 65Kb each, then send them using UDP.


How do you come up with such random numbers? Can you describe ther reasoning behind your decision?

I agree that this will be a great learning experience: designing a communication protocol that uses UDP. Draw a state chart with all possible states (idle, read to receive, reday to send, transmitting the desired file name, transmitting the file data, file sent complete, receving, missed parts, no response from the other end in in xx seconds, etc.etc.) and all possible transitions between them. Design a data structure with useful fields (structure size, connection ID, sequence number, data payload, state, etc.). Simply start programming, try a few things and use what works best. Start simple and add more states as the need arises to handle specific scenarios. Do an internet search to see how tftp does it. All these protocols are very well documented and everything can be found online. It won't be trivial.


omar21 wrote:

Some people told me to wire the file directly to the UDP write VI, but it didn't work, and the same error message appears.


Whay do you listen to "some people". What are their qualifications?


omar21 wrote:

... so shall i use a loop with this configuration?


Anything in LabVIEW (and any other programming environment) that requires repetitive action needs a loop, why is this even a question? Since you don't know the final number of iterations needed at the time the loop starts, you need a while loop.


omar21 wrote:

... or there are other VIs that can be used in creating packets


All you need to do is portion succesive small parts of the file in a suitable data structure that also contains connection information so both sides are aware of the state. All you can transmit is a linear sequence of bytes (a string!), so you need to flatten your data structure to a string and unflatten it back to the data structure on the other end to recover the information. You don't "create packets", that's done automatically by the network stack based on the MTU of the connection. Packet might even get further fragmented by other network devices depending on the needs. LabVIEW only has access to the higher level function, you send and receive strings, that's it. You don't need to worry about the details under the hood.

 


LabVIEW Champion . Do more with less code and in less time .

Member
omar21
Posts: 22
0 Kudos

Re: file transfer using UDP protocol

Actually, it is not a homework, but it is rather a research. I am working on power substations automation using modern IEDs and IEC61850 standard. I am not that expert in data communication and i have just some basics. As i mentioned earlier, one part in our work consists of writing the data into a TDMS file then sending it to a server via ethernet, the transmission should be done as fast as possible, this is why we have chosen UDP. I have downloaded a file transfer application built in LABVIEW 8.5 that uses TCP/IP, and it worked perfectly with my 600Mb file, so i just replaced the TCP VIs with UDP VIs and i ended up with this error message. I understood that the maximum file size that can be sent at once is 65Kb, so i am faced with two solutions: either i increase the size of the buffer up to 600Mb so that it can send my TDMS file at once, or i must devide my file into multiple 65Kb packets and send them directly. so this is the whole story. everybody is invited to contribute in our research :smileyhappy:

Knight of NI
altenbach
Posts: 27,605
0 Kudos

Re: file transfer using UDP protocol

[ Edited ]

omar21 wrote:

... the transmission should be done as fast as possible, this is why we have chosen UDP.


The TCP overhead will be negligible and UDP will not be faster. What makes you think otherwise? What is the speed and latency of your network? How much other traffic is present? Did you measure the transfer speed of the TCP?


LabVIEW Champion . Do more with less code and in less time .

Member
omar21
Posts: 22
0 Kudos

Re: file transfer using UDP protocol

As i told you, I just want to start by sending successfully my TDMS file using UDP, and then we can go for further improvments. i don't wanna see this error message again :smileysad: don't forget, i'm not that expert in data communication, i'm just an automatic control systems engineer

Knight of NI
altenbach
Posts: 27,605

Re: file transfer using UDP protocol


omar21 wrote:

 i'm just an automatic control systems engineer


At this point you simply sound stubborn. :smileywink:

 

All you need is to transfer a file from point A to point B and there are plenty of tools to do that and many have been mentioned. This problem has been solved numerous times since the invention of the internet, and there is a collective experience from very smart network programmers that don't do anything else. You will never achieve these skills unless you focus for years on this particular problem.

 

Insisting on UDP is just silly, it is NOT the right tool.


LabVIEW Champion . Do more with less code and in less time .

Member
omar21
Posts: 22
0 Kudos

Re: file transfer using UDP protocol

for GOD sake, this is my VI, i know TCP is better, but i just wanna see it working with UDP, so tell me how shall i modify it to not see this error message anymore :smileysad:

Knight of NI
RavensFan
Posts: 15,928
0 Kudos

Re: file transfer using UDP protocol

You've also been told that you can't send an entire file at once if it is too big.  You need to break your file into smaller pieces and send those.