bytes missing in serial communication UART - NI Visa READ

[LV_Enthu]

hello everyone,

                     I am using NI Visa read to transfer data from microcontroller to laptop using write-to-binary-file.vi... I have observed a very weird phenomenon, in the fact that some bytes appear to be missing from the data-file. this however, does not occur for all laptops... In some laptops, i have never observed a single byte being missed from the data file...

the OS running on laptop with byte missing is win 7, whereas the laptop running perfectly fine has win xp.

has anyone ever experienced such issue before? i am using Silicon Labs CP2102 as UART-USB converter over a baudrate of 921600 bps... the data size is normally in GB's... and the no. of bytes missing is purely random and varies from 2bytes to 32 bytes,... the way i determine if a byte is missing or not is by right clicking the file, check properties and compare size on disk and size of file ... they shud be same to ensure no data loss...

[JÞB]

Does the tast byte transfered equal the termination charater?

[LV_Enthu]

Actually I am nt using any termination character ... All I hav is a start bit 8 data bits stop bit and parity is enabled...

[RavensFan]

921,600 is extremely fast for a serial connection.  How long is the serial cable?  I wouldn't be surprised if you weren't losing bytes in the serial cable itself due to impedance issues.  Does it work at lower speeds?

[LV_Enthu]

there is a connector from micro-controller to CP2102 which has a wire length of about 5 inches... after the CP2102, its a normal USB A/B type cable... the clock used for 921600 baud is 16MHz... the code works perfectly in some laptops, but not all... and surprisingly, it works in intel core2duo and not on corei7 ... dunno if the processor makes any diff...

[Tunde_S]

Hi LV_Enthu,

I do not believe a difference between the file size and size on disk determines whether bytes are missing from your data file.The difference between size and size on disk can be found here.

If you believe you are losing data, this KnowledgeBase may be helpful.

Regards,

[Timmar]

I have suffered from this one on a compact field point, This may help on a PC as well.

1. The Comms UART has finite space, if you don't empty it it will overflow, losing data.

2. On a compact Field point, the routine to scavange data from the port resides between High Priority and Timed Loop.  By overloading the CPU with timed loops, I was able to force the comms to lose data.

My Solution, 

1. Put your byte handler in a Timed Loop. Buffer is cleared at highest priority.

2. Make Byte handler into a simple consumer, Nothing clever, push the data into a queue.

3. Create a normal to background priority consumer to process the info. 

1a. I use the event model for Comms Data to avoid polling,

Here is a pic of my Comms handler (It is LVOOP, has both read and write and allows for an e-stop and multiple listeners)

[LV_Enthu]

hello Timmar,

I am not doin what you have mentioned in the snapshot below... however, I am using a simple protocol to make sure uC is sending data and LV is receiving it... At the initial stage, LV is sending a cmd to uC to send data, to which uC acks, and upon receivng this ack, LV resends a cmd to uC to send data... 

however, everything is async in this communication medium... I am still confused and clueless why the code works only on 1 laptop and not on others... i also checked the link .. however, I have not implemented any checksum in this comm medium 

[Timmar]

At what point to you lose data?

Is the command-response just for initial handshaking or do you do it for each packet?

Do you wait for the packet to complete before sending the next command?

Your uC could be drowning in commands.

You laptops might have different operating systems, different UARTs

A a general rule, acquiring data at speed AND your writing to disk without a producer consumer infrastructure is a recepie for disaster.

It doesen't matter if you are using Comms or NIDAq, if you don't clear the buffers they will overrun.

The difference is that NI-DAq sends yo a rude error code telling you what has happened.

The Com ports aren't quite so verbose. [There may be some command to check for overruns, I haven't looked into it]

Tim L.

[LV_Enthu]

---

@Timmar wrote:

At what point to you lose data?

 i am not sure at what point i lose my data.. its completely random.. moerover, if i retry downloading data to file, everytime, i lose different no. of bytes... 

Is the command-response just for initial handshaking or do you do it for each packet?

 it is just for intial packets, i do not use any handshaking when i am transferring data from uC to PC; i.e. from external mem in my board -->uC TX buf --> PC

Do you wait for the packet to complete before sending the next command?

 yes, i do.. again, just for the initial routine, not when the data is transmitted from mem to PC

Your uC could be drowning in commands.

no, i am checking for overflow interrupts

You laptops might have different operating systems, different UARTs

Laptops are running diff OS, the code runs perfectly on win xp, but has problem on win 7... however, its running same UART

A a general rule, acquiring data at speed AND your writing to disk without a producer consumer infrastructure is a recepie for disaster.

It doesen't matter if you are using Comms or NIDAq, if you don't clear the buffers they will overrun.

The difference is that NI-DAq sends yo a rude error code telling you what has happened.

The Com ports aren't quite so verbose. [There may be some command to check for overruns, I haven't looked into it]

I am using prod consumer to transfer data from serial port to a file... 

Tim L.

---

Hello Tim,

  i have attached a snspshot of the loop where i m computing conditions for my data transfer...