What is the best way to read high rate serial data?

[Alexlab1]

My goal is to read 14 bytes of binary data from an instrument over RS232 serial (baud 460800) at 2000 Hz. I have not gotten a high speed serial card yet so I am currently using the standard serial port (115200 baud) and reading at 400 Hz. I configure the serial port, flush the buffer then enter a while loop to read the incoming data. I have a visa read 14 bytes per interation every 0.0025 seconds (400 Hz). However it seems the sensor is spitting out data faster than labview can read it because bytes are accumilating at the port. After a while the buffer fills up and the program fails.

Is there a better way to do this?

Would it be better to read larger amounts of data less often, e.g. like 1400 bytes every 0.25 seconds?

Thanks

[johnsold]

Does the instrument send exactly 14 bytes in each message or does it also send one or more characters to separate the messages (such as a line feed or carriage return)?

The way you are building the array for the Bytes at Port Graph will cause memory re-allocations which will eventually slow down the VI and maybe even cause it to fail due to an out of memory error.  Updating the graph and the other indicators also slows things down. Updating those things more often than the frame rate of the display really does not make any sense. Updating faster than the user's eye can follow is rather useless also.

Lynn

[Alexlab1]

12 bytes of data are transmitted with no data separator. Successive points

are separated by two consecutive bytes OxFF (decimal value = 255). So total of 14 bytes per measured data point. You can see the two 255's in the top of the array indicator in the serial_read.jpg image.

I realize the graph and indicators slow the VI down, but I just put them in there for debugging and will remove them in the final program.

[johnsold]

If you remove the shift register, the graph, the Wait, and the indexing tunnel at Data Out, does it still overflow the buffer?

I modified your VI so that it will stop if the buffer starts to fill or an error occurs.

Lynn

[Alexlab1]

Thanks,

Well at 400Hz the while loop can keep up with the sensor, but at 2000Hz (baud 460800) it cannot.

My question is, rather than iterate the while loop at 2000Hz, Is there a better way to recieve and process the data? Should I collect multiple data points in one read operation? for example reading 1400 bytes every 0.25 seconds rather than 14 bytes every 0.0025 seconds? If so, what would be the best way to append that data

[wideofthemark · Accepted solution]

Hi,

I suspect that it would be good to read multiple points in a loop, and keep the loop rate less than 1000Hz.  I found in the past that having something that needs to run faster than about 10ms tends to fall down when windows loses interest in the process.  So yeah pick the slowest that you can get away with reading samples - eg read every 10ms, and read 20 samples each time. You should be able to configure your serial port to be able to buffer the samples i guess.

Also and related to the windows issue - if you use a 'timed' loop rather than a while loop then you can force it to be an integer ms value.  You can also assign it a dedicated processor and set the priority to 100 which means it's less likely to run off and decide to scan viruses on your hard drive or look up your email account or whatever it does when it's bored with your program.  Also it 'tells' you if the loop took longer than expected to run - so you can use this to help iron out bugs etc - and\or report to the user if you have missed some samples.

http://zone.ni.com/reference/en-XX/help/371361H-01/glang/timed_loop/

JP

[wideofthemark]

Also - check out the high res timer vis - they are useful for working out how fast code is\ where bottle necks are etc.

http://www.wideman-one.com/gw/tech/dataacq/labview/perfcounterdemo.htm

[johnsold]

That may be the best approach.  Because so many variables are involved, you will need to experiment to see what combination of number of bytes read per iteration works best. Reading somewhere around half of the buffer may be a good place to start.

You may also find that the best values will change when you get your high speed port.  When selecting the serial card, consider the buffer size because that may have an impact on the optimum performance.  You probably should avoid USB to RS-232 adapters because they tend to be quite fickle.  A PCI card, if that is an option, might be a good choice.

How to append the data? Use parallel loops.  Look at the Producer/Consumer Design Pattern which ships with LV.  In the read loop (Producer) you have only the Read function and an Enqueue.  It reads the number of bytes you find to be nearly optimum as soon as they are available.  No Wait required because the VISA Read will wait for the specified number of bytes.  In the Consumer loop you Dequeue the data.  Append it to a string in a shift register. Use the string manipulation functions to locate the double FF bytes and extract the data accordingly. 

Lynn

[johnsold]

That article on the high res timer is quite old.  Search the Forum for that topic.  Within the past week or so someone posted a modern version.  Note that these are Windows-only VIs.

Lynn

[Henrik_Volkers]

Here is a producer consumer approach.... not tested

The unwrapping of the string could be done in a better way (didn't spend much time on it, not tested!! Just a quick mod from another post)

Can't remember what is faster scan from string or match pattern or regular expression .....

Avoid any frontpanel objects in the serial read loop to keep that thread independent from the GUI thread. Blocksize (128) should be optimized...

EDIT: The scan loop will not work 😞