Serial overrun interrmittent error -1073807252

[Franjo_Tonkovic]

Hi,

I am having problem with serial communication (RS-232) inside of bigger application. I am using cRIO-9024 with chassis 9118 and LabVIEW 2011 SP1.

Serial reading behave totally erratic, sometimes works few for hours without any problem and sometimes has problems from the beginning and I cannot resolve them by open-close connection, flush-clear buffers, VISA Unmap address. When error occurs most of the time I get error -1073807252, but sometimes I get -1073807253 and -1073807232 as well. All these errors are indicating to me that I have problem with data being read from UART.

When error occurs, most of the time Visa read gives me out 16 bytes instead of 200 bytes.

Serial communication is just one part of my application and total CPU usage goes up to 80%. 

I am using usb-serial adapter to send data from my laptop to cRIO.

I tried running cRIO with different VISA drivers, I tried running it with LV2014,  I also tried setting RXFIFO = 1 (as per https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z0000019M1hSAE).... I have reprogrammed serial communication multiple time but no success yet.

Any suggestions what to try next?

Thanks

Regards

Franjo

[Franjo_Tonkovic]

I have created additional test application with delayed acquisition start and it proves that serial readings are good after cRIO start and problems begin after acquisition/processing starts in something like 30% of cases. Before acquisition/processing started CPU usage is very low and after they started it goes to 80%. Attached print screen is created with RXFIFO = 1 set in niserial.dbs on cRIO.

This tests also includes random delay before restarting serial connection, as attached, with idea to catch randomly catch moment to start serial communication when CPU is not fully loaded.

All suggestions are welcome!

Regards

Franjo

[cstorey]

You can find those error values listed here: https://zone.ni.com/reference/en-XX/help/371361J-01/lverror/visa_error_codes/

It seems like you are trying to read data too fast based on those errors.   Perhaps the laptop's USP port or USB adaptor are just too slow for the cRIO rate. 80% CPU usage is probably because you are running the while loop doing the serial read  without any delay.  Perhaps a small delay is required for data to be sent over serial bus before trying to read.   Perhaps try setting the baud rate to something slower and see if the error goes away.  

But you haven't described what data rate you are expecting, how the data is formatted, how the serial bus is initialised, baud rates, what the USB adaptor is exactly, etc...  More details usually get you better advice here. 

Craig

[TSte]

Hi Franjo,

I've tried to sent you a pm, try to check your email. Btw try these two for troubleshooting:

https://knowledge.ni.com/KnowledgeArticleDetails?id=kA00Z0000019L38SAE

http://www.ni.com/tutorial/5008/en/

In the meantime I agree with Craig ("try setting the baud rate to something slower"), so tells us "the data rate you are expecting, how the data is formatted, how the serial bus is initialised, baud rates, what the USB adaptor is exactly, etc.." so more details.

Regards,

Stefano

[Franjo_Tonkovic]

Hi,

First of all thank you all for your suggestions.

I am already fighting this problem for 5 days and found and tried lot of things and I will try to summarize them here.

Till today I was running test on my table in office on the same cRIO configuration as on site and used usb-serial adapter for simulating data. It is low cost (10$) adapter and I do not have detail specs for it. Also the device I am communicating to is not very well documented but I have recorded raw serial data from it and I know that it sends me 200 bytes every 4s and these 200 bytes also contain 00 as internal sync.

Till yesterday I performed tests in the way the I dumped 200 bytes on cRIO port once in 4s and yesterday realized that device actually does not dump 200 bytes at serial port at once but it does it over 200ms. After I changed my simulation code to reproduce the this behavior I started lot of errors during communication and randomly loosing some data in communication (since I am reading slowly changing temperatures it is not so critical) but communication never completely stopped. This test ran over the nigh and I got satisfying results.

Today I went on site and all reading were fine for 1.5h and then I got error -1076807252 and completely stopped receiving data (actually same 16 bytes out of 200bytes). After cRIO restart communication started normally.

You can find my code from today's test attached and also a printscreen from test results.

I am using 57.6kbps and this predefined baud rate and cannot change it, but during my simulation tests in the office I tried lowering baud rate but it didn't help.

Someone mentioned that I am reading data to fast but I think that this problem does not have anything do with it since, I have alocated huge buffer which is 99.9% empty. My understanding is that error is happening on 16 bytes on UART and that CPU does not read those bytes fast enough. As I understand only setting RXFIFO = 1 is influencing this low level hardware readings.

Any comments?

Thanks

Regards

[cstorey]

Well, I understand less about your application and problem now.  You have 2 systems, do both use this USB adaptor?  Or is the USB adaptor a stand-in for the real data source?  Both have the same serial read errors?    

I suspect all of the VISA close/open is having an impact on the data read once you experience the first error.  What are your VISA timeouts?  Set them to quite long (1-2s), perhaps add a delay after you check bytes at port.  (Are you absolutely sure you need to use bytes at port?  Does the device only send data or error messages too?  If so your readings with <200 bytes might be messages and not data.)   

But all we can do is guess since you provide very little detail.  If you want help I suggest you clearly outline the system setup, what the USB adaptor device is doing, the data format (termination chars?) and as much detail as possible.  

I get you're frustrated, but without details its hard to provide advice.  Often when I have something that doesn't work and I take a step back and sped a few minutes clearly explaining my setup, my process and problem to someone I often solve my own problems.  

Craig

[Franjo_Tonkovic]

I am working on this already for a long time and many things happened so it is hard for my to explain everything shortly.

There are 2 systems, one is installed in power plant with  limited access and since I had problems there I asked NI to provide me a loaner which I can use for testing on my table. System in power plant communicates with  Wireless rotor monitoring ( http://www.mikrotrend.com/PDF/wrm-specifications.pdf)  and I do not have second WRM so on my table I use USB-serial adapter to simulate data. I am doing simulation based on raw serial data record from original system in power plant.

I have attached VI snippets and serial settings should be visible there.

Baud rate is fixed to 57.6kbps and as I know there is no termination character. Visa timeout is set to 1s. Buffer is configured huge.

Bytes at port is not necessary but my impression is that I get better reading if I put it there.  Since I do not have termination character, I have set my number of data on Visa Read to be slightly higher than I expect (200 bytes vs 250 bytes) and after I start getting bytes at port I wait for Visa read to timeout.

I am  not aware of any errors being sent over serial, my 16 bytes reading are valid readings of data, I do not see any disturbance in expected values.

Hope this makes it little bit clearer, I have to go on-site...

Regards

[crossrulz]

---

@Franjo_Tonkovic wrote:

Bytes at port is not necessary but my impression is that I get better reading if I put it there.  Since I do not have termination character, I have set my number of data on Visa Read to be slightly higher than I expect (200 bytes vs 250 bytes) and after I start getting bytes at port I wait for Visa read to timeout.

---

That seems like a good way to get a timeout error.  What is the exact protocol of the serial communications?  Since you have no termination character, then there should be some part of the message to state how long the message is.  You should be using that instead of arbitrarily reading X bytes just because it seems to work.