RT program stops running after ~13 hrs. Strange errors in log.

[InfiniteNothing]

I have an RT application that is essentially a datalogger from an 7852R board. The FPGA grabs the data at some interval (20 ms, 10 ms, 20 ms...) and passed the data to Labview with a DMA FIFO. The RT system passes the data to a client running windows through TCPIP. I did a long run test over the weekend (running the program) and it seemed to crash after 13 hours. As far as I can tell, the rt program just stopped working. For the test, I had disconnected the host computer (right click the remote computer in the project and select disconnect) when I connected, it looked the program on the RT box was just waiting to be run. The only way for the RT program to programatically stop running is with a stop button on the front pannel so I'm not sure why it just stopped. I checked my error log and this is what I got:

 2/20/09 6:54:38.924 PM

source\mgcore\MemoryManager.cpp(137) : DWarnInternal: Memory error 2 in DSSetHandleSize

$Id: //labview/branches/Saturn/dev/source/mgcore/MemoryManager.cpp#16 $

0x01EFEB1F - <unknown> + 0

0x01EFF27E - <unknown> + 0

0x01E4A00A - <unknown> + 0

0x188A01AF - <unknown> + 0

0x18F2B2C0 - <unknown> + 0

0x01D3A232 - <unknown> + 0

0x0008E0EC - <unknown> + 0

And that repeats about 50 times. Oddly, I'm not sure the time stamp matches the time of the crash (~6am the next day according to the log files on the client)

I'm running an extended again, this time with "retain wire values".   I was just hoping someone had an idea.

[Ben]

You are probably running out of memory. Monitor your memory usage while the app is running and if it steadily climbing, it will only be a matter of time before the crash.

THose logs are gereally in UTC time.

Have fun!

Ben

[InfiniteNothing]

RE: UTC... I would expect it to be 8 hours ahead, not 12 behind. Oh well, maybe the system clock is off.

Running out of memory seems possible. I'm watching the memory right now with the RT System Manager and it seems stable at ~34% for the last hour. I'll keep an eye on it. I'm wondering if maybe I should do garbage collection once every 4 hours or something. 

Message Edited by InfiniteNothing on 02-23-2009 11:48 AM

[Cory_K]

---

InfiniteNothing wrote:

I'm wondering if maybe I should do garbage collection once every 4 hours or something. 

---

You could just break up the file into several smaller files if memory is an issue.

I know its a pain in the neck, but its an option.

Just come up with some simple logic, For example, start a new file :
- after __ time

- after __ points

- user clicks 'new file' button

 - etc

Just throwing an option out there

Message Edited by Cory K on 02-23-2009 02:16 PM

[InfiniteNothing]

There is no file on the RT system. The RT should just grab the data and send it. There are some fancy queues, and some arrays on shift registers where memory might (but shouldn't accumulate). Actually, I've been watching the program run  (since about the start of this thread) and it's dead solid at 34% memory. There might be some case though where the memory swells (like say, the client disconnects uncleanly.)

Message Edited by InfiniteNothing on 02-23-2009 12:35 PM

[ColeTrain]

InfiniteNothing,

Has your program crashed again since your last post?  If this is still happening, will you attach your error log so we can take a look at what's failing?  Thanks,