How to manage disk use with extra logging enabled

[Staab_Engineering]

As part of a debugging effort to isolate crashes and memory leaks in our LV app, we activated these extra keys in the ni-rt.ini file (at the behest of NI's support staff):

[StartUp]

# Prevents LVRT from rebooting device into safe mode when it crashes

YouOnlyLiveTwice = FALSE

[LVRT]

# These add a bunch more detailed logging that helps debug crashes

DWarnDialogMultiples=True

promoteDWarnInternals=True

DPrintfLogging=True

Debugging=True

numstatusitemstolog=99999

With these keys enabled, we see a new /var/local/natinst/tracelogs directory on the appear, and we see much more output in the /var/local/natinst/log directory. On one of our cRIOs, we saw the /var/local/natinst/log/kern.log file grow to 72 MB, and many of the log files are months old.

How do we prevent the disk from getting filled up with log output? I know that /etc/logrotate.conf and the associated cronjob are responsible for managing normal output, but it seems it doesn't cover this new output. Do we need to modify logrotate.conf in a particular way? How do we flush these directories safely without offending apps that write to them?

[BradM]

kern.log should not be growing with the additions made to the ini file. This is indicative of some OS-level issue. All of this is to say, what is in the kernel log?

We can discuss what needs to be done, but the answer is dependant on the version of the OS you're using (work has been put into cleaning up the system logging, so need to know which version we're starting from)

[Staab_Engineering]

I had so much trouble trying to download the file remotely that I finally just deleted it on the cRIO's disk. I expect that whatever made it grow like that will give me new info soon, so I'll keep you abreast of its contents.

# cat /proc/version

Linux version 3.2.35-rt52-2.10.0f0 (lvbuild@hayes.amer.corp.natinst.com) (gcc version 4.7.2 (GCC) ) #1 SMP PREEMPT RT Mon Dec 8 14:50:27 CST 2014

And MAX says the "firmware" is version 2.5.0f1.

I'm about to format the device and upgrade it to firmware 3.0.1f0.