missing data

[staylor77]

We've a had a few instances of logging successfully started, stopped with live channel data being displayed but not stored in the tdms.  Two recent incidents:

- channel data present for about 80% of the test, stops before logging is stopped.  One channel continues uninterrupted, this is a plugin channel at a lower rate than the missing ones (2kHz).

- as above, but cdaq mdule channels are completely empty from the start.  

I would be grateful for help in avoiding data loss as our tests are costly to set up and run.

[Brad]

What version of FlexLogger are you using? How many channels are you logging? What is the data rate that you are logging (you can hover over the file in data pane while it's being logged and FlexLogger will report the data rate written to the file), how much memory is available, how much CPU load is being used, and how much disk % is being used (use task manager to get this info). We have made several improvements over the last few releases to improve performance so I wouldn't expect this problem from a newer release if the computer has the horse power to handle the throughput. Can you try upgrading to the latest FlexLogger to see how it performs?

You can also use the attached executable inside the zip to get information about the various Processing Elements (PEs) that are running while your FlexLogger project is open. In particular, I would be interested to see a screen capture of what this executable reports when things aren't working well...I would expect the number of items behind to start growing and this could explain why data is missing from the TDMS file if there's a backlog and it doesn't get to the TDMS PE before the test is stopped.

[staylor77]

Thanks for the quick reply

What version of FlexLogger are you using?

2021R2

How many channels are you logging?

What is the data rate that you are logging?

NI9234 modules x 5, 3 channels each, total 15 channels all at 25,6kHz

18 further calculated channels - mean and rms of above

one more plugin channel @ 2kHz

how much memory is available, how much CPU load is being used, and how much disk % is being used

unsure will add these to the project so I can log this too

You can also use the attached executable inside the zip to get information about the various Processing Elements (PEs) that are running while your FlexLogger project is open. 

Thanks I will experiment with this and report back 

[staylor77]

perhaps this warning is a clue.  I'm awaiting some advice from our it team on the hubs/switches question

[staylor77]

the diagnostic vi seems broken I'm afraid.  Flexlogger is open when I run this.

[staylor77]

some more context - this is the same project that has given us problems, logging successfully at the moment when I test

[Brad]

Here's an updated exe that should work and it keeps a log of how many items behind each PE is while your test is running. Make sure to start it after your project is done loading, but before you start the test.

Could you please send us your project so we can try to reproduce the issue in house. Would it be possible to try the latest version of FlexLogger to see how it performs for you?

Let me know how the test goes while using the attached Get PE Stats.exe to monitor if any PEs are getting behind. You can copy the "concatenated string" into a text editor and send me the resulting file for further investigation.

Thanks,

Brad

[staylor77]

This is running just now while logging - test is performing ok at the moment.

DaqPlugin18 ExtTagsSetter1 ExternalTags3 LoggingGate17 LoggingHeartbeat15 Root_Comm_SysSuper ScalingAndCal19 SetpointGen5 SysSuper TDMSLoggingPlugin12 TDMSLoggingPlugin13 TagSetter6 ValenceInput4 alarm14 chanMon7 local_NODE loggingEvents16 niSystemResourceMonitor2 stats11 sync9 testStopTrig8 timerTrig10
1835788 463559 463558 503118 1240755 465011 1835792 463557 1142276 463564 463564 463555 1284590 463552 463553 460659 1386887 1030617 8952208 463550 463558 1622828
2347788 463559 463558 504198 1116915 465490 2347792 463557 1210884 463564 463564 463555 1437230 463552 463553 460659 1526196 1062617 9009808 463550 463558 1145068
2859788 463559 463558 1118054 1241075 465011 2859792 463557 1210884 463564 463564 463555 1590190 463552 463553 460659 2195270 1091417 9067408 463550 463558 1455148
3371788 463559 463558 1120214 1241075 465011 3371792 463557 1279492 463564 463564 463555 1742830 463552 463553 460659 3330796 1120217 9125008 463550 463558 1765868

[staylor77]

Hi, a little update on this problem:

Updated from 2021R2 to 2021R3, and the problem got really bad, I couldn't go 10 minutes without the incoming channel data freezing up.

I was able to run the diagnostic while the channel stream was frozen (it doesn't come back without killing and reopening flexlogger).

while frozen

DaqPlugin21 ExtTagsSetter1 ExternalTags3 FFT18 FFT19 FFT20 LoggingGate17 LoggingHeartbeat15 Root_Comm_SysSuper ScalingAndCal22 SetpointGen5 SysSuper TDMSLoggingPlugin12 TDMSLoggingPlugin13 TagSetter6 ValenceInput2 alarm14 chanMon9 local_NODE loggingEvents16 niSystemResourceMonitor8 stats11 sync4 testStopTrig10 timerTrig7
4073740 463559 463558 2740214 2740214 2740214 504198 1241747 465011 4073744 463557 2043899 463564 463564 463555 1886862 463552 463553 460659 2673859 1142937 9217744 463550 463559 2850379
4073740 463559 463558 2740214 2740214 2740214 504198 1241427 465011 4073744 463557 2043899 463564 463564 463555 1886862 463552 463553 460659 2673436 1142937 9217744 463550 463559 2850379

[staylor77]

Another observation today - flexlogger failing to start, no channels streaming in.

diagnosti vi indicates error below:

NI Skyline Message AMQP (Sync).lvclass:Publish Synchronous Generic.vi:6140001<ERR>
Request timed out.

<b>Complete call chain:</b>
NI Skyline Message AMQP (Sync).lvclass:Publish Synchronous Generic.vi:6140001
Get PE Stats.vi

-251001