PXI Timestamps Offset from Master

[Jagdeb]

I have a PXI 1082 chassis with an 8880 controller, and a 6355 in expansion slot 5. The controller is synchronized via PTP to a master clock on the network, but when I collect samples through DAQmx all the timestamps are 23 seconds early, even though the system time (from get system time vi) is correct. This only happens when building and deploying as a startup program, the timestamps are correct when running from development mode.

There is a near-identical chassis running identical software, but the timestamps always match the system time. The only difference is that the other chassis also has an RTD module in slot 4. The sample clock source is set as "OnboardClock" when creating the DAQmx tasks.

I've been trying to troubleshoot this for a while but I'm quite stuck. I've tried using the Ni-Sync library but it throws an error stating it can't be used on remote systems. I've grabbing the reference clocks and other properties from tasks to compare between PXIs, but they're all identical.

[DavidF_NI]

Hi Jagdeb, 

Are you saying that with a built program the timestamps provided by DAQmx Waveform reads are 23 seconds earlier than the time returned from the Get System Time vi that is run at approximately the same time? 

Best,

David F.

Technical Support Engineer

National Instruments

www.ni.com/support

[Jagdeb]

Yes that's about it.

[DavidF_NI]

Jegdeb, 

So this had me stumped for a bit. Does this happen on the first read? DAQmx will pull the system clock on the first read, but t0 is a calculation after that. 

Best, 

David F.

Technical Support Engineer

National Instruments

www.ni.com/support

[Jagdeb]

Hi David,

Been going back and forth for a while because we have two near-identical PXIs running identical software, and the issue happens on one but not the other. I have been switching out hardware and configurations, reinstalling drivers, etc. trying to narrow down any differences that could cause this, but I went down a similar train of thought with the initial timestamp yesterday and the issue seems to be solved.

The system pretty much logs continuously, in normal operation it'll only set up/start the DAQmx task once and then run for days. I'm not sure where the system time is "saved" between restarts, but I've noticed that Ni-Timesync doesn't seem to actually overwrite it. Every time we start up the PXIs with it enabled, they both end up with a fault/warning in Distributed System Manager saying the system clock has been overwritten. Watching the clock in NI-MAX as it starts up, it's also possible to see it begin with whatever Timezone/System Time I manually save for a few seconds before the ptp synchronization kicks in.

Anyway, I pulled the hardware ref for the clock on the PXI, then used a property node to get ClkSyncEnabled and ClockState under IEEE 1588-2008>Basic Information, and OffsetFromMstr under  IEEE 1588-2008>Synchronization Information. I then prevented the DAQmx task from being initialized until ClkSyncEnabled is True, ClockState is Slave, and OffsetFromMstr is less than 5000 (abs). ClockState alone would probably be sufficient, but pausing for the extra 30-60 seconds on startup while it's fine-tuned doesn't really hurt our use case.

This seems to solve the problem, I guess the DAQmx task was starting before NI-Timesync had a chance to adjust the system clock. I am not sure why this would happen on one system but not the other, or why it would be so consistent. Even when I cut out our (large) program completely and just ran a basic one-channel acquisition loop, PXI1 would always start up fine, and PXI2 would have an incorrect timestamp unless I explicitly waited for the ptp server.

[DavidF_NI]

Jagdeb, 

I am glad you were able to find that workaround and force the Clock Sync to occur before starting the DAQmx task. I am not sure why the systems behave differently. I guess OS schedulers have their own personalities. 

Best,

David F.

Technical Support Engineer

National Instruments

www.ni.com/support