SNTP Plug-in Development Progress

[JodyK]

We have a client who would like to implement SNTP time syncing for their laboratory. They have roughly 10 networked PXI Real-Time systems and have an NTP server running on their local network. The RT nodes are responsible for controlling and acquiring data during both short (24 hour) and long (6 month) duration tests. Clock drift for timestamps becomes an issue on the longer duration tests as well as just on day-to-day operation of the nodes.

I've pulled down the beta plugin and tested it on an RT system here in the office, but I wanted to ask some questions before attempting to integrate this with my customer's system.

• What is the timeline for a final release? Should I expect to see this plugin complete in 6 months? 3 months?
• What is preventing this from being considered a final release? Are there stability issues or missing features?
• How often does syncing occur? Will the clock be synced every hour? Every 10 minutes? Once a day?

Stability is really key to our application. Any data you can provide on how much testing or use this plugin has seen would be incredibley helpful.

Jody Koplo

LabVIEW Architect

DMC

www.dmcinfo.com

[Jeremy_L]

Hi Jody,

1.  Currently we are still evaluating the usability, functionality, and gathering feeback from customers.  We have not received any information from customers requesting a fully released version and timeline, so a tentative release plan would probably be by the end of this year (2011).  If you have an opportunity that requires a final version earlier, please contact your local NI sales representative and we will see what we can do.

2.  We do have some additional work to complete before considering a final release.  For example, we'd like to add the SNTP server IP address as a configuration field in MAX and RT system webpage.  We'd also like to do some more testing to confirm and fully characterize our clock stability.

3.  I believe by default, the sync interval is set to 60 seconds.  However, you can change the sync rate based on your needs by changing the value through an .ini file.  In the future, we may also consider exposing this setting in MAX and RT system webpage for easier customization.  In your environment of 10 systems, it may be okay to change your sync interval to maybe 5 seconds (if your NTP server allows it) and you don't have a lot of additional network traffic.

To change the sync interval, add this key/value to the /ni-rt/system/ni-timesync/ni-timesync-sntp.ini file:

source.sntp.interval=60; (change the number to suit your needs)

Thanks for your interest!

[JodyK]

Jeremy-

Thanks a ton for your reply. I mentioned the possibility of NTP syncing to my customer and they are very interested in implementing it. It would actually solve some additional issues they've been having and make their overall system better at interfacing with some of their older hardware.

With that in mind, it sounds like the largest barrier to a final release is the user interface for configuration. I understand how important this is for general customer release, but it really doesn't matter to this application.

Is there any real risk in integrating the beta with the current systems? I'm mostly concerned about failure modes. Are there any failure modes that result in unexpected behavior (clock being set to 6:00:00.000 PM 12/31/1903 or something similar)? Is there a way for me to read some sort of sync status from my RT code?

It looks like we'll end up moving forward with this. Let me know if there is any feedback I can provide to you guys that will help with development.

Jody Koplo

LabVIEW Architect

DMC

www.dmcinfo.com

[Jeremy_L]

Hi Jody,

There is a property node the user can query with a VI to check sync status and sync offset.  Our notion of being "sync'ed" can be user controlled, but with SNTP, it defaults to 1ms.  The biggest risk today in our opinion is we cannot completely guarantee the performance of the plugin with the limited testing we've done.  We've tested with some normal (what you'd expect in the field) conditions, but not as much on the edge cases.  Given that, we have an idea of our performance, but cannot bound and characterize our stability to the level of claiming a guaranteed number.

With SNTP, a lot can depend on the target, sync interval, and the quality of the SNTP host.  If you are going to deploy this, I would encourage you to check sync status, sync offset, and just system time regularly to ensure that it is still reasonable.

There aren't any known failure modes leading to trampled clock settings that we know of.

On the brighter side, if you are using a local ntp server, it's possible your stability will be much better than normally accessing a ntp server a couple of hops away.

Additionally, it seems with your application targeted for 24-hour to 6-month duration tests, the sync interval of 60 seconds (by default) should be okay, but there will be milliseconds-level drift.  Do you have an idea of what is considered acceptable drift for your application so you can possibly adjust the sync interval to fit your needs?

[JodyK]

Jeremy-

We're actually pretty tolerant to drift. Luckily, the nodes are running independent tests, so there is no real requirement for sync accuracy between them. It's more an issue of the node time drifting by minutes or even hours with respect to the rest of the lab. If we could keep everything plus/minus 1 second I think they would be happy. At plus/minus 10 msec the drift could be considered effectively zero. I guarantee they will ask me what the potential drift is, so it's good that I have an approximation.

I found the property node you mentioned and did some testing in the office and I think I uncovered a bug.

I created a simple real-time executable that reads the system time and all of the SNTP status variables. I then read all of this at 10hz and publish it to the RT Console. Next, I unplugged the network cable. Everything continues to be pushed to screen for a several seconds ( <60 seconds) and then it appears that the property node begins to block the loop execution and I only get a screen update every 60 seconds or so. The SNTP status variables still indicate that everything is "good" and that no synchronization disruption has occurred. Plugging the network back causes the 10hz rate to resume after the next 60 sec timeout

I'm assuming that the first timeout after the disconnect is the next SNTP sync period executing. Then I'm assuming the SNTP call has a 60 second timeout which blocks the property node from completing execution. I'm not sure why my status still returns "good".

Any ideas? I can send you my tester VI but it should be really easy to reproduce. Thanks.

Jody Koplo

LabVIEW Architect

DMC

www.dmcinfo.com

[Jeremy_L]

Hi Jody,

Thanks for the information.  Can you send the VI just to be sure?  We'll take a look at it and see what's going on.  Sorry for the delayed response.

[JodyK]

Sure thing. It's really basic. I built it into an rtexe and ran it on a PXI. The RT Console time display will hang if connectivity is disturbed. If you look closely, you'll notice the cursor executed the carriage return but never printed the time. This is what leads me to believe the property node is hanging execution. This should be easily reproducible.

I've seen a couple other oddities, but nothing I've isolated. We've had some router issues here in the office, so I'll chalk them up to that for now.

Thanks again for being responsive. This is an exciting feature for my customer so it's good to see that it's getting attention from you guys.

Jody Koplo

LabVIEW Architect

DMC

www.dmcinfo.com

[hewu]

About NI-TimeSync 1.2 (beta): The installer of this system software for SNTP synchronization supports LV 2010.

We tried to setup a new configuration under LabVIEW 2011, but this 1.2 installer does not support LV 2011.

Can you create a installer of the NI-TimeSync 1.2 software for LV 2011?

Thank you

Helmut

[JimMacD]

I'm glad I found this page. About a year ago I was looking for a way to sync cRIO platforms with a SNTP server so that timestamps from shared variables would log correctly. I found this web page which lead me to think this was already released: http://zone.ni.com/devzone/cda/tut/p/id/11466  Looks like it was a screenshot from a development version.

When is NI TimeSync 1.2 scheduled to be released?

Thanks

-Jim MacDonald

CLD

[Jeremy_L]

The 1.2 installer marked d15 (currently linked on top of SNTP page) should have LV2011 support.  Please try this update.

Thanks,

Jeremy