Skip to contentSkip to conversation

ntp updates for 2.6.31

GIT PULL Thomas Gleixner 6 participants 21 messages Jun 15, 2009 – Jun 24, 2009 Timekeeping, clocksource core, NTP, alarmtimer

lore
Share & tools
lore
#0 lore raw

Linus,

Please pull the latest timers-for-linus-ntp git tree from:

git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git timers-for-linus-ntp

Thanks,

tglx

------------------>
John Stultz (2):
ntp: adjust SHIFT_PLL to improve NTP convergence
ntp: fix comment typos

 include/linux/timex.h |   42 +++++++++++++++++++++++++++++++-----------
 1 files changed, 31 insertions(+), 11 deletions(-)
include/linux/timex.h +31 -11
index aa3475f..9910e3b 100644
@@ -170,17 +170,37 @@ struct timex {
 #include <asm/timex.h>
 
 /*
- * SHIFT_KG and SHIFT_KF establish the damping of the PLL and are chosen
- * for a slightly underdamped convergence characteristic. SHIFT_KH
- * establishes the damping of the FLL and is chosen by wisdom and black
- * art.
+ * SHIFT_PLL is used as a dampening factor to define how much we
+ * adjust the frequency correction for a given offset in PLL mode.
+ * It also used in dampening the offset correction, to define how
+ * much of the current value in time_offset we correct for each
+ * second. Changing this value changes the stiffness of the ntp
+ * adjustment code. A lower value makes it more flexible, reducing
+ * NTP convergence time. A higher value makes it stiffer, increasing
+ * convergence time, but making the clock more stable.
  *
- * MAXTC establishes the maximum time constant of the PLL. With the
- * SHIFT_KG and SHIFT_KF values given and a time constant range from
- * zero to MAXTC, the PLL will converge in 15 minutes to 16 hours,
- * respectively.
+ * In David Mills' nanokernel reference implementation SHIFT_PLL is 4.
+ * However this seems to increase convergence time much too long.
+ *
+ * https://lists.ntp.org/pipermail/hackers/2008-January/003487.html
+ *
+ * In the above mailing list discussion, it seems the value of 4
+ * was appropriate for other Unix systems with HZ=100, and that
+ * SHIFT_PLL should be decreased as HZ increases. However, Linux's
+ * clock steering implementation is HZ independent.
+ *
+ * Through experimentation, a SHIFT_PLL value of 2 was found to allow
+ * for fast convergence (very similar to the NTPv3 code used prior to
+ * v2.6.19), with good clock stability.
+ *
+ *
+ * SHIFT_FLL is used as a dampening factor to define how much we
+ * adjust the frequency correction for a given offset in FLL mode.
+ * In David Mills' nanokernel reference implementation SHIFT_FLL is 2.
+ *
+ * MAXTC establishes the maximum time constant of the PLL.
  */
-#define SHIFT_PLL	4	/* PLL frequency factor (shift) */
+#define SHIFT_PLL	2	/* PLL frequency factor (shift) */
 #define SHIFT_FLL	2	/* FLL frequency factor (shift) */
 #define MAXTC		10	/* maximum time constant (shift) */
 
@@ -192,10 +212,10 @@ struct timex {
 #define SHIFT_USEC 16		/* frequency offset scale (shift) */
 #define PPM_SCALE ((s64)NSEC_PER_USEC << (NTP_SCALE_SHIFT - SHIFT_USEC))
 #define PPM_SCALE_INV_SHIFT 19
-#define PPM_SCALE_INV ((1ll << (PPM_SCALE_INV_SHIFT + NTP_SCALE_SHIFT)) / \
+#define PPM_SCALE_INV ((1LL << (PPM_SCALE_INV_SHIFT + NTP_SCALE_SHIFT)) / \
 		       PPM_SCALE + 1)
 
-#define MAXPHASE 500000000l	/* max phase error (ns) */
+#define MAXPHASE 500000000L	/* max phase error (ns) */
 #define MAXFREQ 500000		/* max frequency error (ns/s) */
 #define MAXFREQ_SCALED ((s64)MAXFREQ << NTP_SCALE_SHIFT)
 #define MINSEC 256		/* min interval between updates (s) */
#1 lore raw
Quoted 6 lines · Thomas Gleixner

Linus,

Please pull the latest timers-for-linus-ntp git tree from:

  git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git timers-for-linus-ntp

Quoted 8 lines

Thanks,

       tglx

------------------>
John Stultz (2):
     ntp: adjust SHIFT_PLL to improve NTP convergence
     ntp: fix comment typos

Thomas,
Could we hold off on pushing this? I'm still working with Miroslav
to try to work out a solution here from ntpd user-side.

thanks
-john

#2 lore raw
Quoted 11 lines · john stultz
Quoted 10 lines · Thomas Gleixner

Linus,

Please pull the latest timers-for-linus-ntp git tree from:

  git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git timers-for-linus-ntp

Thanks,

       tglx

Quoted 8 lines
Quoted 4 lines

------------------>
John Stultz (2):
     ntp: adjust SHIFT_PLL to improve NTP convergence
     ntp: fix comment typos

Thomas,
  Could we hold off on pushing this? I'm still working with Miroslav to try to work out a solution here from ntpd user-side.

Linus,
You probably didn't see this before merging. Could you yank the
above two patches? Miroslav (RH package maintainer for ntpd), has voiced concerns that the SHIFT_PLL patch breaks the NTP design and is worried it may negatively effect NTP networks of systems running with different SHIFT_PLL values.

While the patch does greatly improve NTP convergence times, and so far no negative results have been seen in tests, its out of an abundance of caution and a desire to keep the adjtimex behavior stable that I requested Thomas and Ingo to hold off on merging this patch, while I work with Miroslav to see if we cannot get the same benefit by adjusting the userspace NTPd.

So if you could revert the two patches until we either sort things out in userspace or I resubmit, I'd appreciate it.

Sorry for the mixup here.

thanks
-john

#3 lore raw
Quoted 7 lines · john stultz

Linus,
You probably didn't see this before merging. Could you yank the
above two patches? Miroslav (RH package maintainer for ntpd), has voiced concerns that the SHIFT_PLL patch breaks the NTP design and is worried it may negatively effect NTP networks of systems running with different SHIFT_PLL values.

Quoted 6 lines

While the patch does greatly improve NTP convergence times, and so far no negative results have been seen in tests, its out of an abundance of caution and a desire to keep the adjtimex behavior stable that I requested Thomas and Ingo to hold off on merging this patch, while I work with Miroslav to see if we cannot get the same benefit by adjusting the userspace NTPd.

As i explained it in previous threads i disagree. The only technically correct direction is to improve NTP stabilization and convergence times as much as possible. [*]

( [*] Without getting into over-compensation and without starting to
oscillate instead of converging - that would be a bug, but such a bug has not been reported so far. )

The 'concern' voiced was that: "what if other OSs converge slower in a cluster and now we have a faster OS in the mix". This absolutely ignores the other 99% of cases where people would have crappier convergence after the revert and for no good reason.

And even regarding that 1% example, well, duh: different OSes have different convergence times, fundamentally so - such as Linux had a very slow convergence time from about 2.6.18 up to recent kernels due to a bug. Now it's converging even faster ...

So i dont think that "Linux is too good" is a good basis to artificially make Linux's NTP code crappier. Really. We dont 'play nice' by being equally crappy.

Each OS should converge back to the correct time _as fast as physically possible_. If this is a problem and if someone wants crappy time and longer periods of convergence for some odd reason then that header file change can be edited by hand even. It's not like it's that hard to change, if there's genuine interest.

So i'm against any revert on this basis. If another basis comes up we can reconsider of course. What do you think?

Ingo

#4 lore raw
Quoted 8 lines · Ingo Molnar

Each OS should converge back to the correct time _as fast as physically possible_. If this is a problem and if someone wants crappy time and longer periods of convergence for some odd reason then that header file change can be edited by hand even. It's not like it's that hard to change, if there's genuine interest.

So i'm against any revert on this basis. If another basis comes up we can reconsider of course. What do you think?

I completely agree.

Consistent convergence across different OSs is a wet dream.

We see even different behaviour across kernel versions :) Also I recently looked at an embedded system running the same kernel version as a PC in the same network. Same version of user space tools. Main difference aside the arch was HZ (100 vs. 1000). The PC convergence time was about 40% higher than the embedded systems.

Thanks,

tglx

#5 lore raw
In reply to Ingo Molnar
Quoted 16 lines · Ingo Molnar
Quoted 13 lines · john stultz

Linus,
You probably didn't see this before merging. Could you yank the
above two patches? Miroslav (RH package maintainer for ntpd), has voiced concerns that the SHIFT_PLL patch breaks the NTP design and is worried it may negatively effect NTP networks of systems running with different SHIFT_PLL values.

While the patch does greatly improve NTP convergence times, and so far no negative results have been seen in tests, its out of an abundance of caution and a desire to keep the adjtimex behavior stable that I requested Thomas and Ingo to hold off on merging this patch, while I work with Miroslav to see if we cannot get the same benefit by adjusting the userspace NTPd.

[..]

Quoted 8 lines

Each OS should converge back to the correct time _as fast as physically possible_. If this is a problem and if someone wants crappy time and longer periods of convergence for some odd reason then that header file change can be edited by hand even. It's not like it's that hard to change, if there's genuine interest.

So i'm against any revert on this basis. If another basis comes up we can reconsider of course. What do you think?

I think the most important one is following the NTP specification.

If Linux really needs to have the fastest PLL, could it be done by modifying the time constant passed in adjtimex structure instead of changing SHIFT_PLL? The PLL response will be exactly the same, but it will allow the applications (and admins) to detect that it is different than expected.

Something like:

kernel/time/ntp.c +2 -0
@@ -425,6 +425,8 @@
 		time_constant = txc->constant;
 		if (!(time_status & STA_NANO))
 			time_constant += 4;
+		/* We want faster PLL */ 
+		time_constant -= 2;
 		time_constant = min(time_constant, (long)MAXTC);
 		time_constant = max(time_constant, 0l);
 	}

Thanks,

Signature · 1 line
Miroslav Lichvar
#6 lore raw
Quoted 42 lines · Miroslav Lichvar
Quoted 16 lines · Ingo Molnar
Quoted 13 lines · john stultz

Linus,
You probably didn't see this before merging. Could you yank the
above two patches? Miroslav (RH package maintainer for ntpd), has voiced concerns that the SHIFT_PLL patch breaks the NTP design and is worried it may negatively effect NTP networks of systems running with different SHIFT_PLL values.

While the patch does greatly improve NTP convergence times, and so far no negative results have been seen in tests, its out of an abundance of caution and a desire to keep the adjtimex behavior stable that I requested Thomas and Ingo to hold off on merging this patch, while I work with Miroslav to see if we cannot get the same benefit by adjusting the userspace NTPd.

[..]

Quoted 8 lines

Each OS should converge back to the correct time _as fast as physically possible_. If this is a problem and if someone wants crappy time and longer periods of convergence for some odd reason then that header file change can be edited by hand even. It's not like it's that hard to change, if there's genuine interest.

So i'm against any revert on this basis. If another basis comes up we can reconsider of course. What do you think?

I think the most important one is following the NTP specification.

If Linux really needs to have the fastest PLL, could it be done by modifying the time constant passed in adjtimex structure instead of changing SHIFT_PLL? The PLL response will be exactly the same, but it will allow the applications (and admins) to detect that it is different than expected.

Something like:

kernel/time/ntp.c line 432

Quoted 8 lines
kernel/time/ntp.c +2 -0
@@ -425,6 +425,8 @@
 		time_constant = txc->constant;
 		if (!(time_status & STA_NANO))
 			time_constant += 4;
+		/* We want faster PLL */ 
+		time_constant -= 2;
 		time_constant = min(time_constant, (long)MAXTC);
 		time_constant = max(time_constant, 0l);
 	}

It looks mathematically equivalent, although I've not had time to test it yet. Probably needs a bigger comment :)

The nice thing with this version is that we're able to expose that the behavior would be different then other systems, but the other side of that coin might be that when the user specifies a time_constant value, the interface will show a different one being used. This might cause some bug reports saying the interface isn't responding properly, or something. Although this is already the case for !STA_NANO, and so far few have noticed.

thanks
-john

#7 lore raw
Quoted 56 lines · John Stultz
Quoted 42 lines · Miroslav Lichvar
Quoted 16 lines · Ingo Molnar
Quoted 13 lines · john stultz

Linus,
You probably didn't see this before merging. Could you yank the
above two patches? Miroslav (RH package maintainer for ntpd), has voiced concerns that the SHIFT_PLL patch breaks the NTP design and is worried it may negatively effect NTP networks of systems running with different SHIFT_PLL values.

While the patch does greatly improve NTP convergence times, and so far no negative results have been seen in tests, its out of an abundance of caution and a desire to keep the adjtimex behavior stable that I requested Thomas and Ingo to hold off on merging this patch, while I work with Miroslav to see if we cannot get the same benefit by adjusting the userspace NTPd.

[..]

Quoted 8 lines

Each OS should converge back to the correct time _as fast as physically possible_. If this is a problem and if someone wants crappy time and longer periods of convergence for some odd reason then that header file change can be edited by hand even. It's not like it's that hard to change, if there's genuine interest.

So i'm against any revert on this basis. If another basis comes up we can reconsider of course. What do you think?

I think the most important one is following the NTP specification.

If Linux really needs to have the fastest PLL, could it be done by modifying the time constant passed in adjtimex structure instead of changing SHIFT_PLL? The PLL response will be exactly the same, but it will allow the applications (and admins) to detect that it is different than expected.

Something like:

kernel/time/ntp.c line 432

Quoted 8 lines
kernel/time/ntp.c +2 -0
@@ -425,6 +425,8 @@
 		time_constant = txc->constant;
 		if (!(time_status & STA_NANO))
 			time_constant += 4;
+		/* We want faster PLL */ 
+		time_constant -= 2;
 		time_constant = min(time_constant, (long)MAXTC);
 		time_constant = max(time_constant, 0l);
 	}

It looks mathematically equivalent, although I've not had time to test it yet. Probably needs a bigger comment :)

Quoted 7 lines

The nice thing with this version is that we're able to expose that the behavior would be different then other systems, but the other side of that coin might be that when the user specifies a time_constant value, the interface will show a different one being used. This might cause some bug reports saying the interface isn't responding properly, or something. Although this is already the case for !STA_NANO, and so far few have noticed.

Sounds good to me. It feels a bit quirky that we 'correct' the user-space provided parameter by 2 ... Definitely needs a big comment.

Ingo

#8 lore raw
In reply to John Stultz
Quoted 25 lines · John Stultz
Quoted 11 lines · Miroslav Lichvar

If Linux really needs to have the fastest PLL, could it be done by modifying the time constant passed in adjtimex structure instead of changing SHIFT_PLL? The PLL response will be exactly the same, but it will allow the applications (and admins) to detect that it is different than expected.

Something like:

kernel/time/ntp.c line 432

Quoted 8 lines
kernel/time/ntp.c +2 -0
@@ -425,6 +425,8 @@
 		time_constant = txc->constant;
 		if (!(time_status & STA_NANO))
 			time_constant += 4;
+		/* We want faster PLL */ 
+		time_constant -= 2;
 		time_constant = min(time_constant, (long)MAXTC);
 		time_constant = max(time_constant, 0l);
 	}

It looks mathematically equivalent, although I've not had time to test it yet. Probably needs a bigger comment :)

Quoted 7 lines

The nice thing with this version is that we're able to expose that the behavior would be different then other systems, but the other side of that coin might be that when the user specifies a time_constant value, the interface will show a different one being used. This might cause some bug reports saying the interface isn't responding properly, or something. Although this is already the case for !STA_NANO, and so far few have noticed.

I have checked the NTP sources and the returned time constant is used only for reporting, at least for NTP it shouldn't cause any problems.

Returning correct time constant will be very useful if NTP developers decide to use lower values or have it configurable as decreasing the constant by another two will make the PLL unstable.

Still, I'd really like to see the original behavior restored. Most of the users complaining about slow convergence are probably just hitting the calibration problem, which needs to be fixed by other means than making PLL faster. Also, users of other systems seem to be happy with their slow convergence. At least that's the impression I have from NTP lists.

Thanks,

Signature · 1 line
Miroslav Lichvar
#9 lore raw
Quoted 6 lines · Miroslav Lichvar

Still, I'd really like to see the original behavior restored. Most of the users complaining about slow convergence are probably just hitting the calibration problem, which needs to be fixed by other means than making PLL faster. Also, users of other systems seem to be happy with their slow convergence. At least that's the impression I have from NTP lists.

Wouldnt the goal be to calibrate as fast as possible? (Without any bad oscillation)

Ingo

#10 lore raw
Quoted 11 lines · Ingo Molnar
Quoted 6 lines · Miroslav Lichvar

Still, I'd really like to see the original behavior restored. Most of the users complaining about slow convergence are probably just hitting the calibration problem, which needs to be fixed by other means than making PLL faster. Also, users of other systems seem to be happy with their slow convergence. At least that's the impression I have from NTP lists.

Wouldnt the goal be to calibrate as fast as possible? (Without any bad oscillation)

I believe he means the TSC calibration error issue, where every boot the TSC calibration varies by 30-80ppm. This makes it hard for systems to stay in NTP sync after a reboot, because ntpd has to search for a new freq (and the SHIFT_PLL & time_constant values control how fast that happens).

While the TSC calibration is an issue, there is also the fact that NTP's slow convergence model (which is "by design", for good or bad) doesn't seem to handle thermal environment changes quickly enough to keep close sync.

Now, weather we fix the change by tweaking ntpd or the kernel, I still think is a question that we've not answered well. Even though I'm of the opinion something needs to change, I'm not yet convinced of which side is the right side to fix. And that is why I requested we hold off on merging the SHIFT_PLL patch.

And really, if you look at Miroslav's patch, which is mathematically equivalent to the SHIFT_PLL change, all we're doing is decreasing what ntpd gave as the time_constant us by two. So the question is, why is that fix best done in the kernel, instead of making ntpd reduce what it passes in to the kernel?

thanks
-john

#11 lore raw
In reply to Ingo Molnar
Quoted 11 lines · Ingo Molnar
Quoted 6 lines · Miroslav Lichvar

Still, I'd really like to see the original behavior restored. Most of the users complaining about slow convergence are probably just hitting the calibration problem, which needs to be fixed by other means than making PLL faster. Also, users of other systems seem to be happy with their slow convergence. At least that's the impression I have from NTP lists.

Wouldnt the goal be to calibrate as fast as possible? (Without any bad oscillation)

Not really. It depends on how noisy is the input signal. On an idle LAN the jitter is just few microseconds, but over internet it easily reaches miliseconds. Over a certain point faster PLL will just make things worse.

PLL is mainly about handling the signal noise, frequency adjusting is secondary. When the noise is very low or the update interval is long enough, the frequency variations caused by temperature changes will dominate the signal noise and this is where FLL should kick in.

The PLL/FLL switching is controlled by update interval. Ideally it would be adaptive, but NTP is not that sophisticated. By default, FLL is enabled when the interval is longer than 2048 seconds. This is of course not the optimal value for all systems.

Unfortunately in kernel it can be configured only to 2048 or 256 and NTP never uses the shorter one. The NTP daemon has its own loop which can be used instead and it allows to use arbitrary values though.

Signature · 1 line
Miroslav Lichvar
#12 lore raw
Quoted 10 lines · Miroslav Lichvar
Quoted 9 lines · Ingo Molnar
Quoted 6 lines · Miroslav Lichvar

Still, I'd really like to see the original behavior restored. Most of the users complaining about slow convergence are probably just hitting the calibration problem, which needs to be fixed by other means than making PLL faster. Also, users of other systems seem to be happy with their slow convergence. At least that's the impression I have from NTP lists.

Quoted 7 lines
Quoted 2 lines

Wouldnt the goal be to calibrate as fast as possible? (Without any bad oscillation)

Not really. It depends on how noisy is the input signal. On an idle LAN the jitter is just few microseconds, but over internet it easily reaches miliseconds. Over a certain point faster PLL will just make things worse.

That is what i called 'bad oscillation' - a 'too fast' PLL that over-compensates and does not converge well enough.

Is there a claim that this change causes that? (John's testing suggested that there's no such effect)

Quoted 11 lines

PLL is mainly about handling the signal noise, frequency adjusting is secondary. When the noise is very low or the update interval is long enough, the frequency variations caused by temperature changes will dominate the signal noise and this is where FLL should kick in.

The PLL/FLL switching is controlled by update interval. Ideally it would be adaptive, but NTP is not that sophisticated. By default, FLL is enabled when the interval is longer than 2048 seconds. This is of course not the optimal value for all systems.

Quoted 4 lines

Unfortunately in kernel it can be configured only to 2048 or 256 and NTP never uses the shorter one. The NTP daemon has its own loop which can be used instead and it allows to use arbitrary values though.

How about going towards the ideal, adaptive design, to which ntpd passes in time samples and which observes noise and converges as quickly as possible (given the noise level) and stays stable once there? I guess we need extensions to the NTP syscall for that.

The NTP code in kernel/time/ntp.c is now reasonably clean for efforts like that.

It would also pave the way to properly support PPS devices in the kernel. Would you be interested in things like this?

Ingo

#13 lore raw
Quoted 8 lines · Ingo Molnar
Quoted 7 lines
Quoted 2 lines

Wouldnt the goal be to calibrate as fast as possible? (Without any bad oscillation)

Not really. It depends on how noisy is the input signal. On an idle LAN the jitter is just few microseconds, but over internet it easily reaches miliseconds. Over a certain point faster PLL will just make things worse.

Quoted 5 lines

That is what i called 'bad oscillation' - a 'too fast' PLL that over-compensates and does not converge well enough.

Is there a claim that this change causes that? (John's testing suggested that there's no such effect)

I think John's tests were done on LAN and in an environment with sudden temperature changes. This is the case where frequency variations strongly dominate the noise and faster PLL performs better.

On the opposite side is an idle machine in a room with stable temperature syncing over wireless or dial-up. I don't have access to such machine, but in simulations (noise with exponential distribution) I see that offset RMS doubles when the time constant is decreased by 2.

Maybe for most of the users the change would be an improvement. I don't have any statistics to back it up or claim otherwise. However, if the constant needs to be adjusted, it's better to do it in NTP.

Quoted 16 lines
Quoted 11 lines

PLL is mainly about handling the signal noise, frequency adjusting is secondary. When the noise is very low or the update interval is long enough, the frequency variations caused by temperature changes will dominate the signal noise and this is where FLL should kick in.

The PLL/FLL switching is controlled by update interval. Ideally it would be adaptive, but NTP is not that sophisticated. By default, FLL is enabled when the interval is longer than 2048 seconds. This is of course not the optimal value for all systems.

Quoted 4 lines

Unfortunately in kernel it can be configured only to 2048 or 256 and NTP never uses the shorter one. The NTP daemon has its own loop which can be used instead and it allows to use arbitrary values though.

Quoted 4 lines

How about going towards the ideal, adaptive design, to which ntpd passes in time samples and which observes noise and converges as quickly as possible (given the noise level) and stays stable once there? I guess we need extensions to the NTP syscall for that.

Not sure how hard that would be. The ntp-hackers list is a better place to discuss such modifications.

Other NTP clients don't have to use the PLL interface. For example, chrony uses only the SINGLESHOT mode and sets the frequency directly. It has an adaptive model using linear regression, it converges really fast and in my tests performs better than NTP.

Quoted 5 lines

The NTP code in kernel/time/ntp.c is now reasonably clean for efforts like that.

It would also pave the way to properly support PPS devices in the kernel. Would you be interested in things like this?

I'm not very familiar with the PPS API, is there something wrong with it?

Signature · 1 line
Miroslav Lichvar
#14 lore raw
Quoted 12 lines · Miroslav Lichvar
Quoted 11 lines · Ingo Molnar
Quoted 7 lines
Quoted 2 lines

Wouldnt the goal be to calibrate as fast as possible? (Without any bad oscillation)

Not really. It depends on how noisy is the input signal. On an idle LAN the jitter is just few microseconds, but over internet it easily reaches miliseconds. Over a certain point faster PLL will just make things worse.

That is what i called 'bad oscillation' - a 'too fast' PLL that over-compensates and does not converge well enough.

Quoted 7 lines
Quoted 2 lines

Is there a claim that this change causes that? (John's testing suggested that there's no such effect)

I think John's tests were done on LAN and in an environment with sudden temperature changes. This is the case where frequency variations strongly dominate the noise and faster PLL performs better.

I'd also expect this to be quite similar to most everyday Linux uses.

Quoted 5 lines

On the opposite side is an idle machine in a room with stable temperature syncing over wireless or dial-up. I don't have access to such machine, but in simulations (noise with exponential distribution) I see that offset RMS doubles when the time constant is decreased by 2.

The thing is, an idle machine in a room with stable temperature is in a good position anyway to have stable time, right? We should rather care about the common-case of temperature variations, reboots, etc.

That is where NTP _helps the most_ - as the physical environment is very entropy laden to begin with.

Quoted 26 lines

Maybe for most of the users the change would be an improvement. I don't have any statistics to back it up or claim otherwise. However, if the constant needs to be adjusted, it's better to do it in NTP.

Quoted 16 lines
Quoted 11 lines

PLL is mainly about handling the signal noise, frequency adjusting is secondary. When the noise is very low or the update interval is long enough, the frequency variations caused by temperature changes will dominate the signal noise and this is where FLL should kick in.

The PLL/FLL switching is controlled by update interval. Ideally it would be adaptive, but NTP is not that sophisticated. By default, FLL is enabled when the interval is longer than 2048 seconds. This is of course not the optimal value for all systems.

Quoted 4 lines

Unfortunately in kernel it can be configured only to 2048 or 256 and NTP never uses the shorter one. The NTP daemon has its own loop which can be used instead and it allows to use arbitrary values though.

Quoted 4 lines

How about going towards the ideal, adaptive design, to which ntpd passes in time samples and which observes noise and converges as quickly as possible (given the noise level) and stays stable once there? I guess we need extensions to the NTP syscall for that.

Quoted 8 lines

Not sure how hard that would be. The ntp-hackers list is a better place to discuss such modifications.

Other NTP clients don't have to use the PLL interface. For example, chrony uses only the SINGLESHOT mode and sets the frequency directly. It has an adaptive model using linear regression, it converges really fast and in my tests performs better than NTP.

That's good. Could this be integrated into the kernel, for even better results?

Quoted 8 lines
Quoted 5 lines

The NTP code in kernel/time/ntp.c is now reasonably clean for efforts like that.

It would also pave the way to properly support PPS devices in the kernel. Would you be interested in things like this?

I'm not very familiar with the PPS API, is there something wrong with it?

The PPS patches i've seen just export IRQ timestamps to user-space.

That is not very robust in my opinion when it comes to do time approximations - to get quick, low-latency action and precise measurements it's best to keep the critical path as short as possible, and within a single source code repository: i.e. within the kernel.

There's little policy really, other than setting some general parameters. NTPd can still provide the raw _network time_ timestamps, as that is probably best fetched by user-space and fed to the kernel.

Ingo

#15 lore raw
Quoted 7 lines · Ingo Molnar
Quoted 4 lines

I think John's tests were done on LAN and in an environment with sudden temperature changes. This is the case where frequency variations strongly dominate the noise and faster PLL performs better.

I'd also expect this to be quite similar to most everyday Linux uses.

I'd say that most users keep the default distribution configs, i.e. syncing over internet to servers from pool.ntp.org. The network jitter is in hundreds of microseconds or even miliseconds and the temperature changes are dominated by the noise.

Quoted 8 lines
Quoted 5 lines

Other NTP clients don't have to use the PLL interface. For example, chrony uses only the SINGLESHOT mode and sets the frequency directly. It has an adaptive model using linear regression, it converges really fast and in my tests performs better than NTP.

That's good. Could this be integrated into the kernel, for even better results?

The code is quite complex with possibly lot of room for improvement. I think it's better to keep it in userspace. There are two things that would help chrony on kernel side though. Supporting nanosecond offset in the SINGLESHOT mode and updating the reported offset with every adjtimex call, not only once per second, so chrony would know exactly how much of the offset was already applied.

Quoted 10 lines
Quoted 2 lines

I'm not very familiar with the PPS API, is there something wrong with it?

The PPS patches i've seen just export IRQ timestamps to user-space.

That is not very robust in my opinion when it comes to do time approximations - to get quick, low-latency action and precise measurements it's best to keep the critical path as short as possible, and within a single source code repository: i.e. within the kernel.

That's what kernel PPS discipline does, it will be probably included later. Its performance is an order or two better than the PLL/FLL discipline.

Signature · 1 line
Miroslav Lichvar
#16 lore raw
Quoted 12 lines · Miroslav Lichvar
Quoted 11 lines
Quoted 2 lines

I'm not very familiar with the PPS API, is there something wrong with it?

The PPS patches i've seen just export IRQ timestamps to user-space.

That is not very robust in my opinion when it comes to do time approximations - to get quick, low-latency action and precise measurements it's best to keep the critical path as short as possible, and within a single source code repository: i.e. within the kernel.

Quoted 3 lines

That's what kernel PPS discipline does, it will be probably included later. Its performance is an order or two better than the PLL/FLL discipline.

Is there some kernel patch i can look at?

Ingo

#17 lore raw
Quoted 15 lines · Ingo Molnar
Quoted 12 lines · Miroslav Lichvar
Quoted 11 lines
Quoted 2 lines

I'm not very familiar with the PPS API, is there something wrong with it?

The PPS patches i've seen just export IRQ timestamps to user-space.

That is not very robust in my opinion when it comes to do time approximations - to get quick, low-latency action and precise measurements it's best to keep the critical path as short as possible, and within a single source code repository: i.e. within the kernel.

Quoted 5 lines
Quoted 3 lines

That's what kernel PPS discipline does, it will be probably included later. Its performance is an order or two better than the PLL/FLL discipline.

Is there some kernel patch i can look at?

It's in the old PPSkit patches for 2.4 kernels, function hardpps().

Signature · 1 line
Miroslav Lichvar
#18 lore raw
In reply to Ingo Molnar
Quoted 8 lines · Ingo Molnar

The PPS patches i've seen just export IRQ timestamps to user-space.

That is not very robust in my opinion when it comes to do time approximations - to get quick, low-latency action and precise measurements it's best to keep the critical path as short as possible, and within a single source code repository: i.e. within the kernel.

Quoted 4 lines

There's little policy really, other than setting some general parameters. NTPd can still provide the raw _network time_ timestamps, as that is probably best fetched by user-space and fed to the kernel.

At some point that stops being NTP. NTP has quite a bit of userland policy for filtering and managing a number of different network clocks (other ntp servers, PPS sources, etc).

Quoted 1 line

From what you're describing (direct offset from a hardware time device

used to steer the clock directly in kernel), you might want to look at the STP code in s390 (stp_sync_clock).

thanks
-john

#19 lore raw
Quoted 2 lines · Ingo Molnar
Quoted 1 line

The PPS patches i've seen just export IRQ timestamps to user-space.

Correct. They improve the sampling information quality and help cut down on jitter between the IRQ sampling and using the timestamp. The jitter is what matters most here and NTP can figure out constant latencies rather well.

Quoted 7 lines

At some point that stops being NTP. NTP has quite a bit of userland policy for filtering and managing a number of different network clocks (other ntp servers, PPS sources, etc).

Quoted 1 line

From what you're describing (direct offset from a hardware time device

used to steer the clock directly in kernel), you might want to look at the STP code in s390 (stp_sync_clock).

And also hardware distributed timing systems like those that distribute a clock with ethernet signals.

Alan

#20 lore raw
Quoted 10 lines · On Wed, 24 Jun 2009 10:29:15 +0100 Alan Cox <[email protected]> wrote:
Quoted 7 lines

At some point that stops being NTP. NTP has quite a bit of userland policy for filtering and managing a number of different network clocks (other ntp servers, PPS sources, etc).

Quoted 1 line

From what you're describing (direct offset from a hardware time device

used to steer the clock directly in kernel), you might want to look at the STP code in s390 (stp_sync_clock).

And also hardware distributed timing systems like those that distribute a clock with ethernet signals.

The STP clock synchronization works below the kernel. Usually we don't notice the clock drift at all, the kernel has the illusion of a perfect clock. Only if the delta is over the clock synchronization tolerance the hardware causes a machine check. Then it becomes the job of the operating system to deal with the clock delta. The current Linux code applies the offset to the hardware clock (which makes the TOD clock non-monotonic) and applies the same offset to the base value sched_clock_base_cc to even out the effect. Then a single shot adjustment is passed to NTP to get the system time in sync with the hardware clock again.

Signature · 4 lines
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.