Improve stability of system clock
Miroslav Lichvar
reviewerv2 cover Improve stability of system clock
v1->v2
- rebased to current code
- improved commit messages
This patch set improves stability and accuracy of the system clock when synchronized to precise time sources like the new PTP KVM clock or NTP/PTP with hardware timestamping. The biggest difference is expected on mostly idle systems running with NOHZ (i.e. having infrequent updates of the clock).
The stability is affected by the error which accumulates in the ntp_error register and its correction, which is performed by adjusting the timekeeping multiplier. After removing the old vsyscalls, which added an error due to nanosecond rounding, there are now three sources of the error:
- alignment of frequency adjustments to ticks
- iterative correction of the multiplier
- limited resolution of the multiplier
Instead of trying to correct the error faster, which is difficult to do without causing instability due to irregular updates of the clock, the patches remove the first two sources of the error. With the only remaining source the correction logic can be significantly simplified and the frequency of the clock is much more stable and accurate.
The freq-step selftest (with PTI disabled to pass the check of the precision of the MONOTONIC_RAW clock) shows a significant improvement:
Before:
Checking response to frequency step:
Step 1st interval 2nd interval
Freq Dev Max Freq Dev Max
40960 +0.131 22 147 +0.136 1 3 [OK]
40960 +0.012 51 347 +0.005 1 3 [OK]
40960 +3.832 25038 171297 -0.169 1 2 [OK]
40960 +2.151 13165 89429 +0.113 1 3 [OK]
40960 -0.095 1 2 -0.095 1 3 [OK]
640 -0.154 1 6 -0.154 1 3 [OK]
640 +0.003 1 6 +0.003 1 3 [OK]
640 -0.131 2 9 -0.131 1 2 [OK]
640 -0.026 19 125 -0.029 1 3 [OK]
640 -0.175 1 3 -0.175 1 3 [OK]
10 +0.001 3 9 +0.000 3 6 [OK]
10 +0.002 7 41 +0.000 2 5 [OK]
10 -0.000 2 5 +0.000 3 5 [OK]
10 +0.001 5 23 -0.000 2 5 [OK]
10 +0.000 2 5 -0.000 6 15 [OK]After:
Checking response to frequency step:
Step 1st interval 2nd interval
Freq Dev Max Freq Dev Max
40960 +0.000 2 5 -0.000 2 5 [OK]
40960 -0.000 2 5 -0.000 3 8 [OK]
40960 -0.000 3 7 +0.000 3 6 [OK]
40960 +0.000 3 6 -0.000 2 6 [OK]
40960 -0.000 2 5 +0.000 3 7 [OK]
640 +0.001 2 9 -0.000 2 5 [OK]
640 -0.000 3 6 +0.000 2 5 [OK]
640 -0.001 2 4 +0.000 1 3 [OK]
640 -0.000 2 5 +0.000 2 4 [OK]
640 +0.000 3 6 -0.000 3 6 [OK]
10 -0.000 2 5 +0.000 2 5 [OK]
10 -0.000 2 5 +0.000 2 5 [OK]
10 +0.000 2 5 +0.000 2 5 [OK]
10 +0.000 2 5 +0.000 2 5 [OK]
10 +0.000 3 12 +0.000 4 11 [OK]Miroslav Lichvar (2):
timekeeping: Don't align frequency adjustments to ticks
timekeeping: Determine multiplier directly from NTP tick length
include/linux/timekeeper_internal.h | 2 + kernel/time/timekeeping.c | 140 ++++++++++++------------------------ 2 files changed, 48 insertions(+), 94 deletions(-)
Signature · 1 line
2.9.5
Miroslav Lichvar
reviewerPATCH v2 1/2 timekeeping: Don't align frequency adjustments to ticks
When the timekeeping multiplier is changed, the NTP error is updated to correct the clock for the delay between the tick and the update of the clock. This error is corrected in later updates and the clock appears as if the frequency was changed exactly on the tick.
Remove this correction to keep the point where the frequency is effectively changed at the time of the update. This removes a major source of the NTP error.
--- kernel/time/timekeeping.c | 3 --- 1 file changed, 3 deletions(-)
kernel/time/timekeeping.c +0 -3
@@ -1860,8 +1860,6 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk,* xtime_nsec_2 = xtime_nsec_1 - offset * Which simplfies to: * xtime_nsec -= offset -* -* XXX - TODO: Doc ntp_error calculation. */ if ((mult_adj >0 ) && (tk->tkr_mono.mult + mult_adj < mult_adj)) {/* NTP adjustment caused clocksource mult overflow */
@@ -1872,7 +1870,6 @@ static __always_inline void timekeeping_apply_adjustment(struct timekeeper *tk, tk->tkr_mono.mult += mult_adj; tk->xtime_interval += interval; tk->tkr_mono.xtime_nsec -= offset; - tk->ntp_error -= (interval - offset) << tk->ntp_error_shift; }/*
Signature · 1 line
2.9.5
Miroslav Lichvar
reviewerPATCH v2 2/2 timekeeping: Determine multiplier directly from NTP tick length
In reply to Miroslav LichvarWhen the length of the NTP tick changes significantly, e.g. when an NTP/PTP application is correcting the initial offset of the clock, a large value may accumulate in the NTP error before the multiplier converges to the correct value. It may then take a very long time (hours or even days) before the error is corrected. This causes the clock to have an unstable frequency offset, which has a negative impact on the stability of synchronization with precise time sources (e.g. NTP/PTP using hardware timestamping or the PTP KVM clock).
Use division to determine the correct multiplier directly from the NTP tick length and replace the iterative approach. This removes the last major source of the NTP error. The only remaining source is now limited resolution of the multiplier, which is corrected by adding 1 to the multiplier when the system clock is behind the NTP time.
--- include/linux/timekeeper_internal.h | 2 + kernel/time/timekeeping.c | 138 ++++++++++++------------------------ 2 files changed, 49 insertions(+), 91 deletions(-)
include/linux/timekeeper_internal.h +2 -0
@@ -117,6 +117,8 @@ struct timekeeper {s64 ntp_error;u32 ntp_error_shift;u32 ntp_err_mult; +/* Flag used to avoid updating NTP twice with same second */ +u32 skip_second_overflow;#ifdef CONFIG_DEBUG_TIMEKEEPING long last_warning;/*
kernel/time/timekeeping.c +47 -91
189 lines not on this page · raw message
Signature · 1 line
2.9.5
Miroslav Lichvar
reviewerv2 cover Improve stability of system clock
In reply to Miroslav Lichvar(resending with fixed CC)
v1->v2
- rebased to current code
- improved commit messages
This patch set improves stability and accuracy of the system clock when synchronized to precise time sources like the new PTP KVM clock or NTP/PTP with hardware timestamping. The biggest difference is expected on mostly idle systems running with NOHZ (i.e. having infrequent updates of the clock).
The stability is affected by the error which accumulates in the ntp_error register and its correction, which is performed by adjusting the timekeeping multiplier. After removing the old vsyscalls, which added an error due to nanosecond rounding, there are now three sources of the error:
- alignment of frequency adjustments to ticks
- iterative correction of the multiplier
- limited resolution of the multiplier
Instead of trying to correct the error faster, which is difficult to do without causing instability due to irregular updates of the clock, the patches remove the first two sources of the error. With the only remaining source the correction logic can be significantly simplified and the frequency of the clock is much more stable and accurate.
The freq-step selftest (with PTI disabled to pass the check of the precision of the MONOTONIC_RAW clock) shows a significant improvement:
Before:
Checking response to frequency step:
Step 1st interval 2nd interval
Freq Dev Max Freq Dev Max
40960 +0.131 22 147 +0.136 1 3 [OK]
40960 +0.012 51 347 +0.005 1 3 [OK]
40960 +3.832 25038 171297 -0.169 1 2 [OK]
40960 +2.151 13165 89429 +0.113 1 3 [OK]
40960 -0.095 1 2 -0.095 1 3 [OK]
640 -0.154 1 6 -0.154 1 3 [OK]
640 +0.003 1 6 +0.003 1 3 [OK]
640 -0.131 2 9 -0.131 1 2 [OK]
640 -0.026 19 125 -0.029 1 3 [OK]
640 -0.175 1 3 -0.175 1 3 [OK]
10 +0.001 3 9 +0.000 3 6 [OK]
10 +0.002 7 41 +0.000 2 5 [OK]
10 -0.000 2 5 +0.000 3 5 [OK]
10 +0.001 5 23 -0.000 2 5 [OK]
10 +0.000 2 5 -0.000 6 15 [OK]After:
Checking response to frequency step:
Step 1st interval 2nd interval
Freq Dev Max Freq Dev Max
40960 +0.000 2 5 -0.000 2 5 [OK]
40960 -0.000 2 5 -0.000 3 8 [OK]
40960 -0.000 3 7 +0.000 3 6 [OK]
40960 +0.000 3 6 -0.000 2 6 [OK]
40960 -0.000 2 5 +0.000 3 7 [OK]
640 +0.001 2 9 -0.000 2 5 [OK]
640 -0.000 3 6 +0.000 2 5 [OK]
640 -0.001 2 4 +0.000 1 3 [OK]
640 -0.000 2 5 +0.000 2 4 [OK]
640 +0.000 3 6 -0.000 3 6 [OK]
10 -0.000 2 5 +0.000 2 5 [OK]
10 -0.000 2 5 +0.000 2 5 [OK]
10 +0.000 2 5 +0.000 2 5 [OK]
10 +0.000 2 5 +0.000 2 5 [OK]
10 +0.000 3 12 +0.000 4 11 [OK]Miroslav Lichvar (2):
timekeeping: Don't align frequency adjustments to ticks
timekeeping: Determine multiplier directly from NTP tick length
include/linux/timekeeper_internal.h | 2 + kernel/time/timekeeping.c | 140 ++++++++++++------------------------ 2 files changed, 48 insertions(+), 94 deletions(-)
Signature · 1 line
2.9.5