Do not mark clocks unstable due to delays for v5.14
v15 cover Do not mark clocks unstable due to delays for v5.14
Hello!
If there is a sufficient delay between reading the watchdog clock and the clock under test, the clock under test will be marked unstable through no fault of its own. This series checks for this, doing limited retries to get a good set of clock reads. If the clock is marked unstable and is marked as being per-CPU, cross-CPU synchronization is checked. This series also provides a clocksource-watchdog-test kernel module that tests this new ability of distinguishing delay-induced clock skew from true clock skew.
Note that "sufficient delay" can be provided by SMIs, NMIs, and of course vCPU preemption.
The patches in this series are as follows:
1. Retry clock read if long delays detected.
2. Check per-CPU clock synchronization when marked unstable.
3. Limit number of CPUs checked for clock synchronization.
4. Reduce clocksource-skew threshold for TSC.
5. Provide kernel module to test clocksource watchdog.
6. Print deviation in nanoseconds for unstable case, courtesy of
Feng Tang.
Changes since v14:
o Start with clocksource.max_cswd_read_retries instead of renaming
this module parameter from clocksource.max_read_retries partway through the series.
Changes since v13:
o Forward-port to v5.13-rc1.
o Add patch 6 from Feng Tang.
Changes since v12, based on feedback from kernel test robot and Stephen Rothwell:
o Export clocksource_verify_percpu().
Changes since v11, based on feedback from Thomas Gleixner:
o Remove the fault-injection code from clocksource.c.
o Create a kernel/time/clocksource-wdtest.c kernel module that
creates its own clocksource structures and injects delays as part of their ->read() functions.
o Make this kernel module splat upon error, for example, when
a clocksource is not marked unstable but should have been.
o Apply a couple more "Link:" fields to all patches.
Changes since v10 based on feedback from Thomas Gleixner, Feng Tang, and Andi Kleen:
o Add an uncertainty_margin field to the clocksource structure to
allow skew cutoffs to be tailored to the pair of clocksources that the watchdog is comparing.
o Manually initialize this uncertainty_margin field for
clocksource_tsc_early and clocksource_jiffies, thus avoiding the need for special-case code to allow for the unusually large skews inherent to these clocksources.
Changes since v9:
o Forgive tsc_early drift, based on feedback from Feng Tang; Xing,
Zhengjun; and Thomas Gleixner.
o Improve CPU selection for clock-synchronization checking.
Changes since v8, based on Thomas Gleixner feedback:
o Reduced clock-skew threshold to 200us and delay limit to 50us.
o Split out a cs_watchdog_read() function.
o Removed the pointless CLOCK_SOURCE_VERIFY_PERCPU from kvm_clock.
o Initialized cs_nsec_max and cs_nsec_min to avoid firsttime checks.
Changes since v7, based on Thomas Gleixner feedback:
o Fix embarrassing git-format-patch operator error.
o Merge pairwise clock-desynchronization checking into the checking
of per-CPU clock synchronization when marked unstable.
o Do selective per-CPU checking rather than blindly checking all
CPUs. Provide a clocksource.verify_n_cpus kernel boot parameter to control this behavior, with the value -1 choosing the old check-all-CPUs behavior. The default is to randomly check 8 CPUs.
o Fix the clock-desynchronization checking to avoid a potential
use-after-free error for dynamically allocated clocksource structures.
o Remove redundance "wdagain_nsec < 0" from clocksource_watchdog()
clocksource skew checking.
o Update commit logs and do code-style updates.
Changes since v5:
o Rebased to v5.12-rc5.
Changes since v4:
o Rebased to v5.12-rc1.
Changes since v3:
o Rebased to v5.11.
o Apply Randy Dunlap feedback.
Changes since v2:
o Rebased to v5.11-rc6.
o Updated Cc: list.
Changes since v1:
o Applied feedback from Rik van Riel.
o Rebased to v5.11-rc3.
o Stripped "RFC" from the subject lines.
Thanx, Paul
------------------------------------------------------------------------
Documentation/admin-guide/kernel-parameters.txt | 16 + arch/x86/kernel/tsc.c | 1 b/Documentation/admin-guide/kernel-parameters.txt | 6 b/arch/x86/kernel/tsc.c | 3 b/include/linux/clocksource.h | 2 b/kernel/time/Makefile | 1 b/kernel/time/clocksource-wdtest.c | 202 ++++++++++++++++++++++ b/kernel/time/clocksource.c | 52 +++++ b/kernel/time/jiffies.c | 15 - b/lib/Kconfig.debug | 12 + include/linux/clocksource.h | 6 kernel/time/clocksource.c | 196 +++++++++++++++++++-- 12 files changed, 479 insertions(+), 33 deletions(-)
PATCH v15 1/6 clocksource: Retry clock read if long delays detected
When the clocksource watchdog marks a clock as unstable, this might be due to that clock being unstable or it might be due to delays that happen to occur between the reads of the two clocks. Yes, interrupts are disabled across those two reads, but there are no shortage of things that can delay interrupts-disabled regions of code ranging from SMI handlers to vCPU preemption. It would be good to have some indication as to why the clock was marked unstable.
Therefore, re-read the watchdog clock on either side of the read from the clock under test. If the watchdog clock shows an excessive time delta between its pair of reads, the reads are retried. The maximum number of retries is specified by a new kernel boot parameter clocksource.max_cswd_read_retries, which defaults to three, that is, up to four reads, one initial and up to three retries. If more than one retry was required, a message is printed on the console (the occasional single retry is expected behavior, especially in guest OSes). If the maximum number of retries is exceeded, the clock under test will be marked unstable. However, the probability of this happening due to various sorts of delays is quite small. In addition, the reason (clock-read delays) for the unstable marking will be apparent.
--- .../admin-guide/kernel-parameters.txt | 6 +++ kernel/time/clocksource.c | 52 ++++++++++++++++--- 2 files changed, 52 insertions(+), 6 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +6 -0
@@ -581,6 +581,12 @@ loops can be debugged more effectively on production systems. + clocksource.max_cswd_read_retries= [KNL] + Number of clocksource_watchdog() retries due to + external delays before the clock will be marked + unstable. Defaults to three retries, that is, + four attempts to read the clock under test. + clearcpuid=BITNUM[,BITNUM...] [X86] Disable CPUID feature X for the kernel. See arch/x86/include/asm/cpufeatures.h for the valid bit
kernel/time/clocksource.c +46 -6
74 lines not on this page · raw message
Signature · 1 line
2.31.1.189.g2e36527f23
[tip: timers/core] clocksource: Retry clock read if long delays detected
The following commit has been merged into the timers/core branch of tip:
The following commit has been merged into the timers/core branch of tip:
Commit-ID: db3a34e17433de2390eb80d436970edcebd0ca3e Gitweb: https://git.kernel.org/tip/db3a34e17433de2390eb80d436970edcebd0ca3e Author: Paul E. McKenney <[email protected]> AuthorDate: Thu, 27 May 2021 12:01:19 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 22 Jun 2021 16:53:16 +02:00
clocksource: Retry clock read if long delays detected
When the clocksource watchdog marks a clock as unstable, this might be due to that clock being unstable or it might be due to delays that happen to occur between the reads of the two clocks. Yes, interrupts are disabled across those two reads, but there are no shortage of things that can delay interrupts-disabled regions of code ranging from SMI handlers to vCPU preemption. It would be good to have some indication as to why the clock was marked unstable.
Therefore, re-read the watchdog clock on either side of the read from the clock under test. If the watchdog clock shows an excessive time delta between its pair of reads, the reads are retried.
The maximum number of retries is specified by a new kernel boot parameter clocksource.max_cswd_read_retries, which defaults to three, that is, up to four reads, one initial and up to three retries. If more than one retry was required, a message is printed on the console (the occasional single retry is expected behavior, especially in guest OSes). If the maximum number of retries is exceeded, the clock under test will be marked unstable. However, the probability of this happening due to various sorts of delays is quite small. In addition, the reason (clock-read delays) for the unstable marking will be apparent.
--- Documentation/admin-guide/kernel-parameters.txt | 6 ++- kernel/time/clocksource.c | 53 ++++++++++++++-- 2 files changed, 53 insertions(+), 6 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +6 -0
@@ -581,6 +581,12 @@ loops can be debugged more effectively on production systems. + clocksource.max_cswd_read_retries= [KNL] + Number of clocksource_watchdog() retries due to + external delays before the clock will be marked + unstable. Defaults to three retries, that is, + four attempts to read the clock under test. + clearcpuid=BITNUM[,BITNUM...] [X86] Disable CPUID feature X for the kernel. See arch/x86/include/asm/cpufeatures.h for the valid bit
kernel/time/clocksource.c +47 -6
75 lines not on this page · raw message
PATCH v15 2/6 clocksource: Check per-CPU clock synchronization when marked unstable
In reply to Paul E. McKenneySome sorts of per-CPU clock sources have a history of going out of synchronization with each other. However, this problem has purportedy been solved in the past ten years. Except that it is all too possible that the problem has instead simply been made less likely, which might mean that some of the occasional "Marking clocksource 'tsc' as unstable" messages might be due to desynchronization. How would anyone know?
Therefore apply CPU-to-CPU synchronization checking to newly unstable clocksource that are marked with the new CLOCK_SOURCE_VERIFY_PERCPU flag. Lists of desynchronized CPUs are printed, with the caveat that if it is the reporting CPU that is itself desynchronized, it will appear that all the other clocks are wrong. Just like in real life.
--- arch/x86/kernel/tsc.c | 3 +- include/linux/clocksource.h | 2 +- kernel/time/clocksource.c | 60 +++++++++++++++++++++++++++++++++++++ 3 files changed, 63 insertions(+), 2 deletions(-)
arch/x86/kernel/tsc.c +2 -1
@@ -1152,7 +1152,8 @@ static struct clocksource clocksource_tsc = { .mask = CLOCKSOURCE_MASK(64), .flags =CLOCK_SOURCE_IS_CONTINUOUS |CLOCK_SOURCE_VALID_FOR_HRES | - CLOCK_SOURCE_MUST_VERIFY, +CLOCK_SOURCE_MUST_VERIFY | + CLOCK_SOURCE_VERIFY_PERCPU, .vdso_clock_mode = VDSO_CLOCKMODE_TSC, .enable = tsc_cs_enable, .resume = tsc_resume,
include/linux/clocksource.h +1 -1
@@ -137,7 +137,7 @@ struct clocksource {#define CLOCK_SOURCE_UNSTABLE 0x40#define CLOCK_SOURCE_SUSPEND_NONSTOP 0x80#define CLOCK_SOURCE_RESELECT 0x100 - +#define CLOCK_SOURCE_VERIFY_PERCPU 0x200/* simplify initialization of mask field */ #define CLOCKSOURCE_MASK (bits) GENMASK_ULL((bits) - 1, 0)
kernel/time/clocksource.c +60 -0
72 lines not on this page · raw message
Signature · 1 line
2.31.1.189.g2e36527f23
[tip: timers/core] clocksource: Check per-CPU clock synchronization when marked unstable
The following commit has been merged into the timers/core branch of tip:
The following commit has been merged into the timers/core branch of tip:
Commit-ID: 7560c02bdffb7c52d1457fa551b9e745d4b9e754 Gitweb: https://git.kernel.org/tip/7560c02bdffb7c52d1457fa551b9e745d4b9e754 Author: Paul E. McKenney <[email protected]> AuthorDate: Thu, 27 May 2021 12:01:20 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 22 Jun 2021 16:53:16 +02:00
clocksource: Check per-CPU clock synchronization when marked unstable
Some sorts of per-CPU clock sources have a history of going out of synchronization with each other. However, this problem has purportedy been solved in the past ten years. Except that it is all too possible that the problem has instead simply been made less likely, which might mean that some of the occasional "Marking clocksource 'tsc' as unstable" messages might be due to desynchronization. How would anyone know?
Therefore apply CPU-to-CPU synchronization checking to newly unstable clocksource that are marked with the new CLOCK_SOURCE_VERIFY_PERCPU flag. Lists of desynchronized CPUs are printed, with the caveat that if it is the reporting CPU that is itself desynchronized, it will appear that all the other clocks are wrong. Just like in real life.
--- arch/x86/kernel/tsc.c | 3 +- include/linux/clocksource.h | 2 +- kernel/time/clocksource.c | 60 ++++++++++++++++++++++++++++++++++++- 3 files changed, 63 insertions(+), 2 deletions(-)
arch/x86/kernel/tsc.c +2 -1
@@ -1152,7 +1152,8 @@ static struct clocksource clocksource_tsc = { .mask = CLOCKSOURCE_MASK(64), .flags =CLOCK_SOURCE_IS_CONTINUOUS |CLOCK_SOURCE_VALID_FOR_HRES | - CLOCK_SOURCE_MUST_VERIFY, +CLOCK_SOURCE_MUST_VERIFY | + CLOCK_SOURCE_VERIFY_PERCPU, .vdso_clock_mode = VDSO_CLOCKMODE_TSC, .enable = tsc_cs_enable, .resume = tsc_resume,
include/linux/clocksource.h +1 -1
@@ -137,7 +137,7 @@ struct clocksource {#define CLOCK_SOURCE_UNSTABLE 0x40#define CLOCK_SOURCE_SUSPEND_NONSTOP 0x80#define CLOCK_SOURCE_RESELECT 0x100 - +#define CLOCK_SOURCE_VERIFY_PERCPU 0x200/* simplify initialization of mask field */ #define CLOCKSOURCE_MASK (bits) GENMASK_ULL((bits) - 1, 0)
kernel/time/clocksource.c +60 -0
72 lines not on this page · raw message
PATCH v15 3/6 clocksource: Limit number of CPUs checked for clock synchronization
In reply to Paul E. McKenneyCurrently, if skew is detected on a clock marked CLOCK_SOURCE_VERIFY_PERCPU, that clock is checked on all CPUs. This is thorough, but might not be what you want on a system with a few tens of CPUs, let alone a few hundred of them.
Therefore, by default check only up to eight randomly chosen CPUs. Also provide a new clocksource.verify_n_cpus kernel boot parameter. A value of -1 says to check all of the CPUs, and a non-negative value says to randomly select that number of CPUs, without concern about selecting the same CPU multiple times. However, make use of a cpumask so that a given CPU will be checked at most once.
--- .../admin-guide/kernel-parameters.txt | 10 +++ kernel/time/clocksource.c | 74 ++++++++++++++++++- 2 files changed, 82 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +10 -0
@@ -587,6 +587,16 @@ unstable. Defaults to three retries, that is, four attempts to read the clock under test. + clocksource.verify_n_cpus= [KNL] + Limit the number of CPUs checked for clocksources + marked with CLOCK_SOURCE_VERIFY_PERCPU that + are marked unstable due to excessive skew. + A negative value says to check all CPUs, while + zero says not to check any. Values larger than + nr_cpu_ids are silently truncated to nr_cpu_ids. + The actual CPUs are chosen randomly, with + no replacement if the same CPU is chosen twice. + clearcpuid=BITNUM[,BITNUM...] [X86] Disable CPUID feature X for the kernel. See arch/x86/include/asm/cpufeatures.h for the valid bit
kernel/time/clocksource.c +72 -2
114 lines not on this page · raw message
Signature · 1 line
2.31.1.189.g2e36527f23
[tip: timers/core] clocksource: Limit number of CPUs checked for clock synchronization
The following commit has been merged into the timers/core branch of tip:
The following commit has been merged into the timers/core branch of tip:
Commit-ID: fa218f1cce6ba40069c8daab8821de7e6be1cdd0 Gitweb: https://git.kernel.org/tip/fa218f1cce6ba40069c8daab8821de7e6be1cdd0 Author: Paul E. McKenney <[email protected]> AuthorDate: Thu, 27 May 2021 12:01:21 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 22 Jun 2021 16:53:16 +02:00
clocksource: Limit number of CPUs checked for clock synchronization
Currently, if skew is detected on a clock marked CLOCK_SOURCE_VERIFY_PERCPU, that clock is checked on all CPUs. This is thorough, but might not be what you want on a system with a few tens of CPUs, let alone a few hundred of them.
Therefore, by default check only up to eight randomly chosen CPUs. Also provide a new clocksource.verify_n_cpus kernel boot parameter. A value of -1 says to check all of the CPUs, and a non-negative value says to randomly select that number of CPUs, without concern about selecting the same CPU multiple times. However, make use of a cpumask so that a given CPU will be checked at most once.
--- Documentation/admin-guide/kernel-parameters.txt | 10 ++- kernel/time/clocksource.c | 74 +++++++++++++++- 2 files changed, 82 insertions(+), 2 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +10 -0
@@ -587,6 +587,16 @@ unstable. Defaults to three retries, that is, four attempts to read the clock under test. + clocksource.verify_n_cpus= [KNL] + Limit the number of CPUs checked for clocksources + marked with CLOCK_SOURCE_VERIFY_PERCPU that + are marked unstable due to excessive skew. + A negative value says to check all CPUs, while + zero says not to check any. Values larger than + nr_cpu_ids are silently truncated to nr_cpu_ids. + The actual CPUs are chosen randomly, with + no replacement if the same CPU is chosen twice. + clearcpuid=BITNUM[,BITNUM...] [X86] Disable CPUID feature X for the kernel. See arch/x86/include/asm/cpufeatures.h for the valid bit
kernel/time/clocksource.c +72 -2
114 lines not on this page · raw message
PATCH v15 4/6 clocksource: Reduce clocksource-skew threshold for TSC
In reply to Paul E. McKenneyCurrently, WATCHDOG_THRESHOLD is set to detect a 62.5-millisecond skew in a 500-millisecond WATCHDOG_INTERVAL. This requires that clocks be skewed by more than 12.5% in order to be marked unstable. Except that a clock that is skewed by that much is probably destroying unsuspecting software right and left. And given that there are now checks for false-positive skews due to delays between reading the two clocks, it should be possible to greatly decrease WATCHDOG_THRESHOLD, at least for fine-grained clocks such as TSC.
Therefore, add a new uncertainty_margin field to the clocksource structure that contains the maximum uncertainty in nanoseconds for the corresponding clock. This field may be initialized manually, as it is for clocksource_tsc_early and clocksource_jiffies, which is copied to refined_jiffies. If the field is not initialized manually, it will be computed at clock-registry time as the period of the clock in question based on the scale and freq parameters to __clocksource_update_freq_scale() function. If either of those two parameters are zero, the tens-of-milliseconds WATCHDOG_THRESHOLD is used as a cowardly alternative to dividing by zero. No matter how the uncertainty_margin field is calculated, it is bounded below by twice WATCHDOG_MAX_SKEW, that is, by 100 microseconds.
Note that manually initialized uncertainty_margin fields are not adjusted, but there is a WARN_ON_ONCE() that triggers if any such field is less than twice WATCHDOG_MAX_SKEW. This WARN_ON_ONCE() is intended to discourage production use of the one-nanosecond uncertainty_margin values that are used to test the clock-skew code itself.
The actual clock-skew check uses the sum of the uncertainty_margin fields of the two clocksource structures being compared. Integer overflow is avoided because the largest computed value of the uncertainty_margin fields is one billion (10^9), and double that value fits into an unsigned int. However, if someone manually specifies (say) UINT_MAX, they will get what they deserve.
Note that the refined_jiffies uncertainty_margin field is initialized to TICK_NSEC, which means that skew checks involving this clocksource will be sufficently forgiving. In a similar vein, the clocksource_tsc_early uncertainty_margin field is initialized to 32*NSEC_PER_MSEC, which replicates the current behavior and allows custom setting if needed in order to address the rare skews detected for this clocksource in current mainline.
--- arch/x86/kernel/tsc.c | 1 + include/linux/clocksource.h | 3 +++ kernel/time/clocksource.c | 48 +++++++++++++++++++++++++++++-------- kernel/time/jiffies.c | 15 ++++++------ 4 files changed, 50 insertions(+), 17 deletions(-)
arch/x86/kernel/tsc.c +1 -0
@@ -1128,6 +1128,7 @@ static int tsc_cs_enable(struct clocksource *cs)static struct clocksource clocksource_tsc_early = { .name ="tsc-early" , .rating =299 , + .uncertainty_margin =32 *NSEC_PER_MSEC , .read = read_tsc, .mask =CLOCKSOURCE_MASK (64 ), .flags = CLOCK_SOURCE_IS_CONTINUOUS |
include/linux/clocksource.h +3 -0
@@ -43,6 +43,8 @@ struct module;* @shift: Cycle to nanosecond divisor (power of two) * @max_idle_ns: Maximum idle time permitted by the clocksource (nsecs) * @maxadj: Maximum adjustment value to mult (~11%) +* @uncertainty_margin: Maximum uncertainty in nanoseconds per half second. +* Zero says to use default WATCHDOG_THRESHOLD. * @archdata: Optional arch-specific data * @max_cycles: Maximum safe cycle value which won't overflow on * multiplication
@@ -98,6 +100,7 @@ struct clocksource {u32 shift;u64 max_idle_ns;u32 maxadj; +u32 uncertainty_margin;#ifdef CONFIG_ARCH_CLOCKSOURCE_DATA struct arch_clocksource_data archdata;#endif
kernel/time/clocksource.c +38 -10
80 lines not on this page · raw message
kernel/time/jiffies.c +8 -7
@@ -49,13 +49,14 @@ static u64 jiffies_read(struct clocksource *cs)* for "tick-less" systems. */ static struct clocksource clocksource_jiffies = { - .name ="jiffies" , - .rating =1 ,/* lowest valid rating*/ - .read = jiffies_read, - .mask =CLOCKSOURCE_MASK (32 ), - .mult =TICK_NSEC <<JIFFIES_SHIFT ,/* details above */ - .shift =JIFFIES_SHIFT , - .max_cycles =10 , + .name ="jiffies" , + .rating =1 ,/* lowest valid rating*/ + .uncertainty_margin =32 *NSEC_PER_MSEC , + .read = jiffies_read, + .mask =CLOCKSOURCE_MASK (32 ), + .mult =TICK_NSEC <<JIFFIES_SHIFT ,/* details above */ + .shift =JIFFIES_SHIFT , + .max_cycles =10 , };__cacheline_aligned_in_smp DEFINE_RAW_SPINLOCK (jiffies_lock );
Signature · 1 line
2.31.1.189.g2e36527f23
[tip: timers/core] clocksource: Reduce clocksource-skew threshold
The following commit has been merged into the timers/core branch of tip:
The following commit has been merged into the timers/core branch of tip:
Commit-ID: 2e27e793e280ff12cb5c202a1214c08b0d3a0f26 Gitweb: https://git.kernel.org/tip/2e27e793e280ff12cb5c202a1214c08b0d3a0f26 Author: Paul E. McKenney <[email protected]> AuthorDate: Thu, 27 May 2021 12:01:22 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 22 Jun 2021 16:53:16 +02:00
clocksource: Reduce clocksource-skew threshold
Currently, WATCHDOG_THRESHOLD is set to detect a 62.5-millisecond skew in a 500-millisecond WATCHDOG_INTERVAL. This requires that clocks be skewed by more than 12.5% in order to be marked unstable. Except that a clock that is skewed by that much is probably destroying unsuspecting software right and left. And given that there are now checks for false-positive skews due to delays between reading the two clocks, it should be possible to greatly decrease WATCHDOG_THRESHOLD, at least for fine-grained clocks such as TSC.
Therefore, add a new uncertainty_margin field to the clocksource structure that contains the maximum uncertainty in nanoseconds for the corresponding clock. This field may be initialized manually, as it is for clocksource_tsc_early and clocksource_jiffies, which is copied to refined_jiffies. If the field is not initialized manually, it will be computed at clock-registry time as the period of the clock in question based on the scale and freq parameters to __clocksource_update_freq_scale() function. If either of those two parameters are zero, the tens-of-milliseconds WATCHDOG_THRESHOLD is used as a cowardly alternative to dividing by zero. No matter how the uncertainty_margin field is calculated, it is bounded below by twice WATCHDOG_MAX_SKEW, that is, by 100 microseconds.
Note that manually initialized uncertainty_margin fields are not adjusted, but there is a WARN_ON_ONCE() that triggers if any such field is less than twice WATCHDOG_MAX_SKEW. This WARN_ON_ONCE() is intended to discourage production use of the one-nanosecond uncertainty_margin values that are used to test the clock-skew code itself.
The actual clock-skew check uses the sum of the uncertainty_margin fields of the two clocksource structures being compared. Integer overflow is avoided because the largest computed value of the uncertainty_margin fields is one billion (10^9), and double that value fits into an unsigned int. However, if someone manually specifies (say) UINT_MAX, they will get what they deserve.
Note that the refined_jiffies uncertainty_margin field is initialized to TICK_NSEC, which means that skew checks involving this clocksource will be sufficently forgiving. In a similar vein, the clocksource_tsc_early uncertainty_margin field is initialized to 32*NSEC_PER_MSEC, which replicates the current behavior and allows custom setting if needed in order to address the rare skews detected for this clocksource in current mainline.
--- arch/x86/kernel/tsc.c | 1 +- include/linux/clocksource.h | 3 ++- kernel/time/clocksource.c | 48 ++++++++++++++++++++++++++++-------- kernel/time/jiffies.c | 15 +++++------ 4 files changed, 50 insertions(+), 17 deletions(-)
arch/x86/kernel/tsc.c +1 -0
@@ -1128,6 +1128,7 @@ static int tsc_cs_enable(struct clocksource *cs)static struct clocksource clocksource_tsc_early = { .name ="tsc-early" , .rating =299 , + .uncertainty_margin =32 *NSEC_PER_MSEC , .read = read_tsc, .mask =CLOCKSOURCE_MASK (64 ), .flags = CLOCK_SOURCE_IS_CONTINUOUS |
include/linux/clocksource.h +3 -0
@@ -43,6 +43,8 @@ struct module;* @shift: Cycle to nanosecond divisor (power of two) * @max_idle_ns: Maximum idle time permitted by the clocksource (nsecs) * @maxadj: Maximum adjustment value to mult (~11%) +* @uncertainty_margin: Maximum uncertainty in nanoseconds per half second. +* Zero says to use default WATCHDOG_THRESHOLD. * @archdata: Optional arch-specific data * @max_cycles: Maximum safe cycle value which won't overflow on * multiplication
@@ -98,6 +100,7 @@ struct clocksource {u32 shift;u64 max_idle_ns;u32 maxadj; +u32 uncertainty_margin;#ifdef CONFIG_ARCH_CLOCKSOURCE_DATA struct arch_clocksource_data archdata;#endif
kernel/time/clocksource.c +38 -10
80 lines not on this page · raw message
kernel/time/jiffies.c +8 -7
@@ -49,13 +49,14 @@ static u64 jiffies_read(struct clocksource *cs)* for "tick-less" systems. */ static struct clocksource clocksource_jiffies = { - .name ="jiffies" , - .rating =1 ,/* lowest valid rating*/ - .read = jiffies_read, - .mask =CLOCKSOURCE_MASK (32 ), - .mult =TICK_NSEC <<JIFFIES_SHIFT ,/* details above */ - .shift =JIFFIES_SHIFT , - .max_cycles =10 , + .name ="jiffies" , + .rating =1 ,/* lowest valid rating*/ + .uncertainty_margin =32 *NSEC_PER_MSEC , + .read = jiffies_read, + .mask =CLOCKSOURCE_MASK (32 ), + .mult =TICK_NSEC <<JIFFIES_SHIFT ,/* details above */ + .shift =JIFFIES_SHIFT , + .max_cycles =10 , };__cacheline_aligned_in_smp DEFINE_RAW_SPINLOCK (jiffies_lock );
PATCH v15 5/6 clocksource: Provide kernel module to test clocksource watchdog
In reply to Paul E. McKenneyWhen the clocksource watchdog marks a clock as unstable, this might be due to that clock being unstable or it might be due to delays that happen to occur between the reads of the two clocks. It would be good to have a way of testing the clocksource watchdog's ability to distinguish between these two causes of clock skew and instability.
Therefore, provide a new clocksource-wdtest module selected by a new TEST_CLOCKSOURCE_WATCHDOG Kconfig option. This module has a single module parameter named "holdoff" that provides the number of seconds of delay before testing should start, which defaults to zero when built as a module and to 10 seconds when built directly into the kernel. Very large systems that boot slowly may need to increase the value of this module parameter.
This module uses hand-crafted clocksource structures to do its testing, thus avoiding messing up timing for the rest of the kernel and for user applications. This module first verifies that the ->uncertainty_margin field of the clocksource structures are set sanely. It then tests the delay-detection capability of the clocksource watchdog, increasing the number of consecutive delays injected, first provoking console messages complaining about the delays and finally forcing a clock-skew event. Unexpected test results cause at least one WARN_ON_ONCE() console splat. If there are no splats, the test has passed. Finally, it fuzzes the value returned from a clocksource to test the clocksource watchdog's ability to detect time skew.
This module checks the state of its clocksource after each test, and uses WARN_ON_ONCE() to emit a console splat if there are any failures. This should enable all types of test frameworks to detect any such failures.
This facility is intended for diagnostic use only, and should be avoided on production systems.
--- .../admin-guide/kernel-parameters.txt | 6 + include/linux/clocksource.h | 3 + kernel/time/Makefile | 1 + kernel/time/clocksource-wdtest.c | 202 ++++++++++++++++++ kernel/time/clocksource.c | 6 +- lib/Kconfig.debug | 12 ++ 6 files changed, 228 insertions(+), 2 deletions(-) create mode 100644 kernel/time/clocksource-wdtest.c
Documentation/admin-guide/kernel-parameters.txt +6 -0
@@ -597,6 +597,12 @@ The actual CPUs are chosen randomly, with no replacement if the same CPU is chosen twice. + clocksource-wdtest.holdoff= [KNL] + Set the time in seconds that the clocksource + watchdog test waits before commencing its tests. + Defaults to zero when built as a module and to + 10 seconds when built into the kernel. + clearcpuid=BITNUM[,BITNUM...] [X86] Disable CPUID feature X for the kernel. See arch/x86/include/asm/cpufeatures.h for the valid bit
include/linux/clocksource.h +3 -0
@@ -291,4 +291,7 @@ static inline void timer_probe(void) {}#define TIMER_ACPI_DECLARE (name, table_id, fn) \ ACPI_DECLARE_PROBE_ENTRY(timer, name, table_id, 0, NULL, 0, fn) +extern ulong max_cswd_read_retries; +void clocksource_verify_percpu (struct clocksource *cs); +#endif /* _LINUX_CLOCKSOURCE_H */
kernel/time/Makefile +1 -0
@@ -21,3 +21,4 @@ obj-$(CONFIG_HAVE_GENERIC_VDSO) += vsyscall.o obj-$(CONFIG_DEBUG_FS) += timekeeping_debug.o obj-$(CONFIG_TEST_UDELAY) += test_udelay.o obj-$(CONFIG_TIME_NS) += namespace.o +obj-$(CONFIG_TEST_CLOCKSOURCE_WATCHDOG) += clocksource-wdtest.o
kernel/time/clocksource-wdtest.c new file +202 -0
202 lines not on this page · raw message
kernel/time/clocksource.c +4 -2
@@ -199,8 +199,9 @@ void clocksource_mark_unstable(struct clocksource *cs)spin_unlock_irqrestore (&watchdog_lock, flags); } -static ulong max_cswd_read_retries =3 ; +ulong max_cswd_read_retries =3 ;module_param (max_cswd_read_retries, ulong,0644 ); +EXPORT_SYMBOL_GPL (max_cswd_read_retries);static int verify_n_cpus =8 ;module_param (verify_n_cpus, int,0644 );
@@ -293,7 +294,7 @@ static void clocksource_verify_one_cpu(void *csin) csnow_mid = cs->read (cs); } -static void clocksource_verify_percpu (struct clocksource *cs) +void clocksource_verify_percpu (struct clocksource *cs) {int64_t cs_nsec, cs_nsec_max =0 , cs_nsec_min =LLONG_MAX ;u64 csnow_begin, csnow_end;
@@ -346,6 +347,7 @@ static void clocksource_verify_percpu(struct clocksource *cs)pr_warn (" CPU %d check durations %lldns - %lldns for clocksource %s.\n" , testcpu, cs_nsec_min, cs_nsec_max, cs->name); } +EXPORT_SYMBOL_GPL (clocksource_verify_percpu);static void clocksource_watchdog (struct timer_list *unused) {
lib/Kconfig.debug +12 -0
@@ -2571,6 +2571,18 @@ config TEST_FPU If unsure, say N. +config TEST_CLOCKSOURCE_WATCHDOG + tristate "Test clocksource watchdog in kernel space" + depends on CLOCKSOURCE_WATCHDOG + help + Enable this option to create a kernel module that will trigger + a test of the clocksource watchdog. This module may be loaded + via modprobe or insmod in which case it will run upon being + loaded, or it may be built in, in which case it will run + shortly after boot. + + If unsure, say N. + endif # RUNTIME_TESTING_MENU config ARCH_USE_MEMTEST
Signature · 1 line
2.31.1.189.g2e36527f23
[tip: timers/core] clocksource: Provide kernel module to test clocksource watchdog
The following commit has been merged into the timers/core branch of tip:
The following commit has been merged into the timers/core branch of tip:
Commit-ID: 1253b9b87e42ab6a3d5c2cb27af2bdd67d7e50ff Gitweb: https://git.kernel.org/tip/1253b9b87e42ab6a3d5c2cb27af2bdd67d7e50ff Author: Paul E. McKenney <[email protected]> AuthorDate: Thu, 27 May 2021 12:01:23 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 22 Jun 2021 16:53:17 +02:00
clocksource: Provide kernel module to test clocksource watchdog
When the clocksource watchdog marks a clock as unstable, this might be due to that clock being unstable or it might be due to delays that happen to occur between the reads of the two clocks. It would be good to have a way of testing the clocksource watchdog's ability to distinguish between these two causes of clock skew and instability.
Therefore, provide a new clocksource-wdtest module selected by a new TEST_CLOCKSOURCE_WATCHDOG Kconfig option. This module has a single module parameter named "holdoff" that provides the number of seconds of delay before testing should start, which defaults to zero when built as a module and to 10 seconds when built directly into the kernel. Very large systems that boot slowly may need to increase the value of this module parameter.
This module uses hand-crafted clocksource structures to do its testing, thus avoiding messing up timing for the rest of the kernel and for user applications. This module first verifies that the ->uncertainty_margin field of the clocksource structures are set sanely. It then tests the delay-detection capability of the clocksource watchdog, increasing the number of consecutive delays injected, first provoking console messages complaining about the delays and finally forcing a clock-skew event. Unexpected test results cause at least one WARN_ON_ONCE() console splat. If there are no splats, the test has passed. Finally, it fuzzes the value returned from a clocksource to test the clocksource watchdog's ability to detect time skew.
This module checks the state of its clocksource after each test, and uses WARN_ON_ONCE() to emit a console splat if there are any failures. This should enable all types of test frameworks to detect any such failures.
This facility is intended for diagnostic use only, and should be avoided on production systems.
--- Documentation/admin-guide/kernel-parameters.txt | 6 +- include/linux/clocksource.h | 3 +- kernel/time/Makefile | 1 +- kernel/time/clocksource-wdtest.c | 202 +++++++++++++++- kernel/time/clocksource.c | 6 +- lib/Kconfig.debug | 12 +- 6 files changed, 228 insertions(+), 2 deletions(-) create mode 100644 kernel/time/clocksource-wdtest.c
Documentation/admin-guide/kernel-parameters.txt +6 -0
@@ -597,6 +597,12 @@ The actual CPUs are chosen randomly, with no replacement if the same CPU is chosen twice. + clocksource-wdtest.holdoff= [KNL] + Set the time in seconds that the clocksource + watchdog test waits before commencing its tests. + Defaults to zero when built as a module and to + 10 seconds when built into the kernel. + clearcpuid=BITNUM[,BITNUM...] [X86] Disable CPUID feature X for the kernel. See arch/x86/include/asm/cpufeatures.h for the valid bit
include/linux/clocksource.h +3 -0
@@ -291,4 +291,7 @@ static inline void timer_probe(void) {}#define TIMER_ACPI_DECLARE (name, table_id, fn) \ ACPI_DECLARE_PROBE_ENTRY(timer, name, table_id, 0, NULL, 0, fn) +extern ulong max_cswd_read_retries; +void clocksource_verify_percpu (struct clocksource *cs); +#endif /* _LINUX_CLOCKSOURCE_H */
kernel/time/Makefile +1 -0
@@ -21,3 +21,4 @@ obj-$(CONFIG_HAVE_GENERIC_VDSO) += vsyscall.o obj-$(CONFIG_DEBUG_FS) += timekeeping_debug.o obj-$(CONFIG_TEST_UDELAY) += test_udelay.o obj-$(CONFIG_TIME_NS) += namespace.o +obj-$(CONFIG_TEST_CLOCKSOURCE_WATCHDOG) += clocksource-wdtest.o
kernel/time/clocksource-wdtest.c new file +202 -0
202 lines not on this page · raw message
kernel/time/clocksource.c +4 -2
@@ -199,8 +199,9 @@ void clocksource_mark_unstable(struct clocksource *cs)spin_unlock_irqrestore (&watchdog_lock, flags); } -static ulong max_cswd_read_retries =3 ; +ulong max_cswd_read_retries =3 ;module_param (max_cswd_read_retries, ulong,0644 ); +EXPORT_SYMBOL_GPL (max_cswd_read_retries);static int verify_n_cpus =8 ;module_param (verify_n_cpus, int,0644 );
@@ -294,7 +295,7 @@ static void clocksource_verify_one_cpu(void *csin) csnow_mid = cs->read (cs); } -static void clocksource_verify_percpu (struct clocksource *cs) +void clocksource_verify_percpu (struct clocksource *cs) {int64_t cs_nsec, cs_nsec_max =0 , cs_nsec_min =LLONG_MAX ;u64 csnow_begin, csnow_end;
@@ -347,6 +348,7 @@ static void clocksource_verify_percpu(struct clocksource *cs)pr_warn (" CPU %d check durations %lldns - %lldns for clocksource %s.\n" , testcpu, cs_nsec_min, cs_nsec_max, cs->name); } +EXPORT_SYMBOL_GPL (clocksource_verify_percpu);static void clocksource_watchdog (struct timer_list *unused) {
lib/Kconfig.debug +12 -0
@@ -2571,6 +2571,18 @@ config TEST_FPU If unsure, say N. +config TEST_CLOCKSOURCE_WATCHDOG + tristate "Test clocksource watchdog in kernel space" + depends on CLOCKSOURCE_WATCHDOG + help + Enable this option to create a kernel module that will trigger + a test of the clocksource watchdog. This module may be loaded + via modprobe or insmod in which case it will run upon being + loaded, or it may be built in, in which case it will run + shortly after boot. + + If unsure, say N. + endif # RUNTIME_TESTING_MENU config ARCH_USE_MEMTEST
PATCH v15 6/6 clocksource: Print deviation in nanoseconds for unstable case
In reply to Paul E. McKenneyFrom: Feng Tang <[email protected]>
Currently when an unstable clocksource is detected, the raw counters of that clocksource and watchdog will be printed, which can only be understood after some math calculation. So print the existing delta in nanoseconds to make it easier for humans to check the results.
--- kernel/time/clocksource.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/time/clocksource.c +4 -4
@@ -406,10 +406,10 @@ static void clocksource_watchdog(struct timer_list *unused)if (abs (cs_nsec - wd_nsec) > md) {pr_warn ("timekeeping watchdog on CPU%d: Marking clocksource '%s' as unstable because the skew is too large:\n" ,smp_processor_id (), cs->name); -pr_warn (" '%s' wd_now: %llx wd_last: %llx mask: %llx\n" , - watchdog->name, wdnow, wdlast, watchdog->mask); -pr_warn (" '%s' cs_now: %llx cs_last: %llx mask: %llx\n" , - cs->name, csnow, cslast, cs->mask); +pr_warn (" '%s' wd_nsec: %lld wd_now: %llx wd_last: %llx mask: %llx\n" , + watchdog->name, wd_nsec, wdnow, wdlast, watchdog->mask); +pr_warn (" '%s' cs_nsec: %lld cs_now: %llx cs_last: %llx mask: %llx\n" , + cs->name, cs_nsec, csnow, cslast, cs->mask);if (curr_clocksource == cs)pr_warn (" '%s' is current clocksource.\n" , cs->name);else if (curr_clocksource)
Signature · 1 line
2.31.1.189.g2e36527f23
[tip: timers/core] clocksource: Print deviation in nanoseconds when a clocksource becomes unstable
The following commit has been merged into the timers/core branch of tip:
The following commit has been merged into the timers/core branch of tip:
Commit-ID: 22a22383371667962b46bd90d534cc57669537ac Gitweb: https://git.kernel.org/tip/22a22383371667962b46bd90d534cc57669537ac Author: Feng Tang <[email protected]> AuthorDate: Thu, 27 May 2021 12:01:24 -07:00 Committer: Thomas Gleixner <[email protected]> CommitterDate: Tue, 22 Jun 2021 16:53:17 +02:00
clocksource: Print deviation in nanoseconds when a clocksource becomes unstable
Currently when an unstable clocksource is detected, the raw counters of that clocksource and watchdog will be printed, which can only be understood after some math calculation.
So print the delta in nanoseconds as well to make it easier for humans to check the results.
[ paulmck: Fix typo. ]
--- kernel/time/clocksource.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
kernel/time/clocksource.c +4 -4
@@ -407,10 +407,10 @@ static void clocksource_watchdog(struct timer_list *unused)if (abs (cs_nsec - wd_nsec) > md) {pr_warn ("timekeeping watchdog on CPU%d: Marking clocksource '%s' as unstable because the skew is too large:\n" ,smp_processor_id (), cs->name); -pr_warn (" '%s' wd_now: %llx wd_last: %llx mask: %llx\n" , - watchdog->name, wdnow, wdlast, watchdog->mask); -pr_warn (" '%s' cs_now: %llx cs_last: %llx mask: %llx\n" , - cs->name, csnow, cslast, cs->mask); +pr_warn (" '%s' wd_nsec: %lld wd_now: %llx wd_last: %llx mask: %llx\n" , + watchdog->name, wd_nsec, wdnow, wdlast, watchdog->mask); +pr_warn (" '%s' cs_nsec: %lld cs_now: %llx cs_last: %llx mask: %llx\n" , + cs->name, cs_nsec, csnow, cslast, cs->mask);if (curr_clocksource == cs)pr_warn (" '%s' is current clocksource.\n" , cs->name);else if (curr_clocksource)