sched/clock: Fix local_clock() before sched_clock_init()
PATCH RESEND 1/1 sched/clock: Fix local_clock() before sched_clock_init()
Have local_clock() return sched_clock() if sched_clock_init() has not yet run. sched_clock_cpu() has this check but it was not included in the new noinstr implementation of local_clock().
The effect can be seen on x86 with CONFIG_PRINTK_TIME enabled, for instance. scd->clock quickly reaches the value of TICK_NSEC and that value is returned until sched_clock_init() runs.
dmesg without this patch:
[ 0.000000] kvm-clock: ...
[ 0.000002] kvm-clock: ...
[ 0.000672] clocksource: ...
[ 0.001000] tsc: ...
[ 0.001000] e820: ...
[ 0.001000] e820: ...
...
[ 0.001000] ..TIMER: ...
[ 0.001000] clocksource: ...
[ 0.378956] Calibrating delay loop ...
[ 0.379955] pid_max: ...dmesg with this patch:
[ 0.000000] kvm-clock: ...
[ 0.000001] kvm-clock: ...
[ 0.000675] clocksource: ...
[ 0.002685] tsc: ...
[ 0.003331] e820: ...
[ 0.004190] e820: ...
...
[ 0.421939] ..TIMER: ...
[ 0.422842] clocksource: ...
[ 0.424582] Calibrating delay loop ...
[ 0.425580] pid_max: ...--- kernel/sched/clock.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/sched/clock.c +3 -0
@@ -300,6 +300,9 @@ noinstr u64 local_clock(void)if (static_branch_likely (&__sched_clock_stable))return sched_clock () + __sched_clock_offset; +if (!static_branch_likely (&sched_clock_running)) +return sched_clock (); +preempt_disable_notrace (); clock =sched_clock_local (this_scd ());preempt_enable_notrace ();
Signature · 1 line
2.39.2
Peter Zijlstra
maintainerQuoted 36 lines · Aaron Thompson
Have
local_clock()returnsched_clock()ifsched_clock_init()has not yet run.sched_clock_cpu()has this check but it was not included in the new noinstr implementation oflocal_clock().The effect can be seen on x86 with
CONFIG_PRINTK_TIMEenabled, for instance.scd->clockquickly reaches the value ofTICK_NSECand that value is returned untilsched_clock_init()runs.dmesg without this patch:
[ 0.000000] kvm-clock: ... [ 0.000002] kvm-clock: ... [ 0.000672] clocksource: ... [ 0.001000] tsc: ... [ 0.001000] e820: ... [ 0.001000] e820: ... ... [ 0.001000] ..TIMER: ... [ 0.001000] clocksource: ... [ 0.378956] Calibrating delay loop ... [ 0.379955] pid_max: ...dmesg with this patch:
[ 0.000000] kvm-clock: ... [ 0.000001] kvm-clock: ... [ 0.000675] clocksource: ... [ 0.002685] tsc: ... [ 0.003331] e820: ... [ 0.004190] e820: ... ... [ 0.421939] ..TIMER: ... [ 0.422842] clocksource: ... [ 0.424582] Calibrating delay loop ... [ 0.425580] pid_max: ...
Quoted 2 lines
Fixes 776f22913b8e ("sched/clock: Make local_clock() noinstr")Signed-off-by Aaron Thompson <[email protected]>
Thanks!
The following commit has been merged into the sched/core branch of tip:
The following commit has been merged into the sched/core branch of tip:
Commit-ID: f31dcb152a3d0816e2f1deab4e64572336da197d Gitweb: https://git.kernel.org/tip/f31dcb152a3d0816e2f1deab4e64572336da197d Author: Aaron Thompson <[email protected]> AuthorDate: Thu, 13 Apr 2023 17:50:12 Committer: Peter Zijlstra <[email protected]> CommitterDate: Fri, 21 Apr 2023 13:24:21 +02:00
sched/clock: Fix local_clock() before sched_clock_init()
Have local_clock() return sched_clock() if sched_clock_init() has not yet run. sched_clock_cpu() has this check but it was not included in the new noinstr implementation of local_clock().
The effect can be seen on x86 with CONFIG_PRINTK_TIME enabled, for instance. scd->clock quickly reaches the value of TICK_NSEC and that value is returned until sched_clock_init() runs.
dmesg without this patch:
[ 0.000000] kvm-clock: ...
[ 0.000002] kvm-clock: ...
[ 0.000672] clocksource: ...
[ 0.001000] tsc: ...
[ 0.001000] e820: ...
[ 0.001000] e820: ...
...
[ 0.001000] ..TIMER: ...
[ 0.001000] clocksource: ...
[ 0.378956] Calibrating delay loop ...
[ 0.379955] pid_max: ...dmesg with this patch:
[ 0.000000] kvm-clock: ...
[ 0.000001] kvm-clock: ...
[ 0.000675] clocksource: ...
[ 0.002685] tsc: ...
[ 0.003331] e820: ...
[ 0.004190] e820: ...
...
[ 0.421939] ..TIMER: ...
[ 0.422842] clocksource: ...
[ 0.424582] Calibrating delay loop ...
[ 0.425580] pid_max: ...--- kernel/sched/clock.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/sched/clock.c +3 -0
@@ -300,6 +300,9 @@ noinstr u64 local_clock(void)if (static_branch_likely (&__sched_clock_stable))return sched_clock () + __sched_clock_offset; +if (!static_branch_likely (&sched_clock_running)) +return sched_clock (); +preempt_disable_notrace (); clock =sched_clock_local (this_scd ());preempt_enable_notrace ();