Fix: use timekeeping_is_busy() to fix ktime_get() hard lockup
RFC PATCH 1/1 Fix: use timekeeping_is_busy() to fix ktime_get() hard lockup
Fix an issue affecting lttng-modules with Linux kernels starting with 3.10.
This patch depends on Linux kernel patch:
"timekeeping: introduce timekeeping_is_busy()"
Starting from Linux kernel commit 06c017fdd4dc48451a29ac37fc1db4a3f86b7f40 "timekeeping: Hold timekeepering locks in do_adjtimex and hardpps" (3.10 kernels), the xtime write seqlock is held across calls to __do_adjtimex(), which includes a call to notify_cmos_timer(), and hence schedule_delayed_work().
This introduces a side-effect for a set of tracepoints, including mainly the workqueue tracepoints: a tracer hooking on those tracepoints and reading current time with ktime_get() will cause hard system LOCKUP such as:
WARNING: CPU: 6 PID: 2258 at kernel/watchdog.c:245 watchdog_overflow_callback+0x93/0x9e() Watchdog detected hard LOCKUP on cpu 6 Modules linked in: lttng_probe_workqueue(O) lttng_probe_vmscan(O) lttng_probe_udp(O) lttng_probe_timer(O) lttng_probe_s] CPU: 6 PID: 2258 Comm: ntpd Tainted: G O 3.11.0 #158 Hardware name: Supermicro X7DAL/X7DAL, BIOS 6.00 12/03/2007 0000000000000000 ffffffff814f83eb ffffffff813b206a ffff88042fd87c78 ffffffff8106a07c 0000000000000000 ffffffff810c94c2 0000000000000000 ffff88041f31bc00 0000000000000000 ffff88042fd87d68 ffff88042fd87ef8 Call Trace: <NMI> [<ffffffff813b206a>] ? dump_stack+0x41/0x51 [<ffffffff8106a07c>] ? warn_slowpath_common+0x79/0x92 [<ffffffff810c94c2>] ? watchdog_overflow_callback+0x93/0x9e [<ffffffff8106a12d>] ? warn_slowpath_fmt+0x45/0x4a [<ffffffff810c94c2>] ? watchdog_overflow_callback+0x93/0x9e [<ffffffff810c942f>] ? watchdog_enable_all_cpus.part.2+0x31/0x31 [<ffffffff810ecc66>] ? __perf_event_overflow+0x12c/0x1ae [<ffffffff810eab60>] ? perf_event_update_userpage+0x13/0xc2 [<ffffffff81016820>] ? intel_pmu_handle_irq+0x26a/0x2fd [<ffffffff813b7a0b>] ? perf_event_nmi_handler+0x24/0x3d [<ffffffff813b728f>] ? nmi_handle.isra.3+0x58/0x12f [<ffffffff813b7a59>] ? perf_ibs_nmi_handler+0x35/0x35 [<ffffffff813b7404>] ? do_nmi+0x9e/0x2bc [<ffffffff813b6af7>] ? end_repeat_nmi+0x1e/0x2e [<ffffffff810a2a33>] ? read_seqcount_begin.constprop.4+0x8/0xf [<ffffffff810a2a33>] ? read_seqcount_begin.constprop.4+0x8/0xf [<ffffffff810a2a33>] ? read_seqcount_begin.constprop.4+0x8/0xf <<EOE>> [<ffffffff810a2d6c>] ? ktime_get+0x23/0x5e [<ffffffffa0314670>] ? lib_ring_buffer_clock_read.isra.28+0x1f/0x21 [lttng_ring_buffer_client_discard] [<ffffffffa0314786>] ? lttng_event_reserve+0x112/0x3f3 [lttng_ring_buffer_client_discard] [<ffffffffa045b1c5>] ? __event_probe__workqueue_queue_work+0x72/0xe0 [lttng_probe_workqueue] [<ffffffff812ef7e9>] ? sock_aio_read.part.10+0x110/0x124 [<ffffffff81133a36>] ? do_sync_readv_writev+0x50/0x76 [<ffffffff8107d514>] ? __queue_work+0x1ab/0x265 [<ffffffff8107da7e>] ? queue_delayed_work_on+0x3f/0x4e [<ffffffff810a473d>] ? __do_adjtimex+0x408/0x413 [<ffffffff810a3e9a>] ? do_adjtimex+0x98/0xee [<ffffffff8106cec6>] ? SYSC_adjtimex+0x32/0x5d [<ffffffff813bb74b>] ? tracesys+0xdd/0xe2
LTTng uses ktime to have the same time-base across kernel and user-space, so traces gathered from LTTng-modules and LTTng-UST can be correlated. We plan on using ktime until a fast, scalable, and fine-grained time-source for tracing that can be used across kernel and user-space, and which does not rely on read seqlock for kernel-level synchronization, makes its way into the kernel.
---
wrapper/trace-clock.h +5 -0
@@ -32,6 +32,7 @@#include <linux/ktime.h> #include <linux/time.h> #include <linux/hrtimer.h> +#include <linux/version.h> #include "random.h" static inline u64 trace_clock_monotonic_wrapper (void )
@@ -45,6 +46,10 @@ static inline u64 trace_clock_monotonic_wrapper(void)if (in_nmi ())return (u64) -EIO ; +#if (LINUX_VERSION_CODE >=KERNEL_VERSION (3 ,10 ,0 )) +if (timekeeping_is_busy ()) +return (u64) -EIO ; +#endif ktime =ktime_get ();return ktime_to_ns (ktime); }
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 30 lines · Mathieu Desnoyers
LTTng uses ktime to have the same time-base across kernel and user-space, so traces gathered from LTTng-modules and LTTng-UST can be correlated. We plan on using ktime until a fast, scalable, and fine-grained time-source for tracing that can be used across kernel and user-space, and which does not rely on read seqlock for kernel-level synchronization, makes its way into the kernel.
Cc Thomas Gleixner <[email protected]>Cc Richard Cochran <[email protected]>Cc Prarit Bhargava <[email protected]>Cc John Stultz <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Steven Rostedt <[email protected]>Cc Ingo Molnar <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>---
wrapper/trace-clock.h +1
-0@@ -32,6 +32,7 @@#include <linux/ktime.h> #include <linux/time.h> #include <linux/hrtimer.h> +#include <linux/version.h> #include "random.h" static inline u64 trace_clock_monotonic_wrapper (void )
wrapper/trace-clock.h static inline u64 trace_clock_monotonic_wrapper(void) · line 55
Quoted 10 lines
wrapper/trace-clock.h +4
-0@@ -45,6 +46,10 @@ static inline u64 trace_clock_monotonic_wrapper(void)if (in_nmi ())return (u64) -EIO ; +#if (LINUX_VERSION_CODE >=KERNEL_VERSION (3 ,10 ,0 )) +if (timekeeping_is_busy ()) +return (u64) -EIO ; +#endif ktime =ktime_get ();return ktime_to_ns (ktime); }
I guess the other question here is should this functionality be pushed down into the timekeeping accessors themselves?
I know any extra checks would probably be considered overhead in some uses, but if we do the check only when we hit contention then it might not be so bad.
thanks
-john
Quoted 46 lines · * John Stultz ([email protected]) wrote:
Quoted 30 lines · Mathieu Desnoyers
LTTng uses ktime to have the same time-base across kernel and user-space, so traces gathered from LTTng-modules and LTTng-UST can be correlated. We plan on using ktime until a fast, scalable, and fine-grained time-source for tracing that can be used across kernel and user-space, and which does not rely on read seqlock for kernel-level synchronization, makes its way into the kernel.
Cc Thomas Gleixner <[email protected]>Cc Richard Cochran <[email protected]>Cc Prarit Bhargava <[email protected]>Cc John Stultz <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Steven Rostedt <[email protected]>Cc Ingo Molnar <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>---
wrapper/trace-clock.h +1
-0@@ -32,6 +32,7 @@#include <linux/ktime.h> #include <linux/time.h> #include <linux/hrtimer.h> +#include <linux/version.h> #include "random.h" static inline u64 trace_clock_monotonic_wrapper (void )wrapper/trace-clock.h static inline u64 trace_clock_monotonic_wrapper(void) · line 55
Quoted 10 lines
wrapper/trace-clock.h +4
-0@@ -45,6 +46,10 @@ static inline u64 trace_clock_monotonic_wrapper(void)if (in_nmi ())return (u64) -EIO ; +#if (LINUX_VERSION_CODE >=KERNEL_VERSION (3 ,10 ,0 )) +if (timekeeping_is_busy ()) +return (u64) -EIO ; +#endif ktime =ktime_get ();return ktime_to_ns (ktime); }I guess the other question here is should this functionality be pushed down into the timekeeping accessors themselves?
Quoted 3 lines
I know any extra checks would probably be considered overhead in some uses, but if we do the check only when we hit contention then it might not be so bad.
I thought about the exact same thing, but wanted to keep my initial kernel patch minimal, so I chose not to touch the fast paths initially.
Indeed, if we only do this check after the seqretry has failed, we should be able to add this check without touching the fast-path.
It might be cleaner to make ktime_get() return an error rather than cause a hard lockup in those cases. Especially if it can be done without performance regression.
Thanks,
Mathieu
Quoted 6 lines
thanks
-john
Signature · 3 lines
Mathieu Desnoyers EfficiOS Inc. http://www.efficios.com
Quoted 54 lines · Mathieu Desnoyers
Quoted 46 lines · * John Stultz ([email protected]) wrote:
Quoted 30 lines · Mathieu Desnoyers
LTTng uses ktime to have the same time-base across kernel and user-space, so traces gathered from LTTng-modules and LTTng-UST can be correlated. We plan on using ktime until a fast, scalable, and fine-grained time-source for tracing that can be used across kernel and user-space, and which does not rely on read seqlock for kernel-level synchronization, makes its way into the kernel.
Cc Thomas Gleixner <[email protected]>Cc Richard Cochran <[email protected]>Cc Prarit Bhargava <[email protected]>Cc John Stultz <[email protected]>Cc Greg Kroah-Hartman <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Steven Rostedt <[email protected]>Cc Ingo Molnar <[email protected]>Signed-off-by Mathieu Desnoyers <[email protected]>---
wrapper/trace-clock.h +1
-0@@ -32,6 +32,7 @@#include <linux/ktime.h> #include <linux/time.h> #include <linux/hrtimer.h> +#include <linux/version.h> #include "random.h" static inline u64 trace_clock_monotonic_wrapper (void )wrapper/trace-clock.h static inline u64 trace_clock_monotonic_wrapper(void) · line 55
Quoted 10 lines
wrapper/trace-clock.h +4
-0@@ -45,6 +46,10 @@ static inline u64 trace_clock_monotonic_wrapper(void)if (in_nmi ())return (u64) -EIO ; +#if (LINUX_VERSION_CODE >=KERNEL_VERSION (3 ,10 ,0 )) +if (timekeeping_is_busy ()) +return (u64) -EIO ; +#endif ktime =ktime_get ();return ktime_to_ns (ktime); }I guess the other question here is should this functionality be pushed down into the timekeeping accessors themselves?
Quoted 3 lines
I know any extra checks would probably be considered overhead in some uses, but if we do the check only when we hit contention then it might not be so bad.
I thought about the exact same thing, but wanted to keep my initial kernel patch minimal, so I chose not to touch the fast paths initially.
Quoted 6 lines
Indeed, if we only do this check after the seqretry has failed, we should be able to add this check without touching the fast-path.
It might be cleaner to make
ktime_get()return an error rather than cause a hard lockup in those cases. Especially if it can be done without performance regression.
Nope. ktime_get() is not going to fail ever. We want to deadlock when its called from inside xtime_lock held code. Simply because it's wrong to do so.
If there are special use cases, i.e. tracing, which need this kind of check, then we rather add a new interface, e.g. ktime_get_tracetime(), than adding a tasteless bogosity like timekeeping_busy().
Thanks,
tglx