Skip to contentSkip to conversation

Sleepable tracepoints

RFC PATCH In review Michael Jeanson 7 participants 20 messages Oct 23, 2020 – Feb 11, 2021

Subsystems (16)
lore
Share & tools
lore
#0 lore raw

cover Sleepable tracepoints

When invoked from system call enter/exit instrumentation, accessing user-space data is a common use-case for tracers. However, tracepoints currently disable preemption around iteration on the registered tracepoint probes and invocation of the probe callbacks, which prevents tracers from handling page faults.

Extend the tracepoint and trace event APIs to allow specific tracer probes to take page faults. Adapt ftrace, perf, and ebpf to allow being called from sleepable context, and convert the system call enter/exit instrumentation to sleepable tracepoints.

This series only implements the tracepoint infrastructure required to allow tracers to handle page faults. Modifying each tracer to handle those page faults would be a next step after we all agree on this piece of instrumentation infrastructure.

This patchset is base on v5.9.1.

Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>

Mathieu Desnoyers (1):
tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

Michael Jeanson (5):
tracing: introduce sleepable tracepoints
tracing: ftrace: add support for sleepable tracepoints
tracing: bpf-trace: add support for sleepable tracepoints
tracing: perf: add support for sleepable tracepoints
tracing: convert sys_enter/exit to sleepable tracepoints

 include/linux/tracepoint-defs.h |  11 ++++
 include/linux/tracepoint.h      | 104 +++++++++++++++++++++-----------
 include/trace/bpf_probe.h       |  23 ++++++-
 include/trace/define_trace.h    |   7 +++
 include/trace/events/syscalls.h |   4 +-
 include/trace/perf.h            |  26 ++++++--
 include/trace/trace_events.h    |  79 ++++++++++++++++++++++--
 init/Kconfig                    |   1 +
 kernel/trace/bpf_trace.c        |   5 +-
 kernel/trace/trace_events.c     |  15 ++++-
 kernel/trace/trace_syscalls.c   |  68 +++++++++++++--------
 kernel/tracepoint.c             | 104 +++++++++++++++++++++++++-------
 12 files changed, 351 insertions(+), 96 deletions(-)
Signature · 1 line
2.25.1
#1 lore raw

RFC PATCH 1/6 tracing: introduce sleepable tracepoints

When invoked from system call enter/exit instrumentation, accessing user-space data is a common use-case for tracers. However, tracepoints currently disable preemption around iteration on the registered tracepoint probes and invocation of the probe callbacks, which prevents tracers from handling page faults.

Extend the tracepoint and trace event APIs to allow defining a sleepable tracepoint which invokes its callback with preemption enabled.

Also extend the tracepoint API to allow tracers to request specific probes to be connected to those sleepable tracepoints. When the TRACEPOINT_MAYSLEEP flag is provided on registration, the probe callback will be called with preemption enabled, and is allowed to take page faults. Sleepable probes can only be registered on sleepable tracepoints and non-sleepable probes on non-sleepable tracepoints.

The tasks trace rcu mechanism is used to synchronize read-side marshalling of the registered probes with respect to sleepable probes unregistration and teardown.

Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>
---
 include/linux/tracepoint-defs.h |  11 ++++
 include/linux/tracepoint.h      |  85 +++++++++++++++++++++-----
 include/trace/define_trace.h    |   7 +++
 include/trace/trace_events.h    |   6 ++
 init/Kconfig                    |   1 +
 kernel/tracepoint.c             | 103 ++++++++++++++++++++++++++------
 6 files changed, 181 insertions(+), 32 deletions(-)
include/linux/tracepoint-defs.h +11 -0
index b29950a19205..87ff40cf343f 100644
@@ -27,12 +27,23 @@ struct tracepoint_func {
 	int prio;
 };
 
+/**
+ * enum tracepoint_flags - Tracepoint flags
+ * @TRACEPOINT_MAYSLEEP: The tracepoint probe callback will be called with
+ *                       preemption enabled, and is allowed to take page
+ *                       faults.
+ */
+enum tracepoint_flags {
+	TRACEPOINT_MAYSLEEP = (1 << 0),
+};
+
 struct tracepoint {
 	const char *name;		/* Tracepoint name */
 	struct static_key key;
 	int (*regfunc)(void);
 	void (*unregfunc)(void);
 	struct tracepoint_func __rcu *funcs;
+	unsigned int flags;
 };
 
 #ifdef CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
include/linux/tracepoint.h +70 -15

217 lines not on this page · raw message

include/trace/define_trace.h +7 -0
index bd75f97867b9..2b6ae7c978b3 100644
@@ -41,6 +41,12 @@
 		assign, print, reg, unreg)			\
 	DEFINE_TRACE_FN(name, reg, unreg)
 
+/* Define a trace event with the MAYSLEEP flag set */
+#undef TRACE_EVENT_FN_MAYSLEEP
+#define TRACE_EVENT_FN_MAYSLEEP(name, proto, args, tstruct,		\
+		assign, print, reg, unreg)			\
+	DEFINE_TRACE_FN_FLAGS(name, reg, unreg, TRACEPOINT_MAYSLEEP)
+
 #undef TRACE_EVENT_FN_COND
 #define TRACE_EVENT_FN_COND(name, proto, args, cond, tstruct,		\
 		assign, print, reg, unreg)			\
@@ -106,6 +112,7 @@
 
 #undef TRACE_EVENT
 #undef TRACE_EVENT_FN
+#undef TRACE_EVENT_FN_MAYSLEEP
 #undef TRACE_EVENT_FN_COND
 #undef TRACE_EVENT_CONDITION
 #undef TRACE_EVENT_NOP
include/trace/trace_events.h +6 -0
index 1bc3e7bba9a4..8b3f4068a702 100644
@@ -138,6 +138,12 @@ TRACE_MAKE_SYSTEM_STR();
 	TRACE_EVENT(name, PARAMS(proto), PARAMS(args),			\
 		PARAMS(tstruct), PARAMS(assign), PARAMS(print))		\
 
+#undef TRACE_EVENT_FN_MAYSLEEP
+#define TRACE_EVENT_FN_MAYSLEEP(name, proto, args, tstruct,		\
+		assign, print, reg, unreg)				\
+	TRACE_EVENT(name, PARAMS(proto), PARAMS(args),			\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print))		\
+
 #undef TRACE_EVENT_FN_COND
 #define TRACE_EVENT_FN_COND(name, proto, args, cond, tstruct,	\
 		assign, print, reg, unreg)				\
init/Kconfig +1 -0
index d6a0b31b13dc..857f57562490 100644
@@ -2018,6 +2018,7 @@ config PROFILING
 #
 config TRACEPOINTS
 	bool
+	select TASKS_TRACE_RCU
 
 endmenu		# General setup
 
kernel/tracepoint.c +86 -17

165 lines not on this page · raw message

Signature · 1 line
2.25.1
#2 lore raw

Re: [RFC PATCH 1/6] tracing: introduce sleepable tracepoints

Quoted 12 lines · Michael Jeanson
include/linux/tracepoint.h +2 -1
-#define __DO_TRACE(tp, proto, args, cond, rcuidle)			\
+#define __DO_TRACE(tp, proto, args, cond, rcuidle, tp_flags)		\
 	do {								\
 		struct tracepoint_func *it_func_ptr;			\
 		void *it_func;						\
 		void *__data;						\
 		int __maybe_unused __idx = 0;				\
+		bool maysleep = (tp_flags) & TRACEPOINT_MAYSLEEP;	\
 									\
 		if (!(cond))						\
 			return;						\

include/linux/tracepoint.h static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) · line 182

Quoted 7 lines
include/linux/tracepoint.h +2 -2
@@ -170,8 +178,13 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		/* srcu can't be used from NMI */			\
 		WARN_ON_ONCE(rcuidle && in_nmi());			\
 									\
-		/* keep srcu and sched-rcu usage consistent */		\
-		preempt_disable_notrace();				\
+		if (maysleep) {						\
+			might_sleep();					\

The main purpose of the patch set is to access user memory in tracepoints, right? In such case I suggest to use stronger might_fault() here.
We used might_sleep() in sleepable bpf and it wasn't enough to catch a combination where sleepable hook was invoked while mm->mmap_lock was taken which may cause a deadlock.

Quoted 5 lines
include/linux/tracepoint.h +5 -0
+			rcu_read_lock_trace();				\
+		} else {						\
+			/* keep srcu and sched-rcu usage consistent */	\
+			preempt_disable_notrace();			\
+		}							\
#3 lore raw

Re: [RFC PATCH 1/6] tracing: introduce sleepable tracepoints

Quoted 16 lines · ----- On Oct 26, 2020, at 6:43 PM, Alexei Starovoitov…
Quoted 15 lines · Michael Jeanson
include/linux/tracepoint.h +2 -1
-#define __DO_TRACE(tp, proto, args, cond, rcuidle)			\
+#define __DO_TRACE(tp, proto, args, cond, rcuidle, tp_flags)		\
 	do {								\
 		struct tracepoint_func *it_func_ptr;			\
 		void *it_func;						\
 		void *__data;						\
 		int __maybe_unused __idx = 0;				\
+		bool maysleep = (tp_flags) & TRACEPOINT_MAYSLEEP;	\
 									\
 		if (!(cond))						\
 			return;						\
@@ -170,8 +178,13 @@ static inline struct tracepoint
 *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		/* srcu can't be used from NMI */			\
 		WARN_ON_ONCE(rcuidle && in_nmi());			\
Quoted 8 lines

include/linux/tracepoint.h static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) · line 182

Quoted 5 lines
include/linux/tracepoint.h +2 -2
 									\
-		/* keep srcu and sched-rcu usage consistent */		\
-		preempt_disable_notrace();				\
+		if (maysleep) {						\
+			might_sleep();					\

The main purpose of the patch set is to access user memory in tracepoints, right?

Yes, exactly.

Quoted 4 lines

In such case I suggest to use stronger might_fault() here. We used might_sleep() in sleepable bpf and it wasn't enough to catch a combination where sleepable hook was invoked while mm->mmap_lock was taken which may cause a deadlock.

Good point! We will do that for the next round.

By the way, we named this "sleepable" tracepoint (with flag TRACEPOINT_MAYSLEEP), but we are open to a better name. Would TRACEPOINT_MAYFAULT be more descriptive ? (a "faultable" tracepoint sounds weird though)

Thanks,

Mathieu

Quoted 6 lines
Quoted 6 lines
include/linux/tracepoint.h +5 -0
+			rcu_read_lock_trace();				\
+		} else {						\
+			/* keep srcu and sched-rcu usage consistent */	\
+			preempt_disable_notrace();			\
+		}							\
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#4 lore raw

Re: [RFC PATCH 1/6] tracing: introduce sleepable tracepoints

Quoted 35 lines · Mathieu Desnoyers
Quoted 16 lines · ----- On Oct 26, 2020, at 6:43 PM, Alexei Starovoitov…
Quoted 15 lines · Michael Jeanson
include/linux/tracepoint.h +2 -1
-#define __DO_TRACE(tp, proto, args, cond, rcuidle)			\
+#define __DO_TRACE(tp, proto, args, cond, rcuidle, tp_flags)		\
 	do {								\
 		struct tracepoint_func *it_func_ptr;			\
 		void *it_func;						\
 		void *__data;						\
 		int __maybe_unused __idx = 0;				\
+		bool maysleep = (tp_flags) & TRACEPOINT_MAYSLEEP;	\
 									\
 		if (!(cond))						\
 			return;						\
@@ -170,8 +178,13 @@ static inline struct tracepoint
 *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		/* srcu can't be used from NMI */			\
 		WARN_ON_ONCE(rcuidle && in_nmi());			\
Quoted 8 lines

include/linux/tracepoint.h static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) · line 182

Quoted 5 lines
include/linux/tracepoint.h +2 -2
 									\
-		/* keep srcu and sched-rcu usage consistent */		\
-		preempt_disable_notrace();				\
+		if (maysleep) {						\
+			might_sleep();					\

The main purpose of the patch set is to access user memory in tracepoints, right?

Yes, exactly.

Quoted 4 lines

In such case I suggest to use stronger might_fault() here. We used might_sleep() in sleepable bpf and it wasn't enough to catch a combination where sleepable hook was invoked while mm->mmap_lock was taken which may cause a deadlock.

Quoted 5 lines

Good point! We will do that for the next round.

By the way, we named this "sleepable" tracepoint (with flag TRACEPOINT_MAYSLEEP), but we are open to a better name. Would TRACEPOINT_MAYFAULT be more descriptive ? (a "faultable" tracepoint sounds weird though)

bpf kept 'sleepable' as a name. 'faultable' is too misleading.

#5 lore raw

Re: [RFC PATCH 1/6] tracing: introduce sleepable tracepoints

Quoted 39 lines · ----- On Oct 28, 2020, at 5:23 PM, Alexei Starovoitov…
Quoted 38 lines · Mathieu Desnoyers

----- On Oct 26, 2020, at 6:43 PM, Alexei Starovoitov

Quoted 16 lines · [email protected] wrote:
Quoted 15 lines · Michael Jeanson
include/linux/tracepoint.h +2 -1
-#define __DO_TRACE(tp, proto, args, cond, rcuidle)			\
+#define __DO_TRACE(tp, proto, args, cond, rcuidle, tp_flags)		\
 	do {								\
 		struct tracepoint_func *it_func_ptr;			\
 		void *it_func;						\
 		void *__data;						\
 		int __maybe_unused __idx = 0;				\
+		bool maysleep = (tp_flags) & TRACEPOINT_MAYSLEEP;	\
 									\
 		if (!(cond))						\
 			return;						\
@@ -170,8 +178,13 @@ static inline struct tracepoint
 *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		/* srcu can't be used from NMI */			\
 		WARN_ON_ONCE(rcuidle && in_nmi());			\
Quoted 8 lines

include/linux/tracepoint.h static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) · line 182

Quoted 5 lines
include/linux/tracepoint.h +2 -2
 									\
-		/* keep srcu and sched-rcu usage consistent */		\
-		preempt_disable_notrace();				\
+		if (maysleep) {						\
+			might_sleep();					\

The main purpose of the patch set is to access user memory in tracepoints, right?

Yes, exactly.

Quoted 4 lines

In such case I suggest to use stronger might_fault() here. We used might_sleep() in sleepable bpf and it wasn't enough to catch a combination where sleepable hook was invoked while mm->mmap_lock was taken which may cause a deadlock.

Good point! We will do that for the next round.

Quoted 7 lines
Quoted 5 lines

By the way, we named this "sleepable" tracepoint (with flag TRACEPOINT_MAYSLEEP),
but we are open to a better name. Would TRACEPOINT_MAYFAULT be more descriptive ?
(a "faultable" tracepoint sounds weird though)

bpf kept 'sleepable' as a name. 'faultable' is too misleading.

We're working on an updated patchset for those "sleepable tracepoints", and considering that those are really "tracepoints allowing page faults", I must admit that I am uncomfortable with the confusion between "sleep" and "fault" in the naming here.

I am tempted to do the following changes:

- Change name from "sleepable tracepoints" to a better suited "tracepoints allowing page faults",
- Use might_fault() rather than might_sleep() in __DO_TRACE(), effectively guaranteeing that all probes connecting to a tracepoint which allows page faults can indeed take page faults.
- Change TRACEPOINT_MAYSLEEP into TRACEPOINT_MAYFAULT.

Any objections ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#6 lore raw

Re: [RFC PATCH 1/6] tracing: introduce sleepable tracepoints

In reply to Mathieu Desnoyers
Quoted 35 lines · Mathieu Desnoyers
Quoted 16 lines · ----- On Oct 26, 2020, at 6:43 PM, Alexei Starovoitov…
Quoted 15 lines · Michael Jeanson
include/linux/tracepoint.h +2 -1
-#define __DO_TRACE(tp, proto, args, cond, rcuidle)			\
+#define __DO_TRACE(tp, proto, args, cond, rcuidle, tp_flags)		\
 	do {								\
 		struct tracepoint_func *it_func_ptr;			\
 		void *it_func;						\
 		void *__data;						\
 		int __maybe_unused __idx = 0;				\
+		bool maysleep = (tp_flags) & TRACEPOINT_MAYSLEEP;	\
 									\
 		if (!(cond))						\
 			return;						\
@@ -170,8 +178,13 @@ static inline struct tracepoint
 *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		/* srcu can't be used from NMI */			\
 		WARN_ON_ONCE(rcuidle && in_nmi());			\
Quoted 8 lines

include/linux/tracepoint.h static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p) · line 182

Quoted 5 lines
include/linux/tracepoint.h +2 -2
 									\
-		/* keep srcu and sched-rcu usage consistent */		\
-		preempt_disable_notrace();				\
+		if (maysleep) {						\
+			might_sleep();					\

The main purpose of the patch set is to access user memory in tracepoints, right?

Yes, exactly.

Quoted 4 lines

In such case I suggest to use stronger might_fault() here. We used might_sleep() in sleepable bpf and it wasn't enough to catch a combination where sleepable hook was invoked while mm->mmap_lock was taken which may cause a deadlock.

Quoted 5 lines

Good point! We will do that for the next round.

By the way, we named this "sleepable" tracepoint (with flag TRACEPOINT_MAYSLEEP), but we are open to a better name. Would TRACEPOINT_MAYFAULT be more descriptive ? (a "faultable" tracepoint sounds weird though)

What about keeping it might_sleep() here and then adding might_fault() in the probe handler? Since the probe handler knows that it may cause page fault, it could itself make sure about it.

One more thought: Should we make _all_ tracepoints sleepable, and then move the preempt_disable() bit to the probe handler as needed? That could simplify the tracepoint API as well. Steven said before that whoever registers probes knows what they are doing so I am ok with that.

No strong feelings one way or the other, for either of these though.

thanks,

- Joel

Quoted 16 lines

Thanks,

Mathieu

Quoted 6 lines
Quoted 6 lines
include/linux/tracepoint.h +5 -0
+			rcu_read_lock_trace();				\
+		} else {						\
+			/* keep srcu and sched-rcu usage consistent */	\
+			preempt_disable_notrace();			\
+		}							\
Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#7 lore raw

RFC PATCH 2/6 tracing: ftrace: add support for sleepable tracepoints

In reply to Michael Jeanson

In preparation for converting system call enter/exit instrumentation into sleepable tracepoints, make sure that ftrace can handle registering to such tracepoints by explicitly disabling preemption within the ftrace tracepoint probes to respect the current expectations within ftrace ring buffer code.

This change does not yet allow ftrace to take page faults per se within its probe, but allows its existing probes to connect to sleepable tracepoints.

Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>
---
 include/trace/trace_events.h | 75 +++++++++++++++++++++++++++++++++---
 kernel/trace/trace_events.c  | 15 +++++++-
 2 files changed, 83 insertions(+), 7 deletions(-)
include/trace/trace_events.h +70 -5
index 8b3f4068a702..b95a9c3d9405 100644
@@ -80,6 +80,16 @@ TRACE_MAKE_SYSTEM_STR();
 			     PARAMS(print));		       \
 	DEFINE_EVENT(name, name, PARAMS(proto), PARAMS(args));
 
+#undef TRACE_EVENT_MAYSLEEP
+#define TRACE_EVENT_MAYSLEEP(name, proto, args, tstruct, assign, print)	\
+	DECLARE_EVENT_CLASS_MAYSLEEP(name,		       \
+			     PARAMS(proto),		       \
+			     PARAMS(args),		       \
+			     PARAMS(tstruct),		       \
+			     PARAMS(assign),		       \
+			     PARAMS(print));		       \
+	DEFINE_EVENT(name, name, PARAMS(proto), PARAMS(args));
+
 
 #undef __field
 #define __field(type, item)		type	item;
@@ -118,6 +128,12 @@ TRACE_MAKE_SYSTEM_STR();
 									\
 	static struct trace_event_class event_class_##name;
 
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(name, proto, args,			\
+		tstruct, assign, print)					\
+	DECLARE_EVENT_CLASS(name, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print))
+
 #undef DEFINE_EVENT
 #define DEFINE_EVENT(template, name, proto, args)	\
 	static struct trace_event_call	__used		\
@@ -141,7 +157,7 @@ TRACE_MAKE_SYSTEM_STR();
 #undef TRACE_EVENT_FN_MAYSLEEP
 #define TRACE_EVENT_FN_MAYSLEEP(name, proto, args, tstruct,		\
 		assign, print, reg, unreg)				\
-	TRACE_EVENT(name, PARAMS(proto), PARAMS(args),			\
+	TRACE_EVENT_MAYSLEEP(name, PARAMS(proto), PARAMS(args),		\
 		PARAMS(tstruct), PARAMS(assign), PARAMS(print))		\
 
 #undef TRACE_EVENT_FN_COND
@@ -212,6 +228,12 @@ TRACE_MAKE_SYSTEM_STR();
 		tstruct;						\
 	};
 
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args,			\
+		tstruct, assign, print)					\
+	DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print))
+
 #undef DEFINE_EVENT
 #define DEFINE_EVENT(template, name, proto, args)
 
@@ -378,6 +400,12 @@ static struct trace_event_functions trace_event_type_funcs_##call = {	\
 	.trace			= trace_raw_output_##call,		\
 };
 
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args,			\
+		tstruct, assign, print)					\
+	DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print))
+
 #undef DEFINE_EVENT_PRINT
 #define DEFINE_EVENT_PRINT(template, call, proto, args, print)		\
 static notrace enum print_line_t					\
@@ -448,6 +476,12 @@ static struct trace_event_fields trace_event_fields_##call[] = {	\
 	tstruct								\
 	{} };
 
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args,			\
+		tstruct, func, print)					\
+	DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(func), PARAMS(print))
+
 #undef DEFINE_EVENT_PRINT
 #define DEFINE_EVENT_PRINT(template, name, proto, args, print)
 
@@ -524,6 +558,12 @@ static inline notrace int trace_event_get_offsets_##call(		\
 	return __data_size;						\
 }
 
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args,			\
+		tstruct, assign, print)					\
+	DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print))
+
 #include TRACE_INCLUDE(TRACE_INCLUDE_FILE)
 
 /*
@@ -673,8 +713,8 @@ static inline notrace int trace_event_get_offsets_##call(		\
 #undef __perf_task
 #define __perf_task(t)	(t)
 
-#undef DECLARE_EVENT_CLASS
-#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print)	\
+#undef _DECLARE_EVENT_CLASS
+#define _DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print, tp_flags)	\
 									\
 static notrace void							\
 trace_event_raw_event_##call(void *__data, proto)			\
@@ -685,8 +725,11 @@ trace_event_raw_event_##call(void *__data, proto)			\
 	struct trace_event_raw_##call *entry;				\
 	int __data_size;						\
 									\
+	if ((tp_flags) & TRACEPOINT_MAYSLEEP)				\
+		preempt_disable_notrace();				\
+									\
 	if (trace_trigger_soft_disabled(trace_file))			\
-		return;							\
+		goto end;						\
 									\
 	__data_size = trace_event_get_offsets_##call(&__data_offsets, args); \
 									\
@@ -694,14 +737,30 @@ trace_event_raw_event_##call(void *__data, proto)			\
 				 sizeof(*entry) + __data_size);		\
 									\
 	if (!entry)							\
-		return;							\
+		goto end;						\
 									\
 	tstruct								\
 									\
 	{ assign; }							\
 									\
 	trace_event_buffer_commit(&fbuffer);				\
+end:									\
+	if ((tp_flags) & TRACEPOINT_MAYSLEEP)				\
+		preempt_enable_notrace();				\
 }
+
+#undef DECLARE_EVENT_CLASS
+#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print)	\
+	_DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+			PARAMS(tstruct), PARAMS(assign),		\
+			PARAMS(print), 0)
+
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args, tstruct, assign, print)	\
+	_DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),			\
+			PARAMS(tstruct), PARAMS(assign),			\
+			PARAMS(print), TRACEPOINT_MAYSLEEP)
+
 /*
  * The ftrace_test_probe is compiled out, it is only here as a build time check
  * to make sure that if the tracepoint handling changes, the ftrace probe will
@@ -748,6 +807,12 @@ static struct trace_event_class __used __refdata event_class_##call = { \
 	_TRACE_PERF_INIT(call)						\
 };
 
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args,			\
+		tstruct, assign, print)					\
+	DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print))
+
 #undef DEFINE_EVENT
 #define DEFINE_EVENT(template, call, proto, args)			\
 									\
kernel/trace/trace_events.c +13 -2
index a85effb2373b..058fe2834f14 100644
@@ -290,9 +290,15 @@ int trace_event_reg(struct trace_event_call *call,
 	WARN_ON(!(call->flags & TRACE_EVENT_FL_TRACEPOINT));
 	switch (type) {
 	case TRACE_REG_REGISTER:
-		return tracepoint_probe_register(call->tp,
+		if (call->tp->flags & TRACEPOINT_MAYSLEEP)
+			return tracepoint_probe_register_maysleep(call->tp,
 						 call->class->probe,
 						 file);
+		else
+			return tracepoint_probe_register(call->tp,
+						 call->class->probe,
+						 file);
+
 	case TRACE_REG_UNREGISTER:
 		tracepoint_probe_unregister(call->tp,
 					    call->class->probe,
@@ -301,7 +307,12 @@ int trace_event_reg(struct trace_event_call *call,
 
 #ifdef CONFIG_PERF_EVENTS
 	case TRACE_REG_PERF_REGISTER:
-		return tracepoint_probe_register(call->tp,
+		if (call->tp->flags & TRACEPOINT_MAYSLEEP)
+			return tracepoint_probe_register_maysleep(call->tp,
+						 call->class->perf_probe,
+						 call);
+		else
+			return tracepoint_probe_register(call->tp,
 						 call->class->perf_probe,
 						 call);
 	case TRACE_REG_PERF_UNREGISTER:
Signature · 1 line
2.25.1
#8 lore raw

RFC PATCH 3/6 tracing: bpf-trace: add support for sleepable tracepoints

In reply to Michael Jeanson

In preparation for converting system call enter/exit instrumentation into sleepable tracepoints, make sure that bpf can handle registering to such tracepoints by explicitly disabling preemption within the bpf tracepoint probes to respect the current expectations within bpf tracing code.

This change does not yet allow bpf to take page faults per se within its probe, but allows its existing probes to connect to sleepable tracepoints.

Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>
---
 include/trace/bpf_probe.h | 23 +++++++++++++++++++++--
 kernel/trace/bpf_trace.c  |  5 ++++-
 2 files changed, 25 insertions(+), 3 deletions(-)
include/trace/bpf_probe.h +21 -2
index 1ce3be63add1..d688cb9b32fe 100644
@@ -55,15 +55,34 @@
 /* tracepoints with more than 12 arguments will hit build error */
 #define CAST_TO_U64(...) CONCATENATE(__CAST, COUNT_ARGS(__VA_ARGS__))(__VA_ARGS__)
 
-#undef DECLARE_EVENT_CLASS
-#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print)	\
+#undef _DECLARE_EVENT_CLASS
+#define _DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print, tp_flags)	\
 static notrace void							\
 __bpf_trace_##call(void *__data, proto)					\
 {									\
 	struct bpf_prog *prog = __data;					\
+									\
+	if ((tp_flags) & TRACEPOINT_MAYSLEEP)				\
+		preempt_disable_notrace();				\
+									\
 	CONCATENATE(bpf_trace_run, COUNT_ARGS(args))(prog, CAST_TO_U64(args));	\
+									\
+	if ((tp_flags) & TRACEPOINT_MAYSLEEP)				\
+		preempt_enable_notrace();				\
 }
 
+#undef DECLARE_EVENT_CLASS
+#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print)	\
+	_DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print),	0)
+
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args, tstruct,	\
+		assign, print)						\
+	_DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print),		\
+		TRACEPOINT_MAYSLEEP)
+
 /*
  * This part is compiled out, it is only here as a build time check
  * to make sure that if the tracepoint handling changes, the
kernel/trace/bpf_trace.c +4 -1
index a8d4f253ed77..54f8b320fe2f 100644
@@ -1947,7 +1947,10 @@ static int __bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog *
 	if (prog->aux->max_tp_access > btp->writable_size)
 		return -EINVAL;
 
-	return tracepoint_probe_register(tp, (void *)btp->bpf_func, prog);
+	if (tp->flags & TRACEPOINT_MAYSLEEP)
+		return tracepoint_probe_register_maysleep(tp, (void *)btp->bpf_func, prog);
+	else
+		return tracepoint_probe_register(tp, (void *)btp->bpf_func, prog);
 }
 
 int bpf_probe_register(struct bpf_raw_event_map *btp, struct bpf_prog *prog)
Signature · 1 line
2.25.1
#9 lore raw

RFC PATCH 4/6 tracing: perf: add support for sleepable tracepoints

In reply to Michael Jeanson

In preparation for converting system call enter/exit instrumentation into sleepable tracepoints, make sure that perf can handle registering to such tracepoints by explicitly disabling preemption within the perf tracepoint probes to respect the current expectations within perf ring buffer code.

This change does not yet allow perf to take page faults per se within its probe, but allows its existing probes to connect to sleepable tracepoints.

Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>
---
 include/trace/perf.h | 26 ++++++++++++++++++++++----
 1 file changed, 22 insertions(+), 4 deletions(-)
include/trace/perf.h +22 -4
index dbc6c74defc3..e1d866c3a076 100644
@@ -27,8 +27,8 @@
 #undef __perf_task
 #define __perf_task(t)	(__task = (t))
 
-#undef DECLARE_EVENT_CLASS
-#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print)	\
+#undef _DECLARE_EVENT_CLASS
+#define _DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print, tp_flags)	\
 static notrace void							\
 perf_trace_##call(void *__data, proto)					\
 {									\
@@ -43,13 +43,16 @@ perf_trace_##call(void *__data, proto)					\
 	int __data_size;						\
 	int rctx;							\
 									\
+	if ((tp_flags) & TRACEPOINT_MAYSLEEP)				\
+		preempt_disable_notrace();				\
+									\
 	__data_size = trace_event_get_offsets_##call(&__data_offsets, args); \
 									\
 	head = this_cpu_ptr(event_call->perf_events);			\
 	if (!bpf_prog_array_valid(event_call) &&			\
 	    __builtin_constant_p(!__task) && !__task &&			\
 	    hlist_empty(head))						\
-		return;							\
+		goto end;						\
 									\
 	__entry_size = ALIGN(__data_size + sizeof(*entry) + sizeof(u32),\
 			     sizeof(u64));				\
@@ -57,7 +60,7 @@ perf_trace_##call(void *__data, proto)					\
 									\
 	entry = perf_trace_buf_alloc(__entry_size, &__regs, &rctx);	\
 	if (!entry)							\
-		return;							\
+		goto end;						\
 									\
 	perf_fetch_caller_regs(__regs);					\
 									\
@@ -68,8 +71,23 @@ perf_trace_##call(void *__data, proto)					\
 	perf_trace_run_bpf_submit(entry, __entry_size, rctx,		\
 				  event_call, __count, __regs,		\
 				  head, __task);			\
+end:									\
+	if ((tp_flags) & TRACEPOINT_MAYSLEEP)				\
+		preempt_enable_notrace();				\
 }
 
+#undef DECLARE_EVENT_CLASS
+#define DECLARE_EVENT_CLASS(call, proto, args, tstruct, assign, print)	\
+	_DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print), 0)
+
+#undef DECLARE_EVENT_CLASS_MAYSLEEP
+#define DECLARE_EVENT_CLASS_MAYSLEEP(call, proto, args, tstruct,	\
+		assign, print)						\
+	_DECLARE_EVENT_CLASS(call, PARAMS(proto), PARAMS(args),		\
+		PARAMS(tstruct), PARAMS(assign), PARAMS(print),		\
+		TRACEPOINT_MAYSLEEP)
+
 /*
  * This part is compiled out, it is only here as a build time check
  * to make sure that if the tracepoint handling changes, the
Signature · 1 line
2.25.1
#10 lore raw

RFC PATCH 5/6 tracing: convert sys_enter/exit to sleepable tracepoints

In reply to Michael Jeanson

Convert the definition of the system call enter/exit tracepoints to sleepable tracepoints now that all upstream tracers handle it.

Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>
---
 include/trace/events/syscalls.h |  4 +-
 kernel/trace/trace_syscalls.c   | 68 ++++++++++++++++++++-------------
 2 files changed, 44 insertions(+), 28 deletions(-)
include/trace/events/syscalls.h +2 -2
index b6e0cbc2c71f..fbb8d8b48f81 100644
@@ -15,7 +15,7 @@
 
 #ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS
 
-TRACE_EVENT_FN(sys_enter,
+TRACE_EVENT_FN_MAYSLEEP(sys_enter,
 
 	TP_PROTO(struct pt_regs *regs, long id),
 
@@ -41,7 +41,7 @@ TRACE_EVENT_FN(sys_enter,
 
 TRACE_EVENT_FLAGS(sys_enter, TRACE_EVENT_FL_CAP_ANY)
 
-TRACE_EVENT_FN(sys_exit,
+TRACE_EVENT_FN_MAYSLEEP(sys_exit,
 
 	TP_PROTO(struct pt_regs *regs, long ret),
 
kernel/trace/trace_syscalls.c +42 -26

215 lines not on this page · raw message

Signature · 1 line
2.25.1
#11 lore raw

RFC PATCH 6/6 tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

In reply to Michael Jeanson

From: Mathieu Desnoyers <[email protected]>

Considering that tracer callbacks expect RCU to be watching (for instance, perf uses rcu_read_lock), we need rcuidle tracepoints to issue rcu_irq_{enter,exit}_irqson around calls to the callbacks. So there is no point in using SRCU anymore given that rcuidle tracepoints need to ensure RCU is watching. Therefore, simply use sched-RCU like normal tracepoints for rcuidle tracepoints.

Cc Michael Jeanson <[email protected]>
Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>
---
 include/linux/tracepoint.h | 33 +++++++--------------------------
 kernel/tracepoint.c        | 25 +++++++++----------------
 2 files changed, 16 insertions(+), 42 deletions(-)
include/linux/tracepoint.h +7 -26
index 0386b54cbcbb..1414b11f864b 100644
@@ -13,7 +13,6 @@
  */
 
 #include <linux/smp.h>
-#include <linux/srcu.h>
 #include <linux/errno.h>
 #include <linux/types.h>
 #include <linux/cpumask.h>
@@ -33,8 +32,6 @@ struct trace_eval_map {
 
 #define TRACEPOINT_DEFAULT_PRIO	10
 
-extern struct srcu_struct tracepoint_srcu;
-
 extern int
 tracepoint_probe_register(struct tracepoint *tp, void *probe, void *data);
 extern int
@@ -86,7 +83,6 @@ int unregister_tracepoint_module_notifier(struct notifier_block *nb)
 static inline void tracepoint_synchronize_unregister(void)
 {
 	synchronize_rcu_tasks_trace();
-	synchronize_srcu(&tracepoint_srcu);
 	synchronize_rcu();
 }
 #else
@@ -175,25 +171,13 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		if (!(cond))						\
 			return;						\
 									\
-		/* srcu can't be used from NMI */			\
-		WARN_ON_ONCE(rcuidle && in_nmi());			\
-									\
-		if (maysleep) {						\
-			might_sleep();					\
+		might_sleep_if(maysleep);				\
+		if (rcuidle)						\
+			rcu_irq_enter_irqson();				\
+		if (maysleep)						\
 			rcu_read_lock_trace();				\
-		} else {						\
-			/* keep srcu and sched-rcu usage consistent */	\
+		else							\
 			preempt_disable_notrace();			\
-		}							\
-									\
-		/*							\
-		 * For rcuidle callers, use srcu since sched-rcu	\
-		 * doesn't work from the idle path.			\
-		 */							\
-		if (rcuidle) {						\
-			__idx = srcu_read_lock_notrace(&tracepoint_srcu);\
-			rcu_irq_enter_irqson();				\
-		}							\
 									\
 		it_func_ptr = rcu_dereference_raw((tp)->funcs);		\
 									\
@@ -205,15 +189,12 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 			} while ((++it_func_ptr)->func);		\
 		}							\
 									\
-		if (rcuidle) {						\
-			rcu_irq_exit_irqson();				\
-			srcu_read_unlock_notrace(&tracepoint_srcu, __idx);\
-		}							\
-									\
 		if (maysleep)						\
 			rcu_read_unlock_trace();			\
 		else							\
 			preempt_enable_notrace();			\
+		if (rcuidle)						\
+			rcu_irq_exit_irqson();				\
 	} while (0)
 
 #ifndef MODULE
kernel/tracepoint.c +9 -16
index 8d8e41c5d8a5..68b4e50798b1 100644
@@ -18,9 +18,6 @@
 extern tracepoint_ptr_t __start___tracepoints_ptrs[];
 extern tracepoint_ptr_t __stop___tracepoints_ptrs[];
 
-DEFINE_SRCU(tracepoint_srcu);
-EXPORT_SYMBOL_GPL(tracepoint_srcu);
-
 /* Set to 1 to enable tracepoint debug output */
 static const int tracepoint_debug;
 
@@ -65,14 +62,9 @@ static void rcu_tasks_trace_free_old_probes(struct rcu_head *head)
 	kfree(container_of(head, struct tp_probes, rcu));
 }
 
-static void srcu_free_old_probes(struct rcu_head *head)
-{
-	call_rcu_tasks_trace(head, rcu_tasks_trace_free_old_probes);
-}
-
 static void rcu_free_old_probes(struct rcu_head *head)
 {
-	call_srcu(&tracepoint_srcu, head, srcu_free_old_probes);
+	call_rcu_tasks_trace(head, rcu_tasks_trace_free_old_probes);
 }
 
 static __init int release_early_probes(void)
@@ -90,7 +82,7 @@ static __init int release_early_probes(void)
 	return 0;
 }
 
-/* SRCU and Tasks Trace RCU are initialized at core_initcall */
+/* Tasks Trace RCU is initialized at core_initcall */
 postcore_initcall(release_early_probes);
 
 static inline void release_probes(struct tracepoint_func *old)
@@ -100,9 +92,8 @@ static inline void release_probes(struct tracepoint_func *old)
 			struct tp_probes, probes[0]);
 
 		/*
-		 * We can't free probes if SRCU and Tasks Trace RCU are not
-		 * initialized yet. Postpone the freeing till after both are
-		 * initialized.
+		 * We can't free probes if Tasks Trace RCU is not initialized yet.
+		 * Postpone the freeing till after Tasks Trace RCU is initialized.
 		 */
 		if (unlikely(!ok_to_free_tracepoints)) {
 			tp_probes->rcu.next = early_probes;
@@ -111,9 +102,11 @@ static inline void release_probes(struct tracepoint_func *old)
 		}
 
 		/*
-		 * Tracepoint probes are protected by sched RCU, SRCU and
-		 * Tasks Trace RCU by chaining the callbacks we cover all three
-		 * cases and wait for all three grace periods.
+		 * Tracepoint probes are protected by both sched RCU and
+		 * Tasks Trace RCU, by calling the Tasks Trace RCU callback in
+		 * the sched RCU callback we cover both cases. So let us chain
+		 * the Tasks Trace RCU and sched RCU callbacks to wait for both
+		 * grace periods.
 		 */
 		call_rcu(&tp_probes->rcu, rcu_free_old_probes);
 	}
Signature · 1 line
2.25.1
#12 lore raw

Re: [RFC PATCH 6/6] tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

Quoted 8 lines · Michael Jeanson

From: Mathieu Desnoyers <[email protected]>

Considering that tracer callbacks expect RCU to be watching (for instance, perf uses rcu_read_lock), we need rcuidle tracepoints to issue rcu_irq_{enter,exit}_irqson around calls to the callbacks. So there is no point in using SRCU anymore given that rcuidle tracepoints need to ensure RCU is watching. Therefore, simply use sched-RCU like normal tracepoints for rcuidle tracepoints.

High level question:

IIRC, doing this increases overhead for general tracing that does not use perf, for 'rcuidle' tracepoints such as the preempt/irq enable/disable tracepoints. I remember adding SRCU because of this reason.

Can the 'rcuidle' information not be pushed down further, such that perf does it because it requires RCU to be watching, so that it does not effect, say, trace events?

thanks,

- Joel

Quoted 167 lines
Cc Michael Jeanson <[email protected]>
Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>
---
 include/linux/tracepoint.h | 33 +++++++--------------------------
 kernel/tracepoint.c        | 25 +++++++++----------------
 2 files changed, 16 insertions(+), 42 deletions(-)
include/linux/tracepoint.h +7 -26
index 0386b54cbcbb..1414b11f864b 100644
@@ -13,7 +13,6 @@
  */
 
 #include <linux/smp.h>
-#include <linux/srcu.h>
 #include <linux/errno.h>
 #include <linux/types.h>
 #include <linux/cpumask.h>
@@ -33,8 +32,6 @@ struct trace_eval_map {
 
 #define TRACEPOINT_DEFAULT_PRIO	10
 
-extern struct srcu_struct tracepoint_srcu;
-
 extern int
 tracepoint_probe_register(struct tracepoint *tp, void *probe, void *data);
 extern int
@@ -86,7 +83,6 @@ int unregister_tracepoint_module_notifier(struct notifier_block *nb)
 static inline void tracepoint_synchronize_unregister(void)
 {
 	synchronize_rcu_tasks_trace();
-	synchronize_srcu(&tracepoint_srcu);
 	synchronize_rcu();
 }
 #else
@@ -175,25 +171,13 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		if (!(cond))						\
 			return;						\
 									\
-		/* srcu can't be used from NMI */			\
-		WARN_ON_ONCE(rcuidle && in_nmi());			\
-									\
-		if (maysleep) {						\
-			might_sleep();					\
+		might_sleep_if(maysleep);				\
+		if (rcuidle)						\
+			rcu_irq_enter_irqson();				\
+		if (maysleep)						\
 			rcu_read_lock_trace();				\
-		} else {						\
-			/* keep srcu and sched-rcu usage consistent */	\
+		else							\
 			preempt_disable_notrace();			\
-		}							\
-									\
-		/*							\
-		 * For rcuidle callers, use srcu since sched-rcu	\
-		 * doesn't work from the idle path.			\
-		 */							\
-		if (rcuidle) {						\
-			__idx = srcu_read_lock_notrace(&tracepoint_srcu);\
-			rcu_irq_enter_irqson();				\
-		}							\
 									\
 		it_func_ptr = rcu_dereference_raw((tp)->funcs);		\
 									\
@@ -205,15 +189,12 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 			} while ((++it_func_ptr)->func);		\
 		}							\
 									\
-		if (rcuidle) {						\
-			rcu_irq_exit_irqson();				\
-			srcu_read_unlock_notrace(&tracepoint_srcu, __idx);\
-		}							\
-									\
 		if (maysleep)						\
 			rcu_read_unlock_trace();			\
 		else							\
 			preempt_enable_notrace();			\
+		if (rcuidle)						\
+			rcu_irq_exit_irqson();				\
 	} while (0)
 
 #ifndef MODULE
kernel/tracepoint.c +9 -16
index 8d8e41c5d8a5..68b4e50798b1 100644
@@ -18,9 +18,6 @@
 extern tracepoint_ptr_t __start___tracepoints_ptrs[];
 extern tracepoint_ptr_t __stop___tracepoints_ptrs[];
 
-DEFINE_SRCU(tracepoint_srcu);
-EXPORT_SYMBOL_GPL(tracepoint_srcu);
-
 /* Set to 1 to enable tracepoint debug output */
 static const int tracepoint_debug;
 
@@ -65,14 +62,9 @@ static void rcu_tasks_trace_free_old_probes(struct rcu_head *head)
 	kfree(container_of(head, struct tp_probes, rcu));
 }
 
-static void srcu_free_old_probes(struct rcu_head *head)
-{
-	call_rcu_tasks_trace(head, rcu_tasks_trace_free_old_probes);
-}
-
 static void rcu_free_old_probes(struct rcu_head *head)
 {
-	call_srcu(&tracepoint_srcu, head, srcu_free_old_probes);
+	call_rcu_tasks_trace(head, rcu_tasks_trace_free_old_probes);
 }
 
 static __init int release_early_probes(void)
@@ -90,7 +82,7 @@ static __init int release_early_probes(void)
 	return 0;
 }
 
-/* SRCU and Tasks Trace RCU are initialized at core_initcall */
+/* Tasks Trace RCU is initialized at core_initcall */
 postcore_initcall(release_early_probes);
 
 static inline void release_probes(struct tracepoint_func *old)
@@ -100,9 +92,8 @@ static inline void release_probes(struct tracepoint_func *old)
 			struct tp_probes, probes[0]);
 
 		/*
-		 * We can't free probes if SRCU and Tasks Trace RCU are not
-		 * initialized yet. Postpone the freeing till after both are
-		 * initialized.
+		 * We can't free probes if Tasks Trace RCU is not initialized yet.
+		 * Postpone the freeing till after Tasks Trace RCU is initialized.
 		 */
 		if (unlikely(!ok_to_free_tracepoints)) {
 			tp_probes->rcu.next = early_probes;
@@ -111,9 +102,11 @@ static inline void release_probes(struct tracepoint_func *old)
 		}
 
 		/*
-		 * Tracepoint probes are protected by sched RCU, SRCU and
-		 * Tasks Trace RCU by chaining the callbacks we cover all three
-		 * cases and wait for all three grace periods.
+		 * Tracepoint probes are protected by both sched RCU and
+		 * Tasks Trace RCU, by calling the Tasks Trace RCU callback in
+		 * the sched RCU callback we cover both cases. So let us chain
+		 * the Tasks Trace RCU and sched RCU callbacks to wait for both
+		 * grace periods.
 		 */
 		call_rcu(&tp_probes->rcu, rcu_free_old_probes);
 	}
Signature · 1 line
2.25.1
#13 lore raw

Re: [RFC PATCH 6/6] tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

Quoted 12 lines · Joel Fernandes
Quoted 8 lines · Michael Jeanson

From: Mathieu Desnoyers <[email protected]>

Considering that tracer callbacks expect RCU to be watching (for instance, perf uses rcu_read_lock), we need rcuidle tracepoints to issue rcu_irq_{enter,exit}_irqson around calls to the callbacks. So there is no point in using SRCU anymore given that rcuidle tracepoints need to ensure RCU is watching. Therefore, simply use sched-RCU like normal tracepoints for rcuidle tracepoints.

High level question:

Quoted 7 lines

IIRC, doing this increases overhead for general tracing that does not use perf, for 'rcuidle' tracepoints such as the preempt/irq enable/disable tracepoints. I remember adding SRCU because of this reason.

Can the 'rcuidle' information not be pushed down further, such that perf does it because it requires RCU to be watching, so that it does not effect, say, trace events?

There's very few trace_.*_rcuidle() users left. We should eradicate them and remove the option. It's bugs to begin with.

#14 lore raw

Re: [RFC PATCH 6/6] tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

Quoted 17 lines · ----- On Oct 26, 2020, at 4:20 AM, Peter Zijlstra [email protected] wrote:
Quoted 16 lines · Joel Fernandes
Quoted 8 lines · Michael Jeanson

From: Mathieu Desnoyers <[email protected]>

Considering that tracer callbacks expect RCU to be watching (for instance, perf uses rcu_read_lock), we need rcuidle tracepoints to issue rcu_irq_{enter,exit}_irqson around calls to the callbacks. So there is no point in using SRCU anymore given that rcuidle tracepoints need to ensure RCU is watching. Therefore, simply use sched-RCU like normal tracepoints for rcuidle tracepoints.

High level question:

IIRC, doing this increases overhead for general tracing that does not use perf, for 'rcuidle' tracepoints such as the preempt/irq enable/disable tracepoints. I remember adding SRCU because of this reason.

Quoted 6 lines
Quoted 3 lines

Can the 'rcuidle' information not be pushed down further, such that perf does it because it requires RCU to be watching, so that it does not effect, say, trace events?

There's very few trace_.*_rcuidle() users left. We should eradicate them and remove the option. It's bugs to begin with.

I agree with Peter. Removing the trace_.*_rcuidle weirdness from the tracepoint API and fixing all callers to ensure they trace from a context where RCU is watching would simplify instrumentation of the Linux kernel, thus making it harder for subtle bugs to hide and be unearthed only when tracing is enabled. This is AFAIU the general approach Thomas Gleixner has been aiming for recently, and I think it is a good thing.

So if we consider this our target, and that the current state of things is that we need to have RCU watching around callback invocation, then removing the dependency on SRCU seems like an overall simplification which does not regress feature-wise nor speed-wise compared with what we have upstream today. The next steps would then be to audit all rcuidle tracepoints and make sure the context where they are placed has RCU watching already, so we can remove the tracepoint rcuidle API. That would effectively remove the calls to rcu_irq_{enter,exit}_irqson from the tracepoint code.

This is however beyond the scope of the proposed patch set.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
maintainer
#15 lore raw

Re: [RFC PATCH 6/6] tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

Quoted 4 lines · On Mon, 26 Oct 2020 10:28:07 -0400 (EDT) Mathieu Desnoyers…

I agree with Peter. Removing the trace_.*_rcuidle weirdness from the tracepoint API and fixing all callers to ensure they trace from a context where RCU is watching would simplify instrumentation of the Linux kernel, thus making it harder for subtle bugs to hide and be unearthed only when tracing is enabled. This is

Note, the lockdep RCU checking of a tracepoint is outside of it being enabled or disable. So if a non rcuidle() tracepoint is in a location that RCU is not watching, it will complain loudly, even if you don't enable that tracepoint.

Quoted 9 lines

AFAIU the general approach Thomas Gleixner has been aiming for recently, and I think it is a good thing.

So if we consider this our target, and that the current state of things is that we need to have RCU watching around callback invocation, then removing the dependency on SRCU seems like an overall simplification which does not regress feature-wise nor speed-wise compared with what we have upstream today. The next steps would then be to audit all rcuidle tracepoints and make sure the context where they are placed has RCU watching already, so we can remove the tracepoint

Just remove the _rcuidle() from them, and lockdep will complain if they are being called without RCU watching.

-- Steve

Quoted 9 lines

rcuidle API. That would effectively remove the calls to rcu_irq_{enter,exit}_irqson from the tracepoint code.

This is however beyond the scope of the proposed patch set.

Thanks,

Mathieu

#16 lore raw

Re: [RFC PATCH 6/6] tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

Quoted 24 lines · ----- On Oct 26, 2020, at 4:44 PM, rostedt [email protected] wrote:
Quoted 5 lines · On Mon, 26 Oct 2020 10:28:07 -0400 (EDT) Mathieu Desnoyers…

I agree with Peter. Removing the trace_.*_rcuidle weirdness from the tracepoint API and fixing all callers to ensure they trace from a context where RCU is watching would simplify instrumentation of the Linux kernel, thus making it harder
for subtle bugs to hide and be unearthed only when tracing is enabled. This is

Note, the lockdep RCU checking of a tracepoint is outside of it being enabled or disable. So if a non rcuidle() tracepoint is in a location that RCU is not watching, it will complain loudly, even if you don't enable that tracepoint.

Quoted 9 lines

AFAIU the general approach Thomas Gleixner has been aiming for recently, and I think it is a good thing.

So if we consider this our target, and that the current state of things is that we need to have RCU watching around callback invocation, then removing the dependency on SRCU seems like an overall simplification which does not regress feature-wise nor speed-wise compared with what we have upstream today. The next steps would then be to audit all rcuidle tracepoints and make sure the context where they are placed has RCU watching already, so we can remove the tracepoint

Quoted 2 lines

Just remove the _rcuidle() from them, and lockdep will complain if they are being called without RCU watching.

That's the easy part. The patch removing rcuidle is attached to this email, and here are the splats I get just when booting the machine (see below). The part which takes more time is fixing those splats and figuring out how to restructure the code. For instance, what should we do about the rcuidle tracepoint within printk() ?

Also, how do we test rcuidle tracepoints triggered only when printk is called from printk warnings ? We'd also need to test on arm32 boards, because some arm architecture code uses rcuidle tracepoints as well.

As this is beyond the scope of this patch set, I can either drop this patch entirely (it's not required for sleepable tracepoints), or keep it as an intermediate cleanup step. Removing rcuidle tracepoints entirely is beyond the effort Michael and I can undertake now.

=============================
WARNING: suspicious RCU usage

5.9.1+ #72 Not tainted
-----------------------------
=============================
./include/trace/events/preemptirq.h:42 suspicious rcu_dereference_check() usage!
WARNING: suspicious RCU usage

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 1
5.9.1+ #72 Not tainted
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.
-----------------------------
./include/trace/events/preemptirq.h:38 suspicious rcu_dereference_check() usage!

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.9.1+ #72

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 1

Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
RCU used illegally from extended quiescent state!
no locks held by swapper/1/0.
 dump_stack+0x8d/0xbb

stack backtrace:
 ? default_idle+0xa/0x20
 trace_hardirqs_on+0xed/0xf0
 default_idle+0xa/0x20
 default_idle_call+0x4f/0x1e0
 do_idle+0x1ef/0x2c0
 cpu_startup_entry+0x19/0x20
 start_kernel+0x578/0x59d
 secondary_startup_64+0xa4/0xb0
CPU: 1 PID: 0 Comm: swapper/1 Not tainted 5.9.1+ #72

=============================
WARNING: suspicious RCU usage
5.9.1+ #72 Not tainted
-----------------------------
./include/trace/events/lock.h:37 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 1
RCU used illegally from extended quiescent state!
no locks held by swapper/0/0.

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.9.1+ #72
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 dump_stack+0x8d/0xbb
 lock_acquire+0x346/0x3b0
 ? __lock_acquire+0x2e7/0x1da0
 _raw_spin_lock+0x2c/0x40
 ? vprintk_emit+0x9f/0x410
 vprintk_emit+0x9f/0x410
 printk+0x52/0x6e
 lockdep_rcu_suspicious+0x1b/0xe0
 ? default_idle+0xa/0x20
 trace_hardirqs_on+0xed/0xf0
 default_idle+0xa/0x20
 default_idle_call+0x4f/0x1e0
 do_idle+0x1ef/0x2c0
 cpu_startup_entry+0x19/0x20
 start_kernel+0x578/0x59d
 secondary_startup_64+0xa4/0xb0

=============================
WARNING: suspicious RCU usage
5.9.1+ #72 Not tainted
-----------------------------
./include/trace/events/lock.h:63 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 1
RCU used illegally from extended quiescent state!
1 lock held by swapper/0/0:
#0: ffffffff97a80158 (logbuf_lock){-...}-{2:2}, at: vprintk_emit+0x9f/0x410

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.9.1+ #72
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 dump_stack+0x8d/0xbb
 lock_release+0x25a/0x280
 _raw_spin_unlock+0x17/0x30
 vprintk_emit+0xdf/0x410
 printk+0x52/0x6e
 lockdep_rcu_suspicious+0x1b/0xe0
 ? default_idle+0xa/0x20
 trace_hardirqs_on+0xed/0xf0
 default_idle+0xa/0x20
 default_idle_call+0x4f/0x1e0
 do_idle+0x1ef/0x2c0
 cpu_startup_entry+0x19/0x20
 start_kernel+0x578/0x59d
 secondary_startup_64+0xa4/0xb0

=============================
WARNING: suspicious RCU usage
5.9.1+ #72 Not tainted
-----------------------------
./include/trace/events/printk.h:33 suspicious rcu_dereference_check() usage!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 1
RCU used illegally from extended quiescent state!
2 locks held by swapper/0/0:
#0: ffffffff97a801a0 (console_lock){+.+.}-{0:0}, at: vprintk_emit+0x126/0x410 #1: ffffffff97a7fec0 (console_owner){....}-{0:0}, at: console_unlock+0x190/0x5d0

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.9.1+ #72
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 dump_stack+0x8d/0xbb
 console_unlock+0x5ad/0x5d0
 vprintk_emit+0x133/0x410
 printk+0x52/0x6e
 lockdep_rcu_suspicious+0x1b/0xe0
 ? default_idle+0xa/0x20
 trace_hardirqs_on+0xed/0xf0
 default_idle+0xa/0x20
 default_idle_call+0x4f/0x1e0
 do_idle+0x1ef/0x2c0
 cpu_startup_entry+0x19/0x20
 start_kernel+0x578/0x59d
 secondary_startup_64+0xa4/0xb0

=============================
WARNING: suspicious RCU usage
5.9.1+ #72 Not tainted
-----------------------------
./include/linux/rcupdate.h:636 rcu_read_lock() used illegally while idle!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 1
RCU used illegally from extended quiescent state!
4 locks held by swapper/0/0:
#0: ffffffff97a801a0 (console_lock){+.+.}-{0:0}, at: vprintk_emit+0x126/0x410 #1: ffffffff97a7fec0 (console_owner){....}-{0:0}, at: console_unlock+0x190/0x5d0 #2: ffffffff97b7d598 (printing_lock){....}-{2:2}, at: vt_console_print+0x7d/0x3e0 #3: ffffffff97a82d80 (rcu_read_lock){....}-{1:2}, at: __atomic_notifier_call_chain+0x5/0x110

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.9.1+ #72
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 dump_stack+0x8d/0xbb
 __atomic_notifier_call_chain+0xd7/0x110
 vt_console_print+0x19e/0x3e0
 console_unlock+0x417/0x5d0
 vprintk_emit+0x133/0x410
 printk+0x52/0x6e
 lockdep_rcu_suspicious+0x1b/0xe0
 ? default_idle+0xa/0x20
 trace_hardirqs_on+0xed/0xf0
 default_idle+0xa/0x20
 default_idle_call+0x4f/0x1e0
 do_idle+0x1ef/0x2c0
 cpu_startup_entry+0x19/0x20
 start_kernel+0x578/0x59d
 secondary_startup_64+0xa4/0xb0

=============================
WARNING: suspicious RCU usage
5.9.1+ #72 Not tainted
-----------------------------
./include/linux/rcupdate.h:685 rcu_read_unlock() used illegally while idle!

other info that might help us debug this:

rcu_scheduler_active = 1, debug_locks = 1
RCU used illegally from extended quiescent state!
4 locks held by swapper/0/0:
#0: ffffffff97a801a0 (console_lock){+.+.}-{0:0}, at: vprintk_emit+0x126/0x410 #1: ffffffff97a7fec0 (console_owner){....}-{0:0}, at: console_unlock+0x190/0x5d0 #2: ffffffff97b7d598 (printing_lock){....}-{2:2}, at: vt_console_print+0x7d/0x3e0 #3: ffffffff97a82d80 (rcu_read_lock){....}-{1:2}, at: __atomic_notifier_call_chain+0x5/0x110

stack backtrace:
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 5.9.1+ #72
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 dump_stack+0x8d/0xbb
 __atomic_notifier_call_chain+0x101/0x110
 vt_console_print+0x19e/0x3e0
 console_unlock+0x417/0x5d0
 vprintk_emit+0x133/0x410
 printk+0x52/0x6e
 lockdep_rcu_suspicious+0x1b/0xe0
 ? default_idle+0xa/0x20
 trace_hardirqs_on+0xed/0xf0
 default_idle+0xa/0x20
 default_idle_call+0x4f/0x1e0
 do_idle+0x1ef/0x2c0
 cpu_startup_entry+0x19/0x20
 start_kernel+0x578/0x59d
 secondary_startup_64+0xa4/0xb0
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 dump_stack+0x8d/0xbb
 ? rcu_idle_exit+0x32/0x40
 trace_hardirqs_off+0xe3/0xf0
 rcu_idle_exit+0x32/0x40
 default_idle_call+0x54/0x1e0
 do_idle+0x1ef/0x2c0
 ? lockdep_hardirqs_on_prepare+0xec/0x180
 cpu_startup_entry+0x19/0x20
 start_secondary+0x11c/0x160
 secondary_startup_64+0xa4/0xb0

Thanks,

Mathieu

Quoted 13 lines

-- Steve

Quoted 9 lines

rcuidle API. That would effectively remove the calls to rcu_irq_{enter,exit}_irqson
from the tracepoint code.

This is however beyond the scope of the proposed patch set.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

From 7a16029eb56921a42e4c16323e0dee128d724c1e Mon Sep 17 00:00:00 2001
From: Mathieu Desnoyers <[email protected]>
Date: Tue, 27 Oct 2020 09:28:46 -0400
Subject: [WIP PATCH 1/1] wip: remove rcuidle tracepoint API

---
 arch/arm/kernel/smp.c             |  6 +++---
 arch/arm/mach-omap2/powerdomain.c | 10 +++++-----
 arch/arm64/kernel/smp.c           |  4 ++--
 drivers/base/power/runtime.c      | 24 ++++++++++++------------
 drivers/clk/clk.c                 |  8 ++++----
 drivers/soc/qcom/rpmh-rsc.c       |  2 +-
 include/linux/tracepoint.h        | 26 ++------------------------
 include/trace/events/preemptirq.h |  8 --------
 kernel/printk/printk.c            |  2 +-
 kernel/trace/trace_preemptirq.c   | 18 ++++++------------
 scripts/tags.sh                   |  2 --
 11 files changed, 36 insertions(+), 74 deletions(-)
arch/arm/kernel/smp.c +3 -3
index 5d9da61eff62..4997a965a38a 100644
@@ -521,7 +521,7 @@ static const char *ipi_types[NR_IPI] __tracepoint_string = {
 
 static void smp_cross_call(const struct cpumask *target, unsigned int ipinr)
 {
-	trace_ipi_raise_rcuidle(target, ipi_types[ipinr]);
+	trace_ipi_raise(target, ipi_types[ipinr]);
 	__smp_cross_call(target, ipinr);
 }
 
@@ -633,7 +633,7 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
 	struct pt_regs *old_regs = set_irq_regs(regs);
 
 	if ((unsigned)ipinr < NR_IPI) {
-		trace_ipi_entry_rcuidle(ipi_types[ipinr]);
+		trace_ipi_entry(ipi_types[ipinr]);
 		__inc_irq_stat(cpu, ipi_irqs[ipinr]);
 	}
 
@@ -694,7 +694,7 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
 	}
 
 	if ((unsigned)ipinr < NR_IPI)
-		trace_ipi_exit_rcuidle(ipi_types[ipinr]);
+		trace_ipi_exit(ipi_types[ipinr]);
 	set_irq_regs(old_regs);
 }
 
arch/arm/mach-omap2/powerdomain.c +5 -5
index 1cbac76136d4..4093efafb168 100644
@@ -187,9 +187,9 @@ static int _pwrdm_state_switch(struct powerdomain *pwrdm, int flag)
 			trace_state = (PWRDM_TRACE_STATES_FLAG |
 				       ((next & OMAP_POWERSTATE_MASK) << 8) |
 				       ((prev & OMAP_POWERSTATE_MASK) << 0));
-			trace_power_domain_target_rcuidle(pwrdm->name,
-							  trace_state,
-							  raw_smp_processor_id());
+			trace_power_domain_target(pwrdm->name,
+						  trace_state,
+						  raw_smp_processor_id());
 		}
 		break;
 	default:
@@ -541,8 +541,8 @@ int pwrdm_set_next_pwrst(struct powerdomain *pwrdm, u8 pwrst)
 
 	if (arch_pwrdm && arch_pwrdm->pwrdm_set_next_pwrst) {
 		/* Trace the pwrdm desired target state */
-		trace_power_domain_target_rcuidle(pwrdm->name, pwrst,
-						  raw_smp_processor_id());
+		trace_power_domain_target(pwrdm->name, pwrst,
+					  raw_smp_processor_id());
 		/* Program the pwrdm desired target state */
 		ret = arch_pwrdm->pwrdm_set_next_pwrst(pwrdm, pwrst);
 	}
arch/arm64/kernel/smp.c +2 -2
index 355ee9eed4dd..dd0415fd9fa8 100644
@@ -896,7 +896,7 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
 	struct pt_regs *old_regs = set_irq_regs(regs);
 
 	if ((unsigned)ipinr < NR_IPI) {
-		trace_ipi_entry_rcuidle(ipi_types[ipinr]);
+		trace_ipi_entry(ipi_types[ipinr]);
 		__inc_irq_stat(cpu, ipi_irqs[ipinr]);
 	}
 
@@ -956,7 +956,7 @@ void handle_IPI(int ipinr, struct pt_regs *regs)
 	}
 
 	if ((unsigned)ipinr < NR_IPI)
-		trace_ipi_exit_rcuidle(ipi_types[ipinr]);
+		trace_ipi_exit(ipi_types[ipinr]);
 	set_irq_regs(old_regs);
 }
 
drivers/base/power/runtime.c +12 -12
index 8143210a5c54..418b3e373231 100644
@@ -401,7 +401,7 @@ static int rpm_idle(struct device *dev, int rpmflags)
 	int (*callback)(struct device *);
 	int retval;
 
-	trace_rpm_idle_rcuidle(dev, rpmflags);
+	trace_rpm_idle(dev, rpmflags);
 	retval = rpm_check_suspend_allowed(dev);
 	if (retval < 0)
 		;	/* Conditions are wrong. */
@@ -437,7 +437,7 @@ static int rpm_idle(struct device *dev, int rpmflags)
 			dev->power.request_pending = true;
 			queue_work(pm_wq, &dev->power.work);
 		}
-		trace_rpm_return_int_rcuidle(dev, _THIS_IP_, 0);
+		trace_rpm_return_int(dev, _THIS_IP_, 0);
 		return 0;
 	}
 
@@ -452,7 +452,7 @@ static int rpm_idle(struct device *dev, int rpmflags)
 	wake_up_all(&dev->power.wait_queue);
 
  out:
-	trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
+	trace_rpm_return_int(dev, _THIS_IP_, retval);
 	return retval ? retval : rpm_suspend(dev, rpmflags | RPM_AUTO);
 }
 
@@ -519,7 +519,7 @@ static int rpm_suspend(struct device *dev, int rpmflags)
 	struct device *parent = NULL;
 	int retval;
 
-	trace_rpm_suspend_rcuidle(dev, rpmflags);
+	trace_rpm_suspend(dev, rpmflags);
 
  repeat:
 	retval = rpm_check_suspend_allowed(dev);
@@ -657,7 +657,7 @@ static int rpm_suspend(struct device *dev, int rpmflags)
 	}
 
  out:
-	trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
+	trace_rpm_return_int(dev, _THIS_IP_, retval);
 
 	return retval;
 
@@ -709,7 +709,7 @@ static int rpm_resume(struct device *dev, int rpmflags)
 	struct device *parent = NULL;
 	int retval = 0;
 
-	trace_rpm_resume_rcuidle(dev, rpmflags);
+	trace_rpm_resume(dev, rpmflags);
 
  repeat:
 	if (dev->power.runtime_error)
@@ -872,7 +872,7 @@ static int rpm_resume(struct device *dev, int rpmflags)
 		spin_lock_irq(&dev->power.lock);
 	}
 
-	trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
+	trace_rpm_return_int(dev, _THIS_IP_, retval);
 
 	return retval;
 }
@@ -1005,7 +1005,7 @@ int __pm_runtime_idle(struct device *dev, int rpmflags)
 
 	if (rpmflags & RPM_GET_PUT) {
 		if (!atomic_dec_and_test(&dev->power.usage_count)) {
-			trace_rpm_usage_rcuidle(dev, rpmflags);
+			trace_rpm_usage(dev, rpmflags);
 			return 0;
 		}
 	}
@@ -1039,7 +1039,7 @@ int __pm_runtime_suspend(struct device *dev, int rpmflags)
 
 	if (rpmflags & RPM_GET_PUT) {
 		if (!atomic_dec_and_test(&dev->power.usage_count)) {
-			trace_rpm_usage_rcuidle(dev, rpmflags);
+			trace_rpm_usage(dev, rpmflags);
 			return 0;
 		}
 	}
@@ -1122,7 +1122,7 @@ int pm_runtime_get_if_active(struct device *dev, bool ign_usage_count)
 	} else {
 		retval = atomic_inc_not_zero(&dev->power.usage_count);
 	}
-	trace_rpm_usage_rcuidle(dev, 0);
+	trace_rpm_usage(dev, 0);
 	spin_unlock_irqrestore(&dev->power.lock, flags);
 
 	return retval;
@@ -1458,7 +1458,7 @@ void pm_runtime_allow(struct device *dev)
 	if (atomic_dec_and_test(&dev->power.usage_count))
 		rpm_idle(dev, RPM_AUTO | RPM_ASYNC);
 	else
-		trace_rpm_usage_rcuidle(dev, RPM_AUTO | RPM_ASYNC);
+		trace_rpm_usage(dev, RPM_AUTO | RPM_ASYNC);
 
  out:
 	spin_unlock_irq(&dev->power.lock);
@@ -1527,7 +1527,7 @@ static void update_autosuspend(struct device *dev, int old_delay, int old_use)
 			atomic_inc(&dev->power.usage_count);
 			rpm_resume(dev, 0);
 		} else {
-			trace_rpm_usage_rcuidle(dev, 0);
+			trace_rpm_usage(dev, 0);
 		}
 	}
 
drivers/clk/clk.c +4 -4
index 0a9261a099bd..1c0996323171 100644
@@ -959,12 +959,12 @@ static void clk_core_disable(struct clk_core *core)
 	if (--core->enable_count > 0)
 		return;
 
-	trace_clk_disable_rcuidle(core);
+	trace_clk_disable(core);
 
 	if (core->ops->disable)
 		core->ops->disable(core->hw);
 
-	trace_clk_disable_complete_rcuidle(core);
+	trace_clk_disable_complete(core);
 
 	clk_core_disable(core->parent);
 }
@@ -1018,12 +1018,12 @@ static int clk_core_enable(struct clk_core *core)
 		if (ret)
 			return ret;
 
-		trace_clk_enable_rcuidle(core);
+		trace_clk_enable(core);
 
 		if (core->ops->enable)
 			ret = core->ops->enable(core->hw);
 
-		trace_clk_enable_complete_rcuidle(core);
+		trace_clk_enable_complete(core);
 
 		if (ret) {
 			clk_core_disable(core->parent);
drivers/soc/qcom/rpmh-rsc.c +1 -1
index ae6675782581..fabe390dce19 100644
@@ -495,7 +495,7 @@ static void __tcs_buffer_write(struct rsc_drv *drv, int tcs_id, int cmd_id,
 		write_tcs_cmd(drv, RSC_DRV_CMD_MSGID, tcs_id, j, msgid);
 		write_tcs_cmd(drv, RSC_DRV_CMD_ADDR, tcs_id, j, cmd->addr);
 		write_tcs_cmd(drv, RSC_DRV_CMD_DATA, tcs_id, j, cmd->data);
-		trace_rpmh_send_msg_rcuidle(drv, tcs_id, j, msgid, cmd);
+		trace_rpmh_send_msg(drv, tcs_id, j, msgid, cmd);
 	}
 
 	write_tcs_reg(drv, RSC_DRV_CMD_WAIT_FOR_CMPL, tcs_id, cmd_complete);
include/linux/tracepoint.h +2 -24
index 1414b11f864b..ee513e696799 100644
@@ -160,7 +160,7 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
  * has a "void" prototype, then it is invalid to declare a function
  * as "(void *, void)".
  */
-#define __DO_TRACE(tp, proto, args, cond, rcuidle, tp_flags)		\
+#define __DO_TRACE(tp, proto, args, cond, tp_flags)			\
 	do {								\
 		struct tracepoint_func *it_func_ptr;			\
 		void *it_func;						\
@@ -172,8 +172,6 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 			return;						\
 									\
 		might_sleep_if(maysleep);				\
-		if (rcuidle)						\
-			rcu_irq_enter_irqson();				\
 		if (maysleep)						\
 			rcu_read_lock_trace();				\
 		else							\
@@ -193,24 +191,8 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 			rcu_read_unlock_trace();			\
 		else							\
 			preempt_enable_notrace();			\
-		if (rcuidle)						\
-			rcu_irq_exit_irqson();				\
 	} while (0)
 
-#ifndef MODULE
-#define __DECLARE_TRACE_RCU(name, proto, args, cond, data_proto, data_args, tp_flags) \
-	static inline void trace_##name##_rcuidle(proto)		\
-	{								\
-		if (static_key_false(&__tracepoint_##name.key))		\
-			__DO_TRACE(&__tracepoint_##name,		\
-				TP_PROTO(data_proto),			\
-				TP_ARGS(data_args),			\
-				TP_CONDITION(cond), 1, tp_flags);	\
-	}
-#else
-#define __DECLARE_TRACE_RCU(name, proto, args, cond, data_proto, data_args, tp_flags)
-#endif
-
 /*
  * Make sure the alignment of the structure in the __tracepoints section will
  * not add unwanted padding between the beginning of the section and the
@@ -231,15 +213,13 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 			__DO_TRACE(&__tracepoint_##name,		\
 				TP_PROTO(data_proto),			\
 				TP_ARGS(data_args),			\
-				TP_CONDITION(cond), 0, tp_flags);	\
+				TP_CONDITION(cond), tp_flags);		\
 		if (IS_ENABLED(CONFIG_LOCKDEP) && (cond)) {		\
 			rcu_read_lock_sched_notrace();			\
 			rcu_dereference_sched(__tracepoint_##name.funcs);\
 			rcu_read_unlock_sched_notrace();		\
 		}							\
 	}								\
-	__DECLARE_TRACE_RCU(name, PARAMS(proto), PARAMS(args),		\
-		PARAMS(cond), PARAMS(data_proto), PARAMS(data_args), tp_flags)	\
 	static inline int						\
 	register_trace_##name(void (*probe)(data_proto), void *data)	\
 	{								\
@@ -310,8 +290,6 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 #define __DECLARE_TRACE(name, proto, args, cond, data_proto, data_args, tp_flags) \
 	static inline void trace_##name(proto)				\
 	{ }								\
-	static inline void trace_##name##_rcuidle(proto)		\
-	{ }								\
 	static inline int						\
 	register_trace_##name(void (*probe)(data_proto),		\
 			      void *data)				\
include/trace/events/preemptirq.h +0 -8
index 3f249e150c0c..f99562d2b496 100644
@@ -43,8 +43,6 @@ DEFINE_EVENT(preemptirq_template, irq_enable,
 #else
 #define trace_irq_enable(...)
 #define trace_irq_disable(...)
-#define trace_irq_enable_rcuidle(...)
-#define trace_irq_disable_rcuidle(...)
 #endif
 
 #ifdef CONFIG_TRACE_PREEMPT_TOGGLE
@@ -58,8 +56,6 @@ DEFINE_EVENT(preemptirq_template, preempt_enable,
 #else
 #define trace_preempt_enable(...)
 #define trace_preempt_disable(...)
-#define trace_preempt_enable_rcuidle(...)
-#define trace_preempt_disable_rcuidle(...)
 #endif
 
 #endif /* _TRACE_PREEMPTIRQ_H */
@@ -69,10 +65,6 @@ DEFINE_EVENT(preemptirq_template, preempt_enable,
 #else /* !CONFIG_PREEMPTIRQ_TRACEPOINTS */
 #define trace_irq_enable(...)
 #define trace_irq_disable(...)
-#define trace_irq_enable_rcuidle(...)
-#define trace_irq_disable_rcuidle(...)
 #define trace_preempt_enable(...)
 #define trace_preempt_disable(...)
-#define trace_preempt_enable_rcuidle(...)
-#define trace_preempt_disable_rcuidle(...)
 #endif
kernel/printk/printk.c +1 -1
index 9b75f6bfc333..ec641a77ad52 100644
@@ -1806,7 +1806,7 @@ static void call_console_drivers(const char *ext_text, size_t ext_len,
 {
 	struct console *con;
 
-	trace_console_rcuidle(text, len);
+	trace_console(text, len);
 
 	for_each_console(con) {
 		if (exclusive_console && con != exclusive_console)
kernel/trace/trace_preemptirq.c +6 -12
index f4938040c228..b89bd133c8fc 100644
@@ -40,8 +40,7 @@ NOKPROBE_SYMBOL(trace_hardirqs_on_prepare);
 void trace_hardirqs_on(void)
 {
 	if (this_cpu_read(tracing_irq_cpu)) {
-		if (!in_nmi())
-			trace_irq_enable_rcuidle(CALLER_ADDR0, CALLER_ADDR1);
+		trace_irq_enable(CALLER_ADDR0, CALLER_ADDR1);
 		tracer_hardirqs_on(CALLER_ADDR0, CALLER_ADDR1);
 		this_cpu_write(tracing_irq_cpu, 0);
 	}
@@ -78,8 +77,7 @@ void trace_hardirqs_off(void)
 	if (!this_cpu_read(tracing_irq_cpu)) {
 		this_cpu_write(tracing_irq_cpu, 1);
 		tracer_hardirqs_off(CALLER_ADDR0, CALLER_ADDR1);
-		if (!in_nmi())
-			trace_irq_disable_rcuidle(CALLER_ADDR0, CALLER_ADDR1);
+		trace_irq_disable(CALLER_ADDR0, CALLER_ADDR1);
 	}
 }
 EXPORT_SYMBOL(trace_hardirqs_off);
@@ -88,8 +86,7 @@ NOKPROBE_SYMBOL(trace_hardirqs_off);
 __visible void trace_hardirqs_on_caller(unsigned long caller_addr)
 {
 	if (this_cpu_read(tracing_irq_cpu)) {
-		if (!in_nmi())
-			trace_irq_enable_rcuidle(CALLER_ADDR0, caller_addr);
+		trace_irq_enable(CALLER_ADDR0, caller_addr);
 		tracer_hardirqs_on(CALLER_ADDR0, caller_addr);
 		this_cpu_write(tracing_irq_cpu, 0);
 	}
@@ -107,8 +104,7 @@ __visible void trace_hardirqs_off_caller(unsigned long caller_addr)
 	if (!this_cpu_read(tracing_irq_cpu)) {
 		this_cpu_write(tracing_irq_cpu, 1);
 		tracer_hardirqs_off(CALLER_ADDR0, caller_addr);
-		if (!in_nmi())
-			trace_irq_disable_rcuidle(CALLER_ADDR0, caller_addr);
+		trace_irq_disable(CALLER_ADDR0, caller_addr);
 	}
 }
 EXPORT_SYMBOL(trace_hardirqs_off_caller);
@@ -119,15 +115,13 @@ NOKPROBE_SYMBOL(trace_hardirqs_off_caller);
 
 void trace_preempt_on(unsigned long a0, unsigned long a1)
 {
-	if (!in_nmi())
-		trace_preempt_enable_rcuidle(a0, a1);
+	trace_preempt_enable(a0, a1);
 	tracer_preempt_on(a0, a1);
 }
 
 void trace_preempt_off(unsigned long a0, unsigned long a1)
 {
-	if (!in_nmi())
-		trace_preempt_disable_rcuidle(a0, a1);
+	trace_preempt_disable(a0, a1);
 	tracer_preempt_off(a0, a1);
 }
 #endif
scripts/tags.sh +0 -2
index 850f4ccb6afc..3057fbe9eba0 100755
@@ -147,9 +147,7 @@ regex_c=(
 	'/^BPF_CALL_[0-9](\([[:alnum:]_]*\).*/\1/'
 	'/^COMPAT_SYSCALL_DEFINE[0-9](\([[:alnum:]_]*\).*/compat_sys_\1/'
 	'/^TRACE_EVENT(\([[:alnum:]_]*\).*/trace_\1/'
-	'/^TRACE_EVENT(\([[:alnum:]_]*\).*/trace_\1_rcuidle/'
 	'/^DEFINE_EVENT([^,)]*, *\([[:alnum:]_]*\).*/trace_\1/'
-	'/^DEFINE_EVENT([^,)]*, *\([[:alnum:]_]*\).*/trace_\1_rcuidle/'
 	'/^DEFINE_INSN_CACHE_OPS(\([[:alnum:]_]*\).*/get_\1_slot/'
 	'/^DEFINE_INSN_CACHE_OPS(\([[:alnum:]_]*\).*/free_\1_slot/'
 	'/^PAGEFLAG(\([[:alnum:]_]*\).*/Page\1/'
Signature · 1 line
2.17.1
#17 lore raw

Re: [RFC PATCH 6/6] tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

In reply to Mathieu Desnoyers
Quoted 42 lines · Mathieu Desnoyers
Quoted 17 lines · ----- On Oct 26, 2020, at 4:20 AM, Peter Zijlstra [email protected] wrote:
Quoted 16 lines · Joel Fernandes
Quoted 8 lines · Michael Jeanson

From: Mathieu Desnoyers <[email protected]>

Considering that tracer callbacks expect RCU to be watching (for instance, perf uses rcu_read_lock), we need rcuidle tracepoints to issue rcu_irq_{enter,exit}_irqson around calls to the callbacks. So there is no point in using SRCU anymore given that rcuidle tracepoints need to ensure RCU is watching. Therefore, simply use sched-RCU like normal tracepoints for rcuidle tracepoints.

High level question:

IIRC, doing this increases overhead for general tracing that does not use perf, for 'rcuidle' tracepoints such as the preempt/irq enable/disable tracepoints. I remember adding SRCU because of this reason.

Quoted 6 lines
Quoted 3 lines

Can the 'rcuidle' information not be pushed down further, such that perf does it because it requires RCU to be watching, so that it does not effect, say, trace events?

There's very few trace_.*_rcuidle() users left. We should eradicate them and remove the option. It's bugs to begin with.

I agree with Peter. Removing the trace_.*_rcuidle weirdness from the tracepoint API and fixing all callers to ensure they trace from a context where RCU is watching would simplify instrumentation of the Linux kernel, thus making it harder for subtle bugs to hide and be unearthed only when tracing is enabled. This is AFAIU the general approach Thomas Gleixner has been aiming for recently, and I think it is a good thing.

So if we consider this our target, and that the current state of things is that we need to have RCU watching around callback invocation, then removing the dependency on SRCU seems like an overall simplification which does not regress feature-wise nor speed-wise compared with what we have upstream today. The next steps would then be to audit all rcuidle tracepoints and make sure the context where they are placed has RCU watching already, so we can remove the tracepoint rcuidle API. That would effectively remove the calls to rcu_irq_{enter,exit}_irqson from the tracepoint code.

Quoted 1 line

This is however beyond the scope of the proposed patch set.

You are right, it doesn't regress speedwise - I got confused since the code was modified to call rcu_enter_irqson() even for the rcuidle case (which I had avoided when I added SRCU). So in current code, SRCU is kind of pointless. I think keep the patch in the series.

thanks,

- Joel

#18 lore raw
In reply to Michael Jeanson
Quoted 10 lines · Michael Jeanson

When invoked from system call enter/exit instrumentation, accessing user-space data is a common use-case for tracers. However, tracepoints currently disable preemption around iteration on the registered tracepoint probes and invocation of the probe callbacks, which prevents tracers from handling page faults.

Extend the tracepoint and trace event APIs to allow specific tracer probes to take page faults. Adapt ftrace, perf, and ebpf to allow being called from sleepable context, and convert the system call enter/exit instrumentation to sleepable tracepoints.

Will this not be a problem for analyse of the trace? It get two relevant times, one it when it is called and one when it returns.

It makes things harder to correlate in what order things happen.

And handling of tracing of contexts that already are not preamptable?

Eg the same tracepoint are used in different places and contexts.

Quoted 45 lines

This series only implements the tracepoint infrastructure required to allow tracers to handle page faults. Modifying each tracer to handle those page faults would be a next step after we all agree on this piece of instrumentation infrastructure.

This patchset is base on v5.9.1.

Cc Steven Rostedt (VMware) <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Alexei Starovoitov <[email protected]>
Cc Yonghong Song <[email protected]>
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Mark Rutland <[email protected]>
Cc Alexander Shishkin <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Joel Fernandes (Google) <[email protected]>

Mathieu Desnoyers (1):
tracing: use sched-RCU instead of SRCU for rcuidle tracepoints

Michael Jeanson (5):
tracing: introduce sleepable tracepoints
tracing: ftrace: add support for sleepable tracepoints
tracing: bpf-trace: add support for sleepable tracepoints
tracing: perf: add support for sleepable tracepoints
tracing: convert sys_enter/exit to sleepable tracepoints

 include/linux/tracepoint-defs.h |  11 ++++
 include/linux/tracepoint.h      | 104 +++++++++++++++++++++-----------
 include/trace/bpf_probe.h       |  23 ++++++-
 include/trace/define_trace.h    |   7 +++
 include/trace/events/syscalls.h |   4 +-
 include/trace/perf.h            |  26 ++++++--
 include/trace/trace_events.h    |  79 ++++++++++++++++++++++--
 init/Kconfig                    |   1 +
 kernel/trace/bpf_trace.c        |   5 +-
 kernel/trace/trace_events.c     |  15 ++++-
 kernel/trace/trace_syscalls.c   |  68 +++++++++++++--------
 kernel/tracepoint.c             | 104 +++++++++++++++++++++++++-------
 12 files changed, 351 insertions(+), 96 deletions(-)
#19 lore raw
Quoted 7 lines · ----- On Oct 26, 2020, at 8:05 AM, peter enderborg [email protected]…
Quoted 6 lines · Michael Jeanson

When invoked from system call enter/exit instrumentation, accessing user-space data is a common use-case for tracers. However, tracepoints currently disable preemption around iteration on the registered tracepoint probes and invocation of the probe callbacks, which prevents tracers from handling page faults.

Quoted 7 lines
Quoted 4 lines

Extend the tracepoint and trace event APIs to allow specific tracer probes to take page faults. Adapt ftrace, perf, and ebpf to allow being called from sleepable context, and convert the system call enter/exit instrumentation to sleepable tracepoints.

Will this not be a problem for analyse of the trace? It get two relevant times, one it when it is called and one when it returns.

It will depend on what the tracer chooses to do. If we call the side-effect of what is being traced a "transaction" (e.g. actually opening a file descriptor and adding it to a process'file descriptor table as the result of an open(2) system call), we have to consider that already today the timestamp which we get is either slightly before or after the actual side-effect of the transaction in the kernel. That is true even without being preemptable.

Sometimes it's not relevant to have a tracepoint before and after the transaction, e.g. when all we care about is to know that the transaction has successfully happened or not.

In the case of system calls, we have sys_enter and sys_exit to mark the beginning and end of the "transaction". Whatever side-effects are done by the system call happens in between.

I think the question here is whether it is relevant to know whether page faults triggered by accessing system call input parameters need to happen after we trace a "system call entry" event. If the tracers care, then it would be up to them to first trace that "system call entry" event, and have a separate event for the argument payload. But there are other ways to identify whether page faults happen within the system call or from user-space, for instance by using the instruction pointer associated with the page fault. So when observing page faults happening before sys enter, but associated with a kernel instruction pointer, a trace analysis tool could theoretically figure out who is to blame for that page fault, *if* it cares.

Quoted 2 lines

It makes things harder to correlate in what order things happen.

The alternative is to have partial payloads like LTTng does today for system call arguments. If reading a string from userspace (e.g. open(2) file name) requires to take a page fault, LTTng truncates the string. This is pretty bad for automated analysis as well.

Quoted 2 lines

And handling of tracing of contexts that already are not preamptable?

The sleepable tracepoints are only meant to be used in contexts which can sleep. For tracepoints placed in non-preemptible contexts, those should never take a page fault to begin with.

Quoted 2 lines

Eg the same tracepoint are used in different places and contexts.

As far as considering that a given tracepoint "name" could be registered to by both a sleepable and non-sleepable tracer probes, I would like to see an actual use-case for this. I don't have any.

I can envision that some tracer code will want to be allowed to work in both sleepable and non-sleepable context, e.g. take page faults in sleepable context (and called from a sleepable tracepoint), but have a truncation behavior when called from non-sleepable context. This can actually be done by looking at the new "TRACEPOINT_MAYSLEEP" tp flag. Passing that tp_flags to code shared between sleepable and non-sleepable probes would allow the callee to know whether it can take a page fault or not.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com