Skip to contentSkip to conversation

ftrace: Remove use of control list and ops

PATCH In review Steven Rostedt 2 participants 14 messages Nov 30, 2015 – Dec 18, 2015

Subsystems (2)
lore
Share & tools
lore

Steven Rostedt

author maintainer
#0 lore raw

PATCH 1/1 ftrace: Remove use of control list and ops

[ Jiri, can you take a look at this. You can also apply it on top of my
branch ftrace/core, and run any specific tests. I just need to nuke that control structure for further updates with ftrace. ]

Currently perf has its own list function within the ftrace infrastructure that seems to be used only to allow for it to have per-cpu disabling as well as a check to make sure that it's not called while RCU is not watching. It uses something called the "control_ops" which is used to iterate over ops under it with the control_list_func().

The problem is that this control_ops and control_list_func unnecessarily complicates the code. By replacing FTRACE_OPS_FL_CONTROL with two new flags (FTRACE_OPS_FL_RCU and FTRACE_OPS_FL_PER_CPU) we can remove all the code that is special with the control ops and add the needed checks within the generic ftrace_list_func().

---
 include/linux/ftrace.h          |  35 +++++------
 kernel/trace/ftrace.c           | 126 ++++++++++++----------------------------
 kernel/trace/trace.h            |   2 -
 kernel/trace/trace_event_perf.c |   2 +-
 4 files changed, 57 insertions(+), 108 deletions(-)
include/linux/ftrace.h +18 -17
index 134f8d45b35b..4736a826baf5 100644
@@ -76,8 +76,8 @@ ftrace_func_t ftrace_ops_get_func(struct ftrace_ops *ops);
  * ENABLED - set/unset when ftrace_ops is registered/unregistered
  * DYNAMIC - set when ftrace_ops is registered to denote dynamically
  *           allocated ftrace_ops which need special care
- * CONTROL - set manualy by ftrace_ops user to denote the ftrace_ops
- *           could be controled by following calls:
+ * PER_CPU - set manualy by ftrace_ops user to denote the ftrace_ops
+ *           could be controlled by following calls:
  *             ftrace_function_local_enable
  *             ftrace_function_local_disable
  * SAVE_REGS - The ftrace_ops wants regs saved at each function called
@@ -121,7 +121,7 @@ ftrace_func_t ftrace_ops_get_func(struct ftrace_ops *ops);
 enum {
 	FTRACE_OPS_FL_ENABLED			= 1 << 0,
 	FTRACE_OPS_FL_DYNAMIC			= 1 << 1,
-	FTRACE_OPS_FL_CONTROL			= 1 << 2,
+	FTRACE_OPS_FL_PER_CPU			= 1 << 2,
 	FTRACE_OPS_FL_SAVE_REGS			= 1 << 3,
 	FTRACE_OPS_FL_SAVE_REGS_IF_SUPPORTED	= 1 << 4,
 	FTRACE_OPS_FL_RECURSION_SAFE		= 1 << 5,
@@ -134,6 +134,7 @@ enum {
 	FTRACE_OPS_FL_ALLOC_TRAMP		= 1 << 12,
 	FTRACE_OPS_FL_IPMODIFY			= 1 << 13,
 	FTRACE_OPS_FL_PID			= 1 << 14,
+	FTRACE_OPS_FL_RCU			= 1 << 15,
 };
 
 #ifdef CONFIG_DYNAMIC_FTRACE
@@ -146,11 +147,11 @@ struct ftrace_ops_hash {
 #endif
 
 /*
- * Note, ftrace_ops can be referenced outside of RCU protection.
- * (Although, for perf, the control ops prevent that). If ftrace_ops is
- * allocated and not part of kernel core data, the unregistering of it will
- * perform a scheduling on all CPUs to make sure that there are no more users.
- * Depending on the load of the system that may take a bit of time.
+ * Note, ftrace_ops can be referenced outside of RCU protection, unless
+ * the RCU flag is set. If ftrace_ops is allocated and not part of kernel
+ * core data, the unregistering of it will perform a scheduling on all CPUs
+ * to make sure that there are no more users. Depending on the load of the
+ * system that may take a bit of time.
  *
  * Any private data added must also take care not to be freed and if private
  * data is added to a ftrace_ops that is in core code, the user of the
@@ -196,34 +197,34 @@ int unregister_ftrace_function(struct ftrace_ops *ops);
 void clear_ftrace_function(void);
 
 /**
- * ftrace_function_local_enable - enable controlled ftrace_ops on current cpu
+ * ftrace_function_local_enable - enable ftrace_ops on current cpu
  *
  * This function enables tracing on current cpu by decreasing
  * the per cpu control variable.
  * It must be called with preemption disabled and only on ftrace_ops
- * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * registered with FTRACE_OPS_FL_PER_CPU. If called without preemption
  * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
  */
 static inline void ftrace_function_local_enable(struct ftrace_ops *ops)
 {
-	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_PER_CPU)))
 		return;
 
 	(*this_cpu_ptr(ops->disabled))--;
 }
 
 /**
- * ftrace_function_local_disable - enable controlled ftrace_ops on current cpu
+ * ftrace_function_local_disable - disable ftrace_ops on current cpu
  *
- * This function enables tracing on current cpu by decreasing
+ * This function disables tracing on current cpu by increasing
  * the per cpu control variable.
  * It must be called with preemption disabled and only on ftrace_ops
- * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * registered with FTRACE_OPS_FL_PER_CPU. If called without preemption
  * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
  */
 static inline void ftrace_function_local_disable(struct ftrace_ops *ops)
 {
-	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL)))
+	if (WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_PER_CPU)))
 		return;
 
 	(*this_cpu_ptr(ops->disabled))++;
@@ -235,12 +236,12 @@ static inline void ftrace_function_local_disable(struct ftrace_ops *ops)
  *
  * This function returns value of ftrace_ops::disabled on current cpu.
  * It must be called with preemption disabled and only on ftrace_ops
- * registered with FTRACE_OPS_FL_CONTROL. If called without preemption
+ * registered with FTRACE_OPS_FL_PER_CPU. If called without preemption
  * disabled, this_cpu_ptr will complain when CONFIG_DEBUG_PREEMPT is enabled.
  */
 static inline int ftrace_function_local_disabled(struct ftrace_ops *ops)
 {
-	WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_CONTROL));
+	WARN_ON_ONCE(!(ops->flags & FTRACE_OPS_FL_PER_CPU));
 	return *this_cpu_ptr(ops->disabled);
 }
 
kernel/trace/ftrace.c +38 -88

233 lines not on this page · raw message

kernel/trace/trace.h +0 -2
index 919d9d07686f..d3980b87bf04 100644
@@ -467,8 +467,6 @@ enum {
 	TRACE_INTERNAL_IRQ_BIT,
 	TRACE_INTERNAL_SIRQ_BIT,
 
-	TRACE_CONTROL_BIT,
-
 	TRACE_BRANCH_BIT,
 /*
  * Abuse of the trace_recursion.
kernel/trace/trace_event_perf.c +1 -1
index abfc903e741e..2649c85cd162 100644
@@ -334,7 +334,7 @@ static int perf_ftrace_function_register(struct perf_event *event)
 {
 	struct ftrace_ops *ops = &event->ftrace_ops;
 
-	ops->flags |= FTRACE_OPS_FL_CONTROL;
+	ops->flags |= FTRACE_OPS_FL_PER_CPU | FTRACE_OPS_FL_RCU;
 	ops->func = perf_ftrace_function_call;
 	return register_ftrace_function(ops);
 }
Signature · 1 line
1.8.3.1
#1 lore raw
Quoted 12 lines · Steven Rostedt

[ Jiri, can you take a look at this. You can also apply it on top of my
branch ftrace/core, and run any specific tests. I just need to nuke that control structure for further updates with ftrace. ]

Currently perf has its own list function within the ftrace infrastructure that seems to be used only to allow for it to have per-cpu disabling as well as a check to make sure that it's not called while RCU is not watching. It uses something called the "control_ops" which is used to iterate over ops under it with the control_list_func().

Quoted 5 lines

The problem is that this control_ops and control_list_func unnecessarily complicates the code. By replacing FTRACE_OPS_FL_CONTROL with two new flags (FTRACE_OPS_FL_RCU and FTRACE_OPS_FL_PER_CPU) we can remove all the code that is special with the control ops and add the needed checks within the generic ftrace_list_func().

hum,
do we need also change for the trampoline, something like below?

I needed attached patch to get the perf ftrace:function event work properly..

thanks,
jirka

---

kernel/trace/ftrace.c +25 -0
index c4d881200d1f..2705ac2f3487 100644
@@ -5179,6 +5179,26 @@ static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
 	trace_clear_recursion(bit);
 }
 
+static void ftrace_ops_per_cpu_func(unsigned long ip, unsigned long parent_ip,
+				   struct ftrace_ops *op, struct pt_regs *regs)
+{
+	int bit;
+
+	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
+	if (bit < 0)
+		return;
+
+	preempt_disable_notrace();
+
+	if ((!(op->flags & FTRACE_OPS_FL_RCU) || rcu_is_watching()) &&
+            (!(op->flags & FTRACE_OPS_FL_PER_CPU) || !ftrace_function_local_disabled(op))) {
+		op->func(ip, parent_ip, op, regs);
+	}
+
+	preempt_enable_notrace();
+	trace_clear_recursion(bit);
+}
+
 /**
  * ftrace_ops_get_func - get the function a trampoline should call
  * @ops: the ops to get the function for
@@ -5192,6 +5212,11 @@ static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
  */
 ftrace_func_t ftrace_ops_get_func(struct ftrace_ops *ops)
 {
+	if (ops->flags & (FTRACE_OPS_FL_PER_CPU|FTRACE_OPS_FL_RCU)) {
+		printk("used per cpu trampoline function\n");
+		return ftrace_ops_per_cpu_func;
+	}
+
 	/*
 	 * If the func handles its own recursion, call it directly.
 	 * Otherwise call the recursion protected function that
author maintainer
#2 lore raw
Quoted 19 lines · On Tue, 1 Dec 2015 14:42:36 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 12 lines · Steven Rostedt

[ Jiri, can you take a look at this. You can also apply it on top of my
branch ftrace/core, and run any specific tests. I just need to nuke that control structure for further updates with ftrace. ]

Currently perf has its own list function within the ftrace infrastructure that seems to be used only to allow for it to have per-cpu disabling as well as a check to make sure that it's not called while RCU is not watching. It uses something called the "control_ops" which is used to iterate over ops under it with the control_list_func().

Quoted 5 lines

The problem is that this control_ops and control_list_func unnecessarily complicates the code. By replacing FTRACE_OPS_FL_CONTROL with two new flags (FTRACE_OPS_FL_RCU and FTRACE_OPS_FL_PER_CPU) we can remove all the code that is special with the control ops and add the needed checks within the generic ftrace_list_func().

Quoted 5 lines

hum,
do we need also change for the trampoline, something like below?

I needed attached patch to get the perf ftrace:function event work properly..

Hmm, I thought that I forced the list function when RCU or PER_CPU was set. Oh wait. I have CONFIG_PREEMPT set, which will change the logic slightly. I'm guessing you have PREEMPT_VOLUNTARY set. I'll try that out.

Quoted 38 lines

thanks,
jirka

---

kernel/trace/ftrace.c +20 -0
index c4d881200d1f..2705ac2f3487 100644
@@ -5179,6 +5179,26 @@ static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
 	trace_clear_recursion(bit);
 }
 
+static void ftrace_ops_per_cpu_func(unsigned long ip, unsigned long parent_ip,
+				   struct ftrace_ops *op, struct pt_regs *regs)
+{
+	int bit;
+
+	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
+	if (bit < 0)
+		return;
+
+	preempt_disable_notrace();
+
+	if ((!(op->flags & FTRACE_OPS_FL_RCU) || rcu_is_watching()) &&
+            (!(op->flags & FTRACE_OPS_FL_PER_CPU) || !ftrace_function_local_disabled(op))) {
+		op->func(ip, parent_ip, op, regs);
+	}
+
+	preempt_enable_notrace();
+	trace_clear_recursion(bit);
+}
+
 /**
  * ftrace_ops_get_func - get the function a trampoline should call
  * @ops: the ops to get the function for

kernel/trace/ftrace.c static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip, · line 5217

Quoted 6 lines
kernel/trace/ftrace.c +3 -0
@@ -5192,6 +5212,11 @@ static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
  */
 ftrace_func_t ftrace_ops_get_func(struct ftrace_ops *ops)
 {
+	if (ops->flags & (FTRACE_OPS_FL_PER_CPU|FTRACE_OPS_FL_RCU)) {
+		printk("used per cpu trampoline function\n");
+		return ftrace_ops_per_cpu_func;

I have a slight different idea on how to handle this.

-- Steve

Quoted 5 lines
kernel/trace/ftrace.c +2 -0
+	}
+
 	/*
 	 * If the func handles its own recursion, call it directly.
 	 * Otherwise call the recursion protected function that
#3 lore raw
Quoted 25 lines · Steven Rostedt
Quoted 22 lines · On Tue, 1 Dec 2015 14:42:36 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 12 lines · Steven Rostedt

[ Jiri, can you take a look at this. You can also apply it on top of my
branch ftrace/core, and run any specific tests. I just need to nuke that control structure for further updates with ftrace. ]

Currently perf has its own list function within the ftrace infrastructure that seems to be used only to allow for it to have per-cpu disabling as well as a check to make sure that it's not called while RCU is not watching. It uses something called the "control_ops" which is used to iterate over ops under it with the control_list_func().

Quoted 5 lines

The problem is that this control_ops and control_list_func unnecessarily complicates the code. By replacing FTRACE_OPS_FL_CONTROL with two new flags (FTRACE_OPS_FL_RCU and FTRACE_OPS_FL_PER_CPU) we can remove all the code that is special with the control ops and add the needed checks within the generic ftrace_list_func().

hum,
do we need also change for the trampoline, something like below?

Quoted 7 lines
Quoted 2 lines

I needed attached patch to get the perf ftrace:function event work properly..

Hmm, I thought that I forced the list function when RCU or PER_CPU was set. Oh wait. I have CONFIG_PREEMPT set, which will change the logic slightly. I'm guessing you have PREEMPT_VOLUNTARY set. I'll try that out.

yep, but the trampoline has separate code path
to set the ops func

jirka

author maintainer
#4 lore raw
Quoted 7 lines · On Tue, 1 Dec 2015 16:57:44 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 4 lines

Hmm, I thought that I forced the list function when RCU or PER_CPU was set. Oh wait. I have CONFIG_PREEMPT set, which will change the logic slightly. I'm guessing you have PREEMPT_VOLUNTARY set. I'll try that out.

yep, but the trampoline has separate code path
to set the ops func

Right, but that only gets called for DYNAMIC ops (which perf is) when CONFIG_PREEMPT is not set. What about this patch instead? (on top of my other one)

It's a lot like yours, but instead of creating a new helper function, I just reuse the recurs_func instead.

-- Steve

kernel/trace/ftrace.c +18 -9
index 8b22e681baf8..05ed87d06bb0 100644
@@ -5230,20 +5230,29 @@ static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip)
 
 /*
  * If there's only one function registered but it does not support
- * recursion, this function will be called by the mcount trampoline.
- * This function will handle recursion protection.
+ * recursion, needs RCU protection and/or requires per cpu handling, then
+ * this function will be called by the mcount trampoline.
  */
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
 
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();
 
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    ftrace_function_local_disabled(op)) {
+		op->func(ip, parent_ip, op, regs);
+	}
+
+	preempt_enable_notrace();
 	trace_clear_recursion(bit);
 }
 
@@ -5261,12 +5270,12 @@ static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
 ftrace_func_t ftrace_ops_get_func(struct ftrace_ops *ops)
 {
 	/*
-	 * If the func handles its own recursion, call it directly.
-	 * Otherwise call the recursion protected function that
-	 * will call the ftrace ops function.
+	 * If the function does not handle recursion, needs to be RCU safe,
+	 * or does per cpu logic, then we need to call the assist handler.
 	 */
-	if (!(ops->flags & FTRACE_OPS_FL_RECURSION_SAFE))
-		return ftrace_ops_recurs_func;
+	if (!(ops->flags & FTRACE_OPS_FL_RECURSION_SAFE) ||
+	    ops->flags & (FTRACE_OPS_FL_RCU | FTRACE_OPS_FL_PER_CPU))
+		return ftrace_ops_assist_func;
 
 	return ops->func;
 }
#5 lore raw
Quoted 39 lines · Steven Rostedt
Quoted 7 lines · On Tue, 1 Dec 2015 16:57:44 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 4 lines

Hmm, I thought that I forced the list function when RCU or PER_CPU was set. Oh wait. I have CONFIG_PREEMPT set, which will change the logic slightly. I'm guessing you have PREEMPT_VOLUNTARY set. I'll try that out.

yep, but the trampoline has separate code path
to set the ops func

Right, but that only gets called for DYNAMIC ops (which perf is) when CONFIG_PREEMPT is not set. What about this patch instead? (on top of my other one)

It's a lot like yours, but instead of creating a new helper function, I just reuse the recurs_func instead.

-- Steve

kernel/trace/ftrace.c +3 -3
index 8b22e681baf8..05ed87d06bb0 100644
@@ -5230,20 +5230,29 @@ static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip)
 
 /*
  * If there's only one function registered but it does not support
- * recursion, this function will be called by the mcount trampoline.
- * This function will handle recursion protection.
+ * recursion, needs RCU protection and/or requires per cpu handling, then
+ * this function will be called by the mcount trampoline.
  */
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;

kernel/trace/ftrace.c static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip) · line 5248

Quoted 10 lines
kernel/trace/ftrace.c +4 -1
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
 
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();

I was wondering about not disabling preemption in the original ftrace_ops_recurs_func function.. thats why I added new one ;-)

I'll test the patch

thanks,
jirka

Quoted 8 lines
kernel/trace/ftrace.c +6 -0
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    ftrace_function_local_disabled(op)) {
+		op->func(ip, parent_ip, op, regs);
+	}
+
+	preempt_enable_notrace();
 	trace_clear_recursion(bit);
author maintainer
#6 lore raw
Quoted 5 lines · On Wed, 2 Dec 2015 08:59:11 +0100 Jiri Olsa <[email protected]> wrote:

kernel/trace/ftrace.c static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip) · line 5248

Quoted 2 lines
kernel/trace/ftrace.c +1 -1
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();  

I was wondering about not disabling preemption in the original ftrace_ops_recurs_func function.. thats why I added new one ;-)

Yeah, I paused on that too. But then I realized that pretty much all callers of function tracing disable preemption too, so it shouldn't cause really any more overhead.

-- Steve

#7 lore raw
In reply to Steven Rostedt

On Tue, Dec 01, 2015 at 11:56:55AM -0500, Steven Rostedt wrote:

SNIP

Quoted 8 lines
(quoted diff) +3 -1
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;

Review

Quoted 10 lines
(quoted diff) +4 -1
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
 
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();
 
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    ftrace_function_local_disabled(op)) {

should be !ftrace_function_local_disabled(op) in here, I passed my test with attached patch

thanks,
jirka

Review

Quoted 6 lines
(quoted diff) +4 -0
+		op->func(ip, parent_ip, op, regs);
+	}
+
+	preempt_enable_notrace();
 	trace_clear_recursion(bit);
 }

SNIP

---

kernel/trace/ftrace.c +2 -2
index 710430fcfd9c..ad0e0e7eaea2 100644
@@ -115,7 +115,7 @@ static struct ftrace_ops *ftrace_ops_list __read_mostly = &ftrace_list_end;
 ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
 static struct ftrace_ops global_ops;
 
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs);
 
 #if ARCH_SUPPORTS_FTRACE_OPS
@@ -5180,7 +5180,7 @@ static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 	preempt_disable_notrace();
 
 	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
-	    ftrace_function_local_disabled(op)) {
+	    !ftrace_function_local_disabled(op)) {
 		op->func(ip, parent_ip, op, regs);
 	}
 
author maintainer
#8 lore raw
Quoted 17 lines · On Wed, 2 Dec 2015 09:58:26 +0100 Jiri Olsa <[email protected]> wrote:

On Tue, Dec 01, 2015 at 11:56:55AM -0500, Steven Rostedt wrote:

SNIP

Quoted 13 lines
(quoted diff) +4 -1
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
Quoted 9 lines

Review

Quoted 5 lines
(quoted diff) +3 -1
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();
 
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    ftrace_function_local_disabled(op)) {  

should be !ftrace_function_local_disabled(op) in here, I passed my test with attached patch

Can you retest with this patch:

-- Steve

kernel/trace/ftrace.c +18 -12
index bc7f4eb6b4b0..e290a30f2d0b 100644
@@ -115,9 +115,6 @@ static struct ftrace_ops *ftrace_ops_list __read_mostly = &ftrace_list_end;
 ftrace_func_t ftrace_trace_function __read_mostly = ftrace_stub;
 static struct ftrace_ops global_ops;
 
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
-				   struct ftrace_ops *op, struct pt_regs *regs);
-
 #if ARCH_SUPPORTS_FTRACE_OPS
 static void ftrace_ops_list_func(unsigned long ip, unsigned long parent_ip,
 				 struct ftrace_ops *op, struct pt_regs *regs);
@@ -5231,20 +5228,29 @@ static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip)
 
 /*
  * If there's only one function registered but it does not support
- * recursion, this function will be called by the mcount trampoline.
- * This function will handle recursion protection.
+ * recursion, needs RCU protection and/or requires per cpu handling, then
+ * this function will be called by the mcount trampoline.
  */
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
 
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();
 
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    !ftrace_function_local_disabled(op)) {
+		op->func(ip, parent_ip, op, regs);
+	}
+
+	preempt_enable_notrace();
 	trace_clear_recursion(bit);
 }
 
@@ -5262,12 +5268,12 @@ static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
 ftrace_func_t ftrace_ops_get_func(struct ftrace_ops *ops)
 {
 	/*
-	 * If the func handles its own recursion, call it directly.
-	 * Otherwise call the recursion protected function that
-	 * will call the ftrace ops function.
+	 * If the function does not handle recursion, needs to be RCU safe,
+	 * or does per cpu logic, then we need to call the assist handler.
 	 */
-	if (!(ops->flags & FTRACE_OPS_FL_RECURSION_SAFE))
-		return ftrace_ops_recurs_func;
+	if (!(ops->flags & FTRACE_OPS_FL_RECURSION_SAFE) ||
+	    ops->flags & (FTRACE_OPS_FL_RCU | FTRACE_OPS_FL_PER_CPU))
+		return ftrace_ops_assist_func;
 
 	return ops->func;
 }
#9 lore raw
Quoted 23 lines · Steven Rostedt
Quoted 20 lines · On Wed, 2 Dec 2015 09:58:26 +0100 Jiri Olsa <[email protected]> wrote:

On Tue, Dec 01, 2015 at 11:56:55AM -0500, Steven Rostedt wrote:

SNIP

Quoted 16 lines
kernel/trace/ftrace.c +5 -2
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
 
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();
Quoted 8 lines
Quoted 6 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    ftrace_function_local_disabled(op)) {  

should be !ftrace_function_local_disabled(op) in here, I passed my test with attached patch

Can you retest with this patch:

sure, but other than the declaration removal it's the same change

jirka

author maintainer
#10 lore raw
Quoted 31 lines · On Wed, 2 Dec 2015 15:50:32 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 30 lines · Steven Rostedt
Quoted 26 lines · On Wed, 2 Dec 2015 09:58:26 +0100 Jiri Olsa <[email protected]> wrote:

On Tue, Dec 01, 2015 at 11:56:55AM -0500, Steven Rostedt wrote:

SNIP

Quoted 8 lines
kernel/trace/ftrace.c +3 -1
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;

kernel/trace/ftrace.c static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip)

Quoted 10 lines
kernel/trace/ftrace.c +4 -1
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
 
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();
 
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    ftrace_function_local_disabled(op)) {    

should be !ftrace_function_local_disabled(op) in here, I passed my test with attached patch

Quoted 4 lines
Quoted 1 line

Can you retest with this patch:

sure, but other than the declaration removal it's the same change

Yep, but I'm paranoid :-) Any change I like to get retested. But if you're fine, can I get a 'Tested-by' from you?

-- Steve

Tested-by
#11 lore raw
Quoted 34 lines · Steven Rostedt
Quoted 31 lines · On Wed, 2 Dec 2015 15:50:32 +0100 Jiri Olsa <[email protected]> wrote:
Quoted 30 lines · Steven Rostedt
Quoted 26 lines · On Wed, 2 Dec 2015 09:58:26 +0100 Jiri Olsa <[email protected]> wrote:

On Tue, Dec 01, 2015 at 11:56:55AM -0500, Steven Rostedt wrote:

SNIP

Quoted 8 lines
kernel/trace/ftrace.c +3 -1
-static void ftrace_ops_recurs_func(unsigned long ip, unsigned long parent_ip,
+static void ftrace_ops_assist_func(unsigned long ip, unsigned long parent_ip,
 				   struct ftrace_ops *op, struct pt_regs *regs)
 {
 	int bit;
 
+	if ((op->flags & FTRACE_OPS_FL_RCU) && !rcu_is_watching())
+		return;

kernel/trace/ftrace.c static void ftrace_ops_no_ops(unsigned long ip, unsigned long parent_ip)

Quoted 10 lines
kernel/trace/ftrace.c +4 -1
+
 	bit = trace_test_and_set_recursion(TRACE_LIST_START, TRACE_LIST_MAX);
 	if (bit < 0)
 		return;
 
-	op->func(ip, parent_ip, op, regs);
+	preempt_disable_notrace();
 
+	if (!(op->flags & FTRACE_OPS_FL_PER_CPU) ||
+	    ftrace_function_local_disabled(op)) {    

should be !ftrace_function_local_disabled(op) in here, I passed my test with attached patch

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Can you retest with this patch:

sure, but other than the declaration removal it's the same change

Yep, but I'm paranoid :-) Any change I like to get retested. But if you're fine, can I get a 'Tested-by' from you?

yep, I re-tested ;-)

thanks,
jirka

#12 lore raw

On Wed, Dec 02, 2015 at 04:23:50PM +0100, Jiri Olsa wrote:

SNIP

Quoted 10 lines
Quoted 10 lines
Quoted 9 lines
Quoted 6 lines
Quoted 4 lines

should be !ftrace_function_local_disabled(op) in here, I passed my test with attached patch

Can you retest with this patch:

sure, but other than the declaration removal it's the same change

Quoted 7 lines
Quoted 2 lines

Yep, but I'm paranoid :-) Any change I like to get retested. But if you're fine, can I get a 'Tested-by' from you?

yep, I re-tested ;-)

hi,
what's the status of this one? I cannot find it out there ;-)

thanks,
jirka

author maintainer
#13 lore raw
Quoted 2 lines · On Fri, 18 Dec 2015 11:18:58 +0100 Jiri Olsa <[email protected]> wrote:

hi,
what's the status of this one? I cannot find it out there ;-)

Yeah, I haven't gotten it through my tests yet. I was testing some upstream bug fixes to push for this -rc, and my test box HD crashed. It's definitely in the queue, but the queue is currently stuck :-/

-- Steve