Skip to contentSkip to conversation

tracepoint: rcuidle: use rcu_is_watching() and tree-rcu

PATCH In review Mathieu Desnoyers 2 participants 6 messages Mar 10, 2020 – Mar 11, 2020 Tracing

lore
Share & tools
lore

Mathieu Desnoyers

author reviewer
#0 lore raw

PATCH 1/1 tracepoint: rcuidle: use rcu_is_watching() and tree-rcu

commit e6753f23d961 ("tracepoint: Make rcuidle tracepoint callers use SRCU") aimed at improving performance of rcuidle tracepoints by using SRCU rather than temporarily enabling tree-rcu every time.

commit 865e63b04e9b ("tracing: Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints") adds back the high-overhead enabling of tree-rcu because perf expects RCU to be watching when called from rcuidle tracepoints.

It turns out that by using "rcu_is_watching()" and conditionally calling the high-overhead rcu_irq_enter/exit_irqson(), the original motivation for using SRCU in the first place disappears.

I suspect that the original benchmarks justifying the introduction of SRCU to handle rcuidle tracepoints was caused by preempt/irq tracepoints, which are typically invoked from contexts that have RCU watching.

CC Thomas Gleixner <[email protected]>
CC Steven Rostedt <[email protected]>
CC Joel Fernandes <[email protected]>
CC "Paul E. McKenney" <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Frederic Weisbecker <[email protected]>
CC Ingo Molnar <[email protected]>
---
 include/linux/tracepoint.h | 25 ++++++++++---------------
 1 file changed, 10 insertions(+), 15 deletions(-)
include/linux/tracepoint.h +10 -15
index 1fb11daa5c53..8e0e94fee29a 100644
@@ -165,25 +165,22 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		void *it_func;						\
 		void *__data;						\
 		int __maybe_unused __idx = 0;				\
+		bool __exit_rcu = false;				\
 									\
 		if (!(cond))						\
 			return;						\
 									\
-		/* srcu can't be used from NMI */			\
-		WARN_ON_ONCE(rcuidle && in_nmi());			\
-									\
-		/* keep srcu and sched-rcu usage consistent */		\
-		preempt_disable_notrace();				\
-									\
 		/*							\
-		 * For rcuidle callers, use srcu since sched-rcu	\
-		 * doesn't work from the idle path.			\
+		 * For rcuidle callers, temporarily enable RCU if	\
+		 * it is not currently watching.			\
 		 */							\
-		if (rcuidle) {						\
-			__idx = srcu_read_lock_notrace(&tracepoint_srcu);\
+		if (rcuidle && !rcu_is_watching()) {			\
 			rcu_irq_enter_irqson();				\
+			__exit_rcu = true;				\
 		}							\
 									\
+		preempt_disable_notrace();				\
+									\
 		it_func_ptr = rcu_dereference_raw((tp)->funcs);		\
 									\
 		if (it_func_ptr) {					\
@@ -194,12 +191,10 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 			} while ((++it_func_ptr)->func);		\
 		}							\
 									\
-		if (rcuidle) {						\
-			rcu_irq_exit_irqson();				\
-			srcu_read_unlock_notrace(&tracepoint_srcu, __idx);\
-		}							\
-									\
 		preempt_enable_notrace();				\
+									\
+		if (__exit_rcu)						\
+			rcu_irq_exit_irqson();				\
 	} while (0)
 
 #ifndef MODULE
Signature · 1 line
2.17.1
#1 lore raw
Quoted 4 lines · Mathieu Desnoyers

commit e6753f23d961 ("tracepoint: Make rcuidle tracepoint callers use SRCU") aimed at improving performance of rcuidle tracepoints by using SRCU rather than temporarily enabling tree-rcu every time.

Quoted 8 lines

commit 865e63b04e9b ("tracing: Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints") adds back the high-overhead enabling of tree-rcu because perf expects RCU to be watching when called from rcuidle tracepoints.

It turns out that by using "rcu_is_watching()" and conditionally calling the high-overhead rcu_irq_enter/exit_irqson(), the original motivation for using SRCU in the first place disappears.

Adding Alexei on CC for his thoughts, given that these were his benchmarks. I believe that he also has additional use cases.

But given the use cases you describe, this seems plausible. This does mean that tracepoints cannot be attached to the CPU-hotplug code that runs on the incoming/outgoing CPU early/late in that process, though that might be OK.

Thanx, Paul

Quoted 73 lines

I suspect that the original benchmarks justifying the introduction of SRCU to handle rcuidle tracepoints was caused by preempt/irq tracepoints, which are typically invoked from contexts that have RCU watching.

CC Thomas Gleixner <[email protected]>
CC Steven Rostedt <[email protected]>
CC Joel Fernandes <[email protected]>
CC "Paul E. McKenney" <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Frederic Weisbecker <[email protected]>
CC Ingo Molnar <[email protected]>
---
 include/linux/tracepoint.h | 25 ++++++++++---------------
 1 file changed, 10 insertions(+), 15 deletions(-)
include/linux/tracepoint.h +10 -15
index 1fb11daa5c53..8e0e94fee29a 100644
@@ -165,25 +165,22 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 		void *it_func;						\
 		void *__data;						\
 		int __maybe_unused __idx = 0;				\
+		bool __exit_rcu = false;				\
 									\
 		if (!(cond))						\
 			return;						\
 									\
-		/* srcu can't be used from NMI */			\
-		WARN_ON_ONCE(rcuidle && in_nmi());			\
-									\
-		/* keep srcu and sched-rcu usage consistent */		\
-		preempt_disable_notrace();				\
-									\
 		/*							\
-		 * For rcuidle callers, use srcu since sched-rcu	\
-		 * doesn't work from the idle path.			\
+		 * For rcuidle callers, temporarily enable RCU if	\
+		 * it is not currently watching.			\
 		 */							\
-		if (rcuidle) {						\
-			__idx = srcu_read_lock_notrace(&tracepoint_srcu);\
+		if (rcuidle && !rcu_is_watching()) {			\
 			rcu_irq_enter_irqson();				\
+			__exit_rcu = true;				\
 		}							\
 									\
+		preempt_disable_notrace();				\
+									\
 		it_func_ptr = rcu_dereference_raw((tp)->funcs);		\
 									\
 		if (it_func_ptr) {					\
@@ -194,12 +191,10 @@ static inline struct tracepoint *tracepoint_ptr_deref(tracepoint_ptr_t *p)
 			} while ((++it_func_ptr)->func);		\
 		}							\
 									\
-		if (rcuidle) {						\
-			rcu_irq_exit_irqson();				\
-			srcu_read_unlock_notrace(&tracepoint_srcu, __idx);\
-		}							\
-									\
 		preempt_enable_notrace();				\
+									\
+		if (__exit_rcu)						\
+			rcu_irq_exit_irqson();				\
 	} while (0)
 
 #ifndef MODULE
Signature · 1 line
2.17.1
author reviewer
#2 lore raw
Quoted 10 lines · ----- On Mar 10, 2020, at 4:53 PM, paulmck [email protected] wrote:
Quoted 9 lines · Mathieu Desnoyers

commit e6753f23d961 ("tracepoint: Make rcuidle tracepoint callers use SRCU") aimed at improving performance of rcuidle tracepoints by using SRCU rather than temporarily enabling tree-rcu every time.

commit 865e63b04e9b ("tracing: Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints") adds back the high-overhead enabling of tree-rcu because perf expects RCU to be watching when called from rcuidle tracepoints.

Quoted 6 lines
Quoted 3 lines

It turns out that by using "rcu_is_watching()" and conditionally calling the high-overhead rcu_irq_enter/exit_irqson(), the original motivation for using SRCU in the first place disappears.

Adding Alexei on CC for his thoughts, given that these were his benchmarks. I believe that he also has additional use cases.

Good point! Sorry I forgot to add Alexei to my CC list for that patch.

Quoted 4 lines

But given the use cases you describe, this seems plausible. This does mean that tracepoints cannot be attached to the CPU-hotplug code that runs on the incoming/outgoing CPU early/late in that process, though that might be OK.

Do you mean standard tracepoints or rcuidle tracepoints ?

Are there any such tracepoints early/late in the cpu hotplug code today ?

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#3 lore raw
Quoted 22 lines · Mathieu Desnoyers
Quoted 10 lines · ----- On Mar 10, 2020, at 4:53 PM, paulmck [email protected] wrote:
Quoted 9 lines · Mathieu Desnoyers

commit e6753f23d961 ("tracepoint: Make rcuidle tracepoint callers use SRCU") aimed at improving performance of rcuidle tracepoints by using SRCU rather than temporarily enabling tree-rcu every time.

commit 865e63b04e9b ("tracing: Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints") adds back the high-overhead enabling of tree-rcu because perf expects RCU to be watching when called from rcuidle tracepoints.

Quoted 6 lines
Quoted 3 lines

It turns out that by using "rcu_is_watching()" and conditionally calling the high-overhead rcu_irq_enter/exit_irqson(), the original motivation for using SRCU in the first place disappears.

Adding Alexei on CC for his thoughts, given that these were his benchmarks. I believe that he also has additional use cases.

Good point! Sorry I forgot to add Alexei to my CC list for that patch.

Quoted 8 lines
Quoted 4 lines

But given the use cases you describe, this seems plausible. This does mean that tracepoints cannot be attached to the CPU-hotplug code that runs on the incoming/outgoing CPU early/late in that process, though that might be OK.

Do you mean standard tracepoints or rcuidle tracepoints ?

Are there any such tracepoints early/late in the cpu hotplug code today ?

I have no idea. You would know better than I. However, I would expect that the same common-code issue that applies to exception-entry/exit code might also apply to the CPU hotplug code.

Thanx, Paul

author reviewer
#4 lore raw
Quoted 28 lines · ----- On Mar 10, 2020, at 7:52 PM, paulmck [email protected] wrote:
Quoted 27 lines · Mathieu Desnoyers
Quoted 10 lines · ----- On Mar 10, 2020, at 4:53 PM, paulmck [email protected] wrote:
Quoted 9 lines · Mathieu Desnoyers

commit e6753f23d961 ("tracepoint: Make rcuidle tracepoint callers use SRCU") aimed at improving performance of rcuidle tracepoints by using SRCU rather than temporarily enabling tree-rcu every time.

commit 865e63b04e9b ("tracing: Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints") adds back the high-overhead enabling of tree-rcu because perf expects RCU to be watching when called from rcuidle tracepoints.

Quoted 6 lines
Quoted 3 lines

It turns out that by using "rcu_is_watching()" and conditionally calling the high-overhead rcu_irq_enter/exit_irqson(), the original motivation for using SRCU in the first place disappears.

Adding Alexei on CC for his thoughts, given that these were his benchmarks. I believe that he also has additional use cases.

Good point! Sorry I forgot to add Alexei to my CC list for that patch.

Quoted 4 lines

But given the use cases you describe, this seems plausible. This does mean that tracepoints cannot be attached to the CPU-hotplug code that runs on the incoming/outgoing CPU early/late in that process, though that might be OK.

Quoted 7 lines
Quoted 3 lines

Do you mean standard tracepoints or rcuidle tracepoints ?

Are there any such tracepoints early/late in the cpu hotplug code today ?

I have no idea. You would know better than I. However, I would expect that the same common-code issue that applies to exception-entry/exit code might also apply to the CPU hotplug code.

I would also expect early/late CPU hotplug states to fall into the same category of "low-level entry/exit" code which Thomas would like to hide from instrumentation.

Thanks,

Mathieu

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#5 lore raw
Quoted 34 lines · Mathieu Desnoyers
Quoted 32 lines · ----- On Mar 10, 2020, at 7:52 PM, paulmck [email protected] wrote:
Quoted 30 lines · Mathieu Desnoyers
Quoted 10 lines · ----- On Mar 10, 2020, at 4:53 PM, paulmck [email protected] wrote:
Quoted 9 lines · Mathieu Desnoyers

commit e6753f23d961 ("tracepoint: Make rcuidle tracepoint callers use SRCU") aimed at improving performance of rcuidle tracepoints by using SRCU rather than temporarily enabling tree-rcu every time.

commit 865e63b04e9b ("tracing: Add back in rcu_irq_enter/exit_irqson() for rcuidle tracepoints") adds back the high-overhead enabling of tree-rcu because perf expects RCU to be watching when called from rcuidle tracepoints.

Quoted 6 lines
Quoted 3 lines

It turns out that by using "rcu_is_watching()" and conditionally calling the high-overhead rcu_irq_enter/exit_irqson(), the original motivation for using SRCU in the first place disappears.

Adding Alexei on CC for his thoughts, given that these were his benchmarks. I believe that he also has additional use cases.

Good point! Sorry I forgot to add Alexei to my CC list for that patch.

Quoted 4 lines

But given the use cases you describe, this seems plausible. This does mean that tracepoints cannot be attached to the CPU-hotplug code that runs on the incoming/outgoing CPU early/late in that process, though that might be OK.

Do you mean standard tracepoints or rcuidle tracepoints ?

Are there any such tracepoints early/late in the cpu hotplug code today ?

Quoted 7 lines
Quoted 3 lines

I have no idea. You would know better than I. However, I would expect that the same common-code issue that applies to exception-entry/exit code might also apply to the CPU hotplug code.

I would also expect early/late CPU hotplug states to fall into the same category of "low-level entry/exit" code which Thomas would like to hide from instrumentation.

That decision I must leave to you guys.

Thanx, Paul