Skip to contentSkip to conversation

perf: Per-pmu-per-cpu contexts

tip:perf/core tip-bot for Peter Zijlstra 4 participants 6 messages Sep 9, 2010 – Sep 10, 2010 perf events

lore
Share & tools
lore
#0 lore raw
Commit-ID:  108b02cfce04ee90b0a07ee0b104baffd39f5934
Gitweb:     http://git.kernel.org/tip/108b02cfce04ee90b0a07ee0b104baffd39f5934
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Mon, 6 Sep 2010 14:32:03 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 9 Sep 2010 20:46:32 +0200

perf: Per-pmu-per-cpu contexts

Allocate per-cpu contexts per pmu.

Cc paulus <[email protected]>
Cc stephane eranian <[email protected]>
Cc Robert Richter <[email protected]>
Cc Frederic Weisbecker <[email protected]>
Cc Lin Ming <[email protected]>
Cc Yanmin <[email protected]>

LKML-Reference: <new-submission>

---
 include/linux/perf_event.h |    4 +-
 kernel/perf_event.c        |  178 +++++++++++++++++++++++++++-----------------
 2 files changed, 113 insertions(+), 69 deletions(-)
include/linux/perf_event.h +3 -1
index fa04537..22155ef 100644
@@ -570,7 +570,8 @@ struct perf_event;
 struct pmu {
 	struct list_head		entry;
 
-	int				*pmu_disable_count;
+	int * __percpu			pmu_disable_count;
+	struct perf_cpu_context * __percpu pmu_cpu_context;
 
 	/*
 	 * Fully disable/enable this PMU, can be used to protect from the PMI
@@ -808,6 +809,7 @@ struct perf_event {
  * Used as a container for task events and CPU events as well:
  */
 struct perf_event_context {
+	struct pmu			*pmu;
 	/*
 	 * Protect the states of the events in the list,
 	 * nr_active, and the list:
kernel/perf_event.c +110 -68

449 lines not on this page · raw message

#1 lore raw
Quoted 9 lines · tip-bot for Peter Zijlstra
kernel/perf_event.c +2 -1
@@ -3745,18 +3757,20 @@ static void perf_event_task_ctx(struct perf_event_context *ctx,
 
 static void perf_event_task_event(struct perf_task_event *task_event)
 {
-	struct perf_cpu_context *cpuctx;
 	struct perf_event_context *ctx = task_event->task_ctx;
+	struct perf_cpu_context *cpuctx;
+	struct pmu *pmu;
 

kernel/perf_event.c static void perf_event_task_ctx(struct perf_event_context *ctx, · line 3770

Quoted 10 lines
kernel/perf_event.c +5 -3
-	rcu_read_lock();
-	cpuctx = &get_cpu_var(perf_cpu_context);
-	perf_event_task_ctx(&cpuctx->ctx, task_event);
+	rcu_read_lock_sched();
+	list_for_each_entry_rcu(pmu, &pmus, entry) {
+		cpuctx = this_cpu_ptr(pmu->pmu_cpu_context);
+		perf_event_task_ctx(&cpuctx->ctx, task_event);
+	}
 	if (!ctx)
 		ctx = rcu_dereference(current->perf_event_ctxp);

So, you say below that it works because synchronize_srcu(), that waits for qs after touching pmus, implies synchronize_sched(), right?

And I guess you picked rcu_read_lock_sched() here because that preempt_disable() at the same time.

That looks complicated but I guess that works.

That said there is also this rcu_dereference(current->perf_event_ctxp). Now, this ctx is released after srcu barrier right? So this should be srcu_dereference(). But then you seem to actually use rcu_read_lock_sched() as it's compatible, so this should be rcu_dereference_sched() ?

With the current state, rcu will whine.
Moreover there seem to be too much game between the different rcu flavours here, and that breaks the reviewers parsing.

#2 lore raw
Quoted 10 lines · Frederic Weisbecker
Quoted 9 lines · tip-bot for Peter Zijlstra
kernel/perf_event.c +2 -1
@@ -3745,18 +3757,20 @@ static void perf_event_task_ctx(struct perf_event_context *ctx,
 
 static void perf_event_task_event(struct perf_task_event *task_event)
 {
-	struct perf_cpu_context *cpuctx;
 	struct perf_event_context *ctx = task_event->task_ctx;
+	struct perf_cpu_context *cpuctx;
+	struct pmu *pmu;
 
Quoted 15 lines

kernel/perf_event.c static void perf_event_task_ctx(struct perf_event_context *ctx, · line 3770

Quoted 10 lines
kernel/perf_event.c +5 -3
-	rcu_read_lock();
-	cpuctx = &get_cpu_var(perf_cpu_context);
-	perf_event_task_ctx(&cpuctx->ctx, task_event);
+	rcu_read_lock_sched();
+	list_for_each_entry_rcu(pmu, &pmus, entry) {
+		cpuctx = this_cpu_ptr(pmu->pmu_cpu_context);
+		perf_event_task_ctx(&cpuctx->ctx, task_event);
+	}
 	if (!ctx)
 		ctx = rcu_dereference(current->perf_event_ctxp);

So, you say below that it works because synchronize_srcu(), that waits for qs after touching pmus, implies synchronize_sched(), right?

Ook... My current plans to fold SRCU into TREE_RCU would invalidate this assumption.

Maybe we need some sort of primitive that concurrently waits for multiple types of RCU grace periods?

Thanx, Paul

Quoted 14 lines

And I guess you picked rcu_read_lock_sched() here because that preempt_disable() at the same time.

That looks complicated but I guess that works.

That said there is also this rcu_dereference(current->perf_event_ctxp). Now, this ctx is released after srcu barrier right? So this should be srcu_dereference(). But then you seem to actually use rcu_read_lock_sched() as it's compatible, so this should be rcu_dereference_sched() ?

With the current state, rcu will whine.
Moreover there seem to be too much game between the different rcu flavours here, and that breaks the reviewers parsing.

maintainer
#3 lore raw
Quoted 8 lines · Paul E. McKenney
Quoted 2 lines

So, you say below that it works because synchronize_srcu(), that waits for qs after touching pmus, implies synchronize_sched(), right?

Ook... My current plans to fold SRCU into TREE_RCU would invalidate this assumption.

Maybe we need some sort of primitive that concurrently waits for multiple types of RCU grace periods?

Nah, but I was thinking that any kind of preemptible rcu sync would imply a sched rcu sync.

If not strictly implied I'd have no problem simply writing:

  synchronize_rcu_sched();
  synchronize_srcu();
maintainer
#5 lore raw
In reply to Frederic Weisbecker
Quoted 10 lines · Frederic Weisbecker
Quoted 9 lines · tip-bot for Peter Zijlstra
kernel/perf_event.c +2 -1
@@ -3745,18 +3757,20 @@ static void perf_event_task_ctx(struct perf_event_context *ctx,
 
 static void perf_event_task_event(struct perf_task_event *task_event)
 {
-	struct perf_cpu_context *cpuctx;
 	struct perf_event_context *ctx = task_event->task_ctx;
+	struct perf_cpu_context *cpuctx;
+	struct pmu *pmu;
 
Quoted 15 lines

kernel/perf_event.c static void perf_event_task_ctx(struct perf_event_context *ctx, · line 3770

Quoted 10 lines
kernel/perf_event.c +5 -3
-	rcu_read_lock();
-	cpuctx = &get_cpu_var(perf_cpu_context);
-	perf_event_task_ctx(&cpuctx->ctx, task_event);
+	rcu_read_lock_sched();
+	list_for_each_entry_rcu(pmu, &pmus, entry) {
+		cpuctx = this_cpu_ptr(pmu->pmu_cpu_context);
+		perf_event_task_ctx(&cpuctx->ctx, task_event);
+	}
 	if (!ctx)
 		ctx = rcu_dereference(current->perf_event_ctxp);

So, you say below that it works because synchronize_srcu(), that waits for qs after touching pmus, implies synchronize_sched(), right?

yep.

Quoted 2 lines

And I guess you picked rcu_read_lock_sched() here because that preempt_disable() at the same time.

Mostly because preemption is already disabled there, and sync_srcu() was implemented using sync_rcu_sched() primitives.

Quoted 1 line

That looks complicated but I guess that works.

Yeah, similar to the event lists which we protect with a mutex and a spinlock, hold either to traverse, hold both to modify.

Depending on the situation we need to traverse the pmu list preemptible -- for example when we need to take the above mentioned mutex (see perf_event_exit_cpu_context), or non-preemptible, the above. So we need to guard it using two different flavours of RCU too.

Quoted 4 lines

That said there is also this rcu_dereference(current->perf_event_ctxp). Now, this ctx is released after srcu barrier right? So this should be srcu_dereference(). But then you seem to actually use rcu_read_lock_sched() as it's compatible, so this should be rcu_dereference_sched() ?

The task context is released using call_rcu(), and should be done under rcu_read_lock(), I guess we should hold both rcu_read_lock() and rcu_read_lock_sched() there to be correct.

Quoted 3 lines

With the current state, rcu will whine.
Moreover there seem to be too much game between the different rcu flavours here, and that breaks the reviewers parsing.

:-)

#4 lore raw
In reply to Peter Zijlstra
Quoted 4 lines · Peter Zijlstra
Quoted 3 lines · Paul E. McKenney
Quoted 2 lines

So, you say below that it works because synchronize_srcu(), that waits for qs after touching pmus, implies synchronize_sched(), right?

Quoted 8 lines
Quoted 5 lines

Ook... My current plans to fold SRCU into TREE_RCU would invalidate this assumption.

Maybe we need some sort of primitive that concurrently waits for multiple types of RCU grace periods?

Nah, but I was thinking that any kind of preemptible rcu sync would imply a sched rcu sync.

Ah!

Although disabling interrupts will block preemptible RCU grace periods in current implementations (but please don't rely on this!), disabling preemption will -not- block preemptible RCU grace periods, even given current TREE_PREEMPT_RCU and TINY_PREEMPT_RCU implementations. Current SRCU grace periods are blocked by disabling preemption, but folding it into the tree/tiny implementations would make SRCU grace periods be no longer blocked by disabling preemption.

This might change if RCU priority boosting is enabled, due to RCU grace-period computation and callback invocation moving to a kthread, but I won't have the guts to make TREE_RCU use kthread by default for some time. (Probably a year or so trouble-free experience with RCU priority boosting/kthreads.)

Quoted 4 lines

If not strictly implied I'd have no problem simply writing:

  synchronize_rcu_sched();
  synchronize_srcu();

If that works for you, then we are set! The only reason to introduce a combined primitive would be if the latency of the above was too large.

Thanx, Paul