Optimize cgroup ctx switch and remove cpuctx->unique_pmu

cpuctx->unique_pmu was originally introduced as a way to identify cpuctxs with shared pmus in order to avoid visiting the same cpuctx more than once in a for_each_pmu loop.

cpuctx->unique_pmu == cpuctx->pmu in non-software task contexts since they have only one pmu per cpuctx. Since perf_pmu_sched_task is only called in hw contexts, this patch replaces cpuctx->unique_pmu by cpuctx->pmu in it.

The change above, together with the previous patch in this series, removed the remaining uses of cpuctx->unique_pmu, so we remove it altogether.

Signed-off-by David Carrillo-Cisneros <[email protected]>

Acked-by Mark Rutland <[email protected]>

kernel/events/core.c +0 -20

@@ -8572,37 +8572,10 @@ static struct perf_cpu_context __percpu *find_pmu_context(int ctxn)
 	return NULL;
 }
 
-static void update_pmu_context(struct pmu *pmu, struct pmu *old_pmu)
-{
-	int cpu;
-
-	for_each_possible_cpu(cpu) {
-		struct perf_cpu_context *cpuctx;
-
-		cpuctx = per_cpu_ptr(pmu->pmu_cpu_context, cpu);
-
-		if (cpuctx->unique_pmu == old_pmu)
-			cpuctx->unique_pmu = pmu;
-	}
-}
-
 static void free_pmu_context(struct pmu *pmu)
 {
-	struct pmu *i;
-
 	mutex_lock(&pmus_lock);
-	/*
-	 * Like a real lame refcount.
-	 */
-	list_for_each_entry(i, &pmus, entry) {

Quoted 46 lines · David Carrillo-Cisneros

cpuctx->unique_pmu was originally introduced as a way to identify cpuctxs with shared pmus in order to avoid visiting the same cpuctx more than once in a for_each_pmu loop.

cpuctx->unique_pmu == cpuctx->pmu in non-software task contexts since they have only one pmu per cpuctx. Since perf_pmu_sched_task is only called in hw contexts, this patch replaces cpuctx->unique_pmu by cpuctx->pmu in it.

The change above, together with the previous patch in this series, removed the remaining uses of cpuctx->unique_pmu, so we remove it altogether.

Signed-off-by David Carrillo-Cisneros <[email protected]>

Acked-by Mark Rutland <[email protected]>

kernel/events/core.c +0 -25

@@ -8572,37 +8572,10 @@ static struct perf_cpu_context __percpu *find_pmu_context(int ctxn)
 	return NULL;
 }
 
-static void update_pmu_context(struct pmu *pmu, struct pmu *old_pmu)
-{
-	int cpu;
-
-	for_each_possible_cpu(cpu) {
-		struct perf_cpu_context *cpuctx;
-
-		cpuctx = per_cpu_ptr(pmu->pmu_cpu_context, cpu);
-
-		if (cpuctx->unique_pmu == old_pmu)
-			cpuctx->unique_pmu = pmu;
-	}
-}
-
 static void free_pmu_context(struct pmu *pmu)
 {
-	struct pmu *i;
-
 	mutex_lock(&pmus_lock);
-	/*
-	 * Like a real lame refcount.
-	 */
-	list_for_each_entry(i, &pmus, entry) {
-		if (i->pmu_cpu_context == pmu->pmu_cpu_context) {
-			update_pmu_context(i, pmu);
-			goto out;
-		}
-	}

kernel/events/core.c static struct perf_cpu_context __percpu *find_pmu_context(int ctxn) · line 8580

Quoted 5 lines

kernel/events/core.c +0 -2

-
 	free_percpu(pmu->pmu_cpu_context);
-out:
 	mutex_unlock(&pmus_lock);
 }

This very much relies on us never calling perf_pmu_unregister() on the software PMUs afaict. A condition not mention in the Changelog.

Quoted 47 lines · David Carrillo-Cisneros

cpuctx->unique_pmu was originally introduced as a way to identify cpuctxs with shared pmus in order to avoid visiting the same cpuctx more than once in a for_each_pmu loop.

cpuctx->unique_pmu == cpuctx->pmu in non-software task contexts since they have only one pmu per cpuctx. Since perf_pmu_sched_task is only called in hw contexts, this patch replaces cpuctx->unique_pmu by cpuctx->pmu in it.

The change above, together with the previous patch in this series, removed the remaining uses of cpuctx->unique_pmu, so we remove it altogether.

Signed-off-by David Carrillo-Cisneros <[email protected]>

Acked-by Mark Rutland <[email protected]>

kernel/events/core.c +0 -25

@@ -8572,37 +8572,10 @@ static struct perf_cpu_context __percpu *find_pmu_context(int ctxn)
      return NULL;
 }
 
-static void update_pmu_context(struct pmu *pmu, struct pmu *old_pmu)
-{
-     int cpu;
-
-     for_each_possible_cpu(cpu) {
-             struct perf_cpu_context *cpuctx;
-
-             cpuctx = per_cpu_ptr(pmu->pmu_cpu_context, cpu);
-
-             if (cpuctx->unique_pmu == old_pmu)
-                     cpuctx->unique_pmu = pmu;
-     }
-}
-
 static void free_pmu_context(struct pmu *pmu)
 {
-     struct pmu *i;
-
      mutex_lock(&pmus_lock);
-     /*
-      * Like a real lame refcount.
-      */
-     list_for_each_entry(i, &pmus, entry) {
-             if (i->pmu_cpu_context == pmu->pmu_cpu_context) {
-                     update_pmu_context(i, pmu);
-                     goto out;
-             }
-     }

kernel/events/core.c static struct perf_cpu_context __percpu *find_pmu_context(int ctxn) · line 8580

Quoted 5 lines

kernel/events/core.c +0 -2

-
      free_percpu(pmu->pmu_cpu_context);
-out:
      mutex_unlock(&pmus_lock);
 }

This very much relies on us never calling perf_pmu_unregister() on the software PMUs afaict. A condition not mention in the Changelog.

Quoted 17 lines · Peter Zijlstra

Quoted 13 lines · David Carrillo-Cisneros

cpuctx->unique_pmu was originally introduced as a way to identify cpuctxs with shared pmus in order to avoid visiting the same cpuctx more than once in a for_each_pmu loop.

cpuctx->unique_pmu == cpuctx->pmu in non-software task contexts since they have only one pmu per cpuctx. Since perf_pmu_sched_task is only called in hw contexts, this patch replaces cpuctx->unique_pmu by cpuctx->pmu in it.

The change above, together with the previous patch in this series, removed the remaining uses of cpuctx->unique_pmu, so we remove it altogether.

Signed-off-by David Carrillo-Cisneros <[email protected]>

Acked-by Mark Rutland <[email protected]>

Quoted 3 lines

This very much relies on us never calling perf_pmu_unregister() on the software PMUs afaict. A condition not mention in the Changelog.

What's a good way to solve this? Update the Changelog or add code to update ctx->pmu?

This issue would go away cleanly if we were to remove the context sharing across pmu's. Would you support work in that direction?