Skip to contentSkip to conversation

perf/cgroups: Don't rotate events for cgroups unnecessarily

PATCH In review Wen Yang 2 participants 4 messages Jun 7, 2021 Stable branch

lore
Share & tools
lore

Wen Yang

author
#0 lore raw

PATCH 1/2 perf/cgroups: Don't rotate events for cgroups unnecessarily

From: Ian Rogers <[email protected]>

[ Upstream commit fd7d55172d1e2e501e6da0a5c1de25f06612dc2e ]

Currently perf_rotate_context assumes that if the context's nr_events != nr_active a rotation is necessary for perf event multiplexing. With cgroups, nr_events is the total count of events for all cgroups and nr_active will not include events in a cgroup other than the current task's. This makes rotation appear necessary for cgroups when it is not.

Add a perf_event_context flag that is set when rotation is necessary. Clear the flag during sched_out and set it when a flexible sched_in fails due to resources.

Cc Alexander Shishkin <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Borislav Petkov <[email protected]>
Cc Jiri Olsa <[email protected]>
Cc Kan Liang <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Namhyung Kim <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Stephane Eranian <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Vince Weaver <[email protected]>
---
 include/linux/perf_event.h |  5 +++++
 kernel/events/core.c       | 42 ++++++++++++++++++++++--------------------
 2 files changed, 27 insertions(+), 20 deletions(-)
include/linux/perf_event.h +5 -0
index c2876e7..ae97f1a 100644
@@ -742,6 +742,11 @@ struct perf_event_context {
 	int				nr_stat;
 	int				nr_freq;
 	int				rotate_disable;
+	/*
+	 * Set when nr_events != nr_active, except tolerant to events not
+	 * necessary to be active due to scheduling constraints, such as cgroups.
+	 */
+	int				rotate_necessary;
 	atomic_t			refcount;
 	struct task_struct		*task;
 
kernel/events/core.c +22 -20

86 lines not on this page · raw message

Signature · 1 line
1.8.3.1
author
#1 lore raw

PATCH 2/2 perf/core: Fix corner case in perf_rotate_context()

From: Song Liu <[email protected]>

[ Upstream commit 7fa343b7fdc4f351de4e3f28d5c285937dd1f42f ]

In perf_rotate_context(), when the first cpu flexible event fail to schedule, cpu_rotate is 1, while cpu_event is NULL. Since cpu_event is NULL, perf_rotate_context will _NOT_ call cpu_ctx_sched_out(), thus cpuctx->ctx.is_active will have EVENT_FLEXIBLE set. Then, the next perf_event_sched_in() will skip all cpu flexible events because of the EVENT_FLEXIBLE bit.

In the next call of perf_rotate_context(), cpu_rotate stays 1, and cpu_event stays NULL, so this process repeats. The end result is, flexible events on this cpu will not be scheduled (until another event being added to the cpuctx).

Here is an easy repro of this issue. On Intel CPUs, where ref-cycles could only use one counter, run one pinned event for ref-cycles, one flexible event for ref-cycles, and one flexible event for cycles. The flexible ref-cycles is never scheduled, which is expected. However, because of this issue, the cycles event is never scheduled either.

 $ perf stat -e ref-cycles:D,ref-cycles,cycles -C 5 -I 1000
           time             counts unit events
    1.000152973         15,412,480      ref-cycles:D
    1.000152973      <not counted>      ref-cycles     (0.00%)
    1.000152973      <not counted>      cycles         (0.00%)
    2.000486957         18,263,120      ref-cycles:D
    2.000486957      <not counted>      ref-cycles     (0.00%)
    2.000486957      <not counted>      cycles         (0.00%)

To fix this, when the flexible_active list is empty, try rotate the first event in the flexible_groups. Also, rename ctx_first_active() to ctx_event_to_rotate(), which is more accurate.

Cc Arnaldo Carvalho de Melo <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Sasha Levin <[email protected]>
Cc Thomas Gleixner <[email protected]>
Fixes 8d5bce0c37fa ("perf/core: Optimize perf_rotate_context() event scheduling")
---
 kernel/events/core.c | 22 +++++++++++++++++-----
 1 file changed, 17 insertions(+), 5 deletions(-)
kernel/events/core.c +17 -5
index ae7dc37..bfb6515 100644
@@ -3676,11 +3676,23 @@ static void rotate_ctx(struct perf_event_context *ctx, struct perf_event *event)
 	perf_event_groups_insert(&ctx->flexible_groups, event);
 }
 
+/* pick an event from the flexible_groups to rotate */
 static inline struct perf_event *
-ctx_first_active(struct perf_event_context *ctx)
+ctx_event_to_rotate(struct perf_event_context *ctx)
 {
-	return list_first_entry_or_null(&ctx->flexible_active,
-					struct perf_event, active_list);
+	struct perf_event *event;
+
+	/* pick the first active flexible event */
+	event = list_first_entry_or_null(&ctx->flexible_active,
+					 struct perf_event, active_list);
+
+	/* if no active flexible event, pick the first event */
+	if (!event) {
+		event = rb_entry_safe(rb_first(&ctx->flexible_groups.tree),
+				      typeof(*event), group_node);
+	}
+
+	return event;
 }
 
 static bool perf_rotate_context(struct perf_cpu_context *cpuctx)
@@ -3705,9 +3717,9 @@ static bool perf_rotate_context(struct perf_cpu_context *cpuctx)
 	perf_pmu_disable(cpuctx->ctx.pmu);
 
 	if (task_rotate)
-		task_event = ctx_first_active(task_ctx);
+		task_event = ctx_event_to_rotate(task_ctx);
 	if (cpu_rotate)
-		cpu_event = ctx_first_active(&cpuctx->ctx);
+		cpu_event = ctx_event_to_rotate(&cpuctx->ctx);
 
 	/*
 	 * As per the order given at ctx_resched() first 'pop' task flexible
Signature · 1 line
1.8.3.1
#2 lore raw
In reply to Wen Yang
Quoted 3 lines · Wen Yang

From: Ian Rogers <[email protected]>

[ Upstream commit fd7d55172d1e2e501e6da0a5c1de25f06612dc2e ]

<snip>

If you look at the releases page of kernel.org, you will see the active kernels (also the front page of kernel.org shows this).

So why are you sending patches for kernels that are long long end-of-life? Who is going to use these patches, and for what trees?

thanks,

greg k-h

author
#3 lore raw
Quoted 10 lines · Greg Kroah-Hartman
Quoted 3 lines · Wen Yang

From: Ian Rogers <[email protected]>

[ Upstream commit fd7d55172d1e2e501e6da0a5c1de25f06612dc2e ]

<snip>

If you look at the releases page of kernel.org, you will see the active kernels (also the front page of kernel.org shows this).

Quoted 7 lines

So why are you sending patches for kernels that are long long end-of-life? Who is going to use these patches, and for what trees?

thanks,

greg k-h

Okay, thank you for your reminder.
The 4.19 kernel is used in our environment (it is also active). This bug exists in versions from 4.19 to 5.2, but in fact it only needs to be fixed on 4.19, thanks

--
Best wishes,
Wen