perf: Fix perf_event_attr::exclusive rotation
Peter Zijlstra
maintainerv2 cover perf: Fix perf_event_attr::exclusive rotation
Hi,
Andi recently added exclusive event group support to tools/perf:
https://lkml.kernel.org/r/[email protected]
and promptly found that they didn't work as specified.
(sorry for the resend, I forgot LKML the first time)
Peter Zijlstra
maintainerPATCH v2 1/4 perf: Simplify group_sched_out()
Since event_sched_out() clears cpuctx->exclusive upon removal of an exclusive event (and only group leaders can be exclusive), there is no point in group_sched_out() trying to do it too. It is impossible for cpuctx->exclusive to still be set here.
--- kernel/events/core.c | 3 --- 1 file changed, 3 deletions(-)
kernel/events/core.c +0 -3
@@ -2312,9 +2312,6 @@ group_sched_out(struct perf_event *groupevent_sched_out (event, cpuctx, ctx);perf_pmu_enable (ctx->pmu); - -if (group_event->attr.exclusive) - cpuctx->exclusive =0 ; }#define DETACH_GROUP 0x01UL
[tip: perf/urgent] perf: Simplify group_sched_out()
The following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 8c7855d82933bab7fa5e96f0e568fc125c2e1ab4 Gitweb: https://git.kernel.org/tip/8c7855d82933bab7fa5e96f0e568fc125c2e1ab4 Author: Peter Zijlstra <[email protected]> AuthorDate: Thu, 29 Oct 2020 16:28:25 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 09 Nov 2020 18:12:35 +01:00
perf: Simplify group_sched_out()
Since event_sched_out() clears cpuctx->exclusive upon removal of an exclusive event (and only group leaders can be exclusive), there is no point in group_sched_out() trying to do it too. It is impossible for cpuctx->exclusive to still be set here.
--- kernel/events/core.c | 3 --- 1 file changed, 3 deletions(-)
kernel/events/core.c +0 -3
@@ -2312,9 +2312,6 @@ group_sched_out(struct perf_event *group_event,event_sched_out (event, cpuctx, ctx);perf_pmu_enable (ctx->pmu); - -if (group_event->attr.exclusive) - cpuctx->exclusive =0 ; }#define DETACH_GROUP 0x01UL
Peter Zijlstra
maintainerPATCH v2 2/4 perf: Simplify group_sched_in()
In reply to Peter ZijlstraCollate the error paths. Code duplication only leads to divergence and extra bugs.
--- kernel/events/core.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
kernel/events/core.c +3 -7
@@ -2580,11 +2580,8 @@ group_sched_in(struct perf_event *group_ pmu->start_txn (pmu,PERF_PMU_TXN_ADD ); -if (event_sched_in (group_event, cpuctx, ctx)) { - pmu->cancel_txn (pmu); -perf_mux_hrtimer_restart (cpuctx); -return -EAGAIN ; - } +if (event_sched_in (group_event, cpuctx, ctx)) + goto error;/* * Schedule in siblings as one group (if any):
@@ -2613,10 +2610,9 @@ group_sched_in(struct perf_event *group_ }event_sched_out (group_event, cpuctx, ctx); +error: pmu->cancel_txn (pmu); -perf_mux_hrtimer_restart (cpuctx); -return -EAGAIN ; }
[tip: perf/urgent] perf: Simplify group_sched_in()
The following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 251ff2d49347793d348babcff745289b11910e96 Gitweb: https://git.kernel.org/tip/251ff2d49347793d348babcff745289b11910e96 Author: Peter Zijlstra <[email protected]> AuthorDate: Thu, 29 Oct 2020 16:29:15 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 09 Nov 2020 18:12:35 +01:00
perf: Simplify group_sched_in()
Collate the error paths. Code duplication only leads to divergence and extra bugs.
--- kernel/events/core.c | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
kernel/events/core.c +3 -7
@@ -2580,11 +2580,8 @@ group_sched_in(struct perf_event *group_event, pmu->start_txn (pmu,PERF_PMU_TXN_ADD ); -if (event_sched_in (group_event, cpuctx, ctx)) { - pmu->cancel_txn (pmu); -perf_mux_hrtimer_restart (cpuctx); -return -EAGAIN ; - } +if (event_sched_in (group_event, cpuctx, ctx)) + goto error;/* * Schedule in siblings as one group (if any):
@@ -2613,10 +2610,9 @@ group_error: }event_sched_out (group_event, cpuctx, ctx); +error: pmu->cancel_txn (pmu); -perf_mux_hrtimer_restart (cpuctx); -return -EAGAIN ; }
Peter Zijlstra
maintainerPATCH v2 3/4 perf: Fix event multiplexing for exclusive groups
In reply to Peter ZijlstraCommit 9e6302056f80 ("perf: Use hrtimers for event multiplexing") placed the hrtimer (re)start call in the wrong place. Instead of capturing all scheduling failures, it only considered the PMU failure.
The result is that groups using perf_event_attr::exclusive are no longer rotated.
--- kernel/events/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/events/core.c +1 -1
@@ -2612,7 +2612,6 @@ group_sched_in(struct perf_event *group_ error: pmu->cancel_txn (pmu); -perf_mux_hrtimer_restart (cpuctx);return -EAGAIN ; }
@@ -3672,6 +3671,7 @@ static int merge_sched_in(struct perf_ev *can_add_hw =0 ; ctx->rotate_necessary =1 ; +perf_mux_hrtimer_restart (cpuctx); }return 0 ;
[tip: perf/urgent] perf: Fix event multiplexing for exclusive groups
The following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 2714c3962f304d031d5016c963c4b459337b0749 Gitweb: https://git.kernel.org/tip/2714c3962f304d031d5016c963c4b459337b0749 Author: Peter Zijlstra <[email protected]> AuthorDate: Thu, 29 Oct 2020 16:29:53 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 09 Nov 2020 18:12:36 +01:00
perf: Fix event multiplexing for exclusive groups
Commit 9e6302056f80 ("perf: Use hrtimers for event multiplexing") placed the hrtimer (re)start call in the wrong place. Instead of capturing all scheduling failures, it only considered the PMU failure.
The result is that groups using perf_event_attr::exclusive are no longer rotated.
--- kernel/events/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/events/core.c +1 -1
@@ -2612,7 +2612,6 @@ group_error: error: pmu->cancel_txn (pmu); -perf_mux_hrtimer_restart (cpuctx);return -EAGAIN ; }
@@ -3672,6 +3671,7 @@ static int merge_sched_in(struct perf_event *event, void *data) *can_add_hw =0 ; ctx->rotate_necessary =1 ; +perf_mux_hrtimer_restart (cpuctx); }return 0 ;
Peter Zijlstra
maintainerPATCH v2 4/4 perf: Tweak perf_event_attr::exclusive semantics
In reply to Peter ZijlstraCurrently perf_event_attr::exclusive can be used to ensure an event(group) is the sole group scheduled on the PMU. One consequence is that when you have a pinned event (say the watchdog) you can no longer have regular exclusive event(group)s.
Inspired by the fact that !pinned events are considered less strict, allow !pinned,exclusive events to share the PMU with pinned,!exclusive events.
Pinned,exclusive is still fully exclusive.
--- kernel/events/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/events/core.c +1 -1
@@ -2637,7 +2637,7 @@ static int group_can_go_on(struct perf_e* If this group is exclusive and there are already * events on the CPU, it can't go on. */ -if (event->attr.exclusive && cpuctx->active_oncpu) +if (event->attr.exclusive && !list_empty (get_event_list (event)))return 0 ;/* * Otherwise, try to add it if all previous groups were able
[tip: perf/urgent] perf: Tweak perf_event_attr::exclusive semantics
The following commit has been merged into the perf/urgent branch of tip:
The following commit has been merged into the perf/urgent branch of tip:
Commit-ID: 1908dc911792067287458fdb0800f036f4f4e0f6 Gitweb: https://git.kernel.org/tip/1908dc911792067287458fdb0800f036f4f4e0f6 Author: Peter Zijlstra <[email protected]> AuthorDate: Thu, 29 Oct 2020 16:32:22 +01:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 09 Nov 2020 18:12:36 +01:00
perf: Tweak perf_event_attr::exclusive semantics
Currently perf_event_attr::exclusive can be used to ensure an event(group) is the sole group scheduled on the PMU. One consequence is that when you have a pinned event (say the watchdog) you can no longer have regular exclusive event(group)s.
Inspired by the fact that !pinned events are considered less strict, allow !pinned,exclusive events to share the PMU with pinned,!exclusive events.
Pinned,exclusive is still fully exclusive.
--- kernel/events/core.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/events/core.c +1 -1
@@ -2637,7 +2637,7 @@ static int group_can_go_on(struct perf_event *event,* If this group is exclusive and there are already * events on the CPU, it can't go on. */ -if (event->attr.exclusive && cpuctx->active_oncpu) +if (event->attr.exclusive && !list_empty (get_event_list (event)))return 0 ;/* * Otherwise, try to add it if all previous groups were able
Quoted 9 lines · Peter Zijlstra
Hi,
Andi recently added exclusive event group support to
tools/perf:https://lkml.kernel.org/r/[email protected]
and promptly found that they didn't work as specified.
(sorry for the resend, I forgot LKML the first time)
hm, it's too late for me to check ;-) but should I be able to do this with exclusive event.. running both command at the same time:
$ sudo ./perf stat -e cycles:e -I 1000 # time counts unit events 1.002430650 33,946,849 cycles:e 2.004920725 502,399,986 cycles:e (67.57%) 3.007087631 859,745,048 cycles:e (50.00%) 4.009078254 845,860,723 cycles:e (50.00%) 5.011086104 838,457,275 cycles:e (50.00%)
$ sudo ./perf stat -e cycles:e -I 1000 # time counts unit events 1.001665466 848,973,404 cycles:e (50.01%) 2.003658048 856,505,255 cycles:e (50.00%) 3.005658022 842,737,973 cycles:e (50.00%) 4.007657797 844,800,598 cycles:e (50.00%)
jirka
Quoted 2 lines
hm, it's too late for me to check ;-) but should I be able to do this with exclusive event.. running both command at the same time:
Yes. The exclusive part only applies during a given context, but the two commands are different contexts.
You would only see a difference when in the same context, and you have multiple groups (or events) that could in theory schedule in parallel
e.g. something like
perf stat -e '{cycles,cycles},{cycles,cycles}' ...
The main motivation is actually that the "multiple groups" algorithm in perf doesn't work all that great: it has quite a few cases where it starves groups or makes the wrong decisions. That is because it is very difficult (likely NP complete) problem and the kernel takes a lot of short cuts to avoid spending too much time on it.
With exclusive it will be possible for a tool to generate "perfect groups" in user space and assume the kernel schedules it dumbly, but at least without any starvation.
-Andi
Peter Zijlstra
maintainerQuoted 5 lines · Andi Kleen
The main motivation is actually that the "multiple groups" algorithm in perf doesn't work all that great: it has quite a few cases where it starves groups or makes the wrong decisions. That is because it is very difficult (likely NP complete) problem and the kernel takes a lot of short cuts to avoid spending too much time on it.
The event scheduling should be starvation free, except in the presence of pinned events.
If you can show starvation without pinned events, it's a bug.
It will also always do equal or better than exclusive mode wrt PMU utilization. Again, if it doesn't it's a bug.
Please provide concrete examples for these two cases, or stop spreading FUD like this.
Quoted 7 lines · Peter Zijlstra
Quoted 5 lines · Andi Kleen
The main motivation is actually that the "multiple groups" algorithm in perf doesn't work all that great: it has quite a few cases where it starves groups or makes the wrong decisions. That is because it is very difficult (likely NP complete) problem and the kernel takes a lot of short cuts to avoid spending too much time on it.
Quoted 7 lines
The event scheduling should be starvation free, except in the presence of pinned events.
If you can show starvation without pinned events, it's a bug.
It will also always do equal or better than exclusive mode wrt PMU utilization. Again, if it doesn't it's a bug.
Simple example (I think we've shown that one before):
(on skylake)
$ cat /proc/sys/kernel/nmi_watchdog 0 $ perf stat -e instructions,cycles,frontend_retired.latency_ge_2,frontend_retired.latency_ge_16 -a sleep 2
Performance counter stats for 'system wide':
654,514,990 instructions # 0.34 insn per cycle (50.67%)
1,924,297,028 cycles (74.28%)
21,708,935 frontend_retired.latency_ge_2 (75.01%)
1,769,952 frontend_retired.latency_ge_16 (24.99%)2.002426541 seconds time elapsed
The second frontend_retired should be both getting 50% and the fixed events should be getting 100%. So several events are starved.
Another similar example is trying to schedule the topdown events on Icelake in parallel to other groups. It works with one extra group, but breaks with two.
(on icelake)
$ cat /proc/sys/kernel/nmi_watchdog
0
$ perf stat -e '{slots,topdown-bad-spec,topdown-be-bound,topdown-fe-bound,topdown-retiring},{branches,branches,branches,branches,branches,branches,branches,branches},{branches,branches,branches,branches,branches,branches,branches,branches}' -a sleep 1Performance counter stats for 'system wide':
71,229,087 slots (60.65%)
5,066,320 topdown-bad-spec # 7.1% bad speculation (60.65%)
35,080,387 topdown-be-bound # 49.2% backend bound (60.65%)
22,769,750 topdown-fe-bound # 32.0% frontend bound (60.65%)
8,336,760 topdown-retiring # 11.7% retiring (60.65%)
424,584 branches (70.00%)
424,584 branches (70.00%)
424,584 branches (70.00%)
424,584 branches (70.00%)
424,584 branches (70.00%)
424,584 branches (70.00%)
424,584 branches (70.00%)
424,584 branches (70.00%)
3,634,075 branches (30.00%)
3,634,075 branches (30.00%)
3,634,075 branches (30.00%)
3,634,075 branches (30.00%)
3,634,075 branches (30.00%)
3,634,075 branches (30.00%)
3,634,075 branches (30.00%)
3,634,075 branches (30.00%)1.001312511 seconds time elapsed
A tool using exclusive hopefully will be able to do better than this.
-Andi
Peter Zijlstra
maintainerQuoted 30 lines · Andi Kleen
Quoted 7 lines · Peter Zijlstra
Quoted 5 lines · Andi Kleen
The main motivation is actually that the "multiple groups" algorithm in perf doesn't work all that great: it has quite a few cases where it starves groups or makes the wrong decisions. That is because it is very difficult (likely NP complete) problem and the kernel takes a lot of short cuts to avoid spending too much time on it.
Quoted 7 lines
The event scheduling should be starvation free, except in the presence of pinned events.
If you can show starvation without pinned events, it's a bug.
It will also always do equal or better than exclusive mode wrt PMU utilization. Again, if it doesn't it's a bug.
Simple example (I think we've shown that one before):
(on skylake)
$ cat /proc/sys/kernel/nmi_watchdog 0 $ perf stat -e instructions,cycles,frontend_retired.latency_ge_2,frontend_retired.latency_ge_16 -a sleep 2Performance counter stats for 'system wide':
654,514,990 instructions # 0.34 insn per cycle (50.67%) 1,924,297,028 cycles (74.28%) 21,708,935 frontend_retired.latency_ge_2 (75.01%) 1,769,952 frontend_retired.latency_ge_16 (24.99%)
Quoted 4 lines
2.002426541 seconds time elapsedThe second
frontend_retiredshould be both getting 50% and the fixed events should be getting 100%. So several events are starved.
*should* how? Also, nothing is 0% so nothing is getting starved.
Quoted 32 lines
Another similar example is trying to schedule the topdown events on Icelake in parallel to other groups. It works with one extra group, but breaks with two.
(on icelake)
$ cat /proc/sys/kernel/nmi_watchdog 0 $ perf stat -e '{slots,topdown-bad-spec,topdown-be-bound,topdown-fe-bound,topdown-retiring},{branches,branches,branches,branches,branches,branches,branches,branches},{branches,branches,branches,branches,branches,branches,branches,branches}' -a sleep 1Performance counter stats for 'system wide':
71,229,087 slots (60.65%) 5,066,320 topdown-bad-spec # 7.1% bad speculation (60.65%) 35,080,387 topdown-be-bound # 49.2% backend bound (60.65%) 22,769,750 topdown-fe-bound # 32.0% frontend bound (60.65%) 8,336,760 topdown-retiring # 11.7% retiring (60.65%) 424,584 branches (70.00%) 424,584 branches (70.00%) 424,584 branches (70.00%) 424,584 branches (70.00%) 424,584 branches (70.00%) 424,584 branches (70.00%) 424,584 branches (70.00%) 424,584 branches (70.00%) 3,634,075 branches (30.00%) 3,634,075 branches (30.00%) 3,634,075 branches (30.00%) 3,634,075 branches (30.00%) 3,634,075 branches (30.00%) 3,634,075 branches (30.00%) 3,634,075 branches (30.00%) 3,634,075 branches (30.00%)
Quoted 3 lines
1.001312511 seconds time elapsedA tool using exclusive hopefully will be able to do better than this.
I don't see how, exclusive will always result in equal or worse PMU utilization, never better.