perf/x86: Package residency counter improvements
Subsystems (3)
cover perf/x86: Package residency counter improvements
Hello,
No changes on this, just resending as 6.6-rc1 is out, and the first post got no feedback.
Original cover letter with more details here [1].
-Tero
[1] https://lore.kernel.org/bpf/[email protected]/T/#mdaea2388b76abc33ebccac3d0a6ccc0ac0def20b
PATCH RESEND 1/2 perf/x86/cstate: Allow reading the package statistics from local CPU
The MSR registers for reading the package residency counters are available on every CPU of the package. To avoid doing unnecessary SMP calls to read the values for these from the various CPUs inside a package, allow reading them from any CPU of the package.
--- arch/x86/events/intel/cstate.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/events/intel/cstate.c +3 -0
@@ -336,6 +336,9 @@ static int cstate_pmu_event_init(struct perf_event *event) cfg =array_index_nospec ((unsigned long )cfg,PERF_CSTATE_PKG_EVENT_MAX );if (!(pkg_msr_mask & (1 << cfg)))return -EINVAL ; + + event->event_caps |=PERF_EV_CAP_READ_ACTIVE_PKG ; + event->hw.event_base = pkg_msr[cfg].msr; cpu =cpumask_any_and (&cstate_pkg_cpu_mask,topology_die_cpumask (event->cpu));
Signature · 1 line
2.40.1
PATCH v2 2/2 perf/core: Allow reading package events from perf_event_read_local
Per-package perf events are typically registered with a single CPU only, however they can be read across all the CPUs within the package. Currently perf_event_read maps the event CPU according to the topology information to avoid an unnecessary SMP call, however perf_event_read_local deals with hard values and rejects a read with a failure if the CPU is not the one exactly registered. Allow similar mapping within the perf_event_read_local if the perf event in question can support this.
This allows users like BPF code to read the package perf events properly across different CPUs within a package.
---
v2:
* prevent illegal array access in case event->oncpu == -1
* split the event->cpu / event->oncpu handling to their own variables
kernel/events/core.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
kernel/events/core.c +15 -3
@@ -4425,6 +4425,9 @@ static int __perf_event_read_cpu(struct perf_event *event, int event_cpu) {u16 local_pkg, event_pkg; +if (event_cpu <0 || event_cpu >= nr_cpu_ids) +return event_cpu; +if (event->group_caps &PERF_EV_CAP_READ_ACTIVE_PKG ) {int local_cpu =smp_processor_id ();
@@ -4528,6 +4531,8 @@ int perf_event_read_local(struct perf_event *event, u64 *value, {unsigned long flags;int ret =0 ; +int event_cpu; +int event_oncpu;/* * Disabling interrupts avoids all counter scheduling (context
@@ -4551,15 +4556,22 @@ int perf_event_read_local(struct perf_event *event, u64 *value, goto out; } +/* +* Get the event CPU numbers, and adjust them to local if the event is +* a per-package event that can be read locally +*/ + event_oncpu =__perf_event_read_cpu (event, event->oncpu); + event_cpu =__perf_event_read_cpu (event, event->cpu); +/* If this is a per-CPU event, it must be for this CPU */ if (!(event->attach_state &PERF_ATTACH_TASK ) && - event->cpu !=smp_processor_id ()) { + event_cpu !=smp_processor_id ()) { ret = -EINVAL ; goto out; }/* If this is a pinned event it must be running on this CPU */ -if (event->attr.pinned && event->oncpu !=smp_processor_id ()) { +if (event->attr.pinned && event_oncpu !=smp_processor_id ()) { ret = -EBUSY ; goto out; }
@@ -4569,7 +4581,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value,* or local to this CPU. Furthermore it means its ACTIVE (otherwise * oncpu == -1). */ -if (event->oncpu ==smp_processor_id ()) +if (event_oncpu ==smp_processor_id ()) event->pmu->read (event); *value =local64_read (&event->count);
Signature · 1 line
2.40.1
Peter Zijlstra
maintainerRe: [PATCHv2 2/2] perf/core: Allow reading package events from perf_event_read_local
Quoted 26 lines · Tero Kristo
Per-package perf events are typically registered with a single CPU only, however they can be read across all the CPUs within the package. Currently
perf_event_readmaps the event CPU according to the topology information to avoid an unnecessary SMP call, howeverperf_event_read_localdeals with hard values and rejects a read with a failure if the CPU is not the one exactly registered. Allow similar mapping within theperf_event_read_localif the perf event in question can support this.This allows users like BPF code to read the package perf events properly across different CPUs within a package.
Signed-off-by Tero Kristo <[email protected]>---
v2:
* prevent illegal array access in caseevent->oncpu== -1
* split theevent->cpu/event->oncpuhandling to their own variableskernel/events/core.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)kernel/events/core.c +0
-0
kernel/events/core.c static int __perf_event_read_cpu(struct perf_event *event, int event_cpu) · line 4429
Quoted 5 lines
kernel/events/core.c +2
-0@@ -4425,6 +4425,9 @@ static int __perf_event_read_cpu(struct perf_event *event, int event_cpu) {u16 local_pkg, event_pkg; +if (event_cpu <0 || event_cpu >= nr_cpu_ids) +return event_cpu;
if ((unsigned)event_cpu >= nr_cpu_ids) return event_cpu;
As you could also find at the current __perf_event_read_cpu() callsite.
kernel/events/core.c int perf_event_read_local(struct perf_event *event, u64 *value, · line 4535
Quoted 10 lines
kernel/events/core.c +3
-0+if (event->group_caps &PERF_EV_CAP_READ_ACTIVE_PKG ) {int local_cpu =smp_processor_id ();@@ -4528,6 +4531,8 @@ int perf_event_read_local(struct perf_event *event, u64 *value, {unsigned long flags;int ret =0 ; +int event_cpu; +int event_oncpu;
You wrecked the x-mas tree :-)
I'll fix both up.
Thanks!
[tip: perf/core] perf/core: Allow reading package events from perf_event_read_local
In reply to Tero KristoThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 1765bb61bb18a7b81f68806de6e8b8f5000f65bf Gitweb: https://git.kernel.org/tip/1765bb61bb18a7b81f68806de6e8b8f5000f65bf Author: Tero Kristo <[email protected]> AuthorDate: Wed, 13 Sep 2023 15:59:56 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 09 Oct 2023 16:12:22 +02:00
perf/core: Allow reading package events from perf_event_read_local
Per-package perf events are typically registered with a single CPU only, however they can be read across all the CPUs within the package. Currently perf_event_read maps the event CPU according to the topology information to avoid an unnecessary SMP call, however perf_event_read_local deals with hard values and rejects a read with a failure if the CPU is not the one exactly registered. Allow similar mapping within the perf_event_read_local if the perf event in question can support this.
This allows users like BPF code to read the package perf events properly across different CPUs within a package.
--- kernel/events/core.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
kernel/events/core.c +15 -3
@@ -4425,6 +4425,9 @@ static int __perf_event_read_cpu(struct perf_event *event, int event_cpu) {u16 local_pkg, event_pkg; +if ((unsigned )event_cpu >= nr_cpu_ids) +return event_cpu; +if (event->group_caps &PERF_EV_CAP_READ_ACTIVE_PKG ) {int local_cpu =smp_processor_id ();
@@ -4527,6 +4530,8 @@ int perf_event_read_local(struct perf_event *event, u64 *value, u64 *enabled, u64 *running) {unsigned long flags; +int event_oncpu; +int event_cpu;int ret =0 ;/*
@@ -4551,15 +4556,22 @@ int perf_event_read_local(struct perf_event *event, u64 *value, goto out; } +/* +* Get the event CPU numbers, and adjust them to local if the event is +* a per-package event that can be read locally +*/ + event_oncpu =__perf_event_read_cpu (event, event->oncpu); + event_cpu =__perf_event_read_cpu (event, event->cpu); +/* If this is a per-CPU event, it must be for this CPU */ if (!(event->attach_state &PERF_ATTACH_TASK ) && - event->cpu !=smp_processor_id ()) { + event_cpu !=smp_processor_id ()) { ret = -EINVAL ; goto out; }/* If this is a pinned event it must be running on this CPU */ -if (event->attr.pinned && event->oncpu !=smp_processor_id ()) { +if (event->attr.pinned && event_oncpu !=smp_processor_id ()) { ret = -EBUSY ; goto out; }
@@ -4569,7 +4581,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value,* or local to this CPU. Furthermore it means its ACTIVE (otherwise * oncpu == -1). */ -if (event->oncpu ==smp_processor_id ()) +if (event_oncpu ==smp_processor_id ()) event->pmu->read (event); *value =local64_read (&event->count);
[tip: perf/core] perf/x86/cstate: Allow reading the package statistics from local CPU
In reply to Tero KristoThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 05276d4831fe023b6674a72bd6b8c5b39796e690 Gitweb: https://git.kernel.org/tip/05276d4831fe023b6674a72bd6b8c5b39796e690 Author: Tero Kristo <[email protected]> AuthorDate: Tue, 12 Sep 2023 15:44:31 +03:00 Committer: Peter Zijlstra <[email protected]> CommitterDate: Mon, 09 Oct 2023 16:12:22 +02:00
perf/x86/cstate: Allow reading the package statistics from local CPU
The MSR registers for reading the package residency counters are available on every CPU of the package. To avoid doing unnecessary SMP calls to read the values for these from the various CPUs inside a package, allow reading them from any CPU of the package.
--- arch/x86/events/intel/cstate.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/events/intel/cstate.c +3 -0
@@ -336,6 +336,9 @@ static int cstate_pmu_event_init(struct perf_event *event) cfg =array_index_nospec ((unsigned long )cfg,PERF_CSTATE_PKG_EVENT_MAX );if (!(pkg_msr_mask & (1 << cfg)))return -EINVAL ; + + event->event_caps |=PERF_EV_CAP_READ_ACTIVE_PKG ; + event->hw.event_base = pkg_msr[cfg].msr; cpu =cpumask_any_and (&cstate_pkg_cpu_mask,topology_die_cpumask (event->cpu));
PATCH RESEND 2/2 perf/core: Allow reading package events from perf_event_read_local
In reply to Tero KristoPer-package perf events are typically registered with a single CPU only, however they can be read across all the CPUs within the package. Currently perf_event_read maps the event CPU according to the topology information to avoid an unnecessary SMP call, however perf_event_read_local deals with hard values and rejects a read with a failure if the CPU is not the one exactly registered. Allow similar mapping within the perf_event_read_local if the perf event in question can support this.
This allows users like BPF code to read the package perf events properly across different CPUs within a package.
--- kernel/events/core.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
kernel/events/core.c +7 -3
@@ -4528,6 +4528,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value, {unsigned long flags;int ret =0 ; +int event_cpu;/* * Disabling interrupts avoids all counter scheduling (context
@@ -4551,15 +4552,18 @@ int perf_event_read_local(struct perf_event *event, u64 *value, goto out; } + event_cpu =READ_ONCE (event->oncpu); + event_cpu =__perf_event_read_cpu (event, event_cpu); +/* If this is a per-CPU event, it must be for this CPU */ if (!(event->attach_state &PERF_ATTACH_TASK ) && - event->cpu !=smp_processor_id ()) { + event_cpu !=smp_processor_id ()) { ret = -EINVAL ; goto out; }/* If this is a pinned event it must be running on this CPU */ -if (event->attr.pinned && event->oncpu !=smp_processor_id ()) { +if (event->attr.pinned && event_cpu !=smp_processor_id ()) { ret = -EBUSY ; goto out; }
@@ -4569,7 +4573,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value,* or local to this CPU. Furthermore it means its ACTIVE (otherwise * oncpu == -1). */ -if (event->oncpu ==smp_processor_id ()) +if (event_cpu ==smp_processor_id ()) event->pmu->read (event); *value =local64_read (&event->count);
Signature · 1 line
2.40.1
Peter Zijlstra
maintainerRe: [RESEND PATCH 2/2] perf/core: Allow reading package events from perf_event_read_local
Quoted 30 lines · Tero Kristo
Per-package perf events are typically registered with a single CPU only, however they can be read across all the CPUs within the package. Currently
perf_event_readmaps the event CPU according to the topology information to avoid an unnecessary SMP call, howeverperf_event_read_localdeals with hard values and rejects a read with a failure if the CPU is not the one exactly registered. Allow similar mapping within theperf_event_read_localif the perf event in question can support this.This allows users like BPF code to read the package perf events properly across different CPUs within a package.
Signed-off-by Tero Kristo <[email protected]>--- kernel/events/core.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)kernel/events/core.c +1
-0@@ -4528,6 +4528,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value, {unsigned long flags;int ret =0 ; +int event_cpu;/* * Disabling interrupts avoids all counter scheduling (context
kernel/events/core.c int perf_event_read_local(struct perf_event *event, u64 *value, · line 4556
Quoted 5 lines
kernel/events/core.c +2
-0@@ -4551,15 +4552,18 @@ int perf_event_read_local(struct perf_event *event, u64 *value, goto out; } + event_cpu =READ_ONCE (event->oncpu); + event_cpu =__perf_event_read_cpu (event, event_cpu);
What happens with __perf_event_read_cpu() when event_cpu == -1 ?
Quoted 27 lines
kernel/events/core.c +4
-3+/* If this is a per-CPU event, it must be for this CPU */ if (!(event->attach_state &PERF_ATTACH_TASK ) && - event->cpu !=smp_processor_id ()) { + event_cpu !=smp_processor_id ()) { ret = -EINVAL ; goto out; }/* If this is a pinned event it must be running on this CPU */ -if (event->attr.pinned && event->oncpu !=smp_processor_id ()) { +if (event->attr.pinned && event_cpu !=smp_processor_id ()) { ret = -EBUSY ; goto out; }@@ -4569,7 +4573,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value,* or local to this CPU. Furthermore it means its ACTIVE (otherwise * oncpu == -1). */ -if (event->oncpu ==smp_processor_id ()) +if (event_cpu ==smp_processor_id ()) event->pmu->read (event); *value =local64_read (&event->count);Signature · 1 line
2.40.1
Re: [RESEND PATCH 2/2] perf/core: Allow reading package events from perf_event_read_local
Quoted 34 lines · Peter Zijlstra
Quoted 33 lines · Tero Kristo
Per-package perf events are typically registered with a single CPU only, however they can be read across all the CPUs within the package. Currently
perf_event_readmaps the event CPU according to the topology information to avoid an unnecessary SMP call, howeverperf_event_read_localdeals with hard values and rejects a read with a failure if the CPU is not the one exactly registered. Allow similar mapping within theperf_event_read_localif the perf event in question can support this.This allows users like BPF code to read the package perf events properly across different CPUs within a package.
Signed-off-by Tero Kristo <[email protected]>--- kernel/events/core.c | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)kernel/events/core.c +1
-0@@ -4528,6 +4528,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value, {unsigned long flags;int ret =0 ; +int event_cpu;/* * Disabling interrupts avoids all counter scheduling (context @@ -4551,15 +4552,18 @@ int perf_event_read_local(struct perf_event *event, u64 *value, goto out; }
Quoted 3 lines
kernel/events/core.c int perf_event_read_local(struct perf_event *event, u64 *value, · line 4556
Quoted 2 lines
kernel/events/core.c +2
-0+ event_cpu =READ_ONCE (event->oncpu); + event_cpu =__perf_event_read_cpu (event, event_cpu);What happens with
__perf_event_read_cpu()whenevent_cpu== -1 ?
Good question. It looks like I need to add a check against that. Will update and send v2 out.
-Tero
Quoted 28 lines
Quoted 28 lines
kernel/events/core.c +4
-3+/* If this is a per-CPU event, it must be for this CPU */ if (!(event->attach_state &PERF_ATTACH_TASK ) && - event->cpu !=smp_processor_id ()) { + event_cpu !=smp_processor_id ()) { ret = -EINVAL ; goto out; }/* If this is a pinned event it must be running on this CPU */ -if (event->attr.pinned && event->oncpu !=smp_processor_id ()) { +if (event->attr.pinned && event_cpu !=smp_processor_id ()) { ret = -EBUSY ; goto out; }@@ -4569,7 +4573,7 @@ int perf_event_read_local(struct perf_event *event, u64 *value,* or local to this CPU. Furthermore it means its ACTIVE (otherwise * oncpu == -1). */ -if (event->oncpu ==smp_processor_id ()) +if (event_cpu ==smp_processor_id ()) event->pmu->read (event); *value =local64_read (&event->count);Signature · 1 line
2.40.1