perf/core: Update perf_adjust_freq_unthr_context()
Namhyung Kim
maintainerPATCH v3 1/2 perf/core: Update perf_adjust_freq_unthr_context()
It was unnecessarily disabling and enabling PMUs for each event. It should be done at PMU level. Add pmu_ctx->nr_freq counter to check it at each PMU. As pmu context has separate active lists for pinned group and flexible group, factor out a new function to do the job.
Another minor optimization is that it can skip PMUs w/ CAP_NO_INTERRUPT even if it needs to unthrottle sampling events.
--- include/linux/perf_event.h | 6 ++++ kernel/events/core.c | 70 ++++++++++++++++++++++++-------------- 2 files changed, 50 insertions(+), 26 deletions(-)
include/linux/perf_event.h +6 -0
@@ -883,6 +883,7 @@ struct perf_event_pmu_context {unsigned int nr_events;unsigned int nr_cgroups; +unsigned int nr_freq;atomic_t refcount;/* event <-> epc */ struct rcu_head rcu_head;
@@ -897,6 +898,11 @@ struct perf_event_pmu_context {int rotate_necessary; }; +static inline bool perf_pmu_ctx_is_active (struct perf_event_pmu_context *epc) +{ +return !list_empty (&epc->flexible_active) || !list_empty (&epc->pinned_active); +} +struct perf_event_groups {struct rb_root tree;u64 index;
kernel/events/core.c +44 -26
114 lines not on this page · raw message
Signature · 1 line
2.43.0.594.gd9cf4e227d-goog
Namhyung Kim
maintainerPATCH v3 2/2 perf/core: Reduce PMU access to adjust sample freq
In perf_adjust_freq_unthr_context(), it first starts the event and then stop unnecessarily to adjust the sampling frequency if the event is throttled.
For a throttled non-frequency event, it doesn't have a freq so no need to adjust. Just starting the event would be ok.
For a frequency event, whether it's throttled or not, it needs to stop before adjusting the frequency. That means it should not start the even if it was throttled. I tried to skip calling the stop callback, but it didn't work well since the event count might not be up to date. It should call the stop callback with PERF_EF_UPDATE anyway.
However not calling start would prevent unnecessary MSR accesses (which can be costly) for already stopped events as stop state is saved in the hw config.
---
Ian and Kan, I decided to leave your Reviewed-by here since it's the basically same. Please tell me if you don't agree.
kernel/events/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/events/core.c +2 -1
@@ -4121,7 +4121,8 @@ static void perf_adjust_freq_unthr_events(struct list_head *event_list)if (hwc->interrupts ==MAX_INTERRUPTS ) { hwc->interrupts =0 ;perf_log_throttle (event,1 ); - event->pmu->start (event,0 ); +if (!event->attr.freq || !event->attr.sample_freq) + event->pmu->start (event,0 ); }if (!event->attr.freq || !event->attr.sample_freq)
Signature · 1 line
2.43.0.594.gd9cf4e227d-goog
[tip: perf/core] perf/core: Reduce PMU access to adjust sample freq
The following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: f38628b06c36222367e26820879789ae59e49f60 Gitweb: https://git.kernel.org/tip/f38628b06c36222367e26820879789ae59e49f60 Author: Namhyung Kim <[email protected]> AuthorDate: Tue, 06 Feb 2024 21:05:45 -08:00 Committer: Ingo Molnar <[email protected]> CommitterDate: Wed, 10 Apr 2024 06:13:57 +02:00
perf/core: Reduce PMU access to adjust sample freq
In perf_adjust_freq_unthr_context(), it first starts the event and then stop unnecessarily to adjust the sampling frequency if the event is throttled.
For a throttled non-frequency event, it doesn't have a freq so no need to adjust. Just starting the event would be ok.
For a frequency event, whether it's throttled or not, it needs to stop before adjusting the frequency. That means it should not start the even if it was throttled. I tried to skip calling the stop callback, but it didn't work well since the event count might not be up to date. It should call the stop callback with PERF_EF_UPDATE anyway.
However not calling start would prevent unnecessary MSR accesses (which can be costly) for already stopped events as stop state is saved in the hw config.
--- kernel/events/core.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/events/core.c +2 -1
@@ -4146,7 +4146,8 @@ static void perf_adjust_freq_unthr_events(struct list_head *event_list)if (hwc->interrupts ==MAX_INTERRUPTS ) { hwc->interrupts =0 ;perf_log_throttle (event,1 ); - event->pmu->start (event,0 ); +if (!event->attr.freq || !event->attr.sample_freq) + event->pmu->start (event,0 ); }if (!event->attr.freq || !event->attr.sample_freq)
[tip: perf/core] perf/core: Optimize perf_adjust_freq_unthr_context()
In reply to Namhyung KimThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 0259bf63f71e2accfeca4a4e346ede8edcc86aab Gitweb: https://git.kernel.org/tip/0259bf63f71e2accfeca4a4e346ede8edcc86aab Author: Namhyung Kim <[email protected]> AuthorDate: Tue, 06 Feb 2024 21:05:44 -08:00 Committer: Ingo Molnar <[email protected]> CommitterDate: Wed, 10 Apr 2024 06:13:57 +02:00
perf/core: Optimize perf_adjust_freq_unthr_context()
It was unnecessarily disabling and enabling PMUs for each event. It should be done at PMU level. Add pmu_ctx->nr_freq counter to check it at each PMU. As PMU context has separate active lists for pinned group and flexible group, factor out a new function to do the job.
Another minor optimization is that it can skip PMUs w/ CAP_NO_INTERRUPT even if it needs to unthrottle sampling events.
--- include/linux/perf_event.h | 6 +++- kernel/events/core.c | 70 +++++++++++++++++++++++-------------- 2 files changed, 50 insertions(+), 26 deletions(-)
include/linux/perf_event.h +6 -0
@@ -883,6 +883,7 @@ struct perf_event_pmu_context {unsigned int nr_events;unsigned int nr_cgroups; +unsigned int nr_freq;atomic_t refcount;/* event <-> epc */ struct rcu_head rcu_head;
@@ -897,6 +898,11 @@ struct perf_event_pmu_context {int rotate_necessary; }; +static inline bool perf_pmu_ctx_is_active (struct perf_event_pmu_context *epc) +{ +return !list_empty (&epc->flexible_active) || !list_empty (&epc->pinned_active); +} +struct perf_event_groups {struct rb_root tree;u64 index;
kernel/events/core.c +44 -26
114 lines not on this page · raw message