perf report/top: Add opt-in --hybrid-merge
Ian Rogers
reviewerv2 cover perf report/top: Add opt-in --hybrid-merge
On a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error.
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
The events to merge are found with first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.
Changes in v2:
- The interactive 'M' keystroke that toggled merging in the hists browser is gone. Merging is now asked for on the command line or through core.hybrid-merge, and the browser instead points out that --hybrid-merge is available when there are events it could merge.
- Fold "perf ui hist: Format group headers iteratively based on proportional visual allocations" into patch 2, as the two only make sense together.
- New patch 5, so that enabling and disabling a merged set touches every event in it. The merged events are made members of a leader but are not a real group, so each still has to be toggled in its own right.
- New patch 7, adding core.hybrid-merge.
- New patch 9, so that "perf test" doesn't inherit the user's ~/.perfconfig. Without it, setting core.hybrid-merge breaks the output matching of unrelated shell tests.
- Patch 8 adds a C test for the merging decision and a report shell test, rather than only extending the perf top test.
- Rewrite the commit messages of patches 2, 3, 5 and 8 to describe the problem being solved rather than the diff.
Testing:
- New "Hybrid event merging" perf test with three subtests: merging events of 2 core PMUs, of 3 core PMUs, and events that must not be merged.
- New "perf report hybrid merge" shell test, and the perf top shell test extended.
- perf test 13, 17, 27, 30, 31, 66, 67, 68, 78, 102, 113 and 172 pass, covering the hists, --hierarchy and addr2line paths this touches.
- Builds in 13 configurations, and every patch builds on its own so the series stays bisectable.
Ian Rogers (9):
perf evlist: Implement evlist__can_merge_hybrid using
first_wildcard_match
perf ui hist: Add support for aggregated total_period and merging
entries
perf tools: Expose opt-in --hybrid-merge
perf Documentation: Add tip for hybrid event merging perf evlist: Toggle merged_hybrid_group properly in enable/disable perf tools: Add TUI hints for --hybrid-merge
perf config: Add core.hybrid-merge to configure event merging perf test: Expand tests for --hybrid-merge
perf test: Isolate test suite from user .perfconfig natively
tools/perf/Documentation/perf-config.txt | 8 + tools/perf/Documentation/perf-report.txt | 5 + tools/perf/Documentation/perf-top.txt | 5 + tools/perf/Documentation/tips.txt | 2 + tools/perf/builtin-report.c | 25 ++ tools/perf/builtin-top.c | 29 ++- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 50 +++- tools/perf/tests/hybrid-merge.c | 206 +++++++++++++++ tools/perf/tests/shell/report_hybrid_merge.sh | 121 +++++++++ tools/perf/tests/shell/top.sh | 73 +++++- tools/perf/tests/tests.h | 1 + tools/perf/ui/browsers/hists.c | 7 +- tools/perf/ui/hist.c | 246 +++++++++++++++++- tools/perf/util/config.c | 10 + tools/perf/util/evlist.c | 230 +++++++++++++++- tools/perf/util/evlist.h | 3 + tools/perf/util/evsel.h | 1 + tools/perf/util/hist.h | 15 +- tools/perf/util/parse-events.c | 4 +- tools/perf/util/pmu.c | 43 ++- tools/perf/util/pmu.h | 3 + tools/perf/util/symbol.c | 1 + tools/perf/util/symbol_conf.h | 1 + 24 files changed, 1046 insertions(+), 44 deletions(-) create mode 100644 tools/perf/tests/hybrid-merge.c create mode 100755 tools/perf/tests/shell/report_hybrid_merge.sh
base-commit: 91b0782fc9e9d2f0a40b5256146e014802fdbb36
prerequisite-patch-id: b6fdc526887b71fb66f5fe0c0d41f7ef9493861e
prerequisite-patch-id: d47077f674c700f4f6296e9367f7ddfe004aea89
prerequisite-patch-id: ed0db23450840601762fe85ca1bef06ce6d28fe7
prerequisite-patch-id: d84e6b96d533ee6ed549e26e94216e2da60f7f74
prerequisite-patch-id: 6e8f19551d771621a5037096626cfe7e271b36f9
prerequisite-patch-id: 3768dfd588c7439deb741bce44c74763011c6be7
prerequisite-patch-id: babac99a4faa3b1525e44d3f34ab33a88103334c
prerequisite-patch-id: 5279827453fea1536abc620aa33c887c44e41ee9
prerequisite-patch-id: bdc0d648a577142b9270fb59b54496bc2bf8d5ea
prerequisite-patch-id: 9d468f113dffb082d74eb6d76e442b6ac69d9983
prerequisite-patch-id: 6f38fc0028b7d5eba7b8b646b402b8187bc3dbfa
prerequisite-patch-id: bc536fd240b5ddd999c781831a6445cb04ace25c
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
Ian Rogers
reviewerPATCH v2 1/9 perf evlist: Implement evlist__can_merge_hybrid using first_wildcard_match
Add logic to dynamically identify mergeable events spawned from the same wildcard alias via first_wildcard_match. first_wildcard_match is populated when events are parsed, in perf report the events come from a file and so the first_wildcard_match is computed by matching event names.
evlist__can_merge_hybrid only tests whether merging is possible and doesn't modify the evlist, so that it may be used as a predicate, for example, to decide whether to display a hint. The linking of all the matching events is done by evlist__merge_hybrid.
is_pmu_core_len identifies core PMUs using the hybrid topology of the perf.data file. Should the file lack the topology, the PMUs of this machine are scanned but only when the file was recorded on the same architecture, as PMU names like "cpu_core" are architecture specific.
--- tools/perf/util/evlist.c | 183 +++++++++++++++++++++++++++++++++ tools/perf/util/evlist.h | 3 + tools/perf/util/evsel.h | 1 + tools/perf/util/hist.h | 1 + tools/perf/util/parse-events.c | 4 +- tools/perf/util/pmu.c | 43 +++++++- tools/perf/util/pmu.h | 3 + tools/perf/util/symbol_conf.h | 1 + 8 files changed, 236 insertions(+), 3 deletions(-)
tools/perf/util/evlist.c +183 -0
@@ -7,6 +7,7 @@*/ #include "evlist.h" +#include <stdio.h> #include <errno.h> #include <inttypes.h> #include <signal.h>
@@ -53,6 +54,7 @@#include "event.h" #include "evsel.h" #include "evsel_fprintf.h" +#include "hist.h" #include "intel-tpebs.h" #include "metricgroup.h" #include "mmap.h"
@@ -125,12 +127,24 @@ struct evlist *evlist__new_default(const struct target *target, bool sample_callif (err) goto out_err; }else { +struct evsel *leader =NULL ; + +/* Fallback for cross-platform file analysis missing the topology header */ while ((pmu =perf_pmus__scan_core (pmu)) !=NULL ) {snprintf (buf,sizeof (buf),"%s/cycles/%s" , pmu->name, can_profile_kernel ?"P" :"Pu" ); err =parse_event (evlist, buf);if (err) goto out_err; + +if (!leader) { + leader =evlist__last (evlist); + }else { +struct evsel *last =evlist__last (evlist); + +if (last != leader) + last->first_wildcard_match = leader; + } } }
@@ -148,6 +162,175 @@ struct evlist *evlist__new_default(const struct target *target, bool sample_callreturn NULL ; } +/** +* evlist__hybrid_matches - find events that may be merged across core PMUs. +* @evlist: The evlist to check. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +* @link: Should the matches be recorded in first_wildcard_match? +* +* perf record, top, etc. set first_wildcard_match in the event parsing. The +* perf.data case (e.g. perf report) recomputes the first_wildcard_match using +* string matches for core events on hybrid systems. +*/ +static bool evlist__hybrid_matches (struct evlist *evlist,struct perf_env *env,bool link) +{ +struct evsel *pos; +unsigned int nr =0 ; +bool found = false; + +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos)) +continue ; + +if (pos->core.leader != &pos->core || pos->core.nr_members >1 ) +return false; + +if (pos->first_wildcard_match) + found = true; + nr++; + } + +/* No events found. */ +if (nr <=1 ) +return false; + +/* Found wildcard events from parsing. */ +if (found) +return true; + +/* Try to find matching core events and set the first_wildcard_match. */ +evlist__for_each_entry (evlist, pos) { +const char *pos_name; +const char *pos_match; +struct evsel *peer; + +if (evsel__is_dummy_event (pos) || pos->first_wildcard_match) +continue ; + + pos_name =evsel__name (pos); + pos_match = pos_name ?strchr (pos_name,'/' ) :NULL ; + +if (!pos_match) +continue ; + +/* If evsel->core.is_pmu_core missing in report, fallback to perf_env */ +if (!evsel__is_hybrid (pos)) { +if (!is_pmu_core_len (env, pos_name, pos_match - pos_name)) +continue ; + } + + peer = pos; +list_for_each_entry_continue (peer, &evlist__core (evlist)->entries, core.node) { +const char *peer_name; +const char *peer_match; + +if (evsel__is_dummy_event (peer) || peer->first_wildcard_match) +continue ; + + peer_name =evsel__name (peer); + peer_match = peer_name ?strchr (peer_name,'/' ) :NULL ; +if (!peer_match) +continue ; + +if (!evsel__is_hybrid (peer)) { +if (!is_pmu_core_len (env, peer_name, + peer_match - peer_name)) +continue ; + } + +if (strcmp (pos_match, peer_match)) +continue ; + + found = true; +if (!link) { +/* Just a test, don't modify the evlist. */ +return true; + } +/* +* Keep looking so that all the events of this name are +* linked, there may be more than 2 core PMUs. +*/ + peer->first_wildcard_match = pos; + } + } +return found; +} + +/** +* evlist__can_merge_hybrid - can events in the evlist be merged across core +* PMUs? The evlist isn't modified. +* @evlist: The evlist to check. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +*/ +bool evlist__can_merge_hybrid (struct evlist *evlist,struct perf_env *env) +{ +return evlist__hybrid_matches (evlist, env,/*link=*/ false); +} + +/* +* evlist__merge_hybrid - group hybrid events logically together. +* @evlist: The evlist containing events to merge. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +* +* Iterates through the evlist and merges associated hybrid events by assigning +* their first_wildcard_match as their core group leader, modifying their +* presentation for a single merged histogram view. +*/ +void evlist__merge_hybrid (struct evlist *evlist,struct perf_env *env) +{ +struct list_head new_list; +struct evsel *member, *mtmp; +struct evsel *pos, *tmp; +int idx =0 ; + +/* Compute first_wildcard_match for events that lack it, say from a file. */ +if (!evlist__hybrid_matches (evlist, env,/*link=*/ true)) +return ; + +evlist__for_each_entry_safe (evlist, tmp, pos) { +struct evsel *leader, *old_leader; + +if (evsel__is_dummy_event (pos)) +continue ; + +if (!pos->first_wildcard_match) +continue ; + + leader =evsel__leader (pos->first_wildcard_match); + old_leader =evsel__leader (pos); +if (old_leader == leader) +continue ; + +if (old_leader != pos) + old_leader->core.nr_members--; + pos->core.leader = &leader->core; + pos->merged_hybrid_group = true; +/* Base is 1 to natively represent the leader */ +if (leader->core.nr_members ==0 ) + leader->core.nr_members =1 ; + leader->core.nr_members++; + } + +INIT_LIST_HEAD (&new_list); + +while (!list_empty (&evlist__core (evlist)->entries)) { + pos = list_first_entry(&evlist__core(evlist)->entries, struct evsel, core.node); +list_move_tail (&pos->core.node, &new_list); + +list_for_each_entry_safe (member, mtmp, &evlist__core (evlist)->entries, core.node) { +if (member->core.leader == &pos->core) +list_move_tail (&member->core.node, &new_list); + } + } +list_splice_init (&new_list, &evlist__core (evlist)->entries); + +evlist__for_each_entry (evlist, pos) + pos->core.idx = idx++; +} +struct evlist *evlist__new_dummy (void ) {struct evlist *evlist =evlist__new ();
tools/perf/util/evlist.h +3 -0
@@ -331,6 +331,9 @@ static inline void evlist__set_selected(struct evlist *evlist, struct evsel *evsstruct evlist *evlist__new (void );struct evlist *evlist__new_default (const struct target *target,bool sample_callchains); +struct perf_env ; +bool evlist__can_merge_hybrid (struct evlist *evlist,struct perf_env *env); +void evlist__merge_hybrid (struct evlist *evlist,struct perf_env *env);struct evlist *evlist__new_dummy (void );struct evlist *evlist__get (struct evlist *evlist);void evlist__put (struct evlist *evlist);
tools/perf/util/evsel.h +1 -0
@@ -53,6 +53,7 @@ struct evsel {int id_pos;int is_pos;unsigned int sample_size; +bool merged_hybrid_group;/* * These fields can be set in the parse-events code or similar.
tools/perf/util/hist.h +1 -0
@@ -130,6 +130,7 @@ struct hists {struct hists_stats stats;u64 event_stream;u16 col_len[HISTC_NR_COLS ]; +bool merge_entries;bool has_callchains;int socket_filter;struct perf_hpp_list *hpp_list;
tools/perf/util/parse-events.c +2 -2
@@ -1683,7 +1683,7 @@ int parse_events_multi_pmu_add(struct parse_events_state *parse_state,strbuf_release (&sb); ok++; } -if (first_wildcard_match ==NULL ) +if (first_wildcard_match ==NULL && !list_empty (list)) first_wildcard_match = container_of(list->prev, struct evsel, core.node); }
@@ -1754,7 +1754,7 @@ int parse_events_multi_pmu_add_or_add_pmu(struct parse_events_state *parse_state ok++; parse_state->wild_card_pmus = true; } -if (first_wildcard_match ==NULL ) { +if (first_wildcard_match ==NULL && !list_empty (*listp)) { first_wildcard_match = container_of((*listp)->prev, struct evsel, core.node); }
tools/perf/util/pmu.c +42 -1
55 lines not on this page · raw message
tools/perf/util/pmu.h +3 -0
@@ -16,6 +16,7 @@struct evsel_config_term ;struct hashmap ;struct perf_cpu_map ; +struct perf_env ;struct print_callbacks ;enum {
@@ -289,6 +290,8 @@ int perf_pmu__for_each_format(struct perf_pmu *pmu, void *state, pmu_format_callu64 perf_pmu__format_unpack (unsigned long *format,u64 config_val);bool is_pmu_core (const char *name); +bool is_pmu_core_len (struct perf_env *env,const char *name,size_t len); +bool perf_pmu__supports_legacy_cache (const struct perf_pmu *pmu);bool perf_pmu__auto_merge_stats (const struct perf_pmu *pmu);bool perf_pmu__have_event (struct perf_pmu *pmu,const char *name);
tools/perf/util/symbol_conf.h +1 -0
@@ -38,6 +38,7 @@ enum symbol__weight_mode {for ((_weight) =WEIGHT_WEIGHT ; (_weight) <=WEIGHT_WEIGHT3 ; (_weight)++) struct symbol_conf { +bool hybrid_merge;bool nanosecs;unsigned short priv_size; bool try_vmlinux_path,
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 1/9] perf evlist: Implement evlist__can_merge_hybrid using first_wildcard_match
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 11 lines
Add logic to dynamically identify mergeable events spawned from the same wildcard alias via
first_wildcard_match.first_wildcard_matchis populated when events are parsed, inperf reportthe events come from a file and so thefirst_wildcard_matchis computed by matching event names.
evlist__can_merge_hybridonly tests whether merging is possible and doesn't modify the evlist, so that it may be used as a predicate, for example, to decide whether to display a hint. The linking of all the matching events is done byevlist__merge_hybrid.
Quoted 6 lines
is_pmu_core_lenidentifies core PMUs using the hybrid topology of the perf.data file. Should the file lack the topology, the PMUs of this machine are scanned but only when the file was recorded on the same architecture, as PMU names like "cpu_core" are architecture specific.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v2 2/9 perf ui hist: Add support for aggregated total_period and merging entries
In reply to Ian RogersAdd the histogram side of hybrid event merging.
When events are merged the entries of the other events are linked to the leader's entries as pairs, and a row is then displayed as the sum over an entry and its pairs. Teach the hpp code to work on that sum:
- The formatting and sorting routines add up the value of an entry and of each of its pairs, so a row shows, and is sorted by, the total across the core PMUs rather than the leader's value alone.
- Percentages are taken against the summed total_period of the merged hists. Computing each event's share against its own PMU's total would leave the column not adding up to 100%. hist_entry__get_percent_limit_merged() does the same for --percent-limit, which would otherwise compare against the wrong total and filter out the wrong entries.
- A header and column width are reserved for the "Total" the merged value is shown under.
evlist__merge_hists_hybrid() links the hists together and resorts the result so that it is ordered by the merged total.
hists__link() is run once for every event merged into the leader, so an entry can be offered as a pair more than once. hist_entry__add_pair() used list_add_tail() unconditionally, which corrupts the list when the entry is already on one. Return early if the entry is already paired with this one, and use list_move_tail() so that an entry linked elsewhere is moved rather than added a second time.
Merging is never combined with --hierarchy, which the merge_entries tests rely on.
--- tools/perf/ui/hist.c | 246 +++++++++++++++++++++++++++++++++++++++-- tools/perf/util/hist.h | 14 ++- 2 files changed, 252 insertions(+), 8 deletions(-)
tools/perf/ui/hist.c +239 -7
@@ -57,8 +57,9 @@ struct hpp_fmt_value { };static int __hpp__fmt (struct perf_hpp *hpp,struct hist_entry *he, -hpp_field_fn get_field,const char *fmt,int len, -hpp_snprint_fn print_fn,enum perf_hpp_fmt_type fmtype) +hpp_field_fn get_field,const char *fmtstr,int len, +hpp_snprint_fn print_fn,enum perf_hpp_fmt_type fmtype, +struct perf_hpp_fmt *fmt__maybe_unused ) {int ret =0 ;struct hists *hists = he->hists;
@@ -98,13 +99,50 @@ static int __hpp__fmt(struct perf_hpp *hpp, struct hist_entry *he, } } +/* Note, merge_entries implies !symbol_conf.report_hierarchy. */ +if (he->hists->merge_entries) { +u64 total_val =0 ; +u64 total_samples =0 ; +u64 total_period =0 ; + +for (i =0 ; i < nr_members; i++) { +struct evsel *member_evsel =hists_to_evsel (values[i].hists); + +struct evsel *he_evsel =hists_to_evsel (he->hists); + +if (member_evsel != he_evsel && + member_evsel->first_wildcard_match != he_evsel) +continue ; + + total_val += values[i].val; + total_samples += values[i].samples; + total_period += fmtype ==PERF_HPP_FMT_TYPE__PERCENT ? +hists__total_period (values[i].hists) : +hists__total_latency (values[i].hists); + } + +if (fmtype ==PERF_HPP_FMT_TYPE__PERCENT || fmtype ==PERF_HPP_FMT_TYPE__LATENCY ) { +double percent =0.0 ; + +if (total_period) + percent =100.0 * total_val / total_period; + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, percent); + }else if (fmtype ==PERF_HPP_FMT_TYPE__AVERAGE ) { +double avg = total_samples ? (1.0 * total_val / total_samples) :0 ; + + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, avg); + }else { + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, total_val); + } + } +for (i =0 ; i < nr_members; i++) {if (symbol_conf.skip_empty && values[i].hists->stats.nr_samples ==0 )continue ; ret +=__hpp__fmt_print (hpp, values[i].hists, values[i].val, - values[i].samples, fmt, len, + values[i].samples, fmtstr, len, print_fn, fmtype); }
@@ -129,7 +167,7 @@ int hpp__fmt(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,if (symbol_conf.field_sep) {return __hpp__fmt (hpp, he, get_field, fmtstr,1 , - print_fn, fmtype); + print_fn, fmtype, fmt); }if (fmtype ==PERF_HPP_FMT_TYPE__PERCENT || fmtype ==PERF_HPP_FMT_TYPE__LATENCY )
@@ -137,7 +175,7 @@ int hpp__fmt(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,else len -=1 ; -return __hpp__fmt (hpp, he, get_field, fmtstr, len, print_fn, fmtype); +return __hpp__fmt (hpp, he, get_field, fmtstr, len, print_fn, fmtype, fmt); } int hpp__fmt_acc(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,
@@ -287,6 +325,35 @@ static int __hpp__sort(struct hist_entry *a, struct hist_entry *b,return __hpp__group_sort_idx (a, b, get_field, symbol_conf.group_sort_idx); } +/* +* Relies on merge_entries being only enabled if there are +* only matching events. If that is ever relaxed will need +* more logic here. merge_entries also implies that +* symbol_conf.report_hierarchy is false. +*/ +if (a->hists->merge_entries && b->hists->merge_entries) { +u64 val_a =get_field (a), val_b =get_field (b); +struct hist_entry *pair; +struct evsel *evsel_a =hists_to_evsel (a->hists); +struct evsel *evsel_b =hists_to_evsel (b->hists); + +list_for_each_entry (pair, &a->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_a) + val_a +=get_field (pair); + } +list_for_each_entry (pair, &b->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_b) + val_b +=get_field (pair); + } + + ret =field_cmp (val_a, val_b); +if (ret) +return ret; + } ret =field_cmp (get_field (a),get_field (b));if (ret || !symbol_conf.event_group)
@@ -323,7 +390,29 @@ static int __hpp__sort_acc(struct hist_entry *a, struct hist_entry *b,/* * Put caller above callee when they have equal period. */ - ret =field_cmp (get_field (a),get_field (b)); +if (a->hists->merge_entries && b->hists->merge_entries) { +u64 val_a =get_field (a), val_b =get_field (b); +struct hist_entry *pair; +struct evsel *evsel_a =hists_to_evsel (a->hists); +struct evsel *evsel_b =hists_to_evsel (b->hists); + +list_for_each_entry (pair, &a->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_a) + val_a +=get_field (pair); + } +list_for_each_entry (pair, &b->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_b) + val_b +=get_field (pair); + } + + ret =field_cmp (val_a, val_b); + }else { + ret =field_cmp (get_field (a),get_field (b)); + }if (ret)return ret;
@@ -386,6 +475,8 @@ static int hpp__width_fn(struct perf_hpp_fmt *fmt, evsel__hists(pos)->stats.nr_samples) nr++; } +if (hists->merge_entries) + nr++;/* Add 1 extra unit of width generically for the 'Total' */ len =max (len, nr * fmt->len); }
@@ -403,8 +494,38 @@ static int hpp__header_fn(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,int len =hpp__width_fn (fmt, hpp, hists);const char *hdr ="" ; -if (line == hists->hpp_list->nr_header_lines -1 ) +if (line == hists->hpp_list->nr_header_lines -1 ) { hdr = fmt->name; +if (hists->merge_entries) { +int w =0 ; +int f_len = fmt->user_len ?: fmt->len; +struct evsel *pos, *evsel =hists_to_evsel (hists); +int string_len = f_len; + +for_each_group_evsel (pos, evsel) { +if (symbol_conf.skip_empty && +evsel__hists (pos)->stats.nr_samples ==0 ) +continue ; + string_len += f_len; + } + +if (len > string_len) { + w +=scnprintf (hpp->buf + w, hpp->size - w,"%*s" , + len - string_len,"" ); + } + + w +=scnprintf (hpp->buf + w, hpp->size - w,"%*.*s" , + f_len, f_len, fmt->name); +for_each_group_evsel (pos, evsel) { +if (symbol_conf.skip_empty && +evsel__hists (pos)->stats.nr_samples ==0 ) +continue ; + w +=scnprintf (hpp->buf + w, hpp->size - w," %*.*s" , + f_len -1 , f_len -1 ,evsel__name (pos)); + } +return w; + } + }return scnprintf (hpp->buf, hpp->size,"%*s" , len, hdr); }
@@ -1271,3 +1392,114 @@ int perf_hpp__alloc_mem_stats(struct perf_hpp_list *list, struct evlist *evlist) }return 0 ; } + +float hist_entry__get_percent_limit_merged (struct hist_entry *he) +{ +struct hist_entry *pair; +u64 period = he->stat.period; +u64 total_period =hists__total_period (he->hists); +struct evsel *evsel =hists_to_evsel (he->hists); +struct evsel *pos; + +/* Accumulate global total_period across all merged hists matching hybrid type */ +for_each_group_member (pos, evsel) { +if (pos->first_wildcard_match == evsel) + total_period +=hists__total_period (evsel__hists (pos)); + } + +if (unlikely (total_period ==0 )) +return 0 ; + +if (symbol_conf.cumulate_callchain) { + period = he->stat_acc->period; +list_for_each_entry (pair, &he->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel) + period += pair->stat_acc->period; + } + }else { +/* Accumulate symbol specific period across pairs matching hybrid type */ +list_for_each_entry (pair, &he->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel) + period += pair->stat.period; + } + } + +return period *100.0 / total_period; +} + +void evlist__merge_hists_hybrid (struct evlist *evlist,bool refresh_hists) +{ +struct evsel *pos; +struct evsel *member; +bool hybrid_group; + +/* +* Merged hists display all the events of a group in a single set of +* entries, which the hierarchy display has no way to render. Keeping +* hists->merge_entries false here means the rest of the display code +* can assume merge_entries implies !symbol_conf.report_hierarchy. +*/ +if (symbol_conf.report_hierarchy) +return ; + +/* Set merge_entries flag strictly on leaders formulated by hybrid topology */ +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos)) +continue ; + +if (pos->core.leader == &pos->core && pos->core.nr_members >1 ) { + hybrid_group = false; + +if (pos->first_wildcard_match || pos->merged_hybrid_group) { + hybrid_group = true; + }else { +for_each_group_member (member, pos) { +if (member->first_wildcard_match || + member->merged_hybrid_group) { + hybrid_group = true; +break ; + } + } + } + +if (hybrid_group) { +evsel__hists (pos)->merge_entries = true; + symbol_conf.event_group = true; + symbol_conf.hybrid_merge = true; +if (!pos->group_name) { + pos->group_name =strdup ("Merged hybrid events" ); +if (!pos->group_name) +pr_warning ("Failed to allocate hybrid group name\n" ); + } + } + } + } + +if (!refresh_hists) +return ; + +evlist__for_each_entry (evlist, pos) { +/* Match histograms dynamically since parsing happened before group toggling */ +if (symbol_conf.event_group && !evsel__is_group_leader (pos)) { +struct hists *leader_hists =evsel__hists (evsel__leader (pos)); +struct hists *hists =evsel__hists (pos); + +hists__match (leader_hists, hists); +hists__link (leader_hists, hists); + } + } + +/* Now that links are formed, safely resort the active tree so the UI renders accurately */ +if (symbol_conf.event_group) { +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos) || !evsel__is_group_leader (pos)) +continue ; +if (pos->core.nr_members >1 &&evsel__hists (pos)->merge_entries) +hists__output_resort (evsel__hists (pos),NULL ); + } + } +}
tools/perf/util/hist.h +13 -1
@@ -339,7 +339,13 @@ static inline struct hist_entry *hist_entry__next_pair(struct hist_entry *he)static inline void hist_entry__add_pair (struct hist_entry *pair,struct hist_entry *he) { -list_add_tail (&pair->pairs.node, &he->pairs.head); +struct list_head *pos; + +list_for_each (pos, &he->pairs.head) { +if (pos == &pair->pairs.node) +return ;/* Already paired */ + } +list_move_tail (&pair->pairs.node, &he->pairs.head); } struct hist_entry *hists__add_entry(struct hists *hists,
@@ -434,11 +440,17 @@ void hists__match(struct hists *leader, struct hists *other);int hists__link (struct hists *leader,struct hists *other);int hists__unlink (struct hists *hists); +float hist_entry__get_percent_limit_merged (struct hist_entry *he); +void evlist__merge_hists_hybrid (struct evlist *evlist,bool refresh); +static inline float hist_entry__get_percent_limit (struct hist_entry *he) {u64 period = he->stat.period;u64 total_period =hists__total_period (he->hists); +if (he->hists->merge_entries) +return hist_entry__get_percent_limit_merged (he); +if (unlikely (total_period ==0 ))return 0 ;
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 2/9] perf ui hist: Add support for aggregated total_period and merging entries
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 17 lines
Add the histogram side of hybrid event merging.
When events are merged the entries of the other events are linked to the leader's entries as pairs, and a row is then displayed as the sum over an entry and its pairs. Teach the hpp code to work on that sum:
- The formatting and sorting routines add up the value of an entry and of each of its pairs, so a row shows, and is sorted by, the total across the core PMUs rather than the leader's value alone.
- Percentages are taken against the summedtotal_periodof the merged hists. Computing each event's share against its own PMU's total would leave the column not adding up to 100%.hist_entry__get_percent_limit_merged()does the same for --percent-limit, which would otherwise compare against the wrong total and filter out the wrong entries.
[ ... ]
Quoted 4 lines
Merging is never combined with --hierarchy, which the
merge_entriestests rely on.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerPATCH v2 3/9 perf tools: Expose opt-in --hybrid-merge
In reply to Ian RogersAdd --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
evlist__merge_hybrid() links the events and evlist__merge_hists_hybrid() links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.
Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
--- tools/perf/Documentation/perf-report.txt | 5 +++++ tools/perf/Documentation/perf-top.txt | 5 +++++ tools/perf/builtin-report.c | 19 +++++++++++++++++++ tools/perf/builtin-top.c | 17 +++++++++++++++++ tools/perf/util/symbol.c | 1 + 5 files changed, 47 insertions(+)
tools/perf/Documentation/perf-report.txt +5 -0
@@ -578,6 +578,11 @@ include::itrace.txt[] --raw-trace:: When displaying traceevent output, do not use print fmt or plugins. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + -H:: --hierarchy:: Enable hierarchical output. In the hierarchy mode, each sort key groups
tools/perf/Documentation/perf-top.txt +5 -0
@@ -49,6 +49,11 @@ Default is to monitor all CPUS. encoding with the layout of the event control registers as described by entries in /sys/bus/event_source/devices/cpu/format/*. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + --filter=<filter>:: Event filter. This option should follow an event selector (-e). For syntax see linkperf:perf-record[1].
tools/perf/builtin-report.c +19 -0
@@ -1114,6 +1114,17 @@ static int __cmd_report(struct report *rep)evlist__for_each_entry (session->evlist, pos) rep->nr_entries +=evsel__hists (pos)->nr_entries; +if (symbol_conf.hybrid_merge) { +struct perf_env *env =perf_session__env (session); + +if (evlist__can_merge_hybrid (session->evlist, env)) { +evlist__merge_hybrid (session->evlist, env); +evlist__merge_hists_hybrid (session->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } +if (use_browser ==0 ) {if (verbose >3 )perf_session__fprintf (session, stdout);
@@ -1449,6 +1460,8 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",
@@ -1548,6 +1561,12 @@ int cmd_report(int argc, const char **argv) report.symbol_filter_str = argv[0 ]; } +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style)
tools/perf/builtin-top.c +17 -0
@@ -1336,6 +1336,15 @@ static int __cmd_top(struct perf_top *top)if (!target__none (&opts->target))evlist__enable (top->evlist); +if (symbol_conf.hybrid_merge) { +if (evlist__can_merge_hybrid (top->evlist,/*env=*/ NULL )) { +evlist__merge_hybrid (top->evlist,/*env=*/ NULL ); +evlist__merge_hists_hybrid (top->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } + ret =-1 ;if (pthread_create (&thread_process,NULL , process_thread, top)) {ui__error ("Could not create process thread.\n" );
@@ -1491,6 +1500,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),
@@ -1679,6 +1690,12 @@ int cmd_top(int argc, const char **argv)if (argc)usage_with_options (top_usage, options); +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style) {
tools/perf/util/symbol.c +1 -0
@@ -74,6 +74,7 @@ struct symbol_conf symbol_conf = { .symfs = "", .symfs_layout_flat = false, .event_group = true, + .hybrid_merge = false, .inline_name = true, .res_sample = 0, .addr2line_timeout_ms = 5 * 1000,
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 3/9] perf tools: Expose opt-in --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Add --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
evlist__merge_hybrid()links the events andevlist__merge_hists_hybrid()links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.
Quoted 5 lines
Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Ian Rogers
reviewerPATCH v2 4/9 perf Documentation: Add tip for hybrid event merging
In reply to Ian RogersAdd informative text outlining the IPC imbalances associated with merging cross-hybrid core events like cycles.
--- tools/perf/Documentation/tips.txt | 1 + 1 file changed, 1 insertion(+)
tools/perf/Documentation/tips.txt +1 -0
@@ -66,3 +66,4 @@ For latency profiling, try: perf record/report --latency For parallelism histogram, try: perf report --hierarchy --sort latency,parallelism,comm,symbol To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s +When merging events like cycles, different core frequencies and instructions per cycle mean the counts may not fairly reflect time spent in a function.
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 4/9] perf Documentation: Add tip for hybrid event merging
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
Add informative text outlining the IPC imbalances associated with merging cross-hybrid core events like cycles.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Ian Rogers
reviewerPATCH v2 5/9 perf evlist: Toggle merged_hybrid_group properly in enable/disable
In reply to Ian RogersMerging makes the merged event the leader of the events of the other core PMUs so their histograms can be linked, but the result isn't a real event group. Each event keeps its own file descriptor and has to be enabled and disabled in its own right.
__evlist__enable(), __evlist__disable() and evlist__is_enabled() skip anything that isn't a group leader, and the first two then walk the group members of the events they do act on. For a merged set that is
backwards: the members are skipped by the leader test, so their file descriptors are never touched, and the walk over the leader's members only updates the bookkeeping in evsel->disabled.
Treat an evsel with merged_hybrid_group set as a leader so that it is enabled and disabled in its own right, and don't walk the members of a merged leader.
perf top enables and disables by event name. A merged member carries the name of its own PMU rather than the name that was asked for, so also match it against its leader's name, otherwise naming the event toggles only part of the merged set.
--- tools/perf/util/evlist.c | 47 +++++++++++++++++++++++++++++----------- 1 file changed, 34 insertions(+), 13 deletions(-)
tools/perf/util/evlist.c +34 -13
@@ -753,7 +753,9 @@ static bool evlist__is_enabled(struct evlist *evlist)struct evsel *pos;evlist__for_each_entry (evlist, pos) { -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if (!pos->core.fd) +continue ; +if (!evsel__is_group_leader (pos) && !pos->merged_hybrid_group)continue ;/* If at least one event is enabled, evlist is enabled. */ if (!pos->disabled)
@@ -767,14 +769,19 @@ static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool exclstruct evsel *pos, *member;struct evlist_cpu_iterator evlist_cpu_itr;bool has_imm = false; +bool match;/* Disable 'immediate' events last */ for (int imm =0 ; imm <=1 ; imm++) {evlist__for_each_cpu (evlist_cpu_itr, evlist) { pos = evlist_cpu_itr.evsel; -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (pos->disabled || !evsel__is_group_leader (pos) || !pos->core.fd) +if (pos->disabled || (!evsel__is_group_leader (pos) && + !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;
@@ -789,16 +796,21 @@ static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool excl }evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ; pos->disabled = true; -for_each_group_member (member, pos) - member->disabled = true; +if (!pos->merged_hybrid_group) { +for_each_group_member (member, pos) + member->disabled = true; + } }/*
@@ -827,28 +839,37 @@ static void __evlist__enable(struct evlist *evlist, char *evsel_name, bool excl_ {struct evsel *pos, *member;struct evlist_cpu_iterator evlist_cpu_itr; +bool match;evlist__for_each_cpu (evlist_cpu_itr, evlist) { pos = evlist_cpu_itr.evsel; -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;evsel__enable_cpu (pos, evlist_cpu_itr.cpu_map_idx); }evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ; pos->disabled = false; +if (!pos->merged_hybrid_group) { +for_each_group_member (member, pos) + member->disabled = false; + } -for_each_group_member (member, pos) - member->disabled = false; }/*
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 5/9] perf evlist: Toggle merged_hybrid_group properly in enable/disable
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] perf: dead code in evlist enable/disable due to flag misunderstanding --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf: dead code in evlist enable/disable due to flag misunderstanding --
commit a1c6041372bff09cc374228542c586fa596b3401
Author: Ian Rogers <[email protected]>
perf evlist: Toggle merged_hybrid_group properly in enable/disable
Merging makes the merged event the leader of the events of the other core PMUs so their histograms can be linked, but the result isn't a real event group. The commit treats an evsel with merged_hybrid_group set as a leader so that it is enabled and disabled in its own right, and updates the logic to avoid walking the members of a merged leader.
Quoted 4 lines
tools/perf/util/evlist.c +0
-0
[ ... ]
Quoted 13 lines
tools/perf/util/evlist.c +5
-2@@ -789,16 +796,21 @@ static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool excl }evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;
tools/perf/util/evlist.c static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool excl · line 813
Quoted 10 lines
tools/perf/util/evlist.c +4
-2if (excl_dummy &&evsel__is_dummy_event (pos))continue ; pos->disabled = true; -for_each_group_member (member, pos) - member->disabled = true; +if (!pos->merged_hybrid_group) { +for_each_group_member (member, pos) + member->disabled = true; + }
[Severity: Medium]
Does this logic correctly prevent walking the members of a merged leader?
The commit message states "don't walk the members of a merged leader". However, the merged_hybrid_group flag is only set on merged members, never on the leader. When __evlist__disable() processes a merged leader, pos->merged_hybrid_group is false, so this code will walk the members anyway.
Conversely, when processing a merged member, the flag is true, skipping the block. But since a member has no members of its own, the walk would have been empty anyway.
Does this check correctly match the intent, or should it be testing if the current group leader is a merged leader instead?
Quoted 3 lines
tools/perf/util/evlist.c +0
-0}/*
[ ... ]
Quoted 1 line
tools/perf/util/evlist.c +0
-0@@ -827,28 +839,37 @@ static void __evlist__enable(struct evlist *evlist, char *evsel_name, bool excl_
[ ... ]
Quoted 7 lines
tools/perf/util/evlist.c +4
-1evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ;
tools/perf/util/evlist.c static void __evlist__enable(struct evlist *evlist, char *evsel_name, bool excl_ · line 871
Quoted 10 lines
tools/perf/util/evlist.c +5
-1-if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ; pos->disabled = false; +if (!pos->merged_hybrid_group) { +for_each_group_member (member, pos) + member->disabled = false; + }
[Severity: Medium]
Does this code also suffer from the same flag misunderstanding in __evlist__enable()?
If pos->merged_hybrid_group is only set on members, this check will similarly fail to prevent the merged leader from walking its members.
Quoted 6 lines
tools/perf/util/evlist.c +0
-2-for_each_group_member (member, pos) - member->disabled = false; }/*
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Ian Rogers
reviewerPATCH v2 6/9 perf tools: Add TUI hints for --hybrid-merge
In reply to Ian RogersAdd dynamic TUI hints prompting users to restart with --hybrid-merge when heterogeneous core PMU events are populated in the sample view.
--- tools/perf/Documentation/tips.txt | 1 + tools/perf/ui/browsers/hists.c | 7 +++++-- 2 files changed, 6 insertions(+), 2 deletions(-)
tools/perf/Documentation/tips.txt +1 -0
@@ -67,3 +67,4 @@ For parallelism histogram, try: perf report --hierarchy --sort latency,paralleli To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s When merging events like cycles, different core frequencies and instructions per cycle mean the counts may not fairly reflect time spent in a function. +Use --hybrid-merge to merge matching events from hybrid cores into one view
tools/perf/ui/browsers/hists.c +5 -2
@@ -3580,9 +3580,12 @@ static int perf_evsel_menu__run(struct evsel_menu *menu,const char *title ="Available samples" ;int delay_secs = hbt ? hbt->refresh :0 ;int key; +const char *help_text ="ESC: exit, ENTER|->: Browse histograms" ; -if (ui_browser__show (&menu->b, title, -"ESC: exit, ENTER|->: Browse histograms" ) <0 ) +if (!symbol_conf.hybrid_merge &&evlist__can_merge_hybrid (evlist, menu->env)) + help_text ="ESC: exit, ENTER|->: Browse. Try --hybrid-merge to combine events." ; + +if (ui_browser__show (&menu->b, title, help_text) <0 )return -1 ;while (1 ) {
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 6/9] perf tools: Add TUI hints for --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
Add dynamic TUI hints prompting users to restart with --hybrid-merge when heterogeneous core PMU events are populated in the sample view.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Ian Rogers
reviewerPATCH v2 7/9 perf config: Add core.hybrid-merge to configure event merging
In reply to Ian RogersProvide a core.hybrid-merge configuration option in .perfconfig to allow enabling hybrid event aggregation by default, avoiding the need to pass --hybrid-merge explicitly on every invocation.
The config value is a default rather than an explicit request, so with --hierarchy it is ignored with a warning, while giving both --hierarchy and --hybrid-merge on the command line remains an error.
perf stat has its own merging options for counting, the sampling core.hybrid-merge deliberately doesn't alter it and its --hybrid-merge must still be given explicitly.
--- tools/perf/Documentation/perf-config.txt | 8 ++++++++ tools/perf/builtin-report.c | 16 +++++++++++----- tools/perf/builtin-top.c | 22 +++++++++++++++------- tools/perf/util/config.c | 10 ++++++++++ 4 files changed, 44 insertions(+), 12 deletions(-)
tools/perf/Documentation/perf-config.txt +8 -0
@@ -217,6 +217,14 @@ core.*:: Sets a timeout (in milliseconds) for parsing 'addr2line' output. The default timeout is 5s. + hybrid-merge:: + Merge matching events from all hybrid core PMUs into one display + by default. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. This applies to + 'perf report' and 'perf top', it is ignored with '--hierarchy' and + doesn't alter 'perf stat' where '--hybrid-merge' must be given + explicitly. + tui.*, gtk.*:: Subcommands that can be configured here are 'top', 'report' and 'annotate'. These values are booleans, for example:
tools/perf/builtin-report.c +11 -5
@@ -1324,6 +1324,7 @@ int cmd_report(int argc, const char **argv)int branch_mode =-1 ;int last_key =0 ;bool branch_call_mode = false; +bool hybrid_merge_set = false;#define CALLCHAIN_DEFAULT_OPT "graph,0.5,caller,function,percent"static const char report_callchain_help[] ="Display call graph (stack chain/backtrace):\n\n" CALLCHAIN_REPORT_HELP
@@ -1460,8 +1461,8 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), -OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, -"merge the same event across hybrid core PMUs" ), +OPT_BOOLEAN_SET (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, &hybrid_merge_set, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",
@@ -1562,9 +1563,14 @@ int cmd_report(int argc, const char **argv) }if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { -pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); - ret = -EINVAL ; - goto exit; +if (hybrid_merge_set) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +/* A config file default shouldn't fail an explicit option. */ +pr_warning ("core.hybrid-merge ignored: --hierarchy cannot display merged hybrid events\n" ); + symbol_conf.hybrid_merge = false; }if (disassembler_style) {
tools/perf/builtin-top.c +15 -7
@@ -1314,9 +1314,11 @@ static int __cmd_top(struct perf_top *top) }/* -* Use global stat_config that is zero meaning aggr_mode is AGGR_NONE -* and hybrid_merge is false. +* Use global stat_config that is zero meaning aggr_mode is AGGR_NONE. +* Merging affects the event names as merged events share a name, all +* other stat_config behavior is unwanted here. */ + stat_config.hybrid_merge = symbol_conf.hybrid_merge;evlist__uniquify_evsel_names (top->evlist, &stat_config); ret =perf_top__start_counters (top);if (ret)
@@ -1493,6 +1495,7 @@ int cmd_top(int argc, const char **argv) .evlistp = &top.evlist, };bool branch_call_mode = false; +bool hybrid_merge_set = false;struct record_opts *opts = &top.record_opts;struct target *target = &opts->target;const char *disassembler_style =NULL , *objdump_path =NULL , *addr2line_path =NULL ;
@@ -1500,8 +1503,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), -OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, -"merge the same event across hybrid core PMUs" ), +OPT_BOOLEAN_SET (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, &hybrid_merge_set, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),
@@ -1691,9 +1694,14 @@ int cmd_top(int argc, const char **argv)usage_with_options (top_usage, options);if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { -pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); - status = -EINVAL ; - goto out_put_evlist; +if (hybrid_merge_set) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +/* A config file default shouldn't fail an explicit option. */ +pr_warning ("core.hybrid-merge ignored: --hierarchy cannot display merged hybrid events\n" ); + symbol_conf.hybrid_merge = false; }if (disassembler_style) {
tools/perf/util/config.c +10 -0
@@ -470,6 +470,16 @@ static int perf_default_core_config(const char *var, const char *value)if (!strcmp (var,"core.addr2line-disable-warn" )) symbol_conf.addr2line_disable_warn =perf_config_bool (var, value); +if (!strcmp (var,"core.hybrid-merge" )) { +/* +* Note, this is for sampling tools like perf report and top. +* perf stat has its own merging options and the +* stat_config.hybrid_merge of "perf stat --hybrid-merge" is +* deliberately not set here. +*/ + symbol_conf.hybrid_merge =perf_config_bool (var, value); + } +/* Add other config variables here. */ return 0 ; }
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 7/9] perf config: Add core.hybrid-merge to configure event merging
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
Provide a core.hybrid-merge configuration option in .perfconfig to allow enabling hybrid event aggregation by default, avoiding the need to pass --hybrid-merge explicitly on every invocation.
The config value is a default rather than an explicit request, so with --hierarchy it is ignored with a warning, while giving both --hierarchy and --hybrid-merge on the command line remains an error.
Quoted 5 lines
perf stat has its own merging options for counting, the sampling core.hybrid-merge deliberately doesn't alter it and its --hybrid-merge must still be given explicitly.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Ian Rogers
reviewerPATCH v2 8/9 perf test: Expand tests for --hybrid-merge
In reply to Ian RogersAdd a "Hybrid event merging" test that checks evlist__can_merge_hybrid() and evlist__merge_hybrid() directly, covering the events a wildcard expanded to over two core PMUs, the same over three, and events that must not be merged.
Add a report_hybrid_merge.sh shell test that records and then reports with --hybrid-merge, and extend top.sh to cover perf top with it.
--- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/hybrid-merge.c | 206 ++++++++++++++++++ tools/perf/tests/shell/report_hybrid_merge.sh | 121 ++++++++++ tools/perf/tests/shell/top.sh | 73 ++++++- tools/perf/tests/tests.h | 1 + 6 files changed, 392 insertions(+), 11 deletions(-) create mode 100644 tools/perf/tests/hybrid-merge.c create mode 100755 tools/perf/tests/shell/report_hybrid_merge.sh
tools/perf/tests/Build +1 -0
@@ -65,6 +65,7 @@ perf-test-y += perf-time-to-tsc.o perf-test-y += dlfilter-test.o perf-test-y += sigtrap.o perf-test-y += event_groups.o +perf-test-y += hybrid-merge.o perf-test-y += symbols.o perf-test-y += util.o perf-test-y += hwmon_pmu.o
tools/perf/tests/builtin-test.c +1 -0
@@ -149,6 +149,7 @@ static struct test_suite *generic_tests[] = { &suite__dlfilter, &suite__sigtrap, &suite__event_groups, + &suite__hybrid_merge, &suite__symbols, &suite__util, &suite__subcmd_help,
tools/perf/tests/hybrid-merge.c new file +206 -0
206 lines not on this page · raw message
tools/perf/tests/shell/report_hybrid_merge.sh new file +121 -0
121 lines not on this page · raw message
tools/perf/tests/shell/top.sh +62 -11
89 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -177,6 +177,7 @@ DECLARE_SUITE(perf_time_to_tsc);DECLARE_SUITE (dlfilter);DECLARE_SUITE (sigtrap);DECLARE_SUITE (event_groups); +DECLARE_SUITE (hybrid_merge);DECLARE_SUITE (symbols);DECLARE_SUITE (util);DECLARE_SUITE (uncore_event_sorting);
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 8/9] perf test: Expand tests for --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Add a "Hybrid event merging" test that checks
evlist__can_merge_hybrid()andevlist__merge_hybrid()directly, covering the events a wildcard expanded to over two core PMUs, the same over three, and events that must not be merged.Add a
report_hybrid_merge.shshell test that records and then reports with --hybrid-merge, and extendtop.shto cover perf top with it.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Ian Rogers
reviewerPATCH v2 9/9 perf test: Isolate test suite from user .perfconfig natively
In reply to Ian RogersRunning 'perf test' should not inherit the user's ~/.perfconfig environment as customizing core properties (like core.hybrid-merge) will trivially break stdout matching checks across the shell suite.
Explicitly set PERF_CONFIG to /dev/null inside cmd_test to globally sandbox the environment for the entire test workflow. Test specific config, like annotate.objdump, is read before the sandboxing so that 'perf test' still honors it.
--- tools/perf/tests/builtin-test.c | 49 +++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 5 deletions(-)
tools/perf/tests/builtin-test.c +44 -5
@@ -14,6 +14,8 @@#include <stdlib.h> #include <string.h> +#include "util/config.h" +#include <dirent.h> #include <linux/kernel.h> #include <linux/string.h>
@@ -1655,11 +1657,30 @@ static int run_workload(const char *work, int argc, const char **argv)return -1 ; } +/* +* Owns the string test_objdump_path points at when it came from the config. It +* is reachable for the lifetime of the process so leak checking won't report +* it. +*/ +static char *test_objdump_config_path; +static int perf_test__config (const char *var,const char *value,void *data__maybe_unused ) { -if (!strcmp (var,"annotate.objdump" )) - test_objdump_path = value; +if (!strcmp (var,"annotate.objdump" )) { +/* +* The config, and so value, is freed by perf_config__exit() +* below, take a copy that lives as long as the tests. +*/ +char *dup =strdup (value); + +if (!dup) +return -ENOMEM ; + +free (test_objdump_config_path); + test_objdump_config_path = dup; + test_objdump_path = dup; + }return 0 ; }
@@ -1755,13 +1776,31 @@ int cmd_test(int argc, const char **argv) };const char *const test_subcommands[] = {"list" ,NULL };struct intlist *skiplist =NULL ; -int ret =hists__init ();struct test_suite **suites; +int ret; -if (ret <0 ) + ret =hists__init (); +if (ret <0 )return ret; -perf_config (perf_test__config,NULL ); +/* Read test related config, like annotate.objdump, before isolating. */ + ret =perf_config (perf_test__config,NULL ); +if (ret <0 ) +return ret; + +/* +* Isolate the test suite from the user's ~/.perfconfig to ensure +* behavior isn't impacted by custom variables (e.g. core.hybrid-merge). +* Setting PERF_CONFIG isolates the perf commands run by the shell +* tests, config_exclusive_filename does the same for this process. +* Config values already read by main's perf_config are reset here, so +* far only the hybrid-merge value is known to alter test output. +*/ +if (setenv ("PERF_CONFIG" ,"/dev/null" ,1 ) <0 ) +return -1 ; + config_exclusive_filename ="/dev/null" ; +perf_config__exit (); + symbol_conf.hybrid_merge = false;/* Unbuffered output */ setvbuf (stdout,NULL , _IONBF,0 );
Signature · 1 line
2.55.0.1032.g73a4cd73de-goog
[email protected]
AI reviewRe: [PATCH v2 9/9] perf test: Isolate test suite from user .perfconfig natively
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Running 'perf test' should not inherit the user's ~/.perfconfig environment as customizing core properties (like core.hybrid-merge) will trivially break stdout matching checks across the shell suite.
Explicitly set
PERF_CONFIGto/dev/nullinsidecmd_testto globally sandbox the environment for the entire test workflow. Test specific config, like annotate.objdump, is read before the sandboxing so that 'perf test' still honors it.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Ian Rogers
reviewerv3 cover perf report/top: Add opt-in --hybrid-merge
In reply to Ian RogersOn a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error.
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
The events to merge are found with first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.
Changes in v3:
- Patch 5: remove the redundant if (!pos->merged_hybrid_group) guard around the group member walk in __evlist__enable() and __evlist__disable(), and update the commit message. In v2, that guard claimed to skip walking the members of a merged leader, but merged_hybrid_group is only set on merged members (which have no group members to walk), never on the leader itself.
Changes in v2:
- The interactive 'M' keystroke that toggled merging in the hists browser is gone. Merging is now asked for on the command line or through core.hybrid-merge, and the browser instead points out that --hybrid-merge is available when there are events it could merge.
- Fold "perf ui hist: Format group headers iteratively based on proportional visual allocations" into patch 2, as the two only make sense together.
- New patch 5, so that enabling and disabling a merged set touches every event in it. The merged events are made members of a leader but are not a real group, so each still has to be toggled in its own right.
- New patch 7, adding core.hybrid-merge.
- New patch 9, so that "perf test" doesn't inherit the user's ~/.perfconfig. Without it, setting core.hybrid-merge breaks the output matching of unrelated shell tests.
- Patch 8 adds a C test for the merging decision and a report shell test, rather than only extending the perf top test.
- Rewrite the commit messages of patches 2, 3, 5 and 8 to describe the problem being solved rather than the diff.
Testing:
- New "Hybrid event merging" perf test with three subtests: merging events of 2 core PMUs, of 3 core PMUs, and events that must not be merged.
- New "perf report hybrid merge" shell test, and the perf top shell test extended.
- perf test 13, 17, 27, 30, 31, 66, 67, 68, 78, 102, 113 and 172 pass, covering the hists, --hierarchy and addr2line paths this touches.
- Builds in 13 configurations, and every patch builds on its own so the series stays bisectable.
Ian Rogers (9):
perf evlist: Implement evlist__can_merge_hybrid using
first_wildcard_match
perf ui hist: Add support for aggregated total_period and merging
entries
perf tools: Expose opt-in --hybrid-merge
perf Documentation: Add tip for hybrid event merging perf evlist: Toggle merged_hybrid_group properly in enable/disable perf tools: Add TUI hints for --hybrid-merge
perf config: Add core.hybrid-merge to configure event merging perf test: Expand tests for --hybrid-merge
perf test: Isolate test suite from user .perfconfig natively
tools/perf/Documentation/perf-config.txt | 8 + tools/perf/Documentation/perf-report.txt | 5 + tools/perf/Documentation/perf-top.txt | 5 + tools/perf/Documentation/tips.txt | 2 + tools/perf/builtin-report.c | 25 ++ tools/perf/builtin-top.c | 29 ++- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 50 +++- tools/perf/tests/hybrid-merge.c | 206 +++++++++++++++ tools/perf/tests/shell/report_hybrid_merge.sh | 121 +++++++++ tools/perf/tests/shell/top.sh | 73 +++++- tools/perf/tests/tests.h | 1 + tools/perf/ui/browsers/hists.c | 7 +- tools/perf/ui/hist.c | 246 +++++++++++++++++- tools/perf/util/config.c | 10 + tools/perf/util/evlist.c | 218 +++++++++++++++- tools/perf/util/evlist.h | 3 + tools/perf/util/evsel.h | 1 + tools/perf/util/hist.h | 15 +- tools/perf/util/parse-events.c | 4 +- tools/perf/util/pmu.c | 43 ++- tools/perf/util/pmu.h | 3 + tools/perf/util/symbol.c | 1 + tools/perf/util/symbol_conf.h | 1 + 24 files changed, 1038 insertions(+), 40 deletions(-) create mode 100644 tools/perf/tests/hybrid-merge.c create mode 100755 tools/perf/tests/shell/report_hybrid_merge.sh
base-commit: 91b0782fc9e9d2f0a40b5256146e014802fdbb36
prerequisite-patch-id: b6fdc526887b71fb66f5fe0c0d41f7ef9493861e
prerequisite-patch-id: d47077f674c700f4f6296e9367f7ddfe004aea89
prerequisite-patch-id: ed0db23450840601762fe85ca1bef06ce6d28fe7
prerequisite-patch-id: d84e6b96d533ee6ed549e26e94216e2da60f7f74
prerequisite-patch-id: 6e8f19551d771621a5037096626cfe7e271b36f9
prerequisite-patch-id: 3768dfd588c7439deb741bce44c74763011c6be7
prerequisite-patch-id: babac99a4faa3b1525e44d3f34ab33a88103334c
prerequisite-patch-id: 5279827453fea1536abc620aa33c887c44e41ee9
prerequisite-patch-id: bdc0d648a577142b9270fb59b54496bc2bf8d5ea
prerequisite-patch-id: 6210f4401a97fc55532f1078089921f64dad7d3e
prerequisite-patch-id: 9dd334df517a2d163e4b73358facd4369635032f
prerequisite-patch-id: bc536fd240b5ddd999c781831a6445cb04ace25c
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Ian Rogers
reviewerPATCH v3 1/9 perf evlist: Implement evlist__can_merge_hybrid using first_wildcard_match
Add logic to dynamically identify mergeable events spawned from the same wildcard alias via first_wildcard_match. first_wildcard_match is populated when events are parsed, in perf report the events come from a file and so the first_wildcard_match is computed by matching event names.
evlist__can_merge_hybrid only tests whether merging is possible and doesn't modify the evlist, so that it may be used as a predicate, for example, to decide whether to display a hint. The linking of all the matching events is done by evlist__merge_hybrid.
is_pmu_core_len identifies core PMUs using the hybrid topology of the perf.data file. Should the file lack the topology, the PMUs of this machine are scanned but only when the file was recorded on the same architecture, as PMU names like "cpu_core" are architecture specific.
--- tools/perf/util/evlist.c | 183 +++++++++++++++++++++++++++++++++ tools/perf/util/evlist.h | 3 + tools/perf/util/evsel.h | 1 + tools/perf/util/hist.h | 1 + tools/perf/util/parse-events.c | 4 +- tools/perf/util/pmu.c | 43 +++++++- tools/perf/util/pmu.h | 3 + tools/perf/util/symbol_conf.h | 1 + 8 files changed, 236 insertions(+), 3 deletions(-)
tools/perf/util/evlist.c +183 -0
@@ -7,6 +7,7 @@*/ #include "evlist.h" +#include <stdio.h> #include <errno.h> #include <inttypes.h> #include <signal.h>
@@ -53,6 +54,7 @@#include "event.h" #include "evsel.h" #include "evsel_fprintf.h" +#include "hist.h" #include "intel-tpebs.h" #include "metricgroup.h" #include "mmap.h"
@@ -125,12 +127,24 @@ struct evlist *evlist__new_default(const struct target *target, bool sample_callif (err) goto out_err; }else { +struct evsel *leader =NULL ; + +/* Fallback for cross-platform file analysis missing the topology header */ while ((pmu =perf_pmus__scan_core (pmu)) !=NULL ) {snprintf (buf,sizeof (buf),"%s/cycles/%s" , pmu->name, can_profile_kernel ?"P" :"Pu" ); err =parse_event (evlist, buf);if (err) goto out_err; + +if (!leader) { + leader =evlist__last (evlist); + }else { +struct evsel *last =evlist__last (evlist); + +if (last != leader) + last->first_wildcard_match = leader; + } } }
@@ -148,6 +162,175 @@ struct evlist *evlist__new_default(const struct target *target, bool sample_callreturn NULL ; } +/** +* evlist__hybrid_matches - find events that may be merged across core PMUs. +* @evlist: The evlist to check. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +* @link: Should the matches be recorded in first_wildcard_match? +* +* perf record, top, etc. set first_wildcard_match in the event parsing. The +* perf.data case (e.g. perf report) recomputes the first_wildcard_match using +* string matches for core events on hybrid systems. +*/ +static bool evlist__hybrid_matches (struct evlist *evlist,struct perf_env *env,bool link) +{ +struct evsel *pos; +unsigned int nr =0 ; +bool found = false; + +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos)) +continue ; + +if (pos->core.leader != &pos->core || pos->core.nr_members >1 ) +return false; + +if (pos->first_wildcard_match) + found = true; + nr++; + } + +/* No events found. */ +if (nr <=1 ) +return false; + +/* Found wildcard events from parsing. */ +if (found) +return true; + +/* Try to find matching core events and set the first_wildcard_match. */ +evlist__for_each_entry (evlist, pos) { +const char *pos_name; +const char *pos_match; +struct evsel *peer; + +if (evsel__is_dummy_event (pos) || pos->first_wildcard_match) +continue ; + + pos_name =evsel__name (pos); + pos_match = pos_name ?strchr (pos_name,'/' ) :NULL ; + +if (!pos_match) +continue ; + +/* If evsel->core.is_pmu_core missing in report, fallback to perf_env */ +if (!evsel__is_hybrid (pos)) { +if (!is_pmu_core_len (env, pos_name, pos_match - pos_name)) +continue ; + } + + peer = pos; +list_for_each_entry_continue (peer, &evlist__core (evlist)->entries, core.node) { +const char *peer_name; +const char *peer_match; + +if (evsel__is_dummy_event (peer) || peer->first_wildcard_match) +continue ; + + peer_name =evsel__name (peer); + peer_match = peer_name ?strchr (peer_name,'/' ) :NULL ; +if (!peer_match) +continue ; + +if (!evsel__is_hybrid (peer)) { +if (!is_pmu_core_len (env, peer_name, + peer_match - peer_name)) +continue ; + } + +if (strcmp (pos_match, peer_match)) +continue ; + + found = true; +if (!link) { +/* Just a test, don't modify the evlist. */ +return true; + } +/* +* Keep looking so that all the events of this name are +* linked, there may be more than 2 core PMUs. +*/ + peer->first_wildcard_match = pos; + } + } +return found; +} + +/** +* evlist__can_merge_hybrid - can events in the evlist be merged across core +* PMUs? The evlist isn't modified. +* @evlist: The evlist to check. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +*/ +bool evlist__can_merge_hybrid (struct evlist *evlist,struct perf_env *env) +{ +return evlist__hybrid_matches (evlist, env,/*link=*/ false); +} + +/* +* evlist__merge_hybrid - group hybrid events logically together. +* @evlist: The evlist containing events to merge. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +* +* Iterates through the evlist and merges associated hybrid events by assigning +* their first_wildcard_match as their core group leader, modifying their +* presentation for a single merged histogram view. +*/ +void evlist__merge_hybrid (struct evlist *evlist,struct perf_env *env) +{ +struct list_head new_list; +struct evsel *member, *mtmp; +struct evsel *pos, *tmp; +int idx =0 ; + +/* Compute first_wildcard_match for events that lack it, say from a file. */ +if (!evlist__hybrid_matches (evlist, env,/*link=*/ true)) +return ; + +evlist__for_each_entry_safe (evlist, tmp, pos) { +struct evsel *leader, *old_leader; + +if (evsel__is_dummy_event (pos)) +continue ; + +if (!pos->first_wildcard_match) +continue ; + + leader =evsel__leader (pos->first_wildcard_match); + old_leader =evsel__leader (pos); +if (old_leader == leader) +continue ; + +if (old_leader != pos) + old_leader->core.nr_members--; + pos->core.leader = &leader->core; + pos->merged_hybrid_group = true; +/* Base is 1 to natively represent the leader */ +if (leader->core.nr_members ==0 ) + leader->core.nr_members =1 ; + leader->core.nr_members++; + } + +INIT_LIST_HEAD (&new_list); + +while (!list_empty (&evlist__core (evlist)->entries)) { + pos = list_first_entry(&evlist__core(evlist)->entries, struct evsel, core.node); +list_move_tail (&pos->core.node, &new_list); + +list_for_each_entry_safe (member, mtmp, &evlist__core (evlist)->entries, core.node) { +if (member->core.leader == &pos->core) +list_move_tail (&member->core.node, &new_list); + } + } +list_splice_init (&new_list, &evlist__core (evlist)->entries); + +evlist__for_each_entry (evlist, pos) + pos->core.idx = idx++; +} +struct evlist *evlist__new_dummy (void ) {struct evlist *evlist =evlist__new ();
tools/perf/util/evlist.h +3 -0
@@ -331,6 +331,9 @@ static inline void evlist__set_selected(struct evlist *evlist, struct evsel *evsstruct evlist *evlist__new (void );struct evlist *evlist__new_default (const struct target *target,bool sample_callchains); +struct perf_env ; +bool evlist__can_merge_hybrid (struct evlist *evlist,struct perf_env *env); +void evlist__merge_hybrid (struct evlist *evlist,struct perf_env *env);struct evlist *evlist__new_dummy (void );struct evlist *evlist__get (struct evlist *evlist);void evlist__put (struct evlist *evlist);
tools/perf/util/evsel.h +1 -0
@@ -53,6 +53,7 @@ struct evsel {int id_pos;int is_pos;unsigned int sample_size; +bool merged_hybrid_group;/* * These fields can be set in the parse-events code or similar.
tools/perf/util/hist.h +1 -0
@@ -130,6 +130,7 @@ struct hists {struct hists_stats stats;u64 event_stream;u16 col_len[HISTC_NR_COLS ]; +bool merge_entries;bool has_callchains;int socket_filter;struct perf_hpp_list *hpp_list;
tools/perf/util/parse-events.c +2 -2
@@ -1683,7 +1683,7 @@ int parse_events_multi_pmu_add(struct parse_events_state *parse_state,strbuf_release (&sb); ok++; } -if (first_wildcard_match ==NULL ) +if (first_wildcard_match ==NULL && !list_empty (list)) first_wildcard_match = container_of(list->prev, struct evsel, core.node); }
@@ -1754,7 +1754,7 @@ int parse_events_multi_pmu_add_or_add_pmu(struct parse_events_state *parse_state ok++; parse_state->wild_card_pmus = true; } -if (first_wildcard_match ==NULL ) { +if (first_wildcard_match ==NULL && !list_empty (*listp)) { first_wildcard_match = container_of((*listp)->prev, struct evsel, core.node); }
tools/perf/util/pmu.c +42 -1
55 lines not on this page · raw message
tools/perf/util/pmu.h +3 -0
@@ -16,6 +16,7 @@struct evsel_config_term ;struct hashmap ;struct perf_cpu_map ; +struct perf_env ;struct print_callbacks ;enum {
@@ -289,6 +290,8 @@ int perf_pmu__for_each_format(struct perf_pmu *pmu, void *state, pmu_format_callu64 perf_pmu__format_unpack (unsigned long *format,u64 config_val);bool is_pmu_core (const char *name); +bool is_pmu_core_len (struct perf_env *env,const char *name,size_t len); +bool perf_pmu__supports_legacy_cache (const struct perf_pmu *pmu);bool perf_pmu__auto_merge_stats (const struct perf_pmu *pmu);bool perf_pmu__have_event (struct perf_pmu *pmu,const char *name);
tools/perf/util/symbol_conf.h +1 -0
@@ -38,6 +38,7 @@ enum symbol__weight_mode {for ((_weight) =WEIGHT_WEIGHT ; (_weight) <=WEIGHT_WEIGHT3 ; (_weight)++) struct symbol_conf { +bool hybrid_merge;bool nanosecs;unsigned short priv_size; bool try_vmlinux_path,
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 1/9] perf evlist: Implement evlist__can_merge_hybrid using first_wildcard_match
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 11 lines
Add logic to dynamically identify mergeable events spawned from the same wildcard alias via
first_wildcard_match.first_wildcard_matchis populated when events are parsed, inperf reportthe events come from a file and so thefirst_wildcard_matchis computed by matching event names.
evlist__can_merge_hybridonly tests whether merging is possible and doesn't modify the evlist, so that it may be used as a predicate, for example, to decide whether to display a hint. The linking of all the matching events is done byevlist__merge_hybrid.
Quoted 6 lines
is_pmu_core_lenidentifies core PMUs using the hybrid topology of the perf.data file. Should the file lack the topology, the PMUs of this machine are scanned but only when the file was recorded on the same architecture, as PMU names like "cpu_core" are architecture specific.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v3 2/9 perf ui hist: Add support for aggregated total_period and merging entries
In reply to Ian RogersAdd the histogram side of hybrid event merging.
When events are merged the entries of the other events are linked to the leader's entries as pairs, and a row is then displayed as the sum over an entry and its pairs. Teach the hpp code to work on that sum:
- The formatting and sorting routines add up the value of an entry and of each of its pairs, so a row shows, and is sorted by, the total across the core PMUs rather than the leader's value alone.
- Percentages are taken against the summed total_period of the merged hists. Computing each event's share against its own PMU's total would leave the column not adding up to 100%. hist_entry__get_percent_limit_merged() does the same for --percent-limit, which would otherwise compare against the wrong total and filter out the wrong entries.
- A header and column width are reserved for the "Total" the merged value is shown under.
evlist__merge_hists_hybrid() links the hists together and resorts the result so that it is ordered by the merged total.
hists__link() is run once for every event merged into the leader, so an entry can be offered as a pair more than once. hist_entry__add_pair() used list_add_tail() unconditionally, which corrupts the list when the entry is already on one. Return early if the entry is already paired with this one, and use list_move_tail() so that an entry linked elsewhere is moved rather than added a second time.
Merging is never combined with --hierarchy, which the merge_entries tests rely on.
--- tools/perf/ui/hist.c | 246 +++++++++++++++++++++++++++++++++++++++-- tools/perf/util/hist.h | 14 ++- 2 files changed, 252 insertions(+), 8 deletions(-)
tools/perf/ui/hist.c +239 -7
@@ -57,8 +57,9 @@ struct hpp_fmt_value { };static int __hpp__fmt (struct perf_hpp *hpp,struct hist_entry *he, -hpp_field_fn get_field,const char *fmt,int len, -hpp_snprint_fn print_fn,enum perf_hpp_fmt_type fmtype) +hpp_field_fn get_field,const char *fmtstr,int len, +hpp_snprint_fn print_fn,enum perf_hpp_fmt_type fmtype, +struct perf_hpp_fmt *fmt__maybe_unused ) {int ret =0 ;struct hists *hists = he->hists;
@@ -98,13 +99,50 @@ static int __hpp__fmt(struct perf_hpp *hpp, struct hist_entry *he, } } +/* Note, merge_entries implies !symbol_conf.report_hierarchy. */ +if (he->hists->merge_entries) { +u64 total_val =0 ; +u64 total_samples =0 ; +u64 total_period =0 ; + +for (i =0 ; i < nr_members; i++) { +struct evsel *member_evsel =hists_to_evsel (values[i].hists); + +struct evsel *he_evsel =hists_to_evsel (he->hists); + +if (member_evsel != he_evsel && + member_evsel->first_wildcard_match != he_evsel) +continue ; + + total_val += values[i].val; + total_samples += values[i].samples; + total_period += fmtype ==PERF_HPP_FMT_TYPE__PERCENT ? +hists__total_period (values[i].hists) : +hists__total_latency (values[i].hists); + } + +if (fmtype ==PERF_HPP_FMT_TYPE__PERCENT || fmtype ==PERF_HPP_FMT_TYPE__LATENCY ) { +double percent =0.0 ; + +if (total_period) + percent =100.0 * total_val / total_period; + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, percent); + }else if (fmtype ==PERF_HPP_FMT_TYPE__AVERAGE ) { +double avg = total_samples ? (1.0 * total_val / total_samples) :0 ; + + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, avg); + }else { + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, total_val); + } + } +for (i =0 ; i < nr_members; i++) {if (symbol_conf.skip_empty && values[i].hists->stats.nr_samples ==0 )continue ; ret +=__hpp__fmt_print (hpp, values[i].hists, values[i].val, - values[i].samples, fmt, len, + values[i].samples, fmtstr, len, print_fn, fmtype); }
@@ -129,7 +167,7 @@ int hpp__fmt(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,if (symbol_conf.field_sep) {return __hpp__fmt (hpp, he, get_field, fmtstr,1 , - print_fn, fmtype); + print_fn, fmtype, fmt); }if (fmtype ==PERF_HPP_FMT_TYPE__PERCENT || fmtype ==PERF_HPP_FMT_TYPE__LATENCY )
@@ -137,7 +175,7 @@ int hpp__fmt(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,else len -=1 ; -return __hpp__fmt (hpp, he, get_field, fmtstr, len, print_fn, fmtype); +return __hpp__fmt (hpp, he, get_field, fmtstr, len, print_fn, fmtype, fmt); } int hpp__fmt_acc(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,
@@ -287,6 +325,35 @@ static int __hpp__sort(struct hist_entry *a, struct hist_entry *b,return __hpp__group_sort_idx (a, b, get_field, symbol_conf.group_sort_idx); } +/* +* Relies on merge_entries being only enabled if there are +* only matching events. If that is ever relaxed will need +* more logic here. merge_entries also implies that +* symbol_conf.report_hierarchy is false. +*/ +if (a->hists->merge_entries && b->hists->merge_entries) { +u64 val_a =get_field (a), val_b =get_field (b); +struct hist_entry *pair; +struct evsel *evsel_a =hists_to_evsel (a->hists); +struct evsel *evsel_b =hists_to_evsel (b->hists); + +list_for_each_entry (pair, &a->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_a) + val_a +=get_field (pair); + } +list_for_each_entry (pair, &b->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_b) + val_b +=get_field (pair); + } + + ret =field_cmp (val_a, val_b); +if (ret) +return ret; + } ret =field_cmp (get_field (a),get_field (b));if (ret || !symbol_conf.event_group)
@@ -323,7 +390,29 @@ static int __hpp__sort_acc(struct hist_entry *a, struct hist_entry *b,/* * Put caller above callee when they have equal period. */ - ret =field_cmp (get_field (a),get_field (b)); +if (a->hists->merge_entries && b->hists->merge_entries) { +u64 val_a =get_field (a), val_b =get_field (b); +struct hist_entry *pair; +struct evsel *evsel_a =hists_to_evsel (a->hists); +struct evsel *evsel_b =hists_to_evsel (b->hists); + +list_for_each_entry (pair, &a->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_a) + val_a +=get_field (pair); + } +list_for_each_entry (pair, &b->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_b) + val_b +=get_field (pair); + } + + ret =field_cmp (val_a, val_b); + }else { + ret =field_cmp (get_field (a),get_field (b)); + }if (ret)return ret;
@@ -386,6 +475,8 @@ static int hpp__width_fn(struct perf_hpp_fmt *fmt, evsel__hists(pos)->stats.nr_samples) nr++; } +if (hists->merge_entries) + nr++;/* Add 1 extra unit of width generically for the 'Total' */ len =max (len, nr * fmt->len); }
@@ -403,8 +494,38 @@ static int hpp__header_fn(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,int len =hpp__width_fn (fmt, hpp, hists);const char *hdr ="" ; -if (line == hists->hpp_list->nr_header_lines -1 ) +if (line == hists->hpp_list->nr_header_lines -1 ) { hdr = fmt->name; +if (hists->merge_entries) { +int w =0 ; +int f_len = fmt->user_len ?: fmt->len; +struct evsel *pos, *evsel =hists_to_evsel (hists); +int string_len = f_len; + +for_each_group_evsel (pos, evsel) { +if (symbol_conf.skip_empty && +evsel__hists (pos)->stats.nr_samples ==0 ) +continue ; + string_len += f_len; + } + +if (len > string_len) { + w +=scnprintf (hpp->buf + w, hpp->size - w,"%*s" , + len - string_len,"" ); + } + + w +=scnprintf (hpp->buf + w, hpp->size - w,"%*.*s" , + f_len, f_len, fmt->name); +for_each_group_evsel (pos, evsel) { +if (symbol_conf.skip_empty && +evsel__hists (pos)->stats.nr_samples ==0 ) +continue ; + w +=scnprintf (hpp->buf + w, hpp->size - w," %*.*s" , + f_len -1 , f_len -1 ,evsel__name (pos)); + } +return w; + } + }return scnprintf (hpp->buf, hpp->size,"%*s" , len, hdr); }
@@ -1271,3 +1392,114 @@ int perf_hpp__alloc_mem_stats(struct perf_hpp_list *list, struct evlist *evlist) }return 0 ; } + +float hist_entry__get_percent_limit_merged (struct hist_entry *he) +{ +struct hist_entry *pair; +u64 period = he->stat.period; +u64 total_period =hists__total_period (he->hists); +struct evsel *evsel =hists_to_evsel (he->hists); +struct evsel *pos; + +/* Accumulate global total_period across all merged hists matching hybrid type */ +for_each_group_member (pos, evsel) { +if (pos->first_wildcard_match == evsel) + total_period +=hists__total_period (evsel__hists (pos)); + } + +if (unlikely (total_period ==0 )) +return 0 ; + +if (symbol_conf.cumulate_callchain) { + period = he->stat_acc->period; +list_for_each_entry (pair, &he->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel) + period += pair->stat_acc->period; + } + }else { +/* Accumulate symbol specific period across pairs matching hybrid type */ +list_for_each_entry (pair, &he->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel) + period += pair->stat.period; + } + } + +return period *100.0 / total_period; +} + +void evlist__merge_hists_hybrid (struct evlist *evlist,bool refresh_hists) +{ +struct evsel *pos; +struct evsel *member; +bool hybrid_group; + +/* +* Merged hists display all the events of a group in a single set of +* entries, which the hierarchy display has no way to render. Keeping +* hists->merge_entries false here means the rest of the display code +* can assume merge_entries implies !symbol_conf.report_hierarchy. +*/ +if (symbol_conf.report_hierarchy) +return ; + +/* Set merge_entries flag strictly on leaders formulated by hybrid topology */ +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos)) +continue ; + +if (pos->core.leader == &pos->core && pos->core.nr_members >1 ) { + hybrid_group = false; + +if (pos->first_wildcard_match || pos->merged_hybrid_group) { + hybrid_group = true; + }else { +for_each_group_member (member, pos) { +if (member->first_wildcard_match || + member->merged_hybrid_group) { + hybrid_group = true; +break ; + } + } + } + +if (hybrid_group) { +evsel__hists (pos)->merge_entries = true; + symbol_conf.event_group = true; + symbol_conf.hybrid_merge = true; +if (!pos->group_name) { + pos->group_name =strdup ("Merged hybrid events" ); +if (!pos->group_name) +pr_warning ("Failed to allocate hybrid group name\n" ); + } + } + } + } + +if (!refresh_hists) +return ; + +evlist__for_each_entry (evlist, pos) { +/* Match histograms dynamically since parsing happened before group toggling */ +if (symbol_conf.event_group && !evsel__is_group_leader (pos)) { +struct hists *leader_hists =evsel__hists (evsel__leader (pos)); +struct hists *hists =evsel__hists (pos); + +hists__match (leader_hists, hists); +hists__link (leader_hists, hists); + } + } + +/* Now that links are formed, safely resort the active tree so the UI renders accurately */ +if (symbol_conf.event_group) { +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos) || !evsel__is_group_leader (pos)) +continue ; +if (pos->core.nr_members >1 &&evsel__hists (pos)->merge_entries) +hists__output_resort (evsel__hists (pos),NULL ); + } + } +}
tools/perf/util/hist.h +13 -1
@@ -339,7 +339,13 @@ static inline struct hist_entry *hist_entry__next_pair(struct hist_entry *he)static inline void hist_entry__add_pair (struct hist_entry *pair,struct hist_entry *he) { -list_add_tail (&pair->pairs.node, &he->pairs.head); +struct list_head *pos; + +list_for_each (pos, &he->pairs.head) { +if (pos == &pair->pairs.node) +return ;/* Already paired */ + } +list_move_tail (&pair->pairs.node, &he->pairs.head); } struct hist_entry *hists__add_entry(struct hists *hists,
@@ -434,11 +440,17 @@ void hists__match(struct hists *leader, struct hists *other);int hists__link (struct hists *leader,struct hists *other);int hists__unlink (struct hists *hists); +float hist_entry__get_percent_limit_merged (struct hist_entry *he); +void evlist__merge_hists_hybrid (struct evlist *evlist,bool refresh); +static inline float hist_entry__get_percent_limit (struct hist_entry *he) {u64 period = he->stat.period;u64 total_period =hists__total_period (he->hists); +if (he->hists->merge_entries) +return hist_entry__get_percent_limit_merged (he); +if (unlikely (total_period ==0 ))return 0 ;
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 2/9] perf ui hist: Add support for aggregated total_period and merging entries
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 17 lines
Add the histogram side of hybrid event merging.
When events are merged the entries of the other events are linked to the leader's entries as pairs, and a row is then displayed as the sum over an entry and its pairs. Teach the hpp code to work on that sum:
- The formatting and sorting routines add up the value of an entry and of each of its pairs, so a row shows, and is sorted by, the total across the core PMUs rather than the leader's value alone.
- Percentages are taken against the summedtotal_periodof the merged hists. Computing each event's share against its own PMU's total would leave the column not adding up to 100%.hist_entry__get_percent_limit_merged()does the same for --percent-limit, which would otherwise compare against the wrong total and filter out the wrong entries.
[ ... ]
Quoted 4 lines
Merging is never combined with --hierarchy, which the
merge_entriestests rely on.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerPATCH v3 3/9 perf tools: Expose opt-in --hybrid-merge
In reply to Ian RogersAdd --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
evlist__merge_hybrid() links the events and evlist__merge_hists_hybrid() links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.
Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
--- tools/perf/Documentation/perf-report.txt | 5 +++++ tools/perf/Documentation/perf-top.txt | 5 +++++ tools/perf/builtin-report.c | 19 +++++++++++++++++++ tools/perf/builtin-top.c | 17 +++++++++++++++++ tools/perf/util/symbol.c | 1 + 5 files changed, 47 insertions(+)
tools/perf/Documentation/perf-report.txt +5 -0
@@ -578,6 +578,11 @@ include::itrace.txt[] --raw-trace:: When displaying traceevent output, do not use print fmt or plugins. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + -H:: --hierarchy:: Enable hierarchical output. In the hierarchy mode, each sort key groups
tools/perf/Documentation/perf-top.txt +5 -0
@@ -49,6 +49,11 @@ Default is to monitor all CPUS. encoding with the layout of the event control registers as described by entries in /sys/bus/event_source/devices/cpu/format/*. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + --filter=<filter>:: Event filter. This option should follow an event selector (-e). For syntax see linkperf:perf-record[1].
tools/perf/builtin-report.c +19 -0
@@ -1114,6 +1114,17 @@ static int __cmd_report(struct report *rep)evlist__for_each_entry (session->evlist, pos) rep->nr_entries +=evsel__hists (pos)->nr_entries; +if (symbol_conf.hybrid_merge) { +struct perf_env *env =perf_session__env (session); + +if (evlist__can_merge_hybrid (session->evlist, env)) { +evlist__merge_hybrid (session->evlist, env); +evlist__merge_hists_hybrid (session->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } +if (use_browser ==0 ) {if (verbose >3 )perf_session__fprintf (session, stdout);
@@ -1449,6 +1460,8 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",
@@ -1548,6 +1561,12 @@ int cmd_report(int argc, const char **argv) report.symbol_filter_str = argv[0 ]; } +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style)
tools/perf/builtin-top.c +17 -0
@@ -1336,6 +1336,15 @@ static int __cmd_top(struct perf_top *top)if (!target__none (&opts->target))evlist__enable (top->evlist); +if (symbol_conf.hybrid_merge) { +if (evlist__can_merge_hybrid (top->evlist,/*env=*/ NULL )) { +evlist__merge_hybrid (top->evlist,/*env=*/ NULL ); +evlist__merge_hists_hybrid (top->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } + ret =-1 ;if (pthread_create (&thread_process,NULL , process_thread, top)) {ui__error ("Could not create process thread.\n" );
@@ -1491,6 +1500,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),
@@ -1679,6 +1690,12 @@ int cmd_top(int argc, const char **argv)if (argc)usage_with_options (top_usage, options); +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style) {
tools/perf/util/symbol.c +1 -0
@@ -74,6 +74,7 @@ struct symbol_conf symbol_conf = { .symfs = "", .symfs_layout_flat = false, .event_group = true, + .hybrid_merge = false, .inline_name = true, .res_sample = 0, .addr2line_timeout_ms = 5 * 1000,
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 3/9] perf tools: Expose opt-in --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Add --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
evlist__merge_hybrid()links the events andevlist__merge_hists_hybrid()links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.
Quoted 5 lines
Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Ian Rogers
reviewerPATCH v3 4/9 perf Documentation: Add tip for hybrid event merging
In reply to Ian RogersAdd informative text outlining the IPC imbalances associated with merging cross-hybrid core events like cycles.
--- tools/perf/Documentation/tips.txt | 1 + 1 file changed, 1 insertion(+)
tools/perf/Documentation/tips.txt +1 -0
@@ -66,3 +66,4 @@ For latency profiling, try: perf record/report --latency For parallelism histogram, try: perf report --hierarchy --sort latency,parallelism,comm,symbol To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s +When merging events like cycles, different core frequencies and instructions per cycle mean the counts may not fairly reflect time spent in a function.
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 4/9] perf Documentation: Add tip for hybrid event merging
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
Add informative text outlining the IPC imbalances associated with merging cross-hybrid core events like cycles.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Ian Rogers
reviewerPATCH v3 5/9 perf evlist: Toggle merged_hybrid_group properly in enable/disable
In reply to Ian RogersMerging makes the merged event the leader of the events of the other core PMUs so their histograms can be linked, but the result isn't a real event group. Each event keeps its own file descriptor and has to be enabled and disabled in its own right.
__evlist__enable(), __evlist__disable() and evlist__is_enabled() skip anything that isn't a group leader, and the first two then walk the group members of the events they do act on. For a merged set that is
backwards: the members are skipped by the leader test, so their file descriptors are never touched, and the walk over the leader's members only updates the bookkeeping in evsel->disabled.
Treat an evsel with merged_hybrid_group set as a leader so that it is enabled and disabled in its own right via its own file descriptors.
perf top enables and disables by event name. A merged member carries the name of its own PMU rather than the name that was asked for, so also match it against its leader's name, otherwise naming the event toggles only part of the merged set.
--- tools/perf/util/evlist.c | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-)
tools/perf/util/evlist.c +26 -9
@@ -753,7 +753,9 @@ static bool evlist__is_enabled(struct evlist *evlist)struct evsel *pos;evlist__for_each_entry (evlist, pos) { -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if (!pos->core.fd) +continue ; +if (!evsel__is_group_leader (pos) && !pos->merged_hybrid_group)continue ;/* If at least one event is enabled, evlist is enabled. */ if (!pos->disabled)
@@ -767,14 +769,19 @@ static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool exclstruct evsel *pos, *member;struct evlist_cpu_iterator evlist_cpu_itr;bool has_imm = false; +bool match;/* Disable 'immediate' events last */ for (int imm =0 ; imm <=1 ; imm++) {evlist__for_each_cpu (evlist_cpu_itr, evlist) { pos = evlist_cpu_itr.evsel; -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (pos->disabled || !evsel__is_group_leader (pos) || !pos->core.fd) +if (pos->disabled || (!evsel__is_group_leader (pos) && + !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;
@@ -789,9 +796,12 @@ static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool excl }evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;
@@ -827,21 +837,28 @@ static void __evlist__enable(struct evlist *evlist, char *evsel_name, bool excl_ {struct evsel *pos, *member;struct evlist_cpu_iterator evlist_cpu_itr; +bool match;evlist__for_each_cpu (evlist_cpu_itr, evlist) { pos = evlist_cpu_itr.evsel; -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;evsel__enable_cpu (pos, evlist_cpu_itr.cpu_map_idx); }evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 5/9] perf evlist: Toggle merged_hybrid_group properly in enable/disable
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines
Merging makes the merged event the leader of the events of the other core PMUs so their histograms can be linked, but the result isn't a real event group. Each event keeps its own file descriptor and has to be enabled and disabled in its own right.
__evlist__enable(),__evlist__disable()andevlist__is_enabled()skip anything that isn't a group leader, and the first two then walk the group members of the events they do act on. For a merged set that is
backwards: the members are skipped by the leader test, so their file descriptors are never touched, and the walk over the leader's members only updates the bookkeeping inevsel->disabled.Treat an evsel with
merged_hybrid_groupset as a leader so that it is enabled and disabled in its own right via its own file descriptors.
Quoted 6 lines
perf top enables and disables by event name. A merged member carries the name of its own PMU rather than the name that was asked for, so also match it against its leader's name, otherwise naming the event toggles only part of the merged set.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Ian Rogers
reviewerPATCH v3 6/9 perf tools: Add TUI hints for --hybrid-merge
In reply to Ian RogersAdd dynamic TUI hints prompting users to restart with --hybrid-merge when heterogeneous core PMU events are populated in the sample view.
--- tools/perf/Documentation/tips.txt | 1 + tools/perf/ui/browsers/hists.c | 7 +++++-- 2 files changed, 6 insertions(+), 2 deletions(-)
tools/perf/Documentation/tips.txt +1 -0
@@ -67,3 +67,4 @@ For parallelism histogram, try: perf report --hierarchy --sort latency,paralleli To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s When merging events like cycles, different core frequencies and instructions per cycle mean the counts may not fairly reflect time spent in a function. +Use --hybrid-merge to merge matching events from hybrid cores into one view
tools/perf/ui/browsers/hists.c +5 -2
@@ -3580,9 +3580,12 @@ static int perf_evsel_menu__run(struct evsel_menu *menu,const char *title ="Available samples" ;int delay_secs = hbt ? hbt->refresh :0 ;int key; +const char *help_text ="ESC: exit, ENTER|->: Browse histograms" ; -if (ui_browser__show (&menu->b, title, -"ESC: exit, ENTER|->: Browse histograms" ) <0 ) +if (!symbol_conf.hybrid_merge &&evlist__can_merge_hybrid (evlist, menu->env)) + help_text ="ESC: exit, ENTER|->: Browse. Try --hybrid-merge to combine events." ; + +if (ui_browser__show (&menu->b, title, help_text) <0 )return -1 ;while (1 ) {
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 6/9] perf tools: Add TUI hints for --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
Add dynamic TUI hints prompting users to restart with --hybrid-merge when heterogeneous core PMU events are populated in the sample view.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Ian Rogers
reviewerPATCH v3 7/9 perf config: Add core.hybrid-merge to configure event merging
In reply to Ian RogersProvide a core.hybrid-merge configuration option in .perfconfig to allow enabling hybrid event aggregation by default, avoiding the need to pass --hybrid-merge explicitly on every invocation.
The config value is a default rather than an explicit request, so with --hierarchy it is ignored with a warning, while giving both --hierarchy and --hybrid-merge on the command line remains an error.
perf stat has its own merging options for counting, the sampling core.hybrid-merge deliberately doesn't alter it and its --hybrid-merge must still be given explicitly.
--- tools/perf/Documentation/perf-config.txt | 8 ++++++++ tools/perf/builtin-report.c | 16 +++++++++++----- tools/perf/builtin-top.c | 22 +++++++++++++++------- tools/perf/util/config.c | 10 ++++++++++ 4 files changed, 44 insertions(+), 12 deletions(-)
tools/perf/Documentation/perf-config.txt +8 -0
@@ -217,6 +217,14 @@ core.*:: Sets a timeout (in milliseconds) for parsing 'addr2line' output. The default timeout is 5s. + hybrid-merge:: + Merge matching events from all hybrid core PMUs into one display + by default. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. This applies to + 'perf report' and 'perf top', it is ignored with '--hierarchy' and + doesn't alter 'perf stat' where '--hybrid-merge' must be given + explicitly. + tui.*, gtk.*:: Subcommands that can be configured here are 'top', 'report' and 'annotate'. These values are booleans, for example:
tools/perf/builtin-report.c +11 -5
@@ -1324,6 +1324,7 @@ int cmd_report(int argc, const char **argv)int branch_mode =-1 ;int last_key =0 ;bool branch_call_mode = false; +bool hybrid_merge_set = false;#define CALLCHAIN_DEFAULT_OPT "graph,0.5,caller,function,percent"static const char report_callchain_help[] ="Display call graph (stack chain/backtrace):\n\n" CALLCHAIN_REPORT_HELP
@@ -1460,8 +1461,8 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), -OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, -"merge the same event across hybrid core PMUs" ), +OPT_BOOLEAN_SET (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, &hybrid_merge_set, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",
@@ -1562,9 +1563,14 @@ int cmd_report(int argc, const char **argv) }if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { -pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); - ret = -EINVAL ; - goto exit; +if (hybrid_merge_set) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +/* A config file default shouldn't fail an explicit option. */ +pr_warning ("core.hybrid-merge ignored: --hierarchy cannot display merged hybrid events\n" ); + symbol_conf.hybrid_merge = false; }if (disassembler_style) {
tools/perf/builtin-top.c +15 -7
@@ -1314,9 +1314,11 @@ static int __cmd_top(struct perf_top *top) }/* -* Use global stat_config that is zero meaning aggr_mode is AGGR_NONE -* and hybrid_merge is false. +* Use global stat_config that is zero meaning aggr_mode is AGGR_NONE. +* Merging affects the event names as merged events share a name, all +* other stat_config behavior is unwanted here. */ + stat_config.hybrid_merge = symbol_conf.hybrid_merge;evlist__uniquify_evsel_names (top->evlist, &stat_config); ret =perf_top__start_counters (top);if (ret)
@@ -1493,6 +1495,7 @@ int cmd_top(int argc, const char **argv) .evlistp = &top.evlist, };bool branch_call_mode = false; +bool hybrid_merge_set = false;struct record_opts *opts = &top.record_opts;struct target *target = &opts->target;const char *disassembler_style =NULL , *objdump_path =NULL , *addr2line_path =NULL ;
@@ -1500,8 +1503,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), -OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, -"merge the same event across hybrid core PMUs" ), +OPT_BOOLEAN_SET (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, &hybrid_merge_set, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),
@@ -1691,9 +1694,14 @@ int cmd_top(int argc, const char **argv)usage_with_options (top_usage, options);if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { -pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); - status = -EINVAL ; - goto out_put_evlist; +if (hybrid_merge_set) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +/* A config file default shouldn't fail an explicit option. */ +pr_warning ("core.hybrid-merge ignored: --hierarchy cannot display merged hybrid events\n" ); + symbol_conf.hybrid_merge = false; }if (disassembler_style) {
tools/perf/util/config.c +10 -0
@@ -470,6 +470,16 @@ static int perf_default_core_config(const char *var, const char *value)if (!strcmp (var,"core.addr2line-disable-warn" )) symbol_conf.addr2line_disable_warn =perf_config_bool (var, value); +if (!strcmp (var,"core.hybrid-merge" )) { +/* +* Note, this is for sampling tools like perf report and top. +* perf stat has its own merging options and the +* stat_config.hybrid_merge of "perf stat --hybrid-merge" is +* deliberately not set here. +*/ + symbol_conf.hybrid_merge =perf_config_bool (var, value); + } +/* Add other config variables here. */ return 0 ; }
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 7/9] perf config: Add core.hybrid-merge to configure event merging
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] perf: spurious hybrid-merge warning on non-hybrid machines when configured globally --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf: spurious hybrid-merge warning on non-hybrid machines when configured globally --
commit 8278af13a38e06a26581491ef0fd6844e19d37a9
Author: Ian Rogers <[email protected]>
perf config: Add core.hybrid-merge to configure event merging
This commit provides a core.hybrid-merge configuration option in .perfconfig to enable hybrid event aggregation by default. It allows users to avoid passing the --hybrid-merge flag on every invocation of perf report or perf top.
Quoted 4 lines
tools/perf/util/config.c +0
-0
[ ... ]
Quoted 7 lines
tools/perf/util/config.c +3
-0@@ -470,6 +470,16 @@ static int perf_default_core_config(const char *var, const char *value)if (!strcmp (var,"core.addr2line-disable-warn" )) symbol_conf.addr2line_disable_warn =perf_config_bool (var, value); +if (!strcmp (var,"core.hybrid-merge" )) { +/* +* Note, this is for sampling tools like perf report and top.
tools/perf/util/config.c static int perf_default_core_config(const char *var, const char *value) · line 485
Quoted 10 lines
tools/perf/util/config.c +7
-0+* perf stat has its own merging options and the +* stat_config.hybrid_merge of "perf stat --hybrid-merge" is +* deliberately not set here. +*/ + symbol_conf.hybrid_merge =perf_config_bool (var, value); + } +/* Add other config variables here. */ return 0 ; }
[Severity: Medium]
When core.hybrid-merge is set globally to true, will this cause spurious blocking UI warnings on non-hybrid machines?
In both __cmd_report() and __cmd_top(), there is existing code that evaluates symbol_conf.hybrid_merge. For example, in __cmd_top():
if (symbol_conf.hybrid_merge) {
if (evlist__can_merge_hybrid(top->evlist, /*env=*/NULL)) {
evlist__merge_hybrid(top->evlist, /*env=*/NULL);
evlist__merge_hists_hybrid(top->evlist, false);
} else {
ui__warning("--hybrid-merge: no events to merge across core PMUs\n");
}
}If symbol_conf.hybrid_merge evaluates to true based on the config default, running perf top or perf report on a non-hybrid system evaluates evlist__can_merge_hybrid() as false and triggers this warning.
Because ui__warning() can create a blocking popup in TUI mode requiring user dismissal, does this break the usability of the tool on generic systems?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Ian Rogers
reviewerPATCH v3 8/9 perf test: Expand tests for --hybrid-merge
In reply to Ian RogersAdd a "Hybrid event merging" test that checks evlist__can_merge_hybrid() and evlist__merge_hybrid() directly, covering the events a wildcard expanded to over two core PMUs, the same over three, and events that must not be merged.
Add a report_hybrid_merge.sh shell test that records and then reports with --hybrid-merge, and extend top.sh to cover perf top with it.
--- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/hybrid-merge.c | 206 ++++++++++++++++++ tools/perf/tests/shell/report_hybrid_merge.sh | 121 ++++++++++ tools/perf/tests/shell/top.sh | 73 ++++++- tools/perf/tests/tests.h | 1 + 6 files changed, 392 insertions(+), 11 deletions(-) create mode 100644 tools/perf/tests/hybrid-merge.c create mode 100755 tools/perf/tests/shell/report_hybrid_merge.sh
tools/perf/tests/Build +1 -0
@@ -65,6 +65,7 @@ perf-test-y += perf-time-to-tsc.o perf-test-y += dlfilter-test.o perf-test-y += sigtrap.o perf-test-y += event_groups.o +perf-test-y += hybrid-merge.o perf-test-y += symbols.o perf-test-y += util.o perf-test-y += hwmon_pmu.o
tools/perf/tests/builtin-test.c +1 -0
@@ -149,6 +149,7 @@ static struct test_suite *generic_tests[] = { &suite__dlfilter, &suite__sigtrap, &suite__event_groups, + &suite__hybrid_merge, &suite__symbols, &suite__util, &suite__subcmd_help,
tools/perf/tests/hybrid-merge.c new file +206 -0
206 lines not on this page · raw message
tools/perf/tests/shell/report_hybrid_merge.sh new file +121 -0
121 lines not on this page · raw message
tools/perf/tests/shell/top.sh +62 -11
89 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -177,6 +177,7 @@ DECLARE_SUITE(perf_time_to_tsc);DECLARE_SUITE (dlfilter);DECLARE_SUITE (sigtrap);DECLARE_SUITE (event_groups); +DECLARE_SUITE (hybrid_merge);DECLARE_SUITE (symbols);DECLARE_SUITE (util);DECLARE_SUITE (uncore_event_sorting);
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 8/9] perf test: Expand tests for --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Add a "Hybrid event merging" test that checks
evlist__can_merge_hybrid()andevlist__merge_hybrid()directly, covering the events a wildcard expanded to over two core PMUs, the same over three, and events that must not be merged.Add a
report_hybrid_merge.shshell test that records and then reports with --hybrid-merge, and extendtop.shto cover perf top with it.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Ian Rogers
reviewerPATCH v3 9/9 perf test: Isolate test suite from user .perfconfig natively
In reply to Ian RogersRunning 'perf test' should not inherit the user's ~/.perfconfig environment as customizing core properties (like core.hybrid-merge) will trivially break stdout matching checks across the shell suite.
Explicitly set PERF_CONFIG to /dev/null inside cmd_test to globally sandbox the environment for the entire test workflow. Test specific config, like annotate.objdump, is read before the sandboxing so that 'perf test' still honors it.
--- tools/perf/tests/builtin-test.c | 49 +++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 5 deletions(-)
tools/perf/tests/builtin-test.c +44 -5
@@ -14,6 +14,8 @@#include <stdlib.h> #include <string.h> +#include "util/config.h" +#include <dirent.h> #include <linux/kernel.h> #include <linux/string.h>
@@ -1655,11 +1657,30 @@ static int run_workload(const char *work, int argc, const char **argv)return -1 ; } +/* +* Owns the string test_objdump_path points at when it came from the config. It +* is reachable for the lifetime of the process so leak checking won't report +* it. +*/ +static char *test_objdump_config_path; +static int perf_test__config (const char *var,const char *value,void *data__maybe_unused ) { -if (!strcmp (var,"annotate.objdump" )) - test_objdump_path = value; +if (!strcmp (var,"annotate.objdump" )) { +/* +* The config, and so value, is freed by perf_config__exit() +* below, take a copy that lives as long as the tests. +*/ +char *dup =strdup (value); + +if (!dup) +return -ENOMEM ; + +free (test_objdump_config_path); + test_objdump_config_path = dup; + test_objdump_path = dup; + }return 0 ; }
@@ -1755,13 +1776,31 @@ int cmd_test(int argc, const char **argv) };const char *const test_subcommands[] = {"list" ,NULL };struct intlist *skiplist =NULL ; -int ret =hists__init ();struct test_suite **suites; +int ret; -if (ret <0 ) + ret =hists__init (); +if (ret <0 )return ret; -perf_config (perf_test__config,NULL ); +/* Read test related config, like annotate.objdump, before isolating. */ + ret =perf_config (perf_test__config,NULL ); +if (ret <0 ) +return ret; + +/* +* Isolate the test suite from the user's ~/.perfconfig to ensure +* behavior isn't impacted by custom variables (e.g. core.hybrid-merge). +* Setting PERF_CONFIG isolates the perf commands run by the shell +* tests, config_exclusive_filename does the same for this process. +* Config values already read by main's perf_config are reset here, so +* far only the hybrid-merge value is known to alter test output. +*/ +if (setenv ("PERF_CONFIG" ,"/dev/null" ,1 ) <0 ) +return -1 ; + config_exclusive_filename ="/dev/null" ; +perf_config__exit (); + symbol_conf.hybrid_merge = false;/* Unbuffered output */ setvbuf (stdout,NULL , _IONBF,0 );
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v3 9/9] perf test: Isolate test suite from user .perfconfig natively
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Running 'perf test' should not inherit the user's ~/.perfconfig environment as customizing core properties (like core.hybrid-merge) will trivially break stdout matching checks across the shell suite.
Explicitly set
PERF_CONFIGto/dev/nullinsidecmd_testto globally sandbox the environment for the entire test workflow. Test specific config, like annotate.objdump, is read before the sandboxing so that 'perf test' still honors it.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Ian Rogers
reviewerv4 cover perf report/top: Add opt-in --hybrid-merge
In reply to Ian RogersOn a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error. For the same reason it is silent when there is nothing to merge, so it doesn't warn on every invocation on a machine with a single core PMU.
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
The events to merge are found with first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.
Changes in v4:
- Patch 7: don't warn that there is nothing to merge when merging came from core.hybrid-merge rather than the command line. In v3, setting the config on a machine with a single core PMU made every perf report and perf top warn, which in the TUI is a popup that has to be dismissed. Track whether the option was given on the command line in symbol_conf so an explicit --hybrid-merge still warns, and document the difference.
Changes in v3:
- Patch 5: remove the redundant if (!pos->merged_hybrid_group) guard around the group member walk in __evlist__enable() and __evlist__disable(), and update the commit message. In v2, that guard claimed to skip walking the members of a merged leader, but merged_hybrid_group is only set on merged members (which have no group members to walk), never on the leader itself.
Changes in v2:
- The interactive 'M' keystroke that toggled merging in the hists browser is gone. Merging is now asked for on the command line or through core.hybrid-merge, and the browser instead points out that --hybrid-merge is available when there are events it could merge.
- Fold "perf ui hist: Format group headers iteratively based on proportional visual allocations" into patch 2, as the two only make sense together.
- New patch 5, so that enabling and disabling a merged set touches every event in it. The merged events are made members of a leader but are not a real group, so each still has to be toggled in its own right.
- New patch 7, adding core.hybrid-merge.
- New patch 9, so that "perf test" doesn't inherit the user's ~/.perfconfig. Without it, setting core.hybrid-merge breaks the output matching of unrelated shell tests.
- Patch 8 adds a C test for the merging decision and a report shell test, rather than only extending the perf top test.
- Rewrite the commit messages of patches 2, 3, 5 and 8 to describe the problem being solved rather than the diff.
Testing:
- New "Hybrid event merging" perf test with three subtests: merging events of 2 core PMUs, of 3 core PMUs, and events that must not be merged.
- New "perf report hybrid merge" shell test, and the perf top shell test extended.
- perf test 13, 17, 27, 30, 31, 66, 67, 68, 78, 102, 113 and 172 pass, covering the hists, --hierarchy and addr2line paths this touches.
- Builds in 13 configurations, and every patch builds on its own so the series stays bisectable.
Ian Rogers (9):
perf evlist: Implement evlist__can_merge_hybrid using
first_wildcard_match
perf ui hist: Add support for aggregated total_period and merging
entries
perf tools: Expose opt-in --hybrid-merge
perf Documentation: Add tip for hybrid event merging perf evlist: Toggle merged_hybrid_group properly in enable/disable perf tools: Add TUI hints for --hybrid-merge
perf config: Add core.hybrid-merge to configure event merging perf test: Expand tests for --hybrid-merge
perf test: Isolate test suite from user .perfconfig natively
tools/perf/Documentation/perf-config.txt | 10 + tools/perf/Documentation/perf-report.txt | 5 + tools/perf/Documentation/perf-top.txt | 5 + tools/perf/Documentation/tips.txt | 2 + tools/perf/builtin-report.c | 31 +++ tools/perf/builtin-top.c | 35 ++- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 50 +++- tools/perf/tests/hybrid-merge.c | 206 +++++++++++++++ tools/perf/tests/shell/report_hybrid_merge.sh | 121 +++++++++ tools/perf/tests/shell/top.sh | 73 +++++- tools/perf/tests/tests.h | 1 + tools/perf/ui/browsers/hists.c | 7 +- tools/perf/ui/hist.c | 246 +++++++++++++++++- tools/perf/util/config.c | 10 + tools/perf/util/evlist.c | 218 +++++++++++++++- tools/perf/util/evlist.h | 3 + tools/perf/util/evsel.h | 1 + tools/perf/util/hist.h | 15 +- tools/perf/util/parse-events.c | 4 +- tools/perf/util/pmu.c | 43 ++- tools/perf/util/pmu.h | 3 + tools/perf/util/symbol.c | 1 + tools/perf/util/symbol_conf.h | 3 + 24 files changed, 1054 insertions(+), 40 deletions(-) create mode 100644 tools/perf/tests/hybrid-merge.c create mode 100755 tools/perf/tests/shell/report_hybrid_merge.sh
base-commit: 91b0782fc9e9d2f0a40b5256146e014802fdbb36
prerequisite-patch-id: b6fdc526887b71fb66f5fe0c0d41f7ef9493861e
prerequisite-patch-id: d47077f674c700f4f6296e9367f7ddfe004aea89
prerequisite-patch-id: ed0db23450840601762fe85ca1bef06ce6d28fe7
prerequisite-patch-id: d84e6b96d533ee6ed549e26e94216e2da60f7f74
prerequisite-patch-id: 6e8f19551d771621a5037096626cfe7e271b36f9
prerequisite-patch-id: 3768dfd588c7439deb741bce44c74763011c6be7
prerequisite-patch-id: babac99a4faa3b1525e44d3f34ab33a88103334c
prerequisite-patch-id: 5279827453fea1536abc620aa33c887c44e41ee9
prerequisite-patch-id: bdc0d648a577142b9270fb59b54496bc2bf8d5ea
prerequisite-patch-id: f840408dcc3cc637a071f2540b31cc7b8d2ca146
prerequisite-patch-id: 8f4f390a5a60f62b61c6c9220e96bfce155fa233
prerequisite-patch-id: bc536fd240b5ddd999c781831a6445cb04ace25c
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Ian Rogers
reviewerPATCH v4 1/9 perf evlist: Implement evlist__can_merge_hybrid using first_wildcard_match
Add logic to dynamically identify mergeable events spawned from the same wildcard alias via first_wildcard_match. first_wildcard_match is populated when events are parsed, in perf report the events come from a file and so the first_wildcard_match is computed by matching event names.
evlist__can_merge_hybrid only tests whether merging is possible and doesn't modify the evlist, so that it may be used as a predicate, for example, to decide whether to display a hint. The linking of all the matching events is done by evlist__merge_hybrid.
is_pmu_core_len identifies core PMUs using the hybrid topology of the perf.data file. Should the file lack the topology, the PMUs of this machine are scanned but only when the file was recorded on the same architecture, as PMU names like "cpu_core" are architecture specific.
--- tools/perf/util/evlist.c | 183 +++++++++++++++++++++++++++++++++ tools/perf/util/evlist.h | 3 + tools/perf/util/evsel.h | 1 + tools/perf/util/hist.h | 1 + tools/perf/util/parse-events.c | 4 +- tools/perf/util/pmu.c | 43 +++++++- tools/perf/util/pmu.h | 3 + tools/perf/util/symbol_conf.h | 1 + 8 files changed, 236 insertions(+), 3 deletions(-)
tools/perf/util/evlist.c +183 -0
@@ -7,6 +7,7 @@*/ #include "evlist.h" +#include <stdio.h> #include <errno.h> #include <inttypes.h> #include <signal.h>
@@ -53,6 +54,7 @@#include "event.h" #include "evsel.h" #include "evsel_fprintf.h" +#include "hist.h" #include "intel-tpebs.h" #include "metricgroup.h" #include "mmap.h"
@@ -125,12 +127,24 @@ struct evlist *evlist__new_default(const struct target *target, bool sample_callif (err) goto out_err; }else { +struct evsel *leader =NULL ; + +/* Fallback for cross-platform file analysis missing the topology header */ while ((pmu =perf_pmus__scan_core (pmu)) !=NULL ) {snprintf (buf,sizeof (buf),"%s/cycles/%s" , pmu->name, can_profile_kernel ?"P" :"Pu" ); err =parse_event (evlist, buf);if (err) goto out_err; + +if (!leader) { + leader =evlist__last (evlist); + }else { +struct evsel *last =evlist__last (evlist); + +if (last != leader) + last->first_wildcard_match = leader; + } } }
@@ -148,6 +162,175 @@ struct evlist *evlist__new_default(const struct target *target, bool sample_callreturn NULL ; } +/** +* evlist__hybrid_matches - find events that may be merged across core PMUs. +* @evlist: The evlist to check. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +* @link: Should the matches be recorded in first_wildcard_match? +* +* perf record, top, etc. set first_wildcard_match in the event parsing. The +* perf.data case (e.g. perf report) recomputes the first_wildcard_match using +* string matches for core events on hybrid systems. +*/ +static bool evlist__hybrid_matches (struct evlist *evlist,struct perf_env *env,bool link) +{ +struct evsel *pos; +unsigned int nr =0 ; +bool found = false; + +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos)) +continue ; + +if (pos->core.leader != &pos->core || pos->core.nr_members >1 ) +return false; + +if (pos->first_wildcard_match) + found = true; + nr++; + } + +/* No events found. */ +if (nr <=1 ) +return false; + +/* Found wildcard events from parsing. */ +if (found) +return true; + +/* Try to find matching core events and set the first_wildcard_match. */ +evlist__for_each_entry (evlist, pos) { +const char *pos_name; +const char *pos_match; +struct evsel *peer; + +if (evsel__is_dummy_event (pos) || pos->first_wildcard_match) +continue ; + + pos_name =evsel__name (pos); + pos_match = pos_name ?strchr (pos_name,'/' ) :NULL ; + +if (!pos_match) +continue ; + +/* If evsel->core.is_pmu_core missing in report, fallback to perf_env */ +if (!evsel__is_hybrid (pos)) { +if (!is_pmu_core_len (env, pos_name, pos_match - pos_name)) +continue ; + } + + peer = pos; +list_for_each_entry_continue (peer, &evlist__core (evlist)->entries, core.node) { +const char *peer_name; +const char *peer_match; + +if (evsel__is_dummy_event (peer) || peer->first_wildcard_match) +continue ; + + peer_name =evsel__name (peer); + peer_match = peer_name ?strchr (peer_name,'/' ) :NULL ; +if (!peer_match) +continue ; + +if (!evsel__is_hybrid (peer)) { +if (!is_pmu_core_len (env, peer_name, + peer_match - peer_name)) +continue ; + } + +if (strcmp (pos_match, peer_match)) +continue ; + + found = true; +if (!link) { +/* Just a test, don't modify the evlist. */ +return true; + } +/* +* Keep looking so that all the events of this name are +* linked, there may be more than 2 core PMUs. +*/ + peer->first_wildcard_match = pos; + } + } +return found; +} + +/** +* evlist__can_merge_hybrid - can events in the evlist be merged across core +* PMUs? The evlist isn't modified. +* @evlist: The evlist to check. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +*/ +bool evlist__can_merge_hybrid (struct evlist *evlist,struct perf_env *env) +{ +return evlist__hybrid_matches (evlist, env,/*link=*/ false); +} + +/* +* evlist__merge_hybrid - group hybrid events logically together. +* @evlist: The evlist containing events to merge. +* @env: The perf_env containing the PMU mapping information, NULL for the +* current machine. +* +* Iterates through the evlist and merges associated hybrid events by assigning +* their first_wildcard_match as their core group leader, modifying their +* presentation for a single merged histogram view. +*/ +void evlist__merge_hybrid (struct evlist *evlist,struct perf_env *env) +{ +struct list_head new_list; +struct evsel *member, *mtmp; +struct evsel *pos, *tmp; +int idx =0 ; + +/* Compute first_wildcard_match for events that lack it, say from a file. */ +if (!evlist__hybrid_matches (evlist, env,/*link=*/ true)) +return ; + +evlist__for_each_entry_safe (evlist, tmp, pos) { +struct evsel *leader, *old_leader; + +if (evsel__is_dummy_event (pos)) +continue ; + +if (!pos->first_wildcard_match) +continue ; + + leader =evsel__leader (pos->first_wildcard_match); + old_leader =evsel__leader (pos); +if (old_leader == leader) +continue ; + +if (old_leader != pos) + old_leader->core.nr_members--; + pos->core.leader = &leader->core; + pos->merged_hybrid_group = true; +/* Base is 1 to natively represent the leader */ +if (leader->core.nr_members ==0 ) + leader->core.nr_members =1 ; + leader->core.nr_members++; + } + +INIT_LIST_HEAD (&new_list); + +while (!list_empty (&evlist__core (evlist)->entries)) { + pos = list_first_entry(&evlist__core(evlist)->entries, struct evsel, core.node); +list_move_tail (&pos->core.node, &new_list); + +list_for_each_entry_safe (member, mtmp, &evlist__core (evlist)->entries, core.node) { +if (member->core.leader == &pos->core) +list_move_tail (&member->core.node, &new_list); + } + } +list_splice_init (&new_list, &evlist__core (evlist)->entries); + +evlist__for_each_entry (evlist, pos) + pos->core.idx = idx++; +} +struct evlist *evlist__new_dummy (void ) {struct evlist *evlist =evlist__new ();
tools/perf/util/evlist.h +3 -0
@@ -331,6 +331,9 @@ static inline void evlist__set_selected(struct evlist *evlist, struct evsel *evsstruct evlist *evlist__new (void );struct evlist *evlist__new_default (const struct target *target,bool sample_callchains); +struct perf_env ; +bool evlist__can_merge_hybrid (struct evlist *evlist,struct perf_env *env); +void evlist__merge_hybrid (struct evlist *evlist,struct perf_env *env);struct evlist *evlist__new_dummy (void );struct evlist *evlist__get (struct evlist *evlist);void evlist__put (struct evlist *evlist);
tools/perf/util/evsel.h +1 -0
@@ -53,6 +53,7 @@ struct evsel {int id_pos;int is_pos;unsigned int sample_size; +bool merged_hybrid_group;/* * These fields can be set in the parse-events code or similar.
tools/perf/util/hist.h +1 -0
@@ -130,6 +130,7 @@ struct hists {struct hists_stats stats;u64 event_stream;u16 col_len[HISTC_NR_COLS ]; +bool merge_entries;bool has_callchains;int socket_filter;struct perf_hpp_list *hpp_list;
tools/perf/util/parse-events.c +2 -2
@@ -1683,7 +1683,7 @@ int parse_events_multi_pmu_add(struct parse_events_state *parse_state,strbuf_release (&sb); ok++; } -if (first_wildcard_match ==NULL ) +if (first_wildcard_match ==NULL && !list_empty (list)) first_wildcard_match = container_of(list->prev, struct evsel, core.node); }
@@ -1754,7 +1754,7 @@ int parse_events_multi_pmu_add_or_add_pmu(struct parse_events_state *parse_state ok++; parse_state->wild_card_pmus = true; } -if (first_wildcard_match ==NULL ) { +if (first_wildcard_match ==NULL && !list_empty (*listp)) { first_wildcard_match = container_of((*listp)->prev, struct evsel, core.node); }
tools/perf/util/pmu.c +42 -1
55 lines not on this page · raw message
tools/perf/util/pmu.h +3 -0
@@ -16,6 +16,7 @@struct evsel_config_term ;struct hashmap ;struct perf_cpu_map ; +struct perf_env ;struct print_callbacks ;enum {
@@ -289,6 +290,8 @@ int perf_pmu__for_each_format(struct perf_pmu *pmu, void *state, pmu_format_callu64 perf_pmu__format_unpack (unsigned long *format,u64 config_val);bool is_pmu_core (const char *name); +bool is_pmu_core_len (struct perf_env *env,const char *name,size_t len); +bool perf_pmu__supports_legacy_cache (const struct perf_pmu *pmu);bool perf_pmu__auto_merge_stats (const struct perf_pmu *pmu);bool perf_pmu__have_event (struct perf_pmu *pmu,const char *name);
tools/perf/util/symbol_conf.h +1 -0
@@ -38,6 +38,7 @@ enum symbol__weight_mode {for ((_weight) =WEIGHT_WEIGHT ; (_weight) <=WEIGHT_WEIGHT3 ; (_weight)++) struct symbol_conf { +bool hybrid_merge;bool nanosecs;unsigned short priv_size; bool try_vmlinux_path,
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 1/9] perf evlist: Implement evlist__can_merge_hybrid using first_wildcard_match
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 11 lines
Add logic to dynamically identify mergeable events spawned from the same wildcard alias via
first_wildcard_match.first_wildcard_matchis populated when events are parsed, inperf reportthe events come from a file and so thefirst_wildcard_matchis computed by matching event names.
evlist__can_merge_hybridonly tests whether merging is possible and doesn't modify the evlist, so that it may be used as a predicate, for example, to decide whether to display a hint. The linking of all the matching events is done byevlist__merge_hybrid.
Quoted 6 lines
is_pmu_core_lenidentifies core PMUs using the hybrid topology of the perf.data file. Should the file lack the topology, the PMUs of this machine are scanned but only when the file was recorded on the same architecture, as PMU names like "cpu_core" are architecture specific.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v4 2/9 perf ui hist: Add support for aggregated total_period and merging entries
In reply to Ian RogersAdd the histogram side of hybrid event merging.
When events are merged the entries of the other events are linked to the leader's entries as pairs, and a row is then displayed as the sum over an entry and its pairs. Teach the hpp code to work on that sum:
- The formatting and sorting routines add up the value of an entry and of each of its pairs, so a row shows, and is sorted by, the total across the core PMUs rather than the leader's value alone.
- Percentages are taken against the summed total_period of the merged hists. Computing each event's share against its own PMU's total would leave the column not adding up to 100%. hist_entry__get_percent_limit_merged() does the same for --percent-limit, which would otherwise compare against the wrong total and filter out the wrong entries.
- A header and column width are reserved for the "Total" the merged value is shown under.
evlist__merge_hists_hybrid() links the hists together and resorts the result so that it is ordered by the merged total.
hists__link() is run once for every event merged into the leader, so an entry can be offered as a pair more than once. hist_entry__add_pair() used list_add_tail() unconditionally, which corrupts the list when the entry is already on one. Return early if the entry is already paired with this one, and use list_move_tail() so that an entry linked elsewhere is moved rather than added a second time.
Merging is never combined with --hierarchy, which the merge_entries tests rely on.
--- tools/perf/ui/hist.c | 246 +++++++++++++++++++++++++++++++++++++++-- tools/perf/util/hist.h | 14 ++- 2 files changed, 252 insertions(+), 8 deletions(-)
tools/perf/ui/hist.c +239 -7
@@ -57,8 +57,9 @@ struct hpp_fmt_value { };static int __hpp__fmt (struct perf_hpp *hpp,struct hist_entry *he, -hpp_field_fn get_field,const char *fmt,int len, -hpp_snprint_fn print_fn,enum perf_hpp_fmt_type fmtype) +hpp_field_fn get_field,const char *fmtstr,int len, +hpp_snprint_fn print_fn,enum perf_hpp_fmt_type fmtype, +struct perf_hpp_fmt *fmt__maybe_unused ) {int ret =0 ;struct hists *hists = he->hists;
@@ -98,13 +99,50 @@ static int __hpp__fmt(struct perf_hpp *hpp, struct hist_entry *he, } } +/* Note, merge_entries implies !symbol_conf.report_hierarchy. */ +if (he->hists->merge_entries) { +u64 total_val =0 ; +u64 total_samples =0 ; +u64 total_period =0 ; + +for (i =0 ; i < nr_members; i++) { +struct evsel *member_evsel =hists_to_evsel (values[i].hists); + +struct evsel *he_evsel =hists_to_evsel (he->hists); + +if (member_evsel != he_evsel && + member_evsel->first_wildcard_match != he_evsel) +continue ; + + total_val += values[i].val; + total_samples += values[i].samples; + total_period += fmtype ==PERF_HPP_FMT_TYPE__PERCENT ? +hists__total_period (values[i].hists) : +hists__total_latency (values[i].hists); + } + +if (fmtype ==PERF_HPP_FMT_TYPE__PERCENT || fmtype ==PERF_HPP_FMT_TYPE__LATENCY ) { +double percent =0.0 ; + +if (total_period) + percent =100.0 * total_val / total_period; + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, percent); + }else if (fmtype ==PERF_HPP_FMT_TYPE__AVERAGE ) { +double avg = total_samples ? (1.0 * total_val / total_samples) :0 ; + + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, avg); + }else { + ret +=hpp__call_print_fn (hpp, print_fn, fmtstr, len, total_val); + } + } +for (i =0 ; i < nr_members; i++) {if (symbol_conf.skip_empty && values[i].hists->stats.nr_samples ==0 )continue ; ret +=__hpp__fmt_print (hpp, values[i].hists, values[i].val, - values[i].samples, fmt, len, + values[i].samples, fmtstr, len, print_fn, fmtype); }
@@ -129,7 +167,7 @@ int hpp__fmt(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,if (symbol_conf.field_sep) {return __hpp__fmt (hpp, he, get_field, fmtstr,1 , - print_fn, fmtype); + print_fn, fmtype, fmt); }if (fmtype ==PERF_HPP_FMT_TYPE__PERCENT || fmtype ==PERF_HPP_FMT_TYPE__LATENCY )
@@ -137,7 +175,7 @@ int hpp__fmt(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,else len -=1 ; -return __hpp__fmt (hpp, he, get_field, fmtstr, len, print_fn, fmtype); +return __hpp__fmt (hpp, he, get_field, fmtstr, len, print_fn, fmtype, fmt); } int hpp__fmt_acc(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,
@@ -287,6 +325,35 @@ static int __hpp__sort(struct hist_entry *a, struct hist_entry *b,return __hpp__group_sort_idx (a, b, get_field, symbol_conf.group_sort_idx); } +/* +* Relies on merge_entries being only enabled if there are +* only matching events. If that is ever relaxed will need +* more logic here. merge_entries also implies that +* symbol_conf.report_hierarchy is false. +*/ +if (a->hists->merge_entries && b->hists->merge_entries) { +u64 val_a =get_field (a), val_b =get_field (b); +struct hist_entry *pair; +struct evsel *evsel_a =hists_to_evsel (a->hists); +struct evsel *evsel_b =hists_to_evsel (b->hists); + +list_for_each_entry (pair, &a->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_a) + val_a +=get_field (pair); + } +list_for_each_entry (pair, &b->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_b) + val_b +=get_field (pair); + } + + ret =field_cmp (val_a, val_b); +if (ret) +return ret; + } ret =field_cmp (get_field (a),get_field (b));if (ret || !symbol_conf.event_group)
@@ -323,7 +390,29 @@ static int __hpp__sort_acc(struct hist_entry *a, struct hist_entry *b,/* * Put caller above callee when they have equal period. */ - ret =field_cmp (get_field (a),get_field (b)); +if (a->hists->merge_entries && b->hists->merge_entries) { +u64 val_a =get_field (a), val_b =get_field (b); +struct hist_entry *pair; +struct evsel *evsel_a =hists_to_evsel (a->hists); +struct evsel *evsel_b =hists_to_evsel (b->hists); + +list_for_each_entry (pair, &a->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_a) + val_a +=get_field (pair); + } +list_for_each_entry (pair, &b->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel_b) + val_b +=get_field (pair); + } + + ret =field_cmp (val_a, val_b); + }else { + ret =field_cmp (get_field (a),get_field (b)); + }if (ret)return ret;
@@ -386,6 +475,8 @@ static int hpp__width_fn(struct perf_hpp_fmt *fmt, evsel__hists(pos)->stats.nr_samples) nr++; } +if (hists->merge_entries) + nr++;/* Add 1 extra unit of width generically for the 'Total' */ len =max (len, nr * fmt->len); }
@@ -403,8 +494,38 @@ static int hpp__header_fn(struct perf_hpp_fmt *fmt, struct perf_hpp *hpp,int len =hpp__width_fn (fmt, hpp, hists);const char *hdr ="" ; -if (line == hists->hpp_list->nr_header_lines -1 ) +if (line == hists->hpp_list->nr_header_lines -1 ) { hdr = fmt->name; +if (hists->merge_entries) { +int w =0 ; +int f_len = fmt->user_len ?: fmt->len; +struct evsel *pos, *evsel =hists_to_evsel (hists); +int string_len = f_len; + +for_each_group_evsel (pos, evsel) { +if (symbol_conf.skip_empty && +evsel__hists (pos)->stats.nr_samples ==0 ) +continue ; + string_len += f_len; + } + +if (len > string_len) { + w +=scnprintf (hpp->buf + w, hpp->size - w,"%*s" , + len - string_len,"" ); + } + + w +=scnprintf (hpp->buf + w, hpp->size - w,"%*.*s" , + f_len, f_len, fmt->name); +for_each_group_evsel (pos, evsel) { +if (symbol_conf.skip_empty && +evsel__hists (pos)->stats.nr_samples ==0 ) +continue ; + w +=scnprintf (hpp->buf + w, hpp->size - w," %*.*s" , + f_len -1 , f_len -1 ,evsel__name (pos)); + } +return w; + } + }return scnprintf (hpp->buf, hpp->size,"%*s" , len, hdr); }
@@ -1271,3 +1392,114 @@ int perf_hpp__alloc_mem_stats(struct perf_hpp_list *list, struct evlist *evlist) }return 0 ; } + +float hist_entry__get_percent_limit_merged (struct hist_entry *he) +{ +struct hist_entry *pair; +u64 period = he->stat.period; +u64 total_period =hists__total_period (he->hists); +struct evsel *evsel =hists_to_evsel (he->hists); +struct evsel *pos; + +/* Accumulate global total_period across all merged hists matching hybrid type */ +for_each_group_member (pos, evsel) { +if (pos->first_wildcard_match == evsel) + total_period +=hists__total_period (evsel__hists (pos)); + } + +if (unlikely (total_period ==0 )) +return 0 ; + +if (symbol_conf.cumulate_callchain) { + period = he->stat_acc->period; +list_for_each_entry (pair, &he->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel) + period += pair->stat_acc->period; + } + }else { +/* Accumulate symbol specific period across pairs matching hybrid type */ +list_for_each_entry (pair, &he->pairs.head, pairs.node) { +struct evsel *pair_evsel =hists_to_evsel (pair->hists); + +if (pair_evsel->first_wildcard_match == evsel) + period += pair->stat.period; + } + } + +return period *100.0 / total_period; +} + +void evlist__merge_hists_hybrid (struct evlist *evlist,bool refresh_hists) +{ +struct evsel *pos; +struct evsel *member; +bool hybrid_group; + +/* +* Merged hists display all the events of a group in a single set of +* entries, which the hierarchy display has no way to render. Keeping +* hists->merge_entries false here means the rest of the display code +* can assume merge_entries implies !symbol_conf.report_hierarchy. +*/ +if (symbol_conf.report_hierarchy) +return ; + +/* Set merge_entries flag strictly on leaders formulated by hybrid topology */ +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos)) +continue ; + +if (pos->core.leader == &pos->core && pos->core.nr_members >1 ) { + hybrid_group = false; + +if (pos->first_wildcard_match || pos->merged_hybrid_group) { + hybrid_group = true; + }else { +for_each_group_member (member, pos) { +if (member->first_wildcard_match || + member->merged_hybrid_group) { + hybrid_group = true; +break ; + } + } + } + +if (hybrid_group) { +evsel__hists (pos)->merge_entries = true; + symbol_conf.event_group = true; + symbol_conf.hybrid_merge = true; +if (!pos->group_name) { + pos->group_name =strdup ("Merged hybrid events" ); +if (!pos->group_name) +pr_warning ("Failed to allocate hybrid group name\n" ); + } + } + } + } + +if (!refresh_hists) +return ; + +evlist__for_each_entry (evlist, pos) { +/* Match histograms dynamically since parsing happened before group toggling */ +if (symbol_conf.event_group && !evsel__is_group_leader (pos)) { +struct hists *leader_hists =evsel__hists (evsel__leader (pos)); +struct hists *hists =evsel__hists (pos); + +hists__match (leader_hists, hists); +hists__link (leader_hists, hists); + } + } + +/* Now that links are formed, safely resort the active tree so the UI renders accurately */ +if (symbol_conf.event_group) { +evlist__for_each_entry (evlist, pos) { +if (evsel__is_dummy_event (pos) || !evsel__is_group_leader (pos)) +continue ; +if (pos->core.nr_members >1 &&evsel__hists (pos)->merge_entries) +hists__output_resort (evsel__hists (pos),NULL ); + } + } +}
tools/perf/util/hist.h +13 -1
@@ -339,7 +339,13 @@ static inline struct hist_entry *hist_entry__next_pair(struct hist_entry *he)static inline void hist_entry__add_pair (struct hist_entry *pair,struct hist_entry *he) { -list_add_tail (&pair->pairs.node, &he->pairs.head); +struct list_head *pos; + +list_for_each (pos, &he->pairs.head) { +if (pos == &pair->pairs.node) +return ;/* Already paired */ + } +list_move_tail (&pair->pairs.node, &he->pairs.head); } struct hist_entry *hists__add_entry(struct hists *hists,
@@ -434,11 +440,17 @@ void hists__match(struct hists *leader, struct hists *other);int hists__link (struct hists *leader,struct hists *other);int hists__unlink (struct hists *hists); +float hist_entry__get_percent_limit_merged (struct hist_entry *he); +void evlist__merge_hists_hybrid (struct evlist *evlist,bool refresh); +static inline float hist_entry__get_percent_limit (struct hist_entry *he) {u64 period = he->stat.period;u64 total_period =hists__total_period (he->hists); +if (he->hists->merge_entries) +return hist_entry__get_percent_limit_merged (he); +if (unlikely (total_period ==0 ))return 0 ;
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 2/9] perf ui hist: Add support for aggregated total_period and merging entries
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 17 lines
Add the histogram side of hybrid event merging.
When events are merged the entries of the other events are linked to the leader's entries as pairs, and a row is then displayed as the sum over an entry and its pairs. Teach the hpp code to work on that sum:
- The formatting and sorting routines add up the value of an entry and of each of its pairs, so a row shows, and is sorted by, the total across the core PMUs rather than the leader's value alone.
- Percentages are taken against the summedtotal_periodof the merged hists. Computing each event's share against its own PMU's total would leave the column not adding up to 100%.hist_entry__get_percent_limit_merged()does the same for --percent-limit, which would otherwise compare against the wrong total and filter out the wrong entries.
[ ... ]
Quoted 4 lines
Merging is never combined with --hierarchy, which the
merge_entriestests rely on.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerPATCH v4 3/9 perf tools: Expose opt-in --hybrid-merge
In reply to Ian RogersAdd --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
evlist__merge_hybrid() links the events and evlist__merge_hists_hybrid() links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.
Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
--- tools/perf/Documentation/perf-report.txt | 5 +++++ tools/perf/Documentation/perf-top.txt | 5 +++++ tools/perf/builtin-report.c | 19 +++++++++++++++++++ tools/perf/builtin-top.c | 17 +++++++++++++++++ tools/perf/util/symbol.c | 1 + 5 files changed, 47 insertions(+)
tools/perf/Documentation/perf-report.txt +5 -0
@@ -578,6 +578,11 @@ include::itrace.txt[] --raw-trace:: When displaying traceevent output, do not use print fmt or plugins. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + -H:: --hierarchy:: Enable hierarchical output. In the hierarchy mode, each sort key groups
tools/perf/Documentation/perf-top.txt +5 -0
@@ -49,6 +49,11 @@ Default is to monitor all CPUS. encoding with the layout of the event control registers as described by entries in /sys/bus/event_source/devices/cpu/format/*. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + --filter=<filter>:: Event filter. This option should follow an event selector (-e). For syntax see linkperf:perf-record[1].
tools/perf/builtin-report.c +19 -0
@@ -1114,6 +1114,17 @@ static int __cmd_report(struct report *rep)evlist__for_each_entry (session->evlist, pos) rep->nr_entries +=evsel__hists (pos)->nr_entries; +if (symbol_conf.hybrid_merge) { +struct perf_env *env =perf_session__env (session); + +if (evlist__can_merge_hybrid (session->evlist, env)) { +evlist__merge_hybrid (session->evlist, env); +evlist__merge_hists_hybrid (session->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } +if (use_browser ==0 ) {if (verbose >3 )perf_session__fprintf (session, stdout);
@@ -1449,6 +1460,8 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",
@@ -1548,6 +1561,12 @@ int cmd_report(int argc, const char **argv) report.symbol_filter_str = argv[0 ]; } +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style)
tools/perf/builtin-top.c +17 -0
@@ -1336,6 +1336,15 @@ static int __cmd_top(struct perf_top *top)if (!target__none (&opts->target))evlist__enable (top->evlist); +if (symbol_conf.hybrid_merge) { +if (evlist__can_merge_hybrid (top->evlist,/*env=*/ NULL )) { +evlist__merge_hybrid (top->evlist,/*env=*/ NULL ); +evlist__merge_hists_hybrid (top->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } + ret =-1 ;if (pthread_create (&thread_process,NULL , process_thread, top)) {ui__error ("Could not create process thread.\n" );
@@ -1491,6 +1500,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),
@@ -1679,6 +1690,12 @@ int cmd_top(int argc, const char **argv)if (argc)usage_with_options (top_usage, options); +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style) {
tools/perf/util/symbol.c +1 -0
@@ -74,6 +74,7 @@ struct symbol_conf symbol_conf = { .symfs = "", .symfs_layout_flat = false, .event_group = true, + .hybrid_merge = false, .inline_name = true, .res_sample = 0, .addr2line_timeout_ms = 5 * 1000,
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 3/9] perf tools: Expose opt-in --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Add --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
evlist__merge_hybrid()links the events andevlist__merge_hists_hybrid()links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.
Quoted 5 lines
Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v4 3/9] perf tools: Expose opt-in --hybrid-merge
In reply to Ian RogersQuoted 5 lines · Ian Rogers
Add --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
Quoted 8 lines
evlist__merge_hybrid()links the events andevlist__merge_hists_hybrid()links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
It would be interesting to have a hotkey to switch to this mode on the fly and back, even if it required to reset everything when doing so, but probably it should be possible on the -> hybrid-merge way.
- Arnaldo
Quoted 142 lines
Signed-off-by Ian Rogers <[email protected]>Assisted-by Antigravity:gemini-3.1-pro--- tools/perf/Documentation/perf-report.txt | 5 +++++ tools/perf/Documentation/perf-top.txt | 5 +++++ tools/perf/builtin-report.c | 19 +++++++++++++++++++ tools/perf/builtin-top.c | 17 +++++++++++++++++ tools/perf/util/symbol.c | 1 + 5 files changed, 47 insertions(+)tools/perf/Documentation/perf-report.txt +5
-0@@ -578,6 +578,11 @@ include::itrace.txt[] --raw-trace:: When displaying traceevent output, do not use print fmt or plugins. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + -H:: --hierarchy:: Enable hierarchical output. In the hierarchy mode, each sort key groupstools/perf/Documentation/perf-top.txt +5
-0@@ -49,6 +49,11 @@ Default is to monitor all CPUS. encoding with the layout of the event control registers as described by entries in /sys/bus/event_source/devices/cpu/format/*. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + --filter=<filter>:: Event filter. This option should follow an event selector (-e). For syntax see linkperf:perf-record[1].tools/perf/builtin-report.c +19
-0@@ -1114,6 +1114,17 @@ static int __cmd_report(struct report *rep)evlist__for_each_entry (session->evlist, pos) rep->nr_entries +=evsel__hists (pos)->nr_entries; +if (symbol_conf.hybrid_merge) { +struct perf_env *env =perf_session__env (session); + +if (evlist__can_merge_hybrid (session->evlist, env)) { +evlist__merge_hybrid (session->evlist, env); +evlist__merge_hists_hybrid (session->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } +if (use_browser ==0 ) {if (verbose >3 )perf_session__fprintf (session, stdout);@@ -1449,6 +1460,8 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",@@ -1548,6 +1561,12 @@ int cmd_report(int argc, const char **argv) report.symbol_filter_str = argv[0 ]; } +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style)tools/perf/builtin-top.c +17
-0@@ -1336,6 +1336,15 @@ static int __cmd_top(struct perf_top *top)if (!target__none (&opts->target))evlist__enable (top->evlist); +if (symbol_conf.hybrid_merge) { +if (evlist__can_merge_hybrid (top->evlist,/*env=*/ NULL )) { +evlist__merge_hybrid (top->evlist,/*env=*/ NULL ); +evlist__merge_hists_hybrid (top->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } + ret =-1 ;if (pthread_create (&thread_process,NULL , process_thread, top)) {ui__error ("Could not create process thread.\n" );@@ -1491,6 +1500,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),@@ -1679,6 +1690,12 @@ int cmd_top(int argc, const char **argv)if (argc)usage_with_options (top_usage, options); +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style) {tools/perf/util/symbol.c +1
-0@@ -74,6 +74,7 @@ struct symbol_conf symbol_conf = { .symfs = "", .symfs_layout_flat = false, .event_group = true, + .hybrid_merge = false, .inline_name = true, .res_sample = 0, .addr2line_timeout_ms = 5 * 1000,Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
Ian Rogers
reviewerRe: [PATCH v4 3/9] perf tools: Expose opt-in --hybrid-merge
Quoted 12 lines · Arnaldo Carvalho de Melo
Quoted 10 lines · Ian Rogers
Add --hybrid-merge to perf report and perf top. It merges the events a wildcard expanded to across the core PMUs, and their histograms, so that a symbol which ran on more than one kind of core is reported once with the total rather than once per PMU.
evlist__merge_hybrid()links the events andevlist__merge_hists_hybrid()links the histograms. There is nothing to merge on a machine with a single core PMU, or when the events didn't come from a wildcard, so warn in that case rather than quietly producing an unmerged report.
Quoted 7 lines
Quoted 3 lines
Merging collapses the per-PMU entries into one set, which doesn't combine with the per-level breakdown of --hierarchy, so asking for both is an error.
It would be interesting to have a hotkey to switch to this mode on the fly and back, even if it required to reset everything when doing so, but probably it should be possible on the -> hybrid-merge way.
In an earlier series I did this with an 'M' key. There were issues with other state based on the unmerged evlist and race conditions during the merge. Ultimately, the command line option was the easiest and most correct choice. We can always add a hotkey later.
Thanks,
Ian
Quoted 144 lines
- Arnaldo
Quoted 142 lines
Signed-off-by Ian Rogers <[email protected]>Assisted-by Antigravity:gemini-3.1-pro--- tools/perf/Documentation/perf-report.txt | 5 +++++ tools/perf/Documentation/perf-top.txt | 5 +++++ tools/perf/builtin-report.c | 19 +++++++++++++++++++ tools/perf/builtin-top.c | 17 +++++++++++++++++ tools/perf/util/symbol.c | 1 + 5 files changed, 47 insertions(+)tools/perf/Documentation/perf-report.txt +5
-0@@ -578,6 +578,11 @@ include::itrace.txt[] --raw-trace:: When displaying traceevent output, do not use print fmt or plugins. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + -H:: --hierarchy:: Enable hierarchical output. In the hierarchy mode, each sort key groupstools/perf/Documentation/perf-top.txt +5
-0@@ -49,6 +49,11 @@ Default is to monitor all CPUS. encoding with the layout of the event control registers as described by entries in /sys/bus/event_source/devices/cpu/format/*. +--hybrid-merge:: + Merge matching events from all hybrid core PMUs into one + display. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. + --filter=<filter>:: Event filter. This option should follow an event selector (-e). For syntax see linkperf:perf-record[1].tools/perf/builtin-report.c +19
-0@@ -1114,6 +1114,17 @@ static int __cmd_report(struct report *rep)evlist__for_each_entry (session->evlist, pos) rep->nr_entries +=evsel__hists (pos)->nr_entries; +if (symbol_conf.hybrid_merge) { +struct perf_env *env =perf_session__env (session); + +if (evlist__can_merge_hybrid (session->evlist, env)) { +evlist__merge_hybrid (session->evlist, env); +evlist__merge_hists_hybrid (session->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } +if (use_browser ==0 ) {if (verbose >3 )perf_session__fprintf (session, stdout);@@ -1449,6 +1460,8 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",@@ -1548,6 +1561,12 @@ int cmd_report(int argc, const char **argv) report.symbol_filter_str = argv[0 ]; } +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style)tools/perf/builtin-top.c +17
-0@@ -1336,6 +1336,15 @@ static int __cmd_top(struct perf_top *top)if (!target__none (&opts->target))evlist__enable (top->evlist); +if (symbol_conf.hybrid_merge) { +if (evlist__can_merge_hybrid (top->evlist,/*env=*/ NULL )) { +evlist__merge_hybrid (top->evlist,/*env=*/ NULL ); +evlist__merge_hists_hybrid (top->evlist, false); + }else { +ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); + } + } + ret =-1 ;if (pthread_create (&thread_process,NULL , process_thread, top)) {ui__error ("Could not create process thread.\n" );@@ -1491,6 +1500,8 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), +OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),@@ -1679,6 +1690,12 @@ int cmd_top(int argc, const char **argv)if (argc)usage_with_options (top_usage, options); +if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +if (disassembler_style) { annotate_opts.disassembler_style =strdup (disassembler_style);if (!annotate_opts.disassembler_style) {tools/perf/util/symbol.c +1
-1@@ -74,6 +74,7 @@ struct symbol_conf symbol_conf = { .symfs = "", .symfs_layout_flat = false, .event_group = true, + .hybrid_merge = false, .inline_name = true, .res_sample = 0, .addr2line_timeout_ms = 5 * 1000, --2.55.0.1082.g2b9226bbc0-goog
Ian Rogers
reviewerPATCH v4 4/9 perf Documentation: Add tip for hybrid event merging
In reply to Ian RogersAdd informative text outlining the IPC imbalances associated with merging cross-hybrid core events like cycles.
--- tools/perf/Documentation/tips.txt | 1 + 1 file changed, 1 insertion(+)
tools/perf/Documentation/tips.txt +1 -0
@@ -66,3 +66,4 @@ For latency profiling, try: perf record/report --latency For parallelism histogram, try: perf report --hierarchy --sort latency,parallelism,comm,symbol To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s +When merging events like cycles, different core frequencies and instructions per cycle mean the counts may not fairly reflect time spent in a function.
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 4/9] perf Documentation: Add tip for hybrid event merging
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
Add informative text outlining the IPC imbalances associated with merging cross-hybrid core events like cycles.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Ian Rogers
reviewerPATCH v4 5/9 perf evlist: Toggle merged_hybrid_group properly in enable/disable
In reply to Ian RogersMerging makes the merged event the leader of the events of the other core PMUs so their histograms can be linked, but the result isn't a real event group. Each event keeps its own file descriptor and has to be enabled and disabled in its own right.
__evlist__enable(), __evlist__disable() and evlist__is_enabled() skip anything that isn't a group leader, and the first two then walk the group members of the events they do act on. For a merged set that is
backwards: the members are skipped by the leader test, so their file descriptors are never touched, and the walk over the leader's members only updates the bookkeeping in evsel->disabled.
Treat an evsel with merged_hybrid_group set as a leader so that it is enabled and disabled in its own right via its own file descriptors.
perf top enables and disables by event name. A merged member carries the name of its own PMU rather than the name that was asked for, so also match it against its leader's name, otherwise naming the event toggles only part of the merged set.
--- tools/perf/util/evlist.c | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-)
tools/perf/util/evlist.c +26 -9
@@ -753,7 +753,9 @@ static bool evlist__is_enabled(struct evlist *evlist)struct evsel *pos;evlist__for_each_entry (evlist, pos) { -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if (!pos->core.fd) +continue ; +if (!evsel__is_group_leader (pos) && !pos->merged_hybrid_group)continue ;/* If at least one event is enabled, evlist is enabled. */ if (!pos->disabled)
@@ -767,14 +769,19 @@ static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool exclstruct evsel *pos, *member;struct evlist_cpu_iterator evlist_cpu_itr;bool has_imm = false; +bool match;/* Disable 'immediate' events last */ for (int imm =0 ; imm <=1 ; imm++) {evlist__for_each_cpu (evlist_cpu_itr, evlist) { pos = evlist_cpu_itr.evsel; -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (pos->disabled || !evsel__is_group_leader (pos) || !pos->core.fd) +if (pos->disabled || (!evsel__is_group_leader (pos) && + !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;
@@ -789,9 +796,12 @@ static void __evlist__disable(struct evlist *evlist, char *evsel_name, bool excl }evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;
@@ -827,21 +837,28 @@ static void __evlist__enable(struct evlist *evlist, char *evsel_name, bool excl_ {struct evsel *pos, *member;struct evlist_cpu_iterator evlist_cpu_itr; +bool match;evlist__for_each_cpu (evlist_cpu_itr, evlist) { pos = evlist_cpu_itr.evsel; -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;evsel__enable_cpu (pos, evlist_cpu_itr.cpu_map_idx); }evlist__for_each_entry (evlist, pos) { -if (evsel__strcmp (pos, evsel_name)) + match = !evsel__strcmp (pos, evsel_name); +if (!match && pos->merged_hybrid_group &&evsel__leader (pos)) + match = !evsel__strcmp (evsel__leader (pos), evsel_name); +if (!match)continue ; -if (!evsel__is_group_leader (pos) || !pos->core.fd) +if ((!evsel__is_group_leader (pos) && !pos->merged_hybrid_group) || !pos->core.fd)continue ;if (excl_dummy &&evsel__is_dummy_event (pos))continue ;
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 5/9] perf evlist: Toggle merged_hybrid_group properly in enable/disable
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 15 lines
Merging makes the merged event the leader of the events of the other core PMUs so their histograms can be linked, but the result isn't a real event group. Each event keeps its own file descriptor and has to be enabled and disabled in its own right.
__evlist__enable(),__evlist__disable()andevlist__is_enabled()skip anything that isn't a group leader, and the first two then walk the group members of the events they do act on. For a merged set that is
backwards: the members are skipped by the leader test, so their file descriptors are never touched, and the walk over the leader's members only updates the bookkeeping inevsel->disabled.Treat an evsel with
merged_hybrid_groupset as a leader so that it is enabled and disabled in its own right via its own file descriptors.
Quoted 6 lines
perf top enables and disables by event name. A merged member carries the name of its own PMU rather than the name that was asked for, so also match it against its leader's name, otherwise naming the event toggles only part of the merged set.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Ian Rogers
reviewerPATCH v4 6/9 perf tools: Add TUI hints for --hybrid-merge
In reply to Ian RogersAdd dynamic TUI hints prompting users to restart with --hybrid-merge when heterogeneous core PMU events are populated in the sample view.
--- tools/perf/Documentation/tips.txt | 1 + tools/perf/ui/browsers/hists.c | 7 +++++-- 2 files changed, 6 insertions(+), 2 deletions(-)
tools/perf/Documentation/tips.txt +1 -0
@@ -67,3 +67,4 @@ For parallelism histogram, try: perf report --hierarchy --sort latency,paralleli To analyze particular parallelism levels, try: perf report --latency --parallelism=32-64 To see how parallelism changes over time, try: perf report -F time,latency,parallelism --time-quantum=1s When merging events like cycles, different core frequencies and instructions per cycle mean the counts may not fairly reflect time spent in a function. +Use --hybrid-merge to merge matching events from hybrid cores into one view
tools/perf/ui/browsers/hists.c +5 -2
@@ -3580,9 +3580,12 @@ static int perf_evsel_menu__run(struct evsel_menu *menu,const char *title ="Available samples" ;int delay_secs = hbt ? hbt->refresh :0 ;int key; +const char *help_text ="ESC: exit, ENTER|->: Browse histograms" ; -if (ui_browser__show (&menu->b, title, -"ESC: exit, ENTER|->: Browse histograms" ) <0 ) +if (!symbol_conf.hybrid_merge &&evlist__can_merge_hybrid (evlist, menu->env)) + help_text ="ESC: exit, ENTER|->: Browse. Try --hybrid-merge to combine events." ; + +if (ui_browser__show (&menu->b, title, help_text) <0 )return -1 ;while (1 ) {
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 6/9] perf tools: Add TUI hints for --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 4 lines
Add dynamic TUI hints prompting users to restart with --hybrid-merge when heterogeneous core PMU events are populated in the sample view.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Ian Rogers
reviewerPATCH v4 7/9 perf config: Add core.hybrid-merge to configure event merging
In reply to Ian RogersProvide a core.hybrid-merge configuration option in .perfconfig to allow enabling hybrid event aggregation by default, avoiding the need to pass --hybrid-merge explicitly on every invocation.
The config value is a default rather than an explicit request, so with --hierarchy it is ignored with a warning, while giving both --hierarchy and --hybrid-merge on the command line remains an error. For the same reason it is silently ignored when there are no events to merge, as would be the case on any machine with a single core PMU, whereas an explicit --hybrid-merge still warns. Record whether the option came from the command line in symbol_conf so both can tell the two apart.
perf stat has its own merging options for counting, the sampling core.hybrid-merge deliberately doesn't alter it and its --hybrid-merge must still be given explicitly.
--- tools/perf/Documentation/perf-config.txt | 10 ++++++++ tools/perf/builtin-report.c | 24 ++++++++++++++----- tools/perf/builtin-top.c | 30 +++++++++++++++++------- tools/perf/util/config.c | 10 ++++++++ tools/perf/util/symbol_conf.h | 2 ++ 5 files changed, 62 insertions(+), 14 deletions(-)
tools/perf/Documentation/perf-config.txt +10 -0
@@ -217,6 +217,16 @@ core.*:: Sets a timeout (in milliseconds) for parsing 'addr2line' output. The default timeout is 5s. + hybrid-merge:: + Merge matching events from all hybrid core PMUs into one display + by default. For example, if a wildcard expands to run on both p-cores and + e-cores, this aggregates them into a single view. This applies to + 'perf report' and 'perf top', it is ignored with '--hierarchy' and + doesn't alter 'perf stat' where '--hybrid-merge' must be given + explicitly. As a default it is silently ignored when there is + nothing to merge, such as on a machine with a single core PMU, + whereas an explicit '--hybrid-merge' warns. + tui.*, gtk.*:: Subcommands that can be configured here are 'top', 'report' and 'annotate'. These values are booleans, for example:
tools/perf/builtin-report.c +18 -6
@@ -1120,7 +1120,13 @@ static int __cmd_report(struct report *rep)if (evlist__can_merge_hybrid (session->evlist, env)) {evlist__merge_hybrid (session->evlist, env);evlist__merge_hists_hybrid (session->evlist, false); - }else { + }else if (symbol_conf.hybrid_merge_set) { +/* +* Only an explicit --hybrid-merge warns. A +* core.hybrid-merge default is silent, as most +* machines aren't hybrid and there is nothing the +* user needs to do about it. +*/ ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); } }
@@ -1460,8 +1466,9 @@ int cmd_report(int argc, const char **argv) parse_branch_mode),OPT_BOOLEAN (0 ,"branch-history" , &branch_call_mode,"add last branch records to call history" ), -OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, -"merge the same event across hybrid core PMUs" ), +OPT_BOOLEAN_SET (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, + &symbol_conf.hybrid_merge_set, +"merge the same event across hybrid core PMUs" ),OPT_STRING (0 ,"objdump" , &objdump_path,"path" ,"objdump binary to use for disassembly and annotations" ), OPT_STRING(0, "addr2line", &addr2line_path, "path",
@@ -1562,9 +1569,14 @@ int cmd_report(int argc, const char **argv) }if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { -pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); - ret = -EINVAL ; - goto exit; +if (symbol_conf.hybrid_merge_set) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + ret = -EINVAL ; + goto exit; + } +/* A config file default shouldn't fail an explicit option. */ +pr_warning ("core.hybrid-merge ignored: --hierarchy cannot display merged hybrid events\n" ); + symbol_conf.hybrid_merge = false; }if (disassembler_style) {
tools/perf/builtin-top.c +22 -8
@@ -1314,9 +1314,11 @@ static int __cmd_top(struct perf_top *top) }/* -* Use global stat_config that is zero meaning aggr_mode is AGGR_NONE -* and hybrid_merge is false. +* Use global stat_config that is zero meaning aggr_mode is AGGR_NONE. +* Merging affects the event names as merged events share a name, all +* other stat_config behavior is unwanted here. */ + stat_config.hybrid_merge = symbol_conf.hybrid_merge;evlist__uniquify_evsel_names (top->evlist, &stat_config); ret =perf_top__start_counters (top);if (ret)
@@ -1340,7 +1342,13 @@ static int __cmd_top(struct perf_top *top)if (evlist__can_merge_hybrid (top->evlist,/*env=*/ NULL )) {evlist__merge_hybrid (top->evlist,/*env=*/ NULL );evlist__merge_hists_hybrid (top->evlist, false); - }else { + }else if (symbol_conf.hybrid_merge_set) { +/* +* Only an explicit --hybrid-merge warns. A +* core.hybrid-merge default is silent, as most +* machines aren't hybrid and there is nothing the +* user needs to do about it. +*/ ui__warning ("--hybrid-merge: no events to merge across core PMUs\n" ); } }
@@ -1500,8 +1508,9 @@ int cmd_top(int argc, const char **argv)OPT_CALLBACK ('e' ,"event" , &parse_events_option_args,"event" ,"event selector. use 'perf list' to list available events" , parse_events_option), -OPT_BOOLEAN (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, -"merge the same event across hybrid core PMUs" ), +OPT_BOOLEAN_SET (0 ,"hybrid-merge" , &symbol_conf.hybrid_merge, + &symbol_conf.hybrid_merge_set, +"merge the same event across hybrid core PMUs" ),OPT_CALLBACK (0 ,"filter" , &top.evlist,"filter" ,"event filter" , parse_filter), OPT_U64('c', "count", &opts->user_interval, "event period to sample"),
@@ -1691,9 +1700,14 @@ int cmd_top(int argc, const char **argv)usage_with_options (top_usage, options);if (symbol_conf.report_hierarchy && symbol_conf.hybrid_merge) { -pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); - status = -EINVAL ; - goto out_put_evlist; +if (symbol_conf.hybrid_merge_set) { +pr_err ("Error: --hierarchy and --hybrid-merge are mutually exclusive.\n" ); + status = -EINVAL ; + goto out_put_evlist; + } +/* A config file default shouldn't fail an explicit option. */ +pr_warning ("core.hybrid-merge ignored: --hierarchy cannot display merged hybrid events\n" ); + symbol_conf.hybrid_merge = false; }if (disassembler_style) {
tools/perf/util/config.c +10 -0
@@ -470,6 +470,16 @@ static int perf_default_core_config(const char *var, const char *value)if (!strcmp (var,"core.addr2line-disable-warn" )) symbol_conf.addr2line_disable_warn =perf_config_bool (var, value); +if (!strcmp (var,"core.hybrid-merge" )) { +/* +* Note, this is for sampling tools like perf report and top. +* perf stat has its own merging options and the +* stat_config.hybrid_merge of "perf stat --hybrid-merge" is +* deliberately not set here. +*/ + symbol_conf.hybrid_merge =perf_config_bool (var, value); + } +/* Add other config variables here. */ return 0 ; }
tools/perf/util/symbol_conf.h +2 -0
@@ -39,6 +39,8 @@ enum symbol__weight_mode {struct symbol_conf {bool hybrid_merge; +/* Was hybrid_merge given on the command line rather than in a config file? */ +bool hybrid_merge_set;bool nanosecs;unsigned short priv_size; bool try_vmlinux_path,
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 7/9] perf config: Add core.hybrid-merge to configure event merging
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 13 lines
Provide a core.hybrid-merge configuration option in .perfconfig to allow enabling hybrid event aggregation by default, avoiding the need to pass --hybrid-merge explicitly on every invocation.
The config value is a default rather than an explicit request, so with --hierarchy it is ignored with a warning, while giving both --hierarchy and --hybrid-merge on the command line remains an error. For the same reason it is silently ignored when there are no events to merge, as would be the case on any machine with a single core PMU, whereas an explicit --hybrid-merge still warns. Record whether the option came from the command line in
symbol_confso both can tell the two apart.
Quoted 5 lines
perf stat has its own merging options for counting, the sampling core.hybrid-merge deliberately doesn't alter it and its --hybrid-merge must still be given explicitly.
Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Ian Rogers
reviewerPATCH v4 8/9 perf test: Expand tests for --hybrid-merge
In reply to Ian RogersAdd a "Hybrid event merging" test that checks evlist__can_merge_hybrid() and evlist__merge_hybrid() directly, covering the events a wildcard expanded to over two core PMUs, the same over three, and events that must not be merged.
Add a report_hybrid_merge.sh shell test that records and then reports with --hybrid-merge, and extend top.sh to cover perf top with it.
--- tools/perf/tests/Build | 1 + tools/perf/tests/builtin-test.c | 1 + tools/perf/tests/hybrid-merge.c | 206 ++++++++++++++++++ tools/perf/tests/shell/report_hybrid_merge.sh | 121 ++++++++++ tools/perf/tests/shell/top.sh | 73 ++++++- tools/perf/tests/tests.h | 1 + 6 files changed, 392 insertions(+), 11 deletions(-) create mode 100644 tools/perf/tests/hybrid-merge.c create mode 100755 tools/perf/tests/shell/report_hybrid_merge.sh
tools/perf/tests/Build +1 -0
@@ -65,6 +65,7 @@ perf-test-y += perf-time-to-tsc.o perf-test-y += dlfilter-test.o perf-test-y += sigtrap.o perf-test-y += event_groups.o +perf-test-y += hybrid-merge.o perf-test-y += symbols.o perf-test-y += util.o perf-test-y += hwmon_pmu.o
tools/perf/tests/builtin-test.c +1 -0
@@ -149,6 +149,7 @@ static struct test_suite *generic_tests[] = { &suite__dlfilter, &suite__sigtrap, &suite__event_groups, + &suite__hybrid_merge, &suite__symbols, &suite__util, &suite__subcmd_help,
tools/perf/tests/hybrid-merge.c new file +206 -0
206 lines not on this page · raw message
tools/perf/tests/shell/report_hybrid_merge.sh new file +121 -0
121 lines not on this page · raw message
tools/perf/tests/shell/top.sh +62 -11
89 lines not on this page · raw message
tools/perf/tests/tests.h +1 -0
@@ -177,6 +177,7 @@ DECLARE_SUITE(perf_time_to_tsc);DECLARE_SUITE (dlfilter);DECLARE_SUITE (sigtrap);DECLARE_SUITE (event_groups); +DECLARE_SUITE (hybrid_merge);DECLARE_SUITE (symbols);DECLARE_SUITE (util);DECLARE_SUITE (uncore_event_sorting);
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 8/9] perf test: Expand tests for --hybrid-merge
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
Add a "Hybrid event merging" test that checks
evlist__can_merge_hybrid()andevlist__merge_hybrid()directly, covering the events a wildcard expanded to over two core PMUs, the same over three, and events that must not be merged.Add a
report_hybrid_merge.shshell test that records and then reports with --hybrid-merge, and extendtop.shto cover perf top with it.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Ian Rogers
reviewerPATCH v4 9/9 perf test: Isolate test suite from user .perfconfig natively
In reply to Ian RogersRunning 'perf test' should not inherit the user's ~/.perfconfig environment as customizing core properties (like core.hybrid-merge) will trivially break stdout matching checks across the shell suite.
Explicitly set PERF_CONFIG to /dev/null inside cmd_test to globally sandbox the environment for the entire test workflow. Test specific config, like annotate.objdump, is read before the sandboxing so that 'perf test' still honors it.
--- tools/perf/tests/builtin-test.c | 49 +++++++++++++++++++++++++++++---- 1 file changed, 44 insertions(+), 5 deletions(-)
tools/perf/tests/builtin-test.c +44 -5
@@ -14,6 +14,8 @@#include <stdlib.h> #include <string.h> +#include "util/config.h" +#include <dirent.h> #include <linux/kernel.h> #include <linux/string.h>
@@ -1655,11 +1657,30 @@ static int run_workload(const char *work, int argc, const char **argv)return -1 ; } +/* +* Owns the string test_objdump_path points at when it came from the config. It +* is reachable for the lifetime of the process so leak checking won't report +* it. +*/ +static char *test_objdump_config_path; +static int perf_test__config (const char *var,const char *value,void *data__maybe_unused ) { -if (!strcmp (var,"annotate.objdump" )) - test_objdump_path = value; +if (!strcmp (var,"annotate.objdump" )) { +/* +* The config, and so value, is freed by perf_config__exit() +* below, take a copy that lives as long as the tests. +*/ +char *dup =strdup (value); + +if (!dup) +return -ENOMEM ; + +free (test_objdump_config_path); + test_objdump_config_path = dup; + test_objdump_path = dup; + }return 0 ; }
@@ -1755,13 +1776,31 @@ int cmd_test(int argc, const char **argv) };const char *const test_subcommands[] = {"list" ,NULL };struct intlist *skiplist =NULL ; -int ret =hists__init ();struct test_suite **suites; +int ret; -if (ret <0 ) + ret =hists__init (); +if (ret <0 )return ret; -perf_config (perf_test__config,NULL ); +/* Read test related config, like annotate.objdump, before isolating. */ + ret =perf_config (perf_test__config,NULL ); +if (ret <0 ) +return ret; + +/* +* Isolate the test suite from the user's ~/.perfconfig to ensure +* behavior isn't impacted by custom variables (e.g. core.hybrid-merge). +* Setting PERF_CONFIG isolates the perf commands run by the shell +* tests, config_exclusive_filename does the same for this process. +* Config values already read by main's perf_config are reset here, so +* far only the hybrid-merge value is known to alter test output. +*/ +if (setenv ("PERF_CONFIG" ,"/dev/null" ,1 ) <0 ) +return -1 ; + config_exclusive_filename ="/dev/null" ; +perf_config__exit (); + symbol_conf.hybrid_merge = false;/* Unbuffered output */ setvbuf (stdout,NULL , _IONBF,0 );
Signature · 1 line
2.55.0.1082.g2b9226bbc0-goog
[email protected]
AI reviewRe: [PATCH v4 9/9] perf test: Isolate test suite from user .perfconfig natively
Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
Running 'perf test' should not inherit the user's ~/.perfconfig environment as customizing core properties (like core.hybrid-merge) will trivially break stdout matching checks across the shell suite.
Explicitly set
PERF_CONFIGto/dev/nullinsidecmd_testto globally sandbox the environment for the entire test workflow. Test specific config, like annotate.objdump, is read before the sandboxing so that 'perf test' still honors it.Signed-off-by Ian Rogers <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Quoted 24 lines · Ian Rogers
On a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error. For the same reason it is silent when there is nothing to merge, so it doesn't warn on every invocation on a machine with a single core PMU.
Quoted 8 lines
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
The events to merge are found with
first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.
Thanks, applied to perf-tools-next, for v7.4.
- Arnaldo
Arnaldo Carvalho de Melo
maintainerQuoted 28 lines · Arnaldo Carvalho de Melo
Quoted 27 lines · Ian Rogers
On a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error. For the same reason it is silent when there is nothing to merge, so it doesn't warn on every invocation on a machine with a single core PMU.
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
Quoted 7 lines
Quoted 5 lines
The events to merge are found with
first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.Thanks, applied to perf-tools-next, for v7.4.
I tried 'perf top --hybrid-merge" on this machine:
acme@x2:~/git/perf-tools-next$ grep -m1 "model name" /proc/cpuinfo model name : Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz acme@x2:~/git/perf-tools-next$
That is hybrid and it didn't merge anything, just told me that:
┌─Warning:──────────────────────────────────────────┐
│--hybrid-merge: no events to merge across core PMUs│
│ │
│ │
│Press any key... │
└───────────────────────────────────────────────────┘What am I missing?
I merged this, so we can continue from there,
- Arnaldo
Ian Rogers
reviewerQuoted 53 lines · Arnaldo Carvalho de Melo
Quoted 28 lines · Arnaldo Carvalho de Melo
Quoted 27 lines · Ian Rogers
On a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error. For the same reason it is silent when there is nothing to merge, so it doesn't warn on every invocation on a machine with a single core PMU.
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
Quoted 7 lines
Quoted 5 lines
The events to merge are found with
first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.Thanks, applied to perf-tools-next, for v7.4.
I tried 'perf top --hybrid-merge" on this machine:
acme@x2:~/git/perf-tools-next$ grep -m1 "model name" /proc/cpuinfo model name : Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz acme@x2:~/git/perf-tools-next$That is hybrid and it didn't merge anything, just told me that:
┌─Warning:──────────────────────────────────────────┐ │--hybrid-merge: no events to merge across core PMUs│ │ │ │ │ │Press any key... │ └───────────────────────────────────────────────────┘
Quoted 3 lines
What am I missing?
I merged this, so we can continue from there,
Are you sure it is hybrid?
https://www.intel.com/content/www/us/en/products/sku/124968/intel-core-i78650u-processor-8m-cache-up-to-4-20-ghz/specifications.html
$ sudo perf top --hybrid-merge
```
Samples: 43K of events 'Merged hybrid events { cpu_atom/cycles/P,
cpu_core/cycles/P }', 1250 Hz, Even
Overhead cpu_ato cpu_cor Shared Object Symbol
3.92% 0.00% 5.84% perf
[.] rb_next
3.50% 0.01% 5.21% perf
[.] __symbols__insert
1.94% 2.20% 1.81% libc.so.6
[.] __memmove_avx_unaligned_e
1.18% 0.56% 1.48% [i915]
[k] fw_domains_get_with_fallb
1.13% 2.43% 0.50% chrome (deleted)
[.] 0x000000000c462020
0.83% 0.00% 1.24% libc.so.6
[.] __strstr_sse2_unaligned
0.78% 0.01% 1.15% perf
[.] str_isascii
0.74% 0.02% 1.10% libc.so.6
[.] __strlen_avx2
0.63% 0.00% 0.94% perf
[.] rb_insert_color
0.61% 1.11% 0.36% libc.so.6
[.] __memset_avx2_unaligned_e
0.61% 0.09% 0.86% libc.so.6
[.] _int_malloc
0.58% 0.73% 0.50% [kernel]
[k] audit_filter_rules.isra.0
0.51% 1.11% 0.22% chrome (deleted)
[.] 0x0000000007139cca
0.49% 0.66% 0.41% [kernel]
[k] kernel_init_pages
0.47% 1.03% 0.20% chrome (deleted)
[.] 0x0000000007139cd4
0.47% 0.52% 0.44% [kernel]
[k] __sched_balance_update_bl
0.46% 0.00% 0.69% perf
[.] dso__load_sym_internal
0.45% 0.79% 0.28% [kernel]
[k] __schedule
0.44% 1.21% 0.06% chrome (deleted)
[.] 0x0000000007139dbe
0.41% 0.00% 0.61% [kernel]
[k] damon_ptep_mkold
0.39% 0.56% 0.31% [kernel]
[k] __audit_filter_op
0.38% 0.00% 0.56% libstdc++.so.6.0.35
[.] 0x00000000000c61cb
0.37% 0.00% 0.54% perf
[.] rust_demangle_legacy_dema
0.35% 0.01% 0.52% libc.so.6
[.] __strncmp_avx2
0.34% 0.00% 0.50% libstdc++.so.6.0.35
[.] 0x00000000000c61ee
0.34% 0.24% 0.38% [kernel]
[k] _raw_spin_lock
0.34% 0.64% 0.19% [kernel]
[k] psi_group_change
0.33% 0.00% 0.50% libstdc++.so.6.0.35
[.] 0x00000000000c61e4
0.30% 0.70% 0.10% chrome (deleted)
[.] 0x0000000007139db3
0.28% 0.38% 0.23% [kernel]
[k] __update_load_avg_cfs_rq
0.28% 0.40% 0.22% [kernel]
[k] native_sched_clock
0.28% 0.00% 0.42% libc.so.6
[.] __libc_calloc2
0.28% 0.00% 0.42% [kernel]
[k] page_vma_mapped_walk
0.26% 0.00% 0.39% perf
[.] symbol__new
0.26% 0.54% 0.12% [kernel]
[k] enqueue_task_fair
0.25% 0.00% 0.37% libstdc++.so.6.0.35
[.] 0x00000000000c86f8
0.25% 0.07% 0.34% [kernel]
[k] __list_del_entry_valid_or
Too slow to read ring buffer (change period (-c/-F) or limit CPUs (-C)
```Thanks,
Ian
Quoted 3 lines
- Arnaldo
Ian Rogers
reviewerQuoted 136 lines · Ian Rogers
Quoted 53 lines · Arnaldo Carvalho de Melo
Quoted 28 lines · Arnaldo Carvalho de Melo
Quoted 27 lines · Ian Rogers
On a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error. For the same reason it is silent when there is nothing to merge, so it doesn't warn on every invocation on a machine with a single core PMU.
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
Quoted 7 lines
Quoted 5 lines
The events to merge are found with
first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.Thanks, applied to perf-tools-next, for v7.4.
I tried 'perf top --hybrid-merge" on this machine:
acme@x2:~/git/perf-tools-next$ grep -m1 "model name" /proc/cpuinfo model name : Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz acme@x2:~/git/perf-tools-next$That is hybrid and it didn't merge anything, just told me that:
┌─Warning:──────────────────────────────────────────┐ │--hybrid-merge: no events to merge across core PMUs│ │ │ │ │ │Press any key... │ └───────────────────────────────────────────────────┘Quoted 3 lines
What am I missing?
I merged this, so we can continue from there,
Are you sure it is hybrid?
https://www.intel.com/content/www/us/en/products/sku/124968/intel-core-i78650u-processor-8m-cache-up-to-4-20-ghz/specifications.html$ sudo perf top --hybrid-merge ``` Samples: 43K of events 'Merged hybrid events { cpu_atom/cycles/P, cpu_core/cycles/P }', 1250 Hz, Even Overhead cpu_ato cpu_cor Shared Object Symbol 3.92% 0.00% 5.84% perf [.] rb_next 3.50% 0.01% 5.21% perf [.] __symbols__insert 1.94% 2.20% 1.81% libc.so.6 [.] __memmove_avx_unaligned_e 1.18% 0.56% 1.48% [i915] [k] fw_domains_get_with_fallb 1.13% 2.43% 0.50% chrome (deleted) [.] 0x000000000c462020 0.83% 0.00% 1.24% libc.so.6 [.] __strstr_sse2_unaligned 0.78% 0.01% 1.15% perf [.] str_isascii 0.74% 0.02% 1.10% libc.so.6 [.] __strlen_avx2 0.63% 0.00% 0.94% perf [.] rb_insert_color 0.61% 1.11% 0.36% libc.so.6 [.] __memset_avx2_unaligned_e 0.61% 0.09% 0.86% libc.so.6 [.] _int_malloc 0.58% 0.73% 0.50% [kernel] [k] audit_filter_rules.isra.0 0.51% 1.11% 0.22% chrome (deleted) [.] 0x0000000007139cca 0.49% 0.66% 0.41% [kernel] [k] kernel_init_pages 0.47% 1.03% 0.20% chrome (deleted) [.] 0x0000000007139cd4 0.47% 0.52% 0.44% [kernel] [k] __sched_balance_update_bl 0.46% 0.00% 0.69% perf [.] dso__load_sym_internal 0.45% 0.79% 0.28% [kernel] [k] __schedule 0.44% 1.21% 0.06% chrome (deleted) [.] 0x0000000007139dbe 0.41% 0.00% 0.61% [kernel] [k] damon_ptep_mkold 0.39% 0.56% 0.31% [kernel] [k] __audit_filter_op 0.38% 0.00% 0.56% libstdc++.so.6.0.35 [.] 0x00000000000c61cb 0.37% 0.00% 0.54% perf [.] rust_demangle_legacy_dema 0.35% 0.01% 0.52% libc.so.6 [.] __strncmp_avx2 0.34% 0.00% 0.50% libstdc++.so.6.0.35 [.] 0x00000000000c61ee 0.34% 0.24% 0.38% [kernel] [k] _raw_spin_lock 0.34% 0.64% 0.19% [kernel] [k] psi_group_change 0.33% 0.00% 0.50% libstdc++.so.6.0.35 [.] 0x00000000000c61e4 0.30% 0.70% 0.10% chrome (deleted) [.] 0x0000000007139db3 0.28% 0.38% 0.23% [kernel] [k] __update_load_avg_cfs_rq 0.28% 0.40% 0.22% [kernel] [k] native_sched_clock 0.28% 0.00% 0.42% libc.so.6 [.] __libc_calloc2 0.28% 0.00% 0.42% [kernel] [k] page_vma_mapped_walk 0.26% 0.00% 0.39% perf [.] symbol__new
Quoted 8 lines
0.26% 0.54% 0.12% [kernel] [k] enqueue_task_fair 0.25% 0.00% 0.37% libstdc++.so.6.0.35 [.] 0x00000000000c86f8 0.25% 0.07% 0.34% [kernel] [k] __list_del_entry_valid_or Too slow to read ring buffer (change period (-c/-F) or limit CPUs (-C) ```
Listing the PMUs would be useful:
``` $ ls /sys/bus/event_source/devices/cpu* /sys/bus/event_source/devices/cpu_atom: caps events freeze_on_smi power subsystem uevent cpus format perf_event_mux_interval_ms rdpmc type /sys/bus/event_source/devices/cpu_core: caps events freeze_on_smi power subsystem uevent cpus format perf_event_mux_interval_ms rdpmc typ ```
Thanks,
Ian
Quoted 55 lines · Ian Rogers
Quoted 53 lines · Arnaldo Carvalho de Melo
Quoted 28 lines · Arnaldo Carvalho de Melo
Quoted 27 lines · Ian Rogers
On a hybrid machine a wildcard event expands to one event per core PMU, so "perf record -e cycles" records cpu_core/cycles/ and cpu_atom/cycles/. perf report and perf top then show one histogram per PMU, and a symbol that ran on both kinds of core is split across them, with each percentage relative to its own PMU's total. There is no way to ask for the single combined profile.
Add --hybrid-merge to perf report and perf top, which merges the histograms of the events that came from the same wildcard into one, with percentages taken against the summed period.
Merging is opt-in because it isn't always the right thing to do. The cores differ in performance, so a merged cycles count mixes work done at different rates and a merged IPC is not the IPC of either core type. perf-tips notes this. When merging is possible but wasn't asked for, the TUI says so, rather than merging silently.
core.hybrid-merge in .perfconfig turns it on by default. As a default rather than an explicit request it yields to the command line: with --hierarchy it is ignored with a warning, whereas asking for both --hierarchy and --hybrid-merge on the command line is an error. For the same reason it is silent when there is nothing to merge, so it doesn't warn on every invocation on a machine with a single core PMU.
perf stat is deliberately untouched. Counting already has its own merging options, and this is aimed at sampling.
Quoted 7 lines
Quoted 5 lines
The events to merge are found with
first_wildcard_match, which parsing fills in. perf report reads events from a file rather than parsing them, so there the wildcard grouping is recovered by matching event names, and core PMUs are identified from the perf.data header topology so that a file recorded on another machine is read correctly.Thanks, applied to perf-tools-next, for v7.4.
I tried 'perf top --hybrid-merge" on this machine:
acme@x2:~/git/perf-tools-next$ grep -m1 "model name" /proc/cpuinfo model name : Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz acme@x2:~/git/perf-tools-next$That is hybrid and it didn't merge anything, just told me that:
┌─Warning:──────────────────────────────────────────┐ │--hybrid-merge: no events to merge across core PMUs│ │ │ │ │ │Press any key... │ └───────────────────────────────────────────────────┘
Quoted 5 lines
Quoted 3 lines
What am I missing?
I merged this, so we can continue from there,
Are you sure it is hybrid?
You're right, this is an older machine, the hybrid one needs repairing the screen and I'm using this one that indeed isn't hybrid, sorry about that.
It's pushed already, I'll be traveling tomorrow but will continue processing patches.
- Arnaldo
Quoted 84 lines
$ sudo perf top --hybrid-merge
```
Samples: 43K of events 'Merged hybrid events { cpu_atom/cycles/P,
cpu_core/cycles/P }', 1250 Hz, Even
Overhead cpu_ato cpu_cor Shared Object Symbol
3.92% 0.00% 5.84% perf
[.] rb_next
3.50% 0.01% 5.21% perf
[.] __symbols__insert
1.94% 2.20% 1.81% libc.so.6
[.] __memmove_avx_unaligned_e
1.18% 0.56% 1.48% [i915]
[k] fw_domains_get_with_fallb
1.13% 2.43% 0.50% chrome (deleted)
[.] 0x000000000c462020
0.83% 0.00% 1.24% libc.so.6
[.] __strstr_sse2_unaligned
0.78% 0.01% 1.15% perf
[.] str_isascii
0.74% 0.02% 1.10% libc.so.6
[.] __strlen_avx2
0.63% 0.00% 0.94% perf
[.] rb_insert_color
0.61% 1.11% 0.36% libc.so.6
[.] __memset_avx2_unaligned_e
0.61% 0.09% 0.86% libc.so.6
[.] _int_malloc
0.58% 0.73% 0.50% [kernel]
[k] audit_filter_rules.isra.0
0.51% 1.11% 0.22% chrome (deleted)
[.] 0x0000000007139cca
0.49% 0.66% 0.41% [kernel]
[k] kernel_init_pages
0.47% 1.03% 0.20% chrome (deleted)
[.] 0x0000000007139cd4
0.47% 0.52% 0.44% [kernel]
[k] __sched_balance_update_bl
0.46% 0.00% 0.69% perf
[.] dso__load_sym_internal
0.45% 0.79% 0.28% [kernel]
[k] __schedule
0.44% 1.21% 0.06% chrome (deleted)
[.] 0x0000000007139dbe
0.41% 0.00% 0.61% [kernel]
[k] damon_ptep_mkold
0.39% 0.56% 0.31% [kernel]
[k] __audit_filter_op
0.38% 0.00% 0.56% libstdc++.so.6.0.35
[.] 0x00000000000c61cb
0.37% 0.00% 0.54% perf
[.] rust_demangle_legacy_dema
0.35% 0.01% 0.52% libc.so.6
[.] __strncmp_avx2
0.34% 0.00% 0.50% libstdc++.so.6.0.35
[.] 0x00000000000c61ee
0.34% 0.24% 0.38% [kernel]
[k] _raw_spin_lock
0.34% 0.64% 0.19% [kernel]
[k] psi_group_change
0.33% 0.00% 0.50% libstdc++.so.6.0.35
[.] 0x00000000000c61e4
0.30% 0.70% 0.10% chrome (deleted)
[.] 0x0000000007139db3
0.28% 0.38% 0.23% [kernel]
[k] __update_load_avg_cfs_rq
0.28% 0.40% 0.22% [kernel]
[k] native_sched_clock
0.28% 0.00% 0.42% libc.so.6
[.] __libc_calloc2
0.28% 0.00% 0.42% [kernel]
[k] page_vma_mapped_walk
0.26% 0.00% 0.39% perf
[.] symbol__new
0.26% 0.54% 0.12% [kernel]
[k] enqueue_task_fair
0.25% 0.00% 0.37% libstdc++.so.6.0.35
[.] 0x00000000000c86f8
0.25% 0.07% 0.34% [kernel]
[k] __list_del_entry_valid_or
Too slow to read ring buffer (change period (-c/-F) or limit CPUs (-C)
```Quoted 6 lines
Thanks,
IanQuoted 3 lines
- Arnaldo
- Arnaldo