perf annotate-data: Fix hangs on broken debug info, AMD mem record
Arnaldo Carvalho de Melo
maintainerv5 cover perf annotate-data: Fix hangs on broken debug info, AMD mem record
Hi,
This originated in another patch series, 'perf tools: Annotate fixes, stdio progress indication, debuginfo-client in more places', and is being split off so that these fixes can be reviewed right away; the debuginfod download feature work from it will come later, separately, based on this series.
- 'perf report -s type' spins forever on the dwz compressed debug info of zlib-ng (libz.so.1): die_collect_vars() saves a type DIE offset that is relative to the file the DIE lives in, the dwz alt file for types shared by several CUs, and resolving it in the main file parses whatever is at that offset, here a typedef whose DW_AT_type refers to itself, making the typedef/qualifier chase spin (patch 3), with the chases bounded so that other kinds of broken debug info don't hang perf either (patches 1, 2 and 4);
- 'perf mem record' requests PERF_SAMPLE_CPU (patch 5) and uses the IBS swfilt filter when the kernel exposes it (patch 6), with the tables carrying the term kept in the arch/x86 code, where the knowledge that IBS needs it stays, as Ravi Bangoria suggested reviewing Namhyung Kim's v6 review remark on this patch (<[email protected]>, replying to <[email protected]>).
About PATCH 5, answering Namhyung Kim's v6 review question (<[email protected]>) about the --sample-cpu default: the data source field is about the memory hierarchy level of the access, it has no record of which CPU issued it, and while the TID is in every sample and in the CTF stream, a thread time-sliced on one CPU or moved between SMT siblings is not told apart by it from cross-core contention, so it is the CPU id that keys it, and it is what the false-sharing detector in pahole needs. The TID is recorded as well.
Requires elfutils 0.160 for dwarf_cu_getdwarf(), so the libdw feature test probes for it and Makefile.config says 0.160: older versions now disable dwarf support with that message instead of failing to link.
Best regards,
- Arnaldo
What changed from v4:
- Added Ravi Bangoria's Reviewed-by (<[email protected]>), given together with his rationale for keeping the IBS swfilt selection
explicit: 'perf record' fails rather than transparently retrying with /swfilt=1/, so that the user is aware software filtering is being used and that it is not overhead free.
- Trimmed the comment on the AMD mem event tables in tools/perf/arch/x86/util/mem-events.c per that review, dropping its last sentence about how 'perf record' handles exclude bits on open failure; the code is unchanged.
- Patch 4: reworded the comment on the truncated flag; the JSON exporter that reads it is added in a later series.
- Patch 6: dropped a leftover no-op hunk from tools/perf/util/mem-events.c; the swfilt selection is all in the arch/x86 code.
Command to see this delta: git diff eb568956a7129c91..HEAD
What changed from v3:
- Dropped patch 5/7, "perf annotate-data: Show the sample count in the data-type browser": it is already in perf-tools-next as commit 9db4e9d6cc7c, so the series is now 6 patches and the subject no longer mentions it.
- Rebased onto current perf-tools-next (base-commit below; v3 was based on 29f320d221c1), patch 4 adapted to the upstream __add_member_cb() cleanup that dropped the member_type local, no behavior change.
What changed from v2:
Only aggregate (struct/union) members are marked as truncated when the MAX_MEMBER_DEPTH limit is reached in the member nesting recursion (patch 4), addressing a sashiko [Medium] review finding (<[email protected]>): the v2 check was made before looking at the member's type, so a primitive field that merely landed on the limit, e.g. an int in a struct nested 31 deep, was marked as truncated and logged even though it has no children to expand. The limit still bounds the recursion, as only aggregates are expanded.
What changed from v1:
Bump MAX_MEMBER_DEPTH from 8 to 32, addressing a review comment from Namhyung.
tools/build/feature/test-libdw.c | 10 ++- tools/perf/Documentation/perf-mem.txt | 4 + tools/perf/Makefile.config | 2 +- tools/perf/arch/x86/util/mem-events.c | 17 ++++ tools/perf/arch/x86/util/mem-events.h | 2 + tools/perf/arch/x86/util/pmu.c | 10 ++- tools/perf/builtin-mem.c | 7 +- tools/perf/tests/shell/test_data_symbol.sh | 6 +- tools/perf/util/annotate-data.c | 41 +++++++-- tools/perf/util/annotate-data.h | 3 + tools/perf/util/dwarf-aux.c | 140 ++++++++++++++++++++++++----- tools/perf/util/dwarf-aux.h | 13 +++ 12 files changed, 217 insertions(+), 38 deletions(-)
v4-head: eb568956a7129c91a8c0ccebb5979440a046c7ef
v3-head: e2504280f5c4086e9851a758ed1bae8df7ef269e
base-commit: edd8a9fe2eca009599e013a29c421c7a6b5ad1b9 --
Arnaldo Carvalho de Melo
maintainerPATCH v5 1/6 perf dwarf-aux: Bound the type chases for broken debug info
From: Arnaldo Carvalho de Melo <[email protected]>
A DIE parsed at an offset that is not the start of a DIE can have a DW_AT_type that refers back to itself, making the typedef/qualifier chases in die_get_real_type(), die_get_pointer_type() and die_get_typename_from_type() spin forever; 'perf report -s type' did that on the dwz compressed debug info of zlib-ng (libz.so.1).
No sane typedef/qualifier chain is 32 DIEs long, so give up on the type with a pr_debug instead of hanging.
--- tools/perf/util/dwarf-aux.c | 84 ++++++++++++++++++++++++++++--------- 1 file changed, 64 insertions(+), 20 deletions(-)
tools/perf/util/dwarf-aux.c +64 -20
@@ -267,16 +267,27 @@ Dwarf_Die *die_get_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem)return NULL ; } +/* +* A DIE parsed at an offset that is not the start of a DIE can refer +* back to itself via DW_AT_type: bound these chases. +*/ +#define MAX_TYPE_CHASE 32 +/* Get a type die, but skip qualifiers */ Dwarf_Die *__die_get_real_type (Dwarf_Die *vr_die,Dwarf_Die *die_mem) { -int tag; +int tag, chase =0 ; do { vr_die = die_get_type(vr_die, die_mem); if (!vr_die) - break; + return NULL; tag = dwarf_tag(vr_die); + if (++chase > MAX_TYPE_CHASE) { + pr_debug("DWARF: qualifier chase limit reached at DIE 0x%lx\n", + (unsigned long)dwarf_dieoffset(vr_die)); + return NULL; + } } while (tag == DW_TAG_const_type || tag == DW_TAG_restrict_type || tag == DW_TAG_volatile_type ||
@@ -297,8 +308,15 @@ Dwarf_Die *__die_get_real_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem)*/ Dwarf_Die *die_get_real_type (Dwarf_Die *vr_die,Dwarf_Die *die_mem) { +int chase =0 ; +do { vr_die =__die_get_real_type (vr_die, die_mem); +if (++chase >MAX_TYPE_CHASE ) { +pr_debug ("DWARF: typedef chase limit reached at DIE 0x%lx\n" , + vr_die ? (unsigned long )dwarf_dieoffset (vr_die) :0 ); +return NULL ; + } }while (vr_die &&dwarf_tag (vr_die) == DW_TAG_typedef);return vr_die;
@@ -315,7 +333,7 @@ Dwarf_Die *die_get_real_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem)*/ Dwarf_Die *die_get_pointer_type (Dwarf_Die *type_die,Dwarf_Die *die_mem) { -int tag; +int tag, chase =0 ; do { tag = dwarf_tag(type_die);
@@ -325,6 +343,11 @@ Dwarf_Die *die_get_pointer_type(Dwarf_Die *type_die, Dwarf_Die *die_mem) tag != DW_TAG_restrict_type && tag != DW_TAG_volatile_type && tag != DW_TAG_shared_type)return NULL ; +if (++chase >MAX_TYPE_CHASE ) { +pr_debug ("DWARF: pointer type chase limit reached at DIE 0x%lx\n" , + (unsigned long )dwarf_dieoffset (type_die)); +return NULL ; + } type_die =die_get_type (type_die, die_mem); }while (type_die);
@@ -1119,17 +1142,25 @@ Dwarf_Die *die_find_member(Dwarf_Die *st_die, const char *name, die_mem); } -/** -* die_get_typename_from_type - Get the name of given type DIE -* @type_die: a type DIE -* @buf: a strbuf for result type name -* -* Get the name of @type_die and stores it to @buf. Return 0 if succeeded. -* and Return -ENOENT if failed to find type name. -* Note that the result will stores typedef name if possible, and stores -* "*(function_type)" if the type is a function pointer. +/* +* The name follows DW_AT_type, so a self-referring DIE makes this +* recurse forever: bound it like the chases above. */ -int die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf) +static int __die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf, +int depth); + +static int __die_get_typename (Dwarf_Die *vr_die,struct strbuf *buf,int depth) +{ +Dwarf_Die type; + +if (__die_get_real_type (vr_die, &type) ==NULL ) +return -ENOENT ; + +return __die_get_typename_from_type (&type, buf, depth); +} + +static int __die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf, +int depth) {int tag, ret;const char *tmp ="" ;
@@ -1156,7 +1187,12 @@ int die_get_typename_from_type(Dwarf_Die *type_die, struct strbuf *buf)/* Write a base name */ return strbuf_addf (buf,"%s%s" , tmp, name ?:"" ); } - ret =die_get_typename (type_die, buf); +if (depth >=MAX_TYPE_CHASE ) { +pr_debug ("DWARF: type name recursion limit reached at DIE 0x%lx\n" , + (unsigned long )dwarf_dieoffset (type_die)); +return -ENOENT ; + } + ret =__die_get_typename (type_die, buf, depth +1 );if (ret <0 ) {/* void pointer has no type attribute */ if (tag == DW_TAG_pointer_type && ret == -ENOENT )
@@ -1167,6 +1203,19 @@ int die_get_typename_from_type(Dwarf_Die *type_die, struct strbuf *buf)return strbuf_addstr (buf, tmp); } +/** +* die_get_typename_from_type - Get the name of a type DIE +* @type_die: a type DIE +* @buf: a strbuf for the result type name +* +* Stores the typedef name if possible, "*(function_type)" for function +* pointers. Returns 0 or -ENOENT. +*/ +int die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf) +{ +return __die_get_typename_from_type (type_die, buf,0 ); +} +/** * die_get_typename - Get the name of given variable DIE * @vr_die: a variable DIE
@@ -1179,12 +1228,7 @@ int die_get_typename_from_type(Dwarf_Die *type_die, struct strbuf *buf)*/ int die_get_typename (Dwarf_Die *vr_die,struct strbuf *buf) { -Dwarf_Die type; - -if (__die_get_real_type (vr_die, &type) ==NULL ) -return -ENOENT ; - -return die_get_typename_from_type (&type, buf); +return __die_get_typename (vr_die, buf,0 ); }/**
Signature · 1 line
2.53.0
Arnaldo Carvalho de Melo
maintainerPATCH v5 2/6 perf dwarf-aux: Add die_same_file() and die_get_type_die()
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
dwarf_dieoffset() is relative to the file the DIE is in, so an offset saved by die_collect_vars()/die_collect_global_vars() is only meaningful in that file: add die_same_file(), comparing the Dwarf each DIE's CU belongs to, and die_get_type_die(), resolving the offset in it with the saved tag as a sanity check. Resolving the offset in the main file instead parses whatever is at it when it came from the dwz common file.
dwarf_cu_getdwarf() is new in elfutils 0.160, so the libdw feature test probes for it and Makefile.config says 0.160.
--- tools/build/feature/test-libdw.c | 10 +++++- tools/perf/Makefile.config | 2 +- tools/perf/util/dwarf-aux.c | 56 ++++++++++++++++++++++++++++++-- tools/perf/util/dwarf-aux.h | 13 ++++++++ 4 files changed, 77 insertions(+), 4 deletions(-)
tools/build/feature/test-libdw.c +9 -1
@@ -49,8 +49,16 @@ int test_elfutils(void)return 0 ; } +/* Needs elfutils 0.160: taking its address fails the probe, not the link. */ +int test_libdw_cu_getdwarf (void ) +{ +void *sym = (void *)dwarf_cu_getdwarf; + +return sym ==NULL ; +} +int main (void ) {return test_libdw () +test_libdw_unwind () +test_libdw_getlocations () + -test_libdw_getcfi () +test_elfutils (); +test_libdw_getcfi () +test_libdw_cu_getdwarf () +test_elfutils (); }
tools/perf/Makefile.config +1 -1
@@ -477,7 +477,7 @@ else else ifneq ($(feature-libdw), 1) ifndef NO_LIBDW - $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.157, disables dwarf support. Please install new elfutils-devel/libdw-dev) + $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.160, disables dwarf support. Please install new elfutils-devel/libdw-dev) NO_LIBDW := 1 endif endif # Dwarf support
tools/perf/util/dwarf-aux.c +54 -2
92 lines not on this page · raw message
tools/perf/util/dwarf-aux.h +13 -0
@@ -152,6 +152,8 @@ int die_get_scopes(Dwarf_Die *cu_die, Dwarf_Addr pc, Dwarf_Die **scopes);struct die_var_type {struct die_var_type *next;u64 die_off; +int die_tag; +bool from_alt;/* die_off is relative to the alt (dwz) file */ u64 addr;u64 end;/* end address of location range */ int reg;
@@ -183,6 +185,17 @@ Dwarf_Die *die_find_variable_by_addr(Dwarf_Die *sc_die, Dwarf_Addr addr,/* Save all variables and parameters in this scope */ void die_collect_vars (Dwarf_Die *sc_die,struct die_var_type **var_types); +/* +* Get the type DIE saved by die_collect_vars()/die_collect_global_vars(): +* @from_alt says which file the offset must be resolved in, @die_tag is +* a sanity check. +*/ +Dwarf_Die *die_get_type_die (Dwarf *dbg,u64 die_off,int die_tag,bool from_alt, +Dwarf_Die *die_mem); + +/* Whether two DIEs live in the same debug file */ +bool die_same_file (Dwarf_Die *die_a,Dwarf_Die *die_b); +/* Save all global variables in this CU */ void die_collect_global_vars (Dwarf_Die *cu_die,struct die_var_type **var_types);
Signature · 1 line
2.53.0
Arnaldo Carvalho de Melo
maintainerPATCH v5 3/6 perf annotate-data: Resolve type DIEs in the debug file they came from
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
'perf report -s type' hangs burning CPU on the dwz compressed debug info of libz.so.1 (zlib-ng): die_collect_vars() saves the dwarf_dieoffset() of the type DIE, which is relative to the file it lives in - the dwz alt file for the types shared by more than one CU - and update_var_state() resolves that offset in the main debug file, parsing whatever is at it, here a typedef whose DW_AT_type refers to itself.
Record whether the type DIE came from the alt file and resolve the offset with die_get_type_die() in that file, with no fallback.
--- tools/perf/util/annotate-data.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)
tools/perf/util/annotate-data.c +15 -6
@@ -739,6 +739,8 @@ struct global_var_entry {u64 start;u64 end;u64 die_offset; +int die_tag; +bool from_alt;/* die_offset is relative to the alt (dwz) file */ };static int global_var_cmp (const void *_key,const struct rb_node *node)
@@ -776,7 +778,7 @@ static struct global_var_entry *global_var__find(struct data_loc_info *dloc, u64 }static bool global_var__add (struct data_loc_info *dloc,u64 addr, -const char *name,Dwarf_Die *type_die) +const char *name,Dwarf_Die *type_die,bool from_alt) {struct dso *dso =map__dso (dloc->ms->map);struct global_var_entry *gvar;
@@ -798,6 +800,8 @@ static bool global_var__add(struct data_loc_info *dloc, u64 addr, gvar->start = addr; gvar->end = addr + size; gvar->die_offset =dwarf_dieoffset (type_die); + gvar->die_tag =dwarf_tag (type_die); + gvar->from_alt = from_alt;rb_add (&gvar->node,dso__global_vars (dso), global_var_less);return true;
@@ -872,12 +876,14 @@ static void global_var__collect(struct data_loc_info *dloc)if (pos->reg !=-1 )continue ; -if (!dwarf_offdie (dwarf, pos->die_off, &type_die)) +if (!die_get_type_die (dwarf, pos->die_off, pos->die_tag, + pos->from_alt, &type_die))continue ;get_global_var_info (dloc, pos->addr, &var_name, &var_offset); -global_var__add (dloc, pos->addr, var_name, &type_die); +global_var__add (dloc, pos->addr, var_name, &type_die, + pos->from_alt); }delete_var_types (var_types);
@@ -902,7 +908,8 @@ bool get_global_var_type(Dwarf_Die *cu_die, struct data_loc_info *dloc, gvar =global_var__find (dloc, var_addr);if (gvar) { -if (!dwarf_offdie (dloc->di->dbg, gvar->die_offset, type_die)) +if (!die_get_type_die (dloc->di->dbg, gvar->die_offset, + gvar->die_tag, gvar->from_alt, type_die))return false; *var_offset = var_addr - gvar->start;
@@ -932,7 +939,8 @@ bool get_global_var_type(Dwarf_Die *cu_die, struct data_loc_info *dloc, ok:/* The address should point to the start of the variable */ -global_var__add (dloc, var_addr - *var_offset, var_name, type_die); +global_var__add (dloc, var_addr - *var_offset, var_name, type_die, + !die_same_file (cu_die, type_die));return true; }
@@ -987,7 +995,8 @@ static void update_var_state(struct type_state *state, struct data_loc_info *dlocontinue ; }/* Get the type DIE using the offset */ -if (!dwarf_offdie (dloc->di->dbg, var->die_off, &mem_die)) +if (!die_get_type_die (dloc->di->dbg, var->die_off, + var->die_tag, var->from_alt, &mem_die))continue ;if (var->reg ==DWARF_REG_FB || var->reg == fbreg || var->reg == state->stack_reg) {
Signature · 1 line
2.53.0
Arnaldo Carvalho de Melo
maintainerPATCH v5 4/6 perf annotate-data: Bound the member nesting recursion
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
Members are added recursively, and a broken DIE can make a member's type point back at one of its own ancestors, recursing until the stack is gone; nothing usable comes out of nesting members 32 deep anyway, so stop there, marking the member as truncated and giving up on member types that don't resolve.
The check is where the children of an aggregate would be expanded, so only struct/union members are marked as truncated.
--- tools/perf/util/annotate-data.c | 20 ++++++++++++++++++-- tools/perf/util/annotate-data.h | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-)
tools/perf/util/annotate-data.c +18 -2
@@ -222,6 +222,12 @@ static bool data_type_less(struct rb_node *node_a, const struct rb_node *node_b)return strcmp (a->self.type_name, b->self.type_name) <0 ; } +/* +* A broken type can point back at one of its own ancestors: bound the +* nesting so it doesn't recurse until the stack is gone. +*/ +#define MAX_MEMBER_DEPTH 32 +/* Recursively add new members for struct/union */ static int __add_member_cb (Dwarf_Die *die,void *arg) {
@@ -236,6 +242,9 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (dwarf_tag (die) != DW_TAG_member)return DIE_FIND_CB_SIBLING ; +if (die_get_real_type (die, &die_mem) ==NULL ) +return DIE_FIND_CB_SIBLING ; + member =zalloc (sizeof (*member));if (member ==NULL )return DIE_FIND_CB_END ;
@@ -248,8 +257,6 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (die_get_typename (die, &sb) <0 )strbuf_add (&sb,"(unknown type)" ,14 ); -die_get_real_type (die, &die_mem); -if (dwarf_aggregate_size (&die_mem, &size) <0 || size ==0 ) {if (dwarf_tag (&die_mem) == DW_TAG_array_type) {/* flex-array? */ die_get_real_type (&die_mem, &die_mem);
@@ -299,6 +306,7 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) } member->size = size; member->offset = loc + parent->offset; + member->depth = parent->depth +1 ;INIT_LIST_HEAD (&member->children);list_for_each_entry_reverse (prev, &parent->children, node) {
@@ -313,6 +321,14 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) member->is_union = true;/* fall through */ case DW_TAG_structure_type: +/* Only aggregates have children to expand, so only they get truncated. */ +if (member->depth >=MAX_MEMBER_DEPTH ) { +/* Consumed by the JSON exporter added in a later series. */ + member->truncated = true; +pr_debug_dtp ("member nesting limit reached at %s\n" , + member->type_name ?:"(unknown type)" ); +break ; + }die_find_child (&die_mem, __add_member_cb, member, &die_mem);break ;default :
tools/perf/util/annotate-data.h +3 -0
@@ -61,6 +61,9 @@ struct annotated_member {int size;bool is_union;bool is_flex_array; +unsigned int depth; +/* Children not expanded because the nesting limit was reached */ +bool truncated; };/**
Signature · 1 line
2.53.0
Arnaldo Carvalho de Melo
maintainerPATCH v5 5/6 perf mem record: Request PERF_SAMPLE_CPU by default
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
The data type profiling per-sample stream keys cross-CPU contention on sample->cpu, which without PERF_SAMPLE_CPU is the "no CPU info" sentinel, making same-instance accesses from different cores indistinguishable from same-CPU traffic. 'perf mem record' already passes -d and -W to the record parser, add --sample-cpu and document it in perf-mem(1).
Also grow rec_argv: the new entry overflows it on PMUs with separate load and store events, as the space for the fixed arguments was not reserved.
--- tools/perf/Documentation/perf-mem.txt | 4 ++++ tools/perf/builtin-mem.c | 7 +++++-- 2 files changed, 9 insertions(+), 2 deletions(-)
tools/perf/Documentation/perf-mem.txt +4 -0
@@ -14,6 +14,10 @@ DESCRIPTION ----------- "perf mem record" runs a command and gathers memory operation data from it, into perf.data. Perf record options are accepted and are passed through. +It also requests the address (-d), the weight (-W, where supported) and the +CPU id (--sample-cpu) of every sampled access by default; the CPU id is what +lets per-sample analysis tell reads and writes to the same data from +different cores apart from same-CPU traffic. "perf mem report" displays the result. It invokes perf report with the right set of options to display a memory access profile. By default, loads
tools/perf/builtin-mem.c +5 -2
@@ -99,8 +99,8 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem, argc =parse_options (argc, argv, options, record_usage,PARSE_OPT_KEEP_UNKNOWN ); -/* Max number of arguments multiplied by number of PMUs that can support them. */ - rec_argc = argc +9 * (perf_pmu__mem_events_num_mem_pmus (pmu) +1 ); +/* Max number of arguments per PMU plus the fixed ones added below. */ + rec_argc = argc +8 +9 * (perf_pmu__mem_events_num_mem_pmus (pmu) +1 );if (mem->cpu_list) rec_argc +=2 ;
@@ -135,6 +135,9 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem, rec_argv[i++] ="-d" ; +/* Cross-CPU contention keys on sample->cpu, -1 without PERF_SAMPLE_CPU. */ + rec_argv[i++] ="--sample-cpu" ; +if (mem->phys_addr) rec_argv[i++] ="--phys-data" ;
Signature · 1 line
2.53.0
Arnaldo Carvalho de Melo
maintainerPATCH v5 6/6 perf mem record: Use the IBS swfilt filter when available
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
IBS events with exclude_{user,kernel} bits, as used for per-thread recording when kernel samples are not allowed, are rejected on hardware without the privilege filter, so per-thread 'perf mem record' fails on AMD:
$ perf mem record -o /dev/null -- true Failure to open event 'ibs_op/ldlat=0/u' on PMU 'ibs_op' which will be removed.
Kernel v6.14 added swfilt, a software privilege filter exposed as the 'swfilt' format term, making those events usable per-thread. Give the ibs_op memory events extra tables with the term, selected in perf_pmu__arch_init() when the PMU exposes it, keeping the names that need system wide mode otherwise; the knowledge that IBS needs this stays in the arch code.
--- tools/perf/arch/x86/util/mem-events.c | 17 +++++++++++++++++ tools/perf/arch/x86/util/mem-events.h | 2 ++ tools/perf/arch/x86/util/pmu.c | 10 ++++++++-- tools/perf/tests/shell/test_data_symbol.sh | 6 ++++-- 4 files changed, 31 insertions(+), 4 deletions(-)
tools/perf/arch/x86/util/mem-events.c +17 -0
@@ -21,14 +21,31 @@ struct perf_mem_event perf_mem_events_intel_aux[PERF_MEM_EVENTS__MAX] = { E(NULL, NULL, NULL, false, 0), }; +/* +* IBS events with exclude_{user,kernel} bits are rejected on hardware +* without the privilege filter unless swfilt is used: the extra tables +* carry the term, selected when the PMU exposes it. +*/ struct perf_mem_event perf_mem_events_amd[PERF_MEM_EVENTS__MAX ] = {E (NULL ,NULL ,NULL , false,0 ),E (NULL ,NULL ,NULL , false,0 ),E ("mem-ldst" ,"%s//" ,NULL , false,0 ), }; +struct perf_mem_event perf_mem_events_amd_swfilt[PERF_MEM_EVENTS__MAX ] = { +E (NULL ,NULL ,NULL , false,0 ), +E (NULL ,NULL ,NULL , false,0 ), +E ("mem-ldst" ,"%s/swfilt=1/" ,NULL , false,0 ), +}; +struct perf_mem_event perf_mem_events_amd_ldlat[PERF_MEM_EVENTS__MAX ] = {E (NULL ,NULL ,NULL , false,0 ),E (NULL ,NULL ,NULL , false,0 ),E ("mem-ldst" ,"%s/ldlat=%u/" ,NULL , true,0 ), }; + +struct perf_mem_event perf_mem_events_amd_ldlat_swfilt[PERF_MEM_EVENTS__MAX ] = { +E (NULL ,NULL ,NULL , false,0 ), +E (NULL ,NULL ,NULL , false,0 ), +E ("mem-ldst" ,"%s/ldlat=%u,swfilt=1/" ,NULL , true,0 ), +};
tools/perf/arch/x86/util/mem-events.h +2 -0
@@ -6,6 +6,8 @@ extern struct perf_mem_event perf_mem_events_intel[PERF_MEM_EVENTS__MAX];extern struct perf_mem_event perf_mem_events_intel_aux[PERF_MEM_EVENTS__MAX ];extern struct perf_mem_event perf_mem_events_amd[PERF_MEM_EVENTS__MAX ]; +extern struct perf_mem_event perf_mem_events_amd_swfilt[PERF_MEM_EVENTS__MAX ];extern struct perf_mem_event perf_mem_events_amd_ldlat[PERF_MEM_EVENTS__MAX ]; +extern struct perf_mem_event perf_mem_events_amd_ldlat_swfilt[PERF_MEM_EVENTS__MAX ];#endif /* _X86_MEM_EVENTS_H */
tools/perf/arch/x86/util/pmu.c +8 -2
@@ -333,6 +333,7 @@ static void uncore_cha_imc_adjust_cpumask_for_snc(struct perf_pmu *pmu, bool chavoid perf_pmu__arch_init (struct perf_pmu *pmu) {struct perf_pmu_caps *ldlat_cap; +bool swfilt_format;if (!strcmp (pmu->name,INTEL_PT_PMU_NAME )) { pmu->auxtrace = true;
@@ -348,7 +349,10 @@ void perf_pmu__arch_init(struct perf_pmu *pmu)if (strcmp (pmu->name,"ibs_op" ))return ; - pmu->mem_events = perf_mem_events_amd; + swfilt_format =perf_pmu__has_format (pmu,"swfilt" ); + pmu->mem_events = swfilt_format ? + perf_mem_events_amd_swfilt : + perf_mem_events_amd;if (!perf_pmu__caps_parse (pmu))return ;
@@ -358,7 +362,9 @@ void perf_pmu__arch_init(struct perf_pmu *pmu)return ; perf_mem_events__loads_ldlat =0 ; - pmu->mem_events = perf_mem_events_amd_ldlat; + pmu->mem_events = swfilt_format ? + perf_mem_events_amd_ldlat_swfilt : + perf_mem_events_amd_ldlat; }else {if (pmu->is_core) {if (perf_pmu__have_event (pmu,"mem-loads-aux" ))
tools/perf/tests/shell/test_data_symbol.sh +4 -2
@@ -65,15 +65,17 @@ if (($is_amd >= 1)); then # --ldlat on AMD: # o Zen4 and earlier uarch does not support ldlat # o Even on supported platforms, it's disabled (--ldlat=0) by default. + # o Kernels with the swfilt term add it even when ldlat is not + # supported, so only check ldlat when the term is present. ldlat=${BASH_REMATCH[1]} - if [[ -n $ldlat ]]; then + if [[ $ldlat == *ldlat=* ]]; then if ! [[ "$ldlat" =~ ldlat=0 ]]; then echo "ERROR: ldlat not initialized to 0?" exit 1 fi mem_events="$(perf mem record -v --ldlat=150 -e list 2>&1)" - if ! [[ "$mem_events" =~ ^mem-ldst.*ibs_op/ldlat=150/.*available ]]; then + if ! [[ "$mem_events" =~ ^mem-ldst.*ibs_op/ldlat=150[,/].*available ]]; then echo "ERROR: --ldlat not honored?" exit 1 fi
Signature · 1 line
2.53.0
[email protected]
AI reviewRe: [PATCH 3/6] perf annotate-data: Resolve type DIEs in the debug file they came from
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 10 lines
From: Arnaldo Carvalho de Melo <[email protected]>
'perf report -s type' hangs burning CPU on the dwz compressed debug info of libz.so.1 (zlib-ng):
die_collect_vars()saves thedwarf_dieoffset()of the type DIE, which is relative to the file it lives in - the dwz alt file for the types shared by more than one CU - andupdate_var_state()resolves that offset in the main debug file, parsing whatever is at it, here a typedef whoseDW_AT_typerefers to itself.
Quoted 7 lines
Record whether the type DIE came from the alt file and resolve the offset with
die_get_type_die()in that file, with no fallback.Fixes 06b2ce75386df04b ("perf annotate-data: Maintain variable type info")Fixes 55ee3d005d62279d ("perf annotate-data: Add a cache for global variable types")Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [PATCH 1/6] perf dwarf-aux: Bound the type chases for broken debug info
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
From: Arnaldo Carvalho de Melo <[email protected]>
A DIE parsed at an offset that is not the start of a DIE can have a
DW_AT_typethat refers back to itself, making the typedef/qualifier chases indie_get_real_type(),die_get_pointer_type()anddie_get_typename_from_type()spin forever; 'perf report -s type' did that on the dwz compressed debug info of zlib-ng (libz.so.1).
Quoted 6 lines
No sane typedef/qualifier chain is 32 DIEs long, so give up on the type with a
pr_debuginstead of hanging.Cc Masami Hiramatsu <[email protected]>Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[email protected]
AI reviewRe: [PATCH 4/6] perf annotate-data: Bound the member nesting recursion
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 8 lines
From: Arnaldo Carvalho de Melo <[email protected]>
Members are added recursively, and a broken DIE can make a member's type point back at one of its own ancestors, recursing until the stack is gone; nothing usable comes out of nesting members 32 deep anyway, so stop there, marking the member as truncated and giving up on member types that don't resolve.
Quoted 5 lines
The check is where the children of an aggregate would be expanded, so only struct/union members are marked as truncated.
Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
[email protected]
AI reviewRe: [PATCH 6/6] perf mem record: Use the IBS swfilt filter when available
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 17 lines
From: Arnaldo Carvalho de Melo <[email protected]>
IBS events with exclude_{user,kernel} bits, as used for per-thread recording when kernel samples are not allowed, are rejected on hardware without the privilege filter, so per-thread 'perf mem record' fails on AMD:
$ perf mem record -o /dev/null -- true Failure to open event 'ibs_op/ldlat=0/u' on PMU 'ibs_op' which will be removed.Kernel v6.14 added swfilt, a software privilege filter exposed as the 'swfilt' format term, making those events usable per-thread. Give the
ibs_opmemory events extra tables with the term, selected inperf_pmu__arch_init()when the PMU exposes it, keeping the names that need system wide mode otherwise; the knowledge that IBS needs this stays in the arch code.
Quoted 5 lines
Suggested-by Namhyung Kim <[email protected]>Suggested-by Ravi Bangoria <[email protected]>Reviewed-by Ravi Bangoria <[email protected]>Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
[email protected]
AI reviewRe: [PATCH 2/6] perf dwarf-aux: Add die_same_file() and die_get_type_die()
In reply to Arnaldo Carvalho de MeloThank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] perf dwarf-aux: truncation of u64 die_off in pr_debug on 32-bit platforms --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] perf dwarf-aux: truncation of u64 die_off in pr_debug on 32-bit platforms --
commit c7e2660460f55978ed41b85fca70ec2e99b2a867
Author: Arnaldo Carvalho de Melo <[email protected]>
perf dwarf-aux: Add die_same_file() and die_get_type_die()
This commit introduces die_same_file() to compare the debug files of two DIEs and die_get_type_die() to resolve a type DIE offset in the correct debug file. This ensures proper offset resolution for DWARF files and DWZ common files.
Quoted 4 lines
tools/perf/util/dwarf-aux.c +0
-0
[ ... ]
tools/perf/util/dwarf-aux.c static int __die_collect_global_vars_cb(Dwarf_Die *die_mem, void *arg) · line 1836
Quoted 12 lines
tools/perf/util/dwarf-aux.c +12
-0+Dwarf_Die *die_get_type_die (Dwarf *dbg,u64 die_off,int die_tag,bool from_alt, +Dwarf_Die *die_mem) +{ +Dwarf *target = dbg; +Dwarf_Die die; + +if (from_alt) { +/* No fallback: an alt offset here parses whatever is at it. */ + target =dwarf_getalt (dbg); +if (target ==NULL ) { +pr_debug ("DWARF: no alt (dwz) debug file to resolve the type DIE at offset 0x%lx in\n" , + (unsigned long )die_off);
[Severity: Low]
Could this truncate the DWARF file offset in tools/perf/util/dwarf-aux.c within die_get_type_die() on 32-bit platforms if the offset exceeds 4GB? The 64-bit integer die_off is cast to unsigned long when logging a failed alt offset resolution, which is only 32 bits on 32-bit architectures, causing debug messages to display an incorrect offset. Should this use PRIx64 instead?
tools/perf/util/dwarf-aux.c static int __die_collect_global_vars_cb(Dwarf_Die *die_mem, void *arg) · line 1847
Quoted 11 lines
tools/perf/util/dwarf-aux.c +11
-0+return NULL ; + } + } + +if (dwarf_offdie (target, die_off, &die) &&dwarf_tag (&die) == die_tag) { + *die_mem = die; +return die_mem; + } + +pr_debug ("DWARF: no DIE with tag %d at offset 0x%lx in the %s debug file\n" , + die_tag, (unsigned long )die_off, from_alt ?"alt" :"main" );
[Severity: Low]
Similarly, when logging a missing DIE at the end of die_get_type_die(), does casting the u64 die_off to unsigned long truncate the upper 32 bits on 32-bit architectures? Consider formatting with PRIx64 to ensure the full 64-bit value is printed correctly.
Quoted 2 lines
tools/perf/util/dwarf-aux.c +2
-0+return NULL ; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
[email protected]
AI reviewRe: [PATCH 5/6] perf mem record: Request PERF_SAMPLE_CPU by default
In reply to Arnaldo Carvalho de MeloSashiko has reviewed this patch and found no issues. It looks great!
Quoted 9 lines
From: Arnaldo Carvalho de Melo <[email protected]>
The data type profiling per-sample stream keys cross-CPU contention on
sample->cpu, which withoutPERF_SAMPLE_CPUis the "no CPU info" sentinel, making same-instance accesses from different cores indistinguishable from same-CPU traffic. 'perf mem record' already passes -d and -W to the record parser, add --sample-cpu and document it in perf-mem(1).
Quoted 6 lines
Also grow
rec_argv: the new entry overflows it on PMUs with separate load and store events, as the space for the fixed arguments was not reserved.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Sashiko has reviewed this patch and found no issues. It looks great!
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Re: [PATCH 1/6] perf dwarf-aux: Bound the type chases for broken debug info
In reply to Arnaldo Carvalho de MeloQuoted 9 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
A DIE parsed at an offset that is not the start of a DIE can have a
DW_AT_typethat refers back to itself, making the typedef/qualifier chases indie_get_real_type(),die_get_pointer_type()anddie_get_typename_from_type()spin forever; 'perf report -s type' did that on the dwz compressed debug info of zlib-ng (libz.so.1).
Quoted 6 lines
No sane typedef/qualifier chain is 32 DIEs long, so give up on the type with a
pr_debuginstead of hanging.Cc Masami Hiramatsu <[email protected]>Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Thanks,
Ian
Quoted 162 lines
--- tools/perf/util/dwarf-aux.c | 84 ++++++++++++++++++++++++++++--------- 1 file changed, 64 insertions(+), 20 deletions(-)tools/perf/util/dwarf-aux.c +64
-21@@ -267,16 +267,27 @@ Dwarf_Die *die_get_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem)return NULL ; } +/* +* A DIE parsed at an offset that is not the start of a DIE can refer +* back to itself via DW_AT_type: bound these chases. +*/ +#define MAX_TYPE_CHASE 32 +/* Get a type die, but skip qualifiers */ Dwarf_Die *__die_get_real_type (Dwarf_Die *vr_die,Dwarf_Die *die_mem) { -int tag; +int tag, chase =0 ; do { vr_die = die_get_type(vr_die, die_mem); if (!vr_die) - break; + return NULL; tag = dwarf_tag(vr_die); + if (++chase > MAX_TYPE_CHASE) { + pr_debug("DWARF: qualifier chase limit reached at DIE 0x%lx\n", + (unsigned long)dwarf_dieoffset(vr_die)); + return NULL; + } } while (tag == DW_TAG_const_type || tag == DW_TAG_restrict_type || tag == DW_TAG_volatile_type ||@@ -297,8 +308,15 @@ Dwarf_Die *__die_get_real_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem)*/ Dwarf_Die *die_get_real_type (Dwarf_Die *vr_die,Dwarf_Die *die_mem) { +int chase =0 ; +do { vr_die =__die_get_real_type (vr_die, die_mem); +if (++chase >MAX_TYPE_CHASE ) { +pr_debug ("DWARF: typedef chase limit reached at DIE 0x%lx\n" , + vr_die ? (unsigned long )dwarf_dieoffset (vr_die) :0 ); +return NULL ; + } }while (vr_die &&dwarf_tag (vr_die) == DW_TAG_typedef);return vr_die;@@ -315,7 +333,7 @@ Dwarf_Die *die_get_real_type(Dwarf_Die *vr_die, Dwarf_Die *die_mem)*/ Dwarf_Die *die_get_pointer_type (Dwarf_Die *type_die,Dwarf_Die *die_mem) { -int tag; +int tag, chase =0 ; do { tag = dwarf_tag(type_die);@@ -325,6 +343,11 @@ Dwarf_Die *die_get_pointer_type(Dwarf_Die *type_die, Dwarf_Die *die_mem) tag != DW_TAG_restrict_type && tag != DW_TAG_volatile_type && tag != DW_TAG_shared_type)return NULL ; +if (++chase >MAX_TYPE_CHASE ) { +pr_debug ("DWARF: pointer type chase limit reached at DIE 0x%lx\n" , + (unsigned long )dwarf_dieoffset (type_die)); +return NULL ; + } type_die =die_get_type (type_die, die_mem); }while (type_die);@@ -1119,17 +1142,25 @@ Dwarf_Die *die_find_member(Dwarf_Die *st_die, const char *name, die_mem); } -/** -* die_get_typename_from_type - Get the name of given type DIE -* @type_die: a type DIE -* @buf: a strbuf for result type name -* -* Get the name of @type_die and stores it to @buf. Return 0 if succeeded. -* and Return -ENOENT if failed to find type name. -* Note that the result will stores typedef name if possible, and stores -* "*(function_type)" if the type is a function pointer. +/* +* The name follows DW_AT_type, so a self-referring DIE makes this +* recurse forever: bound it like the chases above. */ -int die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf) +static int __die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf, +int depth); + +static int __die_get_typename (Dwarf_Die *vr_die,struct strbuf *buf,int depth) +{ +Dwarf_Die type; + +if (__die_get_real_type (vr_die, &type) ==NULL ) +return -ENOENT ; + +return __die_get_typename_from_type (&type, buf, depth); +} + +static int __die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf, +int depth) {int tag, ret;const char *tmp ="" ;@@ -1156,7 +1187,12 @@ int die_get_typename_from_type(Dwarf_Die *type_die, struct strbuf *buf)/* Write a base name */ return strbuf_addf (buf,"%s%s" , tmp, name ?:"" ); } - ret =die_get_typename (type_die, buf); +if (depth >=MAX_TYPE_CHASE ) { +pr_debug ("DWARF: type name recursion limit reached at DIE 0x%lx\n" , + (unsigned long )dwarf_dieoffset (type_die)); +return -ENOENT ; + } + ret =__die_get_typename (type_die, buf, depth +1 );if (ret <0 ) {/* void pointer has no type attribute */ if (tag == DW_TAG_pointer_type && ret == -ENOENT )@@ -1167,6 +1203,19 @@ int die_get_typename_from_type(Dwarf_Die *type_die, struct strbuf *buf)return strbuf_addstr (buf, tmp); } +/** +* die_get_typename_from_type - Get the name of a type DIE +* @type_die: a type DIE +* @buf: a strbuf for the result type name +* +* Stores the typedef name if possible, "*(function_type)" for function +* pointers. Returns 0 or -ENOENT. +*/ +int die_get_typename_from_type (Dwarf_Die *type_die,struct strbuf *buf) +{ +return __die_get_typename_from_type (type_die, buf,0 ); +} +/** * die_get_typename - Get the name of given variable DIE * @vr_die: a variable DIE @@ -1179,12 +1228,7 @@ int die_get_typename_from_type(Dwarf_Die *type_die, struct strbuf *buf)*/ int die_get_typename (Dwarf_Die *vr_die,struct strbuf *buf) { -Dwarf_Die type; - -if (__die_get_real_type (vr_die, &type) ==NULL ) -return -ENOENT ; - -return die_get_typename_from_type (&type, buf); +return __die_get_typename (vr_die, buf,0 ); }/** -- 2.53.0
Re: [PATCH 2/6] perf dwarf-aux: Add die_same_file() and die_get_type_die()
In reply to Arnaldo Carvalho de MeloQuoted 11 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
dwarf_dieoffset()is relative to the file the DIE is in, so an offset saved by die_collect_vars()/die_collect_global_vars() is only meaningful in that file: adddie_same_file(), comparing the Dwarf each DIE's CU belongs to, anddie_get_type_die(), resolving the offset in it with the saved tag as a sanity check. Resolving the offset in the main file instead parses whatever is at it when it came from the dwz common file.
Quoted 6 lines
dwarf_cu_getdwarf()is new in elfutils 0.160, so the libdw feature test probes for it and Makefile.config says 0.160.Cc Masami Hiramatsu <[email protected]>Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Thanks,
Ian
Quoted 178 lines
--- tools/build/feature/test-libdw.c | 10 +++++- tools/perf/Makefile.config | 2 +- tools/perf/util/dwarf-aux.c | 56 ++++++++++++++++++++++++++++++-- tools/perf/util/dwarf-aux.h | 13 ++++++++ 4 files changed, 77 insertions(+), 4 deletions(-)tools/build/feature/test-libdw.c +9
-1@@ -49,8 +49,16 @@ int test_elfutils(void)return 0 ; } +/* Needs elfutils 0.160: taking its address fails the probe, not the link. */ +int test_libdw_cu_getdwarf (void ) +{ +void *sym = (void *)dwarf_cu_getdwarf; + +return sym ==NULL ; +} +int main (void ) {return test_libdw () +test_libdw_unwind () +test_libdw_getlocations () + -test_libdw_getcfi () +test_elfutils (); +test_libdw_getcfi () +test_libdw_cu_getdwarf () +test_elfutils (); }tools/perf/Makefile.config +1
-1@@ -477,7 +477,7 @@ else else ifneq ($(feature-libdw), 1) ifndef NO_LIBDW - $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.157, disables dwarf support. Please install new elfutils-devel/libdw-dev) + $(warning No libdw.h found or old libdw.h found or elfutils is older than 0.160, disables dwarf support. Please install new elfutils-devel/libdw-dev) NO_LIBDW := 1 endif endif # Dwarf supporttools/perf/util/dwarf-aux.c +54
-2@@ -1677,6 +1677,15 @@ Dwarf_Die *die_find_variable_by_addr(Dwarf_Die *sc_die, Dwarf_Addr addr,return result; } +/* +* Whether two DIEs live in the same debug file: dwarf_dieoffset() is +* relative to the file the DIE is in. +*/ +bool die_same_file (Dwarf_Die *die_a,Dwarf_Die *die_b) +{ +return dwarf_cu_getdwarf (die_a->cu) ==dwarf_cu_getdwarf (die_b->cu); +} +static int __die_collect_vars_cb (Dwarf_Die *die_mem,void *arg) {struct die_var_type **var_types = arg;@@ -1721,6 +1730,8 @@ static int __die_collect_vars_cb(Dwarf_Die *die_mem, void *arg) vt->is_reg_var_addr = true; vt->die_off =dwarf_dieoffset (&type_die); + vt->die_tag =dwarf_tag (&type_die); + vt->from_alt = !die_same_file (die_mem, &type_die); vt->addr = start; vt->end = end; vt->has_range = (end !=0 || start !=0 );@@ -1740,7 +1751,8 @@ static int __die_collect_vars_cb(Dwarf_Die *die_mem, void *arg)* * Save all variables and parameters in the @sc_die and save them to @var_types. * The @var_types is a singly-linked list containing type and location info. -* Actual type can be retrieved using dwarf_offdie() with 'die_off' later. +* Actual type can be retrieved using die_get_type_die() with 'die_off', +* 'die_tag' and 'from_alt' later. * * Callers should free @var_types. */ @@ -1786,6 +1798,8 @@ static int __die_collect_global_vars_cb(Dwarf_Die *die_mem, void *arg)return DIE_FIND_CB_END ; vt->die_off =dwarf_dieoffset (&type_die); + vt->die_tag =dwarf_tag (&type_die); + vt->from_alt = !die_same_file (die_mem, &type_die); vt->addr = ops->number; vt->end =0 ; vt->has_range = false;@@ -1797,6 +1811,43 @@ static int __die_collect_global_vars_cb(Dwarf_Die *die_mem, void *arg)return DIE_FIND_CB_SIBLING ; } +/** +* die_get_type_die - Get a type DIE saved by die_collect_vars() +* @dbg: the main debug info +* @die_off: offset of the type DIE, from dwarf_dieoffset() +* @die_tag: tag that DIE had when the offset was saved +* @from_alt: whether the type DIE is in the dwz alt file +* @die_mem: where to store the resulting DIE +* +* Resolve @die_off in the file it was recorded as belonging to, with no +* fallback; @die_tag is a sanity check. +*/ +Dwarf_Die *die_get_type_die (Dwarf *dbg,u64 die_off,int die_tag,bool from_alt, +Dwarf_Die *die_mem) +{ +Dwarf *target = dbg; +Dwarf_Die die; + +if (from_alt) { +/* No fallback: an alt offset here parses whatever is at it. */ + target =dwarf_getalt (dbg); +if (target ==NULL ) { +pr_debug ("DWARF: no alt (dwz) debug file to resolve the type DIE at offset 0x%lx in\n" , + (unsigned long )die_off); +return NULL ; + } + } + +if (dwarf_offdie (target, die_off, &die) &&dwarf_tag (&die) == die_tag) { + *die_mem = die; +return die_mem; + } + +pr_debug ("DWARF: no DIE with tag %d at offset 0x%lx in the %s debug file\n" , + die_tag, (unsigned long )die_off, from_alt ?"alt" :"main" ); +return NULL ; +} +/** * die_collect_global_vars - Save all global variables * @cu_die: a CU DIE @@ -1804,7 +1855,8 @@ static int __die_collect_global_vars_cb(Dwarf_Die *die_mem, void *arg)* * Save all global variables in the @cu_die and save them to @var_types. * The @var_types is a singly-linked list containing type and location info. -* Actual type can be retrieved using dwarf_offdie() with 'die_off' later. +* Actual type can be retrieved using die_get_type_die() with 'die_off', +* 'die_tag' and 'from_alt' later. * * Callers should free @var_types. */ tools/perf/util/dwarf-aux.h +13
-1@@ -152,6 +152,8 @@ int die_get_scopes(Dwarf_Die *cu_die, Dwarf_Addr pc, Dwarf_Die **scopes);struct die_var_type {struct die_var_type *next;u64 die_off; +int die_tag; +bool from_alt;/* die_off is relative to the alt (dwz) file */ u64 addr;u64 end;/* end address of location range */ int reg;@@ -183,6 +185,17 @@ Dwarf_Die *die_find_variable_by_addr(Dwarf_Die *sc_die, Dwarf_Addr addr,/* Save all variables and parameters in this scope */ void die_collect_vars (Dwarf_Die *sc_die,struct die_var_type **var_types); +/* +* Get the type DIE saved by die_collect_vars()/die_collect_global_vars(): +* @from_alt says which file the offset must be resolved in, @die_tag is +* a sanity check. +*/ +Dwarf_Die *die_get_type_die (Dwarf *dbg,u64 die_off,int die_tag,bool from_alt, +Dwarf_Die *die_mem); + +/* Whether two DIEs live in the same debug file */ +bool die_same_file (Dwarf_Die *die_a,Dwarf_Die *die_b); +/* Save all global variables in this CU */ void die_collect_global_vars (Dwarf_Die *cu_die,struct die_var_type **var_types); --2.53.0
Re: [PATCH 3/6] perf annotate-data: Resolve type DIEs in the debug file they came from
In reply to Arnaldo Carvalho de MeloQuoted 11 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
'perf report -s type' hangs burning CPU on the dwz compressed debug info of libz.so.1 (zlib-ng):
die_collect_vars()saves thedwarf_dieoffset()of the type DIE, which is relative to the file it lives in - the dwz alt file for the types shared by more than one CU - andupdate_var_state()resolves that offset in the main debug file, parsing whatever is at it, here a typedef whoseDW_AT_typerefers to itself.
Quoted 7 lines
Record whether the type DIE came from the alt file and resolve the offset with
die_get_type_die()in that file, with no fallback.Fixes 06b2ce75386df04b ("perf annotate-data: Maintain variable type info")Fixes 55ee3d005d62279d ("perf annotate-data: Add a cache for global variable types")Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Thanks,
Ian
Quoted 85 lines
--- tools/perf/util/annotate-data.c | 21 +++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-)tools/perf/util/annotate-data.c +15
-7@@ -739,6 +739,8 @@ struct global_var_entry {u64 start;u64 end;u64 die_offset; +int die_tag; +bool from_alt;/* die_offset is relative to the alt (dwz) file */ };static int global_var_cmp (const void *_key,const struct rb_node *node)@@ -776,7 +778,7 @@ static struct global_var_entry *global_var__find(struct data_loc_info *dloc, u64 }static bool global_var__add (struct data_loc_info *dloc,u64 addr, -const char *name,Dwarf_Die *type_die) +const char *name,Dwarf_Die *type_die,bool from_alt) {struct dso *dso =map__dso (dloc->ms->map);struct global_var_entry *gvar;@@ -798,6 +800,8 @@ static bool global_var__add(struct data_loc_info *dloc, u64 addr, gvar->start = addr; gvar->end = addr + size; gvar->die_offset =dwarf_dieoffset (type_die); + gvar->die_tag =dwarf_tag (type_die); + gvar->from_alt = from_alt;rb_add (&gvar->node,dso__global_vars (dso), global_var_less);return true;@@ -872,12 +876,14 @@ static void global_var__collect(struct data_loc_info *dloc)if (pos->reg !=-1 )continue ; -if (!dwarf_offdie (dwarf, pos->die_off, &type_die)) +if (!die_get_type_die (dwarf, pos->die_off, pos->die_tag, + pos->from_alt, &type_die))continue ;get_global_var_info (dloc, pos->addr, &var_name, &var_offset); -global_var__add (dloc, pos->addr, var_name, &type_die); +global_var__add (dloc, pos->addr, var_name, &type_die, + pos->from_alt); }delete_var_types (var_types);@@ -902,7 +908,8 @@ bool get_global_var_type(Dwarf_Die *cu_die, struct data_loc_info *dloc, gvar =global_var__find (dloc, var_addr);if (gvar) { -if (!dwarf_offdie (dloc->di->dbg, gvar->die_offset, type_die)) +if (!die_get_type_die (dloc->di->dbg, gvar->die_offset, + gvar->die_tag, gvar->from_alt, type_die))return false; *var_offset = var_addr - gvar->start;@@ -932,7 +939,8 @@ bool get_global_var_type(Dwarf_Die *cu_die, struct data_loc_info *dloc, ok:/* The address should point to the start of the variable */ -global_var__add (dloc, var_addr - *var_offset, var_name, type_die); +global_var__add (dloc, var_addr - *var_offset, var_name, type_die, + !die_same_file (cu_die, type_die));return true; }@@ -987,7 +995,8 @@ static void update_var_state(struct type_state *state, struct data_loc_info *dlocontinue ; }/* Get the type DIE using the offset */ -if (!dwarf_offdie (dloc->di->dbg, var->die_off, &mem_die)) +if (!die_get_type_die (dloc->di->dbg, var->die_off, + var->die_tag, var->from_alt, &mem_die))continue ;if (var->reg ==DWARF_REG_FB || var->reg == fbreg || var->reg == state->stack_reg) { --2.53.0
Re: [PATCH 4/6] perf annotate-data: Bound the member nesting recursion
In reply to Arnaldo Carvalho de MeloQuoted 9 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
Members are added recursively, and a broken DIE can make a member's type point back at one of its own ancestors, recursing until the stack is gone; nothing usable comes out of nesting members 32 deep anyway, so stop there, marking the member as truncated and giving up on member types that don't resolve.
Quoted 5 lines
The check is where the children of an aggregate would be expanded, so only struct/union members are marked as truncated.
Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Thanks,
Ian
Quoted 81 lines
--- tools/perf/util/annotate-data.c | 20 ++++++++++++++++++-- tools/perf/util/annotate-data.h | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-)tools/perf/util/annotate-data.c +18
-2@@ -222,6 +222,12 @@ static bool data_type_less(struct rb_node *node_a, const struct rb_node *node_b)return strcmp (a->self.type_name, b->self.type_name) <0 ; } +/* +* A broken type can point back at one of its own ancestors: bound the +* nesting so it doesn't recurse until the stack is gone. +*/ +#define MAX_MEMBER_DEPTH 32 +/* Recursively add new members for struct/union */ static int __add_member_cb (Dwarf_Die *die,void *arg) {@@ -236,6 +242,9 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (dwarf_tag (die) != DW_TAG_member)return DIE_FIND_CB_SIBLING ; +if (die_get_real_type (die, &die_mem) ==NULL ) +return DIE_FIND_CB_SIBLING ; + member =zalloc (sizeof (*member));if (member ==NULL )return DIE_FIND_CB_END ;@@ -248,8 +257,6 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (die_get_typename (die, &sb) <0 )strbuf_add (&sb,"(unknown type)" ,14 ); -die_get_real_type (die, &die_mem); -if (dwarf_aggregate_size (&die_mem, &size) <0 || size ==0 ) {if (dwarf_tag (&die_mem) == DW_TAG_array_type) {/* flex-array? */ die_get_real_type (&die_mem, &die_mem);@@ -299,6 +306,7 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) } member->size = size; member->offset = loc + parent->offset; + member->depth = parent->depth +1 ;INIT_LIST_HEAD (&member->children);list_for_each_entry_reverse (prev, &parent->children, node) {@@ -313,6 +321,14 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) member->is_union = true;/* fall through */ case DW_TAG_structure_type: +/* Only aggregates have children to expand, so only they get truncated. */ +if (member->depth >=MAX_MEMBER_DEPTH ) { +/* Consumed by the JSON exporter added in a later series. */ + member->truncated = true; +pr_debug_dtp ("member nesting limit reached at %s\n" , + member->type_name ?:"(unknown type)" ); +break ; + }die_find_child (&die_mem, __add_member_cb, member, &die_mem);break ;default :tools/perf/util/annotate-data.h +3
-1@@ -61,6 +61,9 @@ struct annotated_member {int size;bool is_union;bool is_flex_array; +unsigned int depth; +/* Children not expanded because the nesting limit was reached */ +bool truncated; };/** -- 2.53.0
Namhyung Kim
maintainerRe: [PATCH 4/6] perf annotate-data: Bound the member nesting recursion
In reply to Arnaldo Carvalho de MeloQuoted 82 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
Members are added recursively, and a broken DIE can make a member's type point back at one of its own ancestors, recursing until the stack is gone; nothing usable comes out of nesting members 32 deep anyway, so stop there, marking the member as truncated and giving up on member types that don't resolve.
The check is where the children of an aggregate would be expanded, so only struct/union members are marked as truncated.
Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/util/annotate-data.c | 20 ++++++++++++++++++-- tools/perf/util/annotate-data.h | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-)tools/perf/util/annotate-data.c +18
-2@@ -222,6 +222,12 @@ static bool data_type_less(struct rb_node *node_a, const struct rb_node *node_b)return strcmp (a->self.type_name, b->self.type_name) <0 ; } +/* +* A broken type can point back at one of its own ancestors: bound the +* nesting so it doesn't recurse until the stack is gone. +*/ +#define MAX_MEMBER_DEPTH 32 +/* Recursively add new members for struct/union */ static int __add_member_cb (Dwarf_Die *die,void *arg) {@@ -236,6 +242,9 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (dwarf_tag (die) != DW_TAG_member)return DIE_FIND_CB_SIBLING ; +if (die_get_real_type (die, &die_mem) ==NULL ) +return DIE_FIND_CB_SIBLING ; + member =zalloc (sizeof (*member));if (member ==NULL )return DIE_FIND_CB_END ;@@ -248,8 +257,6 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (die_get_typename (die, &sb) <0 )strbuf_add (&sb,"(unknown type)" ,14 ); -die_get_real_type (die, &die_mem); -if (dwarf_aggregate_size (&die_mem, &size) <0 || size ==0 ) {if (dwarf_tag (&die_mem) == DW_TAG_array_type) {/* flex-array? */ die_get_real_type (&die_mem, &die_mem);@@ -299,6 +306,7 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) } member->size = size; member->offset = loc + parent->offset; + member->depth = parent->depth +1 ;INIT_LIST_HEAD (&member->children);list_for_each_entry_reverse (prev, &parent->children, node) {@@ -313,6 +321,14 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) member->is_union = true;/* fall through */ case DW_TAG_structure_type: +/* Only aggregates have children to expand, so only they get truncated. */ +if (member->depth >=MAX_MEMBER_DEPTH ) { +/* Consumed by the JSON exporter added in a later series. */ + member->truncated = true; +pr_debug_dtp ("member nesting limit reached at %s\n" , + member->type_name ?:"(unknown type)" ); +break ; + }die_find_child (&die_mem, __add_member_cb, member, &die_mem);break ;default :tools/perf/util/annotate-data.h +0
-0
tools/perf/util/annotate-data.h struct annotated_member { · line 66
Quoted 6 lines
tools/perf/util/annotate-data.h +3
-0@@ -61,6 +61,9 @@ struct annotated_member {int size;bool is_union;bool is_flex_array; +unsigned int depth; +/* Children not expanded because the nesting limit was reached */ +bool truncated;
Nit: it'd be better to switch the order of the last two fields to reduce the padding.
Thanks,
Namhyung
Quoted 6 lines
tools/perf/util/annotate-data.h +0
-0};/** Signature · 1 line
2.53.0
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 4/6] perf annotate-data: Bound the member nesting recursion
Quoted 84 lines · Namhyung Kim
Quoted 83 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
Members are added recursively, and a broken DIE can make a member's type point back at one of its own ancestors, recursing until the stack is gone; nothing usable comes out of nesting members 32 deep anyway, so stop there, marking the member as truncated and giving up on member types that don't resolve.
The check is where the children of an aggregate would be expanded, so only struct/union members are marked as truncated.
Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>--- tools/perf/util/annotate-data.c | 20 ++++++++++++++++++-- tools/perf/util/annotate-data.h | 3 +++ 2 files changed, 21 insertions(+), 2 deletions(-)tools/perf/util/annotate-data.c +18
-2@@ -222,6 +222,12 @@ static bool data_type_less(struct rb_node *node_a, const struct rb_node *node_b)return strcmp (a->self.type_name, b->self.type_name) <0 ; } +/* +* A broken type can point back at one of its own ancestors: bound the +* nesting so it doesn't recurse until the stack is gone. +*/ +#define MAX_MEMBER_DEPTH 32 +/* Recursively add new members for struct/union */ static int __add_member_cb (Dwarf_Die *die,void *arg) {@@ -236,6 +242,9 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (dwarf_tag (die) != DW_TAG_member)return DIE_FIND_CB_SIBLING ; +if (die_get_real_type (die, &die_mem) ==NULL ) +return DIE_FIND_CB_SIBLING ; + member =zalloc (sizeof (*member));if (member ==NULL )return DIE_FIND_CB_END ;@@ -248,8 +257,6 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)if (die_get_typename (die, &sb) <0 )strbuf_add (&sb,"(unknown type)" ,14 ); -die_get_real_type (die, &die_mem); -if (dwarf_aggregate_size (&die_mem, &size) <0 || size ==0 ) {if (dwarf_tag (&die_mem) == DW_TAG_array_type) {/* flex-array? */ die_get_real_type (&die_mem, &die_mem);@@ -299,6 +306,7 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) } member->size = size; member->offset = loc + parent->offset; + member->depth = parent->depth +1 ;INIT_LIST_HEAD (&member->children);list_for_each_entry_reverse (prev, &parent->children, node) {@@ -313,6 +321,14 @@ static int __add_member_cb(Dwarf_Die *die, void *arg) member->is_union = true;/* fall through */ case DW_TAG_structure_type: +/* Only aggregates have children to expand, so only they get truncated. */ +if (member->depth >=MAX_MEMBER_DEPTH ) { +/* Consumed by the JSON exporter added in a later series. */ + member->truncated = true; +pr_debug_dtp ("member nesting limit reached at %s\n" , + member->type_name ?:"(unknown type)" ); +break ; + }die_find_child (&die_mem, __add_member_cb, member, &die_mem);break ;default :tools/perf/util/annotate-data.h +0
-0@@ -61,6 +61,9 @@ struct annotated_member {int size;
Quoted 8 lines
tools/perf/util/annotate-data.h struct annotated_member { · line 66
Quoted 5 lines
tools/perf/util/annotate-data.h +3
-0bool is_union;bool is_flex_array; +unsigned int depth; +/* Children not expanded because the nesting limit was reached */ +bool truncated;Nit: it'd be better to switch the order of the last two fields to reduce the padding.
I should have used pahole... 8-)
Will fix, thanks!
- Arnaldo
Quoted 34 lines · Arnaldo Carvalho de Melo
Hi,
This originated in another patch series, 'perf tools: Annotate fixes, stdio progress indication, debuginfo-client in more places', and is being split off so that these fixes can be reviewed right away; the debuginfod download feature work from it will come later, separately, based on this series.
- 'perf report -s type' spins forever on the dwz compressed debug info of zlib-ng (libz.so.1):
die_collect_vars()saves a type DIE offset that is relative to the file the DIE lives in, the dwz alt file for types shared by several CUs, and resolving it in the main file parses whatever is at that offset, here a typedef whoseDW_AT_typerefers to itself, making the typedef/qualifier chase spin (patch 3), with the chases bounded so that other kinds of broken debug info don't hang perf either (patches 1, 2 and 4);- 'perf mem record' requests
PERF_SAMPLE_CPU(patch 5) and uses the IBS swfilt filter when the kernel exposes it (patch 6), with the tables carrying the term kept in thearch/x86code, where the knowledge that IBS needs it stays, as Ravi Bangoria suggested reviewing Namhyung Kim's v6 review remark on this patch (<[email protected]>, replying to <[email protected]>).About PATCH 5, answering Namhyung Kim's v6 review question (<[email protected]>) about the --sample-cpu default: the data source field is about the memory hierarchy level of the access, it has no record of which CPU issued it, and while the TID is in every sample and in the CTF stream, a thread time-sliced on one CPU or moved between SMT siblings is not told apart by it from cross-core contention, so it is the CPU id that keys it, and it is what the false-sharing detector in pahole needs. The TID is recorded as well.
Quoted 3 lines
Requires elfutils 0.160 for
dwarf_cu_getdwarf(), so the libdw feature test probes for it and Makefile.config says 0.160: older versions now disable dwarf support with that message instead of failing to link.
I have a nitpick on the patch 4, but otherwise looks good to me.
Thanks,
Namhyung
Quoted 65 lines
Best regards,
- Arnaldo
What changed from v4:
- Added Ravi Bangoria's Reviewed-by (<[email protected]>), given together with his rationale for keeping the IBS swfilt selection
explicit: 'perf record' fails rather than transparently retrying with /swfilt=1/, so that the user is aware software filtering is being used and that it is not overhead free.
- Trimmed the comment on the AMD mem event tables intools/perf/arch/x86/util/mem-events.cper that review, dropping its last sentence about how 'perf record' handles exclude bits on open failure; the code is unchanged.
- Patch 4: reworded the comment on the truncated flag; the JSON exporter that reads it is added in a later series.
- Patch 6: dropped a leftover no-op hunk fromtools/perf/util/mem-events.c; the swfilt selection is all in thearch/x86code.
Command to see this delta: git diff eb568956a7129c91..HEADWhat changed from v3:
- Dropped patch 5/7, "perf annotate-data: Show the sample count in the data-type browser": it is already in perf-tools-next as commit
9db4e9d6cc7c, so the series is now 6 patches and the subject no longer mentions it.
- Rebased onto current perf-tools-next (base-commit below; v3 was based on29f320d221c1), patch 4 adapted to the upstream__add_member_cb()cleanup that dropped themember_typelocal, no behavior change.What changed from v2:
Only aggregate (struct/union) members are marked as truncated when the
MAX_MEMBER_DEPTHlimit is reached in the member nesting recursion (patch 4), addressing a sashiko [Medium] review finding (<[email protected]>): the v2 check was made before looking at the member's type, so a primitive field that merely landed on the limit, e.g. an int in a struct nested 31 deep, was marked as truncated and logged even though it has no children to expand. The limit still bounds the recursion, as only aggregates are expanded.What changed from v1:
Bump
MAX_MEMBER_DEPTHfrom 8 to 32, addressing a review comment from Namhyung.tools/build/feature/test-libdw.c | 10 ++- tools/perf/Documentation/perf-mem.txt | 4 + tools/perf/Makefile.config | 2 +- tools/perf/arch/x86/util/mem-events.c | 17 ++++ tools/perf/arch/x86/util/mem-events.h | 2 + tools/perf/arch/x86/util/pmu.c | 10 ++- tools/perf/builtin-mem.c | 7 +- tools/perf/tests/shell/test_data_symbol.sh | 6 +- tools/perf/util/annotate-data.c | 41 +++++++-- tools/perf/util/annotate-data.h | 3 + tools/perf/util/dwarf-aux.c | 140 ++++++++++++++++++++++++----- tools/perf/util/dwarf-aux.h | 13 +++ 12 files changed, 217 insertions(+), 38 deletions(-)v4-head:
eb568956a7129c91a8c0ccebb5979440a046c7ef
v3-head:e2504280f5c4086e9851a758ed1bae8df7ef269e
base-commit:edd8a9fe2eca009599e013a29c421c7a6b5ad1b9--
Re: [PATCH 5/6] perf mem record: Request PERF_SAMPLE_CPU by default
In reply to Arnaldo Carvalho de MeloQuoted 10 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
The data type profiling per-sample stream keys cross-CPU contention on
sample->cpu, which withoutPERF_SAMPLE_CPUis the "no CPU info" sentinel, making same-instance accesses from different cores indistinguishable from same-CPU traffic. 'perf mem record' already passes -d and -W to the record parser, add --sample-cpu and document it in perf-mem(1).
Quoted 6 lines
Also grow
rec_argv: the new entry overflows it on PMUs with separate load and store events, as the space for the fixed arguments was not reserved.Assisted-by LLMSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Thanks,
Ian
Quoted 48 lines
--- tools/perf/Documentation/perf-mem.txt | 4 ++++ tools/perf/builtin-mem.c | 7 +++++-- 2 files changed, 9 insertions(+), 2 deletions(-)tools/perf/Documentation/perf-mem.txt +4
-0@@ -14,6 +14,10 @@ DESCRIPTION ----------- "perf mem record" runs a command and gathers memory operation data from it, into perf.data. Perf record options are accepted and are passed through. +It also requests the address (-d), the weight (-W, where supported) and the +CPU id (--sample-cpu) of every sampled access by default; the CPU id is what +lets per-sample analysis tell reads and writes to the same data from +different cores apart from same-CPU traffic. "perf mem report" displays the result. It invokes perf report with the right set of options to display a memory access profile. By default, loadstools/perf/builtin-mem.c +5
-3@@ -99,8 +99,8 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem, argc =parse_options (argc, argv, options, record_usage,PARSE_OPT_KEEP_UNKNOWN ); -/* Max number of arguments multiplied by number of PMUs that can support them. */ - rec_argc = argc +9 * (perf_pmu__mem_events_num_mem_pmus (pmu) +1 ); +/* Max number of arguments per PMU plus the fixed ones added below. */ + rec_argc = argc +8 +9 * (perf_pmu__mem_events_num_mem_pmus (pmu) +1 );if (mem->cpu_list) rec_argc +=2 ;@@ -135,6 +135,9 @@ static int __cmd_record(int argc, const char **argv, struct perf_mem *mem, rec_argv[i++] ="-d" ; +/* Cross-CPU contention keys on sample->cpu, -1 without PERF_SAMPLE_CPU. */ + rec_argv[i++] ="--sample-cpu" ; +if (mem->phys_addr) rec_argv[i++] ="--phys-data" ; --2.53.0
Arnaldo Carvalho de Melo
maintainerQuoted 35 lines · Namhyung Kim
Quoted 34 lines · Arnaldo Carvalho de Melo
Hi,
This originated in another patch series, 'perf tools: Annotate fixes, stdio progress indication, debuginfo-client in more places', and is being split off so that these fixes can be reviewed right away; the debuginfod download feature work from it will come later, separately, based on this series.
- 'perf report -s type' spins forever on the dwz compressed debug info of zlib-ng (libz.so.1):
die_collect_vars()saves a type DIE offset that is relative to the file the DIE lives in, the dwz alt file for types shared by several CUs, and resolving it in the main file parses whatever is at that offset, here a typedef whoseDW_AT_typerefers to itself, making the typedef/qualifier chase spin (patch 3), with the chases bounded so that other kinds of broken debug info don't hang perf either (patches 1, 2 and 4);- 'perf mem record' requests
PERF_SAMPLE_CPU(patch 5) and uses the IBS swfilt filter when the kernel exposes it (patch 6), with the tables carrying the term kept in thearch/x86code, where the knowledge that IBS needs it stays, as Ravi Bangoria suggested reviewing Namhyung Kim's v6 review remark on this patch (<[email protected]>, replying to <[email protected]>).About PATCH 5, answering Namhyung Kim's v6 review question (<[email protected]>) about the --sample-cpu default: the data source field is about the memory hierarchy level of the access, it has no record of which CPU issued it, and while the TID is in every sample and in the CTF stream, a thread time-sliced on one CPU or moved between SMT siblings is not told apart by it from cross-core contention, so it is the CPU id that keys it, and it is what the false-sharing detector in pahole needs. The TID is recorded as well.
Quoted 7 lines
Quoted 3 lines
Requires elfutils 0.160 for
dwarf_cu_getdwarf(), so the libdw feature test probes for it and Makefile.config says 0.160: older versions now disable dwarf support with that message instead of failing to link.I have a nitpick on the patch 4, but otherwise looks good to me.
Reviewed-by Namhyung Kim <[email protected]>
I'm fixing that one, thanks for reviewing!
- Arnaldo
Quoted 7 lines · Arnaldo Carvalho de Melo
Quoted 6 lines · Namhyung Kim
Quoted 5 lines · Arnaldo Carvalho de Melo
This originated in another patch series, 'perf tools: Annotate fixes, stdio progress indication, debuginfo-client in more places', and is being split off so that these fixes can be reviewed right away; the debuginfod download feature work from it will come later, separately, based on this series.
<SNIP>
Quoted 5 lines
Quoted 3 lines
I have a nitpick on the patch 4, but otherwise looks good to me.
Reviewed-by Namhyung Kim <[email protected]>I'm fixing that one, thanks for reviewing!
Fixed.
Thanks, applied to perf-tools-next, for v7.4.
- Arnaldo
Arnaldo Carvalho de Melo
maintainerRe: [PATCH 4/6] perf annotate-data: Bound the member nesting recursion
In reply to Arnaldo Carvalho de MeloQuoted 14 lines · Arnaldo Carvalho de Melo
Quoted 12 lines · Namhyung Kim
tools/perf/util/annotate-data.h struct annotated_member { · line 66
Quoted 8 lines · Arnaldo Carvalho de Melo
+++
b/tools/perf/util/annotate-data.htools/perf/util/annotate-data.h +3
-0@@ -61,6 +61,9 @@ struct annotated_member {int size;bool is_union;bool is_flex_array; +unsigned int depth; +/* Children not expanded because the nesting limit was reached */ +bool truncated;Nit: it'd be better to switch the order of the last two fields to reduce the padding.
Quoted 1 line
I should have used pahole... 8-)
MAX_MEMBER_DEPTH is 32, so we can make it an u8, did it:
⬢ [acme@toolbx perf-tools-next]$ pahole -C annotated_member ~/bin/perf struct annotated_member {
struct list_head node; /* 0 16 */ struct list_head children; /* 16 16 */ char * type_name; /* 32 8 */ char * var_name; /* 40 8 */ int offset; /* 48 4 */ int size; /* 52 4 */ _Bool is_union; /* 56 1 */ _Bool is_flex_array; /* 57 1 */ _Bool truncated; /* 58 1 */ u8 depth; /* 59 1 */
/* size: 64, cachelines: 1, members: 10 */ /* padding: 4 */ };
⬢ [acme@toolbx perf-tools-next]$
- Arnaldo