Skip to contentSkip to conversation

perf annotate-data: Support flexible array types

PATCH In review Namhyung Kim 3 participants 13 messages Sep 19 perf events

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

v5 cover perf annotate-data: Support flexible array types

Hello,

The flexible arrays are dynamically allocated with different size. So checking with the original type size won't match and cannot find the type if the offset is bigger than the size. This patch series detects those flex-arrays and allows accesses beyong the original size.

I'm not sure what's the best way to add test codes for data type profiling as it seems we need to add a lot more workloads for different cases. Probably we may want to split the workloads as separate binaries.

v5 changes)

* add a cleanup patch
* add is_union and is_flex_array fields to find member correctly

v4: https://lore.kernel.org/r/[email protected]/

* update last member even if the offset is same
* handle type qualifiers in die_has_flex_array()
* add zero-length array members as flexible arrays
* check return value of strbuf_init() for type name

v3: https://lore.kernel.org/r/[email protected]

* patch 1 was merged!
* check if last member is found
* check member location when finding flex-array
* ensure member typename is initiailized
* fix a bug to pass a wrong type in check_variable()
* fix a typo in a comment

v2: https://lore.kernel.org/r/[email protected]

* fix missing index increment in the histogram
* support flex array in union types
* add recursion check in die_has_flex_array()
* check negative index arrays properly
* avoid divide-by-zero when the size is unknown

v1: https://lore.kernel.org/r/[email protected]

Thanks,
Namhyung

Cc Zecheng Li <[email protected]>
Cc Yanbo Zhao <[email protected]>
Cc Tengda Wu <[email protected]>
Cc Shuai Xue <[email protected]>

Namhyung Kim (4):
perf dwarf-aux: Add die_has_flex_array() helper
perf annotate-data: A small cleanup in __add_member_cb() perf annotate-data: Allow out-of-size access for flex-array types perf annotate-data: Adjust type offset for flex-array

 tools/perf/util/annotate-data.c | 192 ++++++++++++++++++++++++--------
 tools/perf/util/annotate-data.h |   6 +
 tools/perf/util/dwarf-aux.c     | 107 ++++++++++++++++++
 tools/perf/util/dwarf-aux.h     |   3 +
 4 files changed, 262 insertions(+), 46 deletions(-)
Signature · 1 line
2.55.0
author maintainer
#1 lore raw

PATCH v5 1/4 perf dwarf-aux: Add die_has_flex_array() helper

The die_has_flex_array() returns true when the given type is a compound type and contains an array at the end. To prevent an infinite recursion add a depth field to the internal function.

---
 tools/perf/util/dwarf-aux.c | 107 ++++++++++++++++++++++++++++++++++++
 tools/perf/util/dwarf-aux.h |   3 +
 2 files changed, 110 insertions(+)
tools/perf/util/dwarf-aux.c +107 -0
index d7160f87ac7d7ab3..1560e721479a1ac2 100644
@@ -7,6 +7,7 @@
 #include <inttypes.h>
 #include <stdbool.h>
 #include <stdlib.h>
+#include <string.h>
 #include "debug.h"
 #include "dwarf-aux.h"
 #include "dwarf-regs.h"
@@ -2180,3 +2181,109 @@ Dwarf_Die *die_deref_ptr_type(Dwarf_Die *ptr_die, int offset,
 
 	return die_get_member_type(&type_die, offset, die_mem);
 }
+
+static bool is_flex_array_member(Dwarf_Die *mb_die)
+{
+	Dwarf_Die type_die;
+	Dwarf_Word size;
+
+	/* get the type of the member */
+	if (die_get_real_type(mb_die, &type_die) == NULL)
+		return false;
+
+	if (dwarf_tag(&type_die) != DW_TAG_array_type)
+		return false;
+
+	return dwarf_aggregate_size(&type_die, &size) < 0 || size == 0;
+}
+
+#define MAX_FLEX_ARRAY_RECURSION  256  /* arbitrary */
+
+static bool die_has_flex_array_recurse(Dwarf_Die *parent_die, int depth)
+{
+	Dwarf_Die die_mem, last_mb;
+	int tag = dwarf_tag(parent_die);
+	bool found = false;
+	Dwarf_Word loc, last_loc = 0;
+
+	if (tag != DW_TAG_structure_type && tag != DW_TAG_union_type)
+		return false;
+
+	/* prevent infinite recursion */
+	if (depth > MAX_FLEX_ARRAY_RECURSION)
+		return false;
+
+	if (dwarf_child(parent_die, &die_mem))
+		return false;
+
+	do {
+		if (dwarf_tag(&die_mem) != DW_TAG_member)
+			continue;
+
+		if (tag == DW_TAG_union_type) {
+			if (is_flex_array_member(&die_mem))
+				return true;
+
+			if (die_get_real_type(&die_mem, &last_mb) &&
+			    die_has_flex_array_recurse(&last_mb, depth + 1))
+				return true;
+		}
+
+		if (tag == DW_TAG_structure_type) {
+			if (die_get_data_member_location(&die_mem, &loc) < 0) {
+				/* ignore bitfields */
+				loc = 0;
+			}
+
+			if (last_loc <= loc) {
+				memcpy(&last_mb, &die_mem, sizeof(last_mb));
+				last_loc = loc;
+			}
+		}
+
+		found = true;
+	} while (dwarf_siblingof(&die_mem, &die_mem) == 0);
+
+	if (tag == DW_TAG_structure_type && found) {
+		if (is_flex_array_member(&last_mb))
+			return true;
+
+		if (die_get_real_type(&last_mb, &die_mem))
+			return die_has_flex_array_recurse(&die_mem, depth + 1);
+	}
+
+	return false;
+}
+
+/**
+ * die_has_flex_array - Check if the given type has a flex-array at the end
+ * @type_die: a pointer to type DIE
+ *
+ * This function returns %true iff @type_die is a struct or union type and has
+ * an array at the end.  Note that the flex-array has no element, it should have
+ * no size and the parent size doesn't include the flex-array.  So it should
+ * locate at the offset of the parent size.
+ *
+ * For simplicity, it assumes the parent size of aligned with the flex-array.
+ */
+bool die_has_flex_array(Dwarf_Die *type_die)
+{
+	Dwarf_Die real_type;
+
+	switch (dwarf_tag(type_die)) {
+	case DW_TAG_typedef:
+	case DW_TAG_const_type:
+	case DW_TAG_restrict_type:
+	case DW_TAG_volatile_type:
+	case DW_TAG_shared_type:
+		if (die_get_real_type(type_die, &real_type) == NULL)
+			return false;
+
+		type_die = &real_type;
+		break;
+	default:
+		break;
+	}
+
+	return die_has_flex_array_recurse(type_die, 0);
+}
tools/perf/util/dwarf-aux.h +3 -0
index 161f0bf980b6ee6a..9b662db710220522 100644
@@ -189,4 +189,7 @@ void die_collect_global_vars(Dwarf_Die *cu_die, struct die_var_type **var_types)
 /* Get the frame base information from CFA */
 int die_get_cfa(Dwarf *dwarf, u64 pc, int *preg, int *poffset);
 
+/* Check whether given type has a flex array */
+bool die_has_flex_array(Dwarf_Die *parent_die);
+
 #endif /* _DWARF_AUX_H */
Signature · 1 line
2.55.0
#2 lore raw

Re: [PATCH v5 1/4] perf dwarf-aux: Add die_has_flex_array() helper

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 6 lines

The die_has_flex_array() returns true when the given type is a compound type and contains an array at the end. To prevent an infinite recursion add a depth field to the internal function.

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#3 lore raw

PATCH v5 2/4 perf annotate-data: A small cleanup in __add_member_cb()

In reply to Namhyung Kim

Check return value of strbuf_init() and die_get_typename() to make sure if typename of a member gets never NULL.

Also remove the check for typedef by just using die_get_real_type(). It can get rid of member_type variable as well.

---
 tools/perf/util/annotate-data.c | 17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
tools/perf/util/annotate-data.c +9 -8
index aff60a630fd05b01..845a5d8c2b84b6a6 100644
@@ -226,7 +226,7 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)
 {
 	struct annotated_member *parent = arg;
 	struct annotated_member *member;
-	Dwarf_Die member_type, die_mem;
+	Dwarf_Die die_mem;
 	Dwarf_Word size, loc, bit_size = 0;
 	Dwarf_Attribute attr;
 	struct strbuf sb;
@@ -239,14 +239,15 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)
 	if (member == NULL)
 		return DIE_FIND_CB_END;
 
-	strbuf_init(&sb, 32);
-	die_get_typename(die, &sb);
+	if (strbuf_init(&sb, 32) < 0) {
+		free(member);
+		return DIE_FIND_CB_END;
+	}
 
-	__die_get_real_type(die, &member_type);
-	if (dwarf_tag(&member_type) == DW_TAG_typedef)
-		die_get_real_type(&member_type, &die_mem);
-	else
-		die_mem = member_type;
+	if (die_get_typename(die, &sb) < 0)
+		strbuf_add(&sb, "(unknown type)", 14);
+
+	die_get_real_type(die, &die_mem);
 
 	if (dwarf_aggregate_size(&die_mem, &size) < 0)
 		size = 0;
Signature · 1 line
2.55.0
#4 lore raw

Re: [PATCH v5 2/4] perf annotate-data: A small cleanup in __add_member_cb()

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 7 lines

Check return value of strbuf_init() and die_get_typename() to make sure if typename of a member gets never NULL.

Also remove the check for typedef by just using die_get_real_type(). It can get rid of member_type variable as well.

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
author maintainer
#5 lore raw

PATCH v5 3/4 perf annotate-data: Allow out-of-size access for flex-array types

In reply to Namhyung Kim

Structs that have a flex array will have accesses beyond its original size as the array was declared as 0 sized. For now, it just allow any offset bigger than the size. It could be refined later.

---
 tools/perf/util/annotate-data.c | 63 ++++++++++++++++++---------------
 tools/perf/util/annotate-data.h |  2 ++
 2 files changed, 36 insertions(+), 29 deletions(-)
tools/perf/util/annotate-data.c +34 -29
index 845a5d8c2b84b6a6..5dd6c6ec2d42451d 100644
@@ -7,6 +7,7 @@
 #include <errno.h>
 #include <stdio.h>
 #include <stdlib.h>
+#include <string.h>
 #include <inttypes.h>
 #include <linux/zalloc.h>
 
@@ -249,8 +250,15 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)
 
 	die_get_real_type(die, &die_mem);
 
-	if (dwarf_aggregate_size(&die_mem, &size) < 0)
-		size = 0;
+	if (dwarf_aggregate_size(&die_mem, &size) < 0 || size == 0) {
+		if (dwarf_tag(&die_mem) == DW_TAG_array_type) { /* flex-array? */
+			die_get_real_type(&die_mem, &die_mem);
+			if (dwarf_aggregate_size(&die_mem, &size) < 0)
+				size = 0;
+		} else {
+			size = 0;
+		}
+	}
 
 	if (dwarf_attr_integrate(die, DW_AT_data_member_location, &attr)) {
 		if (dwarf_formudata(&attr, &loc) != 0) {
@@ -400,6 +408,7 @@ static struct annotated_data_type *dso__findnew_data_type(struct dso *dso,
 	result->self.type_name = type_name;
 	result->self.size = size;
 	INIT_LIST_HEAD(&result->self.children);
+	result->flex_array = die_has_flex_array(type_die);
 
 	if (symbol_conf.annotate_data_member)
 		add_member_types(result, type_die);
@@ -518,13 +527,30 @@ static bool is_better_type(Dwarf_Die *type_a, Dwarf_Die *type_b)
 	return false;
 }
 
+static enum type_match_result check_type_offset(Dwarf_Die *type_die, int offset)
+{
+	Dwarf_Word size;
+
+	/* Get the size of the actual type */
+	if (dwarf_aggregate_size(type_die, &size) < 0)
+		return PERF_TMR_NO_SIZE;
+
+	/* Minimal sanity check */
+	if (offset < 0)
+		return PERF_TMR_BAD_OFFSET;
+
+	if ((unsigned)offset >= size && !die_has_flex_array(type_die))
+		return PERF_TMR_BAD_OFFSET;
+
+	return PERF_TMR_OK;
+}
+
 /* The type info will be saved in @type_die */
 static enum type_match_result check_variable(struct data_loc_info *dloc,
 					     Dwarf_Die *var_die,
 					     Dwarf_Die *type_die, int reg,
 					     int offset, bool is_fbreg)
 {
-	Dwarf_Word size;
 	bool needs_pointer = true;
 	Dwarf_Die sized_type;
 
@@ -555,15 +581,7 @@ static enum type_match_result check_variable(struct data_loc_info *dloc,
 	else
 		sized_type = *type_die;
 
-	/* Get the size of the actual type */
-	if (dwarf_aggregate_size(&sized_type, &size) < 0)
-		return PERF_TMR_NO_SIZE;
-
-	/* Minimal sanity check */
-	if ((unsigned)offset >= size)
-		return PERF_TMR_BAD_OFFSET;
-
-	return PERF_TMR_OK;
+	return check_type_offset(&sized_type, offset);
 }
 
 struct type_state_stack *find_stack_state(struct type_state *state,
@@ -1113,7 +1131,6 @@ static enum type_match_result check_matching_type(struct type_state *state,
 						  struct disasm_line *dl,
 						  Dwarf_Die *type_die)
 {
-	Dwarf_Word size;
 	u32 insn_offset = dl->al.offset;
 	int reg = dloc->op->reg1;
 	int offset = dloc->op->offset;
@@ -1167,12 +1184,7 @@ static enum type_match_result check_matching_type(struct type_state *state,
 		else
 			sized_type = *type_die;
 
-		/* Get the size of the actual type */
-		if (dwarf_aggregate_size(&sized_type, &size) < 0 ||
-		    (unsigned)dloc->type_offset >= size)
-			return PERF_TMR_BAD_OFFSET;
-
-		return PERF_TMR_OK;
+		return check_type_offset(&sized_type, dloc->type_offset);
 	}
 
 	if (state->regs[reg].kind == TSR_KIND_POINTER) {
@@ -1191,12 +1203,7 @@ static enum type_match_result check_matching_type(struct type_state *state,
 
 		dloc->type_offset = dloc->op->offset + state->regs[reg].offset;
 
-		/* Get the size of the actual type */
-		if (dwarf_aggregate_size(type_die, &size) < 0 ||
-		    (unsigned)dloc->type_offset >= size)
-			return PERF_TMR_BAD_OFFSET;
-
-		return PERF_TMR_OK;
+		return check_type_offset(type_die, dloc->type_offset);
 	}
 
 	if (state->regs[reg].kind == TSR_KIND_PERCPU_POINTER) {
@@ -1210,9 +1217,7 @@ static enum type_match_result check_matching_type(struct type_state *state,
 
 		dloc->type_offset = dloc->op->offset;
 
-		/* Get the size of the actual type */
-		if (dwarf_aggregate_size(type_die, &size) < 0 ||
-		    (unsigned)dloc->type_offset >= size)
+		if (check_type_offset(type_die, dloc->type_offset) != PERF_TMR_OK)
 			return PERF_TMR_BAIL_OUT;
 
 		return PERF_TMR_OK;
@@ -1840,7 +1845,7 @@ int annotated_data_type__update_samples(struct annotated_data_type *adt,
 			return -1;
 	}
 
-	if (offset < 0 || offset >= adt->self.size)
+	if (offset < 0 || (offset >= adt->self.size && !adt->flex_array))
 		return -1;
 
 	h = &adt->histograms[evsel->core.idx];
tools/perf/util/annotate-data.h +2 -0
index ca2096a9ee62cbfe..957726334907cc0e 100644
@@ -85,6 +85,7 @@ struct type_hist {
  * struct annotated_data_type - Data type to profile
  * @node: RB-tree node for dso->type_tree
  * @self: Actual type information
+ * @flex_array: Whether it has a flex array
  * @nr_histogram: Number of histogram entries
  * @histograms: An array of histograms
  *
@@ -93,6 +94,7 @@ struct type_hist {
 struct annotated_data_type {
 	struct rb_node node;
 	struct annotated_member self;
+	bool flex_array;
 	int nr_histograms;
 	struct type_hist *histograms;
 };
Signature · 1 line
2.55.0
#6 lore raw

Re: [PATCH v5 3/4] perf annotate-data: Allow out-of-size access for flex-array types

Sashiko has reviewed this patch and found no issues. It looks great!
Quoted 5 lines

Structs that have a flex array will have accesses beyond its original size as the array was declared as 0 sized. For now, it just allow any offset bigger than the size. It could be refined later.

Sashiko has reviewed this patch and found no issues. It looks great!

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
author maintainer
#7 lore raw

PATCH v5 4/4 perf annotate-data: Adjust type offset for flex-array

In reply to Namhyung Kim

The flex array members are located beyond the original type size. Also it needs to adjust the offset in an array to find a corresponding element using module operation. Note that we focus on access to type and field, so array index is not important.

Make sure to find a field name for flex arrays.

---
 tools/perf/util/annotate-data.c | 112 +++++++++++++++++++++++++++++---
 tools/perf/util/annotate-data.h |   4 ++
 2 files changed, 107 insertions(+), 9 deletions(-)
tools/perf/util/annotate-data.c +103 -9
index 5dd6c6ec2d42451d..e9b58a3d587be45b 100644
@@ -226,7 +226,7 @@ static bool data_type_less(struct rb_node *node_a, const struct rb_node *node_b)
 static int __add_member_cb(Dwarf_Die *die, void *arg)
 {
 	struct annotated_member *parent = arg;
-	struct annotated_member *member;
+	struct annotated_member *member, *prev;
 	Dwarf_Die die_mem;
 	Dwarf_Word size, loc, bit_size = 0;
 	Dwarf_Attribute attr;
@@ -253,6 +253,7 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)
 	if (dwarf_aggregate_size(&die_mem, &size) < 0 || size == 0) {
 		if (dwarf_tag(&die_mem) == DW_TAG_array_type) { /* flex-array? */
 			die_get_real_type(&die_mem, &die_mem);
+			member->is_flex_array = true;
 			if (dwarf_aggregate_size(&die_mem, &size) < 0)
 				size = 0;
 		} else {
@@ -299,12 +300,19 @@ static int __add_member_cb(Dwarf_Die *die, void *arg)
 	member->size = size;
 	member->offset = loc + parent->offset;
 	INIT_LIST_HEAD(&member->children);
-	list_add_tail(&member->node, &parent->children);
+
+	list_for_each_entry_reverse(prev, &parent->children, node) {
+		if (prev->offset <= member->offset)
+			break;
+	}
+	list_add(&member->node, &prev->node);
 
 	tag = dwarf_tag(&die_mem);
 	switch (tag) {
-	case DW_TAG_structure_type:
 	case DW_TAG_union_type:
+		member->is_union = true;
+		/* fall through */
+	case DW_TAG_structure_type:
 		die_find_child(&die_mem, __add_member_cb, member, &die_mem);
 		break;
 	default:
@@ -333,20 +341,84 @@ static void delete_members(struct annotated_member *member)
 	}
 }
 
+static struct annotated_member *find_flex_array(struct annotated_member *m)
+{
+	struct annotated_member *child;
+
+	if (list_empty(&m->children))
+		return NULL;
+
+	if (m->is_union) {
+		list_for_each_entry(child, &m->children, node) {
+			if (child->is_flex_array)
+				return child;
+		}
+		list_for_each_entry(child, &m->children, node) {
+			struct annotated_member *grand_child;
+
+			grand_child = find_flex_array(child);
+			if (grand_child)
+				return grand_child;
+		}
+		return NULL;
+	}
+
+	child = list_last_entry(&m->children, struct annotated_member, node);
+	if (child->is_flex_array)
+		return child;
+
+	return find_flex_array(child);
+}
+
+static struct annotated_member *get_flex_array_member(struct annotated_data_type *adt)
+{
+	return find_flex_array(&adt->self);
+}
+
 static int fill_member_name(char *buf, size_t sz, struct annotated_member *m,
-			    int offset, bool first)
+			    int offset, bool first, bool has_flex_array)
 {
 	struct annotated_member *child;
+	bool found = false;
+	int len;
 
 	if (list_empty(&m->children))
 		return 0;
 
 	list_for_each_entry(child, &m->children, node) {
-		int len;
-
 		if (offset < child->offset || offset >= child->offset + child->size)
 			continue;
 
+		found = true;
+		break;
+	}
+
+	if (!found && has_flex_array) {
+		/*
+		 * It may have an intermediate struct that has another struct that
+		 * contains a flex array.  In that case, the outer struct itself is
+		 * has no array and the size is less than the offset so the above
+		 * logic won't find the outer struct at the offset.
+		 */
+		child = find_flex_array(m);
+		if (child == NULL || offset < child->offset)
+			return 0;
+
+		/* find the immediate child that includes a flex array */
+		if (m->is_union) {
+			list_for_each_entry(child, &m->children, node) {
+				if (child->is_flex_array || find_flex_array(child)) {
+					found = true;
+					break;
+				}
+			}
+		} else {
+			child = list_last_entry(&m->children, struct annotated_member, node);
+			found = true;
+		}
+	}
+
+	if (found) {
 		/* It can have anonymous struct/union members */
 		if (child->var_name) {
 			len = scnprintf(buf, sz, "%s%s",
@@ -356,15 +428,18 @@ static int fill_member_name(char *buf, size_t sz, struct annotated_member *m,
 			len = 0;
 		}
 
-		return fill_member_name(buf + len, sz - len, child, offset, first) + len;
+		return fill_member_name(buf + len, sz - len, child, offset, first,
+					has_flex_array) + len;
 	}
+
 	return 0;
 }
 
 int annotated_data_type__get_member_name(struct annotated_data_type *adt,
 					 char *buf, size_t sz, int member_offset)
 {
-	return fill_member_name(buf, sz, &adt->self, member_offset, /*first=*/true);
+	return fill_member_name(buf, sz, &adt->self, member_offset, /*first=*/true,
+				adt->flex_array);
 }
 
 static struct annotated_data_type *dso__findnew_data_type(struct dso *dso,
@@ -1741,6 +1816,7 @@ struct annotated_data_type *find_data_type(struct data_loc_info *dloc)
 {
 	struct dso *dso = map__dso(dloc->ms->map);
 	Dwarf_Die type_die;
+	struct annotated_data_type *result;
 
 	/*
 	 * The type offset is the same as instruction offset by default.
@@ -1753,7 +1829,25 @@ struct annotated_data_type *find_data_type(struct data_loc_info *dloc)
 	if (find_data_type_die(dloc, &type_die) < 0)
 		return NULL;
 
-	return dso__findnew_data_type(dso, &type_die);
+	result = dso__findnew_data_type(dso, &type_die);
+	if (result == NULL)
+		return NULL;
+
+	if (result->flex_array && dloc->type_offset > result->self.size) {
+		struct annotated_member *flex_array = get_flex_array_member(result);
+
+		if (flex_array && flex_array->size > 0) {
+			int offset = dloc->type_offset;
+
+			/* adjust offset in the flex array */
+			offset -= flex_array->offset;
+			offset %= flex_array->size;
+			offset += flex_array->offset;
+
+			dloc->type_offset = offset;
+		}
+	}
+	return result;
 }
 
 static size_t data_type_hash(long key, void *ctx __maybe_unused)
tools/perf/util/annotate-data.h +4 -0
index 957726334907cc0e..14b8113521a927cf 100644
@@ -47,6 +47,8 @@ enum type_state_kind {
  * @var_name: Name of the member variable
  * @offset: Offset from the outer data type
  * @size: Size of the member field
+ * @is_union: Whether it's an union type
+ * @is_flex_array: Whether it's a flex array
  *
  * This represents a member type in a data type.
  */
@@ -57,6 +59,8 @@ struct annotated_member {
 	char *var_name;
 	int offset;
 	int size;
+	bool is_union;
+	bool is_flex_array;
 };
 
 /**
Signature · 1 line
2.55.0
#8 lore raw

Re: [PATCH v5 4/4] perf annotate-data: Adjust type offset for flex-array

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] perf annotate-data: incorrect flex array adjustment skips elements in struct padding --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf annotate-data: incorrect flex array adjustment skips elements in struct padding --

commit b87f5c6310512e6a55305050539cf57984efb087
Author: Namhyung Kim <[email protected]>

perf annotate-data: Adjust type offset for flex-array

The flex array members are located beyond the original type size. Also it needs to adjust the offset in an array to find a corresponding element using module operation. Note that we focus on access to type and field, so array index is not important.

Quoted 4 lines
tools/perf/util/annotate-data.c +0 -0
index 5dd6c6ec2d424..e9b58a3d587be 100644

[ ... ]

Quoted 9 lines
tools/perf/util/annotate-data.c +1 -0
@@ -1741,6 +1816,7 @@ struct annotated_data_type *find_data_type(struct data_loc_info *dloc)
 {
 	struct dso *dso = map__dso(dloc->ms->map);
 	Dwarf_Die type_die;
+	struct annotated_data_type *result;
 
 	/*
 	 * The type offset is the same as instruction offset by default.

tools/perf/util/annotate-data.c struct annotated_data_type *find_data_type(struct data_loc_info *dloc) · line 1836

Quoted 9 lines
tools/perf/util/annotate-data.c +5 -1
@@ -1753,7 +1829,25 @@ struct annotated_data_type *find_data_type(struct data_loc_info *dloc)
 	if (find_data_type_die(dloc, &type_die) < 0)
 		return NULL;
 
-	return dso__findnew_data_type(dso, &type_die);
+	result = dso__findnew_data_type(dso, &type_die);
+	if (result == NULL)
+		return NULL;
+
+	if (result->flex_array && dloc->type_offset > result->self.size) {

[Severity: Medium]
Since result->self.size (derived from DWARF's DW_AT_byte_size) includes trailing padding for alignment, won't this check skip flex array elements that fall within the struct's padded size boundary?

For example, if a struct has an 8-byte aligned member and a trailing flex array of 2-byte structs, the struct size includes up to 6 bytes of trailing padding. Accesses to indices 1 through 3 would remain <= result->self.size and erroneously skip this modulo adjustment.

Does this cause fill_member_name() to fail matching the unadjusted offset against the first element's nested fields, truncating the resolved name to just the outer array name instead of the specific inner field?

Quoted 15 lines
tools/perf/util/annotate-data.c +14 -0
+		struct annotated_member *flex_array = get_flex_array_member(result);
+
+		if (flex_array && flex_array->size > 0) {
+			int offset = dloc->type_offset;
+
+			/* adjust offset in the flex array */
+			offset -= flex_array->offset;
+			offset %= flex_array->size;
+			offset += flex_array->offset;
+
+			dloc->type_offset = offset;
+		}
+	}
+	return result;
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
author maintainer
#9 lore raw

Re: [PATCH v5 4/4] perf annotate-data: Adjust type offset for flex-array

Quoted 45 lines · On Sat, Sep 19, 2026 at 06:46:47AM +0000, [email protected] wrote:

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] perf annotate-data: incorrect flex array adjustment skips elements in struct padding --

commit b87f5c6310512e6a55305050539cf57984efb087
Author: Namhyung Kim <[email protected]>

perf annotate-data: Adjust type offset for flex-array

The flex array members are located beyond the original type size. Also it needs to adjust the offset in an array to find a corresponding element using module operation. Note that we focus on access to type and field, so array index is not important.

Quoted 4 lines
tools/perf/util/annotate-data.c +0 -0
index 5dd6c6ec2d424..e9b58a3d587be 100644

[ ... ]

Quoted 9 lines
tools/perf/util/annotate-data.c +1 -0
@@ -1741,6 +1816,7 @@ struct annotated_data_type *find_data_type(struct data_loc_info *dloc)
 {
 	struct dso *dso = map__dso(dloc->ms->map);
 	Dwarf_Die type_die;
+	struct annotated_data_type *result;
 
 	/*
 	 * The type offset is the same as instruction offset by default.

tools/perf/util/annotate-data.c struct annotated_data_type *find_data_type(struct data_loc_info *dloc) · line 1836

Quoted 9 lines
tools/perf/util/annotate-data.c +5 -1
@@ -1753,7 +1829,25 @@ struct annotated_data_type *find_data_type(struct data_loc_info *dloc)
 	if (find_data_type_die(dloc, &type_die) < 0)
 		return NULL;
 
-	return dso__findnew_data_type(dso, &type_die);
+	result = dso__findnew_data_type(dso, &type_die);
+	if (result == NULL)
+		return NULL;
+
+	if (result->flex_array && dloc->type_offset > result->self.size) {

[Severity: Medium]
Since result->self.size (derived from DWARF's DW_AT_byte_size) includes trailing padding for alignment, won't this check skip flex array elements that fall within the struct's padded size boundary?

Quoted 8 lines

For example, if a struct has an 8-byte aligned member and a trailing flex array of 2-byte structs, the struct size includes up to 6 bytes of trailing padding. Accesses to indices 1 through 3 would remain <= result->self.size and erroneously skip this modulo adjustment.

Does this cause fill_member_name() to fail matching the unadjusted offset against the first element's nested fields, truncating the resolved name to just the outer array name instead of the specific inner field?

As I said, I think practically this is fine and will find a flex array member for most cases.

Thanks,
Namhyung

Quoted 19 lines
Quoted 16 lines
tools/perf/util/annotate-data.c +14 -0
+		struct annotated_member *flex_array = get_flex_array_member(result);
+
+		if (flex_array && flex_array->size > 0) {
+			int offset = dloc->type_offset;
+
+			/* adjust offset in the flex array */
+			offset -= flex_array->offset;
+			offset %= flex_array->size;
+			offset += flex_array->offset;
+
+			dloc->type_offset = offset;
+		}
+	}
+	return result;
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
reviewer
#10 lore raw
In reply to Namhyung Kim
Quoted 11 lines · Namhyung Kim

Hello,

The flexible arrays are dynamically allocated with different size. So checking with the original type size won't match and cannot find the type if the offset is bigger than the size. This patch series detects those flex-arrays and allows accesses beyong the original size.

I'm not sure what's the best way to add test codes for data type profiling as it seems we need to add a lot more workloads for different cases. Probably we may want to split the workloads as separate binaries.

We have tools/perf/tests/shell/data_type_profiling.sh, I wonder can we not just have some workload with a:

```
struct flex_array {
   int len;
   int vals[];
};
```
then create a randomly sized flex array and access  it in a loop something like:
```
struct flex_array *a = calloc(sizeof(*a) + random_number * sizeof(int));
a->len = random_number;
while (!done) { /* done is set by an alarm like with noploop */
   for (int i = 0; i < a->len; i++) {
       a[i]++; /* Expect blame on struct flex_array here */
   }
}
```

ensuring the type profile blames the struct flex_array?

Thanks,
Ian

Quoted 55 lines

v5 changes)

* add a cleanup patch
* add is_union and is_flex_array fields to find member correctly

v4: https://lore.kernel.org/r/[email protected]/

* update last member even if the offset is same
* handle type qualifiers in die_has_flex_array()
* add zero-length array members as flexible arrays
* check return value of strbuf_init() for type name

v3: https://lore.kernel.org/r/[email protected]

* patch 1 was merged!
* check if last member is found
* check member location when finding flex-array
* ensure member typename is initiailized
* fix a bug to pass a wrong type in check_variable()
* fix a typo in a comment

v2: https://lore.kernel.org/r/[email protected]

* fix missing index increment in the histogram
* support flex array in union types
* add recursion check in die_has_flex_array()
* check negative index arrays properly
* avoid divide-by-zero when the size is unknown

v1: https://lore.kernel.org/r/[email protected]

Thanks,
Namhyung

Cc Zecheng Li <[email protected]>
Cc Yanbo Zhao <[email protected]>
Cc Tengda Wu <[email protected]>
Cc Shuai Xue <[email protected]>

Namhyung Kim (4):
perf dwarf-aux: Add die_has_flex_array() helper
perf annotate-data: A small cleanup in __add_member_cb() perf annotate-data: Allow out-of-size access for flex-array types perf annotate-data: Adjust type offset for flex-array

 tools/perf/util/annotate-data.c | 192 ++++++++++++++++++++++++--------
 tools/perf/util/annotate-data.h |   6 +
 tools/perf/util/dwarf-aux.c     | 107 ++++++++++++++++++
 tools/perf/util/dwarf-aux.h     |   3 +
 4 files changed, 262 insertions(+), 46 deletions(-)

--
2.55.0

author maintainer
#11 lore raw
Quoted 31 lines · Ian Rogers
Quoted 11 lines · Namhyung Kim

Hello,

The flexible arrays are dynamically allocated with different size. So checking with the original type size won't match and cannot find the type if the offset is bigger than the size. This patch series detects those flex-arrays and allows accesses beyong the original size.

I'm not sure what's the best way to add test codes for data type profiling as it seems we need to add a lot more workloads for different cases. Probably we may want to split the workloads as separate binaries.

We have tools/perf/tests/shell/data_type_profiling.sh, I wonder can we not just have some workload with a:

```
struct flex_array {
   int len;
   int vals[];
};
```
then create a randomly sized flex array and access  it in a loop something like:
```
struct flex_array *a = calloc(sizeof(*a) + random_number * sizeof(int));
a->len = random_number;
while (!done) { /* done is set by an alarm like with noploop */
   for (int i = 0; i < a->len; i++) {
       a[i]++; /* Expect blame on struct flex_array here */
   }
}
```
Quoted 1 line

ensuring the type profile blames the struct flex_array?

Sure, I can add that.

But I was afraid it'd need many small workloads to check data type profiling behavior for different aspects. Maybe we can think about it later when we need more. :)

Thanks,
Namhyung

reviewer
#12 lore raw
Quoted 35 lines · Namhyung Kim
Quoted 31 lines · Ian Rogers
Quoted 11 lines · Namhyung Kim

Hello,

The flexible arrays are dynamically allocated with different size. So checking with the original type size won't match and cannot find the type if the offset is bigger than the size. This patch series detects those flex-arrays and allows accesses beyong the original size.

I'm not sure what's the best way to add test codes for data type profiling as it seems we need to add a lot more workloads for different cases. Probably we may want to split the workloads as separate binaries.

We have tools/perf/tests/shell/data_type_profiling.sh, I wonder can we not just have some workload with a:

```
struct flex_array {
   int len;
   int vals[];
};
```
then create a randomly sized flex array and access  it in a loop something like:
```
struct flex_array *a = calloc(sizeof(*a) + random_number * sizeof(int));
a->len = random_number;
while (!done) { /* done is set by an alarm like with noploop */
   for (int i = 0; i < a->len; i++) {
       a[i]++; /* Expect blame on struct flex_array here */
   }
}
```
Quoted 1 line

ensuring the type profile blames the struct flex_array?

Quoted 5 lines

Sure, I can add that.

But I was afraid it'd need many small workloads to check data type profiling behavior for different aspects. Maybe we can think about it later when we need more. :)

Agreed, there may be a way to automate that. For the specific bug where flex arrays fail to get type profiles we should be able to avoid reproducing it with a simple test.

Thanks!
Ian

Quoted 3 lines

Thanks,
Namhyung