Skip to contentSkip to conversation

Introduce type match support

PATCH Superseded Daniel Müller 3 participants 19 messages Jun 23, 2022 – Jun 27, 2022

Subsystems (5)
Kselftest BPF BPF selftests libbpf +1 more subsystems

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover Introduce type match support

This patch set proposes the addition of a new way for performing type queries to BPF. It introduces the "type matches" relation, similar to what is already present with "type exists" (in the form of bpf_core_type_exists).

"type exists" performs fairly superficial checking, mostly concerned with whether a type exists in the kernel and is of the same kind (enum/struct/...). Notably, compatibility checks for members of composite types is lacking.

The newly introduced "type matches" (bpf_core_type_matches) fills this gap in that it performs stricter checks: compatibility of members and existence of similarly named enum variants is checked as well. E.g., given these definitions:

	struct task_struct___og { int pid; int tgid; };

	struct task_struct___foo { int foo; }

'task_struct___og' would "match" the kernel type 'task_struct', because the members match up, while 'task_struct___foo' would not match, because the kernel's 'task_struct' has no member named 'foo'.

More precisely, the "type match" relation is defined as follows (copied from source):
- modifiers and typedefs are stripped (and, hence, effectively ignored)
- generally speaking types need to be of same kind (struct vs. struct, union vs. union, etc.)
- exceptions are struct/union behind a pointer which could also match a forward declaration of a struct or union, respectively, and enum vs. enum64 (see below)
Then, depending on type:
- integers:
- match if size and signedness match
- arrays & pointers:
- target types are recursively matched
- structs & unions:
- local members need to exist in target with the same name
- for each member we recursively check match unless it is already behind a pointer, in which case we only check matching names and compatible kind
- enums:
- local variants have to have a match in target by symbolic name (but not numeric value)
- size has to match (but enum may match enum64 and vice versa)
- function pointers:
- number and position of arguments in local type has to match target
- for each argument and the return value we recursively check match

Enabling this feature requires a new relocation to be made known to the compiler. This is being taken care of for LLVM as part of
https://reviews.llvm.org/D126838.

If applied, among other things, usage of this functionality could have helped flag issues such as the one discussed here
https://lore.kernel.org/all/[email protected]/
earlier.

---
Changelog:
v1 -> v2:
- deduplicated and moved core algorithm into relo_core.c
- adjusted bpf_core_names_match to get btf_type passed in
- removed some length equality checks before strncmp usage
- correctly use kflag from targ_t instead of local_t
- added comment for meaning of kflag w/ FWD kind
- __u32 -> u32
- handle BTF_KIND_FWD properly in bpftool marking logic
- rebased

Daniel Müller (9):
bpf: Introduce TYPE_MATCH related constants/macros
bpftool: Honor BPF_CORE_TYPE_MATCHES relocation
bpf: Introduce btf_int_bits() function
libbpf: Add type match support
bpf: Add type match support
libbpf: Honor TYPE_MATCH relocation selftests/bpf: Add type-match checks to type-based tests selftests/bpf: Add test checking more characteristics selftests/bpf: Add nested type to type based tests

 include/linux/btf.h                           |   5 +
 include/uapi/linux/bpf.h                      |   1 +
 kernel/bpf/btf.c                              |   9 +
 tools/bpf/bpftool/gen.c                       | 107 +++++++
 tools/include/uapi/linux/bpf.h                |   1 +
 tools/lib/bpf/bpf_core_read.h                 |  11 +
 tools/lib/bpf/libbpf.c                        |   6 +
 tools/lib/bpf/relo_core.c                     | 292 +++++++++++++++++-
 tools/lib/bpf/relo_core.h                     |   4 +
 .../selftests/bpf/prog_tests/core_reloc.c     |  72 ++++-
 .../progs/btf__core_reloc_type_based___diff.c |   3 +
 .../selftests/bpf/progs/core_reloc_types.h    | 107 ++++++-
 .../bpf/progs/test_core_reloc_type_based.c    |  44 ++-
 13 files changed, 644 insertions(+), 18 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_type_based___diff.c
Signature · 1 line
2.30.2
#1 lore raw

PATCH v2 1/9 bpf: Introduce TYPE_MATCH related constants/macros

In order to provide type match support we require a new type of relocation which, in turn, requires toolchain support. Recent LLVM/Clang versions support a new value for the last argument to the __builtin_preserve_type_info builtin, for example.
With this change we introduce the necessary constants into relevant header files, mirroring what the compiler may support.

---
 include/uapi/linux/bpf.h       | 1 +
 tools/include/uapi/linux/bpf.h | 1 +
 tools/lib/bpf/bpf_core_read.h  | 1 +
 3 files changed, 3 insertions(+)
include/uapi/linux/bpf.h +1 -0
index e81362..42605c 100644
@@ -6782,6 +6782,7 @@ enum bpf_core_relo_kind {
 	BPF_CORE_TYPE_SIZE = 9,              /* type size in bytes */
 	BPF_CORE_ENUMVAL_EXISTS = 10,        /* enum value existence in target kernel */
 	BPF_CORE_ENUMVAL_VALUE = 11,         /* enum value integer value */
+	BPF_CORE_TYPE_MATCHES = 12,          /* type match in target kernel */
 };
 
 /*
tools/include/uapi/linux/bpf.h +1 -0
index e81362..42605c 100644
@@ -6782,6 +6782,7 @@ enum bpf_core_relo_kind {
 	BPF_CORE_TYPE_SIZE = 9,              /* type size in bytes */
 	BPF_CORE_ENUMVAL_EXISTS = 10,        /* enum value existence in target kernel */
 	BPF_CORE_ENUMVAL_VALUE = 11,         /* enum value integer value */
+	BPF_CORE_TYPE_MATCHES = 12,          /* type match in target kernel */
 };
 
 /*
tools/lib/bpf/bpf_core_read.h +1 -0
index fd48b1..2308f49 100644
@@ -29,6 +29,7 @@ enum bpf_type_id_kind {
 enum bpf_type_info_kind {
 	BPF_TYPE_EXISTS = 0,		/* type existence in target kernel */
 	BPF_TYPE_SIZE = 1,		/* type size in target kernel */
+	BPF_TYPE_MATCHES = 2,		/* type match in target kernel */
 };
 
 /* second argument to __builtin_preserve_enum_value() built-in */
Signature · 1 line
2.30.2
#2 lore raw

PATCH v2 2/9 bpftool: Honor BPF_CORE_TYPE_MATCHES relocation

In reply to Daniel Müller

bpftool needs to know about the newly introduced BPF_CORE_TYPE_MATCHES relocation for its 'gen min_core_btf' command to work properly in the present of this relocation.
Specifically, we need to make sure to mark types and fields so that they are present in the minimized BTF for "type match" checks to work out. However, contrary to the existing btfgen_record_field_relo, we need to rely on the BTF -- and not the spec -- to find fields. With this change we handle this new variant correctly. The functionality will be tested with follow on changes to BPF selftests, which already run against a minimized BTF created with bpftool.

---
 tools/bpf/bpftool/gen.c | 107 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)
tools/bpf/bpftool/gen.c +107 -0
index 480cbd8..6cd0ed 100644
@@ -1856,6 +1856,111 @@ static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp
 	return 0;
 }
 
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ *
+ * The `behind_ptr` argument is used to stop marking of composite types reached
+ * through a pointer. This way, we keep can keep BTF size in check while
+ * providing reasonable match semantics.
+ */
+static int btfgen_mark_types_match(struct btfgen_info *info, __u32 type_id, bool behind_ptr)
+{
+	const struct btf_type *btf_type;
+	struct btf *btf = info->src_btf;
+	struct btf_type *cloned_type;
+	int i, err;
+
+	if (type_id == 0)
+		return 0;
+
+	btf_type = btf__type_by_id(btf, type_id);
+	/* mark type on cloned BTF as used */
+	cloned_type = (struct btf_type *)btf__type_by_id(info->marked_btf, type_id);
+	cloned_type->name_off = MARKED;
+
+	switch (btf_kind(btf_type)) {
+	case BTF_KIND_UNKN:
+	case BTF_KIND_INT:
+	case BTF_KIND_FLOAT:
+	case BTF_KIND_ENUM:
+	case BTF_KIND_ENUM64:
+		break;
+	case BTF_KIND_STRUCT:
+	case BTF_KIND_UNION: {
+		struct btf_member *m = btf_members(btf_type);
+		__u16 vlen = btf_vlen(btf_type);
+
+		if (behind_ptr)
+			break;
+
+		for (i = 0; i < vlen; i++, m++) {
+			/* mark member */
+			btfgen_mark_member(info, type_id, i);
+
+			/* mark member's type */
+			err = btfgen_mark_types_match(info, m->type, false);
+			if (err)
+				return err;
+		}
+		break;
+	}
+	case BTF_KIND_CONST:
+	case BTF_KIND_FWD:
+	case BTF_KIND_VOLATILE:
+	case BTF_KIND_TYPEDEF:
+		return btfgen_mark_types_match(info, btf_type->type, false);
+	case BTF_KIND_PTR:
+		return btfgen_mark_types_match(info, btf_type->type, true);
+	case BTF_KIND_ARRAY: {
+		struct btf_array *array;
+
+		array = btf_array(btf_type);
+		/* mark array type */
+		err = btfgen_mark_types_match(info, array->type, false);
+		/* mark array's index type */
+		err = err ? : btfgen_mark_types_match(info, array->index_type, false);
+		if (err)
+			return err;
+		break;
+	}
+	case BTF_KIND_FUNC_PROTO: {
+		__u16 vlen = btf_vlen(btf_type);
+		struct btf_param *param;
+
+		/* mark ret type */
+		err = btfgen_mark_types_match(info, btf_type->type, false);
+		if (err)
+			return err;
+
+		/* mark parameters types */
+		param = btf_params(btf_type);
+		for (i = 0; i < vlen; i++) {
+			err = btfgen_mark_types_match(info, param->type, false);
+			if (err)
+				return err;
+			param++;
+		}
+		break;
+	}
+	/* tells if some other type needs to be handled */
+	default:
+		p_err("unsupported kind: %s (%d)", btf_kind_str(btf_type), type_id);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ */
+static int btfgen_record_types_match_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec)
+{
+	return btfgen_mark_types_match(info, targ_spec->root_type_id, false);
+}
+
 static int btfgen_record_type_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec)
 {
 	return btfgen_mark_type(info, targ_spec->root_type_id, true);
@@ -1882,6 +1987,8 @@ static int btfgen_record_reloc(struct btfgen_info *info, struct bpf_core_spec *r
 	case BPF_CORE_TYPE_EXISTS:
 	case BPF_CORE_TYPE_SIZE:
 		return btfgen_record_type_relo(info, res);
+	case BPF_CORE_TYPE_MATCHES:
+		return btfgen_record_types_match_relo(info, res);
 	case BPF_CORE_ENUMVAL_EXISTS:
 	case BPF_CORE_ENUMVAL_VALUE:
 		return btfgen_record_enumval_relo(info, res);
Signature · 1 line
2.30.2
Acked-by on v2 2/9
#3 lore raw

Re: [PATCH bpf-next v2 2/9] bpftool: Honor BPF_CORE_TYPE_MATCHES relocation

2022-06-23 21:21 UTC+0000 ~ Daniel Müller <[email protected]>

Quoted 22 lines

bpftool needs to know about the newly introduced BPF_CORE_TYPE_MATCHES relocation for its 'gen min_core_btf' command to work properly in the present of this relocation.
Specifically, we need to make sure to mark types and fields so that they are present in the minimized BTF for "type match" checks to work out. However, contrary to the existing btfgen_record_field_relo, we need to rely on the BTF -- and not the spec -- to find fields. With this change we handle this new variant correctly. The functionality will be tested with follow on changes to BPF selftests, which already run against a minimized BTF created with bpftool.

---
 tools/bpf/bpftool/gen.c | 107 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)
tools/bpf/bpftool/gen.c +0 -0
index 480cbd8..6cd0ed 100644

tools/bpf/bpftool/gen.c static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp · line 1864

Quoted 9 lines
tools/bpf/bpftool/gen.c +6 -0
@@ -1856,6 +1856,111 @@ static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp
 	return 0;
 }
 
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ *
+ * The `behind_ptr` argument is used to stop marking of composite types reached
+ * through a pointer. This way, we keep can keep BTF size in check while

Typo, "we keep can keep"

Quoted 85 lines
tools/bpf/bpftool/gen.c +85 -0
+ * providing reasonable match semantics.
+ */
+static int btfgen_mark_types_match(struct btfgen_info *info, __u32 type_id, bool behind_ptr)
+{
+	const struct btf_type *btf_type;
+	struct btf *btf = info->src_btf;
+	struct btf_type *cloned_type;
+	int i, err;
+
+	if (type_id == 0)
+		return 0;
+
+	btf_type = btf__type_by_id(btf, type_id);
+	/* mark type on cloned BTF as used */
+	cloned_type = (struct btf_type *)btf__type_by_id(info->marked_btf, type_id);
+	cloned_type->name_off = MARKED;
+
+	switch (btf_kind(btf_type)) {
+	case BTF_KIND_UNKN:
+	case BTF_KIND_INT:
+	case BTF_KIND_FLOAT:
+	case BTF_KIND_ENUM:
+	case BTF_KIND_ENUM64:
+		break;
+	case BTF_KIND_STRUCT:
+	case BTF_KIND_UNION: {
+		struct btf_member *m = btf_members(btf_type);
+		__u16 vlen = btf_vlen(btf_type);
+
+		if (behind_ptr)
+			break;
+
+		for (i = 0; i < vlen; i++, m++) {
+			/* mark member */
+			btfgen_mark_member(info, type_id, i);
+
+			/* mark member's type */
+			err = btfgen_mark_types_match(info, m->type, false);
+			if (err)
+				return err;
+		}
+		break;
+	}
+	case BTF_KIND_CONST:
+	case BTF_KIND_FWD:
+	case BTF_KIND_VOLATILE:
+	case BTF_KIND_TYPEDEF:
+		return btfgen_mark_types_match(info, btf_type->type, false);
+	case BTF_KIND_PTR:
+		return btfgen_mark_types_match(info, btf_type->type, true);
+	case BTF_KIND_ARRAY: {
+		struct btf_array *array;
+
+		array = btf_array(btf_type);
+		/* mark array type */
+		err = btfgen_mark_types_match(info, array->type, false);
+		/* mark array's index type */
+		err = err ? : btfgen_mark_types_match(info, array->index_type, false);
+		if (err)
+			return err;
+		break;
+	}
+	case BTF_KIND_FUNC_PROTO: {
+		__u16 vlen = btf_vlen(btf_type);
+		struct btf_param *param;
+
+		/* mark ret type */
+		err = btfgen_mark_types_match(info, btf_type->type, false);
+		if (err)
+			return err;
+
+		/* mark parameters types */
+		param = btf_params(btf_type);
+		for (i = 0; i < vlen; i++) {
+			err = btfgen_mark_types_match(info, param->type, false);
+			if (err)
+				return err;
+			param++;
+		}
+		break;
+	}
+	/* tells if some other type needs to be handled */
+	default:
+		p_err("unsupported kind: %s (%d)", btf_kind_str(btf_type), type_id);
+		return -EINVAL;

tools/bpf/bpftool/gen.c static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp · line 1959

Quoted 10 lines
tools/bpf/bpftool/gen.c +10 -0
+	}
+
+	return 0;
+}
+
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ */
+static int btfgen_record_types_match_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec)

Nit: Maybe btfgen_record_type_match_relo() ("type" singular), for consistency with btfgen_record_type_relo()?

Quoted 8 lines
tools/bpf/bpftool/gen.c +4 -0
+{
+	return btfgen_mark_types_match(info, targ_spec->root_type_id, false);
+}
+
 static int btfgen_record_type_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec)
 {
 	return btfgen_mark_type(info, targ_spec->root_type_id, true);

tools/bpf/bpftool/gen.c static int btfgen_record_reloc(struct btfgen_info *info, struct bpf_core_spec *r · line 1994

Quoted 8 lines
tools/bpf/bpftool/gen.c +2 -0
@@ -1882,6 +1987,8 @@ static int btfgen_record_reloc(struct btfgen_info *info, struct bpf_core_spec *r
 	case BPF_CORE_TYPE_EXISTS:
 	case BPF_CORE_TYPE_SIZE:
 		return btfgen_record_type_relo(info, res);
+	case BPF_CORE_TYPE_MATCHES:
+		return btfgen_record_types_match_relo(info, res);
 	case BPF_CORE_ENUMVAL_EXISTS:
 	case BPF_CORE_ENUMVAL_VALUE:
 		return btfgen_record_enumval_relo(info, res);

Aside from the minor nits, the patch looks good to me. Thanks!

#4 lore raw

Re: [PATCH bpf-next v2 2/9] bpftool: Honor BPF_CORE_TYPE_MATCHES relocation

Quoted 26 lines · Quentin Monnet

2022-06-23 21:21 UTC+0000 ~ Daniel Müller <[email protected]>

Quoted 25 lines

bpftool needs to know about the newly introduced BPF_CORE_TYPE_MATCHES relocation for its 'gen min_core_btf' command to work properly in the present of this relocation.
Specifically, we need to make sure to mark types and fields so that they are present in the minimized BTF for "type match" checks to work out. However, contrary to the existing btfgen_record_field_relo, we need to rely on the BTF -- and not the spec -- to find fields. With this change we handle this new variant correctly. The functionality will be tested with follow on changes to BPF selftests, which already run against a minimized BTF created with bpftool.

---
 tools/bpf/bpftool/gen.c | 107 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)
tools/bpf/bpftool/gen.c +0 -0
index 480cbd8..6cd0ed 100644
@@ -1856,6 +1856,111 @@ static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp
 	return 0;
 }
 
Quoted 8 lines

tools/bpf/bpftool/gen.c static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp · line 1864

Quoted 6 lines
tools/bpf/bpftool/gen.c +6 -0
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ *
+ * The `behind_ptr` argument is used to stop marking of composite types reached
+ * through a pointer. This way, we keep can keep BTF size in check while

Typo, "we keep can keep"

Fixed. Thanks!

Quoted 90 lines
Quoted 90 lines
tools/bpf/bpftool/gen.c +90 -0
+ * providing reasonable match semantics.
+ */
+static int btfgen_mark_types_match(struct btfgen_info *info, __u32 type_id, bool behind_ptr)
+{
+	const struct btf_type *btf_type;
+	struct btf *btf = info->src_btf;
+	struct btf_type *cloned_type;
+	int i, err;
+
+	if (type_id == 0)
+		return 0;
+
+	btf_type = btf__type_by_id(btf, type_id);
+	/* mark type on cloned BTF as used */
+	cloned_type = (struct btf_type *)btf__type_by_id(info->marked_btf, type_id);
+	cloned_type->name_off = MARKED;
+
+	switch (btf_kind(btf_type)) {
+	case BTF_KIND_UNKN:
+	case BTF_KIND_INT:
+	case BTF_KIND_FLOAT:
+	case BTF_KIND_ENUM:
+	case BTF_KIND_ENUM64:
+		break;
+	case BTF_KIND_STRUCT:
+	case BTF_KIND_UNION: {
+		struct btf_member *m = btf_members(btf_type);
+		__u16 vlen = btf_vlen(btf_type);
+
+		if (behind_ptr)
+			break;
+
+		for (i = 0; i < vlen; i++, m++) {
+			/* mark member */
+			btfgen_mark_member(info, type_id, i);
+
+			/* mark member's type */
+			err = btfgen_mark_types_match(info, m->type, false);
+			if (err)
+				return err;
+		}
+		break;
+	}
+	case BTF_KIND_CONST:
+	case BTF_KIND_FWD:
+	case BTF_KIND_VOLATILE:
+	case BTF_KIND_TYPEDEF:
+		return btfgen_mark_types_match(info, btf_type->type, false);
+	case BTF_KIND_PTR:
+		return btfgen_mark_types_match(info, btf_type->type, true);
+	case BTF_KIND_ARRAY: {
+		struct btf_array *array;
+
+		array = btf_array(btf_type);
+		/* mark array type */
+		err = btfgen_mark_types_match(info, array->type, false);
+		/* mark array's index type */
+		err = err ? : btfgen_mark_types_match(info, array->index_type, false);
+		if (err)
+			return err;
+		break;
+	}
+	case BTF_KIND_FUNC_PROTO: {
+		__u16 vlen = btf_vlen(btf_type);
+		struct btf_param *param;
+
+		/* mark ret type */
+		err = btfgen_mark_types_match(info, btf_type->type, false);
+		if (err)
+			return err;
+
+		/* mark parameters types */
+		param = btf_params(btf_type);
+		for (i = 0; i < vlen; i++) {
+			err = btfgen_mark_types_match(info, param->type, false);
+			if (err)
+				return err;
+			param++;
+		}
+		break;
+	}
+	/* tells if some other type needs to be handled */
+	default:
+		p_err("unsupported kind: %s (%d)", btf_kind_str(btf_type), type_id);
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
Quoted 8 lines

tools/bpf/bpftool/gen.c static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp · line 1959

Quoted 5 lines
tools/bpf/bpftool/gen.c +5 -0
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ */
+static int btfgen_record_types_match_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec)

Nit: Maybe btfgen_record_type_match_relo() ("type" singular), for consistency with btfgen_record_type_relo()?

Sure, changed.

Quoted 10 lines
Quoted 10 lines
tools/bpf/bpftool/gen.c +4 -0
+{
+	return btfgen_mark_types_match(info, targ_spec->root_type_id, false);
+}
+
 static int btfgen_record_type_relo(struct btfgen_info *info, struct bpf_core_spec *targ_spec)
 {
 	return btfgen_mark_type(info, targ_spec->root_type_id, true);
@@ -1882,6 +1987,8 @@ static int btfgen_record_reloc(struct btfgen_info *info, struct bpf_core_spec *r
 	case BPF_CORE_TYPE_EXISTS:
 	case BPF_CORE_TYPE_SIZE:
Quoted 8 lines

tools/bpf/bpftool/gen.c static int btfgen_record_reloc(struct btfgen_info *info, struct bpf_core_spec *r · line 1994

Quoted 6 lines
tools/bpf/bpftool/gen.c +2 -0
 		return btfgen_record_type_relo(info, res);
+	case BPF_CORE_TYPE_MATCHES:
+		return btfgen_record_types_match_relo(info, res);
 	case BPF_CORE_ENUMVAL_EXISTS:
 	case BPF_CORE_ENUMVAL_VALUE:
 		return btfgen_record_enumval_relo(info, res);

Aside from the minor nits, the patch looks good to me. Thanks!

Thanks for your review!

Daniel

Quoted 1 line
#5 lore raw

Re: [PATCH bpf-next v2 2/9] bpftool: Honor BPF_CORE_TYPE_MATCHES relocation

In reply to Daniel Müller
Quoted 69 lines · Daniel Müller

bpftool needs to know about the newly introduced BPF_CORE_TYPE_MATCHES relocation for its 'gen min_core_btf' command to work properly in the present of this relocation.
Specifically, we need to make sure to mark types and fields so that they are present in the minimized BTF for "type match" checks to work out. However, contrary to the existing btfgen_record_field_relo, we need to rely on the BTF -- and not the spec -- to find fields. With this change we handle this new variant correctly. The functionality will be tested with follow on changes to BPF selftests, which already run against a minimized BTF created with bpftool.

---
 tools/bpf/bpftool/gen.c | 107 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)
tools/bpf/bpftool/gen.c +43 -0
index 480cbd8..6cd0ed 100644
@@ -1856,6 +1856,111 @@ static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp
        return 0;
 }
 
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ *
+ * The `behind_ptr` argument is used to stop marking of composite types reached
+ * through a pointer. This way, we keep can keep BTF size in check while
+ * providing reasonable match semantics.
+ */
+static int btfgen_mark_types_match(struct btfgen_info *info, __u32 type_id, bool behind_ptr)
+{
+       const struct btf_type *btf_type;
+       struct btf *btf = info->src_btf;
+       struct btf_type *cloned_type;
+       int i, err;
+
+       if (type_id == 0)
+               return 0;
+
+       btf_type = btf__type_by_id(btf, type_id);
+       /* mark type on cloned BTF as used */
+       cloned_type = (struct btf_type *)btf__type_by_id(info->marked_btf, type_id);
+       cloned_type->name_off = MARKED;
+
+       switch (btf_kind(btf_type)) {
+       case BTF_KIND_UNKN:
+       case BTF_KIND_INT:
+       case BTF_KIND_FLOAT:
+       case BTF_KIND_ENUM:
+       case BTF_KIND_ENUM64:
+               break;
+       case BTF_KIND_STRUCT:
+       case BTF_KIND_UNION: {
+               struct btf_member *m = btf_members(btf_type);
+               __u16 vlen = btf_vlen(btf_type);
+
+               if (behind_ptr)
+                       break;
+
+               for (i = 0; i < vlen; i++, m++) {
+                       /* mark member */
+                       btfgen_mark_member(info, type_id, i);
+
+                       /* mark member's type */

tools/bpf/bpftool/gen.c static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp · line 1911

Quoted 10 lines
tools/bpf/bpftool/gen.c +10 -0
+                       err = btfgen_mark_types_match(info, m->type, false);
+                       if (err)
+                               return err;
+               }
+               break;
+       }
+       case BTF_KIND_CONST:
+       case BTF_KIND_FWD:
+       case BTF_KIND_VOLATILE:
+       case BTF_KIND_TYPEDEF:

BTF_KIND_RESTRICT is missing?

Quoted 15 lines
tools/bpf/bpftool/gen.c +15 -0
+               return btfgen_mark_types_match(info, btf_type->type, false);
+       case BTF_KIND_PTR:
+               return btfgen_mark_types_match(info, btf_type->type, true);
+       case BTF_KIND_ARRAY: {
+               struct btf_array *array;
+
+               array = btf_array(btf_type);
+               /* mark array type */
+               err = btfgen_mark_types_match(info, array->type, false);
+               /* mark array's index type */
+               err = err ? : btfgen_mark_types_match(info, array->index_type, false);
+               if (err)
+                       return err;
+               break;
+       }

[...]

#6 lore raw

Re: [PATCH bpf-next v2 2/9] bpftool: Honor BPF_CORE_TYPE_MATCHES relocation

Quoted 74 lines · Andrii Nakryiko
Quoted 73 lines · Daniel Müller

bpftool needs to know about the newly introduced BPF_CORE_TYPE_MATCHES relocation for its 'gen min_core_btf' command to work properly in the present of this relocation.
Specifically, we need to make sure to mark types and fields so that they are present in the minimized BTF for "type match" checks to work out. However, contrary to the existing btfgen_record_field_relo, we need to rely on the BTF -- and not the spec -- to find fields. With this change we handle this new variant correctly. The functionality will be tested with follow on changes to BPF selftests, which already run against a minimized BTF created with bpftool.

---
 tools/bpf/bpftool/gen.c | 107 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 107 insertions(+)
tools/bpf/bpftool/gen.c +47 -0
index 480cbd8..6cd0ed 100644
@@ -1856,6 +1856,111 @@ static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp
        return 0;
 }
 
+/* Mark types, members, and member types. Compared to btfgen_record_field_relo,
+ * this function does not rely on the target spec for inferring members, but
+ * uses the associated BTF.
+ *
+ * The `behind_ptr` argument is used to stop marking of composite types reached
+ * through a pointer. This way, we keep can keep BTF size in check while
+ * providing reasonable match semantics.
+ */
+static int btfgen_mark_types_match(struct btfgen_info *info, __u32 type_id, bool behind_ptr)
+{
+       const struct btf_type *btf_type;
+       struct btf *btf = info->src_btf;
+       struct btf_type *cloned_type;
+       int i, err;
+
+       if (type_id == 0)
+               return 0;
+
+       btf_type = btf__type_by_id(btf, type_id);
+       /* mark type on cloned BTF as used */
+       cloned_type = (struct btf_type *)btf__type_by_id(info->marked_btf, type_id);
+       cloned_type->name_off = MARKED;
+
+       switch (btf_kind(btf_type)) {
+       case BTF_KIND_UNKN:
+       case BTF_KIND_INT:
+       case BTF_KIND_FLOAT:
+       case BTF_KIND_ENUM:
+       case BTF_KIND_ENUM64:
+               break;
+       case BTF_KIND_STRUCT:
+       case BTF_KIND_UNION: {
+               struct btf_member *m = btf_members(btf_type);
+               __u16 vlen = btf_vlen(btf_type);
+
+               if (behind_ptr)
+                       break;
+
+               for (i = 0; i < vlen; i++, m++) {
+                       /* mark member */
+                       btfgen_mark_member(info, type_id, i);
+
+                       /* mark member's type */
+                       err = btfgen_mark_types_match(info, m->type, false);
+                       if (err)
+                               return err;
+               }
Quoted 8 lines

tools/bpf/bpftool/gen.c static int btfgen_record_field_relo(struct btfgen_info *info, struct bpf_core_sp · line 1911

Quoted 6 lines
tools/bpf/bpftool/gen.c +6 -0
+               break;
+       }
+       case BTF_KIND_CONST:
+       case BTF_KIND_FWD:
+       case BTF_KIND_VOLATILE:
+       case BTF_KIND_TYPEDEF:

BTF_KIND_RESTRICT is missing?

Good catch. It's missing in btfgen_mark_type as well. Will add it.

Quoted 17 lines
Quoted 15 lines
tools/bpf/bpftool/gen.c +15 -0
+               return btfgen_mark_types_match(info, btf_type->type, false);
+       case BTF_KIND_PTR:
+               return btfgen_mark_types_match(info, btf_type->type, true);
+       case BTF_KIND_ARRAY: {
+               struct btf_array *array;
+
+               array = btf_array(btf_type);
+               /* mark array type */
+               err = btfgen_mark_types_match(info, array->type, false);
+               /* mark array's index type */
+               err = err ? : btfgen_mark_types_match(info, array->index_type, false);
+               if (err)
+                       return err;
+               break;
+       }

[...]

#7 lore raw

PATCH v2 3/9 bpf: Introduce btf_int_bits() function

In reply to Daniel Müller

This change adds the btf_int_bits() function to include/linux/btf.h. It mirrors what already exists in user space and will be required by follow on changes.

---
 include/linux/btf.h | 5 +++++
 1 file changed, 5 insertions(+)
include/linux/btf.h +5 -0
index 1bfed7..54a65a 100644
@@ -242,6 +242,11 @@ static inline u8 btf_int_offset(const struct btf_type *t)
 	return BTF_INT_OFFSET(*(u32 *)(t + 1));
 }
 
+static inline u8 btf_int_bits(const struct btf_type *t)
+{
+	return BTF_INT_BITS(*(u32 *)(t + 1));
+}
+
 static inline u8 btf_int_encoding(const struct btf_type *t)
 {
 	return BTF_INT_ENCODING(*(u32 *)(t + 1));
Signature · 1 line
2.30.2
#8 lore raw

PATCH v2 4/9 libbpf: Add type match support

In reply to Daniel Müller

This patch adds support for the proposed type match relation to relo_core where it is shared between userspace and kernel. A bit more plumbing is necessary and will arrive with subsequent changes to actually use it -- this patch only introduces the main matching algorithm.

The matching relation is defined as follows (copy from source):
- modifiers and typedefs are stripped (and, hence, effectively ignored)
- generally speaking types need to be of same kind (struct vs. struct, union vs. union, etc.)
- exceptions are struct/union behind a pointer which could also match a forward declaration of a struct or union, respectively, and enum vs. enum64 (see below)
Then, depending on type:
- integers:
- match if size and signedness match
- arrays & pointers:
- target types are recursively matched
- structs & unions:
- local members need to exist in target with the same name
- for each member we recursively check match unless it is already behind a pointer, in which case we only check matching names and compatible kind
- enums:
- local variants have to have a match in target by symbolic name (but not numeric value)
- size has to match (but enum may match enum64 and vice versa)
- function pointers:
- number and position of arguments in local type has to match target
- for each argument and the return value we recursively check match

---
 tools/lib/bpf/relo_core.c | 276 ++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/relo_core.h |   2 +
 2 files changed, 278 insertions(+)
tools/lib/bpf/relo_core.c +276 -0
index 6ad3c3..bc5b060 100644
@@ -1330,3 +1330,279 @@ int bpf_core_calc_relo_insn(const char *prog_name,
 
 	return 0;
 }
+
+static bool bpf_core_names_match(const struct btf *local_btf, const struct btf_type *local_t,
+				 const struct btf *targ_btf, const struct btf_type *targ_t)
+{
+	const char *local_n, *targ_n;
+
+	local_n = btf__name_by_offset(local_btf, local_t->name_off);
+	targ_n = btf__name_by_offset(targ_btf, targ_t->name_off);
+
+	return !strncmp(local_n, targ_n, bpf_core_essential_name_len(local_n));
+}
+
+static int bpf_core_enums_match(const struct btf *local_btf, const struct btf_type *local_t,
+				const struct btf *targ_btf, const struct btf_type *targ_t)
+{
+	__u16 local_vlen = btf_vlen(local_t);
+	__u16 targ_vlen = btf_vlen(targ_t);
+	int i, j;
+
+	if (local_t->size != targ_t->size)
+		return 0;
+
+	if (local_vlen > targ_vlen)
+		return 0;
+
+	/* iterate over the local enum's variants and make sure each has
+	 * a symbolic name correspondent in the target
+	 */
+	for (i = 0; i < local_vlen; i++) {
+		bool matched = false;
+		const char *local_n;
+		__u32 local_n_off;
+		size_t local_len;
+
+		local_n_off = btf_is_enum(local_t) ? btf_enum(local_t)[i].name_off :
+						     btf_enum64(local_t)[i].name_off;
+
+		local_n = btf__name_by_offset(local_btf, local_n_off);
+		local_len = bpf_core_essential_name_len(local_n);
+
+		for (j = 0; j < targ_vlen; j++) {
+			const char *targ_n;
+			__u32 targ_n_off;
+
+			targ_n_off = btf_is_enum(targ_t) ? btf_enum(targ_t)[j].name_off :
+							   btf_enum64(targ_t)[j].name_off;
+			targ_n = btf__name_by_offset(targ_btf, targ_n_off);
+
+			if (str_is_empty(targ_n))
+				continue;
+
+			if (!strncmp(local_n, targ_n, local_len)) {
+				matched = true;
+				break;
+			}
+		}
+
+		if (!matched)
+			return 0;
+	}
+	return 1;
+}
+
+static int bpf_core_composites_match(const struct btf *local_btf, const struct btf_type *local_t,
+				     const struct btf *targ_btf, const struct btf_type *targ_t,
+				     int level)
+{
+	const struct btf_member *local_m = btf_members(local_t);
+	__u16 local_vlen = btf_vlen(local_t);
+	__u16 targ_vlen = btf_vlen(targ_t);
+	int i, j, err;
+
+	if (local_vlen > targ_vlen)
+		return 0;
+
+	/* check that all local members have a match in the target */
+	for (i = 0; i < local_vlen; i++, local_m++) {
+		const char *local_n = btf__name_by_offset(local_btf, local_m->name_off);
+		const struct btf_member *targ_m = btf_members(targ_t);
+		bool matched = false;
+
+		for (j = 0; j < targ_vlen; j++, targ_m++) {
+			const char *targ_n = btf__name_by_offset(targ_btf, targ_m->name_off);
+
+			if (str_is_empty(targ_n))
+				continue;
+
+			if (strcmp(local_n, targ_n) != 0)
+				continue;
+
+			err = __bpf_core_types_match(local_btf, local_m->type, targ_btf,
+						     targ_m->type, level - 1);
+			if (err > 0) {
+				matched = true;
+				break;
+			}
+		}
+
+		if (!matched)
+			return 0;
+	}
+	return 1;
+}
+
+/* Check that two types "match".
+ *
+ * The matching relation is defined as follows:
+ * - modifiers and typedefs are stripped (and, hence, effectively ignored)
+ * - generally speaking types need to be of same kind (struct vs. struct, union
+ *   vs. union, etc.)
+ *   - exceptions are struct/union behind a pointer which could also match a
+ *     forward declaration of a struct or union, respectively, and enum vs.
+ *     enum64 (see below)
+ * Then, depending on type:
+ * - integers:
+ *   - match if size and signedness match
+ * - arrays & pointers:
+ *   - target types are recursively matched
+ * - structs & unions:
+ *   - local members need to exist in target with the same name
+ *   - for each member we recursively check match unless it is already behind a
+ *     pointer, in which case we only check matching names and compatible kind
+ * - enums:
+ *   - local variants have to have a match in target by symbolic name (but not
+ *     numeric value)
+ *   - size has to match (but enum may match enum64 and vice versa)
+ * - function pointers:
+ *   - number and position of arguments in local type has to match target
+ *   - for each argument and the return value we recursively check match
+ */
+int __bpf_core_types_match(const struct btf *local_btf, __u32 local_id, const struct btf *targ_btf,
+			   __u32 targ_id, int level)
+{
+	const struct btf_type *local_t, *targ_t, *prev_local_t;
+	int depth = 32; /* max recursion depth */
+	__u16 local_k;
+
+	if (level <= 0)
+		return -EINVAL;
+
+	local_t = btf_type_by_id(local_btf, local_id);
+	targ_t = btf_type_by_id(targ_btf, targ_id);
+
+recur:
+	depth--;
+	if (depth < 0)
+		return -EINVAL;
+
+	prev_local_t = local_t;
+
+	local_t = skip_mods_and_typedefs(local_btf, local_id, &local_id);
+	targ_t = skip_mods_and_typedefs(targ_btf, targ_id, &targ_id);
+	if (!local_t || !targ_t)
+		return -EINVAL;
+
+	if (!bpf_core_names_match(local_btf, local_t, targ_btf, targ_t))
+		return 0;
+
+	local_k = btf_kind(local_t);
+
+	switch (local_k) {
+	case BTF_KIND_UNKN:
+		return local_k == btf_kind(targ_t);
+	case BTF_KIND_FWD: {
+		bool local_f = BTF_INFO_KFLAG(local_t->info);
+		__u16 targ_k = btf_kind(targ_t);
+
+		if (btf_is_ptr(prev_local_t)) {
+			if (local_k == targ_k)
+				return local_f == BTF_INFO_KFLAG(targ_t->info);
+
+			/* for forward declarations kflag dictates whether the
+			 * target is a struct (0) or union (1)
+			 */
+			return (targ_k == BTF_KIND_STRUCT && !local_f) ||
+			       (targ_k == BTF_KIND_UNION && local_f);
+		} else {
+			if (local_k != targ_k)
+				return 0;
+
+			/* match if the forward declaration is for the same kind */
+			return local_f == BTF_INFO_KFLAG(targ_t->info);
+		}
+	}
+	case BTF_KIND_ENUM:
+	case BTF_KIND_ENUM64:
+		if (!btf_is_any_enum(targ_t))
+			return 0;
+
+		return bpf_core_enums_match(local_btf, local_t, targ_btf, targ_t);
+	case BTF_KIND_STRUCT:
+	case BTF_KIND_UNION: {
+		__u16 targ_k = btf_kind(targ_t);
+
+		if (btf_is_ptr(prev_local_t)) {
+			bool targ_f = BTF_INFO_KFLAG(targ_t->info);
+
+			if (local_k == targ_k)
+				return 1;
+
+			if (targ_k != BTF_KIND_FWD)
+				return 0;
+
+			return (local_k == BTF_KIND_UNION) == targ_f;
+		} else {
+			if (local_k != targ_k)
+				return 0;
+
+			return bpf_core_composites_match(local_btf, local_t, targ_btf, targ_t,
+							 level);
+		}
+	}
+	case BTF_KIND_INT: {
+		__u8 local_sgn;
+		__u8 targ_sgn;
+
+		if (local_k != btf_kind(targ_t))
+			return 0;
+
+		local_sgn = btf_int_encoding(local_t) & BTF_INT_SIGNED;
+		targ_sgn = btf_int_encoding(targ_t) & BTF_INT_SIGNED;
+
+		return btf_int_bits(local_t) == btf_int_bits(targ_t) && local_sgn == targ_sgn;
+	}
+	case BTF_KIND_PTR:
+		if (local_k != btf_kind(targ_t))
+			return 0;
+
+		local_id = local_t->type;
+		targ_id = targ_t->type;
+		goto recur;
+	case BTF_KIND_ARRAY: {
+		const struct btf_array *local_array = btf_array(local_t);
+		const struct btf_array *targ_array = btf_array(targ_t);
+
+		if (local_k != btf_kind(targ_t))
+			return 0;
+
+		if (local_array->nelems != targ_array->nelems)
+			return 0;
+
+		local_id = local_array->type;
+		targ_id = targ_array->type;
+		goto recur;
+	}
+	case BTF_KIND_FUNC_PROTO: {
+		struct btf_param *local_p = btf_params(local_t);
+		struct btf_param *targ_p = btf_params(targ_t);
+		__u16 local_vlen = btf_vlen(local_t);
+		__u16 targ_vlen = btf_vlen(targ_t);
+		int i, err;
+
+		if (local_k != btf_kind(targ_t))
+			return 0;
+
+		if (local_vlen != targ_vlen)
+			return 0;
+
+		for (i = 0; i < local_vlen; i++, local_p++, targ_p++) {
+			err = __bpf_core_types_match(local_btf, local_p->type, targ_btf,
+						     targ_p->type, level - 1);
+			if (err <= 0)
+				return err;
+		}
+
+		/* tail recurse for return type check */
+		local_id = local_t->type;
+		targ_id = targ_t->type;
+		goto recur;
+	}
+	default:
+		pr_warn("unexpected kind %s relocated, local [%d], target [%d]\n",
+			btf_kind_str(local_t), local_id, targ_id);
+		return 0;
+	}
+}
tools/lib/bpf/relo_core.h +2 -0
index 7df0da0..289c63 100644
@@ -70,6 +70,8 @@ struct bpf_core_relo_res {
 
 int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
 			      const struct btf *targ_btf, __u32 targ_id);
+int __bpf_core_types_match(const struct btf *local_btf, __u32 local_id, const struct btf *targ_btf,
+			   __u32 targ_id, int level);
 
 size_t bpf_core_essential_name_len(const char *name);
 
Signature · 1 line
2.30.2
#9 lore raw

Re: [PATCH bpf-next v2 4/9] libbpf: Add type match support

Quoted 47 lines · Daniel Müller

This patch adds support for the proposed type match relation to relo_core where it is shared between userspace and kernel. A bit more plumbing is necessary and will arrive with subsequent changes to actually use it -- this patch only introduces the main matching algorithm.

The matching relation is defined as follows (copy from source):
- modifiers and typedefs are stripped (and, hence, effectively ignored)
- generally speaking types need to be of same kind (struct vs. struct, union vs. union, etc.)
- exceptions are struct/union behind a pointer which could also match a forward declaration of a struct or union, respectively, and enum vs. enum64 (see below)
Then, depending on type:
- integers:
- match if size and signedness match
- arrays & pointers:
- target types are recursively matched
- structs & unions:
- local members need to exist in target with the same name
- for each member we recursively check match unless it is already behind a pointer, in which case we only check matching names and compatible kind
- enums:
- local variants have to have a match in target by symbolic name (but not numeric value)
- size has to match (but enum may match enum64 and vice versa)
- function pointers:
- number and position of arguments in local type has to match target
- for each argument and the return value we recursively check match

---
 tools/lib/bpf/relo_core.c | 276 ++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/relo_core.h |   2 +
 2 files changed, 278 insertions(+)
tools/lib/bpf/relo_core.c +2 -0
index 6ad3c3..bc5b060 100644
@@ -1330,3 +1330,279 @@ int bpf_core_calc_relo_insn(const char *prog_name,
 
        return 0;
 }
+
+static bool bpf_core_names_match(const struct btf *local_btf, const struct btf_type *local_t,

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1344

Quoted 10 lines
tools/lib/bpf/relo_core.c +10 -0
+                                const struct btf *targ_btf, const struct btf_type *targ_t)
+{
+       const char *local_n, *targ_n;
+
+       local_n = btf__name_by_offset(local_btf, local_t->name_off);
+       targ_n = btf__name_by_offset(targ_btf, targ_t->name_off);
+
+       return !strncmp(local_n, targ_n, bpf_core_essential_name_len(local_n));
+}
+

we have similar check in existing code in at least two other places (search for strncmp in relo_core.c). But it doesn't always work with btf_type, it sometimes is field name, sometimes is part of core_spec.

so it's confusing that we have this helper used in *one* place, and other places open-code this logic. We can probably have a helper, but it will have to be taking const char * arguments and doing bpf_core_essential_name_len() for both

Quoted 30 lines
tools/lib/bpf/relo_core.c +30 -0
+static int bpf_core_enums_match(const struct btf *local_btf, const struct btf_type *local_t,
+                               const struct btf *targ_btf, const struct btf_type *targ_t)
+{
+       __u16 local_vlen = btf_vlen(local_t);
+       __u16 targ_vlen = btf_vlen(targ_t);
+       int i, j;
+
+       if (local_t->size != targ_t->size)
+               return 0;
+
+       if (local_vlen > targ_vlen)
+               return 0;
+
+       /* iterate over the local enum's variants and make sure each has
+        * a symbolic name correspondent in the target
+        */
+       for (i = 0; i < local_vlen; i++) {
+               bool matched = false;
+               const char *local_n;
+               __u32 local_n_off;
+               size_t local_len;
+
+               local_n_off = btf_is_enum(local_t) ? btf_enum(local_t)[i].name_off :
+                                                    btf_enum64(local_t)[i].name_off;
+
+               local_n = btf__name_by_offset(local_btf, local_n_off);
+               local_len = bpf_core_essential_name_len(local_n);
+
+               for (j = 0; j < targ_vlen; j++) {
+                       const char *targ_n;

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1384

Quoted 10 lines
tools/lib/bpf/relo_core.c +10 -0
+                       __u32 targ_n_off;
+
+                       targ_n_off = btf_is_enum(targ_t) ? btf_enum(targ_t)[j].name_off :
+                                                          btf_enum64(targ_t)[j].name_off;
+                       targ_n = btf__name_by_offset(targ_btf, targ_n_off);
+
+                       if (str_is_empty(targ_n))
+                               continue;
+
+                       if (!strncmp(local_n, targ_n, local_len)) {

and here you open-code name check instead of using your helper ;) but also shouldn't you calculate "essential name len" for target enum as well?.. otherwise local whatever___abc will match whatever123, which won't be right

and I'm not hard-core enough to easily understand !strncmp() (as I also mentioned in another email), I think explicit == 0 is easier to follow for str[n]cmp() APIs.

Quoted 27 lines
tools/lib/bpf/relo_core.c +27 -0
+                               matched = true;
+                               break;
+                       }
+               }
+
+               if (!matched)
+                       return 0;
+       }
+       return 1;
+}
+
+static int bpf_core_composites_match(const struct btf *local_btf, const struct btf_type *local_t,
+                                    const struct btf *targ_btf, const struct btf_type *targ_t,
+                                    int level)
+{
+       const struct btf_member *local_m = btf_members(local_t);
+       __u16 local_vlen = btf_vlen(local_t);
+       __u16 targ_vlen = btf_vlen(targ_t);
+       int i, j, err;
+
+       if (local_vlen > targ_vlen)
+               return 0;
+
+       /* check that all local members have a match in the target */
+       for (i = 0; i < local_vlen; i++, local_m++) {
+               const char *local_n = btf__name_by_offset(local_btf, local_m->name_off);
+               const struct btf_member *targ_m = btf_members(targ_t);

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1421

Quoted 10 lines
tools/lib/bpf/relo_core.c +10 -0
+               bool matched = false;
+
+               for (j = 0; j < targ_vlen; j++, targ_m++) {
+                       const char *targ_n = btf__name_by_offset(targ_btf, targ_m->name_off);
+
+                       if (str_is_empty(targ_n))
+                               continue;
+
+                       if (strcmp(local_n, targ_n) != 0)
+                               continue;

let's have the essential_len logic used consistently for all these field and type name checks?

Quoted 14 lines
tools/lib/bpf/relo_core.c +14 -0
+
+                       err = __bpf_core_types_match(local_btf, local_m->type, targ_btf,
+                                                    targ_m->type, level - 1);
+                       if (err > 0) {
+                               matched = true;
+                               break;
+                       }
+               }
+
+               if (!matched)
+                       return 0;
+       }
+       return 1;
+}

[...]

Quoted 14 lines
tools/lib/bpf/relo_core.c +14 -0
+       depth--;
+       if (depth < 0)
+               return -EINVAL;
+
+       prev_local_t = local_t;
+
+       local_t = skip_mods_and_typedefs(local_btf, local_id, &local_id);
+       targ_t = skip_mods_and_typedefs(targ_btf, targ_id, &targ_id);
+       if (!local_t || !targ_t)
+               return -EINVAL;
+
+       if (!bpf_core_names_match(local_btf, local_t, targ_btf, targ_t))
+               return 0;
+

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1500

Quoted 10 lines
tools/lib/bpf/relo_core.c +10 -0
+       local_k = btf_kind(local_t);
+
+       switch (local_k) {
+       case BTF_KIND_UNKN:
+               return local_k == btf_kind(targ_t);
+       case BTF_KIND_FWD: {
+               bool local_f = BTF_INFO_KFLAG(local_t->info);
+               __u16 targ_k = btf_kind(targ_t);
+
+               if (btf_is_ptr(prev_local_t)) {

this doesn't work in general, you can have PTR -> CONST -> FWD, you need to just remember that you saw PTR in the chain of types

Quoted 16 lines
tools/lib/bpf/relo_core.c +16 -0
+                       if (local_k == targ_k)
+                               return local_f == BTF_INFO_KFLAG(targ_t->info);
+
+                       /* for forward declarations kflag dictates whether the
+                        * target is a struct (0) or union (1)
+                        */
+                       return (targ_k == BTF_KIND_STRUCT && !local_f) ||
+                              (targ_k == BTF_KIND_UNION && local_f);
+               } else {
+                       if (local_k != targ_k)
+                               return 0;
+
+                       /* match if the forward declaration is for the same kind */
+                       return local_f == BTF_INFO_KFLAG(targ_t->info);
+               }
+       }

[...]

#10 lore raw

Re: [PATCH bpf-next v2 4/9] libbpf: Add type match support

Quoted 63 lines · Andrii Nakryiko
Quoted 47 lines · Daniel Müller

This patch adds support for the proposed type match relation to relo_core where it is shared between userspace and kernel. A bit more plumbing is necessary and will arrive with subsequent changes to actually use it -- this patch only introduces the main matching algorithm.

The matching relation is defined as follows (copy from source):
- modifiers and typedefs are stripped (and, hence, effectively ignored)
- generally speaking types need to be of same kind (struct vs. struct, union vs. union, etc.)
- exceptions are struct/union behind a pointer which could also match a forward declaration of a struct or union, respectively, and enum vs. enum64 (see below)
Then, depending on type:
- integers:
- match if size and signedness match
- arrays & pointers:
- target types are recursively matched
- structs & unions:
- local members need to exist in target with the same name
- for each member we recursively check match unless it is already behind a pointer, in which case we only check matching names and compatible kind
- enums:
- local variants have to have a match in target by symbolic name (but not numeric value)
- size has to match (but enum may match enum64 and vice versa)
- function pointers:
- number and position of arguments in local type has to match target
- for each argument and the return value we recursively check match

---
 tools/lib/bpf/relo_core.c | 276 ++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/relo_core.h |   2 +
 2 files changed, 278 insertions(+)
tools/lib/bpf/relo_core.c +2 -0
index 6ad3c3..bc5b060 100644
@@ -1330,3 +1330,279 @@ int bpf_core_calc_relo_insn(const char *prog_name,
 
        return 0;
 }
+
+static bool bpf_core_names_match(const struct btf *local_btf, const struct btf_type *local_t,

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1344

Quoted 10 lines
tools/lib/bpf/relo_core.c +10 -0
+                                const struct btf *targ_btf, const struct btf_type *targ_t)
+{
+       const char *local_n, *targ_n;
+
+       local_n = btf__name_by_offset(local_btf, local_t->name_off);
+       targ_n = btf__name_by_offset(targ_btf, targ_t->name_off);
+
+       return !strncmp(local_n, targ_n, bpf_core_essential_name_len(local_n));
+}
+

we have similar check in existing code in at least two other places (search for strncmp in relo_core.c). But it doesn't always work with btf_type, it sometimes is field name, sometimes is part of core_spec.

Quoted 4 lines

so it's confusing that we have this helper used in *one* place, and other places open-code this logic. We can probably have a helper, but it will have to be taking const char * arguments and doing bpf_core_essential_name_len() for both

Sure.

Quoted 46 lines
Quoted 30 lines
tools/lib/bpf/relo_core.c +30 -0
+static int bpf_core_enums_match(const struct btf *local_btf, const struct btf_type *local_t,
+                               const struct btf *targ_btf, const struct btf_type *targ_t)
+{
+       __u16 local_vlen = btf_vlen(local_t);
+       __u16 targ_vlen = btf_vlen(targ_t);
+       int i, j;
+
+       if (local_t->size != targ_t->size)
+               return 0;
+
+       if (local_vlen > targ_vlen)
+               return 0;
+
+       /* iterate over the local enum's variants and make sure each has
+        * a symbolic name correspondent in the target
+        */
+       for (i = 0; i < local_vlen; i++) {
+               bool matched = false;
+               const char *local_n;
+               __u32 local_n_off;
+               size_t local_len;
+
+               local_n_off = btf_is_enum(local_t) ? btf_enum(local_t)[i].name_off :
+                                                    btf_enum64(local_t)[i].name_off;
+
+               local_n = btf__name_by_offset(local_btf, local_n_off);
+               local_len = bpf_core_essential_name_len(local_n);
+
+               for (j = 0; j < targ_vlen; j++) {
+                       const char *targ_n;

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1384

Quoted 10 lines
tools/lib/bpf/relo_core.c +10 -0
+                       __u32 targ_n_off;
+
+                       targ_n_off = btf_is_enum(targ_t) ? btf_enum(targ_t)[j].name_off :
+                                                          btf_enum64(targ_t)[j].name_off;
+                       targ_n = btf__name_by_offset(targ_btf, targ_n_off);
+
+                       if (str_is_empty(targ_n))
+                               continue;
+
+                       if (!strncmp(local_n, targ_n, local_len)) {

and here you open-code name check instead of using your helper ;) but also shouldn't you calculate "essential name len" for target enum as well?.. otherwise local whatever___abc will match whatever123, which won't be right

Quoted 3 lines

and I'm not hard-core enough to easily understand !strncmp() (as I also mentioned in another email), I think explicit == 0 is easier to follow for str[n]cmp() APIs.

Done.

Quoted 32 lines
Quoted 32 lines
tools/lib/bpf/relo_core.c +32 -0
+                               matched = true;
+                               break;
+                       }
+               }
+
+               if (!matched)
+                       return 0;
+       }
+       return 1;
+}
+
+static int bpf_core_composites_match(const struct btf *local_btf, const struct btf_type *local_t,
+                                    const struct btf *targ_btf, const struct btf_type *targ_t,
+                                    int level)
+{
+       const struct btf_member *local_m = btf_members(local_t);
+       __u16 local_vlen = btf_vlen(local_t);
+       __u16 targ_vlen = btf_vlen(targ_t);
+       int i, j, err;
+
+       if (local_vlen > targ_vlen)
+               return 0;
+
+       /* check that all local members have a match in the target */
+       for (i = 0; i < local_vlen; i++, local_m++) {
+               const char *local_n = btf__name_by_offset(local_btf, local_m->name_off);
+               const struct btf_member *targ_m = btf_members(targ_t);
+               bool matched = false;
+
+               for (j = 0; j < targ_vlen; j++, targ_m++) {
+                       const char *targ_n = btf__name_by_offset(targ_btf, targ_m->name_off);
+
Quoted 8 lines

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1421

Quoted 5 lines
tools/lib/bpf/relo_core.c +5 -0
+                       if (str_is_empty(targ_n))
+                               continue;
+
+                       if (strcmp(local_n, targ_n) != 0)
+                               continue;

let's have the essential_len logic used consistently for all these field and type name checks?

Sounds good.

[...]

Quoted 19 lines
Quoted 19 lines
tools/lib/bpf/relo_core.c +19 -0
+       depth--;
+       if (depth < 0)
+               return -EINVAL;
+
+       prev_local_t = local_t;
+
+       local_t = skip_mods_and_typedefs(local_btf, local_id, &local_id);
+       targ_t = skip_mods_and_typedefs(targ_btf, targ_id, &targ_id);
+       if (!local_t || !targ_t)
+               return -EINVAL;
+
+       if (!bpf_core_names_match(local_btf, local_t, targ_btf, targ_t))
+               return 0;
+
+       local_k = btf_kind(local_t);
+
+       switch (local_k) {
+       case BTF_KIND_UNKN:
+               return local_k == btf_kind(targ_t);
Quoted 8 lines

tools/lib/bpf/relo_core.c int bpf_core_calc_relo_insn(const char *prog_name, · line 1500

Quoted 5 lines
tools/lib/bpf/relo_core.c +5 -0
+       case BTF_KIND_FWD: {
+               bool local_f = BTF_INFO_KFLAG(local_t->info);
+               __u16 targ_k = btf_kind(targ_t);
+
+               if (btf_is_ptr(prev_local_t)) {

this doesn't work in general, you can have PTR -> CONST -> FWD, you need to just remember that you saw PTR in the chain of types

Fair enough; will adjust.

[...]

Thanks,
Daniel

#11 lore raw

PATCH v2 5/9 bpf: Add type match support

In reply to Daniel Müller

This change implements the kernel side of the "type matches" support, just calling the previously added core logic in relo_core.c.

---
 kernel/bpf/btf.c | 9 +++++++++
 1 file changed, 9 insertions(+)
kernel/bpf/btf.c +9 -0
index f08037..d06855d 100644
@@ -7524,6 +7524,15 @@ int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
 					   MAX_TYPES_ARE_COMPAT_DEPTH);
 }
 
+#define MAX_TYPES_MATCH_DEPTH 2
+
+int bpf_core_types_match(const struct btf *local_btf, u32 local_id,
+			 const struct btf *targ_btf, u32 targ_id)
+{
+	return __bpf_core_types_match(local_btf, local_id, targ_btf, targ_id,
+				      MAX_TYPES_MATCH_DEPTH);
+}
+
 static bool bpf_core_is_flavor_sep(const char *s)
 {
 	/* check X___Y name pattern, where X and Y are not underscores */
Signature · 1 line
2.30.2
#12 lore raw

PATCH v2 6/9 libbpf: Honor TYPE_MATCH relocation

In reply to Daniel Müller

This patch finalizes support for the proposed type match relation in libbpf by hooking it up to the TYPE_MATCH relocation. For this relocation to be handled correctly by LLVM we have D126838
(https://reviews.llvm.org/D126838).
The main functionality is present behind the newly introduced bpf_core_type_matches macro (similar to bpf_core_type_exists). This macro can be used to check whether a local type has a "match" in a target BTF.

---
 tools/lib/bpf/bpf_core_read.h | 10 ++++++++++
 tools/lib/bpf/libbpf.c        |  6 ++++++
 tools/lib/bpf/relo_core.c     | 16 ++++++++++++----
 tools/lib/bpf/relo_core.h     |  2 ++
 4 files changed, 30 insertions(+), 4 deletions(-)
tools/lib/bpf/bpf_core_read.h +10 -0
index 2308f49..496e6a 100644
@@ -184,6 +184,16 @@ enum bpf_enum_value_kind {
 #define bpf_core_type_exists(type)					    \
 	__builtin_preserve_type_info(*(typeof(type) *)0, BPF_TYPE_EXISTS)
 
+/*
+ * Convenience macro to check that provided named type
+ * (struct/union/enum/typedef) "matches" that in a target kernel.
+ * Returns:
+ *    1, if the type matches in the target kernel's BTF;
+ *    0, if the type does not match any in the target kernel
+ */
+#define bpf_core_type_matches(type)					    \
+	__builtin_preserve_type_info(*(typeof(type) *)0, BPF_TYPE_MATCHES)
+
 /*
  * Convenience macro to get the byte size of a provided named type
  * (struct/union/enum/typedef) in a target kernel.
tools/lib/bpf/libbpf.c +6 -0
index 49e359c..fee15d 100644
@@ -5805,6 +5805,12 @@ int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
 	}
 }
 
+int bpf_core_types_match(const struct btf *local_btf, __u32 local_id,
+			 const struct btf *targ_btf, __u32 targ_id)
+{
+	return __bpf_core_types_match(local_btf, local_id, targ_btf, targ_id, 32);
+}
+
 static size_t bpf_core_hash_fn(const void *key, void *ctx)
 {
 	return (size_t)key;
tools/lib/bpf/relo_core.c +12 -4
index bc5b060..862185 100644
@@ -95,6 +95,7 @@ static const char *core_relo_kind_str(enum bpf_core_relo_kind kind)
 	case BPF_CORE_TYPE_ID_LOCAL: return "local_type_id";
 	case BPF_CORE_TYPE_ID_TARGET: return "target_type_id";
 	case BPF_CORE_TYPE_EXISTS: return "type_exists";
+	case BPF_CORE_TYPE_MATCHES: return "type_matches";
 	case BPF_CORE_TYPE_SIZE: return "type_size";
 	case BPF_CORE_ENUMVAL_EXISTS: return "enumval_exists";
 	case BPF_CORE_ENUMVAL_VALUE: return "enumval_value";
@@ -123,6 +124,7 @@ static bool core_relo_is_type_based(enum bpf_core_relo_kind kind)
 	case BPF_CORE_TYPE_ID_LOCAL:
 	case BPF_CORE_TYPE_ID_TARGET:
 	case BPF_CORE_TYPE_EXISTS:
+	case BPF_CORE_TYPE_MATCHES:
 	case BPF_CORE_TYPE_SIZE:
 		return true;
 	default:
@@ -171,7 +173,7 @@ static bool core_relo_is_enumval_based(enum bpf_core_relo_kind kind)
  *   - field 'a' access (corresponds to '2' in low-level spec);
  *   - array element #3 access (corresponds to '3' in low-level spec).
  *
- * Type-based relocations (TYPE_EXISTS/TYPE_SIZE,
+ * Type-based relocations (TYPE_EXISTS/TYPE_MATCHES/TYPE_SIZE,
  * TYPE_ID_LOCAL/TYPE_ID_TARGET) don't capture any field information. Their
  * spec and raw_spec are kept empty.
  *
@@ -488,9 +490,14 @@ static int bpf_core_spec_match(struct bpf_core_spec *local_spec,
 	targ_spec->relo_kind = local_spec->relo_kind;
 
 	if (core_relo_is_type_based(local_spec->relo_kind)) {
-		return bpf_core_types_are_compat(local_spec->btf,
-						 local_spec->root_type_id,
-						 targ_btf, targ_id);
+		if (local_spec->relo_kind == BPF_CORE_TYPE_MATCHES)
+			return bpf_core_types_match(local_spec->btf,
+						    local_spec->root_type_id,
+						    targ_btf, targ_id);
+		else
+			return bpf_core_types_are_compat(local_spec->btf,
+							 local_spec->root_type_id,
+							 targ_btf, targ_id);
 	}
 
 	local_acc = &local_spec->spec[0];
@@ -739,6 +746,7 @@ static int bpf_core_calc_type_relo(const struct bpf_core_relo *relo,
 			*validate = false;
 		break;
 	case BPF_CORE_TYPE_EXISTS:
+	case BPF_CORE_TYPE_MATCHES:
 		*val = 1;
 		break;
 	case BPF_CORE_TYPE_SIZE:
tools/lib/bpf/relo_core.h +2 -0
index 289c63..9230234 100644
@@ -72,6 +72,8 @@ int bpf_core_types_are_compat(const struct btf *local_btf, __u32 local_id,
 			      const struct btf *targ_btf, __u32 targ_id);
 int __bpf_core_types_match(const struct btf *local_btf, __u32 local_id, const struct btf *targ_btf,
 			   __u32 targ_id, int level);
+int bpf_core_types_match(const struct btf *local_btf, __u32 local_id, const struct btf *targ_btf,
+			 __u32 targ_id);
 
 size_t bpf_core_essential_name_len(const char *name);
 
Signature · 1 line
2.30.2
#13 lore raw

Re: [PATCH bpf-next v2 6/9] libbpf: Honor TYPE_MATCH relocation

Quoted 5 lines · Daniel Müller

This patch finalizes support for the proposed type match relation in libbpf by hooking it up to the TYPE_MATCH relocation. For this relocation to be handled correctly by LLVM we have D126838
(https://reviews.llvm.org/D126838).

I think we are getting pretty close, is it time to land LLVM diff? Yonghong, WDYT?

Quoted 13 lines

The main functionality is present behind the newly introduced bpf_core_type_matches macro (similar to bpf_core_type_exists). This macro can be used to check whether a local type has a "match" in a target BTF.

---
 tools/lib/bpf/bpf_core_read.h | 10 ++++++++++
 tools/lib/bpf/libbpf.c        |  6 ++++++
 tools/lib/bpf/relo_core.c     | 16 ++++++++++++----
 tools/lib/bpf/relo_core.h     |  2 ++
 4 files changed, 30 insertions(+), 4 deletions(-)

[...]

#14 lore raw

PATCH v2 7/9 selftests/bpf: Add type-match checks to type-based tests

In reply to Daniel Müller

Now that we have type-match logic in both libbpf and the kernel, this change adjusts the existing BPF self tests to check this functionality. Specifically, we extend the existing type-based tests to check the previously introduced bpf_core_type_matches macro.

---
 .../selftests/bpf/prog_tests/core_reloc.c     | 31 ++++++++++++++++--
 .../selftests/bpf/progs/core_reloc_types.h    | 14 +++++++-
 .../bpf/progs/test_core_reloc_type_based.c    | 32 ++++++++++++++++++-
 3 files changed, 73 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/prog_tests/core_reloc.c +29 -2
index 2f92fe..328dd7 100644
@@ -543,7 +543,6 @@ static int __trigger_module_test_read(const struct core_reloc_test_case *test)
 	return 0;
 }
 
-
 static const struct core_reloc_test_case test_cases[] = {
 	/* validate we can find kernel image and use its BTF for relocs */
 	{
@@ -752,7 +751,7 @@ static const struct core_reloc_test_case test_cases[] = {
 	SIZE_CASE(size___diff_offs),
 	SIZE_ERR_CASE(size___err_ambiguous),
 
-	/* validate type existence and size relocations */
+	/* validate type existence, match, and size relocations */
 	TYPE_BASED_CASE(type_based, {
 		.struct_exists = 1,
 		.union_exists = 1,
@@ -765,6 +764,19 @@ static const struct core_reloc_test_case test_cases[] = {
 		.typedef_void_ptr_exists = 1,
 		.typedef_func_proto_exists = 1,
 		.typedef_arr_exists = 1,
+
+		.struct_matches = 1,
+		.union_matches = 1,
+		.enum_matches = 1,
+		.typedef_named_struct_matches = 1,
+		.typedef_anon_struct_matches = 1,
+		.typedef_struct_ptr_matches = 1,
+		.typedef_int_matches = 1,
+		.typedef_enum_matches = 1,
+		.typedef_void_ptr_matches = 1,
+		.typedef_func_proto_matches = 1,
+		.typedef_arr_matches = 1,
+
 		.struct_sz = sizeof(struct a_struct),
 		.union_sz = sizeof(union a_union),
 		.enum_sz = sizeof(enum an_enum),
@@ -792,6 +804,19 @@ static const struct core_reloc_test_case test_cases[] = {
 		.typedef_void_ptr_exists = 1,
 		.typedef_func_proto_exists = 1,
 		.typedef_arr_exists = 1,
+
+		.struct_matches = 0,
+		.union_matches = 0,
+		.enum_matches = 0,
+		.typedef_named_struct_matches = 0,
+		.typedef_anon_struct_matches = 0,
+		.typedef_struct_ptr_matches = 1,
+		.typedef_int_matches = 0,
+		.typedef_enum_matches = 0,
+		.typedef_void_ptr_matches = 1,
+		.typedef_func_proto_matches = 0,
+		.typedef_arr_matches = 0,
+
 		.struct_sz = sizeof(struct a_struct___diff_sz),
 		.union_sz = sizeof(union a_union___diff_sz),
 		.enum_sz = sizeof(enum an_enum___diff_sz),
@@ -806,10 +831,12 @@ static const struct core_reloc_test_case test_cases[] = {
 	}),
 	TYPE_BASED_CASE(type_based___incompat, {
 		.enum_exists = 1,
+		.enum_matches = 1,
 		.enum_sz = sizeof(enum an_enum),
 	}),
 	TYPE_BASED_CASE(type_based___fn_wrong_args, {
 		.struct_exists = 1,
+		.struct_matches = 1,
 		.struct_sz = sizeof(struct a_struct),
 	}),
 
tools/testing/selftests/bpf/progs/core_reloc_types.h +13 -1
index 26e1033..6a44f3e 100644
@@ -860,7 +860,7 @@ struct core_reloc_size___err_ambiguous2 {
 };
 
 /*
- * TYPE EXISTENCE & SIZE
+ * TYPE EXISTENCE, MATCH & SIZE
  */
 struct core_reloc_type_based_output {
 	bool struct_exists;
@@ -875,6 +875,18 @@ struct core_reloc_type_based_output {
 	bool typedef_func_proto_exists;
 	bool typedef_arr_exists;
 
+	bool struct_matches;
+	bool union_matches;
+	bool enum_matches;
+	bool typedef_named_struct_matches;
+	bool typedef_anon_struct_matches;
+	bool typedef_struct_ptr_matches;
+	bool typedef_int_matches;
+	bool typedef_enum_matches;
+	bool typedef_void_ptr_matches;
+	bool typedef_func_proto_matches;
+	bool typedef_arr_matches;
+
 	int struct_sz;
 	int union_sz;
 	int enum_sz;
tools/testing/selftests/bpf/progs/test_core_reloc_type_based.c +31 -1
index fb60f8..325ead 100644
@@ -61,6 +61,18 @@ struct core_reloc_type_based_output {
 	bool typedef_func_proto_exists;
 	bool typedef_arr_exists;
 
+	bool struct_matches;
+	bool union_matches;
+	bool enum_matches;
+	bool typedef_named_struct_matches;
+	bool typedef_anon_struct_matches;
+	bool typedef_struct_ptr_matches;
+	bool typedef_int_matches;
+	bool typedef_enum_matches;
+	bool typedef_void_ptr_matches;
+	bool typedef_func_proto_matches;
+	bool typedef_arr_matches;
+
 	int struct_sz;
 	int union_sz;
 	int enum_sz;
@@ -77,7 +89,13 @@ struct core_reloc_type_based_output {
 SEC("raw_tracepoint/sys_enter")
 int test_core_type_based(void *ctx)
 {
-#if __has_builtin(__builtin_preserve_type_info)
+	/* Support for the BPF_TYPE_MATCHES argument to the
+	 * __builtin_preserve_type_info builtin was added at some point during
+	 * development of clang 15 and it's what we require for this test. Part of it
+	 * could run with merely __builtin_preserve_type_info (which could be checked
+	 * separately), but we have to find an upper bound.
+	 */
+#if __has_builtin(__builtin_preserve_type_info) && __clang_major__ >= 15
 	struct core_reloc_type_based_output *out = (void *)&data.out;
 
 	out->struct_exists = bpf_core_type_exists(struct a_struct);
@@ -92,6 +110,18 @@ int test_core_type_based(void *ctx)
 	out->typedef_func_proto_exists = bpf_core_type_exists(func_proto_typedef);
 	out->typedef_arr_exists = bpf_core_type_exists(arr_typedef);
 
+	out->struct_matches = bpf_core_type_matches(struct a_struct);
+	out->union_matches = bpf_core_type_matches(union a_union);
+	out->enum_matches = bpf_core_type_matches(enum an_enum);
+	out->typedef_named_struct_matches = bpf_core_type_matches(named_struct_typedef);
+	out->typedef_anon_struct_matches = bpf_core_type_matches(anon_struct_typedef);
+	out->typedef_struct_ptr_matches = bpf_core_type_matches(struct_ptr_typedef);
+	out->typedef_int_matches = bpf_core_type_matches(int_typedef);
+	out->typedef_enum_matches = bpf_core_type_matches(enum_typedef);
+	out->typedef_void_ptr_matches = bpf_core_type_matches(void_ptr_typedef);
+	out->typedef_func_proto_matches = bpf_core_type_matches(func_proto_typedef);
+	out->typedef_arr_matches = bpf_core_type_matches(arr_typedef);
+
 	out->struct_sz = bpf_core_type_size(struct a_struct);
 	out->union_sz = bpf_core_type_size(union a_union);
 	out->enum_sz = bpf_core_type_size(enum an_enum);
Signature · 1 line
2.30.2
#15 lore raw

PATCH v2 8/9 selftests/bpf: Add test checking more characteristics

In reply to Daniel Müller

This change adds another type-based self-test that specifically aims to test some more characteristics of the TYPE_MATCH logic. Specifically, it covers a few more potential differences between types, such as different orders, enum variant values, and integer signedness.

---
 .../selftests/bpf/prog_tests/core_reloc.c     | 37 ++++++++++++++
 .../progs/btf__core_reloc_type_based___diff.c |  3 ++
 .../selftests/bpf/progs/core_reloc_types.h    | 51 +++++++++++++++++++
 3 files changed, 91 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/progs/btf__core_reloc_type_based___diff.c
tools/testing/selftests/bpf/prog_tests/core_reloc.c +37 -0
index 328dd7..eb47bf 100644
@@ -792,6 +792,43 @@ static const struct core_reloc_test_case test_cases[] = {
 	TYPE_BASED_CASE(type_based___all_missing, {
 		/* all zeros */
 	}),
+	TYPE_BASED_CASE(type_based___diff, {
+		.struct_exists = 1,
+		.union_exists = 1,
+		.enum_exists = 1,
+		.typedef_named_struct_exists = 1,
+		.typedef_anon_struct_exists = 1,
+		.typedef_struct_ptr_exists = 1,
+		.typedef_int_exists = 1,
+		.typedef_enum_exists = 1,
+		.typedef_void_ptr_exists = 1,
+		.typedef_func_proto_exists = 1,
+		.typedef_arr_exists = 1,
+
+		.struct_matches = 1,
+		.union_matches = 1,
+		.enum_matches = 1,
+		.typedef_named_struct_matches = 1,
+		.typedef_anon_struct_matches = 1,
+		.typedef_struct_ptr_matches = 1,
+		.typedef_int_matches = 0,
+		.typedef_enum_matches = 1,
+		.typedef_void_ptr_matches = 1,
+		.typedef_func_proto_matches = 0,
+		.typedef_arr_matches = 0,
+
+		.struct_sz = sizeof(struct a_struct___diff),
+		.union_sz = sizeof(union a_union___diff),
+		.enum_sz = sizeof(enum an_enum___diff),
+		.typedef_named_struct_sz = sizeof(named_struct_typedef___diff),
+		.typedef_anon_struct_sz = sizeof(anon_struct_typedef___diff),
+		.typedef_struct_ptr_sz = sizeof(struct_ptr_typedef___diff),
+		.typedef_int_sz = sizeof(int_typedef___diff),
+		.typedef_enum_sz = sizeof(enum_typedef___diff),
+		.typedef_void_ptr_sz = sizeof(void_ptr_typedef___diff),
+		.typedef_func_proto_sz = sizeof(func_proto_typedef___diff),
+		.typedef_arr_sz = sizeof(arr_typedef___diff),
+	}),
 	TYPE_BASED_CASE(type_based___diff_sz, {
 		.struct_exists = 1,
 		.union_exists = 1,
tools/testing/selftests/bpf/progs/btf__core_reloc_type_based___diff.c new file +3 -0
new file mode 100644
index 0000000..57ae2c
@@ -0,0 +1,3 @@
+#include "core_reloc_types.h"
+
+void f(struct core_reloc_type_based___diff x) {}
tools/testing/selftests/bpf/progs/core_reloc_types.h +51 -0

57 lines not on this page · raw message

Signature · 1 line
2.30.2
#16 lore raw

PATCH v2 9/9 selftests/bpf: Add nested type to type based tests

In reply to Daniel Müller

This change extends the type based tests with another struct type (in addition to a_struct) to check relocations against: a_complex_struct. This type is nested more deeply to provide additional coverage of certain paths in the type match logic.

---
 .../selftests/bpf/prog_tests/core_reloc.c     |  4 ++
 .../selftests/bpf/progs/core_reloc_types.h    | 62 +++++++++++++------
 .../bpf/progs/test_core_reloc_type_based.c    | 12 ++++
 3 files changed, 58 insertions(+), 20 deletions(-)
tools/testing/selftests/bpf/prog_tests/core_reloc.c +4 -0
index eb47bf..8882c9c 100644
@@ -754,6 +754,7 @@ static const struct core_reloc_test_case test_cases[] = {
 	/* validate type existence, match, and size relocations */
 	TYPE_BASED_CASE(type_based, {
 		.struct_exists = 1,
+		.complex_struct_exists = 1,
 		.union_exists = 1,
 		.enum_exists = 1,
 		.typedef_named_struct_exists = 1,
@@ -766,6 +767,7 @@ static const struct core_reloc_test_case test_cases[] = {
 		.typedef_arr_exists = 1,
 
 		.struct_matches = 1,
+		.complex_struct_matches = 1,
 		.union_matches = 1,
 		.enum_matches = 1,
 		.typedef_named_struct_matches = 1,
@@ -794,6 +796,7 @@ static const struct core_reloc_test_case test_cases[] = {
 	}),
 	TYPE_BASED_CASE(type_based___diff, {
 		.struct_exists = 1,
+		.complex_struct_exists = 1,
 		.union_exists = 1,
 		.enum_exists = 1,
 		.typedef_named_struct_exists = 1,
@@ -806,6 +809,7 @@ static const struct core_reloc_test_case test_cases[] = {
 		.typedef_arr_exists = 1,
 
 		.struct_matches = 1,
+		.complex_struct_matches = 1,
 		.union_matches = 1,
 		.enum_matches = 1,
 		.typedef_named_struct_matches = 1,
tools/testing/selftests/bpf/progs/core_reloc_types.h +42 -20

98 lines not on this page · raw message

tools/testing/selftests/bpf/progs/test_core_reloc_type_based.c +12 -0
index 325ead..d95bc08 100644
@@ -19,6 +19,14 @@ struct a_struct {
 	int x;
 };
 
+struct a_complex_struct {
+	union {
+		struct a_struct *a;
+		void *b;
+	} x;
+	volatile long y;
+};
+
 union a_union {
 	int y;
 	int z;
@@ -50,6 +58,7 @@ typedef char arr_typedef[20];
 
 struct core_reloc_type_based_output {
 	bool struct_exists;
+	bool complex_struct_exists;
 	bool union_exists;
 	bool enum_exists;
 	bool typedef_named_struct_exists;
@@ -62,6 +71,7 @@ struct core_reloc_type_based_output {
 	bool typedef_arr_exists;
 
 	bool struct_matches;
+	bool complex_struct_matches;
 	bool union_matches;
 	bool enum_matches;
 	bool typedef_named_struct_matches;
@@ -99,6 +109,7 @@ int test_core_type_based(void *ctx)
 	struct core_reloc_type_based_output *out = (void *)&data.out;
 
 	out->struct_exists = bpf_core_type_exists(struct a_struct);
+	out->complex_struct_exists = bpf_core_type_exists(struct a_complex_struct);
 	out->union_exists = bpf_core_type_exists(union a_union);
 	out->enum_exists = bpf_core_type_exists(enum an_enum);
 	out->typedef_named_struct_exists = bpf_core_type_exists(named_struct_typedef);
@@ -111,6 +122,7 @@ int test_core_type_based(void *ctx)
 	out->typedef_arr_exists = bpf_core_type_exists(arr_typedef);
 
 	out->struct_matches = bpf_core_type_matches(struct a_struct);
+	out->complex_struct_matches = bpf_core_type_matches(struct a_complex_struct);
 	out->union_matches = bpf_core_type_matches(union a_union);
 	out->enum_matches = bpf_core_type_matches(enum an_enum);
 	out->typedef_named_struct_matches = bpf_core_type_matches(named_struct_typedef);
Signature · 1 line
2.30.2
#17 lore raw

Re: [PATCH bpf-next v2 9/9] selftests/bpf: Add nested type to type based tests

Quoted 8 lines · Daniel Müller

This change extends the type based tests with another struct type (in addition to a_struct) to check relocations against: a_complex_struct. This type is nested more deeply to provide additional coverage of certain paths in the type match logic.

---

I'd like us to have a TYPE_MATCHES test against struct task_struct, something like below:

struct mm_struct___wrong {
    int abc_whatever_should_not_exist;
};
struct task_struct____local {
    int pid;
    struct mm_struct___wrong *mm;
};

and then use struct task_struct____local with bpf_core_type_matches() and check that it succeeds. This will show that TYPE_MATCHES can be used practically. Can you please add it to progs/test_core_reloc_kernel.c?

Quoted 5 lines
 .../selftests/bpf/prog_tests/core_reloc.c     |  4 ++
 .../selftests/bpf/progs/core_reloc_types.h    | 62 +++++++++++++------
 .../bpf/progs/test_core_reloc_type_based.c    | 12 ++++
 3 files changed, 58 insertions(+), 20 deletions(-)

[...]

#18 lore raw

Re: [PATCH bpf-next v2 9/9] selftests/bpf: Add nested type to type based tests

Quoted 23 lines · Andrii Nakryiko
Quoted 8 lines · Daniel Müller

This change extends the type based tests with another struct type (in addition to a_struct) to check relocations against: a_complex_struct. This type is nested more deeply to provide additional coverage of certain paths in the type match logic.

---

I'd like us to have a TYPE_MATCHES test against struct task_struct, something like below:

struct mm_struct___wrong {
    int abc_whatever_should_not_exist;
};
struct task_struct____local {
    int pid;
    struct mm_struct___wrong *mm;
};
Quoted 4 lines

and then use struct task_struct____local with bpf_core_type_matches() and check that it succeeds. This will show that TYPE_MATCHES can be used practically. Can you please add it to progs/test_core_reloc_kernel.c?

Thanks for the suggestion! I will include that test.

[...]

Daniel