Skip to contentSkip to conversation

resolve_btfids: Support for BTF modifications

PATCH Superseded Ihor Solodrai 6 participants 31 messages Nov 27, 2025 – Dec 16, 2025

Subsystems (4)

v2 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Ihor Solodrai

author reviewer
#0 lore raw

v2 cover resolve_btfids: Support for BTF modifications

This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.

Patches #1-#3 in the series are non-functional refactoring in resolve_btfids. The last patch (#4) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. Implementation changes are described in detail in the patch description.

[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/

---

v1->v2:
- gen-btf.sh and other shell script fixes (Donglin)
- update selftests build (Donglin)
- generate .BTF.base only when KBUILD_EXTMOD is set (Alan)
- proper endianness handling for cross-compilation
- change elf_begin mode from ELF_C_RDWR_MMAP to ELF_C_READ_MMAP_PRIVATE
- remove compressed_section_fix()
- nit NULL check in patch #3 (suggested by AI)

v1: https://lore.kernel.org/bpf/[email protected]/

Ihor Solodrai (4):
resolve_btfids: rename object btf field to btf_path
resolve_btfids: factor out load_btf()
resolve_btfids: introduce enum btf_id_kind
resolve_btfids: change in-place update with raw binary output

 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++
 scripts/link-vmlinux.sh              |  42 +---
 tools/bpf/resolve_btfids/main.c      | 331 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 395 insertions(+), 156 deletions(-)
 create mode 100755 scripts/gen-btf.sh
Signature · 1 line
2.52.0
author reviewer
#1 lore raw

PATCH v2 1/4 resolve_btfids: rename object btf field to btf_path

Rename the member of struct object holding the path to BTF data if provided via --btf arg. btf_path is less ambiguous.

---
 tools/bpf/resolve_btfids/main.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
tools/bpf/resolve_btfids/main.c +5 -5
index d47191c6e55e..164f0c941f04 100644
@@ -113,7 +113,7 @@ struct btf_id {
 
 struct object {
 	const char *path;
-	const char *btf;
+	const char *btf_path;
 	const char *base_btf_path;
 
 	struct {
@@ -550,11 +550,11 @@ static int symbols_resolve(struct object *obj)
 		}
 	}
 
-	btf = btf__parse_split(obj->btf ?: obj->path, base_btf);
+	btf = btf__parse_split(obj->btf_path ?: obj->path, base_btf);
 	err = libbpf_get_error(btf);
 	if (err) {
 		pr_err("FAILED: load BTF from %s: %s\n",
-			obj->btf ?: obj->path, strerror(-err));
+			obj->btf_path ?: obj->path, strerror(-err));
 		goto out;
 	}
 
@@ -790,8 +790,8 @@ int main(int argc, const char **argv)
 	struct option btfid_options[] = {
 		OPT_INCR('v', "verbose", &verbose,
 			 "be more verbose (show errors, etc)"),
-		OPT_STRING(0, "btf", &obj.btf, "BTF data",
-			   "BTF data"),
+		OPT_STRING(0, "btf", &obj.btf_path, "file",
+			   "path to a file with input BTF data"),
 		OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
 			   "path of file providing base BTF"),
 		OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
Signature · 1 line
2.52.0
author reviewer
#2 lore raw

PATCH v2 2/4 resolve_btfids: factor out load_btf()

In reply to Ihor Solodrai

Increase the lifetime of parsed BTF in resolve_btfids by factoring load_btf() routine out of symbols_resolve() and storing the base_btf and btf pointers in the struct object.

---
 tools/bpf/resolve_btfids/main.c | 47 ++++++++++++++++++++++++---------
 1 file changed, 34 insertions(+), 13 deletions(-)
tools/bpf/resolve_btfids/main.c +34 -13
index 164f0c941f04..b4caae1170dd 100644
@@ -116,6 +116,9 @@ struct object {
 	const char *btf_path;
 	const char *base_btf_path;
 
+	struct btf *btf;
+	struct btf *base_btf;
+
 	struct {
 		int		 fd;
 		Elf		*elf;
@@ -529,16 +532,10 @@ static int symbols_collect(struct object *obj)
 	return 0;
 }
 
-static int symbols_resolve(struct object *obj)
+static int load_btf(struct object *obj)
 {
-	int nr_typedefs = obj->nr_typedefs;
-	int nr_structs  = obj->nr_structs;
-	int nr_unions   = obj->nr_unions;
-	int nr_funcs    = obj->nr_funcs;
-	struct btf *base_btf = NULL;
-	int err, type_id;
-	struct btf *btf;
-	__u32 nr_types;
+	struct btf *base_btf = NULL, *btf = NULL;
+	int err;
 
 	if (obj->base_btf_path) {
 		base_btf = btf__parse(obj->base_btf_path, NULL);
@@ -546,7 +543,7 @@ static int symbols_resolve(struct object *obj)
 		if (err) {
 			pr_err("FAILED: load base BTF from %s: %s\n",
 			       obj->base_btf_path, strerror(-err));
-			return -1;
+			goto out_err;
 		}
 	}
 
@@ -555,9 +552,30 @@ static int symbols_resolve(struct object *obj)
 	if (err) {
 		pr_err("FAILED: load BTF from %s: %s\n",
 			obj->btf_path ?: obj->path, strerror(-err));
-		goto out;
+		goto out_err;
 	}
 
+	obj->base_btf = base_btf;
+	obj->btf = btf;
+
+	return 0;
+
+out_err:
+	btf__free(base_btf);
+	btf__free(btf);
+	return err;
+}
+
+static int symbols_resolve(struct object *obj)
+{
+	int nr_typedefs = obj->nr_typedefs;
+	int nr_structs  = obj->nr_structs;
+	int nr_unions   = obj->nr_unions;
+	int nr_funcs    = obj->nr_funcs;
+	struct btf *btf = obj->btf;
+	int err, type_id;
+	__u32 nr_types;
+
 	err = -1;
 	nr_types = btf__type_cnt(btf);
 
@@ -615,8 +633,6 @@ static int symbols_resolve(struct object *obj)
 
 	err = 0;
 out:
-	btf__free(base_btf);
-	btf__free(btf);
 	return err;
 }
 
@@ -824,6 +840,9 @@ int main(int argc, const char **argv)
 	if (symbols_collect(&obj))
 		goto out;
 
+	if (load_btf(&obj))
+		goto out;
+
 	if (symbols_resolve(&obj))
 		goto out;
 
@@ -833,6 +852,8 @@ int main(int argc, const char **argv)
 	if (!(fatal_warnings && warnings))
 		err = 0;
 out:
+	btf__free(obj.base_btf);
+	btf__free(obj.btf);
 	if (obj.efile.elf) {
 		elf_end(obj.efile.elf);
 		close(obj.efile.fd);
Signature · 1 line
2.52.0
author reviewer
#3 lore raw

PATCH v2 3/4 resolve_btfids: introduce enum btf_id_kind

In reply to Ihor Solodrai

Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.

---
 tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++-----------
 1 file changed, 42 insertions(+), 20 deletions(-)
tools/bpf/resolve_btfids/main.c +42 -20
index b4caae1170dd..c60d303ca6ed 100644
@@ -98,6 +98,13 @@
 # error "Unknown machine endianness!"
 #endif
 
+enum btf_id_kind {
+	BTF_ID_KIND_NONE,
+	BTF_ID_KIND_SYM,
+	BTF_ID_KIND_SET,
+	BTF_ID_KIND_SET8
+};
+
 struct btf_id {
 	struct rb_node	 rb_node;
 	char		*name;
@@ -105,9 +112,8 @@ struct btf_id {
 		int	 id;
 		int	 cnt;
 	};
-	int		 addr_cnt;
-	bool		 is_set;
-	bool		 is_set8;
+	enum btf_id_kind kind:8;
+	int		 addr_cnt:8;
 	Elf64_Addr	 addr[ADDR_CNT];
 };
 
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
 	return id;
 }
 
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
 	/*
 	 * __BTF_ID__set__name
 	 * name =    ^
 	 * id   =         ^
 	 */
-	char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+	int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+	char *id = name + prefixlen - 1;
 	int len = strlen(name);
+	struct btf_id *btf_id;
 
 	if (id >= name + len) {
 		pr_err("FAILED to parse set name: %s\n", name);
 		return NULL;
 	}
 
-	return btf_id__add(&obj->sets, id, true);
+	btf_id = btf_id__add(&obj->sets, id, true);
+	if (btf_id)
+		btf_id->kind = kind;
+
+	return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+	struct btf_id *btf_id;
 	char *id;
 
 	id = get_id(name + size);
@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 		return NULL;
 	}
 
-	return btf_id__add(root, id, false);
+	btf_id = btf_id__add(root, id, false);
+	if (btf_id)
+		btf_id->kind = BTF_ID_KIND_SYM;
+
+	return btf_id;
 }
 
 /* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
@@ -491,28 +508,24 @@ static int symbols_collect(struct object *obj)
 			id = add_symbol(&obj->funcs, prefix, sizeof(BTF_FUNC) - 1);
 		/* set8 */
 		} else if (!strncmp(prefix, BTF_SET8, sizeof(BTF_SET8) - 1)) {
-			id = add_set(obj, prefix, true);
+			id = add_set(obj, prefix, BTF_ID_KIND_SET8);
 			/*
 			 * SET8 objects store list's count, which is encoded
 			 * in symbol's size, together with 'cnt' field hence
 			 * that - 1.
 			 */
-			if (id) {
+			if (id)
 				id->cnt = sym.st_size / sizeof(uint64_t) - 1;
-				id->is_set8 = true;
-			}
 		/* set */
 		} else if (!strncmp(prefix, BTF_SET, sizeof(BTF_SET) - 1)) {
-			id = add_set(obj, prefix, false);
+			id = add_set(obj, prefix, BTF_ID_KIND_SET);
 			/*
 			 * SET objects store list's count, which is encoded
 			 * in symbol's size, together with 'cnt' field hence
 			 * that - 1.
 			 */
-			if (id) {
+			if (id)
 				id->cnt = sym.st_size / sizeof(int) - 1;
-				id->is_set = true;
-			}
 		} else {
 			pr_err("FAILED unsupported prefix %s\n", prefix);
 			return -1;
@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
 	int i;
 
 	/* For set, set8, id->id may be 0 */
-	if (!id->id && !id->is_set && !id->is_set8) {
+	if (!id->id && id->kind == BTF_ID_KIND_SYM) {
 		pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
 		warnings++;
 	}
@@ -696,6 +709,7 @@ static int sets_patch(struct object *obj)
 {
 	Elf_Data *data = obj->efile.idlist;
 	struct rb_node *next;
+	int cnt;
 
 	next = rb_first(&obj->sets);
 	while (next) {
@@ -715,11 +729,15 @@ static int sets_patch(struct object *obj)
 			return -1;
 		}
 
-		if (id->is_set) {
+		switch (id->kind) {
+		case BTF_ID_KIND_SET:
 			set = data->d_buf + off;
+			cnt = set->cnt;
 			qsort(set->ids, set->cnt, sizeof(set->ids[0]), cmp_id);
-		} else {
+			break;
+		case BTF_ID_KIND_SET8:
 			set8 = data->d_buf + off;
+			cnt = set8->cnt;
 			/*
 			 * Make sure id is at the beginning of the pairs
 			 * struct, otherwise the below qsort would not work.
@@ -744,10 +762,14 @@ static int sets_patch(struct object *obj)
 						bswap_32(set8->pairs[i].flags);
 				}
 			}
+			break;
+		case BTF_ID_KIND_SYM:
+		default:
+			pr_err("Unexpected btf_id_kind %d for set '%s'\n", id->kind, id->name);
+			return -1;
 		}
 
-		pr_debug("sorting  addr %5lu: cnt %6d [%s]\n",
-			 off, id->is_set ? set->cnt : set8->cnt, id->name);
+		pr_debug("sorting  addr %5lu: cnt %6d [%s]\n", off, cnt, id->name);
 
 		next = rb_next(next);
 	}
Signature · 1 line
2.52.0
#4 lore raw

Re: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind

Quoted 8 lines · Ihor Solodrai

Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.

---
 tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++-----------
 1 file changed, 42 insertions(+), 20 deletions(-)

[...]

Quoted 35 lines
tools/bpf/resolve_btfids/main.c +10 -3
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
        /*
         * __BTF_ID__set__name
         * name =    ^
         * id   =         ^
         */
-       char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+       int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+       char *id = name + prefixlen - 1;
        int len = strlen(name);
+       struct btf_id *btf_id;
 
        if (id >= name + len) {
                pr_err("FAILED to parse set name: %s\n", name);
                return NULL;
        }
 
-       return btf_id__add(&obj->sets, id, true);
+       btf_id = btf_id__add(&obj->sets, id, true);
+       if (btf_id)
+               btf_id->kind = kind;
+
+       return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+       struct btf_id *btf_id;
        char *id;
 
        id = get_id(name + size);

tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306

Quoted 7 lines
tools/bpf/resolve_btfids/main.c +3 -1
@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
                return NULL;
        }
 
-       return btf_id__add(root, id, false);
+       btf_id = btf_id__add(root, id, false);
+       if (btf_id)
+               btf_id->kind = BTF_ID_KIND_SYM;

seeing this pattern repeated, wouldn't it make sense to just pass this kind to btf_id__add() and set it there?

Quoted 4 lines
tools/bpf/resolve_btfids/main.c +2 -0
+
+       return btf_id;
 }
 

[...]

tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659

Quoted 6 lines
tools/bpf/resolve_btfids/main.c +1 -1
@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
        int i;
 
        /* For set, set8, id->id may be 0 */
-       if (!id->id && !id->is_set && !id->is_set8) {
+       if (!id->id && id->kind == BTF_ID_KIND_SYM) {

nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?

Quoted 3 lines
tools/bpf/resolve_btfids/main.c +0 -0
                pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
                warnings++;
        }

[...]

author reviewer
#5 lore raw

Re: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind

Quoted 50 lines · Andrii Nakryiko
Quoted 8 lines · Ihor Solodrai

Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.

---
 tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++-----------
 1 file changed, 42 insertions(+), 20 deletions(-)

[...]

Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10 -3
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
        /*
         * __BTF_ID__set__name
         * name =    ^
         * id   =         ^
         */
-       char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+       int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+       char *id = name + prefixlen - 1;
        int len = strlen(name);
+       struct btf_id *btf_id;
 
        if (id >= name + len) {
                pr_err("FAILED to parse set name: %s\n", name);
                return NULL;
        }
 
-       return btf_id__add(&obj->sets, id, true);
+       btf_id = btf_id__add(&obj->sets, id, true);
+       if (btf_id)
+               btf_id->kind = kind;
+
+       return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+       struct btf_id *btf_id;
        char *id;
 
        id = get_id(name + size);
@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
                return NULL;
        }
 
Quoted 7 lines

tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306

Quoted 4 lines
tools/bpf/resolve_btfids/main.c +3 -1
-       return btf_id__add(root, id, false);
+       btf_id = btf_id__add(root, id, false);
+       if (btf_id)
+               btf_id->kind = BTF_ID_KIND_SYM;

seeing this pattern repeated, wouldn't it make sense to just pass this kind to btf_id__add() and set it there?

I like the idea, because we could get rid the "unique" flag then.

But the btf_id__add() does not necessarily create a new struct, and so if we pass the kind in, what do we do with existing objects? Overwrite the kind? If not, do we check for a mismatch?

Quoted 11 lines
Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2 -0
+
+       return btf_id;
 }
 

[...]

Quoted 3 lines
(quoted diff) +0 -0
@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
        int i;
 
Quoted 8 lines

tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659

Quoted 3 lines
tools/bpf/resolve_btfids/main.c +1 -1
        /* For set, set8, id->id may be 0 */
-       if (!id->id && !id->is_set && !id->is_set8) {
+       if (!id->id && id->kind == BTF_ID_KIND_SYM) {

nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?

ok

Quoted 6 lines
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0 -0
                pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
                warnings++;
        }

[...]

#6 lore raw

Re: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind

Quoted 60 lines · Ihor Solodrai
Quoted 50 lines · Andrii Nakryiko
Quoted 8 lines · Ihor Solodrai

Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.

---
 tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++-----------
 1 file changed, 42 insertions(+), 20 deletions(-)

[...]

Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10 -3
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
        /*
         * __BTF_ID__set__name
         * name =    ^
         * id   =         ^
         */
-       char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+       int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+       char *id = name + prefixlen - 1;
        int len = strlen(name);
+       struct btf_id *btf_id;
 
        if (id >= name + len) {
                pr_err("FAILED to parse set name: %s\n", name);
                return NULL;
        }
 
-       return btf_id__add(&obj->sets, id, true);
+       btf_id = btf_id__add(&obj->sets, id, true);
+       if (btf_id)
+               btf_id->kind = kind;
+
+       return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+       struct btf_id *btf_id;
        char *id;
 
        id = get_id(name + size);
@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
                return NULL;
        }
 
Quoted 7 lines

tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306

Quoted 4 lines
tools/bpf/resolve_btfids/main.c +3 -1
-       return btf_id__add(root, id, false);
+       btf_id = btf_id__add(root, id, false);
+       if (btf_id)
+               btf_id->kind = BTF_ID_KIND_SYM;

seeing this pattern repeated, wouldn't it make sense to just pass this kind to btf_id__add() and set it there?

Quoted 6 lines

I like the idea, because we could get rid the "unique" flag then.

But the btf_id__add() does not necessarily create a new struct, and so if we pass the kind in, what do we do with existing objects? Overwrite the kind? If not, do we check for a mismatch?

no idea, don't know code well enough, but your newly added code seems to overwrite the kind always, no?

Quoted 29 lines
Quoted 11 lines
Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2 -0
+
+       return btf_id;
 }
 

[...]

Quoted 3 lines
(quoted diff) +0 -0
@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
        int i;
 
Quoted 8 lines

tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659

Quoted 3 lines
tools/bpf/resolve_btfids/main.c +1 -1
        /* For set, set8, id->id may be 0 */
-       if (!id->id && !id->is_set && !id->is_set8) {
+       if (!id->id && id->kind == BTF_ID_KIND_SYM) {

nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?

ok

Quoted 6 lines
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0 -0
                pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
                warnings++;
        }

[...]

author reviewer
#7 lore raw

Re: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind

Quoted 63 lines · Andrii Nakryiko
Quoted 62 lines · Ihor Solodrai
Quoted 50 lines · Andrii Nakryiko
Quoted 8 lines · Ihor Solodrai

Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.

---
 tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++-----------
 1 file changed, 42 insertions(+), 20 deletions(-)

[...]

Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10 -3
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
        /*
         * __BTF_ID__set__name
         * name =    ^
         * id   =         ^
         */
-       char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+       int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+       char *id = name + prefixlen - 1;
        int len = strlen(name);
+       struct btf_id *btf_id;
 
        if (id >= name + len) {
                pr_err("FAILED to parse set name: %s\n", name);
                return NULL;
        }
 
-       return btf_id__add(&obj->sets, id, true);
+       btf_id = btf_id__add(&obj->sets, id, true);
+       if (btf_id)
+               btf_id->kind = kind;
+
+       return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+       struct btf_id *btf_id;
        char *id;
 
        id = get_id(name + size);
@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
                return NULL;
        }
 
Quoted 7 lines

tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306

Quoted 4 lines
tools/bpf/resolve_btfids/main.c +3 -1
-       return btf_id__add(root, id, false);
+       btf_id = btf_id__add(root, id, false);
+       if (btf_id)
+               btf_id->kind = BTF_ID_KIND_SYM;

seeing this pattern repeated, wouldn't it make sense to just pass this kind to btf_id__add() and set it there?

I like the idea, because we could get rid the "unique" flag then.

Quoted 7 lines
Quoted 4 lines

But the btf_id__add() does not necessarily create a new struct, and so if we pass the kind in, what do we do with existing objects? Overwrite the kind? If not, do we check for a mismatch?

no idea, don't know code well enough, but your newly added code seems to overwrite the kind always, no?

You're right, I am overwriting here, haven't realized that.

I think I'll go with a mismatch check inside btf_id__add() in v3, that would indicate a bug.

Quoted 30 lines
Quoted 30 lines
Quoted 12 lines
Quoted 6 lines
tools/bpf/resolve_btfids/main.c +2 -0
+
+       return btf_id;
 }
 

[...]

Quoted 3 lines
(quoted diff) +0 -0
@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
        int i;
 
Quoted 8 lines

tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659

Quoted 3 lines
tools/bpf/resolve_btfids/main.c +1 -1
        /* For set, set8, id->id may be 0 */
-       if (!id->id && !id->is_set && !id->is_set8) {
+       if (!id->id && id->kind == BTF_ID_KIND_SYM) {

nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?

ok

Quoted 6 lines
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0 -0
                pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
                warnings++;
        }

[...]

maintainer
#8 lore raw

Re: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind

In reply to Ihor Solodrai
Quoted 5 lines · Ihor Solodrai

Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.

---

Just a few nits, looks good to me overall.

Quoted 22 lines
 tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++-----------
 1 file changed, 42 insertions(+), 20 deletions(-)
tools/bpf/resolve_btfids/main.c +7 -0
index b4caae1170dd..c60d303ca6ed 100644
@@ -98,6 +98,13 @@
 # error "Unknown machine endianness!"
 #endif
 
+enum btf_id_kind {
+	BTF_ID_KIND_NONE,
+	BTF_ID_KIND_SYM,
+	BTF_ID_KIND_SET,
+	BTF_ID_KIND_SET8
+};
+
 struct btf_id {
 	struct rb_node	 rb_node;
 	char		*name;

tools/bpf/resolve_btfids/main.c struct btf_id { · line 116

Quoted 8 lines
tools/bpf/resolve_btfids/main.c +2 -3
@@ -105,9 +112,8 @@ struct btf_id {
 		int	 id;
 		int	 cnt;
 	};
-	int		 addr_cnt;
-	bool		 is_set;
-	bool		 is_set8;
+	enum btf_id_kind kind:8;
+	int		 addr_cnt:8;
                                 ^^

Nit: these bitfields are not really necessary:

  $ pahole -C btf_id ./tools/bpf/resolve_btfids/resolve_btfids
  struct btf_id {
          struct rb_node             rb_node __attribute__((__aligned__(8))); /*     0    24 */
          char *                     name;                 /*    24     8 */
          union {
                  int                id;                   /*    32     4 */
                  int                cnt;                  /*    32     4 */
          };                                               /*    32     4 */
          enum btf_id_kind           kind:8;               /*    36: 0  4 */
          int                        addr_cnt:8;           /*    36: 8  4 */
          /* XXX 16 bits hole, try to pack */

          Elf64_Addr                 addr[100];            /*    40   800 */

          /* size: 840, cachelines: 14, members: 6 */
          /* sum members: 836 */
          /* sum bitfield members: 16 bits, bit holes: 1, sum bit holes: 16 bits */
          /* forced alignments: 1 */
          /* last cacheline: 8 bytes */

} __attribute__((__aligned__(8)));

Quoted 7 lines
tools/bpf/resolve_btfids/main.c +0 -0
 	Elf64_Addr	 addr[ADDR_CNT];
 };
 
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
 	return id;
 }
 

tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 276

Quoted 10 lines
tools/bpf/resolve_btfids/main.c +2 -2
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
 	/*
 	 * __BTF_ID__set__name
 	 * name =    ^
 	 * id   =         ^
 	 */
-	char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+	int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");

Nit: Should prefixlen be an input parameter as well? (Or adjust the 'name' at the callsite?)
Otherwise the parameter is still a boolean logically.

Quoted 24 lines
tools/bpf/resolve_btfids/main.c +8 -1
+	char *id = name + prefixlen - 1;
 	int len = strlen(name);
+	struct btf_id *btf_id;
 
 	if (id >= name + len) {
 		pr_err("FAILED to parse set name: %s\n", name);
 		return NULL;
 	}
 
-	return btf_id__add(&obj->sets, id, true);
+	btf_id = btf_id__add(&obj->sets, id, true);
+	if (btf_id)
+		btf_id->kind = kind;
+
+	return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+	struct btf_id *btf_id;
 	char *id;
 
 	id = get_id(name + size);

tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 308

Quoted 9 lines
tools/bpf/resolve_btfids/main.c +5 -1
@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 		return NULL;
 	}
 
-	return btf_id__add(root, id, false);
+	btf_id = btf_id__add(root, id, false);
+	if (btf_id)
+		btf_id->kind = BTF_ID_KIND_SYM;
+
+	return btf_id;

Agree with Andrii regarding 'kind' being a btf_id__add() parameter.

Quoted 72 lines
tools/bpf/resolve_btfids/main.c +12 -11
 }
 
 /* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
@@ -491,28 +508,24 @@ static int symbols_collect(struct object *obj)
 			id = add_symbol(&obj->funcs, prefix, sizeof(BTF_FUNC) - 1);
 		/* set8 */
 		} else if (!strncmp(prefix, BTF_SET8, sizeof(BTF_SET8) - 1)) {
-			id = add_set(obj, prefix, true);
+			id = add_set(obj, prefix, BTF_ID_KIND_SET8);
 			/*
 			 * SET8 objects store list's count, which is encoded
 			 * in symbol's size, together with 'cnt' field hence
 			 * that - 1.
 			 */
-			if (id) {
+			if (id)
 				id->cnt = sym.st_size / sizeof(uint64_t) - 1;
-				id->is_set8 = true;
-			}
 		/* set */
 		} else if (!strncmp(prefix, BTF_SET, sizeof(BTF_SET) - 1)) {
-			id = add_set(obj, prefix, false);
+			id = add_set(obj, prefix, BTF_ID_KIND_SET);
 			/*
 			 * SET objects store list's count, which is encoded
 			 * in symbol's size, together with 'cnt' field hence
 			 * that - 1.
 			 */
-			if (id) {
+			if (id)
 				id->cnt = sym.st_size / sizeof(int) - 1;
-				id->is_set = true;
-			}
 		} else {
 			pr_err("FAILED unsupported prefix %s\n", prefix);
 			return -1;
@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
 	int i;
 
 	/* For set, set8, id->id may be 0 */
-	if (!id->id && !id->is_set && !id->is_set8) {
+	if (!id->id && id->kind == BTF_ID_KIND_SYM) {
 		pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
 		warnings++;
 	}
@@ -696,6 +709,7 @@ static int sets_patch(struct object *obj)
 {
 	Elf_Data *data = obj->efile.idlist;
 	struct rb_node *next;
+	int cnt;
 
 	next = rb_first(&obj->sets);
 	while (next) {
@@ -715,11 +729,15 @@ static int sets_patch(struct object *obj)
 			return -1;
 		}
 
-		if (id->is_set) {
+		switch (id->kind) {
+		case BTF_ID_KIND_SET:
 			set = data->d_buf + off;
+			cnt = set->cnt;
 			qsort(set->ids, set->cnt, sizeof(set->ids[0]), cmp_id);
-		} else {
+			break;
+		case BTF_ID_KIND_SET8:
 			set8 = data->d_buf + off;
+			cnt = set8->cnt;
 			/*
 			 * Make sure id is at the beginning of the pairs
 			 * struct, otherwise the below qsort would not work.

tools/bpf/resolve_btfids/main.c static int sets_patch(struct object *obj) · line 767

Quoted 6 lines
tools/bpf/resolve_btfids/main.c +3 -0
@@ -744,10 +762,14 @@ static int sets_patch(struct object *obj)
 						bswap_32(set8->pairs[i].flags);
 				}
 			}
+			break;
+		case BTF_ID_KIND_SYM:
+		default:

Nit: just default, no need for case BTF_ID_KIND_SYM:?

Quoted 10 lines
tools/bpf/resolve_btfids/main.c +3 -2
+			pr_err("Unexpected btf_id_kind %d for set '%s'\n", id->kind, id->name);
+			return -1;
 		}
 
-		pr_debug("sorting  addr %5lu: cnt %6d [%s]\n",
-			 off, id->is_set ? set->cnt : set8->cnt, id->name);
+		pr_debug("sorting  addr %5lu: cnt %6d [%s]\n", off, cnt, id->name);
 
 		next = rb_next(next);
 	}
author reviewer
#9 lore raw

PATCH v2 4/4 resolve_btfids: change in-place update with raw binary output

In reply to Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).

Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/

---
 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++++++++
 scripts/link-vmlinux.sh              |  42 +-----
 tools/bpf/resolve_btfids/main.c      | 218 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 317 insertions(+), 121 deletions(-)
 create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:	net/sched/act_bpf.c
 F:	net/sched/cls_bpf.c
 F:	samples/bpf/
 F:	scripts/bpf_doc.py
+F:	scripts/gen-btf.sh
 F:	scripts/Makefile.btf
 F:	scripts/pahole-version.sh
 F:	tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..3862fdfa1267 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko = 							\
 	if [ ! -f $(objtree)/vmlinux ]; then				\
 		printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-	else								\
-		LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-		$(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;		\
+	else	\
+		$(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
 	fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +167 -0

167 lines not on this page · raw message

scripts/link-vmlinux.sh +6 -36

71 lines not on this page · raw message

tools/bpf/resolve_btfids/main.c +136 -82

327 lines not on this page · raw message

tools/testing/selftests/bpf/Makefile +5 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)			\
 	$$(call msg,BINARY,,$$@)
 	$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
 	$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+	$(Q)if [ -f [email protected]_ids ]; then \
+		$(OBJCOPY) --update-section [email protected]_ids $$@; \
+	fi
+	$(Q)rm -f [email protected]_ids [email protected]
 	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
 		   $(OUTPUT)/$(if $2,$2/)bpftool
 
Signature · 1 line
2.52.0
#10 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 718 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).

Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/

---
 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++++++++
 scripts/link-vmlinux.sh              |  42 +-----
 tools/bpf/resolve_btfids/main.c      | 218 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 317 insertions(+), 121 deletions(-)
 create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:        net/sched/act_bpf.c
 F:     net/sched/cls_bpf.c
 F:     samples/bpf/
 F:     scripts/bpf_doc.py
+F:     scripts/gen-btf.sh
 F:     scripts/Makefile.btf
 F:     scripts/pahole-version.sh
 F:     tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..3862fdfa1267 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko =                                                     \
        if [ ! -f $(objtree)/vmlinux ]; then                            \
                printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-       else                                                            \
-               LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-               $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;             \
+       else    \
+               $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
        fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +167 -0
new file mode 100755
index 000000000000..2dfb7ab289ca
@@ -0,0 +1,167 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+#   1. pahole generates BTF from DWARF data
+#   2. resolve_btfids applies kernel-specific btf2btf
+#      transformations and computes data for .BTF_ids section
+#   3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+#   - ${1}.btf.o ready for linking into vmlinux
+#   - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+       echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+       exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+       case "$1" in
+       --btf_base)
+               BTF_BASE="$2"
+               shift 2
+               ;;
+       -*)
+               echo "Unknown option: $1" >&2
+               usage
+               ;;
+       *)
+               break
+               ;;
+       esac
+done
+
+if [ $# -ne 1 ]; then
+       usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+       grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+       printf "  %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+       set -x
+       ;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+       exit 0
+fi
+
+gen_btf_data()
+{
+       info BTF "${ELF_FILE}"
+       btf1="${ELF_FILE}.btf.1"
+       ${PAHOLE} -J ${PAHOLE_FLAGS}                    \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf_encode_detached=${btf1}           \
+               "${ELF_FILE}"
+
+       info BTFIDS "${ELF_FILE}"
+       RESOLVE_BTFIDS_OPTS=""
+       if is_enabled CONFIG_WERROR; then
+               RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+       fi
+       if [ -n "${KBUILD_VERBOSE}" ]; then
+               RESOLVE_BTFIDS_OPTS+=" -v "
+       fi
+       ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS}        \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+       local btf_data=${ELF_FILE}.btf.o
+
+       # Create ${btf_data} which contains just .BTF section but no symbols. Add
+       # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+       # deletes all symbols including __start_BTF and __stop_BTF, which will
+       # be redefined in the linker script.
+       info OBJCOPY "${btf_data}"
+       echo "" | ${CC} -c -x c -o ${btf_data} -
+       ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+               --set-section-flags .BTF=alloc,readonly ${btf_data}
+       ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+       # Change e_type to ET_REL so that it can be used to link final vmlinux.
+       # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+       if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+               et_rel='\0\1'
+       else
+               et_rel='\1\0'
+       fi
+       printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+       info OBJCOPY "${ELF_FILE}"
+       ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}
+
+       # a module might not have a .BTF_ids or .BTF.base section
+       local btf_base="${ELF_FILE}.distilled_base.btf"
+       if [ -f "${btf_base}" ]; then
+               ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE}
+       fi
+       local btf_ids="${ELF_FILE}.btf_ids"
+       if [ -f "${btf_ids}" ]; then
+               ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE}
+       fi
+}
+
+cleanup()
+{
+       rm -f "${ELF_FILE}.btf.1"
+       rm -f "${ELF_FILE}.btf"
+       if [ "${BTFGEN_MODE}" = "module" ]; then
+               rm -f "${ELF_FILE}.distilled_base.btf"
+               rm -f "${ELF_FILE}.btf_ids"
+       fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+       BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+       gen_btf_o
+       ;;
+module)
+       embed_btf_data
+       ;;
+esac
+
+exit 0
scripts/link-vmlinux.sh +6 -36
index 433849ff7529..5bea8795f96d 100755
@@ -105,34 +105,6 @@ vmlinux_link()
                ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
 }
 
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
-       local btf_data=${1}.btf.o
-
-       info BTF "${btf_data}"
-       LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
-       # Create ${btf_data} which contains just .BTF section but no symbols. Add
-       # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
-       # deletes all symbols including __start_BTF and __stop_BTF, which will
-       # be redefined in the linker script. Add 2>/dev/null to suppress GNU
-       # objcopy warnings: "empty loadable segment detected at ..."
-       ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
-               --strip-all ${1} "${btf_data}" 2>/dev/null
-       # Change e_type to ET_REL so that it can be used to link final vmlinux.
-       # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
-       if is_enabled CONFIG_CPU_BIG_ENDIAN; then
-               et_rel='\0\1'
-       else
-               et_rel='\1\0'
-       fi
-       printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
-       btf_vmlinux_bin_o=${btf_data}
-}
-
 # Create ${2}.o file with all symbols from the ${1} object file
 kallsyms()
 {
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
 fi
 
 btf_vmlinux_bin_o=
+btfids_vmlinux=
 kallsymso=
 strip_debug=
 generate_map=
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-       if ! gen_btf .tmp_vmlinux1; then
+       if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then
                echo >&2 "Failed to generate BTF for vmlinux"
                echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
                exit 1
        fi
+       btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+       btfids_vmlinux=.tmp_vmlinux1.btf_ids
 fi
 
 if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
 
 vmlinux_link "${VMLINUX}"
 
-# fill in BTF IDs
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-       info BTFIDS "${VMLINUX}"
-       RESOLVE_BTFIDS_ARGS=""
-       if is_enabled CONFIG_WERROR; then
-               RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
-       fi
-       ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+       info OBJCOPY ${btfids_vmlinux}
+       ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
 fi
 
 mksysmap "${VMLINUX}" System.map
tools/bpf/resolve_btfids/main.c +136 -82
index c60d303ca6ed..b8df6256e29e 100644
@@ -71,9 +71,11 @@
 #include <fcntl.h>
 #include <errno.h>
 #include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
 #include <linux/rbtree.h>
 #include <linux/zalloc.h>
 #include <linux/err.h>
+#include <linux/limits.h>
 #include <bpf/btf.h>
 #include <bpf/libbpf.h>
 #include <subcmd/parse-options.h>
@@ -124,6 +126,7 @@ struct object {
 
        struct btf *btf;
        struct btf *base_btf;
+       bool distilled_base;
 
        struct {
                int              fd;
@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
        return btf_id;
 }
 
-/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
-#ifndef SHF_COMPRESSED
-#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */
-#endif
-
-/*
- * The data of compressed section should be aligned to 4
- * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld
- * sets sh_addralign to 1, which makes libelf fail with
- * misaligned section error during the update:
- *    FAILED elf_update(WRITE): invalid section alignment
- *
- * While waiting for ld fix, we fix the compressed sections
- * sh_addralign value manualy.
- */
-static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh)
+static void bswap_32_data(void *data, u32 nr_bytes)
 {
-       int expected = gelf_getclass(elf) == ELFCLASS32 ? 4 : 8;
+       u32 cnt, i;
+       u32 *ptr;
 
-       if (!(sh->sh_flags & SHF_COMPRESSED))
-               return 0;
+       cnt = nr_bytes / sizeof(u32);
+       ptr = data;
 
-       if (sh->sh_addralign == expected)
-               return 0;
-
-       pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n",
-                 sh->sh_addralign, expected);
-
-       sh->sh_addralign = expected;
-
-       if (gelf_update_shdr(scn, sh) == 0) {
-               pr_err("FAILED cannot update section header: %s\n",
-                       elf_errmsg(-1));
-               return -1;
-       }
-       return 0;
+       for (i = 0; i < cnt; i++)
+               ptr[i] = bswap_32(ptr[i]);
 }
 
 static int elf_collect(struct object *obj)
@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)
 
        elf_version(EV_CURRENT);
 
-       elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
+       elf = elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL);
        if (!elf) {
                close(fd);
                pr_err("FAILED cannot create ELF descriptor: %s\n",
@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj)
                        obj->efile.symbols_shndx = idx;
                        obj->efile.strtabidx     = sh.sh_link;
                } else if (!strcmp(name, BTF_IDS_SECTION)) {
+                       /*
+                        * If target endianness differs from host, we need to bswap32
+                        * the .BTF_ids section data on load, because .BTF_ids has
+                        * Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in
+                        * the target endiannes. We repeat this on dump.
+                        */
+                       if (obj->efile.encoding != ELFDATANATIVE) {
+                               pr_debug("bswap_32 .BTF_ids data from target to host endianness\n");
+                               bswap_32_data(data->d_buf, data->d_size);
+                       }
                        obj->efile.idlist       = data;
                        obj->efile.idlist_shndx = idx;
                        obj->efile.idlist_addr  = sh.sh_addr;
-               } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
-                       /* If a .BTF.base section is found, do not resolve
-                        * BTF ids relative to vmlinux; resolve relative
-                        * to the .BTF.base section instead.  btf__parse_split()
-                        * will take care of this once the base BTF it is
-                        * passed is NULL.
-                        */
-                       obj->base_btf_path = NULL;
                }
-
-               if (compressed_section_fix(elf, scn, &sh))
-                       return -1;
        }
 
        return 0;
@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)
        return 0;
 }
 
+static inline bool is_envvar_set(const char *var_name)
+{
+       const char *value = getenv(var_name);
+
+       return value && value[0] != '\0';
+}
+
 static int load_btf(struct object *obj)
 {
        struct btf *base_btf = NULL, *btf = NULL;
@@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
        obj->base_btf = base_btf;
        obj->btf = btf;
 
+       if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {
+               err = btf__distill_base(obj->btf, &base_btf, &btf);
+               if (err) {
+                       pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
+                       goto out_err;
+               }
+
+               btf__free(obj->btf);
+               btf__free(obj->base_btf);
+               obj->btf = btf;
+               obj->base_btf = base_btf;
+               obj->distilled_base = true;
+       }
+
        return 0;
 
 out_err:
@@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)
                         */
                        BUILD_BUG_ON((u32 *)set8->pairs != &set8->pairs[0].id);
                        qsort(set8->pairs, set8->cnt, sizeof(set8->pairs[0]), cmp_id);
-
-                       /*
-                        * When ELF endianness does not match endianness of the
-                        * host, libelf will do the translation when updating
-                        * the ELF. This, however, corrupts SET8 flags which are
-                        * already in the target endianness. So, let's bswap
-                        * them to the host endianness and libelf will then
-                        * correctly translate everything.
-                        */
-                       if (obj->efile.encoding != ELFDATANATIVE) {
-                               int i;
-
-                               set8->flags = bswap_32(set8->flags);
-                               for (i = 0; i < set8->cnt; i++) {
-                                       set8->pairs[i].flags =
-                                               bswap_32(set8->pairs[i].flags);
-                               }
-                       }
                        break;
                case BTF_ID_KIND_SYM:
                default:
@@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)
 
 static int symbols_patch(struct object *obj)
 {
-       off_t err;
-
        if (__symbols_patch(obj, &obj->structs)  ||
            __symbols_patch(obj, &obj->unions)   ||
            __symbols_patch(obj, &obj->typedefs) ||
@@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)
        if (sets_patch(obj))
                return -1;
 
-       /* Set type to ensure endian translation occurs. */
-       obj->efile.idlist->d_type = ELF_T_WORD;
+       return 0;
+}
 
-       elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+       int fd, ret;
 
-       err = elf_update(obj->efile.elf, ELF_C_WRITE);
-       if (err < 0) {
-               pr_err("FAILED elf_update(WRITE): %s\n",
-                       elf_errmsg(-1));
+       fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+       if (fd < 0) {
+               pr_err("Couldn't open %s for writing\n", out_path);
+               return fd;
+       }
+
+       ret = write(fd, data, size);
+       if (ret < 0 || ret != size) {
+               pr_err("Failed to write data to %s\n", out_path);
+               close(fd);
+               unlink(out_path);
+               return -1;
+       }
+
+       close(fd);
+       pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
+
+       return 0;
+}
+
+static int dump_raw_btf_ids(struct object *obj, const char *out_path)
+{
+       Elf_Data *data = obj->efile.idlist;
+       int fd, err;
+
+       if (!data || !data->d_buf) {
+               pr_debug("%s has no BTF_ids data to dump\n", obj->path);
+               return 0;
+       }
+
+       /*
+        * If target endianness differs from host, we need to bswap32 the
+        * .BTF_ids section data before dumping so that the output is in
+        * target endianness.
+        */
+       if (obj->efile.encoding != ELFDATANATIVE) {
+               pr_debug("bswap_32 .BTF_ids data from host to target endianness\n");
+               bswap_32_data(data->d_buf, data->d_size);
+       }
+
+       err = dump_raw_data(out_path, data->d_buf, data->d_size);
+       if (err)
+               return -1;
+
+       return 0;
+}
+
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+       const void *raw_btf_data;
+       u32 raw_btf_size;
+       int fd, err;
+
+       raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+       if (raw_btf_data == NULL) {
+               pr_err("btf__raw_data() failed\n");
+               return -1;
        }
 
-       pr_debug("update %s for %s\n",
-                err >= 0 ? "ok" : "failed", obj->path);
-       return err < 0 ? -1 : 0;
+       err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
+       if (err)
+               return -1;
+
+       return 0;
 }
 
 static const char * const resolve_btfids_usage[] = {
@@ -824,6 +858,7 @@ int main(int argc, const char **argv)
                .funcs    = RB_ROOT,
                .sets     = RB_ROOT,
        };
+       char out_path[PATH_MAX];
        bool fatal_warnings = false;
        struct option btfid_options[] = {
                OPT_INCR('v', "verbose", &verbose,
@@ -836,7 +871,7 @@ int main(int argc, const char **argv)
                            "turn warnings into errors"),
                OPT_END()
        };
-       int err = -1;
+       int err = -1, path_len;
 
        argc = parse_options(argc, argv, btfid_options, resolve_btfids_usage,
                             PARSE_OPT_STOP_AT_NON_OPTION);
@@ -844,6 +879,11 @@ int main(int argc, const char **argv)
                usage_with_options(resolve_btfids_usage, btfid_options);
 
        obj.path = argv[0];
+       strcpy(out_path, obj.path);
+       path_len = strlen(out_path);
+
+       if (load_btf(&obj))
+               goto out;
 
        if (elf_collect(&obj))
                goto out;
@@ -854,23 +894,37 @@ int main(int argc, const char **argv)
         */
        if (obj.efile.idlist_shndx == -1 ||
            obj.efile.symbols_shndx == -1) {
-               pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
-               err = 0;
-               goto out;
+               pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+               goto dump_btf;
        }
 
        if (symbols_collect(&obj))
                goto out;
 
-       if (load_btf(&obj))
-               goto out;
-
        if (symbols_resolve(&obj))
                goto out;
 
        if (symbols_patch(&obj))
                goto out;
 
+       out_path[path_len] = '\0';
+       strcat(out_path, ".btf_ids");
+       if (dump_raw_btf_ids(&obj, out_path))
+               goto out;
+
+dump_btf:
+       out_path[path_len] = '\0';
+       strcat(out_path, ".btf");
+       if (dump_raw_btf(obj.btf, out_path))
+               goto out;
+
+       if (obj.base_btf && obj.distilled_base) {
+               out_path[path_len] = '\0';
+               strcat(out_path, ".distilled_base.btf");
+               if (dump_raw_btf(obj.base_btf, out_path))
+                       goto out;
+       }
+
        if (!(fatal_warnings && warnings))
                err = 0;
 out:
tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

author reviewer
#11 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

author reviewer
#12 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

Quoted 8 lines

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

Hi Donglin,

I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.

In this series, I only have to change the test_progs build recipe.

The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:

static int resolve_symbols(void)

  {
	struct btf *btf;
	int type_id;
	__u32 nr;

	btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */

        [...]

Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.

If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore

This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.

Quoted 17 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

#13 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

Quoted 8 lines

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

Hi Donglin,

I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.

In this series, I only have to change the test_progs build recipe.

The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:

  static int resolve_symbols(void)
  {
        struct btf *btf;
        int type_id;
        __u32 nr;
        btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */

        [...]
Quoted 7 lines

Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.

If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore

Thanks, could the following modification be accepted?

(quoted diff) +0 -0

b/tools/testing/selftests/bpf/.gitignore
index be1ee7ba7ce0..38ac369cd701 100644

tools/testing/selftests/bpf/.gitignore +1 -0
@@ -45,3 +45,4 @@ xdp_synproxy
 xdp_hw_metadata
 xdp_features
 verification_cert.h
+*.btf
(quoted diff) +0 -0

b/tools/testing/selftests/bpf/Makefile
index 2a027ff9ceaf..a1188129229f 100644

tools/testing/selftests/bpf/Makefile +1 -1
@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
        $(Q)if [ -f [email protected]_ids ]; then \
                $(OBJCOPY) --update-section [email protected]_ids $$@; \
        fi
-       $(Q)rm -f [email protected]_ids [email protected]
+       $(Q)rm -f [email protected]_ids
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool

@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)

                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf no_alu32 cpuv4 bpf_gcc             \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
(quoted diff) +0 -0

b/tools/testing/selftests/bpf/prog_tests/resolve_btfids.c index 51544372f52e..00883ff16569 100644

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -101,7 +101,7 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("test_progs.btf");
        if (CHECK(libbpf_get_error(btf), "resolve",
                  "Failed to load BTF from btf_data.bpf.o\n"))
                return -1;

Thanks,
Donglin

Quoted 4 lines

This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.

Thanks, I will take it on.

Quoted 19 lines
Quoted 18 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

author reviewer
#14 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

Quoted 8 lines

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

Hi Donglin,

I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.

In this series, I only have to change the test_progs build recipe.

The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:

  static int resolve_symbols(void)
  {
        struct btf *btf;
        int type_id;
        __u32 nr;
        btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */

        [...]
Quoted 7 lines

Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.

If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore

Thanks, could the following modification be accepted?

tools/testing/selftests/bpf/.gitignore +1 -0
index be1ee7ba7ce0..38ac369cd701 100644
@@ -45,3 +45,4 @@ xdp_synproxy
 xdp_hw_metadata
 xdp_features
 verification_cert.h
+*.btf
tools/testing/selftests/bpf/Makefile +2 -2
index 2a027ff9ceaf..a1188129229f 100644
@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
        $(Q)if [ -f [email protected]_ids ]; then \
                $(OBJCOPY) --update-section [email protected]_ids $$@; \
        fi
-       $(Q)rm -f [email protected]_ids [email protected]
+       $(Q)rm -f [email protected]_ids
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf no_alu32 cpuv4 bpf_gcc             \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 51544372f52e..00883ff16569 100644

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -101,7 +101,7 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("test_progs.btf");

We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (test_progs, test_maps, test_progs-no_alu32 and others).

I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the final TRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option in resolve_btfids
- somehow (env var?) determine what binary is running in the test
- (a hack) in the makefile, copy [email protected] to "test.btf" or similar

IMO the first option is the best, as this makefile code exists because of that specific test.

The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.

Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
        if (CHECK(libbpf_get_error(btf), "resolve",
                  "Failed to load BTF from btf_data.bpf.o\n"))
                return -1;

Thanks,
Donglin

Quoted 1 line
Quoted 5 lines
Quoted 3 lines

This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.

Thanks, I will take it on.

Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

#15 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 168 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

Quoted 8 lines

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

Hi Donglin,

I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.

In this series, I only have to change the test_progs build recipe.

The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:

  static int resolve_symbols(void)
  {
        struct btf *btf;
        int type_id;
        __u32 nr;
        btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */

        [...]
Quoted 7 lines

Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.

If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore

Thanks, could the following modification be accepted?

tools/testing/selftests/bpf/.gitignore +1 -0
index be1ee7ba7ce0..38ac369cd701 100644
@@ -45,3 +45,4 @@ xdp_synproxy
 xdp_hw_metadata
 xdp_features
 verification_cert.h
+*.btf
tools/testing/selftests/bpf/Makefile +2 -2
index 2a027ff9ceaf..a1188129229f 100644
@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
        $(Q)if [ -f [email protected]_ids ]; then \
                $(OBJCOPY) --update-section [email protected]_ids $$@; \
        fi
-       $(Q)rm -f [email protected]_ids [email protected]
+       $(Q)rm -f [email protected]_ids
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf no_alu32 cpuv4 bpf_gcc             \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 51544372f52e..00883ff16569 100644

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -101,7 +101,7 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("test_progs.btf");

We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (test_progs, test_maps, test_progs-no_alu32 and others).

I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the final TRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option in resolve_btfids
- somehow (env var?) determine what binary is running in the test
- (a hack) in the makefile, copy [email protected] to "test.btf" or similar

Quoted 5 lines

IMO the first option is the best, as this makefile code exists because of that specific test.

The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.

Thanks, I understand. Here are the changes based on the first option:

(quoted diff) +0 -0

b/tools/testing/selftests/bpf/Makefile
index 2a027ff9ceaf..751960aeb8e5 100644

tools/testing/selftests/bpf/Makefile +2 -0
@@ -704,6 +704,16 @@ ifneq ($2:$(OUTPUT),:$(shell pwd))
        $(Q)rsync -aq $$^ $(TRUNNER_OUTPUT)/
 endif
 
+ifneq ($(TRUNNER_BINARY),test_maps)
+$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf

$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids:
$(TRUNNER_OUTPUT)/btf_data.bpf.o \
+

                      $(TRUNNER_OUTPUT)/resolve_btfids.test.o    \
+
                      $(RESOLVE_BTFIDS)

+ $(call msg,BTF+IDS,resolve_btfids,$@)
+ $(Q)$(RESOLVE_BTFIDS) --btf $(dir $@)btf_data.bpf.o $(dir
$@)resolve_btfids.test.o
+
+$(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids +endif
+

 # some X.test.o files have runtime dependencies on Y.bpf.o files
 $(OUTPUT)/$(TRUNNER_BINARY): | $(TRUNNER_BPF_OBJS)

@@ -716,11 +726,9 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)

                 \
                             | $(TRUNNER_BINARY)-extras
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS)

$$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
- $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
- $(Q)if [ -f [email protected]_ids ]; then \
- $(OBJCOPY) --update-section [email protected]_ids $$@; \
+ $(Q)if [ "$(TRUNNER_BINARY)" != "test_maps" ]; then \
+ $(OBJCOPY) --update-section
.BTF_ids=$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids $$@; \
fi
- $(Q)rm -f [email protected]_ids [email protected]

        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool

@@ -908,7 +916,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)

                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf *.btf_ids no_alu32 cpuv4 bpf_gcc   \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
(quoted diff) +0 -0

b/tools/testing/selftests/bpf/prog_tests/resolve_btfids.c index 51544372f52e..eef6efc82606 100644

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +2 -2
@@ -101,9 +101,9 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("resolve_btfids.test.o.btf");
        if (CHECK(libbpf_get_error(btf), "resolve",
-                 "Failed to load BTF from btf_data.bpf.o\n"))
+                 "Failed to load BTF from resolve_btfids.test.o.btf\n"))
                return -1;
 
        nr = btf__type_cnt(btf);
Quoted 40 lines
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
        if (CHECK(libbpf_get_error(btf), "resolve",
                  "Failed to load BTF from btf_data.bpf.o\n"))
                return -1;

Thanks,
Donglin

Quoted 1 line
Quoted 5 lines
Quoted 3 lines

This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.

Thanks, I will take it on.

Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

#16 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 197 lines · Donglin Peng
Quoted 168 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

Quoted 8 lines

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

Hi Donglin,

I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.

In this series, I only have to change the test_progs build recipe.

The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:

  static int resolve_symbols(void)
  {
        struct btf *btf;
        int type_id;
        __u32 nr;
        btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */

        [...]
Quoted 7 lines

Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.

If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore

Thanks, could the following modification be accepted?

tools/testing/selftests/bpf/.gitignore +1 -0
index be1ee7ba7ce0..38ac369cd701 100644
@@ -45,3 +45,4 @@ xdp_synproxy
 xdp_hw_metadata
 xdp_features
 verification_cert.h
+*.btf
tools/testing/selftests/bpf/Makefile +2 -2
index 2a027ff9ceaf..a1188129229f 100644
@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
        $(Q)if [ -f [email protected]_ids ]; then \
                $(OBJCOPY) --update-section [email protected]_ids $$@; \
        fi
-       $(Q)rm -f [email protected]_ids [email protected]
+       $(Q)rm -f [email protected]_ids
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf no_alu32 cpuv4 bpf_gcc             \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 51544372f52e..00883ff16569 100644

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -101,7 +101,7 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("test_progs.btf");

We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (test_progs, test_maps, test_progs-no_alu32 and others).

I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the final TRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option in resolve_btfids
- somehow (env var?) determine what binary is running in the test
- (a hack) in the makefile, copy [email protected] to "test.btf" or similar

Quoted 5 lines

IMO the first option is the best, as this makefile code exists because of that specific test.

The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.

Thanks, I understand. Here are the changes based on the first option:

tools/testing/selftests/bpf/Makefile +2 -0
index 2a027ff9ceaf..751960aeb8e5 100644
@@ -704,6 +704,16 @@ ifneq ($2:$(OUTPUT),:$(shell pwd))
        $(Q)rsync -aq $$^ $(TRUNNER_OUTPUT)/
 endif
 
+ifneq ($(TRUNNER_BINARY),test_maps)
+$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf

$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids:
$(TRUNNER_OUTPUT)/btf_data.bpf.o \

tools/testing/selftests/bpf/Makefile +4 -0
+
                      $(TRUNNER_OUTPUT)/resolve_btfids.test.o    \
+
                      $(RESOLVE_BTFIDS)
+       $(call msg,BTF+IDS,resolve_btfids,$@)
+       $(Q)$(RESOLVE_BTFIDS) --btf $(dir $@)btf_data.bpf.o $(dir
Quoted 1 line

$@)resolve_btfids.test.o

Sorry, the above command has some issues. Use the following command instead:
$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)btf_data.bpf.o $(TRUNNER_OUTPUT)resolve_btfids.test.o

Quoted 92 lines
tools/testing/selftests/bpf/Makefile +4 -0
+
+$(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids
+endif
+
 # some X.test.o files have runtime dependencies on Y.bpf.o files
 $(OUTPUT)/$(TRUNNER_BINARY): | $(TRUNNER_BPF_OBJS)
@@ -716,11 +726,9 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
                             | $(TRUNNER_BINARY)-extras
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS)

$$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@

tools/testing/selftests/bpf/Makefile +2 -3
-       $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
-       $(Q)if [ -f [email protected]_ids ]; then \
-               $(OBJCOPY) --update-section [email protected]_ids $$@; \
+       $(Q)if [ "$(TRUNNER_BINARY)" != "test_maps" ]; then \
+               $(OBJCOPY) --update-section

.BTF_ids=$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids $$@; \
fi

tools/testing/selftests/bpf/Makefile +1 -2
-       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
@@ -908,7 +916,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf *.btf_ids no_alu32 cpuv4 bpf_gcc   \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +2 -2
index 51544372f52e..eef6efc82606 100644
@@ -101,9 +101,9 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("resolve_btfids.test.o.btf");
        if (CHECK(libbpf_get_error(btf), "resolve",
-                 "Failed to load BTF from btf_data.bpf.o\n"))
+                 "Failed to load BTF from resolve_btfids.test.o.btf\n"))
                return -1;
 
        nr = btf__type_cnt(btf);
Quoted 41 lines
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
        if (CHECK(libbpf_get_error(btf), "resolve",
                  "Failed to load BTF from btf_data.bpf.o\n"))
                return -1;

Thanks,
Donglin

Quoted 1 line
Quoted 5 lines
Quoted 3 lines

This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.

Thanks, I will take it on.

Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

#17 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

In reply to Ihor Solodrai
Quoted 160 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

Quoted 8 lines

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

Hi Donglin,

I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.

In this series, I only have to change the test_progs build recipe.

The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:

  static int resolve_symbols(void)
  {
        struct btf *btf;
        int type_id;
        __u32 nr;
        btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */

        [...]
Quoted 7 lines

Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.

If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore

Thanks, could the following modification be accepted?

tools/testing/selftests/bpf/.gitignore +1 -0
index be1ee7ba7ce0..38ac369cd701 100644
@@ -45,3 +45,4 @@ xdp_synproxy
 xdp_hw_metadata
 xdp_features
 verification_cert.h
+*.btf
tools/testing/selftests/bpf/Makefile +2 -2
index 2a027ff9ceaf..a1188129229f 100644
@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
        $(Q)if [ -f [email protected]_ids ]; then \
                $(OBJCOPY) --update-section [email protected]_ids $$@; \
        fi
-       $(Q)rm -f [email protected]_ids [email protected]
+       $(Q)rm -f [email protected]_ids
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf no_alu32 cpuv4 bpf_gcc             \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 51544372f52e..00883ff16569 100644

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -101,7 +101,7 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("test_progs.btf");

We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (test_progs, test_maps, test_progs-no_alu32 and others).

Quoted 6 lines

I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the final TRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option in resolve_btfids
- somehow (env var?) determine what binary is running in the test

see cd_flavor_subdir(), each flavor of test_progs has its dedicated subdir (and flavor-less one just runs in selftests' directory). So I think you should be able to hard-code the name. Even for btf_data.bpf.o:

$ find . -name 'btf_data.bpf.o'
./no_alu32/btf_data.bpf.o
./cpuv4/btf_data.bpf.o
./btf_data.bpf.o
Quoted 48 lines

- (a hack) in the makefile, copy [email protected] to "test.btf" or similar

IMO the first option is the best, as this makefile code exists because of that specific test.

The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.

Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
        if (CHECK(libbpf_get_error(btf), "resolve",
                  "Failed to load BTF from btf_data.bpf.o\n"))
                return -1;

Thanks,
Donglin

Quoted 1 line
Quoted 5 lines
Quoted 3 lines

This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.

Thanks, I will take it on.

Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

#18 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 174 lines · Andrii Nakryiko
Quoted 160 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai

[...]

tools/testing/selftests/bpf/Makefile +1 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)

tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)                  \
        $$(call msg,BINARY,,$$@)
        $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
        $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+       $(Q)if [ -f [email protected]_ids ]; then \
+               $(OBJCOPY) --update-section [email protected]_ids $$@; \

I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:

All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL

The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.

Quoted 8 lines

This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.

What do you think of it?

Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.

As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.

resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).

And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.

By the way, the test was written more than 5y ago [1], so it might be outdated too.

Quoted 8 lines

I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.

But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.

I'll try to think of something.

Hi Donglin,

I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.

In this series, I only have to change the test_progs build recipe.

The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:

  static int resolve_symbols(void)
  {
        struct btf *btf;
        int type_id;
        __u32 nr;
        btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */

        [...]
Quoted 7 lines

Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.

If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore

Thanks, could the following modification be accepted?

tools/testing/selftests/bpf/.gitignore +1 -0
index be1ee7ba7ce0..38ac369cd701 100644
@@ -45,3 +45,4 @@ xdp_synproxy
 xdp_hw_metadata
 xdp_features
 verification_cert.h
+*.btf
tools/testing/selftests/bpf/Makefile +2 -2
index 2a027ff9ceaf..a1188129229f 100644
@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
                 \
        $(Q)if [ -f [email protected]_ids ]; then \
                $(OBJCOPY) --update-section [email protected]_ids $$@; \
        fi
-       $(Q)rm -f [email protected]_ids [email protected]
+       $(Q)rm -f [email protected]_ids
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
                 \
        prog_tests/tests.h map_tests/tests.h verifier/tests.h           \
        feature bpftool $(TEST_KMOD_TARGETS)                            \
        $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h   \
-                              no_alu32 cpuv4 bpf_gcc                   \
+                              *.btf no_alu32 cpuv4 bpf_gcc             \
                               liburandom_read.so)                      \
        $(OUTPUT)/FEATURE-DUMP.selftests
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
index 51544372f52e..00883ff16569 100644

tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104

Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -101,7 +101,7 @@ static int resolve_symbols(void)
        int type_id;
        __u32 nr;
 
-       btf = btf__parse_elf("btf_data.bpf.o", NULL);
+       btf = btf__parse_raw("test_progs.btf");

We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (test_progs, test_maps, test_progs-no_alu32 and others).

Quoted 6 lines

I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the final TRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option in resolve_btfids
- somehow (env var?) determine what binary is running in the test

see cd_flavor_subdir(), each flavor of test_progs has its dedicated subdir (and flavor-less one just runs in selftests' directory). So I think you should be able to hard-code the name. Even for btf_data.bpf.o:

Quoted 4 lines
$ find . -name 'btf_data.bpf.o'
./no_alu32/btf_data.bpf.o
./cpuv4/btf_data.bpf.o
./btf_data.bpf.o

I think the first option is feasible. Based on my modifications, the resolve_btfids.test.o.btffile will exist in each flavor's subdirectory. Therefore, we can hard-code the filename resolve_btfids.test.o.btf directly in prog_tests/resolve_btfids.c:

$ find -name resolve_btfids.test.o.btf
./resolve_btfids.test.o.btf
./cpuv4/resolve_btfids.test.o.btf
./no_alu32/resolve_btfids.test.o.btf
Quoted 50 lines
Quoted 50 lines

- (a hack) in the makefile, copy [email protected] to "test.btf" or similar

IMO the first option is the best, as this makefile code exists because of that specific test.

The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.

Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0 -0
        if (CHECK(libbpf_get_error(btf), "resolve",
                  "Failed to load BTF from btf_data.bpf.o\n"))
                return -1;

Thanks,
Donglin

Quoted 1 line
Quoted 5 lines
Quoted 3 lines

This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.

Thanks, I will take it on.

Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines

[1] https://lore.kernel.org/bpf/[email protected]/

Quoted 12 lines

Thanks,
Donglin

Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2 -1
+       fi
+       $(Q)rm -f [email protected]_ids [email protected]
        $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
                   $(OUTPUT)/$(if $2,$2/)bpftool
 
--

2.52.0

maintainer
#19 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

In reply to Ihor Solodrai
Quoted 4 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

Quoted 8 lines

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?

Quoted 8 lines

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

Nit: more context needed, as is the statement raises questions but not
answers them.

Quoted 42 lines

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).

Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.

Quoted 3 lines

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285

Nit: these links are moving target, should refer to a commit or a tag.

Quoted 4 lines

[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/

Quoted 9 lines
---
 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++++++++
 scripts/link-vmlinux.sh              |  42 +-----
 tools/bpf/resolve_btfids/main.c      | 218 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 317 insertions(+), 121 deletions(-)
 create mode 100755 scripts/gen-btf.sh

Since resolve_btfids is now responsible for distilled base generation, does Makefile.btf need modification to remove "--btf_features=distilled_base"?

Quoted 250 lines
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:	net/sched/act_bpf.c
 F:	net/sched/cls_bpf.c
 F:	samples/bpf/
 F:	scripts/bpf_doc.py
+F:	scripts/gen-btf.sh
 F:	scripts/Makefile.btf
 F:	scripts/pahole-version.sh
 F:	tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..3862fdfa1267 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko = 							\
 	if [ ! -f $(objtree)/vmlinux ]; then				\
 		printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-	else								\
-		LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-		$(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;		\
+	else	\
+		$(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
 	fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +167 -0
new file mode 100755
index 000000000000..2dfb7ab289ca
@@ -0,0 +1,167 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+#   1. pahole generates BTF from DWARF data
+#   2. resolve_btfids applies kernel-specific btf2btf
+#      transformations and computes data for .BTF_ids section
+#   3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+#   - ${1}.btf.o ready for linking into vmlinux
+#   - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+	echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+	exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+	case "$1" in
+	--btf_base)
+		BTF_BASE="$2"
+		shift 2
+		;;
+	-*)
+		echo "Unknown option: $1" >&2
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+done
+
+if [ $# -ne 1 ]; then
+	usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+	grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+	printf "  %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+	set -x
+	;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+	exit 0
+fi
+
+gen_btf_data()
+{
+	info BTF "${ELF_FILE}"
+	btf1="${ELF_FILE}.btf.1"
+	${PAHOLE} -J ${PAHOLE_FLAGS}			\
+		${BTF_BASE:+--btf_base ${BTF_BASE}}	\
+		--btf_encode_detached=${btf1}		\
+		"${ELF_FILE}"
+
+	info BTFIDS "${ELF_FILE}"
+	RESOLVE_BTFIDS_OPTS=""
+	if is_enabled CONFIG_WERROR; then
+		RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+	fi
+	if [ -n "${KBUILD_VERBOSE}" ]; then
+		RESOLVE_BTFIDS_OPTS+=" -v "
+	fi
+	${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS}	\
+		${BTF_BASE:+--btf_base ${BTF_BASE}}	\
+		--btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+	local btf_data=${ELF_FILE}.btf.o
+
+	# Create ${btf_data} which contains just .BTF section but no symbols. Add
+	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+	# deletes all symbols including __start_BTF and __stop_BTF, which will
+	# be redefined in the linker script.
+	info OBJCOPY "${btf_data}"
+	echo "" | ${CC} -c -x c -o ${btf_data} -
+	${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+		--set-section-flags .BTF=alloc,readonly ${btf_data}
+	${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+	# Change e_type to ET_REL so that it can be used to link final vmlinux.
+	# GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+	if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+		et_rel='\0\1'
+	else
+		et_rel='\1\0'
+	fi
+	printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+	info OBJCOPY "${ELF_FILE}"
+	${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}
+
+	# a module might not have a .BTF_ids or .BTF.base section
+	local btf_base="${ELF_FILE}.distilled_base.btf"
+	if [ -f "${btf_base}" ]; then
+		${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE}
+	fi
+	local btf_ids="${ELF_FILE}.btf_ids"
+	if [ -f "${btf_ids}" ]; then
+		${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE}
+	fi
+}
+
+cleanup()
+{
+	rm -f "${ELF_FILE}.btf.1"
+	rm -f "${ELF_FILE}.btf"
+	if [ "${BTFGEN_MODE}" = "module" ]; then
+		rm -f "${ELF_FILE}.distilled_base.btf"
+		rm -f "${ELF_FILE}.btf_ids"
+	fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+	BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+	gen_btf_o
+	;;
+module)
+	embed_btf_data
+	;;
+esac
+
+exit 0
scripts/link-vmlinux.sh +1 -28
index 433849ff7529..5bea8795f96d 100755
@@ -105,34 +105,6 @@ vmlinux_link()
 		${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
 }
 
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
-	local btf_data=${1}.btf.o
-
-	info BTF "${btf_data}"
-	LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
-	# Create ${btf_data} which contains just .BTF section but no symbols. Add
-	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
-	# deletes all symbols including __start_BTF and __stop_BTF, which will
-	# be redefined in the linker script. Add 2>/dev/null to suppress GNU
-	# objcopy warnings: "empty loadable segment detected at ..."
-	${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
-		--strip-all ${1} "${btf_data}" 2>/dev/null
-	# Change e_type to ET_REL so that it can be used to link final vmlinux.
-	# GNU ld 2.35+ and lld do not allow an ET_EXEC input.
-	if is_enabled CONFIG_CPU_BIG_ENDIAN; then
-		et_rel='\0\1'
-	else
-		et_rel='\1\0'
-	fi
-	printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
-	btf_vmlinux_bin_o=${btf_data}
-}
-
 # Create ${2}.o file with all symbols from the ${1} object file
 kallsyms()
 {
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
 fi
 
 btf_vmlinux_bin_o=
+btfids_vmlinux=
 kallsymso=
 strip_debug=
 generate_map=

scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200

Quoted 5 lines
scripts/link-vmlinux.sh +1 -1
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-	if ! gen_btf .tmp_vmlinux1; then
+	if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then

Nit: maybe pass output file names as parameters for get-btf.sh?

Quoted 14 lines
scripts/link-vmlinux.sh +2 -1
 		echo >&2 "Failed to generate BTF for vmlinux"
 		echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
 		exit 1
 	fi
+	btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+	btfids_vmlinux=.tmp_vmlinux1.btf_ids
 fi
 
 if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
 
 vmlinux_link "${VMLINUX}"
 
-# fill in BTF IDs

scripts/link-vmlinux.sh fi · line 262

Quoted 10 lines
scripts/link-vmlinux.sh +2 -6
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-	info BTFIDS "${VMLINUX}"
-	RESOLVE_BTFIDS_ARGS=""
-	if is_enabled CONFIG_WERROR; then
-		RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
-	fi
-	${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+	info OBJCOPY ${btfids_vmlinux}
+	${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
 fi

Nit: Maybe do this in get-btf.sh as for modules?
To avoid checking for CONFIG_DEBUG_INFO_BTF in two places.

Quoted 132 lines
scripts/link-vmlinux.sh +0 -0
 mksysmap "${VMLINUX}" System.map
tools/bpf/resolve_btfids/main.c +28 -45
index c60d303ca6ed..b8df6256e29e 100644
@@ -71,9 +71,11 @@
 #include <fcntl.h>
 #include <errno.h>
 #include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
 #include <linux/rbtree.h>
 #include <linux/zalloc.h>
 #include <linux/err.h>
+#include <linux/limits.h>
 #include <bpf/btf.h>
 #include <bpf/libbpf.h>
 #include <subcmd/parse-options.h>
@@ -124,6 +126,7 @@ struct object {
 
 	struct btf *btf;
 	struct btf *base_btf;
+	bool distilled_base;
 
 	struct {
 		int		 fd;
@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 	return btf_id;
 }
 
-/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
-#ifndef SHF_COMPRESSED
-#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */
-#endif
-
-/*
- * The data of compressed section should be aligned to 4
- * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld
- * sets sh_addralign to 1, which makes libelf fail with
- * misaligned section error during the update:
- *    FAILED elf_update(WRITE): invalid section alignment
- *
- * While waiting for ld fix, we fix the compressed sections
- * sh_addralign value manualy.
- */
-static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh)
+static void bswap_32_data(void *data, u32 nr_bytes)
 {
-	int expected = gelf_getclass(elf) == ELFCLASS32 ? 4 : 8;
+	u32 cnt, i;
+	u32 *ptr;
 
-	if (!(sh->sh_flags & SHF_COMPRESSED))
-		return 0;
+	cnt = nr_bytes / sizeof(u32);
+	ptr = data;
 
-	if (sh->sh_addralign == expected)
-		return 0;
-
-	pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n",
-		  sh->sh_addralign, expected);
-
-	sh->sh_addralign = expected;
-
-	if (gelf_update_shdr(scn, sh) == 0) {
-		pr_err("FAILED cannot update section header: %s\n",
-			elf_errmsg(-1));
-		return -1;
-	}
-	return 0;
+	for (i = 0; i < cnt; i++)
+		ptr[i] = bswap_32(ptr[i]);
 }
 
 static int elf_collect(struct object *obj)
@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)
 
 	elf_version(EV_CURRENT);
 
-	elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
+	elf = elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL);
 	if (!elf) {
 		close(fd);
 		pr_err("FAILED cannot create ELF descriptor: %s\n",
@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj)
 			obj->efile.symbols_shndx = idx;
 			obj->efile.strtabidx     = sh.sh_link;
 		} else if (!strcmp(name, BTF_IDS_SECTION)) {
+			/*
+			 * If target endianness differs from host, we need to bswap32
+			 * the .BTF_ids section data on load, because .BTF_ids has
+			 * Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in
+			 * the target endiannes. We repeat this on dump.
+			 */
+			if (obj->efile.encoding != ELFDATANATIVE) {
+				pr_debug("bswap_32 .BTF_ids data from target to host endianness\n");
+				bswap_32_data(data->d_buf, data->d_size);
+			}
 			obj->efile.idlist       = data;
 			obj->efile.idlist_shndx = idx;
 			obj->efile.idlist_addr  = sh.sh_addr;
-		} else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
-			/* If a .BTF.base section is found, do not resolve
-			 * BTF ids relative to vmlinux; resolve relative
-			 * to the .BTF.base section instead.  btf__parse_split()
-			 * will take care of this once the base BTF it is
-			 * passed is NULL.
-			 */
-			obj->base_btf_path = NULL;
 		}
-
-		if (compressed_section_fix(elf, scn, &sh))
-			return -1;
 	}
 
 	return 0;
@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)
 	return 0;
 }
 
+static inline bool is_envvar_set(const char *var_name)
+{
+	const char *value = getenv(var_name);
+
+	return value && value[0] != '\0';
+}
+
 static int load_btf(struct object *obj)
 {
 	struct btf *base_btf = NULL, *btf = NULL;

tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557

Quoted 4 lines
tools/bpf/resolve_btfids/main.c +1 -0
@@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
 	obj->base_btf = base_btf;
 	obj->btf = btf;
 
+	if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {

This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?

Quoted 182 lines
tools/bpf/resolve_btfids/main.c +89 -35
+		err = btf__distill_base(obj->btf, &base_btf, &btf);
+		if (err) {
+			pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
+			goto out_err;
+		}
+
+		btf__free(obj->btf);
+		btf__free(obj->base_btf);
+		obj->btf = btf;
+		obj->base_btf = base_btf;
+		obj->distilled_base = true;
+	}
+
 	return 0;
 
 out_err:
@@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)
 			 */
 			BUILD_BUG_ON((u32 *)set8->pairs != &set8->pairs[0].id);
 			qsort(set8->pairs, set8->cnt, sizeof(set8->pairs[0]), cmp_id);
-
-			/*
-			 * When ELF endianness does not match endianness of the
-			 * host, libelf will do the translation when updating
-			 * the ELF. This, however, corrupts SET8 flags which are
-			 * already in the target endianness. So, let's bswap
-			 * them to the host endianness and libelf will then
-			 * correctly translate everything.
-			 */
-			if (obj->efile.encoding != ELFDATANATIVE) {
-				int i;
-
-				set8->flags = bswap_32(set8->flags);
-				for (i = 0; i < set8->cnt; i++) {
-					set8->pairs[i].flags =
-						bswap_32(set8->pairs[i].flags);
-				}
-			}
 			break;
 		case BTF_ID_KIND_SYM:
 		default:
@@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)
 
 static int symbols_patch(struct object *obj)
 {
-	off_t err;
-
 	if (__symbols_patch(obj, &obj->structs)  ||
 	    __symbols_patch(obj, &obj->unions)   ||
 	    __symbols_patch(obj, &obj->typedefs) ||
@@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)
 	if (sets_patch(obj))
 		return -1;
 
-	/* Set type to ensure endian translation occurs. */
-	obj->efile.idlist->d_type = ELF_T_WORD;
+	return 0;
+}
 
-	elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+	int fd, ret;
 
-	err = elf_update(obj->efile.elf, ELF_C_WRITE);
-	if (err < 0) {
-		pr_err("FAILED elf_update(WRITE): %s\n",
-			elf_errmsg(-1));
+	fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+	if (fd < 0) {
+		pr_err("Couldn't open %s for writing\n", out_path);
+		return fd;
+	}
+
+	ret = write(fd, data, size);
+	if (ret < 0 || ret != size) {
+		pr_err("Failed to write data to %s\n", out_path);
+		close(fd);
+		unlink(out_path);
+		return -1;
+	}
+
+	close(fd);
+	pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
+
+	return 0;
+}
+
+static int dump_raw_btf_ids(struct object *obj, const char *out_path)
+{
+	Elf_Data *data = obj->efile.idlist;
+	int fd, err;
+
+	if (!data || !data->d_buf) {
+		pr_debug("%s has no BTF_ids data to dump\n", obj->path);
+		return 0;
+	}
+
+	/*
+	 * If target endianness differs from host, we need to bswap32 the
+	 * .BTF_ids section data before dumping so that the output is in
+	 * target endianness.
+	 */
+	if (obj->efile.encoding != ELFDATANATIVE) {
+		pr_debug("bswap_32 .BTF_ids data from host to target endianness\n");
+		bswap_32_data(data->d_buf, data->d_size);
+	}
+
+	err = dump_raw_data(out_path, data->d_buf, data->d_size);
+	if (err)
+		return -1;
+
+	return 0;
+}
+
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+	const void *raw_btf_data;
+	u32 raw_btf_size;
+	int fd, err;
+
+	raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+	if (raw_btf_data == NULL) {
+		pr_err("btf__raw_data() failed\n");
+		return -1;
 	}
 
-	pr_debug("update %s for %s\n",
-		 err >= 0 ? "ok" : "failed", obj->path);
-	return err < 0 ? -1 : 0;
+	err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
+	if (err)
+		return -1;
+
+	return 0;
 }
 
 static const char * const resolve_btfids_usage[] = {
@@ -824,6 +858,7 @@ int main(int argc, const char **argv)
 		.funcs    = RB_ROOT,
 		.sets     = RB_ROOT,
 	};
+	char out_path[PATH_MAX];
 	bool fatal_warnings = false;
 	struct option btfid_options[] = {
 		OPT_INCR('v', "verbose", &verbose,
@@ -836,7 +871,7 @@ int main(int argc, const char **argv)
 			    "turn warnings into errors"),
 		OPT_END()
 	};
-	int err = -1;
+	int err = -1, path_len;
 
 	argc = parse_options(argc, argv, btfid_options, resolve_btfids_usage,
 			     PARSE_OPT_STOP_AT_NON_OPTION);
@@ -844,6 +879,11 @@ int main(int argc, const char **argv)
 		usage_with_options(resolve_btfids_usage, btfid_options);
 
 	obj.path = argv[0];
+	strcpy(out_path, obj.path);
+	path_len = strlen(out_path);
+
+	if (load_btf(&obj))
+		goto out;
 
 	if (elf_collect(&obj))
 		goto out;
@@ -854,23 +894,37 @@ int main(int argc, const char **argv)
 	 */
 	if (obj.efile.idlist_shndx == -1 ||
 	    obj.efile.symbols_shndx == -1) {
-		pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
-		err = 0;
-		goto out;
+		pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+		goto dump_btf;
 	}
 
 	if (symbols_collect(&obj))
 		goto out;
 
-	if (load_btf(&obj))

tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 911

Quoted 10 lines
tools/bpf/resolve_btfids/main.c +2 -2
-		goto out;
-
 	if (symbols_resolve(&obj))
 		goto out;
 
 	if (symbols_patch(&obj))
 		goto out;
 
+	out_path[path_len] = '\0';
+	strcat(out_path, ".btf_ids");

Nit: here and below use snprintf() to check for 'out_path' overflow.

Quoted 42 lines
tools/bpf/resolve_btfids/main.c +16 -0
+	if (dump_raw_btf_ids(&obj, out_path))
+		goto out;
+
+dump_btf:
+	out_path[path_len] = '\0';
+	strcat(out_path, ".btf");
+	if (dump_raw_btf(obj.btf, out_path))
+		goto out;
+
+	if (obj.base_btf && obj.distilled_base) {
+		out_path[path_len] = '\0';
+		strcat(out_path, ".distilled_base.btf");
+		if (dump_raw_btf(obj.base_btf, out_path))
+			goto out;
+	}
+
 	if (!(fatal_warnings && warnings))
 		err = 0;
 out:
tools/testing/selftests/bpf/Makefile +5 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)			\
 	$$(call msg,BINARY,,$$@)
 	$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
 	$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+	$(Q)if [ -f [email protected]_ids ]; then \
+		$(OBJCOPY) --update-section [email protected]_ids $$@; \
+	fi
+	$(Q)rm -f [email protected]_ids [email protected]
 	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
 		   $(OUTPUT)/$(if $2,$2/)bpftool
 
author reviewer
#20 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 5 lines · Eduard Zingerman
Quoted 4 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

Hi Eduard, thank you for the review.

Quoted 11 lines
Quoted 8 lines

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?

Sure, makes sense.

Quoted 5 lines
Quoted 5 lines

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

Quoted 7 lines
Quoted 4 lines

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

Nit: more context needed, as is the statement raises questions but not
answers them.

Would you like to see more details about why using libelf is complicated? I don't follow what's unclear here, sorry...

Quoted 43 lines
Quoted 43 lines

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).

Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.

Quoted 5 lines
Quoted 3 lines

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285

Nit: these links are moving target, should refer to a commit or a tag.

Acked

Quoted 15 lines
Quoted 5 lines

[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/

Quoted 9 lines
---
 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++++++++
 scripts/link-vmlinux.sh              |  42 +-----
 tools/bpf/resolve_btfids/main.c      | 218 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 317 insertions(+), 121 deletions(-)
 create mode 100755 scripts/gen-btf.sh
Quoted 2 lines

Since resolve_btfids is now responsible for distilled base generation, does Makefile.btf need modification to remove "--btf_features=distilled_base"?

Yes, good catch.

Quoted 253 lines
Quoted 253 lines
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:	net/sched/act_bpf.c
 F:	net/sched/cls_bpf.c
 F:	samples/bpf/
 F:	scripts/bpf_doc.py
+F:	scripts/gen-btf.sh
 F:	scripts/Makefile.btf
 F:	scripts/pahole-version.sh
 F:	tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..3862fdfa1267 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko = 							\
 	if [ ! -f $(objtree)/vmlinux ]; then				\
 		printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-	else								\
-		LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-		$(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;		\
+	else	\
+		$(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
 	fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +167 -0
new file mode 100755
index 000000000000..2dfb7ab289ca
@@ -0,0 +1,167 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+#   1. pahole generates BTF from DWARF data
+#   2. resolve_btfids applies kernel-specific btf2btf
+#      transformations and computes data for .BTF_ids section
+#   3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+#   - ${1}.btf.o ready for linking into vmlinux
+#   - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+	echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+	exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+	case "$1" in
+	--btf_base)
+		BTF_BASE="$2"
+		shift 2
+		;;
+	-*)
+		echo "Unknown option: $1" >&2
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+done
+
+if [ $# -ne 1 ]; then
+	usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+	grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+	printf "  %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+	set -x
+	;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+	exit 0
+fi
+
+gen_btf_data()
+{
+	info BTF "${ELF_FILE}"
+	btf1="${ELF_FILE}.btf.1"
+	${PAHOLE} -J ${PAHOLE_FLAGS}			\
+		${BTF_BASE:+--btf_base ${BTF_BASE}}	\
+		--btf_encode_detached=${btf1}		\
+		"${ELF_FILE}"
+
+	info BTFIDS "${ELF_FILE}"
+	RESOLVE_BTFIDS_OPTS=""
+	if is_enabled CONFIG_WERROR; then
+		RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+	fi
+	if [ -n "${KBUILD_VERBOSE}" ]; then
+		RESOLVE_BTFIDS_OPTS+=" -v "
+	fi
+	${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS}	\
+		${BTF_BASE:+--btf_base ${BTF_BASE}}	\
+		--btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+	local btf_data=${ELF_FILE}.btf.o
+
+	# Create ${btf_data} which contains just .BTF section but no symbols. Add
+	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+	# deletes all symbols including __start_BTF and __stop_BTF, which will
+	# be redefined in the linker script.
+	info OBJCOPY "${btf_data}"
+	echo "" | ${CC} -c -x c -o ${btf_data} -
+	${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+		--set-section-flags .BTF=alloc,readonly ${btf_data}
+	${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+	# Change e_type to ET_REL so that it can be used to link final vmlinux.
+	# GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+	if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+		et_rel='\0\1'
+	else
+		et_rel='\1\0'
+	fi
+	printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+	info OBJCOPY "${ELF_FILE}"
+	${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}
+
+	# a module might not have a .BTF_ids or .BTF.base section
+	local btf_base="${ELF_FILE}.distilled_base.btf"
+	if [ -f "${btf_base}" ]; then
+		${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE}
+	fi
+	local btf_ids="${ELF_FILE}.btf_ids"
+	if [ -f "${btf_ids}" ]; then
+		${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE}
+	fi
+}
+
+cleanup()
+{
+	rm -f "${ELF_FILE}.btf.1"
+	rm -f "${ELF_FILE}.btf"
+	if [ "${BTFGEN_MODE}" = "module" ]; then
+		rm -f "${ELF_FILE}.distilled_base.btf"
+		rm -f "${ELF_FILE}.btf_ids"
+	fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+	BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+	gen_btf_o
+	;;
+module)
+	embed_btf_data
+	;;
+esac
+
+exit 0
scripts/link-vmlinux.sh +1 -28
index 433849ff7529..5bea8795f96d 100755
@@ -105,34 +105,6 @@ vmlinux_link()
 		${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
 }
 
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
-	local btf_data=${1}.btf.o
-
-	info BTF "${btf_data}"
-	LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
-	# Create ${btf_data} which contains just .BTF section but no symbols. Add
-	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
-	# deletes all symbols including __start_BTF and __stop_BTF, which will
-	# be redefined in the linker script. Add 2>/dev/null to suppress GNU
-	# objcopy warnings: "empty loadable segment detected at ..."
-	${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
-		--strip-all ${1} "${btf_data}" 2>/dev/null
-	# Change e_type to ET_REL so that it can be used to link final vmlinux.
-	# GNU ld 2.35+ and lld do not allow an ET_EXEC input.
-	if is_enabled CONFIG_CPU_BIG_ENDIAN; then
-		et_rel='\0\1'
-	else
-		et_rel='\1\0'
-	fi
-	printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
-	btf_vmlinux_bin_o=${btf_data}
-}
-
 # Create ${2}.o file with all symbols from the ${1} object file
 kallsyms()
 {
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
 fi
 
 btf_vmlinux_bin_o=
+btfids_vmlinux=
 kallsymso=
 strip_debug=
 generate_map=
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
Quoted 5 lines

scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200

Quoted 3 lines
scripts/link-vmlinux.sh +1 -1
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-	if ! gen_btf .tmp_vmlinux1; then
+	if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then

Nit: maybe pass output file names as parameters for get-btf.sh?

I don't see a point in that. The script and callsite will become more complicated, but what is the benefit?

Quoted 14 lines
Quoted 14 lines
scripts/link-vmlinux.sh +2 -0
 		echo >&2 "Failed to generate BTF for vmlinux"
 		echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
 		exit 1
 	fi
+	btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+	btfids_vmlinux=.tmp_vmlinux1.btf_ids
 fi
 
 if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
 
 vmlinux_link "${VMLINUX}"
 
Quoted 14 lines

scripts/link-vmlinux.sh fi · line 262

Quoted 11 lines
scripts/link-vmlinux.sh +2 -7
-# fill in BTF IDs
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-	info BTFIDS "${VMLINUX}"
-	RESOLVE_BTFIDS_ARGS=""
-	if is_enabled CONFIG_WERROR; then
-		RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
-	fi
-	${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+	info OBJCOPY ${btfids_vmlinux}
+	${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
 fi

Nit: Maybe do this in get-btf.sh as for modules?
To avoid checking for CONFIG_DEBUG_INFO_BTF in two places.

IIRC we can't do that, because we have to update .BTF_ids after all the linking steps. I'll double check.

Quoted 136 lines
Quoted 136 lines
scripts/link-vmlinux.sh +0 -0
 mksysmap "${VMLINUX}" System.map
tools/bpf/resolve_btfids/main.c +28 -45
index c60d303ca6ed..b8df6256e29e 100644
@@ -71,9 +71,11 @@
 #include <fcntl.h>
 #include <errno.h>
 #include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
 #include <linux/rbtree.h>
 #include <linux/zalloc.h>
 #include <linux/err.h>
+#include <linux/limits.h>
 #include <bpf/btf.h>
 #include <bpf/libbpf.h>
 #include <subcmd/parse-options.h>
@@ -124,6 +126,7 @@ struct object {
 
 	struct btf *btf;
 	struct btf *base_btf;
+	bool distilled_base;
 
 	struct {
 		int		 fd;
@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 	return btf_id;
 }
 
-/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
-#ifndef SHF_COMPRESSED
-#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */
-#endif
-
-/*
- * The data of compressed section should be aligned to 4
- * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld
- * sets sh_addralign to 1, which makes libelf fail with
- * misaligned section error during the update:
- *    FAILED elf_update(WRITE): invalid section alignment
- *
- * While waiting for ld fix, we fix the compressed sections
- * sh_addralign value manualy.
- */
-static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh)
+static void bswap_32_data(void *data, u32 nr_bytes)
 {
-	int expected = gelf_getclass(elf) == ELFCLASS32 ? 4 : 8;
+	u32 cnt, i;
+	u32 *ptr;
 
-	if (!(sh->sh_flags & SHF_COMPRESSED))
-		return 0;
+	cnt = nr_bytes / sizeof(u32);
+	ptr = data;
 
-	if (sh->sh_addralign == expected)
-		return 0;
-
-	pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n",
-		  sh->sh_addralign, expected);
-
-	sh->sh_addralign = expected;
-
-	if (gelf_update_shdr(scn, sh) == 0) {
-		pr_err("FAILED cannot update section header: %s\n",
-			elf_errmsg(-1));
-		return -1;
-	}
-	return 0;
+	for (i = 0; i < cnt; i++)
+		ptr[i] = bswap_32(ptr[i]);
 }
 
 static int elf_collect(struct object *obj)
@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)
 
 	elf_version(EV_CURRENT);
 
-	elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
+	elf = elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL);
 	if (!elf) {
 		close(fd);
 		pr_err("FAILED cannot create ELF descriptor: %s\n",
@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj)
 			obj->efile.symbols_shndx = idx;
 			obj->efile.strtabidx     = sh.sh_link;
 		} else if (!strcmp(name, BTF_IDS_SECTION)) {
+			/*
+			 * If target endianness differs from host, we need to bswap32
+			 * the .BTF_ids section data on load, because .BTF_ids has
+			 * Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in
+			 * the target endiannes. We repeat this on dump.
+			 */
+			if (obj->efile.encoding != ELFDATANATIVE) {
+				pr_debug("bswap_32 .BTF_ids data from target to host endianness\n");
+				bswap_32_data(data->d_buf, data->d_size);
+			}
 			obj->efile.idlist       = data;
 			obj->efile.idlist_shndx = idx;
 			obj->efile.idlist_addr  = sh.sh_addr;
-		} else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
-			/* If a .BTF.base section is found, do not resolve
-			 * BTF ids relative to vmlinux; resolve relative
-			 * to the .BTF.base section instead.  btf__parse_split()
-			 * will take care of this once the base BTF it is
-			 * passed is NULL.
-			 */
-			obj->base_btf_path = NULL;
 		}
-
-		if (compressed_section_fix(elf, scn, &sh))
-			return -1;
 	}
 
 	return 0;
@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)
 	return 0;
 }
 
+static inline bool is_envvar_set(const char *var_name)
+{
+	const char *value = getenv(var_name);
+
+	return value && value[0] != '\0';
+}
+
 static int load_btf(struct object *obj)
 {
 	struct btf *base_btf = NULL, *btf = NULL;
@@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
 	obj->base_btf = base_btf;
 	obj->btf = btf;
 
Quoted 4 lines

tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557

Quoted 1 line
tools/bpf/resolve_btfids/main.c +1 -0
+	if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {

This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?

Disagree. I intentionally tried to avoid adding options to resolve_btfids, because it's not intendend for general CLI usage (as opposed to pahole, for example). IMO the interface should be as simple as possible.

If we add an option, we still have to check for the env variable somewhere, and then pass the argument through. Why? Just checking an env var when it matters is simpler.

I don't think we want or expect resolve_btfids to run outside of the kernel or selftests build.

Quoted 188 lines
Quoted 188 lines
tools/bpf/resolve_btfids/main.c +89 -37
+		err = btf__distill_base(obj->btf, &base_btf, &btf);
+		if (err) {
+			pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
+			goto out_err;
+		}
+
+		btf__free(obj->btf);
+		btf__free(obj->base_btf);
+		obj->btf = btf;
+		obj->base_btf = base_btf;
+		obj->distilled_base = true;
+	}
+
 	return 0;
 
 out_err:
@@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)
 			 */
 			BUILD_BUG_ON((u32 *)set8->pairs != &set8->pairs[0].id);
 			qsort(set8->pairs, set8->cnt, sizeof(set8->pairs[0]), cmp_id);
-
-			/*
-			 * When ELF endianness does not match endianness of the
-			 * host, libelf will do the translation when updating
-			 * the ELF. This, however, corrupts SET8 flags which are
-			 * already in the target endianness. So, let's bswap
-			 * them to the host endianness and libelf will then
-			 * correctly translate everything.
-			 */
-			if (obj->efile.encoding != ELFDATANATIVE) {
-				int i;
-
-				set8->flags = bswap_32(set8->flags);
-				for (i = 0; i < set8->cnt; i++) {
-					set8->pairs[i].flags =
-						bswap_32(set8->pairs[i].flags);
-				}
-			}
 			break;
 		case BTF_ID_KIND_SYM:
 		default:
@@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)
 
 static int symbols_patch(struct object *obj)
 {
-	off_t err;
-
 	if (__symbols_patch(obj, &obj->structs)  ||
 	    __symbols_patch(obj, &obj->unions)   ||
 	    __symbols_patch(obj, &obj->typedefs) ||
@@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)
 	if (sets_patch(obj))
 		return -1;
 
-	/* Set type to ensure endian translation occurs. */
-	obj->efile.idlist->d_type = ELF_T_WORD;
+	return 0;
+}
 
-	elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+	int fd, ret;
 
-	err = elf_update(obj->efile.elf, ELF_C_WRITE);
-	if (err < 0) {
-		pr_err("FAILED elf_update(WRITE): %s\n",
-			elf_errmsg(-1));
+	fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+	if (fd < 0) {
+		pr_err("Couldn't open %s for writing\n", out_path);
+		return fd;
+	}
+
+	ret = write(fd, data, size);
+	if (ret < 0 || ret != size) {
+		pr_err("Failed to write data to %s\n", out_path);
+		close(fd);
+		unlink(out_path);
+		return -1;
+	}
+
+	close(fd);
+	pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
+
+	return 0;
+}
+
+static int dump_raw_btf_ids(struct object *obj, const char *out_path)
+{
+	Elf_Data *data = obj->efile.idlist;
+	int fd, err;
+
+	if (!data || !data->d_buf) {
+		pr_debug("%s has no BTF_ids data to dump\n", obj->path);
+		return 0;
+	}
+
+	/*
+	 * If target endianness differs from host, we need to bswap32 the
+	 * .BTF_ids section data before dumping so that the output is in
+	 * target endianness.
+	 */
+	if (obj->efile.encoding != ELFDATANATIVE) {
+		pr_debug("bswap_32 .BTF_ids data from host to target endianness\n");
+		bswap_32_data(data->d_buf, data->d_size);
+	}
+
+	err = dump_raw_data(out_path, data->d_buf, data->d_size);
+	if (err)
+		return -1;
+
+	return 0;
+}
+
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+	const void *raw_btf_data;
+	u32 raw_btf_size;
+	int fd, err;
+
+	raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+	if (raw_btf_data == NULL) {
+		pr_err("btf__raw_data() failed\n");
+		return -1;
 	}
 
-	pr_debug("update %s for %s\n",
-		 err >= 0 ? "ok" : "failed", obj->path);
-	return err < 0 ? -1 : 0;
+	err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
+	if (err)
+		return -1;
+
+	return 0;
 }
 
 static const char * const resolve_btfids_usage[] = {
@@ -824,6 +858,7 @@ int main(int argc, const char **argv)
 		.funcs    = RB_ROOT,
 		.sets     = RB_ROOT,
 	};
+	char out_path[PATH_MAX];
 	bool fatal_warnings = false;
 	struct option btfid_options[] = {
 		OPT_INCR('v', "verbose", &verbose,
@@ -836,7 +871,7 @@ int main(int argc, const char **argv)
 			    "turn warnings into errors"),
 		OPT_END()
 	};
-	int err = -1;
+	int err = -1, path_len;
 
 	argc = parse_options(argc, argv, btfid_options, resolve_btfids_usage,
 			     PARSE_OPT_STOP_AT_NON_OPTION);
@@ -844,6 +879,11 @@ int main(int argc, const char **argv)
 		usage_with_options(resolve_btfids_usage, btfid_options);
 
 	obj.path = argv[0];
+	strcpy(out_path, obj.path);
+	path_len = strlen(out_path);
+
+	if (load_btf(&obj))
+		goto out;
 
 	if (elf_collect(&obj))
 		goto out;
@@ -854,23 +894,37 @@ int main(int argc, const char **argv)
 	 */
 	if (obj.efile.idlist_shndx == -1 ||
 	    obj.efile.symbols_shndx == -1) {
-		pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
-		err = 0;
-		goto out;
+		pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+		goto dump_btf;
 	}
 
 	if (symbols_collect(&obj))
 		goto out;
 
-	if (load_btf(&obj))
-		goto out;
-
 	if (symbols_resolve(&obj))
 		goto out;
 
Quoted 7 lines

tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 911

Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2 -0
 	if (symbols_patch(&obj))
 		goto out;
 
+	out_path[path_len] = '\0';
+	strcat(out_path, ".btf_ids");

Nit: here and below use snprintf() to check for 'out_path' overflow.

Thanks for the tip. Will do.

Quoted 43 lines
Quoted 43 lines
tools/bpf/resolve_btfids/main.c +16 -0
+	if (dump_raw_btf_ids(&obj, out_path))
+		goto out;
+
+dump_btf:
+	out_path[path_len] = '\0';
+	strcat(out_path, ".btf");
+	if (dump_raw_btf(obj.btf, out_path))
+		goto out;
+
+	if (obj.base_btf && obj.distilled_base) {
+		out_path[path_len] = '\0';
+		strcat(out_path, ".distilled_base.btf");
+		if (dump_raw_btf(obj.base_btf, out_path))
+			goto out;
+	}
+
 	if (!(fatal_warnings && warnings))
 		err = 0;
 out:
tools/testing/selftests/bpf/Makefile +5 -0
index bac22265e7ff..ec7e2a7721c7 100644
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch
 include ../../../scripts/Makefile.include
 
 CXX ?= $(CROSS_COMPILE)g++
+OBJCOPY ?= $(CROSS_COMPILE)objcopy
 
 CURDIR := $(abspath .)
 TOOLSDIR := $(abspath ../../..)
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)			\
 	$$(call msg,BINARY,,$$@)
 	$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
 	$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
+	$(Q)if [ -f [email protected]_ids ]; then \
+		$(OBJCOPY) --update-section [email protected]_ids $$@; \
+	fi
+	$(Q)rm -f [email protected]_ids [email protected]
 	$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
 		   $(OUTPUT)/$(if $2,$2/)bpftool
 
maintainer
#21 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 33 lines · Ihor Solodrai
Quoted 5 lines · Eduard Zingerman
Quoted 4 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

Hi Eduard, thank you for the review.

Quoted 11 lines
Quoted 8 lines

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?

Sure, makes sense.

Quoted 10 lines
Quoted 9 lines

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

Quoted 5 lines
Quoted 2 lines

Nit: more context needed, as is the statement raises questions but not
answers them.

Would you like to see more details about why using libelf is complicated? I don't follow what's unclear here, sorry...

The claim here is: "libelf API is complicated and bug-prone ... in context of the kernel build". This is a very vague wording. The decision to rely on objcopy/linker comes from a specific needs outlined by Andrii in an off-list discussion. It will be good to have this context captured in the commit message, instead of bluntly stating that libelf is bug-prone.

Quoted 322 lines
Quoted 41 lines
Quoted 41 lines

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).

Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.

Quoted 5 lines
Quoted 3 lines

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285

Nit: these links are moving target, should refer to a commit or a tag.

Acked

Quoted 15 lines
Quoted 5 lines

[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/

Quoted 9 lines
---
 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++++++++
 scripts/link-vmlinux.sh              |  42 +-----
 tools/bpf/resolve_btfids/main.c      | 218 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 317 insertions(+), 121 deletions(-)
 create mode 100755 scripts/gen-btf.sh
Quoted 2 lines

Since resolve_btfids is now responsible for distilled base generation, does Makefile.btf need modification to remove "--btf_features=distilled_base"?

Yes, good catch.

Quoted 253 lines
Quoted 253 lines
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:	net/sched/act_bpf.c
 F:	net/sched/cls_bpf.c
 F:	samples/bpf/
 F:	scripts/bpf_doc.py
+F:	scripts/gen-btf.sh
 F:	scripts/Makefile.btf
 F:	scripts/pahole-version.sh
 F:	tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..3862fdfa1267 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko = 							\
 	if [ ! -f $(objtree)/vmlinux ]; then				\
 		printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-	else								\
-		LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-		$(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;		\
+	else	\
+		$(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
 	fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +167 -0
new file mode 100755
index 000000000000..2dfb7ab289ca
@@ -0,0 +1,167 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+#   1. pahole generates BTF from DWARF data
+#   2. resolve_btfids applies kernel-specific btf2btf
+#      transformations and computes data for .BTF_ids section
+#   3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+#   - ${1}.btf.o ready for linking into vmlinux
+#   - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+	echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+	exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+	case "$1" in
+	--btf_base)
+		BTF_BASE="$2"
+		shift 2
+		;;
+	-*)
+		echo "Unknown option: $1" >&2
+		usage
+		;;
+	*)
+		break
+		;;
+	esac
+done
+
+if [ $# -ne 1 ]; then
+	usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+	grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+	printf "  %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+	set -x
+	;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+	exit 0
+fi
+
+gen_btf_data()
+{
+	info BTF "${ELF_FILE}"
+	btf1="${ELF_FILE}.btf.1"
+	${PAHOLE} -J ${PAHOLE_FLAGS}			\
+		${BTF_BASE:+--btf_base ${BTF_BASE}}	\
+		--btf_encode_detached=${btf1}		\
+		"${ELF_FILE}"
+
+	info BTFIDS "${ELF_FILE}"
+	RESOLVE_BTFIDS_OPTS=""
+	if is_enabled CONFIG_WERROR; then
+		RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+	fi
+	if [ -n "${KBUILD_VERBOSE}" ]; then
+		RESOLVE_BTFIDS_OPTS+=" -v "
+	fi
+	${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS}	\
+		${BTF_BASE:+--btf_base ${BTF_BASE}}	\
+		--btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+	local btf_data=${ELF_FILE}.btf.o
+
+	# Create ${btf_data} which contains just .BTF section but no symbols. Add
+	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+	# deletes all symbols including __start_BTF and __stop_BTF, which will
+	# be redefined in the linker script.
+	info OBJCOPY "${btf_data}"
+	echo "" | ${CC} -c -x c -o ${btf_data} -
+	${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+		--set-section-flags .BTF=alloc,readonly ${btf_data}
+	${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+	# Change e_type to ET_REL so that it can be used to link final vmlinux.
+	# GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+	if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+		et_rel='\0\1'
+	else
+		et_rel='\1\0'
+	fi
+	printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+	info OBJCOPY "${ELF_FILE}"
+	${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}
+
+	# a module might not have a .BTF_ids or .BTF.base section
+	local btf_base="${ELF_FILE}.distilled_base.btf"
+	if [ -f "${btf_base}" ]; then
+		${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE}
+	fi
+	local btf_ids="${ELF_FILE}.btf_ids"
+	if [ -f "${btf_ids}" ]; then
+		${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE}
+	fi
+}
+
+cleanup()
+{
+	rm -f "${ELF_FILE}.btf.1"
+	rm -f "${ELF_FILE}.btf"
+	if [ "${BTFGEN_MODE}" = "module" ]; then
+		rm -f "${ELF_FILE}.distilled_base.btf"
+		rm -f "${ELF_FILE}.btf_ids"
+	fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+	BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+	gen_btf_o
+	;;
+module)
+	embed_btf_data
+	;;
+esac
+
+exit 0
scripts/link-vmlinux.sh +1 -28
index 433849ff7529..5bea8795f96d 100755
@@ -105,34 +105,6 @@ vmlinux_link()
 		${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
 }
 
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
-	local btf_data=${1}.btf.o
-
-	info BTF "${btf_data}"
-	LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
-	# Create ${btf_data} which contains just .BTF section but no symbols. Add
-	# SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
-	# deletes all symbols including __start_BTF and __stop_BTF, which will
-	# be redefined in the linker script. Add 2>/dev/null to suppress GNU
-	# objcopy warnings: "empty loadable segment detected at ..."
-	${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
-		--strip-all ${1} "${btf_data}" 2>/dev/null
-	# Change e_type to ET_REL so that it can be used to link final vmlinux.
-	# GNU ld 2.35+ and lld do not allow an ET_EXEC input.
-	if is_enabled CONFIG_CPU_BIG_ENDIAN; then
-		et_rel='\0\1'
-	else
-		et_rel='\1\0'
-	fi
-	printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
-	btf_vmlinux_bin_o=${btf_data}
-}
-
 # Create ${2}.o file with all symbols from the ${1} object file
 kallsyms()
 {
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
 fi
 
 btf_vmlinux_bin_o=
+btfids_vmlinux=
 kallsymso=
 strip_debug=
 generate_map=
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
Quoted 8 lines
Quoted 5 lines

scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200

Quoted 3 lines
scripts/link-vmlinux.sh +1 -1
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-	if ! gen_btf .tmp_vmlinux1; then
+	if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then

Nit: maybe pass output file names as parameters for get-btf.sh?

I don't see a point in that. The script and callsite will become more complicated, but what is the benefit?

In order to avoid implicit naming conventions. Hence, the reader of the script code has clear understanding about in and out parameters.

Quoted 178 lines
Quoted 14 lines
Quoted 14 lines
scripts/link-vmlinux.sh +2 -0
 		echo >&2 "Failed to generate BTF for vmlinux"
 		echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
 		exit 1
 	fi
+	btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+	btfids_vmlinux=.tmp_vmlinux1.btf_ids
 fi
 
 if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
 
 vmlinux_link "${VMLINUX}"
 
Quoted 14 lines

scripts/link-vmlinux.sh fi · line 262

Quoted 11 lines
scripts/link-vmlinux.sh +2 -7
-# fill in BTF IDs
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-	info BTFIDS "${VMLINUX}"
-	RESOLVE_BTFIDS_ARGS=""
-	if is_enabled CONFIG_WERROR; then
-		RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
-	fi
-	${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+	info OBJCOPY ${btfids_vmlinux}
+	${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
 fi

Nit: Maybe do this in get-btf.sh as for modules?
To avoid checking for CONFIG_DEBUG_INFO_BTF in two places.

IIRC we can't do that, because we have to update .BTF_ids after all the linking steps. I'll double check.

Quoted 136 lines
Quoted 136 lines
scripts/link-vmlinux.sh +0 -0
 mksysmap "${VMLINUX}" System.map
tools/bpf/resolve_btfids/main.c +28 -45
index c60d303ca6ed..b8df6256e29e 100644
@@ -71,9 +71,11 @@
 #include <fcntl.h>
 #include <errno.h>
 #include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
 #include <linux/rbtree.h>
 #include <linux/zalloc.h>
 #include <linux/err.h>
+#include <linux/limits.h>
 #include <bpf/btf.h>
 #include <bpf/libbpf.h>
 #include <subcmd/parse-options.h>
@@ -124,6 +126,7 @@ struct object {
 
 	struct btf *btf;
 	struct btf *base_btf;
+	bool distilled_base;
 
 	struct {
 		int		 fd;
@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 	return btf_id;
 }
 
-/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
-#ifndef SHF_COMPRESSED
-#define SHF_COMPRESSED (1 << 11) /* Section with compressed data. */
-#endif
-
-/*
- * The data of compressed section should be aligned to 4
- * (for 32bit) or 8 (for 64 bit) bytes. The binutils ld
- * sets sh_addralign to 1, which makes libelf fail with
- * misaligned section error during the update:
- *    FAILED elf_update(WRITE): invalid section alignment
- *
- * While waiting for ld fix, we fix the compressed sections
- * sh_addralign value manualy.
- */
-static int compressed_section_fix(Elf *elf, Elf_Scn *scn, GElf_Shdr *sh)
+static void bswap_32_data(void *data, u32 nr_bytes)
 {
-	int expected = gelf_getclass(elf) == ELFCLASS32 ? 4 : 8;
+	u32 cnt, i;
+	u32 *ptr;
 
-	if (!(sh->sh_flags & SHF_COMPRESSED))
-		return 0;
+	cnt = nr_bytes / sizeof(u32);
+	ptr = data;
 
-	if (sh->sh_addralign == expected)
-		return 0;
-
-	pr_debug2(" - fixing wrong alignment sh_addralign %u, expected %u\n",
-		  sh->sh_addralign, expected);
-
-	sh->sh_addralign = expected;
-
-	if (gelf_update_shdr(scn, sh) == 0) {
-		pr_err("FAILED cannot update section header: %s\n",
-			elf_errmsg(-1));
-		return -1;
-	}
-	return 0;
+	for (i = 0; i < cnt; i++)
+		ptr[i] = bswap_32(ptr[i]);
 }
 
 static int elf_collect(struct object *obj)
@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)
 
 	elf_version(EV_CURRENT);
 
-	elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
+	elf = elf_begin(fd, ELF_C_READ_MMAP_PRIVATE, NULL);
 	if (!elf) {
 		close(fd);
 		pr_err("FAILED cannot create ELF descriptor: %s\n",
@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj)
 			obj->efile.symbols_shndx = idx;
 			obj->efile.strtabidx     = sh.sh_link;
 		} else if (!strcmp(name, BTF_IDS_SECTION)) {
+			/*
+			 * If target endianness differs from host, we need to bswap32
+			 * the .BTF_ids section data on load, because .BTF_ids has
+			 * Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in
+			 * the target endiannes. We repeat this on dump.
+			 */
+			if (obj->efile.encoding != ELFDATANATIVE) {
+				pr_debug("bswap_32 .BTF_ids data from target to host endianness\n");
+				bswap_32_data(data->d_buf, data->d_size);
+			}
 			obj->efile.idlist       = data;
 			obj->efile.idlist_shndx = idx;
 			obj->efile.idlist_addr  = sh.sh_addr;
-		} else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
-			/* If a .BTF.base section is found, do not resolve
-			 * BTF ids relative to vmlinux; resolve relative
-			 * to the .BTF.base section instead.  btf__parse_split()
-			 * will take care of this once the base BTF it is
-			 * passed is NULL.
-			 */
-			obj->base_btf_path = NULL;
 		}
-
-		if (compressed_section_fix(elf, scn, &sh))
-			return -1;
 	}
 
 	return 0;
@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)
 	return 0;
 }
 
+static inline bool is_envvar_set(const char *var_name)
+{
+	const char *value = getenv(var_name);
+
+	return value && value[0] != '\0';
+}
+
 static int load_btf(struct object *obj)
 {
 	struct btf *base_btf = NULL, *btf = NULL;
@@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
 	obj->base_btf = base_btf;
 	obj->btf = btf;
 
Quoted 4 lines

tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557

Quoted 1 line
tools/bpf/resolve_btfids/main.c +1 -0
+	if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {

This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?

Disagree. I intentionally tried to avoid adding options to resolve_btfids, because it's not intendend for general CLI usage (as opposed to pahole, for example). IMO the interface should be as simple as possible.

Quoted 6 lines

If we add an option, we still have to check for the env variable somewhere, and then pass the argument through. Why? Just checking an env var when it matters is simpler.

I don't think we want or expect resolve_btfids to run outside of the kernel or selftests build.

This comes to personal opinion, of-course.
So, in my personal opinion, obfuscating a command line tool interface with it being parameterized by both environment variables and command line parameters is rarely justified. In this particular case it will only make life a tad harder for someone debugging resolve_btfids by copy-pasting command from make output.

Hence, I find this piece of code ugly.

[...]

author reviewer
#22 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 40 lines · Eduard Zingerman
Quoted 33 lines · Ihor Solodrai
Quoted 5 lines · Eduard Zingerman
Quoted 4 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

Hi Eduard, thank you for the review.

Quoted 11 lines
Quoted 8 lines

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?

Sure, makes sense.

Quoted 10 lines
Quoted 9 lines

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

Quoted 5 lines
Quoted 2 lines

Nit: more context needed, as is the statement raises questions but not
answers them.

Would you like to see more details about why using libelf is complicated? I don't follow what's unclear here, sorry...

Quoted 6 lines

The claim here is: "libelf API is complicated and bug-prone ... in context of the kernel build". This is a very vague wording. The decision to rely on objcopy/linker comes from a specific needs outlined by Andrii in an off-list discussion. It will be good to have this context captured in the commit message, instead of bluntly stating that libelf is bug-prone.

Ok, it seems you're conflating two separate issues.

There is a requirement to *link* .BTF section into vmlinux, because it must have a SHF_ALLOC flag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.

The patch doesn't change this behavior, and this was (and is) covered in the script comments.

A separate issue is what resolve_btfids does: updates ELF in-place (before the patch) or outputs detached section data (after patch).

The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.

And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes resolve_btfids buggy.

Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 14 lines

[...]
# Create ${2}.o file with all symbols from the ${1} object file kallsyms()
{

scripts/link-vmlinux.sh +1 -0
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
 fi
 
 btf_vmlinux_bin_o=
+btfids_vmlinux=
 kallsymso=
 strip_debug=
 generate_map=

scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200

Quoted 5 lines
scripts/link-vmlinux.sh +1 -1
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-	if ! gen_btf .tmp_vmlinux1; then
+	if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then

Nit: maybe pass output file names as parameters for get-btf.sh?

Quoted 5 lines
Quoted 2 lines

I don't see a point in that. The script and callsite will become more complicated, but what is the benefit?

In order to avoid implicit naming conventions. Hence, the reader of the script code has clear understanding about in and out parameters.

I think implicit naming convention makes sense for this script. The script's top comment describes what it does in detail, including the output naming.

Quoted 44 lines
Quoted 27 lines
Quoted 17 lines

[...]

Quoted 15 lines
tools/bpf/resolve_btfids/main.c +7 -0
+static inline bool is_envvar_set(const char *var_name)
+{
+	const char *value = getenv(var_name);
+
+	return value && value[0] != '\0';
+}
+
 static int load_btf(struct object *obj)
 {
 	struct btf *base_btf = NULL, *btf = NULL;
@@ -571,6 +554,20 @@ static int load_btf(struct object *obj)
 	obj->base_btf = base_btf;
 	obj->btf = btf;
 
Quoted 4 lines

tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557

Quoted 1 line
tools/bpf/resolve_btfids/main.c +1 -0
+	if (obj->base_btf && is_envvar_set("KBUILD_EXTMOD")) {

This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?

Disagree. I intentionally tried to avoid adding options to resolve_btfids, because it's not intendend for general CLI usage (as opposed to pahole, for example). IMO the interface should be as simple as possible.

Quoted 6 lines

If we add an option, we still have to check for the env variable somewhere, and then pass the argument through. Why? Just checking an env var when it matters is simpler.

I don't think we want or expect resolve_btfids to run outside of the kernel or selftests build.

This comes to personal opinion, of-course.
So, in my personal opinion, obfuscating a command line tool interface with it being parameterized by both environment variables and command line parameters is rarely justified. In this particular case it will only make life a tad harder for someone debugging resolve_btfids by copy-pasting command from make output.

Hence, I find this piece of code ugly.

[...]

maintainer
#23 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

On Thu, 2025-12-04 at 09:29 -0800, Ihor Solodrai wrote:

[...]

Quoted 12 lines

Ok, it seems you're conflating two separate issues.

There is a requirement to *link* .BTF section into vmlinux, because it must have a SHF_ALLOC flag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.

The patch doesn't change this behavior, and this was (and is) covered in the script comments.

A separate issue is what resolve_btfids does: updates ELF in-place (before the patch) or outputs detached section data (after patch).

Quoted 6 lines

The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.

And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes resolve_btfids buggy.

So, pahole does the following:
- elf_begin(fildes: fd, cmd: ELF_C_RDWR, ref: NULL);
- selects a section to modify and modifies it
- elf_flagdata(data: btf_data, cmd: ELF_C_SET, flags: ELF_F_DIRTY);
- elf_update(elf, cmd: ELF_C_WRITE)
- elf_end(elf)

What exactly is complicated about that?

[...]

author reviewer
#24 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 23 lines · Eduard Zingerman

On Thu, 2025-12-04 at 09:29 -0800, Ihor Solodrai wrote:

[...]

Quoted 12 lines

Ok, it seems you're conflating two separate issues.

There is a requirement to *link* .BTF section into vmlinux, because it must have a SHF_ALLOC flag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.

The patch doesn't change this behavior, and this was (and is) covered in the script comments.

A separate issue is what resolve_btfids does: updates ELF in-place (before the patch) or outputs detached section data (after patch).

Quoted 6 lines

The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.

And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes resolve_btfids buggy.

Quoted 8 lines

So, pahole does the following:
- elf_begin(fildes: fd, cmd: ELF_C_RDWR, ref: NULL);
- selects a section to modify and modifies it
- elf_flagdata(data: btf_data, cmd: ELF_C_SET, flags: ELF_F_DIRTY);
- elf_update(elf, cmd: ELF_C_WRITE)
- elf_end(elf)

What exactly is complicated about that?

Take a look at the resolve_btfids code that is removed in this patch, as a consequence of switching to read-only ELF.

Also consider that before these changes resolve_btfids had a simple
job: update data buffer of a single section, importantly, without changing its size.

Now let's say we keep "update in-place" approach (which I tried to do, btw). In addition to previous .BTF_ids data update, resolve_btfids may need to either add or update .BTF section changing its size (triggering reorg of sections in ELF, depending on the flags) and add .BTF.base section. There is also a question of how to do it: do we elf_update() multiple times or try to "batch" the updates?

All of this is possible, but the alternative is much simpler:

    ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}

Why re-implement our own incomplete version of objcopy if we can just use it to deal with the details of the ELF update?

Note also that even in pahole "add .BTF section" is implemented via llvm-objcopy call. My guess is: to avoid the headache of figuring out correct libelf usage.

Quoted 2 lines

[...]

maintainer
#25 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 49 lines · Ihor Solodrai
Quoted 23 lines · Eduard Zingerman

On Thu, 2025-12-04 at 09:29 -0800, Ihor Solodrai wrote:

[...]

Quoted 12 lines

Ok, it seems you're conflating two separate issues.

There is a requirement to *link* .BTF section into vmlinux, because it must have a SHF_ALLOC flag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.

The patch doesn't change this behavior, and this was (and is) covered in the script comments.

A separate issue is what resolve_btfids does: updates ELF in-place (before the patch) or outputs detached section data (after patch).

Quoted 6 lines

The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.

And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes resolve_btfids buggy.

Quoted 8 lines

So, pahole does the following:
- elf_begin(fildes: fd, cmd: ELF_C_RDWR, ref: NULL);
- selects a section to modify and modifies it
- elf_flagdata(data: btf_data, cmd: ELF_C_SET, flags: ELF_F_DIRTY);
- elf_update(elf, cmd: ELF_C_WRITE)
- elf_end(elf)

What exactly is complicated about that?

Take a look at the resolve_btfids code that is removed in this patch, as a consequence of switching to read-only ELF.

Also consider that before these changes resolve_btfids had a simple
job: update data buffer of a single section, importantly, without changing its size.

Now let's say we keep "update in-place" approach (which I tried to do, btw). In addition to previous .BTF_ids data update, resolve_btfids may need to either add or update .BTF section changing its size (triggering reorg of sections in ELF, depending on the flags) and add .BTF.base section. There is also a question of how to do it: do we elf_update() multiple times or try to "batch" the updates?

All of this is possible, but the alternative is much simpler:

Quoted 8 lines
    ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}

Why re-implement our own incomplete version of objcopy if we can just use it to deal with the details of the ELF update?

Note also that even in pahole "add .BTF section" is implemented via llvm-objcopy call. My guess is: to avoid the headache of figuring out correct libelf usage.

Please put this motivation in the commit message.

#26 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

In reply to Ihor Solodrai
Quoted 107 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).

Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/

---
 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++++++++
 scripts/link-vmlinux.sh              |  42 +-----
 tools/bpf/resolve_btfids/main.c      | 218 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 317 insertions(+), 121 deletions(-)
 create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:        net/sched/act_bpf.c
 F:     net/sched/cls_bpf.c
 F:     samples/bpf/
 F:     scripts/bpf_doc.py
+F:     scripts/gen-btf.sh
 F:     scripts/Makefile.btf
 F:     scripts/pahole-version.sh
 F:     tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..3862fdfa1267 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko =                                                     \
        if [ ! -f $(objtree)/vmlinux ]; then                            \
                printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-       else                                                            \
-               LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-               $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;             \
+       else    \
+               $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
        fi;
 

[...]

scripts/Makefile.modfinal

Quoted 11 lines
scripts/Makefile.modfinal +11 -0
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+       exit 0
+fi
+
+gen_btf_data()
+{
+       info BTF "${ELF_FILE}"
+       btf1="${ELF_FILE}.btf.1"
+       ${PAHOLE} -J ${PAHOLE_FLAGS}                    \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf_encode_detached=${btf1}           \

please double-check what pahole version has --btf_encode_detached, we might need to change minimal supported pahole version because of this

pw-bot: cr

Quoted 15 lines
scripts/Makefile.modfinal +15 -0
+               "${ELF_FILE}"
+
+       info BTFIDS "${ELF_FILE}"
+       RESOLVE_BTFIDS_OPTS=""
+       if is_enabled CONFIG_WERROR; then
+               RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+       fi
+       if [ -n "${KBUILD_VERBOSE}" ]; then
+               RESOLVE_BTFIDS_OPTS+=" -v "
+       fi
+       ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS}        \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf ${btf1} "${ELF_FILE}"
+}
+

[...]

Quoted 6 lines
scripts/Makefile.modfinal +3 -2
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+       int fd, ret;
 
-       err = elf_update(obj->efile.elf, ELF_C_WRITE);
-       if (err < 0) {

scripts/Makefile.modfinal

Quoted 10 lines
scripts/Makefile.modfinal +8 -2
-               pr_err("FAILED elf_update(WRITE): %s\n",
-                       elf_errmsg(-1));
+       fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+       if (fd < 0) {
+               pr_err("Couldn't open %s for writing\n", out_path);
+               return fd;
+       }
+
+       ret = write(fd, data, size);
+       if (ret < 0 || ret != size) {

use fopen() and fwrite() instead of low-level syscalls? for write() it's "expected" that it might be interrupted and not complete a full write, so you'd need to handle that in a loop properly. With fwrite() I think all this is handled internally, so I'd stick to fopen()'s FILE abstraction and fwrite().

Quoted 5 lines
scripts/Makefile.modfinal +5 -0
+               pr_err("Failed to write data to %s\n", out_path);
+               close(fd);
+               unlink(out_path);
+               return -1;
+       }

[...]

scripts/Makefile.modfinal

Quoted 8 lines
scripts/Makefile.modfinal +8 -0
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+       const void *raw_btf_data;
+       u32 raw_btf_size;
+       int fd, err;
+
+       raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+       if (raw_btf_data == NULL) {

nit: !raw_btf_data, it's C

[...]

scripts/Makefile.modfinal int main(int argc, const char **argv) · line 883

Quoted 6 lines
scripts/Makefile.modfinal +2 -0
@@ -844,6 +879,11 @@ int main(int argc, const char **argv)
                usage_with_options(resolve_btfids_usage, btfid_options);
 
        obj.path = argv[0];
+       strcpy(out_path, obj.path);
+       path_len = strlen(out_path);

Eduard already suggested using snprintf() later in the code, I'd say use snprintf() here as well instead of strcpy(). When working with fixed-sized buffers, snprintf() is the most ergonomic way to deal with that and not trigger unnecessary compiler warnings about possible truncations, out of buffer writes, and stuff like that.

Quoted 6 lines
scripts/Makefile.modfinal +3 -0
+
+       if (load_btf(&obj))
+               goto out;
 
        if (elf_collect(&obj))
                goto out;

[...]

#27 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 111 lines · Andrii Nakryiko
Quoted 107 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).

Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/

---
 MAINTAINERS                          |   1 +
 scripts/Makefile.modfinal            |   5 +-
 scripts/gen-btf.sh                   | 167 ++++++++++++++++++++
 scripts/link-vmlinux.sh              |  42 +-----
 tools/bpf/resolve_btfids/main.c      | 218 +++++++++++++++++----------
 tools/testing/selftests/bpf/Makefile |   5 +
 6 files changed, 317 insertions(+), 121 deletions(-)
 create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:        net/sched/act_bpf.c
 F:     net/sched/cls_bpf.c
 F:     samples/bpf/
 F:     scripts/bpf_doc.py
+F:     scripts/gen-btf.sh
 F:     scripts/Makefile.btf
 F:     scripts/pahole-version.sh
 F:     tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..3862fdfa1267 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko =                                                     \
        if [ ! -f $(objtree)/vmlinux ]; then                            \
                printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-       else                                                            \
-               LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-               $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;             \
+       else    \
+               $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
        fi;
 

[...]

Quoted 15 lines

scripts/Makefile.modfinal

Quoted 11 lines
scripts/Makefile.modfinal +11 -0
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+       exit 0
+fi
+
+gen_btf_data()
+{
+       info BTF "${ELF_FILE}"
+       btf1="${ELF_FILE}.btf.1"
+       ${PAHOLE} -J ${PAHOLE_FLAGS}                    \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf_encode_detached=${btf1}           \

please double-check what pahole version has --btf_encode_detached, we might need to change minimal supported pahole version because of this

yeah, this landed in v1.22 [1]

One thing worth thinking about; are there aspects of the gen_btf.sh script that could be moved to Makefile.btf to avoid having to compute them repeatedly for each module? For example computing resolve_btfids flags based on CONFIG_WERROR could be done there I think. You could also determine whether the script is needed at all in Makefile.btf; i.e.

gen-btf-y =
gen-btf-$(CONFIG_DEBUG_INFO_BTF) = scripts/gen-btf.sh

export GEN_BTF := $(gen-btf-y)

That would allow you to get rid of the is_enabled() I think.

I'm building this now, but I was wondering if the linking/objcopy changes pose any risk to kernel address computations in kallsyms or anything like that? IIRC Stephen ran into some issues with global variable addresses as a consequence of linking BTF sections [2], but not sure if there are additional concerns here.

[1] https://github.com/acmel/dwarves/releases/tag/v1.22
[2] https://lore.kernel.org/bpf/[email protected]/

author reviewer
#28 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 21 lines · Alan Maguire
Quoted 4 lines · Andrii Nakryiko

On Thu, Nov 27, 2025 at 10:53 AM Ihor Solodrai <[email protected]> wrote:

[...]

Quoted 15 lines

scripts/gen-btf.sh line 84

Quoted 11 lines
scripts/gen-btf.sh new file +11 -0
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+       exit 0
+fi
+
+gen_btf_data()
+{
+       info BTF "${ELF_FILE}"
+       btf1="${ELF_FILE}.btf.1"
+       ${PAHOLE} -J ${PAHOLE_FLAGS}                    \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf_encode_detached=${btf1}           \

please double-check what pahole version has --btf_encode_detached, we might need to change minimal supported pahole version because of this

Quoted 1 line

yeah, this landed in v1.22 [1]

Thank you for checking!

Quoted 7 lines

One thing worth thinking about; are there aspects of the gen_btf.sh script that could be moved to Makefile.btf to avoid having to compute them repeatedly for each module? For example computing resolve_btfids flags based on CONFIG_WERROR could be done there I think. You could also determine whether the script is needed at all in Makefile.btf; i.e.

Quoted 6 lines

gen-btf-y =
gen-btf-$(CONFIG_DEBUG_INFO_BTF) = scripts/gen-btf.sh

export GEN_BTF := $(gen-btf-y)

That would allow you to get rid of the is_enabled() I think.

Good point. I'll try moving most relevant flags to Makefile.btf

Quoted 5 lines

I'm building this now, but I was wondering if the linking/objcopy changes pose any risk to kernel address computations in kallsyms or anything like that? IIRC Stephen ran into some issues with global variable addresses as a consequence of linking BTF sections [2], but not sure if there are additional concerns here.

This series doesn't change the fact that .BTF is *linked* into final vmlinux, so a problem described in [2] stands.

That said, AFAIU the suggested change in the linker script will still work.

Quoted 3 lines

[1] https://github.com/acmel/dwarves/releases/tag/v1.22
[2] https://lore.kernel.org/bpf/[email protected]/

#29 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

In reply to Ihor Solodrai
kernel test robot noticed the following build errors:

Hi Ihor,

kernel test robot noticed the following build errors:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Ihor-Solodrai/resolve_btfids-rename-object-btf-field-to-btf_path/20251128-025645
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20251127185242.3954132-5-ihor.solodrai%40linux.dev
patch subject: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
config: arm64-randconfig-004-20251205 (https://download.01.org/0day-ci/archive/20251206/[email protected]/config)
compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 14bf95b06a18b9b59c89601cbc0e5a6f2176b118)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251206/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> ld.lld: error: .tmp_vmlinux1.btf.o is incompatible with aarch64elf
Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author reviewer
#30 lore raw

Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 14 lines · kernel test robot

Hi Ihor,

kernel test robot noticed the following build errors:

[auto build test ERROR on bpf-next/master]

url:    https://github.com/intel-lab-lkp/linux/commits/Ihor-Solodrai/resolve_btfids-rename-object-btf-field-to-btf_path/20251128-025645
base:   https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master
patch link:    https://lore.kernel.org/r/20251127185242.3954132-5-ihor.solodrai%40linux.dev
patch subject: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
config: arm64-randconfig-004-20251205 (https://download.01.org/0day-ci/archive/20251206/[email protected]/config)
compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 14bf95b06a18b9b59c89601cbc0e5a6f2176b118)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251206/[email protected]/reproduce)
Quoted 9 lines

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

Quoted 1 line
Quoted 1 line

ld.lld: error: .tmp_vmlinux1.btf.o is incompatible with aarch64elf

I was able to reproduce this error.

It happens only when cross-compiling with LLVM (which we don't do on BPF CI, btw) because of this line in gen-btf.sh:

	echo "" | ${CC} -c -x c -o ${btf_data} -

The purpose of this command is to produce an "empty" linkable ELF file. The .BTF section is objcopied into it, and then it's linked into the vmlinux.

Before the changes in this patch, the "empty" ELF was produced with objcopy --strip-all on .tmp_vmlinux1, which is a slower operation.

ld.lld fails, because ${CC} without the flags is just clang in this case, and it emits an ELF for host arch, which of course can't be linked. It can be fixed with:

	echo "" | ${CC} ${CLANG_FLAGS} -c -x c -o ${btf_data} -

${CLANG_FLAGS} contains a correct clang --target when cross-compiling.

I'll use this in the upcoming v4 of the series.