Skip to contentSkip to conversation

resolve_btfids: Support for BTF modifications

PATCH Superseded Ihor Solodrai 4 participants 16 messages Nov 26, 2025 – Dec 2, 2025

Subsystems (2)

v1 of 7 Superseded Latest is v7 →

lore
Share & tools
lore

Ihor Solodrai

author reviewer
#0 lore raw

cover resolve_btfids: Support for BTF modifications

This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.

Patches #1-#3 in the series are non-functional refactoring in resolve_btfids. The last patch (#4) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. Implementation changes are described in detail in the patch description.

One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)

Since BTF finalization is now done by resolve_btfids, it requires btf__distill_base() to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag for resolve_btfids.

Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.

However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessary btf__relocate() call on load [6]. It can be avoided by special-casing in-tree module handling in resolve_btfids either with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.

[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358

Ihor Solodrai (4):
resolve_btfids: rename object btf field to btf_path
resolve_btfids: factor out load_btf()
resolve_btfids: introduce enum btf_id_kind
resolve_btfids: change in-place update with raw binary output

 MAINTAINERS                     |   1 +
 scripts/Makefile.modfinal       |   5 +-
 scripts/gen-btf.sh              | 166 ++++++++++++++++++++++
 scripts/link-vmlinux.sh         |  42 +-----
 tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++---------
 5 files changed, 348 insertions(+), 100 deletions(-)
 create mode 100755 scripts/gen-btf.sh
Signature · 1 line
2.52.0
author reviewer
#1 lore raw

PATCH 1/4 resolve_btfids: rename object btf field to btf_path

Rename the member of struct object holding the path to BTF data if provided via --btf arg. btf_path is less ambiguous.

---
 tools/bpf/resolve_btfids/main.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tools/bpf/resolve_btfids/main.c +4 -4
index d47191c6e55e..d2762487ce5f 100644
@@ -113,7 +113,7 @@ struct btf_id {
 
 struct object {
 	const char *path;
-	const char *btf;
+	const char *btf_path;
 	const char *base_btf_path;
 
 	struct {
@@ -550,11 +550,11 @@ static int symbols_resolve(struct object *obj)
 		}
 	}
 
-	btf = btf__parse_split(obj->btf ?: obj->path, base_btf);
+	btf = btf__parse_split(obj->btf_path ?: obj->path, base_btf);
 	err = libbpf_get_error(btf);
 	if (err) {
 		pr_err("FAILED: load BTF from %s: %s\n",
-			obj->btf ?: obj->path, strerror(-err));
+			obj->btf_path ?: obj->path, strerror(-err));
 		goto out;
 	}
 
@@ -790,7 +790,7 @@ int main(int argc, const char **argv)
 	struct option btfid_options[] = {
 		OPT_INCR('v', "verbose", &verbose,
 			 "be more verbose (show errors, etc)"),
-		OPT_STRING(0, "btf", &obj.btf, "BTF data",
+		OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
 			   "BTF data"),
 		OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
 			   "path of file providing base BTF"),
Signature · 1 line
2.52.0
author reviewer
#2 lore raw

PATCH 2/4 resolve_btfids: factor out load_btf()

In reply to Ihor Solodrai

Increase the lifetime of parsed BTF in resolve_btfids by factoring load_btf() routine out of symbols_resolve() and storing the base_btf and btf pointers in the struct object.

---
 tools/bpf/resolve_btfids/main.c | 47 ++++++++++++++++++++++++---------
 1 file changed, 34 insertions(+), 13 deletions(-)
tools/bpf/resolve_btfids/main.c +34 -13
index d2762487ce5f..b7b44e72e765 100644
@@ -116,6 +116,9 @@ struct object {
 	const char *btf_path;
 	const char *base_btf_path;
 
+	struct btf *btf;
+	struct btf *base_btf;
+
 	struct {
 		int		 fd;
 		Elf		*elf;
@@ -529,16 +532,10 @@ static int symbols_collect(struct object *obj)
 	return 0;
 }
 
-static int symbols_resolve(struct object *obj)
+static int load_btf(struct object *obj)
 {
-	int nr_typedefs = obj->nr_typedefs;
-	int nr_structs  = obj->nr_structs;
-	int nr_unions   = obj->nr_unions;
-	int nr_funcs    = obj->nr_funcs;
-	struct btf *base_btf = NULL;
-	int err, type_id;
-	struct btf *btf;
-	__u32 nr_types;
+	struct btf *base_btf = NULL, *btf = NULL;
+	int err;
 
 	if (obj->base_btf_path) {
 		base_btf = btf__parse(obj->base_btf_path, NULL);
@@ -546,7 +543,7 @@ static int symbols_resolve(struct object *obj)
 		if (err) {
 			pr_err("FAILED: load base BTF from %s: %s\n",
 			       obj->base_btf_path, strerror(-err));
-			return -1;
+			goto out_err;
 		}
 	}
 
@@ -555,9 +552,30 @@ static int symbols_resolve(struct object *obj)
 	if (err) {
 		pr_err("FAILED: load BTF from %s: %s\n",
 			obj->btf_path ?: obj->path, strerror(-err));
-		goto out;
+		goto out_err;
 	}
 
+	obj->base_btf = base_btf;
+	obj->btf = btf;
+
+	return 0;
+
+out_err:
+	btf__free(base_btf);
+	btf__free(btf);
+	return err;
+}
+
+static int symbols_resolve(struct object *obj)
+{
+	int nr_typedefs = obj->nr_typedefs;
+	int nr_structs  = obj->nr_structs;
+	int nr_unions   = obj->nr_unions;
+	int nr_funcs    = obj->nr_funcs;
+	struct btf *btf = obj->btf;
+	int err, type_id;
+	__u32 nr_types;
+
 	err = -1;
 	nr_types = btf__type_cnt(btf);
 
@@ -615,8 +633,6 @@ static int symbols_resolve(struct object *obj)
 
 	err = 0;
 out:
-	btf__free(base_btf);
-	btf__free(btf);
 	return err;
 }
 
@@ -824,6 +840,9 @@ int main(int argc, const char **argv)
 	if (symbols_collect(&obj))
 		goto out;
 
+	if (load_btf(&obj))
+		goto out;
+
 	if (symbols_resolve(&obj))
 		goto out;
 
@@ -833,6 +852,8 @@ int main(int argc, const char **argv)
 	if (!(fatal_warnings && warnings))
 		err = 0;
 out:
+	btf__free(obj.base_btf);
+	btf__free(obj.btf);
 	if (obj.efile.elf) {
 		elf_end(obj.efile.elf);
 		close(obj.efile.fd);
Signature · 1 line
2.52.0
author reviewer
#3 lore raw

PATCH 3/4 resolve_btfids: introduce enum btf_id_kind

In reply to Ihor Solodrai

Instead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.

---
 tools/bpf/resolve_btfids/main.c | 61 ++++++++++++++++++++++-----------
 1 file changed, 41 insertions(+), 20 deletions(-)
tools/bpf/resolve_btfids/main.c +41 -20
index b7b44e72e765..7f5a9f7dde7f 100644
@@ -98,6 +98,13 @@
 # error "Unknown machine endianness!"
 #endif
 
+enum btf_id_kind {
+	BTF_ID_KIND_NONE,
+	BTF_ID_KIND_SYM,
+	BTF_ID_KIND_SET,
+	BTF_ID_KIND_SET8
+};
+
 struct btf_id {
 	struct rb_node	 rb_node;
 	char		*name;
@@ -105,9 +112,8 @@ struct btf_id {
 		int	 id;
 		int	 cnt;
 	};
-	int		 addr_cnt;
-	bool		 is_set;
-	bool		 is_set8;
+	enum btf_id_kind kind:8;
+	int		 addr_cnt:8;
 	Elf64_Addr	 addr[ADDR_CNT];
 };
 
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
 	return id;
 }
 
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
 	/*
 	 * __BTF_ID__set__name
 	 * name =    ^
 	 * id   =         ^
 	 */
-	char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+	int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+	char *id = name + prefixlen - 1;
 	int len = strlen(name);
+	struct btf_id *btf_id;
 
 	if (id >= name + len) {
 		pr_err("FAILED to parse set name: %s\n", name);
 		return NULL;
 	}
 
-	return btf_id__add(&obj->sets, id, true);
+	btf_id = btf_id__add(&obj->sets, id, true);
+	if (btf_id)
+		btf_id->kind = kind;
+
+	return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+	struct btf_id *btf_id;
 	char *id;
 
 	id = get_id(name + size);
@@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 		return NULL;
 	}
 
-	return btf_id__add(root, id, false);
+	btf_id = btf_id__add(root, id, false);
+	btf_id->kind = BTF_ID_KIND_SYM;
+
+	return btf_id;
 }
 
 /* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
@@ -491,28 +507,24 @@ static int symbols_collect(struct object *obj)
 			id = add_symbol(&obj->funcs, prefix, sizeof(BTF_FUNC) - 1);
 		/* set8 */
 		} else if (!strncmp(prefix, BTF_SET8, sizeof(BTF_SET8) - 1)) {
-			id = add_set(obj, prefix, true);
+			id = add_set(obj, prefix, BTF_ID_KIND_SET8);
 			/*
 			 * SET8 objects store list's count, which is encoded
 			 * in symbol's size, together with 'cnt' field hence
 			 * that - 1.
 			 */
-			if (id) {
+			if (id)
 				id->cnt = sym.st_size / sizeof(uint64_t) - 1;
-				id->is_set8 = true;
-			}
 		/* set */
 		} else if (!strncmp(prefix, BTF_SET, sizeof(BTF_SET) - 1)) {
-			id = add_set(obj, prefix, false);
+			id = add_set(obj, prefix, BTF_ID_KIND_SET);
 			/*
 			 * SET objects store list's count, which is encoded
 			 * in symbol's size, together with 'cnt' field hence
 			 * that - 1.
 			 */
-			if (id) {
+			if (id)
 				id->cnt = sym.st_size / sizeof(int) - 1;
-				id->is_set = true;
-			}
 		} else {
 			pr_err("FAILED unsupported prefix %s\n", prefix);
 			return -1;
@@ -643,7 +655,7 @@ static int id_patch(struct object *obj, struct btf_id *id)
 	int i;
 
 	/* For set, set8, id->id may be 0 */
-	if (!id->id && !id->is_set && !id->is_set8) {
+	if (!id->id && id->kind == BTF_ID_KIND_SYM) {
 		pr_err("WARN: resolve_btfids: unresolved symbol %s\n", id->name);
 		warnings++;
 	}
@@ -696,6 +708,7 @@ static int sets_patch(struct object *obj)
 {
 	Elf_Data *data = obj->efile.idlist;
 	struct rb_node *next;
+	int cnt;
 
 	next = rb_first(&obj->sets);
 	while (next) {
@@ -715,11 +728,15 @@ static int sets_patch(struct object *obj)
 			return -1;
 		}
 
-		if (id->is_set) {
+		switch (id->kind) {
+		case BTF_ID_KIND_SET:
 			set = data->d_buf + off;
+			cnt = set->cnt;
 			qsort(set->ids, set->cnt, sizeof(set->ids[0]), cmp_id);
-		} else {
+			break;
+		case BTF_ID_KIND_SET8:
 			set8 = data->d_buf + off;
+			cnt = set8->cnt;
 			/*
 			 * Make sure id is at the beginning of the pairs
 			 * struct, otherwise the below qsort would not work.
@@ -744,10 +761,14 @@ static int sets_patch(struct object *obj)
 						bswap_32(set8->pairs[i].flags);
 				}
 			}
+			break;
+		case BTF_ID_KIND_SYM:
+		default:
+			pr_err("Unexpected btf_id_kind %d for set '%s'\n", id->kind, id->name);
+			return -1;
 		}
 
-		pr_debug("sorting  addr %5lu: cnt %6d [%s]\n",
-			 off, id->is_set ? set->cnt : set8->cnt, id->name);
+		pr_debug("sorting  addr %5lu: cnt %6d [%s]\n", off, cnt, id->name);
 
 		next = rb_next(next);
 	}
Signature · 1 line
2.52.0
#4 lore raw

Re: [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind

Can add_symbol() dereference a NULL pointer here? The function btf_id__add() can return NULL on allocation failure (when zalloc fails), but the new code sets btf_id->kind before checking if btf_id…
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0 -0
index b7b44e72e..7f5a9f7dd 100644

[ ... ]

Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10 -3
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
 	return id;
 }
 
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
 	/*
 	 * __BTF_ID__set__name
 	 * name =    ^
 	 * id   =         ^
 	 */
-	char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+	int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+	char *id = name + prefixlen - 1;
 	int len = strlen(name);
+	struct btf_id *btf_id;
 
 	if (id >= name + len) {
 		pr_err("FAILED to parse set name: %s\n", name);
 		return NULL;
 	}
 
-	return btf_id__add(&obj->sets, id, true);
+	btf_id = btf_id__add(&obj->sets, id, true);
+	if (btf_id)
+		btf_id->kind = kind;
+
+	return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+	struct btf_id *btf_id;
 	char *id;
 
 	id = get_id(name + size);

tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 305

Quoted 6 lines
tools/bpf/resolve_btfids/main.c +2 -1
@@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 		return NULL;
 	}
 
-	return btf_id__add(root, id, false);
+	btf_id = btf_id__add(root, id, false);
+	btf_id->kind = BTF_ID_KIND_SYM;
                 ^^^^

Can add_symbol() dereference a NULL pointer here? The function btf_id__add() can return NULL on allocation failure (when zalloc fails), but the new code sets btf_id->kind before checking if btf_id is NULL.

In contrast, add_set() checks for NULL before setting the kind field. Should add_symbol() do the same?

tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 291

Quoted 3 lines
tools/bpf/resolve_btfids/main.c +2 -0
+
+	return btf_id;
 }

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19689674924

author reviewer
#5 lore raw

Re: [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind

Quoted 53 lines · On 11/25/25 6:09 PM, [email protected] wrote:
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0 -0
index b7b44e72e..7f5a9f7dd 100644

[ ... ]

Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10 -3
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)
 	return id;
 }
 
-static struct btf_id *add_set(struct object *obj, char *name, bool is_set8)
+static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind kind)
 {
 	/*
 	 * __BTF_ID__set__name
 	 * name =    ^
 	 * id   =         ^
 	 */
-	char *id = name + (is_set8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__")) - 1;
+	int prefixlen = kind == BTF_ID_KIND_SET8 ? sizeof(BTF_SET8 "__") : sizeof(BTF_SET "__");
+	char *id = name + prefixlen - 1;
 	int len = strlen(name);
+	struct btf_id *btf_id;
 
 	if (id >= name + len) {
 		pr_err("FAILED to parse set name: %s\n", name);
 		return NULL;
 	}
 
-	return btf_id__add(&obj->sets, id, true);
+	btf_id = btf_id__add(&obj->sets, id, true);
+	if (btf_id)
+		btf_id->kind = kind;
+
+	return btf_id;
 }
 
 static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 {
+	struct btf_id *btf_id;
 	char *id;
 
 	id = get_id(name + size);

tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 305

Quoted 6 lines
tools/bpf/resolve_btfids/main.c +2 -1
@@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)
 		return NULL;
 	}
 
-	return btf_id__add(root, id, false);
+	btf_id = btf_id__add(root, id, false);
+	btf_id->kind = BTF_ID_KIND_SYM;
                 ^^^^
Quoted 6 lines

Can add_symbol() dereference a NULL pointer here? The function btf_id__add() can return NULL on allocation failure (when zalloc fails), but the new code sets btf_id->kind before checking if btf_id is NULL.

In contrast, add_set() checks for NULL before setting the kind field. Should add_symbol() do the same?

We pass unique=false to btf_id__add() and that means that it can only return NULL if zalloc() fails, which is unlikely.

Checking for NULL here is little too defensive, but won't hurt.

Quoted 13 lines

tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 291

Quoted 4 lines
tools/bpf/resolve_btfids/main.c +2 -0
+
+	return btf_id;
 }

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19689674924

author reviewer
#6 lore raw

PATCH 4/4 resolve_btfids: change in-place update with raw binary output

In reply to Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a **detached** file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285

---
 MAINTAINERS                     |   1 +
 scripts/Makefile.modfinal       |   5 +-
 scripts/gen-btf.sh              | 166 ++++++++++++++++++++++++++++++++
 scripts/link-vmlinux.sh         |  42 ++------
 tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------
 5 files changed, 272 insertions(+), 66 deletions(-)
 create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:	net/sched/act_bpf.c
 F:	net/sched/cls_bpf.c
 F:	samples/bpf/
 F:	scripts/bpf_doc.py
+F:	scripts/gen-btf.sh
 F:	scripts/Makefile.btf
 F:	scripts/pahole-version.sh
 F:	tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..86f843995556 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko = 							\
 	if [ ! -f $(objtree)/vmlinux ]; then				\
 		printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-	else								\
-		LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-		$(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;		\
+	else	\
+		$(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
 	fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +166 -0

166 lines not on this page · raw message

scripts/link-vmlinux.sh +6 -36

71 lines not on this page · raw message

tools/bpf/resolve_btfids/main.c +97 -27

195 lines not on this page · raw message

Signature · 1 line
2.52.0
#7 lore raw

Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 180 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a **detached** file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285

---
 MAINTAINERS                     |   1 +
 scripts/Makefile.modfinal       |   5 +-
 scripts/gen-btf.sh              | 166 ++++++++++++++++++++++++++++++++
 scripts/link-vmlinux.sh         |  42 ++------
 tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------
 5 files changed, 272 insertions(+), 66 deletions(-)
 create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:        net/sched/act_bpf.c
 F:     net/sched/cls_bpf.c
 F:     samples/bpf/
 F:     scripts/bpf_doc.py
+F:     scripts/gen-btf.sh
 F:     scripts/Makefile.btf
 F:     scripts/pahole-version.sh
 F:     tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..86f843995556 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko =                                                     \
        if [ ! -f $(objtree)/vmlinux ]; then                            \
                printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-       else                                                            \
-               LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-               $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;             \
+       else    \
+               $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
        fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +80 -0
new file mode 100755
index 000000000000..102f8296ae9e
@@ -0,0 +1,166 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+#   1. pahole generates BTF from DWARF data
+#   2. resolve_btfids applies kernel-specific btf2btf
+#      transformations and computes data for .BTF_ids section
+#   3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+#   - ${1}.btf.o ready for linking into vmlinux
+#   - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+       echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+       exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+       case "$1" in
+       --btf_base)
+               BTF_BASE="$2"
+               shift 2
+               ;;
+       -*)
+               echo "Unknown option: $1" >&2
+               usage
+               ;;
+       *)
+               break
+               ;;
+       esac
+done
+
+if [ $# -ne 1 ]; then
+       usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+       grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+       printf "  %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+       set -x
+       ;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+       exit 0
+fi
+
+gen_btf_data()
+{
+       info BTF "${ELF_FILE}"

scripts/gen-btf.sh line 90

Quoted 10 lines
scripts/gen-btf.sh new file +10 -0
+       btf1="${ELF_FILE}.btf.1"
+       ${PAHOLE} -J ${PAHOLE_FLAGS}                    \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf_encode_detached=${btf1}           \
+               "${ELF_FILE}"
+
+       info BTFIDS "${ELF_FILE}"
+       RESOLVE_BTFIDS_OPTS=""
+       if is_enabled CONFIG_WERROR; then
+               RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "

In POSIX sh, +=is undefined[1], and I encountered the following error:

./scripts/gen-btf.sh: 90: RESOLVE_BTFIDS_OPTS+= --fatal_warnings : not found

We should use the following syntax instead:

RESOLVE_BTFIDS_OPTS="${RESOLVE_BTFIDS_OPTS} --fatal_warnings "

[1] https://www.shellcheck.net/wiki/SC3024

Thanks,
Donglin

scripts/gen-btf.sh line 93

Quoted 3 lines
scripts/gen-btf.sh new file +3 -0
+       fi
+       if [ -n "${KBUILD_VERBOSE}" ]; then
+               RESOLVE_BTFIDS_OPTS+=" -v "

Ditto

Thanks,
Donglin

Quoted 40 lines
scripts/gen-btf.sh new file +40 -0
+       fi
+       ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS}        \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+       local btf_data=${ELF_FILE}.btf.o
+
+       # Create ${btf_data} which contains just .BTF section but no symbols. Add
+       # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+       # deletes all symbols including __start_BTF and __stop_BTF, which will
+       # be redefined in the linker script.
+       info OBJCOPY "${btf_data}"
+       echo "" | ${CC} -c -x c -o ${btf_data} -
+       ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+               --set-section-flags .BTF=alloc,readonly ${btf_data}
+       ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+       # Change e_type to ET_REL so that it can be used to link final vmlinux.
+       # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+       if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+               et_rel='\0\1'
+       else
+               et_rel='\1\0'
+       fi
+       printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+       info OBJCOPY "${ELF_FILE}"
+
+       ${OBJCOPY} \
+               --add-section .BTF=${ELF_FILE}.btf                      \
+               --add-section .BTF.base=${ELF_FILE}.distilled_base.btf  \
+               ${ELF_FILE}
+
+       # a module might not have a .BTF_ids section

scripts/gen-btf.sh line 143

Quoted 10 lines
scripts/gen-btf.sh new file +10 -0
+       if [ -f "${ELF_FILE}.btf_ids" ]; then
+               ${OBJCOPY} --update-section .BTF_ids=${ELF_FILE}.btf_ids ${ELF_FILE}
+       fi
+}
+
+cleanup()
+{
+       rm -f "${ELF_FILE}.btf.1"
+       rm -f "${ELF_FILE}.btf"
+       if [ "${BTFGEN_MODE}" == "module" ]; then

In POSIX sh, == in place of = is undefined[2], and I encountered the following error:

./scripts/gen-btf.sh: 143: [: module: unexpected operator

we should use = instead.

[2] https://www.shellcheck.net/wiki/SC3014

Thanks,
Donglin

Quoted 256 lines
scripts/gen-btf.sh new file +23 -0
+               rm -f "${ELF_FILE}.distilled_base.btf"
+               rm -f "${ELF_FILE}.btf_ids"
+       fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+       BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+       gen_btf_o
+       ;;
+module)
+       embed_btf_data
+       ;;
+esac
+
+exit 0
scripts/link-vmlinux.sh +6 -36
index 433849ff7529..728f82af24f6 100755
@@ -105,34 +105,6 @@ vmlinux_link()
                ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
 }
 
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
-       local btf_data=${1}.btf.o
-
-       info BTF "${btf_data}"
-       LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
-       # Create ${btf_data} which contains just .BTF section but no symbols. Add
-       # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
-       # deletes all symbols including __start_BTF and __stop_BTF, which will
-       # be redefined in the linker script. Add 2>/dev/null to suppress GNU
-       # objcopy warnings: "empty loadable segment detected at ..."
-       ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
-               --strip-all ${1} "${btf_data}" 2>/dev/null
-       # Change e_type to ET_REL so that it can be used to link final vmlinux.
-       # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
-       if is_enabled CONFIG_CPU_BIG_ENDIAN; then
-               et_rel='\0\1'
-       else
-               et_rel='\1\0'
-       fi
-       printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
-       btf_vmlinux_bin_o=${btf_data}
-}
-
 # Create ${2}.o file with all symbols from the ${1} object file
 kallsyms()
 {
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
 fi
 
 btf_vmlinux_bin_o=
+btfids_vmlinux=
 kallsymso=
 strip_debug=
 generate_map=
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-       if ! gen_btf .tmp_vmlinux1; then
+       if ! scripts/gen-btf.sh .tmp_vmlinux1; then
                echo >&2 "Failed to generate BTF for vmlinux"
                echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
                exit 1
        fi
+       btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+       btfids_vmlinux=.tmp_vmlinux1.btf_ids
 fi
 
 if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
 
 vmlinux_link "${VMLINUX}"
 
-# fill in BTF IDs
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-       info BTFIDS "${VMLINUX}"
-       RESOLVE_BTFIDS_ARGS=""
-       if is_enabled CONFIG_WERROR; then
-               RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
-       fi
-       ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+       info OBJCOPY ${btfids_vmlinux}
+       ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
 fi
 
 mksysmap "${VMLINUX}" System.map
tools/bpf/resolve_btfids/main.c +74 -21
index 7f5a9f7dde7f..4faf16b1ba6b 100644
@@ -71,9 +71,11 @@
 #include <fcntl.h>
 #include <errno.h>
 #include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
 #include <linux/rbtree.h>
 #include <linux/zalloc.h>
 #include <linux/err.h>
+#include <linux/limits.h>
 #include <bpf/btf.h>
 #include <bpf/libbpf.h>
 #include <subcmd/parse-options.h>
@@ -429,14 +431,6 @@ static int elf_collect(struct object *obj)
                        obj->efile.idlist       = data;
                        obj->efile.idlist_shndx = idx;
                        obj->efile.idlist_addr  = sh.sh_addr;
-               } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
-                       /* If a .BTF.base section is found, do not resolve
-                        * BTF ids relative to vmlinux; resolve relative
-                        * to the .BTF.base section instead.  btf__parse_split()
-                        * will take care of this once the base BTF it is
-                        * passed is NULL.
-                        */
-                       obj->base_btf_path = NULL;
                }
 
                if (compressed_section_fix(elf, scn, &sh))
@@ -570,6 +564,19 @@ static int load_btf(struct object *obj)
        obj->base_btf = base_btf;
        obj->btf = btf;
 
+       if (obj->base_btf) {
+               err = btf__distill_base(obj->btf, &base_btf, &btf);
+               if (err) {
+                       pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
+                       goto out_err;
+               }
+
+               btf__free(obj->btf);
+               btf__free(obj->base_btf);
+               obj->btf = btf;
+               obj->base_btf = base_btf;
+       }
+
        return 0;
 
 out_err:
@@ -777,8 +784,6 @@ static int sets_patch(struct object *obj)
 
 static int symbols_patch(struct object *obj)
 {
-       off_t err;
-
        if (__symbols_patch(obj, &obj->structs)  ||
            __symbols_patch(obj, &obj->unions)   ||
            __symbols_patch(obj, &obj->typedefs) ||
@@ -789,20 +794,67 @@ static int symbols_patch(struct object *obj)
        if (sets_patch(obj))
                return -1;
 
-       /* Set type to ensure endian translation occurs. */
-       obj->efile.idlist->d_type = ELF_T_WORD;
+       return 0;
+}
 
-       elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+       int fd, ret;
 
-       err = elf_update(obj->efile.elf, ELF_C_WRITE);
-       if (err < 0) {
-               pr_err("FAILED elf_update(WRITE): %s\n",
-                       elf_errmsg(-1));
+       fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+       if (fd < 0) {
+               pr_err("Couldn't open %s for writing\n", out_path);
+               return fd;
+       }
+
+       ret = write(fd, data, size);
+       if (ret < 0 || ret != size) {
+               pr_err("Failed to write data to %s\n", out_path);
+               close(fd);
+               unlink(out_path);
+               return -1;
+       }
+
+       close(fd);
+       pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
+
+       return 0;
+}
+
+static int dump_raw_btf_ids(struct object *obj, const char *out_path)
+{
+       Elf_Data *data = obj->efile.idlist;
+       int fd, err;
+
+       if (!data || !data->d_buf) {
+               pr_debug("%s has no BTF_ids data to dump\n", obj->path);
+               return 0;
+       }
+
+       err = dump_raw_data(out_path, data->d_buf, data->d_size);
+       if (err)
+               return -1;
+
+       return 0;
+}
+
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+       const void *raw_btf_data;
+       u32 raw_btf_size;
+       int fd, err;
+
+       raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+       if (raw_btf_data == NULL) {
+               pr_err("btf__raw_data() failed\n");
+               return -1;
        }
 
-       pr_debug("update %s for %s\n",
-                err >= 0 ? "ok" : "failed", obj->path);
-       return err < 0 ? -1 : 0;
+       err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
+       if (err)
+               return -1;
+
+       return 0;
 }
 
 static const char * const resolve_btfids_usage[] = {
@@ -823,12 +875,13 @@ int main(int argc, const char **argv)
                .funcs    = RB_ROOT,
                .sets     = RB_ROOT,
        };
+       char out_path[PATH_MAX];
        bool fatal_warnings = false;
        struct option btfid_options[] = {
                OPT_INCR('v', "verbose", &verbose,
                         "be more verbose (show errors, etc)"),
                OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
-                          "BTF data"),
+                          "input BTF data"),
                OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
                           "path of file providing base BTF"),
                OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,

tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 901

Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2 -0
@@ -844,6 +897,9 @@ int main(int argc, const char **argv)
 
        obj.path = argv[0];
 
+       if (load_btf(&obj))
+               goto out;

I think I can add the BTF sorting function here based on your patch.

Thanks,
Donglin

Quoted 25 lines
tools/bpf/resolve_btfids/main.c +3 -6
+
        if (elf_collect(&obj))
                goto out;
 
@@ -853,23 +909,37 @@ int main(int argc, const char **argv)
         */
        if (obj.efile.idlist_shndx == -1 ||
            obj.efile.symbols_shndx == -1) {
-               pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
-               err = 0;
-               goto out;
+               pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+               goto dump_btf;
        }
 
        if (symbols_collect(&obj))
                goto out;
 
-       if (load_btf(&obj))
-               goto out;
-
        if (symbols_resolve(&obj))
                goto out;
 
        if (symbols_patch(&obj))

tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 932

Quoted 10 lines
tools/bpf/resolve_btfids/main.c +8 -0
                goto out;
 
+       strcpy(out_path, obj.path);
+       strcat(out_path, ".btf_ids");
+       if (dump_raw_btf_ids(&obj, out_path))
+               goto out;
+
+dump_btf:
+       strcpy(out_path, obj.path);
+       strcat(out_path, ".btf");

Do we need to add .btf files to the .gitignore file?

Thanks,
Donglin

Quoted 16 lines
tools/bpf/resolve_btfids/main.c +10 -1
+       if (dump_raw_btf(obj.btf, out_path))
+               goto out;
+
+       if (obj.base_btf) {
+               strcpy(out_path, obj.path);
+               strcat(out_path, ".distilled_base.btf");
+               if (dump_raw_btf(obj.base_btf, out_path))
+                       goto out;
+       }
+
        if (!(fatal_warnings && warnings))
                err = 0;
 out:
--

2.52.0

author reviewer
#8 lore raw

Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 20 lines · Donglin Peng
Quoted 6 lines · Ihor Solodrai

[...]

scripts/gen-btf.sh new file +4 -0
+
+gen_btf_data()
+{
+       info BTF "${ELF_FILE}"

scripts/gen-btf.sh line 90

Quoted 10 lines
scripts/gen-btf.sh new file +10 -0
+       btf1="${ELF_FILE}.btf.1"
+       ${PAHOLE} -J ${PAHOLE_FLAGS}                    \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf_encode_detached=${btf1}           \
+               "${ELF_FILE}"
+
+       info BTFIDS "${ELF_FILE}"
+       RESOLVE_BTFIDS_OPTS=""
+       if is_enabled CONFIG_WERROR; then
+               RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "

In POSIX sh, +=is undefined[1], and I encountered the following error:

Quoted 5 lines

./scripts/gen-btf.sh: 90: RESOLVE_BTFIDS_OPTS+= --fatal_warnings : not found

We should use the following syntax instead:

RESOLVE_BTFIDS_OPTS="${RESOLVE_BTFIDS_OPTS} --fatal_warnings "

Hi Donglin, thanks for taking a look.

These and a couple of other bugs have been caught by CI [1]. I am working on v2.

I changed the script to #!/bin/bash and will run the shellcheck before submitting the next revision [2], when it's ready.

[1] https://github.com/kernel-patches/bpf/actions/runs/19689674975
[2] https://github.com/kernel-patches/bpf/pull/10370

Quoted 6 lines

[1] https://www.shellcheck.net/wiki/SC3024

Thanks,
Donglin

Quoted 1 line

[...]

author reviewer
#9 lore raw

Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output

In reply to Donglin Peng
Quoted 21 lines · Donglin Peng
Quoted 20 lines · Ihor Solodrai

[...]
static const char * const resolve_btfids_usage[] = {

tools/bpf/resolve_btfids/main.c +2 -1
@@ -823,12 +875,13 @@ int main(int argc, const char **argv)
                .funcs    = RB_ROOT,
                .sets     = RB_ROOT,
        };
+       char out_path[PATH_MAX];
        bool fatal_warnings = false;
        struct option btfid_options[] = {
                OPT_INCR('v', "verbose", &verbose,
                         "be more verbose (show errors, etc)"),
                OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
-                          "BTF data"),
+                          "input BTF data"),
                OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
                           "path of file providing base BTF"),
                OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
@@ -844,6 +897,9 @@ int main(int argc, const char **argv)
 
Quoted 6 lines

tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 901

Quoted 4 lines
tools/bpf/resolve_btfids/main.c +2 -0
        obj.path = argv[0];
 
+       if (load_btf(&obj))
+               goto out;

I think I can add the BTF sorting function here based on your patch.

Correct. Any btf2btf transformations will have to happen before the symbols resolution.

Quoted 30 lines

Thanks,
Donglin

Quoted 27 lines
tools/bpf/resolve_btfids/main.c +3 -6
+
        if (elf_collect(&obj))
                goto out;
 
@@ -853,23 +909,37 @@ int main(int argc, const char **argv)
         */
        if (obj.efile.idlist_shndx == -1 ||
            obj.efile.symbols_shndx == -1) {
-               pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
-               err = 0;
-               goto out;
+               pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+               goto dump_btf;
        }
 
        if (symbols_collect(&obj))
                goto out;
 
-       if (load_btf(&obj))
-               goto out;
-
        if (symbols_resolve(&obj))
                goto out;
 
        if (symbols_patch(&obj))
                goto out;
 
Quoted 10 lines

tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 932

Quoted 8 lines
tools/bpf/resolve_btfids/main.c +8 -0
+       strcpy(out_path, obj.path);
+       strcat(out_path, ".btf_ids");
+       if (dump_raw_btf_ids(&obj, out_path))
+               goto out;
+
+dump_btf:
+       strcpy(out_path, obj.path);
+       strcat(out_path, ".btf");

Do we need to add .btf files to the .gitignore file?

I don't know. Probably?

I take care to cleanup temporary files in the gen-btf.sh, but it makes sense to .gitignore them anyways, since those are temporary build artifacts.

Quoted 20 lines

Thanks,
Donglin

Quoted 16 lines
tools/bpf/resolve_btfids/main.c +10 -1
+       if (dump_raw_btf(obj.btf, out_path))
+               goto out;
+
+       if (obj.base_btf) {
+               strcpy(out_path, obj.path);
+               strcat(out_path, ".distilled_base.btf");
+               if (dump_raw_btf(obj.base_btf, out_path))
+                       goto out;
+       }
+
        if (!(fatal_warnings && warnings))
                err = 0;
 out:
--

2.52.0

#10 lore raw

Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output

In reply to Ihor Solodrai
Quoted 46 lines · Ihor Solodrai

Currently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.

This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}

The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.

The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.

There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.

For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.

Before this patch it worked like follows:

* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place

After this patch gen-btf.sh implements the following:

* pahole consumes .tmp_vmlinux1 and produces a **detached** file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in

Quoted 5 lines
    link-vmlinux.sh by objcopy --update-section

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

The Makefile for the bpf selftests also needs be updated too:
https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708

This results in the self-test for resolve_btfids failing:
$./vmtest.sh -- ./test_progs -t resolve_btfids -v
...
test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)

Quoted 504 lines

[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285

---
 MAINTAINERS                     |   1 +
 scripts/Makefile.modfinal       |   5 +-
 scripts/gen-btf.sh              | 166 ++++++++++++++++++++++++++++++++
 scripts/link-vmlinux.sh         |  42 ++------
 tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------
 5 files changed, 272 insertions(+), 66 deletions(-)
 create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
index 48aabeeed029..5cd34419d952 100644
@@ -4672,6 +4672,7 @@ F:        net/sched/act_bpf.c
 F:     net/sched/cls_bpf.c
 F:     samples/bpf/
 F:     scripts/bpf_doc.py
+F:     scripts/gen-btf.sh
 F:     scripts/Makefile.btf
 F:     scripts/pahole-version.sh
 F:     tools/bpf/
scripts/Makefile.modfinal +2 -3
index 542ba462ed3e..86f843995556 100644
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@
       cmd_btf_ko =                                                     \
        if [ ! -f $(objtree)/vmlinux ]; then                            \
                printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \
-       else                                                            \
-               LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \
-               $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@;             \
+       else    \
+               $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \
        fi;
 
 # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +166 -0
new file mode 100755
index 000000000000..102f8296ae9e
@@ -0,0 +1,166 @@
+#!/bin/sh
+# SPDX-License-Identifier: GPL-2.0
+# Copyright (c) 2025 Meta Platforms, Inc. and affiliates.
+#
+# This script generates BTF data for the provided ELF file.
+#
+# Kernel BTF generation involves these conceptual steps:
+#   1. pahole generates BTF from DWARF data
+#   2. resolve_btfids applies kernel-specific btf2btf
+#      transformations and computes data for .BTF_ids section
+#   3. the result gets linked/objcopied into the target binary
+#
+# How step (3) should be done differs between vmlinux, and
+# kernel modules, which is the primary reason for the existence
+# of this script.
+#
+# For modules the script expects vmlinux passed in as --btf_base.
+# Generated .BTF, .BTF.base and .BTF_ids sections become embedded
+# into the input ELF file with objcopy.
+#
+# For vmlinux the input file remains unchanged and two files are produced:
+#   - ${1}.btf.o ready for linking into vmlinux
+#   - ${1}.btf_ids with .BTF_ids data blob
+# This output is consumed by scripts/link-vmlinux.sh
+
+set -e
+
+usage()
+{
+       echo "Usage: $0 [--btf_base <file>] <target ELF file>"
+       exit 1
+}
+
+BTF_BASE=""
+
+while [ $# -gt 0 ]; do
+       case "$1" in
+       --btf_base)
+               BTF_BASE="$2"
+               shift 2
+               ;;
+       -*)
+               echo "Unknown option: $1" >&2
+               usage
+               ;;
+       *)
+               break
+               ;;
+       esac
+done
+
+if [ $# -ne 1 ]; then
+       usage
+fi
+
+ELF_FILE="$1"
+shift
+
+is_enabled() {
+       grep -q "^$1=y" ${objtree}/include/config/auto.conf
+}
+
+info()
+{
+       printf "  %-7s %s\n" "${1}" "${2}"
+}
+
+case "${KBUILD_VERBOSE}" in
+*1*)
+       set -x
+       ;;
+esac
+
+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then
+       exit 0
+fi
+
+gen_btf_data()
+{
+       info BTF "${ELF_FILE}"
+       btf1="${ELF_FILE}.btf.1"
+       ${PAHOLE} -J ${PAHOLE_FLAGS}                    \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf_encode_detached=${btf1}           \
+               "${ELF_FILE}"
+
+       info BTFIDS "${ELF_FILE}"
+       RESOLVE_BTFIDS_OPTS=""
+       if is_enabled CONFIG_WERROR; then
+               RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
+       fi
+       if [ -n "${KBUILD_VERBOSE}" ]; then
+               RESOLVE_BTFIDS_OPTS+=" -v "
+       fi
+       ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS}        \
+               ${BTF_BASE:+--btf_base ${BTF_BASE}}     \
+               --btf ${btf1} "${ELF_FILE}"
+}
+
+gen_btf_o()
+{
+       local btf_data=${ELF_FILE}.btf.o
+
+       # Create ${btf_data} which contains just .BTF section but no symbols. Add
+       # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
+       # deletes all symbols including __start_BTF and __stop_BTF, which will
+       # be redefined in the linker script.
+       info OBJCOPY "${btf_data}"
+       echo "" | ${CC} -c -x c -o ${btf_data} -
+       ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \
+               --set-section-flags .BTF=alloc,readonly ${btf_data}
+       ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data}
+
+       # Change e_type to ET_REL so that it can be used to link final vmlinux.
+       # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
+       if is_enabled CONFIG_CPU_BIG_ENDIAN; then
+               et_rel='\0\1'
+       else
+               et_rel='\1\0'
+       fi
+       printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
+}
+
+embed_btf_data()
+{
+       info OBJCOPY "${ELF_FILE}"
+
+       ${OBJCOPY} \
+               --add-section .BTF=${ELF_FILE}.btf                      \
+               --add-section .BTF.base=${ELF_FILE}.distilled_base.btf  \
+               ${ELF_FILE}
+
+       # a module might not have a .BTF_ids section
+       if [ -f "${ELF_FILE}.btf_ids" ]; then
+               ${OBJCOPY} --update-section .BTF_ids=${ELF_FILE}.btf_ids ${ELF_FILE}
+       fi
+}
+
+cleanup()
+{
+       rm -f "${ELF_FILE}.btf.1"
+       rm -f "${ELF_FILE}.btf"
+       if [ "${BTFGEN_MODE}" == "module" ]; then
+               rm -f "${ELF_FILE}.distilled_base.btf"
+               rm -f "${ELF_FILE}.btf_ids"
+       fi
+}
+trap cleanup EXIT
+
+BTFGEN_MODE="vmlinux"
+if [ -n "${BTF_BASE}" ]; then
+       BTFGEN_MODE="module"
+fi
+
+gen_btf_data
+
+case "${BTFGEN_MODE}" in
+vmlinux)
+       gen_btf_o
+       ;;
+module)
+       embed_btf_data
+       ;;
+esac
+
+exit 0
scripts/link-vmlinux.sh +6 -36
index 433849ff7529..728f82af24f6 100755
@@ -105,34 +105,6 @@ vmlinux_link()
                ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs}
 }
 
-# generate .BTF typeinfo from DWARF debuginfo
-# ${1} - vmlinux image
-gen_btf()
-{
-       local btf_data=${1}.btf.o
-
-       info BTF "${btf_data}"
-       LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1}
-
-       # Create ${btf_data} which contains just .BTF section but no symbols. Add
-       # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all
-       # deletes all symbols including __start_BTF and __stop_BTF, which will
-       # be redefined in the linker script. Add 2>/dev/null to suppress GNU
-       # objcopy warnings: "empty loadable segment detected at ..."
-       ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \
-               --strip-all ${1} "${btf_data}" 2>/dev/null
-       # Change e_type to ET_REL so that it can be used to link final vmlinux.
-       # GNU ld 2.35+ and lld do not allow an ET_EXEC input.
-       if is_enabled CONFIG_CPU_BIG_ENDIAN; then
-               et_rel='\0\1'
-       else
-               et_rel='\1\0'
-       fi
-       printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none
-
-       btf_vmlinux_bin_o=${btf_data}
-}
-
 # Create ${2}.o file with all symbols from the ${1} object file
 kallsyms()
 {
@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then
 fi
 
 btf_vmlinux_bin_o=
+btfids_vmlinux=
 kallsymso=
 strip_debug=
 generate_map=
@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then
 fi
 
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-       if ! gen_btf .tmp_vmlinux1; then
+       if ! scripts/gen-btf.sh .tmp_vmlinux1; then
                echo >&2 "Failed to generate BTF for vmlinux"
                echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF"
                exit 1
        fi
+       btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o
+       btfids_vmlinux=.tmp_vmlinux1.btf_ids
 fi
 
 if is_enabled CONFIG_KALLSYMS; then
@@ -281,14 +256,9 @@ fi
 
 vmlinux_link "${VMLINUX}"
 
-# fill in BTF IDs
 if is_enabled CONFIG_DEBUG_INFO_BTF; then
-       info BTFIDS "${VMLINUX}"
-       RESOLVE_BTFIDS_ARGS=""
-       if is_enabled CONFIG_WERROR; then
-               RESOLVE_BTFIDS_ARGS=" --fatal_warnings "
-       fi
-       ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}"
+       info OBJCOPY ${btfids_vmlinux}
+       ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX}
 fi
 
 mksysmap "${VMLINUX}" System.map
tools/bpf/resolve_btfids/main.c +97 -28
index 7f5a9f7dde7f..4faf16b1ba6b 100644
@@ -71,9 +71,11 @@
 #include <fcntl.h>
 #include <errno.h>
 #include <linux/btf_ids.h>
+#include <linux/kallsyms.h>
 #include <linux/rbtree.h>
 #include <linux/zalloc.h>
 #include <linux/err.h>
+#include <linux/limits.h>
 #include <bpf/btf.h>
 #include <bpf/libbpf.h>
 #include <subcmd/parse-options.h>
@@ -429,14 +431,6 @@ static int elf_collect(struct object *obj)
                        obj->efile.idlist       = data;
                        obj->efile.idlist_shndx = idx;
                        obj->efile.idlist_addr  = sh.sh_addr;
-               } else if (!strcmp(name, BTF_BASE_ELF_SEC)) {
-                       /* If a .BTF.base section is found, do not resolve
-                        * BTF ids relative to vmlinux; resolve relative
-                        * to the .BTF.base section instead.  btf__parse_split()
-                        * will take care of this once the base BTF it is
-                        * passed is NULL.
-                        */
-                       obj->base_btf_path = NULL;
                }
 
                if (compressed_section_fix(elf, scn, &sh))
@@ -570,6 +564,19 @@ static int load_btf(struct object *obj)
        obj->base_btf = base_btf;
        obj->btf = btf;
 
+       if (obj->base_btf) {
+               err = btf__distill_base(obj->btf, &base_btf, &btf);
+               if (err) {
+                       pr_err("FAILED to distill base BTF: %s\n", strerror(errno));
+                       goto out_err;
+               }
+
+               btf__free(obj->btf);
+               btf__free(obj->base_btf);
+               obj->btf = btf;
+               obj->base_btf = base_btf;
+       }
+
        return 0;
 
 out_err:
@@ -777,8 +784,6 @@ static int sets_patch(struct object *obj)
 
 static int symbols_patch(struct object *obj)
 {
-       off_t err;
-
        if (__symbols_patch(obj, &obj->structs)  ||
            __symbols_patch(obj, &obj->unions)   ||
            __symbols_patch(obj, &obj->typedefs) ||
@@ -789,20 +794,67 @@ static int symbols_patch(struct object *obj)
        if (sets_patch(obj))
                return -1;
 
-       /* Set type to ensure endian translation occurs. */
-       obj->efile.idlist->d_type = ELF_T_WORD;
+       return 0;
+}
 
-       elf_flagdata(obj->efile.idlist, ELF_C_SET, ELF_F_DIRTY);
+static int dump_raw_data(const char *out_path, const void *data, u32 size)
+{
+       int fd, ret;
 
-       err = elf_update(obj->efile.elf, ELF_C_WRITE);
-       if (err < 0) {
-               pr_err("FAILED elf_update(WRITE): %s\n",
-                       elf_errmsg(-1));
+       fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640);
+       if (fd < 0) {
+               pr_err("Couldn't open %s for writing\n", out_path);
+               return fd;
+       }
+
+       ret = write(fd, data, size);
+       if (ret < 0 || ret != size) {
+               pr_err("Failed to write data to %s\n", out_path);
+               close(fd);
+               unlink(out_path);
+               return -1;
+       }
+
+       close(fd);
+       pr_debug("Dumped %lu bytes of data to %s\n", size, out_path);
+
+       return 0;
+}
+
+static int dump_raw_btf_ids(struct object *obj, const char *out_path)
+{
+       Elf_Data *data = obj->efile.idlist;
+       int fd, err;
+
+       if (!data || !data->d_buf) {
+               pr_debug("%s has no BTF_ids data to dump\n", obj->path);
+               return 0;
+       }
+
+       err = dump_raw_data(out_path, data->d_buf, data->d_size);
+       if (err)
+               return -1;
+
+       return 0;
+}
+
+static int dump_raw_btf(struct btf *btf, const char *out_path)
+{
+       const void *raw_btf_data;
+       u32 raw_btf_size;
+       int fd, err;
+
+       raw_btf_data = btf__raw_data(btf, &raw_btf_size);
+       if (raw_btf_data == NULL) {
+               pr_err("btf__raw_data() failed\n");
+               return -1;
        }
 
-       pr_debug("update %s for %s\n",
-                err >= 0 ? "ok" : "failed", obj->path);
-       return err < 0 ? -1 : 0;
+       err = dump_raw_data(out_path, raw_btf_data, raw_btf_size);
+       if (err)
+               return -1;
+
+       return 0;
 }
 
 static const char * const resolve_btfids_usage[] = {
@@ -823,12 +875,13 @@ int main(int argc, const char **argv)
                .funcs    = RB_ROOT,
                .sets     = RB_ROOT,
        };
+       char out_path[PATH_MAX];
        bool fatal_warnings = false;
        struct option btfid_options[] = {
                OPT_INCR('v', "verbose", &verbose,
                         "be more verbose (show errors, etc)"),
                OPT_STRING(0, "btf", &obj.btf_path, "BTF data",
-                          "BTF data"),
+                          "input BTF data"),
                OPT_STRING('b', "btf_base", &obj.base_btf_path, "file",
                           "path of file providing base BTF"),
                OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
@@ -844,6 +897,9 @@ int main(int argc, const char **argv)
 
        obj.path = argv[0];
 
+       if (load_btf(&obj))
+               goto out;
+
        if (elf_collect(&obj))
                goto out;
 
@@ -853,23 +909,37 @@ int main(int argc, const char **argv)
         */
        if (obj.efile.idlist_shndx == -1 ||
            obj.efile.symbols_shndx == -1) {
-               pr_debug("Cannot find .BTF_ids or symbols sections, nothing to do\n");
-               err = 0;
-               goto out;
+               pr_debug("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n");
+               goto dump_btf;
        }
 
        if (symbols_collect(&obj))
                goto out;
 
-       if (load_btf(&obj))
-               goto out;
-
        if (symbols_resolve(&obj))
                goto out;
 
        if (symbols_patch(&obj))
                goto out;
 
+       strcpy(out_path, obj.path);
+       strcat(out_path, ".btf_ids");
+       if (dump_raw_btf_ids(&obj, out_path))
+               goto out;
+
+dump_btf:
+       strcpy(out_path, obj.path);
+       strcat(out_path, ".btf");
+       if (dump_raw_btf(obj.btf, out_path))
+               goto out;
+
+       if (obj.base_btf) {
+               strcpy(out_path, obj.path);
+               strcat(out_path, ".distilled_base.btf");
+               if (dump_raw_btf(obj.base_btf, out_path))
+                       goto out;
+       }
+
        if (!(fatal_warnings && warnings))
                err = 0;
 out:
--

2.52.0

author reviewer
#11 lore raw

Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 8 lines · Donglin Peng
Quoted 6 lines · Ihor Solodrai

[...]

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

Quoted 8 lines

The Makefile for the bpf selftests also needs be updated too:
https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708

This results in the self-test for resolve_btfids failing:
$./vmtest.sh -- ./test_progs -t resolve_btfids -v
...
test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)

Good catch, thanks.

I remember I noticed this at some point, and then forgot...

Interestingly this test passes on CI [1]:

2025-11-26T03:09:52.0908317Z #366     reg_bounds_rand_ranges_u64_u64:OK
2025-11-26T03:09:52.0925114Z #367     resolve_btfids:OK
2025-11-26T03:09:52.3904190Z #368/1   res_spin_lock_failure/res_spin_lock_arg:OK

I'll take a closer look.

[1] https://github.com/kernel-patches/bpf/actions/runs/19690981192/job/56406840021

Quoted 4 lines
Quoted 4 lines

[...]

author reviewer
#12 lore raw

Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output

Quoted 22 lines · Ihor Solodrai
Quoted 8 lines · Donglin Peng
Quoted 6 lines · Ihor Solodrai

[...]

For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.

Quoted 8 lines

The Makefile for the bpf selftests also needs be updated too:
https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708

This results in the self-test for resolve_btfids failing:
$./vmtest.sh -- ./test_progs -t resolve_btfids -v
...
test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)

Good catch, thanks.

I remember I noticed this at some point, and then forgot...

Quoted 7 lines

Interestingly this test passes on CI [1]:

2025-11-26T03:09:52.0908317Z #366     reg_bounds_rand_ranges_u64_u64:OK
2025-11-26T03:09:52.0925114Z #367     resolve_btfids:OK
2025-11-26T03:09:52.3904190Z #368/1   res_spin_lock_failure/res_spin_lock_arg:OK

I'll take a closer look.

I figured out why this test was flaky.

Even though I removed elf_update() call from resolve_btfids, the ELF was opened with:

    elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);

And the buffers which resolve_btfids writes to are from Elf_Data returned by elf_getdata(). And so the file might actually get written to in-place, which is why the resolve_btfids test passed for me with no changes to the selftests.

I switched ELF_C_RDWR_MMAP to ELF_C_READ_MMAP_PRIVATE, and then the ELF reliably remains intact (and the test fails). From libelf.h:

ELF_C_READ_MMAP_PRIVATE, /* Read, but memory is writable, results are
not written to the file. */

It makes sense to use this for what resolve_btfids is doing.

I'll fix selftests/bpf/Makefile in the next revision.

Quoted 8 lines

[1] https://github.com/kernel-patches/bpf/actions/runs/19690981192/job/56406840021

Quoted 4 lines
Quoted 4 lines

[...]

#13 lore raw
In reply to Ihor Solodrai
Quoted 17 lines · Ihor Solodrai

This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.

Patches #1-#3 in the series are non-functional refactoring in resolve_btfids. The last patch (#4) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. Implementation changes are described in detail in the patch description.

One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)

Quoted 5 lines

Since BTF finalization is now done by resolve_btfids, it requires btf__distill_base() to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag for resolve_btfids.

hi Ihor,

Can you say more about what constitutes BTF finalization and why BTF distillation prior to finalization (i.e. in pahole) isn't workable? Is it the concern that we eliminate types due to filtering, or is it a problem with sorting/tracking type ids? Are there operations we do/anticipate that make prior distillation infeasbile? Thanks!

Quoted 5 lines

Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.

Quoted 7 lines

However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessary btf__relocate() call on load [6]. It can be avoided by special-casing in-tree module handling in resolve_btfids either with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.

I would say avoid distillation for in-tree modules if possible, as it imposes runtime costs in relocation/type renumbering on module load. For large modules (amdgpu take a bow) that could be non-trivial time-wise. IMO the build-time costs/complexities are worth paying to avoid a runtime tax on module load.

Quoted 21 lines

[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358

Ihor Solodrai (4):
resolve_btfids: rename object btf field to btf_path
resolve_btfids: factor out load_btf()
resolve_btfids: introduce enum btf_id_kind
resolve_btfids: change in-place update with raw binary output

 MAINTAINERS                     |   1 +
 scripts/Makefile.modfinal       |   5 +-
 scripts/gen-btf.sh              | 166 ++++++++++++++++++++++
 scripts/link-vmlinux.sh         |  42 +-----
 tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++---------
 5 files changed, 348 insertions(+), 100 deletions(-)
 create mode 100755 scripts/gen-btf.sh
author reviewer
#14 lore raw
Quoted 23 lines · Alan Maguire
Quoted 17 lines · Ihor Solodrai

This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.

Patches #1-#3 in the series are non-functional refactoring in resolve_btfids. The last patch (#4) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. Implementation changes are described in detail in the patch description.

One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)

Quoted 5 lines

Since BTF finalization is now done by resolve_btfids, it requires btf__distill_base() to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag for resolve_btfids.

Quoted 7 lines

hi Ihor,

Can you say more about what constitutes BTF finalization and why BTF distillation prior to finalization (i.e. in pahole) isn't workable? Is it the concern that we eliminate types due to filtering, or is it a problem with sorting/tracking type ids? Are there operations we do/anticipate that make prior distillation infeasbile? Thanks!

Hi Alan,

That's a good question. AFAIU the distillation should be done on the final BTF, after all the transformations (sorting, adding/removing BTF types) have been applied. At least this way we can be sure that the distilled base is valid.

We certainly want BTF generation process to be the same for modules and the kernel, which means that BTF modifications in resolve_btfids have to be applied to module BTF also.

So the question is whether btf2btf will be safe to do *after* distillation, and that of course depends on the specifics.

Let's say pahole generated BTF for a module and a distilled base. If later some types are removed from module BTF, or a new type is added (that might refer to a type absent in distilled base), is the btf/base pair still valid?

My intuition is that it is more reliable to distill the final-final BTF, and so with resolve_btfids taking over kernel BTF finalization it makes sense to do it there. Otherwise we may be upfront limiting ourselves in how module BTF can be changed in resolve_btfids.

What are the reasons to keep distillation in pahole? It's a simple libbpf API call after all. Anything I might be missing?

Quoted 14 lines
Quoted 6 lines

Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.

Quoted 7 lines

However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessary btf__relocate() call on load [6]. It can be avoided by special-casing in-tree module handling in resolve_btfids either with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.

Quoted 5 lines

I would say avoid distillation for in-tree modules if possible, as it imposes runtime costs in relocation/type renumbering on module load. For large modules (amdgpu take a bow) that could be non-trivial time-wise. IMO the build-time costs/complexities are worth paying to avoid a runtime tax on module load.

Acked. I still would like to avoid passing flags around if possible.

Is it reasonable to simply check for KBUILD_EXTMOD env var from withing resolve_btfids? Any drawbacks to that?

Thanks.

Quoted 23 lines
Quoted 22 lines

[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358

Ihor Solodrai (4):
resolve_btfids: rename object btf field to btf_path
resolve_btfids: factor out load_btf()
resolve_btfids: introduce enum btf_id_kind
resolve_btfids: change in-place update with raw binary output

 MAINTAINERS                     |   1 +
 scripts/Makefile.modfinal       |   5 +-
 scripts/gen-btf.sh              | 166 ++++++++++++++++++++++
 scripts/link-vmlinux.sh         |  42 +-----
 tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++---------
 5 files changed, 348 insertions(+), 100 deletions(-)
 create mode 100755 scripts/gen-btf.sh
#15 lore raw
Quoted 51 lines · Ihor Solodrai
Quoted 23 lines · Alan Maguire
Quoted 17 lines · Ihor Solodrai

This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.

Patches #1-#3 in the series are non-functional refactoring in resolve_btfids. The last patch (#4) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. Implementation changes are described in detail in the patch description.

One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)

Quoted 5 lines

Since BTF finalization is now done by resolve_btfids, it requires btf__distill_base() to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag for resolve_btfids.

Quoted 7 lines

hi Ihor,

Can you say more about what constitutes BTF finalization and why BTF distillation prior to finalization (i.e. in pahole) isn't workable? Is it the concern that we eliminate types due to filtering, or is it a problem with sorting/tracking type ids? Are there operations we do/anticipate that make prior distillation infeasbile? Thanks!

Hi Alan,

That's a good question. AFAIU the distillation should be done on the final BTF, after all the transformations (sorting, adding/removing BTF types) have been applied. At least this way we can be sure that the distilled base is valid.

We certainly want BTF generation process to be the same for modules and the kernel, which means that BTF modifications in resolve_btfids have to be applied to module BTF also.

So the question is whether btf2btf will be safe to do *after* distillation, and that of course depends on the specifics.

Let's say pahole generated BTF for a module and a distilled base. If later some types are removed from module BTF, or a new type is added (that might refer to a type absent in distilled base), is the btf/base pair still valid?

Quoted 9 lines

My intuition is that it is more reliable to distill the final-final BTF, and so with resolve_btfids taking over kernel BTF finalization it makes sense to do it there. Otherwise we may be upfront limiting ourselves in how module BTF can be changed in resolve_btfids.

What are the reasons to keep distillation in pahole? It's a simple libbpf API call after all. Anything I might be missing?

Nope, I think doing distillation as the final step makes sense to me too since we get a clearer picture of the types that the module references. My only reservation was applying distillation to every module (rather than just out-of-tree builds) but it sounds like that's not needed from below.

Quoted 20 lines
Quoted 14 lines
Quoted 6 lines

Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.

Quoted 7 lines

However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessary btf__relocate() call on load [6]. It can be avoided by special-casing in-tree module handling in resolve_btfids either with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.

Quoted 5 lines

I would say avoid distillation for in-tree modules if possible, as it imposes runtime costs in relocation/type renumbering on module load. For large modules (amdgpu take a bow) that could be non-trivial time-wise. IMO the build-time costs/complexities are worth paying to avoid a runtime tax on module load.

Quoted 5 lines

Acked. I still would like to avoid passing flags around if possible.

Is it reasonable to simply check for KBUILD_EXTMOD env var from withing resolve_btfids? Any drawbacks to that?

None that I can think of; sounds like a good approach to me and is basically equivalent to what we do now. I'll take a look at v2. Thanks!

Alan

Quoted 27 lines

Thanks.

Quoted 23 lines
Quoted 22 lines

[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358

Ihor Solodrai (4):
resolve_btfids: rename object btf field to btf_path
resolve_btfids: factor out load_btf()
resolve_btfids: introduce enum btf_id_kind
resolve_btfids: change in-place update with raw binary output

 MAINTAINERS                     |   1 +
 scripts/Makefile.modfinal       |   5 +-
 scripts/gen-btf.sh              | 166 ++++++++++++++++++++++
 scripts/link-vmlinux.sh         |  42 +-----
 tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++---------
 5 files changed, 348 insertions(+), 100 deletions(-)
 create mode 100755 scripts/gen-btf.sh