resolve_btfids: Support for BTF modifications
Ihor Solodrai
reviewercover resolve_btfids: Support for BTF modifications
This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.
Patches #1-#3 in the series are non-functional refactoring in resolve_btfids. The last patch (#4) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. Implementation changes are described in detail in the patch description.
One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)
Since BTF finalization is now done by resolve_btfids, it requires btf__distill_base() to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag for resolve_btfids.
Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.
However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessary btf__relocate() call on load [6]. It can be avoided by special-casing in-tree module handling in resolve_btfids either with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.
[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358
Ihor Solodrai (4):
resolve_btfids: rename object btf field to btf_path
resolve_btfids: factor out load_btf()
resolve_btfids: introduce enum btf_id_kind
resolve_btfids: change in-place update with raw binary output
MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 166 ++++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++--------- 5 files changed, 348 insertions(+), 100 deletions(-) create mode 100755 scripts/gen-btf.sh
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH 1/4 resolve_btfids: rename object btf field to btf_path
Rename the member of struct object holding the path to BTF data if provided via --btf arg. btf_path is less ambiguous.
--- tools/bpf/resolve_btfids/main.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tools/bpf/resolve_btfids/main.c +4 -4
@@ -113,7 +113,7 @@ struct btf_id {struct object {const char *path; -const char *btf; +const char *btf_path;const char *base_btf_path;struct {
@@ -550,11 +550,11 @@ static int symbols_resolve(struct object *obj) } } - btf =btf__parse_split (obj->btf ?: obj->path, base_btf); + btf =btf__parse_split (obj->btf_path ?: obj->path, base_btf); err =libbpf_get_error (btf);if (err) {pr_err ("FAILED: load BTF from %s: %s\n" , - obj->btf ?: obj->path,strerror (-err)); + obj->btf_path ?: obj->path,strerror (-err)); goto out; }
@@ -790,7 +790,7 @@ int main(int argc, const char **argv)struct option btfid_options[] = {OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show errors, etc)" ), -OPT_STRING (0 ,"btf" , &obj.btf,"BTF data" , +OPT_STRING (0 ,"btf" , &obj.btf_path,"BTF data" ,"BTF data" ),OPT_STRING ('b' ,"btf_base" , &obj.base_btf_path,"file" ,"path of file providing base BTF" ),
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH 2/4 resolve_btfids: factor out load_btf()
In reply to Ihor SolodraiIncrease the lifetime of parsed BTF in resolve_btfids by factoring load_btf() routine out of symbols_resolve() and storing the base_btf and btf pointers in the struct object.
--- tools/bpf/resolve_btfids/main.c | 47 ++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 13 deletions(-)
tools/bpf/resolve_btfids/main.c +34 -13
@@ -116,6 +116,9 @@ struct object {const char *btf_path;const char *base_btf_path; +struct btf *btf; +struct btf *base_btf; +struct {int fd;Elf *elf;
@@ -529,16 +532,10 @@ static int symbols_collect(struct object *obj)return 0 ; } -static int symbols_resolve (struct object *obj) +static int load_btf (struct object *obj) { -int nr_typedefs = obj->nr_typedefs; -int nr_structs = obj->nr_structs; -int nr_unions = obj->nr_unions; -int nr_funcs = obj->nr_funcs; -struct btf *base_btf =NULL ; -int err, type_id; -struct btf *btf; -__u32 nr_types; +struct btf *base_btf =NULL , *btf =NULL ; +int err;if (obj->base_btf_path) { base_btf =btf__parse (obj->base_btf_path,NULL );
@@ -546,7 +543,7 @@ static int symbols_resolve(struct object *obj)if (err) {pr_err ("FAILED: load base BTF from %s: %s\n" , obj->base_btf_path,strerror (-err)); -return -1 ; + goto out_err; } }
@@ -555,9 +552,30 @@ static int symbols_resolve(struct object *obj)if (err) {pr_err ("FAILED: load BTF from %s: %s\n" , obj->btf_path ?: obj->path,strerror (-err)); - goto out; + goto out_err; } + obj->base_btf = base_btf; + obj->btf = btf; + +return 0 ; + +out_err: +btf__free (base_btf); +btf__free (btf); +return err; +} + +static int symbols_resolve (struct object *obj) +{ +int nr_typedefs = obj->nr_typedefs; +int nr_structs = obj->nr_structs; +int nr_unions = obj->nr_unions; +int nr_funcs = obj->nr_funcs; +struct btf *btf = obj->btf; +int err, type_id; +__u32 nr_types; + err =-1 ; nr_types =btf__type_cnt (btf);
@@ -615,8 +633,6 @@ static int symbols_resolve(struct object *obj) err =0 ; out: -btf__free (base_btf); -btf__free (btf);return err; }
@@ -824,6 +840,9 @@ int main(int argc, const char **argv)if (symbols_collect (&obj)) goto out; +if (load_btf (&obj)) + goto out; +if (symbols_resolve (&obj)) goto out;
@@ -833,6 +852,8 @@ int main(int argc, const char **argv)if (!(fatal_warnings && warnings)) err =0 ; out: +btf__free (obj.base_btf); +btf__free (obj.btf);if (obj.efile.elf) {elf_end (obj.efile.elf);close (obj.efile.fd);
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH 3/4 resolve_btfids: introduce enum btf_id_kind
In reply to Ihor SolodraiInstead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.
--- tools/bpf/resolve_btfids/main.c | 61 ++++++++++++++++++++++----------- 1 file changed, 41 insertions(+), 20 deletions(-)
tools/bpf/resolve_btfids/main.c +41 -20
@@ -98,6 +98,13 @@# error "Unknown machine endianness!"#endif +enum btf_id_kind { +BTF_ID_KIND_NONE , +BTF_ID_KIND_SYM , +BTF_ID_KIND_SET , +BTF_ID_KIND_SET8 +}; +struct btf_id {struct rb_node rb_node;char *name;
@@ -105,9 +112,8 @@ struct btf_id {int id;int cnt; }; -int addr_cnt; -bool is_set; -bool is_set8; + enum btf_id_kind kind:8; + int addr_cnt:8;Elf64_Addr addr[ADDR_CNT ]; };
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);
@@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; } -return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); + btf_id->kind =BTF_ID_KIND_SYM ; + +return btf_id; }/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
@@ -491,28 +507,24 @@ static int symbols_collect(struct object *obj) id =add_symbol (&obj->funcs, prefix,sizeof (BTF_FUNC ) -1 );/* set8 */ }else if (!strncmp (prefix,BTF_SET8 ,sizeof (BTF_SET8 ) -1 )) { - id =add_set (obj, prefix, true); + id =add_set (obj, prefix,BTF_ID_KIND_SET8 );/* * SET8 objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (uint64_t ) -1 ; - id->is_set8 = true; - }/* set */ }else if (!strncmp (prefix,BTF_SET ,sizeof (BTF_SET ) -1 )) { - id =add_set (obj, prefix, false); + id =add_set (obj, prefix,BTF_ID_KIND_SET );/* * SET objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (int ) -1 ; - id->is_set = true; - } }else {pr_err ("FAILED unsupported prefix %s\n" , prefix);return -1 ;
@@ -643,7 +655,7 @@ static int id_patch(struct object *obj, struct btf_id *id)int i;/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind ==BTF_ID_KIND_SYM ) {pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }
@@ -696,6 +708,7 @@ static int sets_patch(struct object *obj) {Elf_Data *data = obj->efile.idlist;struct rb_node *next; +int cnt; next =rb_first (&obj->sets);while (next) {
@@ -715,11 +728,15 @@ static int sets_patch(struct object *obj)return -1 ; } -if (id->is_set) { +switch (id->kind) { +case BTF_ID_KIND_SET : set = data->d_buf + off; + cnt = set->cnt;qsort (set->ids, set->cnt,sizeof (set->ids[0 ]), cmp_id); - }else { +break ; +case BTF_ID_KIND_SET8 : set8 = data->d_buf + off; + cnt = set8->cnt;/* * Make sure id is at the beginning of the pairs * struct, otherwise the below qsort would not work.
@@ -744,10 +761,14 @@ static int sets_patch(struct object *obj)bswap_32 (set8->pairs[i].flags); } } +break ; +case BTF_ID_KIND_SYM : +default : +pr_err ("Unexpected btf_id_kind %d for set '%s'\n" , id->kind, id->name); +return -1 ; } -pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , - off, id->is_set ? set->cnt : set8->cnt, id->name); +pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , off, cnt, id->name); next =rb_next (next); }
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind
Can add_symbol() dereference a NULL pointer here? The function btf_id__add() can return NULL on allocation failure (when zalloc fails), but the new code sets btf_id->kind before checking if btf_id…
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0
[ ... ]
Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10
-3@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);
tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 305
Quoted 6 lines
tools/bpf/resolve_btfids/main.c +2
-1@@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; } -return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); + btf_id->kind =BTF_ID_KIND_SYM ;
^^^^
Can add_symbol() dereference a NULL pointer here? The function btf_id__add() can return NULL on allocation failure (when zalloc fails), but the new code sets btf_id->kind before checking if btf_id is NULL.
In contrast, add_set() checks for NULL before setting the kind field. Should add_symbol() do the same?
tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 291
Quoted 3 lines
tools/bpf/resolve_btfids/main.c +2
-0+ +return btf_id; }
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/19689674924
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 3/4] resolve_btfids: introduce enum btf_id_kind
Quoted 53 lines · On 11/25/25 6:09 PM, [email protected] wrote:
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0[ ... ]
Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10
-3@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 305
Quoted 6 lines
tools/bpf/resolve_btfids/main.c +2
-1@@ -288,7 +301,10 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; } -return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); + btf_id->kind =BTF_ID_KIND_SYM ;^^^^
Quoted 6 lines
Can
add_symbol()dereference a NULL pointer here? The functionbtf_id__add()can return NULL on allocation failure (when zalloc fails), but the new code setsbtf_id->kindbefore checking ifbtf_idis NULL.In contrast,
add_set()checks for NULL before setting the kind field. Shouldadd_symbol()do the same?
We pass unique=false to btf_id__add() and that means that it can only return NULL if zalloc() fails, which is unlikely.
Checking for NULL here is little too defensive, but won't hurt.
Quoted 13 lines
tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 291
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +2
-0+ +return btf_id; }[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/19689674924
Ihor Solodrai
reviewerPATCH 4/4 resolve_btfids: change in-place update with raw binary output
In reply to Ihor SolodraiCurrently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.
This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.
The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a **detached** file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 166 ++++++++++++++++++++++++++++++++ scripts/link-vmlinux.sh | 42 ++------ tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------ 5 files changed, 272 insertions(+), 66 deletions(-) create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/
scripts/Makefile.modfinal +2 -3
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +166 -0
166 lines not on this page · raw message
scripts/link-vmlinux.sh +6 -36
71 lines not on this page · raw message
tools/bpf/resolve_btfids/main.c +97 -27
195 lines not on this page · raw message
Signature · 1 line
2.52.0
Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 180 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a **detached** file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped inlink-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285Signed-off-by Ihor Solodrai <[email protected]>--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 166 ++++++++++++++++++++++++++++++++ scripts/link-vmlinux.sh | 42 ++------ tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------ 5 files changed, 272 insertions(+), 66 deletions(-) create mode 100755 scripts/gen-btf.shMAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependenciesscripts/gen-btf.sh new file +80
-0@@ -0,0 +1,166 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates. +# +# This script generates BTF data for the provided ELF file. +# +# Kernel BTF generation involves these conceptual steps: +# 1. pahole generates BTF from DWARF data +# 2. resolve_btfids applies kernel-specific btf2btf +# transformations and computes data for .BTF_ids section +# 3. the result gets linked/objcopied into the target binary +# +# How step (3) should be done differs between vmlinux, and +# kernel modules, which is the primary reason for the existence +# of this script. +# +# For modules the script expects vmlinux passed in as --btf_base. +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded +# into the input ELF file with objcopy. +# +# For vmlinux the input file remains unchanged and two files are produced: +# - ${1}.btf.o ready for linking into vmlinux +# - ${1}.btf_ids with .BTF_ids data blob +# This output is consumed by scripts/link-vmlinux.sh + +set -e + +usage() +{ + echo "Usage: $0 [--btf_base <file>] <target ELF file>" + exit 1 +} + +BTF_BASE="" + +while [ $# -gt 0 ]; do + case "$1" in + --btf_base) + BTF_BASE="$2" + shift 2 + ;; + -*) + echo "Unknown option: $1" >&2 + usage + ;; + *) + break + ;; + esac +done + +if [ $# -ne 1 ]; then + usage +fi + +ELF_FILE="$1" +shift + +is_enabled() { + grep -q "^$1=y" ${objtree}/include/config/auto.conf +} + +info() +{ + printf " %-7s %s\n" "${1}" "${2}" +} + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}"
scripts/gen-btf.sh line 90
Quoted 10 lines
scripts/gen-btf.sh new file +10
-0+ btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "
In POSIX sh, +=is undefined[1], and I encountered the following error:
./scripts/gen-btf.sh: 90: RESOLVE_BTFIDS_OPTS+= --fatal_warnings : not found
We should use the following syntax instead:
RESOLVE_BTFIDS_OPTS="${RESOLVE_BTFIDS_OPTS} --fatal_warnings "
[1] https://www.shellcheck.net/wiki/SC3024
Thanks,
Donglin
scripts/gen-btf.sh line 93
Quoted 3 lines
scripts/gen-btf.sh new file +3
-0+ fi + if [ -n "${KBUILD_VERBOSE}" ]; then + RESOLVE_BTFIDS_OPTS+=" -v "
Ditto
Thanks,
Donglin
Quoted 40 lines
scripts/gen-btf.sh new file +40
-0+ fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} + +gen_btf_o() +{ + local btf_data=${ELF_FILE}.btf.o + + # Create ${btf_data} which contains just .BTF section but no symbols. Add + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all + # deletes all symbols including __start_BTF and __stop_BTF, which will + # be redefined in the linker script. + info OBJCOPY "${btf_data}" + echo "" | ${CC} -c -x c -o ${btf_data} - + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \ + --set-section-flags .BTF=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + + # Change e_type to ET_REL so that it can be used to link final vmlinux. + # GNU ld 2.35+ and lld do not allow an ET_EXEC input. + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + et_rel='\0\1' + else + et_rel='\1\0' + fi + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none +} + +embed_btf_data() +{ + info OBJCOPY "${ELF_FILE}" + + ${OBJCOPY} \ + --add-section .BTF=${ELF_FILE}.btf \ + --add-section .BTF.base=${ELF_FILE}.distilled_base.btf \ + ${ELF_FILE} + + # a module might not have a .BTF_ids section
scripts/gen-btf.sh line 143
Quoted 10 lines
scripts/gen-btf.sh new file +10
-0+ if [ -f "${ELF_FILE}.btf_ids" ]; then + ${OBJCOPY} --update-section .BTF_ids=${ELF_FILE}.btf_ids ${ELF_FILE} + fi +} + +cleanup() +{ + rm -f "${ELF_FILE}.btf.1" + rm -f "${ELF_FILE}.btf" + if [ "${BTFGEN_MODE}" == "module" ]; then
In POSIX sh, == in place of = is undefined[2], and I encountered the following error:
./scripts/gen-btf.sh: 143: [: module: unexpected operator
we should use = instead.
[2] https://www.shellcheck.net/wiki/SC3014
Thanks,
Donglin
Quoted 256 lines
scripts/gen-btf.sh new file +23
-0+ rm -f "${ELF_FILE}.distilled_base.btf" + rm -f "${ELF_FILE}.btf_ids" + fi +} +trap cleanup EXIT + +BTFGEN_MODE="vmlinux" +if [ -n "${BTF_BASE}" ]; then + BTFGEN_MODE="module" +fi + +gen_btf_data + +case "${BTFGEN_MODE}" in +vmlinux) + gen_btf_o + ;; +module) + embed_btf_data + ;; +esac + +exit 0scripts/link-vmlinux.sh +6
-36@@ -105,34 +105,6 @@ vmlinux_link() ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} } -# generate .BTF typeinfo from DWARF debuginfo -# ${1} - vmlinux image -gen_btf() -{ - local btf_data=${1}.btf.o - - info BTF "${btf_data}" - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1} - - # Create ${btf_data} which contains just .BTF section but no symbols. Add - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all - # deletes all symbols including __start_BTF and __stop_BTF, which will - # be redefined in the linker script. Add 2>/dev/null to suppress GNU - # objcopy warnings: "empty loadable segment detected at ..." - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \ - --strip-all ${1} "${btf_data}" 2>/dev/null - # Change e_type to ET_REL so that it can be used to link final vmlinux. - # GNU ld 2.35+ and lld do not allow an ET_EXEC input. - if is_enabled CONFIG_CPU_BIG_ENDIAN; then - et_rel='\0\1' - else - et_rel='\1\0' - fi - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none - - btf_vmlinux_bin_o=${btf_data} -} - # Create ${2}.o file with all symbols from the ${1} object file kallsyms() {@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi btf_vmlinux_bin_o= +btfids_vmlinux= kallsymso= strip_debug= generate_map=@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then fi if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! scripts/gen-btf.sh .tmp_vmlinux1; then echo >&2 "Failed to generate BTF for vmlinux" echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF" exit 1 fi + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o + btfids_vmlinux=.tmp_vmlinux1.btf_ids fi if is_enabled CONFIG_KALLSYMS; then@@ -281,14 +256,9 @@ fi vmlinux_link "${VMLINUX}" -# fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS "${VMLINUX}" - RESOLVE_BTFIDS_ARGS="" - if is_enabled CONFIG_WERROR; then - RESOLVE_BTFIDS_ARGS=" --fatal_warnings " - fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" + info OBJCOPY ${btfids_vmlinux} + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX} fi mksysmap "${VMLINUX}" System.maptools/bpf/resolve_btfids/main.c +74
-21@@ -71,9 +71,11 @@#include <fcntl.h> #include <errno.h> #include <linux/btf_ids.h> +#include <linux/kallsyms.h> #include <linux/rbtree.h> #include <linux/zalloc.h> #include <linux/err.h> +#include <linux/limits.h> #include <bpf/btf.h> #include <bpf/libbpf.h> #include <subcmd/parse-options.h> @@ -429,14 +431,6 @@ static int elf_collect(struct object *obj) obj->efile.idlist = data; obj->efile.idlist_shndx = idx; obj->efile.idlist_addr = sh.sh_addr; - }else if (!strcmp (name,BTF_BASE_ELF_SEC )) { -/* If a .BTF.base section is found, do not resolve -* BTF ids relative to vmlinux; resolve relative -* to the .BTF.base section instead. btf__parse_split() -* will take care of this once the base BTF it is -* passed is NULL. -*/ - obj->base_btf_path =NULL ; }if (compressed_section_fix (elf, scn, &sh))@@ -570,6 +564,19 @@ static int load_btf(struct object *obj) obj->base_btf = base_btf; obj->btf = btf; +if (obj->base_btf) { + err =btf__distill_base (obj->btf, &base_btf, &btf); +if (err) { +pr_err ("FAILED to distill base BTF: %s\n" ,strerror (errno)); + goto out_err; + } + +btf__free (obj->btf); +btf__free (obj->base_btf); + obj->btf = btf; + obj->base_btf = base_btf; + } +return 0 ; out_err:@@ -777,8 +784,6 @@ static int sets_patch(struct object *obj)static int symbols_patch (struct object *obj) { -off_t err; - if (__symbols_patch(obj, &obj->structs) || __symbols_patch(obj, &obj->unions) || __symbols_patch(obj, &obj->typedefs) ||@@ -789,20 +794,67 @@ static int symbols_patch(struct object *obj)if (sets_patch (obj))return -1 ; -/* Set type to ensure endian translation occurs. */ - obj->efile.idlist->d_type =ELF_T_WORD ; +return 0 ; +} -elf_flagdata (obj->efile.idlist,ELF_C_SET ,ELF_F_DIRTY ); +static int dump_raw_data (const char *out_path,const void *data,u32 size) +{ +int fd, ret; - err =elf_update (obj->efile.elf,ELF_C_WRITE ); -if (err <0 ) { -pr_err ("FAILED elf_update(WRITE): %s\n" , -elf_errmsg (-1 )); + fd =open (out_path,O_WRONLY |O_CREAT |O_TRUNC ,0640 ); +if (fd <0 ) { +pr_err ("Couldn't open %s for writing\n" , out_path); +return fd; + } + + ret =write (fd, data, size); +if (ret <0 || ret != size) { +pr_err ("Failed to write data to %s\n" , out_path); +close (fd); +unlink (out_path); +return -1 ; + } + +close (fd); +pr_debug ("Dumped %lu bytes of data to %s\n" , size, out_path); + +return 0 ; +} + +static int dump_raw_btf_ids (struct object *obj,const char *out_path) +{ +Elf_Data *data = obj->efile.idlist; +int fd, err; + +if (!data || !data->d_buf) { +pr_debug ("%s has no BTF_ids data to dump\n" , obj->path); +return 0 ; + } + + err =dump_raw_data (out_path, data->d_buf, data->d_size); +if (err) +return -1 ; + +return 0 ; +} + +static int dump_raw_btf (struct btf *btf,const char *out_path) +{ +const void *raw_btf_data; +u32 raw_btf_size; +int fd, err; + + raw_btf_data =btf__raw_data (btf, &raw_btf_size); +if (raw_btf_data ==NULL ) { +pr_err ("btf__raw_data() failed\n" ); +return -1 ; } -pr_debug ("update %s for %s\n" , - err >=0 ?"ok" :"failed" , obj->path); -return err <0 ?-1 :0 ; + err =dump_raw_data (out_path, raw_btf_data, raw_btf_size); +if (err) +return -1 ; + +return 0 ; }static const char *const resolve_btfids_usage[] = {@@ -823,12 +875,13 @@ int main(int argc, const char **argv) .funcs = RB_ROOT, .sets = RB_ROOT, }; +char out_path[PATH_MAX ];bool fatal_warnings = false;struct option btfid_options[] = {OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show errors, etc)" ),OPT_STRING (0 ,"btf" , &obj.btf_path,"BTF data" , -"BTF data" ), +"input BTF data" ),OPT_STRING ('b' ,"btf_base" , &obj.base_btf_path,"file" ,"path of file providing base BTF" ), OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 901
Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2
-0@@ -844,6 +897,9 @@ int main(int argc, const char **argv) obj.path = argv[0 ]; +if (load_btf (&obj)) + goto out;
I think I can add the BTF sorting function here based on your patch.
Thanks,
Donglin
Quoted 25 lines
tools/bpf/resolve_btfids/main.c +3
-6+if (elf_collect (&obj)) goto out;@@ -853,23 +909,37 @@ int main(int argc, const char **argv)*/ if (obj.efile.idlist_shndx ==-1 || obj.efile.symbols_shndx ==-1 ) { -pr_debug ("Cannot find .BTF_ids or symbols sections, nothing to do\n" ); - err =0 ; - goto out; +pr_debug ("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n" ); + goto dump_btf; }if (symbols_collect (&obj)) goto out; -if (load_btf (&obj)) - goto out; -if (symbols_resolve (&obj)) goto out;if (symbols_patch (&obj))
tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 932
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +8
-0goto out; +strcpy (out_path, obj.path); +strcat (out_path,".btf_ids" ); +if (dump_raw_btf_ids (&obj, out_path)) + goto out; + +dump_btf: +strcpy (out_path, obj.path); +strcat (out_path,".btf" );
Do we need to add .btf files to the .gitignore file?
Thanks,
Donglin
Quoted 16 lines
tools/bpf/resolve_btfids/main.c +10
-1+if (dump_raw_btf (obj.btf, out_path)) + goto out; + +if (obj.base_btf) { +strcpy (out_path, obj.path); +strcat (out_path,".distilled_base.btf" ); +if (dump_raw_btf (obj.base_btf, out_path)) + goto out; + } +if (!(fatal_warnings && warnings)) err =0 ; out: --2.52.0
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 20 lines · Donglin Peng
Quoted 6 lines · Ihor Solodrai
[...]
scripts/gen-btf.sh new file +4
-0+ +gen_btf_data() +{ + info BTF "${ELF_FILE}"scripts/gen-btf.sh line 90
Quoted 10 lines
scripts/gen-btf.sh new file +10
-0+ btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings "In POSIX sh, +=is undefined[1], and I encountered the following error:
Quoted 5 lines
./scripts/gen-btf.sh: 90: RESOLVE_BTFIDS_OPTS+= --fatal_warnings : not foundWe should use the following syntax instead:
RESOLVE_BTFIDS_OPTS="${RESOLVE_BTFIDS_OPTS} --fatal_warnings "
Hi Donglin, thanks for taking a look.
These and a couple of other bugs have been caught by CI [1]. I am working on v2.
I changed the script to #!/bin/bash and will run the shellcheck before submitting the next revision [2], when it's ready.
[1] https://github.com/kernel-patches/bpf/actions/runs/19689674975
[2] https://github.com/kernel-patches/bpf/pull/10370
Quoted 6 lines
[1] https://www.shellcheck.net/wiki/SC3024
Thanks,
DonglinQuoted 1 line
[...]
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
In reply to Donglin PengQuoted 21 lines · Donglin Peng
Quoted 20 lines · Ihor Solodrai
[...]
static const char * const resolve_btfids_usage[] = {tools/bpf/resolve_btfids/main.c +2
-1@@ -823,12 +875,13 @@ int main(int argc, const char **argv) .funcs = RB_ROOT, .sets = RB_ROOT, }; +char out_path[PATH_MAX ];bool fatal_warnings = false;struct option btfid_options[] = {OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show errors, etc)" ),OPT_STRING (0 ,"btf" , &obj.btf_path,"BTF data" , -"BTF data" ), +"input BTF data" ),OPT_STRING ('b' ,"btf_base" , &obj.base_btf_path,"file" ,"path of file providing base BTF" ), OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,@@ -844,6 +897,9 @@ int main(int argc, const char **argv)
Quoted 6 lines
tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 901
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +2
-0obj.path = argv[0 ]; +if (load_btf (&obj)) + goto out;I think I can add the BTF sorting function here based on your patch.
Correct. Any btf2btf transformations will have to happen before the symbols resolution.
Quoted 30 lines
Thanks,
DonglinQuoted 27 lines
tools/bpf/resolve_btfids/main.c +3
-6+if (elf_collect (&obj)) goto out;@@ -853,23 +909,37 @@ int main(int argc, const char **argv)*/ if (obj.efile.idlist_shndx ==-1 || obj.efile.symbols_shndx ==-1 ) { -pr_debug ("Cannot find .BTF_ids or symbols sections, nothing to do\n" ); - err =0 ; - goto out; +pr_debug ("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n" ); + goto dump_btf; }if (symbols_collect (&obj)) goto out; -if (load_btf (&obj)) - goto out; -if (symbols_resolve (&obj)) goto out;if (symbols_patch (&obj)) goto out;
Quoted 10 lines
tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 932
Quoted 8 lines
tools/bpf/resolve_btfids/main.c +8
-0+strcpy (out_path, obj.path); +strcat (out_path,".btf_ids" ); +if (dump_raw_btf_ids (&obj, out_path)) + goto out; + +dump_btf: +strcpy (out_path, obj.path); +strcat (out_path,".btf" );Do we need to add .btf files to the .gitignore file?
I don't know. Probably?
I take care to cleanup temporary files in the gen-btf.sh, but it makes sense to .gitignore them anyways, since those are temporary build artifacts.
Quoted 20 lines
Thanks,
DonglinQuoted 16 lines
tools/bpf/resolve_btfids/main.c +10
-1+if (dump_raw_btf (obj.btf, out_path)) + goto out; + +if (obj.base_btf) { +strcpy (out_path, obj.path); +strcat (out_path,".distilled_base.btf" ); +if (dump_raw_btf (obj.base_btf, out_path)) + goto out; + } +if (!(fatal_warnings && warnings)) err =0 ; out: --2.52.0
Re: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
In reply to Ihor SolodraiQuoted 46 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a **detached** file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
Quoted 5 lines
link-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.
The Makefile for the bpf selftests also needs be updated too:
https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708
This results in the self-test for resolve_btfids failing:
$./vmtest.sh -- ./test_progs -t resolve_btfids -v
...
test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)
Quoted 504 lines
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285Signed-off-by Ihor Solodrai <[email protected]>--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 166 ++++++++++++++++++++++++++++++++ scripts/link-vmlinux.sh | 42 ++------ tools/bpf/resolve_btfids/main.c | 124 ++++++++++++++++++------ 5 files changed, 272 insertions(+), 66 deletions(-) create mode 100755 scripts/gen-btf.shMAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(objtree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependenciesscripts/gen-btf.sh new file +166
-0@@ -0,0 +1,166 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates. +# +# This script generates BTF data for the provided ELF file. +# +# Kernel BTF generation involves these conceptual steps: +# 1. pahole generates BTF from DWARF data +# 2. resolve_btfids applies kernel-specific btf2btf +# transformations and computes data for .BTF_ids section +# 3. the result gets linked/objcopied into the target binary +# +# How step (3) should be done differs between vmlinux, and +# kernel modules, which is the primary reason for the existence +# of this script. +# +# For modules the script expects vmlinux passed in as --btf_base. +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded +# into the input ELF file with objcopy. +# +# For vmlinux the input file remains unchanged and two files are produced: +# - ${1}.btf.o ready for linking into vmlinux +# - ${1}.btf_ids with .BTF_ids data blob +# This output is consumed by scripts/link-vmlinux.sh + +set -e + +usage() +{ + echo "Usage: $0 [--btf_base <file>] <target ELF file>" + exit 1 +} + +BTF_BASE="" + +while [ $# -gt 0 ]; do + case "$1" in + --btf_base) + BTF_BASE="$2" + shift 2 + ;; + -*) + echo "Unknown option: $1" >&2 + usage + ;; + *) + break + ;; + esac +done + +if [ $# -ne 1 ]; then + usage +fi + +ELF_FILE="$1" +shift + +is_enabled() { + grep -q "^$1=y" ${objtree}/include/config/auto.conf +} + +info() +{ + printf " %-7s %s\n" "${1}" "${2}" +} + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings " + fi + if [ -n "${KBUILD_VERBOSE}" ]; then + RESOLVE_BTFIDS_OPTS+=" -v " + fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} + +gen_btf_o() +{ + local btf_data=${ELF_FILE}.btf.o + + # Create ${btf_data} which contains just .BTF section but no symbols. Add + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all + # deletes all symbols including __start_BTF and __stop_BTF, which will + # be redefined in the linker script. + info OBJCOPY "${btf_data}" + echo "" | ${CC} -c -x c -o ${btf_data} - + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \ + --set-section-flags .BTF=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + + # Change e_type to ET_REL so that it can be used to link final vmlinux. + # GNU ld 2.35+ and lld do not allow an ET_EXEC input. + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + et_rel='\0\1' + else + et_rel='\1\0' + fi + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none +} + +embed_btf_data() +{ + info OBJCOPY "${ELF_FILE}" + + ${OBJCOPY} \ + --add-section .BTF=${ELF_FILE}.btf \ + --add-section .BTF.base=${ELF_FILE}.distilled_base.btf \ + ${ELF_FILE} + + # a module might not have a .BTF_ids section + if [ -f "${ELF_FILE}.btf_ids" ]; then + ${OBJCOPY} --update-section .BTF_ids=${ELF_FILE}.btf_ids ${ELF_FILE} + fi +} + +cleanup() +{ + rm -f "${ELF_FILE}.btf.1" + rm -f "${ELF_FILE}.btf" + if [ "${BTFGEN_MODE}" == "module" ]; then + rm -f "${ELF_FILE}.distilled_base.btf" + rm -f "${ELF_FILE}.btf_ids" + fi +} +trap cleanup EXIT + +BTFGEN_MODE="vmlinux" +if [ -n "${BTF_BASE}" ]; then + BTFGEN_MODE="module" +fi + +gen_btf_data + +case "${BTFGEN_MODE}" in +vmlinux) + gen_btf_o + ;; +module) + embed_btf_data + ;; +esac + +exit 0scripts/link-vmlinux.sh +6
-36@@ -105,34 +105,6 @@ vmlinux_link() ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} } -# generate .BTF typeinfo from DWARF debuginfo -# ${1} - vmlinux image -gen_btf() -{ - local btf_data=${1}.btf.o - - info BTF "${btf_data}" - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1} - - # Create ${btf_data} which contains just .BTF section but no symbols. Add - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all - # deletes all symbols including __start_BTF and __stop_BTF, which will - # be redefined in the linker script. Add 2>/dev/null to suppress GNU - # objcopy warnings: "empty loadable segment detected at ..." - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \ - --strip-all ${1} "${btf_data}" 2>/dev/null - # Change e_type to ET_REL so that it can be used to link final vmlinux. - # GNU ld 2.35+ and lld do not allow an ET_EXEC input. - if is_enabled CONFIG_CPU_BIG_ENDIAN; then - et_rel='\0\1' - else - et_rel='\1\0' - fi - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none - - btf_vmlinux_bin_o=${btf_data} -} - # Create ${2}.o file with all symbols from the ${1} object file kallsyms() {@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi btf_vmlinux_bin_o= +btfids_vmlinux= kallsymso= strip_debug= generate_map=@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then fi if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! scripts/gen-btf.sh .tmp_vmlinux1; then echo >&2 "Failed to generate BTF for vmlinux" echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF" exit 1 fi + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o + btfids_vmlinux=.tmp_vmlinux1.btf_ids fi if is_enabled CONFIG_KALLSYMS; then@@ -281,14 +256,9 @@ fi vmlinux_link "${VMLINUX}" -# fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS "${VMLINUX}" - RESOLVE_BTFIDS_ARGS="" - if is_enabled CONFIG_WERROR; then - RESOLVE_BTFIDS_ARGS=" --fatal_warnings " - fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" + info OBJCOPY ${btfids_vmlinux} + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX} fi mksysmap "${VMLINUX}" System.maptools/bpf/resolve_btfids/main.c +97
-28@@ -71,9 +71,11 @@#include <fcntl.h> #include <errno.h> #include <linux/btf_ids.h> +#include <linux/kallsyms.h> #include <linux/rbtree.h> #include <linux/zalloc.h> #include <linux/err.h> +#include <linux/limits.h> #include <bpf/btf.h> #include <bpf/libbpf.h> #include <subcmd/parse-options.h> @@ -429,14 +431,6 @@ static int elf_collect(struct object *obj) obj->efile.idlist = data; obj->efile.idlist_shndx = idx; obj->efile.idlist_addr = sh.sh_addr; - }else if (!strcmp (name,BTF_BASE_ELF_SEC )) { -/* If a .BTF.base section is found, do not resolve -* BTF ids relative to vmlinux; resolve relative -* to the .BTF.base section instead. btf__parse_split() -* will take care of this once the base BTF it is -* passed is NULL. -*/ - obj->base_btf_path =NULL ; }if (compressed_section_fix (elf, scn, &sh))@@ -570,6 +564,19 @@ static int load_btf(struct object *obj) obj->base_btf = base_btf; obj->btf = btf; +if (obj->base_btf) { + err =btf__distill_base (obj->btf, &base_btf, &btf); +if (err) { +pr_err ("FAILED to distill base BTF: %s\n" ,strerror (errno)); + goto out_err; + } + +btf__free (obj->btf); +btf__free (obj->base_btf); + obj->btf = btf; + obj->base_btf = base_btf; + } +return 0 ; out_err:@@ -777,8 +784,6 @@ static int sets_patch(struct object *obj)static int symbols_patch (struct object *obj) { -off_t err; - if (__symbols_patch(obj, &obj->structs) || __symbols_patch(obj, &obj->unions) || __symbols_patch(obj, &obj->typedefs) ||@@ -789,20 +794,67 @@ static int symbols_patch(struct object *obj)if (sets_patch (obj))return -1 ; -/* Set type to ensure endian translation occurs. */ - obj->efile.idlist->d_type =ELF_T_WORD ; +return 0 ; +} -elf_flagdata (obj->efile.idlist,ELF_C_SET ,ELF_F_DIRTY ); +static int dump_raw_data (const char *out_path,const void *data,u32 size) +{ +int fd, ret; - err =elf_update (obj->efile.elf,ELF_C_WRITE ); -if (err <0 ) { -pr_err ("FAILED elf_update(WRITE): %s\n" , -elf_errmsg (-1 )); + fd =open (out_path,O_WRONLY |O_CREAT |O_TRUNC ,0640 ); +if (fd <0 ) { +pr_err ("Couldn't open %s for writing\n" , out_path); +return fd; + } + + ret =write (fd, data, size); +if (ret <0 || ret != size) { +pr_err ("Failed to write data to %s\n" , out_path); +close (fd); +unlink (out_path); +return -1 ; + } + +close (fd); +pr_debug ("Dumped %lu bytes of data to %s\n" , size, out_path); + +return 0 ; +} + +static int dump_raw_btf_ids (struct object *obj,const char *out_path) +{ +Elf_Data *data = obj->efile.idlist; +int fd, err; + +if (!data || !data->d_buf) { +pr_debug ("%s has no BTF_ids data to dump\n" , obj->path); +return 0 ; + } + + err =dump_raw_data (out_path, data->d_buf, data->d_size); +if (err) +return -1 ; + +return 0 ; +} + +static int dump_raw_btf (struct btf *btf,const char *out_path) +{ +const void *raw_btf_data; +u32 raw_btf_size; +int fd, err; + + raw_btf_data =btf__raw_data (btf, &raw_btf_size); +if (raw_btf_data ==NULL ) { +pr_err ("btf__raw_data() failed\n" ); +return -1 ; } -pr_debug ("update %s for %s\n" , - err >=0 ?"ok" :"failed" , obj->path); -return err <0 ?-1 :0 ; + err =dump_raw_data (out_path, raw_btf_data, raw_btf_size); +if (err) +return -1 ; + +return 0 ; }static const char *const resolve_btfids_usage[] = {@@ -823,12 +875,13 @@ int main(int argc, const char **argv) .funcs = RB_ROOT, .sets = RB_ROOT, }; +char out_path[PATH_MAX ];bool fatal_warnings = false;struct option btfid_options[] = {OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show errors, etc)" ),OPT_STRING (0 ,"btf" , &obj.btf_path,"BTF data" , -"BTF data" ), +"input BTF data" ),OPT_STRING ('b' ,"btf_base" , &obj.base_btf_path,"file" ,"path of file providing base BTF" ), OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,@@ -844,6 +897,9 @@ int main(int argc, const char **argv) obj.path = argv[0 ]; +if (load_btf (&obj)) + goto out; +if (elf_collect (&obj)) goto out;@@ -853,23 +909,37 @@ int main(int argc, const char **argv)*/ if (obj.efile.idlist_shndx ==-1 || obj.efile.symbols_shndx ==-1 ) { -pr_debug ("Cannot find .BTF_ids or symbols sections, nothing to do\n" ); - err =0 ; - goto out; +pr_debug ("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n" ); + goto dump_btf; }if (symbols_collect (&obj)) goto out; -if (load_btf (&obj)) - goto out; -if (symbols_resolve (&obj)) goto out;if (symbols_patch (&obj)) goto out; +strcpy (out_path, obj.path); +strcat (out_path,".btf_ids" ); +if (dump_raw_btf_ids (&obj, out_path)) + goto out; + +dump_btf: +strcpy (out_path, obj.path); +strcat (out_path,".btf" ); +if (dump_raw_btf (obj.btf, out_path)) + goto out; + +if (obj.base_btf) { +strcpy (out_path, obj.path); +strcat (out_path,".distilled_base.btf" ); +if (dump_raw_btf (obj.base_btf, out_path)) + goto out; + } +if (!(fatal_warnings && warnings)) err =0 ; out: --2.52.0
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 8 lines · Donglin Peng
Quoted 6 lines · Ihor Solodrai
[...]
For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.
Quoted 8 lines
The
Makefilefor the bpf selftests also needs be updated too:
https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708This results in the self-test for
resolve_btfidsfailing:
$./vmtest.sh-- ./test_progs -tresolve_btfids-v
...
test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)
Good catch, thanks.
I remember I noticed this at some point, and then forgot...
Interestingly this test passes on CI [1]:
2025-11-26T03:09:52.0908317Z #366 reg_bounds_rand_ranges_u64_u64:OK 2025-11-26T03:09:52.0925114Z #367 resolve_btfids:OK 2025-11-26T03:09:52.3904190Z #368/1 res_spin_lock_failure/res_spin_lock_arg:OK
I'll take a closer look.
[1] https://github.com/kernel-patches/bpf/actions/runs/19690981192/job/56406840021
Quoted 4 lines
Quoted 4 lines
[...]
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 22 lines · Ihor Solodrai
Quoted 8 lines · Donglin Peng
Quoted 6 lines · Ihor Solodrai
[...]
For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.Quoted 8 lines
The
Makefilefor the bpf selftests also needs be updated too:
https://elixir.bootlin.com/linux/v6.18-rc7/source/tools/testing/selftests/bpf/Makefile#L708This results in the self-test for
resolve_btfidsfailing:
$./vmtest.sh-- ./test_progs -tresolve_btfids-v
...
test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for S (0 != 3)Good catch, thanks.
I remember I noticed this at some point, and then forgot...
Quoted 7 lines
Interestingly this test passes on CI [1]:
2025-11-26T03:09:52.0908317Z #366 reg_bounds_rand_ranges_u64_u64:OK 2025-11-26T03:09:52.0925114Z #367 resolve_btfids:OK 2025-11-26T03:09:52.3904190Z #368/1 res_spin_lock_failure/res_spin_lock_arg:OKI'll take a closer look.
I figured out why this test was flaky.
Even though I removed elf_update() call from resolve_btfids, the ELF was opened with:
elf = elf_begin(fd, ELF_C_RDWR_MMAP, NULL);
And the buffers which resolve_btfids writes to are from Elf_Data returned by elf_getdata(). And so the file might actually get written to in-place, which is why the resolve_btfids test passed for me with no changes to the selftests.
I switched ELF_C_RDWR_MMAP to ELF_C_READ_MMAP_PRIVATE, and then the ELF reliably remains intact (and the test fails). From libelf.h:
ELF_C_READ_MMAP_PRIVATE, /* Read, but memory is writable, results are
not written to the file. */
It makes sense to use this for what resolve_btfids is doing.
I'll fix selftests/bpf/Makefile in the next revision.
Quoted 8 lines
[1] https://github.com/kernel-patches/bpf/actions/runs/19690981192/job/56406840021
Quoted 4 lines
Quoted 4 lines
[...]
Quoted 17 lines · Ihor Solodrai
This series changes
resolve_btfidsand kernel build scripts to enable BTF transformations inresolve_btfids. Main motivation for enhancingresolve_btfidsis to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.Patches #1-#3 in the series are non-functional refactoring in
resolve_btfids. The last patch (#4) makes significant changes inresolve_btfidsand introducesscripts/gen-btf.sh. Implementation changes are described in detail in the patch description.One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)
Quoted 5 lines
Since BTF finalization is now done by
resolve_btfids, it requiresbtf__distill_base()to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag forresolve_btfids.
hi Ihor,
Can you say more about what constitutes BTF finalization and why BTF distillation prior to finalization (i.e. in pahole) isn't workable? Is it the concern that we eliminate types due to filtering, or is it a problem with sorting/tracking type ids? Are there operations we do/anticipate that make prior distillation infeasbile? Thanks!
Quoted 5 lines
Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.
Quoted 7 lines
However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessarybtf__relocate()call on load [6]. It can be avoided by special-casing in-tree module handling inresolve_btfidseither with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.
I would say avoid distillation for in-tree modules if possible, as it imposes runtime costs in relocation/type renumbering on module load. For large modules (amdgpu take a bow) that could be non-trivial time-wise. IMO the build-time costs/complexities are worth paying to avoid a runtime tax on module load.
Quoted 21 lines
[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358Ihor Solodrai (4):
resolve_btfids: rename object btf field tobtf_path
resolve_btfids: factor outload_btf()
resolve_btfids: introduce enumbtf_id_kind
resolve_btfids: change in-place update with raw binary outputMAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 166 ++++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++--------- 5 files changed, 348 insertions(+), 100 deletions(-) create mode 100755 scripts/gen-btf.sh
Ihor Solodrai
reviewerQuoted 23 lines · Alan Maguire
Quoted 17 lines · Ihor Solodrai
This series changes
resolve_btfidsand kernel build scripts to enable BTF transformations inresolve_btfids. Main motivation for enhancingresolve_btfidsis to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.Patches #1-#3 in the series are non-functional refactoring in
resolve_btfids. The last patch (#4) makes significant changes inresolve_btfidsand introducesscripts/gen-btf.sh. Implementation changes are described in detail in the patch description.One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)Quoted 5 lines
Since BTF finalization is now done by
resolve_btfids, it requiresbtf__distill_base()to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag forresolve_btfids.
Quoted 7 lines
hi Ihor,
Can you say more about what constitutes BTF finalization and why BTF distillation prior to finalization (i.e. in pahole) isn't workable? Is it the concern that we eliminate types due to filtering, or is it a problem with sorting/tracking type ids? Are there operations we do/anticipate that make prior distillation infeasbile? Thanks!
Hi Alan,
That's a good question. AFAIU the distillation should be done on the final BTF, after all the transformations (sorting, adding/removing BTF types) have been applied. At least this way we can be sure that the distilled base is valid.
We certainly want BTF generation process to be the same for modules and the kernel, which means that BTF modifications in resolve_btfids have to be applied to module BTF also.
So the question is whether btf2btf will be safe to do *after* distillation, and that of course depends on the specifics.
Let's say pahole generated BTF for a module and a distilled base. If later some types are removed from module BTF, or a new type is added (that might refer to a type absent in distilled base), is the btf/base pair still valid?
My intuition is that it is more reliable to distill the final-final BTF, and so with resolve_btfids taking over kernel BTF finalization it makes sense to do it there. Otherwise we may be upfront limiting ourselves in how module BTF can be changed in resolve_btfids.
What are the reasons to keep distillation in pahole? It's a simple libbpf API call after all. Anything I might be missing?
Quoted 14 lines
Quoted 6 lines
Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.
Quoted 7 lines
However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessarybtf__relocate()call on load [6]. It can be avoided by special-casing in-tree module handling inresolve_btfidseither with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.
Quoted 5 lines
I would say avoid distillation for in-tree modules if possible, as it imposes runtime costs in relocation/type renumbering on module load. For large modules (amdgpu take a bow) that could be non-trivial time-wise. IMO the build-time costs/complexities are worth paying to avoid a runtime tax on module load.
Acked. I still would like to avoid passing flags around if possible.
Is it reasonable to simply check for KBUILD_EXTMOD env var from withing resolve_btfids? Any drawbacks to that?
Thanks.
Quoted 23 lines
Quoted 22 lines
[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358Ihor Solodrai (4):
resolve_btfids: rename object btf field tobtf_path
resolve_btfids: factor outload_btf()
resolve_btfids: introduce enumbtf_id_kind
resolve_btfids: change in-place update with raw binary outputMAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 166 ++++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++--------- 5 files changed, 348 insertions(+), 100 deletions(-) create mode 100755 scripts/gen-btf.sh
Quoted 51 lines · Ihor Solodrai
Quoted 23 lines · Alan Maguire
Quoted 17 lines · Ihor Solodrai
This series changes
resolve_btfidsand kernel build scripts to enable BTF transformations inresolve_btfids. Main motivation for enhancingresolve_btfidsis to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.Patches #1-#3 in the series are non-functional refactoring in
resolve_btfids. The last patch (#4) makes significant changes inresolve_btfidsand introducesscripts/gen-btf.sh. Implementation changes are described in detail in the patch description.One RFC item in this patchset is the --distilled_base [4] handling. Before this patchset .BTF.base was generated and added to target binary by pahole, based on these conditions [5]:
* pahole version >=1.28
* the kernel module is out-of-tree (KBUILD_EXTMOD)Quoted 5 lines
Since BTF finalization is now done by
resolve_btfids, it requiresbtf__distill_base()to happen there. However, in my opinion, it is unnecessary to add and pass through a --distilled_base flag forresolve_btfids.Quoted 7 lines
hi Ihor,
Can you say more about what constitutes BTF finalization and why BTF distillation prior to finalization (i.e. in pahole) isn't workable? Is it the concern that we eliminate types due to filtering, or is it a problem with sorting/tracking type ids? Are there operations we do/anticipate that make prior distillation infeasbile? Thanks!
Hi Alan,
That's a good question. AFAIU the distillation should be done on the final BTF, after all the transformations (sorting, adding/removing BTF types) have been applied. At least this way we can be sure that the distilled base is valid.
We certainly want BTF generation process to be the same for modules and the kernel, which means that BTF modifications in
resolve_btfidshave to be applied to module BTF also.So the question is whether btf2btf will be safe to do *after* distillation, and that of course depends on the specifics.
Let's say pahole generated BTF for a module and a distilled base. If later some types are removed from module BTF, or a new type is added (that might refer to a type absent in distilled base), is the btf/base pair still valid?
Quoted 9 lines
My intuition is that it is more reliable to distill the final-final BTF, and so with
resolve_btfidstaking over kernel BTF finalization it makes sense to do it there. Otherwise we may be upfront limiting ourselves in how module BTF can be changed inresolve_btfids.What are the reasons to keep distillation in pahole? It's a simple libbpf API call after all. Anything I might be missing?
Nope, I think doing distillation as the final step makes sense to me too since we get a clearer picture of the types that the module references. My only reservation was applying distillation to every module (rather than just out-of-tree builds) but it sounds like that's not needed from below.
Quoted 20 lines
Quoted 14 lines
Quoted 6 lines
Logically, any split BTF referring to kernel BTF is not very useful without the .BTF.base, which is why the feature was developed in the first place. Therefore it makes sense to always emit .BTF.base for all modules, unconditionally. This is implemented in the series.
Quoted 7 lines
However it might be argued that .BTF.base is redundant for in-tree
modules: it takes space the module ELF and triggers unnecessarybtf__relocate()call on load [6]. It can be avoided by special-casing in-tree module handling inresolve_btfidseither with a flag or by checking env variables. The trade-off is slight performance impact vs code complexity.Quoted 5 lines
I would say avoid distillation for in-tree modules if possible, as it imposes runtime costs in relocation/type renumbering on module load. For large modules (amdgpu take a bow) that could be non-trivial time-wise. IMO the build-time costs/complexities are worth paying to avoid a runtime tax on module load.
Quoted 5 lines
Acked. I still would like to avoid passing flags around if possible.
Is it reasonable to simply check for
KBUILD_EXTMODenv var from withingresolve_btfids? Any drawbacks to that?
None that I can think of; sounds like a good approach to me and is basically equivalent to what we do now. I'll take a look at v2. Thanks!
Alan
Quoted 27 lines
Thanks.
Quoted 23 lines
Quoted 22 lines
[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
[4] https://docs.kernel.org/bpf/btf.html#btf-base-section
[5] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/Makefile.btf#n29
[6] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/kernel/bpf/btf.c#n6358Ihor Solodrai (4):
resolve_btfids: rename object btf field tobtf_path
resolve_btfids: factor outload_btf()
resolve_btfids: introduce enumbtf_id_kind
resolve_btfids: change in-place update with raw binary outputMAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 166 ++++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 234 +++++++++++++++++++++++--------- 5 files changed, 348 insertions(+), 100 deletions(-) create mode 100755 scripts/gen-btf.sh