resolve_btfids: Support for BTF modifications
Subsystems (7)
Ihor Solodrai
reviewerv4 cover resolve_btfids: Support for BTF modifications
This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.
Patches #1-#4 in the series are non-functional changes in resolve_btfids.
Patch #5 makes kernel build notice pahole version changes between builds.
Patch #6 changes minimum version of pahole required for CONFIG_DEBUG_INFO_BTF to v1.22
Patch #7 makes a small prep change in selftests/bpf build.
The last patch (#8) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. See implementation details in the patch description.
Successful BPF CI run: https://github.com/kernel-patches/bpf/actions/runs/20321339588
[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
---
v3->v4:
- add patch #4: "resolve_btfids: Always build with -Wall -Werror"
- add patch #5: "kbuild: Sync kconfig when PAHOLE_VERSION changes" (Alan)
- fix clang cross-compilation (LKP)
https://lore.kernel.org/bpf/[email protected]/
- remove GEN_BTF env variable (Andrii)
- nits and cleanup in resolve_btfids/main.c (Andrii, Eduard)
- nits in a patch bumping minimum pahole version (Andrii, AI)
v3: https://lore.kernel.org/bpf/[email protected]/
v2->v3:
- add patch #4 bumping minimum pahole version (Andrii, Alan)
- add patch #5 pre-fixing resolve_btfids test (Donglin)
- add GEN_BTF var and assemble RESOLVE_BTFIDS_FLAGS in Makefile.btf (Alan)
- implement --distill_base flag in resolve_btfids, set it depending on KBUILD_EXTMOD in Makefile.btf (Eduard)
- various implementation nits, see the v2 thread for details (Andrii, Eduard)
v2: https://lore.kernel.org/bpf/[email protected]/
v1->v2:
- gen-btf.sh and other shell script fixes (Donglin)
- update selftests build (Donglin)
- generate .BTF.base only when KBUILD_EXTMOD is set (Alan)
- proper endianness handling for cross-compilation
- change elf_begin mode from ELF_C_RDWR_MMAP to ELF_C_READ_MMAP_PRIVATE
- remove compressed_section_fix()
- nit NULL check in patch #3 (suggested by AI)
v1: https://lore.kernel.org/bpf/[email protected]/
Ihor Solodrai (8):
resolve_btfids: Rename object btf field to btf_path
resolve_btfids: Factor out load_btf()
resolve_btfids: Introduce enum btf_id_kind
resolve_btfids: Always build with -Wall -Werror
kbuild: Sync kconfig when PAHOLE_VERSION changes lib/Kconfig.debug: Set the minimum required pahole version to v1.22 selftests/bpf: Run resolve_btfids only for relevant .test.o objects
resolve_btfids: Change in-place update with raw binary output
Documentation/scheduler/sched-ext.rst | 1 - MAINTAINERS | 1 + Makefile | 9 +- init/Kconfig | 2 +- lib/Kconfig.debug | 13 +- scripts/Makefile.btf | 26 +- scripts/Makefile.modfinal | 5 +- scripts/Makefile.vmlinux | 2 +- scripts/gen-btf.sh | 157 ++++++++ scripts/link-vmlinux.sh | 42 +-- tools/bpf/resolve_btfids/Makefile | 3 +- tools/bpf/resolve_btfids/main.c | 356 ++++++++++++------ tools/sched_ext/README.md | 1 - tools/testing/selftests/bpf/.gitignore | 3 + tools/testing/selftests/bpf/Makefile | 11 +- .../selftests/bpf/prog_tests/resolve_btfids.c | 4 +- 16 files changed, 446 insertions(+), 190 deletions(-) create mode 100755 scripts/gen-btf.sh
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v4 1/8 resolve_btfids: Rename object btf field to btf_path
Rename the member of struct object holding the path to BTF data if provided via --btf arg. btf_path is less ambiguous.
--- tools/bpf/resolve_btfids/main.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/bpf/resolve_btfids/main.c +5 -5
@@ -113,7 +113,7 @@ struct btf_id {struct object {const char *path; -const char *btf; +const char *btf_path;const char *base_btf_path;struct {
@@ -550,11 +550,11 @@ static int symbols_resolve(struct object *obj) } } - btf =btf__parse_split (obj->btf ?: obj->path, base_btf); + btf =btf__parse_split (obj->btf_path ?: obj->path, base_btf); err =libbpf_get_error (btf);if (err) {pr_err ("FAILED: load BTF from %s: %s\n" , - obj->btf ?: obj->path,strerror (-err)); + obj->btf_path ?: obj->path,strerror (-err)); goto out; }
@@ -790,8 +790,8 @@ int main(int argc, const char **argv)struct option btfid_options[] = {OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show errors, etc)" ), -OPT_STRING (0 ,"btf" , &obj.btf,"BTF data" , -"BTF data" ), +OPT_STRING (0 ,"btf" , &obj.btf_path,"file" , +"path to a file with input BTF data" ),OPT_STRING ('b' ,"btf_base" , &obj.base_btf_path,"file" ,"path of file providing base BTF" ), OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v4 2/8 resolve_btfids: Factor out load_btf()
In reply to Ihor SolodraiIncrease the lifetime of parsed BTF in resolve_btfids by factoring load_btf() routine out of symbols_resolve() and storing the base_btf and btf pointers in the struct object.
--- tools/bpf/resolve_btfids/main.c | 47 ++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 13 deletions(-)
tools/bpf/resolve_btfids/main.c +34 -13
@@ -116,6 +116,9 @@ struct object {const char *btf_path;const char *base_btf_path; +struct btf *btf; +struct btf *base_btf; +struct {int fd;Elf *elf;
@@ -529,16 +532,10 @@ static int symbols_collect(struct object *obj)return 0 ; } -static int symbols_resolve (struct object *obj) +static int load_btf (struct object *obj) { -int nr_typedefs = obj->nr_typedefs; -int nr_structs = obj->nr_structs; -int nr_unions = obj->nr_unions; -int nr_funcs = obj->nr_funcs; -struct btf *base_btf =NULL ; -int err, type_id; -struct btf *btf; -__u32 nr_types; +struct btf *base_btf =NULL , *btf =NULL ; +int err;if (obj->base_btf_path) { base_btf =btf__parse (obj->base_btf_path,NULL );
@@ -546,7 +543,7 @@ static int symbols_resolve(struct object *obj)if (err) {pr_err ("FAILED: load base BTF from %s: %s\n" , obj->base_btf_path,strerror (-err)); -return -1 ; + goto out_err; } }
@@ -555,9 +552,30 @@ static int symbols_resolve(struct object *obj)if (err) {pr_err ("FAILED: load BTF from %s: %s\n" , obj->btf_path ?: obj->path,strerror (-err)); - goto out; + goto out_err; } + obj->base_btf = base_btf; + obj->btf = btf; + +return 0 ; + +out_err: +btf__free (base_btf); +btf__free (btf); +return err; +} + +static int symbols_resolve (struct object *obj) +{ +int nr_typedefs = obj->nr_typedefs; +int nr_structs = obj->nr_structs; +int nr_unions = obj->nr_unions; +int nr_funcs = obj->nr_funcs; +struct btf *btf = obj->btf; +int err, type_id; +__u32 nr_types; + err =-1 ; nr_types =btf__type_cnt (btf);
@@ -615,8 +633,6 @@ static int symbols_resolve(struct object *obj) err =0 ; out: -btf__free (base_btf); -btf__free (btf);return err; }
@@ -824,6 +840,9 @@ int main(int argc, const char **argv)if (symbols_collect (&obj)) goto out; +if (load_btf (&obj)) + goto out; +if (symbols_resolve (&obj)) goto out;
@@ -833,6 +852,8 @@ int main(int argc, const char **argv)if (!(fatal_warnings && warnings)) err =0 ; out: +btf__free (obj.base_btf); +btf__free (obj.btf);if (obj.efile.elf) {elf_end (obj.efile.elf);close (obj.efile.fd);
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v4 3/8 resolve_btfids: Introduce enum btf_id_kind
In reply to Ihor SolodraiInstead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.
--- tools/bpf/resolve_btfids/main.c | 83 ++++++++++++++++++++++++--------- 1 file changed, 60 insertions(+), 23 deletions(-)
tools/bpf/resolve_btfids/main.c +60 -23
@@ -98,6 +98,13 @@# error "Unknown machine endianness!"#endif +enum btf_id_kind { +BTF_ID_KIND_NONE , +BTF_ID_KIND_SYM , +BTF_ID_KIND_SET , +BTF_ID_KIND_SET8 +}; +struct btf_id {struct rb_node rb_node;char *name;
@@ -105,9 +112,8 @@ struct btf_id {int id;int cnt; }; +enum btf_id_kind kind;int addr_cnt; -bool is_set; -bool is_set8;Elf64_Addr addr[ADDR_CNT ]; };
@@ -197,8 +203,10 @@ static struct btf_id *btf_id__find(struct rb_root *root, const char *name)return NULL ; } -static struct btf_id * -btf_id__add (struct rb_root *root,char *name,bool unique) +static struct btf_id *__btf_id__add (struct rb_root *root, +char *name, +enum btf_id_kind kind, +bool unique) {struct rb_node **p = &root->rb_node;struct rb_node *parent =NULL ;
@@ -221,12 +229,23 @@ btf_id__add(struct rb_root *root, char *name, bool unique)if (id) {pr_debug ("adding symbol %s\n" , name); id->name = name; + id->kind = kind;rb_link_node (&id->rb_node, parent, p);rb_insert_color (&id->rb_node, root); }return id; } +static inline struct btf_id *btf_id__add (struct rb_root *root,char *name,enum btf_id_kind kind) +{ +return __btf_id__add (root, name, kind, false); +} + +static inline struct btf_id *btf_id__add_unique (struct rb_root *root,char *name,enum btf_id_kind kind) +{ +return __btf_id__add (root, name, kind, true); +} +static char *get_id (const char *prefix_end) {/*
@@ -260,22 +279,36 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) { +int len =strlen (name); +int prefixlen; +char *id; +/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; -int len =strlen (name); +switch (kind) { +case BTF_ID_KIND_SET : + prefixlen =sizeof (BTF_SET "__" ) -1 ; +break ; +case BTF_ID_KIND_SET8 : + prefixlen =sizeof (BTF_SET8 "__" ) -1 ; +break ; +default : +pr_err ("Unexpected kind %d passed to %s() for symbol %s\n" , kind, __func__, name); +return NULL ; + } + id = name + prefixlen -1 ;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); +return btf_id__add_unique (&obj->sets, id, kind); }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size)
@@ -288,7 +321,7 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; } -return btf_id__add (root, id, false); +return btf_id__add (root, id,BTF_ID_KIND_SYM ); }/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
@@ -491,35 +524,31 @@ static int symbols_collect(struct object *obj) id =add_symbol (&obj->funcs, prefix,sizeof (BTF_FUNC ) -1 );/* set8 */ }else if (!strncmp (prefix,BTF_SET8 ,sizeof (BTF_SET8 ) -1 )) { - id =add_set (obj, prefix, true); + id =add_set (obj, prefix,BTF_ID_KIND_SET8 );/* * SET8 objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (uint64_t ) -1 ; - id->is_set8 = true; - }/* set */ }else if (!strncmp (prefix,BTF_SET ,sizeof (BTF_SET ) -1 )) { - id =add_set (obj, prefix, false); + id =add_set (obj, prefix,BTF_ID_KIND_SET );/* * SET objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (int ) -1 ; - id->is_set = true; - } }else {pr_err ("FAILED unsupported prefix %s\n" , prefix);return -1 ; }if (!id) -return -ENOMEM ; +return -EINVAL ;if (id->addr_cnt >=ADDR_CNT ) { pr_err("FAILED symbol %s crossed the number of allowed lists\n",
@@ -643,7 +672,7 @@ static int id_patch(struct object *obj, struct btf_id *id)int i;/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind !=BTF_ID_KIND_SET && id->kind !=BTF_ID_KIND_SET8 ) {pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }
@@ -696,6 +725,7 @@ static int sets_patch(struct object *obj) {Elf_Data *data = obj->efile.idlist;struct rb_node *next; +int cnt; next =rb_first (&obj->sets);while (next) {
@@ -715,11 +745,15 @@ static int sets_patch(struct object *obj)return -1 ; } -if (id->is_set) { +switch (id->kind) { +case BTF_ID_KIND_SET : set = data->d_buf + off; + cnt = set->cnt;qsort (set->ids, set->cnt,sizeof (set->ids[0 ]), cmp_id); - }else { +break ; +case BTF_ID_KIND_SET8 : set8 = data->d_buf + off; + cnt = set8->cnt;/* * Make sure id is at the beginning of the pairs * struct, otherwise the below qsort would not work.
@@ -744,10 +778,13 @@ static int sets_patch(struct object *obj)bswap_32 (set8->pairs[i].flags); } } +break ; +default : +pr_err ("Unexpected btf_id_kind %d for set '%s'\n" , id->kind, id->name); +return -1 ; } -pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , - off, id->is_set ? set->cnt : set8->cnt, id->name); +pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , off, cnt, id->name); next =rb_next (next); }
Signature · 1 line
2.52.0
[email protected]
AI reviewRe: [PATCH bpf-next v4 3/8] resolve_btfids: Introduce enum btf_id_kind
Is this off by one? The original code computed:
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0
[ ... ]
Quoted 21 lines
tools/bpf/resolve_btfids/main.c +8
-3@@ -260,22 +279,36 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) { +int len =strlen (name); +int prefixlen; +char *id; +/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; -int len =strlen (name); +switch (kind) { +case BTF_ID_KIND_SET : + prefixlen =sizeof (BTF_SET "__" ) -1 ;
tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 305
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +10
-0+break ; +case BTF_ID_KIND_SET8 : + prefixlen =sizeof (BTF_SET8 "__" ) -1 ; +break ; +default : +pr_err ("Unexpected kind %d passed to %s() for symbol %s\n" , kind, __func__, name); +return NULL ; + } + + id = name + prefixlen -1 ;
Is this off by one? The original code computed:
id = name + sizeof(BTF_SET "__") - 1
For BTF_SET ("set"), this gives: name + sizeof("set__") - 1 = name + 6 - 1 = name + 5
The new code computes:
prefixlen = sizeof(BTF_SET "__") - 1 = 6 - 1 = 5 id = name + prefixlen - 1 = name + 5 - 1 = name + 4
The -1 appears to be applied twice now, once in the prefixlen assignment and again when computing id. Should this line be:
id = name + prefixlen;
tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 312
Quoted 8 lines
tools/bpf/resolve_btfids/main.c +1
-1if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); +return btf_id__add_unique (&obj->sets, id, kind); }
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/20321956014
Ihor Solodrai
reviewerRe: [PATCH bpf-next v4 3/8] resolve_btfids: Introduce enum btf_id_kind
Quoted 47 lines · On 12/17/25 4:47 PM, [email protected] wrote:
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0[ ... ]
Quoted 21 lines
tools/bpf/resolve_btfids/main.c +8
-3@@ -260,22 +279,36 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) { +int len =strlen (name); +int prefixlen; +char *id; +/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; -int len =strlen (name); +switch (kind) { +case BTF_ID_KIND_SET : + prefixlen =sizeof (BTF_SET "__" ) -1 ;tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 305
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +10
-0+break ; +case BTF_ID_KIND_SET8 : + prefixlen =sizeof (BTF_SET8 "__" ) -1 ; +break ; +default : +pr_err ("Unexpected kind %d passed to %s() for symbol %s\n" , kind, __func__, name); +return NULL ; + } + + id = name + prefixlen -1 ;Is this off by one? The original code computed:
id = name + sizeof(BTF_SET "__") - 1For
BTF_SET("set"), this gives: name +sizeof("set__")- 1 = name + 6 - 1 = name + 5The new code computes:
Quoted 7 lines
prefixlen =
sizeof(BTF_SET "__")- 1 = 6 - 1 = 5 id = name + prefixlen - 1 = name + 5 - 1 = name + 4The -1 appears to be applied twice now, once in the prefixlen assignment and again when computing id. Should this line be:
id = name + prefixlen;
Yes, this is an off-by-one bug.
Good catch, Opus 4.5. Sonnet 4.5 missed this in v3.
This was "harmless", since the names stored in the trees don't affect the output. The comparison between the names still works, as they all simply got a "_" prefix. But this only makes the bug sneaky, which is very bad.
The suggested fix is meh though, IMO a better one is:
tools/bpf/resolve_btfids/main.c +2 -2
@@ -295,10 +295,10 @@ static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind k*/ switch (kind) {case BTF_ID_KIND_SET : - prefixlen =sizeof (BTF_SET "__" ) -1 ; + prefixlen =sizeof (BTF_SET "__" );break ;case BTF_ID_KIND_SET8 : - prefixlen =sizeof (BTF_SET8 "__" ) -1 ; + prefixlen =sizeof (BTF_SET8 "__" );break ;default :pr_err ("Unexpected kind %d passed to %s() for symbol %s\n" , kind, __func__, name);
Quoted 16 lines
tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 312
Quoted 9 lines
tools/bpf/resolve_btfids/main.c +1
-1if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); +return btf_id__add_unique (&obj->sets, id, kind); }---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/20321956014
Re: [PATCH bpf-next v4 3/8] resolve_btfids: Introduce enum btf_id_kind
Quoted 59 lines · Ihor Solodrai
Quoted 47 lines · On 12/17/25 4:47 PM, [email protected] wrote:
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0[ ... ]
Quoted 21 lines
tools/bpf/resolve_btfids/main.c +8
-3@@ -260,22 +279,36 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) { +int len =strlen (name); +int prefixlen; +char *id; +/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; -int len =strlen (name); +switch (kind) { +case BTF_ID_KIND_SET : + prefixlen =sizeof (BTF_SET "__" ) -1 ;tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 305
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +10
-0+break ; +case BTF_ID_KIND_SET8 : + prefixlen =sizeof (BTF_SET8 "__" ) -1 ; +break ; +default : +pr_err ("Unexpected kind %d passed to %s() for symbol %s\n" , kind, __func__, name); +return NULL ; + } + + id = name + prefixlen -1 ;Is this off by one? The original code computed:
id = name + sizeof(BTF_SET "__") - 1For
BTF_SET("set"), this gives: name +sizeof("set__")- 1 = name + 6 - 1 = name + 5The new code computes:
Quoted 7 lines
prefixlen =
sizeof(BTF_SET "__")- 1 = 6 - 1 = 5 id = name + prefixlen - 1 = name + 5 - 1 = name + 4The -1 appears to be applied twice now, once in the prefixlen assignment and again when computing id. Should this line be:
id = name + prefixlen;Yes, this is an off-by-one bug.
Quoted 8 lines
Good catch, Opus 4.5. Sonnet 4.5 missed this in v3.
This was "harmless", since the names stored in the trees don't affect the output. The comparison between the names still works, as they all simply got a "_" prefix. But this only makes the bug sneaky, which is very bad.
The suggested fix is meh though, IMO a better one is:
the bug is not in determining prefix length, but in using it to find where id starts in the string, so the fix should be
id = name + prefixlen;
prefixlen is calculated correctly, IMO
Quoted 36 lines
tools/bpf/resolve_btfids/main.c +2
-2@@ -295,10 +295,10 @@ static struct btf_id *add_set(struct object *obj, char *name, enum btf_id_kind k*/ switch (kind) {case BTF_ID_KIND_SET : - prefixlen =sizeof (BTF_SET "__" ) -1 ; + prefixlen =sizeof (BTF_SET "__" );break ;case BTF_ID_KIND_SET8 : - prefixlen =sizeof (BTF_SET8 "__" ) -1 ; + prefixlen =sizeof (BTF_SET8 "__" );break ;default :pr_err ("Unexpected kind %d passed to %s() for symbol %s\n" , kind, __func__, name);Quoted 17 lines
tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 312
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +1
-1if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); +return btf_id__add_unique (&obj->sets, id, kind); }---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.mdCI run summary: https://github.com/kernel-patches/bpf/actions/runs/20321956014
Ihor Solodrai
reviewerRe: [PATCH bpf-next v4 3/8] resolve_btfids: Introduce enum btf_id_kind
Quoted 67 lines · Andrii Nakryiko
Quoted 66 lines · Ihor Solodrai
Quoted 47 lines · On 12/17/25 4:47 PM, [email protected] wrote:
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0[ ... ]
Quoted 21 lines
tools/bpf/resolve_btfids/main.c +8
-3@@ -260,22 +279,36 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) { +int len =strlen (name); +int prefixlen; +char *id; +/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; -int len =strlen (name); +switch (kind) { +case BTF_ID_KIND_SET : + prefixlen =sizeof (BTF_SET "__" ) -1 ;tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 305
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +10
-0+break ; +case BTF_ID_KIND_SET8 : + prefixlen =sizeof (BTF_SET8 "__" ) -1 ; +break ; +default : +pr_err ("Unexpected kind %d passed to %s() for symbol %s\n" , kind, __func__, name); +return NULL ; + } + + id = name + prefixlen -1 ;Is this off by one? The original code computed:
id = name + sizeof(BTF_SET "__") - 1For
BTF_SET("set"), this gives: name +sizeof("set__")- 1 = name + 6 - 1 = name + 5The new code computes:
Quoted 7 lines
prefixlen =
sizeof(BTF_SET "__")- 1 = 6 - 1 = 5 id = name + prefixlen - 1 = name + 5 - 1 = name + 4The -1 appears to be applied twice now, once in the prefixlen assignment and again when computing id. Should this line be:
id = name + prefixlen;Yes, this is an off-by-one bug.
Good catch, Opus 4.5. Sonnet 4.5 missed this in v3.
This was "harmless", since the names stored in the trees don't affect the output. The comparison between the names still works, as they all simply got a "_" prefix. But this only makes the bug sneaky, which is very bad.
Quoted 8 lines
Quoted 1 line
The suggested fix is meh though, IMO a better one is:
the bug is not in determining prefix length, but in using it to find where id starts in the string, so the fix should be
id = name + prefixlen;
prefixlen is calculated correctly, IMO
Aaahh, because the null byte is counted by sizeof, right... I missed that.
Quoted 4 lines
Quoted 4 lines
[...]
Ihor Solodrai
reviewerPATCH v4 4/8 resolve_btfids: Always build with -Wall -Werror
In reply to Ihor Solodrairesolve_btfids builds without compiler warnings currently, so let's enforce this for future changes with '-Wall -Werror' flags [1].
[1] https://lore.kernel.org/bpf/[email protected]/
--- tools/bpf/resolve_btfids/Makefile | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
tools/bpf/resolve_btfids/Makefile +2 -1
@@ -70,7 +70,8 @@ HOSTCFLAGS_resolve_btfids += -g \ -I$(srctree)/tools/include/uapi \ -I$(LIBBPF_INCLUDE) \ -I$(SUBCMD_INCLUDE) \ - $(LIBELF_FLAGS) + $(LIBELF_FLAGS) \ + -Wall -Werror LIBS = $(LIBELF_LIBS) -lz
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v4 5/8 kbuild: Sync kconfig when PAHOLE_VERSION changes
In reply to Ihor SolodraiThis patch implements kconfig re-sync when the pahole version changes between builds, similar to how it happens for compiler version change via CC_VERSION_TEXT.
Define PAHOLE_VERSION in the top-level Makefile and export it for config builds. Set CONFIG_PAHOLE_VERSION default to the exported variable.
Kconfig records the PAHOLE_VERSION value ininclude/config/auto.conf.cmd [1].
The Makefile includes auto.conf.cmd, so if PAHOLE_VERSION changes between builds, make detects a dependency change and triggers syncconfig to update the kconfig [2].
For external module builds, add a warning message in the prepare target, similar to the existing compiler version mismatch warning.
Note that if pahole is not installed or available, PAHOLE_VERSION is set to 0 by pahole-version.sh, so the (un)installation of pahole is treated as a version change.
See previous discussions for context [3].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/preprocess.c?h=v6.18#n91
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n815
[3] https://lore.kernel.org/bpf/[email protected]/
--- Makefile | 9 ++++++++- init/Kconfig | 2 +- 2 files changed, 9 insertions(+), 2 deletions(-)
Makefile +8 -1
@@ -713,6 +713,7 @@ endif # upgrade. CC_VERSION_TEXT = $(subst $(pound),,$(shell LC_ALL=C $(CC) --version 2>/dev/null | head -n 1)) RUSTC_VERSION_TEXT = $(subst $(pound),,$(shell $(RUSTC) --version 2>/dev/null)) +PAHOLE_VERSION = $(shell $(srctree)/scripts/pahole-version.sh $(PAHOLE)) ifneq ($(findstring clang,$(CC_VERSION_TEXT)),) include $(srctree)/scripts/Makefile.clang
@@ -733,7 +734,7 @@ ifdef config-build # KBUILD_DEFCONFIG may point out an alternative default configuration # used for 'make defconfig' include $(srctree)/arch/$(SRCARCH)/Makefile -export KBUILD_DEFCONFIG KBUILD_KCONFIG CC_VERSION_TEXT RUSTC_VERSION_TEXT +export KBUILD_DEFCONFIG KBUILD_KCONFIG CC_VERSION_TEXT RUSTC_VERSION_TEXT PAHOLE_VERSION config: outputmakefile scripts_basic FORCE $(Q)$(MAKE) $(build)=scripts/kconfig $@
@@ -1921,12 +1922,18 @@ clean: private rm-files := Module.symvers modules.nsdeps compile_commands.json PHONY += prepare # now expand this into a simple variable to reduce the cost of shell evaluations prepare: CC_VERSION_TEXT := $(CC_VERSION_TEXT) +prepare: PAHOLE_VERSION := $(PAHOLE_VERSION) prepare: @if [ "$(CC_VERSION_TEXT)" != "$(CONFIG_CC_VERSION_TEXT)" ]; then \ echo >&2 "warning: the compiler differs from the one used to build the kernel"; \ echo >&2 " The kernel was built by: $(CONFIG_CC_VERSION_TEXT)"; \ echo >&2 " You are using: $(CC_VERSION_TEXT)"; \ fi + @if [ "$(PAHOLE_VERSION)" != "$(CONFIG_PAHOLE_VERSION)" ]; then \ + echo >&2 "warning: pahole version differs from the one used to build the kernel"; \ + echo >&2 " The kernel was built with: $(CONFIG_PAHOLE_VERSION)"; \ + echo >&2 " You are using: $(PAHOLE_VERSION)"; \ + fi PHONY += help help:
init/Kconfig +1 -1
@@ -171,7 +171,7 @@ config RUSTC_HAS_FILE_AS_C_STR config PAHOLE_VERSION int - default $(shell,$(srctree)/scripts/pahole-version.sh $(PAHOLE)) + default "$(PAHOLE_VERSION)" config CONSTRUCTORS bool
Signature · 1 line
2.52.0
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 5/8] kbuild: Sync kconfig when PAHOLE_VERSION changes
Quoted 22 lines · Ihor Solodrai
This patch implements kconfig re-sync when the pahole version changes between builds, similar to how it happens for compiler version change via
CC_VERSION_TEXT.Define
PAHOLE_VERSIONin the top-levelMakefileand export it for config builds. SetCONFIG_PAHOLE_VERSIONdefault to the exported variable.
Kconfigrecords thePAHOLE_VERSIONvalue ininclude/config/auto.conf.cmd[1].The
Makefileincludes auto.conf.cmd, so ifPAHOLE_VERSIONchanges between builds, make detects a dependency change and triggers syncconfig to update the kconfig [2].For external module builds, add a warning message in the prepare target, similar to the existing compiler version mismatch warning.
Note that if pahole is not installed or available,
PAHOLE_VERSIONis set to 0 bypahole-version.sh, so the (un)installation of pahole is treated as a version change.
Quoted 8 lines
See previous discussions for context [3].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/preprocess.c?h=v6.18#n91
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n815
[3] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>---
When building BPF selftest modules the pahole version change was detected, but it seems that BTF rebuild was not triggered:
$ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j)
make[1]: Entering directory '/home/ezingerman/bpf-next'
make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods'
CC [M] bpf_testmod.o
CC [M] bpf_test_no_cfi.o
CC [M] bpf_test_modorder_x.o
CC [M] bpf_test_modorder_y.o
CC [M] bpf_test_rqspinlock.o
MODPOST Module.symvers
CC [M] bpf_testmod.mod.o
CC [M] .module-common.o
CC [M] bpf_test_no_cfi.mod.o
CC [M] bpf_test_modorder_x.mod.o
CC [M] bpf_test_modorder_y.mod.o
CC [M] bpf_test_rqspinlock.mod.o
LD [M] bpf_test_modorder_x.ko
LD [M] bpf_testmod.ko
LD [M] bpf_test_modorder_y.ko
LD [M] bpf_test_no_cfi.ko
BTF [M] bpf_test_modorder_x.ko
LD [M] bpf_test_rqspinlock.ko
BTF bpf_test_modorder_x.ko
BTF [M] bpf_test_no_cfi.ko
BTF [M] bpf_test_modorder_y.ko
BTF [M] bpf_testmod.ko
BTF bpf_test_no_cfi.ko
BTF bpf_test_modorder_y.ko
BTF [M] bpf_test_rqspinlock.ko
BTF bpf_testmod.ko
BTF bpf_test_rqspinlock.ko
BTFIDS bpf_test_modorder_x.ko
BTFIDS bpf_test_modorder_y.ko
BTFIDS bpf_test_no_cfi.ko
BTFIDS bpf_testmod.ko
OBJCOPY bpf_test_modorder_x.ko.BTF
BTFIDS bpf_test_rqspinlock.ko
OBJCOPY bpf_test_no_cfi.ko.BTF
OBJCOPY bpf_test_modorder_y.ko.BTF
OBJCOPY bpf_testmod.ko.BTF
OBJCOPY bpf_test_rqspinlock.ko.BTF
make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods'
make[1]: Leaving directory '/home/ezingerman/bpf-next'
[~/bpf-next]
$ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j)
make[1]: Entering directory '/home/ezingerman/bpf-next'
make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods'
make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods'
make[1]: Leaving directory '/home/ezingerman/bpf-next'... update pahole from version 131 to 132 ...
[~/bpf-next]
$ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j)
make[1]: Entering directory '/home/ezingerman/bpf-next'
make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods'
warning: pahole version differs from the one used to build the kernel
The kernel was built with: 131
You are using: 132
make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods'
make[1]: Leaving directory '/home/ezingerman/bpf-next'Is this an expected behavior?
Ihor Solodrai
reviewerRe: [PATCH bpf-next v4 5/8] kbuild: Sync kconfig when PAHOLE_VERSION changes
Quoted 96 lines · Eduard Zingerman
Quoted 22 lines · Ihor Solodrai
This patch implements kconfig re-sync when the pahole version changes between builds, similar to how it happens for compiler version change via
CC_VERSION_TEXT.Define
PAHOLE_VERSIONin the top-levelMakefileand export it for config builds. SetCONFIG_PAHOLE_VERSIONdefault to the exported variable.
Kconfigrecords thePAHOLE_VERSIONvalue ininclude/config/auto.conf.cmd[1].The
Makefileincludes auto.conf.cmd, so ifPAHOLE_VERSIONchanges between builds, make detects a dependency change and triggers syncconfig to update the kconfig [2].For external module builds, add a warning message in the prepare target, similar to the existing compiler version mismatch warning.
Note that if pahole is not installed or available,
PAHOLE_VERSIONis set to 0 bypahole-version.sh, so the (un)installation of pahole is treated as a version change.Quoted 8 lines
See previous discussions for context [3].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/preprocess.c?h=v6.18#n91
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n815
[3] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>---
When building BPF selftest modules the pahole version change was detected, but it seems that BTF rebuild was not triggered:
$ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j) make[1]: Entering directory '/home/ezingerman/bpf-next' make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' CC [M] bpf_testmod.o CC [M] bpf_test_no_cfi.o CC [M] bpf_test_modorder_x.o CC [M] bpf_test_modorder_y.o CC [M] bpf_test_rqspinlock.o MODPOST Module.symvers CC [M] bpf_testmod.mod.o CC [M] .module-common.o CC [M] bpf_test_no_cfi.mod.o CC [M] bpf_test_modorder_x.mod.o CC [M] bpf_test_modorder_y.mod.o CC [M] bpf_test_rqspinlock.mod.o LD [M] bpf_test_modorder_x.ko LD [M] bpf_testmod.ko LD [M] bpf_test_modorder_y.ko LD [M] bpf_test_no_cfi.ko BTF [M] bpf_test_modorder_x.ko LD [M] bpf_test_rqspinlock.ko BTF bpf_test_modorder_x.ko BTF [M] bpf_test_no_cfi.ko BTF [M] bpf_test_modorder_y.ko BTF [M] bpf_testmod.ko BTF bpf_test_no_cfi.ko BTF bpf_test_modorder_y.ko BTF [M] bpf_test_rqspinlock.ko BTF bpf_testmod.ko BTF bpf_test_rqspinlock.ko BTFIDS bpf_test_modorder_x.ko BTFIDS bpf_test_modorder_y.ko BTFIDS bpf_test_no_cfi.ko BTFIDS bpf_testmod.ko OBJCOPY bpf_test_modorder_x.ko.BTF BTFIDS bpf_test_rqspinlock.ko OBJCOPY bpf_test_no_cfi.ko.BTF OBJCOPY bpf_test_modorder_y.ko.BTF OBJCOPY bpf_testmod.ko.BTF OBJCOPY bpf_test_rqspinlock.ko.BTF make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[1]: Leaving directory '/home/ezingerman/bpf-next' [~/bpf-next] $ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j) make[1]: Entering directory '/home/ezingerman/bpf-next' make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[1]: Leaving directory '/home/ezingerman/bpf-next'... update pahole from version 131 to 132 ...
[~/bpf-next]
$ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j) make[1]: Entering directory '/home/ezingerman/bpf-next' make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' warning: pahole version differs from the one used to build the kernel The kernel was built with: 131 You are using: 132 make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[1]: Leaving directory '/home/ezingerman/bpf-next'
Quoted 1 line
Is this an expected behavior?
Yes, it's expected.
I simply repeated the logic used for compiler version change: for external modules only the warning is printed.
See https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n1857
Re: [PATCH bpf-next v4 5/8] kbuild: Sync kconfig when PAHOLE_VERSION changes
Quoted 97 lines · Ihor Solodrai
Quoted 96 lines · Eduard Zingerman
Quoted 22 lines · Ihor Solodrai
This patch implements kconfig re-sync when the pahole version changes between builds, similar to how it happens for compiler version change via
CC_VERSION_TEXT.Define
PAHOLE_VERSIONin the top-levelMakefileand export it for config builds. SetCONFIG_PAHOLE_VERSIONdefault to the exported variable.
Kconfigrecords thePAHOLE_VERSIONvalue ininclude/config/auto.conf.cmd[1].The
Makefileincludes auto.conf.cmd, so ifPAHOLE_VERSIONchanges between builds, make detects a dependency change and triggers syncconfig to update the kconfig [2].For external module builds, add a warning message in the prepare target, similar to the existing compiler version mismatch warning.
Note that if pahole is not installed or available,
PAHOLE_VERSIONis set to 0 bypahole-version.sh, so the (un)installation of pahole is treated as a version change.Quoted 8 lines
See previous discussions for context [3].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/preprocess.c?h=v6.18#n91
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n815
[3] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>---
When building BPF selftest modules the pahole version change was detected, but it seems that BTF rebuild was not triggered:
$ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j) make[1]: Entering directory '/home/ezingerman/bpf-next' make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' CC [M] bpf_testmod.o CC [M] bpf_test_no_cfi.o CC [M] bpf_test_modorder_x.o CC [M] bpf_test_modorder_y.o CC [M] bpf_test_rqspinlock.o MODPOST Module.symvers CC [M] bpf_testmod.mod.o CC [M] .module-common.o CC [M] bpf_test_no_cfi.mod.o CC [M] bpf_test_modorder_x.mod.o CC [M] bpf_test_modorder_y.mod.o CC [M] bpf_test_rqspinlock.mod.o LD [M] bpf_test_modorder_x.ko LD [M] bpf_testmod.ko LD [M] bpf_test_modorder_y.ko LD [M] bpf_test_no_cfi.ko BTF [M] bpf_test_modorder_x.ko LD [M] bpf_test_rqspinlock.ko BTF bpf_test_modorder_x.ko BTF [M] bpf_test_no_cfi.ko BTF [M] bpf_test_modorder_y.ko BTF [M] bpf_testmod.ko BTF bpf_test_no_cfi.ko BTF bpf_test_modorder_y.ko BTF [M] bpf_test_rqspinlock.ko BTF bpf_testmod.ko BTF bpf_test_rqspinlock.ko BTFIDS bpf_test_modorder_x.ko BTFIDS bpf_test_modorder_y.ko BTFIDS bpf_test_no_cfi.ko BTFIDS bpf_testmod.ko OBJCOPY bpf_test_modorder_x.ko.BTF BTFIDS bpf_test_rqspinlock.ko OBJCOPY bpf_test_no_cfi.ko.BTF OBJCOPY bpf_test_modorder_y.ko.BTF OBJCOPY bpf_testmod.ko.BTF OBJCOPY bpf_test_rqspinlock.ko.BTF make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[1]: Leaving directory '/home/ezingerman/bpf-next' [~/bpf-next] $ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j) make[1]: Entering directory '/home/ezingerman/bpf-next' make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[1]: Leaving directory '/home/ezingerman/bpf-next'... update pahole from version 131 to 132 ...
[~/bpf-next]
$ (cd ./tools/testing/selftests/bpf/test_kmods/; make -j) make[1]: Entering directory '/home/ezingerman/bpf-next' make[2]: Entering directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' warning: pahole version differs from the one used to build the kernel The kernel was built with: 131 You are using: 132 make[2]: Leaving directory '/home/ezingerman/bpf-next/tools/testing/selftests/bpf/test_kmods' make[1]: Leaving directory '/home/ezingerman/bpf-next'
Quoted 9 lines
Quoted 1 line
Is this an expected behavior?
Yes, it's expected.
I simply repeated the logic used for compiler version change: for external modules only the warning is printed.
See https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n1857
Ok, it does rebuild BTF for the kernel itself.
Re: [PATCH bpf-next v4 5/8] kbuild: Sync kconfig when PAHOLE_VERSION changes
In reply to Ihor SolodraiQuoted 29 lines · Ihor Solodrai
This patch implements kconfig re-sync when the pahole version changes between builds, similar to how it happens for compiler version change via
CC_VERSION_TEXT.Define
PAHOLE_VERSIONin the top-levelMakefileand export it for config builds. SetCONFIG_PAHOLE_VERSIONdefault to the exported variable.
Kconfigrecords thePAHOLE_VERSIONvalue ininclude/config/auto.conf.cmd[1].The
Makefileincludes auto.conf.cmd, so ifPAHOLE_VERSIONchanges between builds, make detects a dependency change and triggers syncconfig to update the kconfig [2].For external module builds, add a warning message in the prepare target, similar to the existing compiler version mismatch warning.
Note that if pahole is not installed or available,
PAHOLE_VERSIONis set to 0 bypahole-version.sh, so the (un)installation of pahole is treated as a version change.See previous discussions for context [3].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/kconfig/preprocess.c?h=v6.18#n91
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/Makefile?h=v6.18#n815
[3] https://lore.kernel.org/bpf/[email protected]/
Quoted 6 lines
Signed-off-by Ihor Solodrai <[email protected]>--- Makefile | 9 ++++++++- init/Kconfig | 2 +- 2 files changed, 9 insertions(+), 2 deletions(-)
This is great, we should have done that a long time ago :)
[...]
Ihor Solodrai
reviewerPATCH v4 6/8 lib/Kconfig.debug: Set the minimum required pahole version to v1.22
In reply to Ihor SolodraiSubsequent patches in the series change vmlinux linking scripts to unconditionally pass --btf_encode_detached to pahole, which was introduced in v1.22 [1][2].
This change allows to remove PAHOLE_HAS_SPLIT_BTF Kconfig option and other checks of older pahole versions.
[1] https://github.com/acmel/dwarves/releases/tag/v1.22
[2] https://lore.kernel.org/bpf/[email protected]/
--- Documentation/scheduler/sched-ext.rst | 1 - lib/Kconfig.debug | 13 ++++--------- scripts/Makefile.btf | 9 +-------- tools/sched_ext/README.md | 1 - 4 files changed, 5 insertions(+), 19 deletions(-)
Documentation/scheduler/sched-ext.rst +0 -1
@@ -43,7 +43,6 @@ options should be enabled to use sched_ext: CONFIG_DEBUG_INFO_BTF=y CONFIG_BPF_JIT_ALWAYS_ON=y CONFIG_BPF_JIT_DEFAULT_ON=y - CONFIG_PAHOLE_HAS_SPLIT_BTF=y CONFIG_PAHOLE_HAS_BTF_TAG=y sched_ext is used only when the BPF scheduler is loaded and running.
lib/Kconfig.debug +4 -9
@@ -388,18 +388,13 @@ config DEBUG_INFO_BTF depends on !DEBUG_INFO_SPLIT && !DEBUG_INFO_REDUCED depends on !GCC_PLUGIN_RANDSTRUCT || COMPILE_TEST depends on BPF_SYSCALL - depends on PAHOLE_VERSION >= 116 - depends on DEBUG_INFO_DWARF4 || PAHOLE_VERSION >= 121 + depends on PAHOLE_VERSION >= 122 # pahole uses elfutils, which does not have support for Hexagon relocations depends on !HEXAGON help Generate deduplicated BTF type information from DWARF debug info. - Turning this on requires pahole v1.16 or later (v1.21 or later to - support DWARF 5), which will convert DWARF type info into equivalent - deduplicated BTF type info. - -config PAHOLE_HAS_SPLIT_BTF - def_bool PAHOLE_VERSION >= 119 + Turning this on requires pahole v1.22 or later, which will convert + DWARF type info into equivalent deduplicated BTF type info. config PAHOLE_HAS_BTF_TAG def_bool PAHOLE_VERSION >= 123
@@ -421,7 +416,7 @@ config PAHOLE_HAS_LANG_EXCLUDE config DEBUG_INFO_BTF_MODULES bool "Generate BTF type information for kernel modules" default y - depends on DEBUG_INFO_BTF && MODULES && PAHOLE_HAS_SPLIT_BTF + depends on DEBUG_INFO_BTF && MODULES help Generate compact split BTF type information for kernel modules.
scripts/Makefile.btf +1 -8
@@ -7,14 +7,7 @@ JOBS := $(patsubst -j%,%,$(filter -j%,$(MAKEFLAGS))) ifeq ($(call test-le, $(pahole-ver), 125),y) -# pahole 1.18 through 1.21 can't handle zero-sized per-CPU vars -ifeq ($(call test-le, $(pahole-ver), 121),y) -pahole-flags-$(call test-ge, $(pahole-ver), 118) += --skip_encoding_btf_vars -endif - -pahole-flags-$(call test-ge, $(pahole-ver), 121) += --btf_gen_floats - -pahole-flags-$(call test-ge, $(pahole-ver), 122) += -j$(JOBS) +pahole-flags-y += --btf_gen_floats -j$(JOBS) pahole-flags-$(call test-ge, $(pahole-ver), 125) += --skip_encoding_btf_inconsistent_proto --btf_gen_optimized
tools/sched_ext/README.md +0 -1
@@ -65,7 +65,6 @@ It's also recommended that you also include the following Kconfig options: ``` CONFIG_BPF_JIT_ALWAYS_ON=y CONFIG_BPF_JIT_DEFAULT_ON=y -CONFIG_PAHOLE_HAS_SPLIT_BTF=y CONFIG_PAHOLE_HAS_BTF_TAG=y ```
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v4 7/8 selftests/bpf: Run resolve_btfids only for relevant .test.o objects
In reply to Ihor SolodraiA selftest targeting resolve_btfids functionality relies on a resolved .BTF_ids section to be available in the TRUNNER_BINARY. The underlying BTF data is taken from a special BPF program (btf_data.c), and so resolve_btfids is executed as a part of a TRUNNER_BINARY build recipe on the final binary.
Subsequent patches in this series allow resolve_btfids to modify BTF before resolving the symbols, which means that the test needs access to that modified BTF [1]. Currently the test simply reads in btf_data.bpf.o on the assumption that BTF hasn't changed.
Implement resolve_btfids call only for particular test objects (just resolve_btfids.test.o for now). The test objects are linked into the TRUNNER_BINARY, and so .BTF_ids section will be available there.
This will make it trivial for the resolve_btfids test to access BTF modified by resolve_btfids.
[1] https://lore.kernel.org/bpf/CAErzpmvsgSDe-QcWH8SFFErL6y3p3zrqNri5-UHJ9iK2ChyiBw@mail.gmail.com/
--- tools/testing/selftests/bpf/Makefile | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/Makefile +6 -2
@@ -643,6 +643,9 @@ $(TRUNNER_TESTS_HDR): $(TRUNNER_TESTS_DIR)/*.c ) > $$@) endif +$(TRUNNER_OUTPUT)/resolve_btfids.test.o: $(RESOLVE_BTFIDS) $(TRUNNER_OUTPUT)/btf_data.bpf.o +$(TRUNNER_OUTPUT)/resolve_btfids.test.o: private TEST_NEEDS_BTFIDS = 1 + # compile individual test files # Note: we cd into output directory to ensure embedded BPF object is found $(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o: \
@@ -650,6 +653,9 @@ $(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o: \ | $(TRUNNER_OUTPUT)/%.test.d $$(call msg,TEST-OBJ,$(TRUNNER_BINARY),$$@) $(Q)cd $$(@D) && $$(CC) -I. $$(CFLAGS) -MMD -MT $$@ -c $(CURDIR)/$$< $$(LDLIBS) -o $$(@F) + $$(if $$(TEST_NEEDS_BTFIDS), \ + $$(call msg,BTFIDS,$(TRUNNER_BINARY),$$@) \ + $(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@) $(TRUNNER_TEST_OBJS:.o=.d): $(TRUNNER_OUTPUT)/%.test.d: \ $(TRUNNER_TESTS_DIR)/%.c \
@@ -695,13 +701,11 @@ $(OUTPUT)/$(TRUNNER_BINARY): | $(TRUNNER_BPF_OBJS) $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(TRUNNER_EXTRA_OBJS) $$(BPFOBJ) \ $(TRUNNER_LIB_OBJS) \ - $(RESOLVE_BTFIDS) \ $(TRUNNER_BPFTOOL) \ $(OUTPUT)/veristat \ | $(TRUNNER_BINARY)-extras $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ - $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool
Signature · 1 line
2.52.0
Re: [PATCH bpf-next v4 7/8] selftests/bpf: Run resolve_btfids only for relevant .test.o objects
Quoted 15 lines · Ihor Solodrai
A selftest targeting
resolve_btfidsfunctionality relies on a resolved .BTF_ids section to be available in theTRUNNER_BINARY. The underlying BTF data is taken from a special BPF program (btf_data.c), and soresolve_btfidsis executed as a part of aTRUNNER_BINARYbuild recipe on the final binary.Subsequent patches in this series allow
resolve_btfidsto modify BTF before resolving the symbols, which means that the test needs access to that modified BTF [1]. Currently the test simply reads in btf_data.bpf.o on the assumption that BTF hasn't changed.Implement
resolve_btfidscall only for particular test objects (just resolve_btfids.test.o for now). The test objects are linked into theTRUNNER_BINARY, and so .BTF_ids section will be available there.
Quoted 7 lines
This will make it trivial for the
resolve_btfidstest to access BTF modified byresolve_btfids.[1] https://lore.kernel.org/bpf/CAErzpmvsgSDe-QcWH8SFFErL6y3p3zrqNri5-UHJ9iK2ChyiBw@mail.gmail.com/
Signed-off-by Ihor Solodrai <[email protected]>---
[...]
Ihor Solodrai
reviewerPATCH v4 8/8 resolve_btfids: Change in-place update with raw binary output
In reply to Ihor SolodraiCurrently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.
Change resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this, in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.BTF with .BTF section data
* ${1}.BTF_ids with .BTF_ids section data if it existed in ${1}
* ${1}.BTF.base with .BTF.base section data for out-of-tree modules
The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.
The motivation for emitting binary data is that it allows simplifying resolve_btfids implementation by delegating ELF update to the $OBJCOPY tool [1], which is already widely used across the codebase.
There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking. The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked as follows:
* pahole consumed .tmp_vmlinux1 [2] and added .BTF section with llvm-objcopy [3] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [2], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [4], updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For kernel modules, creating a special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with objcopy.
With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows deleting elf_update() call and satellite code (like compressed_section_fix [5]).
Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [6], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n110
[3] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c?h=v1.31#n1803
[4] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/scripts/link-vmlinux.sh?h=v6.18#n284
[5] https://lore.kernel.org/bpf/[email protected]/
[6] https://lore.kernel.org/bpf/[email protected]/
--- MAINTAINERS | 1 + scripts/Makefile.btf | 17 +- scripts/Makefile.modfinal | 5 +- scripts/Makefile.vmlinux | 2 +- scripts/gen-btf.sh | 157 +++++++++++++ scripts/link-vmlinux.sh | 42 +--- tools/bpf/resolve_btfids/main.c | 222 +++++++++++------- tools/testing/selftests/bpf/.gitignore | 3 + tools/testing/selftests/bpf/Makefile | 9 +- .../selftests/bpf/prog_tests/resolve_btfids.c | 4 +- 10 files changed, 331 insertions(+), 131 deletions(-) create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
@@ -4766,6 +4766,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/
scripts/Makefile.btf +12 -5
@@ -1,5 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 +gen-btf-y = +gen-btf-$(CONFIG_DEBUG_INFO_BTF) = $(srctree)/scripts/gen-btf.sh + +export GEN_BTF := $(gen-btf-y) + pahole-ver := $(CONFIG_PAHOLE_VERSION) pahole-flags-y :=
@@ -18,13 +23,15 @@ pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j$(JOBS) --btf_features=enc pahole-flags-$(call test-ge, $(pahole-ver), 130) += --btf_features=attributes -ifneq ($(KBUILD_EXTMOD),) -module-pahole-flags-$(call test-ge, $(pahole-ver), 128) += --btf_features=distilled_base -endif - endif pahole-flags-$(CONFIG_PAHOLE_HAS_LANG_EXCLUDE) += --lang_exclude=rust export PAHOLE_FLAGS := $(pahole-flags-y) -export MODULE_PAHOLE_FLAGS := $(module-pahole-flags-y) + +resolve-btfids-flags-y := +resolve-btfids-flags-$(CONFIG_WERROR) += --fatal_warnings +resolve-btfids-flags-$(if $(KBUILD_EXTMOD),y) += --distill_base +resolve-btfids-flags-$(if $(KBUILD_VERBOSE),y) += --verbose + +export RESOLVE_BTFIDS_FLAGS := $(resolve-btfids-flags-y)
scripts/Makefile.modfinal +2 -3
@@ -42,9 +42,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/Makefile.vmlinux +1 -1
@@ -71,7 +71,7 @@ targets += vmlinux.unstripped .vmlinux.export.o vmlinux.unstripped: scripts/link-vmlinux.sh vmlinux.o .vmlinux.export.o $(KBUILD_LDS) FORCE +$(call if_changed_dep,link_vmlinux) ifdef CONFIG_DEBUG_INFO_BTF -vmlinux.unstripped: $(RESOLVE_BTFIDS) +vmlinux.unstripped: $(RESOLVE_BTFIDS) $(srctree)/scripts/gen-btf.sh endif ifdef CONFIG_BUILDTIME_TABLE_SORT
scripts/gen-btf.sh new file +157 -0
157 lines not on this page · raw message
scripts/link-vmlinux.sh +6 -36
71 lines not on this page · raw message
tools/bpf/resolve_btfids/main.c +141 -81
325 lines not on this page · raw message
tools/testing/selftests/bpf/.gitignore +3 -0
@@ -45,3 +45,6 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.BTF +*.BTF_ids +*.BTF.base
tools/testing/selftests/bpf/Makefile +6 -3
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)
@@ -653,9 +654,10 @@ $(TRUNNER_TEST_OBJS): $(TRUNNER_OUTPUT)/%.test.o: \ | $(TRUNNER_OUTPUT)/%.test.d $$(call msg,TEST-OBJ,$(TRUNNER_BINARY),$$@) $(Q)cd $$(@D) && $$(CC) -I. $$(CFLAGS) -MMD -MT $$@ -c $(CURDIR)/$$< $$(LDLIBS) -o $$(@F) - $$(if $$(TEST_NEEDS_BTFIDS), \ - $$(call msg,BTFIDS,$(TRUNNER_BINARY),$$@) \ - $(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@) + $$(if $$(TEST_NEEDS_BTFIDS), \ + $$(call msg,BTFIDS,$(TRUNNER_BINARY),$$@) \ + $(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@; \ + $(OBJCOPY) --update-section [email protected]_ids $$@) $(TRUNNER_TEST_OBJS:.o=.d): $(TRUNNER_OUTPUT)/%.test.d: \ $(TRUNNER_TESTS_DIR)/%.c \
@@ -894,6 +896,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ + *.BTF *.BTF_ids *.BTF.base \ no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selftests \
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +2 -2
@@ -101,9 +101,9 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("resolve_btfids.test.o.BTF" );if (CHECK (libbpf_get_error (btf),"resolve" , -"Failed to load BTF from btf_data.bpf.o\n" )) +"Failed to load BTF from resolve_btfids.test.o.BTF\n" ))return -1 ; nr =btf__type_cnt (btf);
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerRe: [PATCH bpf-next v4 8/8] resolve_btfids: Change in-place update with raw binary output
scripts/Makefile.btf line 7
Quoted 13 lines · Ihor Solodrai
[...]
scripts/Makefile.btf +5
-0@@ -1,5 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 +gen-btf-y = +gen-btf-$(CONFIG_DEBUG_INFO_BTF) = $(srctree)/scripts/gen-btf.sh + +export GEN_BTF := $(gen-btf-y) +
Ugh... GEN_BTF is not used anywhere in v4.
Looks like I forgot a git add at some point.
I'll wait a bit before sending a v5, in case there is more feedback.
Quoted 14 lines
scripts/Makefile.btf +0
-4pahole-ver := $(CONFIG_PAHOLE_VERSION) pahole-flags-y :=@@ -18,13 +23,15 @@ pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j$(JOBS) --btf_features=enc pahole-flags-$(call test-ge, $(pahole-ver), 130) += --btf_features=attributes -ifneq ($(KBUILD_EXTMOD),) -module-pahole-flags-$(call test-ge, $(pahole-ver), 128) += --btf_features=distilled_base -endif - endif
Re: [PATCH bpf-next v4 8/8] resolve_btfids: Change in-place update with raw binary output
Quoted 19 lines · Ihor Solodrai
scripts/Makefile.btf line 7
Quoted 13 lines · Ihor Solodrai
[...]
scripts/Makefile.btf +5
-0@@ -1,5 +1,10 @@ # SPDX-License-Identifier: GPL-2.0 +gen-btf-y = +gen-btf-$(CONFIG_DEBUG_INFO_BTF) = $(srctree)/scripts/gen-btf.sh + +export GEN_BTF := $(gen-btf-y) +Ugh...
GEN_BTFis not used anywhere in v4.
Looks like I forgot agit addat some point.
Quoted 1 line
I'll wait a bit before sending a v5, in case there is more feedback.
It all looks good to me, so don't wait for any more feedback from my side. If Eduard doesn't find anything in patch #8, please send new revision, thanks!
Quoted 16 lines
Quoted 15 lines
scripts/Makefile.btf +0
-4pahole-ver := $(CONFIG_PAHOLE_VERSION) pahole-flags-y :=@@ -18,13 +23,15 @@ pahole-flags-$(call test-ge, $(pahole-ver), 126) = -j$(JOBS) --btf_features=enc pahole-flags-$(call test-ge, $(pahole-ver), 130) += --btf_features=attributes -ifneq ($(KBUILD_EXTMOD),) -module-pahole-flags-$(call test-ge, $(pahole-ver), 128) += --btf_features=distilled_base -endif - endif
Eduard Zingerman
maintainerRe: [PATCH bpf-next v4 8/8] resolve_btfids: Change in-place update with raw binary output
On Thu, 2025-12-18 at 13:15 -0800, Andrii Nakryiko wrote:
[...]
Quoted 3 lines
It all looks good to me, so don't wait for any more feedback from my side. If Eduard doesn't find anything in patch #8, please send new revision, thanks!
Lgtm, let's wrap this up.
[...]