resolve_btfids: Support for BTF modifications
Subsystems (4)
Ihor Solodrai
reviewerv2 cover resolve_btfids: Support for BTF modifications
This series changes resolve_btfids and kernel build scripts to enable BTF transformations in resolve_btfids. Main motivation for enhancing resolve_btfids is to reduce dependency of the kernel build on pahole capabilities [1] and enable BTF features and optimizations [2][3] particular to the kernel.
Patches #1-#3 in the series are non-functional refactoring in resolve_btfids. The last patch (#4) makes significant changes in resolve_btfids and introduces scripts/gen-btf.sh. Implementation changes are described in detail in the patch description.
[1] https://lore.kernel.org/dwarves/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
[3] https://lore.kernel.org/bpf/[email protected]/
---
v1->v2:
- gen-btf.sh and other shell script fixes (Donglin)
- update selftests build (Donglin)
- generate .BTF.base only when KBUILD_EXTMOD is set (Alan)
- proper endianness handling for cross-compilation
- change elf_begin mode from ELF_C_RDWR_MMAP to ELF_C_READ_MMAP_PRIVATE
- remove compressed_section_fix()
- nit NULL check in patch #3 (suggested by AI)
v1: https://lore.kernel.org/bpf/[email protected]/
Ihor Solodrai (4):
resolve_btfids: rename object btf field to btf_path
resolve_btfids: factor out load_btf()
resolve_btfids: introduce enum btf_id_kind
resolve_btfids: change in-place update with raw binary output
MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++ scripts/link-vmlinux.sh | 42 +--- tools/bpf/resolve_btfids/main.c | 331 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 395 insertions(+), 156 deletions(-) create mode 100755 scripts/gen-btf.sh
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v2 1/4 resolve_btfids: rename object btf field to btf_path
Rename the member of struct object holding the path to BTF data if provided via --btf arg. btf_path is less ambiguous.
--- tools/bpf/resolve_btfids/main.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/bpf/resolve_btfids/main.c +5 -5
@@ -113,7 +113,7 @@ struct btf_id {struct object {const char *path; -const char *btf; +const char *btf_path;const char *base_btf_path;struct {
@@ -550,11 +550,11 @@ static int symbols_resolve(struct object *obj) } } - btf =btf__parse_split (obj->btf ?: obj->path, base_btf); + btf =btf__parse_split (obj->btf_path ?: obj->path, base_btf); err =libbpf_get_error (btf);if (err) {pr_err ("FAILED: load BTF from %s: %s\n" , - obj->btf ?: obj->path,strerror (-err)); + obj->btf_path ?: obj->path,strerror (-err)); goto out; }
@@ -790,8 +790,8 @@ int main(int argc, const char **argv)struct option btfid_options[] = {OPT_INCR ('v' ,"verbose" , &verbose,"be more verbose (show errors, etc)" ), -OPT_STRING (0 ,"btf" , &obj.btf,"BTF data" , -"BTF data" ), +OPT_STRING (0 ,"btf" , &obj.btf_path,"file" , +"path to a file with input BTF data" ),OPT_STRING ('b' ,"btf_base" , &obj.base_btf_path,"file" ,"path of file providing base BTF" ), OPT_BOOLEAN(0, "fatal_warnings", &fatal_warnings,
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v2 2/4 resolve_btfids: factor out load_btf()
In reply to Ihor SolodraiIncrease the lifetime of parsed BTF in resolve_btfids by factoring load_btf() routine out of symbols_resolve() and storing the base_btf and btf pointers in the struct object.
--- tools/bpf/resolve_btfids/main.c | 47 ++++++++++++++++++++++++--------- 1 file changed, 34 insertions(+), 13 deletions(-)
tools/bpf/resolve_btfids/main.c +34 -13
@@ -116,6 +116,9 @@ struct object {const char *btf_path;const char *base_btf_path; +struct btf *btf; +struct btf *base_btf; +struct {int fd;Elf *elf;
@@ -529,16 +532,10 @@ static int symbols_collect(struct object *obj)return 0 ; } -static int symbols_resolve (struct object *obj) +static int load_btf (struct object *obj) { -int nr_typedefs = obj->nr_typedefs; -int nr_structs = obj->nr_structs; -int nr_unions = obj->nr_unions; -int nr_funcs = obj->nr_funcs; -struct btf *base_btf =NULL ; -int err, type_id; -struct btf *btf; -__u32 nr_types; +struct btf *base_btf =NULL , *btf =NULL ; +int err;if (obj->base_btf_path) { base_btf =btf__parse (obj->base_btf_path,NULL );
@@ -546,7 +543,7 @@ static int symbols_resolve(struct object *obj)if (err) {pr_err ("FAILED: load base BTF from %s: %s\n" , obj->base_btf_path,strerror (-err)); -return -1 ; + goto out_err; } }
@@ -555,9 +552,30 @@ static int symbols_resolve(struct object *obj)if (err) {pr_err ("FAILED: load BTF from %s: %s\n" , obj->btf_path ?: obj->path,strerror (-err)); - goto out; + goto out_err; } + obj->base_btf = base_btf; + obj->btf = btf; + +return 0 ; + +out_err: +btf__free (base_btf); +btf__free (btf); +return err; +} + +static int symbols_resolve (struct object *obj) +{ +int nr_typedefs = obj->nr_typedefs; +int nr_structs = obj->nr_structs; +int nr_unions = obj->nr_unions; +int nr_funcs = obj->nr_funcs; +struct btf *btf = obj->btf; +int err, type_id; +__u32 nr_types; + err =-1 ; nr_types =btf__type_cnt (btf);
@@ -615,8 +633,6 @@ static int symbols_resolve(struct object *obj) err =0 ; out: -btf__free (base_btf); -btf__free (btf);return err; }
@@ -824,6 +840,9 @@ int main(int argc, const char **argv)if (symbols_collect (&obj)) goto out; +if (load_btf (&obj)) + goto out; +if (symbols_resolve (&obj)) goto out;
@@ -833,6 +852,8 @@ int main(int argc, const char **argv)if (!(fatal_warnings && warnings)) err =0 ; out: +btf__free (obj.base_btf); +btf__free (obj.btf);if (obj.efile.elf) {elf_end (obj.efile.elf);close (obj.efile.fd);
Signature · 1 line
2.52.0
Ihor Solodrai
reviewerPATCH v2 3/4 resolve_btfids: introduce enum btf_id_kind
In reply to Ihor SolodraiInstead of using multiple flags, make struct btf_id tagged with an enum value indicating its kind in the context of resolve_btfids.
--- tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 20 deletions(-)
tools/bpf/resolve_btfids/main.c +42 -20
@@ -98,6 +98,13 @@# error "Unknown machine endianness!"#endif +enum btf_id_kind { +BTF_ID_KIND_NONE , +BTF_ID_KIND_SYM , +BTF_ID_KIND_SET , +BTF_ID_KIND_SET8 +}; +struct btf_id {struct rb_node rb_node;char *name;
@@ -105,9 +112,8 @@ struct btf_id {int id;int cnt; }; -int addr_cnt; -bool is_set; -bool is_set8; + enum btf_id_kind kind:8; + int addr_cnt:8;Elf64_Addr addr[ADDR_CNT ]; };
@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)return id; } -static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);
@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; } -return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); +if (btf_id) + btf_id->kind =BTF_ID_KIND_SYM ; + +return btf_id; }/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */
@@ -491,28 +508,24 @@ static int symbols_collect(struct object *obj) id =add_symbol (&obj->funcs, prefix,sizeof (BTF_FUNC ) -1 );/* set8 */ }else if (!strncmp (prefix,BTF_SET8 ,sizeof (BTF_SET8 ) -1 )) { - id =add_set (obj, prefix, true); + id =add_set (obj, prefix,BTF_ID_KIND_SET8 );/* * SET8 objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (uint64_t ) -1 ; - id->is_set8 = true; - }/* set */ }else if (!strncmp (prefix,BTF_SET ,sizeof (BTF_SET ) -1 )) { - id =add_set (obj, prefix, false); + id =add_set (obj, prefix,BTF_ID_KIND_SET );/* * SET objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (int ) -1 ; - id->is_set = true; - } }else {pr_err ("FAILED unsupported prefix %s\n" , prefix);return -1 ;
@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)int i;/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind ==BTF_ID_KIND_SYM ) {pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }
@@ -696,6 +709,7 @@ static int sets_patch(struct object *obj) {Elf_Data *data = obj->efile.idlist;struct rb_node *next; +int cnt; next =rb_first (&obj->sets);while (next) {
@@ -715,11 +729,15 @@ static int sets_patch(struct object *obj)return -1 ; } -if (id->is_set) { +switch (id->kind) { +case BTF_ID_KIND_SET : set = data->d_buf + off; + cnt = set->cnt;qsort (set->ids, set->cnt,sizeof (set->ids[0 ]), cmp_id); - }else { +break ; +case BTF_ID_KIND_SET8 : set8 = data->d_buf + off; + cnt = set8->cnt;/* * Make sure id is at the beginning of the pairs * struct, otherwise the below qsort would not work.
@@ -744,10 +762,14 @@ static int sets_patch(struct object *obj)bswap_32 (set8->pairs[i].flags); } } +break ; +case BTF_ID_KIND_SYM : +default : +pr_err ("Unexpected btf_id_kind %d for set '%s'\n" , id->kind, id->name); +return -1 ; } -pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , - off, id->is_set ? set->cnt : set8->cnt, id->name); +pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , off, cnt, id->name); next =rb_next (next); }
Signature · 1 line
2.52.0
Re: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind
Quoted 8 lines · Ihor Solodrai
Instead of using multiple flags, make struct
btf_idtagged with an enum value indicating its kind in the context ofresolve_btfids.Signed-off-by Ihor Solodrai <[email protected]>--- tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 20 deletions(-)
[...]
Quoted 35 lines
tools/bpf/resolve_btfids/main.c +10
-3-static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);
tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306
Quoted 7 lines
tools/bpf/resolve_btfids/main.c +3
-1@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; } -return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); +if (btf_id) + btf_id->kind =BTF_ID_KIND_SYM ;
seeing this pattern repeated, wouldn't it make sense to just pass this kind to btf_id__add() and set it there?
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +2
-0+ +return btf_id; }
[...]
tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659
Quoted 6 lines
tools/bpf/resolve_btfids/main.c +1
-1@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)int i;/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind ==BTF_ID_KIND_SYM ) {
nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?
Quoted 3 lines
tools/bpf/resolve_btfids/main.c +0
-0pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }
[...]
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind
Quoted 50 lines · Andrii Nakryiko
Quoted 8 lines · Ihor Solodrai
Instead of using multiple flags, make struct
btf_idtagged with an enum value indicating its kind in the context ofresolve_btfids.Signed-off-by Ihor Solodrai <[email protected]>--- tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 20 deletions(-)[...]
Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10
-3-static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; }
Quoted 7 lines
tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +3
-1-return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); +if (btf_id) + btf_id->kind =BTF_ID_KIND_SYM ;seeing this pattern repeated, wouldn't it make sense to just pass this kind to
btf_id__add()and set it there?
I like the idea, because we could get rid the "unique" flag then.
But the btf_id__add() does not necessarily create a new struct, and so if we pass the kind in, what do we do with existing objects? Overwrite the kind? If not, do we check for a mismatch?
Quoted 11 lines
Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2
-0+ +return btf_id; }[...]
Quoted 3 lines
(quoted diff) +0
-0@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id) int i;
Quoted 8 lines
tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659
Quoted 3 lines
tools/bpf/resolve_btfids/main.c +1
-1/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind ==BTF_ID_KIND_SYM ) {nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?
ok
Quoted 6 lines
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }[...]
Re: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind
Quoted 60 lines · Ihor Solodrai
Quoted 50 lines · Andrii Nakryiko
Quoted 8 lines · Ihor Solodrai
Instead of using multiple flags, make struct
btf_idtagged with an enum value indicating its kind in the context ofresolve_btfids.Signed-off-by Ihor Solodrai <[email protected]>--- tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 20 deletions(-)[...]
Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10
-3-static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; }Quoted 7 lines
tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +3
-1-return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); +if (btf_id) + btf_id->kind =BTF_ID_KIND_SYM ;seeing this pattern repeated, wouldn't it make sense to just pass this kind to
btf_id__add()and set it there?
Quoted 6 lines
I like the idea, because we could get rid the "unique" flag then.
But the
btf_id__add()does not necessarily create a new struct, and so if we pass the kind in, what do we do with existing objects? Overwrite the kind? If not, do we check for a mismatch?
no idea, don't know code well enough, but your newly added code seems to overwrite the kind always, no?
Quoted 29 lines
Quoted 11 lines
Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2
-0+ +return btf_id; }[...]
Quoted 3 lines
(quoted diff) +0
-0@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id) int i;Quoted 8 lines
tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659
Quoted 3 lines
tools/bpf/resolve_btfids/main.c +1
-1/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind ==BTF_ID_KIND_SYM ) {nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?
ok
Quoted 6 lines
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }[...]
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind
Quoted 63 lines · Andrii Nakryiko
Quoted 62 lines · Ihor Solodrai
Quoted 50 lines · Andrii Nakryiko
Quoted 8 lines · Ihor Solodrai
Instead of using multiple flags, make struct
btf_idtagged with an enum value indicating its kind in the context ofresolve_btfids.Signed-off-by Ihor Solodrai <[email protected]>--- tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 20 deletions(-)[...]
Quoted 38 lines
tools/bpf/resolve_btfids/main.c +10
-3-static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" ); +char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; }Quoted 7 lines
tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 306
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +3
-1-return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); +if (btf_id) + btf_id->kind =BTF_ID_KIND_SYM ;seeing this pattern repeated, wouldn't it make sense to just pass this kind to
btf_id__add()and set it there?I like the idea, because we could get rid the "unique" flag then.
Quoted 7 lines
Quoted 4 lines
But the
btf_id__add()does not necessarily create a new struct, and so if we pass the kind in, what do we do with existing objects? Overwrite the kind? If not, do we check for a mismatch?no idea, don't know code well enough, but your newly added code seems to overwrite the kind always, no?
You're right, I am overwriting here, haven't realized that.
I think I'll go with a mismatch check inside btf_id__add() in v3, that would indicate a bug.
Quoted 30 lines
Quoted 30 lines
Quoted 12 lines
Quoted 6 lines
tools/bpf/resolve_btfids/main.c +2
-0+ +return btf_id; }[...]
Quoted 3 lines
(quoted diff) +0
-0@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id) int i;Quoted 8 lines
tools/bpf/resolve_btfids/main.c static int id_patch(struct object *obj, struct btf_id *id) · line 659
Quoted 3 lines
tools/bpf/resolve_btfids/main.c +1
-1/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind ==BTF_ID_KIND_SYM ) {nit: comment says the exception is specifically for SET and SET8, so I think checking for those two instead of for SYM (implying that only other possible options are set and set8) would be a bit more future-proof?
ok
Quoted 6 lines
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +0
-0pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }[...]
Eduard Zingerman
maintainerRe: [PATCH bpf-next v2 3/4] resolve_btfids: introduce enum btf_id_kind
In reply to Ihor SolodraiQuoted 5 lines · Ihor Solodrai
Instead of using multiple flags, make struct
btf_idtagged with an enum value indicating its kind in the context ofresolve_btfids.Signed-off-by Ihor Solodrai <[email protected]>---
Just a few nits, looks good to me overall.
Quoted 22 lines
tools/bpf/resolve_btfids/main.c | 62 ++++++++++++++++++++++----------- 1 file changed, 42 insertions(+), 20 deletions(-)tools/bpf/resolve_btfids/main.c +7
-0@@ -98,6 +98,13 @@# error "Unknown machine endianness!"#endif +enum btf_id_kind { +BTF_ID_KIND_NONE , +BTF_ID_KIND_SYM , +BTF_ID_KIND_SET , +BTF_ID_KIND_SET8 +}; +struct btf_id {struct rb_node rb_node;char *name;
tools/bpf/resolve_btfids/main.c struct btf_id { · line 116
Quoted 8 lines
tools/bpf/resolve_btfids/main.c +2
-3@@ -105,9 +112,8 @@ struct btf_id {int id;int cnt; }; -int addr_cnt; -bool is_set; -bool is_set8; + enum btf_id_kind kind:8; + int addr_cnt:8;
^^
Nit: these bitfields are not really necessary:
$ pahole -C btf_id ./tools/bpf/resolve_btfids/resolve_btfids
struct btf_id {
struct rb_node rb_node __attribute__((__aligned__(8))); /* 0 24 */
char * name; /* 24 8 */
union {
int id; /* 32 4 */
int cnt; /* 32 4 */
}; /* 32 4 */
enum btf_id_kind kind:8; /* 36: 0 4 */
int addr_cnt:8; /* 36: 8 4 */ /* XXX 16 bits hole, try to pack */
Elf64_Addr addr[100]; /* 40 800 */
/* size: 840, cachelines: 14, members: 6 */
/* sum members: 836 */
/* sum bitfield members: 16 bits, bit holes: 1, sum bit holes: 16 bits */
/* forced alignments: 1 */
/* last cacheline: 8 bytes */} __attribute__((__aligned__(8)));
Quoted 7 lines
tools/bpf/resolve_btfids/main.c +0
-0Elf64_Addr addr[ADDR_CNT ]; };@@ -260,26 +266,33 @@ static char *get_id(const char *prefix_end)return id; }
tools/bpf/resolve_btfids/main.c static char *get_id(const char *prefix_end) · line 276
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +2
-2-static struct btf_id *add_set (struct object *obj,char *name,bool is_set8) +static struct btf_id *add_set (struct object *obj,char *name,enum btf_id_kind kind) {/* * __BTF_ID__set__name * name = ^ * id = ^ */ -char *id = name + (is_set8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" )) -1 ; +int prefixlen = kind ==BTF_ID_KIND_SET8 ?sizeof (BTF_SET8 "__" ) :sizeof (BTF_SET "__" );
Nit: Should prefixlen be an input parameter as well? (Or adjust the 'name' at the callsite?)
Otherwise the parameter is still a boolean logically.
Quoted 24 lines
tools/bpf/resolve_btfids/main.c +8
-1+char *id = name + prefixlen -1 ;int len =strlen (name); +struct btf_id *btf_id;if (id >= name + len) {pr_err ("FAILED to parse set name: %s\n" , name);return NULL ; } -return btf_id__add (&obj->sets, id, true); + btf_id =btf_id__add (&obj->sets, id, true); +if (btf_id) + btf_id->kind = kind; + +return btf_id; }static struct btf_id *add_symbol (struct rb_root *root,char *name,size_t size) { +struct btf_id *btf_id;char *id; id =get_id (name + size);
tools/bpf/resolve_btfids/main.c static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size) · line 308
Quoted 9 lines
tools/bpf/resolve_btfids/main.c +5
-1@@ -288,7 +301,11 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return NULL ; } -return btf_id__add (root, id, false); + btf_id =btf_id__add (root, id, false); +if (btf_id) + btf_id->kind =BTF_ID_KIND_SYM ; + +return btf_id;
Agree with Andrii regarding 'kind' being a btf_id__add() parameter.
Quoted 72 lines
tools/bpf/resolve_btfids/main.c +12
-11}/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */ @@ -491,28 +508,24 @@ static int symbols_collect(struct object *obj) id =add_symbol (&obj->funcs, prefix,sizeof (BTF_FUNC ) -1 );/* set8 */ }else if (!strncmp (prefix,BTF_SET8 ,sizeof (BTF_SET8 ) -1 )) { - id =add_set (obj, prefix, true); + id =add_set (obj, prefix,BTF_ID_KIND_SET8 );/* * SET8 objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (uint64_t ) -1 ; - id->is_set8 = true; - }/* set */ }else if (!strncmp (prefix,BTF_SET ,sizeof (BTF_SET ) -1 )) { - id =add_set (obj, prefix, false); + id =add_set (obj, prefix,BTF_ID_KIND_SET );/* * SET objects store list's count, which is encoded * in symbol's size, together with 'cnt' field hence * that - 1. */ -if (id) { +if (id) id->cnt = sym.st_size /sizeof (int ) -1 ; - id->is_set = true; - } }else {pr_err ("FAILED unsupported prefix %s\n" , prefix);return -1 ;@@ -643,7 +656,7 @@ static int id_patch(struct object *obj, struct btf_id *id)int i;/* For set, set8, id->id may be 0 */ -if (!id->id && !id->is_set && !id->is_set8) { +if (!id->id && id->kind ==BTF_ID_KIND_SYM ) {pr_err ("WARN: resolve_btfids: unresolved symbol %s\n" , id->name); warnings++; }@@ -696,6 +709,7 @@ static int sets_patch(struct object *obj) {Elf_Data *data = obj->efile.idlist;struct rb_node *next; +int cnt; next =rb_first (&obj->sets);while (next) {@@ -715,11 +729,15 @@ static int sets_patch(struct object *obj)return -1 ; } -if (id->is_set) { +switch (id->kind) { +case BTF_ID_KIND_SET : set = data->d_buf + off; + cnt = set->cnt;qsort (set->ids, set->cnt,sizeof (set->ids[0 ]), cmp_id); - }else { +break ; +case BTF_ID_KIND_SET8 : set8 = data->d_buf + off; + cnt = set8->cnt;/* * Make sure id is at the beginning of the pairs * struct, otherwise the below qsort would not work.
tools/bpf/resolve_btfids/main.c static int sets_patch(struct object *obj) · line 767
Quoted 6 lines
tools/bpf/resolve_btfids/main.c +3
-0@@ -744,10 +762,14 @@ static int sets_patch(struct object *obj)bswap_32 (set8->pairs[i].flags); } } +break ; +case BTF_ID_KIND_SYM : +default :
Nit: just default, no need for case BTF_ID_KIND_SYM:?
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +3
-2+pr_err ("Unexpected btf_id_kind %d for set '%s'\n" , id->kind, id->name); +return -1 ; } -pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , - off, id->is_set ? set->cnt : set8->cnt, id->name); +pr_debug ("sorting addr %5lu: cnt %6d [%s]\n" , off, cnt, id->name); next =rb_next (next); }
Ihor Solodrai
reviewerPATCH v2 4/4 resolve_btfids: change in-place update with raw binary output
In reply to Ihor SolodraiCurrently resolve_btfids updates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.
This patch changes resolve_btfids behavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write in resolve_btfids is replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
The execution of resolve_btfids and consumption of its output is orchestrated by scripts/gen-btf.sh introduced in this patch.
The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
There are two distinct paths for BTF generation and resolve_btfids application in the kernel build: for vmlinux and for kernel modules.
For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
* resolve_btfids was executed later on vmlinux.unstripped [3], updating it in-place
After this patch gen-btf.sh implements the following:
* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
* resolve_btfids consumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output of resolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped in
link-vmlinux.sh by objcopy --update-section
For the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by resolve_btfids is straightforward with the objcopy.
With this patch an ELF file becomes effectively read-only within resolve_btfids, which allows to delete elf_update() call and satelite code (like compressed_section_fix [4]).
Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in sets_patch() [5], and then Elf_Type was modified before elf_update() to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/
--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 317 insertions(+), 121 deletions(-) create mode 100755 scripts/gen-btf.sh
MAINTAINERS +1 -0
@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/
scripts/Makefile.modfinal +2 -3
@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependencies
scripts/gen-btf.sh new file +167 -0
167 lines not on this page · raw message
scripts/link-vmlinux.sh +6 -36
71 lines not on this page · raw message
tools/bpf/resolve_btfids/main.c +136 -82
327 lines not on this page · raw message
tools/testing/selftests/bpf/Makefile +5 -0
@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)
@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \ + fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool
Signature · 1 line
2.52.0
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 718 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped inlink-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows to deleteelf_update()call and satelite code (likecompressed_section_fix[4]).Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in
sets_patch()[5], and thenElf_Typewas modified beforeelf_update()to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 317 insertions(+), 121 deletions(-) create mode 100755 scripts/gen-btf.shMAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependenciesscripts/gen-btf.sh new file +167
-0@@ -0,0 +1,167 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates. +# +# This script generates BTF data for the provided ELF file. +# +# Kernel BTF generation involves these conceptual steps: +# 1. pahole generates BTF from DWARF data +# 2. resolve_btfids applies kernel-specific btf2btf +# transformations and computes data for .BTF_ids section +# 3. the result gets linked/objcopied into the target binary +# +# How step (3) should be done differs between vmlinux, and +# kernel modules, which is the primary reason for the existence +# of this script. +# +# For modules the script expects vmlinux passed in as --btf_base. +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded +# into the input ELF file with objcopy. +# +# For vmlinux the input file remains unchanged and two files are produced: +# - ${1}.btf.o ready for linking into vmlinux +# - ${1}.btf_ids with .BTF_ids data blob +# This output is consumed by scripts/link-vmlinux.sh + +set -e + +usage() +{ + echo "Usage: $0 [--btf_base <file>] <target ELF file>" + exit 1 +} + +BTF_BASE="" + +while [ $# -gt 0 ]; do + case "$1" in + --btf_base) + BTF_BASE="$2" + shift 2 + ;; + -*) + echo "Unknown option: $1" >&2 + usage + ;; + *) + break + ;; + esac +done + +if [ $# -ne 1 ]; then + usage +fi + +ELF_FILE="$1" +shift + +is_enabled() { + grep -q "^$1=y" ${objtree}/include/config/auto.conf +} + +info() +{ + printf " %-7s %s\n" "${1}" "${2}" +} + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings " + fi + if [ -n "${KBUILD_VERBOSE}" ]; then + RESOLVE_BTFIDS_OPTS+=" -v " + fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} + +gen_btf_o() +{ + local btf_data=${ELF_FILE}.btf.o + + # Create ${btf_data} which contains just .BTF section but no symbols. Add + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all + # deletes all symbols including __start_BTF and __stop_BTF, which will + # be redefined in the linker script. + info OBJCOPY "${btf_data}" + echo "" | ${CC} -c -x c -o ${btf_data} - + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \ + --set-section-flags .BTF=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + + # Change e_type to ET_REL so that it can be used to link final vmlinux. + # GNU ld 2.35+ and lld do not allow an ET_EXEC input. + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + et_rel='\0\1' + else + et_rel='\1\0' + fi + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none +} + +embed_btf_data() +{ + info OBJCOPY "${ELF_FILE}" + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE} + + # a module might not have a .BTF_ids or .BTF.base section + local btf_base="${ELF_FILE}.distilled_base.btf" + if [ -f "${btf_base}" ]; then + ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE} + fi + local btf_ids="${ELF_FILE}.btf_ids" + if [ -f "${btf_ids}" ]; then + ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE} + fi +} + +cleanup() +{ + rm -f "${ELF_FILE}.btf.1" + rm -f "${ELF_FILE}.btf" + if [ "${BTFGEN_MODE}" = "module" ]; then + rm -f "${ELF_FILE}.distilled_base.btf" + rm -f "${ELF_FILE}.btf_ids" + fi +} +trap cleanup EXIT + +BTFGEN_MODE="vmlinux" +if [ -n "${BTF_BASE}" ]; then + BTFGEN_MODE="module" +fi + +gen_btf_data + +case "${BTFGEN_MODE}" in +vmlinux) + gen_btf_o + ;; +module) + embed_btf_data + ;; +esac + +exit 0scripts/link-vmlinux.sh +6
-36@@ -105,34 +105,6 @@ vmlinux_link() ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} } -# generate .BTF typeinfo from DWARF debuginfo -# ${1} - vmlinux image -gen_btf() -{ - local btf_data=${1}.btf.o - - info BTF "${btf_data}" - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1} - - # Create ${btf_data} which contains just .BTF section but no symbols. Add - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all - # deletes all symbols including __start_BTF and __stop_BTF, which will - # be redefined in the linker script. Add 2>/dev/null to suppress GNU - # objcopy warnings: "empty loadable segment detected at ..." - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \ - --strip-all ${1} "${btf_data}" 2>/dev/null - # Change e_type to ET_REL so that it can be used to link final vmlinux. - # GNU ld 2.35+ and lld do not allow an ET_EXEC input. - if is_enabled CONFIG_CPU_BIG_ENDIAN; then - et_rel='\0\1' - else - et_rel='\1\0' - fi - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none - - btf_vmlinux_bin_o=${btf_data} -} - # Create ${2}.o file with all symbols from the ${1} object file kallsyms() {@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi btf_vmlinux_bin_o= +btfids_vmlinux= kallsymso= strip_debug= generate_map=@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then fi if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then echo >&2 "Failed to generate BTF for vmlinux" echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF" exit 1 fi + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o + btfids_vmlinux=.tmp_vmlinux1.btf_ids fi if is_enabled CONFIG_KALLSYMS; then@@ -281,14 +256,9 @@ fi vmlinux_link "${VMLINUX}" -# fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS "${VMLINUX}" - RESOLVE_BTFIDS_ARGS="" - if is_enabled CONFIG_WERROR; then - RESOLVE_BTFIDS_ARGS=" --fatal_warnings " - fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" + info OBJCOPY ${btfids_vmlinux} + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX} fi mksysmap "${VMLINUX}" System.maptools/bpf/resolve_btfids/main.c +136
-82@@ -71,9 +71,11 @@#include <fcntl.h> #include <errno.h> #include <linux/btf_ids.h> +#include <linux/kallsyms.h> #include <linux/rbtree.h> #include <linux/zalloc.h> #include <linux/err.h> +#include <linux/limits.h> #include <bpf/btf.h> #include <bpf/libbpf.h> #include <subcmd/parse-options.h> @@ -124,6 +126,7 @@ struct object {struct btf *btf;struct btf *base_btf; +bool distilled_base;struct {int fd;@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return btf_id; } -/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */ -#ifndef SHF_COMPRESSED -#define SHF_COMPRESSED (1 << 11)/* Section with compressed data. */ -#endif - -/* -* The data of compressed section should be aligned to 4 -* (for 32bit) or 8 (for 64 bit) bytes. The binutils ld -* sets sh_addralign to 1, which makes libelf fail with -* misaligned section error during the update: -* FAILED elf_update(WRITE): invalid section alignment -* -* While waiting for ld fix, we fix the compressed sections -* sh_addralign value manualy. -*/ -static int compressed_section_fix (Elf *elf,Elf_Scn *scn,GElf_Shdr *sh) +static void bswap_32_data (void *data,u32 nr_bytes) { -int expected =gelf_getclass (elf) ==ELFCLASS32 ?4 :8 ; +u32 cnt, i; +u32 *ptr; -if (!(sh->sh_flags &SHF_COMPRESSED )) -return 0 ; + cnt = nr_bytes /sizeof (u32); + ptr = data; -if (sh->sh_addralign == expected) -return 0 ; - -pr_debug2 (" - fixing wrong alignment sh_addralign %u, expected %u\n" , - sh->sh_addralign, expected); - - sh->sh_addralign = expected; - -if (gelf_update_shdr (scn, sh) ==0 ) { -pr_err ("FAILED cannot update section header: %s\n" , -elf_errmsg (-1 )); -return -1 ; - } -return 0 ; +for (i =0 ; i < cnt; i++) + ptr[i] =bswap_32 (ptr[i]); }static int elf_collect (struct object *obj)@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)elf_version (EV_CURRENT ); - elf =elf_begin (fd,ELF_C_RDWR_MMAP ,NULL ); + elf =elf_begin (fd,ELF_C_READ_MMAP_PRIVATE ,NULL );if (!elf) {close (fd); pr_err("FAILED cannot create ELF descriptor: %s\n",@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj) obj->efile.symbols_shndx = idx; obj->efile.strtabidx = sh.sh_link; }else if (!strcmp (name,BTF_IDS_SECTION )) { +/* +* If target endianness differs from host, we need to bswap32 +* the .BTF_ids section data on load, because .BTF_ids has +* Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in +* the target endiannes. We repeat this on dump. +*/ +if (obj->efile.encoding !=ELFDATANATIVE ) { +pr_debug ("bswap_32 .BTF_ids data from target to host endianness\n" ); +bswap_32_data (data->d_buf, data->d_size); + } obj->efile.idlist = data; obj->efile.idlist_shndx = idx; obj->efile.idlist_addr = sh.sh_addr; - }else if (!strcmp (name,BTF_BASE_ELF_SEC )) { -/* If a .BTF.base section is found, do not resolve -* BTF ids relative to vmlinux; resolve relative -* to the .BTF.base section instead. btf__parse_split() -* will take care of this once the base BTF it is -* passed is NULL. -*/ - obj->base_btf_path =NULL ; } - -if (compressed_section_fix (elf, scn, &sh)) -return -1 ; }return 0 ;@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)return 0 ; } +static inline bool is_envvar_set (const char *var_name) +{ +const char *value =getenv (var_name); + +return value && value[0 ] !='\0' ; +} +static int load_btf (struct object *obj) {struct btf *base_btf =NULL , *btf =NULL ;@@ -571,6 +554,20 @@ static int load_btf(struct object *obj) obj->base_btf = base_btf; obj->btf = btf; +if (obj->base_btf &&is_envvar_set ("KBUILD_EXTMOD" )) { + err =btf__distill_base (obj->btf, &base_btf, &btf); +if (err) { +pr_err ("FAILED to distill base BTF: %s\n" ,strerror (errno)); + goto out_err; + } + +btf__free (obj->btf); +btf__free (obj->base_btf); + obj->btf = btf; + obj->base_btf = base_btf; + obj->distilled_base = true; + } +return 0 ; out_err:@@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)*/ BUILD_BUG_ON ((u32 *)set8->pairs != &set8->pairs[0 ].id);qsort (set8->pairs, set8->cnt,sizeof (set8->pairs[0 ]), cmp_id); - -/* -* When ELF endianness does not match endianness of the -* host, libelf will do the translation when updating -* the ELF. This, however, corrupts SET8 flags which are -* already in the target endianness. So, let's bswap -* them to the host endianness and libelf will then -* correctly translate everything. -*/ -if (obj->efile.encoding !=ELFDATANATIVE ) { -int i; - - set8->flags =bswap_32 (set8->flags); -for (i =0 ; i < set8->cnt; i++) { - set8->pairs[i].flags = -bswap_32 (set8->pairs[i].flags); - } - }break ;case BTF_ID_KIND_SYM :default :@@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)static int symbols_patch (struct object *obj) { -off_t err; - if (__symbols_patch(obj, &obj->structs) || __symbols_patch(obj, &obj->unions) || __symbols_patch(obj, &obj->typedefs) ||@@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)if (sets_patch (obj))return -1 ; -/* Set type to ensure endian translation occurs. */ - obj->efile.idlist->d_type =ELF_T_WORD ; +return 0 ; +} -elf_flagdata (obj->efile.idlist,ELF_C_SET ,ELF_F_DIRTY ); +static int dump_raw_data (const char *out_path,const void *data,u32 size) +{ +int fd, ret; - err =elf_update (obj->efile.elf,ELF_C_WRITE ); -if (err <0 ) { -pr_err ("FAILED elf_update(WRITE): %s\n" , -elf_errmsg (-1 )); + fd =open (out_path,O_WRONLY |O_CREAT |O_TRUNC ,0640 ); +if (fd <0 ) { +pr_err ("Couldn't open %s for writing\n" , out_path); +return fd; + } + + ret =write (fd, data, size); +if (ret <0 || ret != size) { +pr_err ("Failed to write data to %s\n" , out_path); +close (fd); +unlink (out_path); +return -1 ; + } + +close (fd); +pr_debug ("Dumped %lu bytes of data to %s\n" , size, out_path); + +return 0 ; +} + +static int dump_raw_btf_ids (struct object *obj,const char *out_path) +{ +Elf_Data *data = obj->efile.idlist; +int fd, err; + +if (!data || !data->d_buf) { +pr_debug ("%s has no BTF_ids data to dump\n" , obj->path); +return 0 ; + } + +/* +* If target endianness differs from host, we need to bswap32 the +* .BTF_ids section data before dumping so that the output is in +* target endianness. +*/ +if (obj->efile.encoding !=ELFDATANATIVE ) { +pr_debug ("bswap_32 .BTF_ids data from host to target endianness\n" ); +bswap_32_data (data->d_buf, data->d_size); + } + + err =dump_raw_data (out_path, data->d_buf, data->d_size); +if (err) +return -1 ; + +return 0 ; +} + +static int dump_raw_btf (struct btf *btf,const char *out_path) +{ +const void *raw_btf_data; +u32 raw_btf_size; +int fd, err; + + raw_btf_data =btf__raw_data (btf, &raw_btf_size); +if (raw_btf_data ==NULL ) { +pr_err ("btf__raw_data() failed\n" ); +return -1 ; } -pr_debug ("update %s for %s\n" , - err >=0 ?"ok" :"failed" , obj->path); -return err <0 ?-1 :0 ; + err =dump_raw_data (out_path, raw_btf_data, raw_btf_size); +if (err) +return -1 ; + +return 0 ; }static const char *const resolve_btfids_usage[] = {@@ -824,6 +858,7 @@ int main(int argc, const char **argv) .funcs = RB_ROOT, .sets = RB_ROOT, }; +char out_path[PATH_MAX ];bool fatal_warnings = false;struct option btfid_options[] = { OPT_INCR('v', "verbose", &verbose,@@ -836,7 +871,7 @@ int main(int argc, const char **argv) "turn warnings into errors"), OPT_END() }; -int err =-1 ; +int err =-1 , path_len; argc =parse_options (argc, argv, btfid_options, resolve_btfids_usage,PARSE_OPT_STOP_AT_NON_OPTION );@@ -844,6 +879,11 @@ int main(int argc, const char **argv)usage_with_options (resolve_btfids_usage, btfid_options); obj.path = argv[0 ]; +strcpy (out_path, obj.path); + path_len =strlen (out_path); + +if (load_btf (&obj)) + goto out;if (elf_collect (&obj)) goto out;@@ -854,23 +894,37 @@ int main(int argc, const char **argv)*/ if (obj.efile.idlist_shndx ==-1 || obj.efile.symbols_shndx ==-1 ) { -pr_debug ("Cannot find .BTF_ids or symbols sections, nothing to do\n" ); - err =0 ; - goto out; +pr_debug ("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n" ); + goto dump_btf; }if (symbols_collect (&obj)) goto out; -if (load_btf (&obj)) - goto out; -if (symbols_resolve (&obj)) goto out;if (symbols_patch (&obj)) goto out; + out_path[path_len] ='\0' ; +strcat (out_path,".btf_ids" ); +if (dump_raw_btf_ids (&obj, out_path)) + goto out; + +dump_btf: + out_path[path_len] ='\0' ; +strcat (out_path,".btf" ); +if (dump_raw_btf (obj.btf, out_path)) + goto out; + +if (obj.base_btf && obj.distilled_base) { + out_path[path_len] ='\0' ; +strcat (out_path,".distilled_base.btf" ); +if (dump_raw_btf (obj.base_btf, out_path)) + goto out; + } +if (!(fatal_warnings && warnings)) err =0 ; out:tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)
tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \
I encountered a resolve_btfids self-test failure when enabling the BTF sorting feature, with the following error output:
All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAIL
The root cause is that prog_tests/resolve_btfids.c retrieves type IDs from btf_data.bpf.o and compares them against the IDs in test_progs. However, while the IDs in test_progs are sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modify prog_tests/resolve_btfids.c to retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.
What do you think of it?
Thanks,
Donglin
Quoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.
Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within resolve_btfids it's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.
As for the test, we should probably change it to become closer to an actual resolve_btfids use-case. Maybe even replace or remove it.
resolve_btfids operates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).
And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (test_progs / resolve_btfids.test.o), although using proper kernel macros.
By the way, the test was written more than 5y ago [1], so it might be outdated too.
I think the behavior that we care about is already indirectly tested by bpf_testmod module tests, with custom BPF kfuncs and BTF_ID_* declarations etc. If resolve_btfids is broken, those tests will fail.
But it's also reasonable to have some tests targeting resolve_btfids app itself, of course. This one doesn't fit though IMO.
I'll try to think of something.
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within
resolve_btfidsit's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.As for the test, we should probably change it to become closer to an actual
resolve_btfidsuse-case. Maybe even replace or remove it.
resolve_btfidsoperates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (
test_progs/ resolve_btfids.test.o), although using proper kernel macros.By the way, the test was written more than 5y ago [1], so it might be outdated too.
Quoted 8 lines
I think the behavior that we care about is already indirectly tested by
bpf_testmodmodule tests, with custom BPF kfuncs and BTF_ID_* declarations etc. Ifresolve_btfidsis broken, those tests will fail.But it's also reasonable to have some tests targeting
resolve_btfidsapp itself, of course. This one doesn't fit though IMO.I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing resolve_btfids output.
In this series, I only have to change the test_progs build recipe.
The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:
static int resolve_symbols(void)
{
struct btf *btf;
int type_id;
__u32 nr;
btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */
[...]Instead of reading in the source BTF, we have to load .btf produced by resolve_btfids. A complication is that it's going to be a different file for every TRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.
If go this route, I think we should add .btf cleanup to the Makefile and update local .gitignore
This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.
Quoted 17 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within
resolve_btfidsit's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.As for the test, we should probably change it to become closer to an actual
resolve_btfidsuse-case. Maybe even replace or remove it.
resolve_btfidsoperates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (
test_progs/ resolve_btfids.test.o), although using proper kernel macros.By the way, the test was written more than 5y ago [1], so it might be outdated too.
Quoted 8 lines
I think the behavior that we care about is already indirectly tested by
bpf_testmodmodule tests, with custom BPF kfuncs and BTF_ID_* declarations etc. Ifresolve_btfidsis broken, those tests will fail.But it's also reasonable to have some tests targeting
resolve_btfidsapp itself, of course. This one doesn't fit though IMO.I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing
resolve_btfidsoutput.In this series, I only have to change the
test_progsbuild recipe.The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:
static int resolve_symbols(void) { struct btf *btf; int type_id; __u32 nr;btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */ [...]
Quoted 7 lines
Instead of reading in the source BTF, we have to load .btf produced by
resolve_btfids. A complication is that it's going to be a different file for everyTRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.If go this route, I think we should add .btf cleanup to the
Makefileand update local .gitignore
Thanks, could the following modification be accepted?
(quoted diff) +0 -0
b/tools/testing/selftests/bpf/.gitignore
index be1ee7ba7ce0..38ac369cd701 100644
tools/testing/selftests/bpf/.gitignore +1 -0
@@ -45,3 +45,4 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.btf
(quoted diff) +0 -0
b/tools/testing/selftests/bpf/Makefile
index 2a027ff9ceaf..a1188129229f 100644
tools/testing/selftests/bpf/Makefile +1 -1
@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(Q)if [ -f [email protected]_ids ]; then \ $(OBJCOPY) --update-section [email protected]_ids $$@; \ fi - $(Q)rm -f [email protected]_ids [email protected] + $(Q)rm -f [email protected]_ids $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool
@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
\
prog_tests/tests.h map_tests/tests.h verifier/tests.h \
feature bpftool $(TEST_KMOD_TARGETS) \
$(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \
- no_alu32 cpuv4 bpf_gcc \
+ *.btf no_alu32 cpuv4 bpf_gcc \
liburandom_read.so) \
$(OUTPUT)/FEATURE-DUMP.selftests(quoted diff) +0 -0
b/tools/testing/selftests/bpf/prog_tests/resolve_btfids.c index 51544372f52e..00883ff16569 100644
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1 -1
@@ -101,7 +101,7 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("test_progs.btf" );if (CHECK (libbpf_get_error (btf),"resolve" ,"Failed to load BTF from btf_data.bpf.o\n" ))return -1 ;
Thanks,
Donglin
Quoted 4 lines
This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.
Thanks, I will take it on.
Quoted 19 lines
Quoted 18 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within
resolve_btfidsit's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.As for the test, we should probably change it to become closer to an actual
resolve_btfidsuse-case. Maybe even replace or remove it.
resolve_btfidsoperates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (
test_progs/ resolve_btfids.test.o), although using proper kernel macros.By the way, the test was written more than 5y ago [1], so it might be outdated too.
Quoted 8 lines
I think the behavior that we care about is already indirectly tested by
bpf_testmodmodule tests, with custom BPF kfuncs and BTF_ID_* declarations etc. Ifresolve_btfidsis broken, those tests will fail.But it's also reasonable to have some tests targeting
resolve_btfidsapp itself, of course. This one doesn't fit though IMO.I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing
resolve_btfidsoutput.In this series, I only have to change the
test_progsbuild recipe.The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:
static int resolve_symbols(void) { struct btf *btf; int type_id; __u32 nr;btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */ [...]Quoted 7 lines
Instead of reading in the source BTF, we have to load .btf produced by
resolve_btfids. A complication is that it's going to be a different file for everyTRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.If go this route, I think we should add .btf cleanup to the
Makefileand update local .gitignoreThanks, could the following modification be accepted?
tools/testing/selftests/bpf/.gitignore +1
-0@@ -45,3 +45,4 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.btftools/testing/selftests/bpf/Makefile +2
-2@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(Q)if [ -f [email protected]_ids ]; then \ $(OBJCOPY) --update-section [email protected]_ids $$@; \ fi - $(Q)rm -f [email protected]_ids [email protected] + $(Q)rm -f [email protected]_ids $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ - no_alu32 cpuv4 bpf_gcc \ + *.btf no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selfteststools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1
-1@@ -101,7 +101,7 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("test_progs.btf" );
We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (test_progs, test_maps, test_progs-no_alu32 and others).
I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the final TRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option in resolve_btfids
- somehow (env var?) determine what binary is running in the test
- (a hack) in the makefile, copy [email protected] to "test.btf" or similar
IMO the first option is the best, as this makefile code exists because of that specific test.
The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0if (CHECK (libbpf_get_error (btf),"resolve" ,"Failed to load BTF from btf_data.bpf.o\n" ))return -1 ;Thanks,
DonglinQuoted 1 line
Quoted 5 lines
Quoted 3 lines
This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.
Thanks, I will take it on.
Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 168 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within
resolve_btfidsit's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.As for the test, we should probably change it to become closer to an actual
resolve_btfidsuse-case. Maybe even replace or remove it.
resolve_btfidsoperates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (
test_progs/ resolve_btfids.test.o), although using proper kernel macros.By the way, the test was written more than 5y ago [1], so it might be outdated too.
Quoted 8 lines
I think the behavior that we care about is already indirectly tested by
bpf_testmodmodule tests, with custom BPF kfuncs and BTF_ID_* declarations etc. Ifresolve_btfidsis broken, those tests will fail.But it's also reasonable to have some tests targeting
resolve_btfidsapp itself, of course. This one doesn't fit though IMO.I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing
resolve_btfidsoutput.In this series, I only have to change the
test_progsbuild recipe.The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:
static int resolve_symbols(void) { struct btf *btf; int type_id; __u32 nr;btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */ [...]Quoted 7 lines
Instead of reading in the source BTF, we have to load .btf produced by
resolve_btfids. A complication is that it's going to be a different file for everyTRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.If go this route, I think we should add .btf cleanup to the
Makefileand update local .gitignoreThanks, could the following modification be accepted?
tools/testing/selftests/bpf/.gitignore +1
-0@@ -45,3 +45,4 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.btftools/testing/selftests/bpf/Makefile +2
-2@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(Q)if [ -f [email protected]_ids ]; then \ $(OBJCOPY) --update-section [email protected]_ids $$@; \ fi - $(Q)rm -f [email protected]_ids [email protected] + $(Q)rm -f [email protected]_ids $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ - no_alu32 cpuv4 bpf_gcc \ + *.btf no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selfteststools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1
-1@@ -101,7 +101,7 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("test_progs.btf" );We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (
test_progs,test_maps, test_progs-no_alu32 and others).I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the finalTRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option inresolve_btfids
- somehow (env var?) determine what binary is running in the test
- (a hack) in the makefile, copy [email protected] to "test.btf" or similar
Quoted 5 lines
IMO the first option is the best, as this makefile code exists because of that specific test.
The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.
Thanks, I understand. Here are the changes based on the first option:
(quoted diff) +0 -0
b/tools/testing/selftests/bpf/Makefile
index 2a027ff9ceaf..751960aeb8e5 100644
tools/testing/selftests/bpf/Makefile +2 -0
@@ -704,6 +704,16 @@ ifneq ($2:$(OUTPUT),:$(shell pwd)) $(Q)rsync -aq $$^ $(TRUNNER_OUTPUT)/ endif +ifneq ($(TRUNNER_BINARY),test_maps) +$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf
$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids:
$(TRUNNER_OUTPUT)/btf_data.bpf.o \
+
$(TRUNNER_OUTPUT)/resolve_btfids.test.o \
+
$(RESOLVE_BTFIDS)+ $(call msg,BTF+IDS,resolve_btfids,$@)
+ $(Q)$(RESOLVE_BTFIDS) --btf $(dir $@)btf_data.bpf.o $(dir
$@)resolve_btfids.test.o
+
+$(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids +endif
+
# some X.test.o files have runtime dependencies on Y.bpf.o files $(OUTPUT)/$(TRUNNER_BINARY): | $(TRUNNER_BPF_OBJS)
@@ -716,11 +726,9 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS)
\
| $(TRUNNER_BINARY)-extras
$$(call msg,BINARY,,$$@)
$(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS)$$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
- $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@
- $(Q)if [ -f [email protected]_ids ]; then \
- $(OBJCOPY) --update-section [email protected]_ids $$@; \
+ $(Q)if [ "$(TRUNNER_BINARY)" != "test_maps" ]; then \
+ $(OBJCOPY) --update-section
.BTF_ids=$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids $$@; \
fi
- $(Q)rm -f [email protected]_ids [email protected]
$(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \
$(OUTPUT)/$(if $2,$2/)bpftool@@ -908,7 +916,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR)
\
prog_tests/tests.h map_tests/tests.h verifier/tests.h \
feature bpftool $(TEST_KMOD_TARGETS) \
$(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \
- no_alu32 cpuv4 bpf_gcc \
+ *.btf *.btf_ids no_alu32 cpuv4 bpf_gcc \
liburandom_read.so) \
$(OUTPUT)/FEATURE-DUMP.selftests(quoted diff) +0 -0
b/tools/testing/selftests/bpf/prog_tests/resolve_btfids.c index 51544372f52e..eef6efc82606 100644
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +2 -2
@@ -101,9 +101,9 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("resolve_btfids.test.o.btf" );if (CHECK (libbpf_get_error (btf),"resolve" , -"Failed to load BTF from btf_data.bpf.o\n" )) +"Failed to load BTF from resolve_btfids.test.o.btf\n" ))return -1 ; nr =btf__type_cnt (btf);
Quoted 40 lines
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0if (CHECK (libbpf_get_error (btf),"resolve" ,"Failed to load BTF from btf_data.bpf.o\n" ))return -1 ;Thanks,
DonglinQuoted 1 line
Quoted 5 lines
Quoted 3 lines
This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.
Thanks, I will take it on.
Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 197 lines · Donglin Peng
Quoted 168 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within
resolve_btfidsit's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.As for the test, we should probably change it to become closer to an actual
resolve_btfidsuse-case. Maybe even replace or remove it.
resolve_btfidsoperates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (
test_progs/ resolve_btfids.test.o), although using proper kernel macros.By the way, the test was written more than 5y ago [1], so it might be outdated too.
Quoted 8 lines
I think the behavior that we care about is already indirectly tested by
bpf_testmodmodule tests, with custom BPF kfuncs and BTF_ID_* declarations etc. Ifresolve_btfidsis broken, those tests will fail.But it's also reasonable to have some tests targeting
resolve_btfidsapp itself, of course. This one doesn't fit though IMO.I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing
resolve_btfidsoutput.In this series, I only have to change the
test_progsbuild recipe.The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:
static int resolve_symbols(void) { struct btf *btf; int type_id; __u32 nr;btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */ [...]Quoted 7 lines
Instead of reading in the source BTF, we have to load .btf produced by
resolve_btfids. A complication is that it's going to be a different file for everyTRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.If go this route, I think we should add .btf cleanup to the
Makefileand update local .gitignoreThanks, could the following modification be accepted?
tools/testing/selftests/bpf/.gitignore +1
-0@@ -45,3 +45,4 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.btftools/testing/selftests/bpf/Makefile +2
-2@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(Q)if [ -f [email protected]_ids ]; then \ $(OBJCOPY) --update-section [email protected]_ids $$@; \ fi - $(Q)rm -f [email protected]_ids [email protected] + $(Q)rm -f [email protected]_ids $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ - no_alu32 cpuv4 bpf_gcc \ + *.btf no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selfteststools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1
-1@@ -101,7 +101,7 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("test_progs.btf" );We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (
test_progs,test_maps, test_progs-no_alu32 and others).I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the finalTRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option inresolve_btfids
- somehow (env var?) determine what binary is running in the test
- (a hack) in the makefile, copy [email protected] to "test.btf" or similarQuoted 5 lines
IMO the first option is the best, as this makefile code exists because of that specific test.
The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.
Thanks, I understand. Here are the changes based on the first option:
tools/testing/selftests/bpf/Makefile +2
-0@@ -704,6 +704,16 @@ ifneq ($2:$(OUTPUT),:$(shell pwd)) $(Q)rsync -aq $$^ $(TRUNNER_OUTPUT)/ endif +ifneq ($(TRUNNER_BINARY),test_maps) +$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids:
$(TRUNNER_OUTPUT)/btf_data.bpf.o \tools/testing/selftests/bpf/Makefile +4
-0+ $(TRUNNER_OUTPUT)/resolve_btfids.test.o \ + $(RESOLVE_BTFIDS) + $(call msg,BTF+IDS,resolve_btfids,$@) + $(Q)$(RESOLVE_BTFIDS) --btf $(dir $@)btf_data.bpf.o $(dir
Quoted 1 line
$@)resolve_btfids.test.o
Sorry, the above command has some issues. Use the following command instead:
$(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)btf_data.bpf.o $(TRUNNER_OUTPUT)resolve_btfids.test.o
Quoted 92 lines
tools/testing/selftests/bpf/Makefile +4
-0+ +$(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids +endif + # some X.test.o files have runtime dependencies on Y.bpf.o files $(OUTPUT)/$(TRUNNER_BINARY): | $(TRUNNER_BPF_OBJS)@@ -716,11 +726,9 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ | $(TRUNNER_BINARY)-extras $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS)$$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@
tools/testing/selftests/bpf/Makefile +2
-3- $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ - $(Q)if [ -f [email protected]_ids ]; then \ - $(OBJCOPY) --update-section [email protected]_ids $$@; \ + $(Q)if [ "$(TRUNNER_BINARY)" != "test_maps" ]; then \ + $(OBJCOPY) --update-section.BTF_ids=$(TRUNNER_OUTPUT)/resolve_btfids.test.o.btf_ids $$@; \
fitools/testing/selftests/bpf/Makefile +1
-2- $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool@@ -908,7 +916,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ - no_alu32 cpuv4 bpf_gcc \ + *.btf *.btf_ids no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selfteststools/testing/selftests/bpf/prog_tests/resolve_btfids.c +2
-2@@ -101,9 +101,9 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("resolve_btfids.test.o.btf" );if (CHECK (libbpf_get_error (btf),"resolve" , -"Failed to load BTF from btf_data.bpf.o\n" )) +"Failed to load BTF from resolve_btfids.test.o.btf\n" ))return -1 ; nr =btf__type_cnt (btf);Quoted 41 lines
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0if (CHECK (libbpf_get_error (btf),"resolve" ,"Failed to load BTF from btf_data.bpf.o\n" ))return -1 ;Thanks,
DonglinQuoted 1 line
Quoted 5 lines
Quoted 3 lines
This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.
Thanks, I will take it on.
Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
In reply to Ihor SolodraiQuoted 160 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within
resolve_btfidsit's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.As for the test, we should probably change it to become closer to an actual
resolve_btfidsuse-case. Maybe even replace or remove it.
resolve_btfidsoperates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (
test_progs/ resolve_btfids.test.o), although using proper kernel macros.By the way, the test was written more than 5y ago [1], so it might be outdated too.
Quoted 8 lines
I think the behavior that we care about is already indirectly tested by
bpf_testmodmodule tests, with custom BPF kfuncs and BTF_ID_* declarations etc. Ifresolve_btfidsis broken, those tests will fail.But it's also reasonable to have some tests targeting
resolve_btfidsapp itself, of course. This one doesn't fit though IMO.I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing
resolve_btfidsoutput.In this series, I only have to change the
test_progsbuild recipe.The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:
static int resolve_symbols(void) { struct btf *btf; int type_id; __u32 nr;btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */ [...]Quoted 7 lines
Instead of reading in the source BTF, we have to load .btf produced by
resolve_btfids. A complication is that it's going to be a different file for everyTRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.If go this route, I think we should add .btf cleanup to the
Makefileand update local .gitignoreThanks, could the following modification be accepted?
tools/testing/selftests/bpf/.gitignore +1
-0@@ -45,3 +45,4 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.btftools/testing/selftests/bpf/Makefile +2
-2@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(Q)if [ -f [email protected]_ids ]; then \ $(OBJCOPY) --update-section [email protected]_ids $$@; \ fi - $(Q)rm -f [email protected]_ids [email protected] + $(Q)rm -f [email protected]_ids $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ - no_alu32 cpuv4 bpf_gcc \ + *.btf no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selfteststools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1
-1@@ -101,7 +101,7 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("test_progs.btf" );We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (
test_progs,test_maps, test_progs-no_alu32 and others).
Quoted 6 lines
I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the finalTRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option inresolve_btfids
- somehow (env var?) determine what binary is running in the test
see cd_flavor_subdir(), each flavor of test_progs has its dedicated subdir (and flavor-less one just runs in selftests' directory). So I think you should be able to hard-code the name. Even for btf_data.bpf.o:
$ find . -name 'btf_data.bpf.o' ./no_alu32/btf_data.bpf.o ./cpuv4/btf_data.bpf.o ./btf_data.bpf.o
Quoted 48 lines
- (a hack) in the makefile, copy [email protected] to "test.btf" or similar
IMO the first option is the best, as this makefile code exists because of that specific test.
The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0if (CHECK (libbpf_get_error (btf),"resolve" ,"Failed to load BTF from btf_data.bpf.o\n" ))return -1 ;Thanks,
DonglinQuoted 1 line
Quoted 5 lines
Quoted 3 lines
This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.
Thanks, I will take it on.
Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 174 lines · Andrii Nakryiko
Quoted 160 lines · Ihor Solodrai
Quoted 148 lines · Donglin Peng
Quoted 96 lines · Ihor Solodrai
Quoted 65 lines · Ihor Solodrai
Quoted 38 lines · Donglin Peng
Quoted 16 lines · Ihor Solodrai
[...]
tools/testing/selftests/bpf/Makefile +1
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)tools/testing/selftests/bpf/Makefile $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ · line 721
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2
-0@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \I encountered a
resolve_btfidsself-test failure when enabling the BTF sorting feature, with the following error output:All error logs:
resolve_symbols:PASS:resolve 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:PASS:id_check 0 nsec test_resolve_btfids:FAIL:id_check wrong ID for T (7 != 5)
#369 resolve_btfids:FAILThe root cause is that
prog_tests/resolve_btfids.cretrieves type IDs from btf_data.bpf.o and compares them against the IDs intest_progs. However, while the IDs intest_progsare sorted, those in btf_data.bpf.o remain in their original unsorted state, causing the validation to fail.Quoted 8 lines
This presents two potential solutions:
1. Update the relevant .BTF.* section datas in btf_data.bpf.o, including the .BTF and .BTF.ext sections
2. Modifyprog_tests/resolve_btfids.cto retrieve IDs from test_progs.btf instead. However, I discovered that test_progs.btf is deleted in the subsequent code section.What do you think of it?
Within
resolve_btfidsit's clear that we have to update (sort in this case) BTF first, and then resolve the ids based on the changed BTF.As for the test, we should probably change it to become closer to an actual
resolve_btfidsuse-case. Maybe even replace or remove it.
resolve_btfidsoperates on BTF generated by pahole for kernel/module. And the .BTF_ids section makes sense only in kernel space AFAIU (might be wrong, let me know if I am).And in this test we are using BTF produced by LLVM for a BPF program, and then create a .BTF_ids section in a user-space app (
test_progs/ resolve_btfids.test.o), although using proper kernel macros.By the way, the test was written more than 5y ago [1], so it might be outdated too.
Quoted 8 lines
I think the behavior that we care about is already indirectly tested by
bpf_testmodmodule tests, with custom BPF kfuncs and BTF_ID_* declarations etc. Ifresolve_btfidsis broken, those tests will fail.But it's also reasonable to have some tests targeting
resolve_btfidsapp itself, of course. This one doesn't fit though IMO.I'll try to think of something.
Hi Donglin,
I discussed this off-list with Andrii, and we agreed that the selftest itself is reasonable with respect to testing
resolve_btfidsoutput.In this series, I only have to change the
test_progsbuild recipe.The problem that you've encountered I think can be fixed in the test, which is basically what you suggested as option 2:
static int resolve_symbols(void) { struct btf *btf; int type_id; __u32 nr;btf = btf__parse_elf("btf_data.bpf.o", NULL); /* <--- this */ [...]Quoted 7 lines
Instead of reading in the source BTF, we have to load .btf produced by
resolve_btfids. A complication is that it's going to be a different file for everyTRUNNER_BINARY, which has to be accounted for, although the BTF itself would be identical between relevant runners.If go this route, I think we should add .btf cleanup to the
Makefileand update local .gitignoreThanks, could the following modification be accepted?
tools/testing/selftests/bpf/.gitignore +1
-0@@ -45,3 +45,4 @@ xdp_synproxy xdp_hw_metadata xdp_features verification_cert.h +*.btftools/testing/selftests/bpf/Makefile +2
-2@@ -720,7 +720,7 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $(Q)if [ -f [email protected]_ids ]; then \ $(OBJCOPY) --update-section [email protected]_ids $$@; \ fi - $(Q)rm -f [email protected]_ids [email protected] + $(Q)rm -f [email protected]_ids $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool@@ -908,7 +908,7 @@ EXTRA_CLEAN := $(SCRATCH_DIR) $(HOST_SCRATCH_DIR) \ prog_tests/tests.h map_tests/tests.h verifier/tests.h \ feature bpftool $(TEST_KMOD_TARGETS) \ $(addprefix $(OUTPUT)/,*.o *.d *.skel.h *.lskel.h *.subskel.h \ - no_alu32 cpuv4 bpf_gcc \ + *.btf no_alu32 cpuv4 bpf_gcc \ liburandom_read.so) \ $(OUTPUT)/FEATURE-DUMP.selfteststools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0tools/testing/selftests/bpf/prog_tests/resolve_btfids.c static int resolve_symbols(void) · line 104
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +1
-1@@ -101,7 +101,7 @@ static int resolve_symbols(void)int type_id;__u32 nr; - btf =btf__parse_elf ("btf_data.bpf.o" ,NULL ); + btf =btf__parse_raw ("test_progs.btf" );We can't hardcode a filename here, because $(OUTPUT)/$(TRUNNER_BINARY) is a generic rule for a number of different binaries (
test_progs,test_maps, test_progs-no_alu32 and others).Quoted 6 lines
I think there are a few options how to deal with this:
- generate .btf and .btf_ids not for the finalTRUNNER_BINARY, but for a specific test object (resolve_btfids.test.o in this case); then we could load "resolve_btfids.test.o.btf"
- implement an --output-btf option inresolve_btfids
- somehow (env var?) determine what binary is running in the testsee
cd_flavor_subdir(), each flavor oftest_progshas its dedicated subdir (and flavor-less one just runs in selftests' directory). So I think you should be able to hard-code the name. Even for btf_data.bpf.o:
Quoted 4 lines
$ find . -name 'btf_data.bpf.o' ./no_alu32/btf_data.bpf.o ./cpuv4/btf_data.bpf.o ./btf_data.bpf.o
I think the first option is feasible. Based on my modifications, the resolve_btfids.test.o.btffile will exist in each flavor's subdirectory. Therefore, we can hard-code the filename resolve_btfids.test.o.btf directly in prog_tests/resolve_btfids.c:
$ find -name resolve_btfids.test.o.btf ./resolve_btfids.test.o.btf ./cpuv4/resolve_btfids.test.o.btf ./no_alu32/resolve_btfids.test.o.btf
Quoted 50 lines
Quoted 50 lines
- (a hack) in the makefile, copy [email protected] to "test.btf" or similar
IMO the first option is the best, as this makefile code exists because of that specific test.
The --output-btf is okay in principle, but I don't like the idea of adding a cli option that would be used only for one selftest.
Quoted 8 lines
tools/testing/selftests/bpf/prog_tests/resolve_btfids.c +0
-0if (CHECK (libbpf_get_error (btf),"resolve" ,"Failed to load BTF from btf_data.bpf.o\n" ))return -1 ;Thanks,
DonglinQuoted 1 line
Quoted 5 lines
Quoted 3 lines
This change is not strictly necessary in this series, but it is for the BTF sorting series. Let me know if you would like to take this on, so we don't do the same work twice.
Thanks, I will take it on.
Thank you. I think that'll be a patch in the BTF sorting series. You can work on top of this (v2) series for now. The feedback so far has been mostly nits, and I don't expect overall approach to change in v3.
Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 12 lines
Thanks,
DonglinQuoted 8 lines
tools/testing/selftests/bpf/Makefile +2
-1+ fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool --2.52.0
Eduard Zingerman
maintainerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
In reply to Ihor SolodraiQuoted 4 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.
Quoted 8 lines
This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}
Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?
Quoted 8 lines
The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
Nit: more context needed, as is the statement raises questions but not
answers them.
Quoted 42 lines
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped inlink-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows to deleteelf_update()call and satelite code (likecompressed_section_fix[4]).Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in
sets_patch()[5], and thenElf_Typewas modified beforeelf_update()to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.
Quoted 3 lines
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
Nit: these links are moving target, should refer to a commit or a tag.
Quoted 4 lines
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>
Quoted 9 lines
--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 317 insertions(+), 121 deletions(-) create mode 100755 scripts/gen-btf.sh
Since resolve_btfids is now responsible for distilled base generation, does Makefile.btf need modification to remove "--btf_features=distilled_base"?
Quoted 250 lines
MAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependenciesscripts/gen-btf.sh new file +167
-0@@ -0,0 +1,167 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates. +# +# This script generates BTF data for the provided ELF file. +# +# Kernel BTF generation involves these conceptual steps: +# 1. pahole generates BTF from DWARF data +# 2. resolve_btfids applies kernel-specific btf2btf +# transformations and computes data for .BTF_ids section +# 3. the result gets linked/objcopied into the target binary +# +# How step (3) should be done differs between vmlinux, and +# kernel modules, which is the primary reason for the existence +# of this script. +# +# For modules the script expects vmlinux passed in as --btf_base. +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded +# into the input ELF file with objcopy. +# +# For vmlinux the input file remains unchanged and two files are produced: +# - ${1}.btf.o ready for linking into vmlinux +# - ${1}.btf_ids with .BTF_ids data blob +# This output is consumed by scripts/link-vmlinux.sh + +set -e + +usage() +{ + echo "Usage: $0 [--btf_base <file>] <target ELF file>" + exit 1 +} + +BTF_BASE="" + +while [ $# -gt 0 ]; do + case "$1" in + --btf_base) + BTF_BASE="$2" + shift 2 + ;; + -*) + echo "Unknown option: $1" >&2 + usage + ;; + *) + break + ;; + esac +done + +if [ $# -ne 1 ]; then + usage +fi + +ELF_FILE="$1" +shift + +is_enabled() { + grep -q "^$1=y" ${objtree}/include/config/auto.conf +} + +info() +{ + printf " %-7s %s\n" "${1}" "${2}" +} + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings " + fi + if [ -n "${KBUILD_VERBOSE}" ]; then + RESOLVE_BTFIDS_OPTS+=" -v " + fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} + +gen_btf_o() +{ + local btf_data=${ELF_FILE}.btf.o + + # Create ${btf_data} which contains just .BTF section but no symbols. Add + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all + # deletes all symbols including __start_BTF and __stop_BTF, which will + # be redefined in the linker script. + info OBJCOPY "${btf_data}" + echo "" | ${CC} -c -x c -o ${btf_data} - + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \ + --set-section-flags .BTF=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + + # Change e_type to ET_REL so that it can be used to link final vmlinux. + # GNU ld 2.35+ and lld do not allow an ET_EXEC input. + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + et_rel='\0\1' + else + et_rel='\1\0' + fi + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none +} + +embed_btf_data() +{ + info OBJCOPY "${ELF_FILE}" + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE} + + # a module might not have a .BTF_ids or .BTF.base section + local btf_base="${ELF_FILE}.distilled_base.btf" + if [ -f "${btf_base}" ]; then + ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE} + fi + local btf_ids="${ELF_FILE}.btf_ids" + if [ -f "${btf_ids}" ]; then + ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE} + fi +} + +cleanup() +{ + rm -f "${ELF_FILE}.btf.1" + rm -f "${ELF_FILE}.btf" + if [ "${BTFGEN_MODE}" = "module" ]; then + rm -f "${ELF_FILE}.distilled_base.btf" + rm -f "${ELF_FILE}.btf_ids" + fi +} +trap cleanup EXIT + +BTFGEN_MODE="vmlinux" +if [ -n "${BTF_BASE}" ]; then + BTFGEN_MODE="module" +fi + +gen_btf_data + +case "${BTFGEN_MODE}" in +vmlinux) + gen_btf_o + ;; +module) + embed_btf_data + ;; +esac + +exit 0scripts/link-vmlinux.sh +1
-28@@ -105,34 +105,6 @@ vmlinux_link() ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} } -# generate .BTF typeinfo from DWARF debuginfo -# ${1} - vmlinux image -gen_btf() -{ - local btf_data=${1}.btf.o - - info BTF "${btf_data}" - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1} - - # Create ${btf_data} which contains just .BTF section but no symbols. Add - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all - # deletes all symbols including __start_BTF and __stop_BTF, which will - # be redefined in the linker script. Add 2>/dev/null to suppress GNU - # objcopy warnings: "empty loadable segment detected at ..." - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \ - --strip-all ${1} "${btf_data}" 2>/dev/null - # Change e_type to ET_REL so that it can be used to link final vmlinux. - # GNU ld 2.35+ and lld do not allow an ET_EXEC input. - if is_enabled CONFIG_CPU_BIG_ENDIAN; then - et_rel='\0\1' - else - et_rel='\1\0' - fi - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none - - btf_vmlinux_bin_o=${btf_data} -} - # Create ${2}.o file with all symbols from the ${1} object file kallsyms() {@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi btf_vmlinux_bin_o= +btfids_vmlinux= kallsymso= strip_debug= generate_map=
scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200
Quoted 5 lines
scripts/link-vmlinux.sh +1
-1@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then fi if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; then
Nit: maybe pass output file names as parameters for get-btf.sh?
Quoted 14 lines
scripts/link-vmlinux.sh +2
-1echo >&2 "Failed to generate BTF for vmlinux" echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF" exit 1 fi + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o + btfids_vmlinux=.tmp_vmlinux1.btf_ids fi if is_enabled CONFIG_KALLSYMS; then@@ -281,14 +256,9 @@ fi vmlinux_link "${VMLINUX}" -# fill in BTF IDs
scripts/link-vmlinux.sh fi · line 262
Quoted 10 lines
scripts/link-vmlinux.sh +2
-6if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS "${VMLINUX}" - RESOLVE_BTFIDS_ARGS="" - if is_enabled CONFIG_WERROR; then - RESOLVE_BTFIDS_ARGS=" --fatal_warnings " - fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" + info OBJCOPY ${btfids_vmlinux} + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX} fi
Nit: Maybe do this in get-btf.sh as for modules?
To avoid checking for CONFIG_DEBUG_INFO_BTF in two places.
Quoted 132 lines
scripts/link-vmlinux.sh +0
-0mksysmap "${VMLINUX}" System.maptools/bpf/resolve_btfids/main.c +28
-45@@ -71,9 +71,11 @@#include <fcntl.h> #include <errno.h> #include <linux/btf_ids.h> +#include <linux/kallsyms.h> #include <linux/rbtree.h> #include <linux/zalloc.h> #include <linux/err.h> +#include <linux/limits.h> #include <bpf/btf.h> #include <bpf/libbpf.h> #include <subcmd/parse-options.h> @@ -124,6 +126,7 @@ struct object {struct btf *btf;struct btf *base_btf; +bool distilled_base;struct {int fd;@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return btf_id; } -/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */ -#ifndef SHF_COMPRESSED -#define SHF_COMPRESSED (1 << 11)/* Section with compressed data. */ -#endif - -/* -* The data of compressed section should be aligned to 4 -* (for 32bit) or 8 (for 64 bit) bytes. The binutils ld -* sets sh_addralign to 1, which makes libelf fail with -* misaligned section error during the update: -* FAILED elf_update(WRITE): invalid section alignment -* -* While waiting for ld fix, we fix the compressed sections -* sh_addralign value manualy. -*/ -static int compressed_section_fix (Elf *elf,Elf_Scn *scn,GElf_Shdr *sh) +static void bswap_32_data (void *data,u32 nr_bytes) { -int expected =gelf_getclass (elf) ==ELFCLASS32 ?4 :8 ; +u32 cnt, i; +u32 *ptr; -if (!(sh->sh_flags &SHF_COMPRESSED )) -return 0 ; + cnt = nr_bytes /sizeof (u32); + ptr = data; -if (sh->sh_addralign == expected) -return 0 ; - -pr_debug2 (" - fixing wrong alignment sh_addralign %u, expected %u\n" , - sh->sh_addralign, expected); - - sh->sh_addralign = expected; - -if (gelf_update_shdr (scn, sh) ==0 ) { -pr_err ("FAILED cannot update section header: %s\n" , -elf_errmsg (-1 )); -return -1 ; - } -return 0 ; +for (i =0 ; i < cnt; i++) + ptr[i] =bswap_32 (ptr[i]); }static int elf_collect (struct object *obj)@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)elf_version (EV_CURRENT ); - elf =elf_begin (fd,ELF_C_RDWR_MMAP ,NULL ); + elf =elf_begin (fd,ELF_C_READ_MMAP_PRIVATE ,NULL );if (!elf) {close (fd); pr_err("FAILED cannot create ELF descriptor: %s\n",@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj) obj->efile.symbols_shndx = idx; obj->efile.strtabidx = sh.sh_link; }else if (!strcmp (name,BTF_IDS_SECTION )) { +/* +* If target endianness differs from host, we need to bswap32 +* the .BTF_ids section data on load, because .BTF_ids has +* Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in +* the target endiannes. We repeat this on dump. +*/ +if (obj->efile.encoding !=ELFDATANATIVE ) { +pr_debug ("bswap_32 .BTF_ids data from target to host endianness\n" ); +bswap_32_data (data->d_buf, data->d_size); + } obj->efile.idlist = data; obj->efile.idlist_shndx = idx; obj->efile.idlist_addr = sh.sh_addr; - }else if (!strcmp (name,BTF_BASE_ELF_SEC )) { -/* If a .BTF.base section is found, do not resolve -* BTF ids relative to vmlinux; resolve relative -* to the .BTF.base section instead. btf__parse_split() -* will take care of this once the base BTF it is -* passed is NULL. -*/ - obj->base_btf_path =NULL ; } - -if (compressed_section_fix (elf, scn, &sh)) -return -1 ; }return 0 ;@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)return 0 ; } +static inline bool is_envvar_set (const char *var_name) +{ +const char *value =getenv (var_name); + +return value && value[0 ] !='\0' ; +} +static int load_btf (struct object *obj) {struct btf *base_btf =NULL , *btf =NULL ;
tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557
Quoted 4 lines
tools/bpf/resolve_btfids/main.c +1
-0@@ -571,6 +554,20 @@ static int load_btf(struct object *obj) obj->base_btf = base_btf; obj->btf = btf; +if (obj->base_btf &&is_envvar_set ("KBUILD_EXTMOD" )) {
This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?
Quoted 182 lines
tools/bpf/resolve_btfids/main.c +89
-35+ err =btf__distill_base (obj->btf, &base_btf, &btf); +if (err) { +pr_err ("FAILED to distill base BTF: %s\n" ,strerror (errno)); + goto out_err; + } + +btf__free (obj->btf); +btf__free (obj->base_btf); + obj->btf = btf; + obj->base_btf = base_btf; + obj->distilled_base = true; + } +return 0 ; out_err:@@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)*/ BUILD_BUG_ON ((u32 *)set8->pairs != &set8->pairs[0 ].id);qsort (set8->pairs, set8->cnt,sizeof (set8->pairs[0 ]), cmp_id); - -/* -* When ELF endianness does not match endianness of the -* host, libelf will do the translation when updating -* the ELF. This, however, corrupts SET8 flags which are -* already in the target endianness. So, let's bswap -* them to the host endianness and libelf will then -* correctly translate everything. -*/ -if (obj->efile.encoding !=ELFDATANATIVE ) { -int i; - - set8->flags =bswap_32 (set8->flags); -for (i =0 ; i < set8->cnt; i++) { - set8->pairs[i].flags = -bswap_32 (set8->pairs[i].flags); - } - }break ;case BTF_ID_KIND_SYM :default :@@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)static int symbols_patch (struct object *obj) { -off_t err; - if (__symbols_patch(obj, &obj->structs) || __symbols_patch(obj, &obj->unions) || __symbols_patch(obj, &obj->typedefs) ||@@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)if (sets_patch (obj))return -1 ; -/* Set type to ensure endian translation occurs. */ - obj->efile.idlist->d_type =ELF_T_WORD ; +return 0 ; +} -elf_flagdata (obj->efile.idlist,ELF_C_SET ,ELF_F_DIRTY ); +static int dump_raw_data (const char *out_path,const void *data,u32 size) +{ +int fd, ret; - err =elf_update (obj->efile.elf,ELF_C_WRITE ); -if (err <0 ) { -pr_err ("FAILED elf_update(WRITE): %s\n" , -elf_errmsg (-1 )); + fd =open (out_path,O_WRONLY |O_CREAT |O_TRUNC ,0640 ); +if (fd <0 ) { +pr_err ("Couldn't open %s for writing\n" , out_path); +return fd; + } + + ret =write (fd, data, size); +if (ret <0 || ret != size) { +pr_err ("Failed to write data to %s\n" , out_path); +close (fd); +unlink (out_path); +return -1 ; + } + +close (fd); +pr_debug ("Dumped %lu bytes of data to %s\n" , size, out_path); + +return 0 ; +} + +static int dump_raw_btf_ids (struct object *obj,const char *out_path) +{ +Elf_Data *data = obj->efile.idlist; +int fd, err; + +if (!data || !data->d_buf) { +pr_debug ("%s has no BTF_ids data to dump\n" , obj->path); +return 0 ; + } + +/* +* If target endianness differs from host, we need to bswap32 the +* .BTF_ids section data before dumping so that the output is in +* target endianness. +*/ +if (obj->efile.encoding !=ELFDATANATIVE ) { +pr_debug ("bswap_32 .BTF_ids data from host to target endianness\n" ); +bswap_32_data (data->d_buf, data->d_size); + } + + err =dump_raw_data (out_path, data->d_buf, data->d_size); +if (err) +return -1 ; + +return 0 ; +} + +static int dump_raw_btf (struct btf *btf,const char *out_path) +{ +const void *raw_btf_data; +u32 raw_btf_size; +int fd, err; + + raw_btf_data =btf__raw_data (btf, &raw_btf_size); +if (raw_btf_data ==NULL ) { +pr_err ("btf__raw_data() failed\n" ); +return -1 ; } -pr_debug ("update %s for %s\n" , - err >=0 ?"ok" :"failed" , obj->path); -return err <0 ?-1 :0 ; + err =dump_raw_data (out_path, raw_btf_data, raw_btf_size); +if (err) +return -1 ; + +return 0 ; }static const char *const resolve_btfids_usage[] = {@@ -824,6 +858,7 @@ int main(int argc, const char **argv) .funcs = RB_ROOT, .sets = RB_ROOT, }; +char out_path[PATH_MAX ];bool fatal_warnings = false;struct option btfid_options[] = { OPT_INCR('v', "verbose", &verbose,@@ -836,7 +871,7 @@ int main(int argc, const char **argv) "turn warnings into errors"), OPT_END() }; -int err =-1 ; +int err =-1 , path_len; argc =parse_options (argc, argv, btfid_options, resolve_btfids_usage,PARSE_OPT_STOP_AT_NON_OPTION );@@ -844,6 +879,11 @@ int main(int argc, const char **argv)usage_with_options (resolve_btfids_usage, btfid_options); obj.path = argv[0 ]; +strcpy (out_path, obj.path); + path_len =strlen (out_path); + +if (load_btf (&obj)) + goto out;if (elf_collect (&obj)) goto out;@@ -854,23 +894,37 @@ int main(int argc, const char **argv)*/ if (obj.efile.idlist_shndx ==-1 || obj.efile.symbols_shndx ==-1 ) { -pr_debug ("Cannot find .BTF_ids or symbols sections, nothing to do\n" ); - err =0 ; - goto out; +pr_debug ("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n" ); + goto dump_btf; }if (symbols_collect (&obj)) goto out; -if (load_btf (&obj))
tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 911
Quoted 10 lines
tools/bpf/resolve_btfids/main.c +2
-2- goto out; -if (symbols_resolve (&obj)) goto out;if (symbols_patch (&obj)) goto out; + out_path[path_len] ='\0' ; +strcat (out_path,".btf_ids" );
Nit: here and below use snprintf() to check for 'out_path' overflow.
Quoted 42 lines
tools/bpf/resolve_btfids/main.c +16
-0+if (dump_raw_btf_ids (&obj, out_path)) + goto out; + +dump_btf: + out_path[path_len] ='\0' ; +strcat (out_path,".btf" ); +if (dump_raw_btf (obj.btf, out_path)) + goto out; + +if (obj.base_btf && obj.distilled_base) { + out_path[path_len] ='\0' ; +strcat (out_path,".distilled_base.btf" ); +if (dump_raw_btf (obj.base_btf, out_path)) + goto out; + } +if (!(fatal_warnings && warnings)) err =0 ; out:tools/testing/selftests/bpf/Makefile +5
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \ + fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 5 lines · Eduard Zingerman
Quoted 4 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.
Hi Eduard, thank you for the review.
Quoted 11 lines
Quoted 8 lines
This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?
Sure, makes sense.
Quoted 5 lines
Quoted 5 lines
The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.
Quoted 7 lines
Quoted 4 lines
The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
Nit: more context needed, as is the statement raises questions but not
answers them.
Would you like to see more details about why using libelf is complicated? I don't follow what's unclear here, sorry...
Quoted 43 lines
Quoted 43 lines
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped inlink-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows to deleteelf_update()call and satelite code (likecompressed_section_fix[4]).Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in
sets_patch()[5], and thenElf_Typewas modified beforeelf_update()to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.
Quoted 5 lines
Quoted 3 lines
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285Nit: these links are moving target, should refer to a commit or a tag.
Acked
Quoted 15 lines
Quoted 5 lines
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>Quoted 9 lines
--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 317 insertions(+), 121 deletions(-) create mode 100755 scripts/gen-btf.sh
Quoted 2 lines
Since
resolve_btfidsis now responsible for distilled base generation, does Makefile.btf need modification to remove "--btf_features=distilled_base"?
Yes, good catch.
Quoted 253 lines
Quoted 253 lines
MAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependenciesscripts/gen-btf.sh new file +167
-0@@ -0,0 +1,167 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates. +# +# This script generates BTF data for the provided ELF file. +# +# Kernel BTF generation involves these conceptual steps: +# 1. pahole generates BTF from DWARF data +# 2. resolve_btfids applies kernel-specific btf2btf +# transformations and computes data for .BTF_ids section +# 3. the result gets linked/objcopied into the target binary +# +# How step (3) should be done differs between vmlinux, and +# kernel modules, which is the primary reason for the existence +# of this script. +# +# For modules the script expects vmlinux passed in as --btf_base. +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded +# into the input ELF file with objcopy. +# +# For vmlinux the input file remains unchanged and two files are produced: +# - ${1}.btf.o ready for linking into vmlinux +# - ${1}.btf_ids with .BTF_ids data blob +# This output is consumed by scripts/link-vmlinux.sh + +set -e + +usage() +{ + echo "Usage: $0 [--btf_base <file>] <target ELF file>" + exit 1 +} + +BTF_BASE="" + +while [ $# -gt 0 ]; do + case "$1" in + --btf_base) + BTF_BASE="$2" + shift 2 + ;; + -*) + echo "Unknown option: $1" >&2 + usage + ;; + *) + break + ;; + esac +done + +if [ $# -ne 1 ]; then + usage +fi + +ELF_FILE="$1" +shift + +is_enabled() { + grep -q "^$1=y" ${objtree}/include/config/auto.conf +} + +info() +{ + printf " %-7s %s\n" "${1}" "${2}" +} + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings " + fi + if [ -n "${KBUILD_VERBOSE}" ]; then + RESOLVE_BTFIDS_OPTS+=" -v " + fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} + +gen_btf_o() +{ + local btf_data=${ELF_FILE}.btf.o + + # Create ${btf_data} which contains just .BTF section but no symbols. Add + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all + # deletes all symbols including __start_BTF and __stop_BTF, which will + # be redefined in the linker script. + info OBJCOPY "${btf_data}" + echo "" | ${CC} -c -x c -o ${btf_data} - + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \ + --set-section-flags .BTF=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + + # Change e_type to ET_REL so that it can be used to link final vmlinux. + # GNU ld 2.35+ and lld do not allow an ET_EXEC input. + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + et_rel='\0\1' + else + et_rel='\1\0' + fi + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none +} + +embed_btf_data() +{ + info OBJCOPY "${ELF_FILE}" + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE} + + # a module might not have a .BTF_ids or .BTF.base section + local btf_base="${ELF_FILE}.distilled_base.btf" + if [ -f "${btf_base}" ]; then + ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE} + fi + local btf_ids="${ELF_FILE}.btf_ids" + if [ -f "${btf_ids}" ]; then + ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE} + fi +} + +cleanup() +{ + rm -f "${ELF_FILE}.btf.1" + rm -f "${ELF_FILE}.btf" + if [ "${BTFGEN_MODE}" = "module" ]; then + rm -f "${ELF_FILE}.distilled_base.btf" + rm -f "${ELF_FILE}.btf_ids" + fi +} +trap cleanup EXIT + +BTFGEN_MODE="vmlinux" +if [ -n "${BTF_BASE}" ]; then + BTFGEN_MODE="module" +fi + +gen_btf_data + +case "${BTFGEN_MODE}" in +vmlinux) + gen_btf_o + ;; +module) + embed_btf_data + ;; +esac + +exit 0scripts/link-vmlinux.sh +1
-28@@ -105,34 +105,6 @@ vmlinux_link() ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} } -# generate .BTF typeinfo from DWARF debuginfo -# ${1} - vmlinux image -gen_btf() -{ - local btf_data=${1}.btf.o - - info BTF "${btf_data}" - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1} - - # Create ${btf_data} which contains just .BTF section but no symbols. Add - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all - # deletes all symbols including __start_BTF and __stop_BTF, which will - # be redefined in the linker script. Add 2>/dev/null to suppress GNU - # objcopy warnings: "empty loadable segment detected at ..." - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \ - --strip-all ${1} "${btf_data}" 2>/dev/null - # Change e_type to ET_REL so that it can be used to link final vmlinux. - # GNU ld 2.35+ and lld do not allow an ET_EXEC input. - if is_enabled CONFIG_CPU_BIG_ENDIAN; then - et_rel='\0\1' - else - et_rel='\1\0' - fi - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none - - btf_vmlinux_bin_o=${btf_data} -} - # Create ${2}.o file with all symbols from the ${1} object file kallsyms() {@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi btf_vmlinux_bin_o= +btfids_vmlinux= kallsymso= strip_debug= generate_map=@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then fi
Quoted 5 lines
scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200
Quoted 3 lines
scripts/link-vmlinux.sh +1
-1if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; thenNit: maybe pass output file names as parameters for
get-btf.sh?
I don't see a point in that. The script and callsite will become more complicated, but what is the benefit?
Quoted 14 lines
Quoted 14 lines
scripts/link-vmlinux.sh +2
-0echo >&2 "Failed to generate BTF for vmlinux" echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF" exit 1 fi + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o + btfids_vmlinux=.tmp_vmlinux1.btf_ids fi if is_enabled CONFIG_KALLSYMS; then@@ -281,14 +256,9 @@ fi vmlinux_link "${VMLINUX}"
Quoted 14 lines
scripts/link-vmlinux.sh fi · line 262
Quoted 11 lines
scripts/link-vmlinux.sh +2
-7-# fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS "${VMLINUX}" - RESOLVE_BTFIDS_ARGS="" - if is_enabled CONFIG_WERROR; then - RESOLVE_BTFIDS_ARGS=" --fatal_warnings " - fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" + info OBJCOPY ${btfids_vmlinux} + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX} fiNit: Maybe do this in
get-btf.shas for modules?
To avoid checking forCONFIG_DEBUG_INFO_BTFin two places.
IIRC we can't do that, because we have to update .BTF_ids after all the linking steps. I'll double check.
Quoted 136 lines
Quoted 136 lines
scripts/link-vmlinux.sh +0
-0mksysmap "${VMLINUX}" System.maptools/bpf/resolve_btfids/main.c +28
-45@@ -71,9 +71,11 @@#include <fcntl.h> #include <errno.h> #include <linux/btf_ids.h> +#include <linux/kallsyms.h> #include <linux/rbtree.h> #include <linux/zalloc.h> #include <linux/err.h> +#include <linux/limits.h> #include <bpf/btf.h> #include <bpf/libbpf.h> #include <subcmd/parse-options.h> @@ -124,6 +126,7 @@ struct object {struct btf *btf;struct btf *base_btf; +bool distilled_base;struct {int fd;@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return btf_id; } -/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */ -#ifndef SHF_COMPRESSED -#define SHF_COMPRESSED (1 << 11)/* Section with compressed data. */ -#endif - -/* -* The data of compressed section should be aligned to 4 -* (for 32bit) or 8 (for 64 bit) bytes. The binutils ld -* sets sh_addralign to 1, which makes libelf fail with -* misaligned section error during the update: -* FAILED elf_update(WRITE): invalid section alignment -* -* While waiting for ld fix, we fix the compressed sections -* sh_addralign value manualy. -*/ -static int compressed_section_fix (Elf *elf,Elf_Scn *scn,GElf_Shdr *sh) +static void bswap_32_data (void *data,u32 nr_bytes) { -int expected =gelf_getclass (elf) ==ELFCLASS32 ?4 :8 ; +u32 cnt, i; +u32 *ptr; -if (!(sh->sh_flags &SHF_COMPRESSED )) -return 0 ; + cnt = nr_bytes /sizeof (u32); + ptr = data; -if (sh->sh_addralign == expected) -return 0 ; - -pr_debug2 (" - fixing wrong alignment sh_addralign %u, expected %u\n" , - sh->sh_addralign, expected); - - sh->sh_addralign = expected; - -if (gelf_update_shdr (scn, sh) ==0 ) { -pr_err ("FAILED cannot update section header: %s\n" , -elf_errmsg (-1 )); -return -1 ; - } -return 0 ; +for (i =0 ; i < cnt; i++) + ptr[i] =bswap_32 (ptr[i]); }static int elf_collect (struct object *obj)@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)elf_version (EV_CURRENT ); - elf =elf_begin (fd,ELF_C_RDWR_MMAP ,NULL ); + elf =elf_begin (fd,ELF_C_READ_MMAP_PRIVATE ,NULL );if (!elf) {close (fd); pr_err("FAILED cannot create ELF descriptor: %s\n",@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj) obj->efile.symbols_shndx = idx; obj->efile.strtabidx = sh.sh_link; }else if (!strcmp (name,BTF_IDS_SECTION )) { +/* +* If target endianness differs from host, we need to bswap32 +* the .BTF_ids section data on load, because .BTF_ids has +* Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in +* the target endiannes. We repeat this on dump. +*/ +if (obj->efile.encoding !=ELFDATANATIVE ) { +pr_debug ("bswap_32 .BTF_ids data from target to host endianness\n" ); +bswap_32_data (data->d_buf, data->d_size); + } obj->efile.idlist = data; obj->efile.idlist_shndx = idx; obj->efile.idlist_addr = sh.sh_addr; - }else if (!strcmp (name,BTF_BASE_ELF_SEC )) { -/* If a .BTF.base section is found, do not resolve -* BTF ids relative to vmlinux; resolve relative -* to the .BTF.base section instead. btf__parse_split() -* will take care of this once the base BTF it is -* passed is NULL. -*/ - obj->base_btf_path =NULL ; } - -if (compressed_section_fix (elf, scn, &sh)) -return -1 ; }return 0 ;@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)return 0 ; } +static inline bool is_envvar_set (const char *var_name) +{ +const char *value =getenv (var_name); + +return value && value[0 ] !='\0' ; +} +static int load_btf (struct object *obj) {struct btf *base_btf =NULL , *btf =NULL ;@@ -571,6 +554,20 @@ static int load_btf(struct object *obj) obj->base_btf = base_btf; obj->btf = btf;
Quoted 4 lines
tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557
Quoted 1 line
tools/bpf/resolve_btfids/main.c +1
-0+if (obj->base_btf &&is_envvar_set ("KBUILD_EXTMOD" )) {This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?
Disagree. I intentionally tried to avoid adding options to resolve_btfids, because it's not intendend for general CLI usage (as opposed to pahole, for example). IMO the interface should be as simple as possible.
If we add an option, we still have to check for the env variable somewhere, and then pass the argument through. Why? Just checking an env var when it matters is simpler.
I don't think we want or expect resolve_btfids to run outside of the kernel or selftests build.
Quoted 188 lines
Quoted 188 lines
tools/bpf/resolve_btfids/main.c +89
-37+ err =btf__distill_base (obj->btf, &base_btf, &btf); +if (err) { +pr_err ("FAILED to distill base BTF: %s\n" ,strerror (errno)); + goto out_err; + } + +btf__free (obj->btf); +btf__free (obj->base_btf); + obj->btf = btf; + obj->base_btf = base_btf; + obj->distilled_base = true; + } +return 0 ; out_err:@@ -744,24 +741,6 @@ static int sets_patch(struct object *obj)*/ BUILD_BUG_ON ((u32 *)set8->pairs != &set8->pairs[0 ].id);qsort (set8->pairs, set8->cnt,sizeof (set8->pairs[0 ]), cmp_id); - -/* -* When ELF endianness does not match endianness of the -* host, libelf will do the translation when updating -* the ELF. This, however, corrupts SET8 flags which are -* already in the target endianness. So, let's bswap -* them to the host endianness and libelf will then -* correctly translate everything. -*/ -if (obj->efile.encoding !=ELFDATANATIVE ) { -int i; - - set8->flags =bswap_32 (set8->flags); -for (i =0 ; i < set8->cnt; i++) { - set8->pairs[i].flags = -bswap_32 (set8->pairs[i].flags); - } - }break ;case BTF_ID_KIND_SYM :default :@@ -778,8 +757,6 @@ static int sets_patch(struct object *obj)static int symbols_patch (struct object *obj) { -off_t err; - if (__symbols_patch(obj, &obj->structs) || __symbols_patch(obj, &obj->unions) || __symbols_patch(obj, &obj->typedefs) ||@@ -790,20 +767,77 @@ static int symbols_patch(struct object *obj)if (sets_patch (obj))return -1 ; -/* Set type to ensure endian translation occurs. */ - obj->efile.idlist->d_type =ELF_T_WORD ; +return 0 ; +} -elf_flagdata (obj->efile.idlist,ELF_C_SET ,ELF_F_DIRTY ); +static int dump_raw_data (const char *out_path,const void *data,u32 size) +{ +int fd, ret; - err =elf_update (obj->efile.elf,ELF_C_WRITE ); -if (err <0 ) { -pr_err ("FAILED elf_update(WRITE): %s\n" , -elf_errmsg (-1 )); + fd =open (out_path,O_WRONLY |O_CREAT |O_TRUNC ,0640 ); +if (fd <0 ) { +pr_err ("Couldn't open %s for writing\n" , out_path); +return fd; + } + + ret =write (fd, data, size); +if (ret <0 || ret != size) { +pr_err ("Failed to write data to %s\n" , out_path); +close (fd); +unlink (out_path); +return -1 ; + } + +close (fd); +pr_debug ("Dumped %lu bytes of data to %s\n" , size, out_path); + +return 0 ; +} + +static int dump_raw_btf_ids (struct object *obj,const char *out_path) +{ +Elf_Data *data = obj->efile.idlist; +int fd, err; + +if (!data || !data->d_buf) { +pr_debug ("%s has no BTF_ids data to dump\n" , obj->path); +return 0 ; + } + +/* +* If target endianness differs from host, we need to bswap32 the +* .BTF_ids section data before dumping so that the output is in +* target endianness. +*/ +if (obj->efile.encoding !=ELFDATANATIVE ) { +pr_debug ("bswap_32 .BTF_ids data from host to target endianness\n" ); +bswap_32_data (data->d_buf, data->d_size); + } + + err =dump_raw_data (out_path, data->d_buf, data->d_size); +if (err) +return -1 ; + +return 0 ; +} + +static int dump_raw_btf (struct btf *btf,const char *out_path) +{ +const void *raw_btf_data; +u32 raw_btf_size; +int fd, err; + + raw_btf_data =btf__raw_data (btf, &raw_btf_size); +if (raw_btf_data ==NULL ) { +pr_err ("btf__raw_data() failed\n" ); +return -1 ; } -pr_debug ("update %s for %s\n" , - err >=0 ?"ok" :"failed" , obj->path); -return err <0 ?-1 :0 ; + err =dump_raw_data (out_path, raw_btf_data, raw_btf_size); +if (err) +return -1 ; + +return 0 ; }static const char *const resolve_btfids_usage[] = {@@ -824,6 +858,7 @@ int main(int argc, const char **argv) .funcs = RB_ROOT, .sets = RB_ROOT, }; +char out_path[PATH_MAX ];bool fatal_warnings = false;struct option btfid_options[] = { OPT_INCR('v', "verbose", &verbose,@@ -836,7 +871,7 @@ int main(int argc, const char **argv) "turn warnings into errors"), OPT_END() }; -int err =-1 ; +int err =-1 , path_len; argc =parse_options (argc, argv, btfid_options, resolve_btfids_usage,PARSE_OPT_STOP_AT_NON_OPTION );@@ -844,6 +879,11 @@ int main(int argc, const char **argv)usage_with_options (resolve_btfids_usage, btfid_options); obj.path = argv[0 ]; +strcpy (out_path, obj.path); + path_len =strlen (out_path); + +if (load_btf (&obj)) + goto out;if (elf_collect (&obj)) goto out;@@ -854,23 +894,37 @@ int main(int argc, const char **argv)*/ if (obj.efile.idlist_shndx ==-1 || obj.efile.symbols_shndx ==-1 ) { -pr_debug ("Cannot find .BTF_ids or symbols sections, nothing to do\n" ); - err =0 ; - goto out; +pr_debug ("Cannot find .BTF_ids or symbols sections, skip symbols resolution\n" ); + goto dump_btf; }if (symbols_collect (&obj)) goto out; -if (load_btf (&obj)) - goto out; -if (symbols_resolve (&obj)) goto out;
Quoted 7 lines
tools/bpf/resolve_btfids/main.c int main(int argc, const char **argv) · line 911
Quoted 5 lines
tools/bpf/resolve_btfids/main.c +2
-0if (symbols_patch (&obj)) goto out; + out_path[path_len] ='\0' ; +strcat (out_path,".btf_ids" );Nit: here and below use
snprintf()to check for 'out_path' overflow.
Thanks for the tip. Will do.
Quoted 43 lines
Quoted 43 lines
tools/bpf/resolve_btfids/main.c +16
-0+if (dump_raw_btf_ids (&obj, out_path)) + goto out; + +dump_btf: + out_path[path_len] ='\0' ; +strcat (out_path,".btf" ); +if (dump_raw_btf (obj.btf, out_path)) + goto out; + +if (obj.base_btf && obj.distilled_base) { + out_path[path_len] ='\0' ; +strcat (out_path,".distilled_base.btf" ); +if (dump_raw_btf (obj.base_btf, out_path)) + goto out; + } +if (!(fatal_warnings && warnings)) err =0 ; out:tools/testing/selftests/bpf/Makefile +5
-0@@ -4,6 +4,7 @@ include ../../../scripts/Makefile.arch include ../../../scripts/Makefile.include CXX ?= $(CROSS_COMPILE)g++ +OBJCOPY ?= $(CROSS_COMPILE)objcopy CURDIR := $(abspath .) TOOLSDIR := $(abspath ../../..)@@ -716,6 +717,10 @@ $(OUTPUT)/$(TRUNNER_BINARY): $(TRUNNER_TEST_OBJS) \ $$(call msg,BINARY,,$$@) $(Q)$$(CC) $$(CFLAGS) $$(filter %.a %.o,$$^) $$(LDLIBS) $$(LLVM_LDLIBS) $$(LDFLAGS) $$(LLVM_LDFLAGS) -o $$@ $(Q)$(RESOLVE_BTFIDS) --btf $(TRUNNER_OUTPUT)/btf_data.bpf.o $$@ + $(Q)if [ -f [email protected]_ids ]; then \ + $(OBJCOPY) --update-section [email protected]_ids $$@; \ + fi + $(Q)rm -f [email protected]_ids [email protected] $(Q)ln -sf $(if $2,..,.)/tools/build/bpftool/$(USE_BOOTSTRAP)bpftool \ $(OUTPUT)/$(if $2,$2/)bpftool
Eduard Zingerman
maintainerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 33 lines · Ihor Solodrai
Quoted 5 lines · Eduard Zingerman
Quoted 4 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.Hi Eduard, thank you for the review.
Quoted 11 lines
Quoted 8 lines
This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?Sure, makes sense.
Quoted 10 lines
Quoted 9 lines
The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
Quoted 5 lines
Quoted 2 lines
Nit: more context needed, as is the statement raises questions but not
answers them.Would you like to see more details about why using libelf is complicated? I don't follow what's unclear here, sorry...
The claim here is: "libelf API is complicated and bug-prone ... in context of the kernel build". This is a very vague wording. The decision to rely on objcopy/linker comes from a specific needs outlined by Andrii in an off-list discussion. It will be good to have this context captured in the commit message, instead of bluntly stating that libelf is bug-prone.
Quoted 322 lines
Quoted 41 lines
Quoted 41 lines
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped inlink-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows to deleteelf_update()call and satelite code (likecompressed_section_fix[4]).Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in
sets_patch()[5], and thenElf_Typewas modified beforeelf_update()to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.Quoted 5 lines
Quoted 3 lines
[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285Nit: these links are moving target, should refer to a commit or a tag.
Acked
Quoted 15 lines
Quoted 5 lines
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>Quoted 9 lines
--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 317 insertions(+), 121 deletions(-) create mode 100755 scripts/gen-btf.shQuoted 2 lines
Since
resolve_btfidsis now responsible for distilled base generation, does Makefile.btf need modification to remove "--btf_features=distilled_base"?Yes, good catch.
Quoted 253 lines
Quoted 253 lines
MAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependenciesscripts/gen-btf.sh new file +167
-0@@ -0,0 +1,167 @@ +#!/bin/bash +# SPDX-License-Identifier: GPL-2.0 +# Copyright (c) 2025 Meta Platforms, Inc. and affiliates. +# +# This script generates BTF data for the provided ELF file. +# +# Kernel BTF generation involves these conceptual steps: +# 1. pahole generates BTF from DWARF data +# 2. resolve_btfids applies kernel-specific btf2btf +# transformations and computes data for .BTF_ids section +# 3. the result gets linked/objcopied into the target binary +# +# How step (3) should be done differs between vmlinux, and +# kernel modules, which is the primary reason for the existence +# of this script. +# +# For modules the script expects vmlinux passed in as --btf_base. +# Generated .BTF, .BTF.base and .BTF_ids sections become embedded +# into the input ELF file with objcopy. +# +# For vmlinux the input file remains unchanged and two files are produced: +# - ${1}.btf.o ready for linking into vmlinux +# - ${1}.btf_ids with .BTF_ids data blob +# This output is consumed by scripts/link-vmlinux.sh + +set -e + +usage() +{ + echo "Usage: $0 [--btf_base <file>] <target ELF file>" + exit 1 +} + +BTF_BASE="" + +while [ $# -gt 0 ]; do + case "$1" in + --btf_base) + BTF_BASE="$2" + shift 2 + ;; + -*) + echo "Unknown option: $1" >&2 + usage + ;; + *) + break + ;; + esac +done + +if [ $# -ne 1 ]; then + usage +fi + +ELF_FILE="$1" +shift + +is_enabled() { + grep -q "^$1=y" ${objtree}/include/config/auto.conf +} + +info() +{ + printf " %-7s %s\n" "${1}" "${2}" +} + +case "${KBUILD_VERBOSE}" in +*1*) + set -x + ;; +esac + +if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \ + "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings " + fi + if [ -n "${KBUILD_VERBOSE}" ]; then + RESOLVE_BTFIDS_OPTS+=" -v " + fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} + +gen_btf_o() +{ + local btf_data=${ELF_FILE}.btf.o + + # Create ${btf_data} which contains just .BTF section but no symbols. Add + # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all + # deletes all symbols including __start_BTF and __stop_BTF, which will + # be redefined in the linker script. + info OBJCOPY "${btf_data}" + echo "" | ${CC} -c -x c -o ${btf_data} - + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf \ + --set-section-flags .BTF=alloc,readonly ${btf_data} + ${OBJCOPY} --only-section=.BTF --strip-all ${btf_data} + + # Change e_type to ET_REL so that it can be used to link final vmlinux. + # GNU ld 2.35+ and lld do not allow an ET_EXEC input. + if is_enabled CONFIG_CPU_BIG_ENDIAN; then + et_rel='\0\1' + else + et_rel='\1\0' + fi + printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none +} + +embed_btf_data() +{ + info OBJCOPY "${ELF_FILE}" + ${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE} + + # a module might not have a .BTF_ids or .BTF.base section + local btf_base="${ELF_FILE}.distilled_base.btf" + if [ -f "${btf_base}" ]; then + ${OBJCOPY} --add-section .BTF.base=${btf_base} ${ELF_FILE} + fi + local btf_ids="${ELF_FILE}.btf_ids" + if [ -f "${btf_ids}" ]; then + ${OBJCOPY} --update-section .BTF_ids=${btf_ids} ${ELF_FILE} + fi +} + +cleanup() +{ + rm -f "${ELF_FILE}.btf.1" + rm -f "${ELF_FILE}.btf" + if [ "${BTFGEN_MODE}" = "module" ]; then + rm -f "${ELF_FILE}.distilled_base.btf" + rm -f "${ELF_FILE}.btf_ids" + fi +} +trap cleanup EXIT + +BTFGEN_MODE="vmlinux" +if [ -n "${BTF_BASE}" ]; then + BTFGEN_MODE="module" +fi + +gen_btf_data + +case "${BTFGEN_MODE}" in +vmlinux) + gen_btf_o + ;; +module) + embed_btf_data + ;; +esac + +exit 0scripts/link-vmlinux.sh +1
-28@@ -105,34 +105,6 @@ vmlinux_link() ${kallsymso} ${btf_vmlinux_bin_o} ${arch_vmlinux_o} ${ldlibs} } -# generate .BTF typeinfo from DWARF debuginfo -# ${1} - vmlinux image -gen_btf() -{ - local btf_data=${1}.btf.o - - info BTF "${btf_data}" - LLVM_OBJCOPY="${OBJCOPY}" ${PAHOLE} -J ${PAHOLE_FLAGS} ${1} - - # Create ${btf_data} which contains just .BTF section but no symbols. Add - # SHF_ALLOC because .BTF will be part of the vmlinux image. --strip-all - # deletes all symbols including __start_BTF and __stop_BTF, which will - # be redefined in the linker script. Add 2>/dev/null to suppress GNU - # objcopy warnings: "empty loadable segment detected at ..." - ${OBJCOPY} --only-section=.BTF --set-section-flags .BTF=alloc,readonly \ - --strip-all ${1} "${btf_data}" 2>/dev/null - # Change e_type to ET_REL so that it can be used to link final vmlinux. - # GNU ld 2.35+ and lld do not allow an ET_EXEC input. - if is_enabled CONFIG_CPU_BIG_ENDIAN; then - et_rel='\0\1' - else - et_rel='\1\0' - fi - printf "${et_rel}" | dd of="${btf_data}" conv=notrunc bs=1 seek=16 status=none - - btf_vmlinux_bin_o=${btf_data} -} - # Create ${2}.o file with all symbols from the ${1} object file kallsyms() {@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi btf_vmlinux_bin_o= +btfids_vmlinux= kallsymso= strip_debug= generate_map=@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then fi
Quoted 8 lines
Quoted 5 lines
scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200
Quoted 3 lines
scripts/link-vmlinux.sh +1
-1if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; thenNit: maybe pass output file names as parameters for
get-btf.sh?I don't see a point in that. The script and callsite will become more complicated, but what is the benefit?
In order to avoid implicit naming conventions. Hence, the reader of the script code has clear understanding about in and out parameters.
Quoted 178 lines
Quoted 14 lines
Quoted 14 lines
scripts/link-vmlinux.sh +2
-0echo >&2 "Failed to generate BTF for vmlinux" echo >&2 "Try to disable CONFIG_DEBUG_INFO_BTF" exit 1 fi + btf_vmlinux_bin_o=.tmp_vmlinux1.btf.o + btfids_vmlinux=.tmp_vmlinux1.btf_ids fi if is_enabled CONFIG_KALLSYMS; then@@ -281,14 +256,9 @@ fi vmlinux_link "${VMLINUX}"Quoted 14 lines
scripts/link-vmlinux.sh fi · line 262
Quoted 11 lines
scripts/link-vmlinux.sh +2
-7-# fill in BTF IDs if is_enabled CONFIG_DEBUG_INFO_BTF; then - info BTFIDS "${VMLINUX}" - RESOLVE_BTFIDS_ARGS="" - if is_enabled CONFIG_WERROR; then - RESOLVE_BTFIDS_ARGS=" --fatal_warnings " - fi - ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_ARGS} "${VMLINUX}" + info OBJCOPY ${btfids_vmlinux} + ${OBJCOPY} --update-section .BTF_ids=${btfids_vmlinux} ${VMLINUX} fiNit: Maybe do this in
get-btf.shas for modules?
To avoid checking forCONFIG_DEBUG_INFO_BTFin two places.IIRC we can't do that, because we have to update .BTF_ids after all the linking steps. I'll double check.
Quoted 136 lines
Quoted 136 lines
scripts/link-vmlinux.sh +0
-0mksysmap "${VMLINUX}" System.maptools/bpf/resolve_btfids/main.c +28
-45@@ -71,9 +71,11 @@#include <fcntl.h> #include <errno.h> #include <linux/btf_ids.h> +#include <linux/kallsyms.h> #include <linux/rbtree.h> #include <linux/zalloc.h> #include <linux/err.h> +#include <linux/limits.h> #include <bpf/btf.h> #include <bpf/libbpf.h> #include <subcmd/parse-options.h> @@ -124,6 +126,7 @@ struct object {struct btf *btf;struct btf *base_btf; +bool distilled_base;struct {int fd;@@ -308,42 +311,16 @@ static struct btf_id *add_symbol(struct rb_root *root, char *name, size_t size)return btf_id; } -/* Older libelf.h and glibc elf.h might not yet define the ELF compression types. */ -#ifndef SHF_COMPRESSED -#define SHF_COMPRESSED (1 << 11)/* Section with compressed data. */ -#endif - -/* -* The data of compressed section should be aligned to 4 -* (for 32bit) or 8 (for 64 bit) bytes. The binutils ld -* sets sh_addralign to 1, which makes libelf fail with -* misaligned section error during the update: -* FAILED elf_update(WRITE): invalid section alignment -* -* While waiting for ld fix, we fix the compressed sections -* sh_addralign value manualy. -*/ -static int compressed_section_fix (Elf *elf,Elf_Scn *scn,GElf_Shdr *sh) +static void bswap_32_data (void *data,u32 nr_bytes) { -int expected =gelf_getclass (elf) ==ELFCLASS32 ?4 :8 ; +u32 cnt, i; +u32 *ptr; -if (!(sh->sh_flags &SHF_COMPRESSED )) -return 0 ; + cnt = nr_bytes /sizeof (u32); + ptr = data; -if (sh->sh_addralign == expected) -return 0 ; - -pr_debug2 (" - fixing wrong alignment sh_addralign %u, expected %u\n" , - sh->sh_addralign, expected); - - sh->sh_addralign = expected; - -if (gelf_update_shdr (scn, sh) ==0 ) { -pr_err ("FAILED cannot update section header: %s\n" , -elf_errmsg (-1 )); -return -1 ; - } -return 0 ; +for (i =0 ; i < cnt; i++) + ptr[i] =bswap_32 (ptr[i]); }static int elf_collect (struct object *obj)@@ -364,7 +341,7 @@ static int elf_collect(struct object *obj)elf_version (EV_CURRENT ); - elf =elf_begin (fd,ELF_C_RDWR_MMAP ,NULL ); + elf =elf_begin (fd,ELF_C_READ_MMAP_PRIVATE ,NULL );if (!elf) {close (fd); pr_err("FAILED cannot create ELF descriptor: %s\n",@@ -427,21 +404,20 @@ static int elf_collect(struct object *obj) obj->efile.symbols_shndx = idx; obj->efile.strtabidx = sh.sh_link; }else if (!strcmp (name,BTF_IDS_SECTION )) { +/* +* If target endianness differs from host, we need to bswap32 +* the .BTF_ids section data on load, because .BTF_ids has +* Elf_Type = ELF_T_BYTE, and so libelf returns data buffer in +* the target endiannes. We repeat this on dump. +*/ +if (obj->efile.encoding !=ELFDATANATIVE ) { +pr_debug ("bswap_32 .BTF_ids data from target to host endianness\n" ); +bswap_32_data (data->d_buf, data->d_size); + } obj->efile.idlist = data; obj->efile.idlist_shndx = idx; obj->efile.idlist_addr = sh.sh_addr; - }else if (!strcmp (name,BTF_BASE_ELF_SEC )) { -/* If a .BTF.base section is found, do not resolve -* BTF ids relative to vmlinux; resolve relative -* to the .BTF.base section instead. btf__parse_split() -* will take care of this once the base BTF it is -* passed is NULL. -*/ - obj->base_btf_path =NULL ; } - -if (compressed_section_fix (elf, scn, &sh)) -return -1 ; }return 0 ;@@ -545,6 +521,13 @@ static int symbols_collect(struct object *obj)return 0 ; } +static inline bool is_envvar_set (const char *var_name) +{ +const char *value =getenv (var_name); + +return value && value[0 ] !='\0' ; +} +static int load_btf (struct object *obj) {struct btf *base_btf =NULL , *btf =NULL ;@@ -571,6 +554,20 @@ static int load_btf(struct object *obj) obj->base_btf = base_btf; obj->btf = btf;Quoted 4 lines
tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557
Quoted 1 line
tools/bpf/resolve_btfids/main.c +1
-0+if (obj->base_btf &&is_envvar_set ("KBUILD_EXTMOD" )) {This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?
Disagree. I intentionally tried to avoid adding options to
resolve_btfids, because it's not intendend for general CLI usage (as opposed to pahole, for example). IMO the interface should be as simple as possible.
Quoted 6 lines
If we add an option, we still have to check for the env variable somewhere, and then pass the argument through. Why? Just checking an env var when it matters is simpler.
I don't think we want or expect
resolve_btfidsto run outside of the kernel or selftests build.
This comes to personal opinion, of-course.
So, in my personal opinion, obfuscating a command line tool interface with it being parameterized by both environment variables and command line parameters is rarely justified. In this particular case it will only make life a tad harder for someone debugging resolve_btfids by copy-pasting command from make output.
Hence, I find this piece of code ugly.
[...]
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 40 lines · Eduard Zingerman
Quoted 33 lines · Ihor Solodrai
Quoted 5 lines · Eduard Zingerman
Quoted 4 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.Hi Eduard, thank you for the review.
Quoted 11 lines
Quoted 8 lines
This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}Nit: use ${1}.BTF / ${1}.BTF.base / ${1}.BTF_ids, so that each file is
named by it's corresponding section?Sure, makes sense.
Quoted 10 lines
Quoted 9 lines
The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
Quoted 5 lines
Quoted 2 lines
Nit: more context needed, as is the statement raises questions but not
answers them.Would you like to see more details about why using libelf is complicated? I don't follow what's unclear here, sorry...
Quoted 6 lines
The claim here is: "libelf API is complicated and bug-prone ... in context of the kernel build". This is a very vague wording. The decision to rely on objcopy/linker comes from a specific needs outlined by Andrii in an off-list discussion. It will be good to have this context captured in the commit message, instead of bluntly stating that libelf is bug-prone.
Ok, it seems you're conflating two separate issues.
There is a requirement to *link* .BTF section into vmlinux, because it must have a SHF_ALLOC flag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.
The patch doesn't change this behavior, and this was (and is) covered in the script comments.
A separate issue is what resolve_btfids does: updates ELF in-place (before the patch) or outputs detached section data (after patch).
The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.
And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes resolve_btfids buggy.
Quoted 22 lines
Quoted 22 lines
Quoted 21 lines
Quoted 14 lines
[...]
# Create ${2}.o file with all symbols from the ${1} object filekallsyms()
{scripts/link-vmlinux.sh +1
-0@@ -204,6 +176,7 @@ if is_enabled CONFIG_ARCH_WANTS_PRE_LINK_VMLINUX; then fi btf_vmlinux_bin_o= +btfids_vmlinux= kallsymso= strip_debug= generate_map=scripts/link-vmlinux.sh if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then · line 200
Quoted 5 lines
scripts/link-vmlinux.sh +1
-1@@ -224,11 +197,13 @@ if is_enabled CONFIG_KALLSYMS || is_enabled CONFIG_DEBUG_INFO_BTF; then fi if is_enabled CONFIG_DEBUG_INFO_BTF; then - if ! gen_btf .tmp_vmlinux1; then + if ! ${srctree}/scripts/gen-btf.sh .tmp_vmlinux1; thenNit: maybe pass output file names as parameters for
get-btf.sh?
Quoted 5 lines
Quoted 2 lines
I don't see a point in that. The script and callsite will become more complicated, but what is the benefit?
In order to avoid implicit naming conventions. Hence, the reader of the script code has clear understanding about in and out parameters.
I think implicit naming convention makes sense for this script. The script's top comment describes what it does in detail, including the output naming.
Quoted 44 lines
Quoted 27 lines
Quoted 17 lines
[...]
Quoted 15 lines
tools/bpf/resolve_btfids/main.c +7
-0+static inline bool is_envvar_set (const char *var_name) +{ +const char *value =getenv (var_name); + +return value && value[0 ] !='\0' ; +} +static int load_btf (struct object *obj) {struct btf *base_btf =NULL , *btf =NULL ;@@ -571,6 +554,20 @@ static int load_btf(struct object *obj) obj->base_btf = base_btf; obj->btf = btf;Quoted 4 lines
tools/bpf/resolve_btfids/main.c static int load_btf(struct object *obj) · line 557
Quoted 1 line
tools/bpf/resolve_btfids/main.c +1
-0+if (obj->base_btf &&is_envvar_set ("KBUILD_EXTMOD" )) {This is a bit ugly, maybe use a dedicated parameter instead of checking environment variable?
Disagree. I intentionally tried to avoid adding options to
resolve_btfids, because it's not intendend for general CLI usage (as opposed to pahole, for example). IMO the interface should be as simple as possible.Quoted 6 lines
If we add an option, we still have to check for the env variable somewhere, and then pass the argument through. Why? Just checking an env var when it matters is simpler.
I don't think we want or expect
resolve_btfidsto run outside of the kernel or selftests build.This comes to personal opinion, of-course.
So, in my personal opinion, obfuscating a command line tool interface with it being parameterized by both environment variables and command line parameters is rarely justified. In this particular case it will only make life a tad harder for someone debuggingresolve_btfidsby copy-pasting command from make output.Hence, I find this piece of code ugly.
[...]
Eduard Zingerman
maintainerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
On Thu, 2025-12-04 at 09:29 -0800, Ihor Solodrai wrote:
[...]
Quoted 12 lines
Ok, it seems you're conflating two separate issues.
There is a requirement to *link* .BTF section into vmlinux, because it must have a
SHF_ALLOCflag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.The patch doesn't change this behavior, and this was (and is) covered in the script comments.
A separate issue is what
resolve_btfidsdoes: updates ELF in-place (before the patch) or outputs detached section data (after patch).
Quoted 6 lines
The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.
And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes
resolve_btfidsbuggy.
So, pahole does the following:
- elf_begin(fildes: fd, cmd: ELF_C_RDWR, ref: NULL);
- selects a section to modify and modifies it
- elf_flagdata(data: btf_data, cmd: ELF_C_SET, flags: ELF_F_DIRTY);
- elf_update(elf, cmd: ELF_C_WRITE)
- elf_end(elf)
What exactly is complicated about that?
[...]
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 23 lines · Eduard Zingerman
On Thu, 2025-12-04 at 09:29 -0800, Ihor Solodrai wrote:
[...]
Quoted 12 lines
Ok, it seems you're conflating two separate issues.
There is a requirement to *link* .BTF section into vmlinux, because it must have a
SHF_ALLOCflag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.The patch doesn't change this behavior, and this was (and is) covered in the script comments.
A separate issue is what
resolve_btfidsdoes: updates ELF in-place (before the patch) or outputs detached section data (after patch).Quoted 6 lines
The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.
And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes
resolve_btfidsbuggy.
Quoted 8 lines
So, pahole does the following:
-elf_begin(fildes: fd, cmd: ELF_C_RDWR, ref: NULL);
- selects a section to modify and modifies it
-elf_flagdata(data: btf_data, cmd: ELF_C_SET, flags: ELF_F_DIRTY);
-elf_update(elf, cmd: ELF_C_WRITE)
-elf_end(elf)What exactly is complicated about that?
Take a look at the resolve_btfids code that is removed in this patch, as a consequence of switching to read-only ELF.
Also consider that before these changes resolve_btfids had a simple
job: update data buffer of a single section, importantly, without changing its size.
Now let's say we keep "update in-place" approach (which I tried to do, btw). In addition to previous .BTF_ids data update, resolve_btfids may need to either add or update .BTF section changing its size (triggering reorg of sections in ELF, depending on the flags) and add .BTF.base section. There is also a question of how to do it: do we elf_update() multiple times or try to "batch" the updates?
All of this is possible, but the alternative is much simpler:
${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}Why re-implement our own incomplete version of objcopy if we can just use it to deal with the details of the ELF update?
Note also that even in pahole "add .BTF section" is implemented via llvm-objcopy call. My guess is: to avoid the headache of figuring out correct libelf usage.
Quoted 2 lines
[...]
Eduard Zingerman
maintainerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 49 lines · Ihor Solodrai
Quoted 23 lines · Eduard Zingerman
On Thu, 2025-12-04 at 09:29 -0800, Ihor Solodrai wrote:
[...]
Quoted 12 lines
Ok, it seems you're conflating two separate issues.
There is a requirement to *link* .BTF section into vmlinux, because it must have a
SHF_ALLOCflag, which makes objcopying the section data
insufficient: linker has to do some magic under the hood.The patch doesn't change this behavior, and this was (and is) covered in the script comments.
A separate issue is what
resolve_btfidsdoes: updates ELF in-place (before the patch) or outputs detached section data (after patch).Quoted 6 lines
The paragraph in the commit message attempted to explain the decision to output raw section data. And apparently I did a bad job of that. I'll rewrite this part it in the next revision.
And I feel I should clarify that I didn't claim that libelf is buggy. I meant that using it is complicated, which makes
resolve_btfidsbuggy.Quoted 8 lines
So, pahole does the following:
-elf_begin(fildes: fd, cmd: ELF_C_RDWR, ref: NULL);
- selects a section to modify and modifies it
-elf_flagdata(data: btf_data, cmd: ELF_C_SET, flags: ELF_F_DIRTY);
-elf_update(elf, cmd: ELF_C_WRITE)
-elf_end(elf)What exactly is complicated about that?
Take a look at the
resolve_btfidscode that is removed in this patch, as a consequence of switching to read-only ELF.Also consider that before these changes
resolve_btfidshad a simple
job: update data buffer of a single section, importantly, without changing its size.Now let's say we keep "update in-place" approach (which I tried to do, btw). In addition to previous .BTF_ids data update,
resolve_btfidsmay need to either add or update .BTF section changing its size (triggering reorg of sections in ELF, depending on the flags) and add .BTF.base section. There is also a question of how to do it: do weelf_update()multiple times or try to "batch" the updates?All of this is possible, but the alternative is much simpler:
Quoted 8 lines
${OBJCOPY} --add-section .BTF=${ELF_FILE}.btf ${ELF_FILE}Why re-implement our own incomplete version of objcopy if we can just use it to deal with the details of the ELF update?
Note also that even in pahole "add .BTF section" is implemented via llvm-objcopy call. My guess is: to avoid the headache of figuring out correct libelf usage.
Please put this motivation in the commit message.
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
In reply to Ihor SolodraiQuoted 107 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped inlink-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows to deleteelf_update()call and satelite code (likecompressed_section_fix[4]).Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in
sets_patch()[5], and thenElf_Typewas modified beforeelf_update()to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 317 insertions(+), 121 deletions(-) create mode 100755 scripts/gen-btf.shMAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi;
[...]
scripts/Makefile.modfinal
Quoted 11 lines
scripts/Makefile.modfinal +11
-0+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \
please double-check what pahole version has --btf_encode_detached, we might need to change minimal supported pahole version because of this
pw-bot: cr
Quoted 15 lines
scripts/Makefile.modfinal +15
-0+ "${ELF_FILE}" + + info BTFIDS "${ELF_FILE}" + RESOLVE_BTFIDS_OPTS="" + if is_enabled CONFIG_WERROR; then + RESOLVE_BTFIDS_OPTS+=" --fatal_warnings " + fi + if [ -n "${KBUILD_VERBOSE}" ]; then + RESOLVE_BTFIDS_OPTS+=" -v " + fi + ${RESOLVE_BTFIDS} ${RESOLVE_BTFIDS_OPTS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf ${btf1} "${ELF_FILE}" +} +
[...]
Quoted 6 lines
scripts/Makefile.modfinal +3
-2+static int dump_raw_data(const char *out_path, const void *data, u32 size) +{ + int fd, ret; - err = elf_update(obj->efile.elf, ELF_C_WRITE); - if (err < 0) {
scripts/Makefile.modfinal
Quoted 10 lines
scripts/Makefile.modfinal +8
-2- pr_err("FAILED elf_update(WRITE): %s\n", - elf_errmsg(-1)); + fd = open(out_path, O_WRONLY | O_CREAT | O_TRUNC, 0640); + if (fd < 0) { + pr_err("Couldn't open %s for writing\n", out_path); + return fd; + } + + ret = write(fd, data, size); + if (ret < 0 || ret != size) {
use fopen() and fwrite() instead of low-level syscalls? for write() it's "expected" that it might be interrupted and not complete a full write, so you'd need to handle that in a loop properly. With fwrite() I think all this is handled internally, so I'd stick to fopen()'s FILE abstraction and fwrite().
Quoted 5 lines
scripts/Makefile.modfinal +5
-0+ pr_err("Failed to write data to %s\n", out_path); + close(fd); + unlink(out_path); + return -1; + }
[...]
scripts/Makefile.modfinal
Quoted 8 lines
scripts/Makefile.modfinal +8
-0+static int dump_raw_btf(struct btf *btf, const char *out_path) +{ + const void *raw_btf_data; + u32 raw_btf_size; + int fd, err; + + raw_btf_data = btf__raw_data(btf, &raw_btf_size); + if (raw_btf_data == NULL) {
nit: !raw_btf_data, it's C
[...]
scripts/Makefile.modfinal int main(int argc, const char **argv) · line 883
Quoted 6 lines
scripts/Makefile.modfinal +2
-0@@ -844,6 +879,11 @@ int main(int argc, const char **argv) usage_with_options(resolve_btfids_usage, btfid_options); obj.path = argv[0]; + strcpy(out_path, obj.path); + path_len = strlen(out_path);
Eduard already suggested using snprintf() later in the code, I'd say use snprintf() here as well instead of strcpy(). When working with fixed-sized buffers, snprintf() is the most ergonomic way to deal with that and not trigger unnecessary compiler warnings about possible truncations, out of buffer writes, and stuff like that.
Quoted 6 lines
scripts/Makefile.modfinal +3
-0+ + if (load_btf(&obj)) + goto out; if (elf_collect(&obj)) goto out;
[...]
Re: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 111 lines · Andrii Nakryiko
Quoted 107 lines · Ihor Solodrai
Currently
resolve_btfidsupdates .BTF_ids section of an ELF file in-place, based on the contents of provided BTF, usually within the same input file, and optionally a BTF base.This patch changes
resolve_btfidsbehavior to enable BTF transformations as part of its main operation. To achieve this in-place ELF write inresolve_btfidsis replaced with generation of the following binaries:
* ${1}.btf with .BTF section data
* ${1}.distilled_base.btf with .BTF.base section data (for out-of-tree modules)
* ${1}.btf_ids with .BTF_ids section data, if it exists in ${1}The execution of
resolve_btfidsand consumption of its output is orchestrated byscripts/gen-btf.shintroduced in this patch.The rationale for this approach is that updating ELF in-place with libelf API is complicated and bug-prone, especially in the context of the kernel build. On the other hand applying objcopy to manipulate ELF sections is simpler and more reliable.
There are two distinct paths for BTF generation and
resolve_btfidsapplication in the kernel build: for vmlinux and for kernel modules.For the vmlinux binary a .BTF section is added in a roundabout way to ensure correct linking (details below). The patch doesn't change this approach, only the implementation is a little different.
Before this patch it worked like follows:
* pahole consumed .tmp_vmlinux1 [1] and added .BTF section with llvm-objcopy [2] to it
* then everything except the .BTF section was stripped from .tmp_vmlinux1 into a .tmp_vmlinux1.bpf.o object [1], later linked into vmlinux
*resolve_btfidswas executed later on vmlinux.unstripped [3], updating it in-placeAfter this patch
gen-btf.shimplements the following:* pahole consumes .tmp_vmlinux1 and produces a *detached* file with raw BTF data
*resolve_btfidsconsumes .tmp_vmlinux1 and detached BTF to produce (potentially modified) .BTF, and .BTF_ids sections data
* a .tmp_vmlinux1.bpf.o object is then produced with objcopy copying BTF output ofresolve_btfids
* .BTF_ids data gets embedded into vmlinux.unstripped inlink-vmlinux.sh by objcopy --update-sectionFor the kernel modules creating special .bpf.o file is not necessary, and so embedding of sections data produced by
resolve_btfidsis straightforward with the objcopy.With this patch an ELF file becomes effectively read-only within
resolve_btfids, which allows to deleteelf_update()call and satelite code (likecompressed_section_fix[4]).Endianness handling of .BTF_ids data is also changed. Previously the "flags" part of the section was bswapped in
sets_patch()[5], and thenElf_Typewas modified beforeelf_update()to signal to libelf that bswap may be necessary. With this patch we explicitly bswap entire data buffer on load and on dump.[1] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n115
[2] https://git.kernel.org/pub/scm/devel/pahole/pahole.git/tree/btf_encoder.c#n1835
[3] https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf.git/tree/scripts/link-vmlinux.sh#n285
[4] https://lore.kernel.org/bpf/[email protected]/
[5] https://lore.kernel.org/bpf/[email protected]/Signed-off-by Ihor Solodrai <[email protected]>--- MAINTAINERS | 1 + scripts/Makefile.modfinal | 5 +- scripts/gen-btf.sh | 167 ++++++++++++++++++++ scripts/link-vmlinux.sh | 42 +----- tools/bpf/resolve_btfids/main.c | 218 +++++++++++++++++---------- tools/testing/selftests/bpf/Makefile | 5 + 6 files changed, 317 insertions(+), 121 deletions(-) create mode 100755 scripts/gen-btf.shMAINTAINERS +1
-0@@ -4672,6 +4672,7 @@ F: net/sched/act_bpf.c F: net/sched/cls_bpf.c F: samples/bpf/ F: scripts/bpf_doc.py +F: scripts/gen-btf.sh F: scripts/Makefile.btf F: scripts/pahole-version.sh F: tools/bpf/scripts/Makefile.modfinal +2
-3@@ -38,9 +38,8 @@ quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ - else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ - $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ + else \ + $(srctree)/scripts/gen-btf.sh --btf_base $(objtree)/vmlinux $@; \ fi;[...]
Quoted 15 lines
scripts/Makefile.modfinal
Quoted 11 lines
scripts/Makefile.modfinal +11
-0+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \please double-check what pahole version has --btf_encode_detached, we might need to change minimal supported pahole version because of this
yeah, this landed in v1.22 [1]
One thing worth thinking about; are there aspects of the gen_btf.sh script that could be moved to Makefile.btf to avoid having to compute them repeatedly for each module? For example computing resolve_btfids flags based on CONFIG_WERROR could be done there I think. You could also determine whether the script is needed at all in Makefile.btf; i.e.
gen-btf-y =
gen-btf-$(CONFIG_DEBUG_INFO_BTF) = scripts/gen-btf.sh
export GEN_BTF := $(gen-btf-y)
That would allow you to get rid of the is_enabled() I think.
I'm building this now, but I was wondering if the linking/objcopy changes pose any risk to kernel address computations in kallsyms or anything like that? IIRC Stephen ran into some issues with global variable addresses as a consequence of linking BTF sections [2], but not sure if there are additional concerns here.
[1] https://github.com/acmel/dwarves/releases/tag/v1.22
[2] https://lore.kernel.org/bpf/[email protected]/
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 21 lines · Alan Maguire
Quoted 4 lines · Andrii Nakryiko
On Thu, Nov 27, 2025 at 10:53 AM Ihor Solodrai <[email protected]> wrote:
[...]
Quoted 15 lines
scripts/gen-btf.sh line 84
Quoted 11 lines
scripts/gen-btf.sh new file +11
-0+if ! is_enabled CONFIG_DEBUG_INFO_BTF; then + exit 0 +fi + +gen_btf_data() +{ + info BTF "${ELF_FILE}" + btf1="${ELF_FILE}.btf.1" + ${PAHOLE} -J ${PAHOLE_FLAGS} \ + ${BTF_BASE:+--btf_base ${BTF_BASE}} \ + --btf_encode_detached=${btf1} \please double-check what pahole version has --btf_encode_detached, we might need to change minimal supported pahole version because of this
Quoted 1 line
yeah, this landed in v1.22 [1]
Thank you for checking!
Quoted 7 lines
One thing worth thinking about; are there aspects of the
gen_btf.shscript that could be moved to Makefile.btf to avoid having to compute them repeatedly for each module? For example computingresolve_btfidsflags based onCONFIG_WERRORcould be done there I think. You could also determine whether the script is needed at all in Makefile.btf; i.e.
Quoted 6 lines
gen-btf-y =
gen-btf-$(CONFIG_DEBUG_INFO_BTF) =scripts/gen-btf.shexport
GEN_BTF:= $(gen-btf-y)That would allow you to get rid of the
is_enabled()I think.
Good point. I'll try moving most relevant flags to Makefile.btf
Quoted 5 lines
I'm building this now, but I was wondering if the linking/objcopy changes pose any risk to kernel address computations in kallsyms or anything like that? IIRC Stephen ran into some issues with global variable addresses as a consequence of linking BTF sections [2], but not sure if there are additional concerns here.
This series doesn't change the fact that .BTF is *linked* into final vmlinux, so a problem described in [2] stands.
That said, AFAIU the suggested change in the linker script will still work.
kernel test robot
build botRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
In reply to Ihor Solodraikernel test robot noticed the following build errors:
Hi Ihor,
kernel test robot noticed the following build errors:
[auto build test ERROR on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Ihor-Solodrai/resolve_btfids-rename-object-btf-field-to-btf_path/20251128-025645 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20251127185242.3954132-5-ihor.solodrai%40linux.dev patch subject: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output config: arm64-randconfig-004-20251205 (https://download.01.org/0day-ci/archive/20251206/[email protected]/config) compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 14bf95b06a18b9b59c89601cbc0e5a6f2176b118) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251206/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> ld.lld: error: .tmp_vmlinux1.btf.o is incompatible with aarch64elf
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Ihor Solodrai
reviewerRe: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output
Quoted 14 lines · kernel test robot
Hi Ihor,
kernel test robot noticed the following build errors:
[auto build test ERROR on bpf-next/master]
url: https://github.com/intel-lab-lkp/linux/commits/Ihor-Solodrai/resolve_btfids-rename-object-btf-field-to-btf_path/20251128-025645 base: https://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-next.git master patch link: https://lore.kernel.org/r/20251127185242.3954132-5-ihor.solodrai%40linux.dev patch subject: [PATCH bpf-next v2 4/4] resolve_btfids: change in-place update with raw binary output config: arm64-randconfig-004-20251205 (https://download.01.org/0day-ci/archive/20251206/[email protected]/config) compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 14bf95b06a18b9b59c89601cbc0e5a6f2176b118) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20251206/[email protected]/reproduce)
Quoted 9 lines
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/All errors (new ones prefixed by >>):
Quoted 1 line
Quoted 1 line
ld.lld: error: .tmp_vmlinux1.btf.o is incompatible with aarch64elf
I was able to reproduce this error.
It happens only when cross-compiling with LLVM (which we don't do on BPF CI, btw) because of this line in gen-btf.sh:
echo "" | ${CC} -c -x c -o ${btf_data} -The purpose of this command is to produce an "empty" linkable ELF file. The .BTF section is objcopied into it, and then it's linked into the vmlinux.
Before the changes in this patch, the "empty" ELF was produced with objcopy --strip-all on .tmp_vmlinux1, which is a slower operation.
ld.lld fails, because ${CC} without the flags is just clang in this case, and it emits an ELF for host arch, which of course can't be linked. It can be fixed with:
echo "" | ${CC} ${CLANG_FLAGS} -c -x c -o ${btf_data} -${CLANG_FLAGS} contains a correct clang --target when cross-compiling.
I'll use this in the upcoming v4 of the series.