btf_encoder: emit type tags for bpf_arena pointers
Subsystems (2)
Ihor Solodrai
reviewerv4 cover btf_encoder: emit type tags for bpf_arena pointers
This patch series implements emitting appropriate BTF type tags for argument and return types of kfuncs marked with KF_ARENA_* flags.
For additional context see the description of BPF patch "bpf: define KF_ARENA_* flags for bpf_arena kfuncs" [1].
The feature depends on recent changes in libbpf [2].
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
v3->v4:
* Add a patch (#2) replacing compile-time libbpf version checks with runtime checks for symbol availablility
* Add a patch (#3) bumping libbpf submodule commit to latest master
* Modify "btf_encoder: emit type tags for bpf_arena pointers"
(#2->#4) to not use compile time libbpf version checks
v2->v3:
* Nits in patch #1
v1->v2:
* Rewrite patch #1 refactoring btf_encoder__tag_kfuncs(): now the post-processing step is removed entirely, and kfuncs are tagged in btf_encoder__add_func().
* Nits and renames in patch #2
* Add patch #4 editing man pages
v2: https://lore.kernel.org/dwarves/[email protected]/
v1: https://lore.kernel.org/dwarves/[email protected]/
Ihor Solodrai (6):
btf_encoder: refactor btf_encoder__tag_kfuncs()
btf_encoder: use __weak declarations of version-dependent libbpf API
pahole: sync with libbpf mainline
btf_encoder: emit type tags for bpf_arena pointers
pahole: introduce --btf_feature=attributes
man-pages: describe attributes and remove reproducible_build
btf_encoder.c | 328 ++++++++++++++++++++++----------------------- dwarves.h | 13 +- lib/bpf | 2 +- man-pages/pahole.1 | 7 +- pahole.c | 11 +- 5 files changed, 188 insertions(+), 173 deletions(-)
Signature · 1 line
2.48.1
Ihor Solodrai
reviewerPATCH v4 1/6 btf_encoder: refactor btf_encoder__tag_kfuncs()
btf_encoder__tag_kfuncs() is a post-processing step of BTF encoding, executed right before BTF is deduped and dumped to the output.
Rewrite btf_encoder__tag_kfuncs() into btf_encoder__collect_kfuncs(). Now it only reads the .BTF_ids section of the ELF, collecting kfunc information and adding it to corresponding elf_function structs. It is executed in btf_encoder__new() if tag_kfuncs flag is set. This way kfunc information is available within entire lifetime of the btf_encoder.
BTF decl tags for kfuncs are added immediately after the function is added to BTF in btf_encoder__add_func(). It's done by btf__tag_kfunc() factored out from the btf_encoder__tag_kfunc().
As a result btf_encoder__collect_btf_funcs(), struct btf_func type and other relevant code are deleted, as they are no longer necessary.
--- btf_encoder.c | 189 +++++++++++++++----------------------------------- 1 file changed, 54 insertions(+), 135 deletions(-)
btf_encoder.c +54 -135
@@ -89,6 +89,8 @@ struct elf_function {const char *name;char *alias;size_t prefixlen; +bool kfunc; +uint32_t kfunc_flags; };struct elf_secinfo {
@@ -145,11 +147,6 @@ struct btf_encoder {struct list_head elf_functions_list; }; -struct btf_func { -const char *name; -int type_id; -}; -/* Half open interval representing range of addresses containing kfuncs */ struct btf_kfunc_set_range {uint64_t start;
@@ -1178,6 +1175,39 @@ out:return err; } +static int btf__add_kfunc_decl_tag (struct btf *btf,const char *tag,__u32 id,const char *kfunc) +{ +int err =btf__add_decl_tag (btf, tag, id,-1 ); + +if (err <0 ) { +fprintf (stderr,"%s: failed to insert kfunc decl tag for '%s': %d\n" , + __func__, kfunc, err); +return err; + } +return 0 ; +} + +static int btf__tag_kfunc (struct btf *btf,struct elf_function *kfunc,__u32 btf_fn_id) +{ +int err; + +/* Note we are unconditionally adding the btf_decl_tag even +* though vmlinux may already contain btf_decl_tags for kfuncs. +* We are ok to do this b/c we will later btf__dedup() to remove +* any duplicates. +*/ + err =btf__add_kfunc_decl_tag (btf,BTF_KFUNC_TYPE_TAG , btf_fn_id, kfunc->name); +if (err <0 ) +return err; + +if (kfunc->kfunc_flags &KF_FASTCALL ) { + err =btf__add_kfunc_decl_tag (btf,BTF_FASTCALL_TAG , btf_fn_id, kfunc->name); +if (err <0 ) +return err; + } +return 0 ; +} +static int32_t btf_encoder__add_func (struct btf_encoder *encoder,struct btf_encoder_func_state *state) {
@@ -1188,6 +1218,7 @@ static int32_t btf_encoder__add_func(struct btf_encoder *encoder,const char *value;char tmp_value[KSYM_NAME_LEN ];uint16_t idx; +int err; btf_fnproto_id =btf_encoder__add_func_proto (encoder,NULL , state); name = func->alias ?: func->name;
@@ -1199,6 +1230,13 @@ static int32_t btf_encoder__add_func(struct btf_encoder *encoder, name, btf_fnproto_id < 0 ? "proto" : "func");return -1 ; } + +if (func->kfunc && encoder->tag_kfuncs && !encoder->skip_encoding_decl_tag) { + err =btf__tag_kfunc (encoder->btf, func, btf_fn_id); +if (err <0 ) +return err; + } +if (state->nr_annots ==0 )return 0 ;
@@ -1771,116 +1809,10 @@ static char *get_func_name(const char *sym)return func; } -static int btf_func_cmp (const void *_a,const void *_b) -{ -const struct btf_func *a = _a; -const struct btf_func *b = _b; - -return strcmp (a->name, b->name); -} - -/* -* Collects all functions described in BTF. -* Returns non-zero on error. -*/ -static int btf_encoder__collect_btf_funcs (struct btf_encoder *encoder,struct gobuffer *funcs) -{ -struct btf *btf = encoder->btf; -int nr_types, type_id; -int err =-1 ; - -/* First collect all the func entries into an array */ - nr_types =btf__type_cnt (btf); -for (type_id =1 ; type_id < nr_types; type_id++) { -const struct btf_type *type; -struct btf_func func = {}; -const char *name; - - type =btf__type_by_id (btf, type_id); -if (!type) { -fprintf (stderr,"%s: malformed BTF, can't resolve type for ID %d\n" , - __func__, type_id); - err = -EINVAL ; - goto out; - } - -if (!btf_is_func (type)) -continue ; - - name =btf__name_by_offset (btf, type->name_off); -if (!name) { -fprintf (stderr,"%s: malformed BTF, can't resolve name for ID %d\n" , - __func__, type_id); - err = -EINVAL ; - goto out; - } - - func.name = name; - func.type_id = type_id; - err =gobuffer__add (funcs, &func,sizeof (func)); -if (err <0 ) - goto out; - } - -/* Now that we've collected funcs, sort them by name */ -gobuffer__sort (funcs,sizeof (struct btf_func ), btf_func_cmp); - - err =0 ; -out: -return err; -} - -static int btf__add_kfunc_decl_tag (struct btf *btf,const char *tag,__u32 id,const char *kfunc) -{ -int err =btf__add_decl_tag (btf, tag, id,-1 ); - -if (err <0 ) { -fprintf (stderr,"%s: failed to insert kfunc decl tag for '%s': %d\n" , - __func__, kfunc, err); -return err; - } -return 0 ; -} - -static int btf_encoder__tag_kfunc (struct btf_encoder *encoder,struct gobuffer *funcs,const char *kfunc,__u32 flags) -{ -struct btf_func key = { .name = kfunc }; -struct btf *btf = encoder->btf; -struct btf_func *target; -const void *base; -unsigned int cnt; -int err; - - base =gobuffer__entries (funcs); - cnt =gobuffer__nr_entries (funcs); - target =bsearch (&key, base, cnt,sizeof (key), btf_func_cmp); -if (!target) { -fprintf (stderr,"%s: failed to find kfunc '%s' in BTF\n" , __func__, kfunc); -return -1 ; - } - -/* Note we are unconditionally adding the btf_decl_tag even -* though vmlinux may already contain btf_decl_tags for kfuncs. -* We are ok to do this b/c we will later btf__dedup() to remove -* any duplicates. -*/ - err =btf__add_kfunc_decl_tag (btf,BTF_KFUNC_TYPE_TAG , target->type_id, kfunc); -if (err <0 ) -return err; -if (flags &KF_FASTCALL ) { - err =btf__add_kfunc_decl_tag (btf,BTF_FASTCALL_TAG , target->type_id, kfunc); -if (err <0 ) -return err; - } - -return 0 ; -} - -static int btf_encoder__tag_kfuncs (struct btf_encoder *encoder) +static int btf_encoder__collect_kfuncs (struct btf_encoder *encoder) {const char *filename = encoder->source_filename;struct gobuffer btf_kfunc_ranges = {}; -struct gobuffer btf_funcs = {};Elf_Data *symbols =NULL ;Elf_Data *idlist =NULL ;Elf_Scn *symscn =NULL ;
@@ -1977,12 +1909,6 @@ static int btf_encoder__tag_kfuncs(struct btf_encoder *encoder) } nr_syms = shdr.sh_size / shdr.sh_entsize; - err =btf_encoder__collect_btf_funcs (encoder, &btf_funcs); -if (err) { -fprintf (stderr,"%s: failed to collect BTF funcs\n" , __func__); - goto out; - } -/* First collect all kfunc set ranges. * * Note we choose not to sort these ranges and accept a linear
@@ -2015,12 +1941,12 @@ static int btf_encoder__tag_kfuncs(struct btf_encoder *encoder)for (i =0 ; i < nr_syms; i++) {const struct btf_kfunc_set_range *ranges;const struct btf_id_and_flag *pair; +struct elf_function *elf_fn;unsigned int ranges_cnt;char *func, *name;ptrdiff_t off;GElf_Sym sym;bool found; -int err;int j;if (!gelf_getsym (symbols, i, &sym)) {
@@ -2061,18 +1987,16 @@ static int btf_encoder__tag_kfuncs(struct btf_encoder *encoder)continue ; } - err =btf_encoder__tag_kfunc (encoder, &btf_funcs, func, pair->flags); -if (err) { -fprintf (stderr,"%s: failed to tag kfunc '%s'\n" , __func__, func); -free (func); - goto out; + elf_fn =btf_encoder__find_function (encoder, func,0 ); +if (elf_fn) { + elf_fn->kfunc = true; + elf_fn->kfunc_flags = pair->flags; }free (func); } err =0 ; out: -__gobuffer__delete (&btf_funcs);__gobuffer__delete (&btf_kfunc_ranges);if (elf)elf_end (elf);
@@ -2083,7 +2007,6 @@ out:int btf_encoder__encode (struct btf_encoder *encoder,struct conf_load *conf) { -bool should_tag_kfuncs;int err;size_t shndx;
@@ -2099,15 +2022,6 @@ int btf_encoder__encode(struct btf_encoder *encoder, struct conf_load *conf)if (btf__type_cnt (encoder->btf) ==1 )return 0 ; -/* Note vmlinux may already contain btf_decl_tag's for kfuncs. So -* take care to call this before btf_dedup(). -*/ - should_tag_kfuncs = encoder->tag_kfuncs && !encoder->skip_encoding_decl_tag; -if (should_tag_kfuncs &&btf_encoder__tag_kfuncs (encoder)) { -fprintf (stderr,"%s: failed to tag kfuncs!\n" , __func__); -return -1 ; - } -if (btf__dedup (encoder->btf,NULL )) {fprintf (stderr,"%s: btf__dedup failed!\n" , __func__);return -1 ;
@@ -2496,6 +2410,11 @@ struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filenamif (!found_percpu && encoder->verbose)printf ("%s: '%s' doesn't have '%s' section\n" , __func__, cu->filename,PERCPU_SECTION ); +if (encoder->tag_kfuncs) { +if (btf_encoder__collect_kfuncs (encoder)) + goto out_delete; + } +if (encoder->verbose)printf ("File %s:\n" , cu->filename); }
Signature · 1 line
2.48.1
Ihor Solodrai
reviewerPATCH v4 2/6 btf_encoder: use __weak declarations of version-dependent libbpf API
In reply to Ihor SolodraiInstead of compile-time checks for libbpf version, use __weak declarations of the required API functions and do runtime checks at the call sites. This will help with compatibility when libbpf is dynamically linked to pahole [1].
[1] https://lore.kernel.org/dwarves/[email protected]/
--- btf_encoder.c | 48 +++++++++++++++++++----------------------------- dwarves.h | 11 ++++++++++- pahole.c | 2 -- 3 files changed, 29 insertions(+), 32 deletions(-)
btf_encoder.c +19 -29
@@ -34,6 +34,7 @@#include <search.h> /* for tsearch(), tfind() and tdestroy() */ #include <pthread.h> +#define BTF_BASE_ELF_SEC ".BTF.base"#define BTF_IDS_SECTION ".BTF_ids"#define BTF_ID_FUNC_PFX "__BTF_ID__func__"#define BTF_ID_SET8_PFX "__BTF_ID__set8__"
@@ -625,29 +626,6 @@ static int32_t btf_encoder__add_struct(struct btf_encoder *encoder, uint8_t kindreturn id; } -#if LIBBPF_MAJOR_VERSION <1 -static inline int libbpf_err (int ret) -{ -if (ret <0 ) - errno = -ret; -return ret; -} - -static -int btf__add_enum64 (struct btf *btf__maybe_unused ,const char *name__maybe_unused , -__u32 byte_sz__maybe_unused ,bool is_signed__maybe_unused ) -{ -return libbpf_err (-ENOTSUP ); -} - -static -int btf__add_enum64_value (struct btf *btf__maybe_unused ,const char *name__maybe_unused , -__u64 value__maybe_unused ) -{ -return libbpf_err (-ENOTSUP ); -} -#endif -static int32_t btf_encoder__add_enum (struct btf_encoder *encoder,const char *name,struct type *etype,struct conf_load *conf_load) {
@@ -660,8 +638,13 @@ static int32_t btf_encoder__add_enum(struct btf_encoder *encoder, const char *na is_enum32 = size <=4 || conf_load->skip_encoding_btf_enum64;if (is_enum32) id =btf__add_enum (btf, name, size); -else +else if (btf__add_enum64) id =btf__add_enum64 (btf, name, size, etype->is_signed_enum); +else { +fprintf (stderr,"btf__add_enum64 is not available, is libbpf < 1.0?\n" ); +return -ENOTSUP ; + } +if (id >0 ) { t =btf__type_by_id (btf, id);btf_encoder__log_type (encoder, t, false, true,"size=%u" , t->size);
@@ -684,10 +667,14 @@ static int btf_encoder__add_enum_val(struct btf_encoder *encoder, const char *na*/ if (conf_load->skip_encoding_btf_enum64) err =btf__add_enum_value (encoder->btf, name, (uint32_t )value); -else if (etype->size >32 ) - err =btf__add_enum64_value (encoder->btf, name, value); -else +else if (etype->size <=32 ) err =btf__add_enum_value (encoder->btf, name, value); +else if (btf__add_enum64_value) + err =btf__add_enum64_value (encoder->btf, name, value); +else { +fprintf (stderr,"btf__add_enum64_value is not available, is libbpf < 1.0?\n" ); +return -ENOTSUP ; + }if (!err) {if (encoder->verbose) {
@@ -2035,10 +2022,14 @@ int btf_encoder__encode(struct btf_encoder *encoder, struct conf_load *conf)* silently ignore the feature request if libbpf does not * support it. */ -#if LIBBPF_MAJOR_VERSION >=1 &&LIBBPF_MINOR_VERSION >=5 if (encoder->gen_distilled_base) {struct btf *btf =NULL , *distilled_base =NULL ; +if (!btf__distill_base) { +fprintf (stderr,"btf__distill_base is not available, is libbpf < 1.5?\n" ); +return -ENOTSUP ; + } +if (btf__distill_base (encoder->btf, &distilled_base, &btf) <0 ) {fprintf (stderr,"could not generate distilled base BTF: %s\n" ,strerror (errno));
@@ -2051,7 +2042,6 @@ int btf_encoder__encode(struct btf_encoder *encoder, struct conf_load *conf)btf__free (distilled_base);return err; } -#endif err =btf_encoder__write_elf (encoder, encoder->btf,BTF_ELF_SEC ); }
dwarves.h +10 -1
@@ -14,6 +14,7 @@#include <obstack.h> #include <dwarf.h> #include <elfutils/libdwfl.h> +#include <linux/types.h> #include <sys/types.h> #include "dutil.h"
@@ -44,13 +45,21 @@ enum load_steal_kind { LSK__STOP_LOADING, }; +/* +* Weak declarations of libbpf APIs that are version-dependent +*/ +#define __weak __attribute__((weak)) +struct btf ; +__weak extern int btf__add_enum64 (struct btf *btf,const char *name,__u32 byte_sz,bool is_signed); +__weak extern int btf__add_enum64_value (struct btf *btf,const char *name,__u64 value); +__weak extern int btf__distill_base (const struct btf *src_btf,struct btf **new_base_btf,struct btf **new_split_btf); +/* * BTF combines all the types into one big CU using btf_dedup(), so for something * like a allyesconfig vmlinux kernel we can get over 65535 types. */ typedef uint32_t type_id_t ; -struct btf ;struct conf_fprintf ;/** struct conf_load - load configuration
pahole.c +0 -2
@@ -1206,9 +1206,7 @@ struct btf_feature {BTF_DEFAULT_FEATURE (consistent_func, skip_encoding_btf_inconsistent_proto, false),BTF_DEFAULT_FEATURE (decl_tag_kfuncs, btf_decl_tag_kfuncs, false),BTF_NON_DEFAULT_FEATURE (reproducible_build, reproducible_build, false), -#if LIBBPF_MAJOR_VERSION >= 1 && LIBBPF_MINOR_VERSION >= 5BTF_NON_DEFAULT_FEATURE (distilled_base, btf_gen_distilled_base, false), -#endif BTF_NON_DEFAULT_FEATURE(global_var, encode_btf_global_vars, false), };
Signature · 1 line
2.48.1
Ihor Solodrai
reviewerRe: [PATCH dwarves v4 2/6] btf_encoder: use __weak declarations of version-dependent libbpf API
Quoted 27 lines · Ihor Solodrai
Instead of compile-time checks for libbpf version, use
__weakdeclarations of the required API functions and do runtime checks at the call sites. This will help with compatibility when libbpf is dynamically linked to pahole [1].[1] https://lore.kernel.org/dwarves/[email protected]/
Signed-off-by Ihor Solodrai <[email protected]>--- btf_encoder.c | 48 +++++++++++++++++++----------------------------- dwarves.h | 11 ++++++++++- pahole.c | 2 -- 3 files changed, 29 insertions(+), 32 deletions(-)btf_encoder.c +1
-0@@ -34,6 +34,7 @@#include <search.h> /* for tsearch(), tfind() and tdestroy() */ #include <pthread.h> +#define BTF_BASE_ELF_SEC ".BTF.base"#define BTF_IDS_SECTION ".BTF_ids"#define BTF_ID_FUNC_PFX "__BTF_ID__func__"#define BTF_ID_SET8_PFX "__BTF_ID__set8__"
btf_encoder.c static int32_t btf_encoder__add_struct(struct btf_encoder *encoder, uint8_t kind · line 628
Quoted 4 lines
btf_encoder.c +0
-1@@ -625,29 +626,6 @@ static int32_t btf_encoder__add_struct(struct btf_encoder *encoder, uint8_t kindreturn id; } -#if LIBBPF_MAJOR_VERSION <1
There is an identical condition in btf_loader.c, however it guards static functions, for example btf_enum64(). I decided to leave it as is, although I find it unlikely that someone would use libbpf < 1.0.
Quoted 1 line
[...]
Re: [PATCH dwarves v4 2/6] btf_encoder: use __weak declarations of version-dependent libbpf API
Quoted 31 lines · Ihor Solodrai
Quoted 30 lines · Ihor Solodrai
Instead of compile-time checks for libbpf version, use
__weakdeclarations of the required API functions and do runtime checks at the call sites. This will help with compatibility when libbpf is dynamically linked to pahole [1].[1] https://lore.kernel.org/dwarves/[email protected]/
Signed-off-by Ihor Solodrai <[email protected]>--- btf_encoder.c | 48 +++++++++++++++++++----------------------------- dwarves.h | 11 ++++++++++- pahole.c | 2 -- 3 files changed, 29 insertions(+), 32 deletions(-)btf_encoder.c +1
-0@@ -34,6 +34,7 @@#include <search.h> /* for tsearch(), tfind() and tdestroy() */ #include <pthread.h> +#define BTF_BASE_ELF_SEC ".BTF.base"#define BTF_IDS_SECTION ".BTF_ids"#define BTF_ID_FUNC_PFX "__BTF_ID__func__"#define BTF_ID_SET8_PFX "__BTF_ID__set8__"@@ -625,29 +626,6 @@ static int32_t btf_encoder__add_struct(struct btf_encoder *encoder, uint8_t kindreturn id; }
Quoted 7 lines
btf_encoder.c static int32_t btf_encoder__add_struct(struct btf_encoder *encoder, uint8_t kind · line 628
Quoted 1 line
btf_encoder.c +0
-1-#if LIBBPF_MAJOR_VERSION <1 There is an identical condition in
btf_loader.c, however it guards static functions, for examplebtf_enum64(). I decided to leave it as is, although I find it unlikely that someone would use libbpf < 1.0.Quoted 1 line
[...]
yeah, I just noticed we've got a minimum version requirement of 0.4:
pkg_check_modules(LIBBPF REQUIRED libbpf>=0.4.0)
That needs to be revisited in the future to be > v1.0 I would say. To test libbpf shared library support, I tried building with libbpf v1.2, but hit errors around absence of struct btf_enum64. Looks like these compilation errors resulted from not having an up-to-date /usr/include/linux/btf.h (IIRC the libbpf repo ships with a copy, maybe we should do the same?). The errors are not caused by your series, so not something you need to worry about, but more work is needed to better support shared library builds even for libbpf > 1.0 it seems. Thanks!
Alan
Ihor Solodrai
reviewerPATCH v4 3/6 pahole: sync with libbpf mainline
In reply to Ihor SolodraiPull recently added libbpf API functions:
* btf__add_decl_attr()
* btf__add_type_attr()
--- lib/bpf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/bpf +1 -1
@@ -1 +1 @@ -Subproject commit 09b9e83102eb8ab9e540d36b4559c55f3bcdb95d +Subproject commit 42a6ef63161a8dc4288172b27f3870e50b3606f7
Signature · 1 line
2.48.1
Ihor Solodrai
reviewerPATCH v4 4/6 btf_encoder: emit type tags for bpf_arena pointers
In reply to Ihor SolodraiWhen adding a kfunc prototype to BTF, check for the flags indicating bpf_arena pointers and emit a type tag encoding __attribute__((address_space(1))) for them. This also requires updating BTF type ids in the btf_encoder_func_state, which is done as a side effect in the tagging functions.
This feature depends on recent update in libbpf, supporting arbitrarty attribute encoding [1].
[1] https://lore.kernel.org/bpf/[email protected]/
--- btf_encoder.c | 87 ++++++++++++++++++++++++++++++++++++++++++++++++++- dwarves.h | 1 + 2 files changed, 87 insertions(+), 1 deletion(-)
btf_encoder.c +86 -1
@@ -41,7 +41,13 @@#define BTF_SET8_KFUNCS (1 << 0)#define BTF_KFUNC_TYPE_TAG "bpf_kfunc"#define BTF_FASTCALL_TAG "bpf_fastcall" -#define KF_FASTCALL (1 << 12) +#define BPF_ARENA_ATTR "address_space(1)" + +/* kfunc flags, see include/linux/btf.h in the kernel source */ +#define KF_FASTCALL (1 << 12) +#define KF_ARENA_RET (1 << 13) +#define KF_ARENA_ARG1 (1 << 14) +#define KF_ARENA_ARG2 (1 << 15)struct btf_id_and_flag {uint32_t id;
@@ -718,6 +724,81 @@ static int32_t btf_encoder__tag_type(struct btf_encoder *encoder, uint32_t tag_treturn encoder->type_id_off + tag_type; } +static int btf__tag_bpf_arena_ptr (struct btf *btf,int ptr_id) +{ +const struct btf_type *ptr; +int tagged_type_id; + + ptr =btf__type_by_id (btf, ptr_id); +if (!btf_is_ptr (ptr)) +return -EINVAL ; + + tagged_type_id =btf__add_type_attr (btf,BPF_ARENA_ATTR , ptr->type); +if (tagged_type_id <0 ) +return tagged_type_id; + +return btf__add_ptr (btf, tagged_type_id); +} + +static int btf__tag_bpf_arena_arg (struct btf *btf,struct btf_encoder_func_state *state,int idx) +{ +int id; + +if (state->nr_parms <= idx) +return -EINVAL ; + + id =btf__tag_bpf_arena_ptr (btf, state->parms[idx].type_id); +if (id <0 ) { +btf__log_err (btf,BTF_KIND_TYPE_TAG ,BPF_ARENA_ATTR , true, id, +"Error adding BPF_ARENA_ATTR for an argument of kfunc '%s'" , state->elf->name); +return id; + } + state->parms[idx].type_id = id; + +return id; +} + +static int btf__add_bpf_arena_type_tags (struct btf *btf,struct btf_encoder_func_state *state) +{ +uint32_t flags = state->elf->kfunc_flags; +int ret_type_id; +int err; + +if (!btf__add_type_attr) { +fprintf (stderr,"btf__add_type_attr is not available, is libbpf < 1.6?\n" ); +return -ENOTSUP ; + } + +if (KF_ARENA_RET & flags) { + ret_type_id =btf__tag_bpf_arena_ptr (btf, state->ret_type_id); +if (ret_type_id <0 ) { +btf__log_err (btf,BTF_KIND_TYPE_TAG ,BPF_ARENA_ATTR , true, ret_type_id, +"Error adding BPF_ARENA_ATTR for return type of kfunc '%s'" , state->elf->name); +return ret_type_id; + } + state->ret_type_id = ret_type_id; + } + +if (KF_ARENA_ARG1 & flags) { + err =btf__tag_bpf_arena_arg (btf, state,0 ); +if (err <0 ) +return err; + } + +if (KF_ARENA_ARG2 & flags) { + err =btf__tag_bpf_arena_arg (btf, state,1 ); +if (err <0 ) +return err; + } + +return 0 ; +} + +static inline bool is_kfunc_state (struct btf_encoder_func_state *state) +{ +return state && state->elf && state->elf->kfunc; +} +static int32_t btf_encoder__add_func_proto (struct btf_encoder *encoder,struct ftype *ftype,struct btf_encoder_func_state *state) {
@@ -731,6 +812,10 @@ static int32_t btf_encoder__add_func_proto(struct btf_encoder *encoder, struct fassert (ftype !=NULL || state !=NULL ); +if (is_kfunc_state (state) && encoder->tag_kfuncs) +if (btf__add_bpf_arena_type_tags (encoder->btf, state) <0 ) +return -1 ; +/* add btf_type for func_proto */ if (ftype) { btf = encoder->btf;
dwarves.h +1 -0
@@ -52,6 +52,7 @@ enum load_steal_kind {struct btf ;__weak extern int btf__add_enum64 (struct btf *btf,const char *name,__u32 byte_sz,bool is_signed);__weak extern int btf__add_enum64_value (struct btf *btf,const char *name,__u64 value); +__weak extern int btf__add_type_attr (struct btf *btf,const char *value,int ref_type_id);__weak extern int btf__distill_base (const struct btf *src_btf,struct btf **new_base_btf,struct btf **new_split_btf);/*
Signature · 1 line
2.48.1
Ihor Solodrai
reviewerPATCH v4 5/6 pahole: introduce --btf_feature=attributes
In reply to Ihor SolodraiAdd a feature flag "attributes" (default: false) controlling whether pahole is allowed to generate BTF attributes: type tags and decl tags with kind_flag = 1.
This is necessary for backward compatibility, as BPF verifier does not recognize tags with kind_flag = 1 prior to (at least) 6.14-rc1 [1].
[1] https://lore.kernel.org/bpf/[email protected]/
--- btf_encoder.c | 6 ++++-- dwarves.h | 1 + pahole.c | 9 +++++++++ 3 files changed, 14 insertions(+), 2 deletions(-)
btf_encoder.c +4 -2
@@ -137,7 +137,8 @@ struct btf_encoder { gen_floats, skip_encoding_decl_tag, tag_kfuncs, - gen_distilled_base; + gen_distilled_base, + encode_attributes;uint32_t array_index_id;struct elf_secinfo *secinfo;size_t seccnt;
@@ -812,7 +813,7 @@ static int32_t btf_encoder__add_func_proto(struct btf_encoder *encoder, struct fassert (ftype !=NULL || state !=NULL ); -if (is_kfunc_state (state) && encoder->tag_kfuncs) +if (is_kfunc_state (state) && encoder->tag_kfuncs && encoder->encode_attributes)if (btf__add_bpf_arena_type_tags (encoder->btf, state) <0 )return -1 ;
@@ -2405,6 +2406,7 @@ struct btf_encoder *btf_encoder__new(struct cu *cu, const char *detached_filenam encoder->skip_encoding_decl_tag = conf_load->skip_encoding_btf_decl_tag; encoder->tag_kfuncs = conf_load->btf_decl_tag_kfuncs; encoder->gen_distilled_base = conf_load->btf_gen_distilled_base; + encoder->encode_attributes = conf_load->btf_attributes; encoder->verbose = verbose; encoder->has_index_type = false; encoder->need_index_type = false;
dwarves.h +1 -0
@@ -99,6 +99,7 @@ struct conf_load {bool reproducible_build;bool btf_decl_tag_kfuncs;bool btf_gen_distilled_base; +bool btf_attributes;uint8_t hashtable_bits;uint8_t max_hashtable_bits;uint16_t kabi_prefix_len;
pahole.c +9 -0
@@ -1152,6 +1152,7 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;#define ARG_padding_ge 347#define ARG_padding 348#define ARGP_with_embedded_flexible_array 349 +#define ARGP_btf_attributes 350/* --btf_features=feature1[,feature2,..] allows us to specify * a list of requested BTF features or "default" to enable all default
@@ -1208,6 +1209,7 @@ struct btf_feature {BTF_NON_DEFAULT_FEATURE (reproducible_build, reproducible_build, false),BTF_NON_DEFAULT_FEATURE (distilled_base, btf_gen_distilled_base, false),BTF_NON_DEFAULT_FEATURE (global_var, encode_btf_global_vars, false), + BTF_NON_DEFAULT_FEATURE(attributes, btf_attributes, false), };#define BTF_MAX_FEATURE_STR 1024
@@ -1783,6 +1785,11 @@ static const struct argp_option pahole__options[] = { .key = ARGP_running_kernel_vmlinux, .doc = "Search for, possibly getting from a debuginfo server, a vmlinux matching the running kernel build-id (from /sys/kernel/notes)" }, + { + .name = "btf_attributes", + .key = ARGP_btf_attributes, + .doc = "Allow generation of attributes in BTF. Attributes are the type tags and decl tags with the kind_flag set to 1.", + }, { .name = NULL, }
@@ -1977,6 +1984,8 @@ static error_t pahole__options_parser(int key, char *arg,show_supported_btf_features (stdout);exit (0 );case ARGP_btf_features_strict:parse_btf_features (arg, true);break ; +case ARGP_btf_attributes: + conf_load.btf_attributes = true;break ;default :return ARGP_ERR_UNKNOWN ; }
Signature · 1 line
2.48.1
Ihor Solodrai
reviewerPATCH v4 6/6 man-pages: describe attributes and remove reproducible_build
In reply to Ihor SolodraiAdd a description of the new --btf_feature=attributes.
Also remove reproducible_build description, as it is now a moot feature flag.
--- man-pages/pahole.1 | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
man-pages/pahole.1 +4 -3
@@ -327,9 +327,10 @@ Encode BTF using the specified feature list, or specify 'default' for all standa Supported non-standard features (not enabled for 'default') .nf - reproducible_build Ensure generated BTF is consistent every time; - without this parallel BTF encoding can result in - inconsistent BTF ids. + attributes Allow generation of decl_tags and type_tags with + kind_flag = 1. These tags can be used to encode + arbitrary compiler attributes in BTF, but are + backward incompatible. decl_tag_kfuncs Inject a BTF_KIND_DECL_TAG for each discovered kfunc. distilled_base For split BTF, generate a distilled version of the associated base BTF to support later relocation
Signature · 1 line
2.48.1
Ihor Solodrai
reviewerQuoted 31 lines · Ihor Solodrai
This patch series implements emitting appropriate BTF type tags for argument and return types of kfuncs marked with KF_ARENA_* flags.
For additional context see the description of BPF patch "bpf: define KF_ARENA_* flags for
bpf_arenakfuncs" [1].The feature depends on recent changes in libbpf [2].
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
v3->v4:
* Add a patch (#2) replacing compile-time libbpf version checks with runtime checks for symbol availablility
* Add a patch (#3) bumping libbpf submodule commit to latest master
* Modify "btf_encoder: emit type tags forbpf_arenapointers" (#2->#4) to not use compile time libbpf version checks
v2->v3:
* Nits in patch #1
v1->v2:
* Rewrite patch #1 refactoringbtf_encoder__tag_kfuncs(): now the post-processing step is removed entirely, and kfuncs are tagged inbtf_encoder__add_func().
* Nits and renames in patch #2
* Add patch #4 editing man pagesv2: https://lore.kernel.org/dwarves/[email protected]/
v1: https://lore.kernel.org/dwarves/[email protected]/
Quoted 7 lines
Ihor Solodrai (6):
btf_encoder: refactorbtf_encoder__tag_kfuncs()btf_encoder: use__weakdeclarations of version-dependent libbpf API
pahole: sync with libbpf mainlinebtf_encoder: emit type tags forbpf_arenapointers
pahole: introduce --btf_feature=attributes
man-pages: describe attributes and removereproducible_build
Hi Alan, Arnaldo.
This series hasn't received any comments in a while. Do you plan to review/land this?
Thanks.
Quoted 8 lines
btf_encoder.c| 328 ++++++++++++++++++++++-----------------------dwarves.h| 13 +-
lib/bpf | 2 +-
man-pages/pahole.1 | 7 +-pahole.c| 11 +-5 files changed, 188 insertions(+), 173 deletions(-)
Quoted 40 lines · Ihor Solodrai
Quoted 31 lines · Ihor Solodrai
This patch series implements emitting appropriate BTF type tags for argument and return types of kfuncs marked with KF_ARENA_* flags.
For additional context see the description of BPF patch "bpf: define KF_ARENA_* flags for
bpf_arenakfuncs" [1].The feature depends on recent changes in libbpf [2].
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
v3->v4:
* Add a patch (#2) replacing compile-time libbpf version checks with runtime checks for symbol availablility
* Add a patch (#3) bumping libbpf submodule commit to latest master
* Modify "btf_encoder: emit type tags forbpf_arenapointers" (#2->#4) to not use compile time libbpf version checks
v2->v3:
* Nits in patch #1
v1->v2:
* Rewrite patch #1 refactoringbtf_encoder__tag_kfuncs(): now the post-processing step is removed entirely, and kfuncs are tagged inbtf_encoder__add_func().
* Nits and renames in patch #2
* Add patch #4 editing man pagesv2: https://lore.kernel.org/dwarves/[email protected]/
v1: https://lore.kernel.org/dwarves/[email protected]/Quoted 7 lines
Ihor Solodrai (6):
btf_encoder: refactorbtf_encoder__tag_kfuncs()btf_encoder: use__weakdeclarations of version-dependent libbpf API
pahole: sync with libbpf mainlinebtf_encoder: emit type tags forbpf_arenapointers
pahole: introduce --btf_feature=attributes
man-pages: describe attributes and removereproducible_build
Quoted 5 lines
Hi Alan, Arnaldo.
This series hasn't received any comments in a while. Do you plan to review/land this?
Yep, thanks for the reminder; I hit a wall last time I looked a this when testing with a shared library libbpf versus embedded but I can get around that now so I should have the testing done for both modes tomorrow.
Alan
Quoted 10 lines
Thanks.
Quoted 8 lines
btf_encoder.c| 328 ++++++++++++++++++++++-----------------------dwarves.h| 13 +-
lib/bpf | 2 +-
man-pages/pahole.1 | 7 +-pahole.c| 11 +-5 files changed, 188 insertions(+), 173 deletions(-)
Quoted 43 lines · Alan Maguire
Quoted 42 lines · Ihor Solodrai
Quoted 31 lines · Ihor Solodrai
This patch series implements emitting appropriate BTF type tags for argument and return types of kfuncs marked with KF_ARENA_* flags.
For additional context see the description of BPF patch "bpf: define KF_ARENA_* flags for
bpf_arenakfuncs" [1].The feature depends on recent changes in libbpf [2].
[1] https://lore.kernel.org/bpf/[email protected]/
[2] https://lore.kernel.org/bpf/[email protected]/
v3->v4:
* Add a patch (#2) replacing compile-time libbpf version checks with runtime checks for symbol availablility
* Add a patch (#3) bumping libbpf submodule commit to latest master
* Modify "btf_encoder: emit type tags forbpf_arenapointers" (#2->#4) to not use compile time libbpf version checks
v2->v3:
* Nits in patch #1
v1->v2:
* Rewrite patch #1 refactoringbtf_encoder__tag_kfuncs(): now the post-processing step is removed entirely, and kfuncs are tagged inbtf_encoder__add_func().
* Nits and renames in patch #2
* Add patch #4 editing man pagesv2: https://lore.kernel.org/dwarves/[email protected]/
v1: https://lore.kernel.org/dwarves/[email protected]/Quoted 7 lines
Ihor Solodrai (6):
btf_encoder: refactorbtf_encoder__tag_kfuncs()btf_encoder: use__weakdeclarations of version-dependent libbpf API
pahole: sync with libbpf mainlinebtf_encoder: emit type tags forbpf_arenapointers
pahole: introduce --btf_feature=attributes
man-pages: describe attributes and removereproducible_buildHi Alan, Arnaldo.
Quoted 8 lines
Quoted 3 lines
This series hasn't received any comments in a while. Do you plan to review/land this?
Yep, thanks for the reminder; I hit a wall last time I looked a this when testing with a shared library libbpf versus embedded but I can get around that now so I should have the testing done for both modes tomorrow.
hi Ihor, I took a look at the series and merged it with latest next branch; results are in
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4
...if you want to take a look.
There are a few small things I think that it would be good to resolve before landing this.
First, when testing this with -DLIBBPF_EMBEDDED=OFF and a packaged libbpf 1.5 - which means we wouldn't have the latest attributes-related libbpf function; I saw:
BTF .tmp_vmlinux1.btf.obtf__add_type_attr is not available, is libbpf < 1.6?
error: failed to encode function 'bbr_cwnd_event': invalid proto Failed to encode BTF
NM .tmp_vmlinux1.syms
...and we got no BTF as a result. Ideally we'd like pahole to encode but without the attributes feature if not available. Related, we also report features that are not present, i.e. attributes with "--supported_btf_features". So I propose that we make use of the weak declarations being NULL in an optional feature check function. It is optionally declared for a feature, and if declared must return true if the feature is available.
Something like the below works (it's in the next.attributes-v4 branch too for reference) and it resolves the issue of BTF generation failure and accurate supported_btf_features reporting. What do you think? Thanks!
Alan
From: Alan Maguire <[email protected]>
Date: Sun, 23 Mar 2025 11:06:18 +0000
Subject: [PATCH] pahole: add a BTF feature check function
It is used to see if functions that BTF features rely on are really there; weak declarations mean they will be NULL if not in non-embedded linked libbpf.
This gives us more accurate --supported_btf_features reporting also.
--- pahole.c | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-)
pahole.c +0 -0
@@ -1183,10 +1183,31 @@ ARGP_PROGRAM_VERSION_HOOK_DEF =
dwarves_print_version;
* floats, etc. This ensures backwards compatibility. */
#define BTF_DEFAULT_FEATURE(name, alias, initial_value) \
- { #name, #alias, &conf_load.alias, initial_value, true }
+ { #name, #alias, &conf_load.alias, initial_value, true, NULL }
+
+#define BTF_DEFAULT_FEATURE_CHECK(name, alias, initial_value,feature_check) \
+ { #name, #alias, &conf_load.alias, initial_value, true, feature_check }
#define BTF_NON_DEFAULT_FEATURE(name, alias, initial_value) \
- { #name, #alias, &conf_load.alias, initial_value, false }
+ { #name, #alias, &conf_load.alias, initial_value, false, NULL }
+
+#define BTF_NON_DEFAULT_FEATURE_CHECK(name, alias, initial_value, feature_check) \
+ { #name, #alias, &conf_load.alias, initial_value, false, feature_check }
+
+static bool enum64_check(void)
+{
+ return btf__add_enum64 != NULL;
+}
+
+static bool distilled_base_check(void)
+{
+ return btf__distill_base != NULL;
+}
+
+static bool attributes_check(void)
+{
+ return btf__add_type_attr != NULL;
+}
struct btf_feature {
const char *name;
@@ -1196,20 +1217,23 @@ struct btf_feature {
bool default_enabled; /* some nonstandard features may not
* be enabled for --btf_features=default */
+ bool (*feature_check)(void);
} btf_features[] = {
BTF_DEFAULT_FEATURE(encode_force, btf_encode_force, false), BTF_DEFAULT_FEATURE(var, skip_encoding_btf_vars, true), BTF_DEFAULT_FEATURE(float, btf_gen_floats, false), BTF_DEFAULT_FEATURE(decl_tag, skip_encoding_btf_decl_tag, true), BTF_DEFAULT_FEATURE(type_tag, skip_encoding_btf_type_tag, true),
- BTF_DEFAULT_FEATURE(enum64, skip_encoding_btf_enum64, true),
+ BTF_DEFAULT_FEATURE_CHECK(enum64, skip_encoding_btf_enum64, true,enum64_check),
BTF_DEFAULT_FEATURE(optimized_func, btf_gen_optimized, false), BTF_DEFAULT_FEATURE(consistent_func,skip_encoding_btf_inconsistent_proto, false),
BTF_DEFAULT_FEATURE(decl_tag_kfuncs, btf_decl_tag_kfuncs, false), BTF_NON_DEFAULT_FEATURE(reproducible_build, reproducible_build, false),
- BTF_NON_DEFAULT_FEATURE(distilled_base, btf_gen_distilled_base, false),
+ BTF_NON_DEFAULT_FEATURE_CHECK(distilled_base, btf_gen_distilled_base,
false,
+ distilled_base_check),
BTF_NON_DEFAULT_FEATURE(global_var, encode_btf_global_vars, false),
- BTF_NON_DEFAULT_FEATURE(attributes, btf_attributes, false),
+ BTF_NON_DEFAULT_FEATURE_CHECK(attributes, btf_attributes, false,
+ attributes_check),
};
#define BTF_MAX_FEATURE_STR 1024
@@ -1248,7 +1272,8 @@ static void enable_btf_feature(struct btf_feature *feature)
/* switch "initial-off" features on, and "initial-on" features
* off; i.e. negate the initial value.
*/
- *feature->conf_value = !feature->initial_value;
+ if (!feature->feature_check || feature->feature_check())
+ *feature->conf_value = !feature->initial_value;
}
static void show_supported_btf_features(FILE *output)
@@ -1256,6 +1281,8 @@ static void show_supported_btf_features(FILE *output)
int i;
for (i = 0; i < ARRAY_SIZE(btf_features); i++) {
+ if (btf_features[i].feature_check && !btf_features[i].feature_check())
+ continue;
if (i > 0)
fprintf(output, ",");
fprintf(output, "%s", btf_features[i].name);
Signature · 1 line
2.39.3
Quoted 16 lines · Alan Maguire
[...]
hi Ihor, I took a look at the series and merged it with latest next branch; results are in
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4
...if you want to take a look.
There are a few small things I think that it would be good to resolve before landing this.
First, when testing this with -DLIBBPF_EMBEDDED=OFF and a packaged libbpf 1.5 - which means we wouldn't have the latest attributes-related libbpf function; I saw:
Quoted 5 lines
BTF .tmp_vmlinux1.btf.o
btf__add_type_attris not available, is libbpf < 1.6?
error: failed to encode function 'bbr_cwnd_event': invalid proto Failed to encode BTF
NM .tmp_vmlinux1.syms
Hi Alan. Thanks for testing. This is my mistake, I should've checked for attributes feature here:
@@ -731,6 +812,10 @@ static int32_t btf_encoder__add_func_proto(struct btf_encoder *encoder, struct f
assert(ftype != NULL || state != NULL);
+ if (is_kfunc_state(state) && encoder->tag_kfuncs)
+ if (btf__add_bpf_arena_type_tags(encoder->btf, state) < 0)
+ return -1;
Quoted 9 lines
...and we got no BTF as a result. Ideally we'd like pahole to encode but without the attributes feature if not available. Related, we also report features that are not present, i.e. attributes with "--supported_btf_features". So I propose that we make use of the weak declarations being NULL in an optional feature check function. It is optionally declared for a feature, and if declared must return true if the feature is available.
Quoted 3 lines
Something like the below works (it's in the next.attributes-v4 branch too for reference) and it resolves the issue of BTF generation failure and accurate
supported_btf_featuresreporting. What do you think? Thanks!
I think failing fast is a good approach here: check that all requested/default features are available on startup.
Quoted 11 lines
Alan
From: Alan Maguire <[email protected]>
Date: Sun, 23 Mar 2025 11:06:18 +0000
Subject: [PATCH] pahole: add a BTF feature check functionIt is used to see if functions that BTF features rely on are really there; weak declarations mean they will be NULL if not in non-embedded linked libbpf.
Quoted 7 lines
This gives us more accurate --supported_btf_features reporting also.
Signed-off-by Alan Maguire <[email protected]>--- pahole.c | 39 +++++++++++++++++++++++++++++++++------ 1 file changed, 33 insertions(+), 6 deletions(-)
Quoted 91 lines
pahole.c +23
-2@@ -1183,10 +1183,31 @@ ARGP_PROGRAM_VERSION_HOOK_DEF =dwarves_print_version; * floats, etc. This ensures backwards compatibility. */ #define BTF_DEFAULT_FEATURE (name, alias, initial_value) \ - { #name, #alias, &conf_load.alias, initial_value, true } + { #name, #alias, &conf_load.alias, initial_value, true, NULL } + +#define BTF_DEFAULT_FEATURE_CHECK (name, alias, initial_value, feature_check) \ + { #name, #alias, &conf_load.alias, initial_value, true, feature_check }#define BTF_NON_DEFAULT_FEATURE (name, alias, initial_value) \ - { #name, #alias, &conf_load.alias, initial_value, false } + { #name, #alias, &conf_load.alias, initial_value, false, NULL } + +#define BTF_NON_DEFAULT_FEATURE_CHECK (name, alias, initial_value, feature_check) \ + { #name, #alias, &conf_load.alias, initial_value, false, feature_check } + +static bool enum64_check (void ) +{ +return btf__add_enum64 !=NULL ; +} + +static bool distilled_base_check (void ) +{ +return btf__distill_base !=NULL ; +} + +static bool attributes_check (void ) +{ +return btf__add_type_attr !=NULL ; +}struct
btf_feature{pahole.c +6
-3const char *name;@@ -1196,20 +1217,23 @@ struct btf_feature {bool default_enabled;/* some nonstandard features may not * be enabled for --btf_features=default */ +bool (*feature_check)(void ); } btf_features[] = {BTF_DEFAULT_FEATURE (encode_force, btf_encode_force, false),BTF_DEFAULT_FEATURE (var, skip_encoding_btf_vars, true),BTF_DEFAULT_FEATURE (float, btf_gen_floats, false),BTF_DEFAULT_FEATURE (decl_tag, skip_encoding_btf_decl_tag, true),BTF_DEFAULT_FEATURE (type_tag, skip_encoding_btf_type_tag, true), - BTF_DEFAULT_FEATURE(enum64, skip_encoding_btf_enum64, true), +BTF_DEFAULT_FEATURE_CHECK (enum64, skip_encoding_btf_enum64, true, enum64_check),BTF_DEFAULT_FEATURE (optimized_func, btf_gen_optimized, false),BTF_DEFAULT_FEATURE (consistent_func, skip_encoding_btf_inconsistent_proto, false),BTF_DEFAULT_FEATURE (decl_tag_kfuncs, btf_decl_tag_kfuncs, false),BTF_NON_DEFAULT_FEATURE (reproducible_build, reproducible_build, false), -BTF_NON_DEFAULT_FEATURE (distilled_base, btf_gen_distilled_base, false), +BTF_NON_DEFAULT_FEATURE_CHECK (distilled_base, btf_gen_distilled_base, false, + distilled_base_check),BTF_NON_DEFAULT_FEATURE (global_var, encode_btf_global_vars, false), - BTF_NON_DEFAULT_FEATURE(attributes, btf_attributes, false), +BTF_NON_DEFAULT_FEATURE_CHECK (attributes, btf_attributes, false, + attributes_check),};
#define
BTF_MAX_FEATURE_STR1024pahole.c +2
-1@@ -1248,7 +1272,8 @@ static void enable_btf_feature(struct btf_feature *feature)/* switch "initial-off" features on, and "initial-on" features * off; i.e. negate the initial value. */ - *feature->conf_value = !feature->initial_value; +if (!feature->feature_check || feature->feature_check ()) + *feature->conf_value = !feature->initial_value; }static void
show_supported_btf_features(FILE *output)pahole.c +2
-0@@ -1256,6 +1281,8 @@ static void show_supported_btf_features(FILE *output)int i;for (i =0 ; i <ARRAY_SIZE (btf_features); i++) { +if (btf_features[i].feature_check && !btf_features[i].feature_check ()) +continue ;if (i >0 )fprintf (output,"," );fprintf (output,"%s" , btf_features[i].name);
Ihor Solodrai
reviewerQuoted 27 lines · Ihor Solodrai
Quoted 16 lines · Alan Maguire
[...]
hi Ihor, I took a look at the series and merged it with latest next branch; results are in
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4
...if you want to take a look.
There are a few small things I think that it would be good to resolve before landing this.
First, when testing this with -DLIBBPF_EMBEDDED=OFF and a packaged libbpf 1.5 - which means we wouldn't have the latest attributes-related libbpf function; I saw:
Quoted 5 lines
BTF .tmp_vmlinux1.btf.o
btf__add_type_attris not available, is libbpf < 1.6?
error: failed to encode function 'bbr_cwnd_event': invalid proto Failed to encode BTF
NM .tmp_vmlinux1.symsHi Alan. Thanks for testing. This is my mistake, I should've checked for attributes feature here:
Review static int32_t btf_encoder__add_func_proto(struct btf_encoder *encoder, struct f · line 817
Quoted 6 lines
(quoted diff) +3
-0@@ -731,6 +812,10 @@ static int32_t btf_encoder__add_func_proto(struct btf_encoder *encoder, struct f assert(ftype != NULL || state != NULL); + if (is_kfunc_state(state) && encoder->tag_kfuncs) + if (btf__add_bpf_arena_type_tags(encoder->btf, state) < 0) + return -1;
Actually, I added this check in a different patch so the failure must have happened in a different place.
In any case, the point remains that it's better to check for feature availability (hence for API availability) in one place. Your suggestion to add a feature check makes sense to me. Thank you.
Quoted 1 line
[...]
Quoted 38 lines · Ihor Solodrai
Quoted 27 lines · Ihor Solodrai
Quoted 16 lines · Alan Maguire
[...]
hi Ihor, I took a look at the series and merged it with latest next branch; results are in
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4
...if you want to take a look.
There are a few small things I think that it would be good to resolve before landing this.
First, when testing this with -DLIBBPF_EMBEDDED=OFF and a packaged libbpf 1.5 - which means we wouldn't have the latest attributes-related libbpf function; I saw:
Quoted 5 lines
BTF .tmp_vmlinux1.btf.o
btf__add_type_attris not available, is libbpf < 1.6?
error: failed to encode function 'bbr_cwnd_event': invalid proto Failed to encode BTF
NM .tmp_vmlinux1.symsHi Alan. Thanks for testing. This is my mistake, I should've checked for attributes feature here:
Review static int32_t btf_encoder__add_func_proto(struct btf_encoder *encoder, struct f · line 817
Quoted 6 lines
(quoted diff) +3
-0@@ -731,6 +812,10 @@ static int32_t btf_encoder__add_func_proto(struct btf_encoder *encoder, struct f assert(ftype != NULL || state != NULL); + if (is_kfunc_state(state) && encoder->tag_kfuncs) + if (btf__add_bpf_arena_type_tags(encoder->btf, state) < 0) + return -1;Actually, I added this check in a different patch so the failure must have happened in a different place.
Quoted 4 lines
In any case, the point remains that it's better to check for feature availability (hence for API availability) in one place. Your suggestion to add a feature check makes sense to me. Thank you.
Great; so let's do this to land the series. Could you either
- check I merged your patches correctly in the above branch, and if they look good I'll merge them into next and I'll officially send the feature check patch; or if you'd prefer
- send a v5 (perhaps including my feature check patch?)
...whichever approach is easiest for you.
Thanks!
Alan
Ihor Solodrai
reviewerQuoted 11 lines · Alan Maguire
[...]
Great; so let's do this to land the series. Could you either
- check I merged your patches correctly in the above branch, and if they look good I'll merge them into next and I'll officially send the feature check patch; or if you'd prefer
- send a v5 (perhaps including my feature check patch?)...whichever approach is easiest for you.
Hi Alan.
I reviewed the diff between your branch:
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4
and v1.29 + my patchset:
https://github.com/theihor/dwarves/tree/v4.kf-arena
Not a lot of difference besides your patch.
Didn't spot any problems.
I also ran a couple of tests on your branch:
* generate BTF with and without --btf_feature=attributes
* run ./tests/tests on 6.14-rc3 vmlinux (just a build I had at hand)
I think you can apply patches from next.attributes-v4 as is.
Thank you.
Quoted 4 lines
Thanks!
Alan
Quoted 24 lines · Ihor Solodrai
Quoted 11 lines · Alan Maguire
[...]
Great; so let's do this to land the series. Could you either
- check I merged your patches correctly in the above branch, and if they look good I'll merge them into next and I'll officially send the feature check patch; or if you'd prefer
- send a v5 (perhaps including my feature check patch?)...whichever approach is easiest for you.
Hi Alan.
I reviewed the diff between your branch:
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4and v1.29 + my patchset:
https://github.com/theihor/dwarves/tree/v4.kf-arenaNot a lot of difference besides your patch.
Didn't spot any problems.
Quoted 8 lines
I also ran a couple of tests on your branch:
* generate BTF with and without --btf_feature=attributes
* run ./tests/tests on 6.14-rc3 vmlinux (just a build I had at hand)I think you can apply patches from next.attributes-v4 as is.
Thank you.
will do; can I add your Acked-by to the feature check patch? Thanks!
Alan
Ihor Solodrai
reviewerQuoted 29 lines · Alan Maguire
Quoted 28 lines · Ihor Solodrai
Quoted 11 lines · Alan Maguire
[...]
Great; so let's do this to land the series. Could you either
- check I merged your patches correctly in the above branch, and if they look good I'll merge them into next and I'll officially send the feature check patch; or if you'd prefer
- send a v5 (perhaps including my feature check patch?)...whichever approach is easiest for you.
Hi Alan.
I reviewed the diff between your branch:
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4and v1.29 + my patchset:
https://github.com/theihor/dwarves/tree/v4.kf-arenaNot a lot of difference besides your patch.
Didn't spot any problems.I also ran a couple of tests on your branch:
* generate BTF with and without --btf_feature=attributes
* run ./tests/tests on 6.14-rc3 vmlinux (just a build I had at hand)
Quoted 6 lines
Quoted 4 lines
I think you can apply patches from next.attributes-v4 as is.
Thank you.
will do; can I add your Acked-by to the feature check patch? Thanks!
I left an Acked-by here:
https://lore.kernel.org/dwarves/[email protected]/
Quoted 2 lines
Alan
Quoted 32 lines · Ihor Solodrai
Quoted 31 lines · Alan Maguire
Quoted 30 lines · Ihor Solodrai
Quoted 11 lines · Alan Maguire
[...]
Great; so let's do this to land the series. Could you either
- check I merged your patches correctly in the above branch, and if they look good I'll merge them into next and I'll officially send the feature check patch; or if you'd prefer
- send a v5 (perhaps including my feature check patch?)...whichever approach is easiest for you.
Hi Alan.
I reviewed the diff between your branch:
https://web.git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next.attributes-v4and v1.29 + my patchset:
https://github.com/theihor/dwarves/tree/v4.kf-arenaNot a lot of difference besides your patch.
Didn't spot any problems.I also ran a couple of tests on your branch:
* generate BTF with and without --btf_feature=attributes
* run ./tests/tests on 6.14-rc3 vmlinux (just a build I had at hand)I think you can apply patches from next.attributes-v4 as is.
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Thank you.
will do; can I add your Acked-by to the feature check patch? Thanks!
I left an Acked-by here:
https://lore.kernel.org/dwarves/[email protected]/
series applied to next branch at
https://git.kernel.org/pub/scm/devel/pahole/pahole.git , thanks!
Alan
Quoted 2 lines
Quoted 2 lines
Alan