btf: support BTF_KIND_ENUM64
Subsystems (2)
v3 cover btf: support BTF_KIND_ENUM64
Add support for enum64. For 64-bit enumerator value, previously, the value is truncated into 32bit, e.g., for the following enum in linux uapi bpf.h,
enum {
BPF_F_INDEX_MASK = 0xffffffffULL,
BPF_F_CURRENT_CPU = BPF_F_INDEX_MASK,
/* BPF_FUNC_perf_event_output for sk_buff input context. */
BPF_F_CTXLEN_MASK = (0xfffffULL << 32),
};
BPF_F_CTXLEN_MASK will be encoded with 0 with BTF_KIND_ENUM after pahole dwarf-to-btf conversion.
With this patch, the BPF_F_CTXLEN_MASK will be encoded properly with BTF_KIND_ENUM64.
This patch is on top of tmp.master since tmp.master has not been sync'ed with master branch yet.
Changelogs:
v2 -> v3:
- pass struct type/conf_load pointers to btf_encoder__add_enum[_value] to make code easier to understand.
v1 -> v2:
- Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.
Yonghong Song (2):
libbpf: Sync with latest libbpf repo
btf: Support BTF_KIND_ENUM64
btf_encoder.c | 67 +++++++++++++++++++++++++++++++++++------------ btf_encoder.h | 2 +- dwarf_loader.c | 12 +++++++++ dwarves.h | 4 ++- dwarves_fprintf.c | 6 ++++- lib/bpf | 2 +- pahole.c | 10 ++++++- 7 files changed, 81 insertions(+), 22 deletions(-)
Signature · 1 line
2.30.2
PATCH v3 1/2 libbpf: Sync with latest libbpf repo
Sync up to commit
645500dd7d2d ci: blacklist mptcp test on s390x
--- lib/bpf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
lib/bpf +1 -1
@@ -1 +1 @@ -Subproject commit 87dff0a2c775c5943ca9233e69c81a25f2ed1a77 +Subproject commit 645500dd7d2d6b5bb76e4c0375d597d4f0c4814e
Signature · 1 line
2.30.2
PATCH v3 2/2 btf: Support BTF_KIND_ENUM64
In reply to Yonghong SongBTF_KIND_ENUM64 is supported with latest libbpf, which supports 64-bit enum values. Latest libbpf also supports signedness for enum values. Add enum64 support in dwarf-to-btf conversion.
The following is an example of new encoding which covers signed/unsigned enum64/enum variations.
$cat t.c
enum { /* signed, enum64 */
A = -1,
B = 0xffffffff,
} g1;
enum { /* unsigned, enum64 */
C = 1,
D = 0xfffffffff,
} g2;
enum { /* signed, enum */
E = -1,
F = 0xfffffff,
} g3;
enum { /* unsigned, enum */
G = 1,
H = 0xfffffff,
} g4;
$ clang -g -c t.c
$ pahole -JV t.o
btf_encoder__new: 't.o' doesn't have '.data..percpu' section
Found 0 per-CPU variables!
File t.o:
[1] ENUM64 (anon) size=8
A val=-1
B val=4294967295
[2] INT long size=8 nr_bits=64 encoding=SIGNED
[3] ENUM64 (anon) size=8
C val=1
D val=68719476735
[4] INT unsigned long size=8 nr_bits=64 encoding=(none)
[5] ENUM (anon) size=4
E val=-1
F val=268435455
[6] INT int size=4 nr_bits=32 encoding=SIGNED
[7] ENUM (anon) size=4
G val=1
H val=268435455
[8] INT unsigned int size=4 nr_bits=32 encoding=(none)With the flag to skip enum64 encoding,
$ pahole -JV t.o --skip_encoding_btf_enum64
btf_encoder__new: 't.o' doesn't have '.data..percpu' section
Found 0 per-CPU variables!
File t.o:
[1] ENUM (anon) size=8
A val=4294967295
B val=4294967295
[2] INT long size=8 nr_bits=64 encoding=SIGNED
[3] ENUM (anon) size=8
C val=1
D val=4294967295
[4] INT unsigned long size=8 nr_bits=64 encoding=(none)
[5] ENUM (anon) size=4
E val=4294967295
F val=268435455
[6] INT int size=4 nr_bits=32 encoding=SIGNED
[7] ENUM (anon) size=4
G val=1
H val=268435455
[8] INT unsigned int size=4 nr_bits=32 encoding=(none)In the above btf encoding without enum64, all enum types with the same type size as the corresponding enum64. All these enum types have unsigned type (kflag = 0) which is required before kernel enum64 support.
--- btf_encoder.c | 67 +++++++++++++++++++++++++++++++++++------------ btf_encoder.h | 2 +- dwarf_loader.c | 12 +++++++++ dwarves.h | 4 ++- dwarves_fprintf.c | 6 ++++- pahole.c | 10 ++++++- 6 files changed, 80 insertions(+), 21 deletions(-)
btf_encoder.c +50 -17
@@ -144,6 +144,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_FLOAT] = "FLOAT", [BTF_KIND_DECL_TAG] = "DECL_TAG", [BTF_KIND_TYPE_TAG] = "TYPE_TAG", + [BTF_KIND_ENUM64] = "ENUM64", };static const char *btf__printable_name (const struct btf *btf,uint32_t offset)
@@ -490,34 +491,64 @@ static int32_t btf_encoder__add_struct(struct btf_encoder *encoder, uint8_t kindreturn id; } -static int32_t btf_encoder__add_enum (struct btf_encoder *encoder,const char *name,uint32_t bit_size) +static int32_t btf_encoder__add_enum (struct btf_encoder *encoder,const char *name,struct type *etype, +struct conf_load *conf_load) {struct btf *btf = encoder->btf;const struct btf_type *t;int32_t id, size; +bool is_enum32; - size =BITS_ROUNDUP_BYTES (bit_size); - id =btf__add_enum (btf, name, size); + size =BITS_ROUNDUP_BYTES (etype->size); + is_enum32 = size <=4 || conf_load->skip_encoding_btf_enum64; +if (is_enum32) + id =btf__add_enum (btf, name, size); +else + id =btf__add_enum64 (btf, name, size, etype->is_signed_enum);if (id >0 ) { t =btf__type_by_id (btf, id);btf_encoder__log_type (encoder, t, false, true,"size=%u" , t->size); }else { -btf__log_err (btf,BTF_KIND_ENUM , name, true, +btf__log_err (btf, is_enum32 ?BTF_KIND_ENUM :BTF_KIND_ENUM64 , name, true,"size=%u Error emitting BTF type" , size); }return id; } -static int btf_encoder__add_enum_val (struct btf_encoder *encoder,const char *name,int32_t value) +static int btf_encoder__add_enum_val (struct btf_encoder *encoder,const char *name,int64_t value, +struct type *etype,struct conf_load *conf_load) { -int err =btf__add_enum_value (encoder->btf, name, value); +const char *fmt_str; +int err; + +/* If enum64 is not allowed, generate enum32 with unsigned int value. In enum64-supported +* libbpf library, btf__add_enum_value() will set the kflag (sign bit) in common_type +* if the value is negative. +*/ +if (conf_load->skip_encoding_btf_enum64) + err =btf__add_enum_value (encoder->btf, name, (uint32_t )value); +else if (etype->size >32 ) + err =btf__add_enum64_value (encoder->btf, name, value); +else + err =btf__add_enum_value (encoder->btf, name, value);if (!err) { -if (encoder->verbose) -printf ("\t%s val=%d\n" , name, value); +if (encoder->verbose) { +if (conf_load->skip_encoding_btf_enum64) { +printf ("\t%s val=%u\n" , name, (uint32_t )value); + }else { + fmt_str = etype->is_signed_enum ?"\t%s val=%lld\n" :"\t%s val=%llu\n" ; +printf (fmt_str, name, (unsigned long long )value); + } + } }else { -fprintf (stderr,"\t%s val=%d Error emitting BTF enum value\n" , - name, value); +if (conf_load->skip_encoding_btf_enum64) { +fprintf (stderr,"\t%s val=%u Error emitting BTF enum value\n" , name, (uint32_t )value); + }else { + fmt_str = etype->is_signed_enum ?"\t%s val=%lld Error emitting BTF enum value\n" + :"\t%s val=%llu Error emitting BTF enum value\n" ; +fprintf (stderr, fmt_str, name, (unsigned long long )value); + } }return err; }
@@ -844,27 +875,29 @@ static uint32_t array_type__nelems(struct tag *tag)return nelem; } -static int32_t btf_encoder__add_enum_type (struct btf_encoder *encoder,struct tag *tag) +static int32_t btf_encoder__add_enum_type (struct btf_encoder *encoder,struct tag *tag, +struct conf_load *conf_load) {struct type *etype =tag__type (tag);struct enumerator *pos;const char *name =type__name (etype);int32_t type_id; - type_id =btf_encoder__add_enum (encoder, name, etype->size); + type_id =btf_encoder__add_enum (encoder, name, etype, conf_load);if (type_id <0 )return type_id;type__for_each_enumerator (etype, pos) { name =enumerator__name (pos); -if (btf_encoder__add_enum_val (encoder, name, pos->value)) +if (btf_encoder__add_enum_val (encoder, name, pos->value, etype, conf_load))return -1 ; }return type_id; } -static int btf_encoder__encode_tag (struct btf_encoder *encoder,struct tag *tag,uint32_t type_id_off) +static int btf_encoder__encode_tag (struct btf_encoder *encoder,struct tag *tag,uint32_t type_id_off, +struct conf_load *conf_load) {/* single out type 0 as it represents special type "void" */ uint32_t ref_type_id = tag->type ==0 ?0 : type_id_off + tag->type;
@@ -903,7 +936,7 @@ static int btf_encoder__encode_tag(struct btf_encoder *encoder, struct tag *tag, encoder->need_index_type = true;return btf_encoder__add_array (encoder, ref_type_id, encoder->array_index_id,array_type__nelems (tag));case DW_TAG_enumeration_type: -return btf_encoder__add_enum_type (encoder, tag); +return btf_encoder__add_enum_type (encoder, tag, conf_load);case DW_TAG_subroutine_type:return btf_encoder__add_func_proto (encoder,tag__ftype (tag), type_id_off);default :
@@ -1422,7 +1455,7 @@ void btf_encoder__delete(struct btf_encoder *encoder)free (encoder); } -int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu) +int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu,struct conf_load *conf_load) {uint32_t type_id_off =btf__type_cnt (encoder->btf) -1 ;struct llvm_annotation *annot;
@@ -1446,7 +1479,7 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu) }cu__for_each_type (cu, core_id, pos) { - btf_type_id =btf_encoder__encode_tag (encoder, pos, type_id_off); + btf_type_id =btf_encoder__encode_tag (encoder, pos, type_id_off, conf_load);if (btf_type_id <0 ||tag__check_id_drift (pos, core_id, btf_type_id, type_id_off)) {
btf_encoder.h +1 -1
@@ -21,7 +21,7 @@ void btf_encoder__delete(struct btf_encoder *encoder);int btf_encoder__encode (struct btf_encoder *encoder); -int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu); +int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu,struct conf_load *conf_load);void btf_encoders__add (struct list_head *encoders,struct btf_encoder *encoder);
dwarf_loader.c +12 -0
@@ -632,6 +632,18 @@ static void type__init(struct type *type, Dwarf_Die *die, struct cu *cu, struct type->resized =0 ; type->nr_members =0 ; type->nr_static_members =0 ; + type->is_signed_enum =0 ; + +Dwarf_Attribute attr; +if (dwarf_attr (die, DW_AT_type, &attr) !=NULL ) { +Dwarf_Die type_die; +if (dwarf_formref_die (&attr, &type_die) !=NULL ) { +uint64_t encoding =attr_numeric (&type_die, DW_AT_encoding); + +if (encoding == DW_ATE_signed || encoding == DW_ATE_signed_char) + type->is_signed_enum =1 ; + } + } }static struct type *type__new (Dwarf_Die *die,struct cu *cu,struct conf_load *conf)
dwarves.h +3 -1
@@ -65,6 +65,7 @@ struct conf_load {bool skip_encoding_btf_decl_tag;bool skip_missing;bool skip_encoding_btf_type_tag; +bool skip_encoding_btf_enum64;uint8_t hashtable_bits;uint8_t max_hashtable_bits;uint16_t kabi_prefix_len;
@@ -1046,6 +1047,7 @@ struct type { uint8_t definition_emitted:1; uint8_t fwd_decl_emitted:1; uint8_t resized:1; + uint8_t is_signed_enum:1; };void __type__init (struct type *type);
@@ -1365,7 +1367,7 @@ static inline struct string_type *tag__string_type(const struct tag *tag)struct enumerator {struct tag tag;const char *name; -uint32_t value; +uint64_t value;struct tag_cu type_enum;// To cache the type_enum searches };
dwarves_fprintf.c +5 -1
@@ -437,7 +437,11 @@ size_t enumeration__fprintf(const struct tag *tag, const struct conf_fprintf *cotype__for_each_enumerator (type, pos) { printed +=fprintf (fp,"%.*s\t%-*s = " , indent, tabs, max_entry_name_len,enumerator__name (pos)); - printed +=fprintf (fp, conf->hex_fmt ?"%#x" :"%u" , pos->value); +if (conf->hex_fmt) + printed +=fprintf (fp,"%#llx" , (unsigned long long )pos->value); +else + printed +=fprintf (fp, type->is_signed_enum ?"%lld" :"%llu" , + (unsigned long long )pos->value); printed +=fprintf (fp,",\n" ); }
pahole.c +9 -1
@@ -1220,6 +1220,7 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;#define ARGP_compile 334#define ARGP_languages 335#define ARGP_languages_exclude 336 +#define ARGP_skip_encoding_btf_enum64 337static const struct argp_option pahole__options[] = { {
@@ -1622,6 +1623,11 @@ static const struct argp_option pahole__options[] = { .arg = "LANGUAGES", .doc = "Don't consider compilation units written in these languages" }, + { + .name = "skip_encoding_btf_enum64", + .key = ARGP_skip_encoding_btf_enum64, + .doc = "Do not encode ENUM64sin BTF." + }, { .name = NULL, }
@@ -1787,6 +1793,8 @@ static error_t pahole__options_parser(int key, char *arg,/* fallthru */ case ARGP_languages: languages.str = arg;break ; +case ARGP_skip_encoding_btf_enum64: + conf_load.skip_encoding_btf_enum64 = true;break ;default :return ARGP_ERR_UNKNOWN ; }
@@ -3067,7 +3075,7 @@ static enum load_steal_kind pahole_stealer(struct cu *cu, encoder = btf_encoder; } -if (btf_encoder__encode_cu (encoder, cu)) { +if (btf_encoder__encode_cu (encoder, cu, conf_load)) {fprintf (stderr,"Encountered error while encoding BTF.\n" );exit (1 ); }
Signature · 1 line
2.30.2
Re: [PATCH dwarves v3 2/2] btf: Support BTF_KIND_ENUM64
Em Wed, Jun 29, 2022 at 12:12:24AM -0700, Yonghong Song escreveu:
Quoted 7 lines
BTF_KIND_ENUM64is supported with latest libbpf, which supports 64-bit enum values. Latest libbpf also supports signedness for enum values. Add enum64 support in dwarf-to-btf conversion.The following is an example of new encoding which covers signed/unsigned enum64/enum variations.
So, testing this with torvalds/master I'm getting:
FAILED: load BTF from vmlinux: Invalid argument
make[1]: *** [/var/home/acme/git/linux/Makefile:1164: vmlinux] Error 255 make[1]: *** Deleting file 'vmlinux'
make[1]: Leaving directory '/var/home/acme/git/build/v5.19-rc4+'
make: *** [Makefile:219: __sub-make] Error 2
real 8m12.396s
user 183m18.009s
sys 44m27.085s
⬢[acme@toolbox linux]$ find . -name "*.c" | xargs grep "load BTF from vmlinux" ⬢[acme@toolbox linux]$ find . -name "*.c" | xargs grep "load BTF from"./tools/bpf/bpftool/btf.c: p_err("failed to load BTF from %s: %s",./tools/bpf/resolve_btfids/main.c: pr_err("FAILED: load BTF from %s: %s\n",./tools/testing/selftests/bpf/prog_tests/resolve_btfids.c: "Failed to load BTF from btf_data.o\n"))
⬢[acme@toolbox linux]$ vim ./tools/bpf/resolve_btfids/main.c
Which is:
btf = btf__parse_split(obj->btf ?: obj->path, base_btf);
err = libbpf_get_error(btf);
if (err) {
pr_err("FAILED: load BTF from %s: %s\n",
obj->btf ?: obj->path, strerror(-err));
goto out;
}I.e. tools/lib/bpf in torvalds/master doesn´t support BTF_KIND_ENUM64, right? So to build it with a new pahole one needs to ask for --skip_encoding_btf_enum64? How to do this automagically? I.e. its a matter of checking if the in-kernel libbpf has support for it and if not use --skip_encoding_btf_enum64?
I'm now going to try with bpf-next/master
- Arnaldo
Quoted 352 lines
$cat t.c enum { /* signed, enum64 */ A = -1, B = 0xffffffff, } g1; enum { /* unsigned, enum64 */ C = 1, D = 0xfffffffff, } g2; enum { /* signed, enum */ E = -1, F = 0xfffffff, } g3; enum { /* unsigned, enum */ G = 1, H = 0xfffffff, } g4; $ clang -g -c t.c $ pahole -JV t.o btf_encoder__new: 't.o' doesn't have '.data..percpu' section Found 0 per-CPU variables! File t.o: [1] ENUM64 (anon) size=8 A val=-1 B val=4294967295 [2] INT long size=8 nr_bits=64 encoding=SIGNED [3] ENUM64 (anon) size=8 C val=1 D val=68719476735 [4] INT unsigned long size=8 nr_bits=64 encoding=(none) [5] ENUM (anon) size=4 E val=-1 F val=268435455 [6] INT int size=4 nr_bits=32 encoding=SIGNED [7] ENUM (anon) size=4 G val=1 H val=268435455 [8] INT unsigned int size=4 nr_bits=32 encoding=(none)With the flag to skip enum64 encoding,
$ pahole -JV t.o --skip_encoding_btf_enum64 btf_encoder__new: 't.o' doesn't have '.data..percpu' section Found 0 per-CPU variables! File t.o: [1] ENUM (anon) size=8 A val=4294967295 B val=4294967295 [2] INT long size=8 nr_bits=64 encoding=SIGNED [3] ENUM (anon) size=8 C val=1 D val=4294967295 [4] INT unsigned long size=8 nr_bits=64 encoding=(none) [5] ENUM (anon) size=4 E val=4294967295 F val=268435455 [6] INT int size=4 nr_bits=32 encoding=SIGNED [7] ENUM (anon) size=4 G val=1 H val=268435455 [8] INT unsigned int size=4 nr_bits=32 encoding=(none)In the above btf encoding without enum64, all enum types with the same type size as the corresponding enum64. All these enum types have unsigned type (kflag = 0) which is required before kernel enum64 support.
Acked-by Andrii Nakryiko <[email protected]>Signed-off-by Yonghong Song <[email protected]>--- btf_encoder.c | 67 +++++++++++++++++++++++++++++++++++------------ btf_encoder.h | 2 +- dwarf_loader.c | 12 +++++++++ dwarves.h | 4 ++- dwarves_fprintf.c | 6 ++++- pahole.c | 10 ++++++- 6 files changed, 80 insertions(+), 21 deletions(-)btf_encoder.c +50
-17@@ -144,6 +144,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = { [BTF_KIND_FLOAT] = "FLOAT", [BTF_KIND_DECL_TAG] = "DECL_TAG", [BTF_KIND_TYPE_TAG] = "TYPE_TAG", + [BTF_KIND_ENUM64] = "ENUM64", };static const char *btf__printable_name (const struct btf *btf,uint32_t offset)@@ -490,34 +491,64 @@ static int32_t btf_encoder__add_struct(struct btf_encoder *encoder, uint8_t kindreturn id; } -static int32_t btf_encoder__add_enum (struct btf_encoder *encoder,const char *name,uint32_t bit_size) +static int32_t btf_encoder__add_enum (struct btf_encoder *encoder,const char *name,struct type *etype, +struct conf_load *conf_load) {struct btf *btf = encoder->btf;const struct btf_type *t;int32_t id, size; +bool is_enum32; - size =BITS_ROUNDUP_BYTES (bit_size); - id =btf__add_enum (btf, name, size); + size =BITS_ROUNDUP_BYTES (etype->size); + is_enum32 = size <=4 || conf_load->skip_encoding_btf_enum64; +if (is_enum32) + id =btf__add_enum (btf, name, size); +else + id =btf__add_enum64 (btf, name, size, etype->is_signed_enum);if (id >0 ) { t =btf__type_by_id (btf, id);btf_encoder__log_type (encoder, t, false, true,"size=%u" , t->size); }else { -btf__log_err (btf,BTF_KIND_ENUM , name, true, +btf__log_err (btf, is_enum32 ?BTF_KIND_ENUM :BTF_KIND_ENUM64 , name, true,"size=%u Error emitting BTF type" , size); }return id; } -static int btf_encoder__add_enum_val (struct btf_encoder *encoder,const char *name,int32_t value) +static int btf_encoder__add_enum_val (struct btf_encoder *encoder,const char *name,int64_t value, +struct type *etype,struct conf_load *conf_load) { -int err =btf__add_enum_value (encoder->btf, name, value); +const char *fmt_str; +int err; + +/* If enum64 is not allowed, generate enum32 with unsigned int value. In enum64-supported +* libbpf library, btf__add_enum_value() will set the kflag (sign bit) in common_type +* if the value is negative. +*/ +if (conf_load->skip_encoding_btf_enum64) + err =btf__add_enum_value (encoder->btf, name, (uint32_t )value); +else if (etype->size >32 ) + err =btf__add_enum64_value (encoder->btf, name, value); +else + err =btf__add_enum_value (encoder->btf, name, value);if (!err) { -if (encoder->verbose) -printf ("\t%s val=%d\n" , name, value); +if (encoder->verbose) { +if (conf_load->skip_encoding_btf_enum64) { +printf ("\t%s val=%u\n" , name, (uint32_t )value); + }else { + fmt_str = etype->is_signed_enum ?"\t%s val=%lld\n" :"\t%s val=%llu\n" ; +printf (fmt_str, name, (unsigned long long )value); + } + } }else { -fprintf (stderr,"\t%s val=%d Error emitting BTF enum value\n" , - name, value); +if (conf_load->skip_encoding_btf_enum64) { +fprintf (stderr,"\t%s val=%u Error emitting BTF enum value\n" , name, (uint32_t )value); + }else { + fmt_str = etype->is_signed_enum ?"\t%s val=%lld Error emitting BTF enum value\n" + :"\t%s val=%llu Error emitting BTF enum value\n" ; +fprintf (stderr, fmt_str, name, (unsigned long long )value); + } }return err; }@@ -844,27 +875,29 @@ static uint32_t array_type__nelems(struct tag *tag)return nelem; } -static int32_t btf_encoder__add_enum_type (struct btf_encoder *encoder,struct tag *tag) +static int32_t btf_encoder__add_enum_type (struct btf_encoder *encoder,struct tag *tag, +struct conf_load *conf_load) {struct type *etype =tag__type (tag);struct enumerator *pos;const char *name =type__name (etype);int32_t type_id; - type_id =btf_encoder__add_enum (encoder, name, etype->size); + type_id =btf_encoder__add_enum (encoder, name, etype, conf_load);if (type_id <0 )return type_id;type__for_each_enumerator (etype, pos) { name =enumerator__name (pos); -if (btf_encoder__add_enum_val (encoder, name, pos->value)) +if (btf_encoder__add_enum_val (encoder, name, pos->value, etype, conf_load))return -1 ; }return type_id; } -static int btf_encoder__encode_tag (struct btf_encoder *encoder,struct tag *tag,uint32_t type_id_off) +static int btf_encoder__encode_tag (struct btf_encoder *encoder,struct tag *tag,uint32_t type_id_off, +struct conf_load *conf_load) {/* single out type 0 as it represents special type "void" */ uint32_t ref_type_id = tag->type ==0 ?0 : type_id_off + tag->type;@@ -903,7 +936,7 @@ static int btf_encoder__encode_tag(struct btf_encoder *encoder, struct tag *tag, encoder->need_index_type = true;return btf_encoder__add_array (encoder, ref_type_id, encoder->array_index_id,array_type__nelems (tag));case DW_TAG_enumeration_type: -return btf_encoder__add_enum_type (encoder, tag); +return btf_encoder__add_enum_type (encoder, tag, conf_load);case DW_TAG_subroutine_type:return btf_encoder__add_func_proto (encoder,tag__ftype (tag), type_id_off);default :@@ -1422,7 +1455,7 @@ void btf_encoder__delete(struct btf_encoder *encoder)free (encoder); } -int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu) +int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu,struct conf_load *conf_load) {uint32_t type_id_off =btf__type_cnt (encoder->btf) -1 ;struct llvm_annotation *annot;@@ -1446,7 +1479,7 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu) }cu__for_each_type (cu, core_id, pos) { - btf_type_id =btf_encoder__encode_tag (encoder, pos, type_id_off); + btf_type_id =btf_encoder__encode_tag (encoder, pos, type_id_off, conf_load);if (btf_type_id <0 ||tag__check_id_drift (pos, core_id, btf_type_id, type_id_off)) {btf_encoder.h +1
-1@@ -21,7 +21,7 @@ void btf_encoder__delete(struct btf_encoder *encoder);int btf_encoder__encode (struct btf_encoder *encoder); -int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu); +int btf_encoder__encode_cu (struct btf_encoder *encoder,struct cu *cu,struct conf_load *conf_load);void btf_encoders__add (struct list_head *encoders,struct btf_encoder *encoder);dwarf_loader.c +12
-0@@ -632,6 +632,18 @@ static void type__init(struct type *type, Dwarf_Die *die, struct cu *cu, struct type->resized =0 ; type->nr_members =0 ; type->nr_static_members =0 ; + type->is_signed_enum =0 ; + +Dwarf_Attribute attr; +if (dwarf_attr (die, DW_AT_type, &attr) !=NULL ) { +Dwarf_Die type_die; +if (dwarf_formref_die (&attr, &type_die) !=NULL ) { +uint64_t encoding =attr_numeric (&type_die, DW_AT_encoding); + +if (encoding == DW_ATE_signed || encoding == DW_ATE_signed_char) + type->is_signed_enum =1 ; + } + } }static struct type *type__new (Dwarf_Die *die,struct cu *cu,struct conf_load *conf)dwarves.h +3
-1@@ -65,6 +65,7 @@ struct conf_load {bool skip_encoding_btf_decl_tag;bool skip_missing;bool skip_encoding_btf_type_tag; +bool skip_encoding_btf_enum64;uint8_t hashtable_bits;uint8_t max_hashtable_bits;uint16_t kabi_prefix_len;@@ -1046,6 +1047,7 @@ struct type { uint8_t definition_emitted:1; uint8_t fwd_decl_emitted:1; uint8_t resized:1; + uint8_t is_signed_enum:1; };void __type__init (struct type *type);@@ -1365,7 +1367,7 @@ static inline struct string_type *tag__string_type(const struct tag *tag)struct enumerator {struct tag tag;const char *name; -uint32_t value; +uint64_t value;struct tag_cu type_enum;// To cache the type_enum searches };dwarves_fprintf.c +5
-1@@ -437,7 +437,11 @@ size_t enumeration__fprintf(const struct tag *tag, const struct conf_fprintf *cotype__for_each_enumerator (type, pos) { printed +=fprintf (fp,"%.*s\t%-*s = " , indent, tabs, max_entry_name_len,enumerator__name (pos)); - printed +=fprintf (fp, conf->hex_fmt ?"%#x" :"%u" , pos->value); +if (conf->hex_fmt) + printed +=fprintf (fp,"%#llx" , (unsigned long long )pos->value); +else + printed +=fprintf (fp, type->is_signed_enum ?"%lld" :"%llu" , + (unsigned long long )pos->value); printed +=fprintf (fp,",\n" ); }pahole.c +9
-1@@ -1220,6 +1220,7 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;#define ARGP_compile 334#define ARGP_languages 335#define ARGP_languages_exclude 336 +#define ARGP_skip_encoding_btf_enum64 337static const struct argp_option pahole__options[] = { {@@ -1622,6 +1623,11 @@ static const struct argp_option pahole__options[] = { .arg = "LANGUAGES", .doc = "Don't consider compilation units written in these languages" }, + { + .name = "skip_encoding_btf_enum64", + .key = ARGP_skip_encoding_btf_enum64, + .doc = "Do not encode ENUM64sin BTF." + }, { .name = NULL, }@@ -1787,6 +1793,8 @@ static error_t pahole__options_parser(int key, char *arg,/* fallthru */ case ARGP_languages: languages.str = arg;break ; +case ARGP_skip_encoding_btf_enum64: + conf_load.skip_encoding_btf_enum64 = true;break ;default :return ARGP_ERR_UNKNOWN ; }@@ -3067,7 +3075,7 @@ static enum load_steal_kind pahole_stealer(struct cu *cu, encoder = btf_encoder; } -if (btf_encoder__encode_cu (encoder, cu)) { +if (btf_encoder__encode_cu (encoder, cu, conf_load)) {fprintf (stderr,"Encountered error while encoding BTF.\n" );exit (1 ); }Signature · 1 line
2.30.2
Signature · 1 line
- Arnaldo
Re: [PATCH dwarves v3 2/2] btf: Support BTF_KIND_ENUM64
Em Wed, Jun 29, 2022 at 04:12:19PM -0300, Arnaldo Carvalho de Melo escreveu:
Quoted 37 lines
Em Wed, Jun 29, 2022 at 12:12:24AM -0700, Yonghong Song escreveu:
Quoted 7 lines
BTF_KIND_ENUM64is supported with latest libbpf, which supports 64-bit enum values. Latest libbpf also supports signedness for enum values. Add enum64 support in dwarf-to-btf conversion.The following is an example of new encoding which covers signed/unsigned enum64/enum variations.
So, testing this with torvalds/master I'm getting:
FAILED: load BTF from vmlinux: Invalid argument
make[1]: *** [/var/home/acme/git/linux/Makefile:1164: vmlinux] Error 255 make[1]: *** Deleting file 'vmlinux'
make[1]: Leaving directory '/var/home/acme/git/build/v5.19-rc4+'
make: *** [Makefile:219: __sub-make] Error 2real 8m12.396s
user 183m18.009s
sys 44m27.085s
⬢[acme@toolbox linux]$ find . -name "*.c" | xargs grep "load BTF from vmlinux" ⬢[acme@toolbox linux]$ find . -name "*.c" | xargs grep "load BTF from"./tools/bpf/bpftool/btf.c: p_err("failed to load BTF from %s: %s",./tools/bpf/resolve_btfids/main.c: pr_err("FAILED: load BTF from %s: %s\n",./tools/testing/selftests/bpf/prog_tests/resolve_btfids.c: "Failed to load BTF from btf_data.o\n"))
⬢[acme@toolbox linux]$ vim./tools/bpf/resolve_btfids/main.cWhich is:
btf = btf__parse_split(obj->btf ?: obj->path, base_btf); err = libbpf_get_error(btf); if (err) { pr_err("FAILED: load BTF from %s: %s\n", obj->btf ?: obj->path, strerror(-err)); goto out; }
Quoted 7 lines
I.e.
tools/lib/bpfin torvalds/master doesn´t supportBTF_KIND_ENUM64, right? So to build it with a new pahole one needs to ask for --skip_encoding_btf_enum64? How to do this automagically? I.e. its a matter of checking if the in-kernel libbpf has support for it and if not use --skip_encoding_btf_enum64?I'm now going to try with bpf-next/master
Yeah, works with bpf-next:
⬢[acme@toolbox bpf-next]$ pdwtags -F btf ../build/bpf-next/vmlinux | grep -A5 -B5 BPF_F_CTXLEN_MASK
/* 28548 */
enum {
BPF_F_INDEX_MASK = 4294967295,
BPF_F_CURRENT_CPU = 4294967295,
BPF_F_CTXLEN_MASK = 4503595332403200,
} __attribute__((__packed__)); /* size: 8 *//* 28549 */
enum {
BPF_F_GET_BRANCH_RECORDS_SIZE = 1,
⬢[acme@toolbox bpf-next]$
Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
Quoted 15 lines
Add support for enum64. For 64-bit enumerator value, previously, the value is truncated into 32bit, e.g., for the following enum in linux uapi
bpf.h,
enum {
BPF_F_INDEX_MASK=0xffffffffULL,
BPF_F_CURRENT_CPU=BPF_F_INDEX_MASK,
/*BPF_FUNC_perf_event_outputforsk_buffinput context. */BPF_F_CTXLEN_MASK = (0xfffffULL << 32),};
BPF_F_CTXLEN_MASKwill be encoded with 0 withBTF_KIND_ENUMafter pahole dwarf-to-btf conversion.
With this patch, theBPF_F_CTXLEN_MASKwill be encoded properly withBTF_KIND_ENUM64.
Quoted 7 lines
This patch is on top of tmp.master since tmp.master has not been sync'ed with master branch yet.
Changelogs:
v2 -> v3:
- pass struct type/conf_load pointers to btf_encoder__add_enum[_value] to make code easier to understand.
Yeah, that is more clear indeed.
- Arnaldo
Quoted 18 lines
v1 -> v2:
- Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.Yonghong Song (2):
libbpf: Sync with latest libbpf repo
btf: SupportBTF_KIND_ENUM64btf_encoder.c | 67 +++++++++++++++++++++++++++++++++++------------ btf_encoder.h | 2 +- dwarf_loader.c | 12 +++++++++ dwarves.h | 4 ++- dwarves_fprintf.c | 6 ++++- lib/bpf | 2 +- pahole.c | 10 ++++++- 7 files changed, 81 insertions(+), 22 deletions(-)Signature · 1 line
2.30.2
Signature · 1 line
- Arnaldo
Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
Quoted 9 lines
Add support for enum64. For 64-bit enumerator value, previously, the value is truncated into 32bit, e.g., for the following enum in linux uapi
bpf.h,
enum {
BPF_F_INDEX_MASK=0xffffffffULL,
BPF_F_CURRENT_CPU=BPF_F_INDEX_MASK,
/*BPF_FUNC_perf_event_outputforsk_buffinput context. */BPF_F_CTXLEN_MASK = (0xfffffULL << 32),};
Applied, added the entry for skip generating enums to the man page, added support to the pahole BTF loader, used the new pahole to build bpf-next/master, all seems ok, pushing to next on git.kernel.org so that the libbpf github CI can give it a go.
To build with torvalds/master one has to add --skip_encoding_btf_enum64, I think, haven't tested with it, without it isn't working, libbpf complains at that btfids tool.
Please check/test what is in there now:
git://git.kernel.org/pub/scm/devel/pahole/pahole.git next
https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=next
Unless someone screams I plan pushing out a new release, update fedora packages, etc early next week its overdue by now.
- Arnaldo
Quoted 30 lines
BPF_F_CTXLEN_MASKwill be encoded with 0 withBTF_KIND_ENUMafter pahole dwarf-to-btf conversion.
With this patch, theBPF_F_CTXLEN_MASKwill be encoded properly withBTF_KIND_ENUM64.This patch is on top of tmp.master since tmp.master has not been sync'ed with master branch yet.
Changelogs:
v2 -> v3:
- pass struct type/conf_load pointers to btf_encoder__add_enum[_value] to make code easier to understand.
v1 -> v2:
- Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.Yonghong Song (2):
libbpf: Sync with latest libbpf repo
btf: SupportBTF_KIND_ENUM64btf_encoder.c | 67 +++++++++++++++++++++++++++++++++++------------ btf_encoder.h | 2 +- dwarf_loader.c | 12 +++++++++ dwarves.h | 4 ++- dwarves_fprintf.c | 6 ++++- lib/bpf | 2 +- pahole.c | 10 ++++++- 7 files changed, 81 insertions(+), 22 deletions(-)Signature · 1 line
2.30.2
Signature · 1 line
- Arnaldo
Quoted 11 lines · Arnaldo Carvalho de Melo
Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
Quoted 9 lines
Add support for enum64. For 64-bit enumerator value, previously, the value is truncated into 32bit, e.g., for the following enum in linux uapi
bpf.h,
enum {
BPF_F_INDEX_MASK=0xffffffffULL,
BPF_F_CURRENT_CPU=BPF_F_INDEX_MASK,
/*BPF_FUNC_perf_event_outputforsk_buffinput context. */BPF_F_CTXLEN_MASK = (0xfffffULL << 32),};
Quoted 8 lines
Applied, added the entry for skip generating enums to the man page, added support to the pahole BTF loader, used the new pahole to build bpf-next/master, all seems ok, pushing to next on git.kernel.org so that the libbpf github CI can give it a go.
To build with torvalds/master one has to add --skip_encoding_btf_enum64, I think, haven't tested with it, without it isn't working, libbpf complains at that btfids tool.
The patch to support enum64 in btf_laoder looks good to me too. Thanks for checking and additional testing!
Quoted 42 lines
Please check/test what is in there now:
git://git.kernel.org/pub/scm/devel/pahole/pahole.gitnext
https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=nextUnless someone screams I plan pushing out a new release, update fedora packages, etc early next week its overdue by now.
- Arnaldo
Quoted 30 lines
BPF_F_CTXLEN_MASKwill be encoded with 0 withBTF_KIND_ENUMafter pahole dwarf-to-btf conversion.
With this patch, theBPF_F_CTXLEN_MASKwill be encoded properly withBTF_KIND_ENUM64.This patch is on top of tmp.master since tmp.master has not been sync'ed with master branch yet.
Changelogs:
v2 -> v3:
- pass struct type/conf_load pointers to btf_encoder__add_enum[_value] to make code easier to understand.
v1 -> v2:
- Add flag --skip_encoding_btf_enum64 to disable newly-added functionality.Yonghong Song (2):
libbpf: Sync with latest libbpf repo
btf: SupportBTF_KIND_ENUM64btf_encoder.c | 67 +++++++++++++++++++++++++++++++++++------------ btf_encoder.h | 2 +- dwarf_loader.c | 12 +++++++++ dwarves.h | 4 ++- dwarves_fprintf.c | 6 ++++- lib/bpf | 2 +- pahole.c | 10 ++++++- 7 files changed, 81 insertions(+), 22 deletions(-)Signature · 1 line
2.30.2
Quoted 20 lines · Arnaldo Carvalho de Melo
Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
Quoted 9 lines
Add support for enum64. For 64-bit enumerator value, previously, the value is truncated into 32bit, e.g., for the following enum in linux uapi
bpf.h,
enum {
BPF_F_INDEX_MASK=0xffffffffULL,
BPF_F_CURRENT_CPU=BPF_F_INDEX_MASK,
/*BPF_FUNC_perf_event_outputforsk_buffinput context. */BPF_F_CTXLEN_MASK = (0xfffffULL << 32),};
Applied, added the entry for skip generating enums to the man page, added support to the pahole BTF loader, used the new pahole to build bpf-next/master, all seems ok, pushing to next on git.kernel.org so that the libbpf github CI can give it a go.
To build with torvalds/master one has to add --skip_encoding_btf_enum64, I think, haven't tested with it, without it isn't working, libbpf complains at that btfids tool.
Quoted 7 lines
Please check/test what is in there now:
git://git.kernel.org/pub/scm/devel/pahole/pahole.gitnext
https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=nextUnless someone screams I plan pushing out a new release, update fedora packages, etc early next week its overdue by now.
I used this new pahole in kernel build and it looks ok, but I'm getting following warning:
BTFIDS vmlinux
WARN: resolve_btfids: unresolved symbol mptcp_sock
might be specific to my .config, I'll check and let you know
jirka
Jiri Olsa
Tested-byQuoted 29 lines · Jiri Olsa
Quoted 20 lines · Arnaldo Carvalho de Melo
Em Wed, Jun 29, 2022 at 12:12:13AM -0700, Yonghong Song escreveu:
Quoted 9 lines
Add support for enum64. For 64-bit enumerator value, previously, the value is truncated into 32bit, e.g., for the following enum in linux uapi
bpf.h,
enum {
BPF_F_INDEX_MASK=0xffffffffULL,
BPF_F_CURRENT_CPU=BPF_F_INDEX_MASK,
/*BPF_FUNC_perf_event_outputforsk_buffinput context. */BPF_F_CTXLEN_MASK = (0xfffffULL << 32),};
Applied, added the entry for skip generating enums to the man page, added support to the pahole BTF loader, used the new pahole to build bpf-next/master, all seems ok, pushing to next on git.kernel.org so that the libbpf github CI can give it a go.
To build with torvalds/master one has to add --skip_encoding_btf_enum64, I think, haven't tested with it, without it isn't working, libbpf complains at that btfids tool.
Quoted 7 lines
Please check/test what is in there now:
git://git.kernel.org/pub/scm/devel/pahole/pahole.gitnext
https://git.kernel.org/pub/scm/devel/pahole/pahole.git/log/?h=nextUnless someone screams I plan pushing out a new release, update fedora packages, etc early next week its overdue by now.
Quoted 7 lines
I used this new pahole in kernel build and it looks ok, but I'm getting following warning:
BTFIDS vmlinux
WARN:resolve_btfids: unresolved symbolmptcp_sockmight be specific to my .config, I'll check and let you know
ok I have only FWD declaration of mptcp_sock struct, that's why it can't get resolved.. when I enabled CONFIG_MPTCP it goes away
so no problem with pahole ;-)
the problem is with the btf_sock_ids array that carries mptcp_sock BTF id even when the CONFIG_MPTCP option is not enabled
I'll follow up on that in separate email
thanks,
jirka