libbpf: Render decl_tags in btf_dump
Subsystems (4)
Ihor Solodrai
reviewercover libbpf: Render decl_tags in btf_dump
BTF supports arbitrary __attribute__ encoding via decl tags and type tags [1]. However btf_dump facility only outputs the attributes encoded in BTF as type tags, which was implemented in commit 2019c58318b8 ("libbpf: Check the kflag of type tags in btf_dump").
BTF_KIND_DECL_TAG has been left out of btf_dump entirely, independent of the kflag value. An attribute a BTF producer encoded is silently dropped from the C output.
Implement decl_tag support in btf_dump. The btf_dump limits rendered BTF types to records, record members and typedefs. Other types, particularly functions, are deliberately skipped. Therefore decl_tag rendering works only for the same subset of target types.
To be efficient, decl_tag dump support requires a reverse index from the target type id to the list of tags pointing at it.
With decl_tag dump support, provided a validly constructed BTF, it is now possible to obtain C-syntax output like the following example:
struct foo {
int a __attribute__((bar));
int b: 3 __attribute__((bar));
} __attribute__((bar));
typedef struct { ... } __attribute__((bar)) foo_t __attribute__((baz));vmlinux.h generated in-tree remains unchanged: every decl tag in vmlinux BTF is a tag on a FUNC, which is not supported by btf_dump.
[1] https://lore.kernel.org/bpf/[email protected]/
---
The series consists of the following:
* patch #1 is a non-functional refactoring, preparing btf_dump_resize() for decl_tag index build
* patch #2 implements the full decl_tag dump support
* patch #3 adds the selftests to cover the new feature
* patch #4 is an independent selftests change relevant to kflagged decl_tag handling
Ihor Solodrai (4):
libbpf: Walk types in btf_dump_resize(), not in mark_referenced()
libbpf: Render decl_tags in btf_dump
selftests/bpf: Test btf_dump rendering of decl_tags selftests/bpf: Show the tag kflag in the raw BTF dump helper
tools/lib/bpf/btf_dump.c | 210 ++++++++--- tools/testing/selftests/bpf/btf_helpers.c | 10 +- .../selftests/bpf/prog_tests/btf_dump.c | 343 +++++++++++++++++- .../selftests/bpf/prog_tests/btf_write.c | 60 +++ 4 files changed, 560 insertions(+), 63 deletions(-)
base-commit: 10c4f610b215bf961235141161992f010cf7e451
Signature · 1 line
2.55.0
Ihor Solodrai
reviewerPATCH 1/4 libbpf: Walk types in btf_dump_resize(), not in mark_referenced()
btf_dump_mark_referenced() both walks every type added since the last resize and decides what each one references. Move the walk out to btf_dump_resize() and hand the function one type at a time, so a second per-type job can share the same pass instead of adding another one.
No functional change.
--- tools/lib/bpf/btf_dump.c | 116 +++++++++++++++++++-------------------- 1 file changed, 57 insertions(+), 59 deletions(-)
tools/lib/bpf/btf_dump.c +57 -59
@@ -143,7 +143,7 @@ static void btf_dump_printf(const struct btf_dump *d, const char *fmt, ...)va_end (args); } -static int btf_dump_mark_referenced (struct btf_dump *d); +static int btf_dump_mark_referenced (struct btf_dump *d,const struct btf_type *t);static int btf_dump_resize (struct btf_dump *d); struct btf_dump *btf_dump__new(const struct btf *btf,
@@ -195,6 +195,8 @@ struct btf_dump *btf_dump__new(const struct btf *btf,static int btf_dump_resize (struct btf_dump *d) {int err, last_id =btf__type_cnt (d->btf) -1 ; +const struct btf_type *t; +__u32 i;if (last_id <= d->last_id)return 0 ;
@@ -212,10 +214,13 @@ static int btf_dump_resize(struct btf_dump *d) d->type_states[0 ].emit_state =EMITTED ; } -/* eagerly determine referenced types for anon enums */ - err =btf_dump_mark_referenced (d); -if (err) -return err; +for (i = d->last_id +1 ; i <= last_id; i++) { + t =btf__type_by_id (d->btf, i); + + err =btf_dump_mark_referenced (d, t); +if (err) +return err; + } d->last_id = last_id;return 0 ;
@@ -312,68 +317,61 @@ int btf_dump__dump_type(struct btf_dump *d, __u32 id)* top-level anonymous enum won't be referenced by anything, while embedded * one will. */ -static int btf_dump_mark_referenced (struct btf_dump *d) +static int btf_dump_mark_referenced (struct btf_dump *d,const struct btf_type *t) { -int i, j, n =btf__type_cnt (d->btf); -const struct btf_type *t; -__u32 vlen; +__u32 j, vlen =btf_vlen (t); -for (i = d->last_id +1 ; i < n; i++) { - t =btf__type_by_id (d->btf, i); - vlen =btf_vlen (t); - -switch (btf_kind (t)) { -case BTF_KIND_INT : -case BTF_KIND_ENUM : -case BTF_KIND_ENUM64 : -case BTF_KIND_FWD : -case BTF_KIND_FLOAT : -break ; +switch (btf_kind (t)) { +case BTF_KIND_INT : +case BTF_KIND_ENUM : +case BTF_KIND_ENUM64 : +case BTF_KIND_FWD : +case BTF_KIND_FLOAT : +break ; -case BTF_KIND_VOLATILE : -case BTF_KIND_CONST : -case BTF_KIND_RESTRICT : -case BTF_KIND_PTR : -case BTF_KIND_TYPEDEF : -case BTF_KIND_FUNC : -case BTF_KIND_VAR : -case BTF_KIND_DECL_TAG : -case BTF_KIND_TYPE_TAG : - d->type_states[t->type].referenced =1 ; -break ; +case BTF_KIND_VOLATILE : +case BTF_KIND_CONST : +case BTF_KIND_RESTRICT : +case BTF_KIND_PTR : +case BTF_KIND_TYPEDEF : +case BTF_KIND_FUNC : +case BTF_KIND_VAR : +case BTF_KIND_DECL_TAG : +case BTF_KIND_TYPE_TAG : + d->type_states[t->type].referenced =1 ; +break ; -case BTF_KIND_ARRAY : { -const struct btf_array *a =btf_array (t); +case BTF_KIND_ARRAY : { +const struct btf_array *a =btf_array (t); - d->type_states[a->index_type].referenced =1 ; - d->type_states[a->type].referenced =1 ; -break ; - } -case BTF_KIND_STRUCT : -case BTF_KIND_UNION : { -const struct btf_member *m =btf_members (t); + d->type_states[a->index_type].referenced =1 ; + d->type_states[a->type].referenced =1 ; +break ; + } +case BTF_KIND_STRUCT : +case BTF_KIND_UNION : { +const struct btf_member *m =btf_members (t); -for (j =0 ; j < vlen; j++, m++) - d->type_states[m->type].referenced =1 ; -break ; - } -case BTF_KIND_FUNC_PROTO : { -const struct btf_param *p =btf_params (t); +for (j =0 ; j < vlen; j++, m++) + d->type_states[m->type].referenced =1 ; +break ; + } +case BTF_KIND_FUNC_PROTO : { +const struct btf_param *p =btf_params (t); -for (j =0 ; j < vlen; j++, p++) - d->type_states[p->type].referenced =1 ; -break ; - } -case BTF_KIND_DATASEC : { -const struct btf_var_secinfo *v =btf_var_secinfos (t); +for (j =0 ; j < vlen; j++, p++) + d->type_states[p->type].referenced =1 ; +break ; + } +case BTF_KIND_DATASEC : { +const struct btf_var_secinfo *v =btf_var_secinfos (t); -for (j =0 ; j < vlen; j++, v++) - d->type_states[v->type].referenced =1 ; -break ; - } -default : -return -EINVAL ; - } +for (j =0 ; j < vlen; j++, v++) + d->type_states[v->type].referenced =1 ; +break ; + } +default : +return -EINVAL ; }return 0 ; }
Signature · 1 line
2.55.0
Re: [PATCH bpf-next v1 1/4] libbpf: Walk types in btf_dump_resize(), not in mark_referenced()
Quoted 9 lines · Ihor Solodrai
btf_dump_mark_referenced()both walks every type added since the last resize and decides what each one references. Move the walk out tobtf_dump_resize()and hand the function one type at a time, so a second per-type job can share the same pass instead of adding another one.No functional change.
Signed-off-by Ihor Solodrai <[email protected]>---
...
Ihor Solodrai
reviewerPATCH 2/4 libbpf: Render decl_tags in btf_dump
In reply to Ihor SolodraiBTF_KIND_DECL_TAG is an attribute on a declaration: with kind_flag=0 the tag name is the argument of a btf_decl_tag(), and with kind_flag=1 it encodes a complete __attribute__.
When doing btf_dump, render both forms at every declaration btf_dump emits - a record, a record member and a typedef:
struct foo { ... } __attribute__((bar));
struct foo { int a __attribute__((bar)); };
typedef struct { ... } __attribute__((bar)) foo_t;A decl tag is a standalone type pointing at its target. Build an index of decl tags in btf_dump_resize(). Keep it as a flat array sorted by (target ID, tag ID). This allows for a stable emission order in btf_dump_emit_decl_tags(). Tag IDs are unique, so the comparison is a total order and qsort not being stable does not matter.
Only composite and typedef targets are indexed. Valid BTF allows for decl_tags on many types, however btf_dump only supports records, record members and typedefs and ignores datasec, var and func.
btf_dump_emit_decl_tags() binary searches for where the target's entries would begin and walks tags while the target matches. A record shares its target with its members, so the component_idx is matched there.
A record attribute goes after the closing brace, where __attribute__((packed)) already goes. A member attribute goes after the bit-field width: clang rejects one between the declarator and the ':'. A typedef takes it after the declarator, so a typedef of an anonymous record can carry two groups at once, one binding to the record and one to the typedef.
Every decl tag in vmlinux BTF is a bpf_kfunc or bpf_fastcall tag on a FUNC, which has no declaration in the C output, so generated vmlinux.h does not change.
--- tools/lib/bpf/btf_dump.c | 94 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+)
tools/lib/bpf/btf_dump.c +94 -0
@@ -77,6 +77,11 @@ struct btf_dump_data {bool is_array_char; }; +struct decl_tag_desc { +__u32 target_id; +__u32 tag_id; +}; +struct btf_dump {const struct btf *btf;btf_dump_printf_fn_t printf_fn;
@@ -93,6 +98,11 @@ struct btf_dump {const char **cached_names;size_t cached_names_cap; +/* decl tags, sorted by (target ID, tag ID) */ +struct decl_tag_desc *decl_tags; +size_t decl_tags_cnt; +size_t decl_tags_cap; +/* topo-sorted list of dependent type definitions */ __u32 *emit_queue;int emit_queue_cap;
@@ -192,9 +202,37 @@ struct btf_dump *btf_dump__new(const struct btf *btf,return libbpf_err_ptr (err); } +static int btf_dump_cmp_decl_tags (const void *a,const void *b) +{ +const struct decl_tag_desc *x = a, *y = b; + +if (x->target_id != y->target_id) +return x->target_id < y->target_id ?-1 :1 ; +return x->tag_id < y->tag_id ?-1 :1 ; +} + +static int btf_dump_push_decl_tag (struct btf_dump *d,__u32 id,const struct btf_type *t) +{ +const struct btf_type *target =btf__type_by_id (d->btf, t->type); + +if (!target || (!btf_is_composite (target) && !btf_is_typedef (target))) +return 0 ; + +if (libbpf_ensure_mem ((void **)&d->decl_tags, &d->decl_tags_cap, +sizeof (*d->decl_tags), d->decl_tags_cnt +1 )) +return -ENOMEM ; + + d->decl_tags[d->decl_tags_cnt++] = (struct decl_tag_desc ) { + .target_id = t->type, + .tag_id = id, + }; +return 0 ; +} +static int btf_dump_resize (struct btf_dump *d) {int err, last_id =btf__type_cnt (d->btf) -1 ; +size_t cnt = d->decl_tags_cnt;const struct btf_type *t;__u32 i;
@@ -220,8 +258,18 @@ static int btf_dump_resize(struct btf_dump *d) err =btf_dump_mark_referenced (d, t);if (err)return err; + +if (btf_is_decl_tag (t)) { + err =btf_dump_push_decl_tag (d, i, t); +if (err) +return err; + } } +if (d->decl_tags_cnt != cnt) +qsort (d->decl_tags, d->decl_tags_cnt,sizeof (*d->decl_tags), + btf_dump_cmp_decl_tags); + d->last_id = last_id;return 0 ; }
@@ -256,6 +304,7 @@ void btf_dump__free(struct btf_dump *d) } }free (d->cached_names); +free (d->decl_tags);free (d->emit_queue);free (d->decl_stack);btf_dump_free_names (d->type_names);
@@ -962,6 +1011,47 @@ static void btf_dump_emit_struct_fwd(struct btf_dump *d, __u32 id, btf_dump_type_name(d, id)); } +static void btf_dump_emit_decl_tag (struct btf_dump *d,const struct btf_type *t) +{ +const char *name =btf_name_of (d, t->name_off); + +if (btf_kflag (t)) +btf_dump_printf (d," __attribute__((%s))" , name); +else +btf_dump_printf (d," __attribute__((btf_decl_tag(\"%s\")))" , name); +} + +/* +* btf_dump_resize() keeps d->decl_tags sorted by (target ID, tag ID), so the +* tags of one type form a run that binary search finds the start of, in a +* fixed order so that the same BTF always renders the same C. +* +* component_idx is not stored in d->decl_tags: a record and its members share +* a target ID, so it is read from each tag. +*/ +static void btf_dump_emit_decl_tags (struct btf_dump *d,__u32 id,int comp_idx) +{ +size_t i, lo =0 , hi = d->decl_tags_cnt, mid; +const struct btf_type *t; + +/* lower bound on target_id */ +while (lo < hi) { + mid = lo + (hi - lo) /2 ; + +if (d->decl_tags[mid].target_id < id) + lo = mid +1 ; +else + hi = mid; + } + +for (i = lo; i < d->decl_tags_cnt && d->decl_tags[i].target_id == id; i++) { + t =btf__type_by_id (d->btf, d->decl_tags[i].tag_id); + +if (btf_decl_tag (t)->component_idx == comp_idx) +btf_dump_emit_decl_tag (d, t); + } +} + static void btf_dump_emit_struct_def(struct btf_dump *d, __u32 id, const struct btf_type *t,
@@ -1007,6 +1097,8 @@ static void btf_dump_emit_struct_def(struct btf_dump *d, prev_bitfield = false; } +/* after the bit-field width; an attribute cannot precede it */ +btf_dump_emit_decl_tags (d, id, i);btf_dump_printf (d,";" ); }
@@ -1026,6 +1118,7 @@ static void btf_dump_emit_struct_def(struct btf_dump *d, }if (packed)btf_dump_printf (d," __attribute__((packed))" ); +btf_dump_emit_decl_tags (d, id,-1 ); }static const char *missing_base_types[][2 ] = {
@@ -1203,6 +1296,7 @@ static void btf_dump_emit_typedef_def(struct btf_dump *d, __u32 id,btf_dump_printf (d,"typedef " );btf_dump_emit_type_decl (d, t->type, name, lvl); +btf_dump_emit_decl_tags (d, id,-1 ); }static int btf_dump_push_decl_stack_id (struct btf_dump *d,__u32 id)
Signature · 1 line
2.55.0
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] libbpf: btf_dump_cmp_decl_tags violates strict weak ordering in qsort - [Medium] libbpf…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] libbpf: btf_dump_cmp_decl_tags violates strict weak ordering in qsort
- [Medium] libbpf: duplicate tags and memory leak on btf_dump_resize error paths --
commit 869ac84b9144d9ee48198dbc0d57c9510a6f9d98
Author: Ihor Solodrai <[email protected]>
Subject: libbpf: Render decl_tags in btf_dump
This commit adds support for rendering BTF_KIND_DECL_TAG in btf_dump. It builds an array of decl tags sorted by target ID and tag ID, which are then emitted at every declaration.
tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf, · line 208
Quoted 11 lines
tools/lib/bpf/btf_dump.c +7
-0@@ -202,9 +202,15 @@ struct btf_dump *btf_dump__new(const struct btf *btf, +static int btf_dump_cmp_decl_tags (const void *a,const void *b) +{ +const struct decl_tag_desc *x = a, *y = b; + +if (x->target_id != y->target_id) +return x->target_id < y->target_id ?-1 :1 ; +return x->tag_id < y->tag_id ?-1 :1 ;
[Severity: Medium]
Does this comparator satisfy the strict weak ordering requirement for qsort?
If two elements have the same tag_id, this returns 1 instead of 0. Depending on the libc implementation, returning 1 when comparing an element to itself or to a duplicate could break antisymmetry and potentially lead to out of bounds accesses or infinite loops.
Quoted 1 line
tools/lib/bpf/btf_dump.c +1
-0+}
[ ... ]
tools/lib/bpf/btf_dump.c static int btf_dump_resize(struct btf_dump *d) · line 260
Quoted 4 lines
tools/lib/bpf/btf_dump.c +0
-0@@ -258,8 +258,18 @@ static int btf_dump_resize(struct btf_dump *d) err =btf_dump_mark_referenced (d, t);if (err)return err;
[Severity: Medium]
Do we need to roll back d->decl_tags_cnt or free resources here?
If this loop encounters an error and returns early, it leaves the previously pushed decl_tags in the array, but d->last_id isn't updated.
If a user handles the error and resumes using the btf_dump object, won't it re-push those same tags on the next call, causing duplicates and memory leaks?
tools/lib/bpf/btf_dump.c static int btf_dump_resize(struct btf_dump *d) · line 265
Quoted 5 lines
tools/lib/bpf/btf_dump.c +5
-0+ +if (btf_is_decl_tag (t)) { + err =btf_dump_push_decl_tag (d, i, t); +if (err) +return err;
[Severity: Medium]
Similarly, does returning early here lead to the same duplicate tag entries on retry, since d->decl_tags_cnt is not rolled back on error?
Quoted 2 lines
tools/lib/bpf/btf_dump.c +1
-0+ } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Quoted 6 lines · Ihor Solodrai
BTF_KIND_DECL_TAGis an attribute on a declaration: with kind_flag=0 the tag name is the argument of abtf_decl_tag(), and with kind_flag=1 it encodes a complete__attribute__.When doing
btf_dump, render both forms at every declarationbtf_dumpemits - a record, a record member and a typedef:
[...]
Quoted 3 lines
Every decl tag in vmlinux BTF is a
bpf_kfuncorbpf_fastcalltag on a FUNC, which has no declaration in the C output, so generatedvmlinux.hdoes not change.
iirc the goal of this series is "bpftool btf dump file prog.bpf.o format c" and skeletons growing __attribute__((btf_decl_tag("contains:..."))) on members. Pls say so in the commit log.
Otherwise it's 200 lines of code for "generated vmlinux.h doesn't change".
tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf,
Quoted 5 lines
tools/lib/bpf/btf_dump.c +5
-0+static int btf_dump_cmp_decl_tags (const void *a,const void *b) +{ +const struct decl_tag_desc *x = a, *y = b; + + if (x->target_id != y->tar
get_id)
tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf, · line 212
Quoted 3 lines
tools/lib/bpf/btf_dump.c +3
-0+return x->target_id < y->target_id ?-1 :1 ; +return x->tag_id < y->tag_id ?-1 :1 ; +}
nit: return 0 when equal. cmp(x, x) != 0 is asking for trouble.
tools/lib/bpf/btf_dump.c static int btf_dump_resize(struct btf_dump *d) · line 267
Quoted 11 lines
tools/lib/bpf/btf_dump.c +6
-0@@ -220,8 +258,18 @@ static int btf_dump_resize(struct btf_dump *d) err =btf_dump_mark_referenced (d, t);if (err)return err; + +if (btf_is_decl_tag (t)) { + err =btf_dump_push_decl_tag (d, i, t); +if (err) +return err; + } }
\edited robot voice
On either error d->last_id is not advanced but d->decl_tags_cnt keeps whatever was pushed so far, so the next btf_dump__dump_type() walks the same ids again and pushes the same tags twice -> duplicated __attribute__ in the output ?
pw-bot: cr
Ihor Solodrai
reviewerQuoted 14 lines · Alexei Starovoitov
Quoted 6 lines · Ihor Solodrai
BTF_KIND_DECL_TAGis an attribute on a declaration: with kind_flag=0 the tag name is the argument of abtf_decl_tag(), and with kind_flag=1 it encodes a complete__attribute__.When doing
btf_dump, render both forms at every declarationbtf_dumpemits - a record, a record member and a typedef:[...]
Quoted 3 lines
Every decl tag in vmlinux BTF is a
bpf_kfuncorbpf_fastcalltag on a FUNC, which has no declaration in the C output, so generatedvmlinux.hdoes not change.
Quoted 6 lines
iirc the goal of this series is "bpftool btf dump file prog.bpf.o format c" and skeletons growing __attribute__((btf_decl_tag("contains:..."))) on members. Pls say so in the commit log.
Otherwise it's 200 lines of code for "generatedvmlinux.hdoesn't change".
Hi Alexei, thank you for quick review.
The decl_tags is an old generic gap in btf_dump, which gets closed with this patch.
The "contains:" is not the use case I had in mind, but yes, it should be covered as well. The next thing I planned after this lands is a bpftool change to get rid of:
#pragma clang attribute push (__attribute__((preserve_access_index)), apply_to = record)
in vmlinux.h, by emitting the attributes explicitly per record. It's been discussed a long time ago, but never picked up on libbpf / bpftool side [1].
I'll add a paragraph describing potential use cases in v2.
[1] https://lore.kernel.org/bpf/[email protected]/
Quoted 7 lines
tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf,
Quoted 6 lines
tools/lib/bpf/btf_dump.c +5
-0+static int btf_dump_cmp_decl_tags (const void *a,const void *b) +{ +const struct decl_tag_desc *x = a, *y = b; + + if (x->target_id != y->tar
get_id)
Quoted 5 lines
tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf, · line 212
Quoted 3 lines
tools/lib/bpf/btf_dump.c +3
-0+return x->target_id < y->target_id ?-1 :1 ; +return x->tag_id < y->tag_id ?-1 :1 ; +}nit: return 0 when equal.
cmp(x, x)!= 0 is asking for trouble.
For a valid deduped BTF we should never get a duplicate (target_id, tag_id) pair. I'll see if there is a way to detect this and -EINVAL
Quoted 15 lines
tools/lib/bpf/btf_dump.c static int btf_dump_resize(struct btf_dump *d) · line 267
Quoted 12 lines
tools/lib/bpf/btf_dump.c +6
-0@@ -220,8 +258,18 @@ static int btf_dump_resize(struct btf_dump *d) err =btf_dump_mark_referenced (d, t);if (err)return err; + +if (btf_is_decl_tag (t)) { + err =btf_dump_push_decl_tag (d, i, t); +if (err) +return err; + } }\edited robot voice
Quoted 4 lines
On either error
d->last_idis not advanced butd->decl_tags_cntkeeps whatever was pushed so far, so the nextbtf_dump__dump_type()walks the same ids again and pushes the same tags twice -> duplicated__attribute__in the output ?
Yeah, my clanker flagged this too. I decided a mid-dump retry on error is a weird case to support in the context of btf_dump, so ignored it.
Andrii, do you have an opinion?
Quoted 2 lines
pw-bot: cr
Quoted 33 lines · Ihor Solodrai
BTF_KIND_DECL_TAGis an attribute on a declaration: with kind_flag=0 the tag name is the argument of abtf_decl_tag(), and with kind_flag=1 it encodes a complete__attribute__.When doing
btf_dump, render both forms at every declarationbtf_dumpemits - a record, a record member and a typedef:struct foo { ... } __attribute__((bar)); struct foo { int a __attribute__((bar)); }; typedef struct { ... } __attribute__((bar)) foo_t;A decl tag is a standalone type pointing at its target. Build an index of decl tags in
btf_dump_resize(). Keep it as a flat array sorted by (target ID, tag ID). This allows for a stable emission order inbtf_dump_emit_decl_tags(). Tag IDs are unique, so the comparison is a total order and qsort not being stable does not matter.Only composite and typedef targets are indexed. Valid BTF allows for
decl_tagson many types, howeverbtf_dumponly supports records, record members and typedefs and ignores datasec, var and func.
btf_dump_emit_decl_tags()binary searches for where the target's entries would begin and walks tags while the target matches. A record shares its target with its members, so thecomponent_idxis matched there.A record attribute goes after the closing brace, where __attribute__((packed)) already goes. A member attribute goes after the bit-field width: clang rejects one between the declarator and the ':'. A typedef takes it after the declarator, so a typedef of an anonymous record can carry two groups at once, one binding to the record and one to the typedef.
Quoted 6 lines
Every decl tag in vmlinux BTF is a
bpf_kfuncorbpf_fastcalltag on a FUNC, which has no declaration in the C output, so generatedvmlinux.hdoes not change.Signed-off-by Ihor Solodrai <[email protected]>---
Quoted 9 lines
tools/lib/bpf/btf_dump.c +5
-0@@ -962,6 +1011,47 @@ static void btf_dump_emit_struct_fwd(struct btf_dump *d, __u32 id, btf_dump_type_name(d, id)); } +static void btf_dump_emit_decl_tag (struct btf_dump *d,const struct btf_type *t) +{ +const char *name =btf_name_of (d, t->name_off); + +if (btf_kflag (t))
tools/lib/bpf/btf_dump.c static void btf_dump_emit_struct_fwd(struct btf_dump *d, __u32 id, · line 1028
Quoted 10 lines
tools/lib/bpf/btf_dump.c +10
-0+btf_dump_printf (d," __attribute__((%s))" , name); + else + btf_dump_printf(d, " __attribute__((btf_decl_tag(\"%s\")))", name); +} + +/* +* btf_dump_resize() keeps d->decl_tags sorted by (target ID, tag ID), so the +* tags of one type form a run that binary search finds the start of, in a +* fixed order so that the same BTF always renders the same C. +*
--- 8< ---
tools/lib/bpf/btf_dump.c static void btf_dump_emit_struct_fwd(struct btf_dump *d, __u32 id, · line 1030
Quoted 2 lines
tools/lib/bpf/btf_dump.c +2
-0+* component_idx is not stored in d->decl_tags: a record and its members share +* a target ID, so it is read from each tag.
Nit: useless detail, obvious from the code.
--- >8 ---
...
tools/lib/bpf/btf_dump.c static void btf_dump_emit_struct_def(struct btf_dump *d, · line 1100
Quoted 5 lines
tools/lib/bpf/btf_dump.c +1
-0@@ -1007,6 +1097,8 @@ static void btf_dump_emit_struct_def(struct btf_dump *d, prev_bitfield = false; } +/* after the bit-field width; an attribute cannot precede it */
Nit: /* after the bit-field width */ ?
Quoted 4 lines
tools/lib/bpf/btf_dump.c +1
-0+btf_dump_emit_decl_tags (d, id, i);btf_dump_printf (d,";" ); }
...
Quoted 34 lines · Ihor Solodrai
BTF_KIND_DECL_TAGis an attribute on a declaration: with kind_flag=0 the tag name is the argument of abtf_decl_tag(), and with kind_flag=1 it encodes a complete__attribute__.When doing
btf_dump, render both forms at every declarationbtf_dumpemits - a record, a record member and a typedef:struct foo { ... } __attribute__((bar)); struct foo { int a __attribute__((bar)); }; typedef struct { ... } __attribute__((bar)) foo_t;A decl tag is a standalone type pointing at its target. Build an index of decl tags in
btf_dump_resize(). Keep it as a flat array sorted by (target ID, tag ID). This allows for a stable emission order inbtf_dump_emit_decl_tags(). Tag IDs are unique, so the comparison is a total order and qsort not being stable does not matter.Only composite and typedef targets are indexed. Valid BTF allows for
decl_tagson many types, howeverbtf_dumponly supports records, record members and typedefs and ignores datasec, var and func.
btf_dump_emit_decl_tags()binary searches for where the target's entries would begin and walks tags while the target matches. A record shares its target with its members, so thecomponent_idxis matched there.A record attribute goes after the closing brace, where __attribute__((packed)) already goes. A member attribute goes after the bit-field width: clang rejects one between the declarator and the ':'. A typedef takes it after the declarator, so a typedef of an anonymous record can carry two groups at once, one binding to the record and one to the typedef.
Quoted 9 lines
Every decl tag in vmlinux BTF is a
bpf_kfuncorbpf_fastcalltag on a FUNC, which has no declaration in the C output, so generatedvmlinux.hdoes not change.Signed-off-by Ihor Solodrai <[email protected]>--- tools/lib/bpf/btf_dump.c | 94 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+)
series looks good, just small nits below
Quoted 17 lines
tools/lib/bpf/btf_dump.c +5
-0@@ -77,6 +77,11 @@ struct btf_dump_data {bool is_array_char; }; +struct decl_tag_desc { +__u32 target_id; +__u32 tag_id; +}; +struct btf_dump {const struct btf *btf;btf_dump_printf_fn_t printf_fn;
tools/lib/bpf/btf_dump.c struct btf_dump { · line 104
Quoted 7 lines
tools/lib/bpf/btf_dump.c +4
-0@@ -93,6 +98,11 @@ struct btf_dump {const char **cached_names;size_t cached_names_cap; +/* decl tags, sorted by (target ID, tag ID) */ +struct decl_tag_desc *decl_tags; +size_t decl_tags_cnt; +size_t decl_tags_cap;
nit: decl_tag_cnt, decl_tag_cap, "count" expects singular noun before
it: "table count" not "tables count"
Quoted 12 lines
tools/lib/bpf/btf_dump.c +5
-0+/* topo-sorted list of dependent type definitions */ __u32 *emit_queue;int emit_queue_cap;@@ -192,9 +202,37 @@ struct btf_dump *btf_dump__new(const struct btf *btf,return libbpf_err_ptr (err); } +static int btf_dump_cmp_decl_tags (const void *a,const void *b) +{ +const struct decl_tag_desc *x = a, *y = b; +
tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf, · line 218
Quoted 10 lines
tools/lib/bpf/btf_dump.c +10
-0+if (x->target_id != y->target_id) +return x->target_id < y->target_id ?-1 :1 ; +return x->tag_id < y->tag_id ?-1 :1 ; +} + +static int btf_dump_push_decl_tag (struct btf_dump *d,__u32 id,const struct btf_type *t) +{ +const struct btf_type *target =btf__type_by_id (d->btf, t->type); + +if (!target || (!btf_is_composite (target) && !btf_is_typedef (target)))
target shouldn't be null with correct t->type, don't add defensive checks
but also why this artificial limitation on the kind of thing that is tagged? memory savings?
Quoted 13 lines
tools/lib/bpf/btf_dump.c +13
-0+return 0 ; + +if (libbpf_ensure_mem ((void **)&d->decl_tags, &d->decl_tags_cap, +sizeof (*d->decl_tags), d->decl_tags_cnt +1 )) +return -ENOMEM ; + + d->decl_tags[d->decl_tags_cnt++] = (struct decl_tag_desc ) { + .target_id = t->type, + .tag_id = id, + }; +return 0 ; +} +
[...]
Ihor Solodrai
reviewerQuoted 9 lines · Andrii Nakryiko
Quoted 6 lines · Ihor Solodrai
[...]
--- tools/lib/bpf/btf_dump.c | 94 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+)series looks good, just small nits below
Hi Andrii, thanks for the review.
Sorry I didn't reply earlier, wanted to get that REF_TYPE_FRAME stuff out.
Quoted 21 lines
Quoted 21 lines
tools/lib/bpf/btf_dump.c +5
-0@@ -77,6 +77,11 @@ struct btf_dump_data {bool is_array_char; }; +struct decl_tag_desc { +__u32 target_id; +__u32 tag_id; +}; +struct btf_dump {const struct btf *btf;btf_dump_printf_fn_t printf_fn;@@ -93,6 +98,11 @@ struct btf_dump {const char **cached_names;size_t cached_names_cap;
Quoted 7 lines
tools/lib/bpf/btf_dump.c struct btf_dump { · line 104
Quoted 4 lines
tools/lib/bpf/btf_dump.c +4
-0+/* decl tags, sorted by (target ID, tag ID) */ +struct decl_tag_desc *decl_tags; +size_t decl_tags_cnt; +size_t decl_tags_cap;nit:
decl_tag_cnt,decl_tag_cap, "count" expects singular noun before
it: "table count" not "tables count"
ack
Quoted 9 lines
Quoted 9 lines
tools/lib/bpf/btf_dump.c +1
-0+/* topo-sorted list of dependent type definitions */ __u32 *emit_queue;int emit_queue_cap;@@ -192,9 +202,37 @@ struct btf_dump *btf_dump__new(const struct btf *btf,return libbpf_err_ptr (err); }
Quoted 16 lines
Quoted 4 lines
tools/lib/bpf/btf_dump.c +4
-0+static int btf_dump_cmp_decl_tags (const void *a,const void *b) +{ +const struct decl_tag_desc *x = a, *y = b; +tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf, · line 218
Quoted 10 lines
tools/lib/bpf/btf_dump.c +10
-0+if (x->target_id != y->target_id) +return x->target_id < y->target_id ?-1 :1 ; +return x->tag_id < y->tag_id ?-1 :1 ; +} + +static int btf_dump_push_decl_tag (struct btf_dump *d,__u32 id,const struct btf_type *t) +{ +const struct btf_type *target =btf__type_by_id (d->btf, t->type); + +if (!target || (!btf_is_composite (target) && !btf_is_typedef (target)))target shouldn't be null with correct
t->type, don't add defensive checks
According to comments, decl_tags technically can refer to a type that is not the BTF yet:
tools/lib/bpf/btf.c:3314-3323:
/*
* Append new BTF_KIND_DECL_TAG type with:
* - *value* - non-empty/non-NULL string; * - *ref_type_id* - referenced type ID, it might not exist yet; * ...
*/
int btf__add_decl_tag(...)
Granted, I wouldn't expect this to be a real use-case. But if we decide to drop checks like this, then I think we should be explicit about "no incremental dump", and maybe simplify relevant code a bit.
Quoted 3 lines
but also why this artificial limitation on the kind of thing that is tagged? memory savings?
Yeah, I guess the effect of this is memory savings, but that wasn't the motivation. I just thought "why collect tags that we are not going to dump?". But the index can collect everything, it shouldn't affect the dump. Do you think we should do that?
Quoted 17 lines
Quoted 15 lines
tools/lib/bpf/btf_dump.c +13
-0+return 0 ; + +if (libbpf_ensure_mem ((void **)&d->decl_tags, &d->decl_tags_cap, +sizeof (*d->decl_tags), d->decl_tags_cnt +1 )) +return -ENOMEM ; + + d->decl_tags[d->decl_tags_cnt++] = (struct decl_tag_desc ) { + .target_id = t->type, + .tag_id = id, + }; +return 0 ; +} +[...]
Quoted 72 lines · Ihor Solodrai
Quoted 9 lines · Andrii Nakryiko
Quoted 6 lines · Ihor Solodrai
[...]
--- tools/lib/bpf/btf_dump.c | 94 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+)series looks good, just small nits below
Hi Andrii, thanks for the review.
Sorry I didn't reply earlier, wanted to get thatREF_TYPE_FRAMEstuff out.Quoted 21 lines
Quoted 21 lines
tools/lib/bpf/btf_dump.c +5
-0@@ -77,6 +77,11 @@ struct btf_dump_data {bool is_array_char; }; +struct decl_tag_desc { +__u32 target_id; +__u32 tag_id; +}; +struct btf_dump {const struct btf *btf;btf_dump_printf_fn_t printf_fn;@@ -93,6 +98,11 @@ struct btf_dump {const char **cached_names;size_t cached_names_cap;Quoted 7 lines
tools/lib/bpf/btf_dump.c struct btf_dump { · line 104
Quoted 4 lines
tools/lib/bpf/btf_dump.c +4
-0+/* decl tags, sorted by (target ID, tag ID) */ +struct decl_tag_desc *decl_tags; +size_t decl_tags_cnt; +size_t decl_tags_cap;nit:
decl_tag_cnt,decl_tag_cap, "count" expects singular noun before
it: "table count" not "tables count"ack
Quoted 9 lines
Quoted 9 lines
tools/lib/bpf/btf_dump.c +1
-0+/* topo-sorted list of dependent type definitions */ __u32 *emit_queue;int emit_queue_cap;@@ -192,9 +202,37 @@ struct btf_dump *btf_dump__new(const struct btf *btf,return libbpf_err_ptr (err); }Quoted 16 lines
Quoted 4 lines
tools/lib/bpf/btf_dump.c +4
-0+static int btf_dump_cmp_decl_tags (const void *a,const void *b) +{ +const struct decl_tag_desc *x = a, *y = b; +tools/lib/bpf/btf_dump.c struct btf_dump *btf_dump__new(const struct btf *btf, · line 218
Quoted 10 lines
tools/lib/bpf/btf_dump.c +10
-0+if (x->target_id != y->target_id) +return x->target_id < y->target_id ?-1 :1 ; +return x->tag_id < y->tag_id ?-1 :1 ; +} + +static int btf_dump_push_decl_tag (struct btf_dump *d,__u32 id,const struct btf_type *t) +{ +const struct btf_type *target =btf__type_by_id (d->btf, t->type); + +if (!target || (!btf_is_composite (target) && !btf_is_typedef (target)))target shouldn't be null with correct
t->type, don't add defensive checks
Quoted 2 lines
According to comments,
decl_tagstechnically can refer to a type that is not the BTF yet:
that means only during btf construction, I'm not sure how dumper should behave if you pass it decl_tag that points to non-existing type... erroring out would be one way, but silently ignoring is probably not a right way anyways.
but as you said, I don't think anyone should pass half-constructed BTF to dumper and expect that to work... incremental still means that any point btf is correct and complete
Quoted 16 lines
tools/lib/bpf/btf.c:3314-3323:
/*
* Append newBTF_KIND_DECL_TAGtype with:* - *value* - non-empty/non-NULL string; * - *ref_type_id* - referenced type ID, it might not exist yet; * ...*/
intbtf__add_decl_tag(...)Granted, I wouldn't expect this to be a real use-case. But if we decide to drop checks like this, then I think we should be explicit about "no incremental dump", and maybe simplify relevant code a bit.
Quoted 1 line
Quoted 7 lines
Quoted 2 lines
but also why this artificial limitation on the kind of thing that is tagged? memory savings?
Yeah, I guess the effect of this is memory savings, but that wasn't the motivation. I just thought "why collect tags that we are not going to dump?". But the index can collect everything, it shouldn't affect the dump. Do you think we should do that?
I'd collect everything to keep this part of logic complete and forward-compatible without having to revisit it
Quoted 20 lines
Quoted 19 lines
Quoted 17 lines
tools/lib/bpf/btf_dump.c +13
-0+return 0 ; + +if (libbpf_ensure_mem ((void **)&d->decl_tags, &d->decl_tags_cap, +sizeof (*d->decl_tags), d->decl_tags_cnt +1 )) +return -ENOMEM ; + + d->decl_tags[d->decl_tags_cnt++] = (struct decl_tag_desc ) { + .target_id = t->type, + .tag_id = id, + }; +return 0 ; +} +[...]
Ihor Solodrai
reviewerPATCH 3/4 selftests/bpf: Test btf_dump rendering of decl_tags
In reply to Ihor SolodraiCover the decl tag rendering the previous patch added. One subtest builds a BTF with every shape that renders, plus two that must not:
- a named record with both forms of tag
- a typedef'd anonymous record, whose attribute precedes the declarator
- a union
- a record that also carries the derived packed attribute
- an anonymous record inlined at a member
- two records whose tags are added in reverse target order, which is what pins the sort: without it the earlier record loses its attribute
- attributes on a var and on a func, which btf_dump never declares
A second subtest appends types and attributes to a dumper that has already run. Its last phase adds an attribute for a record emitted two dumps earlier and checks nothing is rendered for it: an attribute is part of the type, and btf_dump emits each definition once.
A third covers members. A member attribute and a record one share a key in the index, so it checks they do not leak into each other's position, over a function pointer, an array and a bit-field - declarators the attribute has to follow rather than precede.
A fourth covers typedefs, including a typedef of an anonymous record carrying three groups at once, binding to the member, the record and the typedef.
Switch test_ctx__dump_and_compare() to compare_text_to_expected() while we are here.
--- .../selftests/bpf/prog_tests/btf_dump.c | 343 +++++++++++++++++- 1 file changed, 342 insertions(+), 1 deletion(-)
tools/testing/selftests/bpf/prog_tests/btf_dump.c +342 -1
@@ -224,7 +224,12 @@ static void test_ctx__dump_and_compare(struct test_ctx *t,fflush (t->dump_buf_file); t->dump_buf[t->dump_buf_sz] =0 ;/* some libc implementations don't do this */ -ASSERT_STREQ (t->dump_buf, expected_output, message); +/* +* The mismatch has already been reported, so this only has to +* register the failure. ASSERT_OK() would append a stale errno to it. +*/ + err =compare_text_to_expected (t->dump_buf, expected_output); +ASSERT_EQ (err,0 , message); }static void test_btf_dump_incremental (void )
@@ -328,6 +333,330 @@ static void test_btf_dump_incremental(void)test_ctx__free (&t); } +static void test_btf_dump_decl_tags (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + +/* +* Generate BTF corresponding to the following C code: +* +* struct s1 { int f; } __attribute__((attr1)) +* __attribute__((btf_decl_tag("plain_tag"))); +* typedef struct { int f; } __attribute__((a1)) __attribute__((a2)) t1; +* union u1 { int f; } __attribute__((uattr)); +* struct p1 { char c; int i; } __attribute__((packed)) __attribute__((pattr)); +* struct outer { struct { int g; } __attribute__((inner)) in; }; +* struct s2 { int f; } __attribute__((on_s2)); +* struct s3 { int f; } __attribute__((on_s3)); +* +* plus attributes on a var and a func, which btf_dump does not declare. +*/ + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_int (btf,"char" ,1 ,BTF_INT_CHAR ); +ASSERT_EQ (id,2 ,"char_id" ); + +/* a named record with one attribute of each form */ + id =btf__add_struct (btf,"s1" ,4 ); +ASSERT_EQ (id,3 ,"s1_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s1_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"attr1" ,3 ,-1 ),4 ,"s1_attr" ); +/* no kflag: rendered as a btf_decl_tag(), not as a bare attribute */ +ASSERT_EQ (btf__add_decl_tag (btf,"plain_tag" ,3 ,-1 ),5 ,"s1_plain_tag" ); + +/* +* an anonymous record behind a typedef, with two attributes. This is +* the case that cannot be expressed by a caller appending to the dump: +* the attribute has to go before the declarator. +*/ + id =btf__add_struct (btf,NULL ,4 ); +ASSERT_EQ (id,6 ,"anon_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"anon_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a1" ,6 ,-1 ),7 ,"anon_attr1" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a2" ,6 ,-1 ),8 ,"anon_attr2" ); + id =btf__add_typedef (btf,"t1" ,6 ); +ASSERT_EQ (id,9 ,"typedef_id" ); + +/* unions are records too */ + id =btf__add_union (btf,"u1" ,4 ); +ASSERT_EQ (id,10 ,"u1_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"u1_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"uattr" ,10 ,-1 ),11 ,"u1_attr" ); + +/* composes with the packed attribute, which is derived from layout */ + id =btf__add_struct (btf,"p1" ,5 ); +ASSERT_EQ (id,12 ,"p1_id" ); +ASSERT_OK (btf__add_field (btf,"c" ,2 ,0 ,0 ),"p1_field_c" ); +ASSERT_OK (btf__add_field (btf,"i" ,1 ,8 ,0 ),"p1_field_i" ); +ASSERT_EQ (btf__add_decl_attr (btf,"pattr" ,12 ,-1 ),13 ,"p1_attr" ); + +/* an anonymous record inlined at a member, the third emission position */ + id =btf__add_struct (btf,NULL ,4 ); +ASSERT_EQ (id,14 ,"inner_id" ); +ASSERT_OK (btf__add_field (btf,"g" ,1 ,0 ,0 ),"inner_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"inner" ,14 ,-1 ),15 ,"inner_attr" ); + id =btf__add_struct (btf,"outer" ,4 ); +ASSERT_EQ (id,16 ,"outer_id" ); +ASSERT_OK (btf__add_field (btf,"in" ,14 ,0 ,0 ),"outer_field" ); + +/* not rendered: a var has no declaration in the C output */ + id =btf__add_var (btf,"v" ,BTF_VAR_GLOBAL_ALLOCATED ,3 ); +ASSERT_EQ (id,17 ,"var_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"var_attr" ,17 ,-1 ),18 ,"var_attr" ); +/* not rendered: neither does a func */ + id =btf__add_func_proto (btf,1 ); +ASSERT_EQ (id,19 ,"proto_id" ); + id =btf__add_func (btf,"fn" ,BTF_FUNC_GLOBAL ,19 ); +ASSERT_EQ (id,20 ,"func_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"func_attr" ,20 ,-1 ),21 ,"func_attr" ); + +/* +* Tags added in reverse target order: the tag with the lower type ID +* names the record with the higher one, so the index is in emission +* order only once btf_dump_resize() has sorted it. Without the sort +* the lookup for s2 lands outside its run and drops the attribute. +*/ + id =btf__add_struct (btf,"s2" ,4 ); +ASSERT_EQ (id,22 ,"s2_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s2_field" ); + id =btf__add_struct (btf,"s3" ,4 ); +ASSERT_EQ (id,23 ,"s3_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s3_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"on_s3" ,23 ,-1 ),24 ,"s3_attr" ); +ASSERT_EQ (btf__add_decl_attr (btf,"on_s2" ,22 ,-1 ),25 ,"s2_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s1 {\n" +" int f;\n" +"} __attribute__((attr1)) __attribute__((btf_decl_tag(\"plain_tag\")));\n" +"\n" +"typedef struct {\n" +" int f;\n" +"} __attribute__((a1)) __attribute__((a2)) t1;\n" +"\n" +"union u1 {\n" +" int f;\n" +"} __attribute__((uattr));\n" +"\n" +"struct p1 {\n" +" char c;\n" +" int i;\n" +"} __attribute__((packed)) __attribute__((pattr));\n" +"\n" +"struct outer {\n" +" struct {\n" +" int g;\n" +" } __attribute__((inner)) in;\n" +"};\n" +"\n" +"struct s2 {\n" +" int f;\n" +"} __attribute__((on_s2));\n" +"\n" +"struct s3 {\n" +" int f;\n" +"} __attribute__((on_s3));\n" +"\n" ,"dump_and_compare" ); + +test_ctx__free (&t); +} + +/* +* btf_dump indexes decl attributes incrementally, only walking types appended +* since the last dump. Check that attributes on types added to an existing +* dumper are picked up, and that one added for a record that has already been +* emitted is not - btf_dump emits each definition once. +*/ +static void test_btf_dump_decl_tags_incremental (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_struct (btf,"s1" ,4 ); +ASSERT_EQ (id,2 ,"s1_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s1_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a1" ,2 ,-1 ),3 ,"s1_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s1 {\n" +" int f;\n" +"} __attribute__((a1));\n" +"\n" ,"first_dump" ); + +fseek (t.dump_buf_file,0 ,SEEK_SET ); + +/* a new record and a new attribute for it */ + id =btf__add_struct (btf,"s2" ,4 ); +ASSERT_EQ (id,4 ,"s2_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s2_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a2" ,4 ,-1 ),5 ,"s2_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s2 {\n" +" int f;\n" +"} __attribute__((a2));\n" +"\n" ,"second_dump" ); + +fseek (t.dump_buf_file,0 ,SEEK_SET ); + +/* s1 has already been emitted, so "late" is not rendered anywhere */ +ASSERT_EQ (btf__add_decl_attr (btf,"late" ,2 ,-1 ),6 ,"s1_late_attr" ); + id =btf__add_struct (btf,"s3" ,4 ); +ASSERT_EQ (id,7 ,"s3_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s3_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a3" ,7 ,-1 ),8 ,"s3_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s3 {\n" +" int f;\n" +"} __attribute__((a3));\n" +"\n" ,"third_dump" ); + +test_ctx__free (&t); +} + +/* +* An attribute on a member is emitted at the member declaration, and one on +* the record at the closing brace. Both are keyed by the record in the index, +* so this also checks that they do not leak into each other's position. +*/ +static void test_btf_dump_decl_tags_members (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + +/* +* Generate BTF corresponding to the following C code: +* +* struct m1 { +* int (*fp)(void) __attribute__((fp_attr)); +* int a; +* int b __attribute__((b1)) __attribute__((b2)); +* int arr[4] __attribute__((arr_attr)); +* } __attribute__((rec)); +* struct bits { int x : 3 __attribute__((x_attr)); int y : 29; }; +*/ + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_array (btf,1 ,1 ,4 ); +ASSERT_EQ (id,2 ,"array_id" ); + id =btf__add_func_proto (btf,1 ); +ASSERT_EQ (id,3 ,"proto_id" ); + id =btf__add_ptr (btf,3 ); +ASSERT_EQ (id,4 ,"ptr_id" ); + + id =btf__add_struct (btf,"m1" ,32 ); +ASSERT_EQ (id,5 ,"m1_id" ); +ASSERT_OK (btf__add_field (btf,"fp" ,4 ,0 ,0 ),"m1_fp" ); +ASSERT_OK (btf__add_field (btf,"a" ,1 ,64 ,0 ),"m1_a" ); +ASSERT_OK (btf__add_field (btf,"b" ,1 ,96 ,0 ),"m1_b" ); +ASSERT_OK (btf__add_field (btf,"arr" ,2 ,128 ,0 ),"m1_arr" ); +ASSERT_EQ (btf__add_decl_attr (btf,"rec" ,5 ,-1 ),6 ,"m1_rec_attr" ); +/* the attribute follows the whole declarator, pointer and array alike */ +ASSERT_EQ (btf__add_decl_attr (btf,"fp_attr" ,5 ,0 ),7 ,"m1_fp_attr" ); +/* two on one member, to pin the ordering */ +ASSERT_EQ (btf__add_decl_attr (btf,"b1" ,5 ,2 ),8 ,"m1_b1" ); +ASSERT_EQ (btf__add_decl_attr (btf,"b2" ,5 ,2 ),9 ,"m1_b2" ); +ASSERT_EQ (btf__add_decl_attr (btf,"arr_attr" ,5 ,3 ),10 ,"m1_arr_attr" ); + +/* a bit-field: the attribute has to follow the width */ + id =btf__add_struct (btf,"bits" ,4 ); +ASSERT_EQ (id,11 ,"bits_id" ); +ASSERT_OK (btf__add_field (btf,"x" ,1 ,0 ,3 ),"bits_x" ); +ASSERT_OK (btf__add_field (btf,"y" ,1 ,3 ,29 ),"bits_y" ); +ASSERT_EQ (btf__add_decl_attr (btf,"x_attr" ,11 ,0 ),12 ,"bits_x_attr" ); + +test_ctx__dump_and_compare (&t, +"struct m1 {\n" +" int (*fp)(void) __attribute__((fp_attr));\n" +" int a;\n" +" int b __attribute__((b1)) __attribute__((b2));\n" +" int arr[4] __attribute__((arr_attr));\n" +"} __attribute__((rec));\n" +"\n" +"struct bits {\n" +" int x: 3 __attribute__((x_attr));\n" +" int y: 29;\n" +"};\n" +"\n" ,"dump_and_compare" ); + +test_ctx__free (&t); +} + +/* +* A typedef is the third and last kind btf_dump declares. Its attributes go +* after the declarator, so a typedef of an anonymous record can carry three +* groups in one declaration, binding to three different entities. +*/ +static void test_btf_dump_decl_tags_typedef (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + +/* +* Generate BTF corresponding to the following C code: +* +* typedef int td __attribute__((t1)) +* __attribute__((btf_decl_tag("t2"))); +* typedef struct { +* int f __attribute__((memb_attr)); +* } __attribute__((rec_attr)) both __attribute__((both_attr)); +*/ + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_typedef (btf,"td" ,1 ); +ASSERT_EQ (id,2 ,"td_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"t1" ,2 ,-1 ),3 ,"td_attr1" ); +/* no kflag: rendered as a btf_decl_tag(), not as a bare attribute */ +ASSERT_EQ (btf__add_decl_tag (btf,"t2" ,2 ,-1 ),4 ,"td_attr2" ); + + id =btf__add_struct (btf,NULL ,4 ); +ASSERT_EQ (id,5 ,"anon_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"anon_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"rec_attr" ,5 ,-1 ),6 ,"anon_rec_attr" ); +ASSERT_EQ (btf__add_decl_attr (btf,"memb_attr" ,5 ,0 ),7 ,"anon_memb_attr" ); + id =btf__add_typedef (btf,"both" ,5 ); +ASSERT_EQ (id,8 ,"both_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"both_attr" ,8 ,-1 ),9 ,"both_attr" ); + +test_ctx__dump_and_compare (&t, +"typedef int td __attribute__((t1)) __attribute__((btf_decl_tag(\"t2\")));\n" +"\n" +"typedef struct {\n" +" int f __attribute__((memb_attr));\n" +"} __attribute__((rec_attr)) both __attribute__((both_attr));\n" +"\n" ,"dump_and_compare" ); + +test_ctx__free (&t); +} +static void test_btf_dump_type_tags (void ) {struct test_ctx t = {};
@@ -1109,6 +1438,18 @@ void test_btf_dump() {if (test__start_subtest ("btf_dump: incremental" ))test_btf_dump_incremental (); +if (test__start_subtest ("btf_dump: decl_tags" )) +test_btf_dump_decl_tags (); + +if (test__start_subtest ("btf_dump: decl_tags_incremental" )) +test_btf_dump_decl_tags_incremental (); + +if (test__start_subtest ("btf_dump: decl_tags_members" )) +test_btf_dump_decl_tags_members (); + +if (test__start_subtest ("btf_dump: decl_tags_typedef" )) +test_btf_dump_decl_tags_typedef (); +if (test__start_subtest ("btf_dump: type_tags" ))test_btf_dump_type_tags ();
Signature · 1 line
2.55.0
Re: [PATCH bpf-next v1 3/4] selftests/bpf: Test btf_dump rendering of decl_tags
Quoted 22 lines · Ihor Solodrai
Cover the decl tag rendering the previous patch added. One subtest builds a BTF with every shape that renders, plus two that must not:
- a named record with both forms of tag
- a typedef'd anonymous record, whose attribute precedes the declarator
- a union
- a record that also carries the derived packed attribute
- an anonymous record inlined at a member
- two records whose tags are added in reverse target order, which is what pins the sort: without it the earlier record loses its attribute
- attributes on a var and on a func, whichbtf_dumpnever declaresA second subtest appends types and attributes to a dumper that has already run. Its last phase adds an attribute for a record emitted two dumps earlier and checks nothing is rendered for it: an attribute is part of the type, and
btf_dumpemits each definition once.A third covers members. A member attribute and a record one share a key in the index, so it checks they do not leak into each other's position, over a function pointer, an array and a bit-field - declarators the attribute has to follow rather than precede.
Quoted 8 lines
A fourth covers typedefs, including a typedef of an anonymous record carrying three groups at once, binding to the member, the record and the typedef.
Switch
test_ctx__dump_and_compare()tocompare_text_to_expected()while we are here.Signed-off-by Ihor Solodrai <[email protected]>
Small suggestion; while this test is great at exercising the btf dump internals, adding a bpftool btf dump test to prog_tests/bpftool_btf_dump.c might help illustrate the change end-to-end (if you're planning this as part of the work to eliminate the push/pop attributes feel free to ignore). Thanks!
Quoted 372 lines
--- .../selftests/bpf/prog_tests/btf_dump.c | 343 +++++++++++++++++- 1 file changed, 342 insertions(+), 1 deletion(-)tools/testing/selftests/bpf/prog_tests/btf_dump.c +342
-1@@ -224,7 +224,12 @@ static void test_ctx__dump_and_compare(struct test_ctx *t,fflush (t->dump_buf_file); t->dump_buf[t->dump_buf_sz] =0 ;/* some libc implementations don't do this */ -ASSERT_STREQ (t->dump_buf, expected_output, message); +/* +* The mismatch has already been reported, so this only has to +* register the failure. ASSERT_OK() would append a stale errno to it. +*/ + err =compare_text_to_expected (t->dump_buf, expected_output); +ASSERT_EQ (err,0 , message); }static void test_btf_dump_incremental (void )@@ -328,6 +333,330 @@ static void test_btf_dump_incremental(void)test_ctx__free (&t); } +static void test_btf_dump_decl_tags (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + +/* +* Generate BTF corresponding to the following C code: +* +* struct s1 { int f; } __attribute__((attr1)) +* __attribute__((btf_decl_tag("plain_tag"))); +* typedef struct { int f; } __attribute__((a1)) __attribute__((a2)) t1; +* union u1 { int f; } __attribute__((uattr)); +* struct p1 { char c; int i; } __attribute__((packed)) __attribute__((pattr)); +* struct outer { struct { int g; } __attribute__((inner)) in; }; +* struct s2 { int f; } __attribute__((on_s2)); +* struct s3 { int f; } __attribute__((on_s3)); +* +* plus attributes on a var and a func, which btf_dump does not declare. +*/ + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_int (btf,"char" ,1 ,BTF_INT_CHAR ); +ASSERT_EQ (id,2 ,"char_id" ); + +/* a named record with one attribute of each form */ + id =btf__add_struct (btf,"s1" ,4 ); +ASSERT_EQ (id,3 ,"s1_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s1_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"attr1" ,3 ,-1 ),4 ,"s1_attr" ); +/* no kflag: rendered as a btf_decl_tag(), not as a bare attribute */ +ASSERT_EQ (btf__add_decl_tag (btf,"plain_tag" ,3 ,-1 ),5 ,"s1_plain_tag" ); + +/* +* an anonymous record behind a typedef, with two attributes. This is +* the case that cannot be expressed by a caller appending to the dump: +* the attribute has to go before the declarator. +*/ + id =btf__add_struct (btf,NULL ,4 ); +ASSERT_EQ (id,6 ,"anon_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"anon_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a1" ,6 ,-1 ),7 ,"anon_attr1" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a2" ,6 ,-1 ),8 ,"anon_attr2" ); + id =btf__add_typedef (btf,"t1" ,6 ); +ASSERT_EQ (id,9 ,"typedef_id" ); + +/* unions are records too */ + id =btf__add_union (btf,"u1" ,4 ); +ASSERT_EQ (id,10 ,"u1_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"u1_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"uattr" ,10 ,-1 ),11 ,"u1_attr" ); + +/* composes with the packed attribute, which is derived from layout */ + id =btf__add_struct (btf,"p1" ,5 ); +ASSERT_EQ (id,12 ,"p1_id" ); +ASSERT_OK (btf__add_field (btf,"c" ,2 ,0 ,0 ),"p1_field_c" ); +ASSERT_OK (btf__add_field (btf,"i" ,1 ,8 ,0 ),"p1_field_i" ); +ASSERT_EQ (btf__add_decl_attr (btf,"pattr" ,12 ,-1 ),13 ,"p1_attr" ); + +/* an anonymous record inlined at a member, the third emission position */ + id =btf__add_struct (btf,NULL ,4 ); +ASSERT_EQ (id,14 ,"inner_id" ); +ASSERT_OK (btf__add_field (btf,"g" ,1 ,0 ,0 ),"inner_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"inner" ,14 ,-1 ),15 ,"inner_attr" ); + id =btf__add_struct (btf,"outer" ,4 ); +ASSERT_EQ (id,16 ,"outer_id" ); +ASSERT_OK (btf__add_field (btf,"in" ,14 ,0 ,0 ),"outer_field" ); + +/* not rendered: a var has no declaration in the C output */ + id =btf__add_var (btf,"v" ,BTF_VAR_GLOBAL_ALLOCATED ,3 ); +ASSERT_EQ (id,17 ,"var_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"var_attr" ,17 ,-1 ),18 ,"var_attr" ); +/* not rendered: neither does a func */ + id =btf__add_func_proto (btf,1 ); +ASSERT_EQ (id,19 ,"proto_id" ); + id =btf__add_func (btf,"fn" ,BTF_FUNC_GLOBAL ,19 ); +ASSERT_EQ (id,20 ,"func_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"func_attr" ,20 ,-1 ),21 ,"func_attr" ); + +/* +* Tags added in reverse target order: the tag with the lower type ID +* names the record with the higher one, so the index is in emission +* order only once btf_dump_resize() has sorted it. Without the sort +* the lookup for s2 lands outside its run and drops the attribute. +*/ + id =btf__add_struct (btf,"s2" ,4 ); +ASSERT_EQ (id,22 ,"s2_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s2_field" ); + id =btf__add_struct (btf,"s3" ,4 ); +ASSERT_EQ (id,23 ,"s3_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s3_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"on_s3" ,23 ,-1 ),24 ,"s3_attr" ); +ASSERT_EQ (btf__add_decl_attr (btf,"on_s2" ,22 ,-1 ),25 ,"s2_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s1 {\n" +" int f;\n" +"} __attribute__((attr1)) __attribute__((btf_decl_tag(\"plain_tag\")));\n" +"\n" +"typedef struct {\n" +" int f;\n" +"} __attribute__((a1)) __attribute__((a2)) t1;\n" +"\n" +"union u1 {\n" +" int f;\n" +"} __attribute__((uattr));\n" +"\n" +"struct p1 {\n" +" char c;\n" +" int i;\n" +"} __attribute__((packed)) __attribute__((pattr));\n" +"\n" +"struct outer {\n" +" struct {\n" +" int g;\n" +" } __attribute__((inner)) in;\n" +"};\n" +"\n" +"struct s2 {\n" +" int f;\n" +"} __attribute__((on_s2));\n" +"\n" +"struct s3 {\n" +" int f;\n" +"} __attribute__((on_s3));\n" +"\n" ,"dump_and_compare" ); + +test_ctx__free (&t); +} + +/* +* btf_dump indexes decl attributes incrementally, only walking types appended +* since the last dump. Check that attributes on types added to an existing +* dumper are picked up, and that one added for a record that has already been +* emitted is not - btf_dump emits each definition once. +*/ +static void test_btf_dump_decl_tags_incremental (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_struct (btf,"s1" ,4 ); +ASSERT_EQ (id,2 ,"s1_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s1_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a1" ,2 ,-1 ),3 ,"s1_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s1 {\n" +" int f;\n" +"} __attribute__((a1));\n" +"\n" ,"first_dump" ); + +fseek (t.dump_buf_file,0 ,SEEK_SET ); + +/* a new record and a new attribute for it */ + id =btf__add_struct (btf,"s2" ,4 ); +ASSERT_EQ (id,4 ,"s2_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s2_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a2" ,4 ,-1 ),5 ,"s2_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s2 {\n" +" int f;\n" +"} __attribute__((a2));\n" +"\n" ,"second_dump" ); + +fseek (t.dump_buf_file,0 ,SEEK_SET ); + +/* s1 has already been emitted, so "late" is not rendered anywhere */ +ASSERT_EQ (btf__add_decl_attr (btf,"late" ,2 ,-1 ),6 ,"s1_late_attr" ); + id =btf__add_struct (btf,"s3" ,4 ); +ASSERT_EQ (id,7 ,"s3_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"s3_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"a3" ,7 ,-1 ),8 ,"s3_attr" ); + +test_ctx__dump_and_compare (&t, +"struct s3 {\n" +" int f;\n" +"} __attribute__((a3));\n" +"\n" ,"third_dump" ); + +test_ctx__free (&t); +} + +/* +* An attribute on a member is emitted at the member declaration, and one on +* the record at the closing brace. Both are keyed by the record in the index, +* so this also checks that they do not leak into each other's position. +*/ +static void test_btf_dump_decl_tags_members (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + +/* +* Generate BTF corresponding to the following C code: +* +* struct m1 { +* int (*fp)(void) __attribute__((fp_attr)); +* int a; +* int b __attribute__((b1)) __attribute__((b2)); +* int arr[4] __attribute__((arr_attr)); +* } __attribute__((rec)); +* struct bits { int x : 3 __attribute__((x_attr)); int y : 29; }; +*/ + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_array (btf,1 ,1 ,4 ); +ASSERT_EQ (id,2 ,"array_id" ); + id =btf__add_func_proto (btf,1 ); +ASSERT_EQ (id,3 ,"proto_id" ); + id =btf__add_ptr (btf,3 ); +ASSERT_EQ (id,4 ,"ptr_id" ); + + id =btf__add_struct (btf,"m1" ,32 ); +ASSERT_EQ (id,5 ,"m1_id" ); +ASSERT_OK (btf__add_field (btf,"fp" ,4 ,0 ,0 ),"m1_fp" ); +ASSERT_OK (btf__add_field (btf,"a" ,1 ,64 ,0 ),"m1_a" ); +ASSERT_OK (btf__add_field (btf,"b" ,1 ,96 ,0 ),"m1_b" ); +ASSERT_OK (btf__add_field (btf,"arr" ,2 ,128 ,0 ),"m1_arr" ); +ASSERT_EQ (btf__add_decl_attr (btf,"rec" ,5 ,-1 ),6 ,"m1_rec_attr" ); +/* the attribute follows the whole declarator, pointer and array alike */ +ASSERT_EQ (btf__add_decl_attr (btf,"fp_attr" ,5 ,0 ),7 ,"m1_fp_attr" ); +/* two on one member, to pin the ordering */ +ASSERT_EQ (btf__add_decl_attr (btf,"b1" ,5 ,2 ),8 ,"m1_b1" ); +ASSERT_EQ (btf__add_decl_attr (btf,"b2" ,5 ,2 ),9 ,"m1_b2" ); +ASSERT_EQ (btf__add_decl_attr (btf,"arr_attr" ,5 ,3 ),10 ,"m1_arr_attr" ); + +/* a bit-field: the attribute has to follow the width */ + id =btf__add_struct (btf,"bits" ,4 ); +ASSERT_EQ (id,11 ,"bits_id" ); +ASSERT_OK (btf__add_field (btf,"x" ,1 ,0 ,3 ),"bits_x" ); +ASSERT_OK (btf__add_field (btf,"y" ,1 ,3 ,29 ),"bits_y" ); +ASSERT_EQ (btf__add_decl_attr (btf,"x_attr" ,11 ,0 ),12 ,"bits_x_attr" ); + +test_ctx__dump_and_compare (&t, +"struct m1 {\n" +" int (*fp)(void) __attribute__((fp_attr));\n" +" int a;\n" +" int b __attribute__((b1)) __attribute__((b2));\n" +" int arr[4] __attribute__((arr_attr));\n" +"} __attribute__((rec));\n" +"\n" +"struct bits {\n" +" int x: 3 __attribute__((x_attr));\n" +" int y: 29;\n" +"};\n" +"\n" ,"dump_and_compare" ); + +test_ctx__free (&t); +} + +/* +* A typedef is the third and last kind btf_dump declares. Its attributes go +* after the declarator, so a typedef of an anonymous record can carry three +* groups in one declaration, binding to three different entities. +*/ +static void test_btf_dump_decl_tags_typedef (void ) +{ +struct test_ctx t = {}; +struct btf *btf; +int id; + +if (test_ctx__init (&t)) +return ; + + btf = t.btf; + +/* +* Generate BTF corresponding to the following C code: +* +* typedef int td __attribute__((t1)) +* __attribute__((btf_decl_tag("t2"))); +* typedef struct { +* int f __attribute__((memb_attr)); +* } __attribute__((rec_attr)) both __attribute__((both_attr)); +*/ + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" ); + id =btf__add_typedef (btf,"td" ,1 ); +ASSERT_EQ (id,2 ,"td_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"t1" ,2 ,-1 ),3 ,"td_attr1" ); +/* no kflag: rendered as a btf_decl_tag(), not as a bare attribute */ +ASSERT_EQ (btf__add_decl_tag (btf,"t2" ,2 ,-1 ),4 ,"td_attr2" ); + + id =btf__add_struct (btf,NULL ,4 ); +ASSERT_EQ (id,5 ,"anon_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"anon_field" ); +ASSERT_EQ (btf__add_decl_attr (btf,"rec_attr" ,5 ,-1 ),6 ,"anon_rec_attr" ); +ASSERT_EQ (btf__add_decl_attr (btf,"memb_attr" ,5 ,0 ),7 ,"anon_memb_attr" ); + id =btf__add_typedef (btf,"both" ,5 ); +ASSERT_EQ (id,8 ,"both_id" ); +ASSERT_EQ (btf__add_decl_attr (btf,"both_attr" ,8 ,-1 ),9 ,"both_attr" ); + +test_ctx__dump_and_compare (&t, +"typedef int td __attribute__((t1)) __attribute__((btf_decl_tag(\"t2\")));\n" +"\n" +"typedef struct {\n" +" int f __attribute__((memb_attr));\n" +"} __attribute__((rec_attr)) both __attribute__((both_attr));\n" +"\n" ,"dump_and_compare" ); + +test_ctx__free (&t); +} +static void test_btf_dump_type_tags (void ) {struct test_ctx t = {};@@ -1109,6 +1438,18 @@ void test_btf_dump() {if (test__start_subtest ("btf_dump: incremental" ))test_btf_dump_incremental (); +if (test__start_subtest ("btf_dump: decl_tags" )) +test_btf_dump_decl_tags (); + +if (test__start_subtest ("btf_dump: decl_tags_incremental" )) +test_btf_dump_decl_tags_incremental (); + +if (test__start_subtest ("btf_dump: decl_tags_members" )) +test_btf_dump_decl_tags_members (); + +if (test__start_subtest ("btf_dump: decl_tags_typedef" )) +test_btf_dump_decl_tags_typedef (); +if (test__start_subtest ("btf_dump: type_tags" ))test_btf_dump_type_tags ();
Ihor Solodrai
reviewerRe: [PATCH bpf-next v1 3/4] selftests/bpf: Test btf_dump rendering of decl_tags
Quoted 30 lines · Alan Maguire
Quoted 29 lines · Ihor Solodrai
Cover the decl tag rendering the previous patch added. One subtest builds a BTF with every shape that renders, plus two that must not:
- a named record with both forms of tag
- a typedef'd anonymous record, whose attribute precedes the declarator
- a union
- a record that also carries the derived packed attribute
- an anonymous record inlined at a member
- two records whose tags are added in reverse target order, which is what pins the sort: without it the earlier record loses its attribute
- attributes on a var and on a func, whichbtf_dumpnever declaresA second subtest appends types and attributes to a dumper that has already run. Its last phase adds an attribute for a record emitted two dumps earlier and checks nothing is rendered for it: an attribute is part of the type, and
btf_dumpemits each definition once.A third covers members. A member attribute and a record one share a key in the index, so it checks they do not leak into each other's position, over a function pointer, an array and a bit-field - declarators the attribute has to follow rather than precede.
A fourth covers typedefs, including a typedef of an anonymous record carrying three groups at once, binding to the member, the record and the typedef.
Switch
test_ctx__dump_and_compare()tocompare_text_to_expected()while we are here.
Quoted 7 lines
Quoted 1 line
Signed-off-by Ihor Solodrai <[email protected]>Small suggestion; while this test is great at exercising the btf dump internals, adding a bpftool btf dump test to
prog_tests/bpftool_btf_dump.cmight help illustrate the change end-to-end (if you're planning this as part of the work to eliminate the push/pop attributes feel free to ignore). Thanks!
Hi Alan. Thanks for taking a look.
Yes, that's exactly the plan. btf_dump.c contains tests for libbpf btf_dump primitives, and bpftool_btf_dump.c is for bpftool format c.
So more tests coming soon, no worries.
Quoted 9 lines
Quoted 8 lines
--- .../selftests/bpf/prog_tests/btf_dump.c | 343 +++++++++++++++++- 1 file changed, 342 insertions(+), 1 deletion(-)[...]
Re: [PATCH bpf-next v1 3/4] selftests/bpf: Test btf_dump rendering of decl_tags
In reply to Ihor SolodraiQuoted 26 lines · Ihor Solodrai
Cover the decl tag rendering the previous patch added. One subtest builds a BTF with every shape that renders, plus two that must not:
- a named record with both forms of tag
- a typedef'd anonymous record, whose attribute precedes the declarator
- a union
- a record that also carries the derived packed attribute
- an anonymous record inlined at a member
- two records whose tags are added in reverse target order, which is what pins the sort: without it the earlier record loses its attribute
- attributes on a var and on a func, whichbtf_dumpnever declaresA second subtest appends types and attributes to a dumper that has already run. Its last phase adds an attribute for a record emitted two dumps earlier and checks nothing is rendered for it: an attribute is part of the type, and
btf_dumpemits each definition once.A third covers members. A member attribute and a record one share a key in the index, so it checks they do not leak into each other's position, over a function pointer, an array and a bit-field - declarators the attribute has to follow rather than precede.
A fourth covers typedefs, including a typedef of an anonymous record carrying three groups at once, binding to the member, the record and the typedef.
Quoted 5 lines
Switch
test_ctx__dump_and_compare()tocompare_text_to_expected()while we are here.Signed-off-by Ihor Solodrai <[email protected]>---
Quoted 8 lines
.../selftests/bpf/prog_tests/btf_dump.c | 343 +++++++++++++++++- 1 file changed, 342 insertions(+), 1 deletion(-)tools/testing/selftests/bpf/prog_tests/btf_dump.c +0
-0
tools/testing/selftests/bpf/prog_tests/btf_dump.c static void test_ctx__dump_and_compare(struct test_ctx *t, · line 232
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/btf_dump.c +6
-1@@ -224,7 +224,12 @@ static void test_ctx__dump_and_compare(struct test_ctx *t,fflush (t->dump_buf_file); t->dump_buf[t->dump_buf_sz] =0 ;/* some libc implementations don't do this */ -ASSERT_STREQ (t->dump_buf, expected_output, message); +/* +* The mismatch has already been reported, so this only has to +* register the failure. ASSERT_OK() would append a stale errno to it. +*/ + err =compare_text_to_expected (t->dump_buf, expected_output); +ASSERT_EQ (err,0 , message);
Nit: Every caller of the compare_text_to_expected does such ASSERT,
let's embed the assert in the function and avoid the above comment.
...
tools/testing/selftests/bpf/prog_tests/btf_dump.c static void test_btf_dump_incremental(void) · line 477
Quoted 7 lines
tools/testing/selftests/bpf/prog_tests/btf_dump.c +7
-0+/* +* btf_dump indexes decl attributes incrementally, only walking types appended +* since the last dump. Check that attributes on types added to an existing +* dumper are picked up, and that one added for a record that has already been +* emitted is not - btf_dump emits each definition once. +*/ +static void test_btf_dump_decl_tags_incremental (void )
Nit: I'd drop this test or made it a part of an existing one (w/o the s3 part).
...
Ihor Solodrai
reviewerPATCH 4/4 selftests/bpf: Show the tag kflag in the raw BTF dump helper
In reply to Ihor SolodraiA kflag on BTF_KIND_DECL_TAG or BTF_KIND_TYPE_TAG means the tag string is an __attribute__ list rather than a btf_decl_tag()/btf_type_tag() argument. fprintf_btf_type_raw() did not print it, so a tag added with btf__add_decl_tag() and an attribute added with btf__add_decl_attr() dumped identically apart from their type IDs.
Print " kflag=1" for those two kinds, only when the flag is set, so no existing expected string changes.
Add a btf_write subtest for btf__add_decl_attr() and btf__add_type_attr().
--- tools/testing/selftests/bpf/btf_helpers.c | 10 +++- .../selftests/bpf/prog_tests/btf_write.c | 60 +++++++++++++++++++ 2 files changed, 67 insertions(+), 3 deletions(-)
tools/testing/selftests/bpf/btf_helpers.c +7 -3
@@ -111,9 +111,12 @@ int fprintf_btf_type_raw(FILE *out, const struct btf *btf, __u32 id)case BTF_KIND_VOLATILE :case BTF_KIND_RESTRICT :case BTF_KIND_TYPEDEF : -case BTF_KIND_TYPE_TAG :fprintf (out," type_id=%u" , t->type);break ; +case BTF_KIND_TYPE_TAG : +fprintf (out," type_id=%u%s" , t->type, +btf_kflag (t) ?" kflag=1" :"" ); +break ;case BTF_KIND_ARRAY : {const struct btf_array *arr =btf_array (t);
@@ -200,8 +203,9 @@ int fprintf_btf_type_raw(FILE *out, const struct btf *btf, __u32 id)fprintf (out," size=%u" , t->size);break ;case BTF_KIND_DECL_TAG : -fprintf (out," type_id=%u component_idx=%d" , - t->type,btf_decl_tag (t)->component_idx); +fprintf (out," type_id=%u component_idx=%d%s" , + t->type,btf_decl_tag (t)->component_idx, +btf_kflag (t) ?" kflag=1" :"" );break ;default :break ;
tools/testing/selftests/bpf/prog_tests/btf_write.c +60 -0
70 lines not on this page · raw message
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH bpf-next v1 4/4] selftests/bpf: Show the tag kflag in the raw BTF dump helper
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] bpf: NULL pointer dereference in test_btf_attrs on test failure --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] bpf: NULL pointer dereference in test_btf_attrs on test failure --
commit 551463268433576dd9cc5cc27a8ffb06a64ecabb
Author: Ihor Solodrai <[email protected]>
selftests/bpf: Show the tag kflag in the raw BTF dump helper
This commit updates fprintf_btf_type_raw() to print the kflag for BTF_KIND_DECL_TAG and BTF_KIND_TYPE_TAG types. It also adds a new subtest test_btf_attrs to prog_tests/btf_write.c to verify this behavior.
Quoted 5 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +0
-0@@ -606,6 +606,64 @@ static void test_btf_add_btf_split()
[ ... ]
Quoted 11 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +11
-0+static void test_btf_attrs (void ) +{ +const struct btf_type *t; +struct btf *btf; +int id; + + btf =btf__new_empty (); +if (!ASSERT_OK_PTR (btf,"new_empty" )) +return ; + + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED );
tools/testing/selftests/bpf/prog_tests/btf_write.c static void test_btf_add_btf_split() · line 629
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +10
-0+ASSERT_EQ (id,1 ,"int_id" ); + + id =btf__add_struct (btf,"s" ,4 ); +ASSERT_EQ (id,2 ,"struct_id" ); +ASSERT_OK (btf__add_field (btf,"f" ,1 ,0 ,0 ),"field_ok" ); + + id =btf__add_decl_tag (btf,"tag" ,2 ,-1 ); +ASSERT_EQ (id,3 ,"decl_tag_id" ); + t =btf__type_by_id (btf,3 ); +ASSERT_EQ (btf_kind (t),BTF_KIND_DECL_TAG ,"decl_tag_kind" );
[Severity: Medium]
In BPF selftests, ASSERT_EQ() logs a failure but does not abort the execution of the test block. If btf__add_decl_tag() fails, the subsequent call to btf__type_by_id() will return a NULL pointer.
Will this cause a NULL pointer dereference when t is passed to btf_kind(), which unconditionally dereferences it?
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +4
-0+ASSERT_EQ (btf_kflag (t),0 ,"decl_tag_kflag" ); + + id =btf__add_decl_attr (btf,"tag" ,2 ,-1 ); +ASSERT_EQ (id,4 ,"decl_attr_id" );
tools/testing/selftests/bpf/prog_tests/btf_write.c static void test_btf_add_btf_split() · line 643
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +10
-0+ t =btf__type_by_id (btf,4 ); +ASSERT_EQ (btf_kind (t),BTF_KIND_DECL_TAG ,"decl_attr_kind" ); +ASSERT_EQ (btf_kflag (t),1 ,"decl_attr_kflag" ); +ASSERT_EQ (t->type,2 ,"decl_attr_type" ); +ASSERT_EQ (btf_decl_tag (t)->component_idx,-1 ,"decl_attr_component_idx" ); + + id =btf__add_decl_attr (btf,"member_attr" ,2 ,0 ); +ASSERT_EQ (id,5 ,"member_attr_id" ); +ASSERT_EQ (btf_decl_tag (btf__type_by_id (btf,5 ))->component_idx,0 , +"member_attr_component_idx" );
[Severity: Medium]
Similarly, could chaining a potential NULL return from btf__type_by_id() directly into btf_decl_tag() lead to an out-of-bounds dereference due to pointer arithmetic on NULL?
tools/testing/selftests/bpf/prog_tests/btf_write.c static void test_btf_add_btf_split() · line 647
Quoted 4 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +4
-0+ + id =btf__add_type_tag (btf,"tag" ,1 ); +ASSERT_EQ (id,6 ,"type_tag_id" ); +ASSERT_EQ (btf_kflag (btf__type_by_id (btf,6 )),0 ,"type_tag_kflag" );
[Severity: Medium]
Is it possible that calling btf_kflag() on a potential NULL pointer from btf__type_by_id() results in a segmentation fault if the type addition failed?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH bpf-next v1 4/4] selftests/bpf: Show the tag kflag in the raw BTF dump helper
In reply to Ihor SolodraiQuoted 6 lines · Ihor Solodrai
A kflag on
BTF_KIND_DECL_TAGorBTF_KIND_TYPE_TAGmeans the tag string is an__attribute__list rather than a btf_decl_tag()/btf_type_tag() argument.fprintf_btf_type_raw()did not print it, so a tag added withbtf__add_decl_tag()and an attribute added withbtf__add_decl_attr()dumped identically apart from their type IDs.
Quoted 8 lines
Print " kflag=1" for those two kinds, only when the flag is set, so no existing expected string changes.
Add a
btf_writesubtest forbtf__add_decl_attr()andbtf__add_type_attr().Signed-off-by Ihor Solodrai <[email protected]>---
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +2
-0@@ -606,6 +606,64 @@ static void test_btf_add_btf_split()btf__free (base); } +static void test_btf_attrs (void ) +{
tools/testing/selftests/bpf/prog_tests/btf_write.c static void test_btf_add_btf_split() · line 620
Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/btf_write.c +10
-0+const struct btf_type *t; +struct btf *btf; +int id; + + btf =btf__new_empty (); +if (!ASSERT_OK_PTR (btf,"new_empty" )) +return ; + + id =btf__add_int (btf,"int" ,4 ,BTF_INT_SIGNED ); +ASSERT_EQ (id,1 ,"int_id" );
Note: we have a set of utility macro BTF_*_ENC macros in test_btf.h,
these are more convenient to use in some contexts. (At-least there is no need to assert every time).
...