Skip to contentSkip to conversation

Support for new btf_type_tag encoding

PATCH Superseded Eduard Zingerman 3 participants 6 messages Mar 13, 2023 – Mar 14, 2023 BPF

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Eduard Zingerman

author maintainer
#0 lore raw

cover Support for new btf_type_tag encoding

In recent discussion in BPF mailing list ([1]) participants agreed to add a new DWARF representation for "btf_type_tag" annotations.

Existing representation is DW_TAG_LLVM_annotation object attached as a child to a DW_TAG_pointer_type. It means that "btf_type_tag" annotation is attached to a pointee type.

New representation is DW_TAG_LLVM_annotation object attached as a child to *any* type. It means that "btf_type_tag" annotation is attached to the parent type.

Here is an example:

    int __attribute__((btf_type_tag("tag1"))) *g;

And corresponding DWARF:

    0x0000001e:   DW_TAG_variable
                    DW_AT_name      ("g")
                    DW_AT_type      (0x00000029 "int *")

    0x00000029:   DW_TAG_pointer_type
                    DW_AT_type      (0x00000032 "int")

    0x0000002e:     DW_TAG_LLVM_annotation
                      DW_AT_name    ("btf_type_tag")
                      DW_AT_const_value     ("tag1")

    0x00000032:   DW_TAG_base_type
                    DW_AT_name      ("int")

This patch adds logic necessary to handle such annotations in the pahole tool. Examples like below are supported:

  #define __tag(val) __attribute__((btf_type_tag("__" #val)))
  struct      alpha {};
  union       bravo {};
  enum        charlie { X };

typedef int delta;

  struct echo {
    int          * __tag(a)  a;
    int            __tag(b) *b;
    int            __tag(c)  c;
    void           __tag(d) *d;
    void           __tag(e) *e;
    struct alpha   __tag(f)  f;
    union  bravo   __tag(g)  g;
    enum   charlie __tag(h)  h;
    delta          __tag(i)  i;
    int __tag(j_result) (__tag(j) *j)(int __tag(j_param));
  } g;

Implementation details
----------------------

Although this was not discussed in the mailing list, the proposed implementation acts in a following way (for compatibility reasons):
- both forms could be present in the debug info;
- if any annotations corresponding to the new form are present in the debug info, annotations corresponding to the old form are ignored.

Because the new form of type tags could be applied to any type it is somewhat invasive from implementation point of view:
- Types struct btf_type_tag_type and struct llvm_annotation are consolidated as a single type struct llvm_annotation, in order to reside in a single annots list associated with struct/union/enum or typedef.
- The annots list, used to hold references to annotation objects, is put directly in struct tag.
- At the load phase it is not yet known whether new or old form is used for type tags encoding, so struct llvm_annotation objects are not added to the types table.
- The recode phase now consists of several steps:
- depending on the results of the load phase, struct llvm_annotation objects corresponding to either old or new representations are added to the types table;
- tag__recode_dwarf_type() is executed as usual, but it also

    collects information about type tags;

- references to types that have type tag annotations are updated to point to the first annotation object.

Corresponding clang changes are tracked in [6].

Testing
-------

To verify the changes I used the following:
- Tools:

  - "LLVM-main"   :: LLVM at revision [3];
  - "LLVM-new"    :: LLVM at revision [3] with patches [6] applied;
  - "gcc"         :: GCC version 11.3 (no support for btf_type_tag annotations);

- "pahole-next" :: dwarves at revision [4];
- "pahole-new" :: dwarves at revision [4] + this patch,
- "kernel" :: Linux Kernel bpf-next branch at revision [5]
- test cases:
- kernel build;
- kernel BPF test cases build, BPF tests execution

    (test_verifier, test_progs, test_progs-no_alu32, test_maps);

- btfdiff script (suggested by Arnaldo, [2]).
- tool combinations (kernel compiler / clang for BPF tests / pahole version):
- LLVM-main / LLVM-main / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok (modulo diff #1, see below)
- gcc / LLVM-main / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok
- LLVM-new / LLVM-new / pahole-next
- kernel build : ok with warnings (see warn #1 below)
- bpf tests : ok
- btfdiff : fails (see diff #2 below)
- LLVM-new / LLVM-new / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok (modulo diff #1, see below)
- gcc / LLVM-new / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok

Note on BPF tests: test case verif_scale_loop6 fails for all configurations above and 'LLVM-main / LLVM-main / pahole-next', thus I consider this issue as unrelated.

Diff #1: Difference in flexible printing, several occurrences as below:

@ -10531,7 +10531,7 @ struct bpf_cand_cache {
struct {
const struct btf * btf; /* 16 8 */
u32 id; /* 24 4 */
- } cands[0]; /* 16 0 */
+ } cands[]; /* 16 0 */

          /* size: 16, cachelines: 1, members: 5 */
          /* last cacheline: 16 bytes */

Diff #2: pahole-next does not know how to print type name for types with type tags, when reading from BTF:

@ -72998,8 +73022,8 @ struct sock {
/* --- cacheline 19 boundary (1216 bytes) --- */
int (*sk_backlog_rcv)(struct sock *, struct sk_buff *); /* 1216 8 */
void (*sk_destruct)(struct sock *); /* 1224 8 */

  -       rcu *                      sk_reuseport_cb;      /*  1232     8 */
  -       rcu *                      sk_bpf_storage;       /*  1240     8 */
  +       <ERROR                    > sk_reuseport_cb;     /*  1232     8 */
  +       <ERROR                    > sk_bpf_storage;      /*  1240     8 */

(Also DWARF names refer to TYPE_TAG, not actual type name, fixed in pahole-new).

Warn #1: pahole-next complains about unexpected child tags generated by clang, e.g.:

die__create_new_tag: unspecified_type WITH children!
die__create_new_base_type: DW_TAG_base_type WITH children!

Performance impact
------------------

The update to struct tag might raise concerns regarding memory usage, additional steps in recode phase might raise concerns regarding execution time. Below is statistics collected for Kernel BTF generation.

LLVM-new / LLVM-new / pahole-new:

$ /usr/bin/time -v pahole -J --btf_gen_floats -j --lang_exclude=rust .tmp_vmlinux.btf
    ...
	User time (seconds): 22.29
	System time (seconds): 0.47
	Percent of CPU this job got: 483%
    ...
	Maximum resident set size (kbytes): 714524
    ...

LLVM-new / LLVM-new / pahole-next:

$ /usr/bin/time -v pahole -J --btf_gen_floats -j --lang_exclude=rust .tmp_vmlinux.btf
    ...
	User time (seconds): 20.96
	System time (seconds): 0.44
	Percent of CPU this job got: 473%
    ...
	Maximum resident set size (kbytes): 700848
    ...

Links & revisions
-----------------

[1] Mailing list discussion regarding btf:type_tag

    https://lore.kernel.org/bpf/[email protected]/
[2] Suggestion to use btfdiff
    https://lore.kernel.org/dwarves/[email protected]/T/#mddbfe661e339485fb2b0e706b31329b46bf61bda

[3] f759275c1c8e ("[AMDGPU] Regenerate sdwa-peephole.ll")
[4] a9498899109d ("dwarf_loader: Support for btf:type_tag")
[5] 49b5300f1f8f ("Merge branch 'Support stashing local kptrs with bpf_kptr_xchg'")
[6] LLVM changes to generate btf:type_tag, revisions stack:

    https://reviews.llvm.org/D143966
    https://reviews.llvm.org/D143967
    https://reviews.llvm.org/D145891

Eduard Zingerman (1):
dwarf_loader: Support for btf:type_tag

 btf_encoder.c     |  13 +-
 btf_loader.c      |  15 +-
 dwarf_loader.c    | 763 +++++++++++++++++++++++++++++++++++++---------
 dwarves.c         |   1 +
 dwarves.h         |  68 +++--
 dwarves_fprintf.c |  13 +
 6 files changed, 693 insertions(+), 180 deletions(-)
Signature · 1 line
2.39.1
author maintainer
#1 lore raw

PATCH 1/1 dwarf_loader: Support for btf:type_tag

"btf:type_tag" is an DW_TAG_LLVM_annotation object that encodes btf_type_tag attributes in DWARF. Contrary to existing "btf_type_tag" it allows to associate such attributes with non-pointer types. When "btf:type_tag" is attached to a type it applies to this type.

For example the following C code:

    struct echo {
      int __attribute__((btf_type_tag("__c"))) c;
    }

Produces the following DWARF:

0x29: DW_TAG_structure_type
DW_AT_name ("echo")

0x40: DW_TAG_member

            DW_AT_name    ("c")
            DW_AT_type    (0x8c "int")

0x8c: DW_TAG_base_type

          DW_AT_name      ("int")
          DW_AT_encoding  (DW_ATE_signed)
          DW_AT_byte_size (0x04)

0x90: DW_TAG_LLVM_annotation

            DW_AT_name    ("btf:type_tag")
            DW_AT_const_value     ("__c")

Meaning that type 0x8c is an int with type tag __c. Corresponding BTF looks as follows:

[1] STRUCT 'echo'

        ...
        'c' type_id=8 bits_offset=128

[4] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[8] TYPE_TAG '__c' type_id=4

Void pointers with type tag annotations are represented as objects of unspecified type with name "void", here is an example:

  struct st {
    void __attribute__((btf_type_tag("__d"))) *d;
  }

And corresponding DWARF:

0x29: DW_TAG_structure_type

            DW_AT_name      ("st")

0x49: DW_TAG_member

              DW_AT_name    ("d")
              DW_AT_type    (0xa6 "void *")

0xa6: DW_TAG_pointer_type

            DW_AT_type      (0xaf "void")

0xaf: DW_TAG_unspecified_type
DW_AT_name ("void")

0xb1: DW_TAG_LLVM_annotation

              DW_AT_name    ("btf:type_tag")
              DW_AT_const_value     ("__d")

This commit adds support for DW_TAG_LLVM_annotation "btf:type_tag" attached to the following entities:
- base types;
- arrays;
- pointers;
- structs
- unions;
- enums;
- typedefs.

This is achieved via the following modifications:
- Types struct btf_type_tag_type and struct llvm_annotation are consolidated as a single type struct llvm_annotation, in order to reside in a single annots list associated with struct/union/enum or typedef.
- struct unspecified_type is added to handle void * types annotated with btf:type_tag.
- struct tag is extended with annots list.
- DWARF load phase is modified to fill annots fields for the above mentioned types.
- Recode phase is split in two sub-phases:
- The existing tag__recode_dwarf_type() is executed first;
- Newly added update_btf_type_tag_refs() is executed to update ->type field of each tag if that type refers to an object with btf:type_tag annotation. The id of the type is replaced by id of the type tag.
- When btf:type_tag annotations are present in compilation unit, all btf_type_tag annotations are ignored during BTF dump.

See also:
[1] Mailing list discussion regarding btf:type_tag

    https://lore.kernel.org/bpf/[email protected]/
---
 btf_encoder.c     |  13 +-
 btf_loader.c      |  15 +-
 dwarf_loader.c    | 763 +++++++++++++++++++++++++++++++++++++---------
 dwarves.c         |   1 +
 dwarves.h         |  68 +++--
 dwarves_fprintf.c |  13 +
 6 files changed, 693 insertions(+), 180 deletions(-)
btf_encoder.c +11 -2
index 07a9dc5..b74911b 100644
@@ -899,6 +899,9 @@ static int32_t btf_encoder__add_func(struct btf_encoder *encoder, struct functio
 		return -1;
 	}
 	list_for_each_entry(annot, &fn->annots, node) {
+		if (annot->kind != BTF_DECL_TAG)
+			continue;
+
 		tag_type_id = btf_encoder__add_decl_tag(encoder, annot->value, btf_fn_id,
 							annot->component_idx);
 		if (tag_type_id < 0) {
@@ -1180,7 +1183,7 @@ static int btf_encoder__encode_tag(struct btf_encoder *encoder, struct tag *tag,
 		name = namespace__name(tag__namespace(tag));
 		return btf_encoder__add_ref_type(encoder, BTF_KIND_TYPEDEF, ref_type_id, name, false);
 	case DW_TAG_LLVM_annotation:
-		name = tag__btf_type_tag(tag)->value;
+		name = tag__llvm_annotation(tag)->value;
 		return btf_encoder__add_ref_type(encoder, BTF_KIND_TYPE_TAG, ref_type_id, name, false);
 	case DW_TAG_structure_type:
 	case DW_TAG_union_type:
@@ -1605,6 +1608,9 @@ static int btf_encoder__encode_cu_variables(struct btf_encoder *encoder)
 		}
 
 		list_for_each_entry(annot, &var->annots, node) {
+			if (annot->kind != BTF_DECL_TAG)
+				continue;
+
 			int tag_type_id = btf_encoder__add_decl_tag(encoder, annot->value, id, annot->component_idx);
 			if (tag_type_id < 0) {
 				fprintf(stderr, "error: failed to encode tag '%s' to variable '%s' with component_idx %d\n",
@@ -1798,7 +1804,10 @@ int btf_encoder__encode_cu(struct btf_encoder *encoder, struct cu *cu, struct co
 
 		btf_type_id = encoder->type_id_off + core_id;
 		ns = tag__namespace(pos);
-		list_for_each_entry(annot, &ns->annots, node) {
+		list_for_each_entry(annot, &ns->tag.annots, node) {
+			if (annot->kind != BTF_DECL_TAG)
+				continue;
+
 			tag_type_id = btf_encoder__add_decl_tag(encoder, annot->value, btf_type_id, annot->component_idx);
 			if (tag_type_id < 0) {
 				fprintf(stderr, "error: failed to encode tag '%s' to %s '%s' with component_idx %d\n",
btf_loader.c +11 -4
index e579323..3fe07d0 100644
@@ -429,10 +429,11 @@ static int create_new_tag(struct cu *cu, int type, const struct btf_type *tp, ui
 		return -ENOMEM;
 
 	switch (type) {
-	case BTF_KIND_CONST:	tag->tag = DW_TAG_const_type;	 break;
-	case BTF_KIND_PTR:	tag->tag = DW_TAG_pointer_type;  break;
-	case BTF_KIND_RESTRICT:	tag->tag = DW_TAG_restrict_type; break;
-	case BTF_KIND_VOLATILE:	tag->tag = DW_TAG_volatile_type; break;
+	case BTF_KIND_CONST:	tag->tag = DW_TAG_const_type;	   break;
+	case BTF_KIND_PTR:	tag->tag = DW_TAG_pointer_type;    break;
+	case BTF_KIND_RESTRICT:	tag->tag = DW_TAG_restrict_type;   break;
+	case BTF_KIND_VOLATILE:	tag->tag = DW_TAG_volatile_type;   break;
+	case BTF_KIND_TYPE_TAG:	tag->tag = DW_TAG_LLVM_annotation; break;
 	default:
 		free(tag);
 		printf("%s: Unknown type %d\n\n", __func__, type);
@@ -489,6 +490,12 @@ static int btf__load_types(struct btf *btf, struct cu *cu)
 		case BTF_KIND_PTR:
 		case BTF_KIND_CONST:
 		case BTF_KIND_RESTRICT:
+		/* For type tag it's a bit of a lie.
+		 * In DWARF it is encoded as a child tag of whatever type it
+		 * applies to. Here we load it as a standalone tag with a pointer
+		 * to a next type only to have a valid ID in the types table.
+		 */
+		case BTF_KIND_TYPE_TAG:
 			err = create_new_tag(cu, type, type_ptr, type_index);
 			break;
 		case BTF_KIND_UNKN:
dwarf_loader.c +615 -148

1,080 lines not on this page · raw message

dwarves.c +1 -0
index b43031c..7e66a98 100644
@@ -681,6 +681,7 @@ struct cu *cu__new(const char *name, uint8_t addr_size,
 		cu->dfops	= NULL;
 		INIT_LIST_HEAD(&cu->tags);
 		INIT_LIST_HEAD(&cu->tool_list);
+		INIT_LIST_HEAD(&cu->unspecified_types);
 
 		cu->addr_size = addr_size;
 		cu->extra_dbg_info = 0;
dwarves.h +42 -26

126 lines not on this page · raw message

dwarves_fprintf.c +13 -0
index 30355b4..baa5924 100644
@@ -571,6 +571,7 @@ static const char *__tag__name(const struct tag *tag, const struct cu *cu,
 	case DW_TAG_restrict_type:
 	case DW_TAG_atomic_type:
 	case DW_TAG_unspecified_type:
+	case DW_TAG_LLVM_annotation:
 		type = cu__type(cu, tag->type);
 		if (type == NULL && tag->type != 0)
 			tag__id_not_found_snprintf(bf, len, tag->type);
@@ -782,6 +783,10 @@ next_type:
 			n = tag__has_type_loop(type, ptype, NULL, 0, fp);
 			if (n)
 				return printed + n;
+			if (ptype->tag == DW_TAG_LLVM_annotation) {
+				type = ptype;
+				goto next_type;
+			}
 			if (ptype->tag == DW_TAG_subroutine_type) {
 				printed += ftype__fprintf(tag__ftype(ptype),
 							  cu, name, 0, 1,
@@ -874,6 +879,14 @@ print_modifier: {
 		else
 			printed += enumeration__fprintf(type, &tconf, fp);
 		break;
+	case DW_TAG_LLVM_annotation: {
+		struct tag *ttype = cu__type(cu, type->type);
+		if (ttype) {
+			type = ttype;
+			goto next_type;
+		}
+		goto out_type_not_found;
+	}
 	}
 out:
 	if (type_expanded)
Signature · 1 line
2.39.1
#4 lore raw
In reply to Eduard Zingerman

Em Mon, Mar 13, 2023 at 04:17:43AM +0200, Eduard Zingerman escreveu:

Quoted 55 lines

In recent discussion in BPF mailing list ([1]) participants agreed to add a new DWARF representation for "btf_type_tag" annotations.

Existing representation is DW_TAG_LLVM_annotation object attached as a child to a DW_TAG_pointer_type. It means that "btf_type_tag" annotation is attached to a pointee type.

New representation is DW_TAG_LLVM_annotation object attached as a child to *any* type. It means that "btf_type_tag" annotation is attached to the parent type.

Here is an example:

    int __attribute__((btf_type_tag("tag1"))) *g;

And corresponding DWARF:

    0x0000001e:   DW_TAG_variable
                    DW_AT_name      ("g")
                    DW_AT_type      (0x00000029 "int *")

    0x00000029:   DW_TAG_pointer_type
                    DW_AT_type      (0x00000032 "int")

    0x0000002e:     DW_TAG_LLVM_annotation
                      DW_AT_name    ("btf_type_tag")
                      DW_AT_const_value     ("tag1")

    0x00000032:   DW_TAG_base_type
                    DW_AT_name      ("int")

This patch adds logic necessary to handle such annotations in the pahole tool. Examples like below are supported:

  #define __tag(val) __attribute__((btf_type_tag("__" #val)))
  struct      alpha {};
  union       bravo {};
  enum        charlie { X };

typedef int delta;

  struct echo {
    int          * __tag(a)  a;
    int            __tag(b) *b;
    int            __tag(c)  c;
    void           __tag(d) *d;
    void           __tag(e) *e;
    struct alpha   __tag(f)  f;
    union  bravo   __tag(g)  g;
    enum   charlie __tag(h)  h;
    delta          __tag(i)  i;
    int __tag(j_result) (__tag(j) *j)(int __tag(j_param));
  } g;
Quoted 8 lines

Implementation details
----------------------

Although this was not discussed in the mailing list, the proposed implementation acts in a following way (for compatibility reasons):
- both forms could be present in the debug info;
- if any annotations corresponding to the new form are present in the debug info, annotations corresponding to the old form are ignored.

agreed

Quoted 2 lines

Because the new form of type tags could be applied to any type it is somewhat invasive from implementation point of view:

Can you please see if you can break this patch into a series so that it helps with reviewing and bisection?

Quoted 86 lines

- Types struct btf_type_tag_type and struct llvm_annotation are consolidated as a single type struct llvm_annotation, in order to reside in a single annots list associated with struct/union/enum or typedef.
- The annots list, used to hold references to annotation objects, is put directly in struct tag.
- At the load phase it is not yet known whether new or old form is used for type tags encoding, so struct llvm_annotation objects are not added to the types table.
- The recode phase now consists of several steps:
- depending on the results of the load phase, struct llvm_annotation objects corresponding to either old or new representations are added to the types table;
- tag__recode_dwarf_type() is executed as usual, but it also

    collects information about type tags;

- references to types that have type tag annotations are updated to point to the first annotation object.

Corresponding clang changes are tracked in [6].

Testing
-------

To verify the changes I used the following:
- Tools:

  - "LLVM-main"   :: LLVM at revision [3];
  - "LLVM-new"    :: LLVM at revision [3] with patches [6] applied;
  - "gcc"         :: GCC version 11.3 (no support for btf_type_tag annotations);

- "pahole-next" :: dwarves at revision [4];
- "pahole-new" :: dwarves at revision [4] + this patch,
- "kernel" :: Linux Kernel bpf-next branch at revision [5]
- test cases:
- kernel build;
- kernel BPF test cases build, BPF tests execution

    (test_verifier, test_progs, test_progs-no_alu32, test_maps);

- btfdiff script (suggested by Arnaldo, [2]).
- tool combinations (kernel compiler / clang for BPF tests / pahole version):
- LLVM-main / LLVM-main / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok (modulo diff #1, see below)
- gcc / LLVM-main / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok
- LLVM-new / LLVM-new / pahole-next
- kernel build : ok with warnings (see warn #1 below)
- bpf tests : ok
- btfdiff : fails (see diff #2 below)
- LLVM-new / LLVM-new / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok (modulo diff #1, see below)
- gcc / LLVM-new / pahole-new
- kernel build : ok
- bpf tests : ok
- btfdiff : ok

Note on BPF tests: test case verif_scale_loop6 fails for all configurations above and 'LLVM-main / LLVM-main / pahole-next', thus I consider this issue as unrelated.

Diff #1: Difference in flexible printing, several occurrences as below:

@ -10531,7 +10531,7 @ struct bpf_cand_cache {
struct {
const struct btf * btf; /* 16 8 */
u32 id; /* 24 4 */
- } cands[0]; /* 16 0 */
+ } cands[]; /* 16 0 */

          /* size: 16, cachelines: 1, members: 5 */
          /* last cacheline: 16 bytes */

Diff #2: pahole-next does not know how to print type name for types with type tags, when reading from BTF:

@ -72998,8 +73022,8 @ struct sock {
/* --- cacheline 19 boundary (1216 bytes) --- */
int (*sk_backlog_rcv)(struct sock *, struct sk_buff *); /* 1216 8 */
void (*sk_destruct)(struct sock *); /* 1224 8 */

  -       rcu *                      sk_reuseport_cb;      /*  1232     8 */
  -       rcu *                      sk_bpf_storage;       /*  1240     8 */
  +       <ERROR                    > sk_reuseport_cb;     /*  1232     8 */
  +       <ERROR                    > sk_bpf_storage;      /*  1240     8 */
Quoted 7 lines

(Also DWARF names refer to TYPE_TAG, not actual type name, fixed in pahole-new).

Warn #1: pahole-next complains about unexpected child tags generated by clang, e.g.:

die__create_new_tag: unspecified_type WITH children!
die__create_new_base_type: DW_TAG_base_type WITH children!

Sure, we can remove those if the children is the expected one, leaving the warning maybe for debug sessions.

Thanks for the detailed implementation notes, references and tests performed, please consider breaking it up into smaller pieces or ellaborate on why you think can't be done.

Thanks!

- Arnaldo

Quoted 60 lines

Performance impact
------------------

The update to struct tag might raise concerns regarding memory usage, additional steps in recode phase might raise concerns regarding execution time. Below is statistics collected for Kernel BTF generation.

LLVM-new / LLVM-new / pahole-new:

$ /usr/bin/time -v pahole -J --btf_gen_floats -j --lang_exclude=rust .tmp_vmlinux.btf
    ...
	User time (seconds): 22.29
	System time (seconds): 0.47
	Percent of CPU this job got: 483%
    ...
	Maximum resident set size (kbytes): 714524
    ...

LLVM-new / LLVM-new / pahole-next:

$ /usr/bin/time -v pahole -J --btf_gen_floats -j --lang_exclude=rust .tmp_vmlinux.btf
    ...
	User time (seconds): 20.96
	System time (seconds): 0.44
	Percent of CPU this job got: 473%
    ...
	Maximum resident set size (kbytes): 700848
    ...

Links & revisions
-----------------

[1] Mailing list discussion regarding btf:type_tag

    https://lore.kernel.org/bpf/[email protected]/
[2] Suggestion to use btfdiff
    https://lore.kernel.org/dwarves/[email protected]/T/#mddbfe661e339485fb2b0e706b31329b46bf61bda

[3] f759275c1c8e ("[AMDGPU] Regenerate sdwa-peephole.ll")
[4] a9498899109d ("dwarf_loader: Support for btf:type_tag")
[5] 49b5300f1f8f ("Merge branch 'Support stashing local kptrs with bpf_kptr_xchg'")
[6] LLVM changes to generate btf:type_tag, revisions stack:

    https://reviews.llvm.org/D143966
    https://reviews.llvm.org/D143967
    https://reviews.llvm.org/D145891

Eduard Zingerman (1):
dwarf_loader: Support for btf:type_tag

 btf_encoder.c     |  13 +-
 btf_loader.c      |  15 +-
 dwarf_loader.c    | 763 +++++++++++++++++++++++++++++++++++++---------
 dwarves.c         |   1 +
 dwarves.h         |  68 +++--
 dwarves_fprintf.c |  13 +
 6 files changed, 693 insertions(+), 180 deletions(-)
Signature · 1 line
2.39.1
Signature · 1 line
- Arnaldo
author maintainer
#5 lore raw

On Mon, 2023-03-13 at 17:32 -0300, Arnaldo Carvalho de Melo wrote:
[...]

Quoted 9 lines
Quoted 9 lines

@ -72998,8 +73022,8 @ struct sock {
/* --- cacheline 19 boundary (1216 bytes) --- */
int (*sk_backlog_rcv)(struct sock *, struct sk_buff *); /* 1216 8 */
void (*sk_destruct)(struct sock *); /* 1224 8 */

  -       rcu *                      sk_reuseport_cb;      /*  1232     8 */
  -       rcu *                      sk_bpf_storage;       /*  1240     8 */
  +       <ERROR                    > sk_reuseport_cb;     /*  1232     8 */
  +       <ERROR                    > sk_bpf_storage;      /*  1240     8 */
Quoted 7 lines
Quoted 7 lines

(Also DWARF names refer to TYPE_TAG, not actual type name, fixed in pahole-new).

Warn #1: pahole-next complains about unexpected child tags generated by clang, e.g.:

die__create_new_tag: unspecified_type WITH children!
die__create_new_base_type: DW_TAG_base_type WITH children!

Hi Arnaldo,

Quoted 2 lines

Sure, we can remove those if the children is the expected one, leaving the warning maybe for debug sessions.

Yes, I handle it in the pahole-new. Warnings are not reported for DW_TAG_LLVM_annotation children when these are applicable.

Quoted 4 lines

Thanks for the detailed implementation notes, references and tests performed, please consider breaking it up into smaller pieces or ellaborate on why you think can't be done.

I can split the patch into several parts,
it would be a bit stretched, though:
1. change in btf_loader / dwarves_fprintf to correctly print names for types with type tags;
2. consolidation of struct btf_type_tag_type and struct llvm_annotation;
3. the logic to handle "btf:type_tags";
4. support for "void __tag*" as unspecified type.

Part #3 would still be quite big, +500 LoC or something like this. Will submit v2 today or tomorrow.

Thanks,
Eduard

Quoted 66 lines

Thanks!

- Arnaldo

Quoted 60 lines

Performance impact
------------------

The update to struct tag might raise concerns regarding memory usage, additional steps in recode phase might raise concerns regarding execution time. Below is statistics collected for Kernel BTF generation.

LLVM-new / LLVM-new / pahole-new:

$ /usr/bin/time -v pahole -J --btf_gen_floats -j --lang_exclude=rust .tmp_vmlinux.btf
    ...
	User time (seconds): 22.29
	System time (seconds): 0.47
	Percent of CPU this job got: 483%
    ...
	Maximum resident set size (kbytes): 714524
    ...

LLVM-new / LLVM-new / pahole-next:

$ /usr/bin/time -v pahole -J --btf_gen_floats -j --lang_exclude=rust .tmp_vmlinux.btf
    ...
	User time (seconds): 20.96
	System time (seconds): 0.44
	Percent of CPU this job got: 473%
    ...
	Maximum resident set size (kbytes): 700848
    ...

Links & revisions
-----------------

[1] Mailing list discussion regarding btf:type_tag

    https://lore.kernel.org/bpf/[email protected]/
[2] Suggestion to use btfdiff
    https://lore.kernel.org/dwarves/[email protected]/T/#mddbfe661e339485fb2b0e706b31329b46bf61bda

[3] f759275c1c8e ("[AMDGPU] Regenerate sdwa-peephole.ll")
[4] a9498899109d ("dwarf_loader: Support for btf:type_tag")
[5] 49b5300f1f8f ("Merge branch 'Support stashing local kptrs with bpf_kptr_xchg'")
[6] LLVM changes to generate btf:type_tag, revisions stack:

    https://reviews.llvm.org/D143966
    https://reviews.llvm.org/D143967
    https://reviews.llvm.org/D145891

Eduard Zingerman (1):
dwarf_loader: Support for btf:type_tag

 btf_encoder.c     |  13 +-
 btf_loader.c      |  15 +-
 dwarf_loader.c    | 763 +++++++++++++++++++++++++++++++++++++---------
 dwarves.c         |   1 +
 dwarves.h         |  68 +++--
 dwarves_fprintf.c |  13 +
 6 files changed, 693 insertions(+), 180 deletions(-)
Signature · 1 line
2.39.1
#2 lore raw

Re: [PATCH dwarves 1/1] dwarf_loader: Support for btf:type_tag

In reply to Eduard Zingerman
Quoted 93 lines · Eduard Zingerman

"btf:type_tag" is an DW_TAG_LLVM_annotation object that encodes btf_type_tag attributes in DWARF. Contrary to existing "btf_type_tag" it allows to associate such attributes with non-pointer types. When "btf:type_tag" is attached to a type it applies to this type.

For example the following C code:

    struct echo {
      int __attribute__((btf_type_tag("__c"))) c;
    }

Produces the following DWARF:

0x29: DW_TAG_structure_type
DW_AT_name ("echo")

0x40: DW_TAG_member

            DW_AT_name    ("c")
            DW_AT_type    (0x8c "int")

0x8c: DW_TAG_base_type

          DW_AT_name      ("int")
          DW_AT_encoding  (DW_ATE_signed)
          DW_AT_byte_size (0x04)

0x90: DW_TAG_LLVM_annotation

            DW_AT_name    ("btf:type_tag")
            DW_AT_const_value     ("__c")

Meaning that type 0x8c is an int with type tag __c. Corresponding BTF looks as follows:

[1] STRUCT 'echo'

        ...
        'c' type_id=8 bits_offset=128

[4] INT 'int' size=4 bits_offset=0 nr_bits=32 encoding=SIGNED
[8] TYPE_TAG '__c' type_id=4

Void pointers with type tag annotations are represented as objects of unspecified type with name "void", here is an example:

  struct st {
    void __attribute__((btf_type_tag("__d"))) *d;
  }

And corresponding DWARF:

0x29: DW_TAG_structure_type

            DW_AT_name      ("st")

0x49: DW_TAG_member

              DW_AT_name    ("d")
              DW_AT_type    (0xa6 "void *")

0xa6: DW_TAG_pointer_type

            DW_AT_type      (0xaf "void")

0xaf: DW_TAG_unspecified_type
DW_AT_name ("void")

0xb1: DW_TAG_LLVM_annotation

              DW_AT_name    ("btf:type_tag")
              DW_AT_const_value     ("__d")

This commit adds support for DW_TAG_LLVM_annotation "btf:type_tag" attached to the following entities:
- base types;
- arrays;
- pointers;
- structs
- unions;
- enums;
- typedefs.

This is achieved via the following modifications:
- Types struct btf_type_tag_type and struct llvm_annotation are consolidated as a single type struct llvm_annotation, in order to reside in a single annots list associated with struct/union/enum or typedef.
- struct unspecified_type is added to handle void * types annotated with btf:type_tag.
- struct tag is extended with annots list.
- DWARF load phase is modified to fill annots fields for the above mentioned types.
- Recode phase is split in two sub-phases:
- The existing tag__recode_dwarf_type() is executed first;
- Newly added update_btf_type_tag_refs() is executed to update ->type field of each tag if that type refers to an object with btf:type_tag annotation. The id of the type is replaced by id of the type tag.
- When btf:type_tag annotations are present in compilation unit, all btf_type_tag annotations are ignored during BTF dump.

Quoted 4 lines

See also:
[1] Mailing list discussion regarding btf:type_tag

    https://lore.kernel.org/bpf/[email protected]/

I know there's a v2 coming but one suggestion and one other issue below..

this is a great explanation, thanks for the details! one other thing that might help here is specifying that the solution adopted is option 2 described in that discussion (at least I think it is?).

Quoted 10 lines
---
 btf_encoder.c     |  13 +-
 btf_loader.c      |  15 +-
 dwarf_loader.c    | 763 +++++++++++++++++++++++++++++++++++++---------
 dwarves.c         |   1 +
 dwarves.h         |  68 +++--
 dwarves_fprintf.c |  13 +
 6 files changed, 693 insertions(+), 180 deletions(-)

<snip>

dwarf_loader.c static int die__process_unit(Dwarf_Die *die, struct cu *cu, struct conf_load *co · line 2360

Quoted 9 lines
dwarf_loader.c +8 -1
-static void ftype__recode_dwarf_types(struct tag *tag, struct cu *cu);
+/** Add @tuple to @ctx->mappings array, extend it if necessary. */
+static int push_btf_type_tag_mapping(struct btf_type_tag_mapping *tuple,
+				     struct recode_context *ctx)
+{
+	if (ctx->nr_allocated == ctx->nr_entries) {
+		uint32_t new_nr = ctx->nr_allocated * 2;
+		void *new_array = reallocarray(ctx->mappings, new_nr,
+					       sizeof(ctx->mappings[0]));

older libcs won't have reallocarray; might be good to either replace with realloc or define our own variant in dutil.h like was done with libbpf_reallocarray()?

author maintainer
#3 lore raw

Re: [PATCH dwarves 1/1] dwarf_loader: Support for btf:type_tag

On Tue, 2023-03-14 at 10:30 +0000, Alan Maguire wrote:
[...]

Quoted 10 lines
Quoted 4 lines

See also:
[1] Mailing list discussion regarding btf:type_tag

    https://lore.kernel.org/bpf/[email protected]/

I know there's a v2 coming but one suggestion and one other issue below..

this is a great explanation, thanks for the details! one other thing that might help here is specifying that the solution adopted is option 2 described in that discussion (at least I think it is?).

Hi Alan,

Yes, it's option #2, sorry should have noted it in the original message.

[...]

Quoted 4 lines
Quoted 4 lines
dwarf_loader.c +3 -1
-static void ftype__recode_dwarf_types(struct tag *tag, struct cu *cu);
+/** Add @tuple to @ctx->mappings array, extend it if necessary. */
+static int push_btf_type_tag_mapping(struct btf_type_tag_mapping *tuple,
+				     struct recode_context *ctx)
Quoted 8 lines

dwarf_loader.c static int die__process_unit(Dwarf_Die *die, struct cu *cu, struct conf_load *co · line 2360

Quoted 5 lines
dwarf_loader.c +5 -0
+{
+	if (ctx->nr_allocated == ctx->nr_entries) {
+		uint32_t new_nr = ctx->nr_allocated * 2;
+		void *new_array = reallocarray(ctx->mappings, new_nr,
+					       sizeof(ctx->mappings[0]));

older libcs won't have reallocarray; might be good to either replace with realloc or define our own variant in dutil.h like was done with libbpf_reallocarray()?

Will use realloc(), thank you for the heads-up.

Thanks,
Eduard