Skip to contentSkip to conversation

btf: support btf_type_tag attribute

PATCH In review Yonghong Song 3 participants 11 messages Nov 23, 2021 – Nov 26, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v2 cover btf: support btf_type_tag attribute

btf_type_tag is a new llvm type attribute which is used similar to kernel __user/__rcu attributes. The format of btf_type_tag looks like
__attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like
#define __tag1 __attribute__((btf_type_tag("tag1"))) #define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g;
the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically converts llvm dwarf btf_type_tag attributes to BTF types. Please see individual patches for details.

Changelog:
v1 -> v2:
- reorg an if condition to reduce nesting level.
- add more comments to explain how to chain type tag types.

Yonghong Song (4):
libbpf: sync with latest libbpf repo
dutil: move DW_TAG_LLVM_annotation definition to dutil.h
dwarf_loader: support btf_type_tag attribute
btf_encoder: support btf_type_tag attribute

 btf_encoder.c  |   7 +++
 dutil.h        |   4 ++
 dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
 dwarves.h      |  38 +++++++++++++-
 lib/bpf        |   2 +-
 pahole.c       |   8 +++
 6 files changed, 190 insertions(+), 9 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH v2 1/4 libbpf: sync with latest libbpf repo

Sync up to commit 94a49850c5ee ("Makefile: enforce gnu89 standard"). This is needed to support BTF_KIND_TYPE_TAG.

---
 lib/bpf | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
lib/bpf +1 -1
index f05791d..94a4985 160000
@@ -1 +1 @@
-Subproject commit f05791d8cfcbbf9092b4099b9d011bb72e241ef8
+Subproject commit 94a49850c5ee61ea02dfcbabf48013391e8cecdf
Signature · 1 line
2.30.2
#2 lore raw

PATCH v2 2/4 dutil: move DW_TAG_LLVM_annotation definition to dutil.h

In reply to Yonghong Song

Move DW_TAG_LLVM_annotation definition from dwarf_load.c to dutil.h as it will be used later for btf_encoder.c. There is no functionality change for this patch.

---
 dutil.h        | 4 ++++
 dwarf_loader.c | 4 ----
 2 files changed, 4 insertions(+), 4 deletions(-)
dutil.h +4 -0
index 25691ea..e45bba0 100644
@@ -31,6 +31,10 @@
 
 #define roundup(x,y) ((((x) + ((y) - 1)) / (y)) * (y))
 
+#ifndef DW_TAG_LLVM_annotation
+#define DW_TAG_LLVM_annotation 0x6000
+#endif
+
 static inline __attribute__((const)) bool is_power_of_2(unsigned long n)
 {
         return (n != 0 && ((n & (n - 1)) == 0));
dwarf_loader.c +0 -4
index cf005da..1b07a62 100644
@@ -52,10 +52,6 @@
 #define DW_OP_addrx 0xa1
 #endif
 
-#ifndef DW_TAG_LLVM_annotation
-#define DW_TAG_LLVM_annotation 0x6000
-#endif
-
 static pthread_mutex_t libdw__lock = PTHREAD_MUTEX_INITIALIZER;
 
 static uint32_t hashtags__bits = 12;
Signature · 1 line
2.30.2
#3 lore raw

PATCH v2 3/4 dwarf_loader: support btf_type_tag attribute

In reply to Yonghong Song

LLVM patches ([1] for clang, [2] and [3] for BPF backend) added support for btf_type_tag attributes. The following is
an example:
[$ ~] cat t.c
#define __tag1 __attribute__((btf_type_tag("tag1")))
#define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu"))); [$ ~] clang -O2 -g -c t.c
[$ ~] llvm-dwarfdump --debug-info t.o
t.o: file format elf64-x86-64
...
0x0000001e: DW_TAG_variable

                  DW_AT_name      ("g")
                  DW_AT_type      (0x00000033 "int **")
                  DW_AT_external  (true)
                  DW_AT_decl_file ("/home/yhs/t.c")
                  DW_AT_decl_line (3)
                  DW_AT_location  (DW_OP_addr 0x0)
  0x00000033:   DW_TAG_pointer_type
                  DW_AT_type      (0x0000004b "int *")
  0x00000038:     DW_TAG_LLVM_annotation
                    DW_AT_name    ("btf_type_tag")
                    DW_AT_const_value     ("tag1")
  0x00000041:     DW_TAG_LLVM_annotation
                    DW_AT_name    ("btf_type_tag")
                    DW_AT_const_value     ("tag2")
  0x0000004a:     NULL
  0x0000004b:   DW_TAG_pointer_type
                  DW_AT_type      (0x0000005a "int")
  0x00000050:     DW_TAG_LLVM_annotation
                    DW_AT_name    ("btf_type_tag")
                    DW_AT_const_value     ("tag1")
  0x00000059:     NULL
  0x0000005a:   DW_TAG_base_type
                  DW_AT_name      ("int")
                  DW_AT_encoding  (DW_ATE_signed)
                  DW_AT_byte_size (0x04)

0x00000061: NULL

From the above example, you can see that DW_TAG_pointer_type may contain one or more DW_TAG_LLVM_annotation btf_type_tag tags. If DW_TAG_LLVM_annotation tags are present inside DW_TAG_pointer_type, for BTF encoding, pahole will need to follow [3] to generate a type chain like
var -> ptr -> tag2 -> tag1 -> ptr -> tag1 -> int

This patch implemented dwarf_loader support. If a pointer type contains DW_TAG_LLVM_annotation tags, a new type btf_type_tag_ptr_type will be created which will store the pointer tag itself and all DW_TAG_LLVM_annotation tags. During recoding stage, the type chain will be formed properly based on the above example.

An option "--skip_encoding_btf_type_tag" is added to disable this new functionality.

[1] https://reviews.llvm.org/D111199
[2] https://reviews.llvm.org/D113222
[3] https://reviews.llvm.org/D113496

---
 dwarf_loader.c | 136 +++++++++++++++++++++++++++++++++++++++++++++++--
 dwarves.h      |  33 +++++++++++-
 pahole.c       |   8 +++
 3 files changed, 173 insertions(+), 4 deletions(-)
dwarf_loader.c +133 -3
index 1b07a62..e30b03c 100644
@@ -1206,6 +1206,89 @@ static struct tag *die__create_new_tag(Dwarf_Die *die, struct cu *cu)
 	return tag;
 }
 
+static struct btf_type_tag_ptr_type *die__create_new_btf_type_tag_ptr_type(Dwarf_Die *die, struct cu *cu)
+{
+	struct btf_type_tag_ptr_type *tag;
+
+	tag  = tag__alloc_with_spec(cu, sizeof(struct btf_type_tag_ptr_type));
+	if (tag == NULL)
+		return NULL;
+
+	tag__init(&tag->tag, cu, die);
+	tag->tag.has_btf_type_tag = true;
+	INIT_LIST_HEAD(&tag->tags);
+	return tag;
+}
+
+static struct btf_type_tag_type *die__create_new_btf_type_tag_type(Dwarf_Die *die, struct cu *cu,
+								   struct conf_load *conf)
+{
+	struct btf_type_tag_type *tag;
+
+	tag  = tag__alloc_with_spec(cu, sizeof(struct btf_type_tag_type));
+	if (tag == NULL)
+		return NULL;
+
+	tag__init(&tag->tag, cu, die);
+	tag->value = attr_string(die, DW_AT_const_value, conf);
+	return tag;
+}
+
+static struct tag *die__create_new_pointer_tag(Dwarf_Die *die, struct cu *cu,
+					       struct conf_load *conf)
+{
+	struct btf_type_tag_ptr_type *tag = NULL;
+	struct btf_type_tag_type *annot;
+	Dwarf_Die *cdie, child;
+	const char *name;
+	uint32_t id;
+
+	/* If no child tags or skipping btf_type_tag encoding, just create a new tag
+	 * and return
+	 */
+	if (!dwarf_haschildren(die) || dwarf_child(die, &child) != 0 ||
+	    conf->skip_encoding_btf_type_tag)
+		return tag__new(die, cu);
+
+	/* Otherwise, check DW_TAG_LLVM_annotation child tags */
+	cdie = &child;
+	do {
+		if (dwarf_tag(cdie) != DW_TAG_LLVM_annotation)
+			continue;
+
+		/* Only check btf_type_tag annotations */
+		name = attr_string(cdie, DW_AT_name, conf);
+		if (strcmp(name, "btf_type_tag") != 0)
+			continue;
+
+		if (tag == NULL) {
+			/* Create a btf_type_tag_ptr type. */
+			tag = die__create_new_btf_type_tag_ptr_type(die, cu);
+			if (!tag)
+				return NULL;
+		}
+
+		/* Create a btf_type_tag type for this annotation. */
+		annot = die__create_new_btf_type_tag_type(cdie, cu, conf);
+		if (annot == NULL)
+			return NULL;
+
+		if (cu__table_add_tag(cu, &annot->tag, &id) < 0)
+			return NULL;
+
+		struct dwarf_tag *dtag = annot->tag.priv;
+		dtag->small_id = id;
+		cu__hash(cu, &annot->tag);
+
+		/* For a list of DW_TAG_LLVM_annotation like tag1 -> tag2 -> tag3,
+		 * the tag->tags contains tag3 -> tag2 -> tag1.
+		 */
+		list_add(&annot->node, &tag->tags);
+	} while (dwarf_siblingof(cdie, cdie) == 0);
+
+	return tag ? &tag->tag : tag__new(die, cu);
+}
+
 static struct tag *die__create_new_ptr_to_member_type(Dwarf_Die *die,
 						      struct cu *cu)
 {
@@ -1903,12 +1986,13 @@ static struct tag *__die__process_tag(Dwarf_Die *die, struct cu *cu,
 	case DW_TAG_const_type:
 	case DW_TAG_imported_declaration:
 	case DW_TAG_imported_module:
-	case DW_TAG_pointer_type:
 	case DW_TAG_reference_type:
 	case DW_TAG_restrict_type:
 	case DW_TAG_unspecified_type:
 	case DW_TAG_volatile_type:
 		tag = die__create_new_tag(die, cu);		break;
+	case DW_TAG_pointer_type:
+		tag = die__create_new_pointer_tag(die, cu, conf);	break;
 	case DW_TAG_ptr_to_member_type:
 		tag = die__create_new_ptr_to_member_type(die, cu); break;
 	case DW_TAG_enumeration_type:
@@ -2192,6 +2276,45 @@ static void lexblock__recode_dwarf_types(struct lexblock *tag, struct cu *cu)
 	}
 }
 
+static void dwarf_cu__recode_btf_type_tag_ptr(struct btf_type_tag_ptr_type *tag,
+					      uint32_t pointee_type)
+{
+	struct btf_type_tag_type *annot;
+	struct dwarf_tag *annot_dtag;
+	struct tag *prev_tag;
+
+	/* Given source like
+	 *   int tag1 tag2 tag3 *p;
+	 * the tag->tags contains tag3 -> tag2 -> tag1, the final type chain looks like:
+	 *   pointer -> tag3 -> tag2 -> tag1 -> pointee
+	 *
+	 * Basically it means
+	 *   - '*' applies to "int tag1 tag2 tag3"
+	 *   - tag3 applies to "int tag1 tag2"
+	 *   - tag2 applies to "int tag1"
+	 *   - tag1 applies to "int"
+	 *
+	 * This also makes final source code (format c) easier as we can do
+	 *   emit for "tag3 -> tag2 -> tag1 -> int"
+	 *   emit '*'
+	 *
+	 * For 'tag3 -> tag2 -> tag1 -> int":
+	 *   emit for "tag2 -> tag1 -> int"
+	 *   emit tag3
+	 *
+	 * Eventually we can get the source code like
+	 *   int tag1 tag2 tag3 *p;
+	 * and this matches the user/kernel code.
+	 */
+	prev_tag = &tag->tag;
+	list_for_each_entry(annot, &tag->tags, node) {
+		annot_dtag = annot->tag.priv;
+		prev_tag->type = annot_dtag->small_id;
+		prev_tag = &annot->tag;
+	}
+	prev_tag->type = pointee_type;
+}
+
 static int tag__recode_dwarf_type(struct tag *tag, struct cu *cu)
 {
 	struct dwarf_tag *dtag = tag->priv;
@@ -2301,7 +2424,10 @@ static int tag__recode_dwarf_type(struct tag *tag, struct cu *cu)
 	}
 
 	if (dtag->type.off == 0) {
-		tag->type = 0; /* void */
+		if (tag->tag != DW_TAG_pointer_type || !tag->has_btf_type_tag)
+			tag->type = 0; /* void */
+		else
+			dwarf_cu__recode_btf_type_tag_ptr(tag__btf_type_tag_ptr(tag), 0);
 		return 0;
 	}
 
@@ -2313,7 +2439,11 @@ check_type:
 		return 0;
 	}
 out:
-	tag->type = dtype->small_id;
+	if (tag->tag != DW_TAG_pointer_type || !tag->has_btf_type_tag)
+		tag->type = dtype->small_id;
+	else
+		dwarf_cu__recode_btf_type_tag_ptr(tag__btf_type_tag_ptr(tag), dtype->small_id);
+
 	return 0;
 }
 
dwarves.h +32 -1
index 0d3e204..4425d3c 100644
@@ -63,6 +63,7 @@ struct conf_load {
 	bool			ptr_table_stats;
 	bool			skip_encoding_btf_decl_tag;
 	bool			skip_missing;
+	bool			skip_encoding_btf_type_tag;
 	uint8_t			hashtable_bits;
 	uint8_t			max_hashtable_bits;
 	uint16_t		kabi_prefix_len;
@@ -413,6 +414,7 @@ struct tag {
 	uint16_t	 tag;
 	bool		 visited;
 	bool		 top_level;
+	bool		 has_btf_type_tag;
 	uint16_t	 recursivity_level;
 	void		 *priv;
 };
@@ -533,7 +535,8 @@ static inline int tag__is_tag_type(const struct tag *tag)
 	       tag->tag == DW_TAG_restrict_type ||
 	       tag->tag == DW_TAG_subroutine_type ||
 	       tag->tag == DW_TAG_unspecified_type ||
-	       tag->tag == DW_TAG_volatile_type;
+	       tag->tag == DW_TAG_volatile_type ||
+	       tag->tag == DW_TAG_LLVM_annotation;
 }
 
 static inline const char *tag__decl_file(const struct tag *tag,
@@ -606,6 +609,34 @@ struct llvm_annotation {
 	struct list_head	node;
 };
 
+/** struct btf_type_tag_type - representing a btf_type_tag annotation
+ *
+ * @tag   - DW_TAG_LLVM_annotation tag
+ * @value - btf_type_tag value string
+ * @node  - list_head node
+ */
+struct btf_type_tag_type {
+	struct tag		tag;
+	const char		*value;
+	struct list_head	node;
+};
+
+/** The struct btf_type_tag_ptr_type - type containing both pointer type and
+ *  its btf_type_tag annotations
+ *
+ * @tag  - pointer type tag
+ * @tags - btf_type_tag annotations for the pointer type
+ */
+struct btf_type_tag_ptr_type {
+	struct tag		tag;
+	struct list_head 	tags;
+};
+
+static inline struct btf_type_tag_ptr_type *tag__btf_type_tag_ptr(struct tag *tag)
+{
+	return (struct btf_type_tag_ptr_type *)tag;
+}
+
 /** struct namespace - base class for enums, structs, unions, typedefs, etc
  *
  * @tags - class_member, enumerators, etc
pahole.c +8 -0
index 5fc1cca..f3a51cb 100644
@@ -1126,6 +1126,7 @@ ARGP_PROGRAM_VERSION_HOOK_DEF = dwarves_print_version;
 #define ARGP_devel_stats	   330
 #define ARGP_skip_encoding_btf_decl_tag 331
 #define ARGP_skip_missing          332
+#define ARGP_skip_encoding_btf_type_tag 333
 
 static const struct argp_option pahole__options[] = {
 	{
@@ -1506,6 +1507,11 @@ static const struct argp_option pahole__options[] = {
 		.key  = ARGP_skip_missing,
 		.doc = "skip missing types passed to -C rather than stop",
 	},
+	{
+		.name = "skip_encoding_btf_type_tag",
+		.key  = ARGP_skip_encoding_btf_type_tag,
+		.doc  = "Do not encode TAGs in BTF."
+	},
 	{
 		.name = NULL,
 	}
@@ -1658,6 +1664,8 @@ static error_t pahole__options_parser(int key, char *arg,
 		conf_load.skip_encoding_btf_decl_tag = true;	break;
 	case ARGP_skip_missing:
 		conf_load.skip_missing = true;          break;
+	case ARGP_skip_encoding_btf_type_tag:
+		conf_load.skip_encoding_btf_type_tag = true;	break;
 	default:
 		return ARGP_ERR_UNKNOWN;
 	}
Signature · 1 line
2.30.2
#4 lore raw

PATCH v2 4/4 btf_encoder: support btf_type_tag attribute

In reply to Yonghong Song

The following is an example.
[$ ~] cat t.c
#define __tag1 __attribute__((btf_type_tag("tag1")))
#define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu"))); [$ ~] clang -O2 -g -c t.c
[$ ~] pahole -JV t.o
Found per-CPU symbol 'g' at address 0x0
Found 1 per-CPU variables!
File t.o:
[1] TYPE_TAG tag1 type_id=5
[2] TYPE_TAG tag2 type_id=1
[3] PTR (anon) type_id=2
[4] TYPE_TAG tag1 type_id=6
[5] PTR (anon) type_id=4
[6] INT int size=4 nr_bits=32 encoding=SIGNED
search cu 't.c' for percpu global variables.
Variable 'g' from CU 't.c' at address 0x0 encoded
[7] VAR g type=3 linkage=1
[8] DATASEC .data..percpu size=8 vlen=1

          type=7 offset=0 size=8

[$ ~]

You can see for the source
int __tag1 * __tag1 __tag2 *g __attribute__((section(".data..percpu")));
the following type chain is generated:
var -> ptr -> tag2 -> tag1 -> ptr -> tag1 -> int

The following shows pahole option "--skip_encoding_btf_type_tag" can prevent BTF_KIND_TYPE_TAG generation.
[$ ~] pahole -JV t.o --skip_encoding_btf_type_tag Found per-CPU symbol 'g' at address 0x0
Found 1 per-CPU variables!
File t.o:
[1] PTR (anon) type_id=2
[2] PTR (anon) type_id=3
[3] INT int size=4 nr_bits=32 encoding=SIGNED
search cu 't.c' for percpu global variables. Variable 'g' from CU 't.c' at address 0x0 encoded
[4] VAR g type=1 linkage=1
[5] DATASEC .data..percpu size=8 vlen=1

          type=4 offset=0 size=8

[$ ~]

---
 btf_encoder.c | 7 +++++++
 dwarves.h     | 5 +++++
 2 files changed, 12 insertions(+)
btf_encoder.c +7 -0
index 117656e..9d015f3 100644
@@ -143,6 +143,7 @@ static const char * const btf_kind_str[NR_BTF_KINDS] = {
 	[BTF_KIND_DATASEC]      = "DATASEC",
 	[BTF_KIND_FLOAT]        = "FLOAT",
 	[BTF_KIND_DECL_TAG]     = "DECL_TAG",
+	[BTF_KIND_TYPE_TAG]     = "TYPE_TAG",
 };
 
 static const char *btf__printable_name(const struct btf *btf, uint32_t offset)
@@ -393,6 +394,9 @@ static int32_t btf_encoder__add_ref_type(struct btf_encoder *encoder, uint16_t k
 	case BTF_KIND_TYPEDEF:
 		id = btf__add_typedef(btf, name, type);
 		break;
+	case BTF_KIND_TYPE_TAG:
+		id = btf__add_type_tag(btf, name, type);
+		break;
 	case BTF_KIND_FWD:
 		id = btf__add_fwd(btf, name, kind_flag);
 		break;
@@ -862,6 +866,9 @@ static int btf_encoder__encode_tag(struct btf_encoder *encoder, struct tag *tag,
 	case DW_TAG_typedef:
 		name = namespace__name(tag__namespace(tag));
 		return btf_encoder__add_ref_type(encoder, BTF_KIND_TYPEDEF, ref_type_id, name, false);
+	case DW_TAG_LLVM_annotation:
+		name = tag__btf_type_tag(tag)->value;
+		return btf_encoder__add_ref_type(encoder, BTF_KIND_TYPE_TAG, ref_type_id, name, false);
 	case DW_TAG_structure_type:
 	case DW_TAG_union_type:
 	case DW_TAG_class_type:
dwarves.h +5 -0
index 4425d3c..52d162d 100644
@@ -637,6 +637,11 @@ static inline struct btf_type_tag_ptr_type *tag__btf_type_tag_ptr(struct tag *ta
 	return (struct btf_type_tag_ptr_type *)tag;
 }
 
+static inline struct btf_type_tag_type *tag__btf_type_tag(struct tag *tag)
+{
+	return (struct btf_type_tag_type *)tag;
+}
+
 /** struct namespace - base class for enums, structs, unions, typedefs, etc
  *
  * @tags - class_member, enumerators, etc
Signature · 1 line
2.30.2
Acked-by on the v2 series
#5 lore raw
In reply to Yonghong Song
Quoted 22 lines · Yonghong Song

btf_type_tag is a new llvm type attribute which is used similar to kernel __user/__rcu attributes. The format of btf_type_tag looks like
__attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like
#define __tag1 __attribute__((btf_type_tag("tag1"))) #define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g;
the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically converts llvm dwarf btf_type_tag attributes to BTF types. Please see individual patches for details.

Changelog:
v1 -> v2:
- reorg an if condition to reduce nesting level.
- add more comments to explain how to chain type tag types.

Quoted 6 lines

Yonghong Song (4):
libbpf: sync with latest libbpf repo
dutil: move DW_TAG_LLVM_annotation definition to dutil.h
dwarf_loader: support btf_type_tag attribute
btf_encoder: support btf_type_tag attribute

I thought that v1 was already applied, but either way LGTM. I'm not super familiar with the DWARF loader parts, so I mostly just read it very superficially :)

Quoted 11 lines
 btf_encoder.c  |   7 +++
 dutil.h        |   4 ++
 dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
 dwarves.h      |  38 +++++++++++++-
 lib/bpf        |   2 +-
 pahole.c       |   8 +++
 6 files changed, 190 insertions(+), 9 deletions(-)

--
2.30.2

#6 lore raw

Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:

Quoted 23 lines · Yonghong Song
Quoted 22 lines

btf_type_tag is a new llvm type attribute which is used similar to kernel __user/__rcu attributes. The format of btf_type_tag looks like
__attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like
#define __tag1 __attribute__((btf_type_tag("tag1"))) #define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g;
the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically converts llvm dwarf btf_type_tag attributes to BTF types. Please see individual patches for details.

Changelog:
v1 -> v2:
- reorg an if condition to reduce nesting level.
- add more comments to explain how to chain type tag types.

Quoted 8 lines
Quoted 6 lines

Yonghong Song (4):
libbpf: sync with latest libbpf repo
dutil: move DW_TAG_LLVM_annotation definition to dutil.h
dwarf_loader: support btf_type_tag attribute
btf_encoder: support btf_type_tag attribute

I thought that v1 was already applied, but either way LGTM. I'm not

To the next branch, and the libbpf pahole CI is failing, since a few days, can you please take a look?

Quoted 2 lines

super familiar with the DWARF loader parts, so I mostly just read it very superficially :)

I replaced the patches that changed, re-added the S-o-B for Yonghong and tested it with llvm-project HEAD.

Quoted 1 line

Adding it to the csets.

Thanks!

- Arnaldo

Quoted 12 lines
Quoted 12 lines
 btf_encoder.c  |   7 +++
 dutil.h        |   4 ++
 dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
 dwarves.h      |  38 +++++++++++++-
 lib/bpf        |   2 +-
 pahole.c       |   8 +++
 6 files changed, 190 insertions(+), 9 deletions(-)

--
2.30.2

Signature · 1 line
- Arnaldo
#7 lore raw
Quoted 33 lines · Arnaldo Carvalho de Melo

Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:

Quoted 23 lines · Yonghong Song
Quoted 22 lines

btf_type_tag is a new llvm type attribute which is used similar to kernel __user/__rcu attributes. The format of btf_type_tag looks like
__attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like

   #define __tag1 __attribute__((btf_type_tag("tag1")))
   #define __tag2 __attribute__((btf_type_tag("tag2")))
   int __tag1 * __tag1 __tag2 *g;

the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically converts llvm dwarf btf_type_tag attributes to BTF types. Please see individual patches for details.

Changelog:
v1 -> v2:
- reorg an if condition to reduce nesting level.
- add more comments to explain how to chain type tag types.

Quoted 8 lines
Quoted 6 lines

Yonghong Song (4):
libbpf: sync with latest libbpf repo
dutil: move DW_TAG_LLVM_annotation definition to dutil.h
dwarf_loader: support btf_type_tag attribute
btf_encoder: support btf_type_tag attribute

I thought that v1 was already applied, but either way LGTM. I'm not

Quoted 8 lines

To the next branch, and the libbpf pahole CI is failing, since a few days, can you please take a look?

Quoted 2 lines

super familiar with the DWARF loader parts, so I mostly just read it very superficially :)

I replaced the patches that changed, re-added the S-o-B for Yonghong and tested it with llvm-project HEAD.

Thanks Arnaldo!

Quoted 22 lines
Quoted 2 lines

Adding it to the csets.

Thanks!

- Arnaldo

Quoted 12 lines
Quoted 12 lines
  btf_encoder.c  |   7 +++
  dutil.h        |   4 ++
  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
  dwarves.h      |  38 +++++++++++++-
  lib/bpf        |   2 +-
  pahole.c       |   8 +++
  6 files changed, 190 insertions(+), 9 deletions(-)

--
2.30.2

#8 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 32 lines · Arnaldo Carvalho de Melo

Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:

Quoted 30 lines · Yonghong Song
Quoted 28 lines

btf_type_tag is a new llvm type attribute which is used similar to kernel __user/__rcu attributes. The format of btf_type_tag looks like
__attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like
#define __tag1 __attribute__((btf_type_tag("tag1"))) #define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g;
the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically converts llvm dwarf btf_type_tag attributes to BTF types. Please see individual patches for details.

Changelog:
v1 -> v2:
- reorg an if condition to reduce nesting level.
- add more comments to explain how to chain type tag types.

Yonghong Song (4):
libbpf: sync with latest libbpf repo
dutil: move DW_TAG_LLVM_annotation definition to dutil.h
dwarf_loader: support btf_type_tag attribute
btf_encoder: support btf_type_tag attribute

Quoted 4 lines
Quoted 1 line

I thought that v1 was already applied, but either way LGTM. I'm not

To the next branch, and the libbpf pahole CI is failing, since a few days, can you please take a look?

We've had Clang regression which Yonghong fixed very quickly, but then we were blocked on Clang nightly builds being broken for days. Seems like we got a new Clang today, so hopefully libbpf CI will be back to green again.

Quoted 31 lines
Quoted 3 lines

super familiar with the DWARF loader parts, so I mostly just read it very superficially :)

I replaced the patches that changed, re-added the S-o-B for Yonghong and tested it with llvm-project HEAD.

Quoted 1 line

Adding it to the csets.

Thanks!

- Arnaldo

Quoted 9 lines
Quoted 9 lines
 btf_encoder.c  |   7 +++
 dutil.h        |   4 ++
 dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
 dwarves.h      |  38 +++++++++++++-
 lib/bpf        |   2 +-
 pahole.c       |   8 +++
 6 files changed, 190 insertions(+), 9 deletions(-)
Quoted 3 lines
Quoted 3 lines

--
2.30.2

--

- Arnaldo

#9 lore raw

Em Tue, Nov 23, 2021 at 08:49:17PM -0800, Andrii Nakryiko escreveu:

Quoted 36 lines
Quoted 34 lines · Arnaldo Carvalho de Melo

Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:

Quoted 31 lines · Yonghong Song
Quoted 22 lines

btf_type_tag is a new llvm type attribute which is used similar to kernel __user/__rcu attributes. The format of btf_type_tag looks like
__attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like
#define __tag1 __attribute__((btf_type_tag("tag1"))) #define __tag2 __attribute__((btf_type_tag("tag2")))
int __tag1 * __tag1 __tag2 *g;
the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically converts llvm dwarf btf_type_tag attributes to BTF types. Please see individual patches for details.

Changelog:
v1 -> v2:
- reorg an if condition to reduce nesting level.
- add more comments to explain how to chain type tag types.

Quoted 6 lines

Yonghong Song (4):
libbpf: sync with latest libbpf repo
dutil: move DW_TAG_LLVM_annotation definition to dutil.h
dwarf_loader: support btf_type_tag attribute
btf_encoder: support btf_type_tag attribute

I thought that v1 was already applied, but either way LGTM. I'm not

Quoted 7 lines
Quoted 2 lines

To the next branch, and the libbpf pahole CI is failing, since a few days, can you please take a look?

We've had Clang regression which Yonghong fixed very quickly, but then we were blocked on Clang nightly builds being broken for days. Seems like we got a new Clang today, so hopefully libbpf CI will be back to green again.

It is back to green, so I moved next to master, i.e. this series is now on master.

- Arnaldo

Quoted 31 lines
Quoted 31 lines
Quoted 3 lines

super familiar with the DWARF loader parts, so I mostly just read it very superficially :)

I replaced the patches that changed, re-added the S-o-B for Yonghong and tested it with llvm-project HEAD.

Quoted 1 line

Adding it to the csets.

Thanks!

- Arnaldo

Quoted 9 lines
Quoted 9 lines
 btf_encoder.c  |   7 +++
 dutil.h        |   4 ++
 dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
 dwarves.h      |  38 +++++++++++++-
 lib/bpf        |   2 +-
 pahole.c       |   8 +++
 6 files changed, 190 insertions(+), 9 deletions(-)
Quoted 3 lines
Quoted 3 lines

--
2.30.2

--

- Arnaldo

Signature · 1 line
- Arnaldo
#10 lore raw
Quoted 40 lines · Arnaldo Carvalho de Melo

Em Tue, Nov 23, 2021 at 08:49:17PM -0800, Andrii Nakryiko escreveu:

Quoted 39 lines
Quoted 36 lines · Arnaldo Carvalho de Melo

Em Tue, Nov 23, 2021 at 10:32:18AM -0800, Andrii Nakryiko escreveu:

Quoted 23 lines · Yonghong Song
Quoted 22 lines

btf_type_tag is a new llvm type attribute which is used similar to kernel __user/__rcu attributes. The format of btf_type_tag looks like
__attribute__((btf_type_tag("tag1")))
For the case where the attribute applied to a pointee like

   #define __tag1 __attribute__((btf_type_tag("tag1")))
   #define __tag2 __attribute__((btf_type_tag("tag2")))
   int __tag1 * __tag1 __tag2 *g;

the information will be encoded in dwarf.

In BTF, the attribute is encoded as a new kind BTF_KIND_TYPE_TAG and latest bpf-next supports it.

The patch added support in pahole, specifically converts llvm dwarf btf_type_tag attributes to BTF types. Please see individual patches for details.

Changelog:
v1 -> v2:
- reorg an if condition to reduce nesting level.
- add more comments to explain how to chain type tag types.

Quoted 8 lines
Quoted 6 lines

Yonghong Song (4):
libbpf: sync with latest libbpf repo
dutil: move DW_TAG_LLVM_annotation definition to dutil.h
dwarf_loader: support btf_type_tag attribute
btf_encoder: support btf_type_tag attribute

I thought that v1 was already applied, but either way LGTM. I'm not

To the next branch, and the libbpf pahole CI is failing, since a few days, can you please take a look?

Quoted 7 lines
Quoted 4 lines

We've had Clang regression which Yonghong fixed very quickly, but then we were blocked on Clang nightly builds being broken for days. Seems like we got a new Clang today, so hopefully libbpf CI will be back to green again.

It is back to green, so I moved next to master, i.e. this series is now on master.

Arnaldo,

Is it possible that we could cut a pahole release in the near future?
My kernel using btf_type_tag patch ([1]) needs a pahole version check. In old pahole versions, a warning about pointer type subtags not supported will be emitted and btf_type_tag attributes (as pointer type subtags) will be ignored.

[1] https://lore.kernel.org/bpf/[email protected]/

Thanks!

Quoted 35 lines

- Arnaldo

Quoted 31 lines
Quoted 31 lines
Quoted 3 lines

super familiar with the DWARF loader parts, so I mostly just read it very superficially :)

I replaced the patches that changed, re-added the S-o-B for Yonghong and tested it with llvm-project HEAD.

Quoted 1 line

Adding it to the csets.

Thanks!

- Arnaldo

Quoted 9 lines
Quoted 9 lines
  btf_encoder.c  |   7 +++
  dutil.h        |   4 ++
  dwarf_loader.c | 140 ++++++++++++++++++++++++++++++++++++++++++++++---
  dwarves.h      |  38 +++++++++++++-
  lib/bpf        |   2 +-
  pahole.c       |   8 +++
  6 files changed, 190 insertions(+), 9 deletions(-)
Quoted 3 lines
Quoted 3 lines

--
2.30.2

--

- Arnaldo