Skip to contentSkip to conversation

btf_encoder: Teach pahole to store percpu variables in vmlinux BTF.

PATCH Superseded Hao Luo 4 participants 9 messages Jun 18, 2020 – Jul 3, 2020 7 messages not in the archive

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Hao Luo

author
#0 lore raw

PATCH v4 1/1 btf_encoder: Teach pahole to store percpu variables in vmlinux BTF.

On SMP systems, the global percpu variables are placed in a special '.data..percpu' section, which is stored in a segment whose initial address is set to 0, the addresses of per-CPU variables are relative positive addresses [1].

This patch extracts these variables from vmlinux and places them with their type information in BTF. More specifically, when BTF is encoded, we find the index of the '.data..percpu' section and then traverse the symbol table to find those global objects which are in this section. For each of these objects, we push a BTF_KIND_VAR into the types buffer, and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the CUs have finished processing, we push a BTF_KIND_DATASEC into the btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.7-rc7 linux kernel, I was able to extract 291 such variables. The build time overhead is small and the space overhead is also small.

Testing:

- vmlinux size has increased by ~12kb.
Before:

   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
  After:
   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00

- Common global percpu VARs and DATASEC are found in BTF section.

  $ bpftool btf dump file vmlinux | grep runqueues
  [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17592] STRUCT 'cpu_stopper' size=72 vlen=5
  [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=0 vlen=294

- Tested bpf selftests.

References:
[1] https://lwn.net/Articles/531148/

---
Changelog since v3:
- Correct the size of the DATASEC, which was previously 0 and now is the last var_secinfo's offset + size.
- Add several sanity checks on VARs and DATASEC to ensure that the generated BTF will not be rejected by verifier.
- Tested through a set of selftests and bpf_tracing programs.

Changelog since v2:
- Move finding percpu_shndx and extracting symtab into btfe creation, so we don't have to allocate a new symtab for each CU.
- More debug msg by logging the vars encoded in 'verbose' mode. We probably don't want to log the symbols that are _not_ encoded, since that would be too verbose.
- Calculate var offsets using 'addr - shdr.sh_addr', so it could be generalized to other sections in future.
- Filter out the symbols that are not STT_OBJECT.
- Sort var_secinfos in the DATASEC by their offsets.
- Free 'persec_secinfo' buffer and 'symtab' in btfe deletion.
- Replace the string ".data..percpu" with a constant PERCPU_SECTION.

Changelog since v1:
- Add a ".data..percpu" DATASEC that encodes the found VARs.
- Use percpu section's shndx to find the symbols that are percpu variables.
- Use the correct type to set VAR's linkage.

 btf_encoder.c | 185 ++++++++++++++++++++++++++++++++++++++++++++++++++
 dwarves.c     |   6 ++
 dwarves.h     |   2 +
 libbtf.c      | 127 ++++++++++++++++++++++++++++++++++
 libbtf.h      |  12 ++++
 pahole.c      |   1 +
 6 files changed, 333 insertions(+)
btf_encoder.c +185 -0
index df16ba0..c7401c3 100644
@@ -16,8 +16,44 @@
 #include "elf_symtab.h"
 #include "btf_encoder.h"
 
+#include <ctype.h> /* for isalpha() and isalnum() */
 #include <inttypes.h>
 
+/*
+ * This corresponds to the same macro defined in
+ * include/linux/kallsyms.h
+ */
+#define KSYM_NAME_LEN 128
+
+static bool btf_name_char_ok(char c, bool first, bool dot_ok)
+{
+	if ((first ? !isalpha(c) : !isalnum(c)) &&
+	    c != '_' &&
+	    ((c == '.' && !dot_ok) || c != '.'))
+		return false;
+	return true;
+}
+
+/* Check wether the given name is valid in vmlinux btf. */
+static bool btf_name_valid(const char *p, bool dot_ok)
+{
+	const char *limit;
+
+	if (!btf_name_char_ok(*p, true, dot_ok))
+		return false;
+
+	/* set a limit on identifier length */
+	limit = p + KSYM_NAME_LEN;
+	p++;
+	while (*p && p < limit) {
+		if (!btf_name_char_ok(*p, false, dot_ok))
+			return false;
+		p++;
+	}
+
+	return !*p;
+}
+
 static int tag__check_id_drift(const struct tag *tag,
 			       uint32_t core_id, uint32_t btf_type_id,
 			       uint32_t type_id_off)
@@ -168,6 +204,27 @@ int btf_encoder__encode()
 	return err;
 }
 
+
+#define HASHADDR__BITS 8
+#define HASHADDR__SIZE (1UL << HASHADDR__BITS)
+#define hashaddr__fn(key) hash_64(key, HASHADDR__BITS)
+
+static struct variable *hashaddr__find_variable(const struct hlist_head hashtable[],
+						const uint64_t addr)
+{
+	struct variable *variable;
+	struct hlist_node *pos;
+	uint16_t bucket = hashaddr__fn(addr);
+	const struct hlist_head *head = &hashtable[bucket];
+
+	hlist_for_each_entry(variable, pos, head, tool_hnode) {
+		if (variable->ip.addr == addr)
+			return variable;
+	}
+
+	return NULL;
+}
+
 int cu__encode_btf(struct cu *cu, int verbose)
 {
 	bool add_index_type = false;
@@ -176,6 +233,10 @@ int cu__encode_btf(struct cu *cu, int verbose)
 	struct function *fn;
 	struct tag *pos;
 	int err = 0;
+	struct hlist_head hash_addr[HASHADDR__SIZE];
+	struct variable *var;
+	bool has_global_var = false;
+	GElf_Sym sym;
 
 	if (btfe && strcmp(btfe->filename, cu->filename)) {
 		err = btf_encoder__encode();
@@ -241,6 +302,130 @@ int cu__encode_btf(struct cu *cu, int verbose)
 		}
 	}
 
+
+	if (btfe->percpu_shndx == 0 || !btfe->symtab)
+		goto out;
+
+	if (verbose)
+		printf("search cu '%s' for percpu global variables.\n", cu->name);
+
+	/* cache variables' addresses, preparing for searching in symtab. */
+	for (core_id = 0; core_id < HASHADDR__SIZE; ++core_id)
+		INIT_HLIST_HEAD(&hash_addr[core_id]);
+
+	cu__for_each_variable(cu, core_id, pos) {
+		struct hlist_head *head;
+
+		var = tag__variable(pos);
+		if (var->declaration)
+			continue;
+		/* percpu variables are allocated in global space */
+		if (variable__scope(var) != VSCOPE_GLOBAL)
+			continue;
+		has_global_var = true;
+		head = &hash_addr[hashaddr__fn(var->ip.addr)];
+		hlist_add_head(&var->tool_hnode, head);
+	}
+	if (!has_global_var) {
+		if (verbose)
+			printf("cu has no global variable defined, skip.\n");
+		goto out;
+	}
+
+	/* search within symtab for percpu variables */
+	elf_symtab__for_each_symbol(btfe->symtab, core_id, sym) {
+		uint32_t linkage, type, size, offset;
+		int32_t btf_var_id, btf_var_secinfo_id;
+		uint64_t addr;
+		const char *name;
+
+		/* compare a symbol's shndx to determine if it's a percpu variable */
+		if (elf_sym__section(&sym) != btfe->percpu_shndx)
+			continue;
+		if (elf_sym__type(&sym) != STT_OBJECT)
+			continue;
+
+		addr = elf_sym__value(&sym);
+		/*
+		 * Store only those symbols that have allocated space in the percpu section.
+		 * This excludes the following three types of symbols:
+		 *
+		 *  1. __ADDRESSABLE(sym), which are forcely emitted as symbols.
+		 *  2. __UNIQUE_ID(prefix), which are introduced to generate unique ids.
+		 *  3. __exitcall(fn), functions which are labeled as exit calls.
+		 *
+		 * In addition, the variables defined using DEFINE_PERCPU_FIRST are
+		 * also not included, which currently includes:
+		 *
+		 *  1. fixed_percpu_data
+		 */
+		if (!addr)
+			continue;
+		var = hashaddr__find_variable(hash_addr, addr);
+		if (var == NULL)
+			continue;
+
+		/*
+		 * Valididate the variable's name, the same check is also
+		 * performed in bpf verifier.
+		 */
+		name = variable__name(var, cu);
+		if (!var->name || !btf_name_valid(name, true)) {
+			if (verbose)
+				printf("invalid variable name '%s'.\n", name);
+			continue;
+		}
+
+		/*
+		 * Validate the variable's type, the same check is also
+		 * performed in bpf verifier.
+		 */
+		type = var->ip.tag.type + type_id_off;
+		if (type == 0) {
+			if (verbose)
+				printf("invalid type for variable '%s'.\n", name);
+			continue;
+		}
+
+		/*
+		 * Validate the size of the variable's type, the same check is
+		 * also performed in bpf verifier.
+		 */
+		size = variable__type_size(var, cu);
+		if (size == 0) {
+			if (verbose)
+				printf("invalid type size for variable '%s'.\n", name);
+			continue;
+		}
+
+		if (verbose)
+			printf("symbol '%s' of address 0x%lx encoded\n",
+			       elf_sym__name(&sym, btfe->symtab), addr);
+
+		/* add a BTF_KIND_VAR in btfe->types */
+		linkage = var->external ? BTF_VAR_GLOBAL_ALLOCATED : BTF_VAR_STATIC;
+		btf_var_id = btf_elf__add_var_type(btfe, type, var->name, linkage);
+		if (btf_var_id < 0) {
+			err = -1;
+			printf("error: failed to encode variable '%s'\n", name);
+			break;
+		}
+
+		/*
+		 * add a BTF_VAR_SECINFO in btfe->percpu_secinfo, which will be added into
+		 * btfe->types later when we add BTF_VAR_DATASEC.
+		 */
+		type = btf_var_id;
+		offset = addr - btfe->percpu_base_addr;
+		btf_var_secinfo_id = btf_elf__add_var_secinfo(&btfe->percpu_secinfo,
+							      type, offset, size);
+		if (btf_var_secinfo_id < 0) {
+			err = -1;
+			printf("error: failed to encode var secinfo '%s'\n", name);
+			break;
+		}
+	}
+
 out:
 	if (err)
 		btf_elf__delete(btfe);
dwarves.c +6 -0
index eb7885f..141f688 100644
@@ -978,6 +978,12 @@ const char *variable__type_name(const struct variable *var,
 	return tag != NULL ? tag__name(tag, cu, bf, len, NULL) : NULL;
 }
 
+uint32_t variable__type_size(const struct variable *var, const struct cu *cu)
+{
+	const struct tag *tag = cu__type(cu, var->ip.tag.type);
+	return tag != NULL ? tag__size(tag, cu) : 0;
+}
+
 void class_member__delete(struct class_member *member, struct cu *cu)
 {
 	obstack_free(&cu->obstack, member);
dwarves.h +2 -0
index a772e57..e4b0255 100644
@@ -672,6 +672,8 @@ const char *variable__name(const struct variable *var, const struct cu *cu);
 const char *variable__type_name(const struct variable *var,
 				const struct cu *cu, char *bf, size_t len);
 
+uint32_t variable__type_size(const struct variable *var, const struct cu *cu);
+
 struct lexblock {
 	struct ip_tag	 ip;
 	struct list_head tags;
libbtf.c +127 -0

172 lines not on this page · raw message

libbtf.h +12 -0
index f3c8500..be06480 100644
@@ -20,8 +20,10 @@ struct btf_elf {
 	void		  *priv;
 	Elf		  *elf;
 	GElf_Ehdr	  ehdr;
+	struct elf_symtab *symtab;
 	struct gobuffer	  types;
 	struct gobuffer   *strings;
+	struct gobuffer   percpu_secinfo;
 	char		  *filename;
 	size_t		  size;
 	int		  swapped;
@@ -30,11 +32,15 @@ struct btf_elf {
 	bool		  is_big_endian;
 	bool		  raw_btf; // "/sys/kernel/btf/vmlinux"
 	uint32_t	  type_index;
+	uint32_t	  percpu_shndx;
+	uint64_t	  percpu_base_addr;
 };
 
 extern uint8_t btf_elf__verbose;
 #define btf_elf__verbose_log(fmt, ...) { if (btf_elf__verbose) printf(fmt, __VA_ARGS__); }
 
+#define PERCPU_SECTION ".data..percpu"
+
 struct base_type;
 struct ftype;
 
@@ -55,6 +61,12 @@ int32_t btf_elf__add_enum(struct btf_elf *btf, uint32_t name, uint32_t size,
 int btf_elf__add_enum_val(struct btf_elf *btf, uint32_t name, int32_t value);
 int32_t btf_elf__add_func_proto(struct btf_elf *btf, struct ftype *ftype,
 				uint32_t type_id_off);
+int32_t btf_elf__add_var_type(struct btf_elf *btfe, uint32_t type, uint32_t name_off,
+			      uint32_t linkage);
+int32_t btf_elf__add_var_secinfo(struct gobuffer *buf, uint32_t type,
+				 uint32_t offset, uint32_t size);
+int32_t btf_elf__add_datasec_type(struct btf_elf *btfe, const char *section_name,
+				  struct gobuffer *var_secinfo_buf);
 void btf_elf__set_strings(struct btf_elf *btf, struct gobuffer *strings);
 int  btf_elf__encode(struct btf_elf *btf, uint8_t flags);
 
pahole.c +1 -0
index e2a081b..8407db9 100644
@@ -1084,6 +1084,7 @@ static error_t pahole__options_parser(int key, char *arg,
 	case 'i': find_containers = 1;
 		  class_name = arg;			break;
 	case 'J': btf_encode = 1;
+		  conf_load.get_addr_info = true;
 		  no_bitfield_type_recode = true;	break;
 	case 'l': conf.show_first_biggest_size_base_type_member = 1;	break;
 	case 'M': conf.show_only_data_members = 1;	break;
Signature · 1 line
2.27.0.290.gba653c62da-goog

A message that is not in the archive (sent off-list or not yet received): 20200618074853.133675-1-haoluo-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org

#2 lore raw
Quoted 33 lines · Hao Luo

On SMP systems, the global percpu variables are placed in a special '.data..percpu' section, which is stored in a segment whose initial address is set to 0, the addresses of per-CPU variables are relative positive addresses [1].

This patch extracts these variables from vmlinux and places them with their type information in BTF. More specifically, when BTF is encoded, we find the index of the '.data..percpu' section and then traverse the symbol table to find those global objects which are in this section. For each of these objects, we push a BTF_KIND_VAR into the types buffer, and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the CUs have finished processing, we push a BTF_KIND_DATASEC into the btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.7-rc7 linux kernel, I was able to extract 291 such variables. The build time overhead is small and the space overhead is also small.

Testing:

- vmlinux size has increased by ~12kb.
Before:

   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
  After:
   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00

- Common global percpu VARs and DATASEC are found in BTF section.

  $ bpftool btf dump file vmlinux | grep runqueues
  [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
Quoted 6 lines
  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17592] STRUCT 'cpu_stopper' size=72 vlen=5
  [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=0 vlen=294

probably forgot to update the example, I'd imagine size wouldn't be 0 anymore?

Quoted 7 lines

- Tested bpf selftests.

References:
[1] https://lwn.net/Articles/531148/

---

I'm quite a bit uncomfortable that pahole would just ignore non-conformat variables. We should understand why there might even be those variables with invalid names, how that can happen in C?

Quoted 51 lines

Changelog since v3:
- Correct the size of the DATASEC, which was previously 0 and now is the last var_secinfo's offset + size.
- Add several sanity checks on VARs and DATASEC to ensure that the generated BTF will not be rejected by verifier.
- Tested through a set of selftests and bpf_tracing programs.

Changelog since v2:
- Move finding percpu_shndx and extracting symtab into btfe creation, so we don't have to allocate a new symtab for each CU.
- More debug msg by logging the vars encoded in 'verbose' mode. We probably don't want to log the symbols that are _not_ encoded, since that would be too verbose.
- Calculate var offsets using 'addr - shdr.sh_addr', so it could be generalized to other sections in future.
- Filter out the symbols that are not STT_OBJECT.
- Sort var_secinfos in the DATASEC by their offsets.
- Free 'persec_secinfo' buffer and 'symtab' in btfe deletion.
- Replace the string ".data..percpu" with a constant PERCPU_SECTION.

Changelog since v1:
- Add a ".data..percpu" DATASEC that encodes the found VARs.
- Use percpu section's shndx to find the symbols that are percpu variables.
- Use the correct type to set VAR's linkage.

 btf_encoder.c | 185 ++++++++++++++++++++++++++++++++++++++++++++++++++
 dwarves.c     |   6 ++
 dwarves.h     |   2 +
 libbtf.c      | 127 ++++++++++++++++++++++++++++++++++
 libbtf.h      |  12 ++++
 pahole.c      |   1 +
 6 files changed, 333 insertions(+)
btf_encoder.c +8 -0
index df16ba0..c7401c3 100644
@@ -16,8 +16,44 @@
 #include "elf_symtab.h"
 #include "btf_encoder.h"
 
+#include <ctype.h> /* for isalpha() and isalnum() */
 #include <inttypes.h>
 
+/*
+ * This corresponds to the same macro defined in
+ * include/linux/kallsyms.h
+ */
+#define KSYM_NAME_LEN 128
+
+static bool btf_name_char_ok(char c, bool first, bool dot_ok)

btf_encoder.c line 38

Quoted 10 lines
btf_encoder.c +10 -0
+{
+       if ((first ? !isalpha(c) : !isalnum(c)) &&
+           c != '_' &&
+           ((c == '.' && !dot_ok) || c != '.'))
+               return false;
+       return true;
+}
+
+/* Check wether the given name is valid in vmlinux btf. */
+static bool btf_name_valid(const char *p, bool dot_ok)

dot_ok is always true, you can simplify above function

Quoted 18 lines
btf_encoder.c +18 -0
+{
+       const char *limit;
+
+       if (!btf_name_char_ok(*p, true, dot_ok))
+               return false;
+
+       /* set a limit on identifier length */
+       limit = p + KSYM_NAME_LEN;
+       p++;
+       while (*p && p < limit) {
+               if (!btf_name_char_ok(*p, false, dot_ok))
+                       return false;
+               p++;
+       }
+
+       return !*p;
+}
+

[...]

Quoted 29 lines
btf_encoder.c +29 -0
+       /* search within symtab for percpu variables */
+       elf_symtab__for_each_symbol(btfe->symtab, core_id, sym) {
+               uint32_t linkage, type, size, offset;
+               int32_t btf_var_id, btf_var_secinfo_id;
+               uint64_t addr;
+               const char *name;
+
+               /* compare a symbol's shndx to determine if it's a percpu variable */
+               if (elf_sym__section(&sym) != btfe->percpu_shndx)
+                       continue;
+               if (elf_sym__type(&sym) != STT_OBJECT)
+                       continue;
+
+               addr = elf_sym__value(&sym);
+               /*
+                * Store only those symbols that have allocated space in the percpu section.
+                * This excludes the following three types of symbols:
+                *
+                *  1. __ADDRESSABLE(sym), which are forcely emitted as symbols.
+                *  2. __UNIQUE_ID(prefix), which are introduced to generate unique ids.
+                *  3. __exitcall(fn), functions which are labeled as exit calls.
+                *
+                * In addition, the variables defined using DEFINE_PERCPU_FIRST are
+                * also not included, which currently includes:
+                *
+                *  1. fixed_percpu_data
+                */
+               if (!addr)
+                       continue;

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 373

Quoted 10 lines
btf_encoder.c +10 -0
+               var = hashaddr__find_variable(hash_addr, addr);
+               if (var == NULL)
+                       continue;
+
+               /*
+                * Valididate the variable's name, the same check is also
+                * performed in bpf verifier.
+                */
+               name = variable__name(var, cu);
+               if (!var->name || !btf_name_valid(name, true)) {

It might be ok to filter them out, but I'd rather try to first understand why do we get those invalid names in the first place. Otherwise we might be just papering over the real bug in how we process symbols/variables in DWARF?

Quoted 4 lines
btf_encoder.c +4 -0
+                       if (verbose)
+                               printf("invalid variable name '%s'.\n", name);
+                       continue;
+               }

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 387

Quoted 10 lines
btf_encoder.c +10 -0
+
+               /*
+                * Validate the variable's type, the same check is also
+                * performed in bpf verifier.
+                */
+               type = var->ip.tag.type + type_id_off;
+               if (type == 0) {
+                       if (verbose)
+                               printf("invalid type for variable '%s'.\n", name);
+                       continue;

Again, there might be cases which we want to ignore, but we need to understand what and why those cases are?

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 400

Quoted 13 lines
btf_encoder.c +13 -0
+               }
+
+               /*
+                * Validate the size of the variable's type, the same check is
+                * also performed in bpf verifier.
+                */
+               size = variable__type_size(var, cu);
+               if (size == 0) {
+                       if (verbose)
+                               printf("invalid type size for variable '%s'.\n", name);
+                       continue;
+               }
+

The goal here is not to satisfy the verifier, but to generate BTF from DWARF faithfully. Let's not just ignore "inconvenient" cases and instead investigate why such cases happen.

Quoted 31 lines
btf_encoder.c +28 -0
+               if (verbose)
+                       printf("symbol '%s' of address 0x%lx encoded\n",
+                              elf_sym__name(&sym, btfe->symtab), addr);
+
+               /* add a BTF_KIND_VAR in btfe->types */
+               linkage = var->external ? BTF_VAR_GLOBAL_ALLOCATED : BTF_VAR_STATIC;
+               btf_var_id = btf_elf__add_var_type(btfe, type, var->name, linkage);
+               if (btf_var_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode variable '%s'\n", name);
+                       break;
+               }
+
+               /*
+                * add a BTF_VAR_SECINFO in btfe->percpu_secinfo, which will be added into
+                * btfe->types later when we add BTF_VAR_DATASEC.
+                */
+               type = btf_var_id;
+               offset = addr - btfe->percpu_base_addr;
+               btf_var_secinfo_id = btf_elf__add_var_secinfo(&btfe->percpu_secinfo,
+                                                             type, offset, size);
+               if (btf_var_secinfo_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode var secinfo '%s'\n", name);
+                       break;
+               }
+       }
+
 out:
        if (err)
                btf_elf__delete(btfe);

[...]

A message that is not in the archive (sent off-list or not yet received): CAEf4BzbE2K7uyJ-2d3N+NLrV5OKS0HMm5vKuxMz5bfZkE_ANzw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

author
#4 lore raw

Hi, Andrii,

I agree that we'd better put a hold on this patch until we find out the reason for the 'unconventional' symbols. I'll _try_ to figure it out, but not able to fully commit my time on this patch. I thought I'd better publish this patch as the DATASEC and VARs are generated correctly in format, so that anyone can use it to generate the vmlinux and continue the development on libbpf based on your ksym work (i.e. typed ksyms).

Hao

Quoted 35 lines · Andrii Nakryiko
Quoted 33 lines · Hao Luo

On SMP systems, the global percpu variables are placed in a special '.data..percpu' section, which is stored in a segment whose initial address is set to 0, the addresses of per-CPU variables are relative positive addresses [1].

This patch extracts these variables from vmlinux and places them with their type information in BTF. More specifically, when BTF is encoded, we find the index of the '.data..percpu' section and then traverse the symbol table to find those global objects which are in this section. For each of these objects, we push a BTF_KIND_VAR into the types buffer, and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the CUs have finished processing, we push a BTF_KIND_DATASEC into the btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.7-rc7 linux kernel, I was able to extract 291 such variables. The build time overhead is small and the space overhead is also small.

Testing:

- vmlinux size has increased by ~12kb.
Before:

   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
  After:
   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00

- Common global percpu VARs and DATASEC are found in BTF section.

  $ bpftool btf dump file vmlinux | grep runqueues
  [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
Quoted 9 lines
Quoted 6 lines
  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17592] STRUCT 'cpu_stopper' size=72 vlen=5
  [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=0 vlen=294

probably forgot to update the example, I'd imagine size wouldn't be 0 anymore?

Ah, right, I overlooked the example.

Quoted 212 lines
Quoted 7 lines

- Tested bpf selftests.

References:
[1] https://lwn.net/Articles/531148/

---

I'm quite a bit uncomfortable that pahole would just ignore non-conformat variables. We should understand why there might even be those variables with invalid names, how that can happen in C?

Quoted 51 lines

Changelog since v3:
- Correct the size of the DATASEC, which was previously 0 and now is the last var_secinfo's offset + size.
- Add several sanity checks on VARs and DATASEC to ensure that the generated BTF will not be rejected by verifier.
- Tested through a set of selftests and bpf_tracing programs.

Changelog since v2:
- Move finding percpu_shndx and extracting symtab into btfe creation, so we don't have to allocate a new symtab for each CU.
- More debug msg by logging the vars encoded in 'verbose' mode. We probably don't want to log the symbols that are _not_ encoded, since that would be too verbose.
- Calculate var offsets using 'addr - shdr.sh_addr', so it could be generalized to other sections in future.
- Filter out the symbols that are not STT_OBJECT.
- Sort var_secinfos in the DATASEC by their offsets.
- Free 'persec_secinfo' buffer and 'symtab' in btfe deletion.
- Replace the string ".data..percpu" with a constant PERCPU_SECTION.

Changelog since v1:
- Add a ".data..percpu" DATASEC that encodes the found VARs.
- Use percpu section's shndx to find the symbols that are percpu variables.
- Use the correct type to set VAR's linkage.

 btf_encoder.c | 185 ++++++++++++++++++++++++++++++++++++++++++++++++++
 dwarves.c     |   6 ++
 dwarves.h     |   2 +
 libbtf.c      | 127 ++++++++++++++++++++++++++++++++++
 libbtf.h      |  12 ++++
 pahole.c      |   1 +
 6 files changed, 333 insertions(+)
btf_encoder.c +8 -0
index df16ba0..c7401c3 100644
@@ -16,8 +16,44 @@
 #include "elf_symtab.h"
 #include "btf_encoder.h"
 
+#include <ctype.h> /* for isalpha() and isalnum() */
 #include <inttypes.h>
 
+/*
+ * This corresponds to the same macro defined in
+ * include/linux/kallsyms.h
+ */
+#define KSYM_NAME_LEN 128
+
+static bool btf_name_char_ok(char c, bool first, bool dot_ok)

btf_encoder.c line 38

Quoted 10 lines
btf_encoder.c +10 -0
+{
+       if ((first ? !isalpha(c) : !isalnum(c)) &&
+           c != '_' &&
+           ((c == '.' && !dot_ok) || c != '.'))
+               return false;
+       return true;
+}
+
+/* Check wether the given name is valid in vmlinux btf. */
+static bool btf_name_valid(const char *p, bool dot_ok)

dot_ok is always true, you can simplify above function

Quoted 18 lines
btf_encoder.c +18 -0
+{
+       const char *limit;
+
+       if (!btf_name_char_ok(*p, true, dot_ok))
+               return false;
+
+       /* set a limit on identifier length */
+       limit = p + KSYM_NAME_LEN;
+       p++;
+       while (*p && p < limit) {
+               if (!btf_name_char_ok(*p, false, dot_ok))
+                       return false;
+               p++;
+       }
+
+       return !*p;
+}
+

[...]

Quoted 29 lines
btf_encoder.c +29 -0
+       /* search within symtab for percpu variables */
+       elf_symtab__for_each_symbol(btfe->symtab, core_id, sym) {
+               uint32_t linkage, type, size, offset;
+               int32_t btf_var_id, btf_var_secinfo_id;
+               uint64_t addr;
+               const char *name;
+
+               /* compare a symbol's shndx to determine if it's a percpu variable */
+               if (elf_sym__section(&sym) != btfe->percpu_shndx)
+                       continue;
+               if (elf_sym__type(&sym) != STT_OBJECT)
+                       continue;
+
+               addr = elf_sym__value(&sym);
+               /*
+                * Store only those symbols that have allocated space in the percpu section.
+                * This excludes the following three types of symbols:
+                *
+                *  1. __ADDRESSABLE(sym), which are forcely emitted as symbols.
+                *  2. __UNIQUE_ID(prefix), which are introduced to generate unique ids.
+                *  3. __exitcall(fn), functions which are labeled as exit calls.
+                *
+                * In addition, the variables defined using DEFINE_PERCPU_FIRST are
+                * also not included, which currently includes:
+                *
+                *  1. fixed_percpu_data
+                */
+               if (!addr)
+                       continue;

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 373

Quoted 10 lines
btf_encoder.c +10 -0
+               var = hashaddr__find_variable(hash_addr, addr);
+               if (var == NULL)
+                       continue;
+
+               /*
+                * Valididate the variable's name, the same check is also
+                * performed in bpf verifier.
+                */
+               name = variable__name(var, cu);
+               if (!var->name || !btf_name_valid(name, true)) {

It might be ok to filter them out, but I'd rather try to first understand why do we get those invalid names in the first place. Otherwise we might be just papering over the real bug in how we process symbols/variables in DWARF?

Quoted 4 lines
btf_encoder.c +4 -0
+                       if (verbose)
+                               printf("invalid variable name '%s'.\n", name);
+                       continue;
+               }

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 387

Quoted 10 lines
btf_encoder.c +10 -0
+
+               /*
+                * Validate the variable's type, the same check is also
+                * performed in bpf verifier.
+                */
+               type = var->ip.tag.type + type_id_off;
+               if (type == 0) {
+                       if (verbose)
+                               printf("invalid type for variable '%s'.\n", name);
+                       continue;

Again, there might be cases which we want to ignore, but we need to understand what and why those cases are?

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 400

Quoted 13 lines
btf_encoder.c +13 -0
+               }
+
+               /*
+                * Validate the size of the variable's type, the same check is
+                * also performed in bpf verifier.
+                */
+               size = variable__type_size(var, cu);
+               if (size == 0) {
+                       if (verbose)
+                               printf("invalid type size for variable '%s'.\n", name);
+                       continue;
+               }
+

The goal here is not to satisfy the verifier, but to generate BTF from DWARF faithfully. Let's not just ignore "inconvenient" cases and instead investigate why such cases happen.

Quoted 31 lines
btf_encoder.c +28 -0
+               if (verbose)
+                       printf("symbol '%s' of address 0x%lx encoded\n",
+                              elf_sym__name(&sym, btfe->symtab), addr);
+
+               /* add a BTF_KIND_VAR in btfe->types */
+               linkage = var->external ? BTF_VAR_GLOBAL_ALLOCATED : BTF_VAR_STATIC;
+               btf_var_id = btf_elf__add_var_type(btfe, type, var->name, linkage);
+               if (btf_var_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode variable '%s'\n", name);
+                       break;
+               }
+
+               /*
+                * add a BTF_VAR_SECINFO in btfe->percpu_secinfo, which will be added into
+                * btfe->types later when we add BTF_VAR_DATASEC.
+                */
+               type = btf_var_id;
+               offset = addr - btfe->percpu_base_addr;
+               btf_var_secinfo_id = btf_elf__add_var_secinfo(&btfe->percpu_secinfo,
+                                                             type, offset, size);
+               if (btf_var_secinfo_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode var secinfo '%s'\n", name);
+                       break;
+               }
+       }
+
 out:
        if (err)
                btf_elf__delete(btfe);

[...]

A message that is not in the archive (sent off-list or not yet received): CA+khW7j3AoW8q+Q30RXG-psw4imkBxNqffObnu2dr2=K=oVBCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#6 lore raw
Quoted 11 lines · Hao Luo

Hi, Andrii,

I agree that we'd better put a hold on this patch until we find out the reason for the 'unconventional' symbols. I'll _try_ to figure it out, but not able to fully commit my time on this patch. I thought I'd better publish this patch as the DATASEC and VARs are generated correctly in format, so that anyone can use it to generate the vmlinux and continue the development on libbpf based on your ksym work (i.e. typed ksyms).

Hey Hao,

It's a pity that you are willing to drop this while being half-step away from doing this properly. See my other reply, there is something fishy with variable__name() and variable__type_size() and how it calculates/caches values. ELF itself has all the data (based ELF symbol data) and it seems to be correct. And I was correct to be suspicious about just filtering out such variables, because in my case you'd filter out a good chunk of variables for no good reason: 74 variables, which is a 26% of all per-CPU variables in my kernel.

It's up to you, of course, but it would be nice to fix it up (just switching to elf_sym__size() and elf_sym__name() would be fine, you'd just need to make sure to add elf_sym_name() result into a string buffer; unless Arnaldo has some better alternatives) and publish complete v5, that would get merged into pahole. After that, building on my .ksym work in libbpf should get you to what you need from BPF pretty quickly (plus some kernel-side logic to recognize these per-CPU variables).

Quoted 48 lines

Hao

Quoted 44 lines · Andrii Nakryiko
Quoted 33 lines · Hao Luo

On SMP systems, the global percpu variables are placed in a special '.data..percpu' section, which is stored in a segment whose initial address is set to 0, the addresses of per-CPU variables are relative positive addresses [1].

This patch extracts these variables from vmlinux and places them with their type information in BTF. More specifically, when BTF is encoded, we find the index of the '.data..percpu' section and then traverse the symbol table to find those global objects which are in this section. For each of these objects, we push a BTF_KIND_VAR into the types buffer, and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the CUs have finished processing, we push a BTF_KIND_DATASEC into the btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.7-rc7 linux kernel, I was able to extract 291 such variables. The build time overhead is small and the space overhead is also small.

Testing:

- vmlinux size has increased by ~12kb.
Before:

   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
  After:
   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00

- Common global percpu VARs and DATASEC are found in BTF section.

  $ bpftool btf dump file vmlinux | grep runqueues
  [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
Quoted 6 lines
  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17592] STRUCT 'cpu_stopper' size=72 vlen=5
  [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=0 vlen=294

probably forgot to update the example, I'd imagine size wouldn't be 0 anymore?

[...]

A message that is not in the archive (sent off-list or not yet received): CAEf4Bza20kx3d+84Fg+UKXFYb2vforj55g0zO92Hpz+cM49LQA-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

author
#8 lore raw

Hi Andrii,

Thanks for taking a look! I can post a v5 based on your thorough and detailed analysis this week. Some of my other work on sched consumed most of my bandwidth last week and I very appreciate your help! I apologize if it is felt I dropped this patch and will follow up within this week.

Hao

Quoted 84 lines · Andrii Nakryiko
Quoted 11 lines · Hao Luo

Hi, Andrii,

I agree that we'd better put a hold on this patch until we find out the reason for the 'unconventional' symbols. I'll _try_ to figure it out, but not able to fully commit my time on this patch. I thought I'd better publish this patch as the DATASEC and VARs are generated correctly in format, so that anyone can use it to generate the vmlinux and continue the development on libbpf based on your ksym work (i.e. typed ksyms).

Hey Hao,

It's a pity that you are willing to drop this while being half-step away from doing this properly. See my other reply, there is something fishy with variable__name() and variable__type_size() and how it calculates/caches values. ELF itself has all the data (based ELF symbol data) and it seems to be correct. And I was correct to be suspicious about just filtering out such variables, because in my case you'd filter out a good chunk of variables for no good reason: 74 variables, which is a 26% of all per-CPU variables in my kernel.

It's up to you, of course, but it would be nice to fix it up (just switching to elf_sym__size() and elf_sym__name() would be fine, you'd just need to make sure to add elf_sym_name() result into a string buffer; unless Arnaldo has some better alternatives) and publish complete v5, that would get merged into pahole. After that, building on my .ksym work in libbpf should get you to what you need from BPF pretty quickly (plus some kernel-side logic to recognize these per-CPU variables).

Quoted 48 lines

Hao

Quoted 44 lines · Andrii Nakryiko
Quoted 33 lines · Hao Luo

On SMP systems, the global percpu variables are placed in a special '.data..percpu' section, which is stored in a segment whose initial address is set to 0, the addresses of per-CPU variables are relative positive addresses [1].

This patch extracts these variables from vmlinux and places them with their type information in BTF. More specifically, when BTF is encoded, we find the index of the '.data..percpu' section and then traverse the symbol table to find those global objects which are in this section. For each of these objects, we push a BTF_KIND_VAR into the types buffer, and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the CUs have finished processing, we push a BTF_KIND_DATASEC into the btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.7-rc7 linux kernel, I was able to extract 291 such variables. The build time overhead is small and the space overhead is also small.

Testing:

- vmlinux size has increased by ~12kb.
Before:

   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
  After:
   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00

- Common global percpu VARs and DATASEC are found in BTF section.

  $ bpftool btf dump file vmlinux | grep runqueues
  [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
Quoted 6 lines
  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17592] STRUCT 'cpu_stopper' size=72 vlen=5
  [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=0 vlen=294

probably forgot to update the example, I'd imagine size wouldn't be 0 anymore?

[...]

A message that is not in the archive (sent off-list or not yet received): CA+khW7hqX9fJd2G8-Kze9pMR4hUKV5ROxn1vKYy82+PCzDeCDQ-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#10 lore raw
Quoted 7 lines · Hao Luo

Hi Andrii,

Thanks for taking a look! I can post a v5 based on your thorough and detailed analysis this week. Some of my other work on sched consumed most of my bandwidth last week and I very appreciate your help! I apologize if it is felt I dropped this patch and will follow up within this week.

I'll also try and look at this tomorrow.

- Arnaldo

Quoted 112 lines

Hao

Quoted 7 lines · Andrii Nakryiko
Quoted 5 lines · Hao Luo

Hi, Andrii,

I agree that we'd better put a hold on this patch until we find out the reason for the 'unconventional' symbols. I'll _try_ to figure

it

Quoted 1 line
Quoted 1 line

out, but not able to fully commit my time on this patch. I thought

I'd

Quoted 2 lines
Quoted 2 lines

better publish this patch as the DATASEC and VARs are generated correctly in format, so that anyone can use it to generate the

vmlinux

Quoted 1 line
Quoted 1 line

and continue the development on libbpf based on your ksym work

(i.e.

Quoted 11 lines
Quoted 2 lines

typed ksyms).

Hey Hao,

It's a pity that you are willing to drop this while being half-step away from doing this properly. See my other reply, there is something fishy with variable__name() and variable__type_size() and how it calculates/caches values. ELF itself has all the data (based ELF symbol data) and it seems to be correct. And I was correct to be suspicious about just filtering out such variables, because in my

case

Quoted 10 lines

you'd filter out a good chunk of variables for no good reason: 74 variables, which is a 26% of all per-CPU variables in my kernel.

It's up to you, of course, but it would be nice to fix it up (just switching to elf_sym__size() and elf_sym__name() would be fine, you'd just need to make sure to add elf_sym_name() result into a string buffer; unless Arnaldo has some better alternatives) and publish complete v5, that would get merged into pahole. After that, building on my .ksym work in libbpf should get you to what you need from BPF pretty quickly (plus some kernel-side logic to recognize these

per-CPU

Quoted 8 lines

variables).

Quoted 6 lines

Hao

Quoted 2 lines · Andrii Nakryiko

On Thu, Jun 18, 2020 at 12:49 AM Hao Luo <[email protected]>

Quoted 2 lines · wrote:
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

On SMP systems, the global percpu variables are placed in a

special

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

'.data..percpu' section, which is stored in a segment whose

initial

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

address is set to 0, the addresses of per-CPU variables are

relative

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

positive addresses [1].

This patch extracts these variables from vmlinux and places

them with

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

their type information in BTF. More specifically, when BTF is

encoded,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

we find the index of the '.data..percpu' section and then

traverse

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

the symbol table to find those global objects which are in this

section.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

For each of these objects, we push a BTF_KIND_VAR into the

types buffer,

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When

all the

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

CUs have finished processing, we push a BTF_KIND_DATASEC into

the

Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines

btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.7-rc7 linux kernel, I was able to extract 291 such

variables.

Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line

The build time overhead is small and the space overhead is also

small.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

Testing:

- vmlinux size has increased by ~12kb.
Before:

   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c

13a905c 2d2bf8 00

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

After:

   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c

13a905c 2d5bca 00

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

- Common global percpu VARs and DATASEC are found in BTF

section.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 9 lines
  $ bpftool btf dump file vmlinux | grep runqueues
  [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17592] STRUCT 'cpu_stopper' size=72 vlen=5
  [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=0 vlen=294

probably forgot to update the example, I'd imagine size wouldn't

be 0 anymore?

Quoted 3 lines
Quoted 1 line
Quoted 1 line

[...]

Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
#11 lore raw
In reply to a message not in the archive
Quoted 178 lines · Andrii Nakryiko
Quoted 33 lines · Hao Luo

On SMP systems, the global percpu variables are placed in a special '.data..percpu' section, which is stored in a segment whose initial address is set to 0, the addresses of per-CPU variables are relative positive addresses [1].

This patch extracts these variables from vmlinux and places them with their type information in BTF. More specifically, when BTF is encoded, we find the index of the '.data..percpu' section and then traverse the symbol table to find those global objects which are in this section. For each of these objects, we push a BTF_KIND_VAR into the types buffer, and a BTF_VAR_SECINFO into another buffer, percpu_secinfo. When all the CUs have finished processing, we push a BTF_KIND_DATASEC into the btfe->types buffer, followed by the percpu_secinfo's content.

In a v5.7-rc7 linux kernel, I was able to extract 291 such variables. The build time overhead is small and the space overhead is also small.

Testing:

- vmlinux size has increased by ~12kb.
Before:

   $ readelf -SW vmlinux | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d2bf8 00
  After:
   $ pahole -J vmlinux
   $ readelf -SW vmlinux  | grep BTF
   [25] .BTF              PROGBITS        ffffffff821a905c 13a905c 2d5bca 00

- Common global percpu VARs and DATASEC are found in BTF section.

  $ bpftool btf dump file vmlinux | grep runqueues
  [14098] VAR 'runqueues' type_id=13725, linkage=global-alloc
Quoted 6 lines
  $ bpftool btf dump file vmlinux | grep 'cpu_stopper'
  [17592] STRUCT 'cpu_stopper' size=72 vlen=5
  [17612] VAR 'cpu_stopper' type_id=17592, linkage=static
  $ bpftool btf dump file vmlinux | grep ' DATASEC '
  [63652] DATASEC '.data..percpu' size=0 vlen=294

probably forgot to update the example, I'd imagine size wouldn't be 0 anymore?

Quoted 7 lines

- Tested bpf selftests.

References:
[1] https://lwn.net/Articles/531148/

---

I'm quite a bit uncomfortable that pahole would just ignore non-conformat variables. We should understand why there might even be those variables with invalid names, how that can happen in C?

Quoted 51 lines

Changelog since v3:
- Correct the size of the DATASEC, which was previously 0 and now is the last var_secinfo's offset + size.
- Add several sanity checks on VARs and DATASEC to ensure that the generated BTF will not be rejected by verifier.
- Tested through a set of selftests and bpf_tracing programs.

Changelog since v2:
- Move finding percpu_shndx and extracting symtab into btfe creation, so we don't have to allocate a new symtab for each CU.
- More debug msg by logging the vars encoded in 'verbose' mode. We probably don't want to log the symbols that are _not_ encoded, since that would be too verbose.
- Calculate var offsets using 'addr - shdr.sh_addr', so it could be generalized to other sections in future.
- Filter out the symbols that are not STT_OBJECT.
- Sort var_secinfos in the DATASEC by their offsets.
- Free 'persec_secinfo' buffer and 'symtab' in btfe deletion.
- Replace the string ".data..percpu" with a constant PERCPU_SECTION.

Changelog since v1:
- Add a ".data..percpu" DATASEC that encodes the found VARs.
- Use percpu section's shndx to find the symbols that are percpu variables.
- Use the correct type to set VAR's linkage.

 btf_encoder.c | 185 ++++++++++++++++++++++++++++++++++++++++++++++++++
 dwarves.c     |   6 ++
 dwarves.h     |   2 +
 libbtf.c      | 127 ++++++++++++++++++++++++++++++++++
 libbtf.h      |  12 ++++
 pahole.c      |   1 +
 6 files changed, 333 insertions(+)
btf_encoder.c +8 -0
index df16ba0..c7401c3 100644
@@ -16,8 +16,44 @@
 #include "elf_symtab.h"
 #include "btf_encoder.h"
 
+#include <ctype.h> /* for isalpha() and isalnum() */
 #include <inttypes.h>
 
+/*
+ * This corresponds to the same macro defined in
+ * include/linux/kallsyms.h
+ */
+#define KSYM_NAME_LEN 128
+
+static bool btf_name_char_ok(char c, bool first, bool dot_ok)

btf_encoder.c line 38

Quoted 10 lines
btf_encoder.c +10 -0
+{
+       if ((first ? !isalpha(c) : !isalnum(c)) &&
+           c != '_' &&
+           ((c == '.' && !dot_ok) || c != '.'))
+               return false;
+       return true;
+}
+
+/* Check wether the given name is valid in vmlinux btf. */
+static bool btf_name_valid(const char *p, bool dot_ok)

dot_ok is always true, you can simplify above function

Quoted 18 lines
btf_encoder.c +18 -0
+{
+       const char *limit;
+
+       if (!btf_name_char_ok(*p, true, dot_ok))
+               return false;
+
+       /* set a limit on identifier length */
+       limit = p + KSYM_NAME_LEN;
+       p++;
+       while (*p && p < limit) {
+               if (!btf_name_char_ok(*p, false, dot_ok))
+                       return false;
+               p++;
+       }
+
+       return !*p;
+}
+

[...]

Quoted 36 lines
btf_encoder.c +36 -0
+       /* search within symtab for percpu variables */
+       elf_symtab__for_each_symbol(btfe->symtab, core_id, sym) {
+               uint32_t linkage, type, size, offset;
+               int32_t btf_var_id, btf_var_secinfo_id;
+               uint64_t addr;
+               const char *name;
+
+               /* compare a symbol's shndx to determine if it's a percpu variable */
+               if (elf_sym__section(&sym) != btfe->percpu_shndx)
+                       continue;
+               if (elf_sym__type(&sym) != STT_OBJECT)
+                       continue;
+
+               addr = elf_sym__value(&sym);
+               /*
+                * Store only those symbols that have allocated space in the percpu section.
+                * This excludes the following three types of symbols:
+                *
+                *  1. __ADDRESSABLE(sym), which are forcely emitted as symbols.
+                *  2. __UNIQUE_ID(prefix), which are introduced to generate unique ids.
+                *  3. __exitcall(fn), functions which are labeled as exit calls.
+                *
+                * In addition, the variables defined using DEFINE_PERCPU_FIRST are
+                * also not included, which currently includes:
+                *
+                *  1. fixed_percpu_data
+                */
+               if (!addr)
+                       continue;
+               var = hashaddr__find_variable(hash_addr, addr);
+               if (var == NULL)
+                       continue;
+
+               /*
+                * Valididate the variable's name, the same check is also
+                * performed in bpf verifier.
Quoted 8 lines

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 373

Quoted 3 lines
btf_encoder.c +3 -0
+                */
+               name = variable__name(var, cu);
+               if (!var->name || !btf_name_valid(name, true)) {

It might be ok to filter them out, but I'd rather try to first understand why do we get those invalid names in the first place. Otherwise we might be just papering over the real bug in how we process symbols/variables in DWARF?

Ok, so I played with it a bit locally. Those variables with empty name or 0 size seem to be due to some bugs in variable__name() and variable__type_size() implementations. If you stick to ELF symbol's elf_sym__name() and elf_sym__size(), you'll get valid sizes.

Arnaldo, could you please take a look at why variable__xxx() impls don't work for some variables? I can repro it locally, e.g.:

ELF SYM kstat == <empty>, SZ 48 == 0
invalid type size for variable '(null)'.
invalid variable name '(null)'.

kstat and 48 come from ELF symbol, <empty> and 0 are from variabel__xxx(). And that happens for quite a few variables, actually.

With those fixes, my kernel can be successfully BTF-generated. The number of per-cpu variables jump from 213 with all this filtering to 287 proper variables while using elf_sym__xxx() functions.

Quoted 70 lines
Quoted 5 lines
btf_encoder.c +4 -0
+                       if (verbose)
+                               printf("invalid variable name '%s'.\n", name);
+                       continue;
+               }

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 387

Quoted 10 lines
btf_encoder.c +10 -0
+
+               /*
+                * Validate the variable's type, the same check is also
+                * performed in bpf verifier.
+                */
+               type = var->ip.tag.type + type_id_off;
+               if (type == 0) {
+                       if (verbose)
+                               printf("invalid type for variable '%s'.\n", name);
+                       continue;

Again, there might be cases which we want to ignore, but we need to understand what and why those cases are?

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 400

Quoted 13 lines
btf_encoder.c +13 -0
+               }
+
+               /*
+                * Validate the size of the variable's type, the same check is
+                * also performed in bpf verifier.
+                */
+               size = variable__type_size(var, cu);
+               if (size == 0) {
+                       if (verbose)
+                               printf("invalid type size for variable '%s'.\n", name);
+                       continue;
+               }
+

The goal here is not to satisfy the verifier, but to generate BTF from DWARF faithfully. Let's not just ignore "inconvenient" cases and instead investigate why such cases happen.

Quoted 31 lines
btf_encoder.c +28 -0
+               if (verbose)
+                       printf("symbol '%s' of address 0x%lx encoded\n",
+                              elf_sym__name(&sym, btfe->symtab), addr);
+
+               /* add a BTF_KIND_VAR in btfe->types */
+               linkage = var->external ? BTF_VAR_GLOBAL_ALLOCATED : BTF_VAR_STATIC;
+               btf_var_id = btf_elf__add_var_type(btfe, type, var->name, linkage);
+               if (btf_var_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode variable '%s'\n", name);
+                       break;
+               }
+
+               /*
+                * add a BTF_VAR_SECINFO in btfe->percpu_secinfo, which will be added into
+                * btfe->types later when we add BTF_VAR_DATASEC.
+                */
+               type = btf_var_id;
+               offset = addr - btfe->percpu_base_addr;
+               btf_var_secinfo_id = btf_elf__add_var_secinfo(&btfe->percpu_secinfo,
+                                                             type, offset, size);
+               if (btf_var_secinfo_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode var secinfo '%s'\n", name);
+                       break;
+               }
+       }
+
 out:
        if (err)
                btf_elf__delete(btfe);

[...]

A message that is not in the archive (sent off-list or not yet received): CAEf4Bzbiu8G5UK9TDBRFYGwVk4A4QWoLurZsFKhs439gBXaDCg-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org

#13 lore raw

Em Mon, Jun 29, 2020 at 01:20:02PM -0700, Andrii Nakryiko escreveu:

Quoted 21 lines · Andrii Nakryiko
Quoted 15 lines · Hao Luo

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 373

Quoted 12 lines
btf_encoder.c +12 -0
+               if (!addr)
+                       continue;
+               var = hashaddr__find_variable(hash_addr, addr);
+               if (var == NULL)
+                       continue;
+
+               /*
+                * Valididate the variable's name, the same check is also
+                * performed in bpf verifier.
+                */
+               name = variable__name(var, cu);
+               if (!var->name || !btf_name_valid(name, true)) {
Quoted 4 lines

It might be ok to filter them out, but I'd rather try to first understand why do we get those invalid names in the first place. Otherwise we might be just papering over the real bug in how we process symbols/variables in DWARF?

Quoted 7 lines

Ok, so I played with it a bit locally. Those variables with empty name or 0 size seem to be due to some bugs in variable__name() and variable__type_size() implementations. If you stick to ELF symbol's elf_sym__name() and elf_sym__size(), you'll get valid sizes.

Arnaldo, could you please take a look at why variable__xxx() impls don't work for some variables? I can repro it locally, e.g.:

I'm looking at it, I think this is that maybe not all variables that are in the elf symbol table are encoded in DWARF (maybe some declared in asm files?) and thus we will not find its name in the .debug_str.

ctf_loader.c, that was used as the model for btf_loader.c has this:

static const char *ctf__variable_name(const struct variable *var,
const struct cu *cu)

{
        struct ctf *ctf = cu->priv;

        return ctf->symtab->symstrs->d_buf + var->name;
}

struct debug_fmt_ops ctf__ops = {
.name = "ctf",
.function__name = ctf__function_name,
.load_file = ctf__load_file,
.variable__name = ctf__variable_name,
.strings__ptr = ctf__strings_ptr,
.cu__delete = ctf__cu_delete,
};

And then

const char *variable__name(const struct variable *var, const struct cu *cu)

{
        if (cu->dfops && cu->dfops->variable__name)
                return cu->dfops->variable__name(var, cu);
        return s(cu, var->name);
}

I.e. btf_loader.c and dwarf_loader.c are using the fallback return that assumes all strings are in a single table, the CTF one gets variable names from the ELF symtab, as you seem to be doing by using elf_sym__name() directly.

I'm reading Hao's patch to see how the variables are being created as they are loaded from DWARF, to check this hunch.

- Arnaldo

Quoted 81 lines

ELF SYM kstat == <empty>, SZ 48 == 0
invalid type size for variable '(null)'.
invalid variable name '(null)'.

kstat and 48 come from ELF symbol, <empty> and 0 are from variabel__xxx(). And that happens for quite a few variables, actually.

With those fixes, my kernel can be successfully BTF-generated. The number of per-cpu variables jump from 213 with all this filtering to 287 proper variables while using elf_sym__xxx() functions.

Quoted 70 lines
Quoted 5 lines
btf_encoder.c +4 -0
+                       if (verbose)
+                               printf("invalid variable name '%s'.\n", name);
+                       continue;
+               }

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 387

Quoted 10 lines
btf_encoder.c +10 -0
+
+               /*
+                * Validate the variable's type, the same check is also
+                * performed in bpf verifier.
+                */
+               type = var->ip.tag.type + type_id_off;
+               if (type == 0) {
+                       if (verbose)
+                               printf("invalid type for variable '%s'.\n", name);
+                       continue;

Again, there might be cases which we want to ignore, but we need to understand what and why those cases are?

btf_encoder.c int cu__encode_btf(struct cu *cu, int verbose) · line 400

Quoted 13 lines
btf_encoder.c +13 -0
+               }
+
+               /*
+                * Validate the size of the variable's type, the same check is
+                * also performed in bpf verifier.
+                */
+               size = variable__type_size(var, cu);
+               if (size == 0) {
+                       if (verbose)
+                               printf("invalid type size for variable '%s'.\n", name);
+                       continue;
+               }
+

The goal here is not to satisfy the verifier, but to generate BTF from DWARF faithfully. Let's not just ignore "inconvenient" cases and instead investigate why such cases happen.

Quoted 31 lines
btf_encoder.c +28 -0
+               if (verbose)
+                       printf("symbol '%s' of address 0x%lx encoded\n",
+                              elf_sym__name(&sym, btfe->symtab), addr);
+
+               /* add a BTF_KIND_VAR in btfe->types */
+               linkage = var->external ? BTF_VAR_GLOBAL_ALLOCATED : BTF_VAR_STATIC;
+               btf_var_id = btf_elf__add_var_type(btfe, type, var->name, linkage);
+               if (btf_var_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode variable '%s'\n", name);
+                       break;
+               }
+
+               /*
+                * add a BTF_VAR_SECINFO in btfe->percpu_secinfo, which will be added into
+                * btfe->types later when we add BTF_VAR_DATASEC.
+                */
+               type = btf_var_id;
+               offset = addr - btfe->percpu_base_addr;
+               btf_var_secinfo_id = btf_elf__add_var_secinfo(&btfe->percpu_secinfo,
+                                                             type, offset, size);
+               if (btf_var_secinfo_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode var secinfo '%s'\n", name);
+                       break;
+               }
+       }
+
 out:
        if (err)
                btf_elf__delete(btfe);

[...]

Signature · 1 line
- Arnaldo

A message that is not in the archive (sent off-list or not yet received): [email protected]

author
#15 lore raw

Some update on my side.

I am sending a v5 based on Andrii's suggestion on using elf_sym__name and elf_sym__size. It seems my local build tool has encoded all the symbols into DWARF, therefore, I don't see those "(anon)" variables in either v4 or v5, as seen by Andrii and Arnaldo. But I did come up with a potential concern about the variable's type while writing v5. Right now, the btf_id of the variable's type is retrieved as

type = var->ip.tag.type + type_id_off;

If I understand correctly, this is obtained from DWARF. Since symtab doesn't store the symbol's type (please correct me if I'm wrong), even if we can get the variable's name and size using elf_sym__name and elf_sym__size, we have no way to get the type. I am afraid we may get an incorrect type id using var->ip.tag.type.

So, Andrii, could you please verify whether the encoded VAR's types are still correct for those 'anon' variables? I can't verify as my build tool doesn't generate such variables. If the encoded type ids are still correct, then using sym does do a better job than using DWARF.

I'm also leaving the basic validations on variable's name and size in v5. I think they are needed. It's better having less variables than a malformed btf if things go awry.

Hao

Quoted 158 lines · Arnaldo Carvalho de Melo

Em Mon, Jun 29, 2020 at 01:20:02PM -0700, Andrii Nakryiko escreveu:

Quoted 21 lines · Andrii Nakryiko
Quoted 15 lines · Hao Luo

Review

Quoted 12 lines
(quoted diff) +12 -0
+               if (!addr)
+                       continue;
+               var = hashaddr__find_variable(hash_addr, addr);
+               if (var == NULL)
+                       continue;
+
+               /*
+                * Valididate the variable's name, the same check is also
+                * performed in bpf verifier.
+                */
+               name = variable__name(var, cu);
+               if (!var->name || !btf_name_valid(name, true)) {
Quoted 4 lines

It might be ok to filter them out, but I'd rather try to first understand why do we get those invalid names in the first place. Otherwise we might be just papering over the real bug in how we process symbols/variables in DWARF?

Quoted 7 lines

Ok, so I played with it a bit locally. Those variables with empty name or 0 size seem to be due to some bugs in variable__name() and variable__type_size() implementations. If you stick to ELF symbol's elf_sym__name() and elf_sym__size(), you'll get valid sizes.

Arnaldo, could you please take a look at why variable__xxx() impls don't work for some variables? I can repro it locally, e.g.:

I'm looking at it, I think this is that maybe not all variables that are in the elf symbol table are encoded in DWARF (maybe some declared in asm files?) and thus we will not find its name in the .debug_str.

ctf_loader.c, that was used as the model for btf_loader.c has this:

static const char *ctf__variable_name(const struct variable *var,
const struct cu *cu)

{
        struct ctf *ctf = cu->priv;

        return ctf->symtab->symstrs->d_buf + var->name;
}

struct debug_fmt_ops ctf__ops = {
.name = "ctf",
.function__name = ctf__function_name,
.load_file = ctf__load_file,
.variable__name = ctf__variable_name,
.strings__ptr = ctf__strings_ptr,
.cu__delete = ctf__cu_delete,
};

And then

const char *variable__name(const struct variable *var, const struct cu *cu)

{
        if (cu->dfops && cu->dfops->variable__name)
                return cu->dfops->variable__name(var, cu);
        return s(cu, var->name);
}

I.e. btf_loader.c and dwarf_loader.c are using the fallback return that assumes all strings are in a single table, the CTF one gets variable names from the ELF symtab, as you seem to be doing by using elf_sym__name() directly.

I'm reading Hao's patch to see how the variables are being created as they are loaded from DWARF, to check this hunch.

- Arnaldo

Quoted 4 lines

ELF SYM kstat == <empty>, SZ 48 == 0
invalid type size for variable '(null)'.
invalid variable name '(null)'.

Quoted 77 lines

kstat and 48 come from ELF symbol, <empty> and 0 are from variabel__xxx(). And that happens for quite a few variables, actually.

With those fixes, my kernel can be successfully BTF-generated. The number of per-cpu variables jump from 213 with all this filtering to 287 proper variables while using elf_sym__xxx() functions.

Quoted 70 lines
Quoted 5 lines
(quoted diff) +4 -0
+                       if (verbose)
+                               printf("invalid variable name '%s'.\n", name);
+                       continue;
+               }

Review

Quoted 10 lines
(quoted diff) +10 -0
+
+               /*
+                * Validate the variable's type, the same check is also
+                * performed in bpf verifier.
+                */
+               type = var->ip.tag.type + type_id_off;
+               if (type == 0) {
+                       if (verbose)
+                               printf("invalid type for variable '%s'.\n", name);
+                       continue;

Again, there might be cases which we want to ignore, but we need to understand what and why those cases are?

Review

Quoted 13 lines
(quoted diff) +13 -0
+               }
+
+               /*
+                * Validate the size of the variable's type, the same check is
+                * also performed in bpf verifier.
+                */
+               size = variable__type_size(var, cu);
+               if (size == 0) {
+                       if (verbose)
+                               printf("invalid type size for variable '%s'.\n", name);
+                       continue;
+               }
+

The goal here is not to satisfy the verifier, but to generate BTF from DWARF faithfully. Let's not just ignore "inconvenient" cases and instead investigate why such cases happen.

Quoted 31 lines
(quoted diff) +28 -0
+               if (verbose)
+                       printf("symbol '%s' of address 0x%lx encoded\n",
+                              elf_sym__name(&sym, btfe->symtab), addr);
+
+               /* add a BTF_KIND_VAR in btfe->types */
+               linkage = var->external ? BTF_VAR_GLOBAL_ALLOCATED : BTF_VAR_STATIC;
+               btf_var_id = btf_elf__add_var_type(btfe, type, var->name, linkage);
+               if (btf_var_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode variable '%s'\n", name);
+                       break;
+               }
+
+               /*
+                * add a BTF_VAR_SECINFO in btfe->percpu_secinfo, which will be added into
+                * btfe->types later when we add BTF_VAR_DATASEC.
+                */
+               type = btf_var_id;
+               offset = addr - btfe->percpu_base_addr;
+               btf_var_secinfo_id = btf_elf__add_var_secinfo(&btfe->percpu_secinfo,
+                                                             type, offset, size);
+               if (btf_var_secinfo_id < 0) {
+                       err = -1;
+                       printf("error: failed to encode var secinfo '%s'\n", name);
+                       break;
+               }
+       }
+
 out:
        if (err)
                btf_elf__delete(btfe);

[...]

--

- Arnaldo