Skip to contentSkip to conversation

bpftool: Allow to read btf as raw data

PATCH Applied Jiri Olsa 5 participants 8 messages Oct 24, 2019 – Oct 26, 2019

Subsystems (4)
lore
Share & tools
lore

Jiri Olsa

author reviewer
#0 lore raw

PATCH 1/1 bpftool: Allow to read btf as raw data

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

  $ bpftool btf dump file /sys/kernel/btf/vmlinux
  [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
  [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
  [3] CONST '(anon)' type_id=2

---
v3 changes:
- fix title

v2 changes:
- added is_btf_raw to find out which btf__parse_* function to call
- changed labels and error propagation in btf__parse_raw
- drop the err initialization, which is not needed under this change

 tools/bpf/bpftool/btf.c | 57 ++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 56 insertions(+), 1 deletion(-)
tools/bpf/bpftool/btf.c +56 -1
index 9a9376d1d3df..a7b8bf233cf5 100644
@@ -12,6 +12,9 @@
 #include <libbpf.h>
 #include <linux/btf.h>
 #include <linux/hashtable.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <unistd.h>
 
 #include "btf.h"
 #include "json_writer.h"
@@ -388,6 +391,54 @@ static int dump_btf_c(const struct btf *btf,
 	return err;
 }
 
+static struct btf *btf__parse_raw(const char *file)
+{
+	struct btf *btf;
+	struct stat st;
+	__u8 *buf;
+	FILE *f;
+
+	if (stat(file, &st))
+		return NULL;
+
+	f = fopen(file, "rb");
+	if (!f)
+		return NULL;
+
+	buf = malloc(st.st_size);
+	if (!buf) {
+		btf = ERR_PTR(-ENOMEM);
+		goto exit_close;
+	}
+
+	if ((size_t) st.st_size != fread(buf, 1, st.st_size, f)) {
+		btf = ERR_PTR(-EINVAL);
+		goto exit_free;
+	}
+
+	btf = btf__new(buf, st.st_size);
+
+exit_free:
+	free(buf);
+exit_close:
+	fclose(f);
+	return btf;
+}
+
+static bool is_btf_raw(const char *file)
+{
+	__u16 magic = 0;
+	int fd;
+
+	fd = open(file, O_RDONLY);
+	if (fd < 0)
+		return false;
+
+	read(fd, &magic, sizeof(magic));
+	close(fd);
+	return magic == BTF_MAGIC;
+}
+
 static int do_dump(int argc, char **argv)
 {
 	struct btf *btf = NULL;
@@ -465,7 +516,11 @@ static int do_dump(int argc, char **argv)
 		}
 		NEXT_ARG();
 	} else if (is_prefix(src, "file")) {
-		btf = btf__parse_elf(*argv, NULL);
+		if (is_btf_raw(*argv))
+			btf = btf__parse_raw(*argv);
+		else
+			btf = btf__parse_elf(*argv, NULL);
+
 		if (IS_ERR(btf)) {
 			err = PTR_ERR(btf);
 			btf = NULL;
Signature · 1 line
2.21.0
#1 lore raw
Quoted 9 lines · Jiri Olsa

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

  $ bpftool btf dump file /sys/kernel/btf/vmlinux
  [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
  [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
  [3] CONST '(anon)' type_id=2
Quoted 8 lines

---
v3 changes:
- fix title

v2 changes:
- added is_btf_raw to find out which btf__parse_* function to call
- changed labels and error propagation in btf__parse_raw
- drop the err initialization, which is not needed under this change

Aw, this is v3? Looks like I replied to the older now, such confusion :)

#2 lore raw
Quoted 14 lines · Jakub Kicinski
Quoted 13 lines · Jiri Olsa

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

  $ bpftool btf dump file /sys/kernel/btf/vmlinux
  [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
  [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
  [3] CONST '(anon)' type_id=2

---
v3 changes:
- fix title

Quoted 7 lines
Quoted 4 lines

v2 changes:
- added is_btf_raw to find out which btf__parse_* function to call
- changed labels and error propagation in btf__parse_raw
- drop the err initialization, which is not needed under this change

Aw, this is v3? Looks like I replied to the older now, such confusion :)

yea, that went well.. sry, missed the v3 in title

jirka

Acked-by on v1 1/1
#3 lore raw
In reply to Jiri Olsa
Quoted 8 lines · Jiri Olsa

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

   $ bpftool btf dump file /sys/kernel/btf/vmlinux
   [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
   [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
   [3] CONST '(anon)' type_id=2
Quoted 4 lines

---
v3 changes:
- fix title

The subject (by now) should be "[PATCH v3 bpf-next] ...". Include both version and intended tree (bpf-next usually).

Quoted 9 lines

v2 changes:
- added is_btf_raw to find out which btf__parse_* function to call
- changed labels and error propagation in btf__parse_raw
- drop the err initialization, which is not needed under this change

  tools/bpf/bpftool/btf.c | 57 ++++++++++++++++++++++++++++++++++++++++-
  1 file changed, 56 insertions(+), 1 deletion(-)

Overall looks good. It's kind of sad that we re-open that file twice, but we can optimize that later, if it ever is a problem. We should probably eventually implement btf__parse_raw as libbpf API with mmap2(). But perfect is the enemy of the good, so:

Quoted 5 lines
tools/bpf/bpftool/btf.c +0 -0
index 9a9376d1d3df..a7b8bf233cf5 100644
@@ -12,6 +12,9 @@

[...]

Acked-by on v1 1/1
#4 lore raw
In reply to Jiri Olsa
Quoted 9 lines · Jiri Olsa

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

  $ bpftool btf dump file /sys/kernel/btf/vmlinux
  [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
  [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
  [3] CONST '(anon)' type_id=2
#5 lore raw
Quoted 4 lines · Jakub Kicinski
Quoted 3 lines · Jiri Olsa

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

Quoted 8 lines
Quoted 6 lines
  $ bpftool btf dump file /sys/kernel/btf/vmlinux
  [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
  [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
  [3] CONST '(anon)' type_id=2

[root@ibm-z-107 bpftool]# ./bpftool btf dump file /sys/kernel/btf/vmlinux | head -3
[1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
[2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
[3] CONST '(anon)' type_id=2
[root@ibm-z-107 bpftool]# lscpu | grep Endian
Byte Order: Big Endian

thanks,
jirka

#6 lore raw
Quoted 12 lines · Jiri Olsa
Quoted 11 lines · Jakub Kicinski
Quoted 9 lines · Jiri Olsa

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

  $ bpftool btf dump file /sys/kernel/btf/vmlinux
  [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
  [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
  [3] CONST '(anon)' type_id=2
Quoted 8 lines
Quoted 1 line

[root@ibm-z-107 bpftool]# ./bpftool btf dump file /sys/kernel/btf/vmlinux | head -3
[1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
[2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
[3] CONST '(anon)' type_id=2
[root@ibm-z-107 bpftool]# lscpu | grep Endian
Byte Order: Big Endian

Thanks for checking! I thought the on-disk format is fixed like the ELF magic number. But if two bads make a right then all is good ☺️

maintainer Applied
#7 lore raw
In reply to Jiri Olsa
Quoted 9 lines · Jiri Olsa

The bpftool interface stays the same, but now it's possible to run it over BTF raw data, like:

  $ bpftool btf dump file /sys/kernel/btf/vmlinux
  [1] INT '(anon)' size=4 bits_offset=0 nr_bits=32 encoding=(none)
  [2] INT 'long unsigned int' size=8 bits_offset=0 nr_bits=64 encoding=(none)
  [3] CONST '(anon)' type_id=2

Applied, thanks!