Skip to contentSkip to conversation

perf annotate: Use libcapstone as a disasssembler

PATCH Superseded Namhyung Kim 3 participants 8 messages Mar 28, 2024 – Mar 29, 2024 perf events

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Namhyung Kim

author maintainer
#0 lore raw

cover perf annotate: Use libcapstone as a disasssembler

Hello,

As we've added libcapstone support, it's natural to use it for perf annotate as well. This change added the capstone support on x86 first. Other archs can be added later (by someone who can verify it doesn't break things).

For now it tries to use capstone (if available) before objdump. But it doesn't support source file and line number info. So users should use the objdump (by passing --objdump=PATH option) if they need them. For example, this command line will keep the existing behavior (i.e. using objdump).

  # not to use capstone for disassembly
  $ perf annotate --objdump=objdump

The capstone uses LLVM objdump style output which is slightly different than the GNU objdump. But it should not have differences besides that. I've verified the result of data type profiling and it produced the same output but gave me ~3x speedups.

Thanks,
Namhyung

Cc Changbin Du <[email protected]>

Namhyung Kim (4):
perf annotate: Use ins__is_xxx() if possible
perf annotate: Add and use ins__is_nop()
perf annotate: Split out util/disasm.c
perf annotate: Use libcapstone to disassemble

 tools/perf/util/Build      |    1 +
 tools/perf/util/annotate.c | 1711 ++---------------------------------
 tools/perf/util/annotate.h |   59 +-
 tools/perf/util/disasm.c   | 1739 ++++++++++++++++++++++++++++++++++++
 tools/perf/util/disasm.h   |  112 +++
 5 files changed, 1914 insertions(+), 1708 deletions(-)
 create mode 100644 tools/perf/util/disasm.c
 create mode 100644 tools/perf/util/disasm.h
Signature · 1 line
2.44.0.478.gd926399ef9-goog
author maintainer
#1 lore raw

PATCH 1/4 perf annotate: Use ins__is_xxx() if possible

This is to prepare separation of disasm related code. Use the public ins API instead of checking the internal data structure.

---
 tools/perf/util/annotate.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
tools/perf/util/annotate.c +4 -4
index 64e54ff1aa1d..986c499150ef 100644
@@ -3665,7 +3665,7 @@ int annotate_get_insn_location(struct arch *arch, struct disasm_line *dl,
 	struct annotated_op_loc *op_loc;
 	int i;
 
-	if (!strcmp(dl->ins.name, "lock"))
+	if (ins__is_lock(&dl->ins))
 		ops = dl->ops.locked.ops;
 	else
 		ops = &dl->ops;
@@ -3763,7 +3763,7 @@ static struct disasm_line *find_disasm_line(struct symbol *sym, u64 ip,
 			 * llvm-objdump places "lock" in a separate line and
 			 * in that case, we want to get the next line.
 			 */
-			if (!strcmp(dl->ins.name, "lock") &&
+			if (ins__is_lock(&dl->ins) &&
 			    *dl->ops.raw == '\0' && allow_update) {
 				ip++;
 				continue;
@@ -4093,10 +4093,10 @@ static bool process_basic_block(struct basic_block_data *bb_data,
 		if (dl == last_dl)
 			break;
 		/* 'return' instruction finishes the block */
-		if (dl->ins.ops == &ret_ops)
+		if (ins__is_ret(&dl->ins))
 			break;
 		/* normal instructions are part of the basic block */
-		if (dl->ins.ops != &jump_ops)
+		if (!ins__is_jump(&dl->ins))
 			continue;
 		/* jump to a different function, tail call or return */
 		if (dl->ops.target.outside)
Signature · 1 line
2.44.0.478.gd926399ef9-goog
author maintainer
#2 lore raw

PATCH 2/4 perf annotate: Add and use ins__is_nop()

In reply to Namhyung Kim

Likewise, add ins__is_nop() to check if the current instruction is NOP.

---
 tools/perf/util/annotate.c | 7 ++++++-
 tools/perf/util/annotate.h | 1 +
 2 files changed, 7 insertions(+), 1 deletion(-)
tools/perf/util/annotate.c +6 -1
index 986c499150ef..5d0ca004dcfb 100644
@@ -757,6 +757,11 @@ static struct ins_ops ret_ops = {
 	.scnprintf = ins__raw_scnprintf,
 };
 
+bool ins__is_nop(const struct ins *ins)
+{
+	return ins->ops == &nop_ops;
+}
+
 bool ins__is_ret(const struct ins *ins)
 {
 	return ins->ops == &ret_ops;
@@ -1785,7 +1790,7 @@ static void delete_last_nop(struct symbol *sym)
 		dl = list_entry(list->prev, struct disasm_line, al.node);
 
 		if (dl->ins.ops) {
-			if (dl->ins.ops != &nop_ops)
+			if (!ins__is_nop(&dl->ins))
 				return;
 		} else {
 			if (!strstr(dl->al.line, " nop ") &&
tools/perf/util/annotate.h +1 -0
index 14980b65f812..98f556af637c 100644
@@ -74,6 +74,7 @@ struct ins_ops {
 
 bool ins__is_jump(const struct ins *ins);
 bool ins__is_call(const struct ins *ins);
+bool ins__is_nop(const struct ins *ins);
 bool ins__is_ret(const struct ins *ins);
 bool ins__is_lock(const struct ins *ins);
 int ins__scnprintf(struct ins *ins, char *bf, size_t size, struct ins_operands *ops, int max_ins_name);
Signature · 1 line
2.44.0.478.gd926399ef9-goog
author maintainer
#3 lore raw

PATCH 3/4 perf annotate: Split out util/disasm.c

In reply to Namhyung Kim

The util/annotate.c code has both disassembly and sample annotation related codes. Factor out the disasm part so that it can be handled more easily.

---
 tools/perf/util/Build      |    1 +
 tools/perf/util/annotate.c | 1708 ++----------------------------------
 tools/perf/util/annotate.h |   60 +-
 tools/perf/util/disasm.c   | 1586 +++++++++++++++++++++++++++++++++
 tools/perf/util/disasm.h   |  112 +++
 5 files changed, 1757 insertions(+), 1710 deletions(-)
 create mode 100644 tools/perf/util/disasm.c
 create mode 100644 tools/perf/util/disasm.h
tools/perf/util/Build +1 -0
index e0a723e24503..aec5a590e349 100644
@@ -12,6 +12,7 @@ perf-y += config.o
 perf-y += copyfile.o
 perf-y += ctype.o
 perf-y += db-export.o
+perf-y += disasm.o
 perf-y += env.o
 perf-y += event.o
 perf-y += evlist.o
tools/perf/util/annotate.c +55 -1653

1,761 lines not on this page · raw message

tools/perf/util/annotate.h +3 -57

93 lines not on this page · raw message

tools/perf/util/disasm.c new file +1586 -0

1,139 lines not on this page · raw message

tools/perf/util/disasm.h new file +112 -0

112 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.44.0.478.gd926399ef9-goog

Part of the diff is not shown here. View the raw message.

author maintainer
#4 lore raw

PATCH 4/4 perf annotate: Use libcapstone to disassemble

In reply to Namhyung Kim

Now it can use the capstone library to disassemble the instructions. Let's use that (if available) for perf annotate to speed up. Currently it only supports x86 architecture. With this change I can see ~3x speed up in data type profiling.

But note that capstone cannot give the source file and line number info. For now, users should use the external objdump for that by specifying the --objdump option explicitly.

---
 tools/perf/util/disasm.c | 153 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)
tools/perf/util/disasm.c +153 -0
index 59ac37723990..c58ea6d822ed 100644
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0-only
 #include <ctype.h>
 #include <errno.h>
+#include <fcntl.h>
 #include <inttypes.h>
 #include <libgen.h>
 #include <regex.h>
@@ -18,6 +19,7 @@
 #include "evsel.h"
 #include "map.h"
 #include "maps.h"
+#include "namespaces.h"
 #include "srcline.h"
 #include "symbol.h"
 
@@ -1341,6 +1343,151 @@ symbol__disassemble_bpf_image(struct symbol *sym,
 	return 0;
 }
 
+#ifdef HAVE_LIBCAPSTONE_SUPPORT
+#include <capstone/capstone.h>
+
+static int open_capstone_handle(struct annotate_args *args, bool is_64bit,
+				csh *handle)
+{
+	struct annotation_options *opt = args->options;
+	cs_mode mode = is_64bit ? CS_MODE_64 : CS_MODE_32;
+
+	/* TODO: support more architectures */
+	if (!arch__is(args->arch, "x86"))
+		return -1;
+
+	if (cs_open(CS_ARCH_X86, mode, handle) != CS_ERR_OK)
+		return -1;
+
+	if (!opt->disassembler_style ||
+	    !strcmp(opt->disassembler_style, "att"))
+		cs_option(*handle, CS_OPT_SYNTAX, CS_OPT_SYNTAX_ATT);
+
+	/*
+	 * Resolving address operands to symbols is implemented
+	 * on x86 by investigating instruction details.
+	 */
+	cs_option(*handle, CS_OPT_DETAIL, CS_OPT_ON);
+
+	return 0;
+}
+
+struct find_file_offset_data {
+	u64 ip;
+	u64 offset;
+};
+
+/* This will be called for each PHDR in an ELF binary */
+static int find_file_offset(u64 start, u64 len, u64 pgoff, void *arg)
+{
+	struct find_file_offset_data *data = arg;
+
+	if (start <= data->ip && data->ip < start + len) {
+		data->offset = pgoff + data->ip - start;
+		return 1;
+	}
+	return 0;
+}
+
+static int symbol__disassemble_capstone(char *filename, struct symbol *sym,
+					struct annotate_args *args)
+{
+	struct annotation *notes = symbol__annotation(sym);
+	struct map *map = args->ms.map;
+	struct dso *dso = map__dso(map);
+	struct nscookie nsc;
+	u64 start = map__rip_2objdump(map, sym->start);
+	u64 end = map__rip_2objdump(map, sym->end);
+	u64 len = end - start;
+	u64 offset;
+	int i, fd, count;
+	bool is_64bit = false;
+	bool needs_cs_close = false;
+	u8 *buf = NULL;
+	struct find_file_offset_data data = {
+		.ip = start,
+	};
+	csh handle;
+	cs_insn *insn;
+
+	if (args->options->objdump_path)
+		return -1;
+
+	nsinfo__mountns_enter(dso->nsinfo, &nsc);
+	fd = open(filename, O_RDONLY);
+	nsinfo__mountns_exit(&nsc);
+	if (fd < 0)
+		return -1;
+
+	if (file__read_maps(fd, /*exe=*/true, find_file_offset, &data,
+			    &is_64bit) == 0)
+		goto err;
+
+	if (open_capstone_handle(args, is_64bit, &handle) < 0)
+		goto err;
+
+	needs_cs_close = true;
+
+	buf = malloc(len);
+	if (buf == NULL)
+		goto err;
+
+	count = pread(fd, buf, len, data.offset);
+	close(fd);
+	fd = -1;
+
+	if ((u64)count != len)
+		goto err;
+
+	count = cs_disasm(handle, buf, len, start, len, &insn);
+	for (i = 0, offset = 0; i < count; i++) {
+		char disasm_buf[256];
+		struct disasm_line *dl;
+
+		scnprintf(disasm_buf, sizeof(disasm_buf), "%s %s",
+			  insn[i].mnemonic, insn[i].op_str);
+
+		args->offset = offset;
+		args->line = disasm_buf;
+		args->line_nr = 0;
+		args->fileloc = NULL;
+		args->ms.sym  = sym;
+
+		dl = disasm_line__new(args);
+		if (dl == NULL)
+			goto err;
+
+		annotation_line__add(&dl->al, &notes->src->source);
+
+		offset += insn[i].size;
+	}
+
+out:
+	if (needs_cs_close)
+		cs_close(&handle);
+	free(buf);
+	return count < 0 ? count : 0;
+
+err:
+	if (fd >= 0)
+		close(fd);
+	if (needs_cs_close) {
+		struct disasm_line *dl, *tmp;
+
+		/*
+		 * It probably failed in the middle of the above loop.
+		 * Release any resources it might add.
+		 */
+		list_for_each_entry_safe(dl, tmp, &notes->src->source, al.node) {
+			list_del(&dl->al.node);
+			free(dl);
+		}
+	}
+	count = -1;
+	goto out;
+}
+#endif
+
 /*
  * Possibly create a new version of line with tabs expanded. Returns the
  * existing or new line, storage is updated if a new line is allocated. If
@@ -1463,6 +1610,12 @@ int symbol__disassemble(struct symbol *sym, struct annotate_args *args)
 		strcpy(symfs_filename, tmp);
 	}
 
+#ifdef HAVE_LIBCAPSTONE_SUPPORT
+	err = symbol__disassemble_capstone(symfs_filename, sym, args);
+	if (err == 0)
+		goto out_remove_tmp;
+#endif
+
 	err = asprintf(&command,
 		 "%s %s%s --start-address=0x%016" PRIx64
 		 " --stop-address=0x%016" PRIx64
Signature · 1 line
2.44.0.478.gd926399ef9-goog
reviewer Tested-by on the v1 series
#5 lore raw
In reply to Namhyung Kim
Quoted 15 lines · Namhyung Kim

Hello,

As we've added libcapstone support, it's natural to use it for perf annotate as well. This change added the capstone support on x86 first. Other archs can be added later (by someone who can verify it doesn't break things).

For now it tries to use capstone (if available) before objdump. But it doesn't support source file and line number info. So users should use the objdump (by passing --objdump=PATH option) if they need them. For example, this command line will keep the existing behavior (i.e. using objdump).

  # not to use capstone for disassembly
  $ perf annotate --objdump=objdump
Quoted 7 lines

The capstone uses LLVM objdump style output which is slightly different than the GNU objdump. But it should not have differences besides that. I've verified the result of data type profiling and it produced the same output but gave me ~3x speedups.

Thanks,
Namhyung

I wanted to see what -fsanitize=address would think of libcaptstone and no issues, so:

Some thoughts:
- it may be worth adding to the commit message in "perf annotate:
Split out util/disasm.c" that there's no functional change,
- we lack perf annotate testing,
- it seems capstone should be opt-out rather than opt-in as a library
dependency (given the performance delta and the objdump interactions are at best messy),
- if libcapstone could solve our addr2line issues too this would be great :-).

Thanks,
Ian

Quoted 21 lines
Cc Changbin Du <[email protected]>

Namhyung Kim (4):
perf annotate: Use ins__is_xxx() if possible
perf annotate: Add and use ins__is_nop()
perf annotate: Split out util/disasm.c
perf annotate: Use libcapstone to disassemble

 tools/perf/util/Build      |    1 +
 tools/perf/util/annotate.c | 1711 ++---------------------------------
 tools/perf/util/annotate.h |   59 +-
 tools/perf/util/disasm.c   | 1739 ++++++++++++++++++++++++++++++++++++
 tools/perf/util/disasm.h   |  112 +++
 5 files changed, 1914 insertions(+), 1708 deletions(-)
 create mode 100644 tools/perf/util/disasm.c
 create mode 100644 tools/perf/util/disasm.h

--
2.44.0.478.gd926399ef9-goog

author maintainer
#6 lore raw

Hi Ian,

Quoted 22 lines · Ian Rogers
Quoted 20 lines · Namhyung Kim

Hello,

As we've added libcapstone support, it's natural to use it for perf annotate as well. This change added the capstone support on x86 first. Other archs can be added later (by someone who can verify it doesn't break things).

For now it tries to use capstone (if available) before objdump. But it doesn't support source file and line number info. So users should use the objdump (by passing --objdump=PATH option) if they need them. For example, this command line will keep the existing behavior (i.e. using objdump).

  # not to use capstone for disassembly
  $ perf annotate --objdump=objdump

The capstone uses LLVM objdump style output which is slightly different than the GNU objdump. But it should not have differences besides that. I've verified the result of data type profiling and it produced the same output but gave me ~3x speedups.

Quoted 7 lines
Quoted 2 lines

Thanks,
Namhyung

I wanted to see what -fsanitize=address would think of libcaptstone and no issues, so:

Thanks for checking it!

Quoted 9 lines

Some thoughts:
- it may be worth adding to the commit message in "perf annotate:
Split out util/disasm.c" that there's no functional change,
- we lack perf annotate testing,
- it seems capstone should be opt-out rather than opt-in as a library
dependency (given the performance delta and the objdump interactions are at best messy),
- if libcapstone could solve our addr2line issues too this would be great :-).

I think addr2line should be handled using libdw. I'll take a look.

Thanks,
Namhyung

#7 lore raw
In reply to Namhyung Kim
Quoted 11 lines · Namhyung Kim

Hello,

As we've added libcapstone support, it's natural to use it for perf annotate as well. This change added the capstone support on x86 first. Other archs can be added later (by someone who can verify it doesn't break things).

For now it tries to use capstone (if available) before objdump. But it doesn't support source file and line number info. So users should use the objdump (by passing --objdump=PATH option) if they need them. For example, this command line will keep the existing behavior (i.e. using objdump).

Quoted 7 lines
  # not to use capstone for disassembly
  $ perf annotate --objdump=objdump

The capstone uses LLVM objdump style output which is slightly different than the GNU objdump. But it should not have differences besides that. I've verified the result of data type profiling and it produced the same output but gave me ~3x speedups.

Excellent news, I'll try and test all this soon!

- Arnaldo