Skip to contentSkip to conversation

More dwarf support in python interface

PATCH RESEND In review Andi Kleen 2 participants 12 messages Sep 5, 2024 perf events

lore
Share & tools
lore

Andi Kleen

author
#0 lore raw

PATCH RESEND 1/1 More dwarf support in python interface

[resend because I typoed the mailing list address in the first try. Apologies if you see it twice]

This patch kit adds better support for resolving DWARF from perf script python scripts:

- Add new perf_brstack_srcline / perf_ip_srcline functions to resolve full brstacks or individual IPs.
- Support the DWARF discriminator in addition to the line number
- Report the inline stack to python
- Report the build-id and executable name.

Most of the patch is just plumbing to pass all this information around.

The first patch is a bug fix and can be applied independently.

-andi

author
#1 lore raw

PATCH 1/10 perf: Avoid buffer overflow in python register interface

Running a script that processes PEBS records gives buffer overflows in valgrind. The problem is that the allocation of the register string doesn't include the terminating 0 byte. Fix this. I also replaced the very magic "28" with a more reasonable larger buffer that should fit all registers. There's no need to conserve memory here.

==2106591== Memcheck, a memory error detector
==2106591== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al. ==2106591== Using Valgrind-3.22.0 and LibVEX; rerun with -h for copyright info ==2106591== Command: ../perf script -i tcall.data gcov.py tcall.gcov ==2106591==
==2106591== Invalid write of size 1

==2106591==    at 0x713354: regs_map (trace-event-python.c:748)
==2106591==    by 0x7134EB: set_regs_in_dict (trace-event-python.c:784)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==    by 0x630E7A: __ordered_events__flush (ordered-events.c:324)
==2106591==  Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd
==2106591==    at 0x484280F: malloc (vg_replace_malloc.c:442)
==2106591==    by 0x7134AD: set_regs_in_dict (trace-event-python.c:780)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==    by 0x630E7A: __ordered_events__flush (ordered-events.c:324)
==2106591==
==2106591== Invalid read of size 1
==2106591==    at 0x484B6C6: strlen (vg_replace_strmem.c:502)
==2106591==    by 0x555D494: PyUnicode_FromString (unicodeobject.c:1899)
==2106591==    by 0x7134F7: set_regs_in_dict (trace-event-python.c:786)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==  Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd
==2106591==    at 0x484280F: malloc (vg_replace_malloc.c:442)
==2106591==    by 0x7134AD: set_regs_in_dict (trace-event-python.c:780)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==    by 0x630E7A: __ordered_events__flush (ordered-events.c:324)
==2106591==
==2106591== Invalid write of size 1
==2106591==    at 0x713354: regs_map (trace-event-python.c:748)
==2106591==    by 0x713539: set_regs_in_dict (trace-event-python.c:789)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==    by 0x630E7A: __ordered_events__flush (ordered-events.c:324)
==2106591==  Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd
==2106591==    at 0x484280F: malloc (vg_replace_malloc.c:442)
==2106591==    by 0x7134AD: set_regs_in_dict (trace-event-python.c:780)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==    by 0x630E7A: __ordered_events__flush (ordered-events.c:324)
==2106591==
==2106591== Invalid read of size 1
==2106591==    at 0x484B6C6: strlen (vg_replace_strmem.c:502)
==2106591==    by 0x555D494: PyUnicode_FromString (unicodeobject.c:1899)
==2106591==    by 0x713545: set_regs_in_dict (trace-event-python.c:791)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==  Address 0x7186fe0 is 0 bytes after a block of size 0 alloc'd
==2106591==    at 0x484280F: malloc (vg_replace_malloc.c:442)
==2106591==    by 0x7134AD: set_regs_in_dict (trace-event-python.c:780)
==2106591==    by 0x713E58: get_perf_sample_dict (trace-event-python.c:940)
==2106591==    by 0x716327: python_process_general_event (trace-event-python.c:1499)
==2106591==    by 0x7164E1: python_process_event (trace-event-python.c:1531)
==2106591==    by 0x44F9AF: process_sample_event (builtin-script.c:2549)
==2106591==    by 0x6294DC: evlist__deliver_sample (session.c:1534)
==2106591==    by 0x6296D0: machines__deliver_event (session.c:1573)
==2106591==    by 0x629C39: perf_session__deliver_event (session.c:1655)
==2106591==    by 0x625830: ordered_events__deliver_event (session.c:193)
==2106591==    by 0x630B23: do_flush (ordered-events.c:245)
==2106591==    by 0x630E7A: __ordered_events__flush (ordered-events.c:324)

==2106591==
73056 total, 29 ignored

---
 .../perf/util/scripting-engines/trace-event-python.c  | 11 +++--------
 1 file changed, 3 insertions(+), 8 deletions(-)
tools/perf/util/scripting-engines/trace-event-python.c +3 -8
index 6971dd6c231f..d7183134b669 100644
@@ -762,6 +762,8 @@ static void regs_map(struct regs_dump *regs, uint64_t mask, const char *arch, ch
 	}
 }
 
+#define MAX_REG_SIZE 128
+
 static int set_regs_in_dict(PyObject *dict,
 			     struct perf_sample *sample,
 			     struct evsel *evsel)
@@ -769,14 +771,7 @@ static int set_regs_in_dict(PyObject *dict,
 	struct perf_event_attr *attr = &evsel->core.attr;
 	const char *arch = perf_env__arch(evsel__env(evsel));
 
-	/*
-	 * Here value 28 is a constant size which can be used to print
-	 * one register value and its corresponds to:
-	 * 16 chars is to specify 64 bit register in hexadecimal.
-	 * 2 chars is for appending "0x" to the hexadecimal value and
-	 * 10 chars is for register name.
-	 */
-	int size = __sw_hweight64(attr->sample_regs_intr) * 28;
+	int size = (__sw_hweight64(attr->sample_regs_intr) * MAX_REG_SIZE) + 1;
 	char *bf = malloc(size);
 	if (!bf)
 		return -1;
Signature · 1 line
2.45.2
author
#2 lore raw

PATCH 2/10 perf: Support discriminator in addr2line

In reply to Andi Kleen

Dwarf has the concept of a discriminator to distinguish multiple compiler generated statements in a line. Add support for retrieving the discriminator in addr2line. Will be used in later patches to pass to python scripts.

---
 tools/perf/builtin-script.c                   |  6 ++-
 .../scripts/python/Perf-Trace-Util/Context.c  |  4 +-
 tools/perf/util/dlfilter.c                    |  4 +-
 tools/perf/util/srcline.c                     | 42 ++++++++++++-------
 tools/perf/util/srcline.h                     |  2 +-
 5 files changed, 37 insertions(+), 21 deletions(-)
tools/perf/builtin-script.c +4 -2
index 206b08426555..dc7e5406dae9 100644
@@ -1131,7 +1131,7 @@ static int map__fprintf_srccode(struct map *map, u64 addr, FILE *fp, struct srcc
 {
 	char *srcfile;
 	int ret = 0;
-	unsigned line;
+	unsigned line, disc;
 	int len;
 	char *srccode;
 	struct dso *dso;
@@ -1140,7 +1140,7 @@ static int map__fprintf_srccode(struct map *map, u64 addr, FILE *fp, struct srcc
 		return 0;
 	srcfile = get_srcline_split(dso,
 				    map__rip_2objdump(map, addr),
-				    &line);
+				    &line, &disc);
 	if (!srcfile)
 		return 0;
 
@@ -1157,6 +1157,8 @@ static int map__fprintf_srccode(struct map *map, u64 addr, FILE *fp, struct srcc
 	if (!srccode)
 		goto out_free_line;
 
+	/* Print discriminator too? Maybe with a new option */
+
 	ret = fprintf(fp, "|%-8d %.*s", line, len, srccode);
 
 	if (state) {
tools/perf/scripts/python/Perf-Trace-Util/Context.c +2 -2
index 3954bd1587ce..3aa5e6385c1e 100644
@@ -140,7 +140,7 @@ static PyObject *perf_set_itrace_options(PyObject *obj, PyObject *args)
 static PyObject *perf_sample_src(PyObject *obj, PyObject *args, bool get_srccode)
 {
 	struct scripting_context *c = get_scripting_context(args);
-	unsigned int line = 0;
+	unsigned int line = 0, disc = 0;
 	char *srcfile = NULL;
 	char *srccode = NULL;
 	PyObject *result;
@@ -157,7 +157,7 @@ static PyObject *perf_sample_src(PyObject *obj, PyObject *args, bool get_srccode
 	dso = map ? map__dso(map) : NULL;
 
 	if (dso)
-		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line);
+		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line, &disc);
 
 	if (get_srccode) {
 		if (srcfile)
tools/perf/util/dlfilter.c +2 -2
index 7d180bdaedbc..1a3b5ba92313 100644
@@ -250,7 +250,7 @@ static const char *dlfilter__srcline(void *ctx, __u32 *line_no)
 {
 	struct dlfilter *d = (struct dlfilter *)ctx;
 	struct addr_location *al;
-	unsigned int line = 0;
+	unsigned int line = 0, disc = 0;
 	char *srcfile = NULL;
 	struct map *map;
 	struct dso *dso;
@@ -268,7 +268,7 @@ static const char *dlfilter__srcline(void *ctx, __u32 *line_no)
 	dso = map ? map__dso(map) : NULL;
 
 	if (dso)
-		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line);
+		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line, &disc);
 
 	*line_no = line;
 	return srcfile;
tools/perf/util/srcline.c +28 -14

132 lines not on this page · raw message

tools/perf/util/srcline.h +1 -1
index 75010d39ea28..d2cb90b1177e 100644
@@ -17,7 +17,7 @@ char *__get_srcline(struct dso *dso, u64 addr, struct symbol *sym,
 		  bool show_sym, bool show_addr, bool unwind_inlines,
 		  u64 ip);
 void zfree_srcline(char **srcline);
-char *get_srcline_split(struct dso *dso, u64 addr, unsigned *line);
+char *get_srcline_split(struct dso *dso, u64 addr, unsigned *line, unsigned *disc);
 
 /* insert the srcline into the DSO, which will take ownership */
 void srcline__tree_insert(struct rb_root_cached *tree, u64 addr, char *srcline);
Signature · 1 line
2.45.2
author
#3 lore raw

PATCH 3/10 perf: Plumb passing machine to scripts

In reply to Andi Kleen

Future patches requires the machine pointer in the script events callback. Add the plumbing to pass (and ignore) machine everywhere.

---
 tools/perf/builtin-script.c                            | 2 +-
 tools/perf/util/scripting-engines/trace-event-perl.c   | 5 +++--
 tools/perf/util/scripting-engines/trace-event-python.c | 6 ++++--
 tools/perf/util/trace-event-scripting.c                | 7 +++++--
 tools/perf/util/trace-event.h                          | 7 +++++--
 5 files changed, 18 insertions(+), 9 deletions(-)
tools/perf/builtin-script.c +1 -1
index dc7e5406dae9..c12beceae8b3 100644
@@ -2528,7 +2528,7 @@ static int process_sample_event(const struct perf_tool *tool,
 				thread__resolve(al.thread, &addr_al, sample);
 			addr_al_ptr = &addr_al;
 		}
-		scripting_ops->process_event(event, sample, evsel, &al, addr_al_ptr);
+		scripting_ops->process_event(event, sample, evsel, &al, addr_al_ptr, machine);
 	} else {
 		process_event(scr, sample, evsel, &al, &addr_al, machine);
 	}
tools/perf/util/scripting-engines/trace-event-perl.c +3 -2
index e16257d5ab2c..7b901bf1bdad 100644
@@ -462,9 +462,10 @@ static void perl_process_event(union perf_event *event,
 			       struct perf_sample *sample,
 			       struct evsel *evsel,
 			       struct addr_location *al,
-			       struct addr_location *addr_al)
+			       struct addr_location *addr_al,
+			       struct machine *machine)
 {
-	scripting_context__update(scripting_context, event, sample, evsel, al, addr_al);
+	scripting_context__update(scripting_context, event, sample, evsel, al, addr_al, machine);
 	perl_process_tracepoint(sample, evsel, al);
 	perl_process_event_generic(event, sample, evsel);
 }
tools/perf/util/scripting-engines/trace-event-python.c +4 -2
index d7183134b669..d0ea8bbe962a 100644
@@ -1509,11 +1509,13 @@ static void python_process_event(union perf_event *event,
 				 struct perf_sample *sample,
 				 struct evsel *evsel,
 				 struct addr_location *al,
-				 struct addr_location *addr_al)
+				 struct addr_location *addr_al,
+				 struct machine *machine)
 {
 	struct tables *tables = &tables_global;
 
-	scripting_context__update(scripting_context, event, sample, evsel, al, addr_al);
+	scripting_context__update(scripting_context, event, sample, evsel, al, addr_al,
+				  machine);
 
 	switch (evsel->core.attr.type) {
 	case PERF_TYPE_TRACEPOINT:
tools/perf/util/trace-event-scripting.c +5 -2
index bd0000300c77..79ac0424b43e 100644
@@ -26,7 +26,8 @@ void scripting_context__update(struct scripting_context *c,
 			       struct perf_sample *sample,
 			       struct evsel *evsel,
 			       struct addr_location *al,
-			       struct addr_location *addr_al)
+			       struct addr_location *addr_al,
+			       struct machine *machine)
 {
 	c->event_data = sample->raw_data;
 	c->pevent = NULL;
@@ -39,6 +40,7 @@ void scripting_context__update(struct scripting_context *c,
 	c->evsel = evsel;
 	c->al = al;
 	c->addr_al = addr_al;
+	c->machine = machine;
 }
 
 static int flush_script_unsupported(void)
@@ -55,7 +57,8 @@ static void process_event_unsupported(union perf_event *event __maybe_unused,
 				      struct perf_sample *sample __maybe_unused,
 				      struct evsel *evsel __maybe_unused,
 				      struct addr_location *al __maybe_unused,
-				      struct addr_location *addr_al __maybe_unused)
+				      struct addr_location *addr_al __maybe_unused,
+				      struct machine *machine __maybe_unused)
 {
 }
 
tools/perf/util/trace-event.h +5 -2
index bbf8b26bc8da..c8414c7421ad 100644
@@ -99,7 +99,8 @@ struct scripting_ops {
 			       struct perf_sample *sample,
 			       struct evsel *evsel,
 			       struct addr_location *al,
-			       struct addr_location *addr_al);
+			       struct addr_location *addr_al,
+			       struct machine *machine);
 	void (*process_switch)(union perf_event *event,
 			       struct perf_sample *sample,
 			       struct machine *machine);
@@ -133,6 +134,7 @@ struct scripting_context {
 	struct addr_location *al;
 	struct addr_location *addr_al;
 	struct perf_session *session;
+	struct machine *machine;
 };
 
 void scripting_context__update(struct scripting_context *scripting_context,
@@ -140,7 +142,8 @@ void scripting_context__update(struct scripting_context *scripting_context,
 			       struct perf_sample *sample,
 			       struct evsel *evsel,
 			       struct addr_location *al,
-			       struct addr_location *addr_al);
+			       struct addr_location *addr_al,
+			       struct machine *machine);
 
 int common_pc(struct scripting_context *context);
 int common_flags(struct scripting_context *context);
Signature · 1 line
2.45.2
author
#4 lore raw

PATCH 4/10 perf: Add perf_brstack_srcline to resolve brstack entries

In reply to Andi Kleen

Add a new perf_brstack_srcline python function that resolves a full brstack to filename/line/line/discriminator for each entry.

---
 .../scripts/python/Perf-Trace-Util/Context.c  | 58 +++++++++++++++++++
 1 file changed, 58 insertions(+)
tools/perf/scripts/python/Perf-Trace-Util/Context.c +58 -0
index 3aa5e6385c1e..a95a27895a15 100644
@@ -22,6 +22,7 @@
 #include "../../../util/session.h"
 #include "../../../util/srcline.h"
 #include "../../../util/srccode.h"
+#include "../../../util/dso.h"
 
 #if PY_MAJOR_VERSION < 3
 #define _PyCapsule_GetPointer(arg1, arg2) \
@@ -182,6 +183,61 @@ static PyObject *perf_sample_srccode(PyObject *obj, PyObject *args)
 	return perf_sample_src(obj, args, true);
 }
 
+static PyObject *do_perf_brstack_srcline(struct scripting_context *c, PyObject *ipp)
+{
+	struct addr_location al;
+	struct dso *dso;
+	u8 cpumode;
+	char *srcfile = NULL;
+	PyObject *result = NULL;
+	bool kernel;
+	u64 ip = PyLong_AsUnsignedLongLong(ipp);
+	unsigned line = 0, disc = 0;
+
+	kernel = machine__kernel_ip(c->machine, ip);
+	if (kernel)
+		cpumode = PERF_RECORD_MISC_KERNEL;
+	else
+		cpumode = PERF_RECORD_MISC_USER;
+
+	addr_location__init(&al);
+	if (machine__resolve(c->machine, &al, c->sample) < 0)
+		goto err2;
+	if (!thread__find_map(al.thread, cpumode, ip, &al))
+		goto err;
+	if (!al.map)
+		goto err;
+	dso = map__dso(al.map);
+	if (!dso)
+		goto err;
+	if (dso__data(dso)->status == DSO_DATA_STATUS_ERROR)
+		goto err;
+	srcfile = get_srcline_split(dso, map__rip_2objdump(al.map, al.addr), &line, &disc);
+	if (srcfile)
+		result = Py_BuildValue("(sII)", srcfile, line, disc);
+	free(srcfile);
+err:
+	addr_location__exit(&al);
+err2:
+	if (!result)
+		return Py_BuildValue("");
+	return result;
+}
+
+/* Resolve srcfile/line/disc for brstack from/to. */
+
+static PyObject *perf_brstack_srcline(PyObject *obj, PyObject *args)
+{
+	PyObject *brstack = NULL;
+	PyObject *from, *to;
+	struct scripting_context *c = get_args(args, "brstack", &brstack);
+	if (!c)
+		return NULL;
+	from = do_perf_brstack_srcline(c, PyDict_GetItemString(brstack, "from"));
+	to = do_perf_brstack_srcline(c, PyDict_GetItemString(brstack, "to"));
+	return Py_BuildValue("(OO)", from, to);
+}
+
 static PyMethodDef ContextMethods[] = {
 #ifdef HAVE_LIBTRACEEVENT
 	{ "common_pc", perf_trace_context_common_pc, METH_VARARGS,
@@ -199,6 +255,8 @@ static PyMethodDef ContextMethods[] = {
 	  METH_VARARGS,	"Get source file name and line number."},
 	{ "perf_sample_srccode", perf_sample_srccode,
 	  METH_VARARGS,	"Get source file name, line number and line."},
+	{ "perf_brstack_srcline", perf_brstack_srcline,
+	  METH_VARARGS, "Get source file name, line number, discriminator for from/to of a brstack entry." },
 	{ NULL, NULL, 0, NULL}
 };
 
Signature · 1 line
2.45.2
author
#5 lore raw

PATCH 5/10 perf: Add perf_resolve_ip python interface

In reply to Andi Kleen

Add a perf_resolve_ip python interface to resolve filename/line/disc for specific addresses. I had a case where a script needs to resolve this for the base symbol of a sample. It reuses the same machinery as the earlier added perf_brstack_srcline.

---
 .../scripts/python/Perf-Trace-Util/Context.c  | 19 ++++++++++++++++---
 1 file changed, 16 insertions(+), 3 deletions(-)
tools/perf/scripts/python/Perf-Trace-Util/Context.c +16 -3
index a95a27895a15..417466231f71 100644
@@ -183,7 +183,7 @@ static PyObject *perf_sample_srccode(PyObject *obj, PyObject *args)
 	return perf_sample_src(obj, args, true);
 }
 
-static PyObject *do_perf_brstack_srcline(struct scripting_context *c, PyObject *ipp)
+static PyObject *do_resolve_ip(struct scripting_context *c, PyObject *ipp)
 {
 	struct addr_location al;
 	struct dso *dso;
@@ -233,11 +233,22 @@ static PyObject *perf_brstack_srcline(PyObject *obj, PyObject *args)
 	struct scripting_context *c = get_args(args, "brstack", &brstack);
 	if (!c)
 		return NULL;
-	from = do_perf_brstack_srcline(c, PyDict_GetItemString(brstack, "from"));
-	to = do_perf_brstack_srcline(c, PyDict_GetItemString(brstack, "to"));
+	from = do_resolve_ip(c, PyDict_GetItemString(brstack, "from"));
+	to = do_resolve_ip(c, PyDict_GetItemString(brstack, "to"));
 	return Py_BuildValue("(OO)", from, to);
 }
 
+/* Resolve an numerical address to srcfile/line/disc */
+
+static PyObject *perf_resolve_ip(PyObject *obj, PyObject *args)
+{
+	PyObject *ip;
+	struct scripting_context *c = get_args(args, "ip", &ip);
+	if (!c)
+		return NULL;
+	return do_resolve_ip(c, ip);
+}
+
 static PyMethodDef ContextMethods[] = {
 #ifdef HAVE_LIBTRACEEVENT
 	{ "common_pc", perf_trace_context_common_pc, METH_VARARGS,
@@ -257,6 +268,8 @@ static PyMethodDef ContextMethods[] = {
 	  METH_VARARGS,	"Get source file name, line number and line."},
 	{ "perf_brstack_srcline", perf_brstack_srcline,
 	  METH_VARARGS, "Get source file name, line number, discriminator for from/to of a brstack entry." },
+	{ "perf_resolve_ip", perf_resolve_ip,
+	  METH_VARARGS, "Get source file name, line number, discriminator for numerical IP." },
 	{ NULL, NULL, 0, NULL}
 };
 
Signature · 1 line
2.45.2
author
#6 lore raw

PATCH 6/10 perf: Add plumbling for line/disc for inlines

In reply to Andi Kleen

When resolving inlines save the line number and the discriminator for later use. The line number is already in the string, but it's more convenient to have it as a integer too. Used in future patches.

---
 tools/perf/util/srcline.c | 18 ++++++++++++------
 tools/perf/util/srcline.h |  2 ++
 2 files changed, 14 insertions(+), 6 deletions(-)
tools/perf/util/srcline.c +12 -6
index 07df410b52e4..0b11e3fe3f11 100644
@@ -46,6 +46,7 @@ static const char *srcline_dso_name(struct dso *dso)
 }
 
 static int inline_list__append(struct symbol *symbol, char *srcline,
+			       int line, int disc,
 			       struct inline_node *node)
 {
 	struct inline_list *ilist;
@@ -56,6 +57,8 @@ static int inline_list__append(struct symbol *symbol, char *srcline,
 
 	ilist->symbol = symbol;
 	ilist->srcline = srcline;
+	ilist->disc = disc;
+	ilist->line = line;
 
 	if (callchain_param.order == ORDER_CALLEE)
 		list_add_tail(&ilist->list, &node->val);
@@ -297,7 +300,8 @@ static int inline_list__append_dso_a2l(struct dso *dso,
 	if (a2l->filename)
 		srcline = srcline_from_fileline(a2l->filename, a2l->line);
 
-	return inline_list__append(inline_sym, srcline, node);
+	return inline_list__append(inline_sym, srcline, a2l->line,
+				   a2l->discriminator, node);
 }
 
 static int addr2line(const char *dso_name, u64 addr,
@@ -636,11 +640,12 @@ static int inline_list__append_record(struct dso *dso,
 				      struct symbol *sym,
 				      const char *function,
 				      const char *filename,
-				      unsigned int line_nr)
+				      unsigned int line_nr,
+				      unsigned int disc)
 {
 	struct symbol *inline_sym = new_inline_sym(dso, sym, function);
 
-	return inline_list__append(inline_sym, srcline_from_fileline(filename, line_nr), node);
+	return inline_list__append(inline_sym, srcline_from_fileline(filename, line_nr), line_nr, disc, node);
 }
 
 static int addr2line(const char *dso_name, u64 addr,
@@ -750,7 +755,8 @@ static int addr2line(const char *dso_name, u64 addr,
 		if (node && inline_list__append_record(dso, node, sym,
 						       record_function,
 						       record_filename,
-						       record_line_nr)) {
+						       record_line_nr,
+						       disc)) {
 			ret = 0;
 			goto out;
 		}
@@ -768,12 +774,12 @@ static int addr2line(const char *dso_name, u64 addr,
 						      /*first=*/false,
 						      &record_function,
 						      &record_filename,
-						      &record_line_nr, NULL)) == 1) {
+						      &record_line_nr, &disc)) == 1) {
 		if (unwind_inlines && node && inline_count++ < MAX_INLINE_NEST) {
 			if (inline_list__append_record(dso, node, sym,
 						       record_function,
 						       record_filename,
-						       record_line_nr)) {
+						       record_line_nr, disc)) {
 				ret = 0;
 				goto out;
 			}
tools/perf/util/srcline.h +2 -0
index d2cb90b1177e..5d37fcc7c02c 100644
@@ -32,6 +32,8 @@ extern char *srcline__unknown;
 struct inline_list {
 	struct symbol		*symbol;
 	char			*srcline;
+	int			line;
+	int			disc;
 	struct list_head	list;
 };
 
Signature · 1 line
2.45.2
author
#7 lore raw

PATCH 7/10 perf: Support returning inlines in get_srcline_split

In reply to Andi Kleen

Support filling in an inline_node for get_srcline_split so that callers can generate an inline stack. Adjust all the callers. Used in next patch.

---
 tools/perf/builtin-script.c                         | 2 +-
 tools/perf/scripts/python/Perf-Trace-Util/Context.c | 3 ++-
 tools/perf/util/dlfilter.c                          | 2 +-
 tools/perf/util/srcline.c                           | 5 +++--
 tools/perf/util/srcline.h                           | 5 ++++-
 5 files changed, 11 insertions(+), 6 deletions(-)
tools/perf/builtin-script.c +1 -1
index c12beceae8b3..065b8c15718d 100644
@@ -1140,7 +1140,7 @@ static int map__fprintf_srccode(struct map *map, u64 addr, FILE *fp, struct srcc
 		return 0;
 	srcfile = get_srcline_split(dso,
 				    map__rip_2objdump(map, addr),
-				    &line, &disc);
+				    &line, &disc, NULL);
 	if (!srcfile)
 		return 0;
 
tools/perf/scripts/python/Perf-Trace-Util/Context.c +2 -1
index 417466231f71..1c1d8d08b10e 100644
@@ -158,7 +158,8 @@ static PyObject *perf_sample_src(PyObject *obj, PyObject *args, bool get_srccode
 	dso = map ? map__dso(map) : NULL;
 
 	if (dso)
-		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line, &disc);
+		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line, &disc,
+				NULL);
 
 	if (get_srccode) {
 		if (srcfile)
tools/perf/util/dlfilter.c +1 -1
index 1a3b5ba92313..33d9e917f0df 100644
@@ -268,7 +268,7 @@ static const char *dlfilter__srcline(void *ctx, __u32 *line_no)
 	dso = map ? map__dso(map) : NULL;
 
 	if (dso)
-		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line, &disc);
+		srcfile = get_srcline_split(dso, map__rip_2objdump(map, addr), &line, &disc, NULL);
 
 	*line_no = line;
 	return srcfile;
tools/perf/util/srcline.c +3 -2
index 0b11e3fe3f11..fac552c0a271 100644
@@ -886,7 +886,8 @@ char *__get_srcline(struct dso *dso, u64 addr, struct symbol *sym,
 }
 
 /* Returns filename and fills in line number in line and discriminator in disc */
-char *get_srcline_split(struct dso *dso, u64 addr, unsigned *line, unsigned *disc)
+char *get_srcline_split(struct dso *dso, u64 addr, unsigned *line, unsigned *disc, 
+		        struct inline_node *inline_node)
 {
 	char *file = NULL;
 	const char *dso_name;
@@ -898,7 +899,7 @@ char *get_srcline_split(struct dso *dso, u64 addr, unsigned *line, unsigned *dis
 	if (dso_name == NULL)
 		goto out_err;
 
-	if (!addr2line(dso_name, addr, &file, line, dso, true, NULL, NULL, disc))
+	if (!addr2line(dso_name, addr, &file, line, dso, true, inline_node, NULL, disc))
 		goto out_err;
 
 	dso__set_a2l_fails(dso, 0);
tools/perf/util/srcline.h +4 -1
index 5d37fcc7c02c..27dbe79a63b9 100644
@@ -17,7 +17,10 @@ char *__get_srcline(struct dso *dso, u64 addr, struct symbol *sym,
 		  bool show_sym, bool show_addr, bool unwind_inlines,
 		  u64 ip);
 void zfree_srcline(char **srcline);
-char *get_srcline_split(struct dso *dso, u64 addr, unsigned *line, unsigned *disc);
+
+struct inline_node;
+char *get_srcline_split(struct dso *dso, u64 addr, unsigned *line, unsigned *disc,
+			struct inline_node *inline_node);
 
 /* insert the srcline into the DSO, which will take ownership */
 void srcline__tree_insert(struct rb_root_cached *tree, u64 addr, char *srcline);
Signature · 1 line
2.45.2
author
#8 lore raw

PATCH 8/10 perf: resolve inlines for perf_brstack_srcline/perf_ip_srcline

In reply to Andi Kleen

Extend perf_brstack/ip_srcline to also return the inline stack to the python script. I made it unconditional for now.

---
 .../scripts/python/Perf-Trace-Util/Context.c  | 43 ++++++++++++++++---
 1 file changed, 37 insertions(+), 6 deletions(-)
tools/perf/scripts/python/Perf-Trace-Util/Context.c +37 -6
index 1c1d8d08b10e..a1c3c35bc6e6 100644
@@ -194,7 +194,12 @@ static PyObject *do_resolve_ip(struct scripting_context *c, PyObject *ipp)
 	bool kernel;
 	u64 ip = PyLong_AsUnsignedLongLong(ipp);
 	unsigned line = 0, disc = 0;
+	struct inline_node *inode = calloc(sizeof(struct inline_node), 1);
 
+	if (!inode)
+		goto err2;
+
+	INIT_LIST_HEAD(&inode->val);
 	kernel = machine__kernel_ip(c->machine, ip);
 	if (kernel)
 		cpumode = PERF_RECORD_MISC_KERNEL;
@@ -213,15 +218,41 @@ static PyObject *do_resolve_ip(struct scripting_context *c, PyObject *ipp)
 		goto err;
 	if (dso__data(dso)->status == DSO_DATA_STATUS_ERROR)
 		goto err;
-	srcfile = get_srcline_split(dso, map__rip_2objdump(al.map, al.addr), &line, &disc);
-	if (srcfile)
-		result = Py_BuildValue("(sII)", srcfile, line, disc);
-	free(srcfile);
+	srcfile = get_srcline_split(dso, map__rip_2objdump(al.map, al.addr), &line,
+				    &disc, inode);
+	if (srcfile && srcfile != SRCLINE_UNKNOWN) {
+		if (!list_empty(&inode->val)) {
+			PyObject *inlines;
+			struct inline_list *ilist;
+			int num;
+		       	num = 0;
+			list_for_each_entry (ilist, &inode->val, list)
+				num++;
+			inlines = PyTuple_New(num);
+			if (!inlines)
+				goto out;
+			num = 0;
+			list_for_each_entry (ilist, &inode->val, list) {
+				PyTuple_SetItem(inlines, num,
+						Py_BuildValue("(ssII)",
+							ilist->srcline, ilist->symbol->name,
+							ilist->line, ilist->disc));
+				num++;
+			}
+			result = Py_BuildValue("(sIIO)", srcfile, line, disc, inlines);
+			Py_DECREF(inlines);
+		out:
+		} else {
+			result = Py_BuildValue("(sII)", srcfile, line, disc);
+		}
+	}
+	zfree_srcline(&srcfile);
 err:
 	addr_location__exit(&al);
 err2:
 	if (!result)
 		return Py_BuildValue("");
+	inline_node__delete(inode);
 	return result;
 }
 
@@ -268,9 +299,9 @@ static PyMethodDef ContextMethods[] = {
 	{ "perf_sample_srccode", perf_sample_srccode,
 	  METH_VARARGS,	"Get source file name, line number and line."},
 	{ "perf_brstack_srcline", perf_brstack_srcline,
-	  METH_VARARGS, "Get source file name, line number, discriminator for from/to of a brstack entry." },
+	  METH_VARARGS, "Get source file name, line number, discriminator, inline stack for from/to of a brstack entry." },
 	{ "perf_resolve_ip", perf_resolve_ip,
-	  METH_VARARGS, "Get source file name, line number, discriminator for numerical IP." },
+	  METH_VARARGS, "Get source file name, line number, discriminator, inline stack for numerical IP." },
 	{ NULL, NULL, 0, NULL}
 };
 
Signature · 1 line
2.45.2
author
#9 lore raw

PATCH 9/10 perf: Add build id and filename to perf_brstack/ip_srcline

In reply to Andi Kleen

Add the buildid and executable file name to the tuple returned by the new perf_brstack/ip_srcline python callbacks. This is all useful for some analysis by perf scripts.

---
 .../scripts/python/Perf-Trace-Util/Context.c  | 20 ++++++++++++-------
 1 file changed, 13 insertions(+), 7 deletions(-)
tools/perf/scripts/python/Perf-Trace-Util/Context.c +13 -7
index a1c3c35bc6e6..a64acc3af607 100644
@@ -23,6 +23,7 @@
 #include "../../../util/srcline.h"
 #include "../../../util/srccode.h"
 #include "../../../util/dso.h"
+#include "../../../util/build-id.h"
 
 #if PY_MAJOR_VERSION < 3
 #define _PyCapsule_GetPointer(arg1, arg2) \
@@ -195,6 +196,7 @@ static PyObject *do_resolve_ip(struct scripting_context *c, PyObject *ipp)
 	u64 ip = PyLong_AsUnsignedLongLong(ipp);
 	unsigned line = 0, disc = 0;
 	struct inline_node *inode = calloc(sizeof(struct inline_node), 1);
+	char bid[SBUILD_ID_SIZE];
 
 	if (!inode)
 		goto err2;
@@ -218,6 +220,8 @@ static PyObject *do_resolve_ip(struct scripting_context *c, PyObject *ipp)
 		goto err;
 	if (dso__data(dso)->status == DSO_DATA_STATUS_ERROR)
 		goto err;
+	build_id__sprintf(dso__bid(dso), bid);
+
 	srcfile = get_srcline_split(dso, map__rip_2objdump(al.map, al.addr), &line,
 				    &disc, inode);
 	if (srcfile && srcfile != SRCLINE_UNKNOWN) {
@@ -234,16 +238,18 @@ static PyObject *do_resolve_ip(struct scripting_context *c, PyObject *ipp)
 			num = 0;
 			list_for_each_entry (ilist, &inode->val, list) {
 				PyTuple_SetItem(inlines, num,
-						Py_BuildValue("(ssII)",
-							ilist->srcline, ilist->symbol->name,
-							ilist->line, ilist->disc));
+						Py_BuildValue("(s#IIs)",
+							ilist->srcline,
+							strcspn(ilist->srcline, ":"),
+							ilist->line, ilist->disc,
+							ilist->symbol->name));
 				num++;
 			}
-			result = Py_BuildValue("(sIIO)", srcfile, line, disc, inlines);
+			result = Py_BuildValue("(sIIssO)", srcfile, line, disc, dso->long_name, bid, inlines);
 			Py_DECREF(inlines);
 		out:
 		} else {
-			result = Py_BuildValue("(sII)", srcfile, line, disc);
+			result = Py_BuildValue("(sIIss())", srcfile, line, disc, dso->long_name, bid);
 		}
 	}
 	zfree_srcline(&srcfile);
@@ -299,9 +305,9 @@ static PyMethodDef ContextMethods[] = {
 	{ "perf_sample_srccode", perf_sample_srccode,
 	  METH_VARARGS,	"Get source file name, line number and line."},
 	{ "perf_brstack_srcline", perf_brstack_srcline,
-	  METH_VARARGS, "Get source file name, line number, discriminator, inline stack for from/to of a brstack entry." },
+	  METH_VARARGS, "Get source file name, line number, discriminator, executable, build-id, inline stack for from/to of a brstack entry." },
 	{ "perf_resolve_ip", perf_resolve_ip,
-	  METH_VARARGS, "Get source file name, line number, discriminator, inline stack for numerical IP." },
+	  METH_VARARGS, "Get source file name, line number, discriminator, executable, build-id, inline stack for numerical IP." },
 	{ NULL, NULL, 0, NULL}
 };
 
Signature · 1 line
2.45.2
author
#10 lore raw

PATCH 10/10 perf: Update documentation for new python callbacks

In reply to Andi Kleen

Document perf_brstack_srcline and perf_ip_srcline.

---
 tools/perf/Documentation/perf-script-python.txt | 3 +++
 1 file changed, 3 insertions(+)
tools/perf/Documentation/perf-script-python.txt +3 -0
index 13e37e9385ee..8afa86189c3c 100644
@@ -624,6 +624,9 @@ as perf_trace_context.perf_script_context .
  perf_set_itrace_options(context, itrace_options) - set --itrace options if they have not been set already
  perf_sample_srcline(context) - returns source_file_name, line_number
  perf_sample_srccode(context) - returns source_file_name, line_number, source_line
+ perf_brstack_srcline(context, brstack) - returns source_file_name, line_number, discriminator, executable, build-id, inline-stack for each entry of a brstack from the sample. Inline stack is a list of inlines with filename, line number, discriminator, symbolname for each entry.
+ perf_ip_srcline(context, ip) - returns source_file_name, line_number, discriminator, executable, build-id, inline-stack for a numerical IP address.
+
 
 
 Util.py Module
Signature · 1 line
2.45.2
#11 lore raw
In reply to Andi Kleen
Quoted 12 lines · Andi Kleen

[resend because I typoed the mailing list address in the first try. Apologies if you see it twice]

This patch kit adds better support for resolving DWARF from perf script python scripts:

- Add new perf_brstack_srcline / perf_ip_srcline functions to resolve full brstacks or individual IPs.
- Support the DWARF discriminator in addition to the line number
- Report the inline stack to python
- Report the build-id and executable name.

Quoted 4 lines

Most of the patch is just plumbing to pass all this information around.

The first patch is a bug fix and can be applied independently.

Got it, waiting a bit for the rest.

- Arnaldo