Skip to contentSkip to conversation

tcg: add perfmap and jitdump

PATCH Superseded Ilya Leoshkevich 3 participants 8 messages Jan 11, 2023 – Jan 12, 2023 Arm/Amlogic Meson SoC support

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover tcg: add perfmap and jitdump

v2:
https://lists.gnu.org/archive/html/qemu-devel/2022-11/msg02385.html
https://lists.gnu.org/archive/html/qemu-devel/2023-01/msg01026.html

v2 -> v3:
* Enable only for CONFIG_LINUX (Alex).
* Use qemu_get_thread_id() instead of gettid() (Alex).
* Fix CI (Alex).
https://gitlab.com/iii-i/qemu/-/pipelines/743684604
* Drop unnecessary #includes (Alex).
* Drop the constification change (Alex/Richard).
* Split debuginfo support into a separate patch.
* Fix partial perfmap/jitdump files when terminating due to a signal.
* Fix debuginfo strings being accessed outside of debuginfo lock.
* Fix address resolution with TARGET_TB_PCREL.
* Add DEBUGINFOD_URLS= to the doc; without it perf inject is unacceptably slow.
* Note: it's better to test this with the latest perf (6.2.rc3.g7dd4b804e080 worked fine for me). There has been at least one breakage in the JIT area recently (fixed by 6d518ac7be62).

v1:
https://lists.nongnu.org/archive/html/qemu-devel/2022-10/msg01824.html
https://lists.nongnu.org/archive/html/qemu-devel/2022-11/msg01073.html

v1 -> v2:
* Use QEMU_LOCK_GUARD (Alex).
* Handle TARGET_TB_PCREL (Alex).
* Support ELF -kernels, add a note about this (Alex). Tested with qemu-system-x86_64 and Linux kernel - it's not fast, but it works.
* Minor const correctness and style improvements.

Ilya Leoshkevich (3):
linux-user: Clean up when exiting due to a signal accel/tcg: Add debuginfo support
tcg: add perfmap and jitdump

 accel/tcg/debuginfo.c     |  96 ++++++++++
 accel/tcg/debuginfo.h     |  77 ++++++++
 accel/tcg/meson.build     |   2 +
 accel/tcg/perf.c          | 366 ++++++++++++++++++++++++++++++++++++++
 accel/tcg/perf.h          |  49 +++++
 accel/tcg/translate-all.c |   8 +
 docs/devel/tcg.rst        |  23 +++
 hw/core/loader.c          |   5 +
 linux-user/elfload.c      |   3 +
 linux-user/exit.c         |   2 +
 linux-user/main.c         |  15 ++
 linux-user/meson.build    |   1 +
 linux-user/signal.c       |   8 +-
 meson.build               |   8 +
 qemu-options.hx           |  20 +++
 softmmu/vl.c              |  11 ++
 tcg/tcg.c                 |   2 +
 17 files changed, 693 insertions(+), 3 deletions(-)
 create mode 100644 accel/tcg/debuginfo.c
 create mode 100644 accel/tcg/debuginfo.h
 create mode 100644 accel/tcg/perf.c
 create mode 100644 accel/tcg/perf.h
Signature · 1 line
2.39.0
#1 lore raw

PATCH v3 1/3 linux-user: Clean up when exiting due to a signal

When exiting due to an exit() syscall, qemu-user calls preexit_cleanup(), but this is currently not the case when exiting due to a signal. This leads to various buffers not being flushed (e.g., for gprof, for gcov, and for the upcoming perf support).

Add the missing call.

---
 linux-user/signal.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
linux-user/signal.c +5 -3
index 61c6fa3fcf1..098f3a787db 100644
@@ -695,7 +695,7 @@ void cpu_loop_exit_sigbus(CPUState *cpu, target_ulong addr,
 
 /* abort execution with signal */
 static G_NORETURN
-void dump_core_and_abort(int target_sig)
+void dump_core_and_abort(CPUArchState *cpu_env, int target_sig)
 {
     CPUState *cpu = thread_cpu;
     CPUArchState *env = cpu->env_ptr;
@@ -724,6 +724,8 @@ void dump_core_and_abort(int target_sig)
             target_sig, strsignal(host_sig), "core dumped" );
     }
 
+    preexit_cleanup(cpu_env, 128 + target_sig);
+
     /* The proper exit code for dying from an uncaught signal is
      * -<signal>.  The kernel doesn't allow exit() or _exit() to pass
      * a negative value.  To get the proper exit code we need to
@@ -1058,12 +1060,12 @@ static void handle_pending_signal(CPUArchState *cpu_env, int sig,
                    sig != TARGET_SIGURG &&
                    sig != TARGET_SIGWINCH &&
                    sig != TARGET_SIGCONT) {
-            dump_core_and_abort(sig);
+            dump_core_and_abort(cpu_env, sig);
         }
     } else if (handler == TARGET_SIG_IGN) {
         /* ignore sig */
     } else if (handler == TARGET_SIG_ERR) {
-        dump_core_and_abort(sig);
+        dump_core_and_abort(cpu_env, sig);
     } else {
         /* compute the blocked signals during the handler execution */
         sigset_t *blocked_set;
Signature · 1 line
2.39.0
Reviewed-by on v3 1/3
#2 lore raw

Re: [PATCH v3 1/3] linux-user: Clean up when exiting due to a signal

Quoted 8 lines · Ilya Leoshkevich

When exiting due to an exit() syscall, qemu-user calls preexit_cleanup(), but this is currently not the case when exiting due to a signal. This leads to various buffers not being flushed (e.g., for gprof, for gcov, and for the upcoming perf support).

Add the missing call.

Signature · 2 lines
Alex Bennée
Virtualisation Tech Lead @ Linaro
Reviewed-by on v3 1/3
#3 lore raw

Re: [PATCH v3 1/3] linux-user: Clean up when exiting due to a signal

In reply to Ilya Leoshkevich
Quoted 11 lines · Ilya Leoshkevich

When exiting due to an exit() syscall, qemu-user calls preexit_cleanup(), but this is currently not the case when exiting due to a signal. This leads to various buffers not being flushed (e.g., for gprof, for gcov, and for the upcoming perf support).

Add the missing call.

---
  linux-user/signal.c | 8 +++++---
  1 file changed, 5 insertions(+), 3 deletions(-)

r~

#4 lore raw

PATCH v3 2/3 accel/tcg: Add debuginfo support

In reply to Ilya Leoshkevich

Add libdw-based functions for loading and querying debuginfo. Load debuginfo from the system and the linux-user loaders.

This is useful for the upcoming perf support, which can then put human-readable guest symbols instead of raw guest PCs into perfmap and jitdump files.

---
 accel/tcg/debuginfo.c  | 96 ++++++++++++++++++++++++++++++++++++++++++
 accel/tcg/debuginfo.h  | 77 +++++++++++++++++++++++++++++++++
 accel/tcg/meson.build  |  1 +
 hw/core/loader.c       |  5 +++
 linux-user/elfload.c   |  3 ++
 linux-user/meson.build |  1 +
 meson.build            |  8 ++++
 7 files changed, 191 insertions(+)
 create mode 100644 accel/tcg/debuginfo.c
 create mode 100644 accel/tcg/debuginfo.h
accel/tcg/debuginfo.c new file +96 -0
new file mode 100644
index 00000000000..fee98a8e867
@@ -0,0 +1,96 @@
+/*
+ * Debug information support.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/lockable.h"
+
+#include <elfutils/libdwfl.h>
+
+#include "debuginfo.h"
+
+static QemuMutex lock;
+static Dwfl *dwfl;
+static const Dwfl_Callbacks dwfl_callbacks = {
+    .find_elf = NULL,
+    .find_debuginfo = dwfl_standard_find_debuginfo,
+    .section_address = NULL,
+    .debuginfo_path = NULL,
+};
+
+__attribute__((constructor))
+static void debuginfo_init(void)
+{
+    qemu_mutex_init(&lock);
+}
+
+void debuginfo_report_elf(const char *name, int fd, unsigned long long bias)
+{
+    QEMU_LOCK_GUARD(&lock);
+
+    if (dwfl) {
+        dwfl_report_begin_add(dwfl);
+    } else {
+        dwfl = dwfl_begin(&dwfl_callbacks);
+    }
+
+    if (dwfl) {
+        dwfl_report_elf(dwfl, name, name, fd, bias, true);
+        dwfl_report_end(dwfl, NULL, NULL);
+    }
+}
+
+void debuginfo_lock(void)
+{
+    qemu_mutex_lock(&lock);
+}
+
+void debuginfo_query(struct debuginfo_query *q, size_t n)
+{
+    const char *symbol, *file;
+    Dwfl_Module *dwfl_module;
+    Dwfl_Line *dwfl_line;
+    GElf_Off dwfl_offset;
+    GElf_Sym dwfl_sym;
+    size_t i;
+    int line;
+
+    if (!dwfl) {
+        return;
+    }
+
+    for (i = 0; i < n; i++) {
+        dwfl_module = dwfl_addrmodule(dwfl, q[i].address);
+        if (!dwfl_module) {
+            continue;
+        }
+
+        if (q[i].flags & DEBUGINFO_SYMBOL) {
+            symbol = dwfl_module_addrinfo(dwfl_module, q[i].address,
+                                          &dwfl_offset, &dwfl_sym,
+                                          NULL, NULL, NULL);
+            if (symbol) {
+                q[i].symbol = symbol;
+                q[i].offset = dwfl_offset;
+            }
+        }
+
+        if (q[i].flags & DEBUGINFO_LINE) {
+            dwfl_line = dwfl_module_getsrc(dwfl_module, q[i].address);
+            if (dwfl_line) {
+                file = dwfl_lineinfo(dwfl_line, NULL, &line, 0, NULL, NULL);
+                if (file) {
+                    q[i].file = file;
+                    q[i].line = line;
+                }
+            }
+        }
+    }
+}
+
+void debuginfo_unlock(void)
+{
+    qemu_mutex_unlock(&lock);
+}
accel/tcg/debuginfo.h new file +77 -0

77 lines not on this page · raw message

accel/tcg/meson.build +1 -0
index 75e1dffb4df..55b3b4dd7e3 100644
@@ -12,6 +12,7 @@ tcg_ss.add(files(
 tcg_ss.add(when: 'CONFIG_USER_ONLY', if_true: files('user-exec.c'))
 tcg_ss.add(when: 'CONFIG_SOFTMMU', if_false: files('user-exec-stub.c'))
 tcg_ss.add(when: 'CONFIG_PLUGIN', if_true: [files('plugin-gen.c')])
+tcg_ss.add(when: libdw, if_true: files('debuginfo.c'))
 specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
hw/core/loader.c +5 -0
index 0548830733e..55dbe2e1993 100644
@@ -61,6 +61,7 @@
 #include "hw/boards.h"
 #include "qemu/cutils.h"
 #include "sysemu/runstate.h"
+#include "accel/tcg/debuginfo.h"
 
 #include <zlib.h>
 
@@ -503,6 +504,10 @@ ssize_t load_elf_ram_sym(const char *filename,
                          clear_lsb, data_swab, as, load_rom, sym_cb);
     }
 
+    if (ret != ELF_LOAD_FAILED) {
+        debuginfo_report_elf(filename, fd, 0);
+    }
+
  fail:
     close(fd);
     return ret;
linux-user/elfload.c +3 -0
index 20894b633f5..5928c14dfc9 100644
@@ -19,6 +19,7 @@
 #include "qemu/selfmap.h"
 #include "qapi/error.h"
 #include "target_signal.h"
+#include "accel/tcg/debuginfo.h"
 
 #ifdef _ARCH_PPC64
 #undef ARCH_DLINFO
@@ -3261,6 +3262,8 @@ static void load_elf_image(const char *image_name, int image_fd,
         load_symbols(ehdr, image_fd, load_bias);
     }
 
+    debuginfo_report_elf(image_name, image_fd, load_bias);
+
     mmap_unlock();
 
     close(image_fd);
linux-user/meson.build +1 -0
index de4320af053..7171dc60be2 100644
@@ -22,6 +22,7 @@ linux_user_ss.add(files(
   'uname.c',
 ))
 linux_user_ss.add(rt)
+linux_user_ss.add(libdw)
 
 linux_user_ss.add(when: 'TARGET_HAS_BFLT', if_true: files('flatload.c'))
 linux_user_ss.add(when: 'TARGET_I386', if_true: files('vm86.c'))
meson.build +8 -0
index 175517eafde..cab8c67d961 100644
@@ -1648,6 +1648,12 @@ if libbpf.found() and not cc.links('''
   endif
 endif
 
+# libdw
+libdw = dependency('libdw',
+                   method: 'pkg-config',
+                   kwargs: static_kwargs,
+                   required: false)
+
 #################
 # config-host.h #
 #################
@@ -1923,6 +1929,7 @@ config_host_data.set('CONFIG_DBUS_DISPLAY', dbus_display)
 config_host_data.set('CONFIG_CFI', get_option('cfi'))
 config_host_data.set('CONFIG_SELINUX', selinux.found())
 config_host_data.set('CONFIG_XEN_BACKEND', xen.found())
+config_host_data.set('CONFIG_LIBDW', libdw.found())
 if xen.found()
   # protect from xen.version() having less than three components
   xen_version = xen.version().split('.') + ['0', '0']
@@ -3981,6 +3988,7 @@ summary_info += {'libudev':           libudev}
 # Dummy dependency, keep .found()
 summary_info += {'FUSE lseek':        fuse_lseek.found()}
 summary_info += {'selinux':           selinux}
+summary_info += {'libdw':             libdw}
 summary(summary_info, bool_yn: true, section: 'Dependencies')
 
 if not supported_cpus.contains(cpu)
Signature · 1 line
2.39.0
#5 lore raw

Re: [PATCH v3 2/3] accel/tcg: Add debuginfo support

Quoted 12 lines · Ilya Leoshkevich

ginfo.h

accel/tcg/debuginfo.h new file +10 -0
@@ -0,0 +1,77 @@
+/*
+ * Debug information support.
+ *
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+
+#ifndef ACCEL_TCG_DEBUGINFO_H
+#define ACCEL_TCG_DEBUGINFO_H
+
+/*

accel/tcg/debuginfo.h line 20

Quoted 10 lines
accel/tcg/debuginfo.h new file +10 -0
+ * Debuginfo describing a certain address.
+ */
+struct debuginfo_query {
+    unsigned long long address;  /* Input: address. */
+    int flags;                   /* Input: debuginfo subset. */
+    const char *symbol;          /* Symbol that the address is part of. */
+    unsigned long long offset;   /* Offset from the symbol. */
+    const char *file;            /* Source file associated with the address. */
+    int line;                    /* Line number in the source file. */
+};

s/unsigned long long/uint64_t/g

Otherwise it looks reasonable.

r~

#6 lore raw

PATCH v3 3/3 tcg: add perfmap and jitdump

In reply to Ilya Leoshkevich

Add ability to dump /tmp/perf-<pid>.map and jit-<pid>.dump. The first one allows the perf tool to map samples to each individual translation block. The second one adds the ability to resolve symbol names, line numbers and inspect JITed code.

Example of use:

perf record qemu-x86_64 -perfmap ./a.out
perf report

or

perf record -k 1 qemu-x86_64 -jitdump ./a.out DEBUGINFOD_URLS= perf inject -j -i perf.data -o perf.data.jitted perf report -i perf.data.jitted

---
 accel/tcg/meson.build     |   1 +
 accel/tcg/perf.c          | 366 ++++++++++++++++++++++++++++++++++++++
 accel/tcg/perf.h          |  49 +++++
 accel/tcg/translate-all.c |   8 +
 docs/devel/tcg.rst        |  23 +++
 linux-user/exit.c         |   2 +
 linux-user/main.c         |  15 ++
 qemu-options.hx           |  20 +++
 softmmu/vl.c              |  11 ++
 tcg/tcg.c                 |   2 +
 10 files changed, 497 insertions(+)
 create mode 100644 accel/tcg/perf.c
 create mode 100644 accel/tcg/perf.h
accel/tcg/meson.build +1 -0
index 55b3b4dd7e3..77740b1a0d7 100644
@@ -13,6 +13,7 @@ tcg_ss.add(when: 'CONFIG_USER_ONLY', if_true: files('user-exec.c'))
 tcg_ss.add(when: 'CONFIG_SOFTMMU', if_false: files('user-exec-stub.c'))
 tcg_ss.add(when: 'CONFIG_PLUGIN', if_true: [files('plugin-gen.c')])
 tcg_ss.add(when: libdw, if_true: files('debuginfo.c'))
+tcg_ss.add(when: 'CONFIG_LINUX', if_true: files('perf.c'))
 specific_ss.add_all(when: 'CONFIG_TCG', if_true: tcg_ss)
 
 specific_ss.add(when: ['CONFIG_SOFTMMU', 'CONFIG_TCG'], if_true: files(
accel/tcg/perf.c new file +366 -0

366 lines not on this page · raw message

accel/tcg/perf.h new file +49 -0

49 lines not on this page · raw message

accel/tcg/translate-all.c +8 -0
index 51ac1f6c84a..211afdf6567 100644
@@ -62,6 +62,7 @@
 #include "tb-hash.h"
 #include "tb-context.h"
 #include "internal.h"
+#include "perf.h"
 
 /* Make sure all possible CPU event bits fit in tb->trace_vcpu_dstate */
 QEMU_BUILD_BUG_ON(CPU_TRACE_DSTATE_MAX_EVENTS >
@@ -406,6 +407,13 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
     }
     tb->tc.size = gen_code_size;
 
+    /*
+     * For TARGET_TB_PCREL, attribute all executions of the generated
+     * code to its first mapping.
+     */
+    perf_report_code(pc, tb->icount,
+                     tcg_splitwx_to_rx(gen_code_buf), gen_code_size);
+
 #ifdef CONFIG_PROFILER
     qatomic_set(&prof->code_time, prof->code_time + profile_getclock() - ti);
     qatomic_set(&prof->code_in_len, prof->code_in_len + tb->size);
docs/devel/tcg.rst +23 -0
index 136a7a0d962..b4096a17dfe 100644
@@ -188,3 +188,26 @@ memory areas instead calls out to C code for device emulation.
 Finally, the MMU helps tracking dirty pages and pages pointed to by
 translation blocks.
 
+Profiling JITted code
+---------------------
+
+The Linux ``perf`` tool will treat all JITted code as a single block as
+unlike the main code it can't use debug information to link individual
+program counter samples with larger functions. To overcome this
+limitation you can use the ``-perfmap`` or the ``-jitdump`` option to generate
+map files. ``-perfmap`` is lightweight and produces only guest-host mappings.
+``-jitdump`` additionally saves JITed code and guest debug information (if
+available); its output needs to be integrated with the ``perf.data`` file
+before the final report can be viewed.
+
+.. code::
+
+  perf record $QEMU -perfmap $REMAINING_ARGS
+  perf report
+
+  perf record -k 1 $QEMU -jitdump $REMAINING_ARGS
+  DEBUGINFOD_URLS= perf inject -j -i perf.data -o perf.data.jitted
+  perf report -i perf.data.jitted
+
+Note that qemu-system generates mappings only for ``-kernel`` files in ELF
+format.
linux-user/exit.c +2 -0
index fa6ef0b9b44..607b6da9fc4 100644
@@ -17,6 +17,7 @@
  *  along with this program; if not, see <http://www.gnu.org/licenses/>.
  */
 #include "qemu/osdep.h"
+#include "accel/tcg/perf.h"
 #include "exec/gdbstub.h"
 #include "qemu.h"
 #include "user-internals.h"
@@ -38,4 +39,5 @@ void preexit_cleanup(CPUArchState *env, int code)
 #endif
         gdb_exit(code);
         qemu_plugin_user_exit();
+        perf_exit();
 }
linux-user/main.c +15 -0
index a17fed045bf..4290651c3cf 100644
@@ -53,6 +53,7 @@
 #include "signal-common.h"
 #include "loader.h"
 #include "user-mmap.h"
+#include "accel/tcg/perf.h"
 
 #ifdef CONFIG_SEMIHOSTING
 #include "semihosting/semihost.h"
@@ -423,6 +424,16 @@ static void handle_arg_abi_call0(const char *arg)
 }
 #endif
 
+static void handle_arg_perfmap(const char *arg)
+{
+    perf_enable_perfmap();
+}
+
+static void handle_arg_jitdump(const char *arg)
+{
+    perf_enable_jitdump();
+}
+
 static QemuPluginList plugins = QTAILQ_HEAD_INITIALIZER(plugins);
 
 #ifdef CONFIG_PLUGIN
@@ -493,6 +504,10 @@ static const struct qemu_argument arg_table[] = {
     {"xtensa-abi-call0", "QEMU_XTENSA_ABI_CALL0", false, handle_arg_abi_call0,
      "",           "assume CALL0 Xtensa ABI"},
 #endif
+    {"perfmap",    "QEMU_PERFMAP",     false, handle_arg_perfmap,
+     "",           "Generate a /tmp/perf-${pid}.map file for perf"},
+    {"jitdump",    "QEMU_JITDUMP",     false, handle_arg_jitdump,
+     "",           "Generate a jit-${pid}.dump file for perf"},
     {NULL, NULL, false, NULL, NULL, NULL}
 };
 
qemu-options.hx +20 -0
index 3aa3a2f5a35..d59d19704bc 100644
@@ -4838,6 +4838,26 @@ SRST
     Enable synchronization profiling.
 ERST
 
+#if defined(CONFIG_TCG) && defined(CONFIG_LINUX)
+DEF("perfmap", 0, QEMU_OPTION_perfmap,
+    "-perfmap        generate a /tmp/perf-${pid}.map file for perf\n",
+    QEMU_ARCH_ALL)
+SRST
+``-perfmap``
+    Generate a map file for Linux perf tools that will allow basic profiling
+    information to be broken down into basic blocks.
+ERST
+
+DEF("jitdump", 0, QEMU_OPTION_jitdump,
+    "-jitdump        generate a jit-${pid}.dump file for perf\n",
+    QEMU_ARCH_ALL)
+SRST
+``-jitdump``
+    Generate a dump file for Linux perf tools that maps basic blocks to symbol
+    names, line numbers and JITted code.
+ERST
+#endif
+
 DEFHEADING()
 
 DEFHEADING(Generic object creation:)
softmmu/vl.c +11 -0
index 9bd0e52d016..9177d95d4ec 100644
@@ -96,6 +96,9 @@
 #include "fsdev/qemu-fsdev.h"
 #endif
 #include "sysemu/qtest.h"
+#ifdef CONFIG_TCG
+#include "accel/tcg/perf.h"
+#endif
 
 #include "disas/disas.h"
 
@@ -2926,6 +2929,14 @@ void qemu_init(int argc, char **argv)
             case QEMU_OPTION_DFILTER:
                 qemu_set_dfilter_ranges(optarg, &error_fatal);
                 break;
+#if defined(CONFIG_TCG) && defined(CONFIG_LINUX)
+            case QEMU_OPTION_perfmap:
+                perf_enable_perfmap();
+                break;
+            case QEMU_OPTION_jitdump:
+                perf_enable_jitdump();
+                break;
+#endif
             case QEMU_OPTION_seed:
                 qemu_guest_random_seed_main(optarg, &error_fatal);
                 break;
tcg/tcg.c +2 -0
index da91779890c..9b7df71e7a0 100644
@@ -61,6 +61,7 @@
 #include "exec/log.h"
 #include "tcg/tcg-ldst.h"
 #include "tcg-internal.h"
+#include "accel/tcg/perf.h"
 
 /* Forward declarations for functions declared in tcg-target.c.inc and
    used here. */
@@ -913,6 +914,7 @@ void tcg_prologue_init(TCGContext *s)
 #endif
 
     prologue_size = tcg_current_code_size(s);
+    perf_report_prologue(s->code_gen_ptr, prologue_size);
 
 #ifndef CONFIG_TCG_INTERPRETER
     flush_idcache_range((uintptr_t)tcg_splitwx_to_rx(s->code_buf),
Signature · 1 line
2.39.0
#7 lore raw
Quoted 34 lines · Ilya Leoshkevich

Add ability to dump /tmp/perf-<pid>.map and jit-<pid>.dump. The first one allows the perf tool to map samples to each individual translation block. The second one adds the ability to resolve symbol names, line numbers and inspect JITed code.

Example of use:

    perf record qemu-x86_64 -perfmap ./a.out
    perf report

or

    perf record -k 1 qemu-x86_64 -jitdump ./a.out DEBUGINFOD_URLS= perf inject -j -i perf.data -o perf.data.jitted perf report -i perf.data.jitted

---
 accel/tcg/meson.build     |   1 +
 accel/tcg/perf.c          | 366
++++++++++++++++++++++++++++++++++++++
 accel/tcg/perf.h          |  49 +++++
 accel/tcg/translate-all.c |   8 +
 docs/devel/tcg.rst        |  23 +++
 linux-user/exit.c         |   2 +
 linux-user/main.c         |  15 ++
 qemu-options.hx           |  20 +++
 softmmu/vl.c              |  11 ++
 tcg/tcg.c                 |   2 +

 10 files changed, 497 insertions(+)

 create mode 100644 accel/tcg/perf.c
 create mode 100644 accel/tcg/perf.h

...

Quoted 11 lines
(quoted diff) +11 -0
+void perf_report_code(unsigned long long guest_pc, size_t icount,
+                      const void *start, size_t size)
+{
+    struct debuginfo_query *q;
+    size_t insn;
+
+    if (!perfmap && !jitdump) {
+        return;
+    }
+
+    q = g_try_malloc0_n(icount, sizeof(*q));

Review

Quoted 10 lines
(quoted diff) +10 -0
+    if (!q) {
+        return;
+    }
+
+    debuginfo_lock();
+
+    /* Query debuginfo for each guest instruction. */
+    for (insn = 0; insn < icount; insn++) {
+        q[insn].address = tcg_ctx->gen_insn_data[insn][0] +
+                          (TARGET_TB_PCREL ? guest_pc : 0);

Currently this produces plausibly looking, but actually wrong addresses. This needs to match restore_state_to_opc(), so at least:

accel/tcg/perf.c +0 -0
@@ -325,8 +325,10 @@ void perf_report_code(unsigned long long guest_pc,

size_t icount,

     /* Query debuginfo for each guest instruction. */
     for (insn = 0; insn < icount; insn++) {

- q[insn].address = tcg_ctx->gen_insn_data[insn][0] +
- (TARGET_TB_PCREL ? guest_pc : 0);
+ q[insn].address = tcg_ctx->gen_insn_data[insn][0];
+ if (TARGET_TB_PCREL) {
+ q[insn].address |= (guest_pc & TARGET_PAGE_MASK);
+ }
q[insn].flags = DEBUGINFO_SYMBOL | (jitdump ? DEBUGINFO_LINE :

0);
     }
     debuginfo_query(q, icount);

Apparently even with this there are corner cases, e.g. in x86_restore_state_to_opc() we have:

    if (TARGET_TB_PCREL) {
        env->eip = (env->eip & TARGET_PAGE_MASK) | data[0];
    } else {
        env->eip = data[0] - tb->cs_base;
    }

so if tb->cs_base != 0, the result is still going to be wrong.

I wonder if it would make sense to create a new TCGCPUOps member purely for resolving a PC from data[0]?