Skip to content

RFC perf: Add mmap3 support

Jiri Olsa lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest26 patches 66 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/26bpf: Move stack_map_get_build_id into libinclude/linux/buildid.h kernel/bpf/stackmap.c lib/Makefile +1 more+153 -1401 reply
2/26perf: Introduce mmap3 version of mmap eventinclude/uapi/linux/perf_event.h kernel/events/core.c+57 -825 replies
3/26tools headers uapi: Sync tools/include/uapi/linux/perf_event.htools/include/uapi/linux/perf_event.h+26 -11 reply
4/26perf tools: Add filename__decompress functiontools/perf/util/dso.c tools/perf/util/dso.h+21 -124 replies
5/26perf tools: Add build_id__is_defined functiontools/perf/util/build-id.c tools/perf/util/build-id.h+12 -05 replies
6/26perf tools: Add support to read build id from compressed elftools/perf/util/symbol-elf.c+35 -22 replies
7/26perf tools: Add check for existing link in buildid dirtools/perf/util/build-id.c+19 -12 replies
8/26perf tools: Use struct extra_kernel_map in machine__process_kernel_mmap_eventtools/perf/util/machine.c+33 -290 replies
9/26perf tools: Try load vmlinux from buildid databasetools/perf/util/build-id.c tools/perf/util/build-id.h tools/perf/util/symbol.c+26 -33 replies
10/26perf tools: Enable mmap3 map event when supportedtools/perf/util/evsel.c tools/perf/util/evsel.h tools/perf/util/perf_event_attr_fprintf.c+10 -11 reply
11/26perf tools: Add mmap3 supporttools/lib/perf/include/perf/event.h tools/perf/util/event.c tools/perf/util/event.h +6 more+165 -50 replies
12/26perf tools: Set build id for kernel dso objectstools/perf/util/machine.c+11 -40 replies
13/26perf tools: Plug in mmap3 eventtools/perf/builtin-annotate.c tools/perf/builtin-c2c.c tools/perf/builtin-diff.c +9 more+63 -00 replies
14/26perf tools: Add mmap3 events to --show-mmap-events optiontools/perf/builtin-script.c+33 -02 replies
15/26perf tools: Synthesize proc tasks with mmap3tools/perf/util/mmap.c tools/perf/util/synthetic-events.c+43 -312 replies
16/26perf tools: Synthesize modules with mmap3tools/perf/util/synthetic-events.c+24 -139 replies
17/26perf tools: Synthesize kernel with mmap3tools/perf/util/synthetic-events.c+14 -91 reply
18/26perf tests: Add mmap3 support for perf record testtools/perf/tests/perf-record.c+6 -10 replies
19/26perf tools: Add buildid-list support for mmap3tools/perf/builtin-buildid-list.c+10 -00 replies
20/26perf tools: Add build_id_cache__add functiontools/perf/util/build-id.c tools/perf/util/build-id.h+29 -151 reply
21/26perf tools: Add machine__for_each_dso functiontools/perf/util/machine.c tools/perf/util/machine.h+16 -01 reply
22/26perf tools: Use machine__for_each_dso in perf_session__cache_build_idstools/perf/util/build-id.c+16 -250 replies
23/26perf tools: Add __perf_session__cache_build_ids functiontools/perf/util/build-id.c tools/perf/util/build-id.h+11 -20 replies
24/26perf tools: Add buildid-list --store optiontools/perf/Documentation/perf-buildid-list.txt tools/perf/builtin-buildid-list.c tools/perf/util/probe-event.c+181 -64 replies
25/26perf tools: Move debuginfo download code into get_debuginfotools/perf/util/build-id.c+31 -140 replies
26/26perf tools: Add report --store optiontools/perf/Documentation/perf-report.txt tools/perf/builtin-report.c+20 -00 replies

Reviews trailers per patch

1 of 26 patches has a Reviewed-by or Acked-by.

PatchSong Liu
1/26A-b
2/26
3/26
4/26
5/26
6/26
7/26
8/26
9/26
10/26
11/26
12/26
13/26
14/26
15/26
16/26
17/26
18/26
19/26
20/26
21/26
22/26
23/26
24/26
25/26
26/26

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [RFC 00/26] perf: Add mmap3 support

hi,
while playing with perf daemon support I realized I need the build id data in mmap events, so we don't need to care about removed/updated binaries during long perf runs.

This RFC patchset adds new mmap3 events that copies mmap2 event and adds build id in it. It makes mmap3 the default mmap event for synthesizing kernel/modules/tasks and adds some tooling enhancements to enable the workflow below.

Note that the build id retrieval code is stolen from bpf code, where it's been used (together with file offsets) to replace IPs in user space stack traces. It's now added under lib directory.

On recording server:

- on the recording server we can run record with -B option to skip build id scan:

# perf record -B
^C[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 4.462 MB perf.data ]

# find ~/.debug
find: ‘/root/.debug’: No such file or directory

# perf report
...
97.93% swapper [kernel.kallsyms] [k] native_safe_halt
0.18% sshd [kernel.kallsyms] [k] avtab_search_node
0.14% swapper [kernel.kallsyms] [k] __do_softirq
0.05% swapper [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
0.03% swapper [kernel.kallsyms] [k] finish_task_switch

- display used/hit build ids:

    # perf buildid-list | head -5
    439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
    23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
    d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
    1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
    ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd

- store build id binaries into build id cache:

    # perf buildid-list --store | head -5
    OK   439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
    OK   23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
    OK   d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
    OK   1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
    OK   ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd
    # find ~/.debug | head -5
    /root/.debug
    /root/.debug/[kernel.kallsyms]
    /root/.debug/[kernel.kallsyms]/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46
    /root/.debug/[kernel.kallsyms]/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/kallsyms
    /root/.debug/[kernel.kallsyms]/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/probes

- run debuginfod daemon to provide binaries to another server (below)

    # debuginfod -F /

On another server:

- copy perf.data from 'record' server and run:

    $ find ~/.debug/
    find: ‘/home/jolsa/.debug/’: No such file or directory
    $ perf buildid-list | head -5
    No kallsyms or vmlinux with build-id 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 was found
    439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
    23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
    d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
    1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
    ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd

- report does not show anything (kernel build id does not match):

    $ perf report --stdio
    ...
      97.93%  swapper          [kernel.kallsyms]           [k] 0xffffffffa8b859be
       0.14%  swapper          [kernel.kallsyms]           [k] 0xffffffffa8e00074
       0.11%  sshd             [kernel.kallsyms]           [k] 0xffffffffa855b283
       0.05%  swapper          [kernel.kallsyms]           [k] 0xffffffffa8b85d31
       0.03%  swapper          [kernel.kallsyms]           [k] 0xffffffffa810a220

- store does not work, existing binaries have different build ids:

    $ perf report --store | head -5
    No kallsyms or vmlinux with build-id 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 was found
    FAIL 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
    FAIL 23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
    FAIL d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
    FAIL 1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
    FAIL ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd

- instruct debuginfo client to download them (modules retrieval does not work yet for some reason):

    $ DEBUGINFOD_URLS=http://192.168.122.174:8002 perf report --store  | head -5
    No kallsyms or vmlinux with build-id 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 was found
    OK   439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
    FAIL 23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
    FAIL d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
    FAIL 1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
    OK   ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd

- and report works:

    $ perf report --stdio
    ...
      97.93%  swapper          [kernel.kallsyms]           [k] native_safe_halt
       0.18%  sshd             [kernel.kallsyms]           [k] avtab_search_node
       0.14%  swapper          [kernel.kallsyms]           [k] __do_softirq
       0.05%  swapper          [kernel.kallsyms]           [k] _raw_spin_unlock_irqrestore
       0.03%  swapper          [kernel.kallsyms]           [k] finish_task_switch

- because we have the data in build id cache:

    $ find ~/.debug | head -10
    .../.debug
    .../.debug/home
    .../.debug/home/jolsa
    .../.debug/home/jolsa/.cache
    .../.debug/home/jolsa/.cache/debuginfod_client
    .../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46
    .../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable
    .../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46
    .../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/elf
    .../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/probes

The code still needs some polishing, but I'd like to hear some opinions on the usage workflow, so it could get adjusted early on ;-)

For example: should we make -B default now? what about users that expect build id cache populated? And perhaps some .perfconfig setup possibility for debuginfod server.

Available also in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/mmap3

thanks,
jirka

Cc Alexei Starovoitov <[email protected]>

---
Jiri Olsa (26):
bpf: Move stack_map_get_build_id into lib
perf: Introduce mmap3 version of mmap event
tools headers uapi: Sync tools/include/uapi/linux/perf_event.h perf tools: Add filename__decompress function
perf tools: Add build_id__is_defined function
perf tools: Add support to read build id from compressed elf perf tools: Add check for existing link in buildid dir
perf tools: Use struct extra_kernel_map in machine__process_kernel_mmap_event perf tools: Try load vmlinux from buildid database
perf tools: Enable mmap3 map event when supported
perf tools: Add mmap3 support
perf tools: Set build id for kernel dso objects
perf tools: Plug in mmap3 event
perf tools: Add mmap3 events to --show-mmap-events option
perf tools: Synthesize proc tasks with mmap3
perf tools: Synthesize modules with mmap3
perf tools: Synthesize kernel with mmap3
perf tests: Add mmap3 support for perf record test
perf tools: Add buildid-list support for mmap3
perf tools: Add build_id_cache__add function
perf tools: Add machine__for_each_dso function
perf tools: Use machine__for_each_dso in perf_session__cache_build_ids perf tools: Add __perf_session__cache_build_ids function
perf tools: Add buildid-list --store option
perf tools: Move debuginfo download code into get_debuginfo
perf tools: Add report --store option

 include/linux/buildid.h                        |  11 +++++++
 include/uapi/linux/perf_event.h                |  27 +++++++++++++++-
 kernel/bpf/stackmap.c                          | 143 +++--------------------------------------------------------------------------------
 kernel/events/core.c                           |  38 +++++++++++++++++-----
 lib/Makefile                                   |   3 +-
 lib/buildid.c                                  | 136 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/include/uapi/linux/perf_event.h          |  27 +++++++++++++++-
 tools/lib/perf/include/perf/event.h            |  18 +++++++++++
 tools/perf/Documentation/perf-buildid-list.txt |  12 +++++++
 tools/perf/Documentation/perf-report.txt       |   3 ++
 tools/perf/builtin-annotate.c                  |   1 +
 tools/perf/builtin-buildid-list.c              | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--
 tools/perf/builtin-c2c.c                       |   1 +
 tools/perf/builtin-diff.c                      |   1 +
 tools/perf/builtin-inject.c                    |  38 ++++++++++++++++++++++
 tools/perf/builtin-kmem.c                      |   1 +
 tools/perf/builtin-mem.c                       |   1 +
 tools/perf/builtin-record.c                    |  14 +++++++++
 tools/perf/builtin-report.c                    |  19 +++++++++++
 tools/perf/builtin-script.c                    |  34 ++++++++++++++++++++
 tools/perf/builtin-trace.c                     |   1 +
 tools/perf/tests/perf-record.c                 |   7 ++++-
 tools/perf/util/build-id.c                     | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----------------------------------
 tools/perf/util/build-id.h                     |   8 +++++
 tools/perf/util/data-convert-bt.c              |   1 +
 tools/perf/util/dso.c                          |  31 +++++++++++-------
 tools/perf/util/dso.h                          |   2 ++
 tools/perf/util/event.c                        |  32 +++++++++++++++++++
 tools/perf/util/event.h                        |   5 +++
 tools/perf/util/evsel.c                        |   9 +++++-
 tools/perf/util/evsel.h                        |   1 +
 tools/perf/util/machine.c                      | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------
 tools/perf/util/machine.h                      |   6 ++++
 tools/perf/util/map.c                          |   8 +++--
 tools/perf/util/map.h                          |   2 +-
 tools/perf/util/mmap.c                         |   2 +-
 tools/perf/util/perf_event_attr_fprintf.c      |   1 +
 tools/perf/util/probe-event.c                  |   6 ++--
 tools/perf/util/session.c                      |  28 +++++++++++++++++
 tools/perf/util/symbol-elf.c                   |  37 ++++++++++++++++++++--
 tools/perf/util/symbol.c                       |  14 +++++++++
 tools/perf/util/synthetic-events.c             | 132 ++++++++++++++++++++++++++++++++++++++++++++++------------------------------
 tools/perf/util/tool.h                         |   1 +
 43 files changed, 1059 insertions(+), 316 deletions(-)
 create mode 100644 include/linux/buildid.h
 create mode 100644 lib/buildid.c