RFC perf: Add mmap3 support
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest26 patches 66 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/26 | bpf: Move stack_map_get_build_id into libinclude/ kernel/ lib/ +1 more | +153 | 1 reply | |
| 2/26 | perf: Introduce mmap3 version of mmap eventinclude/ kernel/ | +57 | 25 replies | |
| 3/26 | tools headers uapi: Sync tools/include/uapi/linux/perf_event.htools/ | +26 | 1 reply | |
| 4/26 | perf tools: Add filename__decompress functiontools/ tools/ | +21 | 4 replies | |
| 5/26 | perf tools: Add build_id__is_defined functiontools/ tools/ | +12 | 5 replies | |
| 6/26 | perf tools: Add support to read build id from compressed elftools/ | +35 | 2 replies | |
| 7/26 | perf tools: Add check for existing link in buildid dirtools/ | +19 | 2 replies | |
| 8/26 | perf tools: Use struct extra_kernel_map in machine__process_kernel_mmap_eventtools/ | +33 | 0 replies | |
| 9/26 | perf tools: Try load vmlinux from buildid databasetools/ tools/ tools/ | +26 | 3 replies | |
| 10/26 | perf tools: Enable mmap3 map event when supportedtools/ tools/ tools/ | +10 | 1 reply | |
| 11/26 | perf tools: Add mmap3 supporttools/ tools/ tools/ +6 more | +165 | 0 replies | |
| 12/26 | perf tools: Set build id for kernel dso objectstools/ | +11 | 0 replies | |
| 13/26 | perf tools: Plug in mmap3 eventtools/ tools/ tools/ +9 more | +63 | 0 replies | |
| 14/26 | perf tools: Add mmap3 events to --show-mmap-events optiontools/ | +33 | 2 replies | |
| 15/26 | perf tools: Synthesize proc tasks with mmap3tools/ tools/ | +43 | 2 replies | |
| 16/26 | perf tools: Synthesize modules with mmap3tools/ | +24 | 9 replies | |
| 17/26 | perf tools: Synthesize kernel with mmap3tools/ | +14 | 1 reply | |
| 18/26 | perf tests: Add mmap3 support for perf record testtools/ | +6 | 0 replies | |
| 19/26 | perf tools: Add buildid-list support for mmap3tools/ | +10 | 0 replies | |
| 20/26 | perf tools: Add build_id_cache__add functiontools/ tools/ | +29 | 1 reply | |
| 21/26 | perf tools: Add machine__for_each_dso functiontools/ tools/ | +16 | 1 reply | |
| 22/26 | perf tools: Use machine__for_each_dso in perf_session__cache_build_idstools/ | +16 | 0 replies | |
| 23/26 | perf tools: Add __perf_session__cache_build_ids functiontools/ tools/ | +11 | 0 replies | |
| 24/26 | perf tools: Add buildid-list --store optiontools/ tools/ tools/ | +181 | 4 replies | |
| 25/26 | perf tools: Move debuginfo download code into get_debuginfotools/ | +31 | 0 replies | |
| 26/26 | perf tools: Add report --store optiontools/ tools/ | +20 | 0 replies |
Reviews trailers per patch
1 of 26 patches has a Reviewed-by or Acked-by.
| Patch | Song Liu |
|---|---|
| 1/26 | A-b |
| 2/26 | |
| 3/26 | |
| 4/26 | |
| 5/26 | |
| 6/26 | |
| 7/26 | |
| 8/26 | |
| 9/26 | |
| 10/26 | |
| 11/26 | |
| 12/26 | |
| 13/26 | |
| 14/26 | |
| 15/26 | |
| 16/26 | |
| 17/26 | |
| 18/26 | |
| 19/26 | |
| 20/26 | |
| 21/26 | |
| 22/26 | |
| 23/26 | |
| 24/26 | |
| 25/26 | |
| 26/26 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [RFC 00/26] perf: Add mmap3 support
hi,
while playing with perf daemon support I realized I need the build id data in mmap events, so we don't need to care about removed/updated binaries during long perf runs.
This RFC patchset adds new mmap3 events that copies mmap2 event and adds build id in it. It makes mmap3 the default mmap event for synthesizing kernel/modules/tasks and adds some tooling enhancements to enable the workflow below.
Note that the build id retrieval code is stolen from bpf code, where it's been used (together with file offsets) to replace IPs in user space stack traces. It's now added under lib directory.
On recording server:
- on the recording server we can run record with -B option to skip build id scan:
# perf record -B
^C[ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 4.462 MB perf.data ]
# find ~/.debug
find: ‘/root/.debug’: No such file or directory
# perf report
...
97.93% swapper [kernel.kallsyms] [k] native_safe_halt
0.18% sshd [kernel.kallsyms] [k] avtab_search_node
0.14% swapper [kernel.kallsyms] [k] __do_softirq
0.05% swapper [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
0.03% swapper [kernel.kallsyms] [k] finish_task_switch
- display used/hit build ids:
# perf buildid-list | head -5
439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd- store build id binaries into build id cache:
# perf buildid-list --store | head -5
OK 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
OK 23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
OK d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
OK 1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
OK ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd # find ~/.debug | head -5
/root/.debug
/root/.debug/[kernel.kallsyms]
/root/.debug/[kernel.kallsyms]/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46
/root/.debug/[kernel.kallsyms]/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/kallsyms
/root/.debug/[kernel.kallsyms]/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/probes- run debuginfod daemon to provide binaries to another server (below)
# debuginfod -F /
On another server:
- copy perf.data from 'record' server and run:
$ find ~/.debug/
find: ‘/home/jolsa/.debug/’: No such file or directory $ perf buildid-list | head -5
No kallsyms or vmlinux with build-id 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 was found
439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd- report does not show anything (kernel build id does not match):
$ perf report --stdio
...
97.93% swapper [kernel.kallsyms] [k] 0xffffffffa8b859be
0.14% swapper [kernel.kallsyms] [k] 0xffffffffa8e00074
0.11% sshd [kernel.kallsyms] [k] 0xffffffffa855b283
0.05% swapper [kernel.kallsyms] [k] 0xffffffffa8b85d31
0.03% swapper [kernel.kallsyms] [k] 0xffffffffa810a220- store does not work, existing binaries have different build ids:
$ perf report --store | head -5
No kallsyms or vmlinux with build-id 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 was found
FAIL 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms]
FAIL 23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz
FAIL d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz
FAIL 1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz
FAIL ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd- instruct debuginfo client to download them (modules retrieval does not work yet for some reason):
$ DEBUGINFOD_URLS=http://192.168.122.174:8002 perf report --store | head -5 No kallsyms or vmlinux with build-id 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 was found OK 439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46 [kernel.kallsyms] FAIL 23b87f5b0560481043257e82be670bc97786a171 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/net/ipv4/netfilter/ip_tables.ko.xz FAIL d2b3be372bcdd4ebc15e479d2ff803657de0fd1e /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/drivers/block/virtio_blk.ko.xz FAIL 1466a71bcd0ff5c975ee79b72752137c0143d225 /lib/modules/5.9.0-0.rc3.1.mmap3.fc34.x86_64/kernel/fs/xfs/xfs.ko.xz OK ad60d10b38c93bd8738d5aa594e240f01bb328cd /usr/lib/systemd/systemd
- and report works:
$ perf report --stdio
...
97.93% swapper [kernel.kallsyms] [k] native_safe_halt
0.18% sshd [kernel.kallsyms] [k] avtab_search_node
0.14% swapper [kernel.kallsyms] [k] __do_softirq
0.05% swapper [kernel.kallsyms] [k] _raw_spin_unlock_irqrestore
0.03% swapper [kernel.kallsyms] [k] finish_task_switch- because we have the data in build id cache:
$ find ~/.debug | head -10
.../.debug
.../.debug/home
.../.debug/home/jolsa
.../.debug/home/jolsa/.cache
.../.debug/home/jolsa/.cache/debuginfod_client
.../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46
.../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable
.../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46
.../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/elf
.../.debug/home/jolsa/.cache/debuginfod_client/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/executable/439fe9bdeaed66af2bb8b8de5e650d5ecc3d8d46/probesThe code still needs some polishing, but I'd like to hear some opinions on the usage workflow, so it could get adjusted early on ;-)
For example: should we make -B default now? what about users that expect build id cache populated? And perhaps some .perfconfig setup possibility for debuginfod server.
Available also in:
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
perf/mmap3
thanks,
jirka
---
Jiri Olsa (26):
bpf: Move stack_map_get_build_id into lib
perf: Introduce mmap3 version of mmap event
tools headers uapi: Sync tools/include/uapi/linux/perf_event.h perf tools: Add filename__decompress function
perf tools: Add build_id__is_defined function
perf tools: Add support to read build id from compressed elf perf tools: Add check for existing link in buildid dir
perf tools: Use struct extra_kernel_map in machine__process_kernel_mmap_event perf tools: Try load vmlinux from buildid database
perf tools: Enable mmap3 map event when supported
perf tools: Add mmap3 support
perf tools: Set build id for kernel dso objects
perf tools: Plug in mmap3 event
perf tools: Add mmap3 events to --show-mmap-events option
perf tools: Synthesize proc tasks with mmap3
perf tools: Synthesize modules with mmap3
perf tools: Synthesize kernel with mmap3
perf tests: Add mmap3 support for perf record test
perf tools: Add buildid-list support for mmap3
perf tools: Add build_id_cache__add function
perf tools: Add machine__for_each_dso function
perf tools: Use machine__for_each_dso in perf_session__cache_build_ids perf tools: Add __perf_session__cache_build_ids function
perf tools: Add buildid-list --store option
perf tools: Move debuginfo download code into get_debuginfo
perf tools: Add report --store option
include/linux/buildid.h | 11 +++++++ include/uapi/linux/perf_event.h | 27 +++++++++++++++- kernel/bpf/stackmap.c | 143 +++-------------------------------------------------------------------------------- kernel/events/core.c | 38 +++++++++++++++++----- lib/Makefile | 3 +- lib/buildid.c | 136 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ tools/include/uapi/linux/perf_event.h | 27 +++++++++++++++- tools/lib/perf/include/perf/event.h | 18 +++++++++++ tools/perf/Documentation/perf-buildid-list.txt | 12 +++++++ tools/perf/Documentation/perf-report.txt | 3 ++ tools/perf/builtin-annotate.c | 1 + tools/perf/builtin-buildid-list.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-- tools/perf/builtin-c2c.c | 1 + tools/perf/builtin-diff.c | 1 + tools/perf/builtin-inject.c | 38 ++++++++++++++++++++++ tools/perf/builtin-kmem.c | 1 + tools/perf/builtin-mem.c | 1 + tools/perf/builtin-record.c | 14 +++++++++ tools/perf/builtin-report.c | 19 +++++++++++ tools/perf/builtin-script.c | 34 ++++++++++++++++++++ tools/perf/builtin-trace.c | 1 + tools/perf/tests/perf-record.c | 7 ++++- tools/perf/util/build-id.c | 179 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++---------------------------------- tools/perf/util/build-id.h | 8 +++++ tools/perf/util/data-convert-bt.c | 1 + tools/perf/util/dso.c | 31 +++++++++++------- tools/perf/util/dso.h | 2 ++ tools/perf/util/event.c | 32 +++++++++++++++++++ tools/perf/util/event.h | 5 +++ tools/perf/util/evsel.c | 9 +++++- tools/perf/util/evsel.h | 1 + tools/perf/util/machine.c | 155 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++------------------ tools/perf/util/machine.h | 6 ++++ tools/perf/util/map.c | 8 +++-- tools/perf/util/map.h | 2 +- tools/perf/util/mmap.c | 2 +- tools/perf/util/perf_event_attr_fprintf.c | 1 + tools/perf/util/probe-event.c | 6 ++-- tools/perf/util/session.c | 28 +++++++++++++++++ tools/perf/util/symbol-elf.c | 37 ++++++++++++++++++++-- tools/perf/util/symbol.c | 14 +++++++++ tools/perf/util/synthetic-events.c | 132 ++++++++++++++++++++++++++++++++++++++++++++++------------------------------ tools/perf/util/tool.h | 1 + 43 files changed, 1059 insertions(+), 316 deletions(-) create mode 100644 include/linux/buildid.h create mode 100644 lib/buildid.c