perf/core improvements and fixes
Subsystems (19)
Arnaldo Carvalho de Melo
maintainerHi Ingo/Thomas,
Please consider pulling,
Best regards,
- Arnaldo
Test results at the end of this message, as usual.
The following changes since commit cd0943357bc7570f081701d005318c20982178b8:
Merge tag 'perf-urgent-for-mingo-5.7-20200414' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/urgent (2020-04-16 10:21:31 +0200)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-5.8-20200420
for you to fetch changes up to 12e89e65f446476951f42aedeef56b6bd6f7f1e6:
perf hist: Add fast path for duplicate entries check (2020-04-18 09:05:01 -0300)
----------------------------------------------------------------
perf/core fixes and improvements:
kernel + tools/perf:
Alexey Budankov:
- Introduce CAP_PERFMON to kernel and user space.
callchains:
Adrian Hunter:
- Allow using Intel PT to synthesize callchains for regular events.
Kan Liang:
- Stitch LBR records from multiple samples to get deeper backtraces, there are caveats, see the csets for details.
perf script:
Andreas Gerstmayr:
- Add flamegraph.py script
BPF:
Jiri Olsa:
- Synthesize bpf_trampoline/dispatcher ksymbol events.
perf stat:
Arnaldo Carvalho de Melo:
- Honour --timeout for forked workloads.
Stephane Eranian:
- Force error in fallback on :k events, to avoid counting nothing when the user asks for kernel events but is not allowed to.
perf bench:
Ian Rogers:
- Add event synthesis benchmark.
tools api fs:
Stephane Eranian:
- Make xxx__mountpoint() more scalable
libtraceevent:
He Zhe:
- Handle return value of asprintf.
----------------------------------------------------------------
Adrian Hunter (18):
perf script: Simplify auxiliary event printing functions
perf auxtrace: Add ->evsel_is_auxtrace() callback
perf intel-pt: Implement ->evsel_is_auxtrace() callback
perf intel-bts: Implement ->evsel_is_auxtrace() callback
perf arm-spe: Implement ->evsel_is_auxtrace() callback
perf cs-etm: Implement ->evsel_is_auxtrace() callback
perf s390-cpumsf: Implement ->evsel_is_auxtrace() callback
perf auxtrace: For reporting purposes, un-group AUX area event
perf auxtrace: Add an option to synthesize callchains for regular events perf thread-stack: Add thread_stack__sample_late()
perf evsel: Be consistent when looking which evsel PERF_SAMPLE_ bits are set perf evsel: Add support for synthesized sample type
perf intel-pt: Add support for synthesizing callchains for regular events perf evsel: Move and globalize perf_evsel__find_pmu() and perf_evsel__is_aux_event() perf evlist: Move leader-sampling configuration
perf evsel: Rearrange perf_evsel__config_leader_sampling()
perf evlist: Allow multiple read formats
perf tools: Add support for leader-sampling with AUX area events
Alexey Budankov (12):
capabilities: Introduce CAP_PERFMON to kernel and user space perf/core: Open access to the core for CAP_PERFMON privileged process perf/core: open access to probes for CAP_PERFMON privileged process perf tools: Support CAP_PERFMON capability
drm/i915/perf: Open access for CAP_PERFMON privileged process trace/bpf_trace: Open access for CAP_PERFMON privileged process powerpc/perf: open access for CAP_PERFMON privileged process parisc/perf: open access for CAP_PERFMON privileged process drivers/perf: Open access for CAP_PERFMON privileged process drivers/oprofile: Open access for CAP_PERFMON privileged process doc/admin-guide: Update perf-security.rst with CAP_PERFMON information doc/admin-guide: update kernel.rst with CAP_PERFMON information
Andreas Gerstmayr (1):
perf script: Add flamegraph.py script
Arnaldo Carvalho de Melo (1):
perf stat: Honour --timeout for forked workloads
He Zhe (1):
tools lib traceevent: Take care of return value of asprintf
Ian Rogers (3):
perf bench: Add event synthesis benchmark perf synthetic-events: save 4kb from 2 stack frames perf doc: allow ASCIIDOC_EXTRA to be an argument
Jiri Olsa (6):
perf tools: Synthesize bpf_trampoline/dispatcher ksymbol event perf machine: Set ksymbol dso as loaded on arrival perf annotate: Add basic support for bpf_image
perf expr: Add expr_ prefix for parse_ctx and parse_id perf expr: Add expr_scanner_ctx object
perf parser: Add support to specify rXXX event with pmu
Kajol Jain (1):
perf metrictroup: Split the metricgroup__add_metric function
Kan Liang (15):
perf pmu: Add support for PMU capabilities
perf header: Support CPU PMU capabilities
perf machine: Remove the indent in resolve_lbr_callchain_sample perf machine: Refine the function for LBR call stack reconstruction perf machine: Factor out lbr_callchain_add_kernel_ip() perf machine: Factor out lbr_callchain_add_lbr_ip()
perf thread: Add a knob for LBR stitch approach
perf thread: Save previous sample for LBR stitching approach perf callchain: Save previous cursor nodes for LBR stitching approach perf callchain: Stitch LBR call stack
perf report: Add option to enable the LBR stitching approach perf script: Add option to enable the LBR stitching approach perf top: Add option to enable the LBR stitching approach perf c2c: Add option to enable the LBR stitching approach perf hist: Add fast path for duplicate entries check
Stephane Eranian (2):
tools api fs: Make xxx__mountpoint() more scalable perf stat: Force error in fallback on :k events
Documentation/admin-guide/perf-security.rst | 86 ++-- Documentation/admin-guide/sysctl/kernel.rst | 16 +- arch/parisc/kernel/perf.c | 2 +- arch/powerpc/perf/imc-pmu.c | 4 +- drivers/gpu/drm/i915/i915_perf.c | 13 +- drivers/oprofile/event_buffer.c | 2 +- drivers/perf/arm_spe_pmu.c | 4 +- include/linux/capability.h | 4 + include/linux/perf_event.h | 6 +- include/uapi/linux/capability.h | 8 +- kernel/events/core.c | 6 +- kernel/trace/bpf_trace.c | 2 +- security/selinux/include/classmap.h | 4 +- tools/lib/api/fs/fs.c | 17 + tools/lib/api/fs/fs.h | 12 + tools/lib/traceevent/parse-filter.c | 29 +- tools/perf/Documentation/Makefile | 4 +- tools/perf/Documentation/itrace.txt | 1 + tools/perf/Documentation/perf-bench.txt | 8 + tools/perf/Documentation/perf-c2c.txt | 11 + tools/perf/Documentation/perf-list.txt | 8 + tools/perf/Documentation/perf-report.txt | 11 + tools/perf/Documentation/perf-script.txt | 11 + tools/perf/Documentation/perf-top.txt | 9 + tools/perf/Documentation/perf.data-file-format.txt | 16 + tools/perf/bench/Build | 2 +- tools/perf/bench/bench.h | 2 +- tools/perf/bench/synthesize.c | 101 +++++ tools/perf/builtin-bench.c | 6 + tools/perf/builtin-c2c.c | 12 + tools/perf/builtin-ftrace.c | 5 +- tools/perf/builtin-report.c | 15 +- tools/perf/builtin-script.c | 318 ++++----------- tools/perf/builtin-stat.c | 5 +- tools/perf/builtin-top.c | 11 + tools/perf/design.txt | 3 +- tools/perf/scripts/python/bin/flamegraph-record | 2 + tools/perf/scripts/python/bin/flamegraph-report | 3 + tools/perf/scripts/python/flamegraph.py | 124 ++++++ tools/perf/tests/expr.c | 4 +- tools/perf/tests/parse-events.c | 17 +- tools/perf/util/annotate.c | 20 + tools/perf/util/arm-spe.c | 9 + tools/perf/util/auxtrace.c | 94 +++-- tools/perf/util/auxtrace.h | 14 + tools/perf/util/bpf-event.c | 93 +++++ tools/perf/util/branch.h | 19 +- tools/perf/util/callchain.h | 8 + tools/perf/util/cap.h | 4 + tools/perf/util/cs-etm.c | 11 + tools/perf/util/dso.c | 1 + tools/perf/util/dso.h | 1 + tools/perf/util/env.h | 3 + tools/perf/util/evlist.c | 6 +- tools/perf/util/evsel.c | 35 +- tools/perf/util/evsel.h | 18 +- tools/perf/util/expr.c | 16 +- tools/perf/util/expr.h | 16 +- tools/perf/util/expr.l | 10 +- tools/perf/util/expr.y | 6 +- tools/perf/util/header.c | 108 +++++ tools/perf/util/header.h | 1 + tools/perf/util/hist.c | 23 ++ tools/perf/util/intel-bts.c | 10 + tools/perf/util/intel-pt.c | 95 ++++- tools/perf/util/machine.c | 434 ++++++++++++++++++--- tools/perf/util/metricgroup.c | 60 +-- tools/perf/util/parse-events.l | 1 + tools/perf/util/parse-events.y | 9 + tools/perf/util/pmu.c | 102 +++++ tools/perf/util/pmu.h | 9 + tools/perf/util/record.c | 62 +++ tools/perf/util/s390-cpumcf-kernel.h | 1 + tools/perf/util/s390-cpumsf.c | 11 +- tools/perf/util/sort.c | 2 +- tools/perf/util/sort.h | 2 + tools/perf/util/stat-shadow.c | 2 +- tools/perf/util/symbol.c | 1 + tools/perf/util/synthetic-events.c | 22 +- tools/perf/util/thread-stack.c | 57 +++ tools/perf/util/thread-stack.h | 3 + tools/perf/util/thread.c | 24 ++ tools/perf/util/thread.h | 15 + tools/perf/util/top.h | 1 + tools/perf/util/util.c | 1 + 85 files changed, 1851 insertions(+), 513 deletions(-) create mode 100644 tools/perf/bench/synthesize.c create mode 100755 tools/perf/scripts/python/bin/flamegraph-record create mode 100755 tools/perf/scripts/python/bin/flamegraph-report create mode 100755 tools/perf/scripts/python/flamegraph.py
Test results:
The first ones are container based builds of tools/perf with and without libelf support. Where clang is available, it is also used to build perf with/without libelf, and building with LIBCLANGLLVM=1 (built-in clang) with gcc and clang when clang and its devel libraries are installed.
The objtool and samples/bpf/ builds are disabled now that I'm switching from using the sources in a local volume to fetching them from a http server to build it inside the container, to make it easier to build in a container cluster. Those will come back later.
Several are cross builds, the ones with -x-ARCH and the android one, and those may not have all the features built, due to lack of multi-arch devel packages, available and being used so far on just a few, like debian:experimental-x-{arm64,mipsel}.
The 'perf test' one will perform a variety of tests exercising tools/perf/util/, tools/lib/{bpf,traceevent,etc}, as well as run perf commands with a variety of command line event specifications to then intercept the sys_perf_event syscall to check that the perf_event_attr fields are set up as expected, among a variety of other unit tests.
Then there is the 'make -C tools/perf build-test' ones, that build tools/perf/ with a variety of feature sets, exercising the build with an incomplete set of features as well as with a complete one. It is planned to have it run on each of the containers mentioned above, using some container orchestration infrastructure. Get in contact if interested in helping having this in place.
Ubuntu 19.10 is failing when linking against libllvm, which isn't the default, needs to be investigated, haven't tested with CC=gcc, but should be the same problem:
+ make ARCH= CROSS_COMPILE= EXTRA_CFLAGS= LIBCLANGLLVM=1 -C /git/linux/tools/perf O=/tmp/build/perf CC=clang
.../usr/bin/ld: /usr/lib/llvm-9/lib/libclangAnalysis.a(ExprMutationAnalyzer.cpp.o): in function `clang::ast_matchers::internal::matcher_ignoringImpCasts0Matcher::matches(clang::Expr const&, clang::ast_matchers::internal::ASTMatchFinder*, clang::ast_matchers::internal::BoundNodesTreeBuilder*) const':
(.text._ZNK5clang12ast_matchers8internal32matcher_ignoringImpCasts0Matcher7matchesERKNS_4ExprEPNS1_14ASTMatchFinderEPNS1_21BoundNodesTreeBuilderE[_ZNK5clang12ast_matchers8internal32matcher_ignoringImpCasts0Matcher7matchesERKNS_4ExprEPNS1_14ASTMatchFinderEPNS1_21BoundNodesTreeBuilderE]+0x43): undefined reference to `clang::ast_matchers::internal::DynTypedMatcher::matches(clang::ast_type_traits::DynTypedNode const&, clang::ast_matchers::internal::ASTMatchFinder*, clang::ast_matchers::internal::BoundNodesTreeBuilder*) const' /usr/bin/ld: /usr/lib/llvm-9/lib/libclangAnalysis.a(ExprMutationAnalyzer.cpp.o): in function `clang::ast_matchers::internal::matcher_hasLoopVariable0Matcher::matches(clang::CXXForRangeStmt const&, clang::ast_matchers::internal::ASTMatchFinder*, clang::ast_matchers::internal::BoundNodesTreeBuilder*) const':
(.text._ZNK5clang12ast_matchers8internal31matcher_hasLoopVariable0Matcher7matchesERKNS_15CXXForRangeStmtEPNS1_14ASTMatchFinderEPNS1_21BoundNodesTreeBuilderE[_ZNK5clang12ast_matchers8internal31matcher_hasLoopVariable0Matcher7matchesERKNS_15CXXForRangeStmtEPNS1_14ASTMatchFinderEPNS1_21BoundNodesTreeBuilderE]+0x48): undefined reference to `clang::ast_matchers::internal::DynTypedMatcher::matches(clang::ast_type_traits::DynTypedNode const&, clang::ast_matchers::internal::ASTMatchFinder*, clang::ast_matchers::internal::BoundNodesTreeBuilder*) const'
...
It builds ok with the default set of options.
# export PERF_TARBALL=http://192.168.124.1/perf/perf-5.7.0-rc1.tar.xz # dm 1 alpine:3.4 : Ok gcc (Alpine 5.3.0) 5.3.0, clang version 3.8.0 (tags/RELEASE_380/final) 2 alpine:3.5 : Ok gcc (Alpine 6.2.1) 6.2.1 20160822, clang version 3.8.1 (tags/RELEASE_381/final) 3 alpine:3.6 : Ok gcc (Alpine 6.3.0) 6.3.0, clang version 4.0.0 (tags/RELEASE_400/final) 4 alpine:3.7 : Ok gcc (Alpine 6.4.0) 6.4.0, Alpine clang version 5.0.0 (tags/RELEASE_500/final) (based on LLVM 5.0.0) 5 alpine:3.8 : Ok gcc (Alpine 6.4.0) 6.4.0, Alpine clang version 5.0.1 (tags/RELEASE_501/final) (based on LLVM 5.0.1) 6 alpine:3.9 : Ok gcc (Alpine 8.3.0) 8.3.0, Alpine clang version 5.0.1 (tags/RELEASE_502/final) (based on LLVM 5.0.1) 7 alpine:3.10 : Ok gcc (Alpine 8.3.0) 8.3.0, Alpine clang version 8.0.0 (tags/RELEASE_800/final) (based on LLVM 8.0.0) 8 alpine:3.11 : Ok gcc (Alpine 9.2.0) 9.2.0, Alpine clang version 9.0.0 (https://git.alpinelinux.org/aports f7f0d2c2b8bcd6a5843401a9a702029556492689) (based on LLVM 9.0.0) 9 alpine:edge : Ok gcc (Alpine 9.2.0) 9.2.0, Alpine clang version 9.0.1 (git://git.alpinelinux.org/aports 7c78441134e54efbb34618f457d88c783c913361) (based on LLVM 9.0.1) 10 alt:p8 : Ok x86_64-alt-linux-gcc (GCC) 5.3.1 20151207 (ALT p8 5.3.1-alt3.M80P.1), clang version 3.8.0 (tags/RELEASE_380/final) 11 alt:p9 : Ok x86_64-alt-linux-gcc (GCC) 8.3.1 20190507 (ALT p9 8.3.1-alt5), clang version 7.0.1 12 alt:sisyphus : Ok x86_64-alt-linux-gcc (GCC) 9.2.1 20190827 (ALT Sisyphus 9.2.1-alt2), clang version 7.0.1 13 amazonlinux:1 : Ok gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2), clang version 3.6.2 (tags/RELEASE_362/final) 14 amazonlinux:2 : Ok gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6), clang version 7.0.1 (Amazon Linux 2 7.0.1-1.amzn2.0.2) 15 android-ndk:r12b-arm : Ok arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease) 16 android-ndk:r15c-arm : Ok arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease) 17 centos:5 : Ok gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-55) 18 centos:6 : Ok gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) 19 centos:7 : Ok gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) 20 centos:8 : Ok gcc (GCC) 8.3.1 20190507 (Red Hat 8.3.1-4), clang version 8.0.1 (Red Hat 8.0.1-1.module_el8.1.0+215+a01033fb) 21 clearlinux:latest : Ok gcc (Clear Linux OS for Intel Architecture) 9.2.1 20200214 gcc_9_2_0_release-615-g7866f9ebf1, clang version 9.0.1 22 debian:8 : Ok gcc (Debian 4.9.2-10+deb8u2) 4.9.2, Debian clang version 3.5.0-10 (tags/RELEASE_350/final) (based on LLVM 3.5.0) 23 debian:9 : Ok gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, clang version 3.8.1-24 (tags/RELEASE_381/final) 24 debian:10 : Ok gcc (Debian 8.3.0-6) 8.3.0, clang version 7.0.1-8 (tags/RELEASE_701/final) 25 debian:experimental : Ok gcc (Debian 9.2.1-28) 9.2.1 20200203, clang version 8.0.1-7 (tags/RELEASE_801/final) 26 debian:experimental-x-arm64 : Ok aarch64-linux-gnu-gcc (Debian 8.3.0-19) 8.3.0 27 debian:experimental-x-mips : Ok mips-linux-gnu-gcc (Debian 8.3.0-19) 8.3.0 28 debian:experimental-x-mips64 : Ok mips64-linux-gnuabi64-gcc (Debian 9.2.1-24) 9.2.1 20200117 29 debian:experimental-x-mipsel : Ok mipsel-linux-gnu-gcc (Debian 9.2.1-8) 9.2.1 20190909 30 fedora:20 : Ok gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7) 31 fedora:22 : Ok gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6), clang version 3.5.0 (tags/RELEASE_350/final) 32 fedora:23 : Ok gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6), clang version 3.7.0 (tags/RELEASE_370/final) 33 fedora:24 : Ok gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1), clang version 3.8.1 (tags/RELEASE_381/final) 34 fedora:24-x-ARC-uClibc : Ok arc-linux-gcc (ARCompact ISA Linux uClibc toolchain 2017.09-rc2) 7.1.1 20170710 35 fedora:25 : Ok gcc (GCC) 6.4.1 20170727 (Red Hat 6.4.1-1), clang version 3.9.1 (tags/RELEASE_391/final) 36 fedora:26 : Ok gcc (GCC) 7.3.1 20180130 (Red Hat 7.3.1-2), clang version 4.0.1 (tags/RELEASE_401/final) 37 fedora:27 : Ok gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6), clang version 5.0.2 (tags/RELEASE_502/final) 38 fedora:28 : Ok gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2), clang version 6.0.1 (tags/RELEASE_601/final) 39 fedora:29 : Ok gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2), clang version 7.0.1 (Fedora 7.0.1-6.fc29) 40 fedora:30 : Ok gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1), clang version 8.0.0 (Fedora 8.0.0-3.fc30) 41 fedora:30-x-ARC-glibc : Ok arc-linux-gcc (ARC HS GNU/Linux glibc toolchain 2019.03-rc1) 8.3.1 20190225 42 fedora:30-x-ARC-uClibc : Ok arc-linux-gcc (ARCv2 ISA Linux uClibc toolchain 2019.03-rc1) 8.3.1 20190225 43 fedora:31 : Ok gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1), clang version 9.0.1 (Fedora 9.0.1-2.fc31) 44 fedora:32 : Ok gcc (GCC) 10.0.1 20200216 (Red Hat 10.0.1-0.8), clang version 10.0.0 (Fedora 10.0.0-0.1.rc2.fc32) 45 fedora:rawhide : Ok gcc (GCC) 10.0.1 20200216 (Red Hat 10.0.1-0.8), clang version 10.0.0 (Fedora 10.0.0-0.3.rc2.fc33) 46 gentoo-stage3-amd64:latest : Ok gcc (Gentoo 9.2.0-r2 p3) 9.2.0 47 mageia:5 : Ok gcc (GCC) 4.9.2, clang version 3.5.2 (tags/RELEASE_352/final) 48 mageia:6 : Ok gcc (Mageia 5.5.0-1.mga6) 5.5.0, clang version 3.9.1 (tags/RELEASE_391/final) 49 mageia:7 : Ok gcc (Mageia 8.3.1-0.20190524.1.mga7) 8.3.1 20190524, clang version 8.0.0 (Mageia 8.0.0-1.mga7) 50 manjaro:latest : Ok gcc (GCC) 9.2.0, clang version 9.0.0 (tags/RELEASE_900/final) 51 openmandriva:cooker : Ok gcc (GCC) 10.0.0 20200216 (OpenMandriva), clang version 10.0.0 52 opensuse:15.0 : Ok gcc (SUSE Linux) 7.4.1 20190424 [gcc-7-branch revision 270538], clang version 5.0.1 (tags/RELEASE_501/final 312548) 53 opensuse:15.1 : Ok gcc (SUSE Linux) 7.5.0, clang version 7.0.1 (tags/RELEASE_701/final 349238) 54 opensuse:15.2 : Ok gcc (SUSE Linux) 7.5.0, clang version 7.0.1 (tags/RELEASE_701/final 349238) 55 opensuse:42.3 : Ok gcc (SUSE Linux) 4.8.5, clang version 3.8.0 (tags/RELEASE_380/final 262553) 56 opensuse:tumbleweed : Ok gcc (SUSE Linux) 9.2.1 20200128 [revision 83f65674e78d97d27537361de1a9d74067ff228d], clang version 9.0.1 57 oraclelinux:6 : Ok gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23.0.1) 58 oraclelinux:7 : Ok gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39.0.3) 59 oraclelinux:8 : Ok gcc (GCC) 8.3.1 20190507 (Red Hat 8.3.1-4.5.0.5), clang version 8.0.1 (Red Hat 8.0.1-1.0.1.module+el8.1.0+5428+345cee14) 60 ubuntu:12.04 : Ok gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3, Ubuntu clang version 3.0-6ubuntu3 (tags/RELEASE_30/final) (based on LLVM 3.0) 61 ubuntu:14.04 : Ok gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4 62 ubuntu:16.04 : Ok gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609, clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final) 63 ubuntu:16.04-x-arm : Ok arm-linux-gnueabihf-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 64 ubuntu:16.04-x-arm64 : Ok aarch64-linux-gnu-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 65 ubuntu:16.04-x-powerpc : Ok powerpc-linux-gnu-gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 66 ubuntu:16.04-x-powerpc64 : Ok powerpc64-linux-gnu-gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 67 ubuntu:16.04-x-powerpc64el : Ok powerpc64le-linux-gnu-gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 68 ubuntu:16.04-x-s390 : Ok s390x-linux-gnu-gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609 69 ubuntu:18.04 : Ok gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0, clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final) 70 ubuntu:18.04-x-arm : Ok arm-linux-gnueabihf-gcc (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1) 7.4.0 71 ubuntu:18.04-x-arm64 : Ok aarch64-linux-gnu-gcc (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1) 7.4.0 72 ubuntu:18.04-x-m68k : Ok m68k-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 73 ubuntu:18.04-x-powerpc : Ok powerpc-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 74 ubuntu:18.04-x-powerpc64 : Ok powerpc64-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 75 ubuntu:18.04-x-powerpc64el : Ok powerpc64le-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 76 ubuntu:18.04-x-riscv64 : Ok riscv64-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 77 ubuntu:18.04-x-s390 : Ok s390x-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 78 ubuntu:18.04-x-sh4 : Ok sh4-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 79 ubuntu:18.04-x-sparc64 : Ok sparc64-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0 80 ubuntu:18.10 : Ok gcc (Ubuntu 8.3.0-6ubuntu1~18.10.1) 8.3.0, clang version 7.0.0-3 (tags/RELEASE_700/final) 81 ubuntu:19.04 : Ok gcc (Ubuntu 8.3.0-6ubuntu1) 8.3.0, clang version 8.0.0-3 (tags/RELEASE_800/final) 82 ubuntu:19.04-x-alpha : Ok alpha-linux-gnu-gcc (Ubuntu 8.3.0-6ubuntu1) 8.3.0 83 ubuntu:19.04-x-arm64 : Ok aarch64-linux-gnu-gcc (Ubuntu/Linaro 8.3.0-6ubuntu1) 8.3.0 84 ubuntu:19.04-x-hppa : Ok hppa-linux-gnu-gcc (Ubuntu 8.3.0-6ubuntu1) 8.3.0 85 ubuntu:19.10 : FAIL gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008, clang version 9.0.0-2 (tags/RELEASE_900/final) 86 ubuntu:20.04 : Ok gcc (Ubuntu 9.3.0-8ubuntu1) 9.3.0, clang version 10.0.0-1ubuntu1 #
# uname -a
Linux five 5.5.17-200.fc31.x86_64 #1 SMP Mon Apr 13 15:29:42 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
# git log --oneline -1
12e89e65f446 perf hist: Add fast path for duplicate entries check
# perf version --build-options
perf version 5.7.rc1.g12e89e65f446
dwarf: [ on ] # HAVE_DWARF_SUPPORT
dwarf_getlocations: [ on ] # HAVE_DWARF_GETLOCATIONS_SUPPORT
glibc: [ on ] # HAVE_GLIBC_SUPPORT
gtk2: [ on ] # HAVE_GTK2_SUPPORT
syscall_table: [ on ] # HAVE_SYSCALL_TABLE_SUPPORT
libbfd: [ on ] # HAVE_LIBBFD_SUPPORT
libelf: [ on ] # HAVE_LIBELF_SUPPORT
libnuma: [ on ] # HAVE_LIBNUMA_SUPPORT
numa_num_possible_cpus: [ on ] # HAVE_LIBNUMA_SUPPORT
libperl: [ on ] # HAVE_LIBPERL_SUPPORT
libpython: [ on ] # HAVE_LIBPYTHON_SUPPORT
libslang: [ on ] # HAVE_SLANG_SUPPORT
libcrypto: [ on ] # HAVE_LIBCRYPTO_SUPPORT
libunwind: [ on ] # HAVE_LIBUNWIND_SUPPORT
libdw-dwarf-unwind: [ on ] # HAVE_DWARF_SUPPORT
zlib: [ on ] # HAVE_ZLIB_SUPPORT
lzma: [ on ] # HAVE_LZMA_SUPPORT
get_cpuid: [ on ] # HAVE_AUXTRACE_SUPPORT
bpf: [ on ] # HAVE_LIBBPF_SUPPORT
aio: [ on ] # HAVE_AIO_SUPPORT
zstd: [ on ] # HAVE_ZSTD_SUPPORT
# perf test
1: vmlinux symtab matches kallsyms : Ok
2: Detect openat syscall event : Ok
3: Detect openat syscall event on all cpus : Ok
4: Read samples using the mmap interface : Ok
5: Test data source output : Ok
6: Parse event definition strings : Ok
7: Simple expression parser : Ok
8: PERF_RECORD_* events & perf_sample fields : Ok
9: Parse perf pmu format : Ok
10: PMU events : Ok
11: DSO data read : Ok
12: DSO data cache : Ok
13: DSO data reopen : Ok
14: Roundtrip evsel->name : Ok
15: Parse sched tracepoints fields : Ok
16: syscalls:sys_enter_openat event fields : Ok
17: Setup struct perf_event_attr : Ok
18: Match and link multiple hists : Ok
19: 'import perf' in python : Ok
20: Breakpoint overflow signal handler : Ok
21: Breakpoint overflow sampling : Ok
22: Breakpoint accounting : Ok
23: Watchpoint :
23.1: Read Only Watchpoint : Skip
23.2: Write Only Watchpoint : Ok
23.3: Read / Write Watchpoint : Ok
23.4: Modify Watchpoint : Ok
24: Number of exit events of a simple workload : Ok
25: Software clock events period values : Ok
26: Object code reading : Ok
27: Sample parsing : Ok
28: Use a dummy software event to keep tracking : Ok
29: Parse with no sample_id_all bit set : Ok
30: Filter hist entries : Ok
31: Lookup mmap thread : Ok
32: Share thread maps : Ok
33: Sort output of hist entries : Ok
34: Cumulate child hist entries : Ok
35: Track with sched_switch : Ok
36: Filter fds with revents mask in a fdarray : Ok
37: Add fd to a fdarray, making it autogrow : Ok
38: kmod_path__parse : Ok
39: Thread map : Ok
40: LLVM search and compile :
40.1: Basic BPF llvm compile : Ok
40.2: kbuild searching : Ok
40.3: Compile source for BPF prologue generation : Ok
40.4: Compile source for BPF relocation : Ok
41: Session topology : Ok
42: BPF filter :
42.1: Basic BPF filtering : Ok
42.2: BPF pinning : Ok
42.3: BPF prologue generation : Ok
42.4: BPF relocation checker : Ok
43: Synthesize thread map : Ok
44: Remove thread map : Ok
45: Synthesize cpu map : Ok
46: Synthesize stat config : Ok
47: Synthesize stat : Ok
48: Synthesize stat round : Ok
49: Synthesize attr update : Ok
50: Event times : Ok
51: Read backward ring buffer : Ok
52: Print cpu map : Ok
53: Merge cpu map : Ok
54: Probe SDT events : Ok
55: is_printable_array : Ok
56: Print bitmap : Ok
57: perf hooks : Ok
58: builtin clang support : Skip (not compiled in)
59: unit_number__scnprintf : Ok
60: mem2node : Ok
61: time utils : Ok
62: Test jit_write_elf : Ok
63: maps__merge_in : Ok
64: x86 rdpmc : Ok
65: Convert perf time to TSC : Ok
66: DWARF unwind : Ok
67: x86 instruction decoder - new instructions : Ok
68: Intel PT packet decoder : Ok
69: x86 bp modify : Ok
70: probe libc's inet_pton & backtrace it with ping : Ok
71: Use vfs_getname probe to get syscall args filenames : Ok
72: Check open filename arg using perf trace + vfs_getname: Ok
73: Zstd perf.data compression/decompression : Ok
74: Add vfs_getname probe to get syscall args filenames : Ok
# $ git log --oneline -1 ; make -C tools/perf build-test
12e89e65f446 (HEAD -> perf/core, five/perf/core) perf hist: Add fast path for duplicate entries check
make: Entering directory '/home/acme/git/perf/tools/perf'
- tarpkg: ./tests/perf-targz-src-pkg .
make_no_libbpf_O: make NO_LIBBPF=1
make_no_ui_O: make NO_NEWT=1 NO_SLANG=1 NO_GTK2=1
make_no_newt_O: make NO_NEWT=1
make_with_clangllvm_O: make LIBCLANGLLVM=1
make_no_slang_O: make NO_SLANG=1
make_static_O: make LDFLAGS=-static NO_PERF_READ_VDSO32=1 NO_PERF_READ_VDSOX32=1 NO_JVMTI=1
make_no_backtrace_O: make NO_BACKTRACE=1
make_no_scripts_O: make NO_LIBPYTHON=1 NO_LIBPERL=1
make_install_prefix_O: make install prefix=/tmp/krava
make_no_demangle_O: make NO_DEMANGLE=1
make_no_libdw_dwarf_unwind_O: make NO_LIBDW_DWARF_UNWIND=1
make_perf_o_O: make perf.o
make_cscope_O: make cscope
make_no_libunwind_O: make NO_LIBUNWIND=1
make_install_prefix_slash_O: make install prefix=/tmp/krava/
make_util_pmu_bison_o_O: make util/pmu-bison.o
make_clean_all_O: make clean all
make_no_libnuma_O: make NO_LIBNUMA=1
make_minimal_O: make NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1 NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1 NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1 NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1 NO_LIBCRYPTO=1 NO_SDT=1 NO_JVMTI=1 NO_LIBZSTD=1 NO_LIBCAP=1
make_no_gtk2_O: make NO_GTK2=1
make_tags_O: make tags
make_no_libaudit_O: make NO_LIBAUDIT=1
make_help_O: make help
make_no_libperl_O: make NO_LIBPERL=1
make_install_O: make install
make_no_libelf_O: make NO_LIBELF=1
make_pure_O: make
make_install_bin_O: make install-bin
make_debug_O: make DEBUG=1
make_with_babeltrace_O: make LIBBABELTRACE=1
make_doc_O: make doc
make_no_auxtrace_O: make NO_AUXTRACE=1
make_no_libpython_O: make NO_LIBPYTHON=1
make_util_map_o_O: make util/map.o
make_no_libbionic_O: make NO_LIBBIONIC=1
OK
make: Leaving directory '/home/acme/git/perf/tools/perf'
$Arnaldo Carvalho de Melo
maintainerPATCH 1/60 perf stat: Honour --timeout for forked workloads
From: Arnaldo Carvalho de Melo <[email protected]>
When --timeout is used and a workload is specified to be started by 'perf stat', i.e.
$ perf stat --timeout 1000 sleep 1h
The --timeout wasn't being honoured, i.e. the workload, 'sleep 1h' in the above example, should be terminated after 1000ms, but it wasn't, 'perf stat' was waiting for it to finish.
Fix it by sending a SIGTERM when the timeout expires.
Now it works:
# perf stat -e cycles --timeout 1234 sleep 1h sleep: Terminated
Performance counter stats for 'sleep 1h':
1,066,692 cycles
1.234314838 seconds time elapsed
0.000750000 seconds user
0.000000000 seconds sys#
Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=207243
--- tools/perf/builtin-stat.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/builtin-stat.c +4 -1
@@ -686,8 +686,11 @@ static int __run_perf_stat(int argc, const char **argv, int run_idx)break ; } } -if (child_pid !=-1 ) +if (child_pid !=-1 ) { +if (timeout) +kill (child_pid,SIGTERM );wait4 (child_pid, &status,0 , &stat_config.ru_data); + }if (workload_exec_errno) {const char *emsg =str_error_r (workload_exec_errno, msg,sizeof (msg));
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 2/60 perf tools: Synthesize bpf_trampoline/dispatcher ksymbol event
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Synthesize bpf images (trampolines/dispatchers) on start, as ksymbol events from /proc/kallsyms. Having this perf can recognize samples from those images and perf report and top shows them correctly.
The rest of the ksymbol handling is already in place from for the bpf programs monitoring, so only the initial state was needed.
perf report output:
# Overhead Command Shared Object Symbol
12.37% test_progs [kernel.vmlinux] [k] entry_SYSCALL_64
11.80% test_progs [kernel.vmlinux] [k] syscall_return_via_sysret
9.63% test_progs bpf_prog_bcf7977d3b93787c_prog2 [k] bpf_prog_bcf7977d3b93787c_prog2
6.90% test_progs bpf_trampoline_24456 [k] bpf_trampoline_24456
6.36% test_progs [kernel.vmlinux] [k] memcpy_ermsCommitter notes:
Use scnprintf() instead of strncpy() to overcome this on fedora:32, rawhide and OpenMandriva Cooker:
CC /tmp/build/perf/util/bpf-event.o
In file included from /usr/include/string.h:495,
from /git/linux/tools/lib/bpf/libbpf_common.h:12,
from /git/linux/tools/lib/bpf/bpf.h:31,
from util/bpf-event.c:4:
In function 'strncpy',
inlined from 'process_bpf_image' at util/bpf-event.c:323:2,
inlined from 'kallsyms_process_symbol' at util/bpf-event.c:358:9:
/usr/include/bits/string_fortified.h:106:10: error: '__builtin_strncpy' specified bound 256 equals destination size [-Werror=stringop-truncation]
106 | return __builtin___strncpy_chk (__dest, __src, __len, __bos (__dest));
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~cc1: all warnings being treated as errors
--- tools/perf/util/bpf-event.c | 93 +++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+)
tools/perf/util/bpf-event.c +93 -0
@@ -6,6 +6,9 @@#include <bpf/libbpf.h> #include <linux/btf.h> #include <linux/err.h> +#include <linux/string.h> +#include <internal/lib.h> +#include <symbol/kallsyms.h> #include "bpf-event.h" #include "debug.h" #include "dso.h"
@@ -290,11 +293,82 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_session *session,return err ?-1 :0 ; } +struct kallsyms_parse { +union perf_event *event; +perf_event__handler_t process; +struct machine *machine; +struct perf_tool *tool; +}; + +static int +process_bpf_image (char *name,u64 addr,struct kallsyms_parse *data) +{ +struct machine *machine = data->machine; +union perf_event *event = data->event; +struct perf_record_ksymbol *ksymbol; +int len; + + ksymbol = &event->ksymbol; + + *ksymbol = (struct perf_record_ksymbol ) { + .header = { + .type =PERF_RECORD_KSYMBOL , + .size = offsetof(struct perf_record_ksymbol , name), + }, + .addr = addr, + .len = page_size, + .ksym_type =PERF_RECORD_KSYMBOL_TYPE_BPF , + .flags =0 , + }; + + len =scnprintf (ksymbol->name,KSYM_NAME_LEN ,"%s" , name); + ksymbol->header.size +=PERF_ALIGN (len +1 ,sizeof (u64)); +memset ((void *) event + event->header.size,0 , machine->id_hdr_size); + event->header.size += machine->id_hdr_size; + +return perf_tool__process_synth_event (data->tool, event, machine, + data->process); +} + +static int +kallsyms_process_symbol (void *data,const char *_name, +char type__maybe_unused ,u64 start) +{ +char disp[KSYM_NAME_LEN ]; +char *module, *name; +unsigned long id; +int err =0 ; + + module =strchr (_name,'\t' ); +if (!module) +return 0 ; + +/* We are going after [bpf] module ... */ +if (strcmp (module +1 ,"[bpf]" )) +return 0 ; + + name =memdup (_name, (module - _name) +1 ); +if (!name) +return -ENOMEM ; + + name[module - _name] =0 ; + +/* .. and only for trampolines and dispatchers */ +if ((sscanf (name,"bpf_trampoline_%lu" , &id) ==1 ) || + (sscanf (name,"bpf_dispatcher_%s" , disp) ==1 )) + err =process_bpf_image (name, start, data); + +free (name); +return err; +} +int perf_event__synthesize_bpf_events (struct perf_session *session,perf_event__handler_t process,struct machine *machine,struct record_opts *opts) { +const char *kallsyms_filename ="/proc/kallsyms" ; +struct kallsyms_parse arg;union perf_event *event;__u32 id =0 ;int err;
@@ -303,6 +377,8 @@ int perf_event__synthesize_bpf_events(struct perf_session *session, event =malloc (sizeof (event->bpf) +KSYM_NAME_LEN + machine->id_hdr_size);if (!event)return -1 ; + +/* Synthesize all the bpf programs in system. */ while (true) { err =bpf_prog_get_next_id (id, &id);if (err) {
@@ -335,6 +411,23 @@ int perf_event__synthesize_bpf_events(struct perf_session *session,break ; } } + +/* Synthesize all the bpf images - trampolines/dispatchers. */ +if (symbol_conf.kallsyms_name !=NULL ) + kallsyms_filename = symbol_conf.kallsyms_name; + + arg = (struct kallsyms_parse ) { + .event = event, + .process = process, + .machine = machine, + .tool = session->tool, + }; + +if (kallsyms__parse (kallsyms_filename, &arg, kallsyms_process_symbol)) { +pr_err ("%s: failed to synthesize bpf images: %s\n" , + __func__,strerror (errno)); + } +free (event);return err; }
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 3/60 perf machine: Set ksymbol dso as loaded on arrival
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
There's no special load action for ksymbol data on map__load/dso__load action, where the kernel is getting loaded. It only gets confused with kernel kallsyms/vmlinux load for bpf object, which fails and could mess up with the map.
Disabling any further load of the map for ksymbol related dso/map.
--- tools/perf/util/machine.c | 1 + 1 file changed, 1 insertion(+)
tools/perf/util/machine.c +1 -0
@@ -759,6 +759,7 @@ static int machine__process_ksymbol_register(struct machine *machine, map->start = event->ksymbol.addr; map->end = map->start + event->ksymbol.len;maps__insert (&machine->kmaps, map); +dso__set_loaded (dso); } sym = symbol__new(map->map_ip(map, map->start),
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 4/60 perf annotate: Add basic support for bpf_image
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Add the DSO_BINARY_TYPE__BPF_IMAGE dso binary type to recognize BPF images that carry trampoline or dispatcher.
Upcoming patches will add support to read the image data, store it within the BPF feature in perf.data and display it for annotation purposes.
Currently we only display following message:
# ./perf annotate bpf_trampoline_24456 --stdio
Percent | Source code & Disassembly of . for cycles (504 ...
--------------------------------------------------------------- ...
: to be implemented--- tools/perf/util/annotate.c | 20 ++++++++++++++++++++ tools/perf/util/dso.c | 1 + tools/perf/util/dso.h | 1 + tools/perf/util/machine.c | 11 +++++++++++ tools/perf/util/symbol.c | 1 + 5 files changed, 34 insertions(+)
tools/perf/util/annotate.c +20 -0
@@ -1821,6 +1821,24 @@ static int symbol__disassemble_bpf(struct symbol *sym __maybe_unused, }#endif // defined(HAVE_LIBBFD_SUPPORT) && defined(HAVE_LIBBPF_SUPPORT) +static int +symbol__disassemble_bpf_image (struct symbol *sym, +struct annotate_args *args) +{ +struct annotation *notes =symbol__annotation (sym); +struct disasm_line *dl; + + args->offset =-1 ; + args->line =strdup ("to be implemented" ); + args->line_nr =0 ; + dl =disasm_line__new (args); +if (dl) +annotation_line__add (&dl->al, ¬es->src->source); + +free (args->line); +return 0 ; +} +/* * Possibly create a new version of line with tabs expanded. Returns the * existing or new line, storage is updated if a new line is allocated. If
@@ -1920,6 +1938,8 @@ static int symbol__disassemble(struct symbol *sym, struct annotate_args *args)if (dso->binary_type ==DSO_BINARY_TYPE__BPF_PROG_INFO ) {return symbol__disassemble_bpf (sym, args); + }else if (dso->binary_type ==DSO_BINARY_TYPE__BPF_IMAGE ) { +return symbol__disassemble_bpf_image (sym, args); }else if (dso__is_kcore (dso)) { kce.kcore_filename = symfs_filename; kce.addr =map__rip_2objdump (map, sym->start);
tools/perf/util/dso.c +1 -0
@@ -191,6 +191,7 @@ int dso__read_binary_type_filename(const struct dso *dso,case DSO_BINARY_TYPE__GUEST_KALLSYMS :case DSO_BINARY_TYPE__JAVA_JIT :case DSO_BINARY_TYPE__BPF_PROG_INFO : +case DSO_BINARY_TYPE__BPF_IMAGE :case DSO_BINARY_TYPE__NOT_FOUND : ret =-1 ;break ;
tools/perf/util/dso.h +1 -0
@@ -40,6 +40,7 @@ enum dso_binary_type {DSO_BINARY_TYPE__GUEST_KCORE ,DSO_BINARY_TYPE__OPENEMBEDDED_DEBUGINFO ,DSO_BINARY_TYPE__BPF_PROG_INFO , +DSO_BINARY_TYPE__BPF_IMAGE , DSO_BINARY_TYPE__NOT_FOUND, };
tools/perf/util/machine.c +11 -0
@@ -736,6 +736,12 @@ int machine__process_switch_event(struct machine *machine __maybe_unused,return 0 ; } +static int is_bpf_image (const char *name) +{ +return strncmp (name,"bpf_trampoline_" ,sizeof ("bpf_trampoline_" ) -1 ) || +strncmp (name,"bpf_dispatcher_" ,sizeof ("bpf_dispatcher_" ) -1 ); +} +static int machine__process_ksymbol_register (struct machine *machine,union perf_event *event,struct perf_sample *sample__maybe_unused )
@@ -760,6 +766,11 @@ static int machine__process_ksymbol_register(struct machine *machine, map->end = map->start + event->ksymbol.len;maps__insert (&machine->kmaps, map);dso__set_loaded (dso); + +if (is_bpf_image (event->ksymbol.name)) { + dso->binary_type =DSO_BINARY_TYPE__BPF_IMAGE ; +dso__set_long_name (dso,"" , false); + } } sym = symbol__new(map->map_ip(map, map->start),
tools/perf/util/symbol.c +1 -0
@@ -1544,6 +1544,7 @@ static bool dso__is_compatible_symtab_type(struct dso *dso, bool kmod,return true;case DSO_BINARY_TYPE__BPF_PROG_INFO : +case DSO_BINARY_TYPE__BPF_IMAGE :case DSO_BINARY_TYPE__NOT_FOUND :default :return false;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 5/60 capabilities: Introduce CAP_PERFMON to kernel and user space
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Introduce the CAP_PERFMON capability designed to secure system performance monitoring and observability operations so that CAP_PERFMON can assist CAP_SYS_ADMIN capability in its governing role for performance monitoring and observability subsystems.
CAP_PERFMON hardens system security and integrity during performance monitoring and observability operations by decreasing attack surface that is available to a CAP_SYS_ADMIN privileged process [2]. Providing the access to system performance monitoring and observability operations under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes the operation more secure.
Thus, CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e:
2.2.2.39 principle of least privilege: A security design principle that
states that a process or program be granted only those privileges
(e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
CAP_PERFMON meets the demand to secure system performance monitoring and observability operations for adoption in security sensitive, restricted, multiuser production environments (e.g. HPC clusters, cloud and virtual compute environments), where root or CAP_SYS_ADMIN credentials are not available to mass users of a system, and securely unblocks applicability and scalability of system performance monitoring and observability operations beyond root and CAP_SYS_ADMIN use cases.
CAP_PERFMON takes over CAP_SYS_ADMIN credentials related to system performance monitoring and observability operations and balances amount of CAP_SYS_ADMIN credentials following the recommendations in the capabilities man page [1] for CAP_SYS_ADMIN: "Note: this capability is overloaded; see Notes to kernel developers, below." For backward compatibility reasons access to system performance monitoring and observability subsystems of the kernel remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN capability usage for secure system performance monitoring and observability operations is discouraged with respect to the designed CAP_PERFMON capability.
Although the software running under CAP_PERFMON can not ensure avoidance of related hardware issues, the software can still mitigate these issues following the official hardware issues mitigation procedure [2]. The bugs in the software itself can be fixed following the standard kernel development process [3] to maintain and harden security of system performance monitoring and observability operations.
[1] http://man7.org/linux/man-pages/man7/capabilities.7.html
[2] https://www.kernel.org/doc/html/latest/process/embargoed-hardware-issues.html
[3] https://www.kernel.org/doc/html/latest/admin-guide/security-bugs.html
--- include/linux/capability.h | 4 ++++ include/uapi/linux/capability.h | 8 +++++++- security/selinux/include/classmap.h | 4 ++-- 3 files changed, 13 insertions(+), 3 deletions(-)
include/linux/capability.h +4 -0
@@ -251,6 +251,10 @@ extern bool privileged_wrt_inode_uidgid(struct user_namespace *ns, const structextern bool capable_wrt_inode_uidgid (const struct inode *inode,int cap);extern bool file_ns_capable (const struct file *file,struct user_namespace *ns,int cap);extern bool ptracer_capable (struct task_struct *tsk,struct user_namespace *ns); +static inline bool perfmon_capable (void ) +{ +return capable (CAP_PERFMON ) ||capable (CAP_SYS_ADMIN ); +}/* audit system wants to get cap info from files as well */ extern int get_vfs_caps_from_disk (const struct dentry *dentry,struct cpu_vfs_cap_data *cpu_caps);
include/uapi/linux/capability.h +7 -1
@@ -367,8 +367,14 @@ struct vfs_ns_cap_data {#define CAP_AUDIT_READ 37 +/* +* Allow system performance and observability privileged operations +* using perf_events, i915_perf and other kernel subsystems +*/ + +#define CAP_PERFMON 38 -#define CAP_LAST_CAP CAP_AUDIT_READ +#define CAP_LAST_CAP CAP_PERFMON#define cap_valid (x) ((x) >= 0 && (x) <= CAP_LAST_CAP)
security/selinux/include/classmap.h +2 -2
@@ -27,9 +27,9 @@ "audit_control", "setfcap"#define COMMON_CAP2_PERMS "mac_override", "mac_admin", "syslog", \ - "wake_alarm", "block_suspend", "audit_read" + "wake_alarm", "block_suspend", "audit_read", "perfmon" -#if CAP_LAST_CAP >CAP_AUDIT_READ +#if CAP_LAST_CAP >CAP_PERFMON #error New capability defined, please update COMMON_CAP2_PERMS.#endif
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 6/60 perf/core: Open access to the core for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to monitoring of kernel code, CPUs, tracepoints and namespaces data for a CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons the access to perf_events subsystem remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure perf_events monitoring is discouraged with respect to CAP_PERFMON capability.
--- include/linux/perf_event.h | 6 +++--- kernel/events/core.c | 2 +- 2 files changed, 4 insertions(+), 4 deletions(-)
include/linux/perf_event.h +3 -3
@@ -1305,7 +1305,7 @@ static inline int perf_is_paranoid(void)static inline int perf_allow_kernel (struct perf_event_attr *attr) { -if (sysctl_perf_event_paranoid >1 && !capable (CAP_SYS_ADMIN )) +if (sysctl_perf_event_paranoid >1 && !perfmon_capable ())return -EACCES ;return security_perf_event_open (attr,PERF_SECURITY_KERNEL );
@@ -1313,7 +1313,7 @@ static inline int perf_allow_kernel(struct perf_event_attr *attr)static inline int perf_allow_cpu (struct perf_event_attr *attr) { -if (sysctl_perf_event_paranoid >0 && !capable (CAP_SYS_ADMIN )) +if (sysctl_perf_event_paranoid >0 && !perfmon_capable ())return -EACCES ;return security_perf_event_open (attr,PERF_SECURITY_CPU );
@@ -1321,7 +1321,7 @@ static inline int perf_allow_cpu(struct perf_event_attr *attr)static inline int perf_allow_tracepoint (struct perf_event_attr *attr) { -if (sysctl_perf_event_paranoid >-1 && !capable (CAP_SYS_ADMIN )) +if (sysctl_perf_event_paranoid >-1 && !perfmon_capable ())return -EPERM ;return security_perf_event_open (attr,PERF_SECURITY_TRACEPOINT );
kernel/events/core.c +1 -1
@@ -11504,7 +11504,7 @@ SYSCALL_DEFINE5(perf_event_open, }if (attr.namespaces) { -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EACCES ; }
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 7/60 perf/core: open access to probes for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to monitoring via kprobes and uprobes and eBPF tracing for CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
perf kprobes and uprobes are used by ftrace and eBPF. perf probe uses ftrace to define new kprobe events, and those events are treated as tracepoint events. eBPF defines new probes via perf_event_open interface and then the probes are used in eBPF tracing.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to perf_events subsystem remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure perf_events monitoring is discouraged with respect to CAP_PERFMON capability.
--- kernel/events/core.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/events/core.c +2 -2
@@ -9397,7 +9397,7 @@ static int perf_kprobe_event_init(struct perf_event *event)if (event->attr.type != perf_kprobe.type)return -ENOENT ; -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EACCES ;/*
@@ -9457,7 +9457,7 @@ static int perf_uprobe_event_init(struct perf_event *event)if (event->attr.type != perf_uprobe.type)return -ENOENT ; -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EACCES ;/*
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 8/60 perf tools: Support CAP_PERFMON capability
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Extend error messages to mention CAP_PERFMON capability as an option to substitute CAP_SYS_ADMIN capability for secure system performance monitoring and observability operations. Make perf_event_paranoid_check() and __cmd_ftrace() to be aware of CAP_PERFMON capability.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to perf_events subsystem remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure perf_events monitoring is discouraged with respect to CAP_PERFMON capability.
Committer testing:
Using a libcap with this patch:
diff --git a/libcap/include/uapi/linux/capability.h b/libcap/include/uapi/linux/capability.h
index 78b2fd4c8a95..89b5b0279b60 100644
--- a/libcap/include/uapi/linux/capability.h
+++ b/libcap/include/uapi/linux/capability.h
@@ -366,8 +366,9 @@ struct vfs_ns_cap_data {#define CAP_AUDIT_READ 37
+#define CAP_PERFMON 38
-#define CAP_LAST_CAP CAP_AUDIT_READ
+#define CAP_LAST_CAP CAP_PERFMON
#define cap_valid(x) ((x) >= 0 && (x) <= CAP_LAST_CAP)
Note that using '38' in place of 'cap_perfmon' works to some degree with an old libcap, its only when cap_get_flag() is called that libcap performs an error check based on the maximum value known for capabilities that it will fail.
This makes determining the default of perf_event_attr.exclude_kernel to fail, as it can't determine if CAP_PERFMON is in place.
Using 'perf top -e cycles' avoids the default check and sets perf_event_attr.exclude_kernel to 1.
As root, with a libcap supporting CAP_PERFMON:
# groupadd perf_users # adduser perf -g perf_users # mkdir ~perf/bin # cp ~acme/bin/perf ~perf/bin/ # chgrp perf_users ~perf/bin/perf # setcap "cap_perfmon,cap_sys_ptrace,cap_syslog=ep" ~perf/bin/perf # getcap ~perf/bin/perf /home/perf/bin/perf = cap_sys_ptrace,cap_syslog,cap_perfmon+ep # ls -la ~perf/bin/perf -rwxr-xr-x. 1 root perf_users 16968552 Apr 9 13:10 /home/perf/bin/perf
As the 'perf' user in the 'perf_users' group:
$ perf top -a --stdio Error: Failed to mmap with 1 (Operation not permitted) $
Either add the cap_ipc_lock capability to the perf binary or reduce the ring buffer size to some smaller value:
$ perf top -m10 -a --stdio rounding mmap pages size to 64K (16 pages) Error: Failed to mmap with 1 (Operation not permitted) $ perf top -m4 -a --stdio Error: Failed to mmap with 1 (Operation not permitted) $ perf top -m2 -a --stdio PerfTop: 762 irqs/sec kernel:49.7% exact: 100.0% lost: 0/0 drop: 0/0 [4000Hz cycles], (all, 4 CPUs) ------------------------------------------------------------------------------------------------------
9.83% perf [.] __symbols__insert
8.58% perf [.] rb_next
5.91% [kernel] [k] module_get_kallsym
5.66% [kernel] [k] kallsyms_expand_symbol.constprop.0
3.98% libc-2.29.so [.] __GI_____strtoull_l_internal
3.66% perf [.] rb_insert_color
2.34% [kernel] [k] vsnprintf
2.30% [kernel] [k] string_nocheck
2.16% libc-2.29.so [.] _IO_getdelim
2.15% [kernel] [k] number
2.13% [kernel] [k] format_decode
1.58% libc-2.29.so [.] _IO_feof
1.52% libc-2.29.so [.] __strcmp_avx2
1.50% perf [.] rb_set_parent_color
1.47% libc-2.29.so [.] __libc_calloc
1.24% [kernel] [k] do_syscall_64
1.17% [kernel] [k] __x86_indirect_thunk_rax $ perf record -a sleep 1
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.552 MB perf.data (74 samples) ]
$ perf evlist
cycles
$ perf evlist -v
cycles: size: 120, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|CPU|PERIOD, read_format: ID, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1, ksymbol: 1, bpf_event: 1
$ perf report | head -20
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 74 of event 'cycles'
# Event count (approx.): 15694834
#
# Overhead Command Shared Object Symbol
# ........ ............... .......................... ......................................
#
19.62% perf [kernel.vmlinux] [k] strnlen_user
13.88% swapper [kernel.vmlinux] [k] intel_idle
13.83% ksoftirqd/0 [kernel.vmlinux] [k] pfifo_fast_dequeue
13.51% swapper [kernel.vmlinux] [k] kmem_cache_free
6.31% gnome-shell [kernel.vmlinux] [k] kmem_cache_free
5.66% kworker/u8:3+ix [kernel.vmlinux] [k] delay_tsc
4.42% perf [kernel.vmlinux] [k] __set_cpus_allowed_ptr
3.45% kworker/2:1-eve [kernel.vmlinux] [k] shmem_truncate_range
2.29% gnome-shell libgobject-2.0.so.0.6000.7 [.] g_closure_ref
$--- tools/perf/builtin-ftrace.c | 5 +++-- tools/perf/design.txt | 3 ++- tools/perf/util/cap.h | 4 ++++ tools/perf/util/evsel.c | 10 +++++----- tools/perf/util/util.c | 1 + 5 files changed, 15 insertions(+), 8 deletions(-)
tools/perf/builtin-ftrace.c +3 -2
@@ -284,10 +284,11 @@ static int __cmd_ftrace(struct perf_ftrace *ftrace, int argc, const char **argv) .events = POLLIN, }; - if (!perf_cap__capable(CAP_SYS_ADMIN)) { + if (!(perf_cap__capable(CAP_PERFMON) || + perf_cap__capable(CAP_SYS_ADMIN))) { pr_err("ftrace only works for %s!\n", #ifdef HAVE_LIBCAP_SUPPORT - "users with the SYS_ADMIN capability" + "users with the CAP_PERFMON or CAP_SYS_ADMIN capability" #else "root" #endif
tools/perf/design.txt +2 -1
@@ -258,7 +258,8 @@ gets schedule to. Per task counters can be created by any user, for their own tasks. A 'pid == -1' and 'cpu == x' counter is a per CPU counter that counts -all events on CPU-x. Per CPU counters need CAP_SYS_ADMIN privilege. +all events on CPU-x. Per CPU counters need CAP_PERFMON or CAP_SYS_ADMIN +privilege. The 'flags' parameter is currently unused and must be zero.
tools/perf/util/cap.h +4 -0
@@ -29,4 +29,8 @@ static inline bool perf_cap__capable(int cap __maybe_unused)#define CAP_SYSLOG 34#endif +#ifndef CAP_PERFMON +#define CAP_PERFMON 38 +#endif +#endif /* __PERF_CAP_H */
tools/perf/util/evsel.c +5 -5
@@ -2523,14 +2523,14 @@ int perf_evsel__open_strerror(struct evsel *evsel, struct target *target,"You may not have permission to collect %sstats.\n\n" "Consider tweaking /proc/sys/kernel/perf_event_paranoid,\n" "which controls use of the performance events system by\n" -"unprivileged users (without CAP_SYS_ADMIN).\n\n" +"unprivileged users (without CAP_PERFMON or CAP_SYS_ADMIN).\n\n" "The current value is %d:\n\n" " -1: Allow use of (almost) all events by all users\n" " Ignore mlock limit after perf_event_mlock_kb without CAP_IPC_LOCK\n" -">= 0: Disallow ftrace function tracepoint by users without CAP_SYS_ADMIN\n" -" Disallow raw tracepoint access by users without CAP_SYS_ADMIN\n" -">= 1: Disallow CPU event access by users without CAP_SYS_ADMIN\n" -">= 2: Disallow kernel profiling by users without CAP_SYS_ADMIN\n\n" +">= 0: Disallow ftrace function tracepoint by users without CAP_PERFMON or CAP_SYS_ADMIN\n" +" Disallow raw tracepoint access by users without CAP_SYS_PERFMON or CAP_SYS_ADMIN\n" +">= 1: Disallow CPU event access by users without CAP_PERFMON or CAP_SYS_ADMIN\n" +">= 2: Disallow kernel profiling by users without CAP_PERFMON or CAP_SYS_ADMIN\n\n" "To make this setting permanent, edit /etc/sysctl.conf too, e.g.:\n\n" " kernel.perf_event_paranoid = -1\n" , target->system_wide ? "system-wide " : "",
tools/perf/util/util.c +1 -0
@@ -290,6 +290,7 @@ int perf_event_paranoid(void)bool perf_event_paranoid_check (int max_level) {return perf_cap__capable (CAP_SYS_ADMIN ) || +perf_cap__capable (CAP_PERFMON ) ||perf_event_paranoid () <= max_level; }
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 9/60 drm/i915/perf: Open access for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to i915_perf monitoring for CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to i915_events subsystem remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure i915_events monitoring is discouraged with respect to CAP_PERFMON capability.
--- drivers/gpu/drm/i915/i915_perf.c | 13 ++++++------- 1 file changed, 6 insertions(+), 7 deletions(-)
drivers/gpu/drm/i915/i915_perf.c +6 -7
@@ -3433,10 +3433,10 @@ i915_perf_open_ioctl_locked(struct i915_perf *perf,/* Similar to perf's kernel.perf_paranoid_cpu sysctl option * we check a dev.i915.perf_stream_paranoid sysctl option * to determine if it's ok to access system wide OA counters -* without CAP_SYS_ADMIN privileges. +* without CAP_PERFMON or CAP_SYS_ADMIN privileges. */ if (privileged_op && - i915_perf_stream_paranoid && !capable (CAP_SYS_ADMIN )) { + i915_perf_stream_paranoid && !perfmon_capable ()) {DRM_DEBUG ("Insufficient privileges to open i915 perf stream\n" ); ret = -EACCES ; goto err_ctx;
@@ -3629,9 +3629,8 @@ static int read_properties_unlocked(struct i915_perf *perf, }else oa_freq_hz =0 ; -if (oa_freq_hz > i915_oa_max_sample_rate && - !capable (CAP_SYS_ADMIN )) { -DRM_DEBUG ("OA exponent would exceed the max sampling frequency (sysctl dev.i915.oa_max_sample_rate) %uHz without root privileges\n" , +if (oa_freq_hz > i915_oa_max_sample_rate && !perfmon_capable ()) { +DRM_DEBUG ("OA exponent would exceed the max sampling frequency (sysctl dev.i915.oa_max_sample_rate) %uHz without CAP_PERFMON or CAP_SYS_ADMIN privileges\n" , i915_oa_max_sample_rate);return -EACCES ; }
@@ -4052,7 +4051,7 @@ int i915_perf_add_config_ioctl(struct drm_device *dev, void *data,return -EINVAL ; } -if (i915_perf_stream_paranoid && !capable (CAP_SYS_ADMIN )) { +if (i915_perf_stream_paranoid && !perfmon_capable ()) {DRM_DEBUG ("Insufficient privileges to add i915 OA config\n" );return -EACCES ; }
@@ -4199,7 +4198,7 @@ int i915_perf_remove_config_ioctl(struct drm_device *dev, void *data,return -ENOTSUPP ; } -if (i915_perf_stream_paranoid && !capable (CAP_SYS_ADMIN )) { +if (i915_perf_stream_paranoid && !perfmon_capable ()) {DRM_DEBUG ("Insufficient privileges to remove i915 OA config\n" );return -EACCES ; }
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 10/60 trace/bpf_trace: Open access for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to bpf_trace monitoring for CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to bpf_trace monitoring remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure bpf_trace monitoring is discouraged with respect to CAP_PERFMON capability.
--- kernel/trace/bpf_trace.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/trace/bpf_trace.c +1 -1
@@ -1468,7 +1468,7 @@ int perf_event_query_prog_array(struct perf_event *event, void __user *info)u32 *ids, prog_cnt, ids_len;int ret; -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EPERM ;if (event->attr.type !=PERF_TYPE_TRACEPOINT )return -EINVAL ;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 11/60 powerpc/perf: open access for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to monitoring for CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to the monitoring remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure monitoring is discouraged with respect to CAP_PERFMON capability.
--- arch/powerpc/perf/imc-pmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/powerpc/perf/imc-pmu.c +2 -2
@@ -976,7 +976,7 @@ static int thread_imc_event_init(struct perf_event *event)if (event->attr.type != event->pmu->type)return -ENOENT ; -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EACCES ;/* Sampling not supported */
@@ -1412,7 +1412,7 @@ static int trace_imc_event_init(struct perf_event *event)if (event->attr.type != event->pmu->type)return -ENOENT ; -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EACCES ;/* Return if this is a couting event */
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 12/60 parisc/perf: open access for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to monitoring for CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to the monitoring remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure monitoring is discouraged with respect to CAP_PERFMON capability.
--- arch/parisc/kernel/perf.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/parisc/kernel/perf.c +1 -1
@@ -300,7 +300,7 @@ static ssize_t perf_write(struct file *file, const char __user *buf,else return -EFAULT ; -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EACCES ;if (count !=sizeof (uint32_t ))
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 13/60 drivers/perf: Open access for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to monitoring for CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to the monitoring remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure monitoring is discouraged with respect to CAP_PERFMON capability.
--- drivers/perf/arm_spe_pmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/perf/arm_spe_pmu.c +2 -2
@@ -274,7 +274,7 @@ static u64 arm_spe_event_to_pmscr(struct perf_event *event)if (!attr->exclude_kernel) reg |=BIT (SYS_PMSCR_EL1_E1SPE_SHIFT ); -if (IS_ENABLED (CONFIG_PID_IN_CONTEXTIDR ) &&capable (CAP_SYS_ADMIN )) +if (IS_ENABLED (CONFIG_PID_IN_CONTEXTIDR ) &&perfmon_capable ()) reg |=BIT (SYS_PMSCR_EL1_CX_SHIFT );return reg;
@@ -700,7 +700,7 @@ static int arm_spe_pmu_event_init(struct perf_event *event)return -EOPNOTSUPP ; reg =arm_spe_event_to_pmscr (event); -if (!capable (CAP_SYS_ADMIN ) && +if (!perfmon_capable () && (reg & (BIT (SYS_PMSCR_EL1_PA_SHIFT ) |BIT (SYS_PMSCR_EL1_CX_SHIFT ) |BIT (SYS_PMSCR_EL1_PCT_SHIFT ))))
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 14/60 drivers/oprofile: Open access for CAP_PERFMON privileged process
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Open access to monitoring for CAP_PERFMON privileged process. Providing the access under CAP_PERFMON capability singly, without the rest of CAP_SYS_ADMIN credentials, excludes chances to misuse the credentials and makes operation more secure.
CAP_PERFMON implements the principle of least privilege for performance monitoring and observability operations (POSIX IEEE 1003.1e 2.2.2.39 principle of least privilege: A security design principle that states that a process or program be granted only those privileges (e.g., capabilities) necessary to accomplish its legitimate function, and only for the time that such privileges are actually required)
For backward compatibility reasons access to the monitoring remains open for CAP_SYS_ADMIN privileged processes but CAP_SYS_ADMIN usage for secure monitoring is discouraged with respect to CAP_PERFMON capability.
--- drivers/oprofile/event_buffer.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/oprofile/event_buffer.c +1 -1
@@ -113,7 +113,7 @@ static int event_buffer_open(struct inode *inode, struct file *file) {int err = -EPERM ; -if (!capable (CAP_SYS_ADMIN )) +if (!perfmon_capable ())return -EPERM ;if (test_and_set_bit_lock (0 , &buffer_opened))
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 15/60 doc/admin-guide: Update perf-security.rst with CAP_PERFMON information
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Update perf-security.rst documentation file with the information related to usage of CAP_PERFMON capability to secure performance monitoring and observability operations in system.
Committer notes:
While testing 'perf top' under cap_perfmon I noticed that it needs some more capability and Alexey pointed out cap_ipc_lock, as needed by this kernel chunk:
kernel/events/core.c: 6101
if ((locked > lock_limit) && perf_is_paranoid() &&
!capable(CAP_IPC_LOCK)) {
ret = -EPERM;
goto unlock;
}So I added it to the documentation, and also mentioned that if the libcap version doesn't yet supports 'cap_perfmon', its numeric value can be used instead, i.e. if:
# setcap "cap_perfmon,cap_ipc_lock,cap_sys_ptrace,cap_syslog=ep" perf
Fails, try:
# setcap "38,cap_ipc_lock,cap_sys_ptrace,cap_syslog=ep" perf
I also added a paragraph stating that using an unpatched libcap will fail the check for CAP_PERFMON, as it checks the cap number against a maximum to see if it is valid, which makes it use as the default the 'cycles:u' event, even tho a cap_perfmon capable perf binary can get kernel samples, to workaround that just use, e.g.:
# perf top -e cycles # perf record -e cycles
And it will sample kernel and user modes.
--- Documentation/admin-guide/perf-security.rst | 86 +++++++++++++++------ 1 file changed, 61 insertions(+), 25 deletions(-)
Documentation/admin-guide/perf-security.rst +61 -25
@@ -1,6 +1,6 @@ .. _perf_security: -Perf Events and tool security +Perf events and tool security ============================= Overview
@@ -42,11 +42,11 @@ categories: Data that belong to the fourth category can potentially contain sensitive process data. If PMUs in some monitoring modes capture values of execution context registers or data from process memory then access -to such monitoring capabilities requires to be ordered and secured -properly. So, perf_events/Perf performance monitoring is the subject for -security access control management [5]_ . +to such monitoring modes requires to be ordered and secured properly. +So, perf_events performance monitoring and observability operations are +the subject for security access control management [5]_ . -perf_events/Perf access control +perf_events access control ------------------------------- To perform security checks, the Linux implementation splits processes
@@ -66,11 +66,25 @@ into distinct units, known as capabilities [6]_ , which can be independently enabled and disabled on per-thread basis for processes and files of unprivileged users. -Unprivileged processes with enabled CAP_SYS_ADMIN capability are treated +Unprivileged processes with enabled CAP_PERFMON capability are treated as privileged processes with respect to perf_events performance -monitoring and bypass *scope* permissions checks in the kernel. - -Unprivileged processes using perf_events system call API is also subject +monitoring and observability operations, thus, bypass *scope* permissions +checks in the kernel. CAP_PERFMON implements the principle of least +privilege [13]_ (POSIX 1003.1e: 2.2.2.39) for performance monitoring and +observability operations in the kernel and provides a secure approach to +perfomance monitoring and observability in the system. + +For backward compatibility reasons the access to perf_events monitoring and +observability operations is also open for CAP_SYS_ADMIN privileged +processes but CAP_SYS_ADMIN usage for secure monitoring and observability +use cases is discouraged with respect to the CAP_PERFMON capability. +If system audit records [14]_ for a process using perf_events system call +API contain denial records of acquiring both CAP_PERFMON and CAP_SYS_ADMIN +capabilities then providing the process with CAP_PERFMON capability singly +is recommended as the preferred secure approach to resolve double access +denial logging related to usage of performance monitoring and observability. + +Unprivileged processes using perf_events system call are also subject for PTRACE_MODE_READ_REALCREDS ptrace access mode check [7]_ , whose outcome determines whether monitoring is permitted. So unprivileged processes provided with CAP_SYS_PTRACE capability are effectively
@@ -82,14 +96,14 @@ performance analysis of monitored processes or a system. For example, CAP_SYSLOG capability permits reading kernel space memory addresses from /proc/kallsyms file. -perf_events/Perf privileged users +Privileged Perf users groups --------------------------------- Mechanisms of capabilities, privileged capability-dumb files [6]_ and -file system ACLs [10]_ can be used to create a dedicated group of -perf_events/Perf privileged users who are permitted to execute -performance monitoring without scope limits. The following steps can be -taken to create such a group of privileged Perf users. +file system ACLs [10]_ can be used to create dedicated groups of +privileged Perf users who are permitted to execute performance monitoring +and observability without scope limits. The following steps can be +taken to create such groups of privileged Perf users. 1. Create perf_users group of privileged Perf users, assign perf_users group to Perf tool executable and limit access to the executable for
@@ -108,30 +122,51 @@ taken to create such a group of privileged Perf users. -rwxr-x--- 2 root perf_users 11M Oct 19 15:12 perf 2. Assign the required capabilities to the Perf tool executable file and - enable members of perf_users group with performance monitoring + enable members of perf_users group with monitoring and observability privileges [6]_ : :: - # setcap "cap_sys_admin,cap_sys_ptrace,cap_syslog=ep" perf - # setcap -v "cap_sys_admin,cap_sys_ptrace,cap_syslog=ep" perf + # setcap "cap_perfmon,cap_sys_ptrace,cap_syslog=ep" perf + # setcap -v "cap_perfmon,cap_sys_ptrace,cap_syslog=ep" perf perf: OK # getcap perf - perf = cap_sys_ptrace,cap_sys_admin,cap_syslog+ep + perf = cap_sys_ptrace,cap_syslog,cap_perfmon+ep + +If the libcap installed doesn't yet support "cap_perfmon", use "38" instead, +i.e.: + +:: + + # setcap "38,cap_ipc_lock,cap_sys_ptrace,cap_syslog=ep" perf + +Note that you may need to have 'cap_ipc_lock' in the mix for tools such as +'perf top', alternatively use 'perf top -m N', to reduce the memory that +it uses for the perf ring buffer, see the memory allocation section below. + +Using a libcap without support for CAP_PERFMON will make cap_get_flag(caps, 38, +CAP_EFFECTIVE, &val) fail, which will lead the default event to be 'cycles:u', +so as a workaround explicitly ask for the 'cycles' event, i.e.: + +:: + + # perf top -e cycles + +To get kernel and user samples with a perf binary with just CAP_PERFMON. As a result, members of perf_users group are capable of conducting -performance monitoring by using functionality of the configured Perf -tool executable that, when executes, passes perf_events subsystem scope -checks. +performance monitoring and observability by using functionality of the +configured Perf tool executable that, when executes, passes perf_events +subsystem scope checks. This specific access control management is only available to superuser or root running processes with CAP_SETPCAP, CAP_SETFCAP [6]_ capabilities. -perf_events/Perf unprivileged users +Unprivileged users ----------------------------------- -perf_events/Perf *scope* and *access* control for unprivileged processes +perf_events *scope* and *access* control for unprivileged processes is governed by perf_event_paranoid [2]_ setting: -1:
@@ -166,7 +201,7 @@ is governed by perf_event_paranoid [2]_ setting: perf_event_mlock_kb locking limit is imposed but ignored for unprivileged processes with CAP_IPC_LOCK capability. -perf_events/Perf resource control +Resource control --------------------------------- Open file descriptors
@@ -227,4 +262,5 @@ Bibliography .. [10] `<http://man7.org/linux/man-pages/man5/acl.5.html>`_ .. [11] `<http://man7.org/linux/man-pages/man2/getrlimit.2.html>`_ .. [12] `<http://man7.org/linux/man-pages/man5/limits.conf.5.html>`_ - +.. [13] `<https://sites.google.com/site/fullycapable>`_ +.. [14] `<http://man7.org/linux/man-pages/man8/auditd.8.html>`_
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 16/60 doc/admin-guide: update kernel.rst with CAP_PERFMON information
In reply to Arnaldo Carvalho de MeloFrom: Alexey Budankov <[email protected]>
Update the kernel.rst documentation file with the information related to usage of CAP_PERFMON capability to secure performance monitoring and observability operations in system.
--- Documentation/admin-guide/sysctl/kernel.rst | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
Documentation/admin-guide/sysctl/kernel.rst +11 -5
@@ -730,7 +730,13 @@ perf_event_paranoid =================== Controls use of the performance events system by unprivileged -users (without CAP_SYS_ADMIN). The default value is 2. +users (without CAP_PERFMON). The default value is 2. + +For backward compatibility reasons access to system performance +monitoring and observability remains open for CAP_SYS_ADMIN +privileged processes but CAP_SYS_ADMIN usage for secure system +performance monitoring and observability operations is discouraged +with respect to CAP_PERFMON use cases. === ================================================================== -1 Allow use of (almost) all events by all users.
@@ -739,13 +745,13 @@ users (without CAP_SYS_ADMIN). The default value is 2. ``CAP_IPC_LOCK``. >=0 Disallow ftrace function tracepoint by users without - ``CAP_SYS_ADMIN``. + ``CAP_PERFMON``. - Disallow raw tracepoint access by users without ``CAP_SYS_ADMIN``. + Disallow raw tracepoint access by users without ``CAP_PERFMON``. ->=1 Disallow CPU event access by users without ``CAP_SYS_ADMIN``. +>=1 Disallow CPU event access by users without ``CAP_PERFMON``. ->=2 Disallow kernel profiling by users without ``CAP_SYS_ADMIN``. +>=2 Disallow kernel profiling by users without ``CAP_PERFMON``. === ==================================================================
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 17/60 perf script: Simplify auxiliary event printing functions
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
This simplifies the print functions for the following perf script options:
--show-task-events
--show-namespace-events
--show-cgroup-events
--show-mmap-events
--show-switch-events
--show-lost-events
--show-bpf-events
Example:
# perf record --switch-events -a -e cycles -c 10000 sleep 1 Before: # perf script --show-task-events --show-namespace-events --show-cgroup-events --show-mmap-events --show-switch-events --show-lost-events --show-bpf-events > out-before.txt After: # perf script --show-task-events --show-namespace-events --show-cgroup-events --show-mmap-events --show-switch-events --show-lost-events --show-bpf-events > out-after.txt # diff -s out-before.txt out-after.txt Files out-before.txt and out-after.tx are identical
--- tools/perf/builtin-script.c | 304 ++++++++---------------------------- 1 file changed, 66 insertions(+), 238 deletions(-)
tools/perf/builtin-script.c +66 -238
@@ -2040,7 +2040,7 @@ static int cleanup_scripting(void)static bool filter_cpu (struct perf_sample *sample) { -if (cpu_list) +if (cpu_list && sample->cpu != (u32)-1 )return !test_bit (sample->cpu, cpu_bitmap);return false; }
@@ -2138,41 +2138,59 @@ static int process_attr(struct perf_tool *tool, union perf_event *event,return err; } -static int process_comm_event (struct perf_tool *tool, -union perf_event *event, -struct perf_sample *sample, -struct machine *machine) +static int print_event_with_time (struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct machine *machine, +pid_t pid,pid_t tid,u64 timestamp) { -struct thread *thread; struct perf_script *script = container_of(tool, struct perf_script, tool);struct perf_session *session = script->session;struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); -int ret =-1 ; +struct thread *thread =NULL ; - thread =machine__findnew_thread (machine, event->comm.pid, event->comm.tid); -if (thread ==NULL ) { -pr_debug ("problem processing COMM event, skipping it.\n" ); -return -1 ; +if (evsel && !evsel->core.attr.sample_id_all) { + sample->cpu =0 ; + sample->time = timestamp; + sample->pid = pid; + sample->tid = tid; } -if (perf_event__process_comm (tool, event, sample, machine) <0 ) - goto out; +if (filter_cpu (sample)) +return 0 ; -if (!evsel->core.attr.sample_id_all) { - sample->cpu =0 ; - sample->time =0 ; - sample->tid = event->comm.tid; - sample->pid = event->comm.pid; - } -if (!filter_cpu (sample)) { +if (tid !=-1 ) + thread =machine__findnew_thread (machine, pid, tid); + +if (thread && evsel) {perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_COMM , stdout); -perf_event__fprintf (event, stdout); + event->header.type, stdout); } - ret =0 ; -out: + +perf_event__fprintf (event, stdout); +thread__put (thread); -return ret; + +return 0 ; +} + +static int print_event (struct perf_tool *tool,union perf_event *event, +struct perf_sample *sample,struct machine *machine, +pid_t pid,pid_t tid) +{ +return print_event_with_time (tool, event, sample, machine, pid, tid,0 ); +} + +static int process_comm_event (struct perf_tool *tool, +union perf_event *event, +struct perf_sample *sample, +struct machine *machine) +{ +if (perf_event__process_comm (tool, event, sample, machine) <0 ) +return -1 ; + +return print_event (tool, event, sample, machine, event->comm.pid, + event->comm.tid); } static int process_namespaces_event(struct perf_tool *tool,
@@ -2180,37 +2198,11 @@ static int process_namespaces_event(struct perf_tool *tool, struct perf_sample *sample, struct machine *machine) { - struct thread *thread; - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); -int ret =-1 ; - - thread =machine__findnew_thread (machine, event->namespaces.pid, - event->namespaces.tid); -if (thread ==NULL ) { -pr_debug ("problem processing NAMESPACES event, skipping it.\n" ); -return -1 ; - } - if (perf_event__process_namespaces(tool, event, sample, machine) < 0) - goto out; + return -1; -if (!evsel->core.attr.sample_id_all) { - sample->cpu =0 ; - sample->time =0 ; - sample->tid = event->namespaces.tid; - sample->pid = event->namespaces.pid; - } -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_NAMESPACES , stdout); -perf_event__fprintf (event, stdout); - } - ret =0 ; -out: -thread__put (thread); -return ret; + return print_event(tool, event, sample, machine, event->namespaces.pid, + event->namespaces.tid); } static int process_cgroup_event(struct perf_tool *tool,
@@ -2218,34 +2210,11 @@ static int process_cgroup_event(struct perf_tool *tool, struct perf_sample *sample, struct machine *machine) { - struct thread *thread; - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); -int ret =-1 ; - - thread =machine__findnew_thread (machine, sample->pid, sample->tid); -if (thread ==NULL ) { -pr_debug ("problem processing CGROUP event, skipping it.\n" ); -return -1 ; - } - if (perf_event__process_cgroup(tool, event, sample, machine) < 0) - goto out; + return -1; -if (!evsel->core.attr.sample_id_all) { - sample->cpu =0 ; - sample->time =0 ; - } -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_CGROUP , stdout); -perf_event__fprintf (event, stdout); - } - ret =0 ; -out: -thread__put (thread); -return ret; + return print_event(tool, event, sample, machine, sample->pid, + sample->tid); } static int process_fork_event(struct perf_tool *tool,
@@ -2253,69 +2222,24 @@ static int process_fork_event(struct perf_tool *tool, struct perf_sample *sample, struct machine *machine) { - struct thread *thread; - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); - if (perf_event__process_fork(tool, event, sample, machine) < 0) return -1; - thread =machine__findnew_thread (machine, event->fork.pid, event->fork.tid); -if (thread ==NULL ) { -pr_debug ("problem processing FORK event, skipping it.\n" ); -return -1 ; - } - -if (!evsel->core.attr.sample_id_all) { - sample->cpu =0 ; - sample->time = event->fork.time; - sample->tid = event->fork.tid; - sample->pid = event->fork.pid; - } -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_FORK , stdout); -perf_event__fprintf (event, stdout); - } -thread__put (thread); - -return 0 ; + return print_event_with_time(tool, event, sample, machine, + event->fork.pid, event->fork.tid, + event->fork.time); }static int process_exit_event (struct perf_tool *tool,union perf_event *event,struct perf_sample *sample,struct machine *machine) { -int err =0 ; -struct thread *thread; - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); - - thread =machine__findnew_thread (machine, event->fork.pid, event->fork.tid); -if (thread ==NULL ) { -pr_debug ("problem processing EXIT event, skipping it.\n" ); +/* Print before 'exit' deletes anything */ +if (print_event_with_time (tool, event, sample, machine, event->fork.pid, + event->fork.tid, event->fork.time))return -1 ; - } - -if (!evsel->core.attr.sample_id_all) { - sample->cpu =0 ; - sample->time =0 ; - sample->tid = event->fork.tid; - sample->pid = event->fork.pid; - } -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_EXIT , stdout); -perf_event__fprintf (event, stdout); - } -if (perf_event__process_exit (tool, event, sample, machine) <0 ) - err =-1 ; - -thread__put (thread); -return err; +return perf_event__process_exit (tool, event, sample, machine); } static int process_mmap_event(struct perf_tool *tool,
@@ -2323,33 +2247,11 @@ static int process_mmap_event(struct perf_tool *tool, struct perf_sample *sample, struct machine *machine) { - struct thread *thread; - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); - if (perf_event__process_mmap(tool, event, sample, machine) < 0) return -1; - thread =machine__findnew_thread (machine, event->mmap.pid, event->mmap.tid); -if (thread ==NULL ) { -pr_debug ("problem processing MMAP event, skipping it.\n" ); -return -1 ; - } - -if (!evsel->core.attr.sample_id_all) { - sample->cpu =0 ; - sample->time =0 ; - sample->tid = event->mmap.tid; - sample->pid = event->mmap.pid; - } -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_MMAP , stdout); -perf_event__fprintf (event, stdout); - } -thread__put (thread); -return 0 ; + return print_event(tool, event, sample, machine, event->mmap.pid, + event->mmap.tid); } static int process_mmap2_event(struct perf_tool *tool,
@@ -2357,33 +2259,11 @@ static int process_mmap2_event(struct perf_tool *tool, struct perf_sample *sample, struct machine *machine) { - struct thread *thread; - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); - if (perf_event__process_mmap2(tool, event, sample, machine) < 0) return -1; - thread =machine__findnew_thread (machine, event->mmap2.pid, event->mmap2.tid); -if (thread ==NULL ) { -pr_debug ("problem processing MMAP2 event, skipping it.\n" ); -return -1 ; - } - -if (!evsel->core.attr.sample_id_all) { - sample->cpu =0 ; - sample->time =0 ; - sample->tid = event->mmap2.tid; - sample->pid = event->mmap2.pid; - } -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_MMAP2 , stdout); -perf_event__fprintf (event, stdout); - } -thread__put (thread); -return 0 ; + return print_event(tool, event, sample, machine, event->mmap2.pid, + event->mmap2.tid); } static int process_switch_event(struct perf_tool *tool,
@@ -2391,10 +2271,7 @@ static int process_switch_event(struct perf_tool *tool,struct perf_sample *sample, struct machine *machine) { - struct thread *thread; struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id);if (perf_event__process_switch (tool, event, sample, machine) <0 )return -1 ;
@@ -2405,20 +2282,8 @@ static int process_switch_event(struct perf_tool *tool,if (!script->show_switch_events)return 0 ; - thread =machine__findnew_thread (machine, sample->pid, - sample->tid); -if (thread ==NULL ) { -pr_debug ("problem processing SWITCH event, skipping it.\n" ); -return -1 ; - } - -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_SWITCH , stdout); -perf_event__fprintf (event, stdout); - } -thread__put (thread); -return 0 ; +return print_event (tool, event, sample, machine, sample->pid, + sample->tid); } static int
@@ -2427,23 +2292,8 @@ process_lost_event(struct perf_tool *tool,struct perf_sample *sample, struct machine *machine) { - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); -struct thread *thread; - - thread =machine__findnew_thread (machine, sample->pid, - sample->tid); -if (thread ==NULL ) -return -1 ; - -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, -PERF_RECORD_LOST , stdout); -perf_event__fprintf (event, stdout); - } -thread__put (thread); -return 0 ; + return print_event(tool, event, sample, machine, sample->pid, + sample->tid); } static int
@@ -2462,33 +2312,11 @@ process_bpf_events(struct perf_tool *tool __maybe_unused, struct perf_sample *sample, struct machine *machine) { - struct thread *thread; - struct perf_script *script = container_of(tool, struct perf_script, tool); -struct perf_session *session = script->session; -struct evsel *evsel =perf_evlist__id2evsel (session->evlist, sample->id); - if (machine__process_ksymbol(machine, event, sample) < 0) return -1; -if (!evsel->core.attr.sample_id_all) { -perf_event__fprintf (event, stdout); -return 0 ; - } - - thread =machine__findnew_thread (machine, sample->pid, sample->tid); -if (thread ==NULL ) { -pr_debug ("problem processing MMAP event, skipping it.\n" ); -return -1 ; - } - -if (!filter_cpu (sample)) { -perf_sample__fprintf_start (sample, thread, evsel, - event->header.type, stdout); -perf_event__fprintf (event, stdout); - } - -thread__put (thread); -return 0 ; + return print_event(tool, event, sample, machine, sample->pid, + sample->tid); }static void sig_handler (int sig__maybe_unused )
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 18/60 perf bench: Add event synthesis benchmark
In reply to Arnaldo Carvalho de MeloFrom: Ian Rogers <[email protected]>
Event synthesis may occur at the start or end (tail) of a perf command. In system-wide mode it can scan every process in /proc, which may add seconds of latency before event recording. Add a new benchmark that times how long event synthesis takes with and without data synthesis.
An example execution looks like:
$ perf bench internals synthesize # Running 'internals/synthesize' benchmark: Average synthesis took: 168.253800 usec Average data synthesis took: 208.104700 usec
--- tools/perf/Documentation/perf-bench.txt | 8 ++ tools/perf/bench/Build | 2 +- tools/perf/bench/bench.h | 2 +- tools/perf/bench/synthesize.c | 101 ++++++++++++++++++++++++ tools/perf/builtin-bench.c | 6 ++ 5 files changed, 117 insertions(+), 2 deletions(-) create mode 100644 tools/perf/bench/synthesize.c
tools/perf/Documentation/perf-bench.txt +8 -0
@@ -61,6 +61,9 @@ SUBSYSTEM 'epoll':: Eventpoll (epoll) stressing benchmarks. +'internals':: + Benchmark internal perf functionality. + 'all':: All benchmark subsystems.
@@ -214,6 +217,11 @@ Suite for evaluating concurrent epoll_wait calls. *ctl*:: Suite for evaluating multiple epoll_ctl calls. +SUITES FOR 'internals' +~~~~~~~~~~~~~~~~~~~~~~ +*synthesize*:: +Suite for evaluating perf's event synthesis performance. + SEE ALSO -------- linkperf:perf[1]
tools/perf/bench/Build +1 -1
@@ -6,9 +6,9 @@ perf-y += futex-wake.o perf-y += futex-wake-parallel.o perf-y += futex-requeue.o perf-y += futex-lock-pi.o - perf-y += epoll-wait.o perf-y += epoll-ctl.o +perf-y += synthesize.o perf-$(CONFIG_X86_64) += mem-memcpy-x86-64-lib.o perf-$(CONFIG_X86_64) += mem-memcpy-x86-64-asm.o
tools/perf/bench/bench.h +1 -1
@@ -41,9 +41,9 @@ int bench_futex_wake_parallel(int argc, const char **argv);int bench_futex_requeue (int argc,const char **argv);/* pi futexes */ int bench_futex_lock_pi (int argc,const char **argv); -int bench_epoll_wait (int argc,const char **argv);int bench_epoll_ctl (int argc,const char **argv); +int bench_synthesize (int argc,const char **argv);#define BENCH_FORMAT_DEFAULT_STR "default"#define BENCH_FORMAT_DEFAULT 0
tools/perf/bench/synthesize.c new file +101 -0
101 lines not on this page · raw message
tools/perf/builtin-bench.c +6 -0
@@ -76,6 +76,11 @@ static struct bench epoll_benchmarks[] = { };#endif // HAVE_EVENTFD +static struct bench internals_benchmarks[] = { + {"synthesize" ,"Benchmark perf event synthesis" , bench_synthesize }, + {NULL ,NULL ,NULL } +}; +struct collection {const char *name;const char *summary;
@@ -92,6 +97,7 @@ static struct collection collections[] = { #ifdef HAVE_EVENTFD {"epoll", "Epoll stressing benchmarks", epoll_benchmarks }, #endif + { "internals", "Perf-internals benchmarks", internals_benchmarks }, { "all", "All benchmarks", NULL }, { NULL, NULL, NULL } };
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 19/60 tools api fs: Make xxx__mountpoint() more scalable
In reply to Arnaldo Carvalho de MeloFrom: Stephane Eranian <[email protected]>
The xxx_mountpoint() interface provided by fs.c finds mount points for common pseudo filesystems. The first time xxx_mountpoint() is invoked, it scans the mount table (/proc/mounts) looking for a match. If found, it is cached. The price to scan /proc/mounts is paid once if the mount is found.
When the mount point is not found, subsequent calls to xxx_mountpoint() scan /proc/mounts over and over again. There is no caching.
This causes a scaling issue in perf record with hugeltbfs__mountpoint(). The function is called for each process found in synthesize__mmap_events(). If the machine has thousands of processes and if the /proc/mounts has many entries this could cause major overhead in perf record. We have observed multi-second slowdowns on some configurations.
As an example on a laptop:
Before:
$ sudo umount /dev/hugepages $ strace -e trace=openat -o /tmp/tt perf record -a ls $ fgrep mounts /tmp/tt 285
After:
$ sudo umount /dev/hugepages $ strace -e trace=openat -o /tmp/tt perf record -a ls $ fgrep mounts /tmp/tt 1
One could argue that the non-caching in case the moint point is not found is intentional. That way subsequent calls may discover a moint point if the sysadmin mounts the filesystem. But the same argument could be made against caching the mount point. It could be unmounted causing errors. It all depends on the intent of the interface. This patch assumes it is expected to scan /proc/mounts once. The patch documents the caching behavior in the fs.h header file.
An alternative would be to just fix perf record. But it would solve the problem with hugetlbs__mountpoint() but there could be similar issues (possibly down the line) with other xxx_mountpoint() calls in perf or other tools.
--- tools/lib/api/fs/fs.c | 17 +++++++++++++++++ tools/lib/api/fs/fs.h | 12 ++++++++++++ 2 files changed, 29 insertions(+)
tools/lib/api/fs/fs.c +17 -0
@@ -90,6 +90,7 @@ struct fs {const char *const *mounts;char path[PATH_MAX ];bool found; +bool checked;long magic; };
@@ -111,31 +112,37 @@ static struct fs fs__entries[] = { .name = "sysfs", .mounts = sysfs__fs_known_mountpoints, .magic = SYSFS_MAGIC, + .checked = false, }, [FS__PROCFS] = { .name = "proc", .mounts = procfs__known_mountpoints, .magic = PROC_SUPER_MAGIC, + .checked = false, }, [FS__DEBUGFS] = { .name = "debugfs", .mounts = debugfs__known_mountpoints, .magic = DEBUGFS_MAGIC, + .checked = false, }, [FS__TRACEFS] = { .name = "tracefs", .mounts = tracefs__known_mountpoints, .magic = TRACEFS_MAGIC, + .checked = false, }, [FS__HUGETLBFS] = { .name = "hugetlbfs", .mounts = hugetlbfs__known_mountpoints, .magic = HUGETLBFS_MAGIC, + .checked = false, }, [FS__BPF_FS] = { .name = "bpf", .mounts = bpf_fs__known_mountpoints, .magic = BPF_FS_MAGIC, + .checked = false, }, };
@@ -158,6 +165,7 @@ static bool fs__read_mounts(struct fs *fs) }fclose (fp); + fs->checked = true;return fs->found = found; }
@@ -220,6 +228,7 @@ static bool fs__env_override(struct fs *fs)return false; fs->found = true; + fs->checked = true;strncpy (fs->path, override_path,sizeof (fs->path) -1 ); fs->path[sizeof (fs->path) -1 ] ='\0' ;return true;
@@ -246,6 +255,14 @@ static const char *fs__mountpoint(int idx)if (fs->found)return (const char *)fs->path; +/* the mount point was already checked for the mount point +* but and did not exist, so return NULL to avoid scanning again. +* This makes the found and not found paths cost equivalent +* in case of multiple calls. +*/ +if (fs->checked) +return NULL ; +return fs__get_mountpoint (fs); }
tools/lib/api/fs/fs.h +12 -0
@@ -18,6 +18,18 @@ const char *name##__mount(void); \ bool name##__configured(void); \ +/* +* The xxxx__mountpoint() entry points find the first match mount point for each +* filesystems listed below, where xxxx is the filesystem type. +* +* The interface is as follows: +* +* - If a mount point is found on first call, it is cached and used for all +* subsequent calls. +* +* - If a mount point is not found, NULL is returned on first call and all +* subsequent calls. +*/ FS(sysfs) FS(procfs) FS(debugfs)
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 20/60 perf synthetic-events: save 4kb from 2 stack frames
In reply to Arnaldo Carvalho de MeloFrom: Ian Rogers <[email protected]>
Reuse an existing char buffer to avoid two PATH_MAX sized char buffers.
Reduces stack frame sizes by 4kb.
perf_event__synthesize_mmap_events before 'sub $0x45b8,%rsp' after 'sub $0x35b8,%rsp'.
perf_event__get_comm_ids before 'sub $0x2028,%rsp' after 'sub $0x1028,%rsp'.
The performance impact of this change is negligible.
--- tools/perf/util/synthetic-events.c | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-)
tools/perf/util/synthetic-events.c +10 -12
@@ -71,7 +71,6 @@ int perf_tool__process_synth_event(struct perf_tool *tool,static int perf_event__get_comm_ids (pid_t pid,char *comm,size_t len,pid_t *tgid,pid_t *ppid) { -char filename[PATH_MAX ];char bf[4096 ];int fd;size_t size =0 ;
@@ -81,11 +80,11 @@ static int perf_event__get_comm_ids(pid_t pid, char *comm, size_t len, *tgid =-1 ; *ppid =-1 ; -snprintf (filename,sizeof (filename),"/proc/%d/status" , pid); +snprintf (bf,sizeof (bf),"/proc/%d/status" , pid); - fd =open (filename,O_RDONLY ); + fd =open (bf,O_RDONLY );if (fd <0 ) { -pr_debug ("couldn't open %s\n" , filename); +pr_debug ("couldn't open %s\n" , bf);return -1 ; }
@@ -281,9 +280,9 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool, struct machine *machine, bool mmap_data) { - char filename[PATH_MAX];FILE *fp;unsigned long long t; +char bf[BUFSIZ ];bool truncation = false;unsigned long long timeout = proc_map_timeout *1000000ULL ;int rc =0 ;
@@ -293,15 +292,15 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,if (machine__is_default_guest (machine))return 0 ; -snprintf (filename,sizeof (filename),"%s/proc/%d/task/%d/maps" , - machine->root_dir, pid, pid); +snprintf (bf,sizeof (bf),"%s/proc/%d/task/%d/maps" , + machine->root_dir, pid, pid); - fp =fopen (filename,"r" ); + fp =fopen (bf,"r" );if (fp ==NULL ) {/* * We raced with a task exiting - just return: */ -pr_debug ("couldn't open %s\n" , filename); +pr_debug ("couldn't open %s\n" , bf);return -1 ; }
@@ -309,7 +308,6 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool, t =rdclock ();while (1 ) { -char bf[BUFSIZ ];char prot[5 ];char execname[PATH_MAX ];char anonstr[] ="//anon" ;
@@ -321,10 +319,10 @@ int perf_event__synthesize_mmap_events(struct perf_tool *tool,break ;if ((rdclock () - t) > timeout) { -pr_warning ("Reading %s time out. " +pr_warning ("Reading %s/proc/%d/task/%d/maps time out. " "You may want to increase " "the time limit by --proc-map-timeout\n" , - filename); + machine->root_dir, pid, pid); truncation = true; goto out; }
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 21/60 perf expr: Add expr_ prefix for parse_ctx and parse_id
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Adding expr_ prefix for parse_ctx and parse_id, to straighten out the expr* namespace.
There's no functional change.
--- tools/perf/tests/expr.c | 4 ++-- tools/perf/util/expr.c | 10 +++++----- tools/perf/util/expr.h | 12 ++++++------ tools/perf/util/expr.y | 6 +++--- tools/perf/util/stat-shadow.c | 2 +- 5 files changed, 17 insertions(+), 17 deletions(-)
tools/perf/tests/expr.c +2 -2
@@ -6,7 +6,7 @@#include <string.h> #include <linux/zalloc.h> -static int test (struct parse_ctx *ctx,const char *e,double val2) +static int test (struct expr_parse_ctx *ctx,const char *e,double val2) {double val;
@@ -22,7 +22,7 @@ int test__expr(struct test *t __maybe_unused, int subtest __maybe_unused)const char **other;double val;int i, ret; -struct parse_ctx ctx; +struct expr_parse_ctx ctx;int num_other;expr__ctx_init (&ctx);
tools/perf/util/expr.c +5 -5
@@ -11,7 +11,7 @@ extern int expr_debug;#endif /* Caller must make sure id is allocated */ -void expr__add_id (struct parse_ctx *ctx,const char *name,double val) +void expr__add_id (struct expr_parse_ctx *ctx,const char *name,double val) {int idx;
@@ -21,13 +21,13 @@ void expr__add_id(struct parse_ctx *ctx, const char *name, double val) ctx->ids[idx].val = val; } -void expr__ctx_init (struct parse_ctx *ctx) +void expr__ctx_init (struct expr_parse_ctx *ctx) { ctx->num_ids =0 ; }static int -__expr__parse (double *val,struct parse_ctx *ctx,const char *expr, +__expr__parse (double *val,struct expr_parse_ctx *ctx,const char *expr,int start) {YY_BUFFER_STATE buffer;
@@ -52,7 +52,7 @@ __expr__parse(double *val, struct parse_ctx *ctx, const char *expr,return ret; } -int expr__parse (double *final_val,struct parse_ctx *ctx,const char *expr) +int expr__parse (double *final_val,struct expr_parse_ctx *ctx,const char *expr) {return __expr__parse (final_val, ctx, expr,EXPR_PARSE ) ?-1 :0 ; }
@@ -75,7 +75,7 @@ int expr__find_other(const char *expr, const char *one, const char ***other, int *num_other) {int err, i =0 , j =0 ; -struct parse_ctx ctx; +struct expr_parse_ctx ctx;expr__ctx_init (&ctx); err =__expr__parse (NULL , &ctx, expr,EXPR_OTHER );
tools/perf/util/expr.h +6 -6
@@ -5,19 +5,19 @@#define EXPR_MAX_OTHER 20#define MAX_PARSE_ID EXPR_MAX_OTHER -struct parse_id { +struct expr_parse_id {const char *name;double val; }; -struct parse_ctx { +struct expr_parse_ctx {int num_ids; -struct parse_id ids[MAX_PARSE_ID ]; +struct expr_parse_id ids[MAX_PARSE_ID ]; }; -void expr__ctx_init (struct parse_ctx *ctx); -void expr__add_id (struct parse_ctx *ctx,const char *id,double val); -int expr__parse (double *final_val,struct parse_ctx *ctx,const char *expr); +void expr__ctx_init (struct expr_parse_ctx *ctx); +void expr__add_id (struct expr_parse_ctx *ctx,const char *id,double val); +int expr__parse (double *final_val,struct expr_parse_ctx *ctx,const char *expr);int expr__find_other (const char *expr,const char *one,const char ***other,int *num_other);
tools/perf/util/expr.y +3 -3
@@ -15,7 +15,7 @@ %define api.pure full %parse-param { double *final_val } -%parse-param { struct parse_ctx *ctx } +%parse-param { struct expr_parse_ctx *ctx } %parse-param {void *scanner} %lex-param {void* scanner}
@@ -39,14 +39,14 @@ %{ static void expr_error(double *final_val __maybe_unused, - struct parse_ctx *ctx __maybe_unused, + struct expr_parse_ctx *ctx __maybe_unused, void *scanner, const char *s) { pr_debug("%s\n", s); } -static int lookup_id(struct parse_ctx *ctx, char *id, double *val) +static int lookup_id(struct expr_parse_ctx *ctx, char *id, double *val) { int i;
tools/perf/util/stat-shadow.c +1 -1
@@ -729,7 +729,7 @@ static void generic_metric(struct perf_stat_config *config, struct runtime_stat *st) {print_metric_t print_metric = out->print_metric; -struct parse_ctx pctx; +struct expr_parse_ctx pctx;double ratio, scale;int i;void *ctxp = out->ctx;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 22/60 perf expr: Add expr_scanner_ctx object
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
Add the expr_scanner_ctx object to hold user data for the expr scanner. Currently it holds only start_token, Kajol Jain will use it to hold 24x7 runtime param.
--- tools/perf/util/expr.c | 6 ++++-- tools/perf/util/expr.h | 4 ++++ tools/perf/util/expr.l | 10 +++++----- 3 files changed, 13 insertions(+), 7 deletions(-)
tools/perf/util/expr.c +4 -2
@@ -3,7 +3,6 @@#include <assert.h> #include "expr.h" #include "expr-bison.h" -#define YY_EXTRA_TYPE int#include "expr-flex.h" #ifdef PARSER_DEBUG
@@ -30,11 +29,14 @@ static int __expr__parse(double *val, struct expr_parse_ctx *ctx, const char *expr, int start) { +struct expr_scanner_ctx scanner_ctx = { + .start_token = start, + };YY_BUFFER_STATE buffer;void *scanner;int ret; - ret =expr_lex_init_extra (start, &scanner); + ret =expr_lex_init_extra (&scanner_ctx, &scanner);if (ret)return ret;
tools/perf/util/expr.h +4 -0
@@ -15,6 +15,10 @@ struct expr_parse_ctx {struct expr_parse_id ids[MAX_PARSE_ID ]; }; +struct expr_scanner_ctx { +int start_token; +}; +void expr__ctx_init (struct expr_parse_ctx *ctx);void expr__add_id (struct expr_parse_ctx *ctx,const char *id,double val);int expr__parse (double *final_val,struct expr_parse_ctx *ctx,const char *expr);
tools/perf/util/expr.l +5 -5
@@ -76,13 +76,13 @@ sym [0-9a-zA-Z_\.:@]+ symbol {spec}*{sym}*{spec}*{sym}* %% - { - int start_token; + struct expr_scanner_ctx *sctx = expr_get_extra(yyscanner); - start_token = expr_get_extra(yyscanner); + { + int start_token = sctx->start_token; - if (start_token) { - expr_set_extra(NULL, yyscanner); + if (sctx->start_token) { + sctx->start_token = 0; return start_token; } }
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 23/60 perf metrictroup: Split the metricgroup__add_metric function
In reply to Arnaldo Carvalho de MeloFrom: Kajol Jain <[email protected]>
This patch refactors metricgroup__add_metric function where some part of it move to function metricgroup__add_metric_param. No logic change.
--- tools/perf/util/metricgroup.c | 60 ++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 25 deletions(-)
tools/perf/util/metricgroup.c +35 -25
@@ -485,6 +485,39 @@ static bool metricgroup__has_constraint(struct pmu_event *pe)return false; } +static int __metricgroup__add_metric (struct strbuf *events, +struct list_head *group_list,struct pmu_event *pe) +{ + +const char **ids; +int idnum; +struct egroup *eg; + +if (expr__find_other (pe->metric_expr,NULL , &ids, &idnum) <0 ) +return -EINVAL ; + +if (events->len >0 ) +strbuf_addf (events,"," ); + +if (metricgroup__has_constraint (pe)) +metricgroup__add_metric_non_group (events, ids, idnum); +else +metricgroup__add_metric_weak_group (events, ids, idnum); + + eg =malloc (sizeof (*eg)); +if (!eg) +return -ENOMEM ; + + eg->ids = ids; + eg->idnum = idnum; + eg->metric_name = pe->metric_name; + eg->metric_expr = pe->metric_expr; + eg->metric_unit = pe->unit; +list_add_tail (&eg->nd, group_list); + +return 0 ; +} +static int metricgroup__add_metric (const char *metric,struct strbuf *events,struct list_head *group_list) {
@@ -504,35 +537,12 @@ static int metricgroup__add_metric(const char *metric, struct strbuf *events,continue ;if (match_metric (pe->metric_group, metric) ||match_metric (pe->metric_name, metric)) { -const char **ids; -int idnum; -struct egroup *eg;pr_debug ("metric expr %s for %s\n" , pe->metric_expr, pe->metric_name); -if (expr__find_other (pe->metric_expr, -NULL , &ids, &idnum) <0 ) -continue ; -if (events->len >0 ) -strbuf_addf (events,"," ); - -if (metricgroup__has_constraint (pe)) -metricgroup__add_metric_non_group (events, ids, idnum); -else -metricgroup__add_metric_weak_group (events, ids, idnum); - - eg =malloc (sizeof (struct egroup )); -if (!eg) { - ret = -ENOMEM ; + ret =__metricgroup__add_metric (events, group_list, pe); +if (ret == -ENOMEM )break ; - } - eg->ids = ids; - eg->idnum = idnum; - eg->metric_name = pe->metric_name; - eg->metric_expr = pe->metric_expr; - eg->metric_unit = pe->unit; -list_add_tail (&eg->nd, group_list); - ret =0 ; } }return ret;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 24/60 perf script: Add flamegraph.py script
In reply to Arnaldo Carvalho de MeloFrom: Andreas Gerstmayr <[email protected]>
This script works in tandem with d3-flame-graph to generate flame graphs from perf. It supports two output formats: JSON and HTML (the default). The HTML format will look for a standalone d3-flame-graph template file in /usr/share/d3-flame-graph/d3-flamegraph-base.html and fill in the collected stacks.
Usage:
perf record -a -g -F 99 sleep 60
perf script report flamegraph
Combined:
perf script flamegraph -a -F 99 sleep 60
Committer testing:
Tested both with "PYTHON=python3" and with the default, that uses python2-devel:
Complete set of instructions:
$ mkdir /tmp/build/perf $ make PYTHON=python3 -C tools/perf O=/tmp/build/perf install-bin $ export PATH=~/bin:$PATH $ perf record -a -g -F 99 sleep 60 $ perf script report flamegraph
Now go and open the generated flamegraph.html file in a browser.
At first this required building with PYTHON=python3, but after I reported this Andreas was kind enough to send a patch making it work with both python and python3.
--- .../perf/scripts/python/bin/flamegraph-record | 2 + .../perf/scripts/python/bin/flamegraph-report | 3 + tools/perf/scripts/python/flamegraph.py | 124 ++++++++++++++++++ 3 files changed, 129 insertions(+) create mode 100755 tools/perf/scripts/python/bin/flamegraph-record create mode 100755 tools/perf/scripts/python/bin/flamegraph-report create mode 100755 tools/perf/scripts/python/flamegraph.py
tools/perf/scripts/python/bin/flamegraph-record new file +2 -0
@@ -0,0 +1,2 @@ +#!/usr/bin/sh +perf record -g "$@"
tools/perf/scripts/python/bin/flamegraph-report new file +3 -0
@@ -0,0 +1,3 @@ +#!/usr/bin/sh +# description: create flame graphs +perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
tools/perf/scripts/python/flamegraph.py new file +124 -0
124 lines not on this page · raw message
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 25/60 perf auxtrace: Add ->evsel_is_auxtrace() callback
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Add ->evsel_is_auxtrace() callback to identify if a selected event is an AUX area event.
--- tools/perf/util/auxtrace.c | 9 +++++++++ tools/perf/util/auxtrace.h | 12 ++++++++++++ 2 files changed, 21 insertions(+)
tools/perf/util/auxtrace.c +9 -0
@@ -2577,3 +2577,12 @@ void auxtrace__free(struct perf_session *session)return session->auxtrace->free (session); } + +bool auxtrace__evsel_is_auxtrace (struct perf_session *session, +struct evsel *evsel) +{ +if (!session->auxtrace || !session->auxtrace->evsel_is_auxtrace) +return false; + +return session->auxtrace->evsel_is_auxtrace (session, evsel); +}
tools/perf/util/auxtrace.h +12 -0
@@ -21,6 +21,7 @@union perf_event ;struct perf_session ;struct evlist ; +struct evsel ;struct perf_tool ;struct mmap ;struct perf_sample ;
@@ -166,6 +167,8 @@ struct auxtrace { struct perf_tool *tool);void (*free_events)(struct perf_session *session);void (*free)(struct perf_session *session); +bool (*evsel_is_auxtrace)(struct perf_session *session, +struct evsel *evsel); };/**
@@ -584,6 +587,8 @@ void auxtrace__dump_auxtrace_sample(struct perf_session *session,int auxtrace__flush_events (struct perf_session *session,struct perf_tool *tool);void auxtrace__free_events (struct perf_session *session);void auxtrace__free (struct perf_session *session); +bool auxtrace__evsel_is_auxtrace (struct perf_session *session, +struct evsel *evsel);#define ITRACE_HELP \ " i: synthesize instructions events\n" \
@@ -749,6 +754,13 @@ void auxtrace_index__free(struct list_head *head __maybe_unused) { } +static inline +bool auxtrace__evsel_is_auxtrace (struct perf_session *session__maybe_unused , +struct evsel *evsel__maybe_unused ) +{ +return false; +} +static inline int auxtrace_parse_filters (struct evlist *evlist__maybe_unused ) {
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 26/60 perf intel-pt: Implement ->evsel_is_auxtrace() callback
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Implement ->evsel_is_auxtrace() callback.
--- tools/perf/util/intel-pt.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
tools/perf/util/intel-pt.c +10 -0
@@ -2715,6 +2715,15 @@ static void intel_pt_free(struct perf_session *session)free (pt); } +static bool intel_pt_evsel_is_auxtrace (struct perf_session *session, +struct evsel *evsel) +{ + struct intel_pt *pt = container_of(session->auxtrace, struct intel_pt, + auxtrace); + +return evsel->core.attr.type == pt->pmu_type; +} +static int intel_pt_process_auxtrace_event (struct perf_session *session,union perf_event *event,struct perf_tool *tool__maybe_unused )
@@ -3310,6 +3319,7 @@ int intel_pt_process_auxtrace_info(union perf_event *event, pt->auxtrace.flush_events = intel_pt_flush; pt->auxtrace.free_events = intel_pt_free_events; pt->auxtrace.free = intel_pt_free; + pt->auxtrace.evsel_is_auxtrace = intel_pt_evsel_is_auxtrace; session->auxtrace = &pt->auxtrace;if (dump_trace)
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 27/60 perf intel-bts: Implement ->evsel_is_auxtrace() callback
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Implement ->evsel_is_auxtrace() callback.
--- tools/perf/util/intel-bts.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
tools/perf/util/intel-bts.c +10 -0
@@ -728,6 +728,15 @@ static void intel_bts_free(struct perf_session *session)free (bts); } +static bool intel_bts_evsel_is_auxtrace (struct perf_session *session, +struct evsel *evsel) +{ + struct intel_bts *bts = container_of(session->auxtrace, struct intel_bts, + auxtrace); + +return evsel->core.attr.type == bts->pmu_type; +} +struct intel_bts_synth {struct perf_tool dummy_tool;struct perf_session *session;
@@ -883,6 +892,7 @@ int intel_bts_process_auxtrace_info(union perf_event *event, bts->auxtrace.flush_events = intel_bts_flush; bts->auxtrace.free_events = intel_bts_free_events; bts->auxtrace.free = intel_bts_free; + bts->auxtrace.evsel_is_auxtrace = intel_bts_evsel_is_auxtrace; session->auxtrace = &bts->auxtrace; intel_bts_print_info(&auxtrace_info->priv[0], INTEL_BTS_PMU_TYPE,
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 28/60 perf arm-spe: Implement ->evsel_is_auxtrace() callback
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Implement ->evsel_is_auxtrace() callback.
--- tools/perf/util/arm-spe.c | 9 +++++++++ 1 file changed, 9 insertions(+)
tools/perf/util/arm-spe.c +9 -0
@@ -176,6 +176,14 @@ static void arm_spe_free(struct perf_session *session)free (spe); } +static bool arm_spe_evsel_is_auxtrace (struct perf_session *session, +struct evsel *evsel) +{ + struct arm_spe *spe = container_of(session->auxtrace, struct arm_spe, auxtrace); + +return evsel->core.attr.type == spe->pmu_type; +} +static const char *const arm_spe_info_fmts[] = { [ARM_SPE_PMU_TYPE ] =" PMU Type %" PRId64"\n" , };
@@ -218,6 +226,7 @@ int arm_spe_process_auxtrace_info(union perf_event *event, spe->auxtrace.flush_events = arm_spe_flush; spe->auxtrace.free_events = arm_spe_free_events; spe->auxtrace.free = arm_spe_free; + spe->auxtrace.evsel_is_auxtrace = arm_spe_evsel_is_auxtrace; session->auxtrace = &spe->auxtrace;arm_spe_print_info (&auxtrace_info->priv[0 ]);
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 29/60 perf cs-etm: Implement ->evsel_is_auxtrace() callback
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Implement ->evsel_is_auxtrace() callback.
--- tools/perf/util/cs-etm.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
tools/perf/util/cs-etm.c +11 -0
@@ -631,6 +631,16 @@ static void cs_etm__free(struct perf_session *session)zfree (&aux); } +static bool cs_etm__evsel_is_auxtrace (struct perf_session *session, +struct evsel *evsel) +{ +struct cs_etm_auxtrace *aux =container_of (session->auxtrace, + struct cs_etm_auxtrace, + auxtrace); + +return evsel->core.attr.type == aux->pmu_type; +} +static u8 cs_etm__cpu_mode (struct cs_etm_queue *etmq,u64 address) {struct machine *machine;
@@ -2618,6 +2628,7 @@ int cs_etm__process_auxtrace_info(union perf_event *event, etm->auxtrace.flush_events = cs_etm__flush_events; etm->auxtrace.free_events = cs_etm__free_events; etm->auxtrace.free = cs_etm__free; + etm->auxtrace.evsel_is_auxtrace = cs_etm__evsel_is_auxtrace; session->auxtrace = &etm->auxtrace; etm->unknown_thread =thread__new (999999999 ,999999999 );
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 30/60 perf s390-cpumsf: Implement ->evsel_is_auxtrace() callback
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Implement ->evsel_is_auxtrace() callback.
--- tools/perf/util/s390-cpumcf-kernel.h | 1 + tools/perf/util/s390-cpumsf.c | 9 +++++++++ 2 files changed, 10 insertions(+)
tools/perf/util/s390-cpumcf-kernel.h +1 -0
@@ -11,6 +11,7 @@#define S390_CPUMCF_DIAG_DEF 0xfeef/* Counter diagnostic entry ID */ #define PERF_EVENT_CPUM_CF_DIAG 0xBC000/* Event: Counter sets */ +#define PERF_EVENT_CPUM_SF_DIAG 0xBD000/* Event: Combined-sampling */ struct cf_ctrset_entry {/* CPU-M CF counter set entry (8 byte) */ unsigned int def:16 ;/* 0-15 Data Entry Format */
tools/perf/util/s390-cpumsf.c +9 -0
@@ -1047,6 +1047,14 @@ static void s390_cpumsf_free(struct perf_session *session)free (sf); } +static bool +s390_cpumsf_evsel_is_auxtrace (struct perf_session *session__maybe_unused , +struct evsel *evsel) +{ +return evsel->core.attr.type ==PERF_TYPE_RAW && + evsel->core.attr.config ==PERF_EVENT_CPUM_SF_DIAG ; +} +static int s390_cpumsf_get_type (const char *cpuid) {int ret, family =0 ;
@@ -1142,6 +1150,7 @@ int s390_cpumsf_process_auxtrace_info(union perf_event *event, sf->auxtrace.flush_events = s390_cpumsf_flush; sf->auxtrace.free_events = s390_cpumsf_free_events; sf->auxtrace.free = s390_cpumsf_free; + sf->auxtrace.evsel_is_auxtrace = s390_cpumsf_evsel_is_auxtrace; session->auxtrace = &sf->auxtrace;if (dump_trace)
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 31/60 perf auxtrace: For reporting purposes, un-group AUX area event
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
An AUX area event must be the group leader when recording traces in sample mode, but that does not produce the expected results from 'perf report' because it expects the leader to provide samples.
Rather than teach 'perf report' about AUX area sampling, un-group the AUX area event during processing, making the 2nd event the leader.
Example:
$ perf record -e '{intel_pt//u,branch-misses:u}' -c 1 uname
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.080 MB perf.data ]Before:
$ perf report
Samples: 800 of events 'anon group { intel_pt//u, branch-misses:u }', Event count (approx.): 800
Children Self Command Shared Object Symbol
0.00% 47.50% 0.00% 47.50% uname libc-2.28.so [.] _dl_addr
0.00% 16.38% 0.00% 16.38% uname ld-2.28.so [.] __GI___tunables_init
0.00% 54.75% 0.00% 4.75% uname ld-2.28.so [.] dl_main
0.00% 3.12% 0.00% 3.12% uname ld-2.28.so [.] _dl_map_object_from_fd
0.00% 2.38% 0.00% 2.38% uname ld-2.28.so [.] strcmp
0.00% 2.25% 0.00% 2.25% uname ld-2.28.so [.] _dl_check_map_versions
0.00% 2.00% 0.00% 2.00% uname ld-2.28.so [.] _dl_important_hwcaps
0.00% 2.00% 0.00% 2.00% uname ld-2.28.so [.] _dl_map_object_deps
0.00% 51.50% 0.00% 1.50% uname ld-2.28.so [.] _dl_sysdep_start
0.00% 1.25% 0.00% 1.25% uname ld-2.28.so [.] _dl_load_cache_lookup
0.00% 51.12% 0.00% 1.12% uname ld-2.28.so [.] _dl_start
0.00% 50.88% 0.00% 1.12% uname ld-2.28.so [.] do_lookup_x
0.00% 50.62% 0.00% 1.00% uname ld-2.28.so [.] _dl_lookup_symbol_x
0.00% 1.00% 0.00% 1.00% uname ld-2.28.so [.] _dl_map_object
0.00% 1.00% 0.00% 1.00% uname ld-2.28.so [.] _dl_next_ld_env_entry
0.00% 0.88% 0.00% 0.88% uname ld-2.28.so [.] _dl_cache_libcmp
0.00% 0.88% 0.00% 0.88% uname ld-2.28.so [.] _dl_new_object
0.00% 50.88% 0.00% 0.88% uname ld-2.28.so [.] _dl_relocate_object
0.00% 0.62% 0.00% 0.62% uname ld-2.28.so [.] _dl_init_paths
0.00% 0.62% 0.00% 0.62% uname ld-2.28.so [.] _dl_name_match_p
0.00% 0.50% 0.00% 0.50% uname ld-2.28.so [.] get_common_indeces.constprop.1
0.00% 0.50% 0.00% 0.50% uname ld-2.28.so [.] memmove
0.00% 0.50% 0.00% 0.50% uname ld-2.28.so [.] memset
0.00% 0.50% 0.00% 0.50% uname ld-2.28.so [.] open_verify.constprop.11
0.00% 0.38% 0.00% 0.38% uname ld-2.28.so [.] _dl_check_all_versions
0.00% 0.38% 0.00% 0.38% uname ld-2.28.so [.] _dl_find_dso_for_object
0.00% 0.38% 0.00% 0.38% uname ld-2.28.so [.] init_tls
0.00% 0.25% 0.00% 0.25% uname ld-2.28.so [.] __tunable_get_val
0.00% 0.25% 0.00% 0.25% uname ld-2.28.so [.] _dl_add_to_namespace_list
0.00% 0.25% 0.00% 0.25% uname ld-2.28.so [.] _dl_determine_tlsoffset
0.00% 0.25% 0.00% 0.25% uname ld-2.28.so [.] _dl_discover_osversion
0.00% 0.25% 0.00% 0.25% uname ld-2.28.so [.] calloc@plt
0.00% 0.25% 0.00% 0.25% uname ld-2.28.so [.] malloc
0.00% 0.25% 0.00% 0.25% uname ld-2.28.so [.] malloc@plt
0.00% 0.25% 0.00% 0.25% uname libc-2.28.so [.] _nl_load_locale_from_archive
0.00% 0.25% 0.00% 0.25% uname [unknown] [k] 0xffffffffa3a00010
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] __libc_scratch_buffer_set_array_size
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] _dl_allocate_tls_storage
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] _dl_catch_exception
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] _dl_setup_hash
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] _dl_sort_maps
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] _dl_sysdep_read_whole_file
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] access
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] calloc
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] mmap64
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] openaux
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] rtld_lock_default_lock_recursive
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] rtld_lock_default_unlock_recursive
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] strchr
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] strlen
0.00% 0.12% 0.00% 0.12% uname ld-2.28.so [.] 0x0000000000001080
0.00% 0.12% 0.00% 0.12% uname libc-2.28.so [.] __strchrnul_avx2
0.00% 0.12% 0.00% 0.12% uname libc-2.28.so [.] _nl_normalize_codeset
0.00% 0.12% 0.00% 0.12% uname libc-2.28.so [.] malloc
0.00% 0.12% 0.00% 0.12% uname [unknown] [k] 0xffffffffa3a011f0
0.00% 50.00% 0.00% 0.00% uname ld-2.28.so [.] _dl_start_user
0.00% 50.00% 0.00% 0.00% uname [unknown] [.] 0000000000000000After:
Samples: 800 of event 'branch-misses:u', Event count (approx.): 800
Children Self Command Shared Object Symbol
54.75% 4.75% uname ld-2.28.so [.] dl_main
51.50% 1.50% uname ld-2.28.so [.] _dl_sysdep_start
51.12% 1.12% uname ld-2.28.so [.] _dl_start
50.88% 0.88% uname ld-2.28.so [.] _dl_relocate_object
50.88% 1.12% uname ld-2.28.so [.] do_lookup_x
50.62% 1.00% uname ld-2.28.so [.] _dl_lookup_symbol_x
50.00% 0.00% uname ld-2.28.so [.] _dl_start_user
50.00% 0.00% uname [unknown] [.] 0000000000000000
47.50% 47.50% uname libc-2.28.so [.] _dl_addr
16.38% 16.38% uname ld-2.28.so [.] __GI___tunables_init
3.12% 3.12% uname ld-2.28.so [.] _dl_map_object_from_fd
2.38% 2.38% uname ld-2.28.so [.] strcmp
2.25% 2.25% uname ld-2.28.so [.] _dl_check_map_versions
2.00% 2.00% uname ld-2.28.so [.] _dl_important_hwcaps
2.00% 2.00% uname ld-2.28.so [.] _dl_map_object_deps
1.25% 1.25% uname ld-2.28.so [.] _dl_load_cache_lookup
1.00% 1.00% uname ld-2.28.so [.] _dl_map_object
1.00% 1.00% uname ld-2.28.so [.] _dl_next_ld_env_entry
0.88% 0.88% uname ld-2.28.so [.] _dl_cache_libcmp
0.88% 0.88% uname ld-2.28.so [.] _dl_new_object
0.62% 0.62% uname ld-2.28.so [.] _dl_init_paths
0.62% 0.62% uname ld-2.28.so [.] _dl_name_match_p
0.50% 0.50% uname ld-2.28.so [.] get_common_indeces.constprop.1
0.50% 0.50% uname ld-2.28.so [.] memmove
0.50% 0.50% uname ld-2.28.so [.] memset
0.50% 0.50% uname ld-2.28.so [.] open_verify.constprop.11
0.38% 0.38% uname ld-2.28.so [.] _dl_check_all_versions
0.38% 0.38% uname ld-2.28.so [.] _dl_find_dso_for_object
0.38% 0.38% uname ld-2.28.so [.] init_tls
0.25% 0.25% uname ld-2.28.so [.] __tunable_get_val
0.25% 0.25% uname ld-2.28.so [.] _dl_add_to_namespace_list
0.25% 0.25% uname ld-2.28.so [.] _dl_determine_tlsoffset
0.25% 0.25% uname ld-2.28.so [.] _dl_discover_osversion
0.25% 0.25% uname ld-2.28.so [.] calloc@plt
0.25% 0.25% uname ld-2.28.so [.] malloc
0.25% 0.25% uname ld-2.28.so [.] malloc@plt
0.25% 0.25% uname libc-2.28.so [.] _nl_load_locale_from_archive
0.25% 0.25% uname [unknown] [k] 0xffffffffa3a00010
0.12% 0.12% uname ld-2.28.so [.] __libc_scratch_buffer_set_array_size
0.12% 0.12% uname ld-2.28.so [.] _dl_allocate_tls_storage
0.12% 0.12% uname ld-2.28.so [.] _dl_catch_exception
0.12% 0.12% uname ld-2.28.so [.] _dl_setup_hash
0.12% 0.12% uname ld-2.28.so [.] _dl_sort_maps
0.12% 0.12% uname ld-2.28.so [.] _dl_sysdep_read_whole_file
0.12% 0.12% uname ld-2.28.so [.] access
0.12% 0.12% uname ld-2.28.so [.] calloc
0.12% 0.12% uname ld-2.28.so [.] mmap64
0.12% 0.12% uname ld-2.28.so [.] openaux
0.12% 0.12% uname ld-2.28.so [.] rtld_lock_default_lock_recursive
0.12% 0.12% uname ld-2.28.so [.] rtld_lock_default_unlock_recursive
0.12% 0.12% uname ld-2.28.so [.] strchr
0.12% 0.12% uname ld-2.28.so [.] strlen
0.12% 0.12% uname ld-2.28.so [.] 0x0000000000001080
0.12% 0.12% uname libc-2.28.so [.] __strchrnul_avx2
0.12% 0.12% uname libc-2.28.so [.] _nl_normalize_codeset
0.12% 0.12% uname libc-2.28.so [.] malloc
0.12% 0.12% uname [unknown] [k] 0xffffffffa3a011f0--- tools/perf/util/auxtrace.c | 60 ++++++++++++++++++++++++++++++++++---- 1 file changed, 55 insertions(+), 5 deletions(-)
tools/perf/util/auxtrace.c +55 -5
@@ -1234,29 +1234,79 @@ int perf_event__synthesize_auxtrace_info(struct auxtrace_record *itr,return err; } +static void unleader_evsel (struct evlist *evlist,struct evsel *leader) +{ +struct evsel *new_leader =NULL ; +struct evsel *evsel; + +/* Find new leader for the group */ +evlist__for_each_entry (evlist, evsel) { +if (evsel->leader != leader || evsel == leader) +continue ; +if (!new_leader) + new_leader = evsel; + evsel->leader = new_leader; + } + +/* Update group information */ +if (new_leader) { +zfree (&new_leader->group_name); + new_leader->group_name = leader->group_name; + leader->group_name =NULL ; + + new_leader->core.nr_members = leader->core.nr_members -1 ; + leader->core.nr_members =1 ; + } +} + +static void unleader_auxtrace (struct perf_session *session) +{ +struct evsel *evsel; + +evlist__for_each_entry (session->evlist, evsel) { +if (auxtrace__evsel_is_auxtrace (session, evsel) && +perf_evsel__is_group_leader (evsel)) { +unleader_evsel (session->evlist, evsel); + } + } +} +int perf_event__process_auxtrace_info (struct perf_session *session,union perf_event *event) {enum auxtrace_type type = event->auxtrace_info.type; +int err;if (dump_trace)fprintf (stdout," type: %u\n" , type);switch (type) {case PERF_AUXTRACE_INTEL_PT : -return intel_pt_process_auxtrace_info (event, session); + err =intel_pt_process_auxtrace_info (event, session); +break ;case PERF_AUXTRACE_INTEL_BTS : -return intel_bts_process_auxtrace_info (event, session); + err =intel_bts_process_auxtrace_info (event, session); +break ;case PERF_AUXTRACE_ARM_SPE : -return arm_spe_process_auxtrace_info (event, session); + err =arm_spe_process_auxtrace_info (event, session); +break ;case PERF_AUXTRACE_CS_ETM : -return cs_etm__process_auxtrace_info (event, session); + err =cs_etm__process_auxtrace_info (event, session); +break ;case PERF_AUXTRACE_S390_CPUMSF : -return s390_cpumsf_process_auxtrace_info (event, session); + err =s390_cpumsf_process_auxtrace_info (event, session); +break ;case PERF_AUXTRACE_UNKNOWN :default :return -EINVAL ; } + +if (err) +return err; + +unleader_auxtrace (session); + +return 0 ; } s64 perf_event__process_auxtrace(struct perf_session *session,
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 32/60 perf auxtrace: Add an option to synthesize callchains for regular events
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Currently, callchains can be synthesized only for synthesized events. Add an itrace option to synthesize callchains for regular events.
--- tools/perf/Documentation/itrace.txt | 1 + tools/perf/builtin-report.c | 3 ++- tools/perf/builtin-script.c | 2 +- tools/perf/util/auxtrace.c | 6 +++++- tools/perf/util/auxtrace.h | 2 ++ tools/perf/util/s390-cpumsf.c | 2 +- 6 files changed, 12 insertions(+), 4 deletions(-)
tools/perf/Documentation/itrace.txt +1 -0
@@ -10,6 +10,7 @@ e synthesize error events d create a debug log g synthesize a call chain (use with i or x) + G synthesize a call chain on existing event records l synthesize last branch entries (use with i or x) s skip initial number of events
tools/perf/builtin-report.c +2 -1
@@ -339,6 +339,7 @@ static int report__setup_sample_type(struct report *rep)bool is_pipe =perf_data__is_pipe (session->data);if (session->itrace_synth_opts->callchain || + session->itrace_synth_opts->add_callchain || (!is_pipe &&perf_header__has_feat (&session->header,HEADER_AUXTRACE ) && !session->itrace_synth_opts->set))
@@ -1332,7 +1333,7 @@ int cmd_report(int argc, const char **argv)if (symbol_conf.cumulate_callchain && !callchain_param.order_set) callchain_param.order =ORDER_CALLER ; -if (itrace_synth_opts.callchain && +if ((itrace_synth_opts.callchain || itrace_synth_opts.add_callchain) && (int )itrace_synth_opts.callchain_sz > report.max_stack) report.max_stack = itrace_synth_opts.callchain_sz;
tools/perf/builtin-script.c +1 -1
@@ -3537,7 +3537,7 @@ int cmd_script(int argc, const char **argv)return -1 ; } -if (itrace_synth_opts.callchain && +if ((itrace_synth_opts.callchain || itrace_synth_opts.add_callchain) && itrace_synth_opts.callchain_sz > scripting_max_stack) scripting_max_stack = itrace_synth_opts.callchain_sz;
tools/perf/util/auxtrace.c +5 -1
@@ -1462,8 +1462,12 @@ int itrace_parse_synth_opts(const struct option *opt, const char *str, synth_opts->branches = true; synth_opts->returns = true;break ; +case 'G' :case 'g' : - synth_opts->callchain = true; +if (p[-1 ] =='G' ) + synth_opts->add_callchain = true; +else + synth_opts->callchain = true; synth_opts->callchain_sz =PERF_ITRACE_DEFAULT_CALLCHAIN_SZ ;while (*p ==' ' || *p ==',' )
tools/perf/util/auxtrace.h +2 -0
@@ -74,6 +74,7 @@ enum itrace_period_type {* @calls: limit branch samples to calls (can be combined with @returns) * @returns: limit branch samples to returns (can be combined with @calls) * @callchain: add callchain to 'instructions' events +* @add_callchain: add callchain to existing event records * @thread_stack: feed branches to the thread_stack * @last_branch: add branch context to 'instruction' events * @callchain_sz: maximum callchain size
@@ -101,6 +102,7 @@ struct itrace_synth_opts {bool calls;bool returns;bool callchain; +bool add_callchain;bool thread_stack;bool last_branch;unsigned int callchain_sz;
tools/perf/util/s390-cpumsf.c +1 -1
@@ -1079,7 +1079,7 @@ static bool check_auxtrace_itrace(struct itrace_synth_opts *itops) itops->pwr_events || itops->errors || itops->dont_decode || itops->calls || itops->returns || itops->callchain || itops->thread_stack || - itops->last_branch; + itops->last_branch || itops->add_callchain;if (!ison)return true;pr_err ("Unsupported --itrace options specified\n" );
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 33/60 perf thread-stack: Add thread_stack__sample_late()
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Add a thread stack function to create a call chain for hardware events where the sample records get created some time after the event occurred.
--- tools/perf/util/thread-stack.c | 57 ++++++++++++++++++++++++++++++++++ tools/perf/util/thread-stack.h | 3 ++ 2 files changed, 60 insertions(+)
tools/perf/util/thread-stack.c +57 -0
@@ -497,6 +497,63 @@ void thread_stack__sample(struct thread *thread, int cpu, chain->nr = i; } +/* +* Hardware sample records, created some time after the event occurred, need to +* have subsequent addresses removed from the call chain. +*/ +void thread_stack__sample_late (struct thread *thread,int cpu, +struct ip_callchain *chain,size_t sz, +u64 sample_ip,u64 kernel_start) +{ +struct thread_stack *ts =thread__stack (thread, cpu); +u64 sample_context =callchain_context (sample_ip, kernel_start); +u64 last_context, context, ip; +size_t nr =0 , j; + +if (sz <2 ) { + chain->nr =0 ; +return ; + } + +if (!ts) + goto out; + +/* +* When tracing kernel space, kernel addresses occur at the top of the +* call chain after the event occurred but before tracing stopped. +* Skip them. +*/ +for (j =1 ; j <= ts->cnt; j++) { + ip = ts->stack[ts->cnt - j].ret_addr; + context =callchain_context (ip, kernel_start); +if (context ==PERF_CONTEXT_USER || + (context == sample_context && ip == sample_ip)) +break ; + } + + last_context = sample_ip;/* Use sample_ip as an invalid context */ + +for (; nr < sz && j <= ts->cnt; nr++, j++) { + ip = ts->stack[ts->cnt - j].ret_addr; + context =callchain_context (ip, kernel_start); +if (context != last_context) { +if (nr >= sz -1 ) +break ; + chain->ips[nr++] = context; + last_context = context; + } + chain->ips[nr] = ip; + } +out: +if (nr) { + chain->nr = nr; + }else { + chain->ips[0 ] = sample_context; + chain->ips[1 ] = sample_ip; + chain->nr =2 ; + } +} +struct call_return_processor *call_return_processor__new (int (*process)(struct call_return *cr,u64 *parent_db_id,void *data),void *data)
tools/perf/util/thread-stack.h +3 -0
@@ -85,6 +85,9 @@ int thread_stack__event(struct thread *thread, int cpu, u32 flags, u64 from_ip,void thread_stack__set_trace_nr (struct thread *thread,int cpu,u64 trace_nr);void thread_stack__sample (struct thread *thread,int cpu,struct ip_callchain *chain,size_t sz,u64 ip,u64 kernel_start); +void thread_stack__sample_late (struct thread *thread,int cpu, +struct ip_callchain *chain,size_t sz,u64 ip, +u64 kernel_start);int thread_stack__flush (struct thread *thread);void thread_stack__free (struct thread *thread);size_t thread_stack__depth (struct thread *thread,int cpu);
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 34/60 perf evsel: Be consistent when looking which evsel PERF_SAMPLE_ bits are set
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Using 'type' variable for checking for callchains is equivalent to using evsel__has_callchain(evsel) and is how the other PERF_SAMPLE_ bits are checked in this function, so use it to be consistent.
--- tools/perf/util/evsel.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/util/evsel.c +1 -1
@@ -2136,7 +2136,7 @@ int perf_evsel__parse_sample(struct evsel *evsel, union perf_event *event, } } -if (evsel__has_callchain (evsel)) { +if (type &PERF_SAMPLE_CALLCHAIN ) {const u64 max_callchain_nr =UINT64_MAX /sizeof (u64);OVERFLOW_CHECK_u64 (array);
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 35/60 perf evsel: Add support for synthesized sample type
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
For reporting purposes, an evsel sample can have a callchain synthesized from AUX area data. Add support for keeping track of synthesized sample types. Note, the recorded sample_type cannot be changed because it is needed to continue to parse events.
--- tools/perf/util/evsel.h | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-)
tools/perf/util/evsel.h +14 -1
@@ -104,6 +104,14 @@ struct evsel {perf_evsel__sb_cb_t *cb;void *data; } side_band; +/* +* For reporting purposes, an evsel sample can have a callchain +* synthesized from AUX area data. Keep track of synthesized sample +* types here. Note, the recorded sample_type cannot be changed because +* it is needed to continue to parse events. +* See also evsel__has_callchain(). +*/ +__u64 synth_sample_type; };struct perf_missing_features {
@@ -398,7 +406,12 @@ static inline bool perf_evsel__has_branch_hw_idx(const struct evsel *evsel)static inline bool evsel__has_callchain (const struct evsel *evsel) { -return (evsel->core.attr.sample_type &PERF_SAMPLE_CALLCHAIN ) !=0 ; +/* +* For reporting purposes, an evsel sample can have a recorded callchain +* or a callchain synthesized from AUX area data. +*/ +return evsel->core.attr.sample_type &PERF_SAMPLE_CALLCHAIN || + evsel->synth_sample_type &PERF_SAMPLE_CALLCHAIN ; }struct perf_env *perf_evsel__env (struct evsel *evsel);
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 36/60 perf intel-pt: Add support for synthesizing callchains for regular events
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Currently, callchains can be synthesized only for synthesized events. Support also synthesizing callchains for regular events.
Example:
# perf record --kcore --aux-sample -e '{intel_pt//,cycles}' -c 10000 uname
Linux
[ perf record: Woken up 3 times to write data ]
[ perf record: Captured and wrote 0.532 MB perf.data ]
# perf script --itrace=Ge | head -20
uname 4864 2419025.358181: 10000 cycles:
ffffffffbba56965 apparmor_bprm_committing_creds+0x35 ([kernel.kallsyms])
ffffffffbc400cd5 __indirect_thunk_start+0x5 ([kernel.kallsyms])
ffffffffbba07422 security_bprm_committing_creds+0x22 ([kernel.kallsyms])
ffffffffbb89805d install_exec_creds+0xd ([kernel.kallsyms])
ffffffffbb90d9ac load_elf_binary+0x3ac ([kernel.kallsyms])uname 4864 2419025.358185: 10000 cycles:
ffffffffbba56db0 apparmor_bprm_committed_creds+0x20 ([kernel.kallsyms])
ffffffffbc400cd5 __indirect_thunk_start+0x5 ([kernel.kallsyms])
ffffffffbba07452 security_bprm_committed_creds+0x22 ([kernel.kallsyms])
ffffffffbb89809a install_exec_creds+0x4a ([kernel.kallsyms])
ffffffffbb90d9ac load_elf_binary+0x3ac ([kernel.kallsyms])uname 4864 2419025.358189: 10000 cycles:
ffffffffbb86fdf6 vma_adjust_trans_huge+0x6 ([kernel.kallsyms])
ffffffffbb821660 __vma_adjust+0x160 ([kernel.kallsyms])
ffffffffbb897be7 shift_arg_pages+0x97 ([kernel.kallsyms])
ffffffffbb897ed9 setup_arg_pages+0x1e9 ([kernel.kallsyms])
ffffffffbb90d9f2 load_elf_binary+0x3f2 ([kernel.kallsyms])Committer testing:
# perf record --kcore --aux-sample -e '{intel_pt//,cycles}' -c 10000 uname
Linux
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.233 MB perf.data ]
#Then, before this patch:
# perf script --itrace=Ge | head -20
uname 28642 168664.856384: 10000 cycles: ffffffff9810aeaa commit_creds+0x2a ([kernel.kallsyms])
uname 28642 168664.856388: 10000 cycles: ffffffff982a24f1 mprotect_fixup+0x151 ([kernel.kallsyms])
uname 28642 168664.856392: 10000 cycles: ffffffff982a385b move_page_tables+0xbcb ([kernel.kallsyms])
uname 28642 168664.856396: 10000 cycles: ffffffff982fd4ec __mod_memcg_state+0x1c ([kernel.kallsyms])
uname 28642 168664.856400: 10000 cycles: ffffffff9829fddd do_mmap+0xfd ([kernel.kallsyms])
uname 28642 168664.856404: 10000 cycles: ffffffff9829c879 __vma_adjust+0x479 ([kernel.kallsyms])
uname 28642 168664.856408: 10000 cycles: ffffffff98238e94 __perf_addr_filters_adjust+0x34 ([kernel.kallsyms])
uname 28642 168664.856412: 10000 cycles: ffffffff98a38e0b down_write+0x1b ([kernel.kallsyms])
uname 28642 168664.856416: 10000 cycles: ffffffff983006a0 memcg_kmem_get_cache+0x0 ([kernel.kallsyms])
uname 28642 168664.856421: 10000 cycles: ffffffff98396eaf load_elf_binary+0x92f ([kernel.kallsyms])
uname 28642 168664.856425: 10000 cycles: ffffffff982e0222 kfree+0x62 ([kernel.kallsyms])
uname 28642 168664.856428: 10000 cycles: ffffffff9846dfd4 file_has_perm+0x54 ([kernel.kallsyms])
uname 28642 168664.856433: 10000 cycles: ffffffff98288911 vma_interval_tree_insert+0x51 ([kernel.kallsyms])
uname 28642 168664.856437: 10000 cycles: ffffffff9823e577 perf_event_mmap_output+0x27 ([kernel.kallsyms])
uname 28642 168664.856441: 10000 cycles: ffffffff98a26fa0 xas_load+0x40 ([kernel.kallsyms])
uname 28642 168664.856445: 10000 cycles: ffffffff98004f30 arch_setup_additional_pages+0x0 ([kernel.kallsyms])
uname 28642 168664.856448: 10000 cycles: ffffffff98a297c0 copy_user_generic_unrolled+0xa0 ([kernel.kallsyms])
uname 28642 168664.856452: 10000 cycles: ffffffff9853a87a strnlen_user+0x10a ([kernel.kallsyms])
uname 28642 168664.856456: 10000 cycles: ffffffff986638a7 randomize_page+0x27 ([kernel.kallsyms])
uname 28642 168664.856460: 10000 cycles: ffffffff98a3b645 _raw_spin_lock+0x5 ([kernel.kallsyms])#
And after:
# perf script --itrace=Ge | head -20 uname 28642 168664.856384: 10000 cycles: ffffffff9810aeaa commit_creds+0x2a ([kernel.kallsyms]) ffffffff9831fe87 install_exec_creds+0x17 ([kernel.kallsyms]) ffffffff983968d9 load_elf_binary+0x359 ([kernel.kallsyms]) ffffffff98e00c45 __x86_indirect_thunk_rax+0x5 ([kernel.kallsyms]) ffffffff98e00c45 __x86_indirect_thunk_rax+0x5 ([kernel.kallsyms])
uname 28642 168664.856388: 10000 cycles:
ffffffff982a24f1 mprotect_fixup+0x151 ([kernel.kallsyms]) ffffffff9831fa83 setup_arg_pages+0x123 ([kernel.kallsyms]) ffffffff9839691f load_elf_binary+0x39f ([kernel.kallsyms]) ffffffff98e00c45 __x86_indirect_thunk_rax+0x5 ([kernel.kallsyms]) ffffffff98e00c45 __x86_indirect_thunk_rax+0x5 ([kernel.kallsyms])
uname 28642 168664.856392: 10000 cycles:
ffffffff982a385b move_page_tables+0xbcb ([kernel.kallsyms]) ffffffff9831f889 shift_arg_pages+0xa9 ([kernel.kallsyms]) ffffffff9831fb4f setup_arg_pages+0x1ef ([kernel.kallsyms]) ffffffff9839691f load_elf_binary+0x39f ([kernel.kallsyms]) ffffffff98e00c45 __x86_indirect_thunk_rax+0x5 ([kernel.kallsyms])
#
--- tools/perf/util/intel-pt.c | 68 ++++++++++++++++++++++++++++++++++---- 1 file changed, 61 insertions(+), 7 deletions(-)
tools/perf/util/intel-pt.c +61 -7
@@ -124,6 +124,8 @@ struct intel_pt {struct range *time_ranges;unsigned int range_cnt; + +struct ip_callchain *chain; };enum switch_state {
@@ -868,6 +870,45 @@ static u64 intel_pt_ns_to_ticks(const struct intel_pt *pt, u64 ns) pt->tc.time_mult; } +static struct ip_callchain *intel_pt_alloc_chain (struct intel_pt *pt) +{ +size_t sz =sizeof (struct ip_callchain ); + +/* Add 1 to callchain_sz for callchain context */ + sz += (pt->synth_opts.callchain_sz +1 ) *sizeof (u64); +return zalloc (sz); +} + +static int intel_pt_callchain_init (struct intel_pt *pt) +{ +struct evsel *evsel; + +evlist__for_each_entry (pt->session->evlist, evsel) { +if (!(evsel->core.attr.sample_type &PERF_SAMPLE_CALLCHAIN )) + evsel->synth_sample_type |=PERF_SAMPLE_CALLCHAIN ; + } + + pt->chain =intel_pt_alloc_chain (pt); +if (!pt->chain) +return -ENOMEM ; + +return 0 ; +} + +static void intel_pt_add_callchain (struct intel_pt *pt, +struct perf_sample *sample) +{ +struct thread *thread =machine__findnew_thread (pt->machine, + sample->pid, + sample->tid); + +thread_stack__sample_late (thread, sample->cpu, pt->chain, + pt->synth_opts.callchain_sz +1 , sample->ip, + pt->kernel_start); + + sample->callchain = pt->chain; +} +static struct intel_pt_queue *intel_pt_alloc_queue (struct intel_pt *pt,unsigned int queue_nr) {
@@ -880,11 +921,7 @@ static struct intel_pt_queue *intel_pt_alloc_queue(struct intel_pt *pt,return NULL ;if (pt->synth_opts.callchain) { -size_t sz =sizeof (struct ip_callchain ); - -/* Add 1 to callchain_sz for callchain context */ - sz += (pt->synth_opts.callchain_sz +1 ) *sizeof (u64); - ptq->chain =zalloc (sz); + ptq->chain =intel_pt_alloc_chain (pt);if (!ptq->chain) goto out_free; }
@@ -1992,7 +2029,8 @@ static int intel_pt_sample(struct intel_pt_queue *ptq)if (!(state->type &INTEL_PT_BRANCH ))return 0 ; -if (pt->synth_opts.callchain || pt->synth_opts.thread_stack) +if (pt->synth_opts.callchain || pt->synth_opts.add_callchain || + pt->synth_opts.thread_stack)thread_stack__event (ptq->thread, ptq->cpu, ptq->flags, state->from_ip, state->to_ip, ptq->insn_len, state->trace_nr);
@@ -2639,6 +2677,11 @@ static int intel_pt_process_event(struct perf_session *session,if (err)return err; +if (event->header.type ==PERF_RECORD_SAMPLE ) { +if (pt->synth_opts.add_callchain && !sample->callchain) +intel_pt_add_callchain (pt, sample); + } +if (event->header.type ==PERF_RECORD_AUX && (event->aux.flags &PERF_AUX_FLAG_TRUNCATED ) && pt->synth_opts.errors) {
@@ -2710,6 +2753,7 @@ static void intel_pt_free(struct perf_session *session) session->auxtrace =NULL ;thread__put (pt->unknown_thread);addr_filters__exit (&pt->filts); +zfree (&pt->chain);zfree (&pt->filter);zfree (&pt->time_ranges);free (pt);
@@ -3348,6 +3392,7 @@ int intel_pt_process_auxtrace_info(union perf_event *event, !session->itrace_synth_opts->inject) { pt->synth_opts.branches = false; pt->synth_opts.callchain = true; + pt->synth_opts.add_callchain = true; } pt->synth_opts.thread_stack = session->itrace_synth_opts->thread_stack;
@@ -3380,14 +3425,22 @@ int intel_pt_process_auxtrace_info(union perf_event *event, pt->branches_filter |=PERF_IP_FLAG_RETURN |PERF_IP_FLAG_TRACE_BEGIN ; -if (pt->synth_opts.callchain && !symbol_conf.use_callchain) { +if ((pt->synth_opts.callchain || pt->synth_opts.add_callchain) && + !symbol_conf.use_callchain) { symbol_conf.use_callchain = true;if (callchain_register_param (&callchain_param) <0 ) { symbol_conf.use_callchain = false; pt->synth_opts.callchain = false; + pt->synth_opts.add_callchain = false; } } +if (pt->synth_opts.add_callchain) { + err =intel_pt_callchain_init (pt); +if (err) + goto err_delete_thread; + } + err =intel_pt_synth_events (pt, session);if (err) goto err_delete_thread;
@@ -3410,6 +3463,7 @@ int intel_pt_process_auxtrace_info(union perf_event *event,return 0 ; err_delete_thread: +zfree (&pt->chain);thread__zput (pt->unknown_thread); err_free_queues:intel_pt_log_disable ();
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 37/60 perf evsel: Move and globalize perf_evsel__find_pmu() and perf_evsel__is_aux_event()
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Move and globalize 2 functions from the auxtrace specific sources so that they can be reused.
--- tools/perf/util/auxtrace.c | 19 ------------------- tools/perf/util/evsel.h | 3 +++ tools/perf/util/pmu.c | 20 ++++++++++++++++++++ 3 files changed, 23 insertions(+), 19 deletions(-)
tools/perf/util/auxtrace.c +0 -19
@@ -58,25 +58,6 @@#include "symbol/kallsyms.h" #include <internal/lib.h> -static struct perf_pmu *perf_evsel__find_pmu (struct evsel *evsel) -{ -struct perf_pmu *pmu =NULL ; - -while ((pmu =perf_pmu__scan (pmu)) !=NULL ) { -if (pmu->type == evsel->core.attr.type) -break ; - } - -return pmu; -} - -static bool perf_evsel__is_aux_event (struct evsel *evsel) -{ -struct perf_pmu *pmu =perf_evsel__find_pmu (evsel); - -return pmu && pmu->auxtrace; -} -/* * Make a group from 'leader' to 'last', requiring that the events were not * already grouped to a different leader.
tools/perf/util/evsel.h +3 -0
@@ -158,6 +158,9 @@ int perf_evsel__object_config(size_t object_size,int (*init)(struct evsel *evsel), void (*fini)(struct evsel *evsel)); +struct perf_pmu *perf_evsel__find_pmu (struct evsel *evsel); +bool perf_evsel__is_aux_event (struct evsel *evsel); +struct evsel *perf_evsel__new_idx (struct perf_event_attr *attr,int idx);static inline struct evsel *evsel__new (struct perf_event_attr *attr)
tools/perf/util/pmu.c +20 -0
@@ -18,6 +18,7 @@#include <regex.h> #include <perf/cpumap.h> #include "debug.h" +#include "evsel.h" #include "pmu.h" #include "parse-events.h" #include "header.h"
@@ -884,6 +885,25 @@ struct perf_pmu *perf_pmu__scan(struct perf_pmu *pmu)return NULL ; } +struct perf_pmu *perf_evsel__find_pmu (struct evsel *evsel) +{ +struct perf_pmu *pmu =NULL ; + +while ((pmu =perf_pmu__scan (pmu)) !=NULL ) { +if (pmu->type == evsel->core.attr.type) +break ; + } + +return pmu; +} + +bool perf_evsel__is_aux_event (struct evsel *evsel) +{ +struct perf_pmu *pmu =perf_evsel__find_pmu (evsel); + +return pmu && pmu->auxtrace; +} +struct perf_pmu *perf_pmu__find (const char *name) {struct perf_pmu *pmu;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 38/60 perf evlist: Move leader-sampling configuration
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Move leader-sampling configuration in preparation for adding support for leader sampling with AUX area events.
Committer notes:
It only makes sense when configuring an evsel that is part of an evlist, so the only case where it is called outside perf_evlist__config(), in some 'perf test' entry, is safe, and even there we should just use perf_evlist__config(), but since in that case we have just one evsel in the evlist, it is equivalent.
Also fixed up this problem:
util/record.c: In function ‘perf_evlist__config’:
util/record.c:223:3: error: too many arguments to function ‘perf_evsel__config_leader_sampling’
223 | perf_evsel__config_leader_sampling(evsel, evlist);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ util/record.c:170:13: note: declared here
170 | static void perf_evsel__config_leader_sampling(struct evsel *evsel)
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--- tools/perf/util/evsel.c | 19 ------------------- tools/perf/util/record.c | 29 +++++++++++++++++++++++++++++ 2 files changed, 29 insertions(+), 19 deletions(-)
tools/perf/util/evsel.c +0 -19
@@ -1002,25 +1002,6 @@ void perf_evsel__config(struct evsel *evsel, struct record_opts *opts, } } -/* -* Disable sampling for all group members other -* than leader in case leader 'leads' the sampling. -*/ -if ((leader != evsel) && leader->sample_read) { - attr->freq =0 ; - attr->sample_freq =0 ; - attr->sample_period =0 ; - attr->write_backward =0 ; - -/* -* We don't get sample for slave events, we make them -* when delivering group leader sample. Set the slave -* event to follow the master sample_type to ease up -* report. -*/ - attr->sample_type = leader->core.attr.sample_type; - } -if (opts->no_samples) attr->sample_freq =0 ;
tools/perf/util/record.c +29 -0
@@ -167,6 +167,31 @@ bool perf_can_aux_sample(void)return true; } +static void perf_evsel__config_leader_sampling (struct evsel *evsel) +{ +struct perf_event_attr *attr = &evsel->core.attr; +struct evsel *leader = evsel->leader; + +/* +* Disable sampling for all group members other +* than leader in case leader 'leads' the sampling. +*/ +if (leader != evsel && leader->sample_read) { + attr->freq =0 ; + attr->sample_freq =0 ; + attr->sample_period =0 ; + attr->write_backward =0 ; + +/* +* We don't get sample for slave events, we make them +* when delivering group leader sample. Set the slave +* event to follow the master sample_type to ease up +* report. +*/ + attr->sample_type = leader->core.attr.sample_type; + } +} +void perf_evlist__config (struct evlist *evlist,struct record_opts *opts,struct callchain_param *callchain) {
@@ -193,6 +218,10 @@ void perf_evlist__config(struct evlist *evlist, struct record_opts *opts, evsel->core.attr.comm_exec =1 ; } +/* Configure leader sampling here now that the sample type is known */ +evlist__for_each_entry (evlist, evsel) +perf_evsel__config_leader_sampling (evsel); +if (opts->full_auxtrace) {/* * Need to be able to synthesize and parse selected events with
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 39/60 perf evsel: Rearrange perf_evsel__config_leader_sampling()
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
In preparation for adding support for leader sampling with AUX area events.
--- tools/perf/util/record.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
tools/perf/util/record.c +13 -13
@@ -172,24 +172,24 @@ static void perf_evsel__config_leader_sampling(struct evsel *evsel)struct perf_event_attr *attr = &evsel->core.attr;struct evsel *leader = evsel->leader; +if (leader == evsel || !leader->sample_read) +return ; +/* * Disable sampling for all group members other * than leader in case leader 'leads' the sampling. */ -if (leader != evsel && leader->sample_read) { - attr->freq =0 ; - attr->sample_freq =0 ; - attr->sample_period =0 ; - attr->write_backward =0 ; + attr->freq =0 ; + attr->sample_freq =0 ; + attr->sample_period =0 ; + attr->write_backward =0 ; -/* -* We don't get sample for slave events, we make them -* when delivering group leader sample. Set the slave -* event to follow the master sample_type to ease up -* report. -*/ - attr->sample_type = leader->core.attr.sample_type; - } +/* +* We don't get a sample for slave events, we make them when delivering +* the group leader sample. Set the slave event to follow the master +* sample_type to ease up reporting. +*/ + attr->sample_type = leader->core.attr.sample_type; } void perf_evlist__config(struct evlist *evlist, struct record_opts *opts,
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 40/60 perf evlist: Allow multiple read formats
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
Tools find the correct evsel, and therefore read format, using the event ID, so it isn't necessary for all read formats to be the same. In the case of leader-sampling of AUX area events, dummy tracking events will have a different read format, so relax the validation to become a debug message only.
--- tools/perf/util/evlist.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
tools/perf/util/evlist.c +4 -2
@@ -1131,8 +1131,10 @@ bool perf_evlist__valid_read_format(struct evlist *evlist)u64 sample_type = first->core.attr.sample_type;evlist__for_each_entry (evlist, pos) { -if (read_format != pos->core.attr.read_format) -return false; +if (read_format != pos->core.attr.read_format) { +pr_debug ("Read format differs %#" PRIx64" vs %#" PRIx64"\n" , + read_format, (u64 )pos->core.attr.read_format); + } }/* PERF_SAMPLE_READ imples PERF_FORMAT_ID. */
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 41/60 perf tools: Add support for leader-sampling with AUX area events
In reply to Arnaldo Carvalho de MeloFrom: Adrian Hunter <[email protected]>
When AUX area events are used in sampling mode, they must be the group leader, but the group leader is also used for leader-sampling. However, it is not desirable to use an AUX area event as the leader for leader-sampling, because it doesn't have any samples of its own. To support leader-sampling with AUX area events, use the 2nd event of the group as the "leader" for the purposes of leader-sampling.
Example:
# perf record --kcore --aux-sample -e '{intel_pt//,cycles,instructions}:S' -c 10000 uname
[ perf record: Woken up 3 times to write data ]
[ perf record: Captured and wrote 0.786 MB perf.data ]
# perf report
Samples: 380 of events 'anon group { cycles, instructions }', Event count (approx.): 3026164
Children Self Command Shared Object Symbol
+ 38.76% 42.65% 0.00% 0.00% uname [kernel.kallsyms] [k] __x86_indirect_thunk_rax
+ 35.82% 31.33% 0.00% 0.00% uname ld-2.28.so [.] _dl_start_user
+ 34.29% 29.74% 0.55% 0.47% uname ld-2.28.so [.] _dl_start
+ 33.73% 28.62% 1.60% 0.97% uname ld-2.28.so [.] dl_main
+ 33.19% 29.04% 0.52% 0.32% uname ld-2.28.so [.] _dl_sysdep_start
+ 27.83% 33.74% 0.00% 0.00% uname [kernel.kallsyms] [k] do_syscall_64
+ 26.76% 33.29% 0.00% 0.00% uname [kernel.kallsyms] [k] entry_SYSCALL_64_after_hwframe
+ 23.78% 20.33% 5.97% 5.25% uname [kernel.kallsyms] [k] page_fault
+ 23.18% 24.60% 0.00% 0.00% uname libc-2.28.so [.] __libc_start_main
+ 22.64% 24.37% 0.00% 0.00% uname uname [.] _start
+ 21.04% 23.27% 0.00% 0.00% uname uname [.] main
+ 19.48% 18.08% 3.72% 3.64% uname ld-2.28.so [.] _dl_relocate_object
+ 19.47% 21.81% 0.00% 0.00% uname libc-2.28.so [.] setlocale
+ 19.44% 21.56% 0.52% 0.61% uname libc-2.28.so [.] _nl_find_locale
+ 17.87% 19.66% 0.00% 0.00% uname libc-2.28.so [.] _nl_load_locale_from_archive
+ 15.71% 13.73% 0.53% 0.52% uname [kernel.kallsyms] [k] do_page_fault
+ 15.18% 13.21% 1.03% 0.68% uname [kernel.kallsyms] [k] handle_mm_fault
+ 14.15% 12.53% 1.01% 1.12% uname [kernel.kallsyms] [k] __handle_mm_fault
+ 12.03% 9.67% 0.54% 0.32% uname ld-2.28.so [.] _dl_map_object
+ 10.55% 8.48% 0.00% 0.00% uname ld-2.28.so [.] openaux
+ 10.55% 20.20% 0.52% 0.61% uname libc-2.28.so [.] __run_exit_handlersComnmitter notes:
Fixed up this problem:
util/record.c: In function ‘perf_evlist__config’:
util/record.c:256:3: error: too few arguments to function ‘perf_evsel__config_leader_sampling’
256 | perf_evsel__config_leader_sampling(evsel);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ util/record.c:190:13: note: declared here
190 | static void perf_evsel__config_leader_sampling(struct evsel *evsel,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
--- tools/perf/Documentation/perf-list.txt | 3 ++ tools/perf/util/record.c | 45 ++++++++++++++++++++++---- 2 files changed, 42 insertions(+), 6 deletions(-)
tools/perf/Documentation/perf-list.txt +3 -0
@@ -258,6 +258,9 @@ Normally all events in an event group sample, but with :S only the first event (the leader) samples, and it only reads the values of the other events in the group. +However, in the case AUX area events (e.g. Intel PT or CoreSight), the AUX +area event must be the leader, so then the second event samples, not the first. + OPTIONS -------
tools/perf/util/record.c +39 -6
71 lines not on this page · raw message
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 42/60 perf stat: Force error in fallback on :k events
In reply to Arnaldo Carvalho de MeloFrom: Stephane Eranian <[email protected]>
When it is not possible for a non-privilege perf command to monitor at the kernel level (:k), the fallback code forces a :u. That works if the event was previously monitoring both levels. But if the event was already constrained to kernel only, then it does not make sense to restrict it to user only.
Given the code works by exclusion, a kernel only event would have:
attr->exclude_user = 1
The fallback code would add:
attr->exclude_kernel = 1
In the end the end would not monitor in either the user level or kernel level. In other words, it would count nothing.
An event programmed to monitor kernel only cannot be switched to user only without seriously warning the user.
This patch forces an error in this case to make it clear the request cannot really be satisfied.
Behavior with paranoid 1:
$ sudo bash -c "echo 1 > /proc/sys/kernel/perf_event_paranoid" $ perf stat -e cycles:k sleep 1
Performance counter stats for 'sleep 1':
1,520,413 cycles:k
1.002361664 seconds time elapsed
0.002480000 seconds user
0.000000000 seconds sysOld behavior with paranoid 2:
$ sudo bash -c "echo 2 > /proc/sys/kernel/perf_event_paranoid" $ perf stat -e cycles:k sleep 1 Performance counter stats for 'sleep 1':
0 cycles:ku
1.002358127 seconds time elapsed
0.002384000 seconds user
0.000000000 seconds sysNew behavior with paranoid 2:
$ sudo bash -c "echo 2 > /proc/sys/kernel/perf_event_paranoid" $ perf stat -e cycles:k sleep 1 Error: You may not have permission to collect stats.
Consider tweaking /proc/sys/kernel/perf_event_paranoid, which controls use of the performance events system by unprivileged users (without CAP_PERFMON or CAP_SYS_ADMIN).
The current value is 2:
-1: Allow use of (almost) all events by all users
Ignore mlock limit after perf_event_mlock_kb without CAP_IPC_LOCK
Quoted 1 line
= 0: Disallow ftrace function tracepoint by users without
CAP_PERFMONorCAP_SYS_ADMIN
Disallow raw tracepoint access by users without CAP_SYS_PERFMON or CAP_SYS_ADMIN
Quoted 2 lines
= 1: Disallow CPU event access by users without
CAP_PERFMONorCAP_SYS_ADMIN= 2: Disallow kernel profiling by users withoutCAP_PERFMONorCAP_SYS_ADMIN
To make this setting permanent, edit /etc/sysctl.conf too, e.g.:
kernel.perf_event_paranoid = -1
v2 of this patch addresses the review feedback from [email protected].
--- tools/perf/util/evsel.c | 4 ++++ 1 file changed, 4 insertions(+)
tools/perf/util/evsel.c +4 -0
@@ -2427,6 +2427,10 @@ bool perf_evsel__fallback(struct evsel *evsel, int err,char *new_name;const char *sep =":" ; +/* If event has exclude user then don't exclude kernel. */ +if (evsel->core.attr.exclude_user) +return false; +/* Is there already the separator in the name. */ if (strchr (name,'/' ) ||strchr (name,':' ))
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 43/60 tools lib traceevent: Take care of return value of asprintf
In reply to Arnaldo Carvalho de MeloFrom: He Zhe <[email protected]>
According to the API, if memory allocation wasn't possible, or some other error occurs, asprintf will return -1, and the contents of strp below are undefined.
int asprintf(char **strp, const char *fmt, ...);
This patch takes care of return value of asprintf to make it less error prone and prevent the following build warning.
ignoring return value of ‘asprintf’, declared with attribute warn_unused_result [-Wunused-result]
--- tools/lib/traceevent/parse-filter.c | 29 +++++++++++++++++++---------- 1 file changed, 19 insertions(+), 10 deletions(-)
tools/lib/traceevent/parse-filter.c +19 -10
@@ -1958,7 +1958,8 @@ static char *op_to_str(struct tep_event_filter *filter, struct tep_filter_arg *adefault :break ; } -asprintf (&str, val ?"TRUE" :"FALSE" ); +if (asprintf (&str, val ?"TRUE" :"FALSE" ) <0 ) + str =NULL ;break ; } }
@@ -1976,7 +1977,8 @@ static char *op_to_str(struct tep_event_filter *filter, struct tep_filter_arg *abreak ; } -asprintf (&str,"(%s) %s (%s)" , left, op, right); +if (asprintf (&str,"(%s) %s (%s)" , left, op, right) <0 ) + str =NULL ;break ;case TEP_FILTER_OP_NOT :
@@ -1992,10 +1994,12 @@ static char *op_to_str(struct tep_event_filter *filter, struct tep_filter_arg *a right_val =0 ;if (right_val >=0 ) {/* just return the opposite */ -asprintf (&str, right_val ?"FALSE" :"TRUE" ); +if (asprintf (&str, right_val ?"FALSE" :"TRUE" ) <0 ) + str =NULL ;break ; } -asprintf (&str,"%s(%s)" , op, right); +if (asprintf (&str,"%s(%s)" , op, right) <0 ) + str =NULL ;break ;default :
@@ -2011,7 +2015,8 @@ static char *val_to_str(struct tep_event_filter *filter, struct tep_filter_arg * {char *str =NULL ; -asprintf (&str,"%lld" , arg->value.val); +if (asprintf (&str,"%lld" , arg->value.val) <0 ) + str =NULL ;return str; }
@@ -2069,7 +2074,8 @@ static char *exp_to_str(struct tep_event_filter *filter, struct tep_filter_arg *break ; } -asprintf (&str,"%s %s %s" , lstr, op, rstr); +if (asprintf (&str,"%s %s %s" , lstr, op, rstr) <0 ) + str =NULL ; out:free (lstr);free (rstr);
@@ -2113,7 +2119,8 @@ static char *num_to_str(struct tep_event_filter *filter, struct tep_filter_arg *if (!op) op ="<=" ; -asprintf (&str,"%s %s %s" , lstr, op, rstr); +if (asprintf (&str,"%s %s %s" , lstr, op, rstr) <0 ) + str =NULL ;break ;default :
@@ -2148,8 +2155,9 @@ static char *str_to_str(struct tep_event_filter *filter, struct tep_filter_arg *if (!op) op ="!~" ; -asprintf (&str,"%s %s \"%s\"" , - arg->str.field->name, op, arg->str.val); +if (asprintf (&str,"%s %s \"%s\"" , + arg->str.field->name, op, arg->str.val) <0 ) + str =NULL ;break ;default :
@@ -2165,7 +2173,8 @@ static char *arg_to_str(struct tep_event_filter *filter, struct tep_filter_arg *switch (arg->type) {case TEP_FILTER_ARG_BOOLEAN : -asprintf (&str, arg->boolean.value ?"TRUE" :"FALSE" ); +if (asprintf (&str, arg->boolean.value ?"TRUE" :"FALSE" ) <0 ) + str =NULL ;return str;case TEP_FILTER_ARG_OP :
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 44/60 perf pmu: Add support for PMU capabilities
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
The PMU capabilities information, which is located at /sys/bus/event_source/devices/<dev>/caps, is required by perf tool. For example, the max LBR information is required to stitch LBR call stack.
Add perf_pmu__caps_parse() to parse the PMU capabilities information. The information is stored in a list.
The following patch will store the capabilities information in perf header.
Committer notes:
Here's an example of such directories and its files in an i5 7th gen machine:
[root@seventh ~]# ls -lad /sys/bus/event_source/devices/*/caps drwxr-xr-x. 2 root root 0 Apr 14 13:33 /sys/bus/event_source/devices/cpu/caps drwxr-xr-x. 2 root root 0 Apr 14 13:33 /sys/bus/event_source/devices/intel_pt/caps [root@seventh ~]# ls -la /sys/bus/event_source/devices/intel_pt/caps total 0
drwxr-xr-x. 2 root root 0 Apr 14 13:33 .
drwxr-xr-x. 5 root root 0 Apr 14 13:12 ..
-r--r--r--. 1 root root 4096 Apr 16 13:10 cr3_filtering
-r--r--r--. 1 root root 4096 Apr 16 11:42 cycle_thresholds
-r--r--r--. 1 root root 4096 Apr 16 13:10 ip_filtering
-r--r--r--. 1 root root 4096 Apr 16 13:10 max_subleaf
-r--r--r--. 1 root root 4096 Apr 14 13:33 mtc
-r--r--r--. 1 root root 4096 Apr 14 13:33 mtc_periods
-r--r--r--. 1 root root 4096 Apr 16 13:10 num_address_ranges
-r--r--r--. 1 root root 4096 Apr 16 13:10 output_subsys
-r--r--r--. 1 root root 4096 Apr 16 13:10 payloads_lip
-r--r--r--. 1 root root 4096 Apr 16 13:10 power_event_trace
-r--r--r--. 1 root root 4096 Apr 14 13:33 psb_cyc
-r--r--r--. 1 root root 4096 Apr 14 13:33 psb_periods
-r--r--r--. 1 root root 4096 Apr 16 13:10 ptwrite
-r--r--r--. 1 root root 4096 Apr 16 13:10 single_range_output
-r--r--r--. 1 root root 4096 Apr 16 12:03 topa_multiple_entries -r--r--r--. 1 root root 4096 Apr 16 13:10 topa_output
[root@seventh ~]# cat /sys/bus/event_source/devices/intel_pt/caps/topa_output 1
[root@seventh ~]# cat /sys/bus/event_source/devices/intel_pt/caps/topa_multiple_entries 1
[root@seventh ~]# cat /sys/bus/event_source/devices/intel_pt/caps/mtc
1
[root@seventh ~]# cat /sys/bus/event_source/devices/intel_pt/caps/power_event_trace 0
[root@seventh ~]#
[root@seventh ~]# ls -la /sys/bus/event_source/devices/cpu/caps/ total 0
drwxr-xr-x. 2 root root 0 Apr 14 13:33 .
drwxr-xr-x. 6 root root 0 Apr 14 13:12 ..
-r--r--r--. 1 root root 4096 Apr 16 13:10 branches -r--r--r--. 1 root root 4096 Apr 14 13:33 max_precise -r--r--r--. 1 root root 4096 Apr 16 13:10 pmu_name [root@seventh ~]# cat /sys/bus/event_source/devices/cpu/caps/max_precise 3
[root@seventh ~]# cat /sys/bus/event_source/devices/cpu/caps/branches 32
[root@seventh ~]# cat /sys/bus/event_source/devices/cpu/caps/pmu_name skylake
[root@seventh ~]#
Wow, first time I've heard about /sys/bus/event_source/devices/cpu/caps/max_precise, I think I'll use it! :-)
--- tools/perf/util/pmu.c | 82 +++++++++++++++++++++++++++++++++++++++++++ tools/perf/util/pmu.h | 9 +++++ 2 files changed, 91 insertions(+)
tools/perf/util/pmu.c +82 -0
@@ -850,6 +850,7 @@ static struct perf_pmu *pmu_lookup(const char *name)INIT_LIST_HEAD (&pmu->format);INIT_LIST_HEAD (&pmu->aliases); +INIT_LIST_HEAD (&pmu->caps);list_splice (&format, &pmu->format);list_splice (&aliases, &pmu->aliases);list_add_tail (&pmu->list, &pmus);
@@ -1594,3 +1595,84 @@ int perf_pmu__scan_file(struct perf_pmu *pmu, const char *name, const char *fmt,va_end (args);return ret; } + +static int perf_pmu__new_caps (struct list_head *list,char *name,char *value) +{ +struct perf_pmu_caps *caps =zalloc (sizeof (*caps)); + +if (!caps) +return -ENOMEM ; + + caps->name =strdup (name); +if (!caps->name) + goto free_caps; + caps->value =strndup (value,strlen (value) -1 ); +if (!caps->value) + goto free_name; +list_add_tail (&caps->list, list); +return 0 ; + +free_name: +zfree (caps->name); +free_caps: +free (caps); + +return -ENOMEM ; +} + +/* +* Reading/parsing the given pmu capabilities, which should be located at: +* /sys/bus/event_source/devices/<dev>/caps as sysfs group attributes. +* Return the number of capabilities +*/ +int perf_pmu__caps_parse (struct perf_pmu *pmu) +{ +struct stat st; +char caps_path[PATH_MAX ]; +const char *sysfs =sysfs__mountpoint (); +DIR *caps_dir; +struct dirent *evt_ent; +int nr_caps =0 ; + +if (!sysfs) +return -1 ; + +snprintf (caps_path,PATH_MAX , +"%s" EVENT_SOURCE_DEVICE_PATH "%s/caps" , sysfs, pmu->name); + +if (stat (caps_path, &st) <0 ) +return 0 ;/* no error if caps does not exist */ + + caps_dir =opendir (caps_path); +if (!caps_dir) +return -EINVAL ; + +while ((evt_ent =readdir (caps_dir)) !=NULL ) { +char path[PATH_MAX +NAME_MAX +1 ]; +char *name = evt_ent->d_name; +char value[128 ]; +FILE *file; + +if (!strcmp (name,"." ) || !strcmp (name,".." )) +continue ; + +snprintf (path,sizeof (path),"%s/%s" , caps_path, name); + + file =fopen (path,"r" ); +if (!file) +continue ; + +if (!fgets (value,sizeof (value), file) || + (perf_pmu__new_caps (&pmu->caps, name, value) <0 )) { +fclose (file); +continue ; + } + + nr_caps++; +fclose (file); + } + +closedir (caps_dir); + +return nr_caps; +}
tools/perf/util/pmu.h +9 -0
@@ -22,6 +22,12 @@ enum {struct perf_event_attr ; +struct perf_pmu_caps { +char *name; +char *value; +struct list_head list; +}; +struct perf_pmu {char *name;__u32 type;
@@ -33,6 +39,7 @@ struct perf_pmu {struct perf_cpu_map *cpus;struct list_head format;/* HEAD struct perf_pmu_format -> list */ struct list_head aliases;/* HEAD struct perf_pmu_alias -> list */ +struct list_head caps;/* HEAD struct perf_pmu_caps -> list */ struct list_head list;/* ELEM */ };
@@ -107,4 +114,6 @@ bool pmu_uncore_alias_match(const char *pmu_name, const char *name);int perf_pmu__convert_scale (const char *scale,char **end,double *sval); +int perf_pmu__caps_parse (struct perf_pmu *pmu); +#endif /* __PMU_H */
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 45/60 perf doc: allow ASCIIDOC_EXTRA to be an argument
In reply to Arnaldo Carvalho de MeloFrom: Ian Rogers <[email protected]>
This will allow parent makefiles to pass values to asciidoc.
--- tools/perf/Documentation/Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/Documentation/Makefile +2 -2
@@ -48,7 +48,7 @@ man5dir=$(mandir)/man5 man7dir=$(mandir)/man7 ASCIIDOC=asciidoc -ASCIIDOC_EXTRA = --unsafe -f asciidoc.conf +ASCIIDOC_EXTRA += --unsafe -f asciidoc.conf ASCIIDOC_HTML = xhtml11 MANPAGE_XSL = manpage-normal.xsl XMLTO_EXTRA =
@@ -59,7 +59,7 @@ HTML_REF = origin/html ifdef USE_ASCIIDOCTOR ASCIIDOC = asciidoctor -ASCIIDOC_EXTRA = -a compat-mode +ASCIIDOC_EXTRA += -a compat-mode ASCIIDOC_EXTRA += -I. -rasciidoctor-extensions ASCIIDOC_EXTRA += -a mansource="perf" -a manmanual="perf Manual" ASCIIDOC_HTML = xhtml5
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 46/60 perf parser: Add support to specify rXXX event with pmu
In reply to Arnaldo Carvalho de MeloFrom: Jiri Olsa <[email protected]>
The current rXXXX event specification creates event under PERF_TYPE_RAW pmu type. This change allows to use rXXXX within pmu syntax, so it's type is used via the following syntax:
-e 'cpu/r3c/'
-e 'cpum_cf/r0/'
The XXXX number goes directly to perf_event_attr::config the same way as in '-e rXXXX' event. The perf_event_attr::type is filled with pmu type.
Committer testing:
So, lets see what goes in perf_event_attr::config for, say, the 'instructions' PERF_TYPE_HARDWARE (0) event, first we should look at how to encode this event as a PERF_TYPE_RAW event for this specific CPU, an AMD Ryzen 5:
# cat /sys/devices/cpu/events/instructions event=0xc0 #
Then try with it _and_ the instruction, just to see that they are close enough:
# perf stat -e rc0,instructions sleep 1
Performance counter stats for 'sleep 1':
919,794 rc0
919,898 instructions
1.000754579 seconds time elapsed
0.000715000 seconds user
0.000000000 seconds sys#
Now we should try, before this patch, the PMU event encoding:
# perf stat -e cpu/rc0/ sleep 1
event syntax error: 'cpu/rc0/'
\___ unknown termvalid terms: event,edge,inv,umask,cmask,config,config1,config2,name,period,percore #
Now with this patch, the three ways of specifying the 'instructions' CPU counter are accepted:
# perf stat -e cpu/rc0/,rc0,instructions sleep 1
Performance counter stats for 'sleep 1':
892,948 cpu/rc0/
893,052 rc0
893,156 instructions 1.000931819 seconds time elapsed
0.000916000 seconds user
0.000000000 seconds sys#
--- tools/perf/Documentation/perf-list.txt | 5 +++++ tools/perf/tests/parse-events.c | 17 ++++++++++++++++- tools/perf/util/parse-events.l | 1 + tools/perf/util/parse-events.y | 9 +++++++++ 4 files changed, 31 insertions(+), 1 deletion(-)
tools/perf/Documentation/perf-list.txt +5 -0
@@ -115,6 +115,11 @@ raw encoding of 0x1A8 can be used: perf stat -e r1a8 -a sleep 1 perf record -e r1a8 ... +It's also possible to use pmu syntax: + + perf record -e r1a8 -a sleep 1 + perf record -e cpu/r1a8/ ... + You should refer to the processor specific documentation for getting these details. Some of them are referenced in the SEE ALSO section below.
tools/perf/tests/parse-events.c +16 -1
@@ -1356,6 +1356,16 @@ static int test__checkevent_complex_name(struct evlist *evlist)return 0 ; } +static int test__checkevent_raw_pmu (struct evlist *evlist) +{ +struct evsel *evsel =evlist__first (evlist); + +TEST_ASSERT_VAL ("wrong number of entries" ,1 == evlist->core.nr_entries); +TEST_ASSERT_VAL ("wrong type" ,PERF_TYPE_SOFTWARE == evsel->core.attr.type); +TEST_ASSERT_VAL ("wrong config" ,0x1a == evsel->core.attr.config); +return 0 ; +} +static int test__sym_event_slash (struct evlist *evlist) {struct evsel *evsel =evlist__first (evlist);
@@ -1750,7 +1760,12 @@ static struct evlist_test test__events_pmu[] = { .name = "cpu/name='COMPLEX_CYCLES_NAME:orig=cycles,desc=chip-clock-ticks',period=0x1,event=0x2/ukp", .check = test__checkevent_complex_name, .id = 3, - } + }, + { + .name = "software/r1a/", + .check = test__checkevent_raw_pmu, + .id = 4, + }, };struct terms_test {
tools/perf/util/parse-events.l +1 -0
@@ -286,6 +286,7 @@ no-overwrite { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_NOOVERWRITE); } percore { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_PERCORE); } aux-output { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_AUX_OUTPUT); } aux-sample-size { return term(yyscanner, PARSE_EVENTS__TERM_TYPE_AUX_SAMPLE_SIZE); } +r{num_raw_hex} { return raw(yyscanner); } , { return ','; } "/" { BEGIN(INITIAL); return '/'; } {name_minus} { return str(yyscanner, PE_NAME); }
tools/perf/util/parse-events.y +9 -0
@@ -706,6 +706,15 @@ event_term } event_term: +PE_RAW +{ + struct parse_events_term *term; + + ABORT_ON(parse_events_term__num(&term, PARSE_EVENTS__TERM_TYPE_CONFIG, + NULL, $1, false, &@1, NULL)); + $$ = term; +} +| PE_NAME '=' PE_NAME { struct parse_events_term *term;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 47/60 perf header: Support CPU PMU capabilities
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
To stitch LBR call stack, the max LBR information is required. So the CPU PMU capabilities information has to be stored in perf header.
Add a new feature HEADER_CPU_PMU_CAPS for CPU PMU capabilities. Retrieve all CPU PMU capabilities, not just max LBR information.
Add variable max_branches to facilitate future usage.
Committer testing:
# ls -la /sys/devices/cpu/caps/ total 0 drwxr-xr-x. 2 root root 0 Apr 17 10:53 . drwxr-xr-x. 6 root root 0 Apr 17 07:02 .. -r--r--r--. 1 root root 4096 Apr 17 10:53 max_precise # # cat /sys/devices/cpu/caps/max_precise 0 # perf record sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.033 MB perf.data (7 samples) ] # # perf report --header-only | egrep 'cpu(desc|.*capabilities)' # cpudesc : AMD Ryzen 5 3600X 6-Core Processor # cpu pmu capabilities: max_precise=0 #
And then on an Intel machine:
$ ls -la /sys/devices/cpu/caps/ total 0 drwxr-xr-x. 2 root root 0 Apr 17 10:51 . drwxr-xr-x. 6 root root 0 Apr 17 10:04 .. -r--r--r--. 1 root root 4096 Apr 17 11:37 branches -r--r--r--. 1 root root 4096 Apr 17 10:51 max_precise -r--r--r--. 1 root root 4096 Apr 17 11:37 pmu_name $ cat /sys/devices/cpu/caps/max_precise 3 $ cat /sys/devices/cpu/caps/branches 32 $ cat /sys/devices/cpu/caps/pmu_name skylake $ perf record sleep 1 [ perf record: Woken up 1 times to write data ] [ perf record: Captured and wrote 0.001 MB perf.data (8 samples) ] $ perf report --header-only | egrep 'cpu(desc|.*capabilities)' # cpudesc : Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz # cpu pmu capabilities: branches=32, max_precise=3, pmu_name=skylake $
--- .../Documentation/perf.data-file-format.txt | 16 +++ tools/perf/util/env.h | 3 + tools/perf/util/header.c | 108 ++++++++++++++++++ tools/perf/util/header.h | 1 + 4 files changed, 128 insertions(+)
tools/perf/Documentation/perf.data-file-format.txt +16 -0
@@ -373,6 +373,22 @@ struct { Indicates that trace contains records of PERF_RECORD_COMPRESSED type that have perf_events records in compressed form. + HEADER_CPU_PMU_CAPS = 28, + + A list of cpu PMU capabilities. The format of data is as below. + +struct { + u32 nr_cpu_pmu_caps; + { + char name[]; + char value[]; + } [nr_cpu_pmu_caps] +}; + + +Example: + cpu pmu capabilities: branches=32, max_precise=3, pmu_name=icelake + other bits are reserved and should ignored for now HEADER_FEAT_BITS = 256,
tools/perf/util/env.h +3 -0
@@ -48,6 +48,7 @@ struct perf_env {char *cpuid;unsigned long long total_mem;unsigned int msr_pmu_type; +unsigned int max_branches;int nr_cmdline;int nr_sibling_cores;
@@ -57,12 +58,14 @@ struct perf_env {int nr_memory_nodes;int nr_pmu_mappings;int nr_groups; +int nr_cpu_pmu_caps;char *cmdline;const char **cmdline_argv;char *sibling_cores;char *sibling_dies;char *sibling_threads;char *pmu_mappings; +char *cpu_pmu_caps;struct cpu_topology_map *cpu;struct cpu_cache_level *caches;int caches_cnt;
tools/perf/util/header.c +108 -0
132 lines not on this page · raw message
tools/perf/util/header.h +1 -0
@@ -43,6 +43,7 @@ enum {HEADER_BPF_PROG_INFO ,HEADER_BPF_BTF ,HEADER_COMPRESSED , +HEADER_CPU_PMU_CAPS ,HEADER_LAST_FEATURE , HEADER_FEAT_BITS = 256, };
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 48/60 perf machine: Remove the indent in resolve_lbr_callchain_sample
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
The indent is unnecessary in resolve_lbr_callchain_sample. Removing it will make the following patch simpler.
Current code path for resolve_lbr_callchain_sample()
/* LBR only affects the user callchain */
if (i != chain_nr) {
body of the function
....
return 1;
}
return 0;With the patch,
/* LBR only affects the user callchain */
if (i == chain_nr)
return 0;
body of the function
...
return 1;No functional changes.
--- tools/perf/util/machine.c | 123 +++++++++++++++++++------------------- 1 file changed, 63 insertions(+), 60 deletions(-)
tools/perf/util/machine.c +63 -60
@@ -2208,6 +2208,12 @@ static int resolve_lbr_callchain_sample(struct thread *thread,int chain_nr =min (max_stack, (int )chain->nr), i;u8 cpumode =PERF_RECORD_MISC_USER ;u64 ip, branch_from =0 ; +struct branch_stack *lbr_stack; +struct branch_entry *entries; +int lbr_nr, j, k; +bool branch; +struct branch_flags *flags; +int mix_chain_nr;for (i =0 ; i < chain_nr; i++) {if (chain->ips[i] ==PERF_CONTEXT_USER )
@@ -2215,71 +2221,68 @@ static int resolve_lbr_callchain_sample(struct thread *thread, }/* LBR only affects the user callchain */ -if (i != chain_nr) { -struct branch_stack *lbr_stack = sample->branch_stack; -struct branch_entry *entries =perf_sample__branch_entries (sample); -int lbr_nr = lbr_stack->nr, j, k; -bool branch; -struct branch_flags *flags; -/* -* LBR callstack can only get user call chain. -* The mix_chain_nr is kernel call chain -* number plus LBR user call chain number. -* i is kernel call chain number, -* 1 is PERF_CONTEXT_USER, -* lbr_nr + 1 is the user call chain number. -* For details, please refer to the comments -* in callchain__printf -*/ -int mix_chain_nr = i +1 + lbr_nr +1 ; - -for (j =0 ; j < mix_chain_nr; j++) { -int err; - branch = false; - flags =NULL ; +if (i == chain_nr) +return 0 ; -if (callchain_param.order ==ORDER_CALLEE ) { -if (j < i +1 ) - ip = chain->ips[j]; -else if (j > i +1 ) { - k = j - i -2 ; - ip = entries[k].from; - branch = true; - flags = &entries[k].flags; - }else { - ip = entries[0 ].to; - branch = true; - flags = &entries[0 ].flags; - branch_from = entries[0 ].from; - } + lbr_stack = sample->branch_stack; + entries =perf_sample__branch_entries (sample); + lbr_nr = lbr_stack->nr; +/* +* LBR callstack can only get user call chain. +* The mix_chain_nr is kernel call chain +* number plus LBR user call chain number. +* i is kernel call chain number, +* 1 is PERF_CONTEXT_USER, +* lbr_nr + 1 is the user call chain number. +* For details, please refer to the comments +* in callchain__printf +*/ + mix_chain_nr = i +1 + lbr_nr +1 ; + +for (j =0 ; j < mix_chain_nr; j++) { +int err; + + branch = false; + flags =NULL ; + +if (callchain_param.order ==ORDER_CALLEE ) { +if (j < i +1 ) + ip = chain->ips[j]; +else if (j > i +1 ) { + k = j - i -2 ; + ip = entries[k].from; + branch = true; + flags = &entries[k].flags; }else { -if (j < lbr_nr) { - k = lbr_nr - j -1 ; - ip = entries[k].from; - branch = true; - flags = &entries[k].flags; - } -else if (j > lbr_nr) - ip = chain->ips[i +1 - (j - lbr_nr)]; -else { - ip = entries[0 ].to; - branch = true; - flags = &entries[0 ].flags; - branch_from = entries[0 ].from; - } + ip = entries[0 ].to; + branch = true; + flags = &entries[0 ].flags; + branch_from = entries[0 ].from; + } + }else { +if (j < lbr_nr) { + k = lbr_nr - j -1 ; + ip = entries[k].from; + branch = true; + flags = &entries[k].flags; + }else if (j > lbr_nr) + ip = chain->ips[i +1 - (j - lbr_nr)]; +else { + ip = entries[0 ].to; + branch = true; + flags = &entries[0 ].flags; + branch_from = entries[0 ].from; } - - err =add_callchain_ip (thread, cursor, parent, - root_al, &cpumode, ip, - branch, flags,NULL , - branch_from); -if (err) -return (err <0 ) ? err :0 ; } -return 1 ; - } -return 0 ; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + branch, flags,NULL , + branch_from); +if (err) +return (err <0 ) ? err :0 ; + } +return 1 ; } static int find_prev_cpumode(struct ip_callchain *chain, struct thread *thread,
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 49/60 perf machine: Refine the function for LBR call stack reconstruction
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
LBR only collect the user call stack. To reconstruct a call stack, both kernel call stack and user call stack are required. The function resolve_lbr_callchain_sample() mix the kernel call stack and user call stack.
Now, with the help of HW idx, perf tool can reconstruct a more complete call stack by adding some user call stack from previous sample. However, current implementation is hard to be extended to support it.
Current code path for resolve_lbr_callchain_sample()
for (j = 0; j < mix_chain_nr; j++) {
if (ORDER_CALLEE) {
if (kernel callchain)
Fill callchain info
else if (LBR callchain)
Fill callchain info
} else {
if (LBR callchain)
Fill callchain info
else if (kernel callchain)
Fill callchain info
}
add_callchain_ip();
}With the patch,
if (ORDER_CALLEE) {
for (j = 0; j < NUM of kernel callchain) {
Fill callchain info
add_callchain_ip();
}
for (; j < mix_chain_nr) {
Fill callchain info
add_callchain_ip();
}
} else {
for (; j < NUM of LBR callchain) {
Fill callchain info
add_callchain_ip();
}
for (j = 0; j < mix_chain_nr) {
Fill callchain info
add_callchain_ip();
}
}No functional changes.
--- tools/perf/util/machine.c | 111 ++++++++++++++++++++++++++------------ 1 file changed, 76 insertions(+), 35 deletions(-)
tools/perf/util/machine.c +76 -35
@@ -2214,6 +2214,7 @@ static int resolve_lbr_callchain_sample(struct thread *thread,bool branch;struct branch_flags *flags;int mix_chain_nr; +int err;for (i =0 ; i < chain_nr; i++) {if (chain->ips[i] ==PERF_CONTEXT_USER )
@@ -2239,50 +2240,90 @@ static int resolve_lbr_callchain_sample(struct thread *thread,*/ mix_chain_nr = i +1 + lbr_nr +1 ; -for (j =0 ; j < mix_chain_nr; j++) { -int err; - - branch = false; - flags =NULL ; - -if (callchain_param.order ==ORDER_CALLEE ) { -if (j < i +1 ) - ip = chain->ips[j]; -else if (j > i +1 ) { - k = j - i -2 ; - ip = entries[k].from; - branch = true; - flags = &entries[k].flags; - }else { - ip = entries[0 ].to; - branch = true; - flags = &entries[0 ].flags; - branch_from = entries[0 ].from; - } - }else { -if (j < lbr_nr) { - k = lbr_nr - j -1 ; - ip = entries[k].from; - branch = true; - flags = &entries[k].flags; - }else if (j > lbr_nr) - ip = chain->ips[i +1 - (j - lbr_nr)]; -else { - ip = entries[0 ].to; - branch = true; - flags = &entries[0 ].flags; - branch_from = entries[0 ].from; - } +if (callchain_param.order ==ORDER_CALLEE ) { +/* Add kernel ip */ +for (j =0 ; j < i +1 ; j++) { + ip = chain->ips[j]; + branch = false; + flags =NULL ; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + branch, flags,NULL , + branch_from); +if (err) + goto error; } +/* Add LBR ip from first entries.to */ + ip = entries[0 ].to; + branch = true; + flags = &entries[0 ].flags; + branch_from = entries[0 ].from; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + branch, flags,NULL , + branch_from); +if (err) + goto error; +/* Add LBR ip from entries.from one by one. */ +for (j = i +2 ; j < mix_chain_nr; j++) { + k = j - i -2 ; + ip = entries[k].from; + branch = true; + flags = &entries[k].flags; + + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + branch, flags,NULL , + branch_from); +if (err) + goto error; + } + }else { +/* Add LBR ip from entries.from one by one. */ +for (j =0 ; j < lbr_nr; j++) { + k = lbr_nr - j -1 ; + ip = entries[k].from; + branch = true; + flags = &entries[k].flags; + + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + branch, flags,NULL , + branch_from); +if (err) + goto error; + } + +/* Add LBR ip from first entries.to */ + ip = entries[0 ].to; + branch = true; + flags = &entries[0 ].flags; + branch_from = entries[0 ].from; err =add_callchain_ip (thread, cursor, parent, root_al, &cpumode, ip, branch, flags,NULL , branch_from);if (err) -return (err <0 ) ? err :0 ; + goto error; + +/* Add kernel ip */ +for (j = lbr_nr +1 ; j < mix_chain_nr; j++) { + ip = chain->ips[i +1 - (j - lbr_nr)]; + branch = false; + flags =NULL ; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + branch, flags,NULL , + branch_from); +if (err) + goto error; + } }return 1 ; + +error: +return (err <0 ) ? err :0 ; } static int find_prev_cpumode(struct ip_callchain *chain, struct thread *thread,
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 50/60 perf machine: Factor out lbr_callchain_add_kernel_ip()
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
Both caller and callee needs to add kernel ip to callchain. Factor out lbr_callchain_add_kernel_ip() to improve code readability.
--- tools/perf/util/machine.c | 67 ++++++++++++++++++++++++++------------- 1 file changed, 45 insertions(+), 22 deletions(-)
tools/perf/util/machine.c +45 -22
@@ -2190,6 +2190,40 @@ static int remove_loops(struct branch_entry *l, int nr,return nr; } +static int lbr_callchain_add_kernel_ip (struct thread *thread, +struct callchain_cursor *cursor, +struct perf_sample *sample, +struct symbol **parent, +struct addr_location *root_al, +u64 branch_from, +bool callee,int end) +{ +struct ip_callchain *chain = sample->callchain; +u8 cpumode =PERF_RECORD_MISC_USER ; +int err, i; + +if (callee) { +for (i =0 ; i < end +1 ; i++) { + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, chain->ips[i], + false,NULL ,NULL , branch_from); +if (err) +return err; + } +return 0 ; + } + +for (i = end; i >=0 ; i--) { + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, chain->ips[i], + false,NULL ,NULL , branch_from); +if (err) +return err; + } + +return 0 ; +} +/* * Recolve LBR callstack chain sample * Return:
@@ -2242,17 +2276,12 @@ static int resolve_lbr_callchain_sample(struct thread *thread,if (callchain_param.order ==ORDER_CALLEE ) {/* Add kernel ip */ -for (j =0 ; j < i +1 ; j++) { - ip = chain->ips[j]; - branch = false; - flags =NULL ; - err =add_callchain_ip (thread, cursor, parent, - root_al, &cpumode, ip, - branch, flags,NULL , - branch_from); -if (err) - goto error; - } + err =lbr_callchain_add_kernel_ip (thread, cursor, sample, + parent, root_al, branch_from, + true, i); +if (err) + goto error; +/* Add LBR ip from first entries.to */ ip = entries[0 ].to; branch = true;
@@ -2308,17 +2337,11 @@ static int resolve_lbr_callchain_sample(struct thread *thread, goto error;/* Add kernel ip */ -for (j = lbr_nr +1 ; j < mix_chain_nr; j++) { - ip = chain->ips[i +1 - (j - lbr_nr)]; - branch = false; - flags =NULL ; - err =add_callchain_ip (thread, cursor, parent, - root_al, &cpumode, ip, - branch, flags,NULL , - branch_from); -if (err) - goto error; - } + err =lbr_callchain_add_kernel_ip (thread, cursor, sample, + parent, root_al, branch_from, + false, i); +if (err) + goto error; }return 1 ;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 51/60 perf machine: Factor out lbr_callchain_add_lbr_ip()
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
Both caller and callee needs to add ip from LBR to callchain. Factor out lbr_callchain_add_lbr_ip() to improve code readability.
--- tools/perf/util/machine.c | 143 +++++++++++++++++++------------------- 1 file changed, 73 insertions(+), 70 deletions(-)
tools/perf/util/machine.c +73 -70
@@ -2224,6 +2224,74 @@ static int lbr_callchain_add_kernel_ip(struct thread *thread,return 0 ; } +static int lbr_callchain_add_lbr_ip (struct thread *thread, +struct callchain_cursor *cursor, +struct perf_sample *sample, +struct symbol **parent, +struct addr_location *root_al, +u64 *branch_from, +bool callee) +{ +struct branch_stack *lbr_stack = sample->branch_stack; +struct branch_entry *entries =perf_sample__branch_entries (sample); +u8 cpumode =PERF_RECORD_MISC_USER ; +int lbr_nr = lbr_stack->nr; +struct branch_flags *flags; +int err, i; +u64 ip; + +if (callee) { +/* Add LBR ip from first entries.to */ + ip = entries[0 ].to; + flags = &entries[0 ].flags; + *branch_from = entries[0 ].from; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + true, flags,NULL , + *branch_from); +if (err) +return err; + +/* Add LBR ip from entries.from one by one. */ +for (i =0 ; i < lbr_nr; i++) { + ip = entries[i].from; + flags = &entries[i].flags; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + true, flags,NULL , + *branch_from); +if (err) +return err; + } +return 0 ; + } + +/* Add LBR ip from entries.from one by one. */ +for (i = lbr_nr -1 ; i >=0 ; i--) { + ip = entries[i].from; + flags = &entries[i].flags; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + true, flags,NULL , + *branch_from); +if (err) +return err; + } + +/* Add LBR ip from first entries.to */ + ip = entries[0 ].to; + flags = &entries[0 ].flags; + *branch_from = entries[0 ].from; + err =add_callchain_ip (thread, cursor, parent, + root_al, &cpumode, ip, + true, flags,NULL , + *branch_from); +if (err) +return err; + +return 0 ; +} +/* * Recolve LBR callstack chain sample * Return:
@@ -2240,14 +2308,7 @@ static int resolve_lbr_callchain_sample(struct thread *thread, {struct ip_callchain *chain = sample->callchain;int chain_nr =min (max_stack, (int )chain->nr), i; -u8 cpumode =PERF_RECORD_MISC_USER ; -u64 ip, branch_from =0 ; -struct branch_stack *lbr_stack; -struct branch_entry *entries; -int lbr_nr, j, k; -bool branch; -struct branch_flags *flags; -int mix_chain_nr; +u64 branch_from =0 ;int err;for (i =0 ; i < chain_nr; i++) {
@@ -2259,21 +2320,6 @@ static int resolve_lbr_callchain_sample(struct thread *thread,if (i == chain_nr)return 0 ; - lbr_stack = sample->branch_stack; - entries =perf_sample__branch_entries (sample); - lbr_nr = lbr_stack->nr; -/* -* LBR callstack can only get user call chain. -* The mix_chain_nr is kernel call chain -* number plus LBR user call chain number. -* i is kernel call chain number, -* 1 is PERF_CONTEXT_USER, -* lbr_nr + 1 is the user call chain number. -* For details, please refer to the comments -* in callchain__printf -*/ - mix_chain_nr = i +1 + lbr_nr +1 ; -if (callchain_param.order ==ORDER_CALLEE ) {/* Add kernel ip */ err = lbr_callchain_add_kernel_ip(thread, cursor, sample,
@@ -2282,57 +2328,14 @@ static int resolve_lbr_callchain_sample(struct thread *thread,if (err) goto error; -/* Add LBR ip from first entries.to */ - ip = entries[0 ].to; - branch = true; - flags = &entries[0 ].flags; - branch_from = entries[0 ].from; - err =add_callchain_ip (thread, cursor, parent, - root_al, &cpumode, ip, - branch, flags,NULL , - branch_from); + err =lbr_callchain_add_lbr_ip (thread, cursor, sample, parent, + root_al, &branch_from, true);if (err) goto error; -/* Add LBR ip from entries.from one by one. */ -for (j = i +2 ; j < mix_chain_nr; j++) { - k = j - i -2 ; - ip = entries[k].from; - branch = true; - flags = &entries[k].flags; - - err =add_callchain_ip (thread, cursor, parent, - root_al, &cpumode, ip, - branch, flags,NULL , - branch_from); -if (err) - goto error; - } }else { -/* Add LBR ip from entries.from one by one. */ -for (j =0 ; j < lbr_nr; j++) { - k = lbr_nr - j -1 ; - ip = entries[k].from; - branch = true; - flags = &entries[k].flags; - - err =add_callchain_ip (thread, cursor, parent, - root_al, &cpumode, ip, - branch, flags,NULL , - branch_from); -if (err) - goto error; - } - -/* Add LBR ip from first entries.to */ - ip = entries[0 ].to; - branch = true; - flags = &entries[0 ].flags; - branch_from = entries[0 ].from; - err =add_callchain_ip (thread, cursor, parent, - root_al, &cpumode, ip, - branch, flags,NULL , - branch_from); + err =lbr_callchain_add_lbr_ip (thread, cursor, sample, parent, + root_al, &branch_from, false);if (err) goto error;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 52/60 perf thread: Add a knob for LBR stitch approach
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
The LBR stitch approach should be disabled by default. Because
- The stitching approach base on LBR call stack technology. The known limitations of LBR call stack technology still apply to the approach, e.g. Exception handing such as setjmp/longjmp will have calls/returns not match.
- This approach is not foolproof. There can be cases where it creates incorrect call stacks from incorrect matches. There is no attempt to validate any matches in another way.
The 'lbr_stitch_enable' is used to indicate whether enable LBR stitch approach, which is disabled by default. The following patch will introduce a new option for each tools to enable the LBR stitch approach.
--- tools/perf/util/thread.c | 1 + tools/perf/util/thread.h | 3 +++ 2 files changed, 4 insertions(+)
tools/perf/util/thread.c +1 -0
@@ -47,6 +47,7 @@ struct thread *thread__new(pid_t pid, pid_t tid) thread->tid = tid; thread->ppid =-1 ; thread->cpu =-1 ; + thread->lbr_stitch_enable = false;INIT_LIST_HEAD (&thread->namespaces_list);INIT_LIST_HEAD (&thread->comm_list);init_rwsem (&thread->namespaces_lock);
tools/perf/util/thread.h +3 -0
@@ -46,6 +46,9 @@ struct thread {struct srccode_state srccode_state;bool filter;int filter_entry_depth; + +/* LBR call stack stitch */ +bool lbr_stitch_enable; };struct machine ;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 53/60 perf thread: Save previous sample for LBR stitching approach
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
To retrieve the overwritten LBRs from previous sample for LBR stitching approach, perf has to save the previous sample.
Only allocate the struct lbr_stitch once, when LBR stitching approach is enabled and kernel supports hw_idx.
--- tools/perf/util/machine.c | 23 +++++++++++++++++++++++ tools/perf/util/thread.c | 1 + tools/perf/util/thread.h | 12 ++++++++++++ 3 files changed, 36 insertions(+)
tools/perf/util/machine.c +23 -0
@@ -2292,6 +2292,21 @@ static int lbr_callchain_add_lbr_ip(struct thread *thread,return 0 ; } +static bool alloc_lbr_stitch (struct thread *thread) +{ +if (thread->lbr_stitch) +return true; + + thread->lbr_stitch =zalloc (sizeof (*thread->lbr_stitch)); +if (!thread->lbr_stitch) + goto err; + +err: +pr_warning ("Failed to allocate space for stitched LBRs. Disable LBR stitch\n" ); + thread->lbr_stitch_enable = false; +return false; +} +/* * Recolve LBR callstack chain sample * Return:
@@ -2308,6 +2323,7 @@ static int resolve_lbr_callchain_sample(struct thread *thread, {struct ip_callchain *chain = sample->callchain;int chain_nr =min (max_stack, (int )chain->nr), i; +struct lbr_stitch *lbr_stitch;u64 branch_from =0 ;int err;
@@ -2320,6 +2336,13 @@ static int resolve_lbr_callchain_sample(struct thread *thread,if (i == chain_nr)return 0 ; +if (thread->lbr_stitch_enable && !sample->no_hw_idx && +alloc_lbr_stitch (thread)) { + lbr_stitch = thread->lbr_stitch; + +memcpy (&lbr_stitch->prev_sample, sample,sizeof (*sample)); + } +if (callchain_param.order ==ORDER_CALLEE ) {/* Add kernel ip */ err = lbr_callchain_add_kernel_ip(thread, cursor, sample,
tools/perf/util/thread.c +1 -0
@@ -111,6 +111,7 @@ void thread__delete(struct thread *thread)exit_rwsem (&thread->namespaces_lock);exit_rwsem (&thread->comm_lock); +thread__free_stitch_list (thread);free (thread); }
tools/perf/util/thread.h +12 -0
@@ -5,6 +5,7 @@#include <linux/refcount.h> #include <linux/rbtree.h> #include <linux/list.h> +#include <linux/zalloc.h> #include <stdio.h> #include <unistd.h> #include <sys/types.h>
@@ -13,6 +14,7 @@#include <strlist.h> #include <intlist.h> #include "rwsem.h" +#include "event.h" struct addr_location ;struct map ;
@@ -20,6 +22,10 @@ struct perf_record_namespaces;struct thread_stack ;struct unwind_libunwind_ops ; +struct lbr_stitch { +struct perf_sample prev_sample; +}; + struct thread { union { struct rb_node rb_node;
@@ -49,6 +55,7 @@ struct thread {/* LBR call stack stitch */ bool lbr_stitch_enable; +struct lbr_stitch *lbr_stitch; };struct machine ;
@@ -145,4 +152,9 @@ static inline bool thread__is_filtered(struct thread *thread) return false; } +static inline void thread__free_stitch_list (struct thread *thread) +{ +zfree (&thread->lbr_stitch); +} +#endif /* __PERF_THREAD_H */
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 54/60 perf callchain: Save previous cursor nodes for LBR stitching approach
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
The cursor nodes which generates from sample are eventually added into callchain. To avoid generating cursor nodes from previous samples again, the previous cursor nodes are also saved for LBR stitching approach.
Some option, e.g. hide-unresolved, may hide some LBRs. Add a variable 'valid' in struct callchain_cursor_node to indicate this case. The LBR stitching approach will only append the valid cursor nodes from previous samples later.
--- tools/perf/util/callchain.h | 3 ++ tools/perf/util/machine.c | 76 +++++++++++++++++++++++++++++++++++-- tools/perf/util/thread.h | 8 ++++ 3 files changed, 83 insertions(+), 4 deletions(-)
tools/perf/util/callchain.h +3 -0
@@ -143,6 +143,9 @@ struct callchain_cursor_node {u64 ip;struct map_symbol ms;const char *srcline; +/* Indicate valid cursor node for LBR stitch */ +bool valid; +bool branch;struct branch_flags branch_flags;u64 branch_from;
tools/perf/util/machine.c +72 -4
137 lines not on this page · raw message
tools/perf/util/thread.h +8 -0
@@ -15,6 +15,7 @@#include <intlist.h> #include "rwsem.h" #include "event.h" +#include "callchain.h" struct addr_location ;struct map ;
@@ -24,6 +25,7 @@ struct unwind_libunwind_ops;struct lbr_stitch {struct perf_sample prev_sample; +struct callchain_cursor_node *prev_lbr_cursor; };struct thread {
@@ -154,6 +156,12 @@ static inline bool thread__is_filtered(struct thread *thread)static inline void thread__free_stitch_list (struct thread *thread) { +struct lbr_stitch *lbr_stitch = thread->lbr_stitch; + +if (!lbr_stitch) +return ; + +zfree (&lbr_stitch->prev_lbr_cursor);zfree (&thread->lbr_stitch); }
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 55/60 perf callchain: Stitch LBR call stack
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
In LBR call stack mode, the depth of reconstructed LBR call stack limits to the number of LBR registers.
For example, on skylake, the depth of reconstructed LBR call stack is always <= 32.
# To display the perf.data header info, please use # --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 6K of event 'cycles'
# Event count (approx.): 6487119731
#
# Children Self Command Shared Object Symbol # ........ ........ ............... .................. # ................................
99.97% 99.97% tchain_edit tchain_edit [.] f43
|
--99.64%--f11
f12
f13
f14
f15
f16
f17
f18
f19
f20
f21
f22
f23
f24
f25
f26
f27
f28
f29
f30
f31
f32
f33
f34
f35
f36
f37
f38
f39
f40
f41
f42
f43For a call stack which is deeper than LBR limit, HW will overwrite the LBR register with oldest branch. Only partial call stacks can be reconstructed.
However, the overwritten LBRs may still be retrieved from previous sample. At that moment, HW hasn't overwritten the LBR registers yet. Perf tools can stitch those overwritten LBRs on current call stacks to get a more complete call stack.
To determine if LBRs can be stitched, perf tools need to compare current sample with previous sample.
- They should have identical LBR records (Same from, to and flags values, and the same physical index of LBR registers).
- The searching starts from the base-of-stack of current sample.
Once perf determines to stitch the previous LBRs, the corresponding LBR cursor nodes will be copied to 'lists'. The 'lists' is to track the LBR cursor nodes which are going to be stitched.
When the stitching is over, the nodes will not be freed immediately. They will be moved to 'free_lists'. Next stitching may reuse the space. Both 'lists' and 'free_lists' will be freed when all samples are processed.
Committer notes:
Fix the intel-pt.c initialization of the union with 'struct branch_flags', that breaks the build with its unnamed union on older gcc versions.
Uninline thread__free_stitch_list(), as it grew big and started dragging includes to thread.h, so move it to thread.c where what it needs in terms of headers are already there.
This fixes the build in several systems such as debian:experimental when cross building to the MIPS32 architecture, i.e. in the other cases what was needed was being included by sheer luck.
In file included from builtin-sched.c:11:
util/thread.h: In function 'thread__free_stitch_list':
util/thread.h:169:3: error: implicit declaration of function 'free' [-Werror=implicit-function-declaration]
169 | free(pos);
| ^~~~
util/thread.h:169:3: error: incompatible implicit declaration of built-in function 'free' [-Werror]
util/thread.h:19:1: note: include '<stdlib.h>' or provide a declaration of 'free'
18 | #include "callchain.h"
+++ |+#include <stdlib.h>
19 |
util/thread.h:174:3: error: incompatible implicit declaration of built-in function 'free' [-Werror]
174 | free(pos);
| ^~~~ util/thread.h:174:3: note: include '<stdlib.h>' or provide a declaration of 'free'
--- tools/perf/util/branch.h | 19 +++-- tools/perf/util/callchain.h | 5 ++ tools/perf/util/intel-pt.c | 17 +++-- tools/perf/util/machine.c | 139 +++++++++++++++++++++++++++++++++++- tools/perf/util/thread.c | 22 ++++++ tools/perf/util/thread.h | 14 +--- 6 files changed, 188 insertions(+), 28 deletions(-)
tools/perf/util/branch.h +12 -7
@@ -15,13 +15,18 @@#include "event.h" struct branch_flags { -u64 mispred:1 ; -u64 predicted:1 ; -u64 in_tx:1 ; -u64 abort:1 ; -u64 cycles:16 ; -u64 type:4 ; -u64 reserved:40 ; +union { +u64 value; +struct { +u64 mispred:1 ; +u64 predicted:1 ; +u64 in_tx:1 ; +u64 abort:1 ; +u64 cycles:16 ; +u64 type:4 ; +u64 reserved:40 ; + }; + }; };struct branch_info {
tools/perf/util/callchain.h +5 -0
@@ -154,6 +154,11 @@ struct callchain_cursor_node {struct callchain_cursor_node *next; }; +struct stitch_list { +struct list_head node; +struct callchain_cursor_node cursor; +}; +struct callchain_cursor {u64 nr;struct callchain_cursor_node *first;
tools/perf/util/intel-pt.c +8 -9
@@ -1717,15 +1717,14 @@ static u64 intel_pt_lbr_flags(u64 info)union {struct branch_flags flags;u64 result; - } u = { - .flags = { - .mispred = !!(info &LBR_INFO_MISPRED ), - .predicted = !(info &LBR_INFO_MISPRED ), - .in_tx = !!(info &LBR_INFO_IN_TX ), - .abort = !!(info &LBR_INFO_ABORT ), - .cycles = info &LBR_INFO_CYCLES , - } - }; + } u; + + u.result =0 ; + u.flags.mispred = !!(info &LBR_INFO_MISPRED ); + u.flags.predicted = !(info &LBR_INFO_MISPRED ); + u.flags.in_tx = !!(info &LBR_INFO_IN_TX ); + u.flags.abort = !!(info &LBR_INFO_ABORT ); + u.flags.cycles = info &LBR_INFO_CYCLES ;return u.result; }
tools/perf/util/machine.c +138 -1
176 lines not on this page · raw message
tools/perf/util/thread.c +22 -0
@@ -454,3 +454,25 @@ int thread__memcpy(struct thread *thread, struct machine *machine,return dso__data_read_offset (al.map->dso, machine, offset, buf, len); } + +void thread__free_stitch_list (struct thread *thread) +{ +struct lbr_stitch *lbr_stitch = thread->lbr_stitch; +struct stitch_list *pos, *tmp; + +if (!lbr_stitch) +return ; + +list_for_each_entry_safe (pos, tmp, &lbr_stitch->lists, node) { +list_del_init (&pos->node); +free (pos); + } + +list_for_each_entry_safe (pos, tmp, &lbr_stitch->free_lists, node) { +list_del_init (&pos->node); +free (pos); + } + +zfree (&lbr_stitch->prev_lbr_cursor); +zfree (&thread->lbr_stitch); +}
tools/perf/util/thread.h +3 -11
@@ -5,7 +5,6 @@#include <linux/refcount.h> #include <linux/rbtree.h> #include <linux/list.h> -#include <linux/zalloc.h> #include <stdio.h> #include <unistd.h> #include <sys/types.h>
@@ -24,6 +23,8 @@ struct thread_stack;struct unwind_libunwind_ops ;struct lbr_stitch { +struct list_head lists; +struct list_head free_lists;struct perf_sample prev_sample;struct callchain_cursor_node *prev_lbr_cursor; };
@@ -154,15 +155,6 @@ static inline bool thread__is_filtered(struct thread *thread) return false; } -static inline void thread__free_stitch_list (struct thread *thread) -{ -struct lbr_stitch *lbr_stitch = thread->lbr_stitch; - -if (!lbr_stitch) -return ; - -zfree (&lbr_stitch->prev_lbr_cursor); -zfree (&thread->lbr_stitch); -} +void thread__free_stitch_list (struct thread *thread);#endif /* __PERF_THREAD_H */
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 56/60 perf report: Add option to enable the LBR stitching approach
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
With the LBR stitching approach, the reconstructed LBR call stack can break the HW limitation. However, it may reconstruct invalid call stacks in some cases, e.g. exception handing such as setjmp/longjmp. Also, it may impact the processing time especially when the number of samples with stitched LBRs are huge.
Add an option to enable the approach.
# To display the perf.data header info, please use # --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 6K of event 'cycles'
# Event count (approx.): 6492797701
#
# Children Self Command Shared Object Symbol
# ........ ........ ............... ..................
# .................................
#
99.99% 99.99% tchain_edit tchain_edit [.] f43
|
---main
f1
f2
f3
f4
f5
f6
f7
f8
f9
f10
f11
f12
f13
f14
f15
f16
f17
f18
f19
f20
f21
f22
f23
f24
f25
f26
f27
f28
f29
f30
f31
|
--99.65%--f32
f33
f34
f35
f36
f37
f38
f39
f40
f41
f42
f43Committer testing:
$ perf record --call-graph lbr /wb/tchain_edit [ perf record: Woken up 23 times to write data ] [ perf record: Captured and wrote 5.578 MB perf.data (6839 samples) ] $ perf report --header-only | egrep 'cpu(desc|.*capabilities)' # cpudesc : Intel(R) Core(TM) i5-7500 CPU @ 3.40GHz # cpu pmu capabilities: branches=32, max_precise=3, pmu_name=skylake $
Before:
$ perf report --no-children --stdio
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 6K of event 'cycles:u'
# Event count (approx.): 6459523879
#
# Overhead Command Shared Object Symbol
# ........ ........... ................ .......................
#
99.95% tchain_edit tchain_edit [.] f43
|
--99.92%--f43
f42
f41
f40
f39
f38
f37
f36
f35
f34
f33
f32
f31
f30
f29
f28
f27
f26
f25
f24
f23
f22
f21
f20
f19
f18
f17
f16
f15
f14
f13
f12
f11 0.03% tchain_edit tchain_edit [.] f42
0.01% tchain_edit tchain_edit [.] f41
0.00% tchain_edit tchain_edit [.] f31
0.00% tchain_edit ld-2.29.so [.] _dl_relocate_object
0.00% tchain_edit ld-2.29.so [.] memmove
0.00% tchain_edit [unknown] [k] 0xffffffff93a00b17After:
$ perf report --stitch-lbr --no-children --stdio
# To display the perf.data header info, please use --header/--header-only options.
#
#
# Total Lost Samples: 0
#
# Samples: 6K of event 'cycles:u'
# Event count (approx.): 6459496645
#
# Overhead Command Shared Object Symbol
# ........ ........... ................ ........................
#
99.97% tchain_edit tchain_edit [.] f43
|
--99.93%--f43
f42
f41
f40
f39
f38
f37
f36
f35
f34
f33
f32
f31
f30
f29
f28
f27
f26
f25
f24
f23
f22
f21
f20
f19
f18
f17
f16
f15
f14
f13
f12
f11
f10
f9
f8
f7
f6
f5
f4
f3
f2
f1
main
__libc_start_main 0.02% tchain_edit [unknown] [k] 0xffffffff93a00b17
0.01% tchain_edit tchain_edit [.] f31
0.00% tchain_edit ld-2.29.so [.] _dl_important_hwcaps--- tools/perf/Documentation/perf-report.txt | 11 +++++++++++ tools/perf/builtin-report.c | 12 ++++++++++++ 2 files changed, 23 insertions(+)
tools/perf/Documentation/perf-report.txt +11 -0
@@ -488,6 +488,17 @@ include::itrace.txt[] This option extends the perf report to show reference callgraphs, which collected by reference event, in no callgraph event. +--stitch-lbr:: + Show callgraph with stitched LBRs, which may have more complete + callgraph. The perf.data file must have been obtained using + perf record --call-graph lbr. + Disabled by default. In common cases with call stack overflows, + it can recreate better call stacks than the default lbr call stack + output. But this approach is not full proof. There can be cases + where it creates incorrect call stacks from incorrect matches. + The known limitations include exception handing such as + setjmp/longjmp will have calls/returns not match. + --socket-filter:: Only report the samples on the processor socket that match with this filter
tools/perf/builtin-report.c +12 -0
@@ -84,6 +84,7 @@ struct report {bool header_only;bool nonany_branch_mode;bool group_set; +bool stitch_lbr;int max_stack;struct perf_read_values show_threads_values;struct annotation_options annotation_opts;
@@ -267,6 +268,9 @@ static int process_sample_event(struct perf_tool *tool,return -1 ; } +if (rep->stitch_lbr) + al.thread->lbr_stitch_enable = true; +if (symbol_conf.hide_unresolved && al.sym ==NULL ) goto out_put;
@@ -408,6 +412,12 @@ static int report__setup_sample_type(struct report *rep) callchain_param.record_mode =CALLCHAIN_FP ; } +if (rep->stitch_lbr && (callchain_param.record_mode !=CALLCHAIN_LBR )) { +ui__warning ("Can't find LBR callchain. Switch off --stitch-lbr.\n" +"Please apply --call-graph lbr when recording.\n" ); + rep->stitch_lbr = false; + } +/* ??? handle more cases than just ANY? */ if (!(perf_evlist__combined_branch_type (session->evlist) &PERF_SAMPLE_BRANCH_ANY ))
@@ -1258,6 +1268,8 @@ int cmd_report(int argc, const char **argv) "Show full source file name path for source lines"),OPT_BOOLEAN (0 ,"show-ref-call-graph" , &symbol_conf.show_ref_callgraph,"Show callgraph from reference event" ), +OPT_BOOLEAN (0 ,"stitch-lbr" , &report.stitch_lbr, +"Enable LBR callgraph stitching approach" ),OPT_INTEGER (0 ,"socket-filter" , &report.socket_filter,"only show processor socket that match with this filter" ), OPT_BOOLEAN(0, "raw-trace", &symbol_conf.raw_trace,
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 57/60 perf script: Add option to enable the LBR stitching approach
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
With the LBR stitching approach, the reconstructed LBR call stack can break the HW limitation. However, it may reconstruct invalid call stacks in some cases, e.g. exception handing such as setjmp/longjmp. Also, it may impact the processing time especially when the number of samples with stitched LBRs are huge.
Add an option to enable the approach.
Committer testing:
Using the same perf.data as with the latest cset committer testing section:
$ perf script --stitch-lbr
<SNIP>
tchain_edit 11131 15164.984292: 437491 cycles:u:
401106 f43+0x0 (/wb/tchain_edit)
40114c f42+0x18 (/wb/tchain_edit)
401172 f41+0xe (/wb/tchain_edit)
401194 f40+0x0 (/wb/tchain_edit)
40119b f39+0x0 (/wb/tchain_edit)
4011a2 f38+0x0 (/wb/tchain_edit)
4011a9 f37+0x0 (/wb/tchain_edit)
4011b0 f36+0x0 (/wb/tchain_edit)
4011b7 f35+0x0 (/wb/tchain_edit)
4011be f34+0x0 (/wb/tchain_edit)
4011c5 f33+0x0 (/wb/tchain_edit)
4011cc f32+0x0 (/wb/tchain_edit)
401207 f31+0x34 (/wb/tchain_edit)
401212 f30+0x0 (/wb/tchain_edit)
401219 f29+0x0 (/wb/tchain_edit)
401220 f28+0x0 (/wb/tchain_edit)
401227 f27+0x0 (/wb/tchain_edit)
40122e f26+0x0 (/wb/tchain_edit)
401235 f25+0x0 (/wb/tchain_edit)
40123c f24+0x0 (/wb/tchain_edit)
401243 f23+0x0 (/wb/tchain_edit)
40124a f22+0x0 (/wb/tchain_edit)
401251 f21+0x0 (/wb/tchain_edit)
401258 f20+0x0 (/wb/tchain_edit)
40125f f19+0x0 (/wb/tchain_edit)
401266 f18+0x0 (/wb/tchain_edit)
40126d f17+0x0 (/wb/tchain_edit)
401274 f16+0x0 (/wb/tchain_edit)
40127b f15+0x0 (/wb/tchain_edit)
401282 f14+0x0 (/wb/tchain_edit)
401289 f13+0x0 (/wb/tchain_edit)
401290 f12+0x0 (/wb/tchain_edit)
401297 f11+0x0 (/wb/tchain_edit)
40129e f10+0x0 (/wb/tchain_edit)
4012a5 f9+0x0 (/wb/tchain_edit)
4012ac f8+0x0 (/wb/tchain_edit)
4012b3 f7+0x0 (/wb/tchain_edit)
4012ba f6+0x0 (/wb/tchain_edit)
4012c1 f5+0x0 (/wb/tchain_edit)
4012c8 f4+0x0 (/wb/tchain_edit)
4012cf f3+0x0 (/wb/tchain_edit)
4012d6 f2+0x0 (/wb/tchain_edit)
4012dd f1+0x0 (/wb/tchain_edit)
4012e4 main+0x0 (/wb/tchain_edit)
7f41a5016f41 __libc_start_main+0xf1 (/usr/lib64/libc-2.29.so)
<SNIP>
$--- tools/perf/Documentation/perf-script.txt | 11 +++++++++++ tools/perf/builtin-script.c | 12 ++++++++++++ 2 files changed, 23 insertions(+)
tools/perf/Documentation/perf-script.txt +11 -0
@@ -440,6 +440,17 @@ include::itrace.txt[] --show-on-off-events:: Show the --switch-on/off events too. +--stitch-lbr:: + Show callgraph with stitched LBRs, which may have more complete + callgraph. The perf.data file must have been obtained using + perf record --call-graph lbr. + Disabled by default. In common cases with call stack overflows, + it can recreate better call stacks than the default lbr call stack + output. But this approach is not full proof. There can be cases + where it creates incorrect call stacks from incorrect matches. + The known limitations include exception handing such as + setjmp/longjmp will have calls/returns not match. + SEE ALSO -------- linkperf:perf-record[1], linkperf:perf-script-perl[1],
tools/perf/builtin-script.c +12 -0
@@ -1697,6 +1697,7 @@ struct perf_script {bool show_cgroup_events;bool allocated;bool per_event_dump; +bool stitch_lbr;struct evswitch evswitch;struct perf_cpu_map *cpus;struct perf_thread_map *threads;
@@ -1923,6 +1924,9 @@ static void process_event(struct perf_script *script,if (PRINT_FIELD (IP )) {struct callchain_cursor *cursor =NULL ; +if (script->stitch_lbr) + al->thread->lbr_stitch_enable = true; +if (symbol_conf.use_callchain && sample->callchain &&thread__resolve_callchain (al->thread, &callchain_cursor, evsel, sample,NULL ,NULL , scripting_max_stack) ==0 )
@@ -3170,6 +3174,12 @@ static void script__setup_sample_type(struct perf_script *script)else callchain_param.record_mode =CALLCHAIN_FP ; } + +if (script->stitch_lbr && (callchain_param.record_mode !=CALLCHAIN_LBR )) { +pr_warning ("Can't find LBR callchain. Switch off --stitch-lbr.\n" +"Please apply --call-graph lbr when recording.\n" ); + script->stitch_lbr = false; + } } static int process_stat_round_event(struct perf_session *session,
@@ -3481,6 +3491,8 @@ int cmd_script(int argc, const char **argv) "file", "file saving guest os /proc/kallsyms"),OPT_STRING (0 ,"guestmodules" , &symbol_conf.default_guest_modules,"file" ,"file saving guest os /proc/modules" ), +OPT_BOOLEAN ('\0' ,"stitch-lbr" , &script.stitch_lbr, +"Enable LBR callgraph stitching approach" ),OPTS_EVSWITCH (&script.evswitch), OPT_END() };
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 58/60 perf top: Add option to enable the LBR stitching approach
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
With the LBR stitching approach, the reconstructed LBR call stack can break the HW limitation. However, it may reconstruct invalid call stacks in some cases, e.g. exception handing such as setjmp/longjmp. Also, it may impact the processing time especially when the number of samples with stitched LBRs are huge.
Add an option to enable the approach.
The option must be used with --call-graph lbr.
--- tools/perf/Documentation/perf-top.txt | 9 +++++++++ tools/perf/builtin-top.c | 11 +++++++++++ tools/perf/util/top.h | 1 + 3 files changed, 21 insertions(+)
tools/perf/Documentation/perf-top.txt +9 -0
@@ -319,6 +319,15 @@ Default is to monitor all CPUS. go straight to the histogram browser, just like 'perf top' with no events explicitely specified does. +--stitch-lbr:: + Show callgraph with stitched LBRs, which may have more complete + callgraph. The option must be used with --call-graph lbr recording. + Disabled by default. In common cases with call stack overflows, + it can recreate better call stacks than the default lbr call stack + output. But this approach is not full proof. There can be cases + where it creates incorrect call stacks from incorrect matches. + The known limitations include exception handing such as + setjmp/longjmp will have calls/returns not match. INTERACTIVE PROMPTING KEYS --------------------------
tools/perf/builtin-top.c +11 -0
@@ -33,6 +33,7 @@#include "util/map.h" #include "util/mmap.h" #include "util/session.h" +#include "util/thread.h" #include "util/symbol.h" #include "util/synthetic-events.h" #include "util/top.h"
@@ -775,6 +776,9 @@ static void perf_event__process_sample(struct perf_tool *tool,if (machine__resolve (machine, &al, sample) <0 )return ; +if (top->stitch_lbr) + al.thread->lbr_stitch_enable = true; +if (!machine->kptr_restrict_warned && symbol_conf.kptr_restrict && al.cpumode ==PERF_RECORD_MISC_KERNEL ) {
@@ -1571,6 +1575,8 @@ int cmd_top(int argc, const char **argv)"Sort the output by the event at the index n in group. " "If n is invalid, sort by the first event. " "WARNING: should be used on grouped events."), +OPT_BOOLEAN (0 ,"stitch-lbr" , &top.stitch_lbr, +"Enable LBR callgraph stitching approach" ),OPTS_EVSWITCH (&top.evswitch), OPT_END() };
@@ -1640,6 +1646,11 @@ int cmd_top(int argc, const char **argv) } } +if (top.stitch_lbr && !(callchain_param.record_mode ==CALLCHAIN_LBR )) { +pr_err ("Error: --stitch-lbr must be used with --call-graph lbr\n" ); + goto out_delete_evlist; + } +if (opts->branch_stack && callchain_param.enabled) symbol_conf.show_branchflag_count = true;
tools/perf/util/top.h +1 -0
@@ -36,6 +36,7 @@ struct perf_top {bool use_tui, use_stdio;bool vmlinux_warned;bool dump_symtab; +bool stitch_lbr;struct hist_entry *sym_filter_entry;struct evsel *sym_evsel;struct perf_session *session;
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 59/60 perf c2c: Add option to enable the LBR stitching approach
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
With the LBR stitching approach, the reconstructed LBR call stack can break the HW limitation. However, it may reconstruct invalid call stacks in some cases, e.g. exception handing such as setjmp/longjmp. Also, it may impact the processing time especially when the number of samples with stitched LBRs are huge.
Add an option to enable the approach.
--- tools/perf/Documentation/perf-c2c.txt | 11 +++++++++++ tools/perf/builtin-c2c.c | 12 ++++++++++++ 2 files changed, 23 insertions(+)
tools/perf/Documentation/perf-c2c.txt +11 -0
@@ -111,6 +111,17 @@ REPORT OPTIONS --display:: Switch to HITM type (rmt, lcl) to display and sort on. Total HITMs as default. +--stitch-lbr:: + Show callgraph with stitched LBRs, which may have more complete + callgraph. The perf.data file must have been obtained using + perf c2c record --call-graph lbr. + Disabled by default. In common cases with call stack overflows, + it can recreate better call stacks than the default lbr call stack + output. But this approach is not full proof. There can be cases + where it creates incorrect call stacks from incorrect matches. + The known limitations include exception handing such as + setjmp/longjmp will have calls/returns not match. + C2C RECORD ---------- The perf c2c record command setup options related to HITM cacheline analysis
tools/perf/builtin-c2c.c +12 -0
@@ -95,6 +95,7 @@ struct perf_c2c {bool use_stdio;bool stats_only;bool symbol_full; +bool stitch_lbr;/* HITM shared clines stats */ struct c2c_stats hitm_stats;
@@ -273,6 +274,9 @@ static int process_sample_event(struct perf_tool *tool __maybe_unused,return -1 ; } +if (c2c.stitch_lbr) + al.thread->lbr_stitch_enable = true; + ret =sample__resolve_callchain (sample, &callchain_cursor,NULL , evsel, &al, sysctl_perf_event_max_stack);if (ret)
@@ -2601,6 +2605,12 @@ static int setup_callchain(struct evlist *evlist) } } +if (c2c.stitch_lbr && (mode !=CALLCHAIN_LBR )) { +ui__warning ("Can't find LBR callchain. Switch off --stitch-lbr.\n" +"Please apply --call-graph lbr when recording.\n" ); + c2c.stitch_lbr = false; + } + callchain_param.record_mode = mode; callchain_param.min_percent =0 ;return 0 ;
@@ -2752,6 +2762,8 @@ static int perf_c2c__report(int argc, const char **argv)OPT_STRING ('c' ,"coalesce" , &coalesce,"coalesce fields" ,"coalesce fields: pid,tid,iaddr,dso" ),OPT_BOOLEAN ('f' ,"force" , &symbol_conf.force,"don't complain, do it" ), +OPT_BOOLEAN (0 ,"stitch-lbr" , &c2c.stitch_lbr, +"Enable LBR callgraph stitching approach" ),OPT_PARENT (c2c_options), OPT_END() };
Signature · 1 line
2.21.1
Arnaldo Carvalho de Melo
maintainerPATCH 60/60 perf hist: Add fast path for duplicate entries check
In reply to Arnaldo Carvalho de MeloFrom: Kan Liang <[email protected]>
Perf checks the duplicate entries in a callchain before adding an entry. However the check is very slow especially with deeper call stack. Almost ~50% elapsed time of perf report is spent on the check when the call stack is always depth of 32.
The hist_entry__cmp() is used to compare the new entry with the old entries. It will go through all the available sorts in the sort_list, and call the specific cmp of each sort, which is very slow.
Actually, for most cases, there are no duplicate entries in callchain. The symbols are usually different. It's much faster to do a quick check for symbols first. Only do the full cmp when the symbols are exactly the same.
The quick check is only to check symbols, not dso. Export _sort__sym_cmp.
$ perf record --call-graph lbr ./tchain_edit_64
Without the patch $time perf report --stdio real 0m21.142s user 0m21.110s sys 0m0.033s With the patch $time perf report --stdio real 0m10.977s user 0m10.948s sys 0m0.027s
--- tools/perf/util/hist.c | 23 +++++++++++++++++++++++ tools/perf/util/sort.c | 2 +- tools/perf/util/sort.h | 2 ++ 3 files changed, 26 insertions(+), 1 deletion(-)
tools/perf/util/hist.c +23 -0
@@ -1070,6 +1070,20 @@ iter_next_cumulative_entry(struct hist_entry_iter *iter,return fill_callchain_info (al, node, iter->hide_unresolved); } +static bool +hist_entry__fast__sym_diff (struct hist_entry *left, +struct hist_entry *right) +{ +struct symbol *sym_l = left->ms.sym; +struct symbol *sym_r = right->ms.sym; + +if (!sym_l && !sym_r) +return left->ip != right->ip; + +return !!_sort__sym_cmp (sym_l, sym_r); +} + +static int iter_add_next_cumulative_entry (struct hist_entry_iter *iter,struct addr_location *al)
@@ -1096,6 +1110,7 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter, };int i;struct callchain_cursor cursor; +bool fast =hists__has (he_tmp.hists, sym);callchain_cursor_snapshot (&cursor, &callchain_cursor);
@@ -1106,6 +1121,14 @@ iter_add_next_cumulative_entry(struct hist_entry_iter *iter,* It's possible that it has cycles or recursive calls. */ for (i =0 ; i < iter->curr; i++) { +/* +* For most cases, there are no duplicate entries in callchain. +* The symbols are usually different. Do a quick check for +* symbols first. +*/ +if (fast &&hist_entry__fast__sym_diff (he_cache[i], &he_tmp)) +continue ; +if (hist_entry__cmp (he_cache[i], &he_tmp) ==0 ) {/* to avoid calling callback function */ iter->he =NULL ;
tools/perf/util/sort.c +1 -1
@@ -237,7 +237,7 @@ static int64_t _sort__addr_cmp(u64 left_ip, u64 right_ip)return (int64_t )(right_ip - left_ip); } -static int64_t _sort__sym_cmp (struct symbol *sym_l,struct symbol *sym_r) +int64_t _sort__sym_cmp (struct symbol *sym_l,struct symbol *sym_r) {if (!sym_l || !sym_r)return cmp_null (sym_l, sym_r);
tools/perf/util/sort.h +2 -0
@@ -311,5 +311,7 @@ int64_t sort__daddr_cmp(struct hist_entry *left, struct hist_entry *right);int64_t sort__dcacheline_cmp (struct hist_entry *left,struct hist_entry *right); +int64_t +_sort__sym_cmp (struct symbol *sym_l,struct symbol *sym_r);char *hist_entry__srcline (struct hist_entry *he);#endif /* __PERF_SORT_H */
Signature · 1 line
2.21.1
Quoted 16 lines · Arnaldo Carvalho de Melo
Hi Ingo/Thomas,
Please consider pulling,
Best regards,
- Arnaldo
Test results at the end of this message, as usual.
The following changes since commit
cd0943357bc7570f081701d005318c20982178b8:Merge tag 'perf-urgent-for-mingo-5.7-20200414' of
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linuxinto perf/urgent (2020-04-16 10:21:31 +0200)are available in the Git repository at:
Quoted 7 lines
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.gittags/perf-core-for-mingo-5.8-20200420for you to fetch changes up to
12e89e65f446476951f42aedeef56b6bd6f7f1e6:perf hist: Add fast path for duplicate entries check (2020-04-18 09:05:01 -0300)
85 files changed, 1851 insertions(+), 513 deletions(-)
Pulled, thanks a lot Arnaldo!
Ingo
Hello!
Quoted 20 lines · Ingo Molnar
Quoted 18 lines · Arnaldo Carvalho de Melo
Hi Ingo/Thomas,
Please consider pulling,
Best regards,
- Arnaldo
Test results at the end of this message, as usual.
The following changes since commit
cd0943357bc7570f081701d005318c20982178b8:Merge tag 'perf-urgent-for-mingo-5.7-20200414' of
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linuxinto perf/urgent (2020-04-16 10:21:31 +0200)are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.gittags/perf-core-for-mingo-5.8-20200420
Quoted 7 lines
Quoted 5 lines
for you to fetch changes up to
12e89e65f446476951f42aedeef56b6bd6f7f1e6:perf hist: Add fast path for duplicate entries check (2020-04-18 09:05:01 -0300)
85 files changed, 1851 insertions(+), 513 deletions(-)Pulled, thanks a lot Arnaldo!
Our OpenEmbedded builds detected an issue with 5287f9269206 ("perf
script: Add flamegraph.py script"):
ERROR: perf-1.0-r9 do_package_qa: QA Issue:/usr/libexec/perf-core/scripts/python/bin/flamegraph-report contained in package perf-python requires /usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
This means that there is a new binary pulled in in the shebang line which was unaccounted for: /usr/bin/sh. I don't see any other usage of /usr/bin/sh in the kernel tree (does not even exist on my Ubuntu dev machine) but plenty of /bin/sh. This patch is needed:
-----8<----------8<----------8<-----
(quoted diff) +0 -0
b/tools/perf/scripts/python/bin/flamegraph-record
index 725d66e71570..a2f3fa25ef81 100755
tools/perf/scripts/python/bin/flamegraph-record +1 -1
@@ -1,2 +1,2 @@ -#!/usr/bin/sh +#!/bin/sh perf record -g "$@"
(quoted diff) +0 -0
b/tools/perf/scripts/python/bin/flamegraph-report
index b1a79afd903b..b0177355619b 100755
tools/perf/scripts/python/bin/flamegraph-report +1 -1
@@ -1,3 +1,3 @@ -#!/usr/bin/sh +#!/bin/sh # description: create flame graphs perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
----->8---------->8---------->8-----
Greetings!
Daniel Díaz
[email protected]
Em Thu, Apr 23, 2020 at 04:28:46PM -0500, Daniel Díaz escreveu:
Quoted 11 lines · Ingo Molnar
Quoted 3 lines
Quoted 1 line
85 files changed, 1851 insertions(+), 513 deletions(-)Pulled, thanks a lot Arnaldo!
Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):
ERROR: perf-1.0-r9do_package_qa: QA Issue:/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
yeah, the flamegraph scripts are the outliers, there, everything else is using /bin/bash, so I'll switch to that, ok Andreas?
[acme@quaco perf]$ vim tools/perf/scripts/python/bin/* 34 files to edit
[acme@quaco perf]$ head -1 tools/perf/scripts/python/bin/* ==> tools/perf/scripts/python/bin/compaction-times-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/compaction-times-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/event_analyzing_sample-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/event_analyzing_sample-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/export-to-postgresql-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/export-to-postgresql-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/export-to-sqlite-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/export-to-sqlite-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/failed-syscalls-by-pid-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/failed-syscalls-by-pid-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/flamegraph-record <== #!/usr/bin/sh
==> tools/perf/scripts/python/bin/flamegraph-report <== #!/usr/bin/sh
==> tools/perf/scripts/python/bin/futex-contention-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/futex-contention-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/intel-pt-events-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/intel-pt-events-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/mem-phys-addr-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/mem-phys-addr-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/netdev-times-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/netdev-times-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/net_dropmonitor-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/net_dropmonitor-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/powerpc-hcalls-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/powerpc-hcalls-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/sched-migration-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/sched-migration-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/sctop-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/sctop-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/stackcollapse-record <== #!/bin/sh
==> tools/perf/scripts/python/bin/stackcollapse-report <== #!/bin/sh
==> tools/perf/scripts/python/bin/syscall-counts-by-pid-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/syscall-counts-by-pid-report <== #!/bin/bash
==> tools/perf/scripts/python/bin/syscall-counts-record <== #!/bin/bash
==> tools/perf/scripts/python/bin/syscall-counts-report <== #!/bin/bash
[acme@quaco perf]$
Quoted 14 lines · Arnaldo Carvalho de Melo
Em Thu, Apr 23, 2020 at 04:28:46PM -0500, Daniel Díaz escreveu:
Quoted 11 lines · Ingo Molnar
Quoted 3 lines
Quoted 1 line
85 files changed, 1851 insertions(+), 513 deletions(-)Pulled, thanks a lot Arnaldo!
Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
Quoted 2 lines
yeah, the flamegraph scripts are the outliers, there, everything else is using
/bin/bash, so I'll switch to that, ok Andreas?
Sure, no problem. Thanks!
Cheers,
Andreas
Hello!
Quoted 16 lines · Andreas Gerstmayr
Quoted 14 lines · Arnaldo Carvalho de Melo
Em Thu, Apr 23, 2020 at 04:28:46PM -0500, Daniel Díaz escreveu:
Quoted 11 lines · Ingo Molnar
Quoted 3 lines
Quoted 1 line
85 files changed, 1851 insertions(+), 513 deletions(-)Pulled, thanks a lot Arnaldo!
Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
Quoted 4 lines
Quoted 2 lines
yeah, the flamegraph scripts are the outliers, there, everything else is using
/bin/bash, so I'll switch to that, ok Andreas?Sure, no problem. Thanks!
Just a gentle reminder that this can still be fixed in today's linux-next tree (next-20200504).
Greetings!
Daniel Díaz
[email protected]
Em Mon, May 04, 2020 at 02:07:56PM -0500, Daniel Díaz escreveu:
Quoted 19 lines
Hello!
Quoted 16 lines · Andreas Gerstmayr
Quoted 14 lines · Arnaldo Carvalho de Melo
Em Thu, Apr 23, 2020 at 04:28:46PM -0500, Daniel Díaz escreveu:
Quoted 11 lines · Ingo Molnar
Quoted 3 lines
Quoted 1 line
85 files changed, 1851 insertions(+), 513 deletions(-)Pulled, thanks a lot Arnaldo!
Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
yeah, the flamegraph scripts are the outliers, there, everything else is using
/bin/bash, so I'll switch to that, ok Andreas?Sure, no problem. Thanks!
Just a gentle reminder that this can still be fixed in today's linux-next tree (next-20200504).
Thanks for the reminder, I've just added this to my tree:
commit c74ab13a30d3bec443c116e25b611255c58f32c0
Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue May 5 13:33:12 2020 -0300
perf flamegraph: Use /bin/bash for report script
As all the other tools/perf/scripts/python/bin/*-report scripts, fixing the this problem reported by Daniel Diaz:
Our OpenEmbedded builds detected an issue with 5287f9269206 ("perf
script: Add flamegraph.py script"):
ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-report contained in package perf-python requires /usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
This means that there is a new binary pulled in in the shebang line which was unaccounted for: /usr/bin/sh. I don't see any other usage of /usr/bin/sh in the kernel tree (does not even exist on my Ubuntu dev machine) but plenty of /bin/sh. This patch is needed:
-----8<----------8<----------8<-----
diff --git a/tools/perf/scripts/python/bin/flamegraph-record b/tools/perf/scripts/python/bin/flamegraph-record
index 725d66e71570..a2f3fa25ef81 100755
--- a/tools/perf/scripts/python/bin/flamegraph-record
+++ b/tools/perf/scripts/python/bin/flamegraph-record
@@ -1,2 +1,2 @@
-#!/usr/bin/sh
+#!/bin/sh
perf record -g "$@"
diff --git a/tools/perf/scripts/python/bin/flamegraph-report b/tools/perf/scripts/python/bin/flamegraph-report
index b1a79afd903b..b0177355619b 100755
--- a/tools/perf/scripts/python/bin/flamegraph-report
+++ b/tools/perf/scripts/python/bin/flamegraph-report
@@ -1,3 +1,3 @@
-#!/usr/bin/sh
+#!/bin/sh
# description: create flame graphs
perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
----->8---------->8---------->8-----
tools/perf/scripts/python/bin/flamegraph-report +1 -1
@@ -1,3 +1,3 @@ -#!/usr/bin/sh +#!/bin/bash # description: create flame graphs perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
Hello!
Quoted 88 lines · Arnaldo Carvalho de Melo
Em Mon, May 04, 2020 at 02:07:56PM -0500, Daniel Díaz escreveu:
Quoted 19 lines
Hello!
Quoted 16 lines · Andreas Gerstmayr
Quoted 14 lines · Arnaldo Carvalho de Melo
Em Thu, Apr 23, 2020 at 04:28:46PM -0500, Daniel Díaz escreveu:
Quoted 11 lines · Ingo Molnar
Quoted 3 lines
Quoted 1 line
85 files changed, 1851 insertions(+), 513 deletions(-)Pulled, thanks a lot Arnaldo!
Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
yeah, the flamegraph scripts are the outliers, there, everything else is using
/bin/bash, so I'll switch to that, ok Andreas?Sure, no problem. Thanks!
Just a gentle reminder that this can still be fixed in today's linux-next tree (next-20200504).
Thanks for the reminder, I've just added this to my tree:
commit
c74ab13a30d3bec443c116e25b611255c58f32c0Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue May 5 13:33:12 2020 -0300perf flamegraph: Use
/bin/bashfor report scriptAs all the other
tools/perf/scripts/python/bin/*-reportscripts, fixing the this problem reported by Daniel Diaz:Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):
ERROR: perf-1.0-r9do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]This means that there is a new binary pulled in in the shebang line which was unaccounted for:
/usr/bin/sh. I don't see any other usage of/usr/bin/shin the kernel tree (does not even exist on my Ubuntu dev machine) but plenty of/bin/sh. This patch is needed:
-----8<----------8<----------8<-----
diff --git a/tools/perf/scripts/python/bin/flamegraph-record b/tools/perf/scripts/python/bin/flamegraph-record
index725d66e71570..a2f3fa25ef81100755
--- a/tools/perf/scripts/python/bin/flamegraph-record
+++ b/tools/perf/scripts/python/bin/flamegraph-record
@@ -1,2 +1,2 @@
-#!/usr/bin/sh
+#!/bin/sh
perf record -g "$@"
diff --git a/tools/perf/scripts/python/bin/flamegraph-report b/tools/perf/scripts/python/bin/flamegraph-report
indexb1a79afd903b..b0177355619b100755
--- a/tools/perf/scripts/python/bin/flamegraph-report
+++ b/tools/perf/scripts/python/bin/flamegraph-report
@@ -1,3 +1,3 @@
-#!/usr/bin/sh
+#!/bin/sh
# description: create flame graphs
perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
----->8---------->8---------->8-----Fixes 5287f9269206 ("perf script: Add flamegraph.py script")Reported-by Daniel Díaz <[email protected]>Cc Adrian Hunter <[email protected]>Cc Andreas Gerstmayr <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Link http://lore.kernel.org/lkml/CAEUSe7_wmKS361mKLTB1eYbzYXcKkXdU26BX5BojdKRz8MfPCw@mail.gmail.com open hereSigned-off-by Arnaldo Carvalho de Melo <[email protected]>tools/perf/scripts/python/bin/flamegraph-report +0
-0
tools/perf/scripts/python/bin/flamegraph-report line 3
Quoted 4 lines
tools/perf/scripts/python/bin/flamegraph-report +1
-1@@ -1,3 +1,3 @@ -#!/usr/bin/sh +#!/bin/bash # description: create flame graphs perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
What about flamegraph-record?
Thanks and greetings!
Daniel Díaz
[email protected]
Arnaldo Carvalho de Melo
Acked-by Andreas GerstmayrEm Tue, May 05, 2020 at 11:57:18AM -0500, Daniel Díaz escreveu:
Quoted 87 lines
Hello!
Quoted 83 lines · Arnaldo Carvalho de Melo
Em Mon, May 04, 2020 at 02:07:56PM -0500, Daniel Díaz escreveu:
Quoted 19 lines
Hello!
Quoted 16 lines · Andreas Gerstmayr
Quoted 14 lines · Arnaldo Carvalho de Melo
Em Thu, Apr 23, 2020 at 04:28:46PM -0500, Daniel Díaz escreveu:
Quoted 11 lines · Ingo Molnar
Quoted 3 lines
Quoted 1 line
85 files changed, 1851 insertions(+), 513 deletions(-)Pulled, thanks a lot Arnaldo!
Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
yeah, the flamegraph scripts are the outliers, there, everything else is using
/bin/bash, so I'll switch to that, ok Andreas?Sure, no problem. Thanks!
Just a gentle reminder that this can still be fixed in today's linux-next tree (next-20200504).
Thanks for the reminder, I've just added this to my tree:
commit
c74ab13a30d3bec443c116e25b611255c58f32c0Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue May 5 13:33:12 2020 -0300perf flamegraph: Use
/bin/bashfor report scriptAs all the other
tools/perf/scripts/python/bin/*-reportscripts, fixing the this problem reported by Daniel Diaz:Our OpenEmbedded builds detected an issue with
5287f9269206("perf
script: Addflamegraph.pyscript"):
ERROR: perf-1.0-r9do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-reportcontained in package perf-python requires/usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]This means that there is a new binary pulled in in the shebang line which was unaccounted for:
/usr/bin/sh. I don't see any other usage of/usr/bin/shin the kernel tree (does not even exist on my Ubuntu dev machine) but plenty of/bin/sh. This patch is needed:
-----8<----------8<----------8<-----
diff --git a/tools/perf/scripts/python/bin/flamegraph-record b/tools/perf/scripts/python/bin/flamegraph-record
index725d66e71570..a2f3fa25ef81100755
--- a/tools/perf/scripts/python/bin/flamegraph-record
+++ b/tools/perf/scripts/python/bin/flamegraph-record
@@ -1,2 +1,2 @@
-#!/usr/bin/sh
+#!/bin/sh
perf record -g "$@"
diff --git a/tools/perf/scripts/python/bin/flamegraph-report b/tools/perf/scripts/python/bin/flamegraph-report
indexb1a79afd903b..b0177355619b100755
--- a/tools/perf/scripts/python/bin/flamegraph-report
+++ b/tools/perf/scripts/python/bin/flamegraph-report
@@ -1,3 +1,3 @@
-#!/usr/bin/sh
+#!/bin/sh
# description: create flame graphs
perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
----->8---------->8---------->8-----Fixes 5287f9269206 ("perf script: Add flamegraph.py script")Reported-by Daniel Díaz <[email protected]>Cc Adrian Hunter <[email protected]>Cc Andreas Gerstmayr <[email protected]>Cc Jiri Olsa <[email protected]>Cc Namhyung Kim <[email protected]>Link http://lore.kernel.org/lkml/CAEUSe7_wmKS361mKLTB1eYbzYXcKkXdU26BX5BojdKRz8MfPCw@mail.gmail.com open hereSigned-off-by Arnaldo Carvalho de Melo <[email protected]>
Quoted 11 lines
tools/perf/scripts/python/bin/flamegraph-report line 3
Quoted 9 lines
tools/perf/scripts/python/bin/flamegraph-report +1
-1@@ -1,3 +1,3 @@ -#!/usr/bin/sh +#!/bin/bash # description: create flame graphs perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"What about flamegraph-record?
oops, make that this instead:
commit b3a63d0c17e6e1d23a6b44502b55f066adfd8e6a
Author: Arnaldo Carvalho de Melo <[email protected]> Date: Tue May 5 13:33:12 2020 -0300
perf flamegraph: Use /bin/bash for report and record scripts
As all the other tools/perf/scripts/python/bin/*-{report,record} scripts, fixing the this problem reported by Daniel Diaz:
Our OpenEmbedded builds detected an issue with 5287f9269206 ("perf
script: Add flamegraph.py script"):
ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-report contained in package perf-python requires /usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
This means that there is a new binary pulled in in the shebang line which was unaccounted for: /usr/bin/sh. I don't see any other usage of /usr/bin/sh in the kernel tree (does not even exist on my Ubuntu dev machine) but plenty of /bin/sh. This patch is needed:
-----8<----------8<----------8<-----
diff --git a/tools/perf/scripts/python/bin/flamegraph-record b/tools/perf/scripts/python/bin/flamegraph-record
index 725d66e71570..a2f3fa25ef81 100755
--- a/tools/perf/scripts/python/bin/flamegraph-record
+++ b/tools/perf/scripts/python/bin/flamegraph-record
@@ -1,2 +1,2 @@
-#!/usr/bin/sh
+#!/bin/sh
perf record -g "$@"
diff --git a/tools/perf/scripts/python/bin/flamegraph-report b/tools/perf/scripts/python/bin/flamegraph-report
index b1a79afd903b..b0177355619b 100755
--- a/tools/perf/scripts/python/bin/flamegraph-report
+++ b/tools/perf/scripts/python/bin/flamegraph-report
@@ -1,3 +1,3 @@
-#!/usr/bin/sh
+#!/bin/sh
# description: create flame graphs
perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
----->8---------->8---------->8-----
tools/perf/scripts/python/bin/flamegraph-record +1 -1
@@ -1,2 +1,2 @@ -#!/usr/bin/sh +#!/bin/bash perf record -g "$@"
tools/perf/scripts/python/bin/flamegraph-report +1 -1
@@ -1,3 +1,3 @@ -#!/usr/bin/sh +#!/bin/bash # description: create flame graphs perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
[tip: perf/core] perf flamegraph: Use /bin/bash for report and record scripts
In reply to Daniel DíazThe following commit has been merged into the perf/core branch of tip:
The following commit has been merged into the perf/core branch of tip:
Commit-ID: 19ce2321739da5fc27f6a5ed1e1cb15e384ad030 Gitweb: https://git.kernel.org/tip/19ce2321739da5fc27f6a5ed1e1cb15e384ad030 Author: Arnaldo Carvalho de Melo <[email protected]> AuthorDate: Tue, 05 May 2020 13:33:12 -03:00 Committer: Arnaldo Carvalho de Melo <[email protected]> CommitterDate: Tue, 05 May 2020 16:35:32 -03:00
perf flamegraph: Use /bin/bash for report and record scripts
As all the other tools/perf/scripts/python/bin/*-{report,record} scripts, fixing the this problem reported by Daniel Diaz:
Our OpenEmbedded builds detected an issue with 5287f9269206 ("perf
script: Add flamegraph.py script"):
ERROR: perf-1.0-r9 do_package_qa: QA Issue:
/usr/libexec/perf-core/scripts/python/bin/flamegraph-report contained in package perf-python requires /usr/bin/sh, but no providers found in RDEPENDS_perf-python? [file-rdeps]
This means that there is a new binary pulled in in the shebang line which was unaccounted for: /usr/bin/sh. I don't see any other usage of /usr/bin/sh in the kernel tree (does not even exist on my Ubuntu dev machine) but plenty of /bin/sh. This patch is needed:
-----8<----------8<----------8<-----
diff --git a/tools/perf/scripts/python/bin/flamegraph-record b/tools/perf/scripts/python/bin/flamegraph-record
index 725d66e71570..a2f3fa25ef81 100755
--- a/tools/perf/scripts/python/bin/flamegraph-record
+++ b/tools/perf/scripts/python/bin/flamegraph-record
@@ -1,2 +1,2 @@
-#!/usr/bin/sh
+#!/bin/sh
perf record -g "$@"
diff --git a/tools/perf/scripts/python/bin/flamegraph-report b/tools/perf/scripts/python/bin/flamegraph-report
index b1a79afd903b..b0177355619b 100755
--- a/tools/perf/scripts/python/bin/flamegraph-report
+++ b/tools/perf/scripts/python/bin/flamegraph-report
@@ -1,3 +1,3 @@
-#!/usr/bin/sh
+#!/bin/sh
# description: create flame graphs
perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"
----->8---------->8---------->8-----
--- tools/perf/scripts/python/bin/flamegraph-record | 2 +- tools/perf/scripts/python/bin/flamegraph-report | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
tools/perf/scripts/python/bin/flamegraph-record +1 -1
@@ -1,2 +1,2 @@ -#!/usr/bin/sh +#!/bin/bash perf record -g "$@"
tools/perf/scripts/python/bin/flamegraph-report +1 -1
@@ -1,3 +1,3 @@ -#!/usr/bin/sh +#!/bin/bash # description: create flame graphs perf script -s "$PERF_EXEC_PATH"/scripts/python/flamegraph.py -- "$@"