Skip to contentSkip to conversation

perf/urgent improvements and fixes

GIT PULL In review Arnaldo Carvalho de Melo 5 participants 12 messages Dec 23, 2019 – Feb 10, 2020 perf events

lore
Share & tools
lore

Arnaldo Carvalho de Melo

author maintainer
#0 lore raw

Hi Ingo/Thomas,

Please consider pulling,

Best regards,

- Arnaldo

Test results at the end of this message, as usual.

The following changes since commit 9f0bff1180efc9ea988fed3fd93da7647151ac8b:

perf/core: Add SRCU annotation for pmus list walk (2019-12-17 13:32:46 +0100)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-urgent-for-mingo-5.5-20191223

for you to fetch changes up to 55347ec340af401437680fd0e88df6739a967f9f:

perf hists: Fix variable name's inconsistency in hists__for_each() macro (2019-12-20 18:58:13 -0300)

----------------------------------------------------------------
perf/urgent fixes:

perf report/top:

Arnaldo Carvalho de Melo:

- Fix popup menu for entries in main kernel maps other than the main one, e.g. ".init.text", where a non-initialized pointer was causing segfaults.

Jin Yao:

- Fix incorrectly added dimensions when switching perf.data file to another via the popup menu.

libtraceevent:

Hewenliang:

- Fix memory leakage in filter_event().

perf hists:

Yuya Fujita:

- Fix variable name's inconsistency in hists__for_each() macro.

----------------------------------------------------------------
Arnaldo Carvalho de Melo (1):

      perf map: Set kmap->kmaps backpointer for main kernel map chunks

Hewenliang (1):
tools lib traceevent: Fix memory leakage in filter_event

Jin Yao (1):
perf report: Fix incorrectly added dimensions as switch perf data file

Yuya Fujita (1):
perf hists: Fix variable name's inconsistency in hists__for_each() macro

 tools/lib/traceevent/parse-filter.c | 4 +++-
 tools/perf/builtin-report.c         | 5 ++++-
 tools/perf/util/hist.h              | 4 ++--
 tools/perf/util/symbol-elf.c        | 3 +++
 4 files changed, 12 insertions(+), 4 deletions(-)

Test results:

The first ones are container based builds of tools/perf with and without libelf support. Where clang is available, it is also used to build perf with/without libelf, and building with LIBCLANGLLVM=1 (built-in clang) with gcc and clang when clang and its devel libraries are installed.

The objtool and samples/bpf/ builds are disabled now that I'm switching from using the sources in a local volume to fetching them from a http server to build it inside the container, to make it easier to build in a container cluster. Those will come back later.

Several are cross builds, the ones with -x-ARCH and the android one, and those may not have all the features built, due to lack of multi-arch devel packages, available and being used so far on just a few, like debian:experimental-x-{arm64,mipsel}.

The 'perf test' one will perform a variety of tests exercising tools/perf/util/, tools/lib/{bpf,traceevent,etc}, as well as run perf commands with a variety of command line event specifications to then intercept the sys_perf_event syscall to check that the perf_event_attr fields are set up as expected, among a variety of other unit tests.

Then there is the 'make -C tools/perf build-test' ones, that build tools/perf/ with a variety of feature sets, exercising the build with an incomplete set of features as well as with a complete one. It is planned to have it run on each of the containers mentioned above, using some container orchestration infrastructure. Get in contact if interested in helping having this in place.

Clearlinux is failing when due to:

.gnu.debuglto_.debug_macro' referenced in section .gnu.debuglto_.debug_macro' of /tmp/build/perf/util/scripting-engines/perf-in.o: defined in discarded section `.gnu.debuglto_.debug_macro[wm4.stdcpredef.h.19.8dc41bed5d9037ff9622e015fb5f0ce3]' of /tmp/build/perf/util/scripting-engines/perf-in.o

OpenMandriva Cooker works well with gcc, uncovers a bug where we have to get compiler-clang.h from the kernel sources, will be fixed soon.

With the update of linux/linkage.h to move from ENTRY()/ENDPROC() to SYM_FUNC_START()/etc some of the older containers can't be used with clang, as the minimum version for the constructs used in the new linkage.h is 3.5, older versions (3.4, 3.4.2, etc) end up with:

bench/../../arch/x86/lib/memcpy_64.S:44:14: error: unexpected token in '.type' directive .type MEMCPY STT_FUNC ; .size MEMCPY, .-MEMCPY

               ^
  # export PERF_TARBALL=http://192.168.124.1/perf/perf-5.5.0-rc1.tar.xz
  # dm 
   1 alpine:3.4                    : Ok   gcc (Alpine 5.3.0) 5.3.0, clang version 3.8.0 (tags/RELEASE_380/final)
   2 alpine:3.5                    : Ok   gcc (Alpine 6.2.1) 6.2.1 20160822, clang version 3.8.1 (tags/RELEASE_381/final)
   3 alpine:3.6                    : Ok   gcc (Alpine 6.3.0) 6.3.0, clang version 4.0.0 (tags/RELEASE_400/final)
   4 alpine:3.7                    : Ok   gcc (Alpine 6.4.0) 6.4.0, Alpine clang version 5.0.0 (tags/RELEASE_500/final) (based on LLVM 5.0.0)
   5 alpine:3.8                    : Ok   gcc (Alpine 6.4.0) 6.4.0, Alpine clang version 5.0.1 (tags/RELEASE_501/final) (based on LLVM 5.0.1)
   6 alpine:3.9                    : Ok   gcc (Alpine 8.3.0) 8.3.0, Alpine clang version 5.0.1 (tags/RELEASE_502/final) (based on LLVM 5.0.1)
   7 alpine:3.10                   : Ok   gcc (Alpine 8.3.0) 8.3.0, Alpine clang version 8.0.0 (tags/RELEASE_800/final) (based on LLVM 8.0.0)
   8 alpine:3.11                   : Ok   gcc (Alpine 9.2.0) 9.2.0, Alpine clang version 9.0.0 (https://git.alpinelinux.org/aports f7f0d2c2b8bcd6a5843401a9a702029556492689) (based on LLVM 9.0.0)
   9 alpine:edge                   : Ok   gcc (Alpine 9.2.0) 9.2.0, Alpine clang version 9.0.0 (git://git.alpinelinux.org/aports 25c73ae7b95bdb42ae5f0ceac3b703e766582527) (based on LLVM 9.0.0)
  10 amazonlinux:1                 : Ok   gcc (GCC) 7.2.1 20170915 (Red Hat 7.2.1-2), clang version 3.6.2 (tags/RELEASE_362/final)
  11 amazonlinux:2                 : Ok   gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6), clang version 7.0.1 (Amazon Linux 2 7.0.1-1.amzn2.0.2)
  12 android-ndk:r12b-arm          : Ok   arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease)
  13 android-ndk:r15c-arm          : Ok   arm-linux-androideabi-gcc (GCC) 4.9.x 20150123 (prerelease)
  14 centos:5                      : Ok   gcc (GCC) 4.1.2 20080704 (Red Hat 4.1.2-55)
  15 centos:6                      : Ok   gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23)
  16 centos:7                      : Ok   gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39)
  17 centos:8                      : Ok   gcc (GCC) 8.2.1 20180905 (Red Hat 8.2.1-3), clang version 7.0.1 (tags/RELEASE_701/final)
  18 clearlinux:latest             : FAIL gcc (Clear Linux OS for Intel Architecture) 9.2.1 20191125 gcc-9-branch@278689
  19 debian:8                      : Ok   gcc (Debian 4.9.2-10+deb8u2) 4.9.2, Debian clang version 3.5.0-10 (tags/RELEASE_350/final) (based on LLVM 3.5.0)
  20 debian:9                      : Ok   gcc (Debian 6.3.0-18+deb9u1) 6.3.0 20170516, clang version 3.8.1-24 (tags/RELEASE_381/final)
  21 debian:10                     : Ok   gcc (Debian 8.3.0-6) 8.3.0, clang version 7.0.1-8 (tags/RELEASE_701/final)
  22 debian:experimental           : Ok   gcc (Debian 9.2.1-19) 9.2.1 20191109, clang version 8.0.1-4 (tags/RELEASE_801/final)
  23 debian:experimental-x-arm64   : Ok   aarch64-linux-gnu-gcc (Debian 8.3.0-19) 8.3.0
  24 debian:experimental-x-mips    : Ok   mips-linux-gnu-gcc (Debian 8.3.0-19) 8.3.0
  25 debian:experimental-x-mips64  : Ok   mips64-linux-gnuabi64-gcc (Debian 9.2.1-8) 9.2.1 20190909
  26 debian:experimental-x-mipsel  : Ok   mipsel-linux-gnu-gcc (Debian 9.2.1-8) 9.2.1 20190909
  27 fedora:20                     : Ok   gcc (GCC) 4.8.3 20140911 (Red Hat 4.8.3-7)
  28 fedora:22                     : Ok   gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6), clang version 3.5.0 (tags/RELEASE_350/final)
  29 fedora:23                     : Ok   gcc (GCC) 5.3.1 20160406 (Red Hat 5.3.1-6), clang version 3.7.0 (tags/RELEASE_370/final)
  30 fedora:24                     : Ok   gcc (GCC) 6.3.1 20161221 (Red Hat 6.3.1-1), clang version 3.8.1 (tags/RELEASE_381/final)
  31 fedora:24-x-ARC-uClibc        : Ok   arc-linux-gcc (ARCompact ISA Linux uClibc toolchain 2017.09-rc2) 7.1.1 20170710
  32 fedora:25                     : Ok   gcc (GCC) 6.4.1 20170727 (Red Hat 6.4.1-1), clang version 3.9.1 (tags/RELEASE_391/final)
  33 fedora:26                     : Ok   gcc (GCC) 7.3.1 20180130 (Red Hat 7.3.1-2), clang version 4.0.1 (tags/RELEASE_401/final)
  34 fedora:27                     : Ok   gcc (GCC) 7.3.1 20180712 (Red Hat 7.3.1-6), clang version 5.0.2 (tags/RELEASE_502/final)
  35 fedora:28                     : Ok   gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2), clang version 6.0.1 (tags/RELEASE_601/final)
  36 fedora:29                     : Ok   gcc (GCC) 8.3.1 20190223 (Red Hat 8.3.1-2), clang version 7.0.1 (Fedora 7.0.1-6.fc29)
  37 fedora:30                     : Ok   gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1), clang version 8.0.0 (Fedora 8.0.0-3.fc30)
  38 fedora:30-x-ARC-glibc         : Ok   arc-linux-gcc (ARC HS GNU/Linux glibc toolchain 2019.03-rc1) 8.3.1 20190225
  39 fedora:30-x-ARC-uClibc        : Ok   arc-linux-gcc (ARCv2 ISA Linux uClibc toolchain 2019.03-rc1) 8.3.1 20190225
  40 fedora:31                     : Ok   gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1), clang version 9.0.0 (Fedora 9.0.0-1.fc31)
  41 fedora:32                     : Ok   gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1), clang version 9.0.0 (Fedora 9.0.0-1.fc32)
  42 fedora:rawhide                : Ok   gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1-1), clang version 9.0.0 (Fedora 9.0.0-1.fc32)
  43 gentoo-stage3-amd64:latest    : Ok   gcc (Gentoo 9.2.0-r2 p3) 9.2.0
  44 mageia:5                      : Ok   gcc (GCC) 4.9.2, clang version 3.5.2 (tags/RELEASE_352/final)
  45 mageia:6                      : Ok   gcc (Mageia 5.5.0-1.mga6) 5.5.0, clang version 3.9.1 (tags/RELEASE_391/final)
  46 mageia:7                      : Ok   gcc (Mageia 8.3.1-0.20190524.1.mga7) 8.3.1 20190524, clang version 8.0.0 (Mageia 8.0.0-1.mga7)
  47 manjaro:latest                : Ok   gcc (GCC) 9.2.0, clang version 9.0.0 (tags/RELEASE_900/final)
  48 openmandriva:cooker           : Ok   gcc (GCC) 9.2.1 20191123 (OpenMandriva)
  49 opensuse:15.0                 : Ok   gcc (SUSE Linux) 7.4.1 20190424 [gcc-7-branch revision 270538], clang version 5.0.1 (tags/RELEASE_501/final 312548)
  50 opensuse:15.1                 : Ok   gcc (SUSE Linux) 7.4.1 20190905 [gcc-7-branch revision 275407], clang version 7.0.1 (tags/RELEASE_701/final 349238)
  51 opensuse:15.2                 : Ok   gcc (SUSE Linux) 7.4.1 20190905 [gcc-7-branch revision 275407], clang version 7.0.1 (tags/RELEASE_701/final 349238)
  52 opensuse:42.3                 : Ok   gcc (SUSE Linux) 4.8.5, clang version 3.8.0 (tags/RELEASE_380/final 262553)
  53 opensuse:tumbleweed           : Ok   gcc (SUSE Linux) 9.2.1 20190903 [gcc-9-branch revision 275330], clang version 9.0.0 (tags/RELEASE_900/final 372316)
  54 oraclelinux:6                 : Ok   gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23.0.1)
  55 oraclelinux:7                 : Ok   gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39.0.3)
  56 oraclelinux:8                 : Ok   gcc (GCC) 8.2.1 20180905 (Red Hat 8.2.1-3.0.1), clang version 7.0.1 (tags/RELEASE_701/final)
  57 ubuntu:12.04                  : Ok   gcc (Ubuntu/Linaro 4.6.3-1ubuntu5) 4.6.3, Ubuntu clang version 3.0-6ubuntu3 (tags/RELEASE_30/final) (based on LLVM 3.0)
  58 ubuntu:14.04                  : Ok   gcc (Ubuntu 4.8.4-2ubuntu1~14.04.4) 4.8.4
  59 ubuntu:16.04                  : Ok   gcc (Ubuntu 5.4.0-6ubuntu1~16.04.12) 5.4.0 20160609, clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
  60 ubuntu:16.04-x-arm            : Ok   arm-linux-gnueabihf-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
  61 ubuntu:16.04-x-arm64          : Ok   aarch64-linux-gnu-gcc (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
  62 ubuntu:16.04-x-powerpc        : Ok   powerpc-linux-gnu-gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
  63 ubuntu:16.04-x-powerpc64      : Ok   powerpc64-linux-gnu-gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
  64 ubuntu:16.04-x-powerpc64el    : Ok   powerpc64le-linux-gnu-gcc (Ubuntu/IBM 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
  65 ubuntu:16.04-x-s390           : Ok   s390x-linux-gnu-gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609
  66 ubuntu:18.04                  : Ok   gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0, clang version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
  67 ubuntu:18.04-x-arm            : Ok   arm-linux-gnueabihf-gcc (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1) 7.4.0
  68 ubuntu:18.04-x-arm64          : Ok   aarch64-linux-gnu-gcc (Ubuntu/Linaro 7.4.0-1ubuntu1~18.04.1) 7.4.0
  69 ubuntu:18.04-x-m68k           : Ok   m68k-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  70 ubuntu:18.04-x-powerpc        : Ok   powerpc-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  71 ubuntu:18.04-x-powerpc64      : Ok   powerpc64-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  72 ubuntu:18.04-x-powerpc64el    : Ok   powerpc64le-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  73 ubuntu:18.04-x-riscv64        : Ok   riscv64-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  74 ubuntu:18.04-x-s390           : Ok   s390x-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  75 ubuntu:18.04-x-sh4            : Ok   sh4-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  76 ubuntu:18.04-x-sparc64        : Ok   sparc64-linux-gnu-gcc (Ubuntu 7.4.0-1ubuntu1~18.04.1) 7.4.0
  77 ubuntu:18.10                  : Ok   gcc (Ubuntu 8.3.0-6ubuntu1~18.10.1) 8.3.0, clang version 7.0.0-3 (tags/RELEASE_700/final)
  78 ubuntu:19.04                  : Ok   gcc (Ubuntu 8.3.0-6ubuntu1) 8.3.0, clang version 8.0.0-3 (tags/RELEASE_800/final)
  79 ubuntu:19.04-x-alpha          : Ok   alpha-linux-gnu-gcc (Ubuntu 8.3.0-6ubuntu1) 8.3.0
  80 ubuntu:19.04-x-arm64          : Ok   aarch64-linux-gnu-gcc (Ubuntu/Linaro 8.3.0-6ubuntu1) 8.3.0
  81 ubuntu:19.04-x-hppa           : Ok   hppa-linux-gnu-gcc (Ubuntu 8.3.0-6ubuntu1) 8.3.0
  82 ubuntu:19.10                  : Ok   gcc (Ubuntu 9.2.1-9ubuntu2) 9.2.1 20191008, clang version 9.0.0-2 (tags/RELEASE_900/final)
  #
  # uname -a
  Linux quaco 5.3.16-200.fc30.x86_64 #1 SMP Fri Dec 13 17:48:38 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
  # git log --oneline -1
  55347ec340af perf hists: Fix variable name's inconsistency in hists__for_each() macro
  # perf version --build-options
  perf version 5.5.rc1.g55347ec340af
                   dwarf: [ on  ]  # HAVE_DWARF_SUPPORT
      dwarf_getlocations: [ on  ]  # HAVE_DWARF_GETLOCATIONS_SUPPORT
                   glibc: [ on  ]  # HAVE_GLIBC_SUPPORT
                    gtk2: [ on  ]  # HAVE_GTK2_SUPPORT
           syscall_table: [ on  ]  # HAVE_SYSCALL_TABLE_SUPPORT
                  libbfd: [ on  ]  # HAVE_LIBBFD_SUPPORT
                  libelf: [ on  ]  # HAVE_LIBELF_SUPPORT
                 libnuma: [ on  ]  # HAVE_LIBNUMA_SUPPORT
  numa_num_possible_cpus: [ on  ]  # HAVE_LIBNUMA_SUPPORT
                 libperl: [ on  ]  # HAVE_LIBPERL_SUPPORT
               libpython: [ on  ]  # HAVE_LIBPYTHON_SUPPORT
                libslang: [ on  ]  # HAVE_SLANG_SUPPORT
               libcrypto: [ on  ]  # HAVE_LIBCRYPTO_SUPPORT
               libunwind: [ on  ]  # HAVE_LIBUNWIND_SUPPORT
      libdw-dwarf-unwind: [ on  ]  # HAVE_DWARF_SUPPORT
                    zlib: [ on  ]  # HAVE_ZLIB_SUPPORT
                    lzma: [ on  ]  # HAVE_LZMA_SUPPORT
               get_cpuid: [ on  ]  # HAVE_AUXTRACE_SUPPORT
                     bpf: [ on  ]  # HAVE_LIBBPF_SUPPORT
                     aio: [ on  ]  # HAVE_AIO_SUPPORT
                    zstd: [ on  ]  # HAVE_ZSTD_SUPPORT
  # perf test
   1: vmlinux symtab matches kallsyms                       : Ok
   2: Detect openat syscall event                           : Ok
   3: Detect openat syscall event on all cpus               : Ok
   4: Read samples using the mmap interface                 : Ok
   5: Test data source output                               : Ok
   6: Parse event definition strings                        : Ok
   7: Simple expression parser                              : Ok
   8: PERF_RECORD_* events & perf_sample fields             : Ok
   9: Parse perf pmu format                                 : Ok
  10: DSO data read                                         : Ok
  11: DSO data cache                                        : Ok
  12: DSO data reopen                                       : Ok
  13: Roundtrip evsel->name                                 : Ok
  14: Parse sched tracepoints fields                        : Ok
  15: syscalls:sys_enter_openat event fields                : Ok
  16: Setup struct perf_event_attr                          : Ok
  17: Match and link multiple hists                         : Ok
  18: 'import perf' in python                               : Ok
  19: Breakpoint overflow signal handler                    : Ok
  20: Breakpoint overflow sampling                          : Ok
  21: Breakpoint accounting                                 : Ok
  22: Watchpoint                                            :
  22.1: Read Only Watchpoint                                : Skip
  22.2: Write Only Watchpoint                               : Ok
  22.3: Read / Write Watchpoint                             : Ok
  22.4: Modify Watchpoint                                   : Ok
  23: Number of exit events of a simple workload            : Ok
  24: Software clock events period values                   : Ok
  25: Object code reading                                   : Ok
  26: Sample parsing                                        : Ok
  27: Use a dummy software event to keep tracking           : Ok
  28: Parse with no sample_id_all bit set                   : Ok
  29: Filter hist entries                                   : Ok
  30: Lookup mmap thread                                    : Ok
  31: Share thread maps                                     : Ok
  32: Sort output of hist entries                           : Ok
  33: Cumulate child hist entries                           : Ok
  34: Track with sched_switch                               : Ok
  35: Filter fds with revents mask in a fdarray             : Ok
  36: Add fd to a fdarray, making it autogrow               : Ok
  37: kmod_path__parse                                      : Ok
  38: Thread map                                            : Ok
  39: LLVM search and compile                               :
  39.1: Basic BPF llvm compile                              : Ok
  39.2: kbuild searching                                    : Ok
  39.3: Compile source for BPF prologue generation          : Ok
  39.4: Compile source for BPF relocation                   : Ok
  40: Session topology                                      : Ok
  41: BPF filter                                            :
  41.1: Basic BPF filtering                                 : Ok
  41.2: BPF pinning                                         : Ok
  41.3: BPF prologue generation                             : Ok
  41.4: BPF relocation checker                              : Ok
  42: Synthesize thread map                                 : Ok
  43: Remove thread map                                     : Ok
  44: Synthesize cpu map                                    : Ok
  45: Synthesize stat config                                : Ok
  46: Synthesize stat                                       : Ok
  47: Synthesize stat round                                 : Ok
  48: Synthesize attr update                                : Ok
  49: Event times                                           : Ok
  50: Read backward ring buffer                             : Ok
  51: Print cpu map                                         : Ok
  52: Merge cpu map                                         : Ok
  53: Probe SDT events                                      : Ok
  54: is_printable_array                                    : Ok
  55: Print bitmap                                          : Ok
  56: perf hooks                                            : Ok
  57: builtin clang support                                 : Skip (not compiled in)
  58: unit_number__scnprintf                                : Ok
  59: mem2node                                              : Ok
  60: time utils                                            : Ok
  61: Test jit_write_elf                                    : Ok
  62: maps__merge_in                                        : Ok
  63: x86 rdpmc                                             : Ok
  64: Convert perf time to TSC                              : Ok
  65: DWARF unwind                                          : Ok
  66: x86 instruction decoder - new instructions            : Ok
  67: Intel PT packet decoder                               : Ok
  68: x86 bp modify                                         : Ok
  69: probe libc's inet_pton & backtrace it with ping       : Ok
  70: Use vfs_getname probe to get syscall args filenames   : Ok
  71: Add vfs_getname probe to get syscall args filenames   : Ok
  72: Check open filename arg using perf trace + vfs_getname: Ok
  73: Zstd perf.data compression/decompression              : Ok
  $ make -C tools/perf build-test
  make: Entering directory '/home/acme/git/perf/tools/perf'
  - tarpkg: ./tests/perf-targz-src-pkg .
         make_install_prefix_O: make install prefix=/tmp/krava
            make_install_bin_O: make install-bin
                make_install_O: make install
                    make_doc_O: make doc
           make_no_backtrace_O: make NO_BACKTRACE=1
        make_with_babeltrace_O: make LIBBABELTRACE=1
              make_clean_all_O: make clean all
                 make_perf_o_O: make perf.o
           make_no_libbionic_O: make NO_LIBBIONIC=1
              make_no_libelf_O: make NO_LIBELF=1
             make_no_scripts_O: make NO_LIBPYTHON=1 NO_LIBPERL=1
  make_no_libdw_dwarf_unwind_O: make NO_LIBDW_DWARF_UNWIND=1
                 make_static_O: make LDFLAGS=-static NO_PERF_READ_VDSO32=1 NO_PERF_READ_VDSOX32=1 NO_JVMTI=1
                  make_debug_O: make DEBUG=1
            make_no_demangle_O: make NO_DEMANGLE=1
             make_util_map_o_O: make util/map.o
                make_no_newt_O: make NO_NEWT=1
             make_no_libnuma_O: make NO_LIBNUMA=1
                   make_tags_O: make tags
                make_no_gtk2_O: make NO_GTK2=1
                   make_help_O: make help
              make_no_libbpf_O: make NO_LIBBPF=1
             make_no_libperl_O: make NO_LIBPERL=1
           make_no_libunwind_O: make NO_LIBUNWIND=1
                   make_pure_O: make
            make_no_auxtrace_O: make NO_AUXTRACE=1
                  make_no_ui_O: make NO_NEWT=1 NO_SLANG=1 NO_GTK2=1
            make_no_libaudit_O: make NO_LIBAUDIT=1
       make_util_pmu_bison_o_O: make util/pmu-bison.o
                make_minimal_O: make NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1 NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1 NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1 NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1 NO_LIBCRYPTO=1 NO_SDT=1 NO_JVMTI=1 NO_LIBZSTD=1 NO_LIBCAP=1
   make_install_prefix_slash_O: make install prefix=/tmp/krava/
         make_with_clangllvm_O: make LIBCLANGLLVM=1
           make_no_libpython_O: make NO_LIBPYTHON=1
               make_no_slang_O: make NO_SLANG=1
                 make_cscope_O: make cscope
  OK
  make: Leaving directory '/home/acme/git/perf/tools/perf'
  $
author maintainer
#1 lore raw

PATCH 1/4 tools lib traceevent: Fix memory leakage in filter_event

From: Hewenliang <[email protected]>

It is necessary to call free_arg(arg) when add_filter_type() returns NULL in filter_event().

---
 tools/lib/traceevent/parse-filter.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
tools/lib/traceevent/parse-filter.c +3 -1
index f3cbf86e51ac..20eed719542e 100644
@@ -1228,8 +1228,10 @@ filter_event(struct tep_event_filter *filter, struct tep_event *event,
 	}
 
 	filter_type = add_filter_type(filter, event->id);
-	if (filter_type == NULL)
+	if (filter_type == NULL) {
+		free_arg(arg);
 		return TEP_ERRNO__MEM_ALLOC_FAILED;
+	}
 
 	if (filter_type->filter)
 		free_arg(filter_type->filter);
Signature · 1 line
2.21.1
author maintainer
#2 lore raw

PATCH 2/4 perf report: Fix incorrectly added dimensions as switch perf data file

In reply to Arnaldo Carvalho de Melo

From: Jin Yao <[email protected]>

We observed an issue that was some extra columns displayed after switching perf data file in browser. The steps to reproduce:

1. perf record -a -e cycles,instructions -- sleep 3
2. perf report --group
3. In browser, we use hotkey 's' to switch to another perf.data
4. Now in browser, the extra columns 'Self' and 'Children' are displayed.

The issue is setup_sorting() executed again after repeat path, so dimensions are added again.

This patch checks the last key returned from __cmd_report(). If it's K_SWITCH_INPUT_DATA, skips the setup_sorting().

Fixes ad0de0971b7f ("perf report: Enable the runtime switching of perf data file")
Cc Alexander Shishkin <[email protected]>
Cc Andi Kleen <[email protected]>
Cc Feng Tang <[email protected]>
Cc Jin Yao <[email protected]>
Cc Kan Liang <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 tools/perf/builtin-report.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
tools/perf/builtin-report.c +4 -1
index 387311c67264..de988589d99b 100644
@@ -1076,6 +1076,7 @@ int cmd_report(int argc, const char **argv)
 	struct stat st;
 	bool has_br_stack = false;
 	int branch_mode = -1;
+	int last_key = 0;
 	bool branch_call_mode = false;
 #define CALLCHAIN_DEFAULT_OPT  "graph,0.5,caller,function,percent"
 	static const char report_callchain_help[] = "Display call graph (stack chain/backtrace):\n\n"
@@ -1450,7 +1451,8 @@ int cmd_report(int argc, const char **argv)
 		sort_order = sort_tmp;
 	}
 
-	if (setup_sorting(session->evlist) < 0) {
+	if ((last_key != K_SWITCH_INPUT_DATA) &&
+	    (setup_sorting(session->evlist) < 0)) {
 		if (sort_order)
 			parse_options_usage(report_usage, options, "s", 1);
 		if (field_order)
@@ -1530,6 +1532,7 @@ int cmd_report(int argc, const char **argv)
 	ret = __cmd_report(&report);
 	if (ret == K_SWITCH_INPUT_DATA) {
 		perf_session__delete(session);
+		last_key = K_SWITCH_INPUT_DATA;
 		goto repeat;
 	} else
 		ret = 0;
Signature · 1 line
2.21.1
author maintainer
#3 lore raw

PATCH 3/4 perf map: Set kmap->kmaps backpointer for main kernel map chunks

In reply to Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

When a map is create to represent the main kernel area (vmlinux) with map__new2() we allocate an extra area to store a pointer to the 'struct maps' for the kernel maps, so that we can access that struct when loading ELF files or kallsyms, as we will need to split it in multiple maps, one per kernel module or ELF section (such as ".init.text").

So when map->dso->kernel is non-zero, it is expected that map__kmap(map)->kmaps to be set to the tree of kernel maps (modules, chunks of the main kernel, bpf progs put in place via PERF_RECORD_KSYMBOL, the main kernel).

This was not the case when we were splitting the main kernel into chunks for its ELF sections, which ended up making 'perf report --children' processing a perf.data file with callchains to trip on __map__is_kernel(), when we press ENTER to see the popup menu for main histogram entries that starts at a symbol in the ".init.text" ELF section, e.g.:

- 8.83% 0.00% swapper [kernel.vmlinux].init.text [k] start_kernel
start_kernel
cpu_startup_entry
do_idle
cpuidle_enter
cpuidle_enter_state
intel_idle

Fix it.

---
 tools/perf/util/symbol-elf.c | 3 +++
 1 file changed, 3 insertions(+)
tools/perf/util/symbol-elf.c +3 -0
index 6658fbf196e6..1965aefccb02 100644
@@ -920,6 +920,9 @@ static int dso__process_kernel_symbol(struct dso *dso, struct map *map,
 		if (curr_map == NULL)
 			return -1;
 
+		if (curr_dso->kernel)
+			map__kmap(curr_map)->kmaps = kmaps;
+
 		if (adjust_kernel_syms) {
 			curr_map->start  = shdr->sh_addr + ref_reloc(kmap);
 			curr_map->end	 = curr_map->start + shdr->sh_size;
Signature · 1 line
2.21.1
author maintainer
#10 lore raw

PATCH 4/4 perf hists: Fix variable name's inconsistency in hists__for_each() macro

In reply to Arnaldo Carvalho de Melo

From: Yuya Fujita <[email protected]>

Variable names are inconsistent in hists__for_each macro().

Due to this inconsistency, the macro replaces its second argument with "fmt" regardless of its original name.

So far it works because only "fmt" is passed to the second argument. However, this behavior is not expected and should be fixed.

Fixes f0786af536bb ("perf hists: Introduce hists__for_each_format macro")
Fixes aa6f50af822a ("perf hists: Introduce hists__for_each_sort_list macro")
Cc Peter Zijlstra <[email protected]>
---
 tools/perf/util/hist.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/util/hist.h +2 -2
index 45286900aacb..0aa63aeb58ec 100644
@@ -339,10 +339,10 @@ static inline void perf_hpp__prepend_sort_field(struct perf_hpp_fmt *format)
 	list_for_each_entry_safe(format, tmp, &(_list)->sorts, sort_list)
 
 #define hists__for_each_format(hists, format) \
-	perf_hpp_list__for_each_format((hists)->hpp_list, fmt)
+	perf_hpp_list__for_each_format((hists)->hpp_list, format)
 
 #define hists__for_each_sort_list(hists, format) \
-	perf_hpp_list__for_each_sort_list((hists)->hpp_list, fmt)
+	perf_hpp_list__for_each_sort_list((hists)->hpp_list, format)
 
 extern struct perf_hpp_fmt perf_hpp__format[];
 
Signature · 1 line
2.21.1
#11 lore raw
In reply to Arnaldo Carvalho de Melo
Quoted 61 lines · Arnaldo Carvalho de Melo

Hi Ingo/Thomas,

Please consider pulling,

Best regards,

- Arnaldo

Test results at the end of this message, as usual.

The following changes since commit 9f0bff1180efc9ea988fed3fd93da7647151ac8b:

perf/core: Add SRCU annotation for pmus list walk (2019-12-17 13:32:46 +0100)

are available in the Git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-urgent-for-mingo-5.5-20191223

for you to fetch changes up to 55347ec340af401437680fd0e88df6739a967f9f:

perf hists: Fix variable name's inconsistency in hists__for_each() macro (2019-12-20 18:58:13 -0300)

----------------------------------------------------------------
perf/urgent fixes:

perf report/top:

Arnaldo Carvalho de Melo:

- Fix popup menu for entries in main kernel maps other than the main one, e.g. ".init.text", where a non-initialized pointer was causing segfaults.

Jin Yao:

- Fix incorrectly added dimensions when switching perf.data file to another via the popup menu.

libtraceevent:

Hewenliang:

- Fix memory leakage in filter_event().

perf hists:

Yuya Fujita:

- Fix variable name's inconsistency in hists__for_each() macro.

----------------------------------------------------------------
Arnaldo Carvalho de Melo (1):

      perf map: Set kmap->kmaps backpointer for main kernel map chunks

Hewenliang (1):
tools lib traceevent: Fix memory leakage in filter_event

Jin Yao (1):
perf report: Fix incorrectly added dimensions as switch perf data file

Quoted 8 lines

Yuya Fujita (1):
perf hists: Fix variable name's inconsistency in hists__for_each() macro

 tools/lib/traceevent/parse-filter.c | 4 +++-
 tools/perf/builtin-report.c         | 5 ++++-
 tools/perf/util/hist.h              | 4 ++--
 tools/perf/util/symbol-elf.c        | 3 +++
 4 files changed, 12 insertions(+), 4 deletions(-)

Pulled, thanks a lot Arnaldo!

Ingo

#4 lore raw

Re: [PATCH 3/4] perf map: Set kmap->kmaps backpointer for main kernel map chunks

In reply to Arnaldo Carvalho de Melo

Hi Arnaldo,

Quoted 28 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

When a map is create to represent the main kernel area (vmlinux) with map__new2() we allocate an extra area to store a pointer to the 'struct maps' for the kernel maps, so that we can access that struct when loading ELF files or kallsyms, as we will need to split it in multiple maps, one per kernel module or ELF section (such as ".init.text").

So when map->dso->kernel is non-zero, it is expected that map__kmap(map)->kmaps to be set to the tree of kernel maps (modules, chunks of the main kernel, bpf progs put in place via PERF_RECORD_KSYMBOL, the main kernel).

This was not the case when we were splitting the main kernel into chunks for its ELF sections, which ended up making 'perf report --children' processing a perf.data file with callchains to trip on __map__is_kernel(), when we press ENTER to see the popup menu for main histogram entries that starts at a symbol in the ".init.text" ELF section, e.g.:

- 8.83% 0.00% swapper [kernel.vmlinux].init.text [k] start_kernel
start_kernel
cpu_startup_entry
do_idle
cpuidle_enter
cpuidle_enter_state
intel_idle

Quoted 1 line

Fix it.

perf top from perf/core has started crashing at __map__is_kernel():

(gdb) bt
#0 __map__is_kernel (map=<optimized out>) at util/map.c:935 #1 0x000000000045551d in perf_event__process_sample (machine=0xbab8f8,

      sample=0x7fffe5ffa6d0, evsel=0xba7570, event=0xbcac50, tool=0x7fffffff84e0)
      at builtin-top.c:833
  #2  deliver_event (qe=<optimized out>, qevent=<optimized out>) at builtin-top.c:1192
  #3  0x000000000050b9fb in do_flush (show_progress=false, oe=0x7fffffff87e0)
      at util/ordered-events.c:244
  #4  __ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP,
      timestamp=timestamp@entry=0) at util/ordered-events.c:323
  #5  0x000000000050c1b5 in __ordered_events__flush (timestamp=<optimized out>,
      how=<optimized out>, oe=<optimized out>) at util/ordered-events.c:339
  #6  ordered_events__flush (how=OE_FLUSH__TOP, oe=0x7fffffff87e0) at util/ordered-events.c:341
  #7  ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP)
      at util/ordered-events.c:339
  #8  0x0000000000454e21 in process_thread (arg=0x7fffffff84e0) at builtin-top.c:1104
  #9  0x00007ffff7f2c4e2 in start_thread () from /lib64/libpthread.so.0
  #10 0x00007ffff76086d3 in clone () from /lib64/libc.so.6

I haven't debugged it much but seems like the actual patch that's causing the crash is de90d513b246 ("perf map: Use map->dso->kernel + map__kmaps() in map__kmaps()").

Did you face this / aware of it?

Ravi

#5 lore raw

Re: [PATCH 3/4] perf map: Set kmap->kmaps backpointer for main kernel map chunks

Quoted 54 lines · Ravi Bangoria

Hi Arnaldo,

Quoted 28 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

When a map is create to represent the main kernel area (vmlinux) with map__new2() we allocate an extra area to store a pointer to the 'struct maps' for the kernel maps, so that we can access that struct when loading ELF files or kallsyms, as we will need to split it in multiple maps, one per kernel module or ELF section (such as ".init.text").

So when map->dso->kernel is non-zero, it is expected that map__kmap(map)->kmaps to be set to the tree of kernel maps (modules, chunks of the main kernel, bpf progs put in place via PERF_RECORD_KSYMBOL, the main kernel).

This was not the case when we were splitting the main kernel into chunks for its ELF sections, which ended up making 'perf report --children' processing a perf.data file with callchains to trip on __map__is_kernel(), when we press ENTER to see the popup menu for main histogram entries that starts at a symbol in the ".init.text" ELF section, e.g.:

- 8.83% 0.00% swapper [kernel.vmlinux].init.text [k] start_kernel
start_kernel
cpu_startup_entry
do_idle
cpuidle_enter
cpuidle_enter_state
intel_idle

Quoted 1 line

Fix it.

perf top from perf/core has started crashing at __map__is_kernel():

(gdb) bt
#0 __map__is_kernel (map=<optimized out>) at util/map.c:935 #1 0x000000000045551d in perf_event__process_sample (machine=0xbab8f8,

      sample=0x7fffe5ffa6d0, evsel=0xba7570, event=0xbcac50, tool=0x7fffffff84e0)
      at builtin-top.c:833
  #2  deliver_event (qe=<optimized out>, qevent=<optimized out>) at builtin-top.c:1192
  #3  0x000000000050b9fb in do_flush (show_progress=false, oe=0x7fffffff87e0)
      at util/ordered-events.c:244
  #4  __ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP,
      timestamp=timestamp@entry=0) at util/ordered-events.c:323
  #5  0x000000000050c1b5 in __ordered_events__flush (timestamp=<optimized out>,
      how=<optimized out>, oe=<optimized out>) at util/ordered-events.c:339
  #6  ordered_events__flush (how=OE_FLUSH__TOP, oe=0x7fffffff87e0) at util/ordered-events.c:341
  #7  ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP)
      at util/ordered-events.c:339
  #8  0x0000000000454e21 in process_thread (arg=0x7fffffff84e0) at builtin-top.c:1104
  #9  0x00007ffff7f2c4e2 in start_thread () from /lib64/libpthread.so.0
  #10 0x00007ffff76086d3 in clone () from /lib64/libc.so.6
Quoted 5 lines

I haven't debugged it much but seems like the actual patch that's causing the crash is de90d513b246 ("perf map: Use map->dso->kernel + map__kmaps() in map__kmaps()").

Did you face this / aware of it?

hum, looks like there are few more places where we don't set kmaps pointer, patch below fixes that for me

I'll still need to do more checking and I'll send a fix

jirka

---

tools/perf/util/machine.c +5 -0
index c8c5410315e8..28212193e1e9 100644
@@ -686,6 +686,7 @@ static struct dso *machine__findnew_module_dso(struct machine *machine,
 
 		dso__set_module_info(dso, m, machine);
 		dso__set_long_name(dso, strdup(filename), true);
+		dso->kernel = DSO_TYPE_KERNEL;
 	}
 
 	dso__get(dso);
@@ -774,6 +775,7 @@ static struct map *machine__addnew_module_map(struct machine *machine, u64 start
 					      const char *filename)
 {
 	struct map *map = NULL;
+	struct kmap *kmap;
 	struct kmod_path m;
 	struct dso *dso;
 
@@ -790,6 +792,9 @@ static struct map *machine__addnew_module_map(struct machine *machine, u64 start
 
 	maps__insert(&machine->kmaps, map);
 
+	kmap = map__kmap(map);
+	kmap->kmaps = &machine->kmaps;
+
 	/* Put the map here because maps__insert alread got it */
 	map__put(map);
 out:
tools/perf/util/symbol.c +8 -0
index 3b379b1296f1..c3dd20082734 100644
@@ -862,6 +862,9 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta,
 				return -1;
 			}
 
+			if (ndso->kernel)
+				map__kmap(curr_map)->kmaps = kmaps;
+
 			curr_map->map_ip = curr_map->unmap_ip = identity__map_ip;
 			maps__insert(kmaps, curr_map);
 			++kernel_range;
@@ -1145,11 +1148,13 @@ static int validate_kcore_addresses(const char *kallsyms_filename,
 struct kcore_mapfn_data {
 	struct dso *dso;
 	struct list_head maps;
+	struct maps *kmaps;
 };
 
 static int kcore_mapfn(u64 start, u64 len, u64 pgoff, void *data)
 {
 	struct kcore_mapfn_data *md = data;
+	struct kmap *kmap;
 	struct map *map;
 
 	map = map__new2(start, md->dso);
@@ -1161,6 +1166,8 @@ static int kcore_mapfn(u64 start, u64 len, u64 pgoff, void *data)
 
 	list_add(&map->node, &md->maps);
 
+	kmap = map__kmap(map);
+	kmap->kmaps = md->kmaps;
 	return 0;
 }
 
@@ -1270,6 +1277,7 @@ static int dso__load_kcore(struct dso *dso, struct map *map,
 	if (validate_kcore_addresses(kallsyms_filename, map))
 		return -EINVAL;
 
+	md.kmaps = kmaps;
 	md.dso = dso;
 	INIT_LIST_HEAD(&md.maps);
 
#6 lore raw

Re: [PATCH 3/4] perf map: Set kmap->kmaps backpointer for main kernel map chunks

Quoted 64 lines · Jiri Olsa
Quoted 54 lines · Ravi Bangoria

Hi Arnaldo,

Quoted 28 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

When a map is create to represent the main kernel area (vmlinux) with map__new2() we allocate an extra area to store a pointer to the 'struct maps' for the kernel maps, so that we can access that struct when loading ELF files or kallsyms, as we will need to split it in multiple maps, one per kernel module or ELF section (such as ".init.text").

So when map->dso->kernel is non-zero, it is expected that map__kmap(map)->kmaps to be set to the tree of kernel maps (modules, chunks of the main kernel, bpf progs put in place via PERF_RECORD_KSYMBOL, the main kernel).

This was not the case when we were splitting the main kernel into chunks for its ELF sections, which ended up making 'perf report --children' processing a perf.data file with callchains to trip on __map__is_kernel(), when we press ENTER to see the popup menu for main histogram entries that starts at a symbol in the ".init.text" ELF section, e.g.:

- 8.83% 0.00% swapper [kernel.vmlinux].init.text [k] start_kernel
start_kernel
cpu_startup_entry
do_idle
cpuidle_enter
cpuidle_enter_state
intel_idle

Quoted 1 line

Fix it.

perf top from perf/core has started crashing at __map__is_kernel():

(gdb) bt
#0 __map__is_kernel (map=<optimized out>) at util/map.c:935 #1 0x000000000045551d in perf_event__process_sample (machine=0xbab8f8,

      sample=0x7fffe5ffa6d0, evsel=0xba7570, event=0xbcac50, tool=0x7fffffff84e0)
      at builtin-top.c:833
  #2  deliver_event (qe=<optimized out>, qevent=<optimized out>) at builtin-top.c:1192
  #3  0x000000000050b9fb in do_flush (show_progress=false, oe=0x7fffffff87e0)
      at util/ordered-events.c:244
  #4  __ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP,
      timestamp=timestamp@entry=0) at util/ordered-events.c:323
  #5  0x000000000050c1b5 in __ordered_events__flush (timestamp=<optimized out>,
      how=<optimized out>, oe=<optimized out>) at util/ordered-events.c:339
  #6  ordered_events__flush (how=OE_FLUSH__TOP, oe=0x7fffffff87e0) at util/ordered-events.c:341
  #7  ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP)
      at util/ordered-events.c:339
  #8  0x0000000000454e21 in process_thread (arg=0x7fffffff84e0) at builtin-top.c:1104
  #9  0x00007ffff7f2c4e2 in start_thread () from /lib64/libpthread.so.0
  #10 0x00007ffff76086d3 in clone () from /lib64/libc.so.6
Quoted 5 lines

I haven't debugged it much but seems like the actual patch that's causing the crash is de90d513b246 ("perf map: Use map->dso->kernel + map__kmaps() in map__kmaps()").

Did you face this / aware of it?

hum, looks like there are few more places where we don't set kmaps pointer, patch below fixes that for me

Quoted 6 lines

I'll still need to do more checking and I'll send a fix

jirka

---

I found one more place.. please check the attached patch

jirka

---

tools/perf/util/machine.c +17 -2
index c8c5410315e8..89b0b7713a73 100644
@@ -686,6 +686,7 @@ static struct dso *machine__findnew_module_dso(struct machine *machine,
 
 		dso__set_module_info(dso, m, machine);
 		dso__set_long_name(dso, strdup(filename), true);
+		dso->kernel = DSO_TYPE_KERNEL;
 	}
 
 	dso__get(dso);
@@ -726,13 +727,23 @@ static int machine__process_ksymbol_register(struct machine *machine,
 	struct map *map = maps__find(&machine->kmaps, event->ksymbol.addr);
 
 	if (!map) {
-		map = dso__new_map(event->ksymbol.name);
-		if (!map)
+		struct dso *dso = dso__new(event->ksymbol.name);
+		struct kmap *kmap;
+
+		if (dso) {
+			dso->kernel = DSO_TYPE_KERNEL;
+			map = map__new2(0, dso);
+		}
+
+		if (!dso || !map)
 			return -ENOMEM;
 
 		map->start = event->ksymbol.addr;
 		map->end = map->start + event->ksymbol.len;
 		maps__insert(&machine->kmaps, map);
+
+		kmap = map__kmap(map);
+		kmap->kmaps = &machine->kmaps;
 	}
 
 	sym = symbol__new(map->map_ip(map, map->start),
@@ -774,6 +785,7 @@ static struct map *machine__addnew_module_map(struct machine *machine, u64 start
 					      const char *filename)
 {
 	struct map *map = NULL;
+	struct kmap *kmap;
 	struct kmod_path m;
 	struct dso *dso;
 
@@ -790,6 +802,9 @@ static struct map *machine__addnew_module_map(struct machine *machine, u64 start
 
 	maps__insert(&machine->kmaps, map);
 
+	kmap = map__kmap(map);
+	kmap->kmaps = &machine->kmaps;
+
 	/* Put the map here because maps__insert alread got it */
 	map__put(map);
 out:
tools/perf/util/symbol.c +8 -0
index 3b379b1296f1..c3dd20082734 100644
@@ -862,6 +862,9 @@ static int maps__split_kallsyms(struct maps *kmaps, struct dso *dso, u64 delta,
 				return -1;
 			}
 
+			if (ndso->kernel)
+				map__kmap(curr_map)->kmaps = kmaps;
+
 			curr_map->map_ip = curr_map->unmap_ip = identity__map_ip;
 			maps__insert(kmaps, curr_map);
 			++kernel_range;
@@ -1145,11 +1148,13 @@ static int validate_kcore_addresses(const char *kallsyms_filename,
 struct kcore_mapfn_data {
 	struct dso *dso;
 	struct list_head maps;
+	struct maps *kmaps;
 };
 
 static int kcore_mapfn(u64 start, u64 len, u64 pgoff, void *data)
 {
 	struct kcore_mapfn_data *md = data;
+	struct kmap *kmap;
 	struct map *map;
 
 	map = map__new2(start, md->dso);
@@ -1161,6 +1166,8 @@ static int kcore_mapfn(u64 start, u64 len, u64 pgoff, void *data)
 
 	list_add(&map->node, &md->maps);
 
+	kmap = map__kmap(map);
+	kmap->kmaps = md->kmaps;
 	return 0;
 }
 
@@ -1270,6 +1277,7 @@ static int dso__load_kcore(struct dso *dso, struct map *map,
 	if (validate_kcore_addresses(kallsyms_filename, map))
 		return -EINVAL;
 
+	md.kmaps = kmaps;
 	md.dso = dso;
 	INIT_LIST_HEAD(&md.maps);
 
#7 lore raw

Re: [PATCH 3/4] perf map: Set kmap->kmaps backpointer for main kernel map chunks

On Sun, Feb 09, 2020 at 08:32:38PM +0100, Jiri Olsa wrote:

SNIP

Quoted 31 lines
Quoted 31 lines
Quoted 22 lines

perf top from perf/core has started crashing at __map__is_kernel():

(gdb) bt
#0 __map__is_kernel (map=<optimized out>) at util/map.c:935 #1 0x000000000045551d in perf_event__process_sample (machine=0xbab8f8,

      sample=0x7fffe5ffa6d0, evsel=0xba7570, event=0xbcac50, tool=0x7fffffff84e0)
      at builtin-top.c:833
  #2  deliver_event (qe=<optimized out>, qevent=<optimized out>) at builtin-top.c:1192
  #3  0x000000000050b9fb in do_flush (show_progress=false, oe=0x7fffffff87e0)
      at util/ordered-events.c:244
  #4  __ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP,
      timestamp=timestamp@entry=0) at util/ordered-events.c:323
  #5  0x000000000050c1b5 in __ordered_events__flush (timestamp=<optimized out>,
      how=<optimized out>, oe=<optimized out>) at util/ordered-events.c:339
  #6  ordered_events__flush (how=OE_FLUSH__TOP, oe=0x7fffffff87e0) at util/ordered-events.c:341
  #7  ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP)
      at util/ordered-events.c:339
  #8  0x0000000000454e21 in process_thread (arg=0x7fffffff84e0) at builtin-top.c:1104
  #9  0x00007ffff7f2c4e2 in start_thread () from /lib64/libpthread.so.0
  #10 0x00007ffff76086d3 in clone () from /lib64/libc.so.6
Quoted 5 lines

I haven't debugged it much but seems like the actual patch that's causing the crash is de90d513b246 ("perf map: Use map->dso->kernel + map__kmaps() in map__kmaps()").

Did you face this / aware of it?

hum, looks like there are few more places where we don't set kmaps pointer, patch below fixes that for me

Quoted 8 lines
Quoted 6 lines

I'll still need to do more checking and I'll send a fix

jirka

---

I found one more place.. please check the attached patch

and third time's the charm.. hopefully ;-)

I made the fix more central.. it still needs to be split into several small fixes, but I'm running perf top for few hours now without the crash

jirka

---

tools/perf/util/machine.c +10 -14
index c8c5410315e8..460315476314 100644
@@ -686,6 +686,7 @@ static struct dso *machine__findnew_module_dso(struct machine *machine,
 
 		dso__set_module_info(dso, m, machine);
 		dso__set_long_name(dso, strdup(filename), true);
+		dso->kernel = DSO_TYPE_KERNEL;
 	}
 
 	dso__get(dso);
@@ -726,8 +727,14 @@ static int machine__process_ksymbol_register(struct machine *machine,
 	struct map *map = maps__find(&machine->kmaps, event->ksymbol.addr);
 
 	if (!map) {
-		map = dso__new_map(event->ksymbol.name);
-		if (!map)
+		struct dso *dso = dso__new(event->ksymbol.name);
+
+		if (dso) {
+			dso->kernel = DSO_TYPE_KERNEL;
+			map = map__new2(0, dso);
+		}
+
+		if (!dso || !map)
 			return -ENOMEM;
 
 		map->start = event->ksymbol.addr;
@@ -972,7 +979,6 @@ int machine__create_extra_kernel_map(struct machine *machine,
 
 	kmap = map__kmap(map);
 
-	kmap->kmaps = &machine->kmaps;
 	strlcpy(kmap->name, xm->name, KMAP_NAME_LEN);
 
 	maps__insert(&machine->kmaps, map);
@@ -1082,9 +1088,6 @@ int __weak machine__create_extra_kernel_maps(struct machine *machine __maybe_unu
 static int
 __machine__create_kernel_maps(struct machine *machine, struct dso *kernel)
 {
-	struct kmap *kmap;
-	struct map *map;
-
 	/* In case of renewal the kernel map, destroy previous one */
 	machine__destroy_kernel_maps(machine);
 
@@ -1093,14 +1096,7 @@ __machine__create_kernel_maps(struct machine *machine, struct dso *kernel)
 		return -1;
 
 	machine->vmlinux_map->map_ip = machine->vmlinux_map->unmap_ip = identity__map_ip;
-	map = machine__kernel_map(machine);
-	kmap = map__kmap(map);
-	if (!kmap)
-		return -1;
-
-	kmap->kmaps = &machine->kmaps;
-	maps__insert(&machine->kmaps, map);
-
+	maps__insert(&machine->kmaps, machine->vmlinux_map);
 	return 0;
 }
 
tools/perf/util/map.c +16 -1
index f67960bedebb..a08ca276098e 100644
@@ -375,8 +375,13 @@ struct symbol *map__find_symbol_by_name(struct map *map, const char *name)
 
 struct map *map__clone(struct map *from)
 {
-	struct map *map = memdup(from, sizeof(*map));
+	size_t size = sizeof(struct map);
+	struct map *map;
+
+	if (from->dso && from->dso->kernel)
+		size += sizeof(struct kmap);
 
+	map = memdup(from, size);
 	if (map != NULL) {
 		refcount_set(&map->refcnt, 1);
 		RB_CLEAR_NODE(&map->rb_node);
@@ -538,6 +543,16 @@ void maps__insert(struct maps *maps, struct map *map)
 	__maps__insert(maps, map);
 	++maps->nr_maps;
 
+	if (map->dso && map->dso->kernel) {
+		struct kmap *kmap = map__kmap(map);
+
+		if (kmap)
+			kmap->kmaps = maps;
+		else
+			pr_err("Internal error: kernel dso with non kernel map\n");
+	}
+
+
 	/*
 	 * If we already performed some search by name, then we need to add the just
 	 * inserted map and resort.
Tested-by
#8 lore raw

Re: [PATCH 3/4] perf map: Set kmap->kmaps backpointer for main kernel map chunks

Quoted 42 lines · Jiri Olsa

On Sun, Feb 09, 2020 at 08:32:38PM +0100, Jiri Olsa wrote:

SNIP

Quoted 38 lines
Quoted 37 lines
Quoted 22 lines

perf top from perf/core has started crashing at __map__is_kernel():

   (gdb) bt
   #0  __map__is_kernel (map=<optimized out>) at util/map.c:935
   #1  0x000000000045551d in perf_event__process_sample (machine=0xbab8f8,
       sample=0x7fffe5ffa6d0, evsel=0xba7570, event=0xbcac50, tool=0x7fffffff84e0)
       at builtin-top.c:833
   #2  deliver_event (qe=<optimized out>, qevent=<optimized out>) at builtin-top.c:1192
   #3  0x000000000050b9fb in do_flush (show_progress=false, oe=0x7fffffff87e0)
       at util/ordered-events.c:244
   #4  __ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP,
       timestamp=timestamp@entry=0) at util/ordered-events.c:323
   #5  0x000000000050c1b5 in __ordered_events__flush (timestamp=<optimized out>,
       how=<optimized out>, oe=<optimized out>) at util/ordered-events.c:339
   #6  ordered_events__flush (how=OE_FLUSH__TOP, oe=0x7fffffff87e0) at util/ordered-events.c:341
   #7  ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP)
       at util/ordered-events.c:339
   #8  0x0000000000454e21 in process_thread (arg=0x7fffffff84e0) at builtin-top.c:1104
   #9  0x00007ffff7f2c4e2 in start_thread () from /lib64/libpthread.so.0
   #10 0x00007ffff76086d3 in clone () from /lib64/libc.so.6
Quoted 5 lines

I haven't debugged it much but seems like the actual patch that's causing the crash is de90d513b246 ("perf map: Use map->dso->kernel + map__kmaps() in map__kmaps()").

Did you face this / aware of it?

hum, looks like there are few more places where we don't set kmaps pointer, patch below fixes that for me

I'll still need to do more checking and I'll send a fix

jirka

---

Quoted 7 lines
Quoted 1 line

I found one more place.. please check the attached patch

and third time's the charm.. hopefully ;-)

I made the fix more central.. it still needs to be split into several small fixes, but I'm running perf top for few hours now without the crash

I also kept it running for ~20 mins but didn't see the crash. Feel free to add:

#9 lore raw

Re: [PATCH 3/4] perf map: Set kmap->kmaps backpointer for main kernel map chunks

In reply to Ravi Bangoria

Em Thu, Feb 06, 2020 at 03:10:38PM +0530, Ravi Bangoria escreveu:

Quoted 54 lines

Hi Arnaldo,

Quoted 28 lines · Arnaldo Carvalho de Melo

From: Arnaldo Carvalho de Melo <[email protected]>

When a map is create to represent the main kernel area (vmlinux) with map__new2() we allocate an extra area to store a pointer to the 'struct maps' for the kernel maps, so that we can access that struct when loading ELF files or kallsyms, as we will need to split it in multiple maps, one per kernel module or ELF section (such as ".init.text").

So when map->dso->kernel is non-zero, it is expected that map__kmap(map)->kmaps to be set to the tree of kernel maps (modules, chunks of the main kernel, bpf progs put in place via PERF_RECORD_KSYMBOL, the main kernel).

This was not the case when we were splitting the main kernel into chunks for its ELF sections, which ended up making 'perf report --children' processing a perf.data file with callchains to trip on __map__is_kernel(), when we press ENTER to see the popup menu for main histogram entries that starts at a symbol in the ".init.text" ELF section, e.g.:

- 8.83% 0.00% swapper [kernel.vmlinux].init.text [k] start_kernel
start_kernel
cpu_startup_entry
do_idle
cpuidle_enter
cpuidle_enter_state
intel_idle

Quoted 1 line

Fix it.

perf top from perf/core has started crashing at __map__is_kernel():

(gdb) bt
#0 __map__is_kernel (map=<optimized out>) at util/map.c:935 #1 0x000000000045551d in perf_event__process_sample (machine=0xbab8f8,

      sample=0x7fffe5ffa6d0, evsel=0xba7570, event=0xbcac50, tool=0x7fffffff84e0)
      at builtin-top.c:833
  #2  deliver_event (qe=<optimized out>, qevent=<optimized out>) at builtin-top.c:1192
  #3  0x000000000050b9fb in do_flush (show_progress=false, oe=0x7fffffff87e0)
      at util/ordered-events.c:244
  #4  __ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP,
      timestamp=timestamp@entry=0) at util/ordered-events.c:323
  #5  0x000000000050c1b5 in __ordered_events__flush (timestamp=<optimized out>,
      how=<optimized out>, oe=<optimized out>) at util/ordered-events.c:339
  #6  ordered_events__flush (how=OE_FLUSH__TOP, oe=0x7fffffff87e0) at util/ordered-events.c:341
  #7  ordered_events__flush (oe=oe@entry=0x7fffffff87e0, how=how@entry=OE_FLUSH__TOP)
      at util/ordered-events.c:339
  #8  0x0000000000454e21 in process_thread (arg=0x7fffffff84e0) at builtin-top.c:1104
  #9  0x00007ffff7f2c4e2 in start_thread () from /lib64/libpthread.so.0
  #10 0x00007ffff76086d3 in clone () from /lib64/libc.so.6
Quoted 5 lines

I haven't debugged it much but seems like the actual patch that's causing the crash is de90d513b246 ("perf map: Use map->dso->kernel + map__kmaps() in map__kmaps()").

Did you face this / aware of it?

No, I wasn't aware of this, will check, getting back from vacations today, so may take some time,

- Arnaldo