perf/core improvements and fixes

Hi Ingo,

Please consider pulling, should get linux-next free of perf build fixdep
related race conditions on high core count machines,

- Arnaldo

Test results at the end of this message, as usual.

The following changes since commit 34c4a42791bbc455e65a15d12dcd0b6b3c52ad13:

Merge tag 'perf-core-for-mingo-20161205' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2016-12-06 09:14:56 +0100)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git tags/perf-core-for-mingo-20161207

for you to fetch changes up to 108a7c103b761309ccbd997002e8428808cf1e04:

perf tools: Explicitly document that --children is enabled by default (2016-12-07 12:00:35 -0300)

----------------------------------------------------------------
perf/core improvements and fixes:

Improvements:

- Improve error message when analyzing file with required events in 'perf sched timehist' (David Ahern)

Fixes:

- Force fixdep compilation to be done at the start of the build, fixing some build race conditions in high core count machines (Jiri Olsa)

- Fix handling a zero sample->tid in 'perf sched timehist', as sometimes that isn't the idle thread (Namhyung Kim)

Infrastructure:

- Check minimal accepted LLVM version in its feature check, 3.9 at this time (Wang Nan)

Documentation:

- Explicitly document that --children is enabled by default (Yannick Brosseau)

----------------------------------------------------------------
David Ahern (1):
perf sched timehist: Improve error message when analyzing wrong file

Jiri Olsa (3):
perf tools: Move PERF-VERSION-FILE target into rules area perf tools: Force fixdep compilation at the start of the build perf tools: Move perf build related variables under non fixdep leg

Namhyung Kim (4):
perf sched: Cleanup option processing
perf callchain: Introduce callchain_cursor__copy() perf sched timehist: Handle zero sample->tid properly perf sched timehist: Cleanup idle_max_cpu handling

Wang Nan (1):
perf build: Check LLVM version in feature check

Yannick Brosseau (1):
perf tools: Explicitly document that --children is enabled by default

 tools/build/feature/Makefile              |  8 +++-
 tools/build/feature/test-llvm-version.cpp | 11 +++++
 tools/build/feature/test-llvm.cpp         |  5 +++
 tools/perf/Documentation/perf-report.txt  |  3 +-
 tools/perf/Documentation/perf-top.txt     |  1 +
 tools/perf/Makefile.config                |  8 +++-
 tools/perf/Makefile.perf                  | 68 +++++++++++++++++++++----------
 tools/perf/builtin-sched.c                | 26 ++++++------
 tools/perf/util/callchain.c               | 27 ++++++++++++
 tools/perf/util/callchain.h               |  3 ++
 10 files changed, 122 insertions(+), 38 deletions(-)
 create mode 100644 tools/build/feature/test-llvm-version.cpp

  # uname -a
  Linux jouet 4.8.8-300.fc25.x86_64 #1 SMP Tue Nov 15 18:10:06 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  # perf test
   1: vmlinux symtab matches kallsyms            : Ok
   2: Detect openat syscall event                : Ok
   3: Detect openat syscall event on all cpus    : Ok
   4: Read samples using the mmap interface      : Ok
   5: Parse event definition strings             : Ok
   6: PERF_RECORD_* events & perf_sample fields  : Ok
   7: Parse perf pmu format                      : Ok
   8: DSO data read                              : Ok
   9: DSO data cache                             : Ok
  10: DSO data reopen                            : Ok
  11: Roundtrip evsel->name                      : Ok
  12: Parse sched tracepoints fields             : Ok
  13: syscalls:sys_enter_openat event fields     : Ok
  14: Setup struct perf_event_attr               : Ok
  15: Match and link multiple hists              : Ok
  16: 'import perf' in python                    : Ok
  17: Breakpoint overflow signal handler         : Ok
  18: Breakpoint overflow sampling               : Ok
  19: Number of exit events of a simple workload : Ok
  20: Software clock events period values        : Ok
  21: Object code reading                        : Ok
  22: Sample parsing                             : Ok
  23: Use a dummy software event to keep tracking: Ok
  24: Parse with no sample_id_all bit set        : Ok
  25: Filter hist entries                        : Ok
  26: Lookup mmap thread                         : Ok
  27: Share thread mg                            : Ok
  28: Sort output of hist entries                : Ok
  29: Cumulate child hist entries                : Ok
  30: Track with sched_switch                    : Ok
  31: Filter fds with revents mask in a fdarray  : Ok
  32: Add fd to a fdarray, making it autogrow    : Ok
  33: kmod_path__parse                           : Ok
  34: Thread map                                 : Ok
  35: LLVM search and compile                    :
  35.1: Basic BPF llvm compile                    : Ok
  35.2: kbuild searching                          : Ok
  35.3: Compile source for BPF prologue generation: Ok
  35.4: Compile source for BPF relocation         : Ok
  36: Session topology                           : Ok
  37: BPF filter                                 :
  37.1: Basic BPF filtering                      : Ok
  37.2: BPF prologue generation                  : Ok
  37.3: BPF relocation checker                   : Ok
  38: Synthesize thread map                      : Ok
  39: Synthesize cpu map                         : Ok
  40: Synthesize stat config                     : Ok
  41: Synthesize stat                            : Ok
  42: Synthesize stat round                      : Ok
  43: Synthesize attr update                     : Ok
  44: Event times                                : Ok
  45: Read backward ring buffer                  : Ok
  46: Print cpu map                              : Ok
  47: Probe SDT events                           : Ok
  48: is_printable_array                         : Ok
  49: Print bitmap                               : Ok
  50: perf hooks                                 : Ok
  51: builtin clang support                      :
  51.1: builtin clang compile C source to IR        : Ok
  51.2: builtin clang compile C source to ELF object: Ok
  52: x86 rdpmc                                  : Ok
  53: Convert perf time to TSC                   : Ok
  54: DWARF unwind                               : Ok
  55: x86 instruction decoder - new instructions : Ok
  56: Intel cqm nmi context read                 : Skip
  # 
  # uname -a
  Linux zoo 4.7.3-200.fc24.x86_64 #1 SMP Wed Sep 7 17:31:21 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
  # time dm
   1 alpine:3.4: Ok
   2 android-ndk:r12b-arm: Ok
   3 archlinux:latest: Ok
   4 centos:5: Ok
   5 centos:6: Ok
   6 centos:7: Ok
   7 debian:7: Ok
   8 debian:8: Ok
   9 debian:experimental: Ok
  10 fedora:20: Ok
  11 fedora:21: Ok
  12 fedora:22: Ok
  13 fedora:23: Ok
  14 fedora:24: Ok
  15 fedora:24-x-ARC-uClibc: Ok
  16 fedora:rawhide: Ok
  17 mageia:5: Ok
  18 opensuse:13.2: Ok
  19 opensuse:42.1: Ok
  20 opensuse:tumbleweed: Ok
  21 ubuntu:12.04.5: Ok
  22 ubuntu:14.04: Ok
  23 ubuntu:14.04.4: Ok
  24 ubuntu:15.10: Ok
  25 ubuntu:16.04: Ok
  26 ubuntu:16.04-x-arm: Ok
  27 ubuntu:16.04-x-arm64: Ok
  28 ubuntu:16.04-x-powerpc: Ok
  29 ubuntu:16.04-x-powerpc64: Ok
  30 ubuntu:16.04-x-powerpc64el: Ok
  31 ubuntu:16.04-x-s390: Ok
  32 ubuntu:16.10: Ok
  #
  $ make -C tools/perf build-test
  make: Entering directory '/home/acme/git/linux/tools/perf'
  - tarpkg: ./tests/perf-targz-src-pkg .
                  make_no_ui_O: make NO_NEWT=1 NO_SLANG=1 NO_GTK2=1
                   make_help_O: make help
   make_install_prefix_slash_O: make install prefix=/tmp/krava/
                make_install_O: make install
                 make_static_O: make LDFLAGS=-static
           make_no_libbionic_O: make NO_LIBBIONIC=1
           make_no_backtrace_O: make NO_BACKTRACE=1
           make_no_libunwind_O: make NO_LIBUNWIND=1
                make_minimal_O: make NO_LIBPERL=1 NO_LIBPYTHON=1 NO_NEWT=1 NO_GTK2=1 NO_DEMANGLE=1 NO_LIBELF=1 NO_LIBUNWIND=1 NO_BACKTRACE=1 NO_LIBNUMA=1 NO_LIBAUDIT=1 NO_LIBBIONIC=1 NO_LIBDW_DWARF_UNWIND=1 NO_AUXTRACE=1 NO_LIBBPF=1 NO_LIBCRYPTO=1 NO_SDT=1 NO_JVMTI=1
                    make_doc_O: make doc
              make_no_libbpf_O: make NO_LIBBPF=1
             make_util_map_o_O: make util/map.o
            make_install_bin_O: make install-bin
            make_no_auxtrace_O: make NO_AUXTRACE=1
        make_with_babeltrace_O: make LIBBABELTRACE=1
         make_with_clangllvm_O: make LIBCLANGLLVM=1
                  make_debug_O: make DEBUG=1
              make_no_libelf_O: make NO_LIBELF=1
              make_clean_all_O: make clean all
             make_no_libperl_O: make NO_LIBPERL=1
            make_no_libaudit_O: make NO_LIBAUDIT=1
               make_no_slang_O: make NO_SLANG=1
       make_util_pmu_bison_o_O: make util/pmu-bison.o
                make_no_gtk2_O: make NO_GTK2=1
                   make_pure_O: make
             make_no_scripts_O: make NO_LIBPYTHON=1 NO_LIBPERL=1
         make_install_prefix_O: make install prefix=/tmp/krava
                   make_tags_O: make tags
            make_no_demangle_O: make NO_DEMANGLE=1
                 make_perf_o_O: make perf.o
  make_no_libdw_dwarf_unwind_O: make NO_LIBDW_DWARF_UNWIND=1
             make_no_libnuma_O: make NO_LIBNUMA=1
           make_no_libpython_O: make NO_LIBPYTHON=1
                make_no_newt_O: make NO_NEWT=1
  OK
  make: Leaving directory '/home/acme/git/linux/tools/perf'
  $ 

From: Wang Nan <[email protected]>

Cancel builtin llvm and clang support when LLVM version is less than 3.9.0: following commits uses newer API.

Since Clang/LLVM's API is not guaranteed to be stable, add a test-llvm-version.cpp feature checker, issue warning if LLVM found in compiling environment is not tested yet.

Committer Notes:

Testing it:

Environment:

  $ cat /etc/fedora-release
  Fedora release 25 (Twenty Five)
  $ rpm -q llvm-devel clang-devel
  llvm-devel-3.8.0-1.fc25.x86_64
  clang-devel-3.8.0-2.fc25.x86_64
  $

Before:

  $  make -k LIBCLANGLLVM=1 O=/tmp/build/perf -C tools/perf install-bin
  make: Entering directory '/home/acme/git/linux/tools/perf'
    BUILD:   Doing 'make -j4' parallel build
  Warning: tools/include/uapi/linux/bpf.h differs from kernel
  Warning: tools/arch/arm/include/uapi/asm/kvm.h differs from kernel
    INSTALL  GTK UI
    LINK     /tmp/build/perf/perf
  /tmp/build/perf/libperf.a(libperf-in.o): In function `perf::createCompilerInvocation(llvm::SmallVector<char const*, 16u>, llvm::StringRef&, clang::DiagnosticsEngine&)':
  /home/acme/git/linux/tools/perf/util/c++/clang.cpp:56: undefined reference to `clang::tooling::newInvocation(clang::DiagnosticsEngine*, llvm::SmallVector<char const*, 16u> const&)'
  /tmp/build/perf/libperf.a(libperf-in.o): In function `perf::getModuleFromSource(llvm::SmallVector<char const*, 16u>, llvm::StringRef, llvm::IntrusiveRefCntPtr<clang::vfs::FileSystem>)':
  /home/acme/git/linux/tools/perf/util/c++/clang.cpp:68: undefined reference to `clang::CompilerInstance::CompilerInstance(std::shared_ptr<clang::PCHContainerOperations>, bool)'
  /home/acme/git/linux/tools/perf/util/c++/clang.cpp:69: undefined reference to `clang::CompilerInstance::createDiagnostics(clang::DiagnosticConsumer*, bool)'
  <SNIP>

After:

Makefile.config:807: No suitable libLLVM found, disabling builtin clang and llvm support. Please install llvm-dev(el) (>= 3.9.0)

Updating the environment to a locally built LLVM 4.0 + clang 3.9 (forgot to git pull, duh) combo, all works as expected, it is properly detected and built into the resulting perf binary.

---
 tools/build/feature/Makefile              |  8 ++++++--
 tools/build/feature/test-llvm-version.cpp | 11 +++++++++++
 tools/build/feature/test-llvm.cpp         |  5 +++++
 tools/perf/Makefile.config                |  8 ++++++--
 4 files changed, 28 insertions(+), 4 deletions(-)
 create mode 100644 tools/build/feature/test-llvm-version.cpp

index 303196c16019..b564a2eea039 100644
@@ -231,14 +231,18 @@ $(OUTPUT)test-jvmti.bin:
 	$(BUILD)
 
 $(OUTPUT)test-llvm.bin:
-	$(BUILDXX) -std=gnu++11 					\
+	$(BUILDXX) -std=gnu++11 				\
 		-I$(shell $(LLVM_CONFIG) --includedir) 		\
 		-L$(shell $(LLVM_CONFIG) --libdir)		\
 		$(shell $(LLVM_CONFIG) --libs Core BPF)		\
 		$(shell $(LLVM_CONFIG) --system-libs)
 
+$(OUTPUT)test-llvm-version.bin:
+	$(BUILDXX) -std=gnu++11 				\
+		-I$(shell $(LLVM_CONFIG) --includedir)
+
 $(OUTPUT)test-clang.bin:
-	$(BUILDXX) -std=gnu++11 					\
+	$(BUILDXX) -std=gnu++11 				\
 		-I$(shell $(LLVM_CONFIG) --includedir) 		\
 		-L$(shell $(LLVM_CONFIG) --libdir)		\
 		-Wl,--start-group -lclangBasic -lclangDriver	\

new file mode 100644
index 000000000000..896d31724568
@@ -0,0 +1,11 @@
+#include <cstdio>
+#include "llvm/Config/llvm-config.h"
+
+#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH)
+#define pass int main() {printf("%x\n", NUM_VERSION); return 0;}
+
+#if NUM_VERSION >= 0x030900
+pass
+#else
+# error This LLVM is not tested yet.
+#endif

index d8d2cee35345..455a332dc8a8 100644
@@ -1,5 +1,10 @@
 #include "llvm/Support/ManagedStatic.h"
 #include "llvm/Support/raw_ostream.h"
+#define NUM_VERSION (((LLVM_VERSION_MAJOR) << 16) + (LLVM_VERSION_MINOR << 8) + LLVM_VERSION_PATCH)
+
+#if NUM_VERSION < 0x030900
+# error "LLVM version too low"
+#endif
 int main()
 {
 	llvm::errs() << "Hello World!\n";

index 09c2a9874f2f..76c84f0eec52 100644
@@ -802,12 +802,13 @@ ifdef LIBCLANGLLVM
     msg := $(warning No g++ found, disable clang and llvm support. Please install g++)
   else
     $(call feature_check,llvm)
+    $(call feature_check,llvm-version)
     ifneq ($(feature-llvm), 1)
-      msg := $(warning No libLLVM found, disable clang and llvm support. Please install llvm-dev)
+      msg := $(warning No suitable libLLVM found, disabling builtin clang and LLVM support. Please install llvm-dev(el) (>= 3.9.0))
     else
       $(call feature_check,clang)
       ifneq ($(feature-clang), 1)
-        msg := $(warning No libclang found, disable clang and llvm support. Please install libclang-dev)
+        msg := $(warning No suitable libclang found, disabling builtin clang and LLVM support. Please install libclang-dev(el) (>= 3.9.0))
       else
         CFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT
         CXXFLAGS += -DHAVE_LIBCLANGLLVM_SUPPORT -I$(shell $(LLVM_CONFIG) --includedir)

@@ -816,6 +817,9 @@ ifdef LIBCLANGLLVM
 	USE_CXX = 1
 	USE_LLVM = 1
 	USE_CLANG = 1
+        ifneq ($(feature-llvm-version),1)
+          msg := $(warning This version of LLVM is not tested. May cause build errors)
+        endif
       endif
     endif
   endif

2.9.3

From: Jiri Olsa <[email protected]>

An upcoming fixdep fix needs all targets at the same area, so they'll fit under a signal condition block.

Moving PERF-VERSION-FILE target into rules section.

---
 tools/perf/Makefile.perf | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

index 10495c9dbe71..9e5a6e1a387d 100644
@@ -128,10 +128,6 @@ endif
 # (this improves performance and avoids hard-to-debug behaviour);
 MAKEFLAGS += -r
 
-$(OUTPUT)PERF-VERSION-FILE: ../../.git/HEAD
-	$(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
-	$(Q)touch $(OUTPUT)PERF-VERSION-FILE
-
 # Makefiles suck: This macro sets a default value of $(2) for the
 # variable named by $(1), unless the variable has been set by
 # environment or command line. This is necessary for CC and AR

@@ -505,6 +501,10 @@ $(OUTPUT)common-cmds.h: $(wildcard Documentation/perf-*.txt)
 $(SCRIPTS) : % : %.sh
 	$(QUIET_GEN)$(INSTALL) '[email protected]' '$(OUTPUT)$@'
 
+$(OUTPUT)PERF-VERSION-FILE: ../../.git/HEAD
+	$(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
+	$(Q)touch $(OUTPUT)PERF-VERSION-FILE
+
 # These can record PERF_VERSION
 perf.spec $(SCRIPTS) \
 	: $(OUTPUT)PERF-VERSION-FILE

2.9.3

From: Jiri Olsa <[email protected]>

The fixdep tool needs to be built before everything else, because it fixes every object dependency file.

We handle this currently by making all objects to depend on fixdep, which is error prone and is easily forgotten when new object is added.

Instead of this, this patch force fixdep tool to be built as the first target in the separate make session. This way we don't need to handle extra fixdep dependencies and we are certain there's no fixdep race with any parallel make job.

Committer notes:

Testing it:

Before:

  $ rm -rf /tmp/build/perf/ ; mkdir -p /tmp/build/perf ; make -k O=/tmp/build/perf -C tools/perf install-bin
  make: Entering directory '/home/acme/git/linux/tools/perf'
    BUILD:   Doing 'make -j4' parallel build

Auto-detecting system features:

  ...                         dwarf: [ on  ]
  ...            dwarf_getlocations: [ on  ]
  ...                         glibc: [ on  ]
  ...                          gtk2: [ on  ]
  ...                      libaudit: [ on  ]
  ...                        libbfd: [ on  ]
  ...                        libelf: [ on  ]
  ...                       libnuma: [ on  ]
  ...        numa_num_possible_cpus: [ on  ]
  ...                       libperl: [ on  ]
  ...                     libpython: [ on  ]
  ...                      libslang: [ on  ]
  ...                     libcrypto: [ on  ]
  ...                     libunwind: [ on  ]
  ...            libdw-dwarf-unwind: [ on  ]
  ...                          zlib: [ on  ]
  ...                          lzma: [ on  ]
  ...                     get_cpuid: [ on  ]
  ...                           bpf: [ on  ]

    GEN      /tmp/build/perf/common-cmds.h
    HOSTCC   /tmp/build/perf/fixdep.o
    HOSTLD   /tmp/build/perf/fixdep-in.o
    LINK     /tmp/build/perf/fixdep
    MKDIR    /tmp/build/perf/pmu-events/
    HOSTCC   /tmp/build/perf/pmu-events/json.o
    MKDIR    /tmp/build/perf/pmu-events/
    HOSTCC   /tmp/build/perf/pmu-events/jsmn.o
    HOSTCC   /tmp/build/perf/pmu-events/jevents.o
    HOSTLD   /tmp/build/perf/pmu-events/jevents-in.o
    PERF_VERSION = 4.9.rc8.g868cd5
    CC       /tmp/build/perf/perf-read-vdso32

<SNIP>

After:

  $ rm -rf /tmp/build/perf/ ; mkdir -p /tmp/build/perf ; make -k O=/tmp/build/perf -C tools/perf install-bin
  make: Entering directory '/home/acme/git/linux/tools/perf'
    BUILD:   Doing 'make -j4' parallel build
    HOSTCC   /tmp/build/perf/fixdep.o
    HOSTLD   /tmp/build/perf/fixdep-in.o
    LINK     /tmp/build/perf/fixdep

Auto-detecting system features:

  ...                         dwarf: [ on  ]
  ...            dwarf_getlocations: [ on  ]
  ...                         glibc: [ on  ]
  ...                          gtk2: [ on  ]
  ...                      libaudit: [ on  ]
  ...                        libbfd: [ on  ]
  ...                        libelf: [ on  ]
  ...                       libnuma: [ on  ]
  ...        numa_num_possible_cpus: [ on  ]
  ...                       libperl: [ on  ]
  ...                     libpython: [ on  ]
  ...                      libslang: [ on  ]
  ...                     libcrypto: [ on  ]
  ...                     libunwind: [ on  ]
  ...            libdw-dwarf-unwind: [ on  ]
  ...                          zlib: [ on  ]
  ...                          lzma: [ on  ]
  ...                     get_cpuid: [ on  ]
  ...                           bpf: [ on  ]

    GEN      /tmp/build/perf/common-cmds.h
    MKDIR    /tmp/build/perf/fd/
    CC       /tmp/build/perf/fd/array.o
    LD       /tmp/build/perf/fd/libapi-in.o
    MKDIR    /tmp/build/perf/fs/
    CC       /tmp/build/perf/event-parse.o
    CC       /tmp/build/perf/fs/fs.o
    PERF_VERSION = 4.9.rc8.g57a92f
    CC       /tmp/build/perf/event-plugin.o
    MKDIR    /tmp/build/perf/fs/
    CC       /tmp/build/perf/fs/tracing_path.o

<SNIP>

---
 tools/perf/Makefile.perf | 50 ++++++++++++++++++++++++++++++++++++------------
 1 file changed, 38 insertions(+), 12 deletions(-)

index 9e5a6e1a387d..33b1d9f8555f 100644
@@ -181,6 +181,35 @@ ifeq ($(filter-out $(NON_CONFIG_TARGETS),$(MAKECMDGOALS)),)
 endif
 endif
 
+# The fixdep build - we force fixdep tool to be built as
+# the first target in the separate make session not to be
+# disturbed by any parallel make jobs. Once fixdep is done
+# we issue the requested build with FIXDEP=1 variable.
+#
+# The fixdep build is disabled for $(NON_CONFIG_TARGETS)
+# targets, because it's not necessary.
+
+ifdef FIXDEP
+  force_fixdep := 0
+else
+  force_fixdep := $(config)
+endif
+
+export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
+export HOSTCC HOSTLD HOSTAR
+
+include $(srctree)/tools/build/Makefile.include
+
+ifeq ($(force_fixdep),1)
+goals := $(filter-out all sub-make, $(MAKECMDGOALS))
+
+$(goals) all: sub-make
+
+sub-make: fixdep
+	$(Q)$(MAKE) FIXDEP=1 -f Makefile.perf $(goals)
+
+else # force_fixdep
+
 # Set FEATURE_TESTS to 'all' so all possible feature checkers are executed.
 # Without this setting the output feature dump file misses some features, for
 # example, liberty. Select all checkers so we won't get an incomplete feature

@@ -365,10 +394,6 @@ strip: $(PROGRAMS) $(OUTPUT)perf
 
 PERF_IN := $(OUTPUT)perf-in.o
 
-export srctree OUTPUT RM CC CXX LD AR CFLAGS CXXFLAGS V BISON FLEX AWK
-export HOSTCC HOSTLD HOSTAR
-include $(srctree)/tools/build/Makefile.include
-
 JEVENTS       := $(OUTPUT)pmu-events/jevents
 JEVENTS_IN    := $(OUTPUT)pmu-events/jevents-in.o
 

@@ -487,7 +512,7 @@ $(OUTPUT)perf: $(PERFLIBS) $(PERF_IN) $(PMU_EVENTS_IN) $(LIBTRACEEVENT_DYNAMIC_L
 	$(QUIET_LINK)$(CC) $(CFLAGS) $(LDFLAGS) $(LIBTRACEEVENT_DYNAMIC_LIST_LDFLAGS) \
 		$(PERF_IN) $(PMU_EVENTS_IN) $(LIBS) -o $@
 
-$(GTK_IN): fixdep FORCE
+$(GTK_IN): FORCE
 	$(Q)$(MAKE) $(build)=gtk
 
 $(OUTPUT)libperf-gtk.so: $(GTK_IN) $(PERFLIBS)

@@ -536,7 +561,7 @@ endif
 __build-dir = $(subst $(OUTPUT),,$(dir $@))
 build-dir   = $(if $(__build-dir),$(__build-dir),.)
 
-prepare: $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)common-cmds.h fixdep archheaders
+prepare: $(OUTPUT)PERF-VERSION-FILE $(OUTPUT)common-cmds.h archheaders
 
 $(OUTPUT)%.o: %.c prepare FORCE
 	$(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@

@@ -586,7 +611,7 @@ $(patsubst perf-%,%.o,$(PROGRAMS)): $(wildcard */*.h)
 
 LIBPERF_IN := $(OUTPUT)libperf-in.o
 
-$(LIBPERF_IN): prepare fixdep FORCE
+$(LIBPERF_IN): prepare FORCE
 	$(Q)$(MAKE) $(build)=libperf
 
 $(LIB_FILE): $(LIBPERF_IN)

@@ -594,10 +619,10 @@ $(LIB_FILE): $(LIBPERF_IN)
 
 LIBTRACEEVENT_FLAGS += plugin_dir=$(plugindir_SQ)
 
-$(LIBTRACEEVENT): fixdep FORCE
+$(LIBTRACEEVENT): FORCE
 	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) $(OUTPUT)libtraceevent.a
 
-libtraceevent_plugins: fixdep FORCE
+libtraceevent_plugins: FORCE
 	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) plugins
 
 $(LIBTRACEEVENT_DYNAMIC_LIST): libtraceevent_plugins

@@ -610,21 +635,21 @@ $(LIBTRACEEVENT)-clean:
 install-traceevent-plugins: libtraceevent_plugins
 	$(Q)$(MAKE) -C $(TRACE_EVENT_DIR) $(LIBTRACEEVENT_FLAGS) O=$(OUTPUT) install_plugins
 
-$(LIBAPI): fixdep FORCE
+$(LIBAPI): FORCE
 	$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) $(OUTPUT)libapi.a
 
 $(LIBAPI)-clean:
 	$(call QUIET_CLEAN, libapi)
 	$(Q)$(MAKE) -C $(LIB_DIR) O=$(OUTPUT) clean >/dev/null
 
-$(LIBBPF): fixdep FORCE
+$(LIBBPF): FORCE
 	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) $(OUTPUT)libbpf.a FEATURES_DUMP=$(FEATURE_DUMP_EXPORT)
 
 $(LIBBPF)-clean:
 	$(call QUIET_CLEAN, libbpf)
 	$(Q)$(MAKE) -C $(BPF_DIR) O=$(OUTPUT) clean >/dev/null
 
-$(LIBSUBCMD): fixdep FORCE
+$(LIBSUBCMD): FORCE
 	$(Q)$(MAKE) -C $(SUBCMD_DIR) O=$(OUTPUT) $(OUTPUT)libsubcmd.a
 
 $(LIBSUBCMD)-clean:

@@ -832,3 +857,4 @@ FORCE:
 .PHONY: $(GIT-HEAD-PHONY) TAGS tags cscope FORCE prepare
 .PHONY: libtraceevent_plugins archheaders
 
+endif # force_fixdep

2.9.3

From: Jiri Olsa <[email protected]>

Because there's no need for them in fixdep build.

---
 tools/perf/Makefile.perf | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

index 33b1d9f8555f..8f1c258b151a 100644
@@ -164,11 +164,6 @@ BISON   = bison
 STRIP   = strip
 AWK     = awk
 
-LIB_DIR		= $(srctree)/tools/lib/api/
-TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
-BPF_DIR		= $(srctree)/tools/lib/bpf/
-SUBCMD_DIR	= $(srctree)/tools/lib/subcmd/
-
 # include Makefile.config by default and rule out
 # non-config cases
 config := 1

@@ -210,6 +205,11 @@ sub-make: fixdep
 
 else # force_fixdep
 
+LIB_DIR         = $(srctree)/tools/lib/api/
+TRACE_EVENT_DIR = $(srctree)/tools/lib/traceevent/
+BPF_DIR         = $(srctree)/tools/lib/bpf/
+SUBCMD_DIR      = $(srctree)/tools/lib/subcmd/
+
 # Set FEATURE_TESTS to 'all' so all possible feature checkers are executed.
 # Without this setting the output feature dump file misses some features, for
 # example, liberty. Select all checkers so we won't get an incomplete feature

2.9.3

From: David Ahern <[email protected]>

Arnaldo reported an unhelpful error message when running perf sched timehist on a file that did not contain sched tracepoints:

[root@jouet ~]# perf sched timehist
No trace sample to read. Did you call 'perf record -R'?

[root@jouet ~]# perf evlist -v
cycles:ppp: size: 112, { sample_period, sample_freq }: 4000, sample_type: IP|TID|TIME|CALLCHAIN|CPU|PERIOD, disabled: 1, inherit: 1, mmap: 1, comm: 1, freq: 1, task: 1, precise_ip: 3, sample_id_all: 1, exclude_guest: 1, mmap2: 1, comm_exec: 1

Change the has_traces check to look for the sched_switch event. Analysis for perf sched timehist requires at least this event.

Now when analyzing a file without sched tracepoints you get:

    root@f21-vbox:/tmp$ perf sched timehist
    No sched_switch events found. Have you run 'perf sched record'?

---
 tools/perf/builtin-sched.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

index 870d94cd20ba..84c39f177bef 100644
@@ -2629,8 +2629,12 @@ static int perf_sched__timehist(struct perf_sched *sched)
 	if (perf_session__set_tracepoints_handlers(session, handlers))
 		goto out;
 
-	if (!perf_session__has_traces(session, "record -R"))
+	/* sched_switch event at a minimum needs to exist */
+	if (!perf_evlist__find_tracepoint_by_name(session->evlist,
+						  "sched:sched_switch")) {
+		pr_err("No sched_switch events found. Have you run 'perf sched record'?\n");
 		goto out;
+	}
 
 	if (sched->show_migrations &&
 	    perf_session__set_tracepoints_handlers(session, migrate_handlers))

2.9.3

From: Namhyung Kim <[email protected]>

The -D/--dump-raw-trace option is in the parent option so no need to repeat it. Also move -f/--force option to parent as it's common to handle data file.

---
 tools/perf/builtin-sched.c | 7 ++-----
 1 file changed, 2 insertions(+), 5 deletions(-)

index 84c39f177bef..634d8cfb4693 100644
@@ -2583,6 +2583,7 @@ static int perf_sched__timehist(struct perf_sched *sched)
 	struct perf_data_file file = {
 		.path = input_name,
 		.mode = PERF_DATA_MODE_READ,
+		.force = sched->force,
 	};
 
 	struct perf_session *session;

@@ -2988,6 +2989,7 @@ int cmd_sched(int argc, const char **argv, const char *prefix __maybe_unused)
 		    "be more verbose (show symbol address, etc)"),
 	OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
 		    "dump raw trace in ASCII"),
+	OPT_BOOLEAN('f', "force", &sched.force, "don't complain, do it"),
 	OPT_END()
 	};
 	const struct option latency_options[] = {

@@ -2995,8 +2997,6 @@ int cmd_sched(int argc, const char **argv, const char *prefix __maybe_unused)
 		   "sort by key(s): runtime, switch, avg, max"),
 	OPT_INTEGER('C', "CPU", &sched.profile_cpu,
 		    "CPU to profile on"),
-	OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
-		    "dump raw trace in ASCII"),
 	OPT_BOOLEAN('p', "pids", &sched.skip_merge,
 		    "latency stats per pid instead of per comm"),
 	OPT_PARENT(sched_options)

@@ -3004,9 +3004,6 @@ int cmd_sched(int argc, const char **argv, const char *prefix __maybe_unused)
 	const struct option replay_options[] = {
 	OPT_UINTEGER('r', "repeat", &sched.replay_repeat,
 		     "repeat the workload replay N times (-1: infinite)"),
-	OPT_BOOLEAN('D', "dump-raw-trace", &dump_trace,
-		    "dump raw trace in ASCII"),
-	OPT_BOOLEAN('f', "force", &sched.force, "don't complain, do it"),
 	OPT_PARENT(sched_options)
 	};
 	const struct option map_options[] = {

2.9.3

From: Namhyung Kim <[email protected]>

The callchain_cursor__copy() function is to save current callchain captured by a cursor. It'll be used to keep callchains when switching to idle task for each cpu.

---
 tools/perf/util/callchain.c | 27 +++++++++++++++++++++++++++
 tools/perf/util/callchain.h |  3 +++
 2 files changed, 30 insertions(+)

index 823befd8209a..42922512c1c6 100644
@@ -1234,3 +1234,30 @@ int callchain_node__make_parent_list(struct callchain_node *node)
 	}
 	return -ENOMEM;
 }
+
+int callchain_cursor__copy(struct callchain_cursor *dst,
+			   struct callchain_cursor *src)
+{
+	int rc = 0;
+
+	callchain_cursor_reset(dst);
+	callchain_cursor_commit(src);
+
+	while (true) {
+		struct callchain_cursor_node *node;
+
+		node = callchain_cursor_current(src);
+		if (node == NULL)
+			break;
+
+		rc = callchain_cursor_append(dst, node->ip, node->map, node->sym,
+					     node->branch, &node->branch_flags,
+					     node->nr_loop_iter, node->samples);
+		if (rc)
+			break;
+
+		callchain_cursor_advance(src);
+	}
+
+	return rc;
+}

index d9c70dccf06a..35c8e379530f 100644
@@ -216,6 +216,9 @@ static inline void callchain_cursor_advance(struct callchain_cursor *cursor)
 	cursor->pos++;
 }
 
+int callchain_cursor__copy(struct callchain_cursor *dst,
+			   struct callchain_cursor *src);
+
 struct option;
 struct hist_entry;
 

2.9.3

From: Namhyung Kim <[email protected]>

Sometimes samples have tid of 0 but non-0 pid. It ends up having a new thread of 0 tid/pid (instead of referring idle task) since tid is used to search matching task. But I guess it's wrong to use 0 as a tid when pid is set. This patch uses tid only if it has a non-zero value or same as pid (of 0).

---
 tools/perf/builtin-sched.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

index 634d8cfb4693..c8b3e6c330c3 100644
@@ -2118,7 +2118,9 @@ static struct thread *timehist_get_thread(struct perf_sched *sched,
 			pr_err("Failed to get idle thread for cpu %d.\n", sample->cpu);
 
 	} else {
-		thread = machine__findnew_thread(machine, sample->pid, sample->tid);
+		/* there were samples with tid 0 but non-zero pid */
+		thread = machine__findnew_thread(machine, sample->pid,
+						 sample->tid ?: sample->pid);
 		if (thread == NULL) {
 			pr_debug("Failed to get thread for tid %d. skipping sample.\n",
 				 sample->tid);

2.9.3

From: Namhyung Kim <[email protected]>

It treats the idle_max_cpu little bit confusingly IMHO. Let's make it more straight forward.

---
 tools/perf/builtin-sched.c | 9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

index c8b3e6c330c3..1a3f1be93372 100644
@@ -2010,7 +2010,7 @@ static int init_idle_threads(int ncpu)
 	if (!idle_threads)
 		return -ENOMEM;
 
-	idle_max_cpu = ncpu - 1;
+	idle_max_cpu = ncpu;
 
 	/* allocate the actual thread struct if needed */
 	for (i = 0; i < ncpu; ++i) {

@@ -2031,7 +2031,7 @@ static void free_idle_threads(void)
 	if (idle_threads == NULL)
 		return;
 
-	for (i = 0; i <= idle_max_cpu; ++i) {
+	for (i = 0; i < idle_max_cpu; ++i) {
 		if ((idle_threads[i]))
 			thread__delete(idle_threads[i]);
 	}

@@ -2054,8 +2054,7 @@ static struct thread *get_idle_thread(int cpu)
 			return NULL;
 
 		idle_threads = (struct thread **) p;
-		i = idle_max_cpu ? idle_max_cpu + 1 : 0;
-		for (; i < j; ++i)
+		for (i = idle_max_cpu; i < j; ++i)
 			idle_threads[i] = NULL;
 
 		idle_max_cpu = j;

@@ -2495,7 +2494,7 @@ static void timehist_print_summary(struct perf_sched *sched,
 		return;
 
 	printf("\nIdle stats:\n");
-	for (i = 0; i <= idle_max_cpu; ++i) {
+	for (i = 0; i < idle_max_cpu; ++i) {
 		t = idle_threads[i];
 		if (!t)
 			continue;

2.9.3

From: Yannick Brosseau <[email protected]>

The fact that the --children option is enabled by default is buried deep at the end of the help page, in the overhead calculation section. This make it explicit right where the option is listed, following the same way other default options are described

---
 tools/perf/Documentation/perf-report.txt | 3 ++-
 tools/perf/Documentation/perf-top.txt    | 1 +
 2 files changed, 3 insertions(+), 1 deletion(-)

index 3a166ae4a4d3..f2914f03ae7b 100644
@@ -239,7 +239,8 @@ OPTIONS
 	Accumulate callchain of children to parent entry so that then can
 	show up in the output.  The output will have a new "Children" column
 	and will be sorted on the data.  It requires callchains are recorded.
-	See the `overhead calculation' section for more details.
+	See the `overhead calculation' section for more details. Enabled by
+	default, disable with --no-children.
 
 --max-stack::
 	Set the stack depth limit when parsing the callchain, anything

index 91d638df3a6b..e71d63843f45 100644
@@ -170,6 +170,7 @@ Default is to monitor all CPUS.
 	show up in the output.  The output will have a new "Children" column
 	and will be sorted on the data.  It requires -g/--call-graph option
 	enabled.  See the `overhead calculation' section for more details.
+	Enabled by default, disable with --no-children.
 
 --max-stack::
 	Set the stack depth limit when parsing the callchain, anything

2.9.3

Pulled, thanks a lot Arnaldo!

Ingo