RFC tools/perf: Speed up the build system
Get series with b4
b4 am [email protected]Versions newest first
- v1 RFC Latest50 patches 29 repliesIn review
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/50 | tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORTtools/ tools/ tools/ +27 more | +86 | 0 replies | |
| 2/50 | tools/perf/build: Add feature check core codetools/ tools/ tools/ | +38 | 0 replies | |
| 3/50 | tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automaticallytools/ | +6 | 0 replies | |
| 4/50 | tools/perf/build: Split out feature check: 'libnuma'tools/ tools/ tools/ | +18 | 0 replies | |
| 5/50 | tools/perf/build: Split out feature check: 'stackprotector-all'tools/ tools/ tools/ | +12 | 0 replies | |
| 6/50 | tools/perf/build: Split out feature check: 'stackprotector'tools/ tools/ tools/ | +12 | 0 replies | |
| 7/50 | tools/perf/build: Split out feature check: 'volatile-register-var'tools/ tools/ tools/ | +12 | 0 replies | |
| 8/50 | tools/perf/build: Split out feature check: 'fortify-source'tools/ tools/ tools/ | +12 | 0 replies | |
| 9/50 | tools/perf/build: Split out feature check: 'bionic'tools/ tools/ tools/ | +16 | 0 replies | |
| 10/50 | tools/perf/build: Clean up the libelf logic in config/Makefiletools/ | +29 | 0 replies | |
| 11/50 | tools/perf/build: Split out feature check: 'libelf'tools/ tools/ tools/ | +13 | 0 replies | |
| 12/50 | tools/perf/build: Split out feature check: 'glibc'tools/ tools/ tools/ | +14 | 0 replies | |
| 13/50 | tools/perf/build: Split out feature check: 'dwarf'tools/ tools/ tools/ | +15 | 0 replies | |
| 14/50 | tools/perf/build: Clean up the mmap logic in config/Makefiletools/ | +26 | 0 replies | |
| 15/50 | tools/perf/build: Split out feature check: 'libelf-mmap'tools/ tools/ tools/ | +14 | 0 replies | |
| 16/50 | tools/perf/build: Split out feature check: 'libelf-getphdrnum'tools/ tools/ tools/ | +14 | 0 replies | |
| 17/50 | tools/perf/build: Clean up the libunwind logic in config/Makefiletools/ | +12 | 0 replies | |
| 18/50 | tools/perf/build: Split out feature check: 'libunwind'tools/ tools/ tools/ | +27 | 0 replies | |
| 19/50 | tools/perf/build: Split out feature check: 'libaudit'tools/ tools/ tools/ | +14 | 2 replies | |
| 20/50 | tools/perf/build: Split out feature check: 'libslang'tools/ tools/ tools/ | +13 | 2 replies | |
| 21/50 | tools/perf/build: Split out feature check: 'gtk2'tools/ tools/ tools/ | +15 | 0 replies | |
| 22/50 | tools/perf/build: Split out feature check: 'gtk2-infobar'tools/ tools/ tools/ | +16 | 0 replies | |
| 23/50 | tools/perf/build: Split out feature check: 'libperl'tools/ tools/ tools/ | +24 | 0 replies | |
| 24/50 | tools/perf/build: Split out feature check: 'libpython'tools/ tools/ tools/ | +27 | 0 replies | |
| 25/50 | tools/perf/build: Split out feature check: 'libpython-version'tools/ tools/ tools/ | +16 | 0 replies | |
| 26/50 | tools/perf/build: Split out feature check: 'libbfd'tools/ tools/ tools/ +1 more | +25 | 3 replies | |
| 27/50 | tools/perf/build: Split out feature check: 'on-exit'tools/ tools/ tools/ | +12 | 0 replies | |
| 28/50 | tools/perf/build: Split out feature check: 'backtrace'tools/ tools/ tools/ | +16 | 0 replies | |
| 29/50 | tools/perf: Clean up util/include/linux/compiler.htools/ | +10 | 0 replies | |
| 30/50 | tools/perf: Turn strlcpy() into a __weak functiontools/ tools/ tools/ +3 more | +12 | 0 replies | |
| 31/50 | tools/perf/build: Speed up auto-detection of features by adding a 'test-all' targettools/ tools/ tools/ | +240 | 3 replies | |
| 32/50 | tools/perf/build: Speed up git-version test on re-maketools/ | +13 | 0 replies | |
| 33/50 | tools/perf/build: Speed up the final linktools/ | +7 | 0 replies | |
| 34/50 | tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et altools/ | +4 | 0 replies | |
| 35/50 | tools/perf/build: Invoke feature-checks 'clean' target from the main Makefiletools/ | +9 | 0 replies | |
| 36/50 | tools/perf/build: Speed up auto-detectiontools/ tools/ | +44 | 2 replies | |
| 37/50 | tools/perf/build: Improve printout-of auto-detected featurestools/ | +33 | 5 replies | |
| 38/50 | tools/perf/build: Automatically build in parallel, based on number of CPUs in the systemtools/ | +26 | 0 replies | |
| 39/50 | tools/perf/build: Flip Makefile.parallel and Makefile.perftools/ tools/ tools/ | +855 | 0 replies | |
| 40/50 | tools/perf/build: Standardize the various messages output by parallel maketools/ tools/ | +22 | 0 replies | |
| 41/50 | tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle'tools/ tools/ tools/ | +28 | 0 replies | |
| 42/50 | tools/perf/build: Remove unused config/feature-tests.maktools/ tools/ tools/ | +1 | 0 replies | |
| 43/50 | tools/perf/build: Clean up various testcasestools/ tools/ tools/ +5 more | +47 | 0 replies | |
| 44/50 | tools/perf/build: Collapse the test-all.c testcasetools/ tools/ tools/ | +102 | 2 replies | |
| 45/50 | tools/perf/build: Pass through all targets to Makefile.perftools/ | +7 | 0 replies | |
| 46/50 | tools/perf/build: Make sure autodep feature binaries honor the O= settingtools/ | +8 | 0 replies | |
| 47/50 | tools/perf/build: Exclude MAKEFLAGS from nested invocationtools/ | +13 | 0 replies | |
| 48/50 | tools/perf/build: Fix non-canonical directory names in O=tools/ | +8 | 0 replies | |
| 49/50 | tools/perf/build: Fix O=/some/dir perf.o type of targetstools/ tools/ | +17 | 2 replies | |
| 50/50 | tools/perf/build: Harmonize the style of the feature testcasestools/ tools/ tools/ +9 more | +11 | 0 replies |
Reviews trailers per patch
None of the 50 patches has a Reviewed-by or Acked-by.
| Patch | David Ahern |
|---|---|
| Series | T-b |
| 1/50 | |
| 2/50 | |
| 3/50 | |
| 4/50 | |
| 5/50 | |
| 6/50 | |
| 7/50 | |
| 8/50 | |
| 9/50 | |
| 10/50 | |
| 11/50 | |
| 12/50 | |
| 13/50 | |
| 14/50 | |
| 15/50 | |
| 16/50 | |
| 17/50 | |
| 18/50 | |
| 19/50 | |
| 20/50 | |
| 21/50 | |
| 22/50 | |
| 23/50 | |
| 24/50 | |
| 25/50 | |
| 26/50 | |
| 27/50 | |
| 28/50 | |
| 29/50 | |
| 30/50 | |
| 31/50 | |
| 32/50 | |
| 33/50 | |
| 34/50 | |
| 35/50 | |
| 36/50 | |
| 37/50 | |
| 38/50 | |
| 39/50 | |
| 40/50 | |
| 41/50 | |
| 42/50 | |
| 43/50 | |
| 44/50 | |
| 45/50 | |
| 46/50 | |
| 47/50 | |
| 48/50 | |
| 49/50 | |
| 50/50 |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [RFC PATCH 00/50] tools/perf: Speed up the build system
This series improves various aspects of the tools/perf build system.
The biggest change is the rewrite of the feature detection framework and a conversion of all feature testcases to the new method. The new feature detection machinery relies on three new methods:
- split-out, independent testcases that allow "make -j" parallel builds [ the old build system built the 20+ testcases serially, greatly slowing down the build. ]
- dependency auto-generation via gcc's -MD feature, which allows the 'caching' of testcases.
- a 'test-all' testcase that contains all the other testcases and allows a fast check for the typical 'all dependencies are installed' case.
The series also includes a number of fixes and convenience features:
- auto-detect the number of CPUs on the system and add the -j<cpus> option automatically
- Fix 'make O=/some/dir/ perf.o' type of rules to work in the $(OUTPUT) directory
- Improve documentation builds/install to not require feature-detection
- speed up linking, eliminate spurious rebuilds
Altogether build speed on my testsystem improved by a factor of 2.1x:
[BEFORE]
$ cd tools/perf
$ time make -j12 install
... real 0m27.522s
user 1m59.478s
sys 0m9.714s[AFTER]
$ cd tools/perf
$ time make install
... real 0m12.658s
user 1m39.507s
sys 0m7.706sRe-build speed, which is important to perf developers, increased 5.0x:
[BEFORE]
$ touch perf.c
$ time make
... real 0m9.368s
user 0m8.166s
sys 0m1.015s[AFTER]
$ touch perf.c
$ time make
BUILD: Doing 'make -j12' parallel build
CC perf.o
LINK perf real 0m1.857s
user 0m1.522s
sys 0m0.295sEmpty build speed of 'make' in an already built tree - often done by developers to see whether they are testing the latest binary, increased more than 50-fold:
[BEFORE]
$ time make
... real 0m9.105s
user 0m7.975s
sys 0m0.949s[AFTER]
$ time make
$ time make
BUILD: Doing 'make -j12' parallel build real 0m0.172s
user 0m0.119s
sys 0m0.029sThanks,
Ingo
=============>
Ingo Molnar (50):
tools/perf: Standardize feature support define names to: HAVE_{FEATURE}_SUPPORT tools/perf/build: Add feature check core code
tools/perf/build: Add 'autodep' functionality, generate feature test dependencies automatically tools/perf/build: Split out feature check: 'libnuma'
tools/perf/build: Split out feature check: 'stackprotector-all' tools/perf/build: Split out feature check: 'stackprotector'
tools/perf/build: Split out feature check: 'volatile-register-var' tools/perf/build: Split out feature check: 'fortify-source'
tools/perf/build: Split out feature check: 'bionic'
tools/perf/build: Clean up the libelf logic in config/Makefile tools/perf/build: Split out feature check: 'libelf'
tools/perf/build: Split out feature check: 'glibc'
tools/perf/build: Split out feature check: 'dwarf'
tools/perf/build: Clean up the mmap logic in config/Makefile
tools/perf/build: Split out feature check: 'libelf-mmap'
tools/perf/build: Split out feature check: 'libelf-getphdrnum' tools/perf/build: Clean up the libunwind logic in config/Makefile tools/perf/build: Split out feature check: 'libunwind'
tools/perf/build: Split out feature check: 'libaudit'
tools/perf/build: Split out feature check: 'libslang'
tools/perf/build: Split out feature check: 'gtk2'
tools/perf/build: Split out feature check: 'gtk2-infobar'
tools/perf/build: Split out feature check: 'libperl'
tools/perf/build: Split out feature check: 'libpython'
tools/perf/build: Split out feature check: 'libpython-version' tools/perf/build: Split out feature check: 'libbfd'
tools/perf/build: Split out feature check: 'on-exit'
tools/perf/build: Split out feature check: 'backtrace'
tools/perf: Clean up util/include/linux/compiler.h
tools/perf: Turn strlcpy() into a __weak function
tools/perf/build: Speed up auto-detection of features by adding a 'test-all' target tools/perf/build: Speed up git-version test on re-make
tools/perf/build: Speed up the final link
tools/perf: Fix double/triple-build of the feature detection logic during 'make install' et al tools/perf/build: Invoke feature-checks 'clean' target from the main Makefile tools/perf/build: Speed up auto-detection
tools/perf/build: Improve printout-of auto-detected features
tools/perf/build: Automatically build in parallel, based on number of CPUs in the system tools/perf/build: Flip Makefile.parallel and Makefile.perf
tools/perf/build: Standardize the various messages output by parallel make tools/perf/build: Split out feature checks: 'liberty', 'liberty-z', 'cplus-demangle' tools/perf/build: Remove unused config/feature-tests.mak
tools/perf/build: Clean up various testcases
tools/perf/build: Collapse the test-all.c testcase
tools/perf/build: Pass through all targets to Makefile.perf
tools/perf/build: Make sure autodep feature binaries honor the O= setting tools/perf/build: Exclude MAKEFLAGS from nested invocation
tools/perf/build: Fix non-canonical directory names in O=
tools/perf/build: Fix O=/some/dir perf.o type of targets
tools/perf/build: Harmonize the style of the feature testcases
tools/perf/Documentation/Makefile | 19 +- tools/perf/Makefile | 840 ++--------------------------- tools/perf/Makefile.perf | 855 ++++++++++++++++++++++++++++++ tools/perf/arch/x86/include/perf_regs.h | 6 +- tools/perf/arch/x86/util/unwind.c | 4 +- tools/perf/bench/mem-memcpy-arch.h | 2 +- tools/perf/bench/mem-memcpy.c | 2 +- tools/perf/bench/mem-memset-arch.h | 2 +- tools/perf/bench/mem-memset.c | 2 +- tools/perf/builtin-bench.c | 4 +- tools/perf/builtin-inject.c | 2 +- tools/perf/builtin-probe.c | 14 +- tools/perf/builtin-record.c | 12 +- tools/perf/config/Makefile | 326 +++++++----- tools/perf/config/feature-checks/Makefile | 148 ++++++ tools/perf/config/feature-checks/test-all.c | 106 ++++ tools/perf/config/feature-checks/test-backtrace.c | 13 + tools/perf/config/feature-checks/test-bionic.c | 6 + tools/perf/config/feature-checks/test-cplus-demangle.c | 14 + tools/perf/config/feature-checks/test-dwarf.c | 10 + tools/perf/config/feature-checks/test-fortify-source.c | 6 + tools/perf/config/feature-checks/test-glibc.c | 8 + tools/perf/config/feature-checks/test-gtk2-infobar.c | 11 + tools/perf/config/feature-checks/test-gtk2.c | 10 + tools/perf/config/feature-checks/test-hello.c | 6 + tools/perf/config/feature-checks/test-libaudit.c | 10 + tools/perf/config/feature-checks/test-libbfd.c | 15 + tools/perf/config/feature-checks/test-libelf-getphdrnum.c | 8 + tools/perf/config/feature-checks/test-libelf-mmap.c | 8 + tools/perf/config/feature-checks/test-libelf.c | 8 + tools/perf/config/feature-checks/test-libnuma.c | 9 + tools/perf/config/feature-checks/test-libperl.c | 9 + tools/perf/config/feature-checks/test-libpython-version.c | 10 + tools/perf/config/feature-checks/test-libpython.c | 8 + tools/perf/config/feature-checks/test-libslang.c | 6 + tools/perf/config/feature-checks/test-libunwind.c | 27 + tools/perf/config/feature-checks/test-on-exit.c | 15 + tools/perf/config/feature-checks/test-stackprotector-all.c | 6 + tools/perf/config/feature-checks/test-stackprotector.c | 6 + tools/perf/config/feature-checks/test-volatile-register-var.c | 6 + tools/perf/config/feature-tests.mak | 246 --------- tools/perf/config/utilities.mak | 14 - tools/perf/perf.c | 4 +- tools/perf/ui/gtk/browser.c | 2 +- tools/perf/ui/gtk/gtk.h | 4 +- tools/perf/ui/gtk/util.c | 4 +- tools/perf/ui/ui.h | 4 +- tools/perf/util/annotate.h | 4 +- tools/perf/util/cache.h | 3 +- tools/perf/util/generate-cmdlist.sh | 4 +- tools/perf/util/hist.h | 4 +- tools/perf/util/include/dwarf-regs.h | 2 +- tools/perf/util/include/linux/compiler.h | 19 +- tools/perf/util/map.c | 2 +- tools/perf/util/path.c | 10 +- tools/perf/util/perf_regs.h | 4 +- tools/perf/util/probe-event.c | 4 +- tools/perf/util/probe-finder.h | 4 +- tools/perf/util/symbol-elf.c | 2 +- tools/perf/util/symbol.h | 8 +- tools/perf/util/unwind.h | 4 +- tools/perf/util/util.c | 4 +- 62 files changed, 1678 insertions(+), 1257 deletions(-) create mode 100644 tools/perf/Makefile.perf create mode 100644 tools/perf/config/feature-checks/Makefile create mode 100644 tools/perf/config/feature-checks/test-all.c create mode 100644 tools/perf/config/feature-checks/test-backtrace.c create mode 100644 tools/perf/config/feature-checks/test-bionic.c create mode 100644 tools/perf/config/feature-checks/test-cplus-demangle.c create mode 100644 tools/perf/config/feature-checks/test-dwarf.c create mode 100644 tools/perf/config/feature-checks/test-fortify-source.c create mode 100644 tools/perf/config/feature-checks/test-glibc.c create mode 100644 tools/perf/config/feature-checks/test-gtk2-infobar.c create mode 100644 tools/perf/config/feature-checks/test-gtk2.c create mode 100644 tools/perf/config/feature-checks/test-hello.c create mode 100644 tools/perf/config/feature-checks/test-libaudit.c create mode 100644 tools/perf/config/feature-checks/test-libbfd.c create mode 100644 tools/perf/config/feature-checks/test-libelf-getphdrnum.c create mode 100644 tools/perf/config/feature-checks/test-libelf-mmap.c create mode 100644 tools/perf/config/feature-checks/test-libelf.c create mode 100644 tools/perf/config/feature-checks/test-libnuma.c create mode 100644 tools/perf/config/feature-checks/test-libperl.c create mode 100644 tools/perf/config/feature-checks/test-libpython-version.c create mode 100644 tools/perf/config/feature-checks/test-libpython.c create mode 100644 tools/perf/config/feature-checks/test-libslang.c create mode 100644 tools/perf/config/feature-checks/test-libunwind.c create mode 100644 tools/perf/config/feature-checks/test-on-exit.c create mode 100644 tools/perf/config/feature-checks/test-stackprotector-all.c create mode 100644 tools/perf/config/feature-checks/test-stackprotector.c create mode 100644 tools/perf/config/feature-checks/test-volatile-register-var.c delete mode 100644 tools/perf/config/feature-tests.mak
Signature · 1 line
1.7.11.7