perf build: Reduce build time by one third
Subsystems (3)
Ian Rogers
reviewercover perf build: Reduce build time by one third
This patch series refactors many aspects of the perf build aiming to better encapsulate BPF code generation, remove serial build code and gain build parallelism. The prepare step that blocks the parallel build is reduced to a core 6 smaller dependencies. BPF skeletons are made regular dependencies on the targets that use them. Feature tests and dependencies are reorgnized. The jevents.py script processes json files in parallel and allows the big_c_string to be compiled separately.
On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 36%:
Before:
real 0m29.006s
user 2m46.019s
sys 0m30.610sAfter:
real 0m18.498s
user 2m32.922s
sys 0m27.623sSummary of Patches:
1: bpftool Bootstrap Optimization
- Exempts bpftool bootstrap from non-essential feature tests (LLVM, libbfd, libcap), saving 1.1s of sub-make fork overhead during Kbuild startup.
2-4: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.
5-8: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dump vmlinux.h in the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
9-11: Foundational Linkage & Fast-Path Feature Detection
- Eliminates redundant libbpf sub-make feature checks during static builds.
- Integrates libdebuginfod directly into test-all.c, allowing Make to skip individual feature check sub-make forks during AST parsing on fully configured workstations.
12-13: jevents.py Concurrency & Deduplication
- Splits the massive 2.8 MB big_c_string literal out of pmu-events.c into a dedicated pmu-events-string.c compilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations.
- Pre-populates jevents.py JSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms).
14: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.
Ian Rogers (14):
bpftool build: Restrict feature tests during bootstrap compilation perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak perf build: Move static libbpf dependency out of prepare step perf build: Pre-generate BPF skeletons during umbrella prepare phase perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds tools build: Integrate libdebuginfod into test-all fast path perf pmu-events: Split big_c_string storage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
tools/bpf/bpftool/Makefile | 5 +
tools/build/Makefile.feature | 6 +-
tools/build/feature/Makefile | 2 +-
tools/build/feature/test-all.c | 5 +
tools/perf/Build | 2 +
tools/perf/Makefile.config | 6 +-
tools/perf/Makefile.perf | 427 +-----------------
tools/perf/bench/Build | 6 +
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 110 +++++
tools/perf/builtin-trace.c | 30 +-
tools/perf/pmu-events/Build | 15 +-
tools/perf/pmu-events/jevents.py | 56 ++-
tools/perf/trace/beauty/Build | 280 ++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 +++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 6 +-
tools/perf/trace/beauty/futex_val3.c | 6 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 1 +
tools/perf/trace/beauty/perf_event_open.c | 22 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 22 +
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 17 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 +-
tools/perf/util/env.h | 1 +
40 files changed, 700 insertions(+), 491 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.mak
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH 1/14 bpftool build: Restrict feature tests during bootstrap compilation
When bpf_skel.mak executes 'make -C ../bpf/bpftool bootstrap', bpftool's Makefile unconditionally evaluated feature checks for llvm, libcap, libbfd, and disassembler libraries because the bootstrap target was not exempted.
Since the bootstrap bpftool strictly compiles minimal AST parsing and C code generation logic without linking LLVM or disassembler libraries, these feature check sub-makes are completely redundant.
Exempt the bootstrap target from non-essential feature tests to eliminate unneeded sub-make fork overhead during Kbuild startup.
--- tools/bpf/bpftool/Makefile | 5 +++++ 1 file changed, 5 insertions(+)
tools/bpf/bpftool/Makefile +5 -0
@@ -106,6 +106,10 @@ ifneq ($(SKIP_CRYPTO),1) CRYPTO_LIBS := -lcrypto endif +ifeq ($(MAKECMDGOALS),bootstrap) +FEATURE_TESTS := libelf-zstd +FEATURE_DISPLAY := +else FEATURE_TESTS := clang-bpf-co-re FEATURE_TESTS += llvm FEATURE_TESTS += libcap
@@ -122,6 +126,7 @@ FEATURE_DISPLAY += libcap FEATURE_DISPLAY += libbfd FEATURE_DISPLAY += libbfd-liberty FEATURE_DISPLAY += libbfd-liberty-z +endif check_feat := 1 NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install doc-uninstall
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH 2/14 perf trace beauty: Make beauty generated C code standalone .o files
In reply to Ian RogersPreviously, builtin-trace.c directly included 15 embedded C files (e.g. trace/beauty/mmap.c and fsconfig_arrays.c), which in turn depend on dozens of generated beauty script arrays. To satisfy these embedded inclusions, the global Makefile.perf would define all the generator variables/rules and include them in the massive sequential prepare umbrella target, choking parallel build startup.
Furthermore, tools/perf/util/syscalltbl.c included its own generated mapper, and util/env.c conditionally included arch_errno_names.c inline, splitting consumers across directories and preventing clean Make encapsulation.
Refactor the framework to achieve better encapsulation:
1. Move util/syscalltbl.[ch] into trace/beauty/ to co-locate with all generated code consumers.
2. Create fsconfig.c and flatten embedded beauty .c files to compile as independent standalone objects via trace/beauty/Build, exporting their formatting functions via beauty.h and env.h. Switch arch_errno_names.o and syscalltbl.o assignments directly to perf-util-y and add a top-level recursive kbuild hook (perf-util-$(CONFIG_TRACE) += trace/beauty/) to compile them into libperf-util.a, resolving remote linkage for standalone python extensions.
3. Bridge private opaque references (struct trace) securely via accessors trace__show_zeros() and trace__host(), avoiding header entanglements.
4. Consolidate all generator variables, script paths, and array generation rules entirely out of Makefile.perf and place them directly inside the exact local Build files where their output objects are compiled (trace/beauty/Build and trace/beauty/tracepoints/Build), binding prerequisites locally. Use $(call rule_mkdir) directly inside generator recipes to guarantee dynamic directory creation before script redirection, and append $(call echo-cmd,gen) across all rules to print clean, standardized GEN ... file.c output during compilation.
5. Clean up clean target to recursively remove the generated directory instead of relying on dozens of individual variables.
This unchokes the "prepare" target parallel barrier, allows make to evaluate generation scripts purely locally where consumed, and flattens the tracepoint formatting architecture.
Testing a parallel build (make -j28 all from scratch) shows improvements:
Before:
real 0m28.689s
user 2m38.490s
sys 0m30.148sAfter:
real 0m27.642s
user 2m32.356s
sys 0m26.683sSo reclaiming ~9.6 seconds of raw CPU time and over 1 full second off overall real-world build latency, by overlapping sub-make startup and avoiding top-level double-parsing overhead.
---
tools/perf/Build | 2 +
tools/perf/Makefile.perf | 282 +-----------------
tools/perf/builtin-trace.c | 30 +-
tools/perf/trace/beauty/Build | 280 +++++++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 ++++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 6 +-
tools/perf/trace/beauty/futex_val3.c | 6 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 1 +
tools/perf/trace/beauty/perf_event_open.c | 22 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 22 ++
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 4 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 +-
tools/perf/util/env.h | 1 +
29 files changed, 468 insertions(+), 352 deletions(-)
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)tools/perf/Build +2 -0
@@ -34,6 +34,8 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-$(CONFIG_TRACE) += trace/beauty/ endif +perf-util-y += trace/beauty/ + perf-$(CONFIG_LIBELF) += builtin-probe.o perf-bench-y += bench/
tools/perf/Makefile.perf +2 -280
300 lines not on this page · raw message
tools/perf/builtin-trace.c +13 -17
@@ -60,12 +60,13 @@#include "callchain.h" #include "print_binary.h" #include "string2.h" -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "../perf.h" #include "trace_augment.h" #include "dwarf-regs.h" #include <errno.h> +#include <sys/stat.h> #include <inttypes.h> #include <poll.h> #include <signal.h>
@@ -234,6 +235,16 @@ struct trace {const char *uid_str; }; +bool trace__show_zeros (const struct trace *trace) +{ +return trace->show_zeros; +} + +struct machine *trace__host (const struct trace *trace) +{ +return trace->host; +} +static void trace__load_vmlinux_btf (struct trace *trace__maybe_unused ) {#ifdef HAVE_LIBBPF_SUPPORT
@@ -776,9 +787,7 @@ static const char *fsmount_flags[] = { }; static DEFINE_STRARRAY(fsmount_flags, "FSMOUNT_"); -#include "trace/beauty/generated/fsconfig_arrays.c" -static DEFINE_STRARRAY(fsconfig_cmds, "FSCONFIG_");static const char *epoll_ctl_ops[] = {"ADD" ,"DEL" ,"MOD" , }; static DEFINE_STRARRAY_OFFSET(epoll_ctl_ops, "EPOLL_CTL_", 1);
@@ -1128,20 +1137,7 @@ static bool syscall_arg__strtoul_btf_type(char *bf __maybe_unused, size_t size _ .parm = &strarray__##array, \ .show_zero = true, } -#include "trace/beauty/eventfd.c" -#include "trace/beauty/futex_op.c" -#include "trace/beauty/futex_val3.c" -#include "trace/beauty/mmap.c" -#include "trace/beauty/mode_t.c" -#include "trace/beauty/msg_flags.c" -#include "trace/beauty/open_flags.c" -#include "trace/beauty/perf_event_open.c" -#include "trace/beauty/pid.c" -#include "trace/beauty/sched_policy.c" -#include "trace/beauty/seccomp.c" -#include "trace/beauty/signum.c" -#include "trace/beauty/socket_type.c" -#include "trace/beauty/waitid_options.c" +static const struct syscall_fmt syscall_fmts[] = { { .name ="access" ,
tools/perf/trace/beauty/Build +280 -0
289 lines not on this page · raw message
tools/perf/trace/beauty/arch_errno_names.c +2 -0
@@ -1 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "util/env.h" #include "trace/beauty/generated/arch_errno_name_array.c"
tools/perf/trace/beauty/arch_errno_names.sh +1 -1
@@ -57,7 +57,7 @@ create_arch_errno_table_func() archlist="$1" default="$2" - printf 'static arch_syscalls__strerrno_t *\n' + printf 'arch_syscalls__strerrno_t *\n' printf 'arch_syscalls__strerrno_function(const char *arch)\n' printf '{\n' for arch in $archlist; do
tools/perf/trace/beauty/beauty.h +60 -0
70 lines not on this page · raw message
tools/perf/trace/beauty/eventfd.c +4 -2
@@ -11,7 +11,9 @@#define EFD_CLOEXEC 02000000#endif -static size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="EFD_" ;
@@ -36,4 +38,4 @@ static size_t syscall_arg__scnprintf_eventfd_flags(char *bf, size_t size, structreturn printed; } -#define SCA_EFD_FLAGS syscall_arg__scnprintf_eventfd_flags +
tools/perf/trace/beauty/fsconfig.c new file +5 -0
@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "trace/beauty/generated/fsconfig_arrays.c" + +DEFINE_STRARRAY (fsconfig_cmds,"FSCONFIG_" );
tools/perf/trace/beauty/futex_op.c +4 -2
@@ -17,7 +17,9 @@#define FUTEX_CLOCK_REALTIME 256#endif -static size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="FUTEX_" ;
@@ -60,4 +62,4 @@ static size_t syscall_arg__scnprintf_futex_op(char *bf, size_t size, struct syscreturn printed; } -#define SCA_FUTEX_OP syscall_arg__scnprintf_futex_op +
tools/perf/trace/beauty/futex_val3.c +4 -2
@@ -5,7 +5,9 @@#define FUTEX_BITSET_MATCH_ANY 0xffffffff#endif -static size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) {const char *prefix ="FUTEX_BITSET_" ;unsigned int bitset = arg->val;
@@ -16,4 +18,4 @@ static size_t syscall_arg__scnprintf_futex_val3(char *bf, size_t size, struct syreturn scnprintf (bf, size,"%#xd" , bitset); } -#define SCA_FUTEX_VAL3 syscall_arg__scnprintf_futex_val3 +
tools/perf/trace/beauty/mmap.c +12 -12
@@ -1,5 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 #include <linux/log2.h> +#include "trace/beauty/beauty.h" #include "trace/beauty/generated/mmap_prot_array.c" static DEFINE_STRARRAY(mmap_prot, "PROT_");
@@ -8,8 +9,7 @@ static size_t mmap__scnprintf_prot(unsigned long prot, char *bf, size_t size, bo {return strarray__scnprintf_flags (&strarray__mmap_prot, bf, size, show_prefix, prot); } - -static size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) {unsigned long prot = arg->val;
@@ -19,18 +19,18 @@ static size_t syscall_arg__scnprintf_mmap_prot(char *bf, size_t size, struct sysreturn mmap__scnprintf_prot (prot, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_PROT syscall_arg__scnprintf_mmap_prot +#include "trace/beauty/generated/mmap_flags_array.c" -static DEFINE_STRARRAY(mmap_flags, "MAP_"); +DEFINE_STRARRAY (mmap_flags,"MAP_" );static size_t mmap__scnprintf_flags (unsigned long flags,char *bf,size_t size,bool show_prefix) {return strarray__scnprintf_flags (&strarray__mmap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, +struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -40,7 +40,7 @@ static size_t syscall_arg__scnprintf_mmap_flags(char *bf, size_t size,return mmap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_FLAGS syscall_arg__scnprintf_mmap_flags +#include "trace/beauty/generated/mremap_flags_array.c" static DEFINE_STRARRAY(mremap_flags, "MREMAP_");
@@ -50,7 +50,7 @@ static size_t mremap__scnprintf_flags(unsigned long flags, char *bf, size_t sizereturn strarray__scnprintf_flags (&strarray__mremap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -60,7 +60,7 @@ static size_t syscall_arg__scnprintf_mremap_flags(char *bf, size_t size, structreturn mremap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MREMAP_FLAGS syscall_arg__scnprintf_mremap_flags +static size_t madvise__scnprintf_behavior (int behavior,char *bf,size_t size) {
@@ -73,10 +73,10 @@ static size_t madvise__scnprintf_behavior(int behavior, char *bf, size_t size)return scnprintf (bf, size,"%#" , behavior); } -static size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, +struct syscall_arg *arg) {return madvise__scnprintf_behavior (arg->val, bf, size); } -#define SCA_MADV_BHV syscall_arg__scnprintf_madvise_behavior +
tools/perf/trace/beauty/mode_t.c +4 -2
@@ -20,7 +20,9 @@#define S_IXUGO (S_IXUSR|S_IXGRP|S_IXOTH)#endif -static size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="S_" ;
@@ -68,4 +70,4 @@ static size_t syscall_arg__scnprintf_mode_t(char *bf, size_t size, struct syscalreturn printed; } -#define SCA_MODE_T syscall_arg__scnprintf_mode_t +
tools/perf/trace/beauty/msg_flags.c +5 -3
@@ -27,8 +27,10 @@# define MSG_CMSG_CLOEXEC 0x40000000#endif -static size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="MSG_" ;
@@ -73,4 +75,4 @@ static size_t syscall_arg__scnprintf_msg_flags(char *bf, size_t size,return printed; } -#define SCA_MSG_FLAGS syscall_arg__scnprintf_msg_flags +
tools/perf/trace/beauty/open_flags.c +1 -0
@@ -2,6 +2,7 @@#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> +#include "trace/beauty/beauty.h" #ifndef O_DIRECT #define O_DIRECT 00040000
tools/perf/trace/beauty/perf_event_open.c +12 -10
@@ -1,4 +1,7 @@// SPDX-License-Identifier: LGPL-2.1 +#include <linux/perf_event.h> +#include "util/evsel_fprintf.h" +#ifndef PERF_FLAG_FD_NO_GROUP # define PERF_FLAG_FD_NO_GROUP (1UL << 0)#endif
@@ -15,8 +18,10 @@# define PERF_FLAG_FD_CLOEXEC (1UL << 3)/* O_CLOEXEC */ #endif -static size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="PERF_" ;
@@ -43,7 +48,7 @@ static size_t syscall_arg__scnprintf_perf_flags(char *bf, size_t size,return printed; } -#define SCA_PERF_FLAGS syscall_arg__scnprintf_perf_flags +struct attr_fprintf_args {size_t size, printed;
@@ -76,10 +81,11 @@ static size_t perf_event_attr___scnprintf(struct perf_event_attr *attr, char *bfstatic size_t syscall_arg__scnprintf_augmented_perf_event_attr (struct syscall_arg *arg,char *bf,size_t size) { -return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, arg->trace->show_zeros); +return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, +trace__show_zeros (arg->trace)); } -static size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) {if (arg->augmented.args)return syscall_arg__scnprintf_augmented_perf_event_attr (arg, bf, size);
@@ -87,8 +93,4 @@ static size_t syscall_arg__scnprintf_perf_event_attr(char *bf, size_t size, strureturn scnprintf (bf, size,"%#lx" , arg->val); } -#define SCA_PERF_ATTR syscall_arg__scnprintf_perf_event_attr -// 'argname' is just documentational at this point, to remove the previous comment with that info -#define SCA_PERF_ATTR_FROM_USER (argname) \ - { .scnprintf = SCA_PERF_ATTR, \ - .from_user = true, } +
tools/perf/trace/beauty/pid.c +4 -1
@@ -1,11 +1,14 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "util/machine.h" +#include "util/thread.h" size_t syscall_arg__scnprintf_pid (char *bf,size_t size,struct syscall_arg *arg) {int pid = arg->val;struct trace *trace = arg->trace;size_t printed =scnprintf (bf, size,"%d" , pid); -struct thread *thread =machine__findnew_thread (trace->host, pid, pid); +struct thread *thread =machine__findnew_thread (trace__host (trace), pid, pid);if (thread !=NULL ) {if (!thread__comm_set (thread))
tools/perf/trace/beauty/sched_policy.c +5 -3
@@ -14,8 +14,10 @@#define SCHED_RESET_ON_FORK 0x40000000#endif -static size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SCHED_" ;
@@ -47,4 +49,4 @@ static size_t syscall_arg__scnprintf_sched_policy(char *bf, size_t size,return printed; } -#define SCA_SCHED_POLICY syscall_arg__scnprintf_sched_policy +
tools/perf/trace/beauty/seccomp.c +7 -5
@@ -6,7 +6,9 @@#define SECCOMP_SET_MODE_FILTER 1#endif -static size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_SET_MODE_" ;
@@ -24,14 +26,14 @@ static size_t syscall_arg__scnprintf_seccomp_op(char *bf, size_t size, struct syreturn printed; } -#define SCA_SECCOMP_OP syscall_arg__scnprintf_seccomp_op +#ifndef SECCOMP_FILTER_FLAG_TSYNC #define SECCOMP_FILTER_FLAG_TSYNC 1#endif -static size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_FILTER_FLAG_" ;
@@ -52,4 +54,4 @@ static size_t syscall_arg__scnprintf_seccomp_flags(char *bf, size_t size,return printed; } -#define SCA_SECCOMP_FLAGS syscall_arg__scnprintf_seccomp_flags +
tools/perf/trace/beauty/signum.c +4 -2
@@ -1,7 +1,9 @@// SPDX-License-Identifier: LGPL-2.1 #include <signal.h> -static size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SIG" ;
@@ -54,4 +56,4 @@ static size_t syscall_arg__scnprintf_signum(char *bf, size_t size, struct syscalreturn scnprintf (bf, size,"%#x" , sig); } -#define SCA_SIGNUM syscall_arg__scnprintf_signum +
tools/perf/trace/beauty/socket_type.c +4 -2
@@ -18,7 +18,9 @@#define SOCK_TYPE_MASK 0xf#endif -static size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SOCK_" ;
@@ -60,4 +62,4 @@ static size_t syscall_arg__scnprintf_socket_type(char *bf, size_t size, struct sreturn printed; } -#define SCA_SK_TYPE syscall_arg__scnprintf_socket_type +
tools/perf/trace/beauty/syscalltbl.c renamed from tools/perf/util/syscalltbl.c +0 -0
tools/perf/trace/beauty/syscalltbl.h renamed from tools/perf/util/syscalltbl.h +0 -0
tools/perf/trace/beauty/tracepoints/Build +22 -0
@@ -1,2 +1,24 @@ perf-y += x86_irq_vectors.o perf-y += x86_msr.o + +# --- Local Encapsulated Tracepoint Rules --- +beauty_outdir := $(OUTPUT)trace/beauty/generated +beauty_arch_asm_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/asm/ +x86_arch_asm_dir := $(srctree)/tools/arch/x86/include/asm/ + +x86_arch_irq_vectors_array := $(beauty_outdir)/x86_arch_irq_vectors_array.c +x86_arch_irq_vectors_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_irq_vectors.sh + +$(x86_arch_irq_vectors_array): $(beauty_arch_asm_dir)/irq_vectors.h $(x86_arch_irq_vectors_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_irq_vectors_tbl)' $(beauty_arch_asm_dir) > $@ + +x86_arch_MSRs_array := $(beauty_outdir)/x86_arch_MSRs_array.c +x86_arch_MSRs_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_msr.sh + +$(x86_arch_MSRs_array): $(x86_arch_asm_dir)/msr-index.h $(x86_arch_MSRs_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_MSRs_tbl)' $(x86_arch_asm_dir) > $@ + +$(OUTPUT)trace/beauty/tracepoints/x86_irq_vectors.o: $(x86_arch_irq_vectors_array) +$(OUTPUT)trace/beauty/tracepoints/x86_msr.o: $(x86_arch_MSRs_array)
tools/perf/trace/beauty/waitid_options.c +5 -3
@@ -2,8 +2,10 @@#include <sys/types.h> #include <sys/wait.h> -static size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="W" ;
@@ -26,4 +28,4 @@ static size_t syscall_arg__scnprintf_waitid_options(char *bf, size_t size,return printed; } -#define SCA_WAITID_OPTIONS syscall_arg__scnprintf_waitid_options +
tools/perf/util/Build +3 -1
@@ -75,7 +75,7 @@ perf-util-y += sample.o perf-util-y += sample-raw.o perf-util-y += s390-sample-raw.o perf-util-y += amd-sample-raw.o -perf-util-$(CONFIG_TRACE) += syscalltbl.o + perf-util-y += ordered-events.o perf-util-y += namespaces.o perf-util-y += comm.o
@@ -441,3 +441,5 @@ $(OUTPUT)%.pylint_log: % $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false) perf-util-y += $(PYLINT_TEST_LOGS) + +
tools/perf/util/bpf-trace-summary.c +1 -1
@@ -6,7 +6,7 @@#include <stdlib.h> #include "dwarf-regs.h" /* for EM_HOST */ -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "util/cgroup.h" #include "util/hashmap.h" #include "util/trace.h"
tools/perf/util/env.c +1 -3
@@ -635,9 +635,7 @@ const char *perf_env__arch(struct perf_env *env)return normalize_arch (arch_name); } -#if defined(HAVE_LIBTRACEEVENT ) -#include "trace/beauty/arch_errno_names.c" -#endif +const char *perf_env__arch_strerrno (struct perf_env *env__maybe_unused ,int err__maybe_unused ) {
tools/perf/util/env.h +1 -0
@@ -189,6 +189,7 @@ void cpu_cache_level__free(struct cpu_cache_level *cache);const char *perf_env__arch (struct perf_env *env);const char *perf_env__arch_strerrno (struct perf_env *env,int err); +arch_syscalls__strerrno_t *arch_syscalls__strerrno_function (const char *arch);const char *perf_env__cpuid (struct perf_env *env);const char *perf_env__raw_arch (struct perf_env *env);int perf_env__nr_cpus_avail (struct perf_env *env);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 02/14] perf trace beauty: Make beauty generated C code standalone .o files
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The commit message incorrectly states that it uses a conditional kbuild hook for `CONFIG_TRACE`…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The commit message incorrectly states that it uses a conditional kbuild hook for CONFIG_TRACE, but the patch adds it unconditionally. --
commit b8d8fe14745ce243d9ff4d75f580a2b157dc3294
Author: Ian Rogers <[email protected]>
perf trace beauty: Make beauty generated C code standalone .o files
This commit refactors the perf trace beauty framework to compile generated code as independent standalone objects instead of inline inclusions. It also moves generation rules locally into the target Build files to improve parallel build startup.
Quoted 7 lines
2. Create
fsconfig.cand flatten embedded beauty .c files to compile as independent standalone objects via trace/beauty/Build, exporting their formatting functions viabeauty.handenv.h. Switch arch_errno_names.o and syscalltbl.o assignments directly to perf-util-y and add a top-level recursive kbuild hook (perf-util-$(CONFIG_TRACE) += trace/beauty/) to compile them into libperf-util.a, resolving remote linkage for standalone python extensions.
Does the description regarding perf-util-$(CONFIG_TRACE) match the code? The commit message mentions a conditional kbuild hook, but the patch below adds it unconditionally.
tools/perf/Build ifeq ($(CONFIG_LIBTRACEEVENT),y) · line 39
Quoted 10 lines
tools/perf/Build +2
-0@@ -34,6 +34,8 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-$(CONFIG_TRACE) += trace/beauty/ endif +perf-util-y += trace/beauty/ + perf-$(CONFIG_LIBELF) += builtin-probe.o
Since this is an unconditional assignment to perf-util-y, could the commit message be clarified here, or was the assignment intended to be conditional on CONFIG_TRACE?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerPATCH 3/14 perf build: Decouple pmu-events from prepare umbrella target
In reply to Ian RogersCurrently, the $(LIBPMU_EVENTS_IN) sub-make depends on the massive "prepare" umbrella target. Because "prepare" depends on external libraries (libapi, libperf, etc.) as well as dozens of generated headers, make completely serializes the launch of the pmu-events sub-make behind some of those unrelated prerequisites.
Since pmu-events is a large compilation unit, unblock its startup by binding it directly to only $(LIBPERF) instead of prepare. This allows background python generation scripts to overlap simultaneously with the rest of the build.
Testing a parallel build (make -j28 clean all) shows improvements:
Before:
real 0m27.642s
user 2m32.356s
sys 0m26.683sAfter:
real 0m22.254s
user 2m32.810s
sys 0m24.646sThis reclaims over 5 full seconds of build latency (~19.5% overall reduction) by elevating average CPU concurrency from ~5.5 active cores up to ~8 active cores.
--- tools/perf/Makefile.perf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/Makefile.perf +1 -1
@@ -550,7 +550,7 @@ build := -f $(srctree)/tools/build/Makefile.build dir=. obj $(PERF_IN): prepare FORCE $(Q)$(MAKE) $(build)=perf -$(LIBPMU_EVENTS_IN): FORCE prepare +$(LIBPMU_EVENTS_IN): FORCE $(LIBPERF) $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH 4/14 perf build: Remove empty archheaders target
In reply to Ian RogersRemove empty target that doesn't do anything.
--- tools/perf/Makefile.perf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/Makefile.perf +2 -2
@@ -627,7 +627,7 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) -prepare: $(OUTPUT)PERF-VERSION-FILE archheaders \ +prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ $(LIBAPI) \ $(LIBPERF) \
@@ -1070,7 +1070,7 @@ FORCE: .PHONY: all install clean config-clean strip install-gtk .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell .PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare -.PHONY: archheaders python_perf_target +.PHONY: python_perf_target endif # force_fixdep
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH 5/14 perf build: Move BPF skeleton generation out of Makefile.perf
In reply to Ian RogersCurrently, the top-level Makefile.perf defines a massive global bpf-skel umbrella target that pre-compiles all 12+ BPF skeletons (%.skel.h) upfront before launching sub-makes. This forces unrelated sub-makes to serialize behind bpftool and clang BPF target evaluations, causing parallel build bottlenecks.
Furthermore, bench_uprobe.bpf.c lived inside util/bpf_skel/, breaking conceptual directory encapsulation since it is consumed purely by bench/uprobe.c.
Refactor the BPF skeletons to better achieve directory isolation:
1. Move tools/perf/util/bpf_skel/bench_uprobe.bpf.c directly into tools/perf/bench/bpf_skel/.
2. Extract the skeleton generation infrastructure out of Makefile.perf into a shared inclusion file tools/perf/bpf_skel.mak.
3. Include bpf_skel.mak locally inside tools/perf/util/Build and tools/perf/bench/Build and bind precise local prerequisites.
4. Safely synchronize the shared bpftool bootstrap and vmlinux.h targets via the conditional prepare: umbrella to avoid parallel sub-make races, while evaluating the actual skeletons completely locally on demand. A later patch will move these targets into bpf_skel.mak.
5. Export CLANG from the global Makefile to ensure accurate tool propagation.
6. Clean up Makefile.perf by stripping the global bpf-skel umbrella target and its SKELETONS list.
While removing code from Makefile.perf generally helps build performance, the impact here is minimal. The main motivation for the change is to better encapsulate things in the build and simplify Makefile.perf that has around 50 lines removed.
---
tools/perf/Makefile.perf | 59 ++-----------------
tools/perf/bench/Build | 6 ++
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 55 +++++++++++++++++
tools/perf/util/Build | 15 ++++-
6 files changed, 80 insertions(+), 57 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.maktools/perf/Makefile.perf +4 -55
@@ -274,7 +274,7 @@ ifeq ($(PYLINT),1) PYLINT := $(shell which pylint 2> /dev/null) endif -export srctree OUTPUT RM CC CXX RUSTC LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK +export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT include $(srctree)/tools/build/Makefile.include
@@ -632,8 +632,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \ - $(LIBSYMBOL) \ - bpf-skel + $(LIBSYMBOL) ifdef LIBBPF_STATIC prepare: $(LIBBPF)
@@ -914,44 +913,13 @@ python-clean: SKEL_OUT := $(abspath $(OUTPUT)util/bpf_skel) SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) -SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h -SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h -SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h -SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h -SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h $(SKEL_OUT)/sample_filter.skel.h -SKELETONS += $(SKEL_OUT)/kwork_top.skel.h $(SKEL_OUT)/syscall_summary.skel.h -SKELETONS += $(SKEL_OUT)/bench_uprobe.skel.h -SKELETONS += $(SKEL_OUT)/augmented_raw_syscalls.skel.h $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT) $(LIBSYMBOL_OUTPUT): $(Q)$(MKDIR) -p $@ ifeq ($(CONFIG_PERF_BPF_SKEL),y) +prepare: $(BPFTOOL) $(SKEL_OUT)/vmlinux.h BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool -# Get Clang's default includes on this system, as opposed to those seen by -# '--target=bpf'. This fixes "missing" files on some architectures/distros, -# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc. -# -# Use '-idirafter': Don't interfere with include mechanics except where the -# build would have failed anyways. -define get_sys_includes -$(shell $(1) $(2) -v -E - </dev/null 2>&1 \ - | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \ -$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}') -endef - -ifneq ($(CROSS_COMPILE),) -CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%)) -endif - -CLANG_OPTIONS = -Wall -CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH)) -BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES) -TOOLS_UAPI_INCLUDE := -I$(srctree)/tools/include/uapi - -ifneq ($(WERROR),0) - CLANG_OPTIONS += -Werror -endif $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
@@ -993,29 +961,10 @@ else $(Q)cp "$(VMLINUX_H)" $@ endif -$(SKEL_TMP_OUT)/%.bpf.o: $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h | $(SKEL_TMP_OUT) -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h - $(QUIET_CLANG)$(CLANG) -g -O2 -fno-stack-protector --target=bpf \ - $(CLANG_OPTIONS) $(EXTRA_BPF_FLAGS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ - -include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \ - -fms-extensions -Wno-microsoft-anon-tag \ - -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@ - -$(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o | $(BPFTOOL) - $(QUIET_GENSKEL)$(BPFTOOL) gen skeleton $< > $@ - -bpf-skel: $(SKELETONS) - -.PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o - -else # CONFIG_PERF_BPF_SKEL - -bpf-skel: - endif # CONFIG_PERF_BPF_SKEL bpf-skel-clean: - $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS) $(SKEL_OUT)/vmlinux.h + $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKEL_OUT)/*.skel.h $(SKEL_OUT)/vmlinux.h $(OUTPUT)bench/bpf_skel/*.skel.h pmu-events-clean: ifeq ($(OUTPUT),)
tools/perf/bench/Build +6 -0
@@ -24,3 +24,9 @@ perf-bench-$(CONFIG_X86_64) += mem-memcpy-x86-64-asm.o perf-bench-$(CONFIG_X86_64) += mem-memset-x86-64-asm.o perf-bench-$(CONFIG_NUMA) += numa.o + +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)bench/uprobe.o: $(SKEL_OUT)/bench_uprobe.skel.h +endif
tools/perf/bench/bpf_skel/bench_uprobe.bpf.c renamed from tools/perf/util/bpf_skel/bench_uprobe.bpf.c +0 -0
tools/perf/bench/uprobe.c +1 -1
@@ -44,7 +44,7 @@ static const char * const bench_uprobe_usage[] = { };#ifdef HAVE_BPF_SKEL -#include "bpf_skel/bench_uprobe.skel.h" +#include "bench/bpf_skel/bench_uprobe.skel.h" #define bench_uprobe__attach_uprobe(prog) \ skel->links.prog = bpf_program__attach_uprobe_opts(/*prog=*/skel->progs.prog, \
tools/perf/bpf_skel.mak new file +55 -0
55 lines not on this page · raw message
tools/perf/util/Build +14 -1
@@ -442,4 +442,17 @@ $(OUTPUT)%.pylint_log: % perf-util-y += $(PYLINT_TEST_LOGS) - +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)util/bpf_ftrace.o: $(SKEL_OUT)/func_latency.skel.h +$(OUTPUT)util/bpf-filter.o: $(SKEL_OUT)/sample_filter.skel.h +$(OUTPUT)util/bpf_kwork_top.o: $(SKEL_OUT)/kwork_top.skel.h +$(OUTPUT)util/bpf_off_cpu.o: $(SKEL_OUT)/off_cpu.skel.h +$(OUTPUT)util/bpf-trace-summary.o: $(SKEL_OUT)/syscall_summary.skel.h +$(OUTPUT)util/bpf_counter_cgroup.o: $(SKEL_OUT)/bperf_cgroup.skel.h +$(OUTPUT)util/bpf_trace_augment.o: $(SKEL_OUT)/augmented_raw_syscalls.skel.h +$(OUTPUT)util/bpf_counter.o: $(SKEL_OUT)/bpf_prog_profiler.skel.h $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h +$(OUTPUT)util/bpf_lock_contention.o: $(SKEL_OUT)/lock_contention.skel.h +$(OUTPUT)util/bpf_kwork.o: $(SKEL_OUT)/kwork_trace.skel.h +endif
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 05/14] perf build: Move BPF skeleton generation out of Makefile.perf
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Low] The `prepare` target evaluates the `$(BPFTOOL)` variable before it is defined, causing it to…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Low] The prepare target evaluates the $(BPFTOOL) variable before it is defined, causing it to silently expand to an empty string.
- [Low] The bpf-skel-clean target fails to clean the intermediate .tmp directory for bench skeletons, causing a build artifact leak after make clean. --
commit ba0748386f00351749ad3ff0cb9d88e512a55543
Author: Ian Rogers <[email protected]>
perf build: Move BPF skeleton generation out of Makefile.perf
This commit refactors BPF skeleton generation by extracting the infrastructure out of Makefile.perf into a shared bpf_skel.mak file. It isolates the directory structure and prevents unrelated sub-makes from serializing behind bpftool and clang BPF target evaluations.
Quoted 4 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
Quoted 4 lines
tools/perf/Makefile.perf +0
-0@@ -914,44 +913,13 @@ python-clean: SKEL_OUT := $(abspath $(OUTPUT)util/bpf_skel) SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp)
[ ... ]
Quoted 3 lines
ifeq ($(CONFIG_PERF_BPF_SKEL),y)
+prepare: $(BPFTOOL) $(SKEL_OUT)/vmlinux.h
BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool
Does the prepare target evaluate the $(BPFTOOL) variable before it is defined?
In GNU Make, variables in target prerequisites are expanded immediately. Because the prepare target precedes the BPFTOOL assignment, $(BPFTOOL) silently expands to an empty string. The build seems to succeed only circumstantially because vmlinux.h has a transitive dependency on bpftool.
I note this regression is fixed later in the series by commit e42b6c2f3098a ("perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak").
[ ... ]
tools/perf/Makefile.perf else · line 967
Quoted 3 lines
bpf-skel-clean:
tools/perf/Makefile.perf +1
-1- $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS) $(SKEL_OUT)/vmlinux.h + $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKEL_OUT)/*.skel.h $(SKEL_OUT)/vmlinux.h $(OUTPUT)bench/bpf_skel/*.skel.h
Is the intermediate .tmp directory for bench skeletons left behind during cleanup?
While the target explicitly removes the generated headers from bench/bpf_skel/, it misses the temporary object directory $(OUTPUT)bench/bpf_skel/.tmp where intermediate .bpf.o files are generated, causing a build artifact leak after running make clean.
This regression is also fixed later in the series by commit e42b6c2f3098a ("perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak").
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Ian Rogers
reviewerPATCH 6/14 perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak
In reply to Ian RogersCurrently, bpftool and vmlinux.h are prerequisites of the top-level prepare target in Makefile.perf. This unnecessarily blocks the massive parallel C compilation of libraries (perf-util, perf-ui, pmu-events) during the initial startup phase.
Move all bpftool and vmlinux.h generation rules down into tools/perf/bpf_skel.mak to encapsulate BPF tooling completely within the skeleton framework. Remove them entirely from prepare to unblock immediate parallel build execution.
To prevent parallel sub-makes (perf-util and perf-bench) from racing to build shared prerequisites concurrently, while maintaining strict directory encapsulation without top-level inclusions, serialize bench after the util static archive finishes using an order-only prerequisite:
$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL)
--- tools/perf/Makefile.config | 2 +- tools/perf/Makefile.perf | 61 +++++--------------------------------- tools/perf/bpf_skel.mak | 47 +++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 55 deletions(-)
tools/perf/Makefile.config +1 -1
@@ -736,7 +736,7 @@ ifeq ($(BUILD_BPF_SKEL),1) endif ifndef GEN_VMLINUX_H - VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h + VMLINUX_H_FILE=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h endif dwarf-post-unwind := 1
tools/perf/Makefile.perf +7 -54
88 lines not on this page · raw message
tools/perf/bpf_skel.mak +47 -0
57 lines not on this page · raw message
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 06/14] perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Incomplete variable renaming breaks `make clean` for BPF skeletons on systems without BTF. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Incomplete variable renaming breaks make clean for BPF skeletons on systems without BTF. --
commit e42b6c2f3098a5301a43fefe8d2fed01aa6814b7
Author: Ian Rogers <[email protected]>
perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak
Currently, bpftool and vmlinux.h are prerequisites of the top-level prepare target in Makefile.perf. This unnecessarily blocks the massive parallel C compilation of libraries (perf-util, perf-ui, pmu-events) during the initial startup phase.
Quoted 4 lines
tools/perf/bpf_skel.mak +0
-0
[ ... ]
tools/perf/bpf_skel.mak ifneq ($(WERROR),0) · line 70
Quoted 5 lines
tools/perf/bpf_skel.mak +5
-0+ifeq ($(VMLINUX_H_FILE),) + ifeq ($(VMLINUX_BTF),) + $(error Missing bpftool input for generating vmlinux.h) + endif +endif
Will this unconditional check break make clean on systems without a BTF-enabled vmlinux?
If a user runs make clean CONFIG_PERF_BPF_SKEL=y on such a system, VMLINUX_H_FILE is left empty because the fallback assignment in Makefile.perf's NON_CONFIG_TARGETS block still sets the old VMLINUX_H variable instead of VMLINUX_H_FILE.
Since bpf-skel-clean invokes a sub-make to bpf_skel.mak, and CONFIG_PERF_BPF_SKEL=y is passed, the skeleton block is processed. Because VMLINUX_H_FILE is empty and no system BTF is found, the error check above is triggered, which aborts the clean process.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Ian Rogers
reviewerPATCH 7/14 perf build: Move static libbpf dependency out of prepare step
In reply to Ian RogersThe prepare step is a large serialization point before parallel sub-makes build the perf tool. The libbpf headers are used in the bench and util libraries, as well as individual perf builtin commands. Move the libbpf dependency out of the prepare step and into the dependencies for those targets to avoid it being a source of serialization in the prepare step.
--- tools/perf/Makefile.perf | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
tools/perf/Makefile.perf +3 -7
@@ -547,7 +547,7 @@ export NO_JEVENTS build := -f $(srctree)/tools/build/Makefile.build dir=. obj -$(PERF_IN): prepare FORCE +$(PERF_IN): prepare $(LIBBPF) FORCE $(Q)$(MAKE) $(build)=perf $(LIBPMU_EVENTS_IN): FORCE $(LIBPERF)
@@ -558,7 +558,7 @@ $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN) # The $(LIBPERF_UTIL) dependency is to ensure bpftool and vmlinux.h # aren't racily built for bench/bpf_skel/bench_uprobe.bpf.c -$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL) +$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) | $(LIBPERF_UTIL) $(Q)$(MAKE) $(build)=perf-bench $(LIBPERF_BENCH): $(LIBPERF_BENCH_IN)
@@ -576,7 +576,7 @@ $(LIBPERF_UI_IN): FORCE prepare $(LIBPERF_UI): $(LIBPERF_UI_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_UTIL_IN): FORCE prepare +$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) $(Q)$(MAKE) $(build)=perf-util $(LIBPERF_UTIL): $(LIBPERF_UTIL_IN)
@@ -636,10 +636,6 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBSUBCMD) \ $(LIBSYMBOL) -ifdef LIBBPF_STATIC -prepare: $(LIBBPF) -endif - $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 07/14] perf build: Move static libbpf dependency out of prepare step
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Removing `$(LIBBPF)` from the `prepare` target dependencies creates a parallel build race condition…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Removing $(LIBBPF) from the prepare target dependencies creates a parallel build race condition for $(LIBPERF_TEST_IN), leading to sporadic compilation failures in tests/sigtrap.c. --
commit f64d4116318da61817b68460818308db46b21447
Author: Ian Rogers <[email protected]>
perf build: Move static libbpf dependency out of prepare step
This commit moves the static libbpf dependency out of the prepare step to reduce serialization during parallel builds. The libbpf dependency is explicitly added to the perf, bench, and util targets instead.
Quoted 3 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
tools/perf/Makefile.perf prepare: $(OUTPUT)PERF-VERSION-FILE \ · line 642
Quoted 8 lines
tools/perf/Makefile.perf +0
-4@@ -636,10 +636,6 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBSUBCMD) \ $(LIBSYMBOL) -ifdef LIBBPF_STATIC -prepare: $(LIBBPF) -endif -
Does removing this dependency from prepare create a parallel build race condition for $(LIBPERF_TEST_IN)?
It appears that $(LIBPERF_TEST_IN) is missing the $(LIBBPF) dependency that was added to $(PERF_IN), $(LIBPERF_BENCH_IN), and $(LIBPERF_UTIL_IN).
During a parallel build, $(LIBPERF_TEST_IN) might start compiling test files before $(LIBBPF) finishes installing headers to the include directory.
This could lead to sporadic compilation failures in files like tools/perf/tests/sigtrap.c which includes <bpf/btf.h> when HAVE_BPF_SKEL is enabled.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
Ian Rogers
reviewerPATCH 8/14 perf build: Pre-generate BPF skeletons during umbrella prepare phase
In reply to Ian RogersCurrently, BPF skeleton generation rules (bpf_skel.mak) are evaluated as part of util/Build. However, because LIBPERF_UTIL_IN explicitly depends on the top-level static libbpf archive, Make completely blocked the execution of bpftool bootstrap and skeleton generation until libbpf finished compiling midway through the build.
Since bpftool bootstrap compiles its own independent copy of libbpf.a, it does not depend on the top-level libbpf target.
Decouple early skeleton tooling generation by attaching bpf-skel-prepare to the umbrella prepare target, exporting CONFIG_PERF_BPF_SKEL to ensure accurate feature propagation. This allows Make to compile bpftool and dump vmlinux.h in the background at build startup, eliminating the initial sub-make startup bottleneck before BPF object compilation while keeping 100% of tooling rules perfectly encapsulated in bpf_skel.mak. Provide an empty fallback target to ensure builds succeed when BPF skeletons are disabled.
--- tools/perf/Makefile.perf | 6 +++++- tools/perf/bpf_skel.mak | 8 ++++++++ 2 files changed, 13 insertions(+), 1 deletion(-)
tools/perf/Makefile.perf +5 -1
@@ -275,7 +275,7 @@ ifeq ($(PYLINT),1) endif export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF READELF -export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT +export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT CONFIG_PERF_BPF_SKEL include $(srctree)/tools/build/Makefile.include
@@ -629,8 +629,12 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) +bpf-skel-prepare: + $(Q)$(MAKE) -f bpf_skel.mak bpf-skel-prepare + prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ + bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \
tools/perf/bpf_skel.mak +8 -0
@@ -16,6 +16,10 @@ BPFTOOL := $(SKEL_TOOL_TMP_OUT)/bootstrap/bpftool VMLINUX_H := $(SKEL_TOOL_OUT)/vmlinux.h bpf_skel_deps := $(BPFTOOL) $(VMLINUX_H) +.PHONY: bpf-skel-prepare +bpf-skel-prepare: $(bpf_skel_deps) + @: + define get_sys_includes $(shell $(1) $(2) -v -E - </dev/null 2>&1 \ | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
@@ -94,6 +98,10 @@ $(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o $(BPFTOOL) .PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o +else # CONFIG_PERF_BPF_SKEL +.PHONY: bpf-skel-prepare +bpf-skel-prepare: + @: endif # CONFIG_PERF_BPF_SKEL clean:
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH 9/14 perf build: Move libsymbol dependency out of prepare step
In reply to Ian RogersThe prepare step is a large serialization point before parallel sub-makes build the perf tool. The libsymbol headers are used in the bench and util libraries. Move the libsymbol dependency out of the prepare step and into the dependencies for those targets to avoid it being a source of serialization in the prepare step.
--- tools/perf/Makefile.perf | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
tools/perf/Makefile.perf +3 -4
@@ -558,7 +558,7 @@ $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN) # The $(LIBPERF_UTIL) dependency is to ensure bpftool and vmlinux.h # aren't racily built for bench/bpf_skel/bench_uprobe.bpf.c -$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) | $(LIBPERF_UTIL) +$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) $(LIBSYMBOL) | $(LIBPERF_UTIL) $(Q)$(MAKE) $(build)=perf-bench $(LIBPERF_BENCH): $(LIBPERF_BENCH_IN)
@@ -576,7 +576,7 @@ $(LIBPERF_UI_IN): FORCE prepare $(LIBPERF_UI): $(LIBPERF_UI_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) +$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) $(LIBSYMBOL) $(Q)$(MAKE) $(build)=perf-util $(LIBPERF_UTIL): $(LIBPERF_UTIL_IN)
@@ -637,8 +637,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ - $(LIBSUBCMD) \ - $(LIBSYMBOL) + $(LIBSUBCMD) $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 09/14] perf build: Move libsymbol dependency out of prepare step
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Removing `$(LIBSYMBOL)` from the `prepare` umbrella target breaks the `make check` and partial…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Removing $(LIBSYMBOL) from the prepare umbrella target breaks the make check and partial object compilation rules (e.g., make util/machine.o) on a clean repository. --
commit 807af289bc2dc2091029fa64ee06af014dcf927c
Author: Ian Rogers <[email protected]>
perf build: Move libsymbol dependency out of prepare step
The prepare step is a large serialization point before parallel sub-makes build the perf tool. This commit moves the libsymbol dependency out of the prepare step and into the dependencies for the bench and util libraries to avoid serialization in the prepare step.
Quoted 4 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
tools/perf/Makefile.perf prepare: $(OUTPUT)PERF-VERSION-FILE \ · line 643
Quoted 10 lines
tools/perf/Makefile.perf +1
-2@@ -637,8 +637,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ - $(LIBSUBCMD) \ - $(LIBSYMBOL) + $(LIBSUBCMD) $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Will removing $(LIBSYMBOL) from the prepare target break make check or partial object compilation (e.g., make util/machine.o) on a clean repository?
Files such as tools/perf/util/machine.c directly include <symbol/kallsyms.h>, which is dynamically copied into the include directory by the install_headers target triggered when $(LIBSYMBOL) is evaluated.
Because both make check and partial single-target builds explicitly depend on prepare as their synchronization point, and $(LIBSYMBOL) is no longer part of prepare, does this cause sparse or gcc to fail to resolve <symbol/kallsyms.h> during those builds?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Ian Rogers
reviewerPATCH 10/14 perf build: Remove redundant libbpf feature check for static builds
In reply to Ian RogersBy default, the perf tool compiles and statically links against its own internal copy of libbpf (tools/lib/bpf/libbpf.a), setting LIBBPF_STATIC=1.
Despite this static linkage, Makefile.config unconditionally executed $(call feature_check,libbpf), which forced a synchronous sub-make fork during AST parsing to detect dynamic system libbpf libraries.
As noted in the internal Makefile comments, this check was executed purely so that running make VF=1 would display the detection status of system libbpf. During standard builds without LIBBPF_DYNAMIC=1, the detection result was entirely ignored.
Wrap the libbpf feature check inside LIBBPF_DYNAMIC=1 so Make avoids the redundant sub-make fork overhead during standard static builds.
--- tools/perf/Makefile.config | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
tools/perf/Makefile.config +1 -3
@@ -587,10 +587,8 @@ ifndef NO_LIBELF ifndef NO_LIBBPF ifeq ($(feature-bpf), 1) - # detecting libbpf without LIBBPF_DYNAMIC, so make VF=1 shows libbpf detection status - $(call feature_check,libbpf) - ifdef LIBBPF_DYNAMIC + $(call feature_check,libbpf) ifeq ($(feature-libbpf), 1) EXTLIBS += -lbpf CFLAGS += -DHAVE_LIBBPF_SUPPORT
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH 11/14 tools build: Integrate libdebuginfod into test-all fast path
In reply to Ian RogersBundling libdebuginfod into test-all.c allows Make to avoid an individual feature check sub-make fork during AST parsing. Short-circuit feature_check in Makefile.feature to instantly skip sub-make evaluations when a feature was already pre-detected by test-all.
Trade-offs:
On some Linux distributions (e.g., Debian/Ubuntu), libdebuginfod-dev is packaged separately from libelf-dev and libdw-dev. If a user installs libdw-dev but omits libdebuginfod-dev, adding -ldebuginfod to BUILD_ALL causes test-all to fail, falling back to individual feature tests. However, on fully configured developer workstations, this successfully reclaims sequential startup latency.
--- tools/build/Makefile.feature | 6 ++++-- tools/build/feature/Makefile | 2 +- tools/build/feature/test-all.c | 5 +++++ 3 files changed, 10 insertions(+), 3 deletions(-)
tools/build/Makefile.feature +4 -2
@@ -8,7 +8,9 @@ endif feature_check = $(eval $(feature_check_code)) define feature_check_code - feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + ifeq ($(feature-$(1)),) + feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + endif endef feature_set = $(eval $(feature_set_code))
@@ -100,6 +102,7 @@ FEATURE_TESTS_BASIC := \ disassembler-four-args \ disassembler-init-styled \ file-handle \ + libdebuginfod \ libopenssl # FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list
@@ -125,7 +128,6 @@ FEATURE_TESTS_EXTRA := \ clang \ libbpf \ libpfm4 \ - libdebuginfod \ clang-bpf-co-re \ bpftool-skeletons
tools/build/feature/Makefile +1 -1
@@ -115,7 +115,7 @@ __BUILD = $(CC) $(CFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $( BUILD_BFD = $(BUILD) -DPACKAGE='"perf"' -lbfd -ldl BUILD_ALL = $(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -lslang \ $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -ldl -lz -llzma -lzstd \ - $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) + $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) -ldebuginfod __BUILDXX = $(CXX) $(CXXFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS) BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
tools/build/feature/test-all.c +5 -0
@@ -146,6 +146,10 @@# include "test-libopenssl.c" #undef main +#define main main_test_libdebuginfod +# include "test-libdebuginfod.c" +#undef main +int main (int argc,char *argv[]) {main_test_libpython ();
@@ -178,6 +182,7 @@ int main(int argc, char *argv[])main_test_libzstd ();main_test_libtraceevent ();main_test_libopenssl (); +main_test_libdebuginfod ();return 0 ; }
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 11/14] tools build: Integrate libdebuginfod into test-all fast path
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] Short-circuiting `feature_check` prevents necessary re-evaluation of features falsely assumed…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] Short-circuiting feature_check prevents necessary re-evaluation of features falsely assumed present by test-all, causing fatal build failures.
- [Low] Unconditionally adding -ldebuginfod to BUILD_ALL breaks the test-all fast path even when explicitly disabled via NO_LIBDEBUGINFOD=1. --
commit 53a01ad9ae3a2bcd156b498b52efecdd21be5e21
Author: Ian Rogers <[email protected]>
tools build: Integrate libdebuginfod into test-all fast path
This commit bundles libdebuginfod into test-all.c to avoid an individual feature check sub-make fork. It short-circuits feature_check in Makefile.feature to skip evaluations when a feature was pre-detected.
tools/build/Makefile.feature endif · line 14
Quoted 13 lines
tools/build/Makefile.feature +3
-1@@ -8,7 +8,9 @@ endif feature_check = $(eval $(feature_check_code)) define feature_check_code - feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + ifeq ($(feature-$(1)),) + feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + endif endef
Does this short-circuit prevent the necessary evaluation of features omitted from test-all?
When test-all.bin successfully compiles, Makefile.feature sets all features in FEATURE_TESTS to 1. If features like libbfd or libcpupower are intentionally omitted from test-all.c, they are falsely assumed present.
By skipping the evaluation when feature-$(1) is already set, the explicit correction is missed. Could this cause fatal linker errors on systems that lack these libraries?
[ ... ]
tools/build/feature/Makefile __BUILD = $(CC) $(CFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $( · line 120
Quoted 12 lines
tools/build/feature/Makefile +1
-1@@ -115,7 +115,7 @@ __BUILD = $(CC) $(CFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $( BUILD_BFD = $(BUILD) -DPACKAGE='"perf"' -lbfd -ldl BUILD_ALL = $(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -lslang \ $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -ldl -lz -llzma -lzstd \ - $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) + $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) -ldebuginfod __BUILDXX = $(CXX) $(CXXFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS)
Will adding -ldebuginfod unconditionally here break the test-all fast path even when NO_LIBDEBUGINFOD=1 is explicitly passed?
If a system lacks libdebuginfod-dev but the build is correctly configured to opt out, it appears BUILD_ALL will still attempt to link against -ldebuginfod and fail, forcing a slow fallback to individual feature tests.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Ian Rogers
reviewerPATCH 12/14 perf pmu-events: Split big_c_string storage into standalone compilation unit
In reply to Ian RogersCurrently, jevents.py emits both the massive 2.8 MB big_c_string literal and tens of thousands of compact_pmu_event struct arrays into a single pmu-events.c compilation unit. Compiling this giant file takes ~2.2 seconds on a single CPU core during Kbuild startup.
Refactor jevents.py to emit big_c_string into a dedicated pmu-events-string.c compilation unit. This allows Kbuild to compile pmu-events.o and pmu-events-string.o simultaneously in parallel across two separate CPU cores, preserving 100% string deduplication and zero dynamic ELF relocations while cutting C compilation latency in half.
--- tools/perf/Makefile.perf | 4 ++-- tools/perf/pmu-events/Build | 15 ++++++++++++++- tools/perf/pmu-events/jevents.py | 20 ++++++++++++++++---- 3 files changed, 32 insertions(+), 7 deletions(-)
tools/perf/Makefile.perf +2 -2
@@ -921,7 +921,7 @@ bpf-skel-clean: pmu-events-clean: ifeq ($(OUTPUT),) $(call QUIET_CLEAN, pmu-events) $(RM) \ - pmu-events/pmu-events.c \ + pmu-events/pmu-events*.c \ pmu-events/metric_test.log \ pmu-events/test-empty-pmu-events.c \ pmu-events/empty-pmu-events.log
@@ -929,7 +929,7 @@ ifeq ($(OUTPUT),) -name 'extra-metricgroups.json' -delete else # When an OUTPUT directory is present, clean up the copied pmu-events/arch directory. $(call QUIET_CLEAN, pmu-events) $(RM) -r $(OUTPUT)pmu-events/arch \ - $(OUTPUT)pmu-events/pmu-events.c \ + $(OUTPUT)pmu-events/pmu-events*.c \ $(OUTPUT)pmu-events/metric_test.log \ $(OUTPUT)pmu-events/test-empty-pmu-events.c \ $(OUTPUT)pmu-events/empty-pmu-events.log
tools/perf/pmu-events/Build +14 -1
@@ -1,7 +1,13 @@ EMPTY_PMU_EVENTS_C = pmu-events/empty-pmu-events.c # pmu-events.c will be generated by jevents.py or copied from EMPTY_PMU_EVENTS_C PMU_EVENTS_C = $(OUTPUT)pmu-events/pmu-events.c +PMU_EVENTS_STRING_C = $(OUTPUT)pmu-events/pmu-events-string.c + +ifeq ($(NO_JEVENTS),1) pmu-events-y += pmu-events.o +else +pmu-events-y += pmu-events.o pmu-events-string.o +endif # pmu-events.c file is generated in the OUTPUT directory so it needs a # separate rule to depend on it properly
@@ -9,6 +15,10 @@ $(OUTPUT)pmu-events/pmu-events.o: $(PMU_EVENTS_C) $(call rule_mkdir) $(call if_changed_dep,cc_o_c) +$(OUTPUT)pmu-events/pmu-events-string.o: $(PMU_EVENTS_STRING_C) + $(call rule_mkdir) + $(call if_changed_dep,cc_o_c) + # Message for $(call echo-cmd,cp), possibly remove the src file from # the destination to save space in the build log. quiet_cmd_cp = COPY $(patsubst %$<,%,$@) <- $<
@@ -224,6 +234,9 @@ endif # and inputs are dependencies. $(PMU_EVENTS_C): $(JEVENTS_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ $(PMU_EVENTS_STRING_C) + +$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C) + @: endif # ifeq ($(NO_JEVENTS),1)
tools/perf/pmu-events/jevents.py +16 -4
@@ -1422,6 +1422,8 @@ such as "arm/cortex-a34".''', ) ap.add_argument( 'output_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=sys.stdout) + ap.add_argument( + 'output_string_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=None) _args = ap.parse_args() _args.output_file.write(f"""
@@ -1463,10 +1465,20 @@ struct pmu_table_entry { ftw(arch_path, [], preprocess_one_file) _bcs.compute() - _args.output_file.write('static const char *const big_c_string =\n') - for s in _bcs.big_string: - _args.output_file.write(s) - _args.output_file.write(';\n\n') + if not _args.output_string_file: + _args.output_file.write('static const char *const big_c_string =\n') + for s in _bcs.big_string: + _args.output_file.write(s) + _args.output_file.write(';\n\n') + else: + _args.output_string_file.write('/* SPDX-License-Identifier: GPL-2.0 */\n') + _args.output_string_file.write('/* Autogenerated by jevents.py */\n') + _args.output_string_file.write('const char big_c_string[] =\n') + for s in _bcs.big_string: + _args.output_string_file.write(s) + _args.output_string_file.write(';\n') + _args.output_string_file.close() + _args.output_file.write('extern const char big_c_string[];\n\n') for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], process_one_file)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 12/14] perf pmu-events: Split big_c_string storage into standalone compilation unit
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Low] Missing header declaration for global variable `big_c_string`. - [Low] Flawed multiple-output Make…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Low] Missing header declaration for global variable big_c_string.
- [Low] Flawed multiple-output Make rule causes build failures on configuration changes. --
commit 5d2e7b3d9410693c44dfde4428eb25c3e52c21d2
Author: Ian Rogers <[email protected]>
perf pmu-events: Split big_c_string storage into standalone compilation unit
This commit modifies jevents.py to emit the large big_c_string literal into a dedicated pmu-events-string.c file instead of pmu-events.c. This allows the build system to compile both files in parallel to reduce compilation latency.
Quoted 4 lines
tools/perf/pmu-events/Build +0
-0
[ ... ]
tools/perf/pmu-events/Build endif · line 240
Quoted 9 lines
tools/perf/pmu-events/Build +4
-1@@ -224,6 +234,9 @@ endif # and inputs are dependencies. $(PMU_EVENTS_C): $(JEVENTS_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ $(PMU_EVENTS_STRING_C) + +$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C) + @:
Does this dummy recipe correctly handle configuration changes?
If switching from NO_JEVENTS=1 to NO_JEVENTS=0, pmu-events.c might already exist from a copy of empty-pmu-events.c and have a timestamp newer than JEVENTS_DEPS.
Make could consider pmu-events.c up to date, execute this dummy recipe for the missing pmu-events-string.c, and then fail to compile pmu-events-string.o with a missing file error.
Quoted 4 lines
tools/perf/pmu-events/jevents.py +0
-0
[ ... ]
Quoted 7 lines
tools/perf/pmu-events/jevents.py +0
-3@@ -1463,10 +1465,20 @@ struct pmu_table_entry { ftw(arch_path, [], preprocess_one_file) _bcs.compute() - _args.output_file.write('static const char *const big_c_string =\n') - for s in _bcs.big_string: - _args.output_file.write(s)
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1476
Quoted 10 lines
tools/perf/pmu-events/jevents.py +9
-1- _args.output_file.write(';\n\n') + if not _args.output_string_file: + _args.output_file.write('static const char *const big_c_string =\n') + for s in _bcs.big_string: + _args.output_file.write(s) + _args.output_file.write(';\n\n') + else: + _args.output_string_file.write('/* SPDX-License-Identifier: GPL-2.0 */\n') + _args.output_string_file.write('/* Autogenerated by jevents.py */\n') + _args.output_string_file.write('const char big_c_string[] =\n')
Since big_c_string is now emitted as a non-static global variable in a separate compilation unit, does it need a shared header declaration?
Without a prior declaration included in the generated pmu-events-string.c, compilers may emit -Wmissing-variable-declarations warnings during the build.
Also, is big_c_string too generic of a name to place in the global namespace?
Quoted 5 lines
tools/perf/pmu-events/jevents.py +5
-0+ for s in _bcs.big_string: + _args.output_string_file.write(s) + _args.output_string_file.write(';\n') + _args.output_string_file.close() + _args.output_file.write('extern const char big_c_string[];\n\n')
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=12
Ian Rogers
reviewerPATCH 13/14 perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
In reply to Ian RogersCurrently, jevents.py parses hundreds of JSON event and metric files sequentially across all CPU architectures during Kbuild startup, taking ~3.3 seconds of pure single-core execution time.
Refactor jevents.py to pre-populate its internal JSON AST cache in parallel across all available CPU cores using ProcessPoolExecutor. Configure worker process initializers to guarantee standard event mapping inheritance under spawn multiprocessing semantics, and apply depth filtering to prevent redundant worker tasks. This accelerates jevents.py execution by over 11x (from 3.3s down to ~290ms), fully reclaiming multi-core concurrency during the build generation phase.
--- tools/perf/pmu-events/jevents.py | 36 ++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-)
tools/perf/pmu-events/jevents.py +32 -4
@@ -457,8 +457,8 @@ class JsonEvent: return f'{{ { _bcs.offsets[s] } }}, /* {fix_comment(s)} */\n' -@lru_cache(maxsize=None) -def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: +_json_cache = {} +def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" try: events = json.load(open(path), object_hook=JsonEvent)
@@ -474,12 +474,16 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: if updates: for event in events: if event.metric_name in updates: - # print(f'Updated {event.metric_name} from\n"{event.metric_expr}"\n' - # f'to\n"{updates[event.metric_name]}"') event.metric_expr = updates[event.metric_name] return events +def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: + key = (path, topic) + if key not in _json_cache: + _json_cache[key] = _read_json_events_impl(path, topic) + return _json_cache[key] + def preprocess_arch_std_files(archpath: str) -> None: """Read in all architecture standard events.""" global _arch_std_events
@@ -1381,6 +1385,10 @@ const char *describe_metricgroup(const char *group) } """) +def _parallel_read_json_events(task: Tuple[str, str]) -> Tuple[str, str, Sequence[JsonEvent]]: + path, topic = task + return path, topic, _read_json_events_impl(path, topic) + def main() -> None: global _args
@@ -1459,9 +1467,29 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json) + + def _init_worker(std_events: dict) -> None: + global _arch_std_events + _arch_std_events = std_events + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor: + for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file) _bcs.compute()
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v1 13/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [High] The nested `_init_worker` function causes a fatal `PicklingError` under `spawn` multiprocessing…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [High] The nested _init_worker function causes a fatal PicklingError under spawn multiprocessing semantics.
- [High] Parallelizing the JSON parsing breaks architecture isolation for standard events, leading to state contamination across architectures.
- [Medium] The depth filtering logic is missing from the collect_json callback, causing redundant parallel parsing of deeply nested files. --
commit 90db323cf161fc6e18235c0d700851677b283082
Author: Ian Rogers <[email protected]>
perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
This commit refactors jevents.py to pre-populate its internal JSON AST cache in parallel using ProcessPoolExecutor. It aims to accelerate execution by parallelizing the parsing of JSON event and metric files across available cores.
Quoted 4 lines
tools/perf/pmu-events/jevents.py +0
-0
[ ... ]
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1476
Quoted 11 lines
tools/perf/pmu-events/jevents.py +7
-0@@ -1459,9 +1467,29 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name)))
Is the depth filtering logic missing from collect_json()?
The commit message mentions applying depth filtering to prevent redundant worker tasks. However, collect_json does not seem to limit the directory depth.
This could cause the main thread to queue worker tasks for deeply nested files, which are fully parsed by the worker pool but subsequently ignored during the actual generation phase.
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1481
Quoted 5 lines
tools/perf/pmu-events/jevents.py +2
-0+ for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json)
Does this loop break architecture isolation for standard events?
By looping over all architectures to populate a single global _arch_std_events dictionary before parsing the JSON files, standard events with identical names from later architectures might overwrite definitions from earlier architectures.
When the parallel worker processes evaluate the earlier architecture JSON files, they may use this polluted global dictionary and inherit fields from the later architectures, leading to incorrect PMU tables.
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1487
Quoted 6 lines
tools/perf/pmu-events/jevents.py +6
-0+ + def _init_worker(std_events: dict) -> None: + global _arch_std_events + _arch_std_events = std_events + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor:
Could defining _init_worker locally cause a pickling error under spawn multiprocessing semantics?
_init_worker is defined as a local function inside main(). Under spawn multiprocessing semantics, Python uses pickle to serialize objects to child processes, but pickle cannot serialize local functions.
This regression might result in an attribute error when initializing the executor, which seems to contradict the commit message's claim of supporting spawn semantics.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
Ian Rogers
reviewerPATCH 14/14 perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds
In reply to Ian RogersIn Makefile.perf, ALL_PROGRAMS includes SCRIPTS (perf-archive, perf-iostat). However, unlike PROGRAMS and DLFILTERS, SCRIPTS was not prefixed with $(OUTPUT).
During out-of-tree builds (or when O= is specified), Make checked for the unprefixed target 'tools/perf/perf-archive'. Since the actual script was installed into $(OUTPUT)perf-archive, Make concluded the target was missing and continuously re-executed the script installation rule on every single incremental build.
Prefix SCRIPTS with $(OUTPUT) and update the static pattern rule to ensure Kbuild correctly tracks generated script prerequisites during incremental builds.
--- tools/perf/Makefile.perf | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +3 -3
@@ -405,7 +405,7 @@ python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT # Use the detected configuration -include $(OUTPUT).config-detected -SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) +SCRIPTS = $(addprefix $(OUTPUT),$(patsubst %.sh,%,$(SCRIPT_SH))) PROGRAMS += $(OUTPUT)perf
@@ -592,8 +592,8 @@ $(GTK_IN): FORCE prepare $(OUTPUT)libperf-gtk.so: $(GTK_IN) $(PERFLIBS) $(QUIET_LINK)$(CC) -o $@ -shared $(LDFLAGS) $(filter %.o,$^) $(GTK_LIBS) -$(SCRIPTS) : % : %.sh - $(QUIET_GEN)$(INSTALL) '[email protected]' '$(OUTPUT)$@' +$(SCRIPTS) : $(OUTPUT)% : %.sh + $(QUIET_GEN)$(INSTALL) '$<' '$@' $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE $(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerv2 cover perf build: Reduce build time by nearly half
This patch series refactors Kbuild internals, BPF skeleton generation, Python AST pre-computation, and foundational tooling dependencies across the perf tool build system. By eliminating umbrella target synchronization barriers, decoupling static library prerequisites, parallelizing single-core script generators, and eradicating redundant feature checks, this series unlocks absolute theoretical peak multi-core concurrency during Kbuild startup.
On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 46%:
Before:
real 0m29.006s
user 2m46.019s
sys 0m30.610sAfter:
real 0m15.655s
user 2m43.051s
sys 0m26.437sSaving 13.3 full seconds time per clean build. Furthermore, nothing to build incremental builds are improved by nearly 7x:
Before:
real 0m11.528s
user 0m9.633s
sys 0m6.965sAfter:
real 0m1.665s
user 0m1.501s
sys 0m0.841sSummary of Patches:
1-4: Foundational Tooling & Fast-Path Feature Detection
- Exempts bpftool bootstrap from non-essential feature tests (LLVM, libbfd, libcap), saving 1.1s of sub-make fork overhead during Kbuild startup.
- Integrates libdebuginfod directly into test-all.c, allowing Make to skip individual feature check sub-make forks during AST parsing on fully configured workstations.
- Fixes test-clang-bpf-co-re.bin feature check to correctly generate its target file on disk, allowing Kbuild to perfectly cache the detection result and avoid continuous sub-make re-evaluations.
- Short-circuits CC_NO_CLANG compiler inspection probe in Makefile.include by exporting the cached result, eliminating 40+ redundant compiler forks across the sub-make hierarchy.
5-7: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.
8-11: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dump vmlinux.h in the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
12-13: Foundational Linkage Optimization
- Eliminates redundant libbpf sub-make feature checks during static builds.
- Moves static libsymbol and libbpf library prerequisites out of the prepare step.
14-15: jevents.py Concurrency & Deduplication
- Splits the massive 2.8 MB big_c_string literal out of pmu-events.c into a dedicated pmu-events-string.c compilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations.
- Pre-populates jevents.py JSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms).
16: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.
17-18: AST Parsing Optimization & Shell Fork Eradication
- Converts ZENS, ARMS, and INTELS in pmu-events/Build from recursive assignment (=) to simply expanded assignment (:=) and replaces model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once during AST parsing and evaluates path macros purely in memory, completely eradicating over 7,800 redundant sub-processes during out-of-tree build evaluation.
- Converts llvm-config shell queries in Makefile.config from recursive assignment (=) to simply expanded assignment (:=). This eliminates ~185 redundant sub-processes that were previously executed across object compilation dependency checks.
Changes since v1:
- Reorganized commit order so foundational build system and script infrastructure patches precede perf tool refactoring.
- Added Tested-by tag from James Clark on v1 patches.
- Eliminated redundant llvm-config shell forks and simply expanded PMU directory probing variables, wiping out over 7,800 redundant sub-processes during AST parsing.
- Fixed test-clang-bpf-co-re.bin feature check caching and short-circuited CC_NO_CLANG compiler probes across sub-makes.
Ian Rogers (18):
bpftool build: Restrict feature tests during bootstrap compilation tools build: Integrate libdebuginfod into test-all fast path tools build: Fix test-clang-bpf-co-re.bin to generate target file tools scripts: Short-circuit CC_NO_CLANG compiler probe in
Makefile.include
perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak perf build: Move static libbpf dependency out of prepare step perf build: Pre-generate BPF skeleton tooling during umbrella prepare
phase
perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds perf pmu-events: Split big_c_string storage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
perf pmu-events: Convert recursive shell assignments and macros to
Make built-ins
perf build: Convert llvm-config shell queries to simply expanded
variables
tools/bpf/bpftool/Makefile | 5 +
tools/build/Makefile.feature | 6 +-
tools/build/feature/Makefile | 4 +-
tools/build/feature/test-all.c | 5 +
tools/perf/Build | 2 +
tools/perf/Makefile.config | 19 +-
tools/perf/Makefile.perf | 427 +-----------------
tools/perf/bench/Build | 6 +
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 110 +++++
tools/perf/builtin-trace.c | 30 +-
tools/perf/pmu-events/Build | 25 +-
tools/perf/pmu-events/jevents.py | 56 ++-
tools/perf/trace/beauty/Build | 280 ++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 +++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 6 +-
tools/perf/trace/beauty/futex_val3.c | 6 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 1 +
tools/perf/trace/beauty/perf_event_open.c | 22 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 22 +
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 17 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 +-
tools/perf/util/env.h | 1 +
tools/scripts/Makefile.include | 3 +
41 files changed, 716 insertions(+), 503 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.mak
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 1/18 bpftool build: Restrict feature tests during bootstrap compilation
When bpf_skel.mak executes 'make -C ../bpf/bpftool bootstrap', bpftool's Makefile unconditionally evaluated feature checks for llvm, libcap, libbfd, and disassembler libraries because the bootstrap target was not exempted.
Since the bootstrap bpftool strictly compiles minimal AST parsing and C code generation logic without linking LLVM or disassembler libraries, these feature check sub-makes are completely redundant.
Exempt the bootstrap target from non-essential feature tests to eliminate unneeded sub-make fork overhead during Kbuild startup.
--- tools/bpf/bpftool/Makefile | 5 +++++ 1 file changed, 5 insertions(+)
tools/bpf/bpftool/Makefile +5 -0
@@ -106,6 +106,10 @@ ifneq ($(SKIP_CRYPTO),1) CRYPTO_LIBS := -lcrypto endif +ifeq ($(MAKECMDGOALS),bootstrap) +FEATURE_TESTS := libelf-zstd +FEATURE_DISPLAY := +else FEATURE_TESTS := clang-bpf-co-re FEATURE_TESTS += llvm FEATURE_TESTS += libcap
@@ -122,6 +126,7 @@ FEATURE_DISPLAY += libcap FEATURE_DISPLAY += libbfd FEATURE_DISPLAY += libbfd-liberty FEATURE_DISPLAY += libbfd-liberty-z +endif check_feat := 1 NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install doc-uninstall
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 2/18 tools build: Integrate libdebuginfod into test-all fast path
In reply to Ian RogersBundling libdebuginfod into test-all.c allows Make to avoid an individual feature check sub-make fork during AST parsing. Short-circuit feature_check in Makefile.feature to instantly skip sub-make evaluations when a feature was already pre-detected by test-all.
Trade-offs:
On some Linux distributions (e.g., Debian/Ubuntu), libdebuginfod-dev is packaged separately from libelf-dev and libdw-dev. If a user installs libdw-dev but omits libdebuginfod-dev, adding -ldebuginfod to BUILD_ALL causes test-all to fail, falling back to individual feature tests. However, on fully configured developer workstations, this successfully reclaims sequential startup latency.
--- tools/build/Makefile.feature | 6 ++++-- tools/build/feature/Makefile | 2 +- tools/build/feature/test-all.c | 5 +++++ 3 files changed, 10 insertions(+), 3 deletions(-)
tools/build/Makefile.feature +4 -2
@@ -8,7 +8,9 @@ endif feature_check = $(eval $(feature_check_code)) define feature_check_code - feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + ifeq ($(feature-$(1)),) + feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + endif endef feature_set = $(eval $(feature_set_code))
@@ -100,6 +102,7 @@ FEATURE_TESTS_BASIC := \ disassembler-four-args \ disassembler-init-styled \ file-handle \ + libdebuginfod \ libopenssl # FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list
@@ -125,7 +128,6 @@ FEATURE_TESTS_EXTRA := \ clang \ libbpf \ libpfm4 \ - libdebuginfod \ clang-bpf-co-re \ bpftool-skeletons
tools/build/feature/Makefile +1 -1
@@ -115,7 +115,7 @@ __BUILD = $(CC) $(CFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $( BUILD_BFD = $(BUILD) -DPACKAGE='"perf"' -lbfd -ldl BUILD_ALL = $(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -lslang \ $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -ldl -lz -llzma -lzstd \ - $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) + $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) -ldebuginfod __BUILDXX = $(CXX) $(CXXFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS) BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
tools/build/feature/test-all.c +5 -0
@@ -146,6 +146,10 @@# include "test-libopenssl.c" #undef main +#define main main_test_libdebuginfod +# include "test-libdebuginfod.c" +#undef main +int main (int argc,char *argv[]) {main_test_libpython ();
@@ -178,6 +182,7 @@ int main(int argc, char *argv[])main_test_libzstd ();main_test_libtraceevent ();main_test_libopenssl (); +main_test_libdebuginfod ();return 0 ; }
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 02/18] tools build: Integrate libdebuginfod into test-all fast path
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The `ifeq` short-circuit in `feature_check_code` fails to prevent unconditional sub-make…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The ifeq short-circuit in feature_check_code fails to prevent unconditional sub-make evaluations due to eager expansion in $(eval). --
commit e9f96ae598e8b4643d43a93ea3da9bfe004d8b24
Author: Ian Rogers <[email protected]>
tools build: Integrate libdebuginfod into test-all fast path
This commit bundles libdebuginfod into the test-all fast path to reclaim sequential startup latency. It also attempts to short-circuit the feature_check evaluations to skip sub-make forks when a feature is already pre-detected.
tools/build/Makefile.feature endif · line 14
Quoted 13 lines
tools/build/Makefile.feature +3
-1@@ -8,7 +8,9 @@ endif feature_check = $(eval $(feature_check_code)) define feature_check_code - feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + ifeq ($(feature-$(1)),) + feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + endif endef
Does this $(shell ...) invocation get evaluated unconditionally before the ifeq is parsed?
Since feature_check_code is passed to the $(eval ...) function:
feature_check = $(eval $(feature_check_code))
GNU Make expands the arguments to eval completely before parsing them as Makefile syntax.
Because the $(shell ...) call is not escaped (for example, as $$(shell ...)), it appears it will be executed during the initial expansion phase, bypassing the ifeq condition entirely.
Could this result in the sub-make fork still running unconditionally, defeating the intended optimization?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerPATCH v2 3/18 tools build: Fix test-clang-bpf-co-re.bin to generate target file
In reply to Ian RogersIn tools/build/feature/Makefile, the test-clang-bpf-co-re.bin feature check compiled test-clang-bpf-co-re.c assembly on stdout and piped it to grep, but never redirected the matched output to create the target binary () on disk.
Because the target file was never created, Kbuild could never cache the result of the check. Consequently, Make treated the prerequisite as missing and continuously re-executed the Clang BPF backend compilation check on every single sub-make evaluation during build startup.
Redirect grep output to to guarantee the file is created on disk upon success, allowing Kbuild to perfectly cache the detection result and avoid redundant sub-make forks.
--- tools/build/feature/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/build/feature/Makefile +1 -1
@@ -385,7 +385,7 @@ $(OUTPUT)test-libzstd.bin: $(OUTPUT)test-clang-bpf-co-re.bin: $(CLANG) -S -g --target=bpf -o - $(patsubst %.bin,%.c,$(@F)) | \ - grep BTF_KIND_VAR + grep BTF_KIND_VAR > $@ $(OUTPUT)test-file-handle.bin: $(BUILD)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 03/18] tools build: Fix test-clang-bpf-co-re.bin to generate target file
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Unconditionally redirecting the feature check pipeline directly to the target file (`> $@`) causes…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Unconditionally redirecting the feature check pipeline directly to the target file (> $@) causes Kbuild to permanently cache failed detections as successful. --
commit c952ccdf6ac3df2b44fe927cf8066c42efc727a9
Author: Ian Rogers <[email protected]>
tools build: Fix test-clang-bpf-co-re.bin to generate target file
This commit updates the test-clang-bpf-co-re.bin feature check to redirect the grep output to the target file. This ensures the target file is created on disk upon success, allowing Kbuild to cache the result.
tools/build/feature/Makefile $(OUTPUT)test-libzstd.bin: · line 388
Quoted 10 lines
tools/build/feature/Makefile +1
-1@@ -385,7 +385,7 @@ $(OUTPUT)test-libzstd.bin: $(OUTPUT)test-clang-bpf-co-re.bin: $(CLANG) -S -g --target=bpf -o - $(patsubst %.bin,%.c,$(@F)) | \ - grep BTF_KIND_VAR + grep BTF_KIND_VAR > $@
Could this unconditional redirection cause Kbuild to permanently cache failed detections as successful?
If the host compiler does not support BPF CO-RE, grep fails to find BTF_KIND_VAR and exits with 1. However, the shell processes the redirection by creating the target file before execution.
Since tools/build/feature/Makefile lacks the GNU Make .DELETE_ON_ERROR:
special target, a failed grep leaves a 0-byte $@ target file on disk.
On subsequent evaluations, make checks test-clang-bpf-co-re.bin, sees the file exists, and considers the target up to date. This causes make to exit with 0, which breaks negative feature caching as Kbuild incorrectly interprets this as feature-clang-bpf-co-re=1.
Would it be better to write to a temporary file and move it on success (e.g., > [email protected] && mv [email protected] $@) to ensure the target is only created when the feature is actually supported?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Ian Rogers
reviewerPATCH v2 4/18 tools scripts: Short-circuit CC_NO_CLANG compiler probe in Makefile.include
In reply to Ian RogersIn Makefile.include, CC_NO_CLANG unconditionally executed a GCC/Clang compiler inspection fork (gcc -dM -E ...) during Makefile parse time. Because Makefile.include is included across more than 40 sub-makes and component directories throughout the Kbuild tree, Make continuously re-executed this compiler probe dozens of times during incremental builds.
Wrap CC_NO_CLANG with a conditional check and export the variable to sub-makes. This guarantees Kbuild evaluates the compiler backend exactly once during top-level Make invocation, eliminating 40+ redundant shell forks across the build tree.
--- tools/scripts/Makefile.include | 3 +++ 1 file changed, 3 insertions(+)
tools/scripts/Makefile.include +3 -0
@@ -74,7 +74,10 @@ $(call allow-override,CXX,$(CROSS_COMPILE)g++) $(call allow-override,STRIP,$(CROSS_COMPILE)strip) endif +ifeq ($(CC_NO_CLANG),) CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) +endif +export CC_NO_CLANG ifneq ($(LLVM),) HOSTAR ?= $(LLVM_PREFIX)llvm-ar$(LLVM_SUFFIX)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 04/18] tools scripts: Short-circuit CC_NO_CLANG compiler probe in Makefile.include
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Exporting `CC_NO_CLANG` across sub-makes breaks cross-compilation when target and host compilers…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Exporting CC_NO_CLANG across sub-makes breaks cross-compilation when target and host compilers differ (e.g., CC=gcc and HOSTCC=clang).
- [Medium] The empty check ifeq ($(CC_NO_CLANG),) blindly trusts the user's external shell environment, introducing build fragility. --
commit fd712ba4b295d8ef072e15714d284857b59a5cfa
Author: Ian Rogers <[email protected]>
tools scripts: Short-circuit CC_NO_CLANG compiler probe in Makefile.include
This commit wraps the CC_NO_CLANG evaluation in a conditional check and exports it. This caching prevents Make from re-executing the compiler probe multiple times during incremental builds.
Quoted 4 lines
tools/scripts/Makefile.include +0
-0
[ ... ]
tools/scripts/Makefile.include $(call allow-override,CXX,$(CROSS_COMPILE)g++) · line 77
Quoted 5 lines
tools/scripts/Makefile.include +1
-0@@ -74,7 +74,10 @@ $(call allow-override,CXX,$(CROSS_COMPILE)g++) $(call allow-override,STRIP,$(CROSS_COMPILE)strip) endif +ifeq ($(CC_NO_CLANG),)
Does this empty check run the risk of blindly trusting the external shell environment? If CC_NO_CLANG happens to be exported in the user's shell, the probe is completely skipped. Would checking the origin of the variable, such as using ifeq ($(origin CC_NO_CLANG), undefined), prevent this from causing unintended build misconfigurations?
tools/scripts/Makefile.include $(call allow-override,CXX,$(CROSS_COMPILE)g++) · line 80
Quoted 3 lines
tools/scripts/Makefile.include +2
-0CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?) +endif +export CC_NO_CLANG
Could exporting CC_NO_CLANG cause regressions during cross-compilation when CC and HOSTCC use different compilers?
Many tools build host components by invoking sub-makes with an overridden compiler, like CC="$(HOSTCC)" in tools/objtool or libsubcmd. Because CC_NO_CLANG is exported here, these sub-makes will inherit the parent's cached value and skip their own compiler probes.
If CC=gcc and HOSTCC=clang, the sub-make will inherit CC_NO_CLANG=1, which incorrectly appends GCC-specific flags (like -Wstrict-aliasing=3) for the Clang build, potentially breaking it. Conversely, if CC=clang and HOSTCC=gcc, sub-makes might inherit CC_NO_CLANG=0 and attempt to use Clang-specific flags like --target= with GCC.
Quoted 3 lines
tools/scripts/Makefile.include +0
-0ifneq ($(LLVM),) HOSTAR ?= $(LLVM_PREFIX)llvm-ar$(LLVM_SUFFIX)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Ian Rogers
reviewerPATCH v2 5/18 perf trace beauty: Make beauty generated C code standalone .o files
In reply to Ian RogersPreviously, builtin-trace.c directly included 15 embedded C files (e.g. trace/beauty/mmap.c and fsconfig_arrays.c), which in turn depend on dozens of generated beauty script arrays. To satisfy these embedded inclusions, the global Makefile.perf would define all the generator variables/rules and include them in the massive sequential prepare umbrella target, choking parallel build startup.
Furthermore, tools/perf/util/syscalltbl.c included its own generated mapper, and util/env.c conditionally included arch_errno_names.c inline, splitting consumers across directories and preventing clean Make encapsulation.
Refactor the framework to achieve better encapsulation:
1. Move util/syscalltbl.[ch] into trace/beauty/ to co-locate with all generated code consumers.
2. Create fsconfig.c and flatten embedded beauty .c files to compile as independent standalone objects via trace/beauty/Build, exporting their formatting functions via beauty.h and env.h. Switch arch_errno_names.o and syscalltbl.o assignments directly to perf-util-y and add a top-level recursive kbuild hook (perf-util-$(CONFIG_TRACE) += trace/beauty/) to compile them into libperf-util.a, resolving remote linkage for standalone python extensions.
3. Bridge private opaque references (struct trace) securely via accessors trace__show_zeros() and trace__host(), avoiding header entanglements.
4. Consolidate all generator variables, script paths, and array generation rules entirely out of Makefile.perf and place them directly inside the exact local Build files where their output objects are compiled (trace/beauty/Build and trace/beauty/tracepoints/Build), binding prerequisites locally. Use $(call rule_mkdir) directly inside generator recipes to guarantee dynamic directory creation before script redirection, and append $(call echo-cmd,gen) across all rules to print clean, standardized GEN ... file.c output during compilation.
5. Clean up clean target to recursively remove the generated directory instead of relying on dozens of individual variables.
This unchokes the "prepare" target parallel barrier, allows make to evaluate generation scripts purely locally where consumed, and flattens the tracepoint formatting architecture.
Testing a parallel build (make -j28 all from scratch) shows improvements:
Before:
real 0m28.689s
user 2m38.490s
sys 0m30.148sAfter:
real 0m27.642s
user 2m32.356s
sys 0m26.683sSo reclaiming ~9.6 seconds of raw CPU time and over 1 full second off overall real-world build latency, by overlapping sub-make startup and avoiding top-level double-parsing overhead.
---
tools/perf/Build | 2 +
tools/perf/Makefile.perf | 282 +-----------------
tools/perf/builtin-trace.c | 30 +-
tools/perf/trace/beauty/Build | 280 +++++++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 ++++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 6 +-
tools/perf/trace/beauty/futex_val3.c | 6 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 1 +
tools/perf/trace/beauty/perf_event_open.c | 22 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 22 ++
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 4 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 +-
tools/perf/util/env.h | 1 +
29 files changed, 468 insertions(+), 352 deletions(-)
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)tools/perf/Build +2 -0
@@ -34,6 +34,8 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-$(CONFIG_TRACE) += trace/beauty/ endif +perf-util-y += trace/beauty/ + perf-$(CONFIG_LIBELF) += builtin-probe.o perf-bench-y += bench/
tools/perf/Makefile.perf +2 -280
300 lines not on this page · raw message
tools/perf/builtin-trace.c +13 -17
@@ -60,12 +60,13 @@#include "callchain.h" #include "print_binary.h" #include "string2.h" -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "../perf.h" #include "trace_augment.h" #include "dwarf-regs.h" #include <errno.h> +#include <sys/stat.h> #include <inttypes.h> #include <poll.h> #include <signal.h>
@@ -234,6 +235,16 @@ struct trace {const char *uid_str; }; +bool trace__show_zeros (const struct trace *trace) +{ +return trace->show_zeros; +} + +struct machine *trace__host (const struct trace *trace) +{ +return trace->host; +} +static void trace__load_vmlinux_btf (struct trace *trace__maybe_unused ) {#ifdef HAVE_LIBBPF_SUPPORT
@@ -776,9 +787,7 @@ static const char *fsmount_flags[] = { }; static DEFINE_STRARRAY(fsmount_flags, "FSMOUNT_"); -#include "trace/beauty/generated/fsconfig_arrays.c" -static DEFINE_STRARRAY(fsconfig_cmds, "FSCONFIG_");static const char *epoll_ctl_ops[] = {"ADD" ,"DEL" ,"MOD" , }; static DEFINE_STRARRAY_OFFSET(epoll_ctl_ops, "EPOLL_CTL_", 1);
@@ -1128,20 +1137,7 @@ static bool syscall_arg__strtoul_btf_type(char *bf __maybe_unused, size_t size _ .parm = &strarray__##array, \ .show_zero = true, } -#include "trace/beauty/eventfd.c" -#include "trace/beauty/futex_op.c" -#include "trace/beauty/futex_val3.c" -#include "trace/beauty/mmap.c" -#include "trace/beauty/mode_t.c" -#include "trace/beauty/msg_flags.c" -#include "trace/beauty/open_flags.c" -#include "trace/beauty/perf_event_open.c" -#include "trace/beauty/pid.c" -#include "trace/beauty/sched_policy.c" -#include "trace/beauty/seccomp.c" -#include "trace/beauty/signum.c" -#include "trace/beauty/socket_type.c" -#include "trace/beauty/waitid_options.c" +static const struct syscall_fmt syscall_fmts[] = { { .name ="access" ,
tools/perf/trace/beauty/Build +280 -0
289 lines not on this page · raw message
tools/perf/trace/beauty/arch_errno_names.c +2 -0
@@ -1 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "util/env.h" #include "trace/beauty/generated/arch_errno_name_array.c"
tools/perf/trace/beauty/arch_errno_names.sh +1 -1
@@ -57,7 +57,7 @@ create_arch_errno_table_func() archlist="$1" default="$2" - printf 'static arch_syscalls__strerrno_t *\n' + printf 'arch_syscalls__strerrno_t *\n' printf 'arch_syscalls__strerrno_function(const char *arch)\n' printf '{\n' for arch in $archlist; do
tools/perf/trace/beauty/beauty.h +60 -0
70 lines not on this page · raw message
tools/perf/trace/beauty/eventfd.c +4 -2
@@ -11,7 +11,9 @@#define EFD_CLOEXEC 02000000#endif -static size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="EFD_" ;
@@ -36,4 +38,4 @@ static size_t syscall_arg__scnprintf_eventfd_flags(char *bf, size_t size, structreturn printed; } -#define SCA_EFD_FLAGS syscall_arg__scnprintf_eventfd_flags +
tools/perf/trace/beauty/fsconfig.c new file +5 -0
@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "trace/beauty/generated/fsconfig_arrays.c" + +DEFINE_STRARRAY (fsconfig_cmds,"FSCONFIG_" );
tools/perf/trace/beauty/futex_op.c +4 -2
@@ -17,7 +17,9 @@#define FUTEX_CLOCK_REALTIME 256#endif -static size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="FUTEX_" ;
@@ -60,4 +62,4 @@ static size_t syscall_arg__scnprintf_futex_op(char *bf, size_t size, struct syscreturn printed; } -#define SCA_FUTEX_OP syscall_arg__scnprintf_futex_op +
tools/perf/trace/beauty/futex_val3.c +4 -2
@@ -5,7 +5,9 @@#define FUTEX_BITSET_MATCH_ANY 0xffffffff#endif -static size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) {const char *prefix ="FUTEX_BITSET_" ;unsigned int bitset = arg->val;
@@ -16,4 +18,4 @@ static size_t syscall_arg__scnprintf_futex_val3(char *bf, size_t size, struct syreturn scnprintf (bf, size,"%#xd" , bitset); } -#define SCA_FUTEX_VAL3 syscall_arg__scnprintf_futex_val3 +
tools/perf/trace/beauty/mmap.c +12 -12
@@ -1,5 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 #include <linux/log2.h> +#include "trace/beauty/beauty.h" #include "trace/beauty/generated/mmap_prot_array.c" static DEFINE_STRARRAY(mmap_prot, "PROT_");
@@ -8,8 +9,7 @@ static size_t mmap__scnprintf_prot(unsigned long prot, char *bf, size_t size, bo {return strarray__scnprintf_flags (&strarray__mmap_prot, bf, size, show_prefix, prot); } - -static size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) {unsigned long prot = arg->val;
@@ -19,18 +19,18 @@ static size_t syscall_arg__scnprintf_mmap_prot(char *bf, size_t size, struct sysreturn mmap__scnprintf_prot (prot, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_PROT syscall_arg__scnprintf_mmap_prot +#include "trace/beauty/generated/mmap_flags_array.c" -static DEFINE_STRARRAY(mmap_flags, "MAP_"); +DEFINE_STRARRAY (mmap_flags,"MAP_" );static size_t mmap__scnprintf_flags (unsigned long flags,char *bf,size_t size,bool show_prefix) {return strarray__scnprintf_flags (&strarray__mmap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, +struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -40,7 +40,7 @@ static size_t syscall_arg__scnprintf_mmap_flags(char *bf, size_t size,return mmap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_FLAGS syscall_arg__scnprintf_mmap_flags +#include "trace/beauty/generated/mremap_flags_array.c" static DEFINE_STRARRAY(mremap_flags, "MREMAP_");
@@ -50,7 +50,7 @@ static size_t mremap__scnprintf_flags(unsigned long flags, char *bf, size_t sizereturn strarray__scnprintf_flags (&strarray__mremap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -60,7 +60,7 @@ static size_t syscall_arg__scnprintf_mremap_flags(char *bf, size_t size, structreturn mremap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MREMAP_FLAGS syscall_arg__scnprintf_mremap_flags +static size_t madvise__scnprintf_behavior (int behavior,char *bf,size_t size) {
@@ -73,10 +73,10 @@ static size_t madvise__scnprintf_behavior(int behavior, char *bf, size_t size)return scnprintf (bf, size,"%#" , behavior); } -static size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, +struct syscall_arg *arg) {return madvise__scnprintf_behavior (arg->val, bf, size); } -#define SCA_MADV_BHV syscall_arg__scnprintf_madvise_behavior +
tools/perf/trace/beauty/mode_t.c +4 -2
@@ -20,7 +20,9 @@#define S_IXUGO (S_IXUSR|S_IXGRP|S_IXOTH)#endif -static size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="S_" ;
@@ -68,4 +70,4 @@ static size_t syscall_arg__scnprintf_mode_t(char *bf, size_t size, struct syscalreturn printed; } -#define SCA_MODE_T syscall_arg__scnprintf_mode_t +
tools/perf/trace/beauty/msg_flags.c +5 -3
@@ -27,8 +27,10 @@# define MSG_CMSG_CLOEXEC 0x40000000#endif -static size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="MSG_" ;
@@ -73,4 +75,4 @@ static size_t syscall_arg__scnprintf_msg_flags(char *bf, size_t size,return printed; } -#define SCA_MSG_FLAGS syscall_arg__scnprintf_msg_flags +
tools/perf/trace/beauty/open_flags.c +1 -0
@@ -2,6 +2,7 @@#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> +#include "trace/beauty/beauty.h" #ifndef O_DIRECT #define O_DIRECT 00040000
tools/perf/trace/beauty/perf_event_open.c +12 -10
@@ -1,4 +1,7 @@// SPDX-License-Identifier: LGPL-2.1 +#include <linux/perf_event.h> +#include "util/evsel_fprintf.h" +#ifndef PERF_FLAG_FD_NO_GROUP # define PERF_FLAG_FD_NO_GROUP (1UL << 0)#endif
@@ -15,8 +18,10 @@# define PERF_FLAG_FD_CLOEXEC (1UL << 3)/* O_CLOEXEC */ #endif -static size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="PERF_" ;
@@ -43,7 +48,7 @@ static size_t syscall_arg__scnprintf_perf_flags(char *bf, size_t size,return printed; } -#define SCA_PERF_FLAGS syscall_arg__scnprintf_perf_flags +struct attr_fprintf_args {size_t size, printed;
@@ -76,10 +81,11 @@ static size_t perf_event_attr___scnprintf(struct perf_event_attr *attr, char *bfstatic size_t syscall_arg__scnprintf_augmented_perf_event_attr (struct syscall_arg *arg,char *bf,size_t size) { -return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, arg->trace->show_zeros); +return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, +trace__show_zeros (arg->trace)); } -static size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) {if (arg->augmented.args)return syscall_arg__scnprintf_augmented_perf_event_attr (arg, bf, size);
@@ -87,8 +93,4 @@ static size_t syscall_arg__scnprintf_perf_event_attr(char *bf, size_t size, strureturn scnprintf (bf, size,"%#lx" , arg->val); } -#define SCA_PERF_ATTR syscall_arg__scnprintf_perf_event_attr -// 'argname' is just documentational at this point, to remove the previous comment with that info -#define SCA_PERF_ATTR_FROM_USER (argname) \ - { .scnprintf = SCA_PERF_ATTR, \ - .from_user = true, } +
tools/perf/trace/beauty/pid.c +4 -1
@@ -1,11 +1,14 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "util/machine.h" +#include "util/thread.h" size_t syscall_arg__scnprintf_pid (char *bf,size_t size,struct syscall_arg *arg) {int pid = arg->val;struct trace *trace = arg->trace;size_t printed =scnprintf (bf, size,"%d" , pid); -struct thread *thread =machine__findnew_thread (trace->host, pid, pid); +struct thread *thread =machine__findnew_thread (trace__host (trace), pid, pid);if (thread !=NULL ) {if (!thread__comm_set (thread))
tools/perf/trace/beauty/sched_policy.c +5 -3
@@ -14,8 +14,10 @@#define SCHED_RESET_ON_FORK 0x40000000#endif -static size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SCHED_" ;
@@ -47,4 +49,4 @@ static size_t syscall_arg__scnprintf_sched_policy(char *bf, size_t size,return printed; } -#define SCA_SCHED_POLICY syscall_arg__scnprintf_sched_policy +
tools/perf/trace/beauty/seccomp.c +7 -5
@@ -6,7 +6,9 @@#define SECCOMP_SET_MODE_FILTER 1#endif -static size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_SET_MODE_" ;
@@ -24,14 +26,14 @@ static size_t syscall_arg__scnprintf_seccomp_op(char *bf, size_t size, struct syreturn printed; } -#define SCA_SECCOMP_OP syscall_arg__scnprintf_seccomp_op +#ifndef SECCOMP_FILTER_FLAG_TSYNC #define SECCOMP_FILTER_FLAG_TSYNC 1#endif -static size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_FILTER_FLAG_" ;
@@ -52,4 +54,4 @@ static size_t syscall_arg__scnprintf_seccomp_flags(char *bf, size_t size,return printed; } -#define SCA_SECCOMP_FLAGS syscall_arg__scnprintf_seccomp_flags +
tools/perf/trace/beauty/signum.c +4 -2
@@ -1,7 +1,9 @@// SPDX-License-Identifier: LGPL-2.1 #include <signal.h> -static size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SIG" ;
@@ -54,4 +56,4 @@ static size_t syscall_arg__scnprintf_signum(char *bf, size_t size, struct syscalreturn scnprintf (bf, size,"%#x" , sig); } -#define SCA_SIGNUM syscall_arg__scnprintf_signum +
tools/perf/trace/beauty/socket_type.c +4 -2
@@ -18,7 +18,9 @@#define SOCK_TYPE_MASK 0xf#endif -static size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SOCK_" ;
@@ -60,4 +62,4 @@ static size_t syscall_arg__scnprintf_socket_type(char *bf, size_t size, struct sreturn printed; } -#define SCA_SK_TYPE syscall_arg__scnprintf_socket_type +
tools/perf/trace/beauty/syscalltbl.c renamed from tools/perf/util/syscalltbl.c +0 -0
tools/perf/trace/beauty/syscalltbl.h renamed from tools/perf/util/syscalltbl.h +0 -0
tools/perf/trace/beauty/tracepoints/Build +22 -0
@@ -1,2 +1,24 @@ perf-y += x86_irq_vectors.o perf-y += x86_msr.o + +# --- Local Encapsulated Tracepoint Rules --- +beauty_outdir := $(OUTPUT)trace/beauty/generated +beauty_arch_asm_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/asm/ +x86_arch_asm_dir := $(srctree)/tools/arch/x86/include/asm/ + +x86_arch_irq_vectors_array := $(beauty_outdir)/x86_arch_irq_vectors_array.c +x86_arch_irq_vectors_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_irq_vectors.sh + +$(x86_arch_irq_vectors_array): $(beauty_arch_asm_dir)/irq_vectors.h $(x86_arch_irq_vectors_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_irq_vectors_tbl)' $(beauty_arch_asm_dir) > $@ + +x86_arch_MSRs_array := $(beauty_outdir)/x86_arch_MSRs_array.c +x86_arch_MSRs_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_msr.sh + +$(x86_arch_MSRs_array): $(x86_arch_asm_dir)/msr-index.h $(x86_arch_MSRs_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_MSRs_tbl)' $(x86_arch_asm_dir) > $@ + +$(OUTPUT)trace/beauty/tracepoints/x86_irq_vectors.o: $(x86_arch_irq_vectors_array) +$(OUTPUT)trace/beauty/tracepoints/x86_msr.o: $(x86_arch_MSRs_array)
tools/perf/trace/beauty/waitid_options.c +5 -3
@@ -2,8 +2,10 @@#include <sys/types.h> #include <sys/wait.h> -static size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="W" ;
@@ -26,4 +28,4 @@ static size_t syscall_arg__scnprintf_waitid_options(char *bf, size_t size,return printed; } -#define SCA_WAITID_OPTIONS syscall_arg__scnprintf_waitid_options +
tools/perf/util/Build +3 -1
@@ -75,7 +75,7 @@ perf-util-y += sample.o perf-util-y += sample-raw.o perf-util-y += s390-sample-raw.o perf-util-y += amd-sample-raw.o -perf-util-$(CONFIG_TRACE) += syscalltbl.o + perf-util-y += ordered-events.o perf-util-y += namespaces.o perf-util-y += comm.o
@@ -441,3 +441,5 @@ $(OUTPUT)%.pylint_log: % $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false) perf-util-y += $(PYLINT_TEST_LOGS) + +
tools/perf/util/bpf-trace-summary.c +1 -1
@@ -6,7 +6,7 @@#include <stdlib.h> #include "dwarf-regs.h" /* for EM_HOST */ -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "util/cgroup.h" #include "util/hashmap.h" #include "util/trace.h"
tools/perf/util/env.c +1 -3
@@ -635,9 +635,7 @@ const char *perf_env__arch(struct perf_env *env)return normalize_arch (arch_name); } -#if defined(HAVE_LIBTRACEEVENT ) -#include "trace/beauty/arch_errno_names.c" -#endif +const char *perf_env__arch_strerrno (struct perf_env *env__maybe_unused ,int err__maybe_unused ) {
tools/perf/util/env.h +1 -0
@@ -189,6 +189,7 @@ void cpu_cache_level__free(struct cpu_cache_level *cache);const char *perf_env__arch (struct perf_env *env);const char *perf_env__arch_strerrno (struct perf_env *env,int err); +arch_syscalls__strerrno_t *arch_syscalls__strerrno_function (const char *arch);const char *perf_env__cpuid (struct perf_env *env);const char *perf_env__raw_arch (struct perf_env *env);int perf_env__nr_cpus_avail (struct perf_env *env);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 05/18] perf trace beauty: Make beauty generated C code standalone .o files
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The commit message claims to add `perf-util-$(CONFIG_TRACE) += trace/beauty/`, but the actual…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The commit message claims to add perf-util-$(CONFIG_TRACE) += trace/beauty/, but the actual patch uses perf-util-y += trace/beauty/. --
commit 64f2b861c8affb434c0dc658c744d4faf0fb24fa
Author: Ian Rogers <[email protected]>
perf trace beauty: Make beauty generated C code standalone .o files
This commit refactors the perf trace beauty framework to compile generated C code and formatting scripts as standalone object files, rather than embedding them directly. This encapsulation improves parallel build performance and cleans up Makefile dependencies.
Quoted 7 lines
2. Create
fsconfig.cand flatten embedded beauty .c files to compile as independent standalone objects via trace/beauty/Build, exporting their formatting functions viabeauty.handenv.h. Switch arch_errno_names.o and syscalltbl.o assignments directly to perf-util-y and add a top-level recursive kbuild hook (perf-util-$(CONFIG_TRACE) += trace/beauty/) to compile them into libperf-util.a, resolving remote linkage for standalone python extensions.
Is this description accurate regarding the recursive kbuild hook? The message states it adds perf-util-$(CONFIG_TRACE) += trace/beauty/, but looking at the actual patch, it is added unconditionally:
tools/perf/Build ifeq ($(CONFIG_LIBTRACEEVENT),y) · line 39
Quoted 11 lines
tools/perf/Build +2
-0@@ -34,6 +34,8 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-$(CONFIG_TRACE) += trace/beauty/ endif +perf-util-y += trace/beauty/ + perf-$(CONFIG_LIBELF) += builtin-probe.o
[ ... ]
Functionally, using perf-util-y appears correct since arch_errno_names.o and syscalltbl.o are needed by util/env.c and util/bpf-trace-summary.c respectively, which are compiled into libperf-util.a regardless of whether CONFIG_TRACE is enabled.
Could the commit message be updated to reflect the actual implementation?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Ian Rogers
reviewerPATCH v2 6/18 perf build: Decouple pmu-events from prepare umbrella target
In reply to Ian RogersCurrently, the $(LIBPMU_EVENTS_IN) sub-make depends on the massive "prepare" umbrella target. Because "prepare" depends on external libraries (libapi, libperf, etc.) as well as dozens of generated headers, make completely serializes the launch of the pmu-events sub-make behind some of those unrelated prerequisites.
Since pmu-events is a large compilation unit, unblock its startup by binding it directly to only $(LIBPERF) instead of prepare. This allows background python generation scripts to overlap simultaneously with the rest of the build.
Testing a parallel build (make -j28 clean all) shows improvements:
Before:
real 0m27.642s
user 2m32.356s
sys 0m26.683sAfter:
real 0m22.254s
user 2m32.810s
sys 0m24.646sThis reclaims over 5 full seconds of build latency (~19.5% overall reduction) by elevating average CPU concurrency from ~5.5 active cores up to ~8 active cores.
--- tools/perf/Makefile.perf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/Makefile.perf +1 -1
@@ -550,7 +550,7 @@ build := -f $(srctree)/tools/build/Makefile.build dir=. obj $(PERF_IN): prepare FORCE $(Q)$(MAKE) $(build)=perf -$(LIBPMU_EVENTS_IN): FORCE prepare +$(LIBPMU_EVENTS_IN): FORCE $(LIBPERF) $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 7/18 perf build: Remove empty archheaders target
In reply to Ian RogersRemove empty target that doesn't do anything.
--- tools/perf/Makefile.perf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/Makefile.perf +2 -2
@@ -627,7 +627,7 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) -prepare: $(OUTPUT)PERF-VERSION-FILE archheaders \ +prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ $(LIBAPI) \ $(LIBPERF) \
@@ -1070,7 +1070,7 @@ FORCE: .PHONY: all install clean config-clean strip install-gtk .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell .PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare -.PHONY: archheaders python_perf_target +.PHONY: python_perf_target endif # force_fixdep
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 8/18 perf build: Move BPF skeleton generation out of Makefile.perf
In reply to Ian RogersCurrently, the top-level Makefile.perf defines a massive global bpf-skel umbrella target that pre-compiles all 12+ BPF skeletons (%.skel.h) upfront before launching sub-makes. This forces unrelated sub-makes to serialize behind bpftool and clang BPF target evaluations, causing parallel build bottlenecks.
Furthermore, bench_uprobe.bpf.c lived inside util/bpf_skel/, breaking conceptual directory encapsulation since it is consumed purely by bench/uprobe.c.
Refactor the BPF skeletons to better achieve directory isolation:
1. Move tools/perf/util/bpf_skel/bench_uprobe.bpf.c directly into tools/perf/bench/bpf_skel/.
2. Extract the skeleton generation infrastructure out of Makefile.perf into a shared inclusion file tools/perf/bpf_skel.mak.
3. Include bpf_skel.mak locally inside tools/perf/util/Build and tools/perf/bench/Build and bind precise local prerequisites.
4. Safely synchronize the shared bpftool bootstrap and vmlinux.h targets via the conditional prepare: umbrella to avoid parallel sub-make races, while evaluating the actual skeletons completely locally on demand. A later patch will move these targets into bpf_skel.mak.
5. Export CLANG from the global Makefile to ensure accurate tool propagation.
6. Clean up Makefile.perf by stripping the global bpf-skel umbrella target and its SKELETONS list.
While removing code from Makefile.perf generally helps build performance, the impact here is minimal. The main motivation for the change is to better encapsulate things in the build and simplify Makefile.perf that has around 50 lines removed.
---
tools/perf/Makefile.perf | 59 ++-----------------
tools/perf/bench/Build | 6 ++
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 55 +++++++++++++++++
tools/perf/util/Build | 15 ++++-
6 files changed, 80 insertions(+), 57 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.maktools/perf/Makefile.perf +4 -55
@@ -274,7 +274,7 @@ ifeq ($(PYLINT),1) PYLINT := $(shell which pylint 2> /dev/null) endif -export srctree OUTPUT RM CC CXX RUSTC LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK +export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT include $(srctree)/tools/build/Makefile.include
@@ -632,8 +632,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \ - $(LIBSYMBOL) \ - bpf-skel + $(LIBSYMBOL) ifdef LIBBPF_STATIC prepare: $(LIBBPF)
@@ -914,44 +913,13 @@ python-clean: SKEL_OUT := $(abspath $(OUTPUT)util/bpf_skel) SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) -SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h -SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h -SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h -SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h -SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h $(SKEL_OUT)/sample_filter.skel.h -SKELETONS += $(SKEL_OUT)/kwork_top.skel.h $(SKEL_OUT)/syscall_summary.skel.h -SKELETONS += $(SKEL_OUT)/bench_uprobe.skel.h -SKELETONS += $(SKEL_OUT)/augmented_raw_syscalls.skel.h $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT) $(LIBSYMBOL_OUTPUT): $(Q)$(MKDIR) -p $@ ifeq ($(CONFIG_PERF_BPF_SKEL),y) +prepare: $(BPFTOOL) $(SKEL_OUT)/vmlinux.h BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool -# Get Clang's default includes on this system, as opposed to those seen by -# '--target=bpf'. This fixes "missing" files on some architectures/distros, -# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc. -# -# Use '-idirafter': Don't interfere with include mechanics except where the -# build would have failed anyways. -define get_sys_includes -$(shell $(1) $(2) -v -E - </dev/null 2>&1 \ - | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \ -$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}') -endef - -ifneq ($(CROSS_COMPILE),) -CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%)) -endif - -CLANG_OPTIONS = -Wall -CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH)) -BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES) -TOOLS_UAPI_INCLUDE := -I$(srctree)/tools/include/uapi - -ifneq ($(WERROR),0) - CLANG_OPTIONS += -Werror -endif $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
@@ -993,29 +961,10 @@ else $(Q)cp "$(VMLINUX_H)" $@ endif -$(SKEL_TMP_OUT)/%.bpf.o: $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h | $(SKEL_TMP_OUT) -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h - $(QUIET_CLANG)$(CLANG) -g -O2 -fno-stack-protector --target=bpf \ - $(CLANG_OPTIONS) $(EXTRA_BPF_FLAGS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ - -include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \ - -fms-extensions -Wno-microsoft-anon-tag \ - -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@ - -$(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o | $(BPFTOOL) - $(QUIET_GENSKEL)$(BPFTOOL) gen skeleton $< > $@ - -bpf-skel: $(SKELETONS) - -.PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o - -else # CONFIG_PERF_BPF_SKEL - -bpf-skel: - endif # CONFIG_PERF_BPF_SKEL bpf-skel-clean: - $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS) $(SKEL_OUT)/vmlinux.h + $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKEL_OUT)/*.skel.h $(SKEL_OUT)/vmlinux.h $(OUTPUT)bench/bpf_skel/*.skel.h pmu-events-clean: ifeq ($(OUTPUT),)
tools/perf/bench/Build +6 -0
@@ -24,3 +24,9 @@ perf-bench-$(CONFIG_X86_64) += mem-memcpy-x86-64-asm.o perf-bench-$(CONFIG_X86_64) += mem-memset-x86-64-asm.o perf-bench-$(CONFIG_NUMA) += numa.o + +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)bench/uprobe.o: $(SKEL_OUT)/bench_uprobe.skel.h +endif
tools/perf/bench/bpf_skel/bench_uprobe.bpf.c renamed from tools/perf/util/bpf_skel/bench_uprobe.bpf.c +0 -0
tools/perf/bench/uprobe.c +1 -1
@@ -44,7 +44,7 @@ static const char * const bench_uprobe_usage[] = { };#ifdef HAVE_BPF_SKEL -#include "bpf_skel/bench_uprobe.skel.h" +#include "bench/bpf_skel/bench_uprobe.skel.h" #define bench_uprobe__attach_uprobe(prog) \ skel->links.prog = bpf_program__attach_uprobe_opts(/*prog=*/skel->progs.prog, \
tools/perf/bpf_skel.mak new file +55 -0
55 lines not on this page · raw message
tools/perf/util/Build +14 -1
@@ -442,4 +442,17 @@ $(OUTPUT)%.pylint_log: % perf-util-y += $(PYLINT_TEST_LOGS) - +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)util/bpf_ftrace.o: $(SKEL_OUT)/func_latency.skel.h +$(OUTPUT)util/bpf-filter.o: $(SKEL_OUT)/sample_filter.skel.h +$(OUTPUT)util/bpf_kwork_top.o: $(SKEL_OUT)/kwork_top.skel.h +$(OUTPUT)util/bpf_off_cpu.o: $(SKEL_OUT)/off_cpu.skel.h +$(OUTPUT)util/bpf-trace-summary.o: $(SKEL_OUT)/syscall_summary.skel.h +$(OUTPUT)util/bpf_counter_cgroup.o: $(SKEL_OUT)/bperf_cgroup.skel.h +$(OUTPUT)util/bpf_trace_augment.o: $(SKEL_OUT)/augmented_raw_syscalls.skel.h +$(OUTPUT)util/bpf_counter.o: $(SKEL_OUT)/bpf_prog_profiler.skel.h $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h +$(OUTPUT)util/bpf_lock_contention.o: $(SKEL_OUT)/lock_contention.skel.h +$(OUTPUT)util/bpf_kwork.o: $(SKEL_OUT)/kwork_trace.skel.h +endif
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 08/18] perf build: Move BPF skeleton generation out of Makefile.perf
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Low] The `bpf-skel-clean` target does not clean intermediate `.bpf.o` files generated for benchmark…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Low] The bpf-skel-clean target does not clean intermediate .bpf.o files generated for benchmark skeletons.
- [Low] The bpf_skel_deps variable is defined but never utilized.
--
commit 82e063c8cb877e030e9f5ae7b0984031f34bbb1e
Author: Ian Rogers <[email protected]>
perf build: Move BPF skeleton generation out of Makefile.perf
This refactors the BPF skeleton generation to better achieve directory isolation. It moves the generation infrastructure out of Makefile.perf into a shared bpf_skel.mak, and adjusts local prerequisites.
Quoted 4 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
tools/perf/Makefile.perf else · line 967
Quoted 3 lines
bpf-skel-clean:
tools/perf/Makefile.perf +1
-1- $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS) $(SKEL_OUT)/vmlinux.h + $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKEL_OUT)/*.skel.h $(SKEL_OUT)/vmlinux.h $(OUTPUT)bench/bpf_skel/*.skel.h
Does the bpf-skel-clean target leave behind intermediate .bpf.o files for the benchmark skeletons?
Since bench_uprobe.bpf.c compilation was moved out of util/bpf_skel/.tmp/ and directly into bench/bpf_skel/.tmp/, it seems the updated clean target only removes $(SKEL_TMP_OUT) (which resolves to util/bpf_skel/.tmp) and the .skel.h files. It appears to miss the new .tmp directory in the bench/ tree where Kbuild outputs the intermediate .bpf.o artifacts, which could leave files on disk after a clean.
I noticed this was addressed in a later commit ("perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak").
[ ... ]
Quoted 5 lines
tools/perf/bpf_skel.mak new file +0
-0
[ ... ]
tools/perf/bpf_skel.mak line 17
Quoted 4 lines
tools/perf/bpf_skel.mak new file +4
-0+ifeq ($(CONFIG_PERF_BPF_SKEL),y) +BPFTOOL := $(SKEL_TOOL_TMP_OUT)/bootstrap/bpftool +VMLINUX_H := $(SKEL_TOOL_OUT)/vmlinux.h +bpf_skel_deps := $(BPFTOOL) $(VMLINUX_H)
Is bpf_skel_deps used anywhere after being defined?
It looks like the pattern rules duplicate $(BPFTOOL) and $(VMLINUX_H) as explicit prerequisites instead, leaving bpf_skel_deps unused.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Ian Rogers
reviewerPATCH v2 9/18 perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak
In reply to Ian RogersCurrently, bpftool and vmlinux.h are prerequisites of the top-level prepare target in Makefile.perf. This unnecessarily blocks the massive parallel C compilation of libraries (perf-util, perf-ui, pmu-events) during the initial startup phase.
Move all bpftool and vmlinux.h generation rules down into tools/perf/bpf_skel.mak to encapsulate BPF tooling completely within the skeleton framework. Remove them entirely from prepare to unblock immediate parallel build execution.
To prevent parallel sub-makes (perf-util and perf-bench) from racing to build shared prerequisites concurrently, while maintaining strict directory encapsulation without top-level inclusions, serialize bench after the util static archive finishes using an order-only prerequisite:
$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL)
--- tools/perf/Makefile.config | 2 +- tools/perf/Makefile.perf | 61 +++++--------------------------------- tools/perf/bpf_skel.mak | 47 +++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 55 deletions(-)
tools/perf/Makefile.config +1 -1
@@ -736,7 +736,7 @@ ifeq ($(BUILD_BPF_SKEL),1) endif ifndef GEN_VMLINUX_H - VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h + VMLINUX_H_FILE=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h endif dwarf-post-unwind := 1
tools/perf/Makefile.perf +7 -54
88 lines not on this page · raw message
tools/perf/bpf_skel.mak +47 -0
57 lines not on this page · raw message
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 10/18 perf build: Move static libbpf dependency out of prepare step
In reply to Ian RogersThe prepare step is a large serialization point before parallel sub-makes build the perf tool. The libbpf headers are used in the bench and util libraries, as well as individual perf builtin commands. Move the libbpf dependency out of the prepare step and into the dependencies for those targets to avoid it being a source of serialization in the prepare step.
--- tools/perf/Makefile.perf | 10 +++------- 1 file changed, 3 insertions(+), 7 deletions(-)
tools/perf/Makefile.perf +3 -7
@@ -547,7 +547,7 @@ export NO_JEVENTS build := -f $(srctree)/tools/build/Makefile.build dir=. obj -$(PERF_IN): prepare FORCE +$(PERF_IN): prepare $(LIBBPF) FORCE $(Q)$(MAKE) $(build)=perf $(LIBPMU_EVENTS_IN): FORCE $(LIBPERF)
@@ -558,7 +558,7 @@ $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN) # The $(LIBPERF_UTIL) dependency is to ensure bpftool and vmlinux.h # aren't racily built for bench/bpf_skel/bench_uprobe.bpf.c -$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL) +$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) | $(LIBPERF_UTIL) $(Q)$(MAKE) $(build)=perf-bench $(LIBPERF_BENCH): $(LIBPERF_BENCH_IN)
@@ -576,7 +576,7 @@ $(LIBPERF_UI_IN): FORCE prepare $(LIBPERF_UI): $(LIBPERF_UI_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_UTIL_IN): FORCE prepare +$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) $(Q)$(MAKE) $(build)=perf-util $(LIBPERF_UTIL): $(LIBPERF_UTIL_IN)
@@ -636,10 +636,6 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBSUBCMD) \ $(LIBSYMBOL) -ifdef LIBBPF_STATIC -prepare: $(LIBBPF) -endif - $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Re: [PATCH v2 10/18] perf build: Move static libbpf dependency out of prepare step
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Missing $(LIBBPF) dependency in $(LIBPERF_TEST_IN) target causing parallel build race condition. --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Missing $(LIBBPF) dependency in $(LIBPERF_TEST_IN) target causing parallel build race condition. --
commit e65547c4ccfbe83e535bb280ee457abfc8a0629d
Author: Ian Rogers <[email protected]>
perf build: Move static libbpf dependency out of prepare step
This commit removes the static libbpf dependency from the prepare step and moves it into specific targets like bench, util, and the main perf binary. This change aims to reduce serialization during the early build phases to improve parallel build performance.
Quoted 4 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
tools/perf/Makefile.perf prepare: $(OUTPUT)PERF-VERSION-FILE \ · line 642
Quoted 8 lines
tools/perf/Makefile.perf +0
-4@@ -636,10 +636,6 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBSUBCMD) \ $(LIBSYMBOL) -ifdef LIBBPF_STATIC -prepare: $(LIBBPF) -endif -
Looking at tools/perf/tests/sigtrap.c, it is built unconditionally and includes <bpf/btf.h> when HAVE_BPF_SKEL is defined.
Because $(LIBBPF) is removed from the prepare step, there is no longer a guarantee that libbpf headers are installed before the tests compile unless explicitly specified.
Does $(LIBPERF_TEST_IN) also need an explicit dependency on $(LIBBPF)?
Could a parallel build attempt to compile sigtrap.c before the concurrent $(LIBBPF) target finishes installing its headers into the include directory, causing an intermittent missing header regression during make -j builds?
Quoted 2 lines
tools/perf/Makefile.perf +0
-0$(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
Ian Rogers
reviewerPATCH v2 11/18 perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase
In reply to Ian RogersCurrently, BPF skeleton generation rules (bpf_skel.mak) are evaluated as part of util/Build. However, because LIBPERF_UTIL_IN explicitly depends on the top-level static libbpf archive, Make completely blocked the execution of bpftool bootstrap and skeleton generation until libbpf finished compiling midway through the build.
Since bpftool bootstrap compiles its own independent copy of libbpf.a, it does not depend on the top-level libbpf target.
Decouple early skeleton tooling generation by attaching bpf-skel-prepare to the umbrella prepare target, exporting CONFIG_PERF_BPF_SKEL to ensure accurate feature propagation. This allows Make to compile bpftool and dump vmlinux.h in the background at build startup, eliminating the initial sub-make startup bottleneck before BPF object compilation while keeping 100% of tooling rules perfectly encapsulated in bpf_skel.mak. Provide an empty fallback target to ensure builds succeed when BPF skeletons are disabled.
--- tools/perf/Makefile.perf | 6 +++++- tools/perf/bpf_skel.mak | 8 ++++++++ 2 files changed, 13 insertions(+), 1 deletion(-)
tools/perf/Makefile.perf +5 -1
@@ -275,7 +275,7 @@ ifeq ($(PYLINT),1) endif export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF READELF -export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT +export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT CONFIG_PERF_BPF_SKEL include $(srctree)/tools/build/Makefile.include
@@ -629,8 +629,12 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) +bpf-skel-prepare: + $(Q)$(MAKE) -f bpf_skel.mak bpf-skel-prepare + prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ + bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \
tools/perf/bpf_skel.mak +8 -0
@@ -16,6 +16,10 @@ BPFTOOL := $(SKEL_TOOL_TMP_OUT)/bootstrap/bpftool VMLINUX_H := $(SKEL_TOOL_OUT)/vmlinux.h bpf_skel_deps := $(BPFTOOL) $(VMLINUX_H) +.PHONY: bpf-skel-prepare +bpf-skel-prepare: $(bpf_skel_deps) + @: + define get_sys_includes $(shell $(1) $(2) -v -E - </dev/null 2>&1 \ | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
@@ -94,6 +98,10 @@ $(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o $(BPFTOOL) .PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o +else # CONFIG_PERF_BPF_SKEL +.PHONY: bpf-skel-prepare +bpf-skel-prepare: + @: endif # CONFIG_PERF_BPF_SKEL clean:
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 11/18] perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] The new `bpf-skel-prepare` target in `Makefile.perf` is a pseudo-target but is missing a `.PHONY`…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] The new bpf-skel-prepare target in Makefile.perf is a pseudo-target but is missing a .PHONY declaration. --
commit 13c9711193709348ce5b064c3bbf8b66453f9cb5
Author: Ian Rogers <[email protected]>
perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase
This commit decouples early skeleton tooling generation by attaching a bpf-skel-prepare target to the umbrella prepare phase. This allows Make to compile bpftool and dump vmlinux.h in the background at build startup.
Quoted 4 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
tools/perf/Makefile.perf endif · line 633
Quoted 6 lines
tools/perf/Makefile.perf +2
-0@@ -629,8 +629,12 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) +bpf-skel-prepare: + $(Q)$(MAKE) -f bpf_skel.mak bpf-skel-prepare
Should bpf-skel-prepare be marked as .PHONY in Makefile.perf?
Without a .PHONY declaration, if a file or directory named bpf-skel-prepare accidentally exists in the build directory, Make will consider the target up-to-date and silently skip it.
While bpf-skel-prepare is marked .PHONY inside bpf_skel.mak, the top-level Makefile.perf evaluates the target first before the child make is invoked.
Quoted 5 lines
tools/perf/Makefile.perf +2
-0+ prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ + bpf-skel-prepare \ $(LIBAPI) \
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Ian Rogers
reviewerPATCH v2 12/18 perf build: Move libsymbol dependency out of prepare step
In reply to Ian RogersThe prepare step is a large serialization point before parallel sub-makes build the perf tool. The libsymbol headers are used in the bench and util libraries. Move the libsymbol dependency out of the prepare step and into the dependencies for those targets to avoid it being a source of serialization in the prepare step.
--- tools/perf/Makefile.perf | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
tools/perf/Makefile.perf +3 -4
@@ -558,7 +558,7 @@ $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN) # The $(LIBPERF_UTIL) dependency is to ensure bpftool and vmlinux.h # aren't racily built for bench/bpf_skel/bench_uprobe.bpf.c -$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) | $(LIBPERF_UTIL) +$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) $(LIBSYMBOL) | $(LIBPERF_UTIL) $(Q)$(MAKE) $(build)=perf-bench $(LIBPERF_BENCH): $(LIBPERF_BENCH_IN)
@@ -576,7 +576,7 @@ $(LIBPERF_UI_IN): FORCE prepare $(LIBPERF_UI): $(LIBPERF_UI_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) +$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) $(LIBSYMBOL) $(Q)$(MAKE) $(build)=perf-util $(LIBPERF_UTIL): $(LIBPERF_UTIL_IN)
@@ -637,8 +637,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ - $(LIBSUBCMD) \ - $(LIBSYMBOL) + $(LIBSUBCMD) $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 13/18 perf build: Remove redundant libbpf feature check for static builds
In reply to Ian RogersBy default, the perf tool compiles and statically links against its own internal copy of libbpf (tools/lib/bpf/libbpf.a), setting LIBBPF_STATIC=1.
Despite this static linkage, Makefile.config unconditionally executed $(call feature_check,libbpf), which forced a synchronous sub-make fork during AST parsing to detect dynamic system libbpf libraries.
As noted in the internal Makefile comments, this check was executed purely so that running make VF=1 would display the detection status of system libbpf. During standard builds without LIBBPF_DYNAMIC=1, the detection result was entirely ignored.
Wrap the libbpf feature check inside LIBBPF_DYNAMIC=1 so Make avoids the redundant sub-make fork overhead during standard static builds.
--- tools/perf/Makefile.config | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
tools/perf/Makefile.config +1 -3
@@ -587,10 +587,8 @@ ifndef NO_LIBELF ifndef NO_LIBBPF ifeq ($(feature-bpf), 1) - # detecting libbpf without LIBBPF_DYNAMIC, so make VF=1 shows libbpf detection status - $(call feature_check,libbpf) - ifdef LIBBPF_DYNAMIC + $(call feature_check,libbpf) ifeq ($(feature-libbpf), 1) EXTLIBS += -lbpf CFLAGS += -DHAVE_LIBBPF_SUPPORT
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 14/18 perf pmu-events: Split big_c_string storage into standalone compilation unit
In reply to Ian RogersCurrently, jevents.py emits both the massive 2.8 MB big_c_string literal and tens of thousands of compact_pmu_event struct arrays into a single pmu-events.c compilation unit. Compiling this giant file takes ~2.2 seconds on a single CPU core during Kbuild startup.
Refactor jevents.py to emit big_c_string into a dedicated pmu-events-string.c compilation unit. This allows Kbuild to compile pmu-events.o and pmu-events-string.o simultaneously in parallel across two separate CPU cores, preserving 100% string deduplication and zero dynamic ELF relocations while cutting C compilation latency in half.
--- tools/perf/Makefile.perf | 4 ++-- tools/perf/pmu-events/Build | 15 +++++++++++++-- tools/perf/pmu-events/jevents.py | 20 ++++++++++++++++---- 3 files changed, 31 insertions(+), 8 deletions(-)
tools/perf/Makefile.perf +2 -2
@@ -921,7 +921,7 @@ bpf-skel-clean: pmu-events-clean: ifeq ($(OUTPUT),) $(call QUIET_CLEAN, pmu-events) $(RM) \ - pmu-events/pmu-events.c \ + pmu-events/pmu-events*.c \ pmu-events/metric_test.log \ pmu-events/test-empty-pmu-events.c \ pmu-events/empty-pmu-events.log
@@ -929,7 +929,7 @@ ifeq ($(OUTPUT),) -name 'extra-metricgroups.json' -delete else # When an OUTPUT directory is present, clean up the copied pmu-events/arch directory. $(call QUIET_CLEAN, pmu-events) $(RM) -r $(OUTPUT)pmu-events/arch \ - $(OUTPUT)pmu-events/pmu-events.c \ + $(OUTPUT)pmu-events/pmu-events*.c \ $(OUTPUT)pmu-events/metric_test.log \ $(OUTPUT)pmu-events/test-empty-pmu-events.c \ $(OUTPUT)pmu-events/empty-pmu-events.log
tools/perf/pmu-events/Build +13 -2
@@ -1,7 +1,12 @@ EMPTY_PMU_EVENTS_C = pmu-events/empty-pmu-events.c # pmu-events.c will be generated by jevents.py or copied from EMPTY_PMU_EVENTS_C PMU_EVENTS_C = $(OUTPUT)pmu-events/pmu-events.c +PMU_EVENTS_STRING_C = $(OUTPUT)pmu-events/pmu-events-string.c + pmu-events-y += pmu-events.o +ifneq ($(NO_JEVENTS),1) +pmu-events-y += pmu-events-string.o +endif # pmu-events.c file is generated in the OUTPUT directory so it needs a # separate rule to depend on it properly
@@ -9,6 +14,10 @@ $(OUTPUT)pmu-events/pmu-events.o: $(PMU_EVENTS_C) $(call rule_mkdir) $(call if_changed_dep,cc_o_c) +$(OUTPUT)pmu-events/pmu-events-string.o: $(PMU_EVENTS_STRING_C) + $(call rule_mkdir) + $(call if_changed_dep,cc_o_c) + # Message for $(call echo-cmd,cp), possibly remove the src file from # the destination to save space in the build log. quiet_cmd_cp = COPY $(patsubst %$<,%,$@) <- $<
@@ -118,6 +127,7 @@ CUR_OUT_JSON := $(shell [ -d $(OUT_DIR) ] && find $(OUT_DIR) -type f) # Things in the OUTPUT directory but shouldn't be there as computed by # OUT_JSON and GEN_JSON. + ORPHAN_FILES := $(filter-out $(OUT_JSON) $(GEN_JSON),$(CUR_OUT_JSON)) # Message for $(call echo-cmd,mkd). There is already a mkdir message
@@ -222,8 +232,9 @@ endif # Finally, the rule to build pmu-events.c using jevents.py. All test # and inputs are dependencies. -$(PMU_EVENTS_C): $(JEVENTS_DEPS) +$(PMU_EVENTS_STRING_C) $(PMU_EVENTS_C) &: $(JEVENTS_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) \ + $(OUT_DIR) $(PMU_EVENTS_C) $(PMU_EVENTS_STRING_C) endif # ifeq ($(NO_JEVENTS),1)
tools/perf/pmu-events/jevents.py +16 -4
@@ -1422,6 +1422,8 @@ such as "arm/cortex-a34".''', ) ap.add_argument( 'output_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=sys.stdout) + ap.add_argument( + 'output_string_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=None) _args = ap.parse_args() _args.output_file.write(f"""
@@ -1463,10 +1465,20 @@ struct pmu_table_entry { ftw(arch_path, [], preprocess_one_file) _bcs.compute() - _args.output_file.write('static const char *const big_c_string =\n') - for s in _bcs.big_string: - _args.output_file.write(s) - _args.output_file.write(';\n\n') + if not _args.output_string_file: + _args.output_file.write('static const char *const big_c_string =\n') + for s in _bcs.big_string: + _args.output_file.write(s) + _args.output_file.write(';\n\n') + else: + _args.output_string_file.write('/* SPDX-License-Identifier: GPL-2.0 */\n') + _args.output_string_file.write('/* Autogenerated by jevents.py */\n') + _args.output_string_file.write('const char big_c_string[] =\n') + for s in _bcs.big_string: + _args.output_string_file.write(s) + _args.output_string_file.write(';\n') + _args.output_string_file.close() + _args.output_file.write('extern const char big_c_string[];\n\n') for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], process_one_file)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 14/18] perf pmu-events: Split big_c_string storage into standalone compilation unit
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] The use of the `&:` grouped target syntax in the Makefile breaks compatibility with GNU Make < 4.3…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The use of the &: grouped target syntax in the Makefile breaks compatibility with GNU Make < 4.3 and causes race conditions during parallel builds.
- [Low] The newly generated pmu-events-string.c is not added to tools/perf/.gitignore, resulting in an untracked file after an in-tree build. --
commit e5e31a4f692737db305f31f8a7cee6c36037c141
Author: Ian Rogers <[email protected]>
perf pmu-events: Split big_c_string storage into standalone compilation unit
Currently, jevents.py emits both the massive 2.8 MB big_c_string literal and tens of thousands of compact_pmu_event struct arrays into a single pmu-events.c compilation unit. This commit refactors jevents.py to emit big_c_string into a dedicated pmu-events-string.c compilation unit. This allows Kbuild to compile them simultaneously in parallel, cutting C compilation latency.
tools/perf/pmu-events/Build line 5
Quoted 10 lines
tools/perf/pmu-events/Build +2
-0@@ -1,7 +1,12 @@ EMPTY_PMU_EVENTS_C = pmu-events/empty-pmu-events.c # pmu-events.c will be generated by jevents.py or copied from EMPTY_PMU_EVENTS_C PMU_EVENTS_C = $(OUTPUT)pmu-events/pmu-events.c +PMU_EVENTS_STRING_C = $(OUTPUT)pmu-events/pmu-events-string.c +
Does this new generated file need to be added to tools/perf/.gitignore?
Looking at tools/perf/.gitignore, pmu-events/pmu-events.c is explicitly ignored, but the new string file is not:
tools/perf/.gitignore:
...
pmu-events/pmu-events.c
pmu-events/jevents
pmu-events/metric_test.log
...Compiling perf in-tree might leave the working directory dirty with an untracked pmu-events-string.c file.
Quoted 7 lines
tools/perf/pmu-events/Build +3
-0pmu-events-y += pmu-events.o +ifneq ($(NO_JEVENTS),1) +pmu-events-y += pmu-events-string.o +endif # pmu-events.c file is generated in the OUTPUT directory so it needs a # separate rule to depend on it properly
[ ... ]
tools/perf/pmu-events/Build endif · line 236
Quoted 7 lines
tools/perf/pmu-events/Build +1
-1@@ -222,8 +232,9 @@ endif # Finally, the rule to build pmu-events.c using jevents.py. All test # and inputs are dependencies. -$(PMU_EVENTS_C): $(JEVENTS_DEPS) +$(PMU_EVENTS_STRING_C) $(PMU_EVENTS_C) &: $(JEVENTS_DEPS) $(call rule_mkdir)
Will this rule work correctly on systems using GNU Make older than 4.3?
The top-level Makefile enforces a minimum requirement of GNU Make 4.0, but the &: grouped target feature was introduced in GNU Make 4.3.
On older, supported versions of Make (like 4.2.1), the &: syntax is not recognized. Make parses & as a literal target and interprets the line as having three independent targets: $(PMU_EVENTS_STRING_C), $(PMU_EVENTS_C), and &.
During a parallel build, could this result in multiple jevents.py processes spawning simultaneously? It seems they might write to pmu-events.c and pmu-events-string.c concurrently, potentially interleaving output and corrupting the generated C source files.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
Ian Rogers
reviewerPATCH v2 15/18 perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
In reply to Ian RogersCurrently, jevents.py parses hundreds of JSON event and metric files sequentially across all CPU architectures during Kbuild startup, taking ~3.3 seconds of pure single-core execution time.
Refactor jevents.py to pre-populate its internal JSON AST cache in parallel across all available CPU cores using ProcessPoolExecutor. Configure worker process initializers to guarantee standard event mapping inheritance under spawn multiprocessing semantics, and apply depth filtering to prevent redundant worker tasks. This accelerates jevents.py execution by over 11x (from 3.3s down to ~290ms), fully reclaiming multi-core concurrency during the build generation phase.
--- tools/perf/pmu-events/jevents.py | 36 ++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-)
tools/perf/pmu-events/jevents.py +32 -4
@@ -457,8 +457,8 @@ class JsonEvent: return f'{{ { _bcs.offsets[s] } }}, /* {fix_comment(s)} */\n' -@lru_cache(maxsize=None) -def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: +_json_cache = {} +def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" try: events = json.load(open(path), object_hook=JsonEvent)
@@ -474,12 +474,16 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: if updates: for event in events: if event.metric_name in updates: - # print(f'Updated {event.metric_name} from\n"{event.metric_expr}"\n' - # f'to\n"{updates[event.metric_name]}"') event.metric_expr = updates[event.metric_name] return events +def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: + key = (path, topic) + if key not in _json_cache: + _json_cache[key] = _read_json_events_impl(path, topic) + return _json_cache[key] + def preprocess_arch_std_files(archpath: str) -> None: """Read in all architecture standard events.""" global _arch_std_events
@@ -1381,6 +1385,10 @@ const char *describe_metricgroup(const char *group) } """) +def _parallel_read_json_events(task: Tuple[str, str]) -> Tuple[str, str, Sequence[JsonEvent]]: + path, topic = task + return path, topic, _read_json_events_impl(path, topic) + def main() -> None: global _args
@@ -1459,9 +1467,29 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json) + + def _init_worker(std_events: dict) -> None: + global _arch_std_events + _arch_std_events = std_events + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor: + for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file) _bcs.compute()
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v2 15/18] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] The `_init_worker` function is defined as a local function, which breaks the build under `spawn`…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] The _init_worker function is defined as a local function, which breaks the build under spawn multiprocessing semantics because it cannot be pickled.
- [Medium] Parallelizing the JSON parsing breaks architecture isolation, allowing standard events from later architectures to overwrite those of earlier architectures.
- [Low] The collect_json callback omits the directory depth filter, leading to redundant parsing of deeply nested JSON files.
--
commit a8c22dabeaaa676d9d6b5a7b1c0a3df0bfcd2345
Author: Ian Rogers <[email protected]>
perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
This commit refactors jevents.py to pre-populate its internal JSON AST cache in parallel across all available CPU cores using ProcessPoolExecutor. It aims to accelerate the execution time by reclaiming multi-core concurrency during the build generation phase.
Quoted 3 lines
tools/perf/pmu-events/jevents.py +0
-0
[ ... ]
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1475
Quoted 10 lines
tools/perf/pmu-events/jevents.py +6
-0@@ -1459,9 +1467,29 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'):
The commit message states the code will "apply depth filtering to prevent redundant worker tasks".
Should there be a check on the depth of the directory structure here, to avoid redundantly parsing deeply nested JSON files?
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1481
Quoted 6 lines
tools/perf/pmu-events/jevents.py +3
-0+ tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json)
By preprocessing all architecture standard files before the worker pool evaluates the tasks, does this break architecture isolation?
Because archs.sort() evaluates architectures alphabetically, it appears that architectures processed later might overwrite overlapping standard event definitions of earlier architectures in the global _arch_std_events dictionary.
When the worker pool executes, will earlier architectures incorrectly resolve standard events against these overwritten definitions?
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1483
Quoted 2 lines
tools/perf/pmu-events/jevents.py +2
-0+ + def _init_worker(std_events: dict) -> None:
Is it possible to use a local function as an initializer for ProcessPoolExecutor when using the spawn multiprocessing start method?
The commit message notes it guarantees inheritance under spawn semantics. However, Python's multiprocessing framework uses pickle to serialize the initializer function for child processes, and local/nested functions cannot be pickled. Will this cause an AttributeError and crash the build?
Quoted 6 lines
tools/perf/pmu-events/jevents.py +6
-0+ global _arch_std_events + _arch_std_events = std_events + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor: + for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=15
Ian Rogers
reviewerPATCH v2 16/18 perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds
In reply to Ian RogersIn Makefile.perf, ALL_PROGRAMS includes SCRIPTS (perf-archive, perf-iostat). However, unlike PROGRAMS and DLFILTERS, SCRIPTS was not prefixed with $(OUTPUT).
During out-of-tree builds (or when O= is specified), Make checked for the unprefixed target 'tools/perf/perf-archive'. Since the actual script was installed into $(OUTPUT)perf-archive, Make concluded the target was missing and continuously re-executed the script installation rule on every single incremental build.
Prefix SCRIPTS with $(OUTPUT) and update the static pattern rule to ensure Kbuild correctly tracks generated script prerequisites during incremental builds.
--- tools/perf/Makefile.perf | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +3 -3
@@ -405,7 +405,7 @@ python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT # Use the detected configuration -include $(OUTPUT).config-detected -SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) +SCRIPTS = $(addprefix $(OUTPUT),$(patsubst %.sh,%,$(SCRIPT_SH))) PROGRAMS += $(OUTPUT)perf
@@ -592,8 +592,8 @@ $(GTK_IN): FORCE prepare $(OUTPUT)libperf-gtk.so: $(GTK_IN) $(PERFLIBS) $(QUIET_LINK)$(CC) -o $@ -shared $(LDFLAGS) $(filter %.o,$^) $(GTK_LIBS) -$(SCRIPTS) : % : %.sh - $(QUIET_GEN)$(INSTALL) '[email protected]' '$(OUTPUT)$@' +$(SCRIPTS) : $(OUTPUT)% : %.sh + $(QUIET_GEN)$(INSTALL) '$<' '$@' $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE $(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 17/18 perf pmu-events: Convert recursive shell assignments and macros to Make built-ins
In reply to Ian RogersIn pmu-events/Build, ZENS, ARMS, and INTELS were assigned using recursive assignment (=), and model_name/vendor_name were evaluated using shell macros (echo ... | sed ...).
Because these variables were expanded inside the COPY_RULE dependency evaluation loop across hundreds of PMU JSON files and inside every metric generation recipe, Kbuild continuously re-executed 'ls', 'grep', and 'sed' shell forks thousands of times during AST parsing and execution.
Convert ZENS, ARMS, and INTELS to simply expanded variables (:=) and replace model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once when the Build file is parsed and evaluates path macros purely in memory, completely eliminating over 7,800 redundant sub-processes during build startup.
--- tools/perf/pmu-events/Build | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/pmu-events/Build +5 -5
@@ -56,12 +56,12 @@ $(LEGACY_CACHE_JSON): $(LEGACY_CACHE_PY) # Python to generate architectural metrics GEN_METRIC_DEPS := pmu-events/metric.py pmu-events/common_metrics.py # Functions to extract the model from an extra-metrics.json or extra-metricgroups.json path. -model_name = $(shell echo $(1)|sed -e 's@.\+/\(.*\)/extra-metric.*\.json@\1@') -vendor_name = $(shell echo $(1)|sed -e 's@.\+/\(.*\)/[^/]*/extra-metric.*\.json@\1@') +model_name = $(notdir $(patsubst %/,%,$(dir $(1)))) +vendor_name = $(notdir $(patsubst %/,%,$(dir $(patsubst %/,%,$(dir $(1)))))) ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),x86 all)) # Generate AMD Json -ZENS = $(shell ls -d pmu-events/arch/x86/amdzen*) +ZENS := $(shell ls -d pmu-events/arch/x86/amdzen*) ZEN_METRICS = $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metrics.json) ZEN_METRICGROUPS = $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(ZEN_METRICS) $(ZEN_METRICGROUPS)
@@ -78,7 +78,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all)) # Generate ARM Json -ARMS = $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json) ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(ARM_METRICS) $(ARM_METRICGROUPS)
@@ -95,7 +95,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),x86 all)) # Generate Intel Json -INTELS = $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapfile.csv) +INTELS := $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapfile.csv) INTEL_METRICS = $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metrics.json) INTEL_METRICGROUPS = $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(INTEL_METRICS) $(INTEL_METRICGROUPS)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v2 18/18 perf build: Convert llvm-config shell queries to simply expanded variables
In reply to Ian RogersIn Makefile.config, CFLAGS, CXXFLAGS, LIBLLVM, and EXTLIBS were assigned using recursive expansion or appended with raw $(shell $(LLVM_CONFIG) ...) calls. Because these variables were expanded during dependency evaluation across every single object file compilation rule, Kbuild continuously re-executed llvm-config forks nearly 200 times during incremental builds.
Convert llvm-config shell queries to simply expanded variables (:=) to ensure Make evaluates LLVM compiler flags and library paths exactly once when Makefile.config is parsed, eliminating ~185 redundant sub-processes during build startup.
--- tools/perf/Makefile.config | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
tools/perf/Makefile.config +8 -5
@@ -951,11 +951,14 @@ ifndef NO_LIBLLVM $(call feature_check,llvm-perf) ifeq ($(feature-llvm-perf), 1) CFLAGS += -DHAVE_LIBLLVM_SUPPORT - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) - CXXFLAGS += -DHAVE_LIBLLVM_SUPPORT - CXXFLAGS += $(shell $(LLVM_CONFIG) --cxxflags) - LIBLLVM = $(shell $(LLVM_CONFIG) --libs all) $(shell $(LLVM_CONFIG) --system-libs) - EXTLIBS += -L$(shell $(LLVM_CONFIG) --libdir) $(LIBLLVM) + LLVM_CFLAGS := $(shell $(LLVM_CONFIG) --cflags 2>/dev/null) + LLVM_CXXFLAGS := $(shell $(LLVM_CONFIG) --cxxflags 2>/dev/null) + LLVM_LIBLLVM := $(shell $(LLVM_CONFIG) --libs all 2>/dev/null) $(shell $(LLVM_CONFIG) --system-libs 2>/dev/null) + LLVM_LIBDIR := $(shell $(LLVM_CONFIG) --libdir 2>/dev/null) + CFLAGS += $(LLVM_CFLAGS) + CXXFLAGS += -DHAVE_LIBLLVM_SUPPORT $(LLVM_CXXFLAGS) + LIBLLVM := $(LLVM_LIBLLVM) + EXTLIBS += -L$(LLVM_LIBDIR) $(LIBLLVM) EXTLIBS += -lstdc++ $(call detected,CONFIG_LIBLLVM) else
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerv3 cover perf build: Reduce build time by nearly half
In reply to Ian RogersThis patch series refactors Kbuild internals, BPF skeleton generation, Python AST pre-computation, and foundational tooling dependencies across the perf tool build system. By eliminating umbrella target synchronization barriers, decoupling static library prerequisites, parallelizing single-core script generators, and eradicating redundant feature checks, this series unlocks absolute theoretical peak multi-core concurrency during Kbuild startup.
On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 49%:
Before:
real 0m29.006s
user 2m46.019s
sys 0m30.610sAfter:
real 0m14.782s
user 2m39.527s
sys 0m22.938sSaving 14.2 full seconds time per clean build. Furthermore, nothing to build incremental builds are improved by nearly 7x:
Before:
real 0m11.528s
user 0m9.633s
sys 0m6.965sAfter:
real 0m1.729s
user 0m1.600s
sys 0m0.884sSummary of Patches:
1-3: Foundational Tooling & Fast-Path Feature Detection
- Exempts bpftool bootstrap from non-essential feature tests (LLVM, libbfd, libcap), saving 1.1s of sub-make fork overhead during Kbuild startup.
- Integrates libdebuginfod directly into test-all.c, allowing Make to skip individual feature check sub-make forks during AST parsing on fully configured workstations. Escapes $(shell ...) macro expansion to prevent unconditional sub-make forks.
- Fixes test-clang-bpf-co-re.bin feature check to correctly generate its target file on disk via atomic move (> [email protected] && mv [email protected] $@), allowing Kbuild to perfectly cache the detection result and avoid continuous sub-make re-evaluations.
4-6: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.
7-10: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dump vmlinux.h in the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
- Ensures benchmark skeleton intermediate .bpf.o files are cleanly removed during make clean, and adds bpf-skel-prepare to .PHONY.
11-12: Foundational Linkage Optimization
- Eliminates redundant libbpf sub-make feature checks during static builds.
- Moves static libsymbol and libbpf library prerequisites out of the prepare step, ensuring libbpf headers are installed before compiling BPF-dependent tests.
13-14: jevents.py Concurrency & Deduplication
- Splits the massive 2.8 MB big_c_string literal out of pmu-events.c into a dedicated pmu-events-string.c compilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations. Adds pmu-events-string.c to .gitignore and uses Make 4.0 compatible dependency chaining.
- Pre-populates jevents.py JSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms). Moves _init_worker to top-level scope to ensure clean pickling under spawn multiprocessing start methods.
15: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.
16-17: AST Parsing Optimization & Shell Fork Eradication
- Converts ZENS, ARMS, and INTELS in pmu-events/Build from recursive assignment (=) to simply expanded assignment (:=) and replaces model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once during AST parsing and evaluates path macros purely in memory, completely eradicating over 7,800 redundant sub-processes during out-of-tree build evaluation.
- Converts llvm-config shell queries in Makefile.config from recursive assignment (=) to simply expanded assignment (:=). This eliminates ~185 redundant sub-processes that were previously executed across object compilation dependency checks.
Changes since v2:
- Dropped Patch 4 (tools scripts: Short-circuit CC_NO_CLANG compiler probe in Makefile.include) to prevent potential cross-compilation regressions when CC and HOSTCC use different compilers.
- tools build (Patch 2): Escaped $(shell ...) macro expansion as $$(shell ...) inside define feature_check_code to safely defer sub-make execution until after eval parses the ifeq guard.
- tools build (Patch 3): Refactored test-clang-bpf-co-re.bin feature check recipe to redirect grep output to a temporary file and atomically move it upon success (> [email protected] && mv [email protected] $@), preventing Kbuild from permanently caching failed detections due to 0-byte files.
- perf trace beauty (Patch 4): Updated commit description to accurately reflect the unconditional top-level recursive kbuild hook (perf-util-y += trace/beauty/).
- perf build (Patch 7): Added $(OUTPUT)bench/bpf_skel/.tmp to bpf-skel-clean in Makefile.perf to ensure intermediate benchmark skeleton .bpf.o artifacts are cleanly removed during make clean. Removed unused bpf_skel_deps variable from bpf_skel.mak.
- perf build (Patch 9): Added $(LIBBPF) as an explicit prerequisite to $(LIBPERF_TEST_IN) in Makefile.perf to guarantee libbpf headers are fully installed before compiling sigtrap.c or other BPF-dependent tests during parallel builds.
- perf build (Patch 10): Added bpf-skel-prepare to the .PHONY target list in Makefile.perf to ensure Make never incorrectly skips the target if a file or directory named bpf-skel-prepare accidentally exists in the build tree.
- perf pmu-events (Patch 13): Added pmu-events/pmu-events-string.c to tools/perf/.gitignore. Replaced grouped targets (&:) with Make 4.0 compatible dependency chaining to guarantee backward compatibility with older Make versions (like 4.2.1) and prevent parallel builds from spawning multiple concurrent jevents.py processes.
- perf pmu-events (Patch 14): Moved _init_worker from local main() scope to the top-level module scope in jevents.py to ensure it can be cleanly pickled when ProcessPoolExecutor uses the spawn multiprocessing start method (avoiding AttributeError crashes).
Ian Rogers (17):
bpftool build: Restrict feature tests during bootstrap compilation tools build: Integrate libdebuginfod into test-all fast path tools build: Fix test-clang-bpf-co-re.bin to generate target file perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak perf build: Move static libbpf dependency out of prepare step perf build: Pre-generate BPF skeleton tooling during umbrella prepare
phase
perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds perf pmu-events: Split big_c_string storage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
perf pmu-events: Convert recursive shell assignments and macros to
Make built-ins
perf build: Convert llvm-config shell queries to simply expanded
variables
tools/bpf/bpftool/Makefile | 5 +
tools/build/Makefile.feature | 6 +-
tools/build/feature/Makefile | 4 +-
tools/build/feature/test-all.c | 5 +
tools/perf/.gitignore | 1 +
tools/perf/Build | 2 +
tools/perf/Makefile.config | 19 +-
tools/perf/Makefile.perf | 431 ++----------------
tools/perf/bench/Build | 6 +
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 109 +++++
tools/perf/builtin-trace.c | 30 +-
tools/perf/pmu-events/Build | 26 +-
tools/perf/pmu-events/jevents.py | 56 ++-
tools/perf/trace/beauty/Build | 280 ++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 +++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 6 +-
tools/perf/trace/beauty/futex_val3.c | 6 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 1 +
tools/perf/trace/beauty/perf_event_open.c | 22 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 22 +
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 17 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 +-
tools/perf/util/env.h | 1 +
41 files changed, 717 insertions(+), 504 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.mak
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 1/17 bpftool build: Restrict feature tests during bootstrap compilation
When bpf_skel.mak executes 'make -C ../bpf/bpftool bootstrap', bpftool's Makefile unconditionally evaluated feature checks for llvm, libcap, libbfd, and disassembler libraries because the bootstrap target was not exempted.
Since the bootstrap bpftool strictly compiles minimal AST parsing and C code generation logic without linking LLVM or disassembler libraries, these feature check sub-makes are completely redundant.
Exempt the bootstrap target from non-essential feature tests to eliminate unneeded sub-make fork overhead during Kbuild startup.
--- tools/bpf/bpftool/Makefile | 5 +++++ 1 file changed, 5 insertions(+)
tools/bpf/bpftool/Makefile +5 -0
@@ -106,6 +106,10 @@ ifneq ($(SKIP_CRYPTO),1) CRYPTO_LIBS := -lcrypto endif +ifeq ($(MAKECMDGOALS),bootstrap) +FEATURE_TESTS := libelf-zstd +FEATURE_DISPLAY := +else FEATURE_TESTS := clang-bpf-co-re FEATURE_TESTS += llvm FEATURE_TESTS += libcap
@@ -122,6 +126,7 @@ FEATURE_DISPLAY += libcap FEATURE_DISPLAY += libbfd FEATURE_DISPLAY += libbfd-liberty FEATURE_DISPLAY += libbfd-liberty-z +endif check_feat := 1 NON_CHECK_FEAT_TARGETS := clean uninstall doc doc-clean doc-install doc-uninstall
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 2/17 tools build: Integrate libdebuginfod into test-all fast path
In reply to Ian RogersBundling libdebuginfod into test-all.c allows Make to avoid an individual feature check sub-make fork during AST parsing. Short-circuit feature_check in Makefile.feature to instantly skip sub-make evaluations when a feature was already pre-detected by test-all.
Trade-offs:
On some Linux distributions (e.g., Debian/Ubuntu), libdebuginfod-dev is packaged separately from libelf-dev and libdw-dev. If a user installs libdw-dev but omits libdebuginfod-dev, adding -ldebuginfod to BUILD_ALL causes test-all to fail, falling back to individual feature tests. However, on fully configured developer workstations, this successfully reclaims sequential startup latency.
--- tools/build/Makefile.feature | 6 ++++-- tools/build/feature/Makefile | 2 +- tools/build/feature/test-all.c | 5 +++++ 3 files changed, 10 insertions(+), 3 deletions(-)
tools/build/Makefile.feature +4 -2
@@ -8,7 +8,9 @@ endif feature_check = $(eval $(feature_check_code)) define feature_check_code - feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + ifeq ($(feature-$(1)),) + feature-$(1) := $$(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + endif endef feature_set = $(eval $(feature_set_code))
@@ -100,6 +102,7 @@ FEATURE_TESTS_BASIC := \ disassembler-four-args \ disassembler-init-styled \ file-handle \ + libdebuginfod \ libopenssl # FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA is the complete list
@@ -125,7 +128,6 @@ FEATURE_TESTS_EXTRA := \ clang \ libbpf \ libpfm4 \ - libdebuginfod \ clang-bpf-co-re \ bpftool-skeletons
tools/build/feature/Makefile +1 -1
@@ -115,7 +115,7 @@ __BUILD = $(CC) $(CFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $( BUILD_BFD = $(BUILD) -DPACKAGE='"perf"' -lbfd -ldl BUILD_ALL = $(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -lslang \ $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -ldl -lz -llzma -lzstd \ - $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) + $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) -ldebuginfod __BUILDXX = $(CXX) $(CXXFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS) BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
tools/build/feature/test-all.c +5 -0
@@ -146,6 +146,10 @@# include "test-libopenssl.c" #undef main +#define main main_test_libdebuginfod +# include "test-libdebuginfod.c" +#undef main +int main (int argc,char *argv[]) {main_test_libpython ();
@@ -178,6 +182,7 @@ int main(int argc, char *argv[])main_test_libzstd ();main_test_libtraceevent ();main_test_libopenssl (); +main_test_libdebuginfod ();return 0 ; }
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v3 02/17] tools build: Integrate libdebuginfod into test-all fast path
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The use of `$$(shell ...)` with unescaped variables like `$(CC)` inside a block evaluated by…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The use of $$(shell ...) with unescaped variables like $(CC) inside a block evaluated by $(eval) causes a double-expansion bug that corrupts compiler flags containing $. --
commit 8e0d941d5a7133e71fe2afde9b8f70d691de8879
Author: Ian Rogers <[email protected]>
tools build: Integrate libdebuginfod into test-all fast path
This commit integrates libdebuginfod into the test-all fast path. It also updates feature_check_code to short-circuit the feature check and avoid a sub-make fork if a feature was already pre-detected.
tools/build/Makefile.feature endif · line 12
Quoted 11 lines
tools/build/Makefile.feature +2
-1@@ -8,7 +8,9 @@ endif feature_check = $(eval $(feature_check_code)) define feature_check_code - feature-$(1) := $(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0) + ifeq ($(feature-$(1)),) + feature-$(1) := $$(shell $(MAKE) OUTPUT=$(OUTPUT_FEATURES) CC="$(CC)" CXX="$(CXX)" CFLAGS="$(EXTRA_CFLAGS) $(FEATURE_CHECK_CFLAGS-$(1))" CXXFLAGS="$(EXTRA_CXXFLAGS) $(FEATURE_CHECK_CXXFLAGS-$(1))" LDFLAGS="$(LDFLAGS) $(FEATURE_CHECK_LDFLAGS-$(1))" -C $(feature_dir) $(OUTPUT_FEATURES)test-$1.bin >/dev/null 2>/dev/null && echo 1 || echo 0)
Does leaving variables like $(CC), $(EXTRA_CFLAGS), and $(LDFLAGS) unescaped inside the $$(shell ...) block cause a double-expansion issue?
When eval parses the assignment, these variables are expanded in the first pass, embedding their literal values into the string.
During the second pass, when the shell function is evaluated, Make scans the embedded string again. If any compiler or linker flags contain a literal $ (such as -Wl,-rpath=$$ORIGIN), they will be incorrectly expanded during this second pass (for example, replacing $O with the value of the output directory variable O), corrupting the flags.
Quoted 2 lines
tools/build/Makefile.feature +1
-0+ endif endef
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Ian Rogers
reviewerPATCH v3 3/17 tools build: Fix test-clang-bpf-co-re.bin to generate target file
In reply to Ian RogersIn tools/build/feature/Makefile, the test-clang-bpf-co-re.bin feature check compiled test-clang-bpf-co-re.c assembly on stdout and piped it to grep, but never redirected the matched output to create the target binary () on disk.
Because the target file was never created, Kbuild could never cache the result of the check. Consequently, Make treated the prerequisite as missing and continuously re-executed the Clang BPF backend compilation check on every single sub-make evaluation during build startup.
Redirect grep output to to guarantee the file is created on disk upon success, allowing Kbuild to perfectly cache the detection result and avoid redundant sub-make forks.
--- tools/build/feature/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/build/feature/Makefile +1 -1
@@ -385,7 +385,7 @@ $(OUTPUT)test-libzstd.bin: $(OUTPUT)test-clang-bpf-co-re.bin: $(CLANG) -S -g --target=bpf -o - $(patsubst %.bin,%.c,$(@F)) | \ - grep BTF_KIND_VAR + grep BTF_KIND_VAR > [email protected] && mv [email protected] $@ $(OUTPUT)test-file-handle.bin: $(BUILD)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v3 03/17] tools build: Fix test-clang-bpf-co-re.bin to generate target file
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [Medium] Broken dependency tracking for `test-clang-bpf-co-re.bin` due to missing `.d` generation and…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [Medium] Broken dependency tracking for test-clang-bpf-co-re.bin due to missing .d generation and prerequisites.
- [Low] Incomplete implementation: test-bpftool-skeletons.bin suffers from the identical missing target file flaw. --
commit 57e84770100052aa7d6b943326d9f33e586ed45a
Author: Ian Rogers <[email protected]>
tools build: Fix test-clang-bpf-co-re.bin to generate target file
This commit fixes the test-clang-bpf-co-re.bin feature check to ensure the target file is created on disk upon success. This allows Kbuild to cache the detection result and avoids redundant sub-make forks.
Quoted 4 lines
Because the target file was never created,
Kbuildcould never cache the result of the check. Consequently, Make treated the prerequisite as missing and continuously re-executed the Clang BPF backend compilation check on every single sub-make evaluation during build startup.
The commit message identifies that failing to create the target file breaks Make's caching. In the same Makefile, test-bpftool-skeletons.bin redirects its output but never creates the $@ target file:
$(OUTPUT)test-bpftool-skeletons.bin:
$(SYSTEM_BPFTOOL) version | grep '^features:.*skeletons' \ > $(@:.bin=.make.output) 2>&1
Should test-bpftool-skeletons.bin also be updated to ensure the target file is touched upon success, to prevent the same redundant sub-make fork overhead?
tools/build/feature/Makefile $(OUTPUT)test-libzstd.bin: · line 388
Quoted 10 lines
tools/build/feature/Makefile +1
-1@@ -385,7 +385,7 @@ $(OUTPUT)test-libzstd.bin: $(OUTPUT)test-clang-bpf-co-re.bin: $(CLANG) -S -g --target=bpf -o - $(patsubst %.bin,%.c,$(@F)) | \ - grep BTF_KIND_VAR + grep BTF_KIND_VAR > [email protected] && mv [email protected] $@
Since this manual $(CLANG) invocation doesn't use the -MD flag (unlike standard targets using $(BUILD)), no .d dependency file is generated. Additionally, there is no explicit prerequisite on the corresponding .c source file.
Now that the target file is created on disk, will Make consider it up-to-date indefinitely and silently ignore future modifications to test-clang-bpf-co-re.c during incremental builds?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Ian Rogers
reviewerPATCH v3 4/17 perf trace beauty: Make beauty generated C code standalone .o files
In reply to Ian RogersPreviously, builtin-trace.c directly included 15 embedded C files (e.g. trace/beauty/mmap.c and fsconfig_arrays.c), which in turn depend on dozens of generated beauty script arrays. To satisfy these embedded inclusions, the global Makefile.perf would define all the generator variables/rules and include them in the massive sequential prepare umbrella target, choking parallel build startup.
Furthermore, tools/perf/util/syscalltbl.c included its own generated mapper, and util/env.c conditionally included arch_errno_names.c inline, splitting consumers across directories and preventing clean Make encapsulation.
Refactor the framework to achieve better encapsulation:
1. Move util/syscalltbl.[ch] into trace/beauty/ to co-locate with all generated code consumers.
2. Create fsconfig.c and flatten embedded beauty .c files to compile as independent standalone objects via trace/beauty/Build, exporting their formatting functions via beauty.h and env.h. Switch arch_errno_names.o and syscalltbl.o assignments directly to perf-util-y and add an unconditional top-level recursive kbuild hook (perf-util-y += trace/beauty/) to compile them into libperf-util.a, resolving remote linkage for util/env.c, util/bpf-trace-summary.c, and standalone python extensions.
3. Bridge private opaque references (struct trace) securely via accessors trace__show_zeros() and trace__host(), avoiding header entanglements.
4. Consolidate all generator variables, script paths, and array generation rules entirely out of Makefile.perf and place them directly inside the exact local Build files where their output objects are compiled (trace/beauty/Build and trace/beauty/tracepoints/Build), binding prerequisites locally. Use directly inside
generator recipes to guarantee dynamic directory creation before script redirection, and append across all rules to print
clean, standardized GEN ... file.c output during compilation.
5. Clean up clean target to recursively remove the generated directory instead of relying on dozens of individual variables.
This unchokes the "prepare" target parallel barrier, allows make to evaluate generation scripts purely locally where consumed, and flattens the tracepoint formatting architecture.
Testing a parallel build (make -j28 all from scratch) shows improvements:
Before:
real 0m28.689s
user 2m38.490s
sys 0m30.148sAfter:
real 0m27.642s
user 2m32.356s
sys 0m26.683sSo reclaiming ~9.6 seconds of raw CPU time and over 1 full second off overall real-world build latency, by overlapping sub-make startup and avoiding top-level double-parsing overhead.
---
tools/perf/Build | 2 +
tools/perf/Makefile.perf | 282 +-----------------
tools/perf/builtin-trace.c | 30 +-
tools/perf/trace/beauty/Build | 280 +++++++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 ++++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 6 +-
tools/perf/trace/beauty/futex_val3.c | 6 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 1 +
tools/perf/trace/beauty/perf_event_open.c | 22 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 22 ++
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 4 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 +-
tools/perf/util/env.h | 1 +
29 files changed, 468 insertions(+), 352 deletions(-)
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)tools/perf/Build +2 -0
@@ -34,6 +34,8 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-$(CONFIG_TRACE) += trace/beauty/ endif +perf-util-y += trace/beauty/ + perf-$(CONFIG_LIBELF) += builtin-probe.o perf-bench-y += bench/
tools/perf/Makefile.perf +2 -280
300 lines not on this page · raw message
tools/perf/builtin-trace.c +13 -17
@@ -60,12 +60,13 @@#include "callchain.h" #include "print_binary.h" #include "string2.h" -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "../perf.h" #include "trace_augment.h" #include "dwarf-regs.h" #include <errno.h> +#include <sys/stat.h> #include <inttypes.h> #include <poll.h> #include <signal.h>
@@ -234,6 +235,16 @@ struct trace {const char *uid_str; }; +bool trace__show_zeros (const struct trace *trace) +{ +return trace->show_zeros; +} + +struct machine *trace__host (const struct trace *trace) +{ +return trace->host; +} +static void trace__load_vmlinux_btf (struct trace *trace__maybe_unused ) {#ifdef HAVE_LIBBPF_SUPPORT
@@ -776,9 +787,7 @@ static const char *fsmount_flags[] = { }; static DEFINE_STRARRAY(fsmount_flags, "FSMOUNT_"); -#include "trace/beauty/generated/fsconfig_arrays.c" -static DEFINE_STRARRAY(fsconfig_cmds, "FSCONFIG_");static const char *epoll_ctl_ops[] = {"ADD" ,"DEL" ,"MOD" , }; static DEFINE_STRARRAY_OFFSET(epoll_ctl_ops, "EPOLL_CTL_", 1);
@@ -1128,20 +1137,7 @@ static bool syscall_arg__strtoul_btf_type(char *bf __maybe_unused, size_t size _ .parm = &strarray__##array, \ .show_zero = true, } -#include "trace/beauty/eventfd.c" -#include "trace/beauty/futex_op.c" -#include "trace/beauty/futex_val3.c" -#include "trace/beauty/mmap.c" -#include "trace/beauty/mode_t.c" -#include "trace/beauty/msg_flags.c" -#include "trace/beauty/open_flags.c" -#include "trace/beauty/perf_event_open.c" -#include "trace/beauty/pid.c" -#include "trace/beauty/sched_policy.c" -#include "trace/beauty/seccomp.c" -#include "trace/beauty/signum.c" -#include "trace/beauty/socket_type.c" -#include "trace/beauty/waitid_options.c" +static const struct syscall_fmt syscall_fmts[] = { { .name ="access" ,
tools/perf/trace/beauty/Build +280 -0
289 lines not on this page · raw message
tools/perf/trace/beauty/arch_errno_names.c +2 -0
@@ -1 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "util/env.h" #include "trace/beauty/generated/arch_errno_name_array.c"
tools/perf/trace/beauty/arch_errno_names.sh +1 -1
@@ -57,7 +57,7 @@ create_arch_errno_table_func() archlist="$1" default="$2" - printf 'static arch_syscalls__strerrno_t *\n' + printf 'arch_syscalls__strerrno_t *\n' printf 'arch_syscalls__strerrno_function(const char *arch)\n' printf '{\n' for arch in $archlist; do
tools/perf/trace/beauty/beauty.h +60 -0
70 lines not on this page · raw message
tools/perf/trace/beauty/eventfd.c +4 -2
@@ -11,7 +11,9 @@#define EFD_CLOEXEC 02000000#endif -static size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="EFD_" ;
@@ -36,4 +38,4 @@ static size_t syscall_arg__scnprintf_eventfd_flags(char *bf, size_t size, structreturn printed; } -#define SCA_EFD_FLAGS syscall_arg__scnprintf_eventfd_flags +
tools/perf/trace/beauty/fsconfig.c new file +5 -0
@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "trace/beauty/generated/fsconfig_arrays.c" + +DEFINE_STRARRAY (fsconfig_cmds,"FSCONFIG_" );
tools/perf/trace/beauty/futex_op.c +4 -2
@@ -17,7 +17,9 @@#define FUTEX_CLOCK_REALTIME 256#endif -static size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="FUTEX_" ;
@@ -60,4 +62,4 @@ static size_t syscall_arg__scnprintf_futex_op(char *bf, size_t size, struct syscreturn printed; } -#define SCA_FUTEX_OP syscall_arg__scnprintf_futex_op +
tools/perf/trace/beauty/futex_val3.c +4 -2
@@ -5,7 +5,9 @@#define FUTEX_BITSET_MATCH_ANY 0xffffffff#endif -static size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) {const char *prefix ="FUTEX_BITSET_" ;unsigned int bitset = arg->val;
@@ -16,4 +18,4 @@ static size_t syscall_arg__scnprintf_futex_val3(char *bf, size_t size, struct syreturn scnprintf (bf, size,"%#xd" , bitset); } -#define SCA_FUTEX_VAL3 syscall_arg__scnprintf_futex_val3 +
tools/perf/trace/beauty/mmap.c +12 -12
@@ -1,5 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 #include <linux/log2.h> +#include "trace/beauty/beauty.h" #include "trace/beauty/generated/mmap_prot_array.c" static DEFINE_STRARRAY(mmap_prot, "PROT_");
@@ -8,8 +9,7 @@ static size_t mmap__scnprintf_prot(unsigned long prot, char *bf, size_t size, bo {return strarray__scnprintf_flags (&strarray__mmap_prot, bf, size, show_prefix, prot); } - -static size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) {unsigned long prot = arg->val;
@@ -19,18 +19,18 @@ static size_t syscall_arg__scnprintf_mmap_prot(char *bf, size_t size, struct sysreturn mmap__scnprintf_prot (prot, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_PROT syscall_arg__scnprintf_mmap_prot +#include "trace/beauty/generated/mmap_flags_array.c" -static DEFINE_STRARRAY(mmap_flags, "MAP_"); +DEFINE_STRARRAY (mmap_flags,"MAP_" );static size_t mmap__scnprintf_flags (unsigned long flags,char *bf,size_t size,bool show_prefix) {return strarray__scnprintf_flags (&strarray__mmap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, +struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -40,7 +40,7 @@ static size_t syscall_arg__scnprintf_mmap_flags(char *bf, size_t size,return mmap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_FLAGS syscall_arg__scnprintf_mmap_flags +#include "trace/beauty/generated/mremap_flags_array.c" static DEFINE_STRARRAY(mremap_flags, "MREMAP_");
@@ -50,7 +50,7 @@ static size_t mremap__scnprintf_flags(unsigned long flags, char *bf, size_t sizereturn strarray__scnprintf_flags (&strarray__mremap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -60,7 +60,7 @@ static size_t syscall_arg__scnprintf_mremap_flags(char *bf, size_t size, structreturn mremap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MREMAP_FLAGS syscall_arg__scnprintf_mremap_flags +static size_t madvise__scnprintf_behavior (int behavior,char *bf,size_t size) {
@@ -73,10 +73,10 @@ static size_t madvise__scnprintf_behavior(int behavior, char *bf, size_t size)return scnprintf (bf, size,"%#" , behavior); } -static size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, +struct syscall_arg *arg) {return madvise__scnprintf_behavior (arg->val, bf, size); } -#define SCA_MADV_BHV syscall_arg__scnprintf_madvise_behavior +
tools/perf/trace/beauty/mode_t.c +4 -2
@@ -20,7 +20,9 @@#define S_IXUGO (S_IXUSR|S_IXGRP|S_IXOTH)#endif -static size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="S_" ;
@@ -68,4 +70,4 @@ static size_t syscall_arg__scnprintf_mode_t(char *bf, size_t size, struct syscalreturn printed; } -#define SCA_MODE_T syscall_arg__scnprintf_mode_t +
tools/perf/trace/beauty/msg_flags.c +5 -3
@@ -27,8 +27,10 @@# define MSG_CMSG_CLOEXEC 0x40000000#endif -static size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="MSG_" ;
@@ -73,4 +75,4 @@ static size_t syscall_arg__scnprintf_msg_flags(char *bf, size_t size,return printed; } -#define SCA_MSG_FLAGS syscall_arg__scnprintf_msg_flags +
tools/perf/trace/beauty/open_flags.c +1 -0
@@ -2,6 +2,7 @@#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> +#include "trace/beauty/beauty.h" #ifndef O_DIRECT #define O_DIRECT 00040000
tools/perf/trace/beauty/perf_event_open.c +12 -10
@@ -1,4 +1,7 @@// SPDX-License-Identifier: LGPL-2.1 +#include <linux/perf_event.h> +#include "util/evsel_fprintf.h" +#ifndef PERF_FLAG_FD_NO_GROUP # define PERF_FLAG_FD_NO_GROUP (1UL << 0)#endif
@@ -15,8 +18,10 @@# define PERF_FLAG_FD_CLOEXEC (1UL << 3)/* O_CLOEXEC */ #endif -static size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="PERF_" ;
@@ -43,7 +48,7 @@ static size_t syscall_arg__scnprintf_perf_flags(char *bf, size_t size,return printed; } -#define SCA_PERF_FLAGS syscall_arg__scnprintf_perf_flags +struct attr_fprintf_args {size_t size, printed;
@@ -76,10 +81,11 @@ static size_t perf_event_attr___scnprintf(struct perf_event_attr *attr, char *bfstatic size_t syscall_arg__scnprintf_augmented_perf_event_attr (struct syscall_arg *arg,char *bf,size_t size) { -return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, arg->trace->show_zeros); +return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, +trace__show_zeros (arg->trace)); } -static size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) {if (arg->augmented.args)return syscall_arg__scnprintf_augmented_perf_event_attr (arg, bf, size);
@@ -87,8 +93,4 @@ static size_t syscall_arg__scnprintf_perf_event_attr(char *bf, size_t size, strureturn scnprintf (bf, size,"%#lx" , arg->val); } -#define SCA_PERF_ATTR syscall_arg__scnprintf_perf_event_attr -// 'argname' is just documentational at this point, to remove the previous comment with that info -#define SCA_PERF_ATTR_FROM_USER (argname) \ - { .scnprintf = SCA_PERF_ATTR, \ - .from_user = true, } +
tools/perf/trace/beauty/pid.c +4 -1
@@ -1,11 +1,14 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "util/machine.h" +#include "util/thread.h" size_t syscall_arg__scnprintf_pid (char *bf,size_t size,struct syscall_arg *arg) {int pid = arg->val;struct trace *trace = arg->trace;size_t printed =scnprintf (bf, size,"%d" , pid); -struct thread *thread =machine__findnew_thread (trace->host, pid, pid); +struct thread *thread =machine__findnew_thread (trace__host (trace), pid, pid);if (thread !=NULL ) {if (!thread__comm_set (thread))
tools/perf/trace/beauty/sched_policy.c +5 -3
@@ -14,8 +14,10 @@#define SCHED_RESET_ON_FORK 0x40000000#endif -static size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SCHED_" ;
@@ -47,4 +49,4 @@ static size_t syscall_arg__scnprintf_sched_policy(char *bf, size_t size,return printed; } -#define SCA_SCHED_POLICY syscall_arg__scnprintf_sched_policy +
tools/perf/trace/beauty/seccomp.c +7 -5
@@ -6,7 +6,9 @@#define SECCOMP_SET_MODE_FILTER 1#endif -static size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_SET_MODE_" ;
@@ -24,14 +26,14 @@ static size_t syscall_arg__scnprintf_seccomp_op(char *bf, size_t size, struct syreturn printed; } -#define SCA_SECCOMP_OP syscall_arg__scnprintf_seccomp_op +#ifndef SECCOMP_FILTER_FLAG_TSYNC #define SECCOMP_FILTER_FLAG_TSYNC 1#endif -static size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_FILTER_FLAG_" ;
@@ -52,4 +54,4 @@ static size_t syscall_arg__scnprintf_seccomp_flags(char *bf, size_t size,return printed; } -#define SCA_SECCOMP_FLAGS syscall_arg__scnprintf_seccomp_flags +
tools/perf/trace/beauty/signum.c +4 -2
@@ -1,7 +1,9 @@// SPDX-License-Identifier: LGPL-2.1 #include <signal.h> -static size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SIG" ;
@@ -54,4 +56,4 @@ static size_t syscall_arg__scnprintf_signum(char *bf, size_t size, struct syscalreturn scnprintf (bf, size,"%#x" , sig); } -#define SCA_SIGNUM syscall_arg__scnprintf_signum +
tools/perf/trace/beauty/socket_type.c +4 -2
@@ -18,7 +18,9 @@#define SOCK_TYPE_MASK 0xf#endif -static size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SOCK_" ;
@@ -60,4 +62,4 @@ static size_t syscall_arg__scnprintf_socket_type(char *bf, size_t size, struct sreturn printed; } -#define SCA_SK_TYPE syscall_arg__scnprintf_socket_type +
tools/perf/trace/beauty/syscalltbl.c renamed from tools/perf/util/syscalltbl.c +0 -0
tools/perf/trace/beauty/syscalltbl.h renamed from tools/perf/util/syscalltbl.h +0 -0
tools/perf/trace/beauty/tracepoints/Build +22 -0
@@ -1,2 +1,24 @@ perf-y += x86_irq_vectors.o perf-y += x86_msr.o + +# --- Local Encapsulated Tracepoint Rules --- +beauty_outdir := $(OUTPUT)trace/beauty/generated +beauty_arch_asm_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/asm/ +x86_arch_asm_dir := $(srctree)/tools/arch/x86/include/asm/ + +x86_arch_irq_vectors_array := $(beauty_outdir)/x86_arch_irq_vectors_array.c +x86_arch_irq_vectors_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_irq_vectors.sh + +$(x86_arch_irq_vectors_array): $(beauty_arch_asm_dir)/irq_vectors.h $(x86_arch_irq_vectors_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_irq_vectors_tbl)' $(beauty_arch_asm_dir) > $@ + +x86_arch_MSRs_array := $(beauty_outdir)/x86_arch_MSRs_array.c +x86_arch_MSRs_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_msr.sh + +$(x86_arch_MSRs_array): $(x86_arch_asm_dir)/msr-index.h $(x86_arch_MSRs_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_MSRs_tbl)' $(x86_arch_asm_dir) > $@ + +$(OUTPUT)trace/beauty/tracepoints/x86_irq_vectors.o: $(x86_arch_irq_vectors_array) +$(OUTPUT)trace/beauty/tracepoints/x86_msr.o: $(x86_arch_MSRs_array)
tools/perf/trace/beauty/waitid_options.c +5 -3
@@ -2,8 +2,10 @@#include <sys/types.h> #include <sys/wait.h> -static size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, -struct syscall_arg *arg) +#include "trace/beauty/beauty.h" + +size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="W" ;
@@ -26,4 +28,4 @@ static size_t syscall_arg__scnprintf_waitid_options(char *bf, size_t size,return printed; } -#define SCA_WAITID_OPTIONS syscall_arg__scnprintf_waitid_options +
tools/perf/util/Build +3 -1
@@ -75,7 +75,7 @@ perf-util-y += sample.o perf-util-y += sample-raw.o perf-util-y += s390-sample-raw.o perf-util-y += amd-sample-raw.o -perf-util-$(CONFIG_TRACE) += syscalltbl.o + perf-util-y += ordered-events.o perf-util-y += namespaces.o perf-util-y += comm.o
@@ -441,3 +441,5 @@ $(OUTPUT)%.pylint_log: % $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false) perf-util-y += $(PYLINT_TEST_LOGS) + +
tools/perf/util/bpf-trace-summary.c +1 -1
@@ -6,7 +6,7 @@#include <stdlib.h> #include "dwarf-regs.h" /* for EM_HOST */ -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "util/cgroup.h" #include "util/hashmap.h" #include "util/trace.h"
tools/perf/util/env.c +1 -3
@@ -635,9 +635,7 @@ const char *perf_env__arch(struct perf_env *env)return normalize_arch (arch_name); } -#if defined(HAVE_LIBTRACEEVENT ) -#include "trace/beauty/arch_errno_names.c" -#endif +const char *perf_env__arch_strerrno (struct perf_env *env__maybe_unused ,int err__maybe_unused ) {
tools/perf/util/env.h +1 -0
@@ -189,6 +189,7 @@ void cpu_cache_level__free(struct cpu_cache_level *cache);const char *perf_env__arch (struct perf_env *env);const char *perf_env__arch_strerrno (struct perf_env *env,int err); +arch_syscalls__strerrno_t *arch_syscalls__strerrno_function (const char *arch);const char *perf_env__cpuid (struct perf_env *env);const char *perf_env__raw_arch (struct perf_env *env);int perf_env__nr_cpus_avail (struct perf_env *env);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 5/17 perf build: Decouple pmu-events from prepare umbrella target
In reply to Ian RogersCurrently, the $(LIBPMU_EVENTS_IN) sub-make depends on the massive "prepare" umbrella target. Because "prepare" depends on external libraries (libapi, libperf, etc.) as well as dozens of generated headers, make completely serializes the launch of the pmu-events sub-make behind some of those unrelated prerequisites.
Since pmu-events is a large compilation unit, unblock its startup by binding it directly to only $(LIBPERF) instead of prepare. This allows background python generation scripts to overlap simultaneously with the rest of the build.
Testing a parallel build (make -j28 clean all) shows improvements:
Before:
real 0m27.642s
user 2m32.356s
sys 0m26.683sAfter:
real 0m22.254s
user 2m32.810s
sys 0m24.646sThis reclaims over 5 full seconds of build latency (~19.5% overall reduction) by elevating average CPU concurrency from ~5.5 active cores up to ~8 active cores.
--- tools/perf/Makefile.perf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/Makefile.perf +1 -1
@@ -550,7 +550,7 @@ build := -f $(srctree)/tools/build/Makefile.build dir=. obj $(PERF_IN): prepare FORCE $(Q)$(MAKE) $(build)=perf -$(LIBPMU_EVENTS_IN): FORCE prepare +$(LIBPMU_EVENTS_IN): FORCE $(LIBPERF) $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 6/17 perf build: Remove empty archheaders target
In reply to Ian RogersRemove empty target that doesn't do anything.
--- tools/perf/Makefile.perf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/Makefile.perf +2 -2
@@ -627,7 +627,7 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) -prepare: $(OUTPUT)PERF-VERSION-FILE archheaders \ +prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ $(LIBAPI) \ $(LIBPERF) \
@@ -1070,7 +1070,7 @@ FORCE: .PHONY: all install clean config-clean strip install-gtk .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell .PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare -.PHONY: archheaders python_perf_target +.PHONY: python_perf_target endif # force_fixdep
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 7/17 perf build: Move BPF skeleton generation out of Makefile.perf
In reply to Ian RogersCurrently, the top-level Makefile.perf defines a massive global bpf-skel umbrella target that pre-compiles all 12+ BPF skeletons (%.skel.h) upfront before launching sub-makes. This forces unrelated sub-makes to serialize behind bpftool and clang BPF target evaluations, causing parallel build bottlenecks.
Furthermore, bench_uprobe.bpf.c lived inside util/bpf_skel/, breaking conceptual directory encapsulation since it is consumed purely by bench/uprobe.c.
Refactor the BPF skeletons to better achieve directory isolation:
1. Move tools/perf/util/bpf_skel/bench_uprobe.bpf.c directly into tools/perf/bench/bpf_skel/.
2. Extract the skeleton generation infrastructure out of Makefile.perf into a shared inclusion file tools/perf/bpf_skel.mak.
3. Include bpf_skel.mak locally inside tools/perf/util/Build and tools/perf/bench/Build and bind precise local prerequisites.
4. Safely synchronize the shared bpftool bootstrap and vmlinux.h targets via the conditional prepare: umbrella to avoid parallel sub-make races, while evaluating the actual skeletons completely locally on demand. A later patch will move these targets into bpf_skel.mak.
5. Export CLANG from the global Makefile to ensure accurate tool propagation.
6. Clean up Makefile.perf by stripping the global bpf-skel umbrella target and its SKELETONS list.
While removing code from Makefile.perf generally helps build performance, the impact here is minimal. The main motivation for the change is to better encapsulate things in the build and simplify Makefile.perf that has around 50 lines removed.
---
tools/perf/Makefile.perf | 59 ++-----------------
tools/perf/bench/Build | 6 ++
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 54 +++++++++++++++++
tools/perf/util/Build | 15 ++++-
6 files changed, 79 insertions(+), 57 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.maktools/perf/Makefile.perf +4 -55
@@ -274,7 +274,7 @@ ifeq ($(PYLINT),1) PYLINT := $(shell which pylint 2> /dev/null) endif -export srctree OUTPUT RM CC CXX RUSTC LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK +export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT include $(srctree)/tools/build/Makefile.include
@@ -632,8 +632,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \ - $(LIBSYMBOL) \ - bpf-skel + $(LIBSYMBOL) ifdef LIBBPF_STATIC prepare: $(LIBBPF)
@@ -914,44 +913,13 @@ python-clean: SKEL_OUT := $(abspath $(OUTPUT)util/bpf_skel) SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) -SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h -SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h -SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h -SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h -SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h $(SKEL_OUT)/sample_filter.skel.h -SKELETONS += $(SKEL_OUT)/kwork_top.skel.h $(SKEL_OUT)/syscall_summary.skel.h -SKELETONS += $(SKEL_OUT)/bench_uprobe.skel.h -SKELETONS += $(SKEL_OUT)/augmented_raw_syscalls.skel.h $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT) $(LIBSYMBOL_OUTPUT): $(Q)$(MKDIR) -p $@ ifeq ($(CONFIG_PERF_BPF_SKEL),y) +prepare: $(BPFTOOL) $(SKEL_OUT)/vmlinux.h BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool -# Get Clang's default includes on this system, as opposed to those seen by -# '--target=bpf'. This fixes "missing" files on some architectures/distros, -# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc. -# -# Use '-idirafter': Don't interfere with include mechanics except where the -# build would have failed anyways. -define get_sys_includes -$(shell $(1) $(2) -v -E - </dev/null 2>&1 \ - | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \ -$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}') -endef - -ifneq ($(CROSS_COMPILE),) -CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%)) -endif - -CLANG_OPTIONS = -Wall -CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH)) -BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES) -TOOLS_UAPI_INCLUDE := -I$(srctree)/tools/include/uapi - -ifneq ($(WERROR),0) - CLANG_OPTIONS += -Werror -endif $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
@@ -993,29 +961,10 @@ else $(Q)cp "$(VMLINUX_H)" $@ endif -$(SKEL_TMP_OUT)/%.bpf.o: $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h | $(SKEL_TMP_OUT) -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h - $(QUIET_CLANG)$(CLANG) -g -O2 -fno-stack-protector --target=bpf \ - $(CLANG_OPTIONS) $(EXTRA_BPF_FLAGS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ - -include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \ - -fms-extensions -Wno-microsoft-anon-tag \ - -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@ - -$(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o | $(BPFTOOL) - $(QUIET_GENSKEL)$(BPFTOOL) gen skeleton $< > $@ - -bpf-skel: $(SKELETONS) - -.PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o - -else # CONFIG_PERF_BPF_SKEL - -bpf-skel: - endif # CONFIG_PERF_BPF_SKEL bpf-skel-clean: - $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS) $(SKEL_OUT)/vmlinux.h + $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKEL_OUT)/*.skel.h $(SKEL_OUT)/vmlinux.h $(OUTPUT)bench/bpf_skel/*.skel.h $(OUTPUT)bench/bpf_skel/.tmp pmu-events-clean: ifeq ($(OUTPUT),)
tools/perf/bench/Build +6 -0
@@ -24,3 +24,9 @@ perf-bench-$(CONFIG_X86_64) += mem-memcpy-x86-64-asm.o perf-bench-$(CONFIG_X86_64) += mem-memset-x86-64-asm.o perf-bench-$(CONFIG_NUMA) += numa.o + +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)bench/uprobe.o: $(SKEL_OUT)/bench_uprobe.skel.h +endif
tools/perf/bench/bpf_skel/bench_uprobe.bpf.c renamed from tools/perf/util/bpf_skel/bench_uprobe.bpf.c +0 -0
tools/perf/bench/uprobe.c +1 -1
@@ -44,7 +44,7 @@ static const char * const bench_uprobe_usage[] = { };#ifdef HAVE_BPF_SKEL -#include "bpf_skel/bench_uprobe.skel.h" +#include "bench/bpf_skel/bench_uprobe.skel.h" #define bench_uprobe__attach_uprobe(prog) \ skel->links.prog = bpf_program__attach_uprobe_opts(/*prog=*/skel->progs.prog, \
tools/perf/bpf_skel.mak new file +54 -0
54 lines not on this page · raw message
tools/perf/util/Build +14 -1
@@ -442,4 +442,17 @@ $(OUTPUT)%.pylint_log: % perf-util-y += $(PYLINT_TEST_LOGS) - +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)util/bpf_ftrace.o: $(SKEL_OUT)/func_latency.skel.h +$(OUTPUT)util/bpf-filter.o: $(SKEL_OUT)/sample_filter.skel.h +$(OUTPUT)util/bpf_kwork_top.o: $(SKEL_OUT)/kwork_top.skel.h +$(OUTPUT)util/bpf_off_cpu.o: $(SKEL_OUT)/off_cpu.skel.h +$(OUTPUT)util/bpf-trace-summary.o: $(SKEL_OUT)/syscall_summary.skel.h +$(OUTPUT)util/bpf_counter_cgroup.o: $(SKEL_OUT)/bperf_cgroup.skel.h +$(OUTPUT)util/bpf_trace_augment.o: $(SKEL_OUT)/augmented_raw_syscalls.skel.h +$(OUTPUT)util/bpf_counter.o: $(SKEL_OUT)/bpf_prog_profiler.skel.h $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h +$(OUTPUT)util/bpf_lock_contention.o: $(SKEL_OUT)/lock_contention.skel.h +$(OUTPUT)util/bpf_kwork.o: $(SKEL_OUT)/kwork_trace.skel.h +endif
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 8/17 perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak
In reply to Ian RogersCurrently, bpftool and vmlinux.h are prerequisites of the top-level prepare target in Makefile.perf. This unnecessarily blocks the massive parallel C compilation of libraries (perf-util, perf-ui, pmu-events) during the initial startup phase.
Move all bpftool and vmlinux.h generation rules down into tools/perf/bpf_skel.mak to encapsulate BPF tooling completely within the skeleton framework. Remove them entirely from prepare to unblock immediate parallel build execution.
To prevent parallel sub-makes (perf-util and perf-bench) from racing to build shared prerequisites concurrently, while maintaining strict directory encapsulation without top-level inclusions, serialize bench after the util static archive finishes using an order-only prerequisite:
$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL)
--- tools/perf/Makefile.config | 2 +- tools/perf/Makefile.perf | 61 +++++--------------------------------- tools/perf/bpf_skel.mak | 47 +++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 55 deletions(-)
tools/perf/Makefile.config +1 -1
@@ -736,7 +736,7 @@ ifeq ($(BUILD_BPF_SKEL),1) endif ifndef GEN_VMLINUX_H - VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h + VMLINUX_H_FILE=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h endif dwarf-post-unwind := 1
tools/perf/Makefile.perf +7 -54
88 lines not on this page · raw message
tools/perf/bpf_skel.mak +47 -0
57 lines not on this page · raw message
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 9/17 perf build: Move static libbpf dependency out of prepare step
In reply to Ian RogersThe prepare step is a large serialization point before parallel sub-makes build the perf tool. The libbpf headers are used in the bench and util libraries, as well as individual perf builtin commands. Move the libbpf dependency out of the prepare step and into the dependencies for those targets to avoid it being a source of serialization in the prepare step.
--- tools/perf/Makefile.perf | 12 ++++-------- 1 file changed, 4 insertions(+), 8 deletions(-)
tools/perf/Makefile.perf +4 -8
@@ -547,7 +547,7 @@ export NO_JEVENTS build := -f $(srctree)/tools/build/Makefile.build dir=. obj -$(PERF_IN): prepare FORCE +$(PERF_IN): prepare $(LIBBPF) FORCE $(Q)$(MAKE) $(build)=perf $(LIBPMU_EVENTS_IN): FORCE $(LIBPERF)
@@ -558,13 +558,13 @@ $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN) # The $(LIBPERF_UTIL) dependency is to ensure bpftool and vmlinux.h # aren't racily built for bench/bpf_skel/bench_uprobe.bpf.c -$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL) +$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) | $(LIBPERF_UTIL) $(Q)$(MAKE) $(build)=perf-bench $(LIBPERF_BENCH): $(LIBPERF_BENCH_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_TEST_IN): FORCE prepare +$(LIBPERF_TEST_IN): FORCE prepare $(LIBBPF) $(Q)$(MAKE) $(build)=perf-test $(LIBPERF_TEST): $(LIBPERF_TEST_IN)
@@ -576,7 +576,7 @@ $(LIBPERF_UI_IN): FORCE prepare $(LIBPERF_UI): $(LIBPERF_UI_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_UTIL_IN): FORCE prepare +$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) $(Q)$(MAKE) $(build)=perf-util $(LIBPERF_UTIL): $(LIBPERF_UTIL_IN)
@@ -636,10 +636,6 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBSUBCMD) \ $(LIBSYMBOL) -ifdef LIBBPF_STATIC -prepare: $(LIBBPF) -endif - $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v3 09/17] perf build: Move static libbpf dependency out of prepare step
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Removing $(LIBBPF) from `prepare` breaks the compilation of individual object files (e.g., `make…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Removing $(LIBBPF) from prepare breaks the compilation of individual object files (e.g., make util/bpf_counter.o) and the check target. --
commit 37c1263c37c9d7f14cf72b110d0a12d1429501e0
Author: Ian Rogers <[email protected]>
perf build: Move static libbpf dependency out of prepare step
This commit moves the libbpf dependency from the general prepare step to the specific targets that require it. This reduces serialization during the prepare step to speed up parallel builds.
Quoted 3 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
tools/perf/Makefile.perf prepare: $(OUTPUT)PERF-VERSION-FILE \ · line 640
Quoted 10 lines
tools/perf/Makefile.perf +0
-4@@ -636,10 +636,6 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBSUBCMD) \ $(LIBSYMBOL) -ifdef LIBBPF_STATIC -prepare: $(LIBBPF) -endif - $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Does removing this dependency from the prepare step break the compilation of individual object files and the check target?
Since the %.o compilation rule shown above and the check target rely on prepare, it looks like running make util/bpf_counter.o or make check in a clean tree will proceed without first building libbpf.a and its headers.
Could this result in build failures for any C files that include BPF headers before the library is fully built?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=9
Ian Rogers
reviewerPATCH v3 10/17 perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase
In reply to Ian RogersCurrently, BPF skeleton generation rules (bpf_skel.mak) are evaluated as part of util/Build. However, because LIBPERF_UTIL_IN explicitly depends on the top-level static libbpf archive, Make completely blocked the execution of bpftool bootstrap and skeleton generation until libbpf finished compiling midway through the build.
Since bpftool bootstrap compiles its own independent copy of libbpf.a, it does not depend on the top-level libbpf target.
Decouple early skeleton tooling generation by attaching bpf-skel-prepare to the umbrella prepare target, exporting CONFIG_PERF_BPF_SKEL to ensure accurate feature propagation. This allows Make to compile bpftool and dump vmlinux.h in the background at build startup, eliminating the initial sub-make startup bottleneck before BPF object compilation while keeping 100% of tooling rules perfectly encapsulated in bpf_skel.mak. Provide an empty fallback target to ensure builds succeed when BPF skeletons are disabled.
--- tools/perf/Makefile.perf | 8 ++++++-- tools/perf/bpf_skel.mak | 8 ++++++++ 2 files changed, 14 insertions(+), 2 deletions(-)
tools/perf/Makefile.perf +6 -2
@@ -275,7 +275,7 @@ ifeq ($(PYLINT),1) endif export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF READELF -export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT +export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT CONFIG_PERF_BPF_SKEL include $(srctree)/tools/build/Makefile.include
@@ -629,8 +629,12 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) +bpf-skel-prepare: + $(Q)$(MAKE) -f bpf_skel.mak bpf-skel-prepare + prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ + bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \
@@ -967,7 +971,7 @@ FORCE: .PHONY: all install clean config-clean strip install-gtk .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell -.PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare +.PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare bpf-skel-prepare .PHONY: python_perf_target endif # force_fixdep
tools/perf/bpf_skel.mak +8 -0
@@ -15,6 +15,10 @@ ifeq ($(CONFIG_PERF_BPF_SKEL),y) BPFTOOL := $(SKEL_TOOL_TMP_OUT)/bootstrap/bpftool VMLINUX_H := $(SKEL_TOOL_OUT)/vmlinux.h +.PHONY: bpf-skel-prepare +bpf-skel-prepare: $(bpf_skel_deps) + @: + define get_sys_includes $(shell $(1) $(2) -v -E - </dev/null 2>&1 \ | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
@@ -93,6 +97,10 @@ $(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o $(BPFTOOL) .PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o +else # CONFIG_PERF_BPF_SKEL +.PHONY: bpf-skel-prepare +bpf-skel-prepare: + @: endif # CONFIG_PERF_BPF_SKEL clean:
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v3 10/17] perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The `bpf-skel-prepare` target relies on an undefined variable `$(bpf_skel_deps)`, causing the…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The bpf-skel-prepare target relies on an undefined variable $(bpf_skel_deps), causing the pre-generation to be silently skipped and introducing a parallel build race condition. --
commit aa0df16c70b52c768c1153976cf365e149c64c2f
Author: Ian Rogers <[email protected]>
perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase
This commit decouples BPF skeleton tooling generation by attaching a new bpf-skel-prepare target to the umbrella prepare target. This is intended to compile bpftool and dump vmlinux.h early to eliminate sub-make startup bottlenecks.
tools/perf/bpf_skel.mak ifeq ($(CONFIG_PERF_BPF_SKEL),y) · line 19
Quoted 10 lines
tools/perf/bpf_skel.mak +2
-0@@ -15,6 +15,10 @@ ifeq ($(CONFIG_PERF_BPF_SKEL),y) BPFTOOL := $(SKEL_TOOL_TMP_OUT)/bootstrap/bpftool VMLINUX_H := $(SKEL_TOOL_OUT)/vmlinux.h +.PHONY: bpf-skel-prepare +bpf-skel-prepare: $(bpf_skel_deps)
Is $(bpf_skel_deps) defined anywhere?
It looks like this variable is never defined, meaning bpf-skel-prepare evaluates with no prerequisites and executes as a no-op during the prepare phase.
If bpftool and vmlinux.h are not pre-generated here, will concurrent sub-makes that independently include bpf_skel.mak simultaneously attempt to evaluate and build the missing $(BPFTOOL) dependency?
This could cause multiple independent Make instances to concurrently enter ../bpf/bpftool and write to the same output directories, leading to parallel build corruption and failures.
Quoted 1 line
tools/perf/bpf_skel.mak +1
-0+ @:
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
Ian Rogers
reviewerPATCH v3 11/17 perf build: Move libsymbol dependency out of prepare step
In reply to Ian RogersThe prepare step is a large serialization point before parallel sub-makes build the perf tool. The libsymbol headers are used in the bench and util libraries. Move the libsymbol dependency out of the prepare step and into the dependencies for those targets to avoid it being a source of serialization in the prepare step.
--- tools/perf/Makefile.perf | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
tools/perf/Makefile.perf +3 -4
@@ -558,7 +558,7 @@ $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN) # The $(LIBPERF_UTIL) dependency is to ensure bpftool and vmlinux.h # aren't racily built for bench/bpf_skel/bench_uprobe.bpf.c -$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) | $(LIBPERF_UTIL) +$(LIBPERF_BENCH_IN): FORCE prepare $(LIBBPF) $(LIBSYMBOL) | $(LIBPERF_UTIL) $(Q)$(MAKE) $(build)=perf-bench $(LIBPERF_BENCH): $(LIBPERF_BENCH_IN)
@@ -576,7 +576,7 @@ $(LIBPERF_UI_IN): FORCE prepare $(LIBPERF_UI): $(LIBPERF_UI_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) +$(LIBPERF_UTIL_IN): FORCE prepare $(LIBBPF) $(LIBSYMBOL) $(Q)$(MAKE) $(build)=perf-util $(LIBPERF_UTIL): $(LIBPERF_UTIL_IN)
@@ -637,8 +637,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ - $(LIBSUBCMD) \ - $(LIBSYMBOL) + $(LIBSUBCMD) $(OUTPUT)%.o: %.c prepare FORCE $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=$(build-dir) $@
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 12/17 perf build: Remove redundant libbpf feature check for static builds
In reply to Ian RogersBy default, the perf tool compiles and statically links against its own internal copy of libbpf (tools/lib/bpf/libbpf.a), setting LIBBPF_STATIC=1.
Despite this static linkage, Makefile.config unconditionally executed $(call feature_check,libbpf), which forced a synchronous sub-make fork during AST parsing to detect dynamic system libbpf libraries.
As noted in the internal Makefile comments, this check was executed purely so that running make VF=1 would display the detection status of system libbpf. During standard builds without LIBBPF_DYNAMIC=1, the detection result was entirely ignored.
Wrap the libbpf feature check inside LIBBPF_DYNAMIC=1 so Make avoids the redundant sub-make fork overhead during standard static builds.
--- tools/perf/Makefile.config | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
tools/perf/Makefile.config +1 -3
@@ -587,10 +587,8 @@ ifndef NO_LIBELF ifndef NO_LIBBPF ifeq ($(feature-bpf), 1) - # detecting libbpf without LIBBPF_DYNAMIC, so make VF=1 shows libbpf detection status - $(call feature_check,libbpf) - ifdef LIBBPF_DYNAMIC + $(call feature_check,libbpf) ifeq ($(feature-libbpf), 1) EXTLIBS += -lbpf CFLAGS += -DHAVE_LIBBPF_SUPPORT
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 13/17 perf pmu-events: Split big_c_string storage into standalone compilation unit
In reply to Ian RogersCurrently, jevents.py emits both the massive 2.8 MB big_c_string literal and tens of thousands of compact_pmu_event struct arrays into a single pmu-events.c compilation unit. Compiling this giant file takes ~2.2 seconds on a single CPU core during Kbuild startup.
Refactor jevents.py to emit big_c_string into a dedicated pmu-events-string.c compilation unit. This allows Kbuild to compile pmu-events.o and pmu-events-string.o simultaneously in parallel across two separate CPU cores, preserving 100% string deduplication and zero dynamic ELF relocations while cutting C compilation latency in half.
Add pmu-events-string.c to tools/perf/.gitignore to ensure in-tree Kbuild runs do not leave untracked generated files in the working directory.
To guarantee 100% backward compatibility with GNU Make 4.0+ (avoiding the Make 4.3+ grouped target &: syntax which causes older Make versions like 4.2.1 to spawn multiple concurrent jevents.py processes during parallel builds), implement a robust dependency chaining pattern:
$(PMU_EVENTS_C): $(JEVENTS_DEPS)
$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C)
@:
This ensures jevents.py is invoked exactly once. If jevents.py aborts early, Make's .DELETE_ON_ERROR: purges pmu-events.c, guaranteeing that subsequent Make invocations correctly re-execute the script and overwrite pmu-events-string.c, keeping the build fully self-correcting.
--- tools/perf/.gitignore | 1 + tools/perf/Makefile.perf | 4 ++-- tools/perf/pmu-events/Build | 16 +++++++++++++++- tools/perf/pmu-events/jevents.py | 20 ++++++++++++++++---- 4 files changed, 34 insertions(+), 7 deletions(-)
tools/perf/.gitignore +1 -0
@@ -38,6 +38,7 @@ arch/*/include/generated/ trace/beauty/generated/ pmu-events/arch/common/common/legacy-cache.json pmu-events/pmu-events.c +pmu-events/pmu-events-string.c pmu-events/jevents pmu-events/metric_test.log pmu-events/empty-pmu-events.log
tools/perf/Makefile.perf +2 -2
@@ -921,7 +921,7 @@ bpf-skel-clean: pmu-events-clean: ifeq ($(OUTPUT),) $(call QUIET_CLEAN, pmu-events) $(RM) \ - pmu-events/pmu-events.c \ + pmu-events/pmu-events*.c \ pmu-events/metric_test.log \ pmu-events/test-empty-pmu-events.c \ pmu-events/empty-pmu-events.log
@@ -929,7 +929,7 @@ ifeq ($(OUTPUT),) -name 'extra-metricgroups.json' -delete else # When an OUTPUT directory is present, clean up the copied pmu-events/arch directory. $(call QUIET_CLEAN, pmu-events) $(RM) -r $(OUTPUT)pmu-events/arch \ - $(OUTPUT)pmu-events/pmu-events.c \ + $(OUTPUT)pmu-events/pmu-events*.c \ $(OUTPUT)pmu-events/metric_test.log \ $(OUTPUT)pmu-events/test-empty-pmu-events.c \ $(OUTPUT)pmu-events/empty-pmu-events.log
tools/perf/pmu-events/Build +15 -1
@@ -1,7 +1,12 @@ EMPTY_PMU_EVENTS_C = pmu-events/empty-pmu-events.c # pmu-events.c will be generated by jevents.py or copied from EMPTY_PMU_EVENTS_C PMU_EVENTS_C = $(OUTPUT)pmu-events/pmu-events.c +PMU_EVENTS_STRING_C = $(OUTPUT)pmu-events/pmu-events-string.c + pmu-events-y += pmu-events.o +ifneq ($(NO_JEVENTS),1) +pmu-events-y += pmu-events-string.o +endif # pmu-events.c file is generated in the OUTPUT directory so it needs a # separate rule to depend on it properly
@@ -9,6 +14,10 @@ $(OUTPUT)pmu-events/pmu-events.o: $(PMU_EVENTS_C) $(call rule_mkdir) $(call if_changed_dep,cc_o_c) +$(OUTPUT)pmu-events/pmu-events-string.o: $(PMU_EVENTS_STRING_C) + $(call rule_mkdir) + $(call if_changed_dep,cc_o_c) + # Message for $(call echo-cmd,cp), possibly remove the src file from # the destination to save space in the build log. quiet_cmd_cp = COPY $(patsubst %$<,%,$@) <- $<
@@ -118,6 +127,7 @@ CUR_OUT_JSON := $(shell [ -d $(OUT_DIR) ] && find $(OUT_DIR) -type f) # Things in the OUTPUT directory but shouldn't be there as computed by # OUT_JSON and GEN_JSON. + ORPHAN_FILES := $(filter-out $(OUT_JSON) $(GEN_JSON),$(CUR_OUT_JSON)) # Message for $(call echo-cmd,mkd). There is already a mkdir message
@@ -224,6 +234,10 @@ endif # and inputs are dependencies. $(PMU_EVENTS_C): $(JEVENTS_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) \ + $(OUT_DIR) $(PMU_EVENTS_C) $(PMU_EVENTS_STRING_C) + +$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C) + @: endif # ifeq ($(NO_JEVENTS),1)
tools/perf/pmu-events/jevents.py +16 -4
@@ -1422,6 +1422,8 @@ such as "arm/cortex-a34".''', ) ap.add_argument( 'output_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=sys.stdout) + ap.add_argument( + 'output_string_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=None) _args = ap.parse_args() _args.output_file.write(f"""
@@ -1463,10 +1465,20 @@ struct pmu_table_entry { ftw(arch_path, [], preprocess_one_file) _bcs.compute() - _args.output_file.write('static const char *const big_c_string =\n') - for s in _bcs.big_string: - _args.output_file.write(s) - _args.output_file.write(';\n\n') + if not _args.output_string_file: + _args.output_file.write('static const char *const big_c_string =\n') + for s in _bcs.big_string: + _args.output_file.write(s) + _args.output_file.write(';\n\n') + else: + _args.output_string_file.write('/* SPDX-License-Identifier: GPL-2.0 */\n') + _args.output_string_file.write('/* Autogenerated by jevents.py */\n') + _args.output_string_file.write('const char big_c_string[] =\n') + for s in _bcs.big_string: + _args.output_string_file.write(s) + _args.output_string_file.write(';\n') + _args.output_string_file.close() + _args.output_file.write('extern const char big_c_string[];\n\n') for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], process_one_file)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v3 13/17] perf pmu-events: Split big_c_string storage into standalone compilation unit
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] Incremental build breakage causing `perf` to be unconditionally relinked on every `make`…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] Incremental build breakage causing perf to be unconditionally relinked on every make invocation.
- [Low] Missing variable declaration for big_c_string causes a compiler warning/error on Clang.
- [Low] Fragile multiple-output dependency chaining causes an unrecoverable build loop if the secondary file is deleted. --
commit c21d41481809344c6220821a1680882429693c05
Author: Ian Rogers <[email protected]>
perf pmu-events: Split big_c_string storage into standalone compilation unit
This commit refactors jevents.py to output the big_c_string literal into a separate pmu-events-string.c file. It also updates the perf Makefile to build this new file, using a dependency chain pattern to avoid multiple jevents.py executions in parallel builds.
Quoted 4 lines
tools/perf/pmu-events/Build +0
-0
[ ... ]
tools/perf/pmu-events/Build endif · line 241
Quoted 10 lines
tools/perf/pmu-events/Build +5
-1@@ -224,6 +234,10 @@ endif # and inputs are dependencies. $(PMU_EVENTS_C): $(JEVENTS_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) \ + $(OUT_DIR) $(PMU_EVENTS_C) $(PMU_EVENTS_STRING_C) + +$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C) + @:
If a user manually deletes pmu-events-string.c, won't Make execute this dummy recipe, assume the file was recreated, and then fail during compilation?
This could leave the build in an unrecoverable state until pmu-events.c is also deleted, which seems contrary to the fully self-correcting goal mentioned in the commit message.
Quoted 4 lines
tools/perf/pmu-events/jevents.py +0
-0
[ ... ]
Quoted 10 lines
tools/perf/pmu-events/jevents.py +2
-4@@ -1463,10 +1465,20 @@ struct pmu_table_entry { ftw(arch_path, [], preprocess_one_file) _bcs.compute() - _args.output_file.write('static const char *const big_c_string =\n') - for s in _bcs.big_string: - _args.output_file.write(s) - _args.output_file.write(';\n\n') + if not _args.output_string_file: + _args.output_file.write('static const char *const big_c_string =\n')
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1479
Quoted 10 lines
tools/perf/pmu-events/jevents.py +10
-0+ for s in _bcs.big_string: + _args.output_file.write(s) + _args.output_file.write(';\n\n') + else: + _args.output_string_file.write('/* SPDX-License-Identifier: GPL-2.0 */\n') + _args.output_string_file.write('/* Autogenerated by jevents.py */\n') + _args.output_string_file.write('const char big_c_string[] =\n') + for s in _bcs.big_string: + _args.output_string_file.write(s) + _args.output_string_file.write(';\n')
Will compiling this file with Clang trigger a missing-variable-declarations warning?
It looks like big_c_string is defined globally here without a prior extern declaration in the same compilation unit, which could cause a build failure when -Werror is enabled.
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1480
Quoted 1 line
tools/perf/pmu-events/jevents.py +1
-0+ _args.output_string_file.close()
Does closing output_string_file here cause an incremental build breakage?
Since output_file is written to during the subsequent ftw calls that parse the architecture JSON files, output_string_file will end up with an older modification timestamp than output_file (which corresponds to pmu-events.c).
Combined with the Makefile rule $(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C) having a dummy @: recipe, Make will see pmu-events-string.c is older than pmu-events.c on every invocation, and unconditionally rebuild pmu-events-string.o and relink the perf binary.
Quoted 4 lines
tools/perf/pmu-events/jevents.py +1
-0+ _args.output_file.write('extern const char big_c_string[];\n\n') for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], process_one_file)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=13
Ian Rogers
reviewerPATCH v3 14/17 perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
In reply to Ian RogersCurrently, jevents.py parses hundreds of JSON event and metric files sequentially across all CPU architectures during Kbuild startup, taking ~3.3 seconds of pure single-core execution time.
Refactor jevents.py to pre-populate its internal JSON AST cache in parallel across all available CPU cores using ProcessPoolExecutor. Define the worker process initializer _init_worker at the top-level module scope to guarantee flawless pickling and standard event mapping inheritance under spawn multiprocessing semantics (avoiding AttributeError crashes when spawn is used instead of fork). This accelerates jevents.py execution by over 11x (from 3.3s down to ~290ms), fully reclaiming multi-core concurrency during the build generation phase.
--- tools/perf/pmu-events/jevents.py | 36 ++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-)
tools/perf/pmu-events/jevents.py +32 -4
@@ -457,8 +457,8 @@ class JsonEvent: return f'{{ { _bcs.offsets[s] } }}, /* {fix_comment(s)} */\n' -@lru_cache(maxsize=None) -def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: +_json_cache = {} +def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" try: events = json.load(open(path), object_hook=JsonEvent)
@@ -474,12 +474,16 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: if updates: for event in events: if event.metric_name in updates: - # print(f'Updated {event.metric_name} from\n"{event.metric_expr}"\n' - # f'to\n"{updates[event.metric_name]}"') event.metric_expr = updates[event.metric_name] return events +def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: + key = (path, topic) + if key not in _json_cache: + _json_cache[key] = _read_json_events_impl(path, topic) + return _json_cache[key] + def preprocess_arch_std_files(archpath: str) -> None: """Read in all architecture standard events.""" global _arch_std_events
@@ -1381,6 +1385,14 @@ const char *describe_metricgroup(const char *group) } """) +def _parallel_read_json_events(task: Tuple[str, str]) -> Tuple[str, str, Sequence[JsonEvent]]: + path, topic = task + return path, topic, _read_json_events_impl(path, topic) + +def _init_worker(std_events: dict) -> None: + global _arch_std_events + _arch_std_events = std_events + def main() -> None: global _args
@@ -1459,9 +1471,25 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json) + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor: + for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file) _bcs.compute()
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v3 14/17] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Pre-populating `_arch_std_events` for all architectures upfront before parallel parsing causes…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Pre-populating _arch_std_events for all architectures upfront before parallel parsing causes standard events to be overwritten by subsequent architectures, breaking per-architecture scoping. --
commit 91f4ede8d30987c122141ee70c9172917d76aef9
Author: Ian Rogers <[email protected]>
perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
This commit parallelizes the parsing of JSON event and metric files in jevents.py using ProcessPoolExecutor. It introduces a worker initializer to inherit standard event mappings, significantly reducing execution time.
Quoted 4 lines
tools/perf/pmu-events/jevents.py +0
-0
[ ... ]
Quoted 8 lines
tools/perf/pmu-events/jevents.py +4
-0@@ -1459,9 +1471,25 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0:
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1487
Quoted 10 lines
tools/perf/pmu-events/jevents.py +7
-0+ return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json) + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor:
Does this loop overwrite standard events from earlier architectures?
In the original sequential code, _arch_std_events was populated for a single architecture and immediately used for parsing. By hoisting preprocess_arch_std_files into this loop for all architectures upfront, and since _arch_std_events is a single global dictionary that is not cleared between iterations, identically named standard events from later architectures in the archs list will overwrite those from earlier architectures.
When the worker processes are initialized with this completely merged state in _init_worker, it appears they will incorrectly resolve ArchStdEvent references using the later architecture definitions. Can this be isolated per architecture instead?
Quoted 6 lines
tools/perf/pmu-events/jevents.py +5
-0+ for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=14
Ian Rogers
reviewerPATCH v3 15/17 perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds
In reply to Ian RogersIn Makefile.perf, ALL_PROGRAMS includes SCRIPTS (perf-archive, perf-iostat). However, unlike PROGRAMS and DLFILTERS, SCRIPTS was not prefixed with $(OUTPUT).
During out-of-tree builds (or when O= is specified), Make checked for the unprefixed target 'tools/perf/perf-archive'. Since the actual script was installed into $(OUTPUT)perf-archive, Make concluded the target was missing and continuously re-executed the script installation rule on every single incremental build.
Prefix SCRIPTS with $(OUTPUT) and update the static pattern rule to ensure Kbuild correctly tracks generated script prerequisites during incremental builds.
--- tools/perf/Makefile.perf | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +3 -3
@@ -405,7 +405,7 @@ python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT # Use the detected configuration -include $(OUTPUT).config-detected -SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) +SCRIPTS = $(addprefix $(OUTPUT),$(patsubst %.sh,%,$(SCRIPT_SH))) PROGRAMS += $(OUTPUT)perf
@@ -592,8 +592,8 @@ $(GTK_IN): FORCE prepare $(OUTPUT)libperf-gtk.so: $(GTK_IN) $(PERFLIBS) $(QUIET_LINK)$(CC) -o $@ -shared $(LDFLAGS) $(filter %.o,$^) $(GTK_LIBS) -$(SCRIPTS) : % : %.sh - $(QUIET_GEN)$(INSTALL) '[email protected]' '$(OUTPUT)$@' +$(SCRIPTS) : $(OUTPUT)% : %.sh + $(QUIET_GEN)$(INSTALL) '$<' '$@' $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE $(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 16/17 perf pmu-events: Convert recursive shell assignments and macros to Make built-ins
In reply to Ian RogersIn pmu-events/Build, ZENS, ARMS, and INTELS were assigned using recursive assignment (=), and model_name/vendor_name were evaluated using shell macros (echo ... | sed ...).
Because these variables were expanded inside the COPY_RULE dependency evaluation loop across hundreds of PMU JSON files and inside every metric generation recipe, Kbuild continuously re-executed 'ls', 'grep', and 'sed' shell forks thousands of times during AST parsing and execution.
Convert ZENS, ARMS, and INTELS to simply expanded variables (:=) and replace model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once when the Build file is parsed and evaluates path macros purely in memory, completely eliminating over 7,800 redundant sub-processes during build startup.
--- tools/perf/pmu-events/Build | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/pmu-events/Build +5 -5
@@ -56,12 +56,12 @@ $(LEGACY_CACHE_JSON): $(LEGACY_CACHE_PY) # Python to generate architectural metrics GEN_METRIC_DEPS := pmu-events/metric.py pmu-events/common_metrics.py # Functions to extract the model from an extra-metrics.json or extra-metricgroups.json path. -model_name = $(shell echo $(1)|sed -e 's@.\+/\(.*\)/extra-metric.*\.json@\1@') -vendor_name = $(shell echo $(1)|sed -e 's@.\+/\(.*\)/[^/]*/extra-metric.*\.json@\1@') +model_name = $(notdir $(patsubst %/,%,$(dir $(1)))) +vendor_name = $(notdir $(patsubst %/,%,$(dir $(patsubst %/,%,$(dir $(1)))))) ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),x86 all)) # Generate AMD Json -ZENS = $(shell ls -d pmu-events/arch/x86/amdzen*) +ZENS := $(shell ls -d pmu-events/arch/x86/amdzen*) ZEN_METRICS = $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metrics.json) ZEN_METRICGROUPS = $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(ZEN_METRICS) $(ZEN_METRICGROUPS)
@@ -78,7 +78,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all)) # Generate ARM Json -ARMS = $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json) ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(ARM_METRICS) $(ARM_METRICGROUPS)
@@ -95,7 +95,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),x86 all)) # Generate Intel Json -INTELS = $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapfile.csv) +INTELS := $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapfile.csv) INTEL_METRICS = $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metrics.json) INTEL_METRICGROUPS = $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(INTEL_METRICS) $(INTEL_METRICGROUPS)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v3 17/17 perf build: Convert llvm-config shell queries to simply expanded variables
In reply to Ian RogersIn Makefile.config, CFLAGS, CXXFLAGS, LIBLLVM, and EXTLIBS were assigned using recursive expansion or appended with raw $(shell $(LLVM_CONFIG) ...) calls. Because these variables were expanded during dependency evaluation across every single object file compilation rule, Kbuild continuously re-executed llvm-config forks nearly 200 times during incremental builds.
Convert llvm-config shell queries to simply expanded variables (:=) to ensure Make evaluates LLVM compiler flags and library paths exactly once when Makefile.config is parsed, eliminating ~185 redundant sub-processes during build startup.
--- tools/perf/Makefile.config | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
tools/perf/Makefile.config +8 -5
@@ -951,11 +951,14 @@ ifndef NO_LIBLLVM $(call feature_check,llvm-perf) ifeq ($(feature-llvm-perf), 1) CFLAGS += -DHAVE_LIBLLVM_SUPPORT - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) - CXXFLAGS += -DHAVE_LIBLLVM_SUPPORT - CXXFLAGS += $(shell $(LLVM_CONFIG) --cxxflags) - LIBLLVM = $(shell $(LLVM_CONFIG) --libs all) $(shell $(LLVM_CONFIG) --system-libs) - EXTLIBS += -L$(shell $(LLVM_CONFIG) --libdir) $(LIBLLVM) + LLVM_CFLAGS := $(shell $(LLVM_CONFIG) --cflags 2>/dev/null) + LLVM_CXXFLAGS := $(shell $(LLVM_CONFIG) --cxxflags 2>/dev/null) + LLVM_LIBLLVM := $(shell $(LLVM_CONFIG) --libs all 2>/dev/null) $(shell $(LLVM_CONFIG) --system-libs 2>/dev/null) + LLVM_LIBDIR := $(shell $(LLVM_CONFIG) --libdir 2>/dev/null) + CFLAGS += $(LLVM_CFLAGS) + CXXFLAGS += -DHAVE_LIBLLVM_SUPPORT $(LLVM_CXXFLAGS) + LIBLLVM := $(LLVM_LIBLLVM) + EXTLIBS += -L$(LLVM_LIBDIR) $(LIBLLVM) EXTLIBS += -lstdc++ $(call detected,CONFIG_LIBLLVM) else
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Namhyung Kim
maintainerRe: [PATCH v3 00/17] perf build: Reduce build time by nearly half
In reply to Ian RogersQuoted 24 lines · Ian Rogers
This patch series refactors
Kbuildinternals, BPF skeleton generation, Python AST pre-computation, and foundational tooling dependencies across the perf tool build system. By eliminating umbrella target synchronization barriers, decoupling static library prerequisites, parallelizing single-core script generators, and eradicating redundant feature checks, this series unlocks absolute theoretical peak multi-core concurrency duringKbuildstartup.On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 49%:
Before:
real 0m29.006s user 2m46.019s sys 0m30.610sAfter:
real 0m14.782s user 2m39.527s sys 0m22.938sSaving 14.2 full seconds time per clean build. Furthermore, nothing to build incremental builds are improved by nearly 7x:
Before:
Quoted 8 lines
real 0m11.528s user 0m9.633s sys 0m6.965sAfter:
real 0m1.729s user 0m1.600s sys 0m0.884s
I've quickly checked it with latency profiling like below:
$ perf record --latency -- make -C tools/perf
$ perf report --latency -s comm
The result looks like this.
Before:
#
# Samples: 715K of event 'cpu/cycles/Pu'
# Event count (approx.): 422452811481
#
# Latency Overhead Command
# ........ ........ ...............
#
45.28% 71.33% cc1
34.48% 16.92% python3
11.15% 2.21% ld
2.58% 1.51% x86_64-linux-gn
2.22% 0.99% cc1plus
0.71% 0.63% sh
0.69% 0.14% llvm-config
0.62% 0.56% clang
0.57% 4.40% shellcheck
0.44% 0.12% perl After:
#
# Samples: 709K of event 'cpu/cycles/Pu'
# Event count (approx.): 416654798495
#
# Latency Overhead Command
# ........ ........ ...............
#
64.99% 71.16% cc1
15.07% 1.81% ld
7.14% 17.59% python3
3.66% 1.53% x86_64-linux-gn
3.48% 0.75% cc1plus
1.11% 4.43% shellcheck
1.09% 0.74% sh
0.86% 0.59% clang
0.77% 0.12% perl
0.45% 0.23% make Now I see a big drop in the latency from python. And the llvm-config doesn't show up in the top 10.
Thanks,
Namhyung
Quoted 179 lines
Summary of Patches:
1-3: Foundational Tooling & Fast-Path Feature Detection
- Exempts bpftool bootstrap from non-essential feature tests (LLVM, libbfd, libcap), saving 1.1s of sub-make fork overhead duringKbuildstartup.
- Integrates libdebuginfod directly intotest-all.c, allowing Make to skip individual feature check sub-make forks during AST parsing on fully configured workstations. Escapes $(shell ...) macro expansion to prevent unconditional sub-make forks.
- Fixes test-clang-bpf-co-re.bin feature check to correctly generate its target file on disk via atomic move (> [email protected] && mv [email protected] $@), allowingKbuildto perfectly cache the detection result and avoid continuous sub-make re-evaluations.4-6: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.7-10: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dumpvmlinux.hin the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
- Ensures benchmark skeleton intermediate .bpf.o files are cleanly removed during make clean, and adds bpf-skel-prepare to .PHONY.11-12: Foundational Linkage Optimization
- Eliminates redundant libbpf sub-make feature checks during static builds.
- Moves static libsymbol and libbpf library prerequisites out of the prepare step, ensuring libbpf headers are installed before compiling BPF-dependent tests.13-14:
jevents.pyConcurrency & Deduplication
- Splits the massive 2.8 MBbig_c_stringliteral out ofpmu-events.cinto a dedicatedpmu-events-string.ccompilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations. Addspmu-events-string.cto .gitignore and uses Make 4.0 compatible dependency chaining.
- Pre-populatesjevents.pyJSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms). Moves_init_workerto top-level scope to ensure clean pickling under spawn multiprocessing start methods.15: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.16-17: AST Parsing Optimization & Shell Fork Eradication
- Converts ZENS, ARMS, and INTELS in pmu-events/Build from recursive assignment (=) to simply expanded assignment (:=) and replaces model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once during AST parsing and evaluates path macros purely in memory, completely eradicating over 7,800 redundant sub-processes during out-of-tree build evaluation.
- Converts llvm-config shell queries in Makefile.config from recursive assignment (=) to simply expanded assignment (:=). This eliminates ~185 redundant sub-processes that were previously executed across object compilation dependency checks.Changes since v2:
- Dropped Patch 4 (tools scripts: Short-circuitCC_NO_CLANGcompiler probe in Makefile.include) to prevent potential cross-compilation regressions when CC and HOSTCC use different compilers.
- tools build (Patch 2): Escaped $(shell ...) macro expansion as $$(shell ...) inside definefeature_check_codeto safely defer sub-make execution until after eval parses the ifeq guard.
- tools build (Patch 3): Refactored test-clang-bpf-co-re.bin feature check recipe to redirect grep output to a temporary file and atomically move it upon success (> [email protected] && mv [email protected] $@), preventingKbuildfrom permanently caching failed detections due to 0-byte files.
- perf trace beauty (Patch 4): Updated commit description to accurately reflect the unconditional top-level recursive kbuild hook (perf-util-y += trace/beauty/).
- perf build (Patch 7): Added $(OUTPUT)bench/bpf_skel/.tmp to bpf-skel-clean in Makefile.perf to ensure intermediate benchmark skeleton .bpf.o artifacts are cleanly removed during make clean. Removed unusedbpf_skel_depsvariable from bpf_skel.mak.
- perf build (Patch 9): Added $(LIBBPF) as an explicit prerequisite to $(LIBPERF_TEST_IN) in Makefile.perf to guarantee libbpf headers are fully installed before compilingsigtrap.cor other BPF-dependent tests during parallel builds.
- perf build (Patch 10): Added bpf-skel-prepare to the .PHONY target list in Makefile.perf to ensure Make never incorrectly skips the target if a file or directory named bpf-skel-prepare accidentally exists in the build tree.
- perf pmu-events (Patch 13): Addedpmu-events/pmu-events-string.ctotools/perf/.gitignore. Replaced grouped targets (&:) with Make 4.0 compatible dependency chaining to guarantee backward compatibility with older Make versions (like 4.2.1) and prevent parallel builds from spawning multiple concurrentjevents.pyprocesses.
- perf pmu-events (Patch 14): Moved_init_workerfrom localmain()scope to the top-level module scope injevents.pyto ensure it can be cleanly pickled when ProcessPoolExecutor uses the spawn multiprocessing start method (avoiding AttributeError crashes).Ian Rogers (17):
bpftool build: Restrict feature tests during bootstrap compilation tools build: Integrate libdebuginfod into test-all fast path tools build: Fix test-clang-bpf-co-re.bin to generate target file perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulatevmlinux.hand bpftool in bpf_skel.mak perf build: Move static libbpf dependency out of prepare step perf build: Pre-generate BPF skeleton tooling during umbrella prepare
phase
perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds perf pmu-events: Splitbig_c_stringstorage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
perf pmu-events: Convert recursive shell assignments and macros to
Make built-ins
perf build: Convert llvm-config shell queries to simply expanded
variablestools/bpf/bpftool/Makefile | 5 + tools/build/Makefile.feature | 6 +- tools/build/feature/Makefile | 4 +- tools/build/feature/test-all.c | 5 + tools/perf/.gitignore | 1 + tools/perf/Build | 2 + tools/perf/Makefile.config | 19 +- tools/perf/Makefile.perf | 431 ++---------------- tools/perf/bench/Build | 6 + .../bpf_skel/bench_uprobe.bpf.c | 0 tools/perf/bench/uprobe.c | 2 +- tools/perf/bpf_skel.mak | 109 +++++ tools/perf/builtin-trace.c | 30 +- tools/perf/pmu-events/Build | 26 +- tools/perf/pmu-events/jevents.py | 56 ++- tools/perf/trace/beauty/Build | 280 ++++++++++++ tools/perf/trace/beauty/arch_errno_names.c | 2 + tools/perf/trace/beauty/arch_errno_names.sh | 2 +- tools/perf/trace/beauty/beauty.h | 60 +++ tools/perf/trace/beauty/eventfd.c | 6 +- tools/perf/trace/beauty/fsconfig.c | 5 + tools/perf/trace/beauty/futex_op.c | 6 +- tools/perf/trace/beauty/futex_val3.c | 6 +- tools/perf/trace/beauty/mmap.c | 24 +- tools/perf/trace/beauty/mode_t.c | 6 +- tools/perf/trace/beauty/msg_flags.c | 8 +- tools/perf/trace/beauty/open_flags.c | 1 + tools/perf/trace/beauty/perf_event_open.c | 22 +- tools/perf/trace/beauty/pid.c | 5 +- tools/perf/trace/beauty/sched_policy.c | 8 +- tools/perf/trace/beauty/seccomp.c | 12 +- tools/perf/trace/beauty/signum.c | 6 +- tools/perf/trace/beauty/socket_type.c | 6 +- .../perf/{util => trace/beauty}/syscalltbl.c | 0 .../perf/{util => trace/beauty}/syscalltbl.h | 0 tools/perf/trace/beauty/tracepoints/Build | 22 + tools/perf/trace/beauty/waitid_options.c | 8 +- tools/perf/util/Build | 17 +- tools/perf/util/bpf-trace-summary.c | 2 +- tools/perf/util/env.c | 4 +- tools/perf/util/env.h | 1 + 41 files changed, 717 insertions(+), 504 deletions(-) rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%) create mode 100644 tools/perf/bpf_skel.mak create mode 100644 tools/perf/trace/beauty/fsconfig.c rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%) rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerRe: [PATCH v3 00/17] perf build: Reduce build time by nearly half
Quoted 81 lines · Namhyung Kim
Quoted 24 lines · Ian Rogers
This patch series refactors
Kbuildinternals, BPF skeleton generation, Python AST pre-computation, and foundational tooling dependencies across the perf tool build system. By eliminating umbrella target synchronization barriers, decoupling static library prerequisites, parallelizing single-core script generators, and eradicating redundant feature checks, this series unlocks absolute theoretical peak multi-core concurrency duringKbuildstartup.On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 49%:
Before:
real 0m29.006s user 2m46.019s sys 0m30.610sAfter:
real 0m14.782s user 2m39.527s sys 0m22.938sSaving 14.2 full seconds time per clean build. Furthermore, nothing to build incremental builds are improved by nearly 7x:
Before:
Quoted 8 lines
real 0m11.528s user 0m9.633s sys 0m6.965sAfter:
real 0m1.729s user 0m1.600s sys 0m0.884sI've quickly checked it with latency profiling like below:
$ perf record --latency -- make -C tools/perf$ perf report --latency -s commThe result looks like this.
Before:
#
# Samples: 715K of event 'cpu/cycles/Pu'
# Event count (approx.): 422452811481
## Latency Overhead Command # ........ ........ ............... # 45.28% 71.33% cc1 34.48% 16.92% python3 11.15% 2.21% ld 2.58% 1.51% x86_64-linux-gn 2.22% 0.99% cc1plus 0.71% 0.63% sh 0.69% 0.14% llvm-config 0.62% 0.56% clang 0.57% 4.40% shellcheck 0.44% 0.12% perlAfter:
#
# Samples: 709K of event 'cpu/cycles/Pu'
# Event count (approx.): 416654798495
## Latency Overhead Command # ........ ........ ............... # 64.99% 71.16% cc1 15.07% 1.81% ld 7.14% 17.59% python3 3.66% 1.53% x86_64-linux-gn 3.48% 0.75% cc1plus 1.11% 4.43% shellcheck 1.09% 0.74% sh 0.86% 0.59% clang 0.77% 0.12% perl 0.45% 0.23% make
Quoted 2 lines
Now I see a big drop in the latency from python. And the llvm-config doesn't show up in the top 10.
This looks good. What is "x86_64-linux-gn", and since we default off LIBPERL, why does perl show up in the commands?
Thanks,
Ian
Quoted 182 lines
Thanks,
NamhyungQuoted 179 lines
Summary of Patches:
1-3: Foundational Tooling & Fast-Path Feature Detection
- Exempts bpftool bootstrap from non-essential feature tests (LLVM, libbfd, libcap), saving 1.1s of sub-make fork overhead duringKbuildstartup.
- Integrates libdebuginfod directly intotest-all.c, allowing Make to skip individual feature check sub-make forks during AST parsing on fully configured workstations. Escapes $(shell ...) macro expansion to prevent unconditional sub-make forks.
- Fixes test-clang-bpf-co-re.bin feature check to correctly generate its target file on disk via atomic move (> [email protected] && mv [email protected] $@), allowingKbuildto perfectly cache the detection result and avoid continuous sub-make re-evaluations.4-6: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.7-10: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dumpvmlinux.hin the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
- Ensures benchmark skeleton intermediate .bpf.o files are cleanly removed during make clean, and adds bpf-skel-prepare to .PHONY.11-12: Foundational Linkage Optimization
- Eliminates redundant libbpf sub-make feature checks during static builds.
- Moves static libsymbol and libbpf library prerequisites out of the prepare step, ensuring libbpf headers are installed before compiling BPF-dependent tests.13-14:
jevents.pyConcurrency & Deduplication
- Splits the massive 2.8 MBbig_c_stringliteral out ofpmu-events.cinto a dedicatedpmu-events-string.ccompilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations. Addspmu-events-string.cto .gitignore and uses Make 4.0 compatible dependency chaining.
- Pre-populatesjevents.pyJSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms). Moves_init_workerto top-level scope to ensure clean pickling under spawn multiprocessing start methods.15: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.16-17: AST Parsing Optimization & Shell Fork Eradication
- Converts ZENS, ARMS, and INTELS in pmu-events/Build from recursive assignment (=) to simply expanded assignment (:=) and replaces model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once during AST parsing and evaluates path macros purely in memory, completely eradicating over 7,800 redundant sub-processes during out-of-tree build evaluation.
- Converts llvm-config shell queries in Makefile.config from recursive assignment (=) to simply expanded assignment (:=). This eliminates ~185 redundant sub-processes that were previously executed across object compilation dependency checks.Changes since v2:
- Dropped Patch 4 (tools scripts: Short-circuitCC_NO_CLANGcompiler probe in Makefile.include) to prevent potential cross-compilation regressions when CC and HOSTCC use different compilers.
- tools build (Patch 2): Escaped $(shell ...) macro expansion as $$(shell ...) inside definefeature_check_codeto safely defer sub-make execution until after eval parses the ifeq guard.
- tools build (Patch 3): Refactored test-clang-bpf-co-re.bin feature check recipe to redirect grep output to a temporary file and atomically move it upon success (> [email protected] && mv [email protected] $@), preventingKbuildfrom permanently caching failed detections due to 0-byte files.
- perf trace beauty (Patch 4): Updated commit description to accurately reflect the unconditional top-level recursive kbuild hook (perf-util-y += trace/beauty/).
- perf build (Patch 7): Added $(OUTPUT)bench/bpf_skel/.tmp to bpf-skel-clean in Makefile.perf to ensure intermediate benchmark skeleton .bpf.o artifacts are cleanly removed during make clean. Removed unusedbpf_skel_depsvariable from bpf_skel.mak.
- perf build (Patch 9): Added $(LIBBPF) as an explicit prerequisite to $(LIBPERF_TEST_IN) in Makefile.perf to guarantee libbpf headers are fully installed before compilingsigtrap.cor other BPF-dependent tests during parallel builds.
- perf build (Patch 10): Added bpf-skel-prepare to the .PHONY target list in Makefile.perf to ensure Make never incorrectly skips the target if a file or directory named bpf-skel-prepare accidentally exists in the build tree.
- perf pmu-events (Patch 13): Addedpmu-events/pmu-events-string.ctotools/perf/.gitignore. Replaced grouped targets (&:) with Make 4.0 compatible dependency chaining to guarantee backward compatibility with older Make versions (like 4.2.1) and prevent parallel builds from spawning multiple concurrentjevents.pyprocesses.
- perf pmu-events (Patch 14): Moved_init_workerfrom localmain()scope to the top-level module scope injevents.pyto ensure it can be cleanly pickled when ProcessPoolExecutor uses the spawn multiprocessing start method (avoiding AttributeError crashes).Ian Rogers (17):
bpftool build: Restrict feature tests during bootstrap compilation tools build: Integrate libdebuginfod into test-all fast path tools build: Fix test-clang-bpf-co-re.bin to generate target file perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulatevmlinux.hand bpftool in bpf_skel.mak perf build: Move static libbpf dependency out of prepare step perf build: Pre-generate BPF skeleton tooling during umbrella prepare
phase
perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds perf pmu-events: Splitbig_c_stringstorage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
perf pmu-events: Convert recursive shell assignments and macros to
Make built-ins
perf build: Convert llvm-config shell queries to simply expanded
variablestools/bpf/bpftool/Makefile | 5 + tools/build/Makefile.feature | 6 +- tools/build/feature/Makefile | 4 +- tools/build/feature/test-all.c | 5 + tools/perf/.gitignore | 1 + tools/perf/Build | 2 + tools/perf/Makefile.config | 19 +- tools/perf/Makefile.perf | 431 ++---------------- tools/perf/bench/Build | 6 + .../bpf_skel/bench_uprobe.bpf.c | 0 tools/perf/bench/uprobe.c | 2 +- tools/perf/bpf_skel.mak | 109 +++++ tools/perf/builtin-trace.c | 30 +- tools/perf/pmu-events/Build | 26 +- tools/perf/pmu-events/jevents.py | 56 ++- tools/perf/trace/beauty/Build | 280 ++++++++++++ tools/perf/trace/beauty/arch_errno_names.c | 2 + tools/perf/trace/beauty/arch_errno_names.sh | 2 +- tools/perf/trace/beauty/beauty.h | 60 +++ tools/perf/trace/beauty/eventfd.c | 6 +- tools/perf/trace/beauty/fsconfig.c | 5 + tools/perf/trace/beauty/futex_op.c | 6 +- tools/perf/trace/beauty/futex_val3.c | 6 +- tools/perf/trace/beauty/mmap.c | 24 +- tools/perf/trace/beauty/mode_t.c | 6 +- tools/perf/trace/beauty/msg_flags.c | 8 +- tools/perf/trace/beauty/open_flags.c | 1 + tools/perf/trace/beauty/perf_event_open.c | 22 +- tools/perf/trace/beauty/pid.c | 5 +- tools/perf/trace/beauty/sched_policy.c | 8 +- tools/perf/trace/beauty/seccomp.c | 12 +- tools/perf/trace/beauty/signum.c | 6 +- tools/perf/trace/beauty/socket_type.c | 6 +- .../perf/{util => trace/beauty}/syscalltbl.c | 0 .../perf/{util => trace/beauty}/syscalltbl.h | 0 tools/perf/trace/beauty/tracepoints/Build | 22 + tools/perf/trace/beauty/waitid_options.c | 8 +- tools/perf/util/Build | 17 +- tools/perf/util/bpf-trace-summary.c | 2 +- tools/perf/util/env.c | 4 +- tools/perf/util/env.h | 1 + 41 files changed, 717 insertions(+), 504 deletions(-) rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%) create mode 100644 tools/perf/bpf_skel.mak create mode 100644 tools/perf/trace/beauty/fsconfig.c rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%) rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)--
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerRe: [PATCH v3 00/17] perf build: Reduce build time by nearly half
Quoted 83 lines · Ian Rogers
Quoted 81 lines · Namhyung Kim
Quoted 24 lines · Ian Rogers
This patch series refactors
Kbuildinternals, BPF skeleton generation, Python AST pre-computation, and foundational tooling dependencies across the perf tool build system. By eliminating umbrella target synchronization barriers, decoupling static library prerequisites, parallelizing single-core script generators, and eradicating redundant feature checks, this series unlocks absolute theoretical peak multi-core concurrency duringKbuildstartup.On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 49%:
Before:
real 0m29.006s user 2m46.019s sys 0m30.610sAfter:
real 0m14.782s user 2m39.527s sys 0m22.938sSaving 14.2 full seconds time per clean build. Furthermore, nothing to build incremental builds are improved by nearly 7x:
Before:
Quoted 8 lines
real 0m11.528s user 0m9.633s sys 0m6.965sAfter:
real 0m1.729s user 0m1.600s sys 0m0.884sI've quickly checked it with latency profiling like below:
$ perf record --latency -- make -C tools/perf$ perf report --latency -s commThe result looks like this.
Before:
#
# Samples: 715K of event 'cpu/cycles/Pu'
# Event count (approx.): 422452811481
## Latency Overhead Command # ........ ........ ............... # 45.28% 71.33% cc1 34.48% 16.92% python3 11.15% 2.21% ld 2.58% 1.51% x86_64-linux-gn 2.22% 0.99% cc1plus 0.71% 0.63% sh 0.69% 0.14% llvm-config 0.62% 0.56% clang 0.57% 4.40% shellcheck 0.44% 0.12% perlAfter:
#
# Samples: 709K of event 'cpu/cycles/Pu'
# Event count (approx.): 416654798495
## Latency Overhead Command # ........ ........ ............... # 64.99% 71.16% cc1 15.07% 1.81% ld 7.14% 17.59% python3 3.66% 1.53% x86_64-linux-gn 3.48% 0.75% cc1plus 1.11% 4.43% shellcheck 1.09% 0.74% sh 0.86% 0.59% clang 0.77% 0.12% perl 0.45% 0.23% make
Quoted 8 lines
Quoted 2 lines
Now I see a big drop in the latency from python. And the llvm-config doesn't show up in the top 10.
This looks good. What is "x86_64-linux-gn", and since we default off LIBPERL, why does perl show up in the commands?
Thanks,
Ian
So, Sashiko reviews caught 2 regressions I introduced in v3. Given that people seem reasonably happy I propose the following for v4 to make the easy bits easy to land:
1) Drop patch 1 the bpftool change - hopefully someone on the BPF side can pull in an equivalent change so that we're not testing libbfd for disassembly support in a bootstrap version of bpftool that doesn't support disassembly.
2) Drop patch 2 on feature testing debuginfod and avoiding repeated tests we can come back to it.
3) Fix patch 3 "tools build: Fix test-clang-bpf-co-re.bin" to cover the other missed case and to make the dependency tracking better. We shouldn't have to re-feature test BPF features on every incremental build.
4) Patch 4 to 8, keep everything seems happy.
5) Drop patch 9 "perf build: Move static libbpf dependency out of prepare step." There's parallelism in the prepare step, so this likely wasn't gaining much build time and Sashiko seems to find many potential not really issues that suggest the gain isn't worth the pain.
6) Patch 10 needs a fix for a variable dropped from v2.
7) Patch 11 to 12, keep everything seems happy.
8) Patch 13, the header file declaring the variable seems to have gone AWOL. Needs a fix.
9) Patch 14, Sashiko is warning "Does this loop overwrite standard events from earlier architectures?" which would be a worry if (1) we cared much about building for >1 architecture, or (2) more than just arm64 were using the architecture standard files. I think we can address this problem later.
10) Patch 15 to 17, keep everything seems happy.
Thanks,
Ian
Quoted 182 lines
Quoted 182 lines
Thanks,
NamhyungQuoted 179 lines
Summary of Patches:
1-3: Foundational Tooling & Fast-Path Feature Detection
- Exempts bpftool bootstrap from non-essential feature tests (LLVM, libbfd, libcap), saving 1.1s of sub-make fork overhead duringKbuildstartup.
- Integrates libdebuginfod directly intotest-all.c, allowing Make to skip individual feature check sub-make forks during AST parsing on fully configured workstations. Escapes $(shell ...) macro expansion to prevent unconditional sub-make forks.
- Fixes test-clang-bpf-co-re.bin feature check to correctly generate its target file on disk via atomic move (> [email protected] && mv [email protected] $@), allowingKbuildto perfectly cache the detection result and avoid continuous sub-make re-evaluations.4-6: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.7-10: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dumpvmlinux.hin the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
- Ensures benchmark skeleton intermediate .bpf.o files are cleanly removed during make clean, and adds bpf-skel-prepare to .PHONY.11-12: Foundational Linkage Optimization
- Eliminates redundant libbpf sub-make feature checks during static builds.
- Moves static libsymbol and libbpf library prerequisites out of the prepare step, ensuring libbpf headers are installed before compiling BPF-dependent tests.13-14:
jevents.pyConcurrency & Deduplication
- Splits the massive 2.8 MBbig_c_stringliteral out ofpmu-events.cinto a dedicatedpmu-events-string.ccompilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations. Addspmu-events-string.cto .gitignore and uses Make 4.0 compatible dependency chaining.
- Pre-populatesjevents.pyJSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms). Moves_init_workerto top-level scope to ensure clean pickling under spawn multiprocessing start methods.15: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.16-17: AST Parsing Optimization & Shell Fork Eradication
- Converts ZENS, ARMS, and INTELS in pmu-events/Build from recursive assignment (=) to simply expanded assignment (:=) and replaces model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once during AST parsing and evaluates path macros purely in memory, completely eradicating over 7,800 redundant sub-processes during out-of-tree build evaluation.
- Converts llvm-config shell queries in Makefile.config from recursive assignment (=) to simply expanded assignment (:=). This eliminates ~185 redundant sub-processes that were previously executed across object compilation dependency checks.Changes since v2:
- Dropped Patch 4 (tools scripts: Short-circuitCC_NO_CLANGcompiler probe in Makefile.include) to prevent potential cross-compilation regressions when CC and HOSTCC use different compilers.
- tools build (Patch 2): Escaped $(shell ...) macro expansion as $$(shell ...) inside definefeature_check_codeto safely defer sub-make execution until after eval parses the ifeq guard.
- tools build (Patch 3): Refactored test-clang-bpf-co-re.bin feature check recipe to redirect grep output to a temporary file and atomically move it upon success (> [email protected] && mv [email protected] $@), preventingKbuildfrom permanently caching failed detections due to 0-byte files.
- perf trace beauty (Patch 4): Updated commit description to accurately reflect the unconditional top-level recursive kbuild hook (perf-util-y += trace/beauty/).
- perf build (Patch 7): Added $(OUTPUT)bench/bpf_skel/.tmp to bpf-skel-clean in Makefile.perf to ensure intermediate benchmark skeleton .bpf.o artifacts are cleanly removed during make clean. Removed unusedbpf_skel_depsvariable from bpf_skel.mak.
- perf build (Patch 9): Added $(LIBBPF) as an explicit prerequisite to $(LIBPERF_TEST_IN) in Makefile.perf to guarantee libbpf headers are fully installed before compilingsigtrap.cor other BPF-dependent tests during parallel builds.
- perf build (Patch 10): Added bpf-skel-prepare to the .PHONY target list in Makefile.perf to ensure Make never incorrectly skips the target if a file or directory named bpf-skel-prepare accidentally exists in the build tree.
- perf pmu-events (Patch 13): Addedpmu-events/pmu-events-string.ctotools/perf/.gitignore. Replaced grouped targets (&:) with Make 4.0 compatible dependency chaining to guarantee backward compatibility with older Make versions (like 4.2.1) and prevent parallel builds from spawning multiple concurrentjevents.pyprocesses.
- perf pmu-events (Patch 14): Moved_init_workerfrom localmain()scope to the top-level module scope injevents.pyto ensure it can be cleanly pickled when ProcessPoolExecutor uses the spawn multiprocessing start method (avoiding AttributeError crashes).Ian Rogers (17):
bpftool build: Restrict feature tests during bootstrap compilation tools build: Integrate libdebuginfod into test-all fast path tools build: Fix test-clang-bpf-co-re.bin to generate target file perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulatevmlinux.hand bpftool in bpf_skel.mak perf build: Move static libbpf dependency out of prepare step perf build: Pre-generate BPF skeleton tooling during umbrella prepare
phase
perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds perf pmu-events: Splitbig_c_stringstorage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
perf pmu-events: Convert recursive shell assignments and macros to
Make built-ins
perf build: Convert llvm-config shell queries to simply expanded
variablestools/bpf/bpftool/Makefile | 5 + tools/build/Makefile.feature | 6 +- tools/build/feature/Makefile | 4 +- tools/build/feature/test-all.c | 5 + tools/perf/.gitignore | 1 + tools/perf/Build | 2 + tools/perf/Makefile.config | 19 +- tools/perf/Makefile.perf | 431 ++---------------- tools/perf/bench/Build | 6 + .../bpf_skel/bench_uprobe.bpf.c | 0 tools/perf/bench/uprobe.c | 2 +- tools/perf/bpf_skel.mak | 109 +++++ tools/perf/builtin-trace.c | 30 +- tools/perf/pmu-events/Build | 26 +- tools/perf/pmu-events/jevents.py | 56 ++- tools/perf/trace/beauty/Build | 280 ++++++++++++ tools/perf/trace/beauty/arch_errno_names.c | 2 + tools/perf/trace/beauty/arch_errno_names.sh | 2 +- tools/perf/trace/beauty/beauty.h | 60 +++ tools/perf/trace/beauty/eventfd.c | 6 +- tools/perf/trace/beauty/fsconfig.c | 5 + tools/perf/trace/beauty/futex_op.c | 6 +- tools/perf/trace/beauty/futex_val3.c | 6 +- tools/perf/trace/beauty/mmap.c | 24 +- tools/perf/trace/beauty/mode_t.c | 6 +- tools/perf/trace/beauty/msg_flags.c | 8 +- tools/perf/trace/beauty/open_flags.c | 1 + tools/perf/trace/beauty/perf_event_open.c | 22 +- tools/perf/trace/beauty/pid.c | 5 +- tools/perf/trace/beauty/sched_policy.c | 8 +- tools/perf/trace/beauty/seccomp.c | 12 +- tools/perf/trace/beauty/signum.c | 6 +- tools/perf/trace/beauty/socket_type.c | 6 +- .../perf/{util => trace/beauty}/syscalltbl.c | 0 .../perf/{util => trace/beauty}/syscalltbl.h | 0 tools/perf/trace/beauty/tracepoints/Build | 22 + tools/perf/trace/beauty/waitid_options.c | 8 +- tools/perf/util/Build | 17 +- tools/perf/util/bpf-trace-summary.c | 2 +- tools/perf/util/env.c | 4 +- tools/perf/util/env.h | 1 + 41 files changed, 717 insertions(+), 504 deletions(-) rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%) create mode 100644 tools/perf/bpf_skel.mak create mode 100644 tools/perf/trace/beauty/fsconfig.c rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%) rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)--
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerv4 cover perf build: Reduce build time by nearly half
This patch series refactors Kbuild internals, BPF skeleton generation, Python AST pre-computation, and foundational tooling dependencies across the perf tool build system. By eliminating umbrella target synchronization barriers, decoupling static library prerequisites, parallelizing single-core script generators, and eradicating redundant feature checks, this series unlocks absolute theoretical peak multi-core concurrency during Kbuild startup.
On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 44%:
Before:
real 0m29.006s
user 2m46.019s
sys 0m30.610sAfter:
real 0m16.091s
user 2m40.135s
sys 0m25.740sSaving 12.9 full seconds time per clean build. Furthermore, nothing to build incremental builds are improved by nearly 7x:
Before:
real 0m11.528s
user 0m9.633s
sys 0m6.965sAfter:
real 0m1.717s
user 0m1.682s
sys 0m0.960sSummary of Patches:
1: Fast-Path Feature Detection
- Refactors test-clang-bpf-co-re.bin and test-bpftool-skeletons.bin feature checks to redirect grep output to .make.output and touch $@ upon success (> $(@:.bin=.make.output) 2>&1 && touch $@). This perfectly matches standard Kbuild feature check conventions and ensures the target files are touched on disk purely upon success, allowing Kbuild to cache positive detections and avoid continuous sub-make re-evaluations during incremental builds. For test-clang-bpf-co-re.bin, adds explicit source prerequisite test-clang-bpf-co-re.c and simplifies Clang recipe using $<.
2-4: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.
5-7: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dump vmlinux.h in the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
- Ensures benchmark skeleton intermediate .bpf.o files are cleanly removed during make clean, and adds bpf-skel-prepare to .PHONY.
8-9: Foundational Linkage Optimization
- Moves static libsymbol library prerequisites out of the prepare step.
- Eliminates redundant libbpf sub-make feature checks during static builds.
10-11: jevents.py Concurrency & Deduplication
- Splits the massive 2.8 MB big_c_string literal out of pmu-events.c into a dedicated pmu-events-string.c compilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations. Adds pmu-events-string.c to .gitignore, includes pmu-events.h for global extern declarations, defers file closures to ensure identical timestamps, and uses Make 4.0 compatible dependency chaining with robust self-correction checks.
- Pre-populates jevents.py JSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms). Moves _init_worker to top-level scope to ensure clean pickling under spawn multiprocessing start methods.
12: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.
13-14: AST Parsing Optimization & Shell Fork Eradication
- Converts ZENS, ARMS, and INTELS in pmu-events/Build from recursive assignment (=) to simply expanded assignment (:=) and replaces model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once during AST parsing and evaluates path macros purely in memory, completely eradicating over 7,800 redundant sub-processes during out-of-tree build evaluation.
- Converts llvm-config shell queries in Makefile.config from recursive assignment (=) to simply expanded assignment (:=). This eliminates ~185 redundant sub-processes that were previously executed across object compilation dependency checks.
Changes since v3:
- Streamlined series to 14 patches by dropping Patches 1, 2, and 9 to focus on the most uncontroversial, high-impact architectural gains.
- tools build (Patch 1): Refactored test-bpftool-skeletons.bin and test-clang-bpf-co-re.bin feature check recipes to match standard Kbuild conventions by redirecting grep output to .make.output and touching $@ upon success (> $(@:.bin=.make.output) 2>&1 && touch $@). Added explicit source file prerequisite test-clang-bpf-co-re.c and simplified Clang recipe using $<.
- perf build (Patch 7): Fixed missing prerequisite on bpf-skel-prepare in bpf_skel.mak by making it depend directly on explicit $(BPFTOOL) $(VMLINUX_H) prerequisites, preventing it from executing as a no-op during prepare.
- perf pmu-events (Patch 10): Added extern const char big_c_string[]; declaration to pmu-events.h and included it in output_string_file to satisfy Clang -Wmissing-variable-declarations compiler warnings. Deferred closing output_string_file until the absolute tail of main() to ensure identical timestamps with output_file, preventing redundant incremental rebuilds. Updated the secondary target rule in pmu-events/Build to verify the file exists on disk and force a rebuild if manually deleted, ensuring 100% self-correcting builds.
Ian Rogers (14):
tools build: Fix feature checks to touch target files on success perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak perf build: Pre-generate BPF skeleton tooling during umbrella prepare
phase
perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds perf pmu-events: Split big_c_string storage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
perf pmu-events: Convert recursive shell assignments and macros to
Make built-ins
perf build: Convert llvm-config shell queries to simply expanded
variables
tools/build/feature/Makefile | 8 +-
tools/perf/.gitignore | 1 +
tools/perf/Build | 2 +
tools/perf/Makefile.config | 19 +-
tools/perf/Makefile.perf | 423 ++----------------
tools/perf/bench/Build | 6 +
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 109 +++++
tools/perf/builtin-trace.c | 32 +-
tools/perf/pmu-events/Build | 26 +-
tools/perf/pmu-events/jevents.py | 57 ++-
tools/perf/pmu-events/pmu-events.h | 2 +
tools/perf/trace/beauty/Build | 276 ++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 +++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 5 +-
tools/perf/trace/beauty/futex_val3.c | 5 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 2 +
tools/perf/trace/beauty/perf_event_open.c | 21 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 21 +
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 17 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 -
tools/perf/util/env.h | 1 +
39 files changed, 685 insertions(+), 514 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.mak
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 1/14 tools build: Fix feature checks to touch target files on success
In tools/build/feature/Makefile, test-clang-bpf-co-re.bin and test-bpftool-skeletons.bin redirected grep output but never touched or created the $@ target file upon success.
Because the target file was never created on disk, Kbuild could never cache the result of the check. Consequently, Make treated the prerequisite as missing and continuously re-executed the Clang BPF backend and bpftool feature checks on every single sub-make evaluation during build startup.
Refactor both feature check recipes to redirect grep output to .make.output and touch $@ upon success (> $(@:.bin=.make.output) 2>&1 && touch $@). This perfectly matches standard Kbuild feature check conventions and ensures the target file is touched on disk purely upon success, allowing Kbuild to perfectly cache positive detections and avoid redundant sub-make forks. For test-clang-bpf-co-re.bin, add the explicit source file prerequisite test-clang-bpf-co-re.c and simplify the Clang recipe using $<.
--- tools/build/feature/Makefile | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
tools/build/feature/Makefile +4 -4
@@ -383,9 +383,9 @@ $(OUTPUT)test-libaio.bin: $(OUTPUT)test-libzstd.bin: $(BUILD) -lzstd -$(OUTPUT)test-clang-bpf-co-re.bin: - $(CLANG) -S -g --target=bpf -o - $(patsubst %.bin,%.c,$(@F)) | \ - grep BTF_KIND_VAR +$(OUTPUT)test-clang-bpf-co-re.bin: test-clang-bpf-co-re.c + $(CLANG) -S -g --target=bpf -o - $< | \ + grep BTF_KIND_VAR > $(@:.bin=.make.output) 2>&1 && touch $@ $(OUTPUT)test-file-handle.bin: $(BUILD)
@@ -398,7 +398,7 @@ $(OUTPUT)test-libopenssl.bin: $(OUTPUT)test-bpftool-skeletons.bin: $(SYSTEM_BPFTOOL) version | grep '^features:.*skeletons' \ - > $(@:.bin=.make.output) 2>&1 + > $(@:.bin=.make.output) 2>&1 && touch $@ # Testing Rust is special: we don't compile anything, it's enough to check the # compiler presence. Compiling a test code for this purposes is problematic,
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v4 01/14] tools build: Fix feature checks to touch target files on success
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] Build Artifact Leak / Make Clean Breakage - [Medium] Error Reporting / Diagnostics logic fails to…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] Build Artifact Leak / Make Clean Breakage
- [Medium] Error Reporting / Diagnostics logic fails to capture actual compiler/tool errors --
commit 566bad233a12ce681515edad91fc35069a93ec9e
Author: Ian Rogers <[email protected]>
tools build: Fix feature checks to touch target files on success
This commit modifies the clang bpf and bpftool feature checks to physically touch their target files on disk upon success. It also adds redirection to create a .make.output file, allowing Kbuild to cache positive detections and avoid redundant sub-make executions during build startup.
Quoted 3 lines
tools/build/feature/Makefile +0
-0
[ ... ]
tools/build/feature/Makefile $(OUTPUT)test-libaio.bin: · line 388
Quoted 10 lines
tools/build/feature/Makefile +3
-3@@ -383,9 +383,9 @@ $(OUTPUT)test-libaio.bin: $(OUTPUT)test-libzstd.bin: $(BUILD) -lzstd -$(OUTPUT)test-clang-bpf-co-re.bin: - $(CLANG) -S -g --target=bpf -o - $(patsubst %.bin,%.c,$(@F)) | \ - grep BTF_KIND_VAR +$(OUTPUT)test-clang-bpf-co-re.bin: test-clang-bpf-co-re.c + $(CLANG) -S -g --target=bpf -o - $< | \ + grep BTF_KIND_VAR > $(@:.bin=.make.output) 2>&1 && touch $@
Does this redirection only capture the output of grep?
Because the output redirection is applied strictly to the final command in the shell pipeline, the stderr from the compiler might flow to the parent shell instead. Since the parent Kbuild feature check suppresses output, this regression could prevent actual compiler errors from being captured in the .make.output file, defeating its purpose for diagnostics.
[ ... ]
tools/build/feature/Makefile $(OUTPUT)test-libopenssl.bin: · line 401
Quoted 6 lines
tools/build/feature/Makefile +1
-1@@ -398,7 +398,7 @@ $(OUTPUT)test-libopenssl.bin: $(OUTPUT)test-bpftool-skeletons.bin: $(SYSTEM_BPFTOOL) version | grep '^features:.*skeletons' \ - > $(@:.bin=.make.output) 2>&1 + > $(@:.bin=.make.output) 2>&1 && touch $@
Since test-bpftool-skeletons.bin is now physically created on disk upon success, does it need to be added to the FILES list at the top of the Makefile?
The make clean target relies exclusively on the FILES variable to remove generated artifacts. If this binary is missing from that list, the generated file will permanently survive make clean. This regression could permanently poison the Kbuild cache, causing it to treat the feature as present even if the underlying bpftool environment is later changed or uninstalled.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Ian Rogers
reviewerPATCH v4 2/14 perf trace beauty: Make beauty generated C code standalone .o files
In reply to Ian RogersPreviously, builtin-trace.c directly included 15 embedded C files (e.g. trace/beauty/mmap.c and fsconfig_arrays.c), which in turn depend on dozens of generated beauty script arrays. To satisfy these embedded inclusions, the global Makefile.perf would define all the generator variables/rules and include them in the prepare umbrella target, choking parallel build startup.
Furthermore, tools/perf/util/syscalltbl.c included its own generated mapper, and util/env.c conditionally included arch_errno_names.c inline, splitting consumers across directories and preventing clean Make encapsulation.
Refactor the framework to achieve better encapsulation:
1. Move util/syscalltbl.[ch] into trace/beauty/ to co-locate with all generated code consumers.
2. Create fsconfig.c and flatten embedded beauty .c files to compile as independent standalone objects via trace/beauty/Build, exporting their formatting functions via beauty.h and env.h. Switch arch_errno_names.o and syscalltbl.o assignments directly to perf-util-y and add an unconditional top-level recursive kbuild hook (perf-util-y += trace/beauty/) to compile them into libperf-util.a, resolving remote linkage for util/env.c, util/bpf-trace-summary.c, and standalone python extensions.
3. Bridge private opaque references (struct trace) securely via accessors trace__show_zeros() and trace__host(), avoiding header entanglements.
4. Consolidate all generator variables, script paths, and array generation rules entirely out of Makefile.perf and place them directly inside the exact local Build files where their output objects are compiled (trace/beauty/Build and trace/beauty/tracepoints/Build), binding prerequisites locally. Use directly inside
generator recipes to guarantee dynamic directory creation before script redirection, and append across all rules to print
clean, standardized GEN ... file.c output during compilation.
5. Clean up clean target to recursively remove the generated directory instead of relying on dozens of individual variables.
This unchokes the "prepare" target parallel barrier, allows make to evaluate generation scripts purely locally where consumed, and flattens the tracepoint formatting architecture.
Testing a parallel build (make -j28 all from scratch) shows improvements:
Before:
real 0m28.689s
user 2m38.490s
sys 0m30.148sAfter:
real 0m27.642s
user 2m32.356s
sys 0m26.683sSo reclaiming ~9.6 seconds of raw CPU time and over 1 full second off overall real-world build latency, by overlapping sub-make startup and avoiding top-level double-parsing overhead.
---
tools/perf/Build | 2 +
tools/perf/Makefile.perf | 282 +-----------------
tools/perf/builtin-trace.c | 32 +-
tools/perf/trace/beauty/Build | 276 +++++++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 ++++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 5 +-
tools/perf/trace/beauty/futex_val3.c | 5 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 2 +
tools/perf/trace/beauty/perf_event_open.c | 21 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 21 ++
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 4 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 -
tools/perf/util/env.h | 1 +
29 files changed, 447 insertions(+), 368 deletions(-)
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)tools/perf/Build +2 -0
@@ -34,6 +34,8 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-$(CONFIG_TRACE) += trace/beauty/ endif +perf-util-y += trace/beauty/ + perf-$(CONFIG_LIBELF) += builtin-probe.o perf-bench-y += bench/
tools/perf/Makefile.perf +2 -280
300 lines not on this page · raw message
tools/perf/builtin-trace.c +12 -20
@@ -60,12 +60,13 @@#include "callchain.h" #include "print_binary.h" #include "string2.h" -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "../perf.h" #include "trace_augment.h" #include "dwarf-regs.h" #include <errno.h> +#include <sys/stat.h> #include <inttypes.h> #include <poll.h> #include <signal.h>
@@ -234,6 +235,16 @@ struct trace {const char *uid_str; }; +bool trace__show_zeros (const struct trace *trace) +{ +return trace->show_zeros; +} + +struct machine *trace__host (const struct trace *trace) +{ +return trace->host; +} +static void trace__load_vmlinux_btf (struct trace *trace__maybe_unused ) {#ifdef HAVE_LIBBPF_SUPPORT
@@ -776,10 +787,6 @@ static const char *fsmount_flags[] = { }; static DEFINE_STRARRAY(fsmount_flags, "FSMOUNT_"); -#include "trace/beauty/generated/fsconfig_arrays.c" - -static DEFINE_STRARRAY(fsconfig_cmds, "FSCONFIG_"); -static const char *epoll_ctl_ops[] = {"ADD" ,"DEL" ,"MOD" , }; static DEFINE_STRARRAY_OFFSET(epoll_ctl_ops, "EPOLL_CTL_", 1);
@@ -1128,21 +1135,6 @@ static bool syscall_arg__strtoul_btf_type(char *bf __maybe_unused, size_t size _ .parm = &strarray__##array, \ .show_zero = true, } -#include "trace/beauty/eventfd.c" -#include "trace/beauty/futex_op.c" -#include "trace/beauty/futex_val3.c" -#include "trace/beauty/mmap.c" -#include "trace/beauty/mode_t.c" -#include "trace/beauty/msg_flags.c" -#include "trace/beauty/open_flags.c" -#include "trace/beauty/perf_event_open.c" -#include "trace/beauty/pid.c" -#include "trace/beauty/sched_policy.c" -#include "trace/beauty/seccomp.c" -#include "trace/beauty/signum.c" -#include "trace/beauty/socket_type.c" -#include "trace/beauty/waitid_options.c" -static const struct syscall_fmt syscall_fmts[] = { { .name ="access" , .arg = { [1 ] = { .scnprintf =SCA_ACCMODE ,/* mode */ }, }, },
tools/perf/trace/beauty/Build +276 -0
285 lines not on this page · raw message
tools/perf/trace/beauty/arch_errno_names.c +2 -0
@@ -1 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "util/env.h" #include "trace/beauty/generated/arch_errno_name_array.c"
tools/perf/trace/beauty/arch_errno_names.sh +1 -1
@@ -57,7 +57,7 @@ create_arch_errno_table_func() archlist="$1" default="$2" - printf 'static arch_syscalls__strerrno_t *\n' + printf 'arch_syscalls__strerrno_t *\n' printf 'arch_syscalls__strerrno_function(const char *arch)\n' printf '{\n' for arch in $archlist; do
tools/perf/trace/beauty/beauty.h +60 -0
70 lines not on this page · raw message
tools/perf/trace/beauty/eventfd.c +3 -3
@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#ifndef EFD_SEMAPHORE #define EFD_SEMAPHORE 1#endif
@@ -11,7 +13,7 @@#define EFD_CLOEXEC 02000000#endif -static size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="EFD_" ;
@@ -35,5 +37,3 @@ static size_t syscall_arg__scnprintf_eventfd_flags(char *bf, size_t size, structreturn printed; } - -#define SCA_EFD_FLAGS syscall_arg__scnprintf_eventfd_flags
tools/perf/trace/beauty/fsconfig.c new file +5 -0
@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "trace/beauty/generated/fsconfig_arrays.c" + +DEFINE_STRARRAY (fsconfig_cmds,"FSCONFIG_" );
tools/perf/trace/beauty/futex_op.c +2 -3
@@ -1,4 +1,5 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" #include <linux/futex.h> #ifndef FUTEX_WAIT_BITSET
@@ -17,7 +18,7 @@#define FUTEX_CLOCK_REALTIME 256#endif -static size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="FUTEX_" ;
@@ -59,5 +60,3 @@ static size_t syscall_arg__scnprintf_futex_op(char *bf, size_t size, struct syscreturn printed; } - -#define SCA_FUTEX_OP syscall_arg__scnprintf_futex_op
tools/perf/trace/beauty/futex_val3.c +2 -3
@@ -1,11 +1,12 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" #include <linux/futex.h> #ifndef FUTEX_BITSET_MATCH_ANY #define FUTEX_BITSET_MATCH_ANY 0xffffffff#endif -static size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) {const char *prefix ="FUTEX_BITSET_" ;unsigned int bitset = arg->val;
@@ -15,5 +16,3 @@ static size_t syscall_arg__scnprintf_futex_val3(char *bf, size_t size, struct syreturn scnprintf (bf, size,"%#xd" , bitset); } - -#define SCA_FUTEX_VAL3 syscall_arg__scnprintf_futex_val3
tools/perf/trace/beauty/mmap.c +11 -13
@@ -1,5 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 #include <linux/log2.h> +#include "trace/beauty/beauty.h" #include "trace/beauty/generated/mmap_prot_array.c" static DEFINE_STRARRAY(mmap_prot, "PROT_");
@@ -8,8 +9,7 @@ static size_t mmap__scnprintf_prot(unsigned long prot, char *bf, size_t size, bo {return strarray__scnprintf_flags (&strarray__mmap_prot, bf, size, show_prefix, prot); } - -static size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) {unsigned long prot = arg->val;
@@ -19,18 +19,18 @@ static size_t syscall_arg__scnprintf_mmap_prot(char *bf, size_t size, struct sysreturn mmap__scnprintf_prot (prot, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_PROT syscall_arg__scnprintf_mmap_prot +#include "trace/beauty/generated/mmap_flags_array.c" -static DEFINE_STRARRAY(mmap_flags, "MAP_"); +DEFINE_STRARRAY (mmap_flags,"MAP_" );static size_t mmap__scnprintf_flags (unsigned long flags,char *bf,size_t size,bool show_prefix) {return strarray__scnprintf_flags (&strarray__mmap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, +struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -40,7 +40,7 @@ static size_t syscall_arg__scnprintf_mmap_flags(char *bf, size_t size,return mmap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_FLAGS syscall_arg__scnprintf_mmap_flags +#include "trace/beauty/generated/mremap_flags_array.c" static DEFINE_STRARRAY(mremap_flags, "MREMAP_");
@@ -50,7 +50,7 @@ static size_t mremap__scnprintf_flags(unsigned long flags, char *bf, size_t sizereturn strarray__scnprintf_flags (&strarray__mremap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) {unsigned long flags = arg->val;
@@ -60,7 +60,7 @@ static size_t syscall_arg__scnprintf_mremap_flags(char *bf, size_t size, structreturn mremap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MREMAP_FLAGS syscall_arg__scnprintf_mremap_flags +static size_t madvise__scnprintf_behavior (int behavior,char *bf,size_t size) {
@@ -73,10 +73,8 @@ static size_t madvise__scnprintf_behavior(int behavior, char *bf, size_t size)return scnprintf (bf, size,"%#" , behavior); } -static size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, +struct syscall_arg *arg) {return madvise__scnprintf_behavior (arg->val, bf, size); } - -#define SCA_MADV_BHV syscall_arg__scnprintf_madvise_behavior
tools/perf/trace/beauty/mode_t.c +3 -3
@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/stat.h> #include <unistd.h>
@@ -20,7 +22,7 @@#define S_IXUGO (S_IXUSR|S_IXGRP|S_IXOTH)#endif -static size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="S_" ;
@@ -67,5 +69,3 @@ static size_t syscall_arg__scnprintf_mode_t(char *bf, size_t size, struct syscalreturn printed; } - -#define SCA_MODE_T syscall_arg__scnprintf_mode_t
tools/perf/trace/beauty/msg_flags.c +4 -4
@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/socket.h>
@@ -27,8 +29,8 @@# define MSG_CMSG_CLOEXEC 0x40000000#endif -static size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="MSG_" ;
@@ -72,5 +74,3 @@ static size_t syscall_arg__scnprintf_msg_flags(char *bf, size_t size,return printed; } - -#define SCA_MSG_FLAGS syscall_arg__scnprintf_msg_flags
tools/perf/trace/beauty/open_flags.c +2 -0
@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h>
tools/perf/trace/beauty/perf_event_open.c +10 -11
@@ -1,4 +1,8 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "util/evsel_fprintf.h" +#include <linux/perf_event.h> +#ifndef PERF_FLAG_FD_NO_GROUP # define PERF_FLAG_FD_NO_GROUP (1UL << 0)#endif
@@ -15,8 +19,8 @@# define PERF_FLAG_FD_CLOEXEC (1UL << 3)/* O_CLOEXEC */ #endif -static size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="PERF_" ;
@@ -43,7 +47,7 @@ static size_t syscall_arg__scnprintf_perf_flags(char *bf, size_t size,return printed; } -#define SCA_PERF_FLAGS syscall_arg__scnprintf_perf_flags +struct attr_fprintf_args {size_t size, printed;
@@ -76,19 +80,14 @@ static size_t perf_event_attr___scnprintf(struct perf_event_attr *attr, char *bfstatic size_t syscall_arg__scnprintf_augmented_perf_event_attr (struct syscall_arg *arg,char *bf,size_t size) { -return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, arg->trace->show_zeros); +return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, +trace__show_zeros (arg->trace)); } -static size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) {if (arg->augmented.args)return syscall_arg__scnprintf_augmented_perf_event_attr (arg, bf, size);return scnprintf (bf, size,"%#lx" , arg->val); } - -#define SCA_PERF_ATTR syscall_arg__scnprintf_perf_event_attr -// 'argname' is just documentational at this point, to remove the previous comment with that info -#define SCA_PERF_ATTR_FROM_USER (argname) \ - { .scnprintf = SCA_PERF_ATTR, \ - .from_user = true, }
tools/perf/trace/beauty/pid.c +4 -1
@@ -1,11 +1,14 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "util/machine.h" +#include "util/thread.h" size_t syscall_arg__scnprintf_pid (char *bf,size_t size,struct syscall_arg *arg) {int pid = arg->val;struct trace *trace = arg->trace;size_t printed =scnprintf (bf, size,"%d" , pid); -struct thread *thread =machine__findnew_thread (trace->host, pid, pid); +struct thread *thread =machine__findnew_thread (trace__host (trace), pid, pid);if (thread !=NULL ) {if (!thread__comm_set (thread))
tools/perf/trace/beauty/sched_policy.c +4 -4
@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sched.h> /*
@@ -14,8 +16,8 @@#define SCHED_RESET_ON_FORK 0x40000000#endif -static size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SCHED_" ;
@@ -46,5 +48,3 @@ static size_t syscall_arg__scnprintf_sched_policy(char *bf, size_t size,return printed; } - -#define SCA_SCHED_POLICY syscall_arg__scnprintf_sched_policy
tools/perf/trace/beauty/seccomp.c +6 -6
@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#ifndef SECCOMP_SET_MODE_STRICT #define SECCOMP_SET_MODE_STRICT 0#endif
@@ -6,7 +8,7 @@#define SECCOMP_SET_MODE_FILTER 1#endif -static size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_SET_MODE_" ;
@@ -24,14 +26,14 @@ static size_t syscall_arg__scnprintf_seccomp_op(char *bf, size_t size, struct syreturn printed; } -#define SCA_SECCOMP_OP syscall_arg__scnprintf_seccomp_op +#ifndef SECCOMP_FILTER_FLAG_TSYNC #define SECCOMP_FILTER_FLAG_TSYNC 1#endif -static size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_FILTER_FLAG_" ;
@@ -51,5 +53,3 @@ static size_t syscall_arg__scnprintf_seccomp_flags(char *bf, size_t size,return printed; } - -#define SCA_SECCOMP_FLAGS syscall_arg__scnprintf_seccomp_flags
tools/perf/trace/beauty/signum.c +3 -3
@@ -1,7 +1,9 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <signal.h> -static size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SIG" ;
@@ -53,5 +55,3 @@ static size_t syscall_arg__scnprintf_signum(char *bf, size_t size, struct syscalreturn scnprintf (bf, size,"%#x" , sig); } - -#define SCA_SIGNUM syscall_arg__scnprintf_signum
tools/perf/trace/beauty/socket_type.c +3 -3
@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/socket.h>
@@ -18,7 +20,7 @@#define SOCK_TYPE_MASK 0xf#endif -static size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SOCK_" ;
@@ -59,5 +61,3 @@ static size_t syscall_arg__scnprintf_socket_type(char *bf, size_t size, struct sreturn printed; } - -#define SCA_SK_TYPE syscall_arg__scnprintf_socket_type
tools/perf/trace/beauty/syscalltbl.c renamed from tools/perf/util/syscalltbl.c +0 -0
tools/perf/trace/beauty/syscalltbl.h renamed from tools/perf/util/syscalltbl.h +0 -0
tools/perf/trace/beauty/tracepoints/Build +21 -0
@@ -1,2 +1,23 @@ perf-y += x86_irq_vectors.o perf-y += x86_msr.o + +beauty_outdir := $(OUTPUT)trace/beauty/generated +beauty_arch_asm_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/asm/ +x86_arch_asm_dir := $(srctree)/tools/arch/x86/include/asm/ + +x86_arch_irq_vectors_array := $(beauty_outdir)/x86_arch_irq_vectors_array.c +x86_arch_irq_vectors_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_irq_vectors.sh + +$(x86_arch_irq_vectors_array): $(beauty_arch_asm_dir)/irq_vectors.h $(x86_arch_irq_vectors_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_irq_vectors_tbl)' $(beauty_arch_asm_dir) > $@ + +x86_arch_MSRs_array := $(beauty_outdir)/x86_arch_MSRs_array.c +x86_arch_MSRs_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_msr.sh + +$(x86_arch_MSRs_array): $(x86_arch_asm_dir)/msr-index.h $(x86_arch_MSRs_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_MSRs_tbl)' $(x86_arch_asm_dir) > $@ + +$(OUTPUT)trace/beauty/tracepoints/x86_irq_vectors.o: $(x86_arch_irq_vectors_array) +$(OUTPUT)trace/beauty/tracepoints/x86_msr.o: $(x86_arch_MSRs_array)
tools/perf/trace/beauty/waitid_options.c +4 -4
@@ -1,9 +1,11 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/wait.h> -static size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="W" ;
@@ -25,5 +27,3 @@ static size_t syscall_arg__scnprintf_waitid_options(char *bf, size_t size,return printed; } - -#define SCA_WAITID_OPTIONS syscall_arg__scnprintf_waitid_options
tools/perf/util/Build +3 -1
@@ -75,7 +75,7 @@ perf-util-y += sample.o perf-util-y += sample-raw.o perf-util-y += s390-sample-raw.o perf-util-y += amd-sample-raw.o -perf-util-$(CONFIG_TRACE) += syscalltbl.o + perf-util-y += ordered-events.o perf-util-y += namespaces.o perf-util-y += comm.o
@@ -441,3 +441,5 @@ $(OUTPUT)%.pylint_log: % $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false) perf-util-y += $(PYLINT_TEST_LOGS) + +
tools/perf/util/bpf-trace-summary.c +1 -1
@@ -6,7 +6,7 @@#include <stdlib.h> #include "dwarf-regs.h" /* for EM_HOST */ -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "util/cgroup.h" #include "util/hashmap.h" #include "util/trace.h"
tools/perf/util/env.c +0 -4
@@ -635,10 +635,6 @@ const char *perf_env__arch(struct perf_env *env)return normalize_arch (arch_name); } -#if defined(HAVE_LIBTRACEEVENT ) -#include "trace/beauty/arch_errno_names.c" -#endif -const char *perf_env__arch_strerrno (struct perf_env *env__maybe_unused ,int err__maybe_unused ) {#if defined(HAVE_LIBTRACEEVENT )
tools/perf/util/env.h +1 -0
@@ -189,6 +189,7 @@ void cpu_cache_level__free(struct cpu_cache_level *cache);const char *perf_env__arch (struct perf_env *env);const char *perf_env__arch_strerrno (struct perf_env *env,int err); +arch_syscalls__strerrno_t *arch_syscalls__strerrno_function (const char *arch);const char *perf_env__cpuid (struct perf_env *env);const char *perf_env__raw_arch (struct perf_env *env);int perf_env__nr_cpus_avail (struct perf_env *env);
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Namhyung Kim
maintainerRe: [PATCH v4 02/14] perf trace beauty: Make beauty generated C code standalone .o files
Hi Ian,
Quoted 44 lines · Ian Rogers
Previously,
builtin-trace.cdirectly included 15 embedded C files (e.g.trace/beauty/mmap.candfsconfig_arrays.c), which in turn depend on dozens of generated beauty script arrays. To satisfy these embedded inclusions, the global Makefile.perf would define all the generator variables/rules and include them in the prepare umbrella target, choking parallel build startup.Furthermore,
tools/perf/util/syscalltbl.cincluded its own generated mapper, andutil/env.cconditionally includedarch_errno_names.cinline, splitting consumers across directories and preventing clean Make encapsulation.Refactor the framework to achieve better encapsulation:
1. Move util/syscalltbl.[ch] into trace/beauty/ to co-locate with all generated code consumers.
2. Createfsconfig.cand flatten embedded beauty .c files to compile as independent standalone objects via trace/beauty/Build, exporting their formatting functions viabeauty.handenv.h. Switch arch_errno_names.o and syscalltbl.o assignments directly to perf-util-y and add an unconditional top-level recursive kbuild hook (perf-util-y += trace/beauty/) to compile them into libperf-util.a, resolving remote linkage forutil/env.c,util/bpf-trace-summary.c, and standalone python extensions.
3. Bridge private opaque references (struct trace) securely via accessorstrace__show_zeros()andtrace__host(), avoiding header entanglements.
4. Consolidate all generator variables, script paths, and array generation rules entirely out of Makefile.perf and place them directly inside the exact local Build files where their output objects are compiled (trace/beauty/Build and trace/beauty/tracepoints/Build), binding prerequisites locally. Use directly inside
generator recipes to guarantee dynamic directory creation before script redirection, and append across all rules to print
clean, standardized GEN ...file.coutput during compilation.
5. Clean up clean target to recursively remove the generated directory instead of relying on dozens of individual variables.This unchokes the "prepare" target parallel barrier, allows make to evaluate generation scripts purely locally where consumed, and flattens the tracepoint formatting architecture.
Testing a parallel build (make -j28 all from scratch) shows improvements:
Before:real 0m28.689s user 2m38.490s sys 0m30.148s
Quoted 8 lines
After:
real 0m27.642s user 2m32.356s sys 0m26.683sSo reclaiming ~9.6 seconds of raw CPU time and over 1 full second off overall real-world build latency, by overlapping sub-make startup and avoiding top-level double-parsing overhead.
In addition, it'd be nice to prepend the inclusion path for trace/beauty directory for the generated codes. But this would be a separate work.
Thanks,
Namhyung
Quoted 1,368 lines
Tested-by James Clark <[email protected]>Assisted-by Gemini:gemini-3.1-pro-previewSigned-off-by Ian Rogers <[email protected]>--- tools/perf/Build | 2 + tools/perf/Makefile.perf | 282 +----------------- tools/perf/builtin-trace.c | 32 +- tools/perf/trace/beauty/Build | 276 +++++++++++++++++ tools/perf/trace/beauty/arch_errno_names.c | 2 + tools/perf/trace/beauty/arch_errno_names.sh | 2 +- tools/perf/trace/beauty/beauty.h | 60 ++++ tools/perf/trace/beauty/eventfd.c | 6 +- tools/perf/trace/beauty/fsconfig.c | 5 + tools/perf/trace/beauty/futex_op.c | 5 +- tools/perf/trace/beauty/futex_val3.c | 5 +- tools/perf/trace/beauty/mmap.c | 24 +- tools/perf/trace/beauty/mode_t.c | 6 +- tools/perf/trace/beauty/msg_flags.c | 8 +- tools/perf/trace/beauty/open_flags.c | 2 + tools/perf/trace/beauty/perf_event_open.c | 21 +- tools/perf/trace/beauty/pid.c | 5 +- tools/perf/trace/beauty/sched_policy.c | 8 +- tools/perf/trace/beauty/seccomp.c | 12 +- tools/perf/trace/beauty/signum.c | 6 +- tools/perf/trace/beauty/socket_type.c | 6 +- .../perf/{util => trace/beauty}/syscalltbl.c | 0 .../perf/{util => trace/beauty}/syscalltbl.h | 0 tools/perf/trace/beauty/tracepoints/Build | 21 ++ tools/perf/trace/beauty/waitid_options.c | 8 +- tools/perf/util/Build | 4 +- tools/perf/util/bpf-trace-summary.c | 2 +- tools/perf/util/env.c | 4 - tools/perf/util/env.h | 1 + 29 files changed, 447 insertions(+), 368 deletions(-) create mode 100644 tools/perf/trace/beauty/fsconfig.c rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%) rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)tools/perf/Build +2
-0@@ -34,6 +34,8 @@ ifeq ($(CONFIG_LIBTRACEEVENT),y) perf-$(CONFIG_TRACE) += trace/beauty/ endif +perf-util-y += trace/beauty/ + perf-$(CONFIG_LIBELF) += builtin-probe.o perf-bench-y += bench/tools/perf/Makefile.perf +2
-280@@ -509,225 +509,6 @@ arm64-sysreg-defs-clean: $(Q)$(MAKE) -C $(arm64_gen_sysreg_dir) O=$(arm64_gen_sysreg_outdir) \ prefix= subdir= clean > /dev/null -beauty_linux_dir := $(srctree)/tools/perf/trace/beauty/include/linux/ -beauty_uapi_linux_dir := $(srctree)/tools/perf/trace/beauty/include/uapi/linux/ -beauty_uapi_sound_dir := $(srctree)/tools/perf/trace/beauty/include/uapi/sound/ -beauty_arch_asm_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/asm/ -beauty_x86_arch_asm_uapi_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/uapi/asm/ - -linux_uapi_dir := $(srctree)/tools/include/uapi/linux -asm_generic_uapi_dir := $(srctree)/tools/include/uapi/asm-generic -arch_asm_uapi_dir := $(srctree)/tools/arch/$(SRCARCH)/include/uapi/asm/ -x86_arch_asm_dir := $(srctree)/tools/arch/x86/include/asm/ - -beauty_outdir := $(OUTPUT)trace/beauty/generated -beauty_ioctl_outdir := $(beauty_outdir)/ioctl - -# Create output directory if not already present -$(shell [ -d '$(beauty_ioctl_outdir)' ] || mkdir -p '$(beauty_ioctl_outdir)') - -syscall_array := $(beauty_outdir)/syscalltbl.c -syscall_tbl := $(srctree)/tools/perf/trace/beauty/syscalltbl.sh -syscall_tbl_data := $(srctree)/tools/scripts/syscall.tbl \ - $(wildcard $(srctree)/tools/perf/arch/*/entry/syscalls/syscall*.tbl) - -$(syscall_array): $(syscall_tbl) $(syscall_tbl_data) - $(Q)$(SHELL) '$(syscall_tbl)' $(srctree)/tools $@ - -fs_at_flags_array := $(beauty_outdir)/fs_at_flags_array.c -fs_at_flags_tbl := $(srctree)/tools/perf/trace/beauty/fs_at_flags.sh - -$(fs_at_flags_array): $(beauty_uapi_linux_dir)/fcntl.h $(fs_at_flags_tbl) - $(Q)$(SHELL) '$(fs_at_flags_tbl)' $(beauty_uapi_linux_dir) > $@ - -clone_flags_array := $(beauty_outdir)/clone_flags_array.c -clone_flags_tbl := $(srctree)/tools/perf/trace/beauty/clone.sh - -$(clone_flags_array): $(beauty_uapi_linux_dir)/sched.h $(clone_flags_tbl) - $(Q)$(SHELL) '$(clone_flags_tbl)' $(beauty_uapi_linux_dir) > $@ - -drm_ioctl_array := $(beauty_ioctl_outdir)/drm_ioctl_array.c -drm_hdr_dir := $(srctree)/tools/perf/trace/beauty/include/uapi/drm -drm_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/drm_ioctl.sh - -$(drm_ioctl_array): $(drm_hdr_dir)/drm.h $(drm_hdr_dir)/i915_drm.h $(drm_ioctl_tbl) - $(Q)$(SHELL) '$(drm_ioctl_tbl)' $(drm_hdr_dir) > $@ - -fadvise_advice_array := $(beauty_outdir)/fadvise_advice_array.c -fadvise_advice_tbl := $(srctree)/tools/perf/trace/beauty/fadvise.sh - -$(fadvise_advice_array): $(beauty_uapi_linux_dir)/fadvise.h $(fadvise_advice_tbl) - $(Q)$(SHELL) '$(fadvise_advice_tbl)' $(beauty_uapi_linux_dir) > $@ - -fsmount_arrays := $(beauty_outdir)/fsmount_arrays.c -fsmount_tbls := $(srctree)/tools/perf/trace/beauty/fsmount.sh - -$(fsmount_arrays): $(beauty_uapi_linux_dir)/mount.h $(fsmount_tbls) - $(Q)$(SHELL) '$(fsmount_tbls)' $(beauty_uapi_linux_dir) > $@ - -fspick_arrays := $(beauty_outdir)/fspick_arrays.c -fspick_tbls := $(srctree)/tools/perf/trace/beauty/fspick.sh - -$(fspick_arrays): $(beauty_uapi_linux_dir)/mount.h $(fspick_tbls) - $(Q)$(SHELL) '$(fspick_tbls)' $(beauty_uapi_linux_dir) > $@ - -fsconfig_arrays := $(beauty_outdir)/fsconfig_arrays.c -fsconfig_tbls := $(srctree)/tools/perf/trace/beauty/fsconfig.sh - -$(fsconfig_arrays): $(beauty_uapi_linux_dir)/mount.h $(fsconfig_tbls) - $(Q)$(SHELL) '$(fsconfig_tbls)' $(beauty_uapi_linux_dir) > $@ - -pkey_alloc_access_rights_array := $(beauty_outdir)/pkey_alloc_access_rights_array.c -asm_generic_hdr_dir := $(srctree)/tools/include/uapi/asm-generic/ -pkey_alloc_access_rights_tbl := $(srctree)/tools/perf/trace/beauty/pkey_alloc_access_rights.sh - -$(pkey_alloc_access_rights_array): $(asm_generic_hdr_dir)/mman-common.h $(pkey_alloc_access_rights_tbl) - $(Q)$(SHELL) '$(pkey_alloc_access_rights_tbl)' $(asm_generic_hdr_dir) > $@ - -sndrv_ctl_ioctl_array := $(beauty_ioctl_outdir)/sndrv_ctl_ioctl_array.c -sndrv_ctl_hdr_dir := $(srctree)/tools/include/uapi/sound -sndrv_ctl_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/sndrv_ctl_ioctl.sh - -$(sndrv_ctl_ioctl_array): $(beauty_uapi_sound_dir)/asound.h $(sndrv_ctl_ioctl_tbl) - $(Q)$(SHELL) '$(sndrv_ctl_ioctl_tbl)' $(beauty_uapi_sound_dir) > $@ - -sndrv_pcm_ioctl_array := $(beauty_ioctl_outdir)/sndrv_pcm_ioctl_array.c -sndrv_pcm_hdr_dir := $(srctree)/tools/include/uapi/sound -sndrv_pcm_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/sndrv_pcm_ioctl.sh - -$(sndrv_pcm_ioctl_array): $(beauty_uapi_sound_dir)/asound.h $(sndrv_pcm_ioctl_tbl) - $(Q)$(SHELL) '$(sndrv_pcm_ioctl_tbl)' $(beauty_uapi_sound_dir) > $@ - -kcmp_type_array := $(beauty_outdir)/kcmp_type_array.c -kcmp_hdr_dir := $(srctree)/tools/include/uapi/linux/ -kcmp_type_tbl := $(srctree)/tools/perf/trace/beauty/kcmp_type.sh - -$(kcmp_type_array): $(kcmp_hdr_dir)/kcmp.h $(kcmp_type_tbl) - $(Q)$(SHELL) '$(kcmp_type_tbl)' $(kcmp_hdr_dir) > $@ - -kvm_ioctl_array := $(beauty_ioctl_outdir)/kvm_ioctl_array.c -kvm_hdr_dir := $(srctree)/tools/include/uapi/linux -kvm_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/kvm_ioctl.sh - -$(kvm_ioctl_array): $(kvm_hdr_dir)/kvm.h $(kvm_ioctl_tbl) - $(Q)$(SHELL) '$(kvm_ioctl_tbl)' $(kvm_hdr_dir) > $@ - -socket_arrays := $(beauty_outdir)/socket.c -socket_tbl := $(srctree)/tools/perf/trace/beauty/socket.sh - -$(socket_arrays): $(linux_uapi_dir)/in.h $(beauty_linux_dir)/socket.h $(socket_tbl) - $(Q)$(SHELL) '$(socket_tbl)' $(linux_uapi_dir) $(beauty_linux_dir) > $@ - -sockaddr_arrays := $(beauty_outdir)/sockaddr.c -sockaddr_tbl := $(srctree)/tools/perf/trace/beauty/sockaddr.sh - -$(sockaddr_arrays): $(beauty_linux_dir)/socket.h $(sockaddr_tbl) - $(Q)$(SHELL) '$(sockaddr_tbl)' $(beauty_linux_dir) > $@ - -vhost_virtio_ioctl_array := $(beauty_ioctl_outdir)/vhost_virtio_ioctl_array.c -vhost_virtio_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/vhost_virtio_ioctl.sh - -$(vhost_virtio_ioctl_array): $(beauty_uapi_linux_dir)/vhost.h $(vhost_virtio_ioctl_tbl) - $(Q)$(SHELL) '$(vhost_virtio_ioctl_tbl)' $(beauty_uapi_linux_dir) > $@ - -perf_ioctl_array := $(beauty_ioctl_outdir)/perf_ioctl_array.c -perf_hdr_dir := $(srctree)/tools/include/uapi/linux -perf_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/perf_ioctl.sh - -$(perf_ioctl_array): $(perf_hdr_dir)/perf_event.h $(perf_ioctl_tbl) - $(Q)$(SHELL) '$(perf_ioctl_tbl)' $(perf_hdr_dir) > $@ - -madvise_behavior_array := $(beauty_outdir)/madvise_behavior_array.c -madvise_hdr_dir := $(srctree)/tools/include/uapi/asm-generic/ -madvise_behavior_tbl := $(srctree)/tools/perf/trace/beauty/madvise_behavior.sh - -$(madvise_behavior_array): $(madvise_hdr_dir)/mman-common.h $(madvise_behavior_tbl) - $(Q)$(SHELL) '$(madvise_behavior_tbl)' $(madvise_hdr_dir) > $@ - -mmap_flags_array := $(beauty_outdir)/mmap_flags_array.c -mmap_flags_tbl := $(srctree)/tools/perf/trace/beauty/mmap_flags.sh - -$(mmap_flags_array): $(linux_uapi_dir)/mman.h $(asm_generic_uapi_dir)/mman.h $(asm_generic_uapi_dir)/mman-common.h $(mmap_flags_tbl) - $(Q)$(SHELL) '$(mmap_flags_tbl)' $(linux_uapi_dir) $(asm_generic_uapi_dir) $(arch_asm_uapi_dir) > $@ - -mremap_flags_array := $(beauty_outdir)/mremap_flags_array.c -mremap_flags_tbl := $(srctree)/tools/perf/trace/beauty/mremap_flags.sh - -$(mremap_flags_array): $(linux_uapi_dir)/mman.h $(mremap_flags_tbl) - $(Q)$(SHELL) '$(mremap_flags_tbl)' $(linux_uapi_dir) > $@ - -mount_flags_array := $(beauty_outdir)/mount_flags_array.c -mount_flags_tbl := $(srctree)/tools/perf/trace/beauty/mount_flags.sh - -$(mount_flags_array): $(beauty_uapi_linux_dir)/mount.h $(mount_flags_tbl) - $(Q)$(SHELL) '$(mount_flags_tbl)' $(beauty_uapi_linux_dir) > $@ - -move_mount_flags_array := $(beauty_outdir)/move_mount_flags_array.c -move_mount_flags_tbl := $(srctree)/tools/perf/trace/beauty/move_mount_flags.sh - -$(move_mount_flags_array): $(beauty_uapi_linux_dir)/mount.h $(move_mount_flags_tbl) - $(Q)$(SHELL) '$(move_mount_flags_tbl)' $(beauty_uapi_linux_dir) > $@ - -mmap_prot_array := $(beauty_outdir)/mmap_prot_array.c -mmap_prot_tbl := $(srctree)/tools/perf/trace/beauty/mmap_prot.sh - -$(mmap_prot_array): $(asm_generic_uapi_dir)/mman.h $(asm_generic_uapi_dir)/mman-common.h $(mmap_prot_tbl) - $(Q)$(SHELL) '$(mmap_prot_tbl)' $(asm_generic_uapi_dir) $(arch_asm_uapi_dir) > $@ - -prctl_option_array := $(beauty_outdir)/prctl_option_array.c -prctl_option_tbl := $(srctree)/tools/perf/trace/beauty/prctl_option.sh - -$(prctl_option_array): $(beauty_uapi_linux_dir)/prctl.h $(prctl_option_tbl) - $(Q)$(SHELL) '$(prctl_option_tbl)' $(beauty_uapi_linux_dir) > $@ - -usbdevfs_ioctl_array := $(beauty_ioctl_outdir)/usbdevfs_ioctl_array.c -usbdevfs_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/usbdevfs_ioctl.sh - -$(usbdevfs_ioctl_array): $(beauty_uapi_linux_dir)/usbdevice_fs.h $(usbdevfs_ioctl_tbl) - $(Q)$(SHELL) '$(usbdevfs_ioctl_tbl)' $(beauty_uapi_linux_dir) > $@ - -x86_arch_prctl_code_array := $(beauty_outdir)/x86_arch_prctl_code_array.c -x86_arch_prctl_code_tbl := $(srctree)/tools/perf/trace/beauty/x86_arch_prctl.sh - -$(x86_arch_prctl_code_array): $(beauty_x86_arch_asm_uapi_dir)/prctl.h $(x86_arch_prctl_code_tbl) - $(Q)$(SHELL) '$(x86_arch_prctl_code_tbl)' $(beauty_x86_arch_asm_uapi_dir) > $@ - -x86_arch_irq_vectors_array := $(beauty_outdir)/x86_arch_irq_vectors_array.c -x86_arch_irq_vectors_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_irq_vectors.sh - -$(x86_arch_irq_vectors_array): $(beauty_arch_asm_dir)/irq_vectors.h $(x86_arch_irq_vectors_tbl) - $(Q)$(SHELL) '$(x86_arch_irq_vectors_tbl)' $(beauty_arch_asm_dir) > $@ - -x86_arch_MSRs_array := $(beauty_outdir)/x86_arch_MSRs_array.c -x86_arch_MSRs_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_msr.sh - -$(x86_arch_MSRs_array): $(x86_arch_asm_dir)/msr-index.h $(x86_arch_MSRs_tbl) - $(Q)$(SHELL) '$(x86_arch_MSRs_tbl)' $(x86_arch_asm_dir) > $@ - -rename_flags_array := $(beauty_outdir)/rename_flags_array.c -rename_flags_tbl := $(srctree)/tools/perf/trace/beauty/rename_flags.sh - -$(rename_flags_array): $(beauty_uapi_linux_dir)/fs.h $(rename_flags_tbl) - $(Q)$(SHELL) '$(rename_flags_tbl)' $(beauty_uapi_linux_dir) > $@ - -arch_errno_name_array := $(beauty_outdir)/arch_errno_name_array.c -arch_errno_hdr_dir := $(srctree)/tools -arch_errno_tbl := $(srctree)/tools/perf/trace/beauty/arch_errno_names.sh - -$(arch_errno_name_array): $(arch_errno_tbl) - $(Q)$(SHELL) '$(arch_errno_tbl)' '$(patsubst -%,,$(CC))' $(arch_errno_hdr_dir) > $@ - -statx_mask_array := $(beauty_outdir)/statx_mask_array.c -statx_mask_tbl := $(srctree)/tools/perf/trace/beauty/statx_mask.sh - -$(statx_mask_array): $(beauty_uapi_linux_dir)/stat.h $(statx_mask_tbl) - $(Q)$(SHELL) '$(statx_mask_tbl)' $(beauty_uapi_linux_dir) > $@ - -sync_file_range_arrays := $(beauty_outdir)/sync_file_range_arrays.c -sync_file_range_tbls := $(srctree)/tools/perf/trace/beauty/sync_file_range.sh - -$(sync_file_range_arrays): $(beauty_uapi_linux_dir)/fs.h $(sync_file_range_tbls) - $(Q)$(SHELL) '$(sync_file_range_tbls)' $(beauty_uapi_linux_dir) > $@ TESTS_CORESIGHT_DIR := $(srctree)/tools/perf/tests/shell/coresight@@ -848,38 +629,6 @@ build-dir = $(or $(__build-dir),.) prepare: $(OUTPUT)PERF-VERSION-FILE archheaders \ arm64-sysreg-defs \ - $(syscall_array) \ - $(fs_at_flags_array) \ - $(clone_flags_array) \ - $(drm_ioctl_array) \ - $(fadvise_advice_array) \ - $(fsconfig_arrays) \ - $(fsmount_arrays) \ - $(fspick_arrays) \ - $(pkey_alloc_access_rights_array) \ - $(sndrv_pcm_ioctl_array) \ - $(sndrv_ctl_ioctl_array) \ - $(kcmp_type_array) \ - $(kvm_ioctl_array) \ - $(socket_arrays) \ - $(sockaddr_arrays) \ - $(vhost_virtio_ioctl_array) \ - $(madvise_behavior_array) \ - $(mmap_flags_array) \ - $(mmap_prot_array) \ - $(mremap_flags_array) \ - $(mount_flags_array) \ - $(move_mount_flags_array) \ - $(perf_ioctl_array) \ - $(prctl_option_array) \ - $(usbdevfs_ioctl_array) \ - $(x86_arch_irq_vectors_array) \ - $(x86_arch_MSRs_array) \ - $(x86_arch_prctl_code_array) \ - $(rename_flags_array) \ - $(arch_errno_name_array) \ - $(statx_mask_array) \ - $(sync_file_range_arrays) \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \@@ -1299,35 +1048,8 @@ clean:: $(LIBAPI)-clean $(LIBBPF)-clean $(LIBSUBCMD)-clean $(LIBSYMBOL)-clean $( TAGS tags cscope* $(OUTPUT)PERF-VERSION-FILE \ $(OUTPUT)FEATURE-DUMP $(OUTPUT)util/*-bison* $(OUTPUT)util/*-flex* \ $(OUTPUT)util/intel-pt-decoder/inat-tables.c \ - $(OUTPUT)tests/llvm-src-{base,kbuild,prologue,relocation}.c \ - $(OUTPUT)$(fadvise_advice_array) \ - $(OUTPUT)$(fsconfig_arrays) \ - $(OUTPUT)$(fsmount_arrays) \ - $(OUTPUT)$(fspick_arrays) \ - $(OUTPUT)$(madvise_behavior_array) \ - $(OUTPUT)$(mmap_flags_array) \ - $(OUTPUT)$(mmap_prot_array) \ - $(OUTPUT)$(mremap_flags_array) \ - $(OUTPUT)$(mount_flags_array) \ - $(OUTPUT)$(move_mount_flags_array) \ - $(OUTPUT)$(drm_ioctl_array) \ - $(OUTPUT)$(pkey_alloc_access_rights_array) \ - $(OUTPUT)$(sndrv_ctl_ioctl_array) \ - $(OUTPUT)$(sndrv_pcm_ioctl_array) \ - $(OUTPUT)$(kvm_ioctl_array) \ - $(OUTPUT)$(kcmp_type_array) \ - $(OUTPUT)$(socket_arrays) \ - $(OUTPUT)$(sockaddr_arrays) \ - $(OUTPUT)$(vhost_virtio_ioctl_array) \ - $(OUTPUT)$(perf_ioctl_array) \ - $(OUTPUT)$(prctl_option_array) \ - $(OUTPUT)$(usbdevfs_ioctl_array) \ - $(OUTPUT)$(x86_arch_irq_vectors_array) \ - $(OUTPUT)$(x86_arch_MSRs_array) \ - $(OUTPUT)$(x86_arch_prctl_code_array) \ - $(OUTPUT)$(rename_flags_array) \ - $(OUTPUT)$(arch_errno_name_array) \ - $(OUTPUT)$(sync_file_range_arrays) + $(OUTPUT)tests/llvm-src-{base,kbuild,prologue,relocation}.c + $(Q)$(RM) -r $(OUTPUT)trace/beauty/generated $(call QUIET_CLEAN, Documentation) \ $(MAKE) -C $(DOC_DIR) O=$(OUTPUT) clean >/dev/nulltools/perf/builtin-trace.c +12
-20@@ -60,12 +60,13 @@#include "callchain.h" #include "print_binary.h" #include "string2.h" -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "../perf.h" #include "trace_augment.h" #include "dwarf-regs.h" #include <errno.h> +#include <sys/stat.h> #include <inttypes.h> #include <poll.h> #include <signal.h> @@ -234,6 +235,16 @@ struct trace {const char *uid_str; }; +bool trace__show_zeros (const struct trace *trace) +{ +return trace->show_zeros; +} + +struct machine *trace__host (const struct trace *trace) +{ +return trace->host; +} +static void trace__load_vmlinux_btf (struct trace *trace__maybe_unused ) {#ifdef HAVE_LIBBPF_SUPPORT @@ -776,10 +787,6 @@ static const char *fsmount_flags[] = { }; static DEFINE_STRARRAY(fsmount_flags, "FSMOUNT_"); -#include "trace/beauty/generated/fsconfig_arrays.c" - -static DEFINE_STRARRAY(fsconfig_cmds, "FSCONFIG_"); -static const char *epoll_ctl_ops[] = {"ADD" ,"DEL" ,"MOD" , }; static DEFINE_STRARRAY_OFFSET(epoll_ctl_ops, "EPOLL_CTL_", 1);@@ -1128,21 +1135,6 @@ static bool syscall_arg__strtoul_btf_type(char *bf __maybe_unused, size_t size _ .parm = &strarray__##array, \ .show_zero = true, } -#include "trace/beauty/eventfd.c" -#include "trace/beauty/futex_op.c" -#include "trace/beauty/futex_val3.c" -#include "trace/beauty/mmap.c" -#include "trace/beauty/mode_t.c" -#include "trace/beauty/msg_flags.c" -#include "trace/beauty/open_flags.c" -#include "trace/beauty/perf_event_open.c" -#include "trace/beauty/pid.c" -#include "trace/beauty/sched_policy.c" -#include "trace/beauty/seccomp.c" -#include "trace/beauty/signum.c" -#include "trace/beauty/socket_type.c" -#include "trace/beauty/waitid_options.c" -static const struct syscall_fmt syscall_fmts[] = { { .name ="access" , .arg = { [1 ] = { .scnprintf =SCA_ACCMODE ,/* mode */ }, }, },tools/perf/trace/beauty/Build +276
-0@@ -19,6 +19,23 @@ perf-y += socket.o perf-y += statx.o perf-y += sync_file_range.o perf-y += timespec.o +perf-util-y += syscalltbl.o +perf-y += fsconfig.o +perf-y += eventfd.o +perf-y += futex_op.o +perf-y += futex_val3.o +perf-y += mmap.o +perf-y += mode_t.o +perf-y += msg_flags.o +perf-y += open_flags.o +perf-y += perf_event_open.o +perf-y += pid.o +perf-y += sched_policy.o +perf-y += seccomp.o +perf-y += signum.o +perf-y += socket_type.o +perf-y += waitid_options.o +perf-util-y += arch_errno_names.o perf-y += tracepoints/ ifdef SHELLCHECK@@ -34,3 +51,262 @@ $(OUTPUT)%.shellcheck_log: % $(Q)$(call echo-cmd,test)$(SHELLCHECK) "$<" > $@ || (cat $@ && rm $@ && false) perf-y += $(SHELL_TEST_LOGS) + +beauty_linux_dir := $(srctree)/tools/perf/trace/beauty/include/linux/ +beauty_uapi_linux_dir := $(srctree)/tools/perf/trace/beauty/include/uapi/linux/ +beauty_uapi_sound_dir := $(srctree)/tools/perf/trace/beauty/include/uapi/sound/ +beauty_arch_asm_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/asm/ +beauty_x86_arch_asm_uapi_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/uapi/asm/ + +linux_uapi_dir := $(srctree)/tools/include/uapi/linux +asm_generic_uapi_dir := $(srctree)/tools/include/uapi/asm-generic +arch_asm_uapi_dir := $(srctree)/tools/arch/$(SRCARCH)/include/uapi/asm/ +x86_arch_asm_dir := $(srctree)/tools/arch/x86/include/asm/ + +beauty_outdir := $(OUTPUT)trace/beauty/generated +beauty_ioctl_outdir := $(beauty_outdir)/ioctl + +syscall_array := $(beauty_outdir)/syscalltbl.c +syscall_tbl := $(srctree)/tools/perf/trace/beauty/syscalltbl.sh +syscall_tbl_data := $(srctree)/tools/scripts/syscall.tbl \ + $(wildcard $(srctree)/tools/perf/arch/*/entry/syscalls/syscall*.tbl) + +$(syscall_array): $(syscall_tbl) $(syscall_tbl_data) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(syscall_tbl)' $(srctree)/tools $@ + +fs_at_flags_array := $(beauty_outdir)/fs_at_flags_array.c +fs_at_flags_tbl := $(srctree)/tools/perf/trace/beauty/fs_at_flags.sh + +$(fs_at_flags_array): $(beauty_uapi_linux_dir)/fcntl.h $(fs_at_flags_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(fs_at_flags_tbl)' $(beauty_uapi_linux_dir) > $@ + +clone_flags_array := $(beauty_outdir)/clone_flags_array.c +clone_flags_tbl := $(srctree)/tools/perf/trace/beauty/clone.sh + +$(clone_flags_array): $(beauty_uapi_linux_dir)/sched.h $(clone_flags_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(clone_flags_tbl)' $(beauty_uapi_linux_dir) > $@ + +drm_ioctl_array := $(beauty_ioctl_outdir)/drm_ioctl_array.c +drm_hdr_dir := $(srctree)/tools/perf/trace/beauty/include/uapi/drm +drm_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/drm_ioctl.sh + +$(drm_ioctl_array): $(drm_hdr_dir)/drm.h $(drm_hdr_dir)/i915_drm.h $(drm_ioctl_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(drm_ioctl_tbl)' $(drm_hdr_dir) > $@ + +fadvise_advice_array := $(beauty_outdir)/fadvise_advice_array.c +fadvise_advice_tbl := $(srctree)/tools/perf/trace/beauty/fadvise.sh + +$(fadvise_advice_array): $(beauty_uapi_linux_dir)/fadvise.h $(fadvise_advice_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(fadvise_advice_tbl)' $(beauty_uapi_linux_dir) > $@ + +fsmount_arrays := $(beauty_outdir)/fsmount_arrays.c +fsmount_tbls := $(srctree)/tools/perf/trace/beauty/fsmount.sh + +$(fsmount_arrays): $(beauty_uapi_linux_dir)/mount.h $(fsmount_tbls) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(fsmount_tbls)' $(beauty_uapi_linux_dir) > $@ + +fspick_arrays := $(beauty_outdir)/fspick_arrays.c +fspick_tbls := $(srctree)/tools/perf/trace/beauty/fspick.sh + +$(fspick_arrays): $(beauty_uapi_linux_dir)/mount.h $(fspick_tbls) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(fspick_tbls)' $(beauty_uapi_linux_dir) > $@ + +fsconfig_arrays := $(beauty_outdir)/fsconfig_arrays.c +fsconfig_tbls := $(srctree)/tools/perf/trace/beauty/fsconfig.sh + +$(fsconfig_arrays): $(beauty_uapi_linux_dir)/mount.h $(fsconfig_tbls) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(fsconfig_tbls)' $(beauty_uapi_linux_dir) > $@ + +pkey_alloc_access_rights_array := $(beauty_outdir)/pkey_alloc_access_rights_array.c +asm_generic_hdr_dir := $(srctree)/tools/include/uapi/asm-generic/ +pkey_alloc_access_rights_tbl := $(srctree)/tools/perf/trace/beauty/pkey_alloc_access_rights.sh + +$(pkey_alloc_access_rights_array): $(asm_generic_hdr_dir)/mman-common.h $(pkey_alloc_access_rights_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(pkey_alloc_access_rights_tbl)' $(asm_generic_hdr_dir) > $@ + +sndrv_ctl_ioctl_array := $(beauty_ioctl_outdir)/sndrv_ctl_ioctl_array.c +sndrv_ctl_hdr_dir := $(srctree)/tools/include/uapi/sound +sndrv_ctl_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/sndrv_ctl_ioctl.sh + +$(sndrv_ctl_ioctl_array): $(beauty_uapi_sound_dir)/asound.h $(sndrv_ctl_ioctl_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(sndrv_ctl_ioctl_tbl)' $(beauty_uapi_sound_dir) > $@ + +sndrv_pcm_ioctl_array := $(beauty_ioctl_outdir)/sndrv_pcm_ioctl_array.c +sndrv_pcm_hdr_dir := $(srctree)/tools/include/uapi/sound +sndrv_pcm_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/sndrv_pcm_ioctl.sh + +$(sndrv_pcm_ioctl_array): $(beauty_uapi_sound_dir)/asound.h $(sndrv_pcm_ioctl_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(sndrv_pcm_ioctl_tbl)' $(beauty_uapi_sound_dir) > $@ + +kcmp_type_array := $(beauty_outdir)/kcmp_type_array.c +kcmp_hdr_dir := $(srctree)/tools/include/uapi/linux/ +kcmp_type_tbl := $(srctree)/tools/perf/trace/beauty/kcmp_type.sh + +$(kcmp_type_array): $(kcmp_hdr_dir)/kcmp.h $(kcmp_type_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(kcmp_type_tbl)' $(kcmp_hdr_dir) > $@ + +kvm_ioctl_array := $(beauty_ioctl_outdir)/kvm_ioctl_array.c +kvm_hdr_dir := $(srctree)/tools/include/uapi/linux +kvm_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/kvm_ioctl.sh + +$(kvm_ioctl_array): $(kvm_hdr_dir)/kvm.h $(kvm_ioctl_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(kvm_ioctl_tbl)' $(kvm_hdr_dir) > $@ + +socket_arrays := $(beauty_outdir)/socket.c +socket_tbl := $(srctree)/tools/perf/trace/beauty/socket.sh + +$(socket_arrays): $(linux_uapi_dir)/in.h $(beauty_linux_dir)/socket.h $(socket_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(socket_tbl)' $(linux_uapi_dir) $(beauty_linux_dir) > $@ + +sockaddr_arrays := $(beauty_outdir)/sockaddr.c +sockaddr_tbl := $(srctree)/tools/perf/trace/beauty/sockaddr.sh + +$(sockaddr_arrays): $(beauty_linux_dir)/socket.h $(sockaddr_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(sockaddr_tbl)' $(beauty_linux_dir) > $@ + +vhost_virtio_ioctl_array := $(beauty_ioctl_outdir)/vhost_virtio_ioctl_array.c +vhost_virtio_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/vhost_virtio_ioctl.sh + +$(vhost_virtio_ioctl_array): $(beauty_uapi_linux_dir)/vhost.h $(vhost_virtio_ioctl_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(vhost_virtio_ioctl_tbl)' $(beauty_uapi_linux_dir) > $@ + +perf_ioctl_array := $(beauty_ioctl_outdir)/perf_ioctl_array.c +perf_hdr_dir := $(srctree)/tools/include/uapi/linux +perf_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/perf_ioctl.sh + +$(perf_ioctl_array): $(perf_hdr_dir)/perf_event.h $(perf_ioctl_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(perf_ioctl_tbl)' $(perf_hdr_dir) > $@ + +madvise_behavior_array := $(beauty_outdir)/madvise_behavior_array.c +madvise_hdr_dir := $(srctree)/tools/include/uapi/asm-generic/ +madvise_behavior_tbl := $(srctree)/tools/perf/trace/beauty/madvise_behavior.sh + +$(madvise_behavior_array): $(madvise_hdr_dir)/mman-common.h $(madvise_behavior_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(madvise_behavior_tbl)' $(madvise_hdr_dir) > $@ + +mmap_flags_array := $(beauty_outdir)/mmap_flags_array.c +mmap_flags_tbl := $(srctree)/tools/perf/trace/beauty/mmap_flags.sh + +$(mmap_flags_array): $(linux_uapi_dir)/mman.h $(asm_generic_uapi_dir)/mman.h $(asm_generic_uapi_dir)/mman-common.h $(mmap_flags_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(mmap_flags_tbl)' $(linux_uapi_dir) $(asm_generic_uapi_dir) $(arch_asm_uapi_dir) > $@ + +mremap_flags_array := $(beauty_outdir)/mremap_flags_array.c +mremap_flags_tbl := $(srctree)/tools/perf/trace/beauty/mremap_flags.sh + +$(mremap_flags_array): $(linux_uapi_dir)/mman.h $(mremap_flags_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(mremap_flags_tbl)' $(linux_uapi_dir) > $@ + +mount_flags_array := $(beauty_outdir)/mount_flags_array.c +mount_flags_tbl := $(srctree)/tools/perf/trace/beauty/mount_flags.sh + +$(mount_flags_array): $(beauty_uapi_linux_dir)/mount.h $(mount_flags_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(mount_flags_tbl)' $(beauty_uapi_linux_dir) > $@ + +move_mount_flags_array := $(beauty_outdir)/move_mount_flags_array.c +move_mount_flags_tbl := $(srctree)/tools/perf/trace/beauty/move_mount_flags.sh + +$(move_mount_flags_array): $(beauty_uapi_linux_dir)/mount.h $(move_mount_flags_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(move_mount_flags_tbl)' $(beauty_uapi_linux_dir) > $@ + +mmap_prot_array := $(beauty_outdir)/mmap_prot_array.c +mmap_prot_tbl := $(srctree)/tools/perf/trace/beauty/mmap_prot.sh + +$(mmap_prot_array): $(asm_generic_uapi_dir)/mman.h $(asm_generic_uapi_dir)/mman-common.h $(mmap_prot_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(mmap_prot_tbl)' $(asm_generic_uapi_dir) $(arch_asm_uapi_dir) > $@ + +prctl_option_array := $(beauty_outdir)/prctl_option_array.c +prctl_option_tbl := $(srctree)/tools/perf/trace/beauty/prctl_option.sh + +$(prctl_option_array): $(beauty_uapi_linux_dir)/prctl.h $(prctl_option_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(prctl_option_tbl)' $(beauty_uapi_linux_dir) > $@ + +usbdevfs_ioctl_array := $(beauty_ioctl_outdir)/usbdevfs_ioctl_array.c +usbdevfs_ioctl_tbl := $(srctree)/tools/perf/trace/beauty/usbdevfs_ioctl.sh + +$(usbdevfs_ioctl_array): $(beauty_uapi_linux_dir)/usbdevice_fs.h $(usbdevfs_ioctl_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(usbdevfs_ioctl_tbl)' $(beauty_uapi_linux_dir) > $@ + +x86_arch_prctl_code_array := $(beauty_outdir)/x86_arch_prctl_code_array.c +x86_arch_prctl_code_tbl := $(srctree)/tools/perf/trace/beauty/x86_arch_prctl.sh + +$(x86_arch_prctl_code_array): $(beauty_x86_arch_asm_uapi_dir)/prctl.h $(x86_arch_prctl_code_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_prctl_code_tbl)' $(beauty_x86_arch_asm_uapi_dir) > $@ + + +rename_flags_array := $(beauty_outdir)/rename_flags_array.c +rename_flags_tbl := $(srctree)/tools/perf/trace/beauty/rename_flags.sh + +$(rename_flags_array): $(beauty_uapi_linux_dir)/fs.h $(rename_flags_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(rename_flags_tbl)' $(beauty_uapi_linux_dir) > $@ + + +statx_mask_array := $(beauty_outdir)/statx_mask_array.c +statx_mask_tbl := $(srctree)/tools/perf/trace/beauty/statx_mask.sh + +$(statx_mask_array): $(beauty_uapi_linux_dir)/stat.h $(statx_mask_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(statx_mask_tbl)' $(beauty_uapi_linux_dir) > $@ + +sync_file_range_arrays := $(beauty_outdir)/sync_file_range_arrays.c +sync_file_range_tbls := $(srctree)/tools/perf/trace/beauty/sync_file_range.sh + +$(sync_file_range_arrays): $(beauty_uapi_linux_dir)/fs.h $(sync_file_range_tbls) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(sync_file_range_tbls)' $(beauty_uapi_linux_dir) > $@ + +$(OUTPUT)trace/beauty/syscalltbl.o: $(syscall_array) +$(OUTPUT)trace/beauty/fsconfig.o: $(fsconfig_arrays) +$(OUTPUT)trace/beauty/clone.o: $(clone_flags_array) +$(OUTPUT)trace/beauty/fs_at_flags.o: $(fs_at_flags_array) +$(OUTPUT)trace/beauty/fsmount.o: $(fsmount_arrays) +$(OUTPUT)trace/beauty/fspick.o: $(fspick_arrays) +$(OUTPUT)trace/beauty/ioctl.o: $(drm_ioctl_array) $(sndrv_pcm_ioctl_array) $(sndrv_ctl_ioctl_array) $(kvm_ioctl_array) $(vhost_virtio_ioctl_array) $(perf_ioctl_array) $(usbdevfs_ioctl_array) +$(OUTPUT)trace/beauty/kcmp.o: $(kcmp_type_array) +$(OUTPUT)trace/beauty/mmap.o: $(mmap_prot_array) $(mmap_flags_array) $(mremap_flags_array) $(madvise_behavior_array) +$(OUTPUT)trace/beauty/mount_flags.o: $(mount_flags_array) +$(OUTPUT)trace/beauty/move_mount.o: $(move_mount_flags_array) +$(OUTPUT)trace/beauty/pkey_alloc.o: $(pkey_alloc_access_rights_array) +$(OUTPUT)trace/beauty/prctl.o: $(prctl_option_array) +$(OUTPUT)trace/beauty/renameat.o: $(rename_flags_array) +$(OUTPUT)trace/beauty/sockaddr.o: $(sockaddr_arrays) +$(OUTPUT)trace/beauty/socket.o: $(socket_arrays) +$(OUTPUT)trace/beauty/statx.o: $(statx_mask_array) +$(OUTPUT)trace/beauty/sync_file_range.o: $(sync_file_range_arrays) +$(OUTPUT)trace/beauty/arch_prctl.o: $(x86_arch_prctl_code_array) + +arch_errno_name_array := $(beauty_outdir)/arch_errno_name_array.c +arch_errno_hdr_dir := $(srctree)/tools +arch_errno_tbl := $(srctree)/tools/perf/trace/beauty/arch_errno_names.sh + +$(arch_errno_name_array): $(arch_errno_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(arch_errno_tbl)' '$(patsubst -%,,$(CC))' $(arch_errno_hdr_dir) > $@ + +$(OUTPUT)trace/beauty/arch_errno_names.o: $(arch_errno_name_array)tools/perf/trace/beauty/arch_errno_names.c +2
-0@@ -1 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 +#include "util/env.h" #include "trace/beauty/generated/arch_errno_name_array.c" tools/perf/trace/beauty/arch_errno_names.sh +1
-1@@ -57,7 +57,7 @@ create_arch_errno_table_func() archlist="$1" default="$2" - printf 'static arch_syscalls__strerrno_t *\n' + printf 'arch_syscalls__strerrno_t *\n' printf 'arch_syscalls__strerrno_function(const char *arch)\n' printf '{\n' for arch in $archlist; dotools/perf/trace/beauty/beauty.h +60
-0@@ -35,6 +35,8 @@ bool strarray__strtoul(struct strarray *sa, char *bf, size_t size, u64 *ret);bool strarray__strtoul_flags (struct strarray *sa,char *bf,size_t size,u64 *ret);struct trace ; +bool trace__show_zeros (const struct trace *trace); +struct machine *trace__host (const struct trace *trace);struct thread ;struct file {@@ -265,4 +267,62 @@ size_t open__scnprintf_flags(unsigned long flags, char *bf, size_t size, bool shvoid syscall_arg__set_ret_scnprintf (struct syscall_arg *arg,size_t (*ret_scnprintf)(char *bf,size_t size,struct syscall_arg *arg)); +extern struct strarray strarray__fsconfig_cmds; + +size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_EFD_FLAGS syscall_arg__scnprintf_eventfd_flags + +size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_FUTEX_OP syscall_arg__scnprintf_futex_op + +size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_FUTEX_VAL3 syscall_arg__scnprintf_futex_val3 + +size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_MMAP_PROT syscall_arg__scnprintf_mmap_prot + +extern struct strarray strarray__mmap_flags; + +size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_MMAP_FLAGS syscall_arg__scnprintf_mmap_flags + +size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_MREMAP_FLAGS syscall_arg__scnprintf_mremap_flags + +size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_MADV_BHV syscall_arg__scnprintf_madvise_behavior + +size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_MODE_T syscall_arg__scnprintf_mode_t + +size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_MSG_FLAGS syscall_arg__scnprintf_msg_flags + +size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_PERF_FLAGS syscall_arg__scnprintf_perf_flags + +size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_PERF_ATTR syscall_arg__scnprintf_perf_event_attr +#define SCA_PERF_ATTR_FROM_USER (argname) \ + { .scnprintf = SCA_PERF_ATTR, \ + .from_user = true, } + +size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_SCHED_POLICY syscall_arg__scnprintf_sched_policy + +size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_SECCOMP_OP syscall_arg__scnprintf_seccomp_op + +size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_SECCOMP_FLAGS syscall_arg__scnprintf_seccomp_flags + +size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_SIGNUM syscall_arg__scnprintf_signum + +size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_SK_TYPE syscall_arg__scnprintf_socket_type + +size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size,struct syscall_arg *arg); +#define SCA_WAITID_OPTIONS syscall_arg__scnprintf_waitid_options +#endif /* _PERF_TRACE_BEAUTY_H */ tools/perf/trace/beauty/eventfd.c +3
-3@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#ifndef EFD_SEMAPHORE #define EFD_SEMAPHORE 1#endif @@ -11,7 +13,7 @@#define EFD_CLOEXEC 02000000#endif -static size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_eventfd_flags (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="EFD_" ;@@ -35,5 +37,3 @@ static size_t syscall_arg__scnprintf_eventfd_flags(char *bf, size_t size, structreturn printed; } - -#define SCA_EFD_FLAGS syscall_arg__scnprintf_eventfd_flagstools/perf/trace/beauty/fsconfig.c new file +5
-0@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "trace/beauty/generated/fsconfig_arrays.c" + +DEFINE_STRARRAY (fsconfig_cmds,"FSCONFIG_" );tools/perf/trace/beauty/futex_op.c +2
-3@@ -1,4 +1,5 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" #include <linux/futex.h> #ifndef FUTEX_WAIT_BITSET @@ -17,7 +18,7 @@#define FUTEX_CLOCK_REALTIME 256#endif -static size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_futex_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="FUTEX_" ;@@ -59,5 +60,3 @@ static size_t syscall_arg__scnprintf_futex_op(char *bf, size_t size, struct syscreturn printed; } - -#define SCA_FUTEX_OP syscall_arg__scnprintf_futex_optools/perf/trace/beauty/futex_val3.c +2
-3@@ -1,11 +1,12 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" #include <linux/futex.h> #ifndef FUTEX_BITSET_MATCH_ANY #define FUTEX_BITSET_MATCH_ANY 0xffffffff#endif -static size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_futex_val3 (char *bf,size_t size,struct syscall_arg *arg) {const char *prefix ="FUTEX_BITSET_" ;unsigned int bitset = arg->val;@@ -15,5 +16,3 @@ static size_t syscall_arg__scnprintf_futex_val3(char *bf, size_t size, struct syreturn scnprintf (bf, size,"%#xd" , bitset); } - -#define SCA_FUTEX_VAL3 syscall_arg__scnprintf_futex_val3tools/perf/trace/beauty/mmap.c +11
-13@@ -1,5 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 #include <linux/log2.h> +#include "trace/beauty/beauty.h" #include "trace/beauty/generated/mmap_prot_array.c" static DEFINE_STRARRAY(mmap_prot, "PROT_");@@ -8,8 +9,7 @@ static size_t mmap__scnprintf_prot(unsigned long prot, char *bf, size_t size, bo {return strarray__scnprintf_flags (&strarray__mmap_prot, bf, size, show_prefix, prot); } - -static size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_prot (char *bf,size_t size,struct syscall_arg *arg) {unsigned long prot = arg->val;@@ -19,18 +19,18 @@ static size_t syscall_arg__scnprintf_mmap_prot(char *bf, size_t size, struct sysreturn mmap__scnprintf_prot (prot, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_PROT syscall_arg__scnprintf_mmap_prot +#include "trace/beauty/generated/mmap_flags_array.c" -static DEFINE_STRARRAY(mmap_flags, "MAP_"); +DEFINE_STRARRAY (mmap_flags,"MAP_" );static size_t mmap__scnprintf_flags (unsigned long flags,char *bf,size_t size,bool show_prefix) {return strarray__scnprintf_flags (&strarray__mmap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mmap_flags (char *bf,size_t size, +struct syscall_arg *arg) {unsigned long flags = arg->val;@@ -40,7 +40,7 @@ static size_t syscall_arg__scnprintf_mmap_flags(char *bf, size_t size,return mmap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MMAP_FLAGS syscall_arg__scnprintf_mmap_flags +#include "trace/beauty/generated/mremap_flags_array.c" static DEFINE_STRARRAY(mremap_flags, "MREMAP_");@@ -50,7 +50,7 @@ static size_t mremap__scnprintf_flags(unsigned long flags, char *bf, size_t sizereturn strarray__scnprintf_flags (&strarray__mremap_flags, bf, size, show_prefix, flags); } -static size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mremap_flags (char *bf,size_t size,struct syscall_arg *arg) {unsigned long flags = arg->val;@@ -60,7 +60,7 @@ static size_t syscall_arg__scnprintf_mremap_flags(char *bf, size_t size, structreturn mremap__scnprintf_flags (flags, bf, size, arg->show_string_prefix); } -#define SCA_MREMAP_FLAGS syscall_arg__scnprintf_mremap_flags +static size_t madvise__scnprintf_behavior (int behavior,char *bf,size_t size) {@@ -73,10 +73,8 @@ static size_t madvise__scnprintf_behavior(int behavior, char *bf, size_t size)return scnprintf (bf, size,"%#" , behavior); } -static size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_madvise_behavior (char *bf,size_t size, +struct syscall_arg *arg) {return madvise__scnprintf_behavior (arg->val, bf, size); } - -#define SCA_MADV_BHV syscall_arg__scnprintf_madvise_behaviortools/perf/trace/beauty/mode_t.c +3
-3@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/stat.h> #include <unistd.h> @@ -20,7 +22,7 @@#define S_IXUGO (S_IXUSR|S_IXGRP|S_IXOTH)#endif -static size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_mode_t (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="S_" ;@@ -67,5 +69,3 @@ static size_t syscall_arg__scnprintf_mode_t(char *bf, size_t size, struct syscalreturn printed; } - -#define SCA_MODE_T syscall_arg__scnprintf_mode_ttools/perf/trace/beauty/msg_flags.c +4
-4@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/socket.h> @@ -27,8 +29,8 @@# define MSG_CMSG_CLOEXEC 0x40000000#endif -static size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_msg_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="MSG_" ;@@ -72,5 +74,3 @@ static size_t syscall_arg__scnprintf_msg_flags(char *bf, size_t size,return printed; } - -#define SCA_MSG_FLAGS syscall_arg__scnprintf_msg_flagstools/perf/trace/beauty/open_flags.c +2
-0@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/stat.h> #include <fcntl.h> tools/perf/trace/beauty/perf_event_open.c +10
-11@@ -1,4 +1,8 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "util/evsel_fprintf.h" +#include <linux/perf_event.h> +#ifndef PERF_FLAG_FD_NO_GROUP # define PERF_FLAG_FD_NO_GROUP (1UL << 0)#endif @@ -15,8 +19,8 @@# define PERF_FLAG_FD_CLOEXEC (1UL << 3)/* O_CLOEXEC */ #endif -static size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_perf_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="PERF_" ;@@ -43,7 +47,7 @@ static size_t syscall_arg__scnprintf_perf_flags(char *bf, size_t size,return printed; } -#define SCA_PERF_FLAGS syscall_arg__scnprintf_perf_flags +struct attr_fprintf_args {size_t size, printed;@@ -76,19 +80,14 @@ static size_t perf_event_attr___scnprintf(struct perf_event_attr *attr, char *bfstatic size_t syscall_arg__scnprintf_augmented_perf_event_attr (struct syscall_arg *arg,char *bf,size_t size) { -return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, arg->trace->show_zeros); +return perf_event_attr___scnprintf ((void *)arg->augmented.args->value, bf, size, +trace__show_zeros (arg->trace)); } -static size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_perf_event_attr (char *bf,size_t size,struct syscall_arg *arg) {if (arg->augmented.args)return syscall_arg__scnprintf_augmented_perf_event_attr (arg, bf, size);return scnprintf (bf, size,"%#lx" , arg->val); } - -#define SCA_PERF_ATTR syscall_arg__scnprintf_perf_event_attr -// 'argname' is just documentational at this point, to remove the previous comment with that info -#define SCA_PERF_ATTR_FROM_USER (argname) \ - { .scnprintf = SCA_PERF_ATTR, \ - .from_user = true, }tools/perf/trace/beauty/pid.c +4
-1@@ -1,11 +1,14 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include "util/machine.h" +#include "util/thread.h" size_t syscall_arg__scnprintf_pid (char *bf,size_t size,struct syscall_arg *arg) {int pid = arg->val;struct trace *trace = arg->trace;size_t printed =scnprintf (bf, size,"%d" , pid); -struct thread *thread =machine__findnew_thread (trace->host, pid, pid); +struct thread *thread =machine__findnew_thread (trace__host (trace), pid, pid);if (thread !=NULL ) {if (!thread__comm_set (thread))tools/perf/trace/beauty/sched_policy.c +4
-4@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sched.h> /* @@ -14,8 +16,8 @@#define SCHED_RESET_ON_FORK 0x40000000#endif -static size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_sched_policy (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SCHED_" ;@@ -46,5 +48,3 @@ static size_t syscall_arg__scnprintf_sched_policy(char *bf, size_t size,return printed; } - -#define SCA_SCHED_POLICY syscall_arg__scnprintf_sched_policytools/perf/trace/beauty/seccomp.c +6
-6@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#ifndef SECCOMP_SET_MODE_STRICT #define SECCOMP_SET_MODE_STRICT 0#endif @@ -6,7 +8,7 @@#define SECCOMP_SET_MODE_FILTER 1#endif -static size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_seccomp_op (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_SET_MODE_" ;@@ -24,14 +26,14 @@ static size_t syscall_arg__scnprintf_seccomp_op(char *bf, size_t size, struct syreturn printed; } -#define SCA_SECCOMP_OP syscall_arg__scnprintf_seccomp_op +#ifndef SECCOMP_FILTER_FLAG_TSYNC #define SECCOMP_FILTER_FLAG_TSYNC 1#endif -static size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_seccomp_flags (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SECCOMP_FILTER_FLAG_" ;@@ -51,5 +53,3 @@ static size_t syscall_arg__scnprintf_seccomp_flags(char *bf, size_t size,return printed; } - -#define SCA_SECCOMP_FLAGS syscall_arg__scnprintf_seccomp_flagstools/perf/trace/beauty/signum.c +3
-3@@ -1,7 +1,9 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <signal.h> -static size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_signum (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SIG" ;@@ -53,5 +55,3 @@ static size_t syscall_arg__scnprintf_signum(char *bf, size_t size, struct syscalreturn scnprintf (bf, size,"%#x" , sig); } - -#define SCA_SIGNUM syscall_arg__scnprintf_signumtools/perf/trace/beauty/socket_type.c +3
-3@@ -1,4 +1,6 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/socket.h> @@ -18,7 +20,7 @@#define SOCK_TYPE_MASK 0xf#endif -static size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) +size_t syscall_arg__scnprintf_socket_type (char *bf,size_t size,struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="SOCK_" ;@@ -59,5 +61,3 @@ static size_t syscall_arg__scnprintf_socket_type(char *bf, size_t size, struct sreturn printed; } - -#define SCA_SK_TYPE syscall_arg__scnprintf_socket_typetools/perf/trace/beauty/syscalltbl.c renamed from tools/perf/util/syscalltbl.c +0
-0tools/perf/trace/beauty/syscalltbl.h renamed from tools/perf/util/syscalltbl.h +0
-0tools/perf/trace/beauty/tracepoints/Build +21
-0@@ -1,2 +1,23 @@ perf-y += x86_irq_vectors.o perf-y += x86_msr.o + +beauty_outdir := $(OUTPUT)trace/beauty/generated +beauty_arch_asm_dir := $(srctree)/tools/perf/trace/beauty/arch/x86/include/asm/ +x86_arch_asm_dir := $(srctree)/tools/arch/x86/include/asm/ + +x86_arch_irq_vectors_array := $(beauty_outdir)/x86_arch_irq_vectors_array.c +x86_arch_irq_vectors_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_irq_vectors.sh + +$(x86_arch_irq_vectors_array): $(beauty_arch_asm_dir)/irq_vectors.h $(x86_arch_irq_vectors_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_irq_vectors_tbl)' $(beauty_arch_asm_dir) > $@ + +x86_arch_MSRs_array := $(beauty_outdir)/x86_arch_MSRs_array.c +x86_arch_MSRs_tbl := $(srctree)/tools/perf/trace/beauty/tracepoints/x86_msr.sh + +$(x86_arch_MSRs_array): $(x86_arch_asm_dir)/msr-index.h $(x86_arch_MSRs_tbl) + $(call rule_mkdir) + $(Q)$(call echo-cmd,gen)$(SHELL) '$(x86_arch_MSRs_tbl)' $(x86_arch_asm_dir) > $@ + +$(OUTPUT)trace/beauty/tracepoints/x86_irq_vectors.o: $(x86_arch_irq_vectors_array) +$(OUTPUT)trace/beauty/tracepoints/x86_msr.o: $(x86_arch_MSRs_array)tools/perf/trace/beauty/waitid_options.c +4
-4@@ -1,9 +1,11 @@// SPDX-License-Identifier: LGPL-2.1 +#include "trace/beauty/beauty.h" +#include <sys/types.h> #include <sys/wait.h> -static size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, -struct syscall_arg *arg) +size_t syscall_arg__scnprintf_waitid_options (char *bf,size_t size, +struct syscall_arg *arg) {bool show_prefix = arg->show_string_prefix;const char *prefix ="W" ;@@ -25,5 +27,3 @@ static size_t syscall_arg__scnprintf_waitid_options(char *bf, size_t size,return printed; } - -#define SCA_WAITID_OPTIONS syscall_arg__scnprintf_waitid_optionstools/perf/util/Build +3
-1@@ -75,7 +75,7 @@ perf-util-y += sample.o perf-util-y += sample-raw.o perf-util-y += s390-sample-raw.o perf-util-y += amd-sample-raw.o -perf-util-$(CONFIG_TRACE) += syscalltbl.o + perf-util-y += ordered-events.o perf-util-y += namespaces.o perf-util-y += comm.o@@ -441,3 +441,5 @@ $(OUTPUT)%.pylint_log: % $(Q)$(call echo-cmd,test)pylint "$<" > $@ || (cat $@ && rm $@ && false) perf-util-y += $(PYLINT_TEST_LOGS) + +tools/perf/util/bpf-trace-summary.c +1
-1@@ -6,7 +6,7 @@#include <stdlib.h> #include "dwarf-regs.h" /* for EM_HOST */ -#include "syscalltbl.h" +#include "trace/beauty/syscalltbl.h" #include "util/cgroup.h" #include "util/hashmap.h" #include "util/trace.h" tools/perf/util/env.c +0
-4@@ -635,10 +635,6 @@ const char *perf_env__arch(struct perf_env *env)return normalize_arch (arch_name); } -#if defined(HAVE_LIBTRACEEVENT ) -#include "trace/beauty/arch_errno_names.c" -#endif -const char *perf_env__arch_strerrno (struct perf_env *env__maybe_unused ,int err__maybe_unused ) {#if defined(HAVE_LIBTRACEEVENT )tools/perf/util/env.h +1
-0@@ -189,6 +189,7 @@ void cpu_cache_level__free(struct cpu_cache_level *cache);const char *perf_env__arch (struct perf_env *env);const char *perf_env__arch_strerrno (struct perf_env *env,int err); +arch_syscalls__strerrno_t *arch_syscalls__strerrno_function (const char *arch);const char *perf_env__cpuid (struct perf_env *env);const char *perf_env__raw_arch (struct perf_env *env);int perf_env__nr_cpus_avail (struct perf_env *env);Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 3/14 perf build: Decouple pmu-events from prepare umbrella target
In reply to Ian RogersCurrently, the $(LIBPMU_EVENTS_IN) sub-make depends on the massive "prepare" umbrella target. Because "prepare" depends on external libraries (libapi, libperf, etc.) as well as dozens of generated headers, make completely serializes the launch of the pmu-events sub-make behind some of those unrelated prerequisites.
Since pmu-events is a large compilation unit, unblock its startup by binding it directly to only $(LIBPERF) instead of prepare. This allows background python generation scripts to overlap simultaneously with the rest of the build.
Testing a parallel build (make -j28 clean all) shows improvements:
Before:
real 0m27.642s
user 2m32.356s
sys 0m26.683sAfter:
real 0m22.254s
user 2m32.810s
sys 0m24.646sThis reclaims over 5 full seconds of build latency (~19.5% overall reduction) by elevating average CPU concurrency from ~5.5 active cores up to ~8 active cores.
--- tools/perf/Makefile.perf | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/perf/Makefile.perf +1 -1
@@ -550,7 +550,7 @@ build := -f $(srctree)/tools/build/Makefile.build dir=. obj $(PERF_IN): prepare FORCE $(Q)$(MAKE) $(build)=perf -$(LIBPMU_EVENTS_IN): FORCE prepare +$(LIBPMU_EVENTS_IN): FORCE $(LIBPERF) $(Q)$(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 4/14 perf build: Remove empty archheaders target
In reply to Ian RogersRemove empty target that doesn't do anything.
--- tools/perf/Makefile.perf | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
tools/perf/Makefile.perf +2 -2
@@ -627,7 +627,7 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) -prepare: $(OUTPUT)PERF-VERSION-FILE archheaders \ +prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ $(LIBAPI) \ $(LIBPERF) \
@@ -1070,7 +1070,7 @@ FORCE: .PHONY: all install clean config-clean strip install-gtk .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell .PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare -.PHONY: archheaders python_perf_target +.PHONY: python_perf_target endif # force_fixdep
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 5/14 perf build: Move BPF skeleton generation out of Makefile.perf
In reply to Ian RogersCurrently, the top-level Makefile.perf defines a massive global bpf-skel umbrella target that pre-compiles all 12+ BPF skeletons (%.skel.h) upfront before launching sub-makes. This forces unrelated sub-makes to serialize behind bpftool and clang BPF target evaluations, causing parallel build bottlenecks.
Furthermore, bench_uprobe.bpf.c lived inside util/bpf_skel/, breaking conceptual directory encapsulation since it is consumed purely by bench/uprobe.c.
Refactor the BPF skeletons to better achieve directory isolation:
1. Move tools/perf/util/bpf_skel/bench_uprobe.bpf.c directly into tools/perf/bench/bpf_skel/.
2. Extract the skeleton generation infrastructure out of Makefile.perf into a shared inclusion file tools/perf/bpf_skel.mak.
3. Include bpf_skel.mak locally inside tools/perf/util/Build and tools/perf/bench/Build and bind precise local prerequisites.
4. Safely synchronize the shared bpftool bootstrap and vmlinux.h targets via the conditional prepare: umbrella to avoid parallel sub-make races, while evaluating the actual skeletons completely locally on demand. A later patch will move these targets into bpf_skel.mak.
5. Export CLANG from the global Makefile to ensure accurate tool propagation.
6. Clean up Makefile.perf by stripping the global bpf-skel umbrella target and its SKELETONS list.
While removing code from Makefile.perf generally helps build performance, the impact here is minimal. The main motivation for the change is to better encapsulate things in the build and simplify Makefile.perf that has around 50 lines removed.
---
tools/perf/Makefile.perf | 59 ++-----------------
tools/perf/bench/Build | 6 ++
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 54 +++++++++++++++++
tools/perf/util/Build | 15 ++++-
6 files changed, 79 insertions(+), 57 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.maktools/perf/Makefile.perf +4 -55
@@ -274,7 +274,7 @@ ifeq ($(PYLINT),1) PYLINT := $(shell which pylint 2> /dev/null) endif -export srctree OUTPUT RM CC CXX RUSTC LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK +export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT include $(srctree)/tools/build/Makefile.include
@@ -632,8 +632,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \ - $(LIBSYMBOL) \ - bpf-skel + $(LIBSYMBOL) ifdef LIBBPF_STATIC prepare: $(LIBBPF)
@@ -914,44 +913,13 @@ python-clean: SKEL_OUT := $(abspath $(OUTPUT)util/bpf_skel) SKEL_TMP_OUT := $(abspath $(SKEL_OUT)/.tmp) -SKELETONS := $(SKEL_OUT)/bpf_prog_profiler.skel.h -SKELETONS += $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h -SKELETONS += $(SKEL_OUT)/bperf_cgroup.skel.h $(SKEL_OUT)/func_latency.skel.h -SKELETONS += $(SKEL_OUT)/off_cpu.skel.h $(SKEL_OUT)/lock_contention.skel.h -SKELETONS += $(SKEL_OUT)/kwork_trace.skel.h $(SKEL_OUT)/sample_filter.skel.h -SKELETONS += $(SKEL_OUT)/kwork_top.skel.h $(SKEL_OUT)/syscall_summary.skel.h -SKELETONS += $(SKEL_OUT)/bench_uprobe.skel.h -SKELETONS += $(SKEL_OUT)/augmented_raw_syscalls.skel.h $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_OUTPUT) $(LIBSYMBOL_OUTPUT): $(Q)$(MKDIR) -p $@ ifeq ($(CONFIG_PERF_BPF_SKEL),y) +prepare: $(BPFTOOL) $(SKEL_OUT)/vmlinux.h BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool -# Get Clang's default includes on this system, as opposed to those seen by -# '--target=bpf'. This fixes "missing" files on some architectures/distros, -# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc. -# -# Use '-idirafter': Don't interfere with include mechanics except where the -# build would have failed anyways. -define get_sys_includes -$(shell $(1) $(2) -v -E - </dev/null 2>&1 \ - | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \ -$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}') -endef - -ifneq ($(CROSS_COMPILE),) -CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%)) -endif - -CLANG_OPTIONS = -Wall -CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH)) -BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES) -TOOLS_UAPI_INCLUDE := -I$(srctree)/tools/include/uapi - -ifneq ($(WERROR),0) - CLANG_OPTIONS += -Werror -endif $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
@@ -993,29 +961,10 @@ else $(Q)cp "$(VMLINUX_H)" $@ endif -$(SKEL_TMP_OUT)/%.bpf.o: $(OUTPUT)PERF-VERSION-FILE util/bpf_skel/perf_version.h | $(SKEL_TMP_OUT) -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h - $(QUIET_CLANG)$(CLANG) -g -O2 -fno-stack-protector --target=bpf \ - $(CLANG_OPTIONS) $(EXTRA_BPF_FLAGS) $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ - -include $(OUTPUT)PERF-VERSION-FILE -include util/bpf_skel/perf_version.h \ - -fms-extensions -Wno-microsoft-anon-tag \ - -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@ - -$(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o | $(BPFTOOL) - $(QUIET_GENSKEL)$(BPFTOOL) gen skeleton $< > $@ - -bpf-skel: $(SKELETONS) - -.PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o - -else # CONFIG_PERF_BPF_SKEL - -bpf-skel: - endif # CONFIG_PERF_BPF_SKEL bpf-skel-clean: - $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKELETONS) $(SKEL_OUT)/vmlinux.h + $(call QUIET_CLEAN, bpf-skel) $(RM) -r $(SKEL_TMP_OUT) $(SKEL_OUT)/*.skel.h $(SKEL_OUT)/vmlinux.h $(OUTPUT)bench/bpf_skel/*.skel.h $(OUTPUT)bench/bpf_skel/.tmp pmu-events-clean: ifeq ($(OUTPUT),)
tools/perf/bench/Build +6 -0
@@ -24,3 +24,9 @@ perf-bench-$(CONFIG_X86_64) += mem-memcpy-x86-64-asm.o perf-bench-$(CONFIG_X86_64) += mem-memset-x86-64-asm.o perf-bench-$(CONFIG_NUMA) += numa.o + +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)bench/uprobe.o: $(SKEL_OUT)/bench_uprobe.skel.h +endif
tools/perf/bench/bpf_skel/bench_uprobe.bpf.c renamed from tools/perf/util/bpf_skel/bench_uprobe.bpf.c +0 -0
tools/perf/bench/uprobe.c +1 -1
@@ -44,7 +44,7 @@ static const char * const bench_uprobe_usage[] = { };#ifdef HAVE_BPF_SKEL -#include "bpf_skel/bench_uprobe.skel.h" +#include "bench/bpf_skel/bench_uprobe.skel.h" #define bench_uprobe__attach_uprobe(prog) \ skel->links.prog = bpf_program__attach_uprobe_opts(/*prog=*/skel->progs.prog, \
tools/perf/bpf_skel.mak new file +54 -0
54 lines not on this page · raw message
tools/perf/util/Build +14 -1
@@ -442,4 +442,17 @@ $(OUTPUT)%.pylint_log: % perf-util-y += $(PYLINT_TEST_LOGS) - +ifeq ($(CONFIG_PERF_BPF_SKEL),y) +include $(srctree)/tools/perf/bpf_skel.mak + +$(OUTPUT)util/bpf_ftrace.o: $(SKEL_OUT)/func_latency.skel.h +$(OUTPUT)util/bpf-filter.o: $(SKEL_OUT)/sample_filter.skel.h +$(OUTPUT)util/bpf_kwork_top.o: $(SKEL_OUT)/kwork_top.skel.h +$(OUTPUT)util/bpf_off_cpu.o: $(SKEL_OUT)/off_cpu.skel.h +$(OUTPUT)util/bpf-trace-summary.o: $(SKEL_OUT)/syscall_summary.skel.h +$(OUTPUT)util/bpf_counter_cgroup.o: $(SKEL_OUT)/bperf_cgroup.skel.h +$(OUTPUT)util/bpf_trace_augment.o: $(SKEL_OUT)/augmented_raw_syscalls.skel.h +$(OUTPUT)util/bpf_counter.o: $(SKEL_OUT)/bpf_prog_profiler.skel.h $(SKEL_OUT)/bperf_leader.skel.h $(SKEL_OUT)/bperf_follower.skel.h +$(OUTPUT)util/bpf_lock_contention.o: $(SKEL_OUT)/lock_contention.skel.h +$(OUTPUT)util/bpf_kwork.o: $(SKEL_OUT)/kwork_trace.skel.h +endif
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 6/14 perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak
In reply to Ian RogersCurrently, bpftool and vmlinux.h are prerequisites of the top-level prepare target in Makefile.perf. This unnecessarily blocks the massive parallel C compilation of libraries (perf-util, perf-ui, pmu-events) during the initial startup phase.
Move all bpftool and vmlinux.h generation rules down into tools/perf/bpf_skel.mak to encapsulate BPF tooling completely within the skeleton framework. Remove them entirely from prepare to unblock immediate parallel build execution.
To prevent parallel sub-makes (perf-util and perf-bench) from racing to build shared prerequisites concurrently, while maintaining strict directory encapsulation without top-level inclusions, serialize bench after the util static archive finishes using an order-only prerequisite:
$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL)
--- tools/perf/Makefile.config | 2 +- tools/perf/Makefile.perf | 61 +++++--------------------------------- tools/perf/bpf_skel.mak | 47 +++++++++++++++++++++++++++++ 3 files changed, 55 insertions(+), 55 deletions(-)
tools/perf/Makefile.config +1 -1
@@ -736,7 +736,7 @@ ifeq ($(BUILD_BPF_SKEL),1) endif ifndef GEN_VMLINUX_H - VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h + VMLINUX_H_FILE=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h endif dwarf-post-unwind := 1
tools/perf/Makefile.perf +7 -54
88 lines not on this page · raw message
tools/perf/bpf_skel.mak +47 -0
57 lines not on this page · raw message
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 7/14 perf build: Pre-generate BPF skeleton tooling during umbrella prepare phase
In reply to Ian RogersCurrently, BPF skeleton generation rules (bpf_skel.mak) are evaluated as part of util/Build. However, because LIBPERF_UTIL_IN explicitly depends on the top-level static libbpf archive, Make completely blocked the execution of bpftool bootstrap and skeleton generation until libbpf finished compiling midway through the build.
Since bpftool bootstrap compiles its own independent copy of libbpf.a, it does not depend on the top-level libbpf target.
Decouple early skeleton tooling generation by attaching bpf-skel-prepare to the umbrella prepare target, exporting CONFIG_PERF_BPF_SKEL to ensure accurate feature propagation. This allows Make to compile bpftool and dump vmlinux.h in the background at build startup, eliminating the initial sub-make startup bottleneck before BPF object compilation while keeping 100% of tooling rules perfectly encapsulated in bpf_skel.mak. Provide an empty fallback target to ensure builds succeed when BPF skeletons are disabled.
--- tools/perf/Makefile.perf | 8 ++++++-- tools/perf/bpf_skel.mak | 8 ++++++++ 2 files changed, 14 insertions(+), 2 deletions(-)
tools/perf/Makefile.perf +6 -2
@@ -275,7 +275,7 @@ ifeq ($(PYLINT),1) endif export srctree OUTPUT RM CC CXX RUSTC CLANG LD AR CFLAGS CXXFLAGS RUST_FLAGS V BISON FLEX AWK LIBBPF READELF -export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT +export HOSTCC HOSTLD HOSTAR HOSTCFLAGS SHELLCHECK MYPY PYLINT CONFIG_PERF_BPF_SKEL include $(srctree)/tools/build/Makefile.include
@@ -629,8 +629,12 @@ endif __build-dir = $(subst $(OUTPUT),,$(dir $@)) build-dir = $(or $(__build-dir),.) +bpf-skel-prepare: + $(Q)$(MAKE) -f bpf_skel.mak bpf-skel-prepare + prepare: $(OUTPUT)PERF-VERSION-FILE \ arm64-sysreg-defs \ + bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ $(LIBSUBCMD) \
@@ -971,7 +975,7 @@ FORCE: .PHONY: all install clean config-clean strip install-gtk .PHONY: shell_compatibility_test please_set_SHELL_PATH_to_a_more_modern_shell -.PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare +.PHONY: .FORCE-PERF-VERSION-FILE TAGS tags cscope FORCE prepare bpf-skel-prepare .PHONY: python_perf_target endif # force_fixdep
tools/perf/bpf_skel.mak +8 -0
@@ -15,6 +15,10 @@ ifeq ($(CONFIG_PERF_BPF_SKEL),y) BPFTOOL := $(SKEL_TOOL_TMP_OUT)/bootstrap/bpftool VMLINUX_H := $(SKEL_TOOL_OUT)/vmlinux.h +.PHONY: bpf-skel-prepare +bpf-skel-prepare: $(BPFTOOL) $(VMLINUX_H) + @: + define get_sys_includes $(shell $(1) $(2) -v -E - </dev/null 2>&1 \ | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
@@ -93,6 +97,10 @@ $(SKEL_OUT)/%.skel.h: $(SKEL_TMP_OUT)/%.bpf.o $(BPFTOOL) .PRECIOUS: $(SKEL_TMP_OUT)/%.bpf.o +else # CONFIG_PERF_BPF_SKEL +.PHONY: bpf-skel-prepare +bpf-skel-prepare: + @: endif # CONFIG_PERF_BPF_SKEL clean:
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 8/14 perf build: Move libsymbol dependency out of prepare step
In reply to Ian RogersThe prepare step is a large serialization point before parallel sub-makes build the perf tool. The libsymbol headers are used in the bench and util libraries. Move the libsymbol dependency out of the prepare step and into the dependencies for those targets to avoid it being a source of serialization in the prepare step.
--- tools/perf/Makefile.perf | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-)
tools/perf/Makefile.perf +3 -4
@@ -558,7 +558,7 @@ $(LIBPMU_EVENTS): $(LIBPMU_EVENTS_IN) # The $(LIBPERF_UTIL) dependency is to ensure bpftool and vmlinux.h # aren't racily built for bench/bpf_skel/bench_uprobe.bpf.c -$(LIBPERF_BENCH_IN): FORCE prepare | $(LIBPERF_UTIL) +$(LIBPERF_BENCH_IN): FORCE prepare $(LIBSYMBOL) | $(LIBPERF_UTIL) $(Q)$(MAKE) $(build)=perf-bench $(LIBPERF_BENCH): $(LIBPERF_BENCH_IN)
@@ -576,7 +576,7 @@ $(LIBPERF_UI_IN): FORCE prepare $(LIBPERF_UI): $(LIBPERF_UI_IN) $(QUIET_AR)$(RM) $@ && $(AR) rcs $@ $< -$(LIBPERF_UTIL_IN): FORCE prepare +$(LIBPERF_UTIL_IN): FORCE prepare $(LIBSYMBOL) $(Q)$(MAKE) $(build)=perf-util $(LIBPERF_UTIL): $(LIBPERF_UTIL_IN)
@@ -637,8 +637,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ - $(LIBSUBCMD) \ - $(LIBSYMBOL) + $(LIBSUBCMD) ifdef LIBBPF_STATIC prepare: $(LIBBPF)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v4 08/14] perf build: Move libsymbol dependency out of prepare step
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Removing `$(LIBSYMBOL)` from the `prepare` step breaks standalone Makefile targets such as `make…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Removing $(LIBSYMBOL) from the prepare step breaks standalone Makefile targets such as make check and direct object compilation (e.g., make util/symbol.o). --
commit 75b0ba18e6b9ffc69fa79d6bb98e33c664d4de91
Author: Ian Rogers <[email protected]>
perf build: Move libsymbol dependency out of prepare step
This commit removes the libsymbol dependency from the global prepare step and moves it directly into the bench and util library dependencies. This is intended to reduce build serialization.
Quoted 4 lines
tools/perf/Makefile.perf +0
-0
[ ... ]
tools/perf/Makefile.perf prepare: $(OUTPUT)PERF-VERSION-FILE \ · line 640
Quoted 7 lines
tools/perf/Makefile.perf +1
-2@@ -637,8 +637,7 @@ prepare: $(OUTPUT)PERF-VERSION-FILE \ bpf-skel-prepare \ $(LIBAPI) \ $(LIBPERF) \ - $(LIBSUBCMD) \ - $(LIBSYMBOL) + $(LIBSUBCMD)
Does removing $(LIBSYMBOL) from the prepare step break standalone Makefile targets such as make check or direct object compilation like make util/symbol.o?
The prepare target serves as an umbrella dependency to ensure all necessary headers are generated and installed before compilation begins. For targets that rely solely on prepare, such as the pattern rule $(OUTPUT)%.o: %.c prepare FORCE, removing this dependency might cause the build to execute without the libsymbol headers being available.
Running make check or make util/symbol.o in a clean tree could cause sparse or gcc to fail with a missing header error when processing files that include symbol/kallsyms.h.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=8
Ian Rogers
reviewerPATCH v4 9/14 perf build: Remove redundant libbpf feature check for static builds
In reply to Ian RogersBy default, the perf tool compiles and statically links against its own internal copy of libbpf (tools/lib/bpf/libbpf.a), setting LIBBPF_STATIC=1.
Despite this static linkage, Makefile.config unconditionally executed $(call feature_check,libbpf), which forced a synchronous sub-make fork during AST parsing to detect dynamic system libbpf libraries.
As noted in the internal Makefile comments, this check was executed purely so that running make VF=1 would display the detection status of system libbpf. During standard builds without LIBBPF_DYNAMIC=1, the detection result was entirely ignored.
Wrap the libbpf feature check inside LIBBPF_DYNAMIC=1 so Make avoids the redundant sub-make fork overhead during standard static builds.
--- tools/perf/Makefile.config | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
tools/perf/Makefile.config +1 -3
@@ -587,10 +587,8 @@ ifndef NO_LIBELF ifndef NO_LIBBPF ifeq ($(feature-bpf), 1) - # detecting libbpf without LIBBPF_DYNAMIC, so make VF=1 shows libbpf detection status - $(call feature_check,libbpf) - ifdef LIBBPF_DYNAMIC + $(call feature_check,libbpf) ifeq ($(feature-libbpf), 1) EXTLIBS += -lbpf CFLAGS += -DHAVE_LIBBPF_SUPPORT
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 10/14 perf pmu-events: Split big_c_string storage into standalone compilation unit
In reply to Ian RogersCurrently, jevents.py emits both the massive 2.8 MB big_c_string literal and tens of thousands of compact_pmu_event struct arrays into a single pmu-events.c compilation unit. Compiling this giant file takes ~2.2 seconds on a single CPU core during Kbuild startup.
Refactor jevents.py to emit big_c_string into a dedicated pmu-events-string.c compilation unit. This allows Kbuild to compile pmu-events.o and pmu-events-string.o simultaneously in parallel across two separate CPU cores, preserving 100% string deduplication and zero dynamic ELF relocations while cutting C compilation latency in half.
Add pmu-events-string.c to tools/perf/.gitignore to ensure in-tree Kbuild runs do not leave untracked generated files in the working directory.
To guarantee 100% backward compatibility with GNU Make 4.0+ (avoiding the Make 4.3+ grouped target &: syntax which causes older Make versions like 4.2.1 to spawn multiple concurrent jevents.py processes during parallel builds), implement a robust dependency chaining pattern:
$(PMU_EVENTS_C): $(JEVENTS_DEPS)
$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C)
@:
This ensures jevents.py is invoked exactly once. If jevents.py aborts early, Make's .DELETE_ON_ERROR: purges pmu-events.c, guaranteeing that subsequent Make invocations correctly re-execute the script and overwrite pmu-events-string.c, keeping the build fully self-correcting.
--- tools/perf/.gitignore | 1 + tools/perf/Makefile.perf | 4 ++-- tools/perf/pmu-events/Build | 16 +++++++++++++++- tools/perf/pmu-events/jevents.py | 21 +++++++++++++++++---- tools/perf/pmu-events/pmu-events.h | 2 ++ 5 files changed, 37 insertions(+), 7 deletions(-)
tools/perf/.gitignore +1 -0
@@ -38,6 +38,7 @@ arch/*/include/generated/ trace/beauty/generated/ pmu-events/arch/common/common/legacy-cache.json pmu-events/pmu-events.c +pmu-events/pmu-events-string.c pmu-events/jevents pmu-events/metric_test.log pmu-events/empty-pmu-events.log
tools/perf/Makefile.perf +2 -2
@@ -925,7 +925,7 @@ bpf-skel-clean: pmu-events-clean: ifeq ($(OUTPUT),) $(call QUIET_CLEAN, pmu-events) $(RM) \ - pmu-events/pmu-events.c \ + pmu-events/pmu-events*.c \ pmu-events/metric_test.log \ pmu-events/test-empty-pmu-events.c \ pmu-events/empty-pmu-events.log
@@ -933,7 +933,7 @@ ifeq ($(OUTPUT),) -name 'extra-metricgroups.json' -delete else # When an OUTPUT directory is present, clean up the copied pmu-events/arch directory. $(call QUIET_CLEAN, pmu-events) $(RM) -r $(OUTPUT)pmu-events/arch \ - $(OUTPUT)pmu-events/pmu-events.c \ + $(OUTPUT)pmu-events/pmu-events*.c \ $(OUTPUT)pmu-events/metric_test.log \ $(OUTPUT)pmu-events/test-empty-pmu-events.c \ $(OUTPUT)pmu-events/empty-pmu-events.log
tools/perf/pmu-events/Build +15 -1
@@ -1,7 +1,12 @@ EMPTY_PMU_EVENTS_C = pmu-events/empty-pmu-events.c # pmu-events.c will be generated by jevents.py or copied from EMPTY_PMU_EVENTS_C PMU_EVENTS_C = $(OUTPUT)pmu-events/pmu-events.c +PMU_EVENTS_STRING_C = $(OUTPUT)pmu-events/pmu-events-string.c + pmu-events-y += pmu-events.o +ifneq ($(NO_JEVENTS),1) +pmu-events-y += pmu-events-string.o +endif # pmu-events.c file is generated in the OUTPUT directory so it needs a # separate rule to depend on it properly
@@ -9,6 +14,10 @@ $(OUTPUT)pmu-events/pmu-events.o: $(PMU_EVENTS_C) $(call rule_mkdir) $(call if_changed_dep,cc_o_c) +$(OUTPUT)pmu-events/pmu-events-string.o: $(PMU_EVENTS_STRING_C) + $(call rule_mkdir) + $(call if_changed_dep,cc_o_c) + # Message for $(call echo-cmd,cp), possibly remove the src file from # the destination to save space in the build log. quiet_cmd_cp = COPY $(patsubst %$<,%,$@) <- $<
@@ -118,6 +127,7 @@ CUR_OUT_JSON := $(shell [ -d $(OUT_DIR) ] && find $(OUT_DIR) -type f) # Things in the OUTPUT directory but shouldn't be there as computed by # OUT_JSON and GEN_JSON. + ORPHAN_FILES := $(filter-out $(OUT_JSON) $(GEN_JSON),$(CUR_OUT_JSON)) # Message for $(call echo-cmd,mkd). There is already a mkdir message
@@ -224,6 +234,10 @@ endif # and inputs are dependencies. $(PMU_EVENTS_C): $(JEVENTS_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) \ + $(OUT_DIR) $(PMU_EVENTS_C) $(PMU_EVENTS_STRING_C) + +$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C) + $(Q)test -f $@ || { rm -f $<; $(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events; } endif # ifeq ($(NO_JEVENTS),1)
tools/perf/pmu-events/jevents.py +17 -4
@@ -1422,6 +1422,8 @@ such as "arm/cortex-a34".''', ) ap.add_argument( 'output_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=sys.stdout) + ap.add_argument( + 'output_string_file', type=argparse.FileType('w', encoding='utf-8'), nargs='?', default=None) _args = ap.parse_args() _args.output_file.write(f"""
@@ -1463,10 +1465,19 @@ struct pmu_table_entry { ftw(arch_path, [], preprocess_one_file) _bcs.compute() - _args.output_file.write('static const char *const big_c_string =\n') - for s in _bcs.big_string: - _args.output_file.write(s) - _args.output_file.write(';\n\n') + if not _args.output_string_file: + _args.output_file.write('static const char *const big_c_string =\n') + for s in _bcs.big_string: + _args.output_file.write(s) + _args.output_file.write(';\n\n') + else: + _args.output_string_file.write('/* SPDX-License-Identifier: GPL-2.0 */\n') + _args.output_string_file.write('/* Autogenerated by jevents.py */\n') + _args.output_string_file.write('#include <pmu-events/pmu-events.h>\n') + _args.output_string_file.write('const char big_c_string[] =\n') + for s in _bcs.big_string: + _args.output_string_file.write(s) + _args.output_string_file.write(';\n') for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], process_one_file)
@@ -1476,6 +1487,8 @@ struct pmu_table_entry { print_mapping_table(archs) print_system_mapping_table() print_metricgroups() + if _args.output_string_file: + _args.output_string_file.close() if __name__ == '__main__': main()
tools/perf/pmu-events/pmu-events.h +2 -0
@@ -141,4 +141,6 @@ int pmu_for_each_sys_metric(pmu_metric_iter_fn fn, void *data);const char *describe_metricgroup (const char *group); +extern const char big_c_string[]; +#endif
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v4 10/14] perf pmu-events: Split big_c_string storage into standalone compilation unit
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Medium] Race condition in parallel builds causing non-deterministic 'No such file or directory' errors.…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Medium] Race condition in parallel builds causing non-deterministic 'No such file or directory' errors.
- [Medium] Meaningful code and commit message mismatch regarding dependency chaining.
- [Low] NO_JEVENTS=1 fallback and default jevents.py mode fail to compile due to big_c_string linkage conflict. --
commit b03f8034531e9c582fd4777daf59f93349bcf916
Author: Ian Rogers <[email protected]>
perf pmu-events: Split big_c_string storage into standalone compilation unit
This commit splits the generation of big_c_string into a dedicated compilation unit to speed up Kbuild. It modifies jevents.py to optionally generate this file, adds it to the build system, and updates the pmu-events.h header.
Quoted 7 lines
To guarantee 100% backward compatibility with GNU Make 4.0+ (avoiding the Make 4.3+ grouped target &: syntax which causes older Make versions like 4.2.1 to spawn multiple concurrent
jevents.pyprocesses during parallel builds), implement a robust dependency chaining pattern:
$(PMU_EVENTS_C): $(JEVENTS_DEPS)
$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C)
@:
The commit message states that a no-op recipe (@:) is used for dependency chaining, but the actual code implements a file-recovery shell script instead. Could the commit message be updated to match the code?
Quoted 4 lines
tools/perf/pmu-events/Build +0
-0
[ ... ]
tools/perf/pmu-events/Build endif · line 241
Quoted 10 lines
tools/perf/pmu-events/Build +5
-1@@ -224,6 +234,10 @@ endif # and inputs are dependencies. $(PMU_EVENTS_C): $(JEVENTS_DEPS) $(call rule_mkdir) - $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) $(OUT_DIR) $@ + $(Q)$(call echo-cmd,gen)$(PYTHON) $(JEVENTS_PY) $(JEVENTS_ARCH) $(JEVENTS_MODEL) \ + $(OUT_DIR) $(PMU_EVENTS_C) $(PMU_EVENTS_STRING_C) + +$(PMU_EVENTS_STRING_C): $(PMU_EVENTS_C) + $(Q)test -f $@ || { rm -f $<; $(MAKE) -f $(srctree)/tools/build/Makefile.build dir=pmu-events obj=pmu-events; }
Does this introduce a race condition during parallel builds?
If pmu-events-string.c is missing but pmu-events.c is up-to-date, a parallel build could execute this recipe concurrently with the compilation of pmu-events.o. The rm -f $< would remove pmu-events.c while the C compiler is trying to read it, leading to a "No such file or directory" error.
tools/perf/pmu-events/pmu-events.h int pmu_for_each_sys_metric(pmu_metric_iter_fn fn, void *data); · line 144
Quoted 9 lines
tools/perf/pmu-events/pmu-events.h +1
-0@@ -141,4 +141,6 @@ int pmu_for_each_sys_metric(pmu_metric_iter_fn fn, void *data);const char *describe_metricgroup (const char *group); +extern const char big_c_string[];
Will this cause a compilation error when building with NO_JEVENTS=1?
The build uses empty-pmu-events.c which still defines big_c_string as:
static const char *const big_c_string = ...;
Including this header will result in a conflicting type and linkage error between const char[] and const char *const, and between extern and static. The same conflict appears to happen when invoking jevents.py without the output_string_file argument.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=10
Ian Rogers
reviewerPATCH v4 11/14 perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
In reply to Ian RogersCurrently, jevents.py parses hundreds of JSON event and metric files sequentially across all CPU architectures during Kbuild startup, taking ~3.3 seconds of pure single-core execution time.
Refactor jevents.py to pre-populate its internal JSON AST cache in parallel across all available CPU cores using ProcessPoolExecutor. Define the worker process initializer _init_worker at the top-level module scope to guarantee flawless pickling and standard event mapping inheritance under spawn multiprocessing semantics (avoiding AttributeError crashes when spawn is used instead of fork). This accelerates jevents.py execution by over 11x (from 3.3s down to ~290ms), fully reclaiming multi-core concurrency during the build generation phase.
--- tools/perf/pmu-events/jevents.py | 36 ++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-)
tools/perf/pmu-events/jevents.py +32 -4
@@ -457,8 +457,8 @@ class JsonEvent: return f'{{ { _bcs.offsets[s] } }}, /* {fix_comment(s)} */\n' -@lru_cache(maxsize=None) -def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: +_json_cache = {} +def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" try: events = json.load(open(path), object_hook=JsonEvent)
@@ -474,12 +474,16 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: if updates: for event in events: if event.metric_name in updates: - # print(f'Updated {event.metric_name} from\n"{event.metric_expr}"\n' - # f'to\n"{updates[event.metric_name]}"') event.metric_expr = updates[event.metric_name] return events +def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: + key = (path, topic) + if key not in _json_cache: + _json_cache[key] = _read_json_events_impl(path, topic) + return _json_cache[key] + def preprocess_arch_std_files(archpath: str) -> None: """Read in all architecture standard events.""" global _arch_std_events
@@ -1381,6 +1385,14 @@ const char *describe_metricgroup(const char *group) } """) +def _parallel_read_json_events(task: Tuple[str, str]) -> Tuple[str, str, Sequence[JsonEvent]]: + path, topic = task + return path, topic, _read_json_events_impl(path, topic) + +def _init_worker(std_events: dict) -> None: + global _arch_std_events + _arch_std_events = std_events + def main() -> None: global _args
@@ -1459,9 +1471,25 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json) + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor: + for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file) _bcs.compute()
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
[email protected]
AI reviewRe: [PATCH v4 11/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] The use of `ProcessPoolExecutor` without a `max_workers` limit violates GNU Make's jobserver…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] The use of ProcessPoolExecutor without a max_workers limit violates GNU Make's jobserver concurrency constraints, causing system oversubscription. --
commit 09eba7d34c7e309d0f6449fe513b92dfbec5e85c
Author: Ian Rogers <[email protected]>
perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
This commit refactors jevents.py to pre-populate its internal JSON AST cache in parallel across available CPU cores using ProcessPoolExecutor, reducing build generation time.
Quoted 3 lines
tools/perf/pmu-events/jevents.py +0
-0
[ ... ]
Quoted 1 line
tools/perf/pmu-events/jevents.py +0
-0@@ -1459,9 +1471,25 @@ struct pmu_table_entry {
[ ... ]
tools/perf/pmu-events/jevents.py struct pmu_table_entry { · line 1487
Quoted 6 lines
for arch in archs:
arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path)tools/perf/pmu-events/jevents.py +3
-0+ ftw(arch_path, [], collect_json) + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor:
Does this bypass GNU Make's jobserver concurrency constraints? By instantiating ProcessPoolExecutor without a max_workers limit, it will default to spawning workers for all available CPU cores.
If a user explicitly requests a restricted build with make -j1, or if this runs alongside numerous other compilation jobs, could it lead to extreme CPU contention and potential out-of-memory failures?
Quoted 6 lines
tools/perf/pmu-events/jevents.py +5
-0+ for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file)
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=11
Namhyung Kim
maintainerRe: [PATCH v4 11/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
In reply to Ian RogersQuoted 3 lines · Ian Rogers
Currently,
jevents.pyparses hundreds of JSON event and metric files sequentially across all CPU architectures duringKbuildstartup, taking ~3.3 seconds of pure single-core execution time.
I'm curious about this. Does it really need to parse all architectures? Oh... is it for perf stat record/report?
Thanks,
Namhyung
Quoted 95 lines
Refactor
jevents.pyto pre-populate its internal JSON AST cache in parallel across all available CPU cores using ProcessPoolExecutor. Define the worker process initializer_init_workerat the top-level module scope to guarantee flawless pickling and standard event mapping inheritance under spawn multiprocessing semantics (avoiding AttributeError crashes when spawn is used instead of fork). This acceleratesjevents.pyexecution by over 11x (from 3.3s down to ~290ms), fully reclaiming multi-core concurrency during the build generation phase.Tested-by James Clark <[email protected]>Assisted-by Gemini:gemini-3.1-pro-previewSigned-off-by Ian Rogers <[email protected]>--- tools/perf/pmu-events/jevents.py | 36 ++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-)tools/perf/pmu-events/jevents.py +32
-4@@ -457,8 +457,8 @@ class JsonEvent: return f'{{ { _bcs.offsets[s] } }}, /* {fix_comment(s)} */\n' -@lru_cache(maxsize=None) -def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: +_json_cache = {} +def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" try: events = json.load(open(path), object_hook=JsonEvent)@@ -474,12 +474,16 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: if updates: for event in events: if event.metric_name in updates: - # print(f'Updated {event.metric_name} from\n"{event.metric_expr}"\n' - # f'to\n"{updates[event.metric_name]}"') event.metric_expr = updates[event.metric_name] return events +def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: + key = (path, topic) + if key not in _json_cache: + _json_cache[key] = _read_json_events_impl(path, topic) + return _json_cache[key] + def preprocess_arch_std_files(archpath: str) -> None: """Read in all architecture standard events.""" global _arch_std_events@@ -1381,6 +1385,14 @@ const char *describe_metricgroup(const char *group) } """) +def _parallel_read_json_events(task: Tuple[str, str]) -> Tuple[str, str, Sequence[JsonEvent]]: + path, topic = task + return path, topic, _read_json_events_impl(path, topic) + +def _init_worker(std_events: dict) -> None: + global _arch_std_events + _arch_std_events = std_events + def main() -> None: global _args@@ -1459,9 +1471,25 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json) + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor: + for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file) _bcs.compute()Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerRe: [PATCH v4 11/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Quoted 8 lines · Namhyung Kim
Quoted 3 lines · Ian Rogers
Currently,
jevents.pyparses hundreds of JSON event and metric files sequentially across all CPU architectures duringKbuildstartup, taking ~3.3 seconds of pure single-core execution time.I'm curious about this. Does it really need to parse all architectures? Oh... is it for perf stat record/report?
So it is controlled by JEVENTS_ARCH and JEVENTS_MODEL:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/Build?h=perf-tools-next#n24
The default for JEVENTS_ARCH is SRCARCH (e.g. x86) while JEVENTS_MODEL defaults to "all". We have a build test for JEVENTS_ARCH is "all":
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/make?h=perf-tools-next#n74
So saying "all CPU architectures" is a bit over the top, but not wholly inaccurate.
Fwiw, I created a script to convert CPUIDs into models:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/models.py?h=perf-tools-next
which maybe could be used to reduce the perf binary size by building only the native CPU's JSON. It is a useful script for debugging :-)
Thanks,
Ian
Quoted 98 lines
Thanks,
NamhyungQuoted 95 lines
Refactor
jevents.pyto pre-populate its internal JSON AST cache in parallel across all available CPU cores using ProcessPoolExecutor. Define the worker process initializer_init_workerat the top-level module scope to guarantee flawless pickling and standard event mapping inheritance under spawn multiprocessing semantics (avoiding AttributeError crashes when spawn is used instead of fork). This acceleratesjevents.pyexecution by over 11x (from 3.3s down to ~290ms), fully reclaiming multi-core concurrency during the build generation phase.Tested-by James Clark <[email protected]>Assisted-by Gemini:gemini-3.1-pro-previewSigned-off-by Ian Rogers <[email protected]>--- tools/perf/pmu-events/jevents.py | 36 ++++++++++++++++++++++++++++---- 1 file changed, 32 insertions(+), 4 deletions(-)tools/perf/pmu-events/jevents.py +32
-5@@ -457,8 +457,8 @@ class JsonEvent: return f'{{ { _bcs.offsets[s] } }}, /* {fix_comment(s)} */\n' -@lru_cache(maxsize=None) -def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: +_json_cache = {} +def _read_json_events_impl(path: str, topic: str) -> Sequence[JsonEvent]: """Read json events from the specified file.""" try: events = json.load(open(path), object_hook=JsonEvent)@@ -474,12 +474,16 @@ def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: if updates: for event in events: if event.metric_name in updates: - # print(f'Updated {event.metric_name} from\n"{event.metric_expr}"\n' - # f'to\n"{updates[event.metric_name]}"') event.metric_expr = updates[event.metric_name] return events +def read_json_events(path: str, topic: str) -> Sequence[JsonEvent]: + key = (path, topic) + if key not in _json_cache: + _json_cache[key] = _read_json_events_impl(path, topic) + return _json_cache[key] + def preprocess_arch_std_files(archpath: str) -> None: """Read in all architecture standard events.""" global _arch_std_events@@ -1381,6 +1385,14 @@ const char *describe_metricgroup(const char *group) } """) +def _parallel_read_json_events(task: Tuple[str, str]) -> Tuple[str, str, Sequence[JsonEvent]]: + path, topic = task + return path, topic, _read_json_events_impl(path, topic) + +def _init_worker(std_events: dict) -> None: + global _arch_std_events + _arch_std_events = std_events + def main() -> None: global _args@@ -1459,9 +1471,25 @@ struct pmu_table_entry { raise IOError(f'Missing architecture directory \'{_args.arch}\'') archs.sort() + import concurrent.futures + tasks = [] + def collect_json(parents: Sequence[str], item: os.DirEntry) -> None: + if len(parents) == 0: + return + if item.is_file() and item.name.endswith('.json') and not item.name.endswith('metricgroups.json'): + tasks.append((item.path, get_topic(item.name))) + for arch in archs: arch_path = f'{_args.starting_dir}/{arch}' preprocess_arch_std_files(arch_path) + ftw(arch_path, [], collect_json) + + with concurrent.futures.ProcessPoolExecutor(initializer=_init_worker, initargs=(_arch_std_events,)) as executor: + for path, topic, events in executor.map(_parallel_read_json_events, tasks): + _json_cache[(path, topic)] = events + + for arch in archs: + arch_path = f'{_args.starting_dir}/{arch}' ftw(arch_path, [], preprocess_one_file) _bcs.compute() --2.54.0.563.g4f69b47b94-goog
Namhyung Kim
maintainerRe: [PATCH v4 11/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Quoted 10 lines · Ian Rogers
Quoted 8 lines · Namhyung Kim
Quoted 3 lines · Ian Rogers
Currently,
jevents.pyparses hundreds of JSON event and metric files sequentially across all CPU architectures duringKbuildstartup, taking ~3.3 seconds of pure single-core execution time.I'm curious about this. Does it really need to parse all architectures? Oh... is it for perf stat record/report?
Quoted 7 lines
So it is controlled by
JEVENTS_ARCHandJEVENTS_MODEL:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/Build?h=perf-tools-next#n24
The default forJEVENTS_ARCHis SRCARCH (e.g. x86) whileJEVENTS_MODELdefaults to "all". We have a build test forJEVENTS_ARCHis "all":
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/make?h=perf-tools-next#n74
So saying "all CPU architectures" is a bit over the top, but not wholly inaccurate.
Ok, so it's all x86 models. I think it's ok to have "all" for build tests.
Quoted 5 lines
Fwiw, I created a script to convert CPUIDs into models:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/models.py?h=perf-tools-next
which maybe could be used to reduce the perf binary size by building only the native CPU's JSON. It is a useful script for debugging :-)
Sounds like it should be used by default to pick the current model.
Thanks,
Namhyung
Ian Rogers
reviewerRe: [PATCH v4 11/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Quoted 24 lines · Namhyung Kim
Quoted 10 lines · Ian Rogers
Quoted 8 lines · Namhyung Kim
Quoted 3 lines · Ian Rogers
Currently,
jevents.pyparses hundreds of JSON event and metric files sequentially across all CPU architectures duringKbuildstartup, taking ~3.3 seconds of pure single-core execution time.I'm curious about this. Does it really need to parse all architectures? Oh... is it for perf stat record/report?
Quoted 7 lines
So it is controlled by
JEVENTS_ARCHandJEVENTS_MODEL:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/Build?h=perf-tools-next#n24
The default forJEVENTS_ARCHis SRCARCH (e.g. x86) whileJEVENTS_MODELdefaults to "all". We have a build test forJEVENTS_ARCHis "all":
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/make?h=perf-tools-next#n74
So saying "all CPU architectures" is a bit over the top, but not wholly inaccurate.Ok, so it's all x86 models. I think it's ok to have "all" for build tests.
Quoted 1 line
Quoted 6 lines
Quoted 4 lines
Fwiw, I created a script to convert CPUIDs into models:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/models.py?h=perf-tools-next
which maybe could be used to reduce the perf binary size by building only the native CPU's JSON. It is a useful script for debugging :-)Sounds like it should be used by default to pick the current model.
Having a build of perf tied to a particular model like Skylake, doesn't seem very good for distributions. I think the default should try to match what we hope the distributions will do, so an x86 binary should run on all x86 models.
Thanks,
Ian
Quoted 3 lines
Thanks,
Namhyung
Namhyung Kim
maintainerRe: [PATCH v4 11/14] perf pmu-events: Parallelize JSON and metric pre-computation in jevents.py
Quoted 30 lines · Ian Rogers
Quoted 29 lines · Namhyung Kim
Quoted 10 lines · Ian Rogers
Quoted 8 lines · Namhyung Kim
Quoted 3 lines · Ian Rogers
Currently,
jevents.pyparses hundreds of JSON event and metric files sequentially across all CPU architectures duringKbuildstartup, taking ~3.3 seconds of pure single-core execution time.I'm curious about this. Does it really need to parse all architectures? Oh... is it for perf stat record/report?
Quoted 7 lines
So it is controlled by
JEVENTS_ARCHandJEVENTS_MODEL:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/Build?h=perf-tools-next#n24
The default forJEVENTS_ARCHis SRCARCH (e.g. x86) whileJEVENTS_MODELdefaults to "all". We have a build test forJEVENTS_ARCHis "all":
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/make?h=perf-tools-next#n74
So saying "all CPU architectures" is a bit over the top, but not wholly inaccurate.Ok, so it's all x86 models. I think it's ok to have "all" for build tests.
Quoted 5 lines
Fwiw, I created a script to convert CPUIDs into models:
https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/pmu-events/models.py?h=perf-tools-next
which maybe could be used to reduce the perf binary size by building only the native CPU's JSON. It is a useful script for debugging :-)
Quoted 6 lines
Quoted 1 line
Sounds like it should be used by default to pick the current model.
Having a build of perf tied to a particular model like Skylake, doesn't seem very good for distributions. I think the default should try to match what we hope the distributions will do, so an x86 binary should run on all x86 models.
Oh right, distros will need it.
Thanks,
Namhyung
Ian Rogers
reviewerPATCH v4 12/14 perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds
In reply to Ian RogersIn Makefile.perf, ALL_PROGRAMS includes SCRIPTS (perf-archive, perf-iostat). However, unlike PROGRAMS and DLFILTERS, SCRIPTS was not prefixed with $(OUTPUT).
During out-of-tree builds (or when O= is specified), Make checked for the unprefixed target 'tools/perf/perf-archive'. Since the actual script was installed into $(OUTPUT)perf-archive, Make concluded the target was missing and continuously re-executed the script installation rule on every single incremental build.
Prefix SCRIPTS with $(OUTPUT) and update the static pattern rule to ensure Kbuild correctly tracks generated script prerequisites during incremental builds.
--- tools/perf/Makefile.perf | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +3 -3
@@ -405,7 +405,7 @@ python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT # Use the detected configuration -include $(OUTPUT).config-detected -SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) +SCRIPTS = $(addprefix $(OUTPUT),$(patsubst %.sh,%,$(SCRIPT_SH))) PROGRAMS += $(OUTPUT)perf
@@ -592,8 +592,8 @@ $(GTK_IN): FORCE prepare $(OUTPUT)libperf-gtk.so: $(GTK_IN) $(PERFLIBS) $(QUIET_LINK)$(CC) -o $@ -shared $(LDFLAGS) $(filter %.o,$^) $(GTK_LIBS) -$(SCRIPTS) : % : %.sh - $(QUIET_GEN)$(INSTALL) '[email protected]' '$(OUTPUT)$@' +$(SCRIPTS) : $(OUTPUT)% : %.sh + $(QUIET_GEN)$(INSTALL) '$<' '$@' $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE $(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Namhyung Kim
maintainerRe: [PATCH v4 12/14] perf build: Prefix SCRIPTS with output directory to fix continuous rebuilds
Quoted 3 lines · Ian Rogers
In Makefile.perf,
ALL_PROGRAMSincludes SCRIPTS (perf-archive, perf-iostat). However, unlike PROGRAMS and DLFILTERS, SCRIPTS was not prefixed with $(OUTPUT).
Looks like a bug.
Thanks,
Namhyung
Quoted 45 lines
During out-of-tree builds (or when O= is specified), Make checked for the unprefixed target '
tools/perf/perf-archive'. Since the actual script was installed into $(OUTPUT)perf-archive, Make concluded the target was missing and continuously re-executed the script installation rule on every single incremental build.Prefix SCRIPTS with $(OUTPUT) and update the static pattern rule to ensure
Kbuildcorrectly tracks generated script prerequisites during incremental builds.Tested-by James Clark <[email protected]>Assisted-by Gemini:gemini-3.1-pro-previewSigned-off-by Ian Rogers <[email protected]>--- tools/perf/Makefile.perf | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)tools/perf/Makefile.perf +3
-3@@ -405,7 +405,7 @@ python-clean := $(call QUIET_CLEAN, python) $(RM) -r $(PYTHON_EXTBUILD) $(OUTPUT # Use the detected configuration -include $(OUTPUT).config-detected -SCRIPTS = $(patsubst %.sh,%,$(SCRIPT_SH)) +SCRIPTS = $(addprefix $(OUTPUT),$(patsubst %.sh,%,$(SCRIPT_SH))) PROGRAMS += $(OUTPUT)perf@@ -592,8 +592,8 @@ $(GTK_IN): FORCE prepare $(OUTPUT)libperf-gtk.so: $(GTK_IN) $(PERFLIBS) $(QUIET_LINK)$(CC) -o $@ -shared $(LDFLAGS) $(filter %.o,$^) $(GTK_LIBS) -$(SCRIPTS) : % : %.sh - $(QUIET_GEN)$(INSTALL) '[email protected]' '$(OUTPUT)$@' +$(SCRIPTS) : $(OUTPUT)% : %.sh + $(QUIET_GEN)$(INSTALL) '$<' '$@' $(OUTPUT)PERF-VERSION-FILE: .FORCE-PERF-VERSION-FILE $(Q)$(SHELL_PATH) util/PERF-VERSION-GEN $(OUTPUT)Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 13/14 perf pmu-events: Convert recursive shell assignments and macros to Make built-ins
In reply to Ian RogersIn pmu-events/Build, ZENS, ARMS, and INTELS were assigned using recursive assignment (=), and model_name/vendor_name were evaluated using shell macros (echo ... | sed ...).
Because these variables were expanded inside the COPY_RULE dependency evaluation loop across hundreds of PMU JSON files and inside every metric generation recipe, Kbuild continuously re-executed 'ls', 'grep', and 'sed' shell forks thousands of times during AST parsing and execution.
Convert ZENS, ARMS, and INTELS to simply expanded variables (:=) and replace model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once when the Build file is parsed and evaluates path macros purely in memory, completely eliminating over 7,800 redundant sub-processes during build startup.
--- tools/perf/pmu-events/Build | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
tools/perf/pmu-events/Build +5 -5
@@ -56,12 +56,12 @@ $(LEGACY_CACHE_JSON): $(LEGACY_CACHE_PY) # Python to generate architectural metrics GEN_METRIC_DEPS := pmu-events/metric.py pmu-events/common_metrics.py # Functions to extract the model from an extra-metrics.json or extra-metricgroups.json path. -model_name = $(shell echo $(1)|sed -e 's@.\+/\(.*\)/extra-metric.*\.json@\1@') -vendor_name = $(shell echo $(1)|sed -e 's@.\+/\(.*\)/[^/]*/extra-metric.*\.json@\1@') +model_name = $(notdir $(patsubst %/,%,$(dir $(1)))) +vendor_name = $(notdir $(patsubst %/,%,$(dir $(patsubst %/,%,$(dir $(1)))))) ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),x86 all)) # Generate AMD Json -ZENS = $(shell ls -d pmu-events/arch/x86/amdzen*) +ZENS := $(shell ls -d pmu-events/arch/x86/amdzen*) ZEN_METRICS = $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metrics.json) ZEN_METRICGROUPS = $(foreach x,$(ZENS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(ZEN_METRICS) $(ZEN_METRICGROUPS)
@@ -78,7 +78,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),arm64 all)) # Generate ARM Json -ARMS = $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) +ARMS := $(shell ls -d pmu-events/arch/arm64/arm/*|grep -v cmn) ARM_METRICS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metrics.json) ARM_METRICGROUPS = $(foreach x,$(ARMS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(ARM_METRICS) $(ARM_METRICGROUPS)
@@ -95,7 +95,7 @@ endif ifeq ($(JEVENTS_ARCH),$(filter $(JEVENTS_ARCH),x86 all)) # Generate Intel Json -INTELS = $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapfile.csv) +INTELS := $(shell ls -d pmu-events/arch/x86/*|grep -v amdzen|grep -v mapfile.csv) INTEL_METRICS = $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metrics.json) INTEL_METRICGROUPS = $(foreach x,$(INTELS),$(OUTPUT)$(x)/extra-metricgroups.json) GEN_JSON += $(INTEL_METRICS) $(INTEL_METRICGROUPS)
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v4 14/14 perf build: Convert llvm-config shell queries to simply expanded variables
In reply to Ian RogersIn Makefile.config, CFLAGS, CXXFLAGS, LIBLLVM, and EXTLIBS were assigned using recursive expansion or appended with raw $(shell $(LLVM_CONFIG) ...) calls. Because these variables were expanded during dependency evaluation across every single object file compilation rule, Kbuild continuously re-executed llvm-config forks nearly 200 times during incremental builds.
Convert llvm-config shell queries to simply expanded variables (:=) to ensure Make evaluates LLVM compiler flags and library paths exactly once when Makefile.config is parsed, eliminating ~185 redundant sub-processes during build startup.
--- tools/perf/Makefile.config | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
tools/perf/Makefile.config +8 -5
@@ -951,11 +951,14 @@ ifndef NO_LIBLLVM $(call feature_check,llvm-perf) ifeq ($(feature-llvm-perf), 1) CFLAGS += -DHAVE_LIBLLVM_SUPPORT - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) - CXXFLAGS += -DHAVE_LIBLLVM_SUPPORT - CXXFLAGS += $(shell $(LLVM_CONFIG) --cxxflags) - LIBLLVM = $(shell $(LLVM_CONFIG) --libs all) $(shell $(LLVM_CONFIG) --system-libs) - EXTLIBS += -L$(shell $(LLVM_CONFIG) --libdir) $(LIBLLVM) + LLVM_CFLAGS := $(shell $(LLVM_CONFIG) --cflags 2>/dev/null) + LLVM_CXXFLAGS := $(shell $(LLVM_CONFIG) --cxxflags 2>/dev/null) + LLVM_LIBLLVM := $(shell $(LLVM_CONFIG) --libs all 2>/dev/null) $(shell $(LLVM_CONFIG) --system-libs 2>/dev/null) + LLVM_LIBDIR := $(shell $(LLVM_CONFIG) --libdir 2>/dev/null) + CFLAGS += $(LLVM_CFLAGS) + CXXFLAGS += -DHAVE_LIBLLVM_SUPPORT $(LLVM_CXXFLAGS) + LIBLLVM := $(LLVM_LIBLLVM) + EXTLIBS += -L$(LLVM_LIBDIR) $(LIBLLVM) EXTLIBS += -lstdc++ $(call detected,CONFIG_LIBLLVM) else
Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerv5 cover perf build: Reduce build time by nearly half
In reply to Ian RogersThis patch series refactors Kbuild internals, BPF skeleton generation, Python AST pre-computation, and foundational tooling dependencies across the perf tool build system. By eliminating umbrella target synchronization barriers, decoupling static library prerequisites, parallelizing single-core script generators, and eradicating redundant feature checks, this series unlocks greater concurrency during Kbuild startup.
On a 28-core build workstation (make -j28 all from scratch), clean build latency improves by over 44%:
Before:
real 0m29.006s
user 2m46.019s
sys 0m30.610sAfter:
real 0m16.091s
user 2m40.135s
sys 0m25.740sSaving 12.9 full seconds time per clean build. Furthermore, nothing to build incremental builds are improved by nearly 7x:
Before:
real 0m11.528s
user 0m9.633s
sys 0m6.965sAfter:
real 0m1.717s
user 0m1.682s
sys 0m0.960sSummary of Patches:
1: Fast-Path Feature Detection
- Refactors test-clang-bpf-co-re.bin and test-bpftool-skeletons.bin feature checks to group shell pipelines within curly braces and redirect both stdout and stderr to .make.output before touching $@ purely upon success
(> $(@:.bin=.make.output) 2>&1 && touch $@). Grouping the pipeline ({ cmd1 | cmd2; }) ensures that compiler stderr is successfully captured in .make.output rather than escaping to the parent shell. This perfectly matches standard Kbuild feature check conventions and ensures the target files are touched on disk purely upon success, allowing Kbuild to cache positive detections and avoid continuous sub-make re-evaluations during incremental builds. Adds test-bpftool-skeletons.bin to the clean FILES list and explicit source prerequisite test-clang-bpf-co-re.c.
2-4: Flattening Umbrella Prepare Barriers
- builtin-trace embedded inclusions and pmu-events generation are completely decoupled from the sequential "prepare" umbrella target, eliminating Make AST double-parsing overhead and unchoking parallel compilation barriers.
5-7: Decoupling & Pre-generating BPF Skeletons
- BPF skeleton rules are extracted out of Makefile.perf into bpf_skel.mak.
- Decouples bpftool bootstrap from top-level static libbpf dependencies, attaching bpf-skel-prepare directly to the umbrella prepare target. This allows Make to pre-compile bpftool and dump vmlinux.h in the background at build startup, removing the 7-second serialization bottleneck before BPF object compilation.
- Ensures benchmark skeleton intermediate .bpf.o files are cleanly removed during make clean, and adds bpf-skel-prepare to .PHONY.
8-9: Foundational Linkage Optimization
- Moves static libsymbol library prerequisites out of the prepare step.
- Eliminates redundant libbpf sub-make feature checks during static builds.
10-11: jevents.py Concurrency & Deduplication
- Splits the massive 2.8 MB big_c_string literal out of pmu-events.c into a dedicated pmu-events-string.c compilation unit. This slices C compilation latency in half by compiling string and struct tables simultaneously across separate CPU cores while preserving zero dynamic ELF relocations. Adds pmu-events-string.c to .gitignore, declares extern const char big_c_string[]; locally inside output_string_file and output_file when split to prevent linkage conflicts with empty-pmu-events.c, defers file closures to ensure identical timestamps, and uses canonical Make 4.0 @: dependency chaining.
- Pre-populates jevents.py JSON ASTs and metric formulas in parallel across all available CPU cores using ProcessPoolExecutor (accelerating Python execution by 11x, from 3.3s down to ~290ms). Moves _init_worker to top-level scope to ensure clean pickling under spawn multiprocessing start methods.
12: Out-of-Tree Incremental Rebuild Fix
- Prefixes SCRIPTS (perf-archive, perf-iostat) with $(OUTPUT) to prevent Make from continuously re-executing script installation rules on already built out-of-tree builds.
13-14: AST Parsing Optimization & Shell Fork Eradication
- Converts ZENS, ARMS, and INTELS in pmu-events/Build from recursive assignment (=) to simply expanded assignment (:=) and replaces model_name/vendor_name with pure GNU Make string functions. This guarantees Make executes directory probing shell forks exactly once during AST parsing and evaluates path macros purely in memory, completely eradicating over 7,800 redundant sub-processes during out-of-tree build evaluation.
- Converts llvm-config shell queries in Makefile.config from recursive assignment (=) to simply expanded assignment (:=). This eliminates ~185 redundant sub-processes that were previously executed across object compilation dependency checks.
Changes since v4:
- tools build (Patch 1): Refactored test-bpftool-skeletons.bin and test-clang-bpf-co-re.bin feature check recipes to group the shell pipeline within curly braces ({ cmd1 | cmd2; }) so that compiler stderr is successfully captured in .make.output rather than escaping to the parent shell. Added test-bpftool-skeletons.bin to the clean FILES list in feature/Makefile so that make clean correctly purges the generated binary and prevents permanent feature cache poisoning.
- perf pmu-events (Patch 10): Reverted secondary target rule in pmu-events/Build back to the canonical Make 4.0 @: dependency chaining pattern to prevent concurrency race conditions during parallel compilation. Removed global extern const char big_c_string[]; declaration from pmu-events.h and instead emitted the extern declaration locally inside output_string_file and output_file when split, preventing type and linkage conflicts with empty-pmu-events.c when building with NO_JEVENTS=1.
Ian Rogers (14):
tools build: Fix feature checks to touch target files on success perf trace beauty: Make beauty generated C code standalone .o files perf build: Decouple pmu-events from prepare umbrella target perf build: Remove empty archheaders target
perf build: Move BPF skeleton generation out of Makefile.perf perf build: Encapsulate vmlinux.h and bpftool in bpf_skel.mak perf build: Pre-generate BPF skeleton tooling during umbrella prepare
phase
perf build: Move libsymbol dependency out of prepare step perf build: Remove redundant libbpf feature check for static builds perf pmu-events: Split big_c_string storage into standalone
compilation unit
perf pmu-events: Parallelize JSON and metric pre-computation in
jevents.py
perf build: Prefix SCRIPTS with output directory to fix continuous
rebuilds
perf pmu-events: Convert recursive shell assignments and macros to
Make built-ins
perf build: Convert llvm-config shell queries to simply expanded
variables
tools/build/feature/Makefile | 13 +-
tools/perf/.gitignore | 1 +
tools/perf/Build | 2 +
tools/perf/Makefile.config | 19 +-
tools/perf/Makefile.perf | 423 ++----------------
tools/perf/bench/Build | 6 +
.../bpf_skel/bench_uprobe.bpf.c | 0
tools/perf/bench/uprobe.c | 2 +-
tools/perf/bpf_skel.mak | 109 +++++
tools/perf/builtin-trace.c | 32 +-
tools/perf/pmu-events/Build | 26 +-
tools/perf/pmu-events/jevents.py | 58 ++-
tools/perf/trace/beauty/Build | 276 ++++++++++++
tools/perf/trace/beauty/arch_errno_names.c | 2 +
tools/perf/trace/beauty/arch_errno_names.sh | 2 +-
tools/perf/trace/beauty/beauty.h | 60 +++
tools/perf/trace/beauty/eventfd.c | 6 +-
tools/perf/trace/beauty/fsconfig.c | 5 +
tools/perf/trace/beauty/futex_op.c | 5 +-
tools/perf/trace/beauty/futex_val3.c | 5 +-
tools/perf/trace/beauty/mmap.c | 24 +-
tools/perf/trace/beauty/mode_t.c | 6 +-
tools/perf/trace/beauty/msg_flags.c | 8 +-
tools/perf/trace/beauty/open_flags.c | 2 +
tools/perf/trace/beauty/perf_event_open.c | 21 +-
tools/perf/trace/beauty/pid.c | 5 +-
tools/perf/trace/beauty/sched_policy.c | 8 +-
tools/perf/trace/beauty/seccomp.c | 12 +-
tools/perf/trace/beauty/signum.c | 6 +-
tools/perf/trace/beauty/socket_type.c | 6 +-
.../perf/{util => trace/beauty}/syscalltbl.c | 0
.../perf/{util => trace/beauty}/syscalltbl.h | 0
tools/perf/trace/beauty/tracepoints/Build | 21 +
tools/perf/trace/beauty/waitid_options.c | 8 +-
tools/perf/util/Build | 17 +-
tools/perf/util/bpf-trace-summary.c | 2 +-
tools/perf/util/env.c | 4 -
tools/perf/util/env.h | 1 +
38 files changed, 687 insertions(+), 516 deletions(-)
rename tools/perf/{util => bench}/bpf_skel/bench_uprobe.bpf.c (100%)
create mode 100644 tools/perf/bpf_skel.mak
create mode 100644 tools/perf/trace/beauty/fsconfig.c
rename tools/perf/{util => trace/beauty}/syscalltbl.c (100%)
rename tools/perf/{util => trace/beauty}/syscalltbl.h (100%)Signature · 1 line
2.54.0.563.g4f69b47b94-goog
Ian Rogers
reviewerPATCH v5 1/14 tools build: Fix feature checks to touch target files on success
In tools/build/feature/Makefile, test-clang-bpf-co-re.bin and test-bpftool-skeletons.bin redirected grep output but never touched or created the $@ target file upon success.
Because the target file was never created on disk, Kbuild could never cache the result of the check. Consequently, Make treated the prerequisite as missing and continuously re-executed the Clang BPF backend and bpftool feature checks on every single sub-make evaluation during build startup.
Refactor both feature check recipes to group the shell pipeline within curly braces and redirect both stdout and stderr to .make.output before touching $@ purely upon success (> $(@:.bin=.make.output) 2>&1 && touch $@). Grouping the pipeline ({ cmd1 | cmd2; }) ensures that compiler stderr is successfully captured in .make.output rather than escaping to the parent shell. This perfectly matches standard Kbuild feature check conventions, allowing Make to cache positive detections and avoid redundant sub-make forks during incremental builds. Add test-bpftool-skeletons.bin to the FILES list so that make clean correctly purges the generated binary. For test-clang-bpf-co-re.bin, add explicit source file prerequisite test-clang-bpf-co-re.c and simplify the Clang recipe using $<.
--- tools/build/feature/Makefile | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
tools/build/feature/Makefile +7 -6
@@ -75,7 +75,8 @@ FILES= \ test-file-handle.bin \ test-libpfm4.bin \ test-rust.bin \ - test-libopenssl.bin + test-libopenssl.bin \ + test-bpftool-skeletons.bin FILES := $(addprefix $(OUTPUT),$(FILES))
@@ -383,9 +384,9 @@ $(OUTPUT)test-libaio.bin: $(OUTPUT)test-libzstd.bin: $(BUILD) -lzstd -$(OUTPUT)test-clang-bpf-co-re.bin: - $(CLANG) -S -g --target=bpf -o - $(patsubst %.bin,%.c,$(@F)) | \ - grep BTF_KIND_VAR +$(OUTPUT)test-clang-bpf-co-re.bin: test-clang-bpf-co-re.c + { $(CLANG) -S -g --target=bpf -o - $< | \ + grep BTF_KIND_VAR; } > $(@:.bin=.make.output) 2>&1 && touch $@ $(OUTPUT)test-file-handle.bin: $(BUILD)
@@ -397,8 +398,8 @@ $(OUTPUT)test-libopenssl.bin: $(BUILD) $(shell $(PKG_CONFIG) --libs --cflags openssl 2>/dev/null) $(OUTPUT)test-bpftool-skeletons.bin: - $(SYSTEM_BPFTOOL) version | grep '^features:.*skeletons' \ - > $(@:.bin=.make.output) 2>&1 + { $(SYSTEM_BPFTOOL) version | grep '^features:.*skeletons'; } \ + > $(@:.bin=.make.output) 2>&1 && touch $@ # Testing Rust is special: we don't compile anything, it's enough to check the # compiler presence. Compiling a test code for this purposes is problematic,
Signature · 1 line
2.54.0.563.g4f69b47b94-goog