Fast path feature detection fixes
cover Fast path feature detection fixes
Hi,
This is on top of a previous patch I sent today for libunwind
opt-in fallout, I'll continue checking if what is being enabled when test-all.c builds is the right set of features to be declared as detected on this fast path method.
- Arnaldo
Arnaldo Carvalho de Melo (2):
tools build feature: Add some comments to explain the FEATURE_TESTS logic tools build feature: Don't set feature-libcap=1 if libcap-devel isn't available
tools/build/Makefile.feature | 37 ++++++++++++++++++++++++++++++++++-- tools/build/feature/Makefile | 2 +- 2 files changed, 36 insertions(+), 3 deletions(-)
Signature · 1 line
2.47.0
PATCH 1/2 tools build feature: Add some comments to explain the FEATURE_TESTS logic
From: Arnaldo Carvalho de Melo <[email protected]>
The tools/build/feature/test-all.c works in conjunction with the tools/build/Makefile.feature FEATURE_TESTS_BASIC and FEATURE_TESTS_EXTRA contents, so that if test-all.c manages to be built, we go on and iterate all entries in FEATURE_TESTS_BASIC + FEATURE_TESTS_EXTRA setting then to 1.
To test this:
$ rm -rf /tmp/b ; mkdir /tmp/b ; make -C tools/perf O=/tmp/b feature-dump $ cat /tmp/b/feature/test-all.make.output $ ldd /tmp/b/feature/test-all.bin linux-vdso.so.1 (0x00007f2a47a67000) libdw.so.1 => /lib64/libdw.so.1 (0x00007f2a477cf000) libpython3.12.so.1.0 => /lib64/libpython3.12.so.1.0 (0x00007f2a471fe000) libm.so.6 => /lib64/libm.so.6 (0x00007f2a4711a000) libtraceevent.so.1 => /lib64/libtraceevent.so.1 (0x00007f2a470f2000) libtracefs.so.1 => /lib64/libtracefs.so.1 (0x00007f2a470cb000) libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f2a46c1b000) libz.so.1 => /lib64/libz.so.1 (0x00007f2a46bf8000) libbabeltrace-ctf.so.1 => /lib64/libbabeltrace-ctf.so.1 (0x00007f2a46bad000) libcapstone.so.5 => /lib64/libcapstone.so.5 (0x00007f2a464b8000) libopencsd_c_api.so.1 => /lib64/libopencsd_c_api.so.1 (0x00007f2a464a8000) libopencsd.so.1 => /lib64/libopencsd.so.1 (0x00007f2a46422000) libelf.so.1 => /lib64/libelf.so.1 (0x00007f2a46406000) libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f2a463f6000) libslang.so.2 => /lib64/libslang.so.2 (0x00007f2a46113000) libperl.so.5.38 => /lib64/libperl.so.5.38 (0x00007f2a45d74000) libc.so.6 => /lib64/libc.so.6 (0x00007f2a45b83000) liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f2a45b50000) libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f2a45a91000) libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f2a45a7b000) /lib64/ld-linux-x86-64.so.2 (0x00007f2a47a69000) libbabeltrace.so.1 => /lib64/libbabeltrace.so.1 (0x00007f2a45a6b000) libpopt.so.0 => /lib64/libpopt.so.0 (0x00007f2a45a5b000) libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f2a45a51000) libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f2a45a4a000) libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f2a458fa000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2a45696000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2a45668000) libcrypt.so.2 => /lib64/libcrypt.so.2 (0x00007f2a45630000) libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f2a45590000) $ head /tmp/b/FEATURE-DUMP feature-backtrace=1 feature-libdw=1 feature-eventfd=1 feature-fortify-source=1 feature-get_current_dir_name=1 feature-gettid=1 feature-glibc=1 feature-libbfd=1 feature-libbfd-buildid=1 feature-libcap=1 $
There are inconsistencies that are being audited, as can be seen above with the libcap case, that is not linked with test-all.bin nor is present in test-all.c, so shouldn't be set as present. Further patches are going to address those inconsistencies, but lets document this a bit more to reduce the chances of this happening again.
--- tools/build/Makefile.feature | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)
tools/build/Makefile.feature +35 -0
@@ -28,6 +28,41 @@ endef # the rule that uses them - an example for that is the 'bionic' # feature check. ] # +# These + the ones in FEATURE_TESTS_EXTRA are included in +# tools/build/feature/test-all.c and we try to build it all together +# then setting all those features to '1' meaning they are all enabled. +# +# There are things like fortify-source that will be set to 1 because test-all +# is built with the flags needed to test if its enabled, resulting in +# +# $ rm -rf /tmp/b ; mkdir /tmp/b ; make -C tools/perf O=/tmp/b feature-dump +# $ grep fortify-source /tmp/b/FEATURE-DUMP +# feature-fortify-source=1 +# $ +# +# All the others should have lines in tools/build/feature/test-all.c like: +# +# #define main main_test_disassembler_init_styled +# # include "test-disassembler-init-styled.c" +# #undef main +# +# #define main main_test_libzstd +# # include "test-libzstd.c" +# #undef main +# +# int main(int argc, char *argv[]) +# { +# main_test_disassembler_four_args(); +# main_test_libzstd(); +# return 0; +# } +# +# If the sample above works, then we end up with these lines in the FEATURE-DUMP +# file: +# +# feature-disassembler-four-args=1 +# feature-libzstd=1 +# FEATURE_TESTS_BASIC := \ backtrace \ libdw \
Signature · 1 line
2.47.0
Re: [PATCH 1/2] tools build feature: Add some comments to explain the FEATURE_TESTS logic
Quoted 8 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
The
tools/build/feature/test-all.cworks in conjunction with thetools/build/Makefile.featureFEATURE_TESTS_BASICandFEATURE_TESTS_EXTRAcontents, so that iftest-all.cmanages to be built, we go on and iterate all entries inFEATURE_TESTS_BASIC+FEATURE_TESTS_EXTRAsetting then to 1.
nit: s/then/them/
Quoted 54 lines
To test this:
$ rm -rf /tmp/b ; mkdir /tmp/b ; make -C tools/perf O=/tmp/b feature-dump $ cat /tmp/b/feature/test-all.make.output $ ldd /tmp/b/feature/test-all.bin linux-vdso.so.1 (0x00007f2a47a67000) libdw.so.1 => /lib64/libdw.so.1 (0x00007f2a477cf000) libpython3.12.so.1.0 => /lib64/libpython3.12.so.1.0 (0x00007f2a471fe000) libm.so.6 => /lib64/libm.so.6 (0x00007f2a4711a000) libtraceevent.so.1 => /lib64/libtraceevent.so.1 (0x00007f2a470f2000) libtracefs.so.1 => /lib64/libtracefs.so.1 (0x00007f2a470cb000) libcrypto.so.3 => /lib64/libcrypto.so.3 (0x00007f2a46c1b000) libz.so.1 => /lib64/libz.so.1 (0x00007f2a46bf8000) libbabeltrace-ctf.so.1 => /lib64/libbabeltrace-ctf.so.1 (0x00007f2a46bad000) libcapstone.so.5 => /lib64/libcapstone.so.5 (0x00007f2a464b8000) libopencsd_c_api.so.1 => /lib64/libopencsd_c_api.so.1 (0x00007f2a464a8000) libopencsd.so.1 => /lib64/libopencsd.so.1 (0x00007f2a46422000) libelf.so.1 => /lib64/libelf.so.1 (0x00007f2a46406000) libnuma.so.1 => /lib64/libnuma.so.1 (0x00007f2a463f6000) libslang.so.2 => /lib64/libslang.so.2 (0x00007f2a46113000) libperl.so.5.38 => /lib64/libperl.so.5.38 (0x00007f2a45d74000) libc.so.6 => /lib64/libc.so.6 (0x00007f2a45b83000) liblzma.so.5 => /lib64/liblzma.so.5 (0x00007f2a45b50000) libzstd.so.1 => /lib64/libzstd.so.1 (0x00007f2a45a91000) libbz2.so.1 => /lib64/libbz2.so.1 (0x00007f2a45a7b000) /lib64/ld-linux-x86-64.so.2 (0x00007f2a47a69000) libbabeltrace.so.1 => /lib64/libbabeltrace.so.1 (0x00007f2a45a6b000) libpopt.so.0 => /lib64/libpopt.so.0 (0x00007f2a45a5b000) libuuid.so.1 => /lib64/libuuid.so.1 (0x00007f2a45a51000) libgmodule-2.0.so.0 => /lib64/libgmodule-2.0.so.0 (0x00007f2a45a4a000) libglib-2.0.so.0 => /lib64/libglib-2.0.so.0 (0x00007f2a458fa000) libstdc++.so.6 => /lib64/libstdc++.so.6 (0x00007f2a45696000) libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00007f2a45668000) libcrypt.so.2 => /lib64/libcrypt.so.2 (0x00007f2a45630000) libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x00007f2a45590000) $ head /tmp/b/FEATURE-DUMP feature-backtrace=1 feature-libdw=1 feature-eventfd=1 feature-fortify-source=1 feature-get_current_dir_name=1 feature-gettid=1 feature-glibc=1 feature-libbfd=1 feature-libbfd-buildid=1 feature-libcap=1 $There are inconsistencies that are being audited, as can be seen above with the libcap case, that is not linked with test-all.bin nor is present in
test-all.c, so shouldn't be set as present. Further patches are going to address those inconsistencies, but lets document this a bit more to reduce the chances of this happening again.
Quoted 7 lines
Cc Adrian Hunter <[email protected]>Cc Ian Rogers <[email protected]>Cc James Clark <[email protected]>Cc Jiri Olsa <[email protected]>Cc Kan Liang <[email protected]>Cc Namhyung Kim <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>
Thanks,
Ian
Quoted 53 lines
--- tools/build/Makefile.feature | 35 +++++++++++++++++++++++++++++++++++ 1 file changed, 35 insertions(+)tools/build/Makefile.feature +35
-1@@ -28,6 +28,41 @@ endef # the rule that uses them - an example for that is the 'bionic' # feature check. ] # +# These + the ones in FEATURE_TESTS_EXTRA are included in +# tools/build/feature/test-all.c and we try to build it all together +# then setting all those features to '1' meaning they are all enabled. +# +# There are things like fortify-source that will be set to 1 because test-all +# is built with the flags needed to test if its enabled, resulting in +# +# $ rm -rf /tmp/b ; mkdir /tmp/b ; make -C tools/perf O=/tmp/b feature-dump +# $ grep fortify-source /tmp/b/FEATURE-DUMP +# feature-fortify-source=1 +# $ +# +# All the others should have lines in tools/build/feature/test-all.c like: +# +# #define main main_test_disassembler_init_styled +# # include "test-disassembler-init-styled.c" +# #undef main +# +# #define main main_test_libzstd +# # include "test-libzstd.c" +# #undef main +# +# int main(int argc, char *argv[]) +# { +# main_test_disassembler_four_args(); +# main_test_libzstd(); +# return 0; +# } +# +# If the sample above works, then we end up with these lines in the FEATURE-DUMP +# file: +# +# feature-disassembler-four-args=1 +# feature-libzstd=1 +# FEATURE_TESTS_BASIC := \ backtrace \ libdw \ --2.47.0
PATCH 2/2 tools build feature: Don't set feature-libcap=1 if libcap-devel isn't available
In reply to Arnaldo Carvalho de MeloFrom: Arnaldo Carvalho de Melo <[email protected]>
libcap isn't tested in the tools/build/feature/test-all.c fast path feature detection process, so don't set it as available if test-all manages to build.
There are other users of this feature detection mechanism, and they explicitely ask for libcap to be tested, so are not affected by this patch, for instance, with this patch in place:
$ make -C tools/bpf/bpftool/ clean <SNIP> make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' ⬢ [acme@toolbox perf-tools-next]$ make -C tools/bpf/bpftool/ make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'
Auto-detecting system features:
... clang-bpf-co-re: [ on ]
... llvm: [ on ]
... libcap: [ on ]
... libbfd: [ on ]
... libelf-zstd: [ on ]
<SNIP>
LINK bpftool
make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'
$
$ sudo rpm -e libcap-devel
$ make -C tools/bpf/bpftool/
<SNIP>
make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' Auto-detecting system features:
... clang-bpf-co-re: [ on ]
... llvm: [ on ]
... libcap: [ OFF ]
... libbfd: [ on ]
... libelf-zstd: [ on ]
$
--- tools/build/Makefile.feature | 2 -- tools/build/feature/Makefile | 2 +- 2 files changed, 1 insertion(+), 3 deletions(-)
tools/build/Makefile.feature +0 -2
@@ -73,7 +73,6 @@ FEATURE_TESTS_BASIC := \ glibc \ libbfd \ libbfd-buildid \ - libcap \ libelf \ libelf-getphdrnum \ libelf-gelf_getnote \
@@ -150,7 +149,6 @@ FEATURE_DISPLAY ?= \ glibc \ libbfd \ libbfd-buildid \ - libcap \ libelf \ libnuma \ numa_num_possible_cpus \
tools/build/feature/Makefile +1 -1
@@ -110,7 +110,7 @@ all: $(FILES) __BUILD = $(CC) $(CFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.c,$(@F)) $(LDFLAGS) BUILD = $(__BUILD) > $(@:.bin=.make.output) 2>&1 BUILD_BFD = $(BUILD) -DPACKAGE='"perf"' -lbfd -ldl - BUILD_ALL = $(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -lslang $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl -lz -llzma -lzstd -lcap + BUILD_ALL = $(BUILD) -fstack-protector-all -O2 -D_FORTIFY_SOURCE=2 -ldw -lelf -lnuma -lelf -lslang $(FLAGS_PERL_EMBED) $(FLAGS_PYTHON_EMBED) -DPACKAGE='"perf"' -lbfd -ldl -lz -llzma -lzstd __BUILDXX = $(CXX) $(CXXFLAGS) -MD -Wall -Werror -o $@ $(patsubst %.bin,%.cpp,$(@F)) $(LDFLAGS) BUILDXX = $(__BUILDXX) > $(@:.bin=.make.output) 2>&1
Signature · 1 line
2.47.0
Re: [PATCH 2/2] tools build feature: Don't set feature-libcap=1 if libcap-devel isn't available
Quoted 41 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
libcap isn't tested in the
tools/build/feature/test-all.cfast path feature detection process, so don't set it as available if test-all manages to build.There are other users of this feature detection mechanism, and they explicitely ask for libcap to be tested, so are not affected by this patch, for instance, with this patch in place:
$ make -C tools/bpf/bpftool/ clean <SNIP> make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' ⬢ [acme@toolbox perf-tools-next]$ make -C tools/bpf/bpftool/ make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'Auto-detecting system features:
... clang-bpf-co-re: [ on ] ... llvm: [ on ] ... libcap: [ on ] ... libbfd: [ on ] ... libelf-zstd: [ on ] <SNIP> LINK bpftool make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' $ $ sudo rpm -e libcap-devel $ make -C tools/bpf/bpftool/ <SNIP> make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'Auto-detecting system features:
... clang-bpf-co-re: [ on ]
... llvm: [ on ]
... libcap: [ OFF ]
... libbfd: [ on ]
... libelf-zstd: [ on ]$
Quoted 8 lines
Cc Adrian Hunter <[email protected]>Cc Ian Rogers <[email protected]>Cc James Clark <[email protected]>Cc Jiri Olsa <[email protected]>Cc Kan Liang <[email protected]>Cc Namhyung Kim <[email protected]>Cc Quentin Monnet <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>
Thanks,
Ian
Quentin Monnet
Acked-byRe: [PATCH 2/2] tools build feature: Don't set feature-libcap=1 if libcap-devel isn't available
2024-12-11 20:25 UTC-0800 ~ Ian Rogers <[email protected]>
Quoted 52 lines
Quoted 41 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
libcap isn't tested in the
tools/build/feature/test-all.cfast path feature detection process, so don't set it as available if test-all manages to build.There are other users of this feature detection mechanism, and they explicitely ask for libcap to be tested, so are not affected by this patch, for instance, with this patch in place:
$ make -C tools/bpf/bpftool/ clean <SNIP> make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' ⬢ [acme@toolbox perf-tools-next]$ make -C tools/bpf/bpftool/ make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'Auto-detecting system features:
... clang-bpf-co-re: [ on ] ... llvm: [ on ] ... libcap: [ on ] ... libbfd: [ on ] ... libelf-zstd: [ on ] <SNIP> LINK bpftool make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' $ $ sudo rpm -e libcap-devel $ make -C tools/bpf/bpftool/ <SNIP> make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'Auto-detecting system features:
... clang-bpf-co-re: [ on ]
... llvm: [ on ]
... libcap: [ OFF ]
... libbfd: [ on ]
... libelf-zstd: [ on ]$
Quoted 8 lines
Cc Adrian Hunter <[email protected]>Cc Ian Rogers <[email protected]>Cc James Clark <[email protected]>Cc Jiri Olsa <[email protected]>Cc Kan Liang <[email protected]>Cc Namhyung Kim <[email protected]>Cc Quentin Monnet <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>
Quoted 1 line
Reviewed-by Ian Rogers <[email protected]>
Thanks Arnaldo for checking the bpftool build :)
Quentin
Re: [PATCH 2/2] tools build feature: Don't set feature-libcap=1 if libcap-devel isn't available
Quoted 53 lines · Quentin Monnet
2024-12-11 20:25 UTC-0800 ~ Ian Rogers <[email protected]>
Quoted 52 lines
Quoted 49 lines · Arnaldo Carvalho de Melo
From: Arnaldo Carvalho de Melo <[email protected]>
libcap isn't tested in the
tools/build/feature/test-all.cfast path feature detection process, so don't set it as available if test-all manages to build.There are other users of this feature detection mechanism, and they explicitely ask for libcap to be tested, so are not affected by this patch, for instance, with this patch in place:
$ make -C tools/bpf/bpftool/ clean <SNIP> make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' ⬢ [acme@toolbox perf-tools-next]$ make -C tools/bpf/bpftool/ make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'Auto-detecting system features:
... clang-bpf-co-re: [ on ] ... llvm: [ on ] ... libcap: [ on ] ... libbfd: [ on ] ... libelf-zstd: [ on ] <SNIP> LINK bpftool make: Leaving directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool' $ $ sudo rpm -e libcap-devel $ make -C tools/bpf/bpftool/ <SNIP> make: Entering directory '/home/acme/git/perf-tools-next/tools/bpf/bpftool'Auto-detecting system features:
... clang-bpf-co-re: [ on ]
... llvm: [ on ]
... libcap: [ OFF ]
... libbfd: [ on ]
... libelf-zstd: [ on ]$
Cc Adrian Hunter <[email protected]>Cc Ian Rogers <[email protected]>Cc James Clark <[email protected]>Cc Jiri Olsa <[email protected]>Cc Kan Liang <[email protected]>Cc Namhyung Kim <[email protected]>Cc Quentin Monnet <[email protected]>Signed-off-by Arnaldo Carvalho de Melo <[email protected]>
Quoted 6 lines
Quoted 1 line
Reviewed-by Ian Rogers <[email protected]>Acked-by Quentin Monnet <[email protected]>Thanks Arnaldo for checking the bpftool build :)
Thanks! Added to the cset.
- Arnaldo