selftests/bpf: Improve building with extra
Subsystems (4)
cover selftests/bpf: Improve building with extra
When trying to build BPF selftests with additional compiler and linker flags, we're running into multiple problems. This series addresses all of them:
- CFLAGS are not passed to sub-makes of bpftool and libbpf. This is a problem when compiling with PIE as libbpf.a ends up being non-PIE and cannot be linked with other binaries (patch #1).
- bpftool Makefile runs llvm-config --cflags and appends the result to CFLAGS. The result typically contains -D_GNU_SOURCE which may be already set in CFLAGS. That causes a compilation error (patch #2).
- Some GCC flags are not supported by Clang but there are binaries which are always built with Clang but reuse user-defined CFLAGS. When CFLAGS contain such flags, compilation fails (patch #3).
Viktor Malik (3):
selftests/bpf: Allow building with extra flags
bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile selftests/bpf: Allow ignoring some flags for Clang builds
tools/bpf/bpftool/Makefile | 8 ++++++- tools/testing/selftests/bpf/Makefile | 31 +++++++++++++++++++--------- 2 files changed, 28 insertions(+), 11 deletions(-)
Signature · 1 line
2.47.0
PATCH 1/3 selftests/bpf: Allow building with extra flags
In order to specify extra compilation or linking flags to BPF selftests, it is possible to set EXTRA_CFLAGS and EXTRA_LDFLAGS from the command line. The problem is that they are not propagated to sub-make calls (runqslower, bpftool, libbpf) and in the better case are not applied, in the worse case cause the entire build fail.
Propagate EXTRA_CFLAGS and EXTRA_LDFLAGS to the sub-makes.
This, for instance, allows to build selftests as PIE with
$ make EXTRA_CFLAGS='-fPIE' EXTRA_LDFLAGS='-pie'
Without this change, the command would fail because libbpf.a would not be built with -fPIE and other PIE binaries would not link against it.
The only problem is that we have to explicitly provide empty EXTRA_CFLAGS='' and EXTRA_LDFLAGS='' to the builds of kernel modules (bpf_testmod and bpf_test_no_cfi) as we don't want to build modules with flags used for userspace (the above example would fail as kernel doesn't support PIE).
--- tools/testing/selftests/bpf/Makefile | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-)
tools/testing/selftests/bpf/Makefile +14 -8
@@ -294,13 +294,17 @@ $(OUTPUT)/sign-file: ../../../../scripts/sign-file.c $(OUTPUT)/bpf_testmod.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_testmod/Makefile bpf_testmod/*.[ch]) $(call msg,MOD,,$@) $(Q)$(RM) bpf_testmod/bpf_testmod.ko # force re-compilation - $(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_testmod + $(Q)$(MAKE) $(submake_extras) -C bpf_testmod \ + RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) \ + EXTRA_CFLAGS='' EXTRA_LDFLAGS='' $(Q)cp bpf_testmod/bpf_testmod.ko $@ $(OUTPUT)/bpf_test_no_cfi.ko: $(VMLINUX_BTF) $(RESOLVE_BTFIDS) $(wildcard bpf_test_no_cfi/Makefile bpf_test_no_cfi/*.[ch]) $(call msg,MOD,,$@) $(Q)$(RM) bpf_test_no_cfi/bpf_test_no_cfi.ko # force re-compilation - $(Q)$(MAKE) $(submake_extras) RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) -C bpf_test_no_cfi + $(Q)$(MAKE) $(submake_extras) -C bpf_test_no_cfi \ + RESOLVE_BTFIDS=$(RESOLVE_BTFIDS) \ + EXTRA_CFLAGS='' EXTRA_LDFLAGS='' $(Q)cp bpf_test_no_cfi/bpf_test_no_cfi.ko $@ DEFAULT_BPFTOOL := $(HOST_SCRATCH_DIR)/sbin/bpftool
@@ -319,8 +323,8 @@ $(OUTPUT)/runqslower: $(BPFOBJ) | $(DEFAULT_BPFTOOL) $(RUNQSLOWER_OUTPUT) BPFTOOL_OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \ BPFOBJ_OUTPUT=$(BUILD_DIR)/libbpf/ \ BPFOBJ=$(BPFOBJ) BPF_INCLUDE=$(INCLUDE_DIR) \ - EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)' \ - EXTRA_LDFLAGS='$(SAN_LDFLAGS)' && \ + EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' && \ cp $(RUNQSLOWER_OUTPUT)runqslower $@ TEST_GEN_PROGS_EXTENDED += $(TRUNNER_BPFTOOL)
@@ -354,7 +358,8 @@ $(DEFAULT_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) \ $(HOST_BPFOBJ) | $(HOST_BUILD_DIR)/bpftool $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \ ARCH= CROSS_COMPILE= CC="$(HOSTCC)" LD="$(HOSTLD)" \ - EXTRA_CFLAGS='-g $(OPT_FLAGS)' \ + EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)' \ + EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)' \ OUTPUT=$(HOST_BUILD_DIR)/bpftool/ \ LIBBPF_OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \ LIBBPF_DESTDIR=$(HOST_SCRATCH_DIR)/ \
@@ -365,7 +370,8 @@ $(CROSS_BPFTOOL): $(wildcard $(BPFTOOLDIR)/*.[ch] $(BPFTOOLDIR)/Makefile) \ $(BPFOBJ) | $(BUILD_DIR)/bpftool $(Q)$(MAKE) $(submake_extras) -C $(BPFTOOLDIR) \ ARCH=$(ARCH) CROSS_COMPILE=$(CROSS_COMPILE) \ - EXTRA_CFLAGS='-g $(OPT_FLAGS)' \ + EXTRA_CFLAGS='-g $(OPT_FLAGS) $(EXTRA_CFLAGS)' \ + EXTRA_LDFLAGS='$(EXTRA_LDFLAGS)' \ OUTPUT=$(BUILD_DIR)/bpftool/ \ LIBBPF_OUTPUT=$(BUILD_DIR)/libbpf/ \ LIBBPF_DESTDIR=$(SCRATCH_DIR)/ \
@@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ $(APIDIR)/linux/bpf.h \ | $(BUILD_DIR)/libbpf $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \ - EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)' \ - EXTRA_LDFLAGS='$(SAN_LDFLAGS)' \ + EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' \ DESTDIR=$(SCRATCH_DIR) prefix= all install_headers ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
Signature · 1 line
2.47.0
PATCH 3/3 selftests/bpf: Allow ignoring some flags for Clang builds
In reply to Viktor MalikThere exist compiler flags supported by GCC but not supported by Clang (e.g. -specs=...). Currently, these cannot be passed to BPF selftests builds, even when building with GCC, as some binaries (urandom_read and liburandom_read.so) are always built with Clang and the unsupported flags make the compilation fail (as -Werror is turned on).
Add new Makefile variable CLANG_FILTEROUT_FLAGS which can be used by users to specify which flags (from the user-provided CFLAGS or LDFLAGS) should be filtered out for Clang invocations.
This allows to do things like:
$ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \
CLANG_FILTEROUT_FLAGS="-specs=%" \
make -C tools/testing/selftests/bpfWithout this patch, the compilation would fail with:
[...]
clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1
[...]
--- tools/testing/selftests/bpf/Makefile | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
tools/testing/selftests/bpf/Makefile +7 -2
@@ -31,6 +31,11 @@ SAN_LDFLAGS ?= $(SAN_CFLAGS) RELEASE ?= OPT_FLAGS ?= $(if $(RELEASE),-O2,-O0) +# Some flags supported by GCC are not supported by Clang. +# This variable allows users to specify such flags so that they can use custom +# CFLAGS and binaries built with Clang do not fail to build. +CLANG_FILTEROUT_FLAGS ?= + LIBELF_CFLAGS := $(shell $(PKG_CONFIG) libelf --cflags 2>/dev/null) LIBELF_LIBS := $(shell $(PKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)
@@ -271,7 +276,7 @@ endif $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom_read.map $(call msg,LIB,,$@) $(Q)$(CLANG) $(CLANG_TARGET_ARCH) \ - $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \ + $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) \ $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \ -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \ -Wl,--version-script=liburandom_read.map \
@@ -280,7 +285,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_read.so $(call msg,BINARY,,$@) $(Q)$(CLANG) $(CLANG_TARGET_ARCH) \ - $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \ + $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \ -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \ -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \ -Wl,-rpath=. -o $@
Signature · 1 line
2.47.0
PATCH 2/3 bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
In reply to Viktor MalikWhen building selftests with CFLAGS set via env variable, the value of CFLAGS is propagated into bpftool Makefile (called from selftests Makefile). This makes the compilation fail as _GNU_SOURCE is defined two times - once from selftests Makefile (by including lib.mk) and once from bpftool Makefile (by calling llvm-config --cflags):
$ CFLAGS="" make -C tools/testing/selftests/bpf
[...]
CC /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o
<command-line>: error: "_GNU_SOURCE" redefined [-Werror]
<command-line>: note: this is the location of the previous definition
cc1: all warnings being treated as errors
[...]Let bpftool Makefile check if _GNU_SOURCE is already defined and if so, do not let llvm-config add it again.
--- tools/bpf/bpftool/Makefile | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
tools/bpf/bpftool/Makefile +7 -1
@@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1) # If LLVM is available, use it for JIT disassembly CFLAGS += -DHAVE_LLVM_SUPPORT LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + # When bpftool build is called from another Makefile which already sets + # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict. + ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),) + CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) + else + CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + endif LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS)) ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static) LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS))
Signature · 1 line
2.47.0
Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
2024-10-15 08:54 UTC+0200 ~ Viktor Malik <[email protected]>
Quoted 27 lines
When building selftests with CFLAGS set via env variable, the value of CFLAGS is propagated into bpftool
Makefile(called from selftestsMakefile). This makes the compilation fail as_GNU_SOURCEis defined two times - once from selftestsMakefile(by including lib.mk) and once from bpftoolMakefile(by callingllvm-config --cflags):$ CFLAGS="" make -C tools/testing/selftests/bpf [...] CC /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o <command-line>: error: "_GNU_SOURCE" redefined [-Werror] <command-line>: note: this is the location of the previous definition cc1: all warnings being treated as errors [...]Let bpftool
Makefilecheck if_GNU_SOURCEis already defined and if so, do not let llvm-config add it again.Signed-off-by Viktor Malik <[email protected]>--- tools/bpf/bpftool/Makefile | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)tools/bpf/bpftool/Makefile +0
-0
tools/bpf/bpftool/Makefile ifeq ($(feature-llvm),1) · line 151
Quoted 6 lines
tools/bpf/bpftool/Makefile +2
-1@@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1) # If LLVM is available, use it for JIT disassembly CFLAGS += -DHAVE_LLVM_SUPPORT LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + # When bpftool build is called from another Makefile which already sets + # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict.
Thanks! Can you please make your comment more explicit and mention the file tools/testing/selftests/lib.mk and/or the use case addressed (building bpftool from selftests), given that you match on the exact string "-D_GNU_SOURCE="? Your check won't skip adding the duplicate definition if someone passes "-D_GNU_SOURCE", without the "=", by calling the Makefile from another path; that's fine, but I don't want users to read the Makefile and expect it to remove the second definition in such a case.
tools/bpf/bpftool/Makefile ifeq ($(feature-llvm),1) · line 156
Quoted 5 lines
tools/bpf/bpftool/Makefile +5
-0+ ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),) + CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) + else + CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + endif
Looks good otherwise:
Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
Quoted 36 lines · Quentin Monnet
2024-10-15 08:54 UTC+0200 ~ Viktor Malik <[email protected]>
Quoted 27 lines
When building selftests with CFLAGS set via env variable, the value of CFLAGS is propagated into bpftool
Makefile(called from selftestsMakefile). This makes the compilation fail as_GNU_SOURCEis defined two times - once from selftestsMakefile(by including lib.mk) and once from bpftoolMakefile(by callingllvm-config --cflags):$ CFLAGS="" make -C tools/testing/selftests/bpf [...] CC /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o <command-line>: error: "_GNU_SOURCE" redefined [-Werror] <command-line>: note: this is the location of the previous definition cc1: all warnings being treated as errors [...]Let bpftool
Makefilecheck if_GNU_SOURCEis already defined and if so, do not let llvm-config add it again.Signed-off-by Viktor Malik <[email protected]>--- tools/bpf/bpftool/Makefile | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)tools/bpf/bpftool/Makefile +0
-0tools/bpf/bpftool/Makefile ifeq ($(feature-llvm),1) · line 151
Quoted 6 lines
tools/bpf/bpftool/Makefile +2
-1@@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1) # If LLVM is available, use it for JIT disassembly CFLAGS += -DHAVE_LLVM_SUPPORT LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + # When bpftool build is called from another Makefile which already sets + # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict.
Quoted 8 lines
Thanks! Can you please make your comment more explicit and mention the file
tools/testing/selftests/lib.mkand/or the use case addressed (building bpftool from selftests), given that you match on the exact string "-D_GNU_SOURCE="? Your check won't skip adding the duplicate definition if someone passes "-D_GNU_SOURCE", without the "=", by calling theMakefilefrom another path; that's fine, but I don't want users to read theMakefileand expect it to remove the second definition in such a case.
Right, that's a good point, I'll expand the comment.
Thanks!
Quoted 12 lines
tools/bpf/bpftool/Makefile ifeq ($(feature-llvm),1) · line 156
Quoted 7 lines
tools/bpf/bpftool/Makefile +5
-0+ ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),) + CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags)) + else + CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + endifLooks good otherwise:
Reviewed-by Quentin Monnet <[email protected]>
Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
In reply to Viktor MalikQuoted 28 lines · Viktor Malik
When building selftests with CFLAGS set via env variable, the value of CFLAGS is propagated into bpftool
Makefile(called from selftestsMakefile). This makes the compilation fail as_GNU_SOURCEis defined two times - once from selftestsMakefile(by including lib.mk) and once from bpftoolMakefile(by callingllvm-config --cflags):$ CFLAGS="" make -C tools/testing/selftests/bpf [...] CC /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o <command-line>: error: "_GNU_SOURCE" redefined [-Werror] <command-line>: note: this is the location of the previous definition cc1: all warnings being treated as errors [...]Let bpftool
Makefilecheck if_GNU_SOURCEis already defined and if so, do not let llvm-config add it again.Signed-off-by Viktor Malik <[email protected]>--- tools/bpf/bpftool/Makefile | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)tools/bpf/bpftool/Makefile +0
-0
tools/bpf/bpftool/Makefile ifeq ($(feature-llvm),1) · line 153
Quoted 8 lines
tools/bpf/bpftool/Makefile +4
-1@@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1) # If LLVM is available, use it for JIT disassembly CFLAGS += -DHAVE_LLVM_SUPPORT LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + # When bpftool build is called from another Makefile which already sets + # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict. + ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),) + CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))
why not always do filter-out and avoid this ugly ifneq?
Quoted 9 lines
tools/bpf/bpftool/Makefile +3
-1+ else + CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + endif LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS)) ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static) LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) --2.47.0
Re: [PATCH bpf-next 3/3] selftests/bpf: Allow ignoring some flags for Clang builds
In reply to Viktor MalikQuoted 13 lines · Viktor Malik
There exist compiler flags supported by GCC but not supported by Clang (e.g. -specs=...). Currently, these cannot be passed to BPF selftests builds, even when building with GCC, as some binaries (
urandom_readand liburandom_read.so) are always built with Clang and the unsupported flags make the compilation fail (as -Werror is turned on).Add new
MakefilevariableCLANG_FILTEROUT_FLAGSwhich can be used by users to specify which flags (from the user-provided CFLAGS or LDFLAGS) should be filtered out for Clang invocations.This allows to do things like:
Quoted 8 lines
$ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \ CLANG_FILTEROUT_FLAGS="-specs=%" \ make -C tools/testing/selftests/bpfWithout this patch, the compilation would fail with:
[...]
clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
maybe we should just not error out (i.e., enable
-Wno-unused-command-line-argument)?
Quoted 45 lines
make: *** [Makefile:273: /bpf-next/tools/testing/selftests/bpf/liburandom_read.so] Error 1 [...]
Signed-off-by Viktor Malik <[email protected]>--- tools/testing/selftests/bpf/Makefile | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)tools/testing/selftests/bpf/Makefile +7
-3@@ -31,6 +31,11 @@ SAN_LDFLAGS ?= $(SAN_CFLAGS) RELEASE ?= OPT_FLAGS ?= $(if $(RELEASE),-O2,-O0) +# Some flags supported by GCC are not supported by Clang. +# This variable allows users to specify such flags so that they can use custom +# CFLAGS and binaries built with Clang do not fail to build. +CLANG_FILTEROUT_FLAGS ?= + LIBELF_CFLAGS := $(shell $(PKG_CONFIG) libelf --cflags 2>/dev/null) LIBELF_LIBS := $(shell $(PKG_CONFIG) libelf --libs 2>/dev/null || echo -lelf)@@ -271,7 +276,7 @@ endif $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom_read.map $(call msg,LIB,,$@) $(Q)$(CLANG) $(CLANG_TARGET_ARCH) \ - $(filter-out -static,$(CFLAGS) $(LDFLAGS)) \ + $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) \ $(filter %.c,$^) $(filter-out -static,$(LDLIBS)) \ -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \ -Wl,--version-script=liburandom_read.map \@@ -280,7 +285,7 @@ $(OUTPUT)/liburandom_read.so: urandom_read_lib1.c urandom_read_lib2.c liburandom $(OUTPUT)/urandom_read: urandom_read.c urandom_read_aux.c $(OUTPUT)/liburandom_read.so $(call msg,BINARY,,$@) $(Q)$(CLANG) $(CLANG_TARGET_ARCH) \ - $(filter-out -static,$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \ + $(filter-out -static $(CLANG_FILTEROUT_FLAGS),$(CFLAGS) $(LDFLAGS)) $(filter %.c,$^) \ -lurandom_read $(filter-out -static,$(LDLIBS)) -L$(OUTPUT) \ -fuse-ld=$(LLD) -Wl,-znoseparate-code -Wl,--build-id=sha1 \ -Wl,-rpath=. -o $@ --2.47.0
Eduard Zingerman
maintainerRe: [PATCH bpf-next 3/3] selftests/bpf: Allow ignoring some flags for Clang builds
Quoted 18 lines · Andrii Nakryiko
Quoted 17 lines · Viktor Malik
There exist compiler flags supported by GCC but not supported by Clang (e.g. -specs=...). Currently, these cannot be passed to BPF selftests builds, even when building with GCC, as some binaries (
urandom_readand liburandom_read.so) are always built with Clang and the unsupported flags make the compilation fail (as -Werror is turned on).Add new
MakefilevariableCLANG_FILTEROUT_FLAGSwhich can be used by users to specify which flags (from the user-provided CFLAGS or LDFLAGS) should be filtered out for Clang invocations.This allows to do things like:
$ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \ CLANG_FILTEROUT_FLAGS="-specs=%" \ make -C tools/testing/selftests/bpf
Quoted 7 lines
Quoted 4 lines
Without this patch, the compilation would fail with:
[...]
clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]maybe we should just not error out (i.e., enable
-Wno-unused-command-line-argument)?
I agree with Andrii, grepping for FILTEROUT in kernel source code does not show anything similar to this. Are such filter-out variables some kind of convention?
Another option might be to remove -Werror and add it on CI via EXTRA_CFLAGS.
[...]
Re: [PATCH bpf-next 1/3] selftests/bpf: Allow building with extra flags
In reply to Viktor MalikOn Tue, 2024-10-15 at 08:54 +0200, Viktor Malik wrote:
Tried this, seems to work as expected, but see below.
[...]
tools/testing/selftests/bpf/Makefile $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ · line 401
Quoted 11 lines
tools/testing/selftests/bpf/Makefile +2
-2@@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ $(APIDIR)/linux/bpf.h \ | $(BUILD_DIR)/libbpf $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \ - EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)' \ - EXTRA_LDFLAGS='$(SAN_LDFLAGS)' \ + EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' \ DESTDIR=$(SCRATCH_DIR) prefix= all install_headers ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
Note, that there is also the following code just below this hunk:
ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
$(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \
$(APIDIR)/linux/bpf.h \
| $(HOST_BUILD_DIR)/libbpf
$(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \
EXTRA_CFLAGS='-g $(OPT_FLAGS)' ARCH= CROSS_COMPILE= \
^^^^^^^^^^^^
needs an update?
OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \
CC="$(HOSTCC)" LD="$(HOSTLD)" \
DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headersendif
Re: [PATCH bpf-next 2/3] bpftool: Prevent setting duplicate _GNU_SOURCE in Makefile
In reply to Andrii NakryikoQuoted 24 lines · Andrii Nakryiko
Quoted 23 lines · Viktor Malik
When building selftests with CFLAGS set via env variable, the value of CFLAGS is propagated into bpftool
Makefile(called from selftestsMakefile). This makes the compilation fail as_GNU_SOURCEis defined two times - once from selftestsMakefile(by including lib.mk) and once from bpftoolMakefile(by callingllvm-config --cflags):$ CFLAGS="" make -C tools/testing/selftests/bpf [...] CC /bpf-next/tools/testing/selftests/bpf/tools/build/bpftool/btf.o <command-line>: error: "_GNU_SOURCE" redefined [-Werror] <command-line>: note: this is the location of the previous definition cc1: all warnings being treated as errors [...]Let bpftool
Makefilecheck if_GNU_SOURCEis already defined and if so, do not let llvm-config add it again.Signed-off-by Viktor Malik <[email protected]>--- tools/bpf/bpftool/Makefile | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
Quoted 15 lines
tools/bpf/bpftool/Makefile ifeq ($(feature-llvm),1) · line 153
Quoted 13 lines
tools/bpf/bpftool/Makefile +4
-1@@ -147,7 +147,13 @@ ifeq ($(feature-llvm),1) # If LLVM is available, use it for JIT disassembly CFLAGS += -DHAVE_LLVM_SUPPORT LLVM_CONFIG_LIB_COMPONENTS := mcdisassembler all-targets - CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + # When bpftool build is called from another Makefile which already sets + # -D_GNU_SOURCE, do not let llvm-config add it again as it will cause conflict. + ifneq ($(filter -D_GNU_SOURCE=,$(CFLAGS)),) + CFLAGS += $(filter-out -D_GNU_SOURCE,$(shell $(LLVM_CONFIG) --cflags))why not always do filter-out and avoid this ugly ifneq?
Because in that case, _GNU_SOURCE would not be defined for some builds (e.g. plain bpftool build). I'm not entirely sure what the implications are so I wanted to stay on the safe side. Anyways, I gave it a try and bpftool builds without _GNU_SOURCE just fine so I think that we can drop the ifneq.
Quoted 11 lines
Quoted 10 lines
tools/bpf/bpftool/Makefile +3
-1+ else + CFLAGS += $(shell $(LLVM_CONFIG) --cflags) + endif LIBS += $(shell $(LLVM_CONFIG) --libs $(LLVM_CONFIG_LIB_COMPONENTS)) ifeq ($(shell $(LLVM_CONFIG) --shared-mode),static) LIBS += $(shell $(LLVM_CONFIG) --system-libs $(LLVM_CONFIG_LIB_COMPONENTS)) --2.47.0
Re: [PATCH bpf-next 1/3] selftests/bpf: Allow building with extra flags
In reply to Eduard ZingermanQuoted 22 lines · Eduard Zingerman
On Tue, 2024-10-15 at 08:54 +0200, Viktor Malik wrote:
Tried this, seems to work as expected, but see below.
Tested-by Eduard Zingerman <[email protected]>[...]
tools/testing/selftests/bpf/Makefile $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ · line 401
Quoted 11 lines
tools/testing/selftests/bpf/Makefile +2
-2@@ -388,8 +394,8 @@ $(BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ $(APIDIR)/linux/bpf.h \ | $(BUILD_DIR)/libbpf $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) OUTPUT=$(BUILD_DIR)/libbpf/ \ - EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS)' \ - EXTRA_LDFLAGS='$(SAN_LDFLAGS)' \ + EXTRA_CFLAGS='-g $(OPT_FLAGS) $(SAN_CFLAGS) $(EXTRA_CFLAGS)' \ + EXTRA_LDFLAGS='$(SAN_LDFLAGS) $(EXTRA_LDFLAGS)' \ DESTDIR=$(SCRATCH_DIR) prefix= all install_headers ifneq ($(BPFOBJ),$(HOST_BPFOBJ))Note, that there is also the following code just below this hunk:
Quoted 8 lines
ifneq ($(BPFOBJ),$(HOST_BPFOBJ))
$(HOST_BPFOBJ): $(wildcard $(BPFDIR)/*.[ch] $(BPFDIR)/Makefile) \ $(APIDIR)/linux/bpf.h \ | $(HOST_BUILD_DIR)/libbpf $(Q)$(MAKE) $(submake_extras) -C $(BPFDIR) \ EXTRA_CFLAGS='-g $(OPT_FLAGS)' ARCH= CROSS_COMPILE= \ ^^^^^^^^^^^^ needs an update?
Yes, good catch, I'll add it.
Thanks!
Quoted 7 lines
OUTPUT=$(HOST_BUILD_DIR)/libbpf/ \ CC="$(HOSTCC)" LD="$(HOSTLD)" \ DESTDIR=$(HOST_SCRATCH_DIR)/ prefix= all install_headersendif
Re: [PATCH bpf-next 3/3] selftests/bpf: Allow ignoring some flags for Clang builds
In reply to Eduard ZingermanQuoted 24 lines · Eduard Zingerman
Quoted 23 lines · Andrii Nakryiko
Quoted 21 lines · Viktor Malik
There exist compiler flags supported by GCC but not supported by Clang (e.g. -specs=...). Currently, these cannot be passed to BPF selftests builds, even when building with GCC, as some binaries (
urandom_readand liburandom_read.so) are always built with Clang and the unsupported flags make the compilation fail (as -Werror is turned on).Add new
MakefilevariableCLANG_FILTEROUT_FLAGSwhich can be used by users to specify which flags (from the user-provided CFLAGS or LDFLAGS) should be filtered out for Clang invocations.This allows to do things like:
$ CFLAGS="-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1" \ CLANG_FILTEROUT_FLAGS="-specs=%" \ make -C tools/testing/selftests/bpfWithout this patch, the compilation would fail with:
[...]
clang: error: argument unused during compilation: '-specs=/usr/lib/rpm/redhat/redhat-hardened-cc1' [-Werror,-Wunused-command-line-argument]
Quoted 8 lines
Quoted 2 lines
maybe we should just not error out (i.e., enable
-Wno-unused-command-line-argument)?I agree with Andrii, grepping for FILTEROUT in kernel source code does not show anything similar to this. Are such filter-out variables some kind of convention?
Another option might be to remove
-Werrorand add it on CI viaEXTRA_CFLAGS.
Enabling -Wno-unused-command-line-argument is the simplest way here, let's do that.
Thank you both for suggestions!
Quoted 3 lines
[...]