Skip to contentSkip to conversation

bpf: tools: support build selftests/bpf with clang

PATCH In review Yonghong Song 5 participants 18 messages Apr 13, 2021 – Apr 16, 2021

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v3 cover bpf: tools: support build selftests/bpf with clang

To build kernel with clang, people typically use
make -j60 LLVM=1 LLVM_IAS=1
LLVM_IAS=1 is not required for non-LTO build but
is required for LTO build. In my environment,
I am always having LLVM_IAS=1 regardless of
whether LTO is enabled or not.

After kernel is build with clang, the following command can be used to build selftests with clang:
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1

I am using latest bpf-next kernel code base and
latest clang built from source from
https://github.com/llvm/llvm-project.git
Using earlier version of llvm may have compilation errors, see
tools/testing/selftests/bpf
due to continuous development in llvm bpf features and selftests to use these features.

To run bpf selftest properly, you need have certain necessary kernel configs like at:
bpf-next:tools/testing/selftests/bpf/config
(not that this is not a complete .config file and some other configs
might still be needed.)

Currently, using the above command, some compilations still use gcc and there are also compilation errors and warnings. This patch set intends to fix these issues.
Patch #1 and #2 fixed the issue so clang/clang++ is used instead of gcc/g++. Patch #3 fixed a compilation failure. Patch #4 and #5 fixed various compiler warnings.

Changelog:
v2 -> v3:
. more test environment description in cover letter. (Sedat) . use a different fix, but similar to other use in selftests/bpf
Makefile, to exclude header files from CXX compilation command line. (Andrii)
. fix codes instead of adding -Wno-format-security. (Andrii)
v1 -> v2:
. add -Wno-unused-command-line-argument and -Wno-format-security
for clang only as (1). gcc does not exhibit those warnings, and (2). -Wno-unused-command-line-argument is only supported by clang. (Sedat)

Yonghong Song (5):
selftests: set CC to clang in lib.mk if LLVM is set
tools: allow proper CC/CXX/... override with LLVM=1 in
Makefile.include
selftests/bpf: fix test_cpp compilation failure with clang selftests/bpf: silence clang compilation warnings
bpftool: fix a clang compilation warning

 tools/bpf/bpftool/net.c                              |  2 +-
 tools/scripts/Makefile.include                       | 12 ++++++++++--
 tools/testing/selftests/bpf/Makefile                 |  7 ++++++-
 tools/testing/selftests/bpf/prog_tests/fexit_sleep.c |  4 ++--
 .../selftests/bpf/prog_tests/ns_current_pid_tgid.c   |  4 ++--
 tools/testing/selftests/lib.mk                       |  4 ++++
 6 files changed, 25 insertions(+), 8 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH v3 1/5 selftests: set CC to clang in lib.mk if LLVM is set

selftests/bpf/Makefile includes lib.mk. With the following command
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 V=1
some files are still compiled with gcc. This patch fixed lib.mk issue which sets CC to gcc in all cases.

---
 tools/testing/selftests/lib.mk | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/lib.mk +4 -0
index a5ce26d548e4..9a41d8bb9ff1 100644
@@ -1,6 +1,10 @@
 # This mimics the top-level Makefile. We do it explicitly here so that this
 # Makefile can operate with or without the kbuild infrastructure.
+ifneq ($(LLVM),)
+CC := clang
+else
 CC := $(CROSS_COMPILE)gcc
+endif
 
 ifeq (0,$(MAKELEVEL))
     ifeq ($(OUTPUT),)
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH bpf-next v3 1/5] selftests: set CC to clang in lib.mk if LLVM is set

Quoted 18 lines · Yonghong Song

selftests/bpf/Makefile includes lib.mk. With the following command
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 V=1
some files are still compiled with gcc. This patch fixed lib.mk issue which sets CC to gcc in all cases.

---
 tools/testing/selftests/lib.mk | 4 ++++
 1 file changed, 4 insertions(+)
tools/testing/selftests/lib.mk +0 -0
index a5ce26d548e4..9a41d8bb9ff1 100644

tools/testing/selftests/lib.mk line 4

Quoted 4 lines
tools/testing/selftests/lib.mk +2 -0
@@ -1,6 +1,10 @@
 # This mimics the top-level Makefile. We do it explicitly here so that this
 # Makefile can operate with or without the kbuild infrastructure.
+ifneq ($(LLVM),)
+CC := clang

Does this mean that cross-compilation with Clang doesn't work at all or is achieved in some other way?

Quoted 9 lines
tools/testing/selftests/lib.mk +2 -1
+else
 CC := $(CROSS_COMPILE)gcc
+endif
 
 ifeq (0,$(MAKELEVEL))
     ifeq ($(OUTPUT),)
--

2.30.2

#3 lore raw

Re: [PATCH bpf-next v3 1/5] selftests: set CC to clang in lib.mk if LLVM is set

Quoted 15 lines · Andrii Nakryiko
Quoted 13 lines · Yonghong Song

selftests/bpf/Makefile includes lib.mk. With the following command
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 V=1
some files are still compiled with gcc. This patch fixed lib.mk issue which sets CC to gcc in all cases.

---
 tools/testing/selftests/lib.mk | 4 ++++
 1 file changed, 4 insertions(+)
Quoted 12 lines

tools/testing/selftests/lib.mk line 4

Quoted 9 lines
tools/testing/selftests/lib.mk +2 -0
index a5ce26d548e4..9a41d8bb9ff1 100644
@@ -1,6 +1,10 @@
 # This mimics the top-level Makefile. We do it explicitly here so that this
 # Makefile can operate with or without the kbuild infrastructure.
+ifneq ($(LLVM),)
+CC := clang

Does this mean that cross-compilation with Clang doesn't work at all or is achieved in some other way?

Right, this probably doesn't support cross compilation w/ Clang. Rather than invoke $(CROSS_COMPILE) clang, you'd do clang --target=$(CROSS_COMPILE). Even then, cross linking executables is hairy. But at least this should enable native compilation, which is a start.

Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/lib.mk +2 -1
+else
 CC := $(CROSS_COMPILE)gcc
+endif
 
 ifeq (0,$(MAKELEVEL))
     ifeq ($(OUTPUT),)
--

2.30.2

Signature · 2 lines
Thanks,
~Nick Desaulniers
#4 lore raw

Re: [PATCH bpf-next v3 1/5] selftests: set CC to clang in lib.mk if LLVM is set

Quoted 30 lines · Nick Desaulniers
Quoted 15 lines · Andrii Nakryiko
Quoted 13 lines · Yonghong Song

selftests/bpf/Makefile includes lib.mk. With the following command
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 V=1
some files are still compiled with gcc. This patch fixed lib.mk issue which sets CC to gcc in all cases.

---
  tools/testing/selftests/lib.mk | 4 ++++
  1 file changed, 4 insertions(+)
Quoted 12 lines

tools/testing/selftests/lib.mk line 4

Quoted 9 lines
tools/testing/selftests/lib.mk +2 -0
index a5ce26d548e4..9a41d8bb9ff1 100644
@@ -1,6 +1,10 @@
  # This mimics the top-level Makefile. We do it explicitly here so that this
  # Makefile can operate with or without the kbuild infrastructure.
+ifneq ($(LLVM),)
+CC := clang

Does this mean that cross-compilation with Clang doesn't work at all or is achieved in some other way?

Quoted 5 lines

Right, this probably doesn't support cross compilation w/ Clang. Rather than invoke $(CROSS_COMPILE) clang, you'd do clang --target=$(CROSS_COMPILE). Even then, cross linking executables is hairy. But at least this should enable native compilation, which is a start.

See https://clang.llvm.org/docs/CrossCompilation.html.
As Nick said, clang prefers --target=$(CROSS_COMPILE) to indicate cross compilation. User can pass additional flags (CFLAGS) for cross compilation for the time being. This is the same as main kernel Makefile.

ifneq ($(LLVM),)

CC              = clang
LD              = ld.lld
AR              = llvm-ar
NM              = llvm-nm
OBJCOPY         = llvm-objcopy
OBJDUMP         = llvm-objdump
READELF         = llvm-readelf
STRIP           = llvm-strip
else
CC              = $(CROSS_COMPILE)gcc
LD              = $(CROSS_COMPILE)ld
AR              = $(CROSS_COMPILE)ar
NM              = $(CROSS_COMPILE)nm
OBJCOPY         = $(CROSS_COMPILE)objcopy
OBJDUMP         = $(CROSS_COMPILE)objdump
READELF         = $(CROSS_COMPILE)readelf
STRIP           = $(CROSS_COMPILE)strip

endif

Quoted 14 lines
Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/lib.mk +2 -1
+else
  CC := $(CROSS_COMPILE)gcc
+endif
 
  ifeq (0,$(MAKELEVEL))
      ifeq ($(OUTPUT),)
--

2.30.2

#5 lore raw

Re: [PATCH bpf-next v3 1/5] selftests: set CC to clang in lib.mk if LLVM is set

Quoted 64 lines · Yonghong Song
Quoted 30 lines · Nick Desaulniers
Quoted 15 lines · Andrii Nakryiko
Quoted 13 lines · Yonghong Song

selftests/bpf/Makefile includes lib.mk. With the following command
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 V=1
some files are still compiled with gcc. This patch fixed lib.mk issue which sets CC to gcc in all cases.

---
  tools/testing/selftests/lib.mk | 4 ++++
  1 file changed, 4 insertions(+)
Quoted 12 lines

tools/testing/selftests/lib.mk line 4

Quoted 9 lines
tools/testing/selftests/lib.mk +2 -0
index a5ce26d548e4..9a41d8bb9ff1 100644
@@ -1,6 +1,10 @@
  # This mimics the top-level Makefile. We do it explicitly here so that this
  # Makefile can operate with or without the kbuild infrastructure.
+ifneq ($(LLVM),)
+CC := clang

Does this mean that cross-compilation with Clang doesn't work at all or is achieved in some other way?

Quoted 5 lines

Right, this probably doesn't support cross compilation w/ Clang. Rather than invoke $(CROSS_COMPILE) clang, you'd do clang --target=$(CROSS_COMPILE). Even then, cross linking executables is hairy. But at least this should enable native compilation, which is a start.

See https://clang.llvm.org/docs/CrossCompilation.html.
As Nick said, clang prefers --target=$(CROSS_COMPILE) to indicate cross compilation. User can pass additional flags (CFLAGS) for cross compilation for the time being. This is the same as main kernel Makefile.

ifneq ($(LLVM),)

CC              = clang
LD              = ld.lld
AR              = llvm-ar
NM              = llvm-nm
OBJCOPY         = llvm-objcopy
OBJDUMP         = llvm-objdump
READELF         = llvm-readelf
STRIP           = llvm-strip
else
CC              = $(CROSS_COMPILE)gcc
LD              = $(CROSS_COMPILE)ld
AR              = $(CROSS_COMPILE)ar
NM              = $(CROSS_COMPILE)nm
OBJCOPY         = $(CROSS_COMPILE)objcopy
OBJDUMP         = $(CROSS_COMPILE)objdump
READELF         = $(CROSS_COMPILE)readelf
STRIP           = $(CROSS_COMPILE)strip
Quoted 1 line

endif

Right, then later in the top level Makefile to achieve cross compilation support
569 ifneq ($(CROSS_COMPILE),)
570 CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%))
...

The top level Makefile has this all wired up correctly; I wonder if tools/ could make use of the existing exports, rather than redoing all of that work?

Quoted 15 lines
Quoted 15 lines
Quoted 12 lines
Quoted 12 lines
tools/testing/selftests/lib.mk +2 -1
+else
  CC := $(CROSS_COMPILE)gcc
+endif
 
  ifeq (0,$(MAKELEVEL))
      ifeq ($(OUTPUT),)
--

2.30.2

Signature · 2 lines
Thanks,
~Nick Desaulniers
#6 lore raw

Re: [PATCH bpf-next v3 1/5] selftests: set CC to clang in lib.mk if LLVM is set

In reply to Yonghong Song
Quoted 64 lines · Yonghong Song
Quoted 30 lines · Nick Desaulniers
Quoted 15 lines · Andrii Nakryiko
Quoted 13 lines · Yonghong Song

selftests/bpf/Makefile includes lib.mk. With the following command
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 V=1
some files are still compiled with gcc. This patch fixed lib.mk issue which sets CC to gcc in all cases.

---
  tools/testing/selftests/lib.mk | 4 ++++
  1 file changed, 4 insertions(+)
Quoted 12 lines

tools/testing/selftests/lib.mk line 4

Quoted 9 lines
tools/testing/selftests/lib.mk +2 -0
index a5ce26d548e4..9a41d8bb9ff1 100644
@@ -1,6 +1,10 @@
  # This mimics the top-level Makefile. We do it explicitly here so that this
  # Makefile can operate with or without the kbuild infrastructure.
+ifneq ($(LLVM),)
+CC := clang

Does this mean that cross-compilation with Clang doesn't work at all or is achieved in some other way?

Quoted 5 lines

Right, this probably doesn't support cross compilation w/ Clang. Rather than invoke $(CROSS_COMPILE) clang, you'd do clang --target=$(CROSS_COMPILE). Even then, cross linking executables is hairy. But at least this should enable native compilation, which is a start.

See https://clang.llvm.org/docs/CrossCompilation.html.
As Nick said, clang prefers --target=$(CROSS_COMPILE) to indicate cross compilation. User can pass additional flags (CFLAGS) for cross compilation for the time being. This is the same as main kernel Makefile.

ifneq ($(LLVM),)

CC              = clang
LD              = ld.lld
AR              = llvm-ar
NM              = llvm-nm
OBJCOPY         = llvm-objcopy
OBJDUMP         = llvm-objdump
READELF         = llvm-readelf
STRIP           = llvm-strip
else
CC              = $(CROSS_COMPILE)gcc
LD              = $(CROSS_COMPILE)ld
AR              = $(CROSS_COMPILE)ar
NM              = $(CROSS_COMPILE)nm
OBJCOPY         = $(CROSS_COMPILE)objcopy
OBJDUMP         = $(CROSS_COMPILE)objdump
READELF         = $(CROSS_COMPILE)readelf
STRIP           = $(CROSS_COMPILE)strip
Quoted 2 lines

endif

Ok, sounds good. We can improve cross-compilation separately.

Quoted 14 lines
Quoted 14 lines
Quoted 11 lines
Quoted 11 lines
tools/testing/selftests/lib.mk +2 -1
+else
  CC := $(CROSS_COMPILE)gcc
+endif
 
  ifeq (0,$(MAKELEVEL))
      ifeq ($(OUTPUT),)
--

2.30.2

#7 lore raw

PATCH v3 2/5 tools: allow proper CC/CXX/... override with LLVM=1 in Makefile.include

In reply to Yonghong Song

selftests/bpf/Makefile includes tools/scripts/Makefile.include. With the following command
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1 V=1
some files are still compiled with gcc. This patch fixed the case if CC/AR/LD/CXX/STRIP is allowed to be overridden, it will be written to clang/llvm-ar/..., instead of gcc binaries. The definition of CC_NO_CLANG is also relocated to the place after the above CC is defined.

---
 tools/scripts/Makefile.include | 12 ++++++++++--
 1 file changed, 10 insertions(+), 2 deletions(-)
tools/scripts/Makefile.include +10 -2
index a402f32a145c..91130648d8e6 100644
@@ -39,8 +39,6 @@ EXTRA_WARNINGS += -Wundef
 EXTRA_WARNINGS += -Wwrite-strings
 EXTRA_WARNINGS += -Wformat
 
-CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?)
-
 # Makefiles suck: This macro sets a default value of $(2) for the
 # variable named by $(1), unless the variable has been set by
 # environment or command line. This is necessary for CC and AR
@@ -52,12 +50,22 @@ define allow-override
     $(eval $(1) = $(2)))
 endef
 
+ifneq ($(LLVM),)
+$(call allow-override,CC,clang)
+$(call allow-override,AR,llvm-ar)
+$(call allow-override,LD,ld.lld)
+$(call allow-override,CXX,clang++)
+$(call allow-override,STRIP,llvm-strip)
+else
 # Allow setting various cross-compile vars or setting CROSS_COMPILE as a prefix.
 $(call allow-override,CC,$(CROSS_COMPILE)gcc)
 $(call allow-override,AR,$(CROSS_COMPILE)ar)
 $(call allow-override,LD,$(CROSS_COMPILE)ld)
 $(call allow-override,CXX,$(CROSS_COMPILE)g++)
 $(call allow-override,STRIP,$(CROSS_COMPILE)strip)
+endif
+
+CC_NO_CLANG := $(shell $(CC) -dM -E -x c /dev/null | grep -Fq "__clang__"; echo $$?)
 
 ifneq ($(LLVM),)
 HOSTAR  ?= llvm-ar
Signature · 1 line
2.30.2
#8 lore raw

PATCH v3 3/5 selftests/bpf: fix test_cpp compilation failure with clang

In reply to Yonghong Song

With clang compiler:
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
the test_cpp build failed due to the failure:
warning: treating 'c-header' input as 'c++-header' when in C++ mode, this behavior is deprecated [-Wdeprecated]
clang-13: error: cannot specify -o when generating multiple output files

test_cpp compilation flag looks like:
clang++ -g -Og -rdynamic -Wall -I<...> ... \
-Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program \
test_cpp.cpp <...>/test_core_extern.skel.h <...>/libbpf.a <...>/test_stub.o \
-lcap -lelf -lz -lrt -lpthread -o <...>/test_cpp

The clang++ compiler complains the header file in the command line and also failed the compilation due to this.
Let us remove the header file from the command line which is not intended any way, and this fixed the compilation problem.

---
 tools/testing/selftests/bpf/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/Makefile +1 -1
index 6448c626498f..dcc2dc1f2a86 100644
@@ -481,7 +481,7 @@ $(OUTPUT)/test_verifier: test_verifier.c verifier/tests.h $(BPFOBJ) | $(OUTPUT)
 # Make sure we are able to include and link libbpf against c++.
 $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
 	$(call msg,CXX,,$@)
-	$(Q)$(CXX) $(CFLAGS) $^ $(LDLIBS) -o $@
+	$(Q)$(CXX) $(CFLAGS) $(filter %.a %.o %.cpp,$^) $(LDLIBS) -o $@
 
 # Benchmark runner
 $(OUTPUT)/bench_%.o: benchs/bench_%.c bench.h
Signature · 1 line
2.30.2
Acked-by on v3 3/5
#9 lore raw

Re: [PATCH bpf-next v3 3/5] selftests/bpf: fix test_cpp compilation failure with clang

Quoted 14 lines · Yonghong Song

With clang compiler:
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
the test_cpp build failed due to the failure:
warning: treating 'c-header' input as 'c++-header' when in C++ mode, this behavior is deprecated [-Wdeprecated]
clang-13: error: cannot specify -o when generating multiple output files

test_cpp compilation flag looks like:
clang++ -g -Og -rdynamic -Wall -I<...> ... \
-Dbpf_prog_load=bpf_prog_test_load -Dbpf_load_program=bpf_test_load_program \
test_cpp.cpp <...>/test_core_extern.skel.h <...>/libbpf.a <...>/test_stub.o \
-lcap -lelf -lz -lrt -lpthread -o <...>/test_cpp

Quoted 7 lines

The clang++ compiler complains the header file in the command line and also failed the compilation due to this.
Let us remove the header file from the command line which is not intended any way, and this fixed the compilation problem.

---

Quoted 19 lines
 tools/testing/selftests/bpf/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/selftests/bpf/Makefile +1 -2
index 6448c626498f..dcc2dc1f2a86 100644
@@ -481,7 +481,7 @@ $(OUTPUT)/test_verifier: test_verifier.c verifier/tests.h $(BPFOBJ) | $(OUTPUT)
 # Make sure we are able to include and link libbpf against c++.
 $(OUTPUT)/test_cpp: test_cpp.cpp $(OUTPUT)/test_core_extern.skel.h $(BPFOBJ)
        $(call msg,CXX,,$@)
-       $(Q)$(CXX) $(CFLAGS) $^ $(LDLIBS) -o $@
+       $(Q)$(CXX) $(CFLAGS) $(filter %.a %.o %.cpp,$^) $(LDLIBS) -o $@
 
 # Benchmark runner
 $(OUTPUT)/bench_%.o: benchs/bench_%.c bench.h
--

2.30.2

#10 lore raw

PATCH v3 4/5 selftests/bpf: silence clang compilation warnings

In reply to Yonghong Song

With clang compiler:
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
Some linker flags are not used/effective for some binaries and we have warnings like:
warning: -lelf: 'linker' input unused [-Wunused-command-line-argument]

We also have warnings like:
.../selftests/bpf/prog_tests/ns_current_pid_tgid.c:74:57: note: treat the string as an argument to avoid this

        if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", strerror(errno)))
                                                               ^
                                                               "%s",
  .../selftests/bpf/test_progs.h:129:35: note: expanded from macro 'CHECK'
        _CHECK(condition, tag, duration, format)
                                         ^
  .../selftests/bpf/test_progs.h:108:21: note: expanded from macro '_CHECK'
                fprintf(stdout, ##format);                              \
                                  ^

The first warning can be silenced with clang option -Wno-unused-command-line-argument. For the second warning, source codes are modified as suggested by the compiler to silence the warning. Since gcc does not support the option -Wno-unused-command-line-argument and the warning only happens with clang compiler, the option -Wno-unused-command-line-argument is enabled only when clang compiler is used.

---
 tools/testing/selftests/bpf/Makefile                         | 5 +++++
 tools/testing/selftests/bpf/prog_tests/fexit_sleep.c         | 4 ++--
 tools/testing/selftests/bpf/prog_tests/ns_current_pid_tgid.c | 4 ++--
 3 files changed, 9 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/Makefile +5 -0
index dcc2dc1f2a86..c45ae13b88a0 100644
@@ -28,6 +28,11 @@ CFLAGS += -g -Og -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS)		\
 	  -Dbpf_load_program=bpf_test_load_program
 LDLIBS += -lcap -lelf -lz -lrt -lpthread
 
+# Silence some warnings when compiled with clang
+ifneq ($(LLVM),)
+CFLAGS += -Wno-unused-command-line-argument
+endif
+
 # Order correspond to 'make run_tests' order
 TEST_GEN_PROGS = test_verifier test_tag test_maps test_lru_map test_lpm_map test_progs \
 	test_verifier_log test_dev_cgroup \
tools/testing/selftests/bpf/prog_tests/fexit_sleep.c +2 -2
index 6c4d42a2386f..ccc7e8a34ab6 100644
@@ -39,7 +39,7 @@ void test_fexit_sleep(void)
 		goto cleanup;
 
 	cpid = clone(do_sleep, child_stack + STACK_SIZE, CLONE_FILES | SIGCHLD, fexit_skel);
-	if (CHECK(cpid == -1, "clone", strerror(errno)))
+	if (CHECK(cpid == -1, "clone", "%s\n", strerror(errno)))
 		goto cleanup;
 
 	/* wait until first sys_nanosleep ends and second sys_nanosleep starts */
@@ -65,7 +65,7 @@ void test_fexit_sleep(void)
 	/* kill the thread to unwind sys_nanosleep stack through the trampoline */
 	kill(cpid, 9);
 
-	if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", strerror(errno)))
+	if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", "%s\n", strerror(errno)))
 		goto cleanup;
 	if (CHECK(WEXITSTATUS(wstatus) != 0, "exitstatus", "failed"))
 		goto cleanup;
tools/testing/selftests/bpf/prog_tests/ns_current_pid_tgid.c +2 -2
index 31a3114906e2..2535788e135f 100644
@@ -68,10 +68,10 @@ static void test_ns_current_pid_tgid_new_ns(void)
 	cpid = clone(test_current_pid_tgid, child_stack + STACK_SIZE,
 		     CLONE_NEWPID | SIGCHLD, NULL);
 
-	if (CHECK(cpid == -1, "clone", strerror(errno)))
+	if (CHECK(cpid == -1, "clone", "%s\n", strerror(errno)))
 		return;
 
-	if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", strerror(errno)))
+	if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", "%s\n", strerror(errno)))
 		return;
 
 	if (CHECK(WEXITSTATUS(wstatus) != 0, "newns_pidtgid", "failed"))
Signature · 1 line
2.30.2
Acked-by on v3 4/5
#11 lore raw

Re: [PATCH bpf-next v3 4/5] selftests/bpf: silence clang compilation warnings

Quoted 26 lines · Yonghong Song

With clang compiler:
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
Some linker flags are not used/effective for some binaries and we have warnings like:
warning: -lelf: 'linker' input unused [-Wunused-command-line-argument]

We also have warnings like:
.../selftests/bpf/prog_tests/ns_current_pid_tgid.c:74:57: note: treat the string as an argument to avoid this

        if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", strerror(errno)))
                                                               ^
                                                               "%s",
  .../selftests/bpf/test_progs.h:129:35: note: expanded from macro 'CHECK'
        _CHECK(condition, tag, duration, format)
                                         ^
  .../selftests/bpf/test_progs.h:108:21: note: expanded from macro '_CHECK'
                fprintf(stdout, ##format);                              \
                                  ^

The first warning can be silenced with clang option -Wno-unused-command-line-argument. For the second warning, source codes are modified as suggested by the compiler to silence the warning. Since gcc does not support the option -Wno-unused-command-line-argument and the warning only happens with clang compiler, the option -Wno-unused-command-line-argument is enabled only when clang compiler is used.

Quoted 2 lines

---

LGTM, please see the question below.

Quoted 10 lines
 tools/testing/selftests/bpf/Makefile                         | 5 +++++
 tools/testing/selftests/bpf/prog_tests/fexit_sleep.c         | 4 ++--
 tools/testing/selftests/bpf/prog_tests/ns_current_pid_tgid.c | 4 ++--
 3 files changed, 9 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/Makefile +0 -0
index dcc2dc1f2a86..c45ae13b88a0 100644

tools/testing/selftests/bpf/Makefile CFLAGS += -g -Og -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS) \ · line 32

Quoted 5 lines
tools/testing/selftests/bpf/Makefile +2 -0
@@ -28,6 +28,11 @@ CFLAGS += -g -Og -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS)           \
          -Dbpf_load_program=bpf_test_load_program
 LDLIBS += -lcap -lelf -lz -lrt -lpthread
 
+# Silence some warnings when compiled with clang
+ifneq ($(LLVM),)

This won't handle the case where someone does make CC=clang, right? Do we care at all?

Quoted 3 lines
tools/testing/selftests/bpf/Makefile +3 -0
+CFLAGS += -Wno-unused-command-line-argument
+endif
+

[...]

#12 lore raw

Re: [PATCH bpf-next v3 4/5] selftests/bpf: silence clang compilation warnings

Quoted 48 lines · Andrii Nakryiko
Quoted 26 lines · Yonghong Song

With clang compiler:
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
Some linker flags are not used/effective for some binaries and we have warnings like:
warning: -lelf: 'linker' input unused [-Wunused-command-line-argument]

We also have warnings like:
.../selftests/bpf/prog_tests/ns_current_pid_tgid.c:74:57: note: treat the string as an argument to avoid this

        if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", strerror(errno)))
                                                               ^
                                                               "%s",
  .../selftests/bpf/test_progs.h:129:35: note: expanded from macro 'CHECK'
        _CHECK(condition, tag, duration, format)
                                         ^
  .../selftests/bpf/test_progs.h:108:21: note: expanded from macro '_CHECK'
                fprintf(stdout, ##format);                              \
                                  ^

The first warning can be silenced with clang option -Wno-unused-command-line-argument. For the second warning, source codes are modified as suggested by the compiler to silence the warning. Since gcc does not support the option -Wno-unused-command-line-argument and the warning only happens with clang compiler, the option -Wno-unused-command-line-argument is enabled only when clang compiler is used.

Quoted 2 lines

---

LGTM, please see the question below.

Quoted 13 lines
 tools/testing/selftests/bpf/Makefile                         | 5 +++++
 tools/testing/selftests/bpf/prog_tests/fexit_sleep.c         | 4 ++--
 tools/testing/selftests/bpf/prog_tests/ns_current_pid_tgid.c | 4 ++--
 3 files changed, 9 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/Makefile +0 -0
index dcc2dc1f2a86..c45ae13b88a0 100644
@@ -28,6 +28,11 @@ CFLAGS += -g -Og -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS)           \
          -Dbpf_load_program=bpf_test_load_program
 LDLIBS += -lcap -lelf -lz -lrt -lpthread
 
Quoted 5 lines

tools/testing/selftests/bpf/Makefile CFLAGS += -g -Og -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS) \ · line 32

Quoted 2 lines
tools/testing/selftests/bpf/Makefile +2 -0
+# Silence some warnings when compiled with clang
+ifneq ($(LLVM),)

This won't handle the case where someone does make CC=clang, right? Do we care at all?

Right; it would be better to check CC=clang and have LLVM=1 enable CC=clang from a higher level Makefile (since tools/build/Makefile seems to override the top level $CC). See the top level Makefile L448-456. Then it should work for CC=clang or LLVM=1.

Quoted 7 lines
Quoted 5 lines
tools/testing/selftests/bpf/Makefile +3 -0
+CFLAGS += -Wno-unused-command-line-argument
+endif
+

[...]

Signature · 2 lines
Thanks,
~Nick Desaulniers
#13 lore raw

Re: [PATCH bpf-next v3 4/5] selftests/bpf: silence clang compilation warnings

Quoted 56 lines · Nick Desaulniers
Quoted 48 lines · Andrii Nakryiko
Quoted 26 lines · Yonghong Song

With clang compiler:
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
Some linker flags are not used/effective for some binaries and we have warnings like:
warning: -lelf: 'linker' input unused [-Wunused-command-line-argument]

We also have warnings like:

   .../selftests/bpf/prog_tests/ns_current_pid_tgid.c:74:57: note: treat the string as an argument to avoid this
         if (CHECK(waitpid(cpid, &wstatus, 0) == -1, "waitpid", strerror(errno)))
                                                                ^
                                                                "%s",
   .../selftests/bpf/test_progs.h:129:35: note: expanded from macro 'CHECK'
         _CHECK(condition, tag, duration, format)
                                          ^
   .../selftests/bpf/test_progs.h:108:21: note: expanded from macro '_CHECK'
                 fprintf(stdout, ##format);                              \
                                   ^

The first warning can be silenced with clang option -Wno-unused-command-line-argument. For the second warning, source codes are modified as suggested by the compiler to silence the warning. Since gcc does not support the option -Wno-unused-command-line-argument and the warning only happens with clang compiler, the option -Wno-unused-command-line-argument is enabled only when clang compiler is used.

Quoted 2 lines

---

LGTM, please see the question below.

Quoted 13 lines
  tools/testing/selftests/bpf/Makefile                         | 5 +++++
  tools/testing/selftests/bpf/prog_tests/fexit_sleep.c         | 4 ++--
  tools/testing/selftests/bpf/prog_tests/ns_current_pid_tgid.c | 4 ++--
  3 files changed, 9 insertions(+), 4 deletions(-)
tools/testing/selftests/bpf/Makefile +0 -0
index dcc2dc1f2a86..c45ae13b88a0 100644
@@ -28,6 +28,11 @@ CFLAGS += -g -Og -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS)           \
           -Dbpf_load_program=bpf_test_load_program
  LDLIBS += -lcap -lelf -lz -lrt -lpthread
 
Quoted 5 lines

tools/testing/selftests/bpf/Makefile CFLAGS += -g -Og -rdynamic -Wall $(GENFLAGS) $(SAN_CFLAGS) \ · line 32

Quoted 2 lines
tools/testing/selftests/bpf/Makefile +2 -0
+# Silence some warnings when compiled with clang
+ifneq ($(LLVM),)

This won't handle the case where someone does make CC=clang, right? Do we care at all?

Quoted 4 lines

Right; it would be better to check CC=clang and have LLVM=1 enable CC=clang from a higher level Makefile (since tools/build/Makefile seems to override the top level $CC). See the top level Makefile L448-456. Then it should work for CC=clang or LLVM=1.

I would consider LLVM=1 is recommended and should be supported first. Support for CC=clang CXX=clang++ etc. can be done later if people find cases where LLVM=1 does not fit their need.

Quoted 11 lines
Quoted 8 lines
Quoted 6 lines
tools/testing/selftests/bpf/Makefile +3 -0
+CFLAGS += -Wno-unused-command-line-argument
+endif
+

[...]

#14 lore raw

PATCH v3 5/5 bpftool: fix a clang compilation warning

In reply to Yonghong Song

With clang compiler:
make -j60 LLVM=1 LLVM_IAS=1 <=== compile kernel

  # build selftests/bpf or bpftool
  make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1
  make -j60 -C tools/bpf/bpftool LLVM=1 LLVM_IAS=1
the following compilation warning showed up,
  net.c:160:37: warning: comparison of integers of different signs: '__u32' (aka 'unsigned int') and 'int' [-Wsign-compare]
                for (nh = (struct nlmsghdr *)buf; NLMSG_OK(nh, len);
                                                  ^~~~~~~~~~~~~~~~~
  .../tools/include/uapi/linux/netlink.h:99:24: note: expanded from macro 'NLMSG_OK'
                           (nlh)->nlmsg_len <= (len))
                           ~~~~~~~~~~~~~~~~ ^   ~~~

In this particular case, "len" is defined as "int" and (nlh)->nlmsg_len is "unsigned int". The macro NLMSG_OK is defined as below in uapi/linux/netlink.h.

  #define NLMSG_OK(nlh,len) ((len) >= (int)sizeof(struct nlmsghdr) && \
                             (nlh)->nlmsg_len >= sizeof(struct nlmsghdr) && \
                             (nlh)->nlmsg_len <= (len))

The clang compiler complains the comparision "(nlh)->nlmsg_len <= (len))", but in bpftool/net.c, it is already ensured that "len > 0" must be true. So theoretically the compiler could deduce that comparison of "(nlh)->nlmsg_len" and "len" is okay, but this really depends on compiler internals. Let us add an explicit type conversion (from "int" to "unsigned int") for "len" in NLMSG_OK to silence this warning right now.

---
 tools/bpf/bpftool/net.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tools/bpf/bpftool/net.c +1 -1
index ff3aa0cf3997..f836d115d7d6 100644
@@ -157,7 +157,7 @@ static int netlink_recv(int sock, __u32 nl_pid, __u32 seq,
 		if (len == 0)
 			break;
 
-		for (nh = (struct nlmsghdr *)buf; NLMSG_OK(nh, len);
+		for (nh = (struct nlmsghdr *)buf; NLMSG_OK(nh, (unsigned int)len);
 		     nh = NLMSG_NEXT(nh, len)) {
 			if (nh->nlmsg_pid != nl_pid) {
 				ret = -LIBBPF_ERRNO__WRNGPID;
Signature · 1 line
2.30.2
Acked-by on the v3 series
#15 lore raw
In reply to Yonghong Song
Quoted 25 lines · Yonghong Song

To build kernel with clang, people typically use
make -j60 LLVM=1 LLVM_IAS=1
LLVM_IAS=1 is not required for non-LTO build but
is required for LTO build. In my environment,
I am always having LLVM_IAS=1 regardless of
whether LTO is enabled or not.

After kernel is build with clang, the following command can be used to build selftests with clang:
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1

I am using latest bpf-next kernel code base and
latest clang built from source from
https://github.com/llvm/llvm-project.git
Using earlier version of llvm may have compilation errors, see
tools/testing/selftests/bpf
due to continuous development in llvm bpf features and selftests to use these features.

To run bpf selftest properly, you need have certain necessary kernel configs like at:
bpf-next:tools/testing/selftests/bpf/config
(not that this is not a complete .config file and some other configs
might still be needed.)

Quoted 6 lines

Currently, using the above command, some compilations still use gcc and there are also compilation errors and warnings. This patch set intends to fix these issues.
Patch #1 and #2 fixed the issue so clang/clang++ is used instead of gcc/g++. Patch #3 fixed a compilation failure. Patch #4 and #5 fixed various compiler warnings.

#16 lore raw
In reply to Yonghong Song
Quoted 32 lines · Yonghong Song

To build kernel with clang, people typically use
make -j60 LLVM=1 LLVM_IAS=1
LLVM_IAS=1 is not required for non-LTO build but
is required for LTO build. In my environment,
I am always having LLVM_IAS=1 regardless of
whether LTO is enabled or not.

After kernel is build with clang, the following command can be used to build selftests with clang:
make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1

I am using latest bpf-next kernel code base and
latest clang built from source from
https://github.com/llvm/llvm-project.git
Using earlier version of llvm may have compilation errors, see
tools/testing/selftests/bpf
due to continuous development in llvm bpf features and selftests to use these features.

To run bpf selftest properly, you need have certain necessary kernel configs like at:
bpf-next:tools/testing/selftests/bpf/config
(not that this is not a complete .config file and some other configs
might still be needed.)

Currently, using the above command, some compilations still use gcc and there are also compilation errors and warnings. This patch set intends to fix these issues.
Patch #1 and #2 fixed the issue so clang/clang++ is used instead of gcc/g++. Patch #3 fixed a compilation failure. Patch #4 and #5 fixed various compiler warnings.

Quoted 7 lines

Changelog:
v2 -> v3:
. more test environment description in cover letter. (Sedat) . use a different fix, but similar to other use in selftests/bpf
Makefile, to exclude header files from CXX compilation command line. (Andrii)
. fix codes instead of adding -Wno-format-security. (Andrii)

I struggled to tweak my llvm setup, but at the end it compiled and selftests/bpf/test_progs passed compiled by clang,
so I've applied to bpf-next.

The things I've seen:
1.
include <iostream> not found due to my setup quirks. 2.
diff selftests/bpf/tools/build/libbpf/libbpf_global_syms.tmp diff selftests/bpf/tools/build/libbpf/libbpf_versioned_syms.tmp
btf__set_pointer_size
btf__str_by_offset
btf__type_by_id
+LIBBPF_0.0.1
+LIBBPF_0.0.2
and this was happening with packaged llvm builds,
but my own llvm build was fine, so I didn't debug further.

3.
clang-12: error: unsupported option '-mrecord-mcount' for target 'x86_64-unknown-linux-gnu' due to kernel not built with clang.

I suspect followups will be needed to make it bulletproof.

#17 lore raw
Quoted 41 lines · Alexei Starovoitov
Quoted 32 lines · Yonghong Song

To build kernel with clang, people typically use
   make -j60 LLVM=1 LLVM_IAS=1
LLVM_IAS=1 is not required for non-LTO build but
is required for LTO build. In my environment,
I am always having LLVM_IAS=1 regardless of
whether LTO is enabled or not.

After kernel is build with clang, the following command can be used to build selftests with clang:
   make -j60 -C tools/testing/selftests/bpf LLVM=1 LLVM_IAS=1

I am using latest bpf-next kernel code base and
latest clang built from source from
   https://github.com/llvm/llvm-project.git
Using earlier version of llvm may have compilation errors, see tools/testing/selftests/bpf
due to continuous development in llvm bpf features and selftests to use these features.

To run bpf selftest properly, you need have certain necessary kernel configs like at:
   bpf-next:tools/testing/selftests/bpf/config
(not that this is not a complete .config file and some other configs might still be needed.)

Currently, using the above command, some compilations still use gcc and there are also compilation errors and warnings. This patch set intends to fix these issues.
Patch #1 and #2 fixed the issue so clang/clang++ is used instead of gcc/g++. Patch #3 fixed a compilation failure. Patch #4 and #5 fixed various compiler warnings.

Quoted 7 lines

Changelog:
   v2 -> v3:
     . more test environment description in cover letter. (Sedat) . use a different fix, but similar to other use in selftests/bpf Makefile, to exclude header files from CXX compilation command line. (Andrii)
     . fix codes instead of adding -Wno-format-security. (Andrii)

Quoted 7 lines

I struggled to tweak my llvm setup, but at the end it compiled and selftests/bpf/test_progs passed compiled by clang,
so I've applied to bpf-next.

The things I've seen:
1.
include <iostream> not found due to my setup quirks.

This header file is included by test_cpp.cpp. Probably some clang stdc++ setup issue.

Quoted 10 lines

2.
diff selftests/bpf/tools/build/libbpf/libbpf_global_syms.tmp diff selftests/bpf/tools/build/libbpf/libbpf_versioned_syms.tmp btf__set_pointer_size
btf__str_by_offset
btf__type_by_id

(quoted diff) +2 -0
+LIBBPF_0.0.1
+LIBBPF_0.0.2

and this was happening with packaged llvm builds, but my own llvm build was fine, so I didn't debug further.

I am also use latest llvm as I can easily verify all selftests passed.

Quoted 4 lines

3.
clang-12: error: unsupported option '-mrecord-mcount' for target 'x86_64-unknown-linux-gnu' due to kernel not built with clang.

Top level Makefile has
Makefile: CC_FLAGS_FTRACE += -mrecord-mcount
tools Makefile has some dependency on top level Makefile.

Quoted 2 lines

I suspect followups will be needed to make it bulletproof.

Definitely, Nick has libz issues, cross compile is not supported, tools build depends on kernel build, both or neither with LLVM=1, ... clang build got some traction recently and hopefully more people can start to contribute to make the build process more robust.