Bring back vmlinux.h generation
Subsystems (2)
Ian Rogers
reviewerv2 cover Bring back vmlinux.h generation
Commit 760ebc45746b ("perf lock contention: Add empty 'struct rq' to satisfy libbpf 'runqueue' type verification") inadvertently created a declaration of 'struct rq' that conflicted with a generated vmlinux.h's:
```
util/bpf_skel/lock_contention.bpf.c:419:8: error: redefinition of 'rq'
struct rq {};
^
/tmp/perf/util/bpf_skel/.tmp/../vmlinux.h:45630:8: note: previous definition is here
struct rq {
^
1 error generated.
```Fix the issue by moving the declaration to vmlinux.h. So this can't happen again, bring back build support for generating vmlinux.h then add build tests.
v2. Rebase on perf-tools-next. Add Andrii's acked-by. Add patch to
filter out kernels that lack a .BTF section and cause the build to break.
Ian Rogers (4):
perf build: Add ability to build with a generated vmlinux.h perf bpf: Move the declaration of struct rq
perf test: Add build tests for BUILD_BPF_SKEL perf build: Filter out BTF sources without a .BTF section
tools/perf/Makefile.config | 4 +++
tools/perf/Makefile.perf | 33 ++++++++++++++++++-
tools/perf/tests/make | 4 +++
tools/perf/util/bpf_skel/.gitignore | 1 +
.../perf/util/bpf_skel/lock_contention.bpf.c | 2 --
.../util/bpf_skel/{ => vmlinux}/vmlinux.h | 10 ++++++
6 files changed, 51 insertions(+), 3 deletions(-)
rename tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h (90%)Signature · 1 line
2.41.0.rc0.172.g3f132b7071-goog
Ian Rogers
reviewerPATCH v2 1/4 perf build: Add ability to build with a generated vmlinux.h
Commit a887466562b4 ("perf bpf skels: Stop using vmlinux.h generated from BTF, use subset of used structs + CO-RE") made it so that vmlinux.h was uncondtionally included from tools/perf/util/vmlinux.h. This change reverts part of that change (so that vmlinux.h is once again generated) and makes it so that the vmlinux.h used at build time is selected from the VMLINUX_H variable. By default the VMLINUX_H variable is set to the vmlinux.h added in change a887466562b4, but if GEN_VMLINUX_H=1 is passed on the build command line then the previous generation behavior kicks in.
The build with GEN_VMLINUX_H=1 currently fails with:
```
util/bpf_skel/lock_contention.bpf.c:419:8: error: redefinition of 'rq'
struct rq {};
^
/tmp/perf/util/bpf_skel/.tmp/../vmlinux.h:45630:8: note: previous definition is here
struct rq {
^
1 error generated.
```---
tools/perf/Makefile.config | 4 ++++
tools/perf/Makefile.perf | 16 +++++++++++++++-
tools/perf/util/bpf_skel/.gitignore | 1 +
tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h | 0
4 files changed, 20 insertions(+), 1 deletion(-)
rename tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h (100%)tools/perf/Makefile.config +4 -0
@@ -680,6 +680,10 @@ ifdef BUILD_BPF_SKEL CFLAGS += -DHAVE_BPF_SKEL endif +ifndef GEN_VMLINUX_H + VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h +endif + dwarf-post-unwind := 1 dwarf-post-unwind-text := BUG
tools/perf/Makefile.perf +15 -1
@@ -1080,7 +1080,21 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) | $(SKEL_TMP_OUT) +VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ + $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ + ../../vmlinux \ + /sys/kernel/btf/vmlinux \ + /boot/vmlinux-$(shell uname -r) +VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +$(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) +ifeq ($(VMLINUX_H),) + $(QUIET_GEN)$(BPFTOOL) btf dump file $< format c > $@ +else + $(Q)cp "$(VMLINUX_H)" $@ +endif + +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) $(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -Wall -Werror $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@
tools/perf/util/bpf_skel/.gitignore +1 -0
@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only .tmp *.skel.h +vmlinux.h
tools/perf/util/bpf_skel/vmlinux/vmlinux.h renamed from tools/perf/util/bpf_skel/vmlinux.h +0 -0
Signature · 1 line
2.41.0.rc0.172.g3f132b7071-goog
Re: [PATCH v2 1/4] perf build: Add ability to build with a generated vmlinux.h
Quoted 23 lines · Ian Rogers
Commit
a887466562b4("perf bpf skels: Stop usingvmlinux.hgenerated from BTF, use subset of used structs + CO-RE") made it so thatvmlinux.hwas uncondtionally included fromtools/perf/util/vmlinux.h. This change reverts part of that change (so thatvmlinux.his once again generated) and makes it so that thevmlinux.hused at build time is selected from theVMLINUX_Hvariable. By default theVMLINUX_Hvariable is set to thevmlinux.hadded in changea887466562b4, but if GEN_VMLINUX_H=1 is passed on the build command line then the previous generation behavior kicks in.The build with GEN_VMLINUX_H=1 currently fails with:
``` util/bpf_skel/lock_contention.bpf.c:419:8: error: redefinition of 'rq' struct rq {}; ^ /tmp/perf/util/bpf_skel/.tmp/../vmlinux.h:45630:8: note: previous definition is here struct rq { ^ 1 error generated. ```Signed-off-by Ian Rogers <[email protected]>Acked-by Andrii Nakryiko <[email protected]>
Quoted 7 lines
---
tools/perf/Makefile.config | 4 ++++
tools/perf/Makefile.perf | 16 +++++++++++++++-
tools/perf/util/bpf_skel/.gitignore | 1 +
tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h | 0
4 files changed, 20 insertions(+), 1 deletion(-)
rename tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h (100%)looks good, but I don't understand why you moved the vmlinux.h
jirka
Quoted 59 lines
tools/perf/Makefile.config +4
-0@@ -680,6 +680,10 @@ ifdef BUILD_BPF_SKEL CFLAGS += -DHAVE_BPF_SKEL endif +ifndef GEN_VMLINUX_H + VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h +endif + dwarf-post-unwind := 1 dwarf-post-unwind-text := BUGtools/perf/Makefile.perf +15
-1@@ -1080,7 +1080,21 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) | $(SKEL_TMP_OUT) +VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ + $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ + ../../vmlinux \ + /sys/kernel/btf/vmlinux \ + /boot/vmlinux-$(shell uname -r) +VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +$(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) +ifeq ($(VMLINUX_H),) + $(QUIET_GEN)$(BPFTOOL) btf dump file $< format c > $@ +else + $(Q)cp "$(VMLINUX_H)" $@ +endif + +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) $(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -Wall -Werror $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@tools/perf/util/bpf_skel/.gitignore +1
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only .tmp *.skel.h +vmlinux.htools/perf/util/bpf_skel/vmlinux/vmlinux.h renamed from tools/perf/util/bpf_skel/vmlinux.h +0
-0Signature · 1 line
2.41.0.rc0.172.g3f132b7071-goog
Ian Rogers
reviewerRe: [PATCH v2 1/4] perf build: Add ability to build with a generated vmlinux.h
Quoted 33 lines · Jiri Olsa
Quoted 23 lines · Ian Rogers
Commit
a887466562b4("perf bpf skels: Stop usingvmlinux.hgenerated from BTF, use subset of used structs + CO-RE") made it so thatvmlinux.hwas uncondtionally included fromtools/perf/util/vmlinux.h. This change reverts part of that change (so thatvmlinux.his once again generated) and makes it so that thevmlinux.hused at build time is selected from theVMLINUX_Hvariable. By default theVMLINUX_Hvariable is set to thevmlinux.hadded in changea887466562b4, but if GEN_VMLINUX_H=1 is passed on the build command line then the previous generation behavior kicks in.The build with GEN_VMLINUX_H=1 currently fails with:
``` util/bpf_skel/lock_contention.bpf.c:419:8: error: redefinition of 'rq' struct rq {}; ^ /tmp/perf/util/bpf_skel/.tmp/../vmlinux.h:45630:8: note: previous definition is here struct rq { ^ 1 error generated. ```Signed-off-by Ian Rogers <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Quoted 7 lines
--- tools/perf/Makefile.config | 4 ++++ tools/perf/Makefile.perf | 16 +++++++++++++++- tools/perf/util/bpf_skel/.gitignore | 1 + tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h | 0 4 files changed, 20 insertions(+), 1 deletion(-) rename tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h (100%)
Quoted 3 lines
looks good, but I don't understand why you moved the
vmlinux.hjirka
Dumb reason, as headers in the same directory take priority, I had to move the vmlinux.h out of the directory with the C code for skeletons so that it could be selected via a -I.
Thanks,
Ian
Quoted 59 lines
Quoted 59 lines
tools/perf/Makefile.config +4
-0@@ -680,6 +680,10 @@ ifdef BUILD_BPF_SKEL CFLAGS += -DHAVE_BPF_SKEL endif +ifndef GEN_VMLINUX_H + VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h +endif + dwarf-post-unwind := 1 dwarf-post-unwind-text := BUGtools/perf/Makefile.perf +15
-1@@ -1080,7 +1080,21 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) | $(SKEL_TMP_OUT) +VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ + $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ + ../../vmlinux \ + /sys/kernel/btf/vmlinux \ + /boot/vmlinux-$(shell uname -r) +VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +$(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) +ifeq ($(VMLINUX_H),) + $(QUIET_GEN)$(BPFTOOL) btf dump file $< format c > $@ +else + $(Q)cp "$(VMLINUX_H)" $@ +endif + +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) $(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -Wall -Werror $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@tools/perf/util/bpf_skel/.gitignore +1
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only .tmp *.skel.h +vmlinux.htools/perf/util/bpf_skel/vmlinux/vmlinux.h renamed from tools/perf/util/bpf_skel/vmlinux.h +0
-1--2.41.0.rc0.172.g3f132b7071-goog
Arnaldo Carvalho de Melo
maintainerRe: [PATCH v2 1/4] perf build: Add ability to build with a generated vmlinux.h
Em Mon, Jun 05, 2023 at 09:25:54PM -0700, Ian Rogers escreveu:
Quoted 34 lines · Jiri Olsa
Quoted 33 lines · Ian Rogers
Quoted 30 lines
Commit
a887466562b4("perf bpf skels: Stop usingvmlinux.hgenerated from BTF, use subset of used structs + CO-RE") made it so thatvmlinux.hwas uncondtionally included fromtools/perf/util/vmlinux.h. This change reverts part of that change (so thatvmlinux.his once again generated) and makes it so that thevmlinux.hused at build time is selected from theVMLINUX_Hvariable. By default theVMLINUX_Hvariable is set to thevmlinux.hadded in changea887466562b4, but if GEN_VMLINUX_H=1 is passed on the build command line then the previous generation behavior kicks in.The build with GEN_VMLINUX_H=1 currently fails with:
``` util/bpf_skel/lock_contention.bpf.c:419:8: error: redefinition of 'rq' struct rq {}; ^ /tmp/perf/util/bpf_skel/.tmp/../vmlinux.h:45630:8: note: previous definition is here struct rq { ^ 1 error generated. ```Signed-off-by Ian Rogers <[email protected]>Acked-by Andrii Nakryiko <[email protected]>--- tools/perf/Makefile.config | 4 ++++ tools/perf/Makefile.perf | 16 +++++++++++++++- tools/perf/util/bpf_skel/.gitignore | 1 + tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h | 0 4 files changed, 20 insertions(+), 1 deletion(-) rename tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h (100%)
Quoted 7 lines
Quoted 3 lines
looks good, but I don't understand why you moved the
vmlinux.hjirka
Dumb reason, as headers in the same directory take priority, I had to move the
vmlinux.hout of the directory with the C code for skeletons so that it could be selected via a -I.
Can this be in a separate patch, i.e. moving vmlinux to a separate directory? I was going to cherry pick the 'struct rq' fix but then it touches the vmlinux/vmlinux.h file that is in this first patch that has review comments.
- Arnaldo
Quoted 62 lines
Thanks,
IanQuoted 59 lines
Quoted 59 lines
tools/perf/Makefile.config +4
-0@@ -680,6 +680,10 @@ ifdef BUILD_BPF_SKEL CFLAGS += -DHAVE_BPF_SKEL endif +ifndef GEN_VMLINUX_H + VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h +endif + dwarf-post-unwind := 1 dwarf-post-unwind-text := BUGtools/perf/Makefile.perf +15
-1@@ -1080,7 +1080,21 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) | $(SKEL_TMP_OUT) +VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ + $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ + ../../vmlinux \ + /sys/kernel/btf/vmlinux \ + /boot/vmlinux-$(shell uname -r) +VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +$(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) +ifeq ($(VMLINUX_H),) + $(QUIET_GEN)$(BPFTOOL) btf dump file $< format c > $@ +else + $(Q)cp "$(VMLINUX_H)" $@ +endif + +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) $(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -Wall -Werror $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@tools/perf/util/bpf_skel/.gitignore +1
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only .tmp *.skel.h +vmlinux.htools/perf/util/bpf_skel/vmlinux/vmlinux.h renamed from tools/perf/util/bpf_skel/vmlinux.h +0
-1--2.41.0.rc0.172.g3f132b7071-goog
Signature · 1 line
- Arnaldo
Ian Rogers
reviewerRe: [PATCH v2 1/4] perf build: Add ability to build with a generated vmlinux.h
Quoted 40 lines · Arnaldo Carvalho de Melo
Em Mon, Jun 05, 2023 at 09:25:54PM -0700, Ian Rogers escreveu:
Quoted 38 lines · Jiri Olsa
Quoted 36 lines · Ian Rogers
Quoted 23 lines
Commit
a887466562b4("perf bpf skels: Stop usingvmlinux.hgenerated from BTF, use subset of used structs + CO-RE") made it so thatvmlinux.hwas uncondtionally included fromtools/perf/util/vmlinux.h. This change reverts part of that change (so thatvmlinux.his once again generated) and makes it so that thevmlinux.hused at build time is selected from theVMLINUX_Hvariable. By default theVMLINUX_Hvariable is set to thevmlinux.hadded in changea887466562b4, but if GEN_VMLINUX_H=1 is passed on the build command line then the previous generation behavior kicks in.The build with GEN_VMLINUX_H=1 currently fails with:
``` util/bpf_skel/lock_contention.bpf.c:419:8: error: redefinition of 'rq' struct rq {}; ^ /tmp/perf/util/bpf_skel/.tmp/../vmlinux.h:45630:8: note: previous definition is here struct rq { ^ 1 error generated. ```Signed-off-by Ian Rogers <[email protected]>Acked-by Andrii Nakryiko <[email protected]>Quoted 7 lines
--- tools/perf/Makefile.config | 4 ++++ tools/perf/Makefile.perf | 16 +++++++++++++++- tools/perf/util/bpf_skel/.gitignore | 1 + tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h | 0 4 files changed, 20 insertions(+), 1 deletion(-) rename tools/perf/util/bpf_skel/{ => vmlinux}/vmlinux.h (100%)looks good, but I don't understand why you moved the
vmlinux.hjirka
Quoted 8 lines
Quoted 3 lines
Dumb reason, as headers in the same directory take priority, I had to move the
vmlinux.hout of the directory with the C code for skeletons so that it could be selected via a -I.Can this be in a separate patch, i.e. moving vmlinux to a separate directory? I was going to cherry pick the 'struct rq' fix but then it touches the
vmlinux/vmlinux.hfile that is in this first patch that has review comments.
I think the comments were more of a question of why the move? Hopefully I answered that, the move is necessary to defeat the include path order preferring files in the same directory. We need the move, we could vary the directory name.
Thanks,
Ian
Quoted 68 lines
- Arnaldo
Quoted 46 lines
Thanks,
IanQuoted 43 lines
Quoted 43 lines
tools/perf/Makefile.config +4
-0@@ -680,6 +680,10 @@ ifdef BUILD_BPF_SKEL CFLAGS += -DHAVE_BPF_SKEL endif +ifndef GEN_VMLINUX_H + VMLINUX_H=$(src-perf)/util/bpf_skel/vmlinux/vmlinux.h +endif + dwarf-post-unwind := 1 dwarf-post-unwind-text := BUGtools/perf/Makefile.perf +15
-1@@ -1080,7 +1080,21 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) | $(SKEL_TMP_OUT) +VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ + $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ + ../../vmlinux \ + /sys/kernel/btf/vmlinux \ + /boot/vmlinux-$(shell uname -r) +VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +$(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) +ifeq ($(VMLINUX_H),) + $(QUIET_GEN)$(BPFTOOL) btf dump file $< format c > $@ +else + $(Q)cp "$(VMLINUX_H)" $@ +endif + +$(SKEL_TMP_OUT)/%.bpf.o: util/bpf_skel/%.bpf.c $(LIBBPF) $(SKEL_OUT)/vmlinux.h | $(SKEL_TMP_OUT) $(QUIET_CLANG)$(CLANG) -g -O2 -target bpf -Wall -Werror $(BPF_INCLUDE) $(TOOLS_UAPI_INCLUDE) \ -c $(filter util/bpf_skel/%.bpf.c,$^) -o $@Quoted 16 lines
Quoted 8 lines
Quoted 8 lines
tools/perf/util/bpf_skel/.gitignore +0
-0@@ -1,3 +1,4 @@ # SPDX-License-Identifier: GPL-2.0-only .tmp *.skel.hQuoted 8 lines
Quoted 8 lines
tools/perf/util/bpf_skel/.gitignore +1
-0+vmlinux.htools/perf/util/bpf_skel/vmlinux/vmlinux.h renamed from tools/perf/util/bpf_skel/vmlinux.h +0
-1--2.41.0.rc0.172.g3f132b7071-goog
--
- Arnaldo
kernel test robot
build botRe: [PATCH v2 1/4] perf build: Add ability to build with a generated vmlinux.h
In reply to Ian Rogerskernel test robot noticed the following build warnings:
Hi Ian,
kernel test robot noticed the following build warnings:
[auto build test WARNING on acme/perf/core]
[also build test WARNING on tip/master tip/auto-latest linus/master v6.4-rc5 next-20230606] [cannot apply to tip/perf/core]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Ian-Rogers/perf-build-Add-ability-to-build-with-a-generated-vmlinux-h/20230606-043106 base: https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git perf/core patch link: https://lore.kernel.org/r/20230605202712.1690876-2-irogers%40google.com patch subject: [PATCH v2 1/4] perf build: Add ability to build with a generated vmlinux.h config: i386-randconfig-i052-20230605 (https://download.01.org/0day-ci/archive/20230606/[email protected]/config) compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project.git 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce (this is a W=1 build):
mkdir -p ~/bin
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross
git remote add acme https://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git
git fetch acme perf/core
git checkout acme/perf/core
b4 shazam https://lore.kernel.org/r/[email protected]
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang ~/bin/make.cross W=1 O=build_dir ARCH=i386 olddefconfig COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang ~/bin/make.cross W=1 O=build_dir ARCH=i386 SHELL=/bin/bash
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> tools/perf/util/bpf_skel/vmlinux/vmlinux.h: warning: ignored by one of the .gitignore files
tools/testing/selftests/arm64/tags/.gitignore: warning: ignored by one of the .gitignore files tools/testing/selftests/arm64/tags/Makefile: warning: ignored by one of the .gitignore files tools/testing/selftests/arm64/tags/run_tags_test.sh: warning: ignored by one of the .gitignore files tools/testing/selftests/arm64/tags/tags_test.c: warning: ignored by one of the .gitignore files tools/testing/selftests/kvm/.gitignore: warning: ignored by one of the .gitignore files tools/testing/selftests/kvm/Makefile: warning: ignored by one of the .gitignore files tools/testing/selftests/kvm/config: warning: ignored by one of the .gitignore files tools/testing/selftests/kvm/settings: warning: ignored by one of the .gitignore files
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Ian Rogers
reviewerPATCH v2 2/4 perf bpf: Move the declaration of struct rq
In reply to Ian Rogersstruct rq is defined in vmlinux.h when the vmlinux.h is generated, this causes a redefinition failure if it is declared in lock_contention.bpf.c. Move the definition to vmlinux.h for consistency with the generated version.
--- tools/perf/util/bpf_skel/lock_contention.bpf.c | 2 -- tools/perf/util/bpf_skel/vmlinux/vmlinux.h | 10 ++++++++++ 2 files changed, 10 insertions(+), 2 deletions(-)
tools/perf/util/bpf_skel/lock_contention.bpf.c +0 -2
@@ -416,8 +416,6 @@ int contention_end(u64 *ctx)return 0 ; } -struct rq {}; - extern struct rq runqueues __ksym;struct rq___old {
tools/perf/util/bpf_skel/vmlinux/vmlinux.h +10 -0
@@ -171,4 +171,14 @@ struct bpf_perf_event_data_kern {struct perf_sample_data *data;struct perf_event *event; } __attribute__((preserve_access_index)); + +/* +* If 'struct rq' isn't defined for lock_contention.bpf.c, for the sake of +* rq___old and rq___new, then the type for the 'runqueue' variable ends up +* being a forward declaration (BTF_KIND_FWD) while the kernel has it defined +* (BTF_KIND_STRUCT). The definition appears in vmlinux.h rather than +* lock_contention.bpf.c for consistency with a generated vmlinux.h. +*/ +struct rq {}; +#endif // __VMLINUX_H
Signature · 1 line
2.41.0.rc0.172.g3f132b7071-goog
Ian Rogers
reviewerPATCH v2 3/4 perf test: Add build tests for BUILD_BPF_SKEL
In reply to Ian RogersAdd tests with and without generating vmlinux.h.
--- tools/perf/tests/make | 4 ++++ 1 file changed, 4 insertions(+)
tools/perf/tests/make +4 -0
@@ -69,6 +69,8 @@ make_clean_all := clean all make_python_perf_so := $(python_perf_so) make_debug := DEBUG=1 make_nondistro := BUILD_NONDISTRO=1 +make_bpf_skel := BUILD_BPF_SKEL=1 +make_gen_vmlinux_h := BUILD_BPF_SKEL=1 GEN_VMLINUX_H=1 make_no_libperl := NO_LIBPERL=1 make_no_libpython := NO_LIBPYTHON=1 make_no_scripts := NO_LIBPYTHON=1 NO_LIBPERL=1
@@ -136,6 +138,8 @@ endif run += make_python_perf_so run += make_debug run += make_nondistro +run += make_build_bpf_skel +run += make_gen_vmlinux_h run += make_no_libperl run += make_no_libpython run += make_no_scripts
Signature · 1 line
2.41.0.rc0.172.g3f132b7071-goog
Ian Rogers
reviewerPATCH v2 4/4 perf build: Filter out BTF sources without a .BTF section
In reply to Ian RogersIf generating vmlinux.h, make the code to generate it more tolerant by filtering out paths to kernels that lack a .BTF section.
--- tools/perf/Makefile.perf | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
tools/perf/Makefile.perf +20 -3
@@ -193,6 +193,7 @@ FLEX ?= flex BISON ?= bison STRIP = strip AWK = awk +READELF ?= readelf # include Makefile.config by default and rule out # non-config cases
@@ -1080,12 +1081,28 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ +# Paths to search for a kernel to generate vmlinux.h from. +VMLINUX_BTF_ELF_PATHS ?= $(if $(O),$(O)/vmlinux) \ $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ ../../vmlinux \ - /sys/kernel/btf/vmlinux \ /boot/vmlinux-$(shell uname -r) -VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +# Paths to BTF information. +VMLINUX_BTF_BTF_PATHS ?= /sys/kernel/btf/vmlinux + +# Filter out kernels that don't exist or without a BTF section. +VMLINUX_BTF_ELF_ABSPATHS ?= $(abspath $(wildcard $(VMLINUX_BTF_ELF_PATHS))) +VMLINUX_BTF_PATHS ?= $(shell for file in $(VMLINUX_BTF_ELF_ABSPATHS); \ + do \ + if [ -f $$file ] && ($(READELF) -t "$$file" | grep .BTF); \ + then \ + echo "$$file"; \ + fi; \ + done) \ + $(wildcard $(VMLINUX_BTF_BTF_PATHS)) + +# Select the first as the source of vmlinux.h. +VMLINUX_BTF ?= $(firstword $(VMLINUX_BTF_PATHS)) $(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) ifeq ($(VMLINUX_H),)
Signature · 1 line
2.41.0.rc0.172.g3f132b7071-goog
Namhyung Kim
maintainerRe: [PATCH v2 4/4] perf build: Filter out BTF sources without a .BTF section
Hi Ian,
Quoted 32 lines · Ian Rogers
If generating
vmlinux.h, make the code to generate it more tolerant by filtering out paths to kernels that lack a .BTF section.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/Makefile.perf | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)tools/perf/Makefile.perf +3
-2@@ -193,6 +193,7 @@ FLEX ?= flex BISON ?= bison STRIP = strip AWK = awk +READELF ?= readelf # include Makefile.config by default and rule out # non-config cases@@ -1080,12 +1081,28 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ +# Paths to search for a kernel to generate vmlinux.h from. +VMLINUX_BTF_ELF_PATHS ?= $(if $(O),$(O)/vmlinux) \ $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ ../../vmlinux \ - /sys/kernel/btf/vmlinux \ /boot/vmlinux-$(shell uname -r)
tools/perf/Makefile.perf $(BPFTOOL): | $(SKEL_TMP_OUT) · line 1097
Quoted 10 lines
tools/perf/Makefile.perf +9
-1-VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +# Paths to BTF information. +VMLINUX_BTF_BTF_PATHS ?= /sys/kernel/btf/vmlinux + +# Filter out kernels that don't exist or without a BTF section. +VMLINUX_BTF_ELF_ABSPATHS ?= $(abspath $(wildcard $(VMLINUX_BTF_ELF_PATHS))) +VMLINUX_BTF_PATHS ?= $(shell for file in $(VMLINUX_BTF_ELF_ABSPATHS); \ + do \ + if [ -f $$file ] && ($(READELF) -t "$$file" | grep .BTF); \
Wouldn't it be readelf -S instead? Also I think grep needs -q to suppress output.
tools/perf/Makefile.perf $(BPFTOOL): | $(SKEL_TMP_OUT) · line 1102
Quoted 5 lines
tools/perf/Makefile.perf +5
-0+ then \ + echo "$$file"; \ + fi; \ + done) \ + $(wildcard $(VMLINUX_BTF_BTF_PATHS))
This changes the order of processing the sysfs file. But I'm not sure it matters much as both /boot/vmlinux and sysfs should refer to the running kernel.
Thanks,
Namhyung
Quoted 9 lines
tools/perf/Makefile.perf +3
-1+ +# Select the first as the source of vmlinux.h. +VMLINUX_BTF ?= $(firstword $(VMLINUX_BTF_PATHS)) $(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) ifeq ($(VMLINUX_H),) --2.41.0.rc0.172.g3f132b7071-goog
Ian Rogers
reviewerRe: [PATCH v2 4/4] perf build: Filter out BTF sources without a .BTF section
Quoted 41 lines · Namhyung Kim
Hi Ian,
Quoted 37 lines · Ian Rogers
If generating
vmlinux.h, make the code to generate it more tolerant by filtering out paths to kernels that lack a .BTF section.Signed-off-by Ian Rogers <[email protected]>--- tools/perf/Makefile.perf | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)tools/perf/Makefile.perf +7
-3@@ -193,6 +193,7 @@ FLEX ?= flex BISON ?= bison STRIP = strip AWK = awk +READELF ?= readelf # include Makefile.config by default and rule out # non-config cases@@ -1080,12 +1081,28 @@ $(BPFTOOL): | $(SKEL_TMP_OUT) $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \ OUTPUT=$(SKEL_TMP_OUT)/ bootstrap -VMLINUX_BTF_PATHS ?= $(if $(O),$(O)/vmlinux) \ +# Paths to search for a kernel to generate vmlinux.h from. +VMLINUX_BTF_ELF_PATHS ?= $(if $(O),$(O)/vmlinux) \ $(if $(KBUILD_OUTPUT),$(KBUILD_OUTPUT)/vmlinux) \ ../../vmlinux \ - /sys/kernel/btf/vmlinux \ /boot/vmlinux-$(shell uname -r) -VMLINUX_BTF ?= $(abspath $(firstword $(wildcard $(VMLINUX_BTF_PATHS)))) + +# Paths to BTF information. +VMLINUX_BTF_BTF_PATHS ?= /sys/kernel/btf/vmlinux +
Quoted 8 lines
tools/perf/Makefile.perf $(BPFTOOL): | $(SKEL_TMP_OUT) · line 1097
Quoted 5 lines
tools/perf/Makefile.perf +5
-0+# Filter out kernels that don't exist or without a BTF section. +VMLINUX_BTF_ELF_ABSPATHS ?= $(abspath $(wildcard $(VMLINUX_BTF_ELF_PATHS))) +VMLINUX_BTF_PATHS ?= $(shell for file in $(VMLINUX_BTF_ELF_ABSPATHS); \ + do \ + if [ -f $$file ] && ($(READELF) -t "$$file" | grep .BTF); \Wouldn't it be
readelf -Sinstead? Also I think grep needs -q to suppress output.
Makes sense, I can change it in v3.
Quoted 9 lines
tools/perf/Makefile.perf $(BPFTOOL): | $(SKEL_TMP_OUT) · line 1102
Quoted 5 lines
tools/perf/Makefile.perf +5
-0+ then \ + echo "$$file"; \ + fi; \ + done) \ + $(wildcard $(VMLINUX_BTF_BTF_PATHS))This changes the order of processing the sysfs file. But I'm not sure it matters much as both
/boot/vmlinuxand sysfs should refer to the running kernel.
Agreed. readelf fails for /sys/kernel/btf/vmlinux as it isn't an elf file and I'm not sure it is worth worrying too much about the order here.
Thanks,
Ian
Quoted 13 lines
Thanks,
NamhyungQuoted 9 lines
tools/perf/Makefile.perf +3
-1+ +# Select the first as the source of vmlinux.h. +VMLINUX_BTF ?= $(firstword $(VMLINUX_BTF_PATHS)) $(SKEL_OUT)/vmlinux.h: $(VMLINUX_BTF) $(BPFTOOL) ifeq ($(VMLINUX_H),) --2.41.0.rc0.172.g3f132b7071-goog