Skip to contentSkip to conversation

perf build: Add system include paths to BPF builds

PATCH In review Ian Rogers 3 participants 4 messages May 6, 2023 – May 11, 2023

Subsystems (3)
lore
Share & tools
lore

Ian Rogers

author reviewer
#0 lore raw

PATCH 1/1 perf build: Add system include paths to BPF builds

There are insufficient headers in tools/include to satisfy building BPF programs and their header dependencies. Add the system include paths from the non-BPF clang compile so that these headers can be found.

This code was taken from:
tools/testing/selftests/bpf/Makefile

---
 tools/perf/Makefile.perf | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
tools/perf/Makefile.perf +19 -1
index 61c33d100b2b..37befdfa8ac8 100644
@@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
 
 ifndef NO_BPF_SKEL
 BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool
-BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE)
+
+# Get Clang's default includes on this system, as opposed to those seen by
+# '-target bpf'. This fixes "missing" files on some architectures/distros,
+# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc.
+#
+# Use '-idirafter': Don't interfere with include mechanics except where the
+# build would have failed anyways.
+define get_sys_includes
+$(shell $(1) $(2) -v -E - </dev/null 2>&1 \
+	| sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
+$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}')
+endef
+
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+
+CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))
+BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES)
 
 $(BPFTOOL): | $(SKEL_TMP_OUT)
 	$(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
Signature · 1 line
2.40.1.521.gf1e218fcd8-goog
maintainer Tested-by on v1 1/1
#1 lore raw

Hi Ian,

Quoted 10 lines · Ian Rogers

There are insufficient headers in tools/include to satisfy building BPF programs and their header dependencies. Add the system include paths from the non-BPF clang compile so that these headers can be found.

This code was taken from:
tools/testing/selftests/bpf/Makefile

Thanks, this fixes the BPF build failure on my machine.

Thanks,
Namhyung

Quoted 38 lines
---
 tools/perf/Makefile.perf | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
tools/perf/Makefile.perf +19 -2
index 61c33d100b2b..37befdfa8ac8 100644
@@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
 
 ifndef NO_BPF_SKEL
 BPFTOOL := $(SKEL_TMP_OUT)/bootstrap/bpftool
-BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE)
+
+# Get Clang's default includes on this system, as opposed to those seen by
+# '-target bpf'. This fixes "missing" files on some architectures/distros,
+# such as asm/byteorder.h, asm/socket.h, asm/sockios.h, sys/cdefs.h etc.
+#
+# Use '-idirafter': Don't interfere with include mechanics except where the
+# build would have failed anyways.
+define get_sys_includes
+$(shell $(1) $(2) -v -E - </dev/null 2>&1 \
+       | sed -n '/<...> search starts here:/,/End of search list./{ s| \(/.*\)|-idirafter \1|p }') \
+$(shell $(1) $(2) -dM -E - </dev/null | grep '__riscv_xlen ' | awk '{printf("-D__riscv_xlen=%d -D__BITS_PER_LONG=%d", $$3, $$3)}')
+endef
+
+ifneq ($(CROSS_COMPILE),)
+CLANG_TARGET_ARCH = --target=$(notdir $(CROSS_COMPILE:%-=%))
+endif
+
+CLANG_SYS_INCLUDES = $(call get_sys_includes,$(CLANG),$(CLANG_TARGET_ARCH))
+BPF_INCLUDE := -I$(SKEL_TMP_OUT)/.. -I$(LIBBPF_INCLUDE) $(CLANG_SYS_INCLUDES)
 
 $(BPFTOOL): | $(SKEL_TMP_OUT)
        $(Q)CFLAGS= $(MAKE) -C ../bpf/bpftool \
--

2.40.1.521.gf1e218fcd8-goog

#2 lore raw
In reply to Ian Rogers

Em Fri, May 05, 2023 at 07:14:50PM -0700, Ian Rogers escreveu:

Quoted 18 lines

There are insufficient headers in tools/include to satisfy building BPF programs and their header dependencies. Add the system include paths from the non-BPF clang compile so that these headers can be found.

This code was taken from:
tools/testing/selftests/bpf/Makefile

---
 tools/perf/Makefile.perf | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
tools/perf/Makefile.perf +0 -0
index 61c33d100b2b..37befdfa8ac8 100644

tools/perf/Makefile.perf $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_ · line 1058

Quoted 2 lines
tools/perf/Makefile.perf +0 -0
@@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
 
 ifndef NO_BPF_SKEL

So this patch was done before the reverts, I adjusted it to what is upstream and to another patch that makes the build use the headers from the perf sources instead of the system's (linux/bpf.h and linux/perf_event.h, from vmlinux.h), please take a look at the patch below, I'm also trying to figure out that other problem you pointed with linux/types.s :-\

What I have now in tmp.perf-tools:

⬢[acme@toolbox perf-tools]$ git log --oneline torvalds/master..
a2af0f6b8ef7ea40 (HEAD -> perf-tools) perf build: Add system include paths to BPF builds 5be6cecda0802f23 perf bpf skels: Make vmlinux.h use bpf.h and perf_event.h in source directory 7d161165d9072dcb perf parse-events: Do not break up AUX event group
a468085011ea8bba perf test test_intel_pt.sh: Test sample mode with event with PMU name 123361659fa405de perf evsel: Modify group pmu name for software events
34e82891d995ab89 tools arch x86: Sync the msr-index.h copy with the kernel sources 705049ca4f5b7b00 tools headers kvm: Sync uapi/{asm/linux} kvm.h headers with the kernel sources 8d6a41c8065e1120 tools include UAPI: Sync the sound/asound.h copy with the kernel sources 92b8e61e88351091 tools headers UAPI: Sync the linux/const.h with the kernel headers e7ec3a249c38a9c9 tools headers UAPI: Sync the i915_drm.h with the kernel sources e6232180e524e112 tools headers UAPI: Sync the drm/drm.h with the kernel sources 5d1ac59ff7445e51 tools headers UAPI: Sync the linux/in.h with the kernel sources b0618f38e2ab8ce3 perf build: Gracefully fail the build if BUILD_BPF_SKEL=1 is specified and clang isn't available 5f0b89e632ed81b6 perf test java symbol: Remove needless debuginfod queries
327daf34554d20a6 perf parse-events: Don't reorder ungrouped events by PMU
ccc66c6092802d68 perf metric: JSON flag to not group events if gathering a metric group 1b114824106ca468 perf stat: Introduce skippable evsels
2a939c8695035b11 perf metric: Change divide by zero and !support events behavior
⬢[acme@toolbox perf-tools]$

Please help me test this,

Regards,

- Arnaldo

author reviewer
#3 lore raw
Quoted 53 lines · Arnaldo Carvalho de Melo

Em Fri, May 05, 2023 at 07:14:50PM -0700, Ian Rogers escreveu:

Quoted 18 lines

There are insufficient headers in tools/include to satisfy building BPF programs and their header dependencies. Add the system include paths from the non-BPF clang compile so that these headers can be found.

This code was taken from:
tools/testing/selftests/bpf/Makefile

---
 tools/perf/Makefile.perf | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)
tools/perf/Makefile.perf +0 -0
index 61c33d100b2b..37befdfa8ac8 100644

tools/perf/Makefile.perf $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_ · line 1058

Quoted 2 lines
tools/perf/Makefile.perf +0 -0
@@ -1057,7 +1057,25 @@ $(SKEL_TMP_OUT) $(LIBAPI_OUTPUT) $(LIBBPF_OUTPUT) $(LIBPERF_OUTPUT) $(LIBSUBCMD_
 
 ifndef NO_BPF_SKEL

So this patch was done before the reverts, I adjusted it to what is upstream and to another patch that makes the build use the headers from the perf sources instead of the system's (linux/bpf.h and linux/perf_event.h, from vmlinux.h), please take a look at the patch below, I'm also trying to figure out that other problem you pointed with linux/types.s :-\

What I have now in tmp.perf-tools:

⬢[acme@toolbox perf-tools]$ git log --oneline torvalds/master..
a2af0f6b8ef7ea40 (HEAD -> perf-tools) perf build: Add system include paths to BPF builds 5be6cecda0802f23 perf bpf skels: Make vmlinux.h use bpf.h and perf_event.h in source directory 7d161165d9072dcb perf parse-events: Do not break up AUX event group
a468085011ea8bba perf test test_intel_pt.sh: Test sample mode with event with PMU name 123361659fa405de perf evsel: Modify group pmu name for software events
34e82891d995ab89 tools arch x86: Sync the msr-index.h copy with the kernel sources 705049ca4f5b7b00 tools headers kvm: Sync uapi/{asm/linux} kvm.h headers with the kernel sources 8d6a41c8065e1120 tools include UAPI: Sync the sound/asound.h copy with the kernel sources 92b8e61e88351091 tools headers UAPI: Sync the linux/const.h with the kernel headers e7ec3a249c38a9c9 tools headers UAPI: Sync the i915_drm.h with the kernel sources e6232180e524e112 tools headers UAPI: Sync the drm/drm.h with the kernel sources 5d1ac59ff7445e51 tools headers UAPI: Sync the linux/in.h with the kernel sources b0618f38e2ab8ce3 perf build: Gracefully fail the build if BUILD_BPF_SKEL=1 is specified and clang isn't available 5f0b89e632ed81b6 perf test java symbol: Remove needless debuginfod queries
327daf34554d20a6 perf parse-events: Don't reorder ungrouped events by PMU
ccc66c6092802d68 perf metric: JSON flag to not group events if gathering a metric group 1b114824106ca468 perf stat: Introduce skippable evsels
2a939c8695035b11 perf metric: Change divide by zero and !support events behavior
⬢[acme@toolbox perf-tools]$

Quoted 1 line

Please help me test this,

build-test and compiling with/without BPF skeletons looked okay in perf test on my Debian derived distro.

Thanks,
Ian

Quoted 5 lines

Regards,

- Arnaldo