Skip to contentSkip to conversation

kbuild: build the userspace tools in tools/ with the kernel

PATCH In review Sasha Levin 4 participants 19 messages Nov 12 – Nov 16

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

v4 cover kbuild: build the userspace tools in tools/ with the kernel

The userspace programs under tools/ are disconnected from the rest of the kernel's build system. Each has its own makefiles and build framework, and "make" never builds them: apart from the few host tools the build itself needs, such as objtool and resolve_btfids, nothing under tools/ is compiled as part of a kernel build.

That makes them easy to break. Most people, and most of the build bots, check a change with "make allmodconfig && make", which doesn't cover tools/ at all. A change that breaks a tool passes every build check and is only noticed when someone happens to build that tool by hand, often much later.

We keep seeing this in tools/testing/: selftests that stop building after a change elsewhere in the tree, or after a backport to a stable tree, and stay broken until someone tries to build them. Others brought up the same concern at LPC.

This series lets the kernel build also build the tools it is configured for:

- tools/Kconfig adds a "Build userspace tools from tools/" menu (CONFIG_TOOLS_BUILD) with one entry per tool and per selftests directory, so .config records what gets built, and a tool that breaks can be turned off on its own.

- "make tools_build" builds the selected tools for the target, each with its own build system as it is built standalone, into tools-build/ in the object tree.

- With CONFIG_TOOLS_BUILD=y, "make all" builds them too. allmodconfig and allyesconfig enable every entry that applies to the target when the compiler can link glibc programs for it (currently x86_64 and arm64), so the usual build test catches tools breakage. Other toolchains are unaffected, as are "make vmlinux", "make modules" and external module builds.

This is useful beyond build testing: the tools are built for the same target, from the same tree, as the kernel they go with.

The selftests need "selftests: lib.mk: define LINK.c, COMPILE.S and LINK.S when make has none" [1], which is not in mainline yet. Without it, most of them fail to build in tools_build.

I used LLM coding assistants to write and review these patches. I reviewed and tested every change.

Changes in v4:
- 2/7: Check for glibc with the 64-bit flag that CC_CAN_LINK uses, so a compiler that defaults to 32-bit is not taken for one without glibc. Rebuild a tool when the compiler changes. The rebuild check only looks at files, not at directories.
- 4/7: perf can only be built without MINIMAL=1 when pkg-config, libelf and libtraceevent are found for the target, and never with COMPILE_TEST, so a randconfig build does not fail over a missing library.
- 5/7: Keep the vDSO selftest from running "make headers" in the kernel build a second time. Leave the headers the kernel build generates out of the rebuild check, which rebuilt the vDSO and x86 selftests on every build in the source tree. Rebuild the selftests when a makefile under tools/scripts or scripts/subarch.include changes.

Changes in v3:
- Rebase onto v7.4-rc1 and drop the RFC prefix.
- 3/7: The perf-tools-next changes are in mainline now, so MINIMAL=1 sets NO_PYTHON_MODULE instead of NO_LIBPYTHON and also sets NO_MYPY and NO_PYLINT. This is the version that was posted on its own [2]. Add the Acked-by tags from Namhyung and Arnaldo.
- 4/7: Add Namhyung's Acked-by.
- 5/7: Add Shuah's Reviewed-by. Add entries for the selftests new in v7.4 that build with only a C library: filesystems/configfs, filesystems/file_stressor, filesystems/freeze and nfsd.

Changes in v2:
- 1/7: Add Thomas's Reviewed-by.
- 2/7: Put each "depends on" condition on its own line. Allow UML: the tools are built for the architecture UML runs on (HEADER_ARCH), as their build systems do not know ARCH=um. Drop the PYTHONDONTWRITEBYTECODE export, as 5479e1a13b75 ("perf tools: Put Python bytecode in output directory") in perf-tools-next keeps perf's bytecode in its output directory.
- 3/7: New. Add MINIMAL=1 to perf, which turns off every optional feature, as Ian suggested.
- 4/7: Build perf with MINIMAL=1 instead of a list of NO_* variables.
- 5/7: As with "make kselftest-all", a selftest that fails to build no longer fails the build unless FORCE_TARGETS is set, and kselftest_deps.sh lists the libraries it needs that the compiler cannot link with. Also rebuild the selftests when headers under tools/arch change, the vDSO test when kernel headers change, and the x86 tests when the x86 headers they include change.

v3: https://lore.kernel.org/r/[email protected]
v2: https://lore.kernel.org/r/[email protected]
v1: https://lore.kernel.org/r/[email protected]

[1] https://lore.kernel.org/r/[email protected]
[2] https://lore.kernel.org/r/[email protected]

Sasha Levin (7):
kbuild: factor the userspace ABI flags out into KBUILD_USERABI
kbuild: add "make tools_build" to build userspace tools from tools/ perf build: Add MINIMAL=1 to build without optional features
tools: build perf in tools_build
tools: build kernel selftests in tools_build
kbuild: build the tools selected in tools/Kconfig as part of "make
all"
Documentation: describe building tools with the kernel

 .gitignore                                 |   5 +
 Documentation/dev-tools/kselftest.rst      |   4 +
 Documentation/kbuild/kbuild.rst            |   4 +-
 Documentation/process/submit-checklist.rst |   5 +
 Makefile                                   |  24 +-
 lib/Kconfig.debug                          |   2 +
 tools/Kconfig                              | 350 +++++++++++++++++++++
 tools/Makefile.kbuild                      | 219 +++++++++++++
 tools/perf/Makefile.config                 |   8 +-
 tools/perf/Makefile.perf                   |  42 ++-
 tools/perf/tests/make                      |   8 +-
 11 files changed, 653 insertions(+), 18 deletions(-)
 create mode 100644 tools/Kconfig
 create mode 100644 tools/Makefile.kbuild

base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
prerequisite-patch-id: 68eb8785244b88ec7e696e716420e2d9b1367dbc
prerequisite-patch-id: 37c57cf4c2a9700788cd1755912753f363cbb3f9
prerequisite-patch-id: 5243b7b03f2c12a6e66c353b0c083f7f21fd7ba2
prerequisite-patch-id: 7153a847ba7b05e49a670caba5d5b8fa2508eff1
prerequisite-patch-id: 67a995618952454c30eb77960a5bf6f897d22547
prerequisite-patch-id: bc69d99570e4326ce7736b244838e1e97cc13358
prerequisite-patch-id: 7a28872d234cdfa7bd5ec851e9dde01a65e91130
prerequisite-patch-id: a71f47de3fd339d680c9f18e80f5bcde7eae0abf

Signature · 1 line
2.53.0
#1 lore raw

PATCH v4 1/7 kbuild: factor the userspace ABI flags out into KBUILD_USERABI

KBUILD_USERCFLAGS and KBUILD_USERLDFLAGS pick up the flags that select the target ABI for userspace programs: --target= under clang, -m32/-m64, or CONFIG_ARCH_USERFLAGS. Collect those in KBUILD_USERABI and export it, so programs built outside of kbuild's userprogs can be compiled for the same ABI without also inheriting kbuild's warning and language flags.

No functional change: KBUILD_USERCFLAGS and KBUILD_USERLDFLAGS keep the same value.

---
 Makefile | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
Makefile +7 -5
index f895752d4b05f..58bc69b4996e1 100644
@@ -656,7 +656,7 @@ export CPP AR NM STRIP OBJCOPY OBJDUMP READELF PAHOLE RESOLVE_BTFIDS LEX YACC AW
 export PERL PYTHON3 CHECK CHECKFLAGS MAKE UTS_MACHINE HOSTCXX
 export KGZIP KPGZIP KBZIP2 KLZOP LZMA LZ4 XZ ZSTD TAR
 export KBUILD_HOSTCXXFLAGS KBUILD_HOSTLDFLAGS KBUILD_HOSTLDLIBS KBUILD_PROCMACROLDFLAGS LDFLAGS_MODULE
-export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS
+export KBUILD_USERCFLAGS KBUILD_USERLDFLAGS KBUILD_USERABI
 
 export KBUILD_CPPFLAGS NOSTDINC_FLAGS LINUXINCLUDE OBJCOPYFLAGS KBUILD_LDFLAGS
 export KBUILD_CFLAGS CFLAGS_KERNEL CFLAGS_MODULE
@@ -1262,15 +1262,17 @@ endif
 USERFLAGS_FROM_KERNEL := --target=%
 
 ifdef CONFIG_ARCH_USERFLAGS
-KBUILD_USERCFLAGS += $(CONFIG_ARCH_USERFLAGS)
-KBUILD_USERLDFLAGS += $(CONFIG_ARCH_USERFLAGS)
+KBUILD_USERABI := $(CONFIG_ARCH_USERFLAGS)
 else
+KBUILD_USERABI :=
 # If not overridden also inherit the bit size
 USERFLAGS_FROM_KERNEL += -m32 -m64
 endif
 
-KBUILD_USERCFLAGS  += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
-KBUILD_USERLDFLAGS += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
+KBUILD_USERABI += $(filter $(USERFLAGS_FROM_KERNEL), $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS))
+
+KBUILD_USERCFLAGS  += $(KBUILD_USERABI)
+KBUILD_USERLDFLAGS += $(KBUILD_USERABI)
 
 # userspace programs are linked via the compiler, use the correct linker
 ifdef CONFIG_CC_IS_CLANG
Signature · 1 line
2.53.0
#2 lore raw

Re: [PATCH v4 1/7] kbuild: factor the userspace ABI flags out into KBUILD_USERABI

Sashiko has reviewed this patch and found no issues.
Quoted 10 lines

KBUILD_USERCFLAGS and KBUILD_USERLDFLAGS pick up the flags that select the target ABI for userspace programs: --target= under clang, -m32/-m64, or CONFIG_ARCH_USERFLAGS. Collect those in KBUILD_USERABI and export it, so programs built outside of kbuild's userprogs can be compiled for the same ABI without also inheriting kbuild's warning and language flags.

No functional change: KBUILD_USERCFLAGS and KBUILD_USERLDFLAGS keep the same value.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

PATCH v4 2/7 kbuild: add "make tools_build" to build userspace tools from tools/

In reply to Sasha Levin

The programs under tools/ have their own build systems and are not built by "make", so nothing builds them together with the kernel they belong to, and breakage in them is only noticed when someone happens to build them by hand, sometimes many stable releases later.

Add a "Build userspace tools from tools/" menu (CONFIG_TOOLS_BUILD) with an entry for each tool, and a "tools_build" target that builds the selected ones for the target:

- Each tool is built by its own build system, as it is built standalone, into tools-build/ in the object tree. "make clean" removes that directory without running the tools' own clean rules.

- The compiler is the one CONFIG_CC_CAN_LINK was tested with: $(CC), USERCFLAGS and KBUILD_USERABI, so cross builds, including LLVM=1 ARCH=<arch> without CROSS_COMPILE, produce target binaries. USERLDFLAGS reach the tools as LDFLAGS.

- kbuild's own state (srctree, objtree, VPATH, sub_make_done, ...) and the user's CFLAGS/LDFLAGS/O= from the environment or the command line are kept away from the tools.

- Under UML, ARCH and SRCARCH are set to the architecture UML runs on (HEADER_ARCH): a UML kernel runs that architecture's programs, and the tools' build systems do not know ARCH=um.

- A tool is rebuilt from scratch when its make arguments or the compiler change. "make -n" shows the build without removing or recording anything.

- On failure, the message names the option that skips the tool.

The menu depends on CC_CAN_LINK, so toolchains without a C library are unaffected, and is limited to glibc toolchains for x86_64 and arm64, which is what it has been tested with.

The first entries are the tools that only need a C library: bootconfig, counter, dma, gpio, hv, iio, spi and usb.

---
 .gitignore            |  5 +++
 Makefile              |  8 +++-
 lib/Kconfig.debug     |  2 +
 tools/Kconfig         | 73 +++++++++++++++++++++++++++++++++++++
 tools/Makefile.kbuild | 85 +++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 172 insertions(+), 1 deletion(-)
 create mode 100644 tools/Kconfig
 create mode 100644 tools/Makefile.kbuild
.gitignore +5 -0
index f3efefe2db77a..f2f0ccc39dd7b 100644
@@ -102,6 +102,11 @@ modules.order
 #
 /tar-install/
 
+#
+# userspace tools (make tools_build)
+#
+/tools-build/
+
 #
 # pacman files (make pacman-pkg)
 #
Makefile +7 -1
index 58bc69b4996e1..7919263484f0a 100644
@@ -1645,6 +1645,11 @@ tools/%: FORCE
 	$(Q)mkdir -p $(objtree)/tools
 	$(Q)$(MAKE) O=$(abspath $(objtree)) subdir=tools -C $(srctree)/tools/ $*
 
+# The userspace programs selected in tools/Kconfig, built for the target
+PHONY += tools_build
+tools_build: headers
+	$(Q)$(MAKE) -f $(srctree)/tools/Makefile.kbuild
+
 # ---------------------------------------------------------------------------
 # Kernel selftest
 
@@ -1785,7 +1790,7 @@ CLEAN_FILES += vmlinux.symvers modules-only.symvers \
 	       modules.builtin.ranges vmlinux.o.map vmlinux.unstripped \
 	       vmlinux.BTF.inline \
 	       vmlinux.thinlto-index builtin.order \
-	       compile_commands.json rust/test \
+	       compile_commands.json rust/test tools-build \
 	       rust-project.json .vmlinux.objs .vmlinux.export.c \
 	       .builtin-dtbs-list .builtin-dtbs.S sbom-*.spdx.json
 
@@ -1909,6 +1914,7 @@ help:
 	@echo  '  clang-tidy        - Check with clang-tidy'
 	@echo  ''
 	@echo  'Tools:'
+	@echo  '  tools_build     - Build the userspace tools selected in tools/Kconfig'
 	@echo  '  nsdeps          - Generate missing symbol namespace dependencies'
 	@echo  '  sbom            - Generate Software Bill of Materials'
 	@echo  ''
lib/Kconfig.debug +2 -0
index 2d22638ec13a6..cffc089bdb964 100644
@@ -1945,6 +1945,8 @@ config PROVIDE_OHCI1394_DMA_INIT
 
 source "samples/Kconfig"
 
+source "tools/Kconfig"
+
 config ARCH_HAS_DEVMEM_IS_ALLOWED
 	bool
 
tools/Kconfig new file +73 -0

73 lines not on this page · raw message

tools/Makefile.kbuild new file +85 -0

85 lines not on this page · raw message

Signature · 1 line
2.53.0
#4 lore raw

Re: [PATCH v4 2/7] kbuild: add "make tools_build" to build userspace tools from tools/

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Low] kbuild: USERLDFLAGS in LDFLAGS is ignored when building tools/dma --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Low] kbuild: USERLDFLAGS in LDFLAGS is ignored when building tools/dma
--

commit d48c343f29bb15ff9a8fc256221e365cbc49ffb2
Author: Sasha Levin <[email protected]>

kbuild: add "make tools_build" to build userspace tools from tools/

This commit adds a CONFIG_TOOLS_BUILD menu and a tools_build make target to build selected userspace programs under tools/ into tools-build/ in the object tree.

Quoted 5 lines
tools/Makefile.kbuild new file +0 -0
new file mode 100644
index 0000000000000..7a26e7c3d5769

[ ... ]

Quoted 7 lines
tools/Makefile.kbuild new file +7 -0
+# The compiler CONFIG_CC_CAN_LINK was tested with. The tools take linker
+# flags from LDFLAGS.
+tools-cc := $(CC) $(USERCFLAGS) $(KBUILD_USERABI)
+ifneq ($(USERLDFLAGS),)
+override LDFLAGS := $(USERLDFLAGS)
+export LDFLAGS
+endif

[ ... ]

Quoted 1 line

+tools-$(CONFIG_TOOLS_BUILD_DMA) += dma

[Severity: Low]
Does building tools/dma drop USERLDFLAGS?

tools/Makefile.kbuild passes USERLDFLAGS only through LDFLAGS, but tools/dma/Makefile compiles and links dma_map_benchmark in one step without referencing $(LDFLAGS):

tools/dma/Makefile:

	$(OUTPUT)$(DMA_MAP_BENCHMARK): prepare FORCE
		$(CC) $(CFLAGS) $(DMA_MAP_BENCHMARK).c -o $@

When USERLDFLAGS is needed to link target userspace binaries (such as --sysroot or -fuse-ld=lld validated by CONFIG_CC_CAN_LINK), building tools/dma with CONFIG_TOOLS_BUILD_DMA=y will ignore those flags and can fail to link.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
#5 lore raw

PATCH v4 3/7 perf build: Add MINIMAL=1 to build without optional features

In reply to Sasha Levin

Turning off every optional feature takes a long list of NO_* variables, and the list has to follow features as they are added and renamed. The one in tests/make for make_minimal has fallen behind: it leaves libtraceevent, jevents, slang, AIO, LZMA, zlib, libLLVM and Rust to be detected, so a make_minimal build still needs libtraceevent and python.

Add MINIMAL=1, which turns off every optional feature, the BPF skeletons, and the shellcheck, mypy and pylint runs. perf then builds, natively or cross, with no library beyond the C library and without python, pkg-config, clang or rustc, and the result does not depend on what is installed on the build machine.

A MINIMAL=1 build also leaves out the notices that post unwind and python support are disabled, which it asked for, and the one about the pkg-config search path for cross builds, as pkg-config is not needed then.

It is handled in Makefile.perf rather than Makefile.config, as Makefile.config is included after the pkg-config, shellcheck, mypy and pylint setup.

Use it for make_minimal in tests/make.

---
 tools/perf/Makefile.config |  8 ++++++--
 tools/perf/Makefile.perf   | 42 ++++++++++++++++++++++++++++++++++++--
 tools/perf/tests/make      |  8 +-------
 3 files changed, 47 insertions(+), 11 deletions(-)
tools/perf/Makefile.config +6 -2
index b4764f69f47f4..4af7934887539 100644
@@ -753,7 +753,9 @@ dwarf-post-unwind-text := BUG
 # setup DWARF post unwinder
 ifdef NO_LIBUNWIND
   ifdef NO_LIBDW
-    $(warning Disabling post unwind, no support found.)
+    ifneq ($(MINIMAL),1)
+      $(warning Disabling post unwind, no support found.)
+    endif
     dwarf-post-unwind := 0
   else
     dwarf-post-unwind-text := libdw
@@ -828,7 +830,9 @@ endif
 
 PYTHON_EXTENSION_SUFFIX := '.so'
 ifdef NO_PYTHON_MODULE
-  $(call disable-python,Python support disabled by user)
+  ifneq ($(MINIMAL),1)
+    $(call disable-python,Python support disabled by user)
+  endif
 else
 
   ifndef PYTHON
tools/perf/Makefile.perf +40 -2

60 lines not on this page · raw message

tools/perf/tests/make +1 -7
index 7524de5228029..bef5599b95d6f 100644
@@ -119,13 +119,7 @@ make_install_prefix       := install prefix=/tmp/krava
 make_install_prefix_slash := install prefix=/tmp/krava/
 make_static         := LDFLAGS=-static NO_PERF_READ_VDSO32=1 NO_PERF_READ_VDSOX32=1 NO_JVMTI=1 NO_LIBTRACEEVENT=1 NO_LIBELF=1
 
-# all the NO_* variable combined
-make_minimal        := NO_PYTHON_MODULE=1
-make_minimal        += NO_DEMANGLE=1 NO_LIBELF=1 NO_BACKTRACE=1
-make_minimal        += NO_LIBNUMA=1 NO_LIBBIONIC=1 NO_LIBDW=1
-make_minimal        += NO_LIBBPF=1
-make_minimal        += NO_SDT=1 NO_JVMTI=1 NO_LIBZSTD=1
-make_minimal        += NO_CAPSTONE=1
+make_minimal        := MINIMAL=1
 
 # binutils 2_42 and newer have bfd_thread_init()
 new_libbfd := $(shell echo '#include <bfd.h>' | $(CC) -E -x c - | grep bfd_thread_init)
Signature · 1 line
2.53.0
#6 lore raw

Re: [PATCH v4 3/7] perf build: Add MINIMAL=1 to build without optional features

Sashiko has reviewed this patch and found no issues.
Quoted 21 lines

Turning off every optional feature takes a long list of NO_* variables, and the list has to follow features as they are added and renamed. The one in tests/make for make_minimal has fallen behind: it leaves libtraceevent, jevents, slang, AIO, LZMA, zlib, libLLVM and Rust to be detected, so a make_minimal build still needs libtraceevent and python.

Add MINIMAL=1, which turns off every optional feature, the BPF skeletons, and the shellcheck, mypy and pylint runs. perf then builds, natively or cross, with no library beyond the C library and without python, pkg-config, clang or rustc, and the result does not depend on what is installed on the build machine.

A MINIMAL=1 build also leaves out the notices that post unwind and python support are disabled, which it asked for, and the one about the pkg-config search path for cross builds, as pkg-config is not needed then.

It is handled in Makefile.perf rather than Makefile.config, as Makefile.config is included after the pkg-config, shellcheck, mypy and pylint setup.

Quoted 5 lines

Use it for make_minimal in tests/make.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
#7 lore raw

PATCH v4 4/7 tools: build perf in tools_build

In reply to Sasha Levin

Add CONFIG_TOOLS_BUILD_PERF. perf is built as it is built standalone, with optional features enabled when their libraries are installed, and with perf's WERROR setting following CONFIG_WERROR. The feature table is left out of the kernel build log.

CONFIG_TOOLS_BUILD_PERF_MINIMAL builds it with MINIMAL=1 instead, which turns every optional feature off, so the result does not depend on the build machine. Without MINIMAL=1 perf does not build when pkg-config, libelf or libtraceevent is missing, so the option can only be turned off when Kconfig finds all three for the target, and never with CONFIG_COMPILE_TEST. A build test, randconfig included, then does not fail over a library the machine does not have.

---
 tools/Kconfig         | 24 ++++++++++++++++++++++++
 tools/Makefile.kbuild | 10 ++++++++++
 2 files changed, 34 insertions(+)
tools/Kconfig +24 -0
index 04896e6b0ca51..ab1e15beb9fe3 100644
@@ -60,6 +60,30 @@ config TOOLS_BUILD_IIO
 	help
 	  Build tools/iio (lsiio, iio_event_monitor, iio_generic_buffer).
 
+config TOOLS_BUILD_PERF
+	bool "perf"
+	help
+	  Build tools/perf. As in a standalone perf build, optional
+	  features are enabled when the libraries they need are installed.
+
+# What perf does not build without, unless it is built with MINIMAL=1
+config TOOLS_BUILD_PERF_LIBS
+	def_bool $(success,command -v $(CROSS_COMPILE)pkg-config || command -v pkg-config) && \
+		 $(success,echo 'int main(void) { return 0; }' | $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag) `$(CROSS_COMPILE)pkg-config --cflags --libs libtraceevent 2>/dev/null || pkg-config --cflags --libs libtraceevent` -include libelf.h -include trace-seq.h -x c - -o /dev/null -lelf)
+
+config TOOLS_BUILD_PERF_MINIMAL
+	bool "Build perf with all optional features disabled" if TOOLS_BUILD_PERF_LIBS && !COMPILE_TEST
+	depends on TOOLS_BUILD_PERF
+	default !TOOLS_BUILD_PERF_LIBS || COMPILE_TEST
+	help
+	  Build perf with MINIMAL=1, which turns off every optional
+	  feature, so that the result does not depend on which libraries
+	  the build machine has installed. This is always on when
+	  pkg-config, libelf or libtraceevent is missing for the target,
+	  as perf does not build without them otherwise, and with
+	  COMPILE_TEST, which keeps build tests such as allmodconfig the
+	  same on every machine.
+
 config TOOLS_BUILD_SPI
 	bool "SPI tools"
 	help
tools/Makefile.kbuild +10 -0
index 7a26e7c3d5769..5cf8441989fac 100644
@@ -79,6 +79,16 @@ PHONY += $(tool-targets)
 $(tool-targets): tool-%: FORCE
 	$(call build,$(outdir)/$*,-C $(toolsdir)/$* O=$(outdir)/$* CC='$(call escsq,$(tools-cc))',,TOOLS_BUILD_$(shell echo $* | tr a-z A-Z))
 
+# perf, without the feature table in the kernel build log
+perf-flags := CC='$(call escsq,$(tools-cc))' FEATURE_DISPLAY= WERROR=$(if $(CONFIG_WERROR),1,0)
+perf-flags-$(CONFIG_TOOLS_BUILD_PERF_MINIMAL) += MINIMAL=1
+
+__tools_build: $(if $(CONFIG_TOOLS_BUILD_PERF),perf)
+
+PHONY += perf
+perf: FORCE
+	$(call build,$(outdir)/perf,-C $(toolsdir)/perf -f Makefile.perf O=$(outdir)/perf $(perf-flags) $(perf-flags-y),,TOOLS_BUILD_PERF)
+
 PHONY += FORCE
 FORCE:
 
Signature · 1 line
2.53.0
#8 lore raw

Re: [PATCH v4 4/7] tools: build perf in tools_build

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider: - [Medium] tools: missing python-config and Python 3.9 check in TOOLS_BUILD_PERF_LIBS - [Medium] tools…

Thank you for your contribution! Sashiko AI review found 2 potential issues to consider:
- [Medium] tools: missing python-config and Python 3.9 check in TOOLS_BUILD_PERF_LIBS
- [Medium] tools: TOOLS_BUILD_PERF_LIBS probe mismatches tools/perf pkg-config and link checks
--

commit c19562a2a39be79f326367f65fd8b1eb86f840c4
Author: Sasha Levin <[email protected]>

tools: build perf in tools_build

This commit adds CONFIG_TOOLS_BUILD_PERF and CONFIG_TOOLS_BUILD_PERF_MINIMAL to build tools/perf as part of tools_build.

Quoted 8 lines
tools/Kconfig +0 -0
index 04896e6b0ca51..ab1e15beb9fe3 100644
@@ -60,6 +60,30 @@ config TOOLS_BUILD_IIO
 	help
 	  Build tools/iio (lsiio, iio_event_monitor, iio_generic_buffer).
 

tools/Kconfig config TOOLS_BUILD_IIO · line 72

Quoted 10 lines
tools/Kconfig +10 -0
+config TOOLS_BUILD_PERF
+	bool "perf"
+	help
+	  Build tools/perf. As in a standalone perf build, optional
+	  features are enabled when the libraries they need are installed.
+
+# What perf does not build without, unless it is built with MINIMAL=1
+config TOOLS_BUILD_PERF_LIBS
+	def_bool $(success,command -v $(CROSS_COMPILE)pkg-config || command -v pkg-config) && \
+		 $(success,echo 'int main(void) { return 0; }' | $(CC) $(CLANG_FLAGS) $(USERCFLAGS) $(USERLDFLAGS) $(m64-flag) `$(CROSS_COMPILE)pkg-config --cflags --libs libtraceevent 2>/dev/null || pkg-config --cflags --libs libtraceevent` -include libelf.h -include trace-seq.h -x c - -o /dev/null -lelf)

[Severity: Medium]
Can TOOLS_BUILD_PERF_LIBS evaluate to y when tools/perf feature detection for libtraceevent or libelf will fail?

During a cross-build where $(CROSS_COMPILE)pkg-config is installed on the host but target libtraceevent.pc is missing, $(CROSS_COMPILE)pkg-config fails and the || fallback runs the host pkg-config, returning host flags, whereas tools/perf/Makefile.perf uses $(CROSS_COMPILE)pkg-config exclusively when present. Conversely, when $(CROSS_COMPILE)pkg-config is absent, tools/perf/Makefile.perf exports PKG_CONFIG_LIBDIR for the target multiarch directories, while the Kconfig check runs host pkg-config with no PKG_CONFIG_LIBDIR set.

In addition, because pkg-config runs inside backticks without an explicit -ltraceevent on the compiler command line, a failing pkg-config expands to an empty string and still succeeds if trace-seq.h is in the include path. Passing --libs before -x c - and compiling an empty main() without calling symbols such as elf_begin() or trace_seq_init() also succeeds with static archives even when transitive dependencies required by tools/build/feature/ checks are missing.

tools/Kconfig config TOOLS_BUILD_IIO · line 77

Quoted 5 lines
tools/Kconfig +5 -0
+
+config TOOLS_BUILD_PERF_MINIMAL
+	bool "Build perf with all optional features disabled" if TOOLS_BUILD_PERF_LIBS && !COMPILE_TEST
+	depends on TOOLS_BUILD_PERF
+	default !TOOLS_BUILD_PERF_LIBS || COMPILE_TEST

[Severity: Medium]
Does building perf with CONFIG_TOOLS_BUILD_PERF_MINIMAL=n fail when python3-config is missing or Python is older than 3.9?

Without MINIMAL=1 (which sets NO_JEVENTS=1), x86_64 and arm64 perf builds enable jevents generation:

tools/perf/Makefile.config:

    ...
    override PYTHON_CONFIG := $(call get-executable-or-default,PYTHON_CONFIG,$(PYTHON_AUTO))
    override PYTHON := $(call get-executable-or-default,PYTHON,$(subst -config,,$(PYTHON_CONFIG)))
    ...
    ifneq ($(NO_JEVENTS),1)
      NO_JEVENTS := 0
      ifndef PYTHON
        $(error ERROR: No python interpreter needed for jevents generation. Install python or build with NO_JEVENTS=1.)
      else
        ifneq ($(PYTHON_39_GOOD), 1)
          $(error ERROR: Python interpreter needed for jevents generation too old (older than 3.9). Install a newer python or build with NO_JEVENTS=1.)
        endif
      endif
    endif

Because Makefile.config derives PYTHON from PYTHON_CONFIG (python3-config or python-config, typically packaged in python3-dev / python3-devel rather than the base python3 package) and TOOLS_BUILD_PERF_LIBS does not check for it or Python >= 3.9, TOOLS_BUILD_PERF_MINIMAL defaults to n and the perf build aborts when pkg-config, libelf, and libtraceevent are installed without python3-config.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
#9 lore raw

PATCH v4 5/7 tools: build kernel selftests in tools_build

In reply to Sasha Levin

Add a "Kernel selftests" sub-menu with one entry per selftests directory that builds with only a C library and does not write into the source tree. Each is built as "make kselftest-all" builds it, with KHDR_INCLUDES pointing at the UAPI headers installed in the object tree, but by its own make, so the selected ones build in parallel. Selftests whose Makefile does not use KHDR_INCLUDES still build against the C library's copy of the UAPI headers.

As with "make kselftest-all", a selftest that fails to build does not fail the build unless FORCE_TARGETS is set. kselftest_deps.sh then lists the libraries the selftest needs that the compiler cannot link with.

The selftests do not track header dependencies, so a selftest is built again from scratch when its own directory, a header or makefile any selftest may share, or the kernel sources and headers it includes (the vDSO test includes the architecture's vDSO code) are newer than its last build.

Selftests that need libraries beyond the C library, a vmlinux with BTF, or a sanitizer runtime are left out for now.

---
 tools/Kconfig         | 253 ++++++++++++++++++++++++++++++++++++++++++
 tools/Makefile.kbuild | 132 +++++++++++++++++++++-
 2 files changed, 381 insertions(+), 4 deletions(-)
tools/Kconfig +253 -0
index ab1e15beb9fe3..f97f7efa3c73c 100644
@@ -94,4 +94,257 @@ config TOOLS_BUILD_USB
 	help
 	  Build tools/usb (testusb, ffs-test).
 
+menuconfig TOOLS_BUILD_SELFTESTS
+	bool "Kernel selftests"
+	help
+	  Build the selected targets under tools/testing/selftests, as
+	  "make kselftest-all" would, without running them. As there, a
+	  selftest that fails to build does not fail the build unless
+	  FORCE_TARGETS=1 is passed to make.
+
+if TOOLS_BUILD_SELFTESTS
+
+config TOOLS_BUILD_SELFTESTS_ACCT
+	bool "acct"
+
+config TOOLS_BUILD_SELFTESTS_ALLOC_TAG
+	bool "alloc_tag"
+
+config TOOLS_BUILD_SELFTESTS_BREAKPOINTS
+	bool "breakpoints"
+
+config TOOLS_BUILD_SELFTESTS_CACHESTAT
+	bool "cachestat"
+
+config TOOLS_BUILD_SELFTESTS_CGROUP
+	bool "cgroup"
+
+config TOOLS_BUILD_SELFTESTS_CONNECTOR
+	bool "connector"
+
+config TOOLS_BUILD_SELFTESTS_CORE
+	bool "core"
+
+config TOOLS_BUILD_SELFTESTS_COREDUMP
+	bool "coredump"
+
+config TOOLS_BUILD_SELFTESTS_DAMON
+	bool "damon"
+
+config TOOLS_BUILD_SELFTESTS_DMABUF_HEAPS
+	bool "dmabuf-heaps"
+
+config TOOLS_BUILD_SELFTESTS_DRIVERS_DMA_BUF
+	bool "drivers/dma-buf"
+
+config TOOLS_BUILD_SELFTESTS_DRIVERS_NTSYNC
+	bool "drivers/ntsync"
+
+config TOOLS_BUILD_SELFTESTS_EFIVARFS
+	bool "efivarfs"
+
+config TOOLS_BUILD_SELFTESTS_FILELOCK
+	bool "filelock"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_BINDERFS
+	bool "filesystems/binderfs"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_CONFIGFS
+	bool "filesystems/configfs"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_EPOLL
+	bool "filesystems/epoll"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_EVENTFD
+	bool "filesystems/eventfd"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_FAILFS
+	bool "filesystems/failfs"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_FAT
+	bool "filesystems/fat"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_FILE_STRESSOR
+	bool "filesystems/file_stressor"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_FREEZE
+	bool "filesystems/freeze"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_MNTNS_UNBINDABLE
+	bool "filesystems/mntns_unbindable"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_MOUNT_CYCLE
+	bool "filesystems/mount_cycle"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_NSFS
+	bool "filesystems/nsfs"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_UMOUNT_PROPAGATION
+	bool "filesystems/umount_propagation"
+
+config TOOLS_BUILD_SELFTESTS_FILESYSTEMS_XATTR
+	bool "filesystems/xattr"
+
+config TOOLS_BUILD_SELFTESTS_FIRMWARE
+	bool "firmware"
+
+config TOOLS_BUILD_SELFTESTS_FPU
+	bool "fpu"
+
+config TOOLS_BUILD_SELFTESTS_FTRACE
+	bool "ftrace"
+
+config TOOLS_BUILD_SELFTESTS_GPIO
+	bool "gpio"
+
+config TOOLS_BUILD_SELFTESTS_INTEL_PSTATE
+	bool "intel_pstate"
+	depends on X86
+
+config TOOLS_BUILD_SELFTESTS_IPC
+	bool "ipc"
+
+config TOOLS_BUILD_SELFTESTS_IR
+	bool "ir"
+
+config TOOLS_BUILD_SELFTESTS_KCMP
+	bool "kcmp"
+
+config TOOLS_BUILD_SELFTESTS_KEXEC
+	bool "kexec"
+	depends on X86_64
+
+config TOOLS_BUILD_SELFTESTS_KSELFTEST_HARNESS
+	bool "kselftest_harness"
+
+config TOOLS_BUILD_SELFTESTS_LSM
+	bool "lsm"
+
+config TOOLS_BUILD_SELFTESTS_MEDIA_TESTS
+	bool "media_tests"
+
+config TOOLS_BUILD_SELFTESTS_MEMBARRIER
+	bool "membarrier"
+
+config TOOLS_BUILD_SELFTESTS_MINCORE
+	bool "mincore"
+
+config TOOLS_BUILD_SELFTESTS_MOUNT
+	bool "mount"
+
+config TOOLS_BUILD_SELFTESTS_MOUNT_SETATTR
+	bool "mount_setattr"
+
+config TOOLS_BUILD_SELFTESTS_MOVE_MOUNT_SET_GROUP
+	bool "move_mount_set_group"
+
+config TOOLS_BUILD_SELFTESTS_MSEAL_SYSTEM_MAPPINGS
+	bool "mseal_system_mappings"
+
+config TOOLS_BUILD_SELFTESTS_NCI
+	bool "nci"
+
+config TOOLS_BUILD_SELFTESTS_NET_CAN
+	bool "net/can"
+
+config TOOLS_BUILD_SELFTESTS_NET_FORWARDING
+	bool "net/forwarding"
+
+config TOOLS_BUILD_SELFTESTS_NET_MPTCP
+	bool "net/mptcp"
+
+config TOOLS_BUILD_SELFTESTS_NFSD
+	bool "nfsd"
+
+config TOOLS_BUILD_SELFTESTS_PCI_ENDPOINT
+	bool "pci_endpoint"
+
+config TOOLS_BUILD_SELFTESTS_PERF_EVENTS
+	bool "perf_events"
+
+config TOOLS_BUILD_SELFTESTS_PID_NAMESPACE
+	bool "pid_namespace"
+
+config TOOLS_BUILD_SELFTESTS_PIDFD
+	bool "pidfd"
+
+config TOOLS_BUILD_SELFTESTS_PIPE
+	bool "pipe"
+
+config TOOLS_BUILD_SELFTESTS_PROC
+	bool "proc"
+
+config TOOLS_BUILD_SELFTESTS_PTP
+	bool "ptp"
+
+config TOOLS_BUILD_SELFTESTS_PTRACE
+	bool "ptrace"
+
+config TOOLS_BUILD_SELFTESTS_RESCTRL
+	bool "resctrl"
+	depends on X86
+
+config TOOLS_BUILD_SELFTESTS_RING_BUFFER
+	bool "ring-buffer"
+
+config TOOLS_BUILD_SELFTESTS_RLIMITS
+	bool "rlimits"
+
+config TOOLS_BUILD_SELFTESTS_RSEQ
+	bool "rseq"
+
+config TOOLS_BUILD_SELFTESTS_RTC
+	bool "rtc"
+
+config TOOLS_BUILD_SELFTESTS_SCHED
+	bool "sched"
+
+config TOOLS_BUILD_SELFTESTS_SIGNAL
+	bool "signal"
+
+config TOOLS_BUILD_SELFTESTS_SIZE
+	bool "size"
+
+config TOOLS_BUILD_SELFTESTS_SPLICE
+	bool "splice"
+
+config TOOLS_BUILD_SELFTESTS_SYNC
+	bool "sync"
+
+config TOOLS_BUILD_SELFTESTS_SYSCALL_USER_DISPATCH
+	bool "syscall_user_dispatch"
+
+config TOOLS_BUILD_SELFTESTS_TDX
+	bool "tdx"
+	depends on X86
+
+config TOOLS_BUILD_SELFTESTS_THERMAL_INTEL_POWER_FLOOR
+	bool "thermal/intel/power_floor"
+	depends on X86
+
+config TOOLS_BUILD_SELFTESTS_THERMAL_INTEL_WORKLOAD_HINT
+	bool "thermal/intel/workload_hint"
+	depends on X86
+
+config TOOLS_BUILD_SELFTESTS_TIMENS
+	bool "timens"
+
+config TOOLS_BUILD_SELFTESTS_TMPFS
+	bool "tmpfs"
+
+config TOOLS_BUILD_SELFTESTS_USER_EVENTS
+	bool "user_events"
+
+config TOOLS_BUILD_SELFTESTS_VDSO
+	bool "vDSO"
+
+config TOOLS_BUILD_SELFTESTS_WATCHDOG
+	bool "watchdog"
+
+config TOOLS_BUILD_SELFTESTS_X86
+	bool "x86"
+	depends on X86
+
+endif # TOOLS_BUILD_SELFTESTS
+
 endif # TOOLS_BUILD
tools/Makefile.kbuild +128 -4

155 lines not on this page · raw message

Signature · 1 line
2.53.0
#10 lore raw

Re: [PATCH v4 5/7] tools: build kernel selftests in tools_build

Sashiko has reviewed this patch and found no issues.
Quoted 18 lines

Add a "Kernel selftests" sub-menu with one entry per selftests directory that builds with only a C library and does not write into the source tree. Each is built as "make kselftest-all" builds it, with KHDR_INCLUDES pointing at the UAPI headers installed in the object tree, but by its own make, so the selected ones build in parallel. Selftests whose Makefile does not use KHDR_INCLUDES still build against the C library's copy of the UAPI headers.

As with "make kselftest-all", a selftest that fails to build does not fail the build unless FORCE_TARGETS is set. kselftest_deps.sh then lists the libraries the selftest needs that the compiler cannot link with.

The selftests do not track header dependencies, so a selftest is built again from scratch when its own directory, a header or makefile any selftest may share, or the kernel sources and headers it includes (the vDSO test includes the architecture's vDSO code) are newer than its last build.

Quoted 4 lines

Selftests that need libraries beyond the C library, a vmlinux with BTF, or a sanitizer runtime are left out for now.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
#11 lore raw

PATCH v4 6/7 kbuild: build the tools selected in tools/Kconfig as part of "make all"

In reply to Sasha Levin

With CONFIG_TOOLS_BUILD set, make "all" depend on tools_build, so the selected tools are built with every kernel build, and allmodconfig and allyesconfig builds catch changes that break them.

"make vmlinux", "make modules", single target and external module builds are unaffected.

---
 Makefile | 4 ++++
 1 file changed, 4 insertions(+)
Makefile +4 -0
index 7919263484f0a..2a69f4c73a641 100644
@@ -1650,6 +1650,10 @@ PHONY += tools_build
 tools_build: headers
 	$(Q)$(MAKE) -f $(srctree)/tools/Makefile.kbuild
 
+ifdef CONFIG_TOOLS_BUILD
+all: tools_build
+endif
+
 # ---------------------------------------------------------------------------
 # Kernel selftest
 
Signature · 1 line
2.53.0
#12 lore raw

Re: [PATCH v4 6/7] kbuild: build the tools selected in tools/Kconfig as part of "make all"

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] kbuild: make -n all fails in tools_build when CONFIG_TOOLS_BUILD_PERF=y --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] kbuild: make -n all fails in tools_build when CONFIG_TOOLS_BUILD_PERF=y
--

commit 0273f2e0753bd56565e4e6fbd57c3d5604e7d261
Author: Sasha Levin <[email protected]>

kbuild: build the tools selected in tools/Kconfig as part of "make all"

This commit makes the "all" target depend on "tools_build" when CONFIG_TOOLS_BUILD is enabled so that selected tools are built during standard kernel builds.

Quoted 5 lines
Makefile +0 -0
index 7919263484f0a..2a69f4c73a641 100644

Makefile PHONY += tools_build · line 1659

Quoted 10 lines
Makefile +4 -0
@@ -1650,6 +1650,10 @@ PHONY += tools_build
 tools_build: headers
 	$(Q)$(MAKE) -f $(srctree)/tools/Makefile.kbuild
 
+ifdef CONFIG_TOOLS_BUILD
+all: tools_build
+endif
+
 # ---------------------------------------------------------------------------
 # Kernel selftest
 

[Severity: Medium]
Does this break "make -n all" on a clean tree when CONFIG_TOOLS_BUILD_PERF is enabled?

Because tools_build and the "build" macro in tools/Makefile.kbuild invoke $(MAKE), GNU Make executes them during a dry run, creating directories under tools-build/ and descending into tools/perf/Makefile.perf:

tools/perf/Makefile.perf:
$(LIBAPI): FORCE | $(LIBAPI_OUTPUT)

	$(Q)$(MAKE) -C $(LIBAPI_DIR) O=$(LIBAPI_OUTPUT) \
		DESTDIR=$(LIBAPI_DESTDIR) prefix= subdir= \
		$@ install_headers

Under "make -n", the $(Q)$(MKDIR) -p $@ recipe for the $(LIBAPI_OUTPUT) order-only prerequisite is skipped, while the $(LIBAPI) sub-make still runs and aborts in tools/scripts/Makefile.include with "O=.../libapi does not exist".

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
#13 lore raw

PATCH v4 7/7 Documentation: describe building tools with the kernel

In reply to Sasha Levin

Mention CONFIG_TOOLS_BUILD and "make tools_build" in the submit checklist, ask for libc-only selftests to be added to tools/Kconfig, and note that USERCFLAGS and USERLDFLAGS also apply to the tools.

---
 Documentation/dev-tools/kselftest.rst      | 4 ++++
 Documentation/kbuild/kbuild.rst            | 4 +++-
 Documentation/process/submit-checklist.rst | 5 +++++
 3 files changed, 12 insertions(+), 1 deletion(-)
Documentation/dev-tools/kselftest.rst +4 -0
index 0772525cbc627..9dc771ee185fb 100644
@@ -371,6 +371,10 @@ Contributing new tests (details)
 
     TARGETS += android
 
+ * If the test builds with only a C library, add an entry for it to the
+   "Kernel selftests" menu in tools/Kconfig and to tools/Makefile.kbuild,
+   so that it is built together with the kernel.
+
  * All changes should pass::
 
     kselftest-{all,install,clean,gen_tar}
Documentation/kbuild/kbuild.rst +3 -1
index e71a692a559ac..ea48a11b513e7 100644
@@ -131,12 +131,14 @@ of the same name.
 
 USERCFLAGS
 ----------
-Additional options used for $(CC) when compiling userprogs.
+Additional options used for $(CC) when compiling userprogs and the tools
+selected in tools/Kconfig.
 
 USERLDFLAGS
 -----------
 Additional options used for $(LD) when linking userprogs. userprogs are linked
 with CC, so $(USERLDFLAGS) should include "-Wl," prefix as applicable.
+The tools selected in tools/Kconfig get these options in LDFLAGS.
 
 KBUILD_KCONFIG
 --------------
Documentation/process/submit-checklist.rst +5 -0
index beb7f94279fdb..9857b05824c2d 100644
@@ -91,6 +91,11 @@ Build your code
      Use ``make htmldocs`` or ``make pdfdocs`` to check the build and
      fix any issues.
 
+  e) Changes to tools/ or to UAPI headers do not break the userspace tools
+     selected by ``CONFIG_TOOLS_BUILD``. ``allmodconfig`` turns it on when
+     the compiler can link glibc programs for the target (currently x86_64
+     and arm64), and ``make tools_build`` builds only the tools.
+
 2) Builds on multiple CPU architectures by using local cross-compile tools
    or some other build farm.
    Note that testing against architectures of different word sizes
Signature · 1 line
2.53.0
#14 lore raw

Re: [PATCH v4 7/7] Documentation: describe building tools with the kernel

Sashiko has reviewed this patch and found no issues.
Quoted 5 lines

Mention CONFIG_TOOLS_BUILD and "make tools_build" in the submit checklist, ask for libc-only selftests to be added to tools/Kconfig, and note that USERCFLAGS and USERLDFLAGS also apply to the tools.

Sashiko has reviewed this patch and found no issues.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=7
maintainer
#15 lore raw
In reply to Sasha Levin
Quoted 10 lines · On Thu, 12 Nov 2026 14:54:20 -0500 Sasha Levin <[email protected]> wrote:

The userspace programs under tools/ are disconnected from the rest of the kernel's build system. Each has its own makefiles and build framework, and "make" never builds them: apart from the few host tools the build itself needs, such as objtool and resolve_btfids, nothing under tools/ is compiled as part of a kernel build.

That makes them easy to break. Most people, and most of the build bots, check a change with "make allmodconfig && make", which doesn't cover tools/ at all. A change that breaks a tool passes every build check and is only noticed when someone happens to build that tool by hand, often much later.

ooh.

A pet peeve is CONFIG_TRIM_UNUSED_KSYMS. It zaps exports which aren't used by the current module set.

So the 20-odd selftests which build modules and then modprobe them can find that some of the exports which they need were zapped by CONFIG_TRIM_UNUSED_KSYMS, so their modprobe fails. This happens if the current module set happens to not use certain exports which the selftests need.

CONFIG_TRIM_UNUSED_KSYMS has a whitelist dont-zap-this thing but I don't know how to populate that from the 20-odd selftests modules.

From a high level, I think all this happens because selftests are considered "out of tree" for kbuild purposes.

Does this patchset fix this CONFIG_TRIM_UNUSED_KSYMS-vs-sefltests thing?

#16 lore raw
Quoted 3 lines · Andrew Morton

So the 20-odd selftests which build modules and then modprobe them can find that some of the exports which they need were zapped by CONFIG_TRIM_UNUSED_KSYMS, so their modprobe fails.

[...]

Quoted 1 line

Does this patchset fix this CONFIG_TRIM_UNUSED_KSYMS-vs-sefltests thing?

No, not yet, but almost :)

This only builds userspace, and the selftests that build their own modules (bpf, livepatch, mm, net/bench, sched_ext) aren't there quite yet.

It does get us closer though. With this in, a selftest is a Kconfig entry, so the kernel build knows which ones it's going to build. For the ones with modules we could then either build the modules together with the kernel's, so modpost sees what they import, or generate the whitelist from them.

I haven't tried either yet. I can look at it once this foundation is in.

Signature · 2 lines
Thanks,
Sasha
#17 lore raw
In reply to Sasha Levin

Hi Nathan, Nicolas,

Will you be picking this up through the kbuild tree?

3/7 has acks from Arnaldo and Namhyung, 4/7 from Namhyung, and Shuah and Thomas reviewed 5/7 and 1/7. What's left without a review is the kbuild
side: 2/7, 6/7 and 7/7.

If you'd rather see it go in some other way, or want something done differently, let me know.

Signature · 2 lines
Thanks,
Sasha
#18 lore raw

Hi Sasha,

Quoted 8 lines · Sasha Levin

Will you be picking this up through the kbuild tree?

3/7 has acks from Arnaldo and Namhyung, 4/7 from Namhyung, and Shuah and Thomas reviewed 5/7 and 1/7. What's left without a review is the kbuild
side: 2/7, 6/7 and 7/7.

If you'd rather see it go in some other way, or want something done differently, let me know.

I intend to take a look at this either this week or next. I assume we will want to take this via the Kbuild if/when we are happy with the implementation but I cannot make any guarantee. I totally empathize with the reasons for wanting this listed in the commit message but it is a big commitment on both Kbuild's part and other maintainers to integrate tools/ into the build in this manner.

Signature · 2 lines
Cheers,
Nathan