Skip to contentSkip to conversation

kbuild: reduce single-build dependencies

PATCH Superseded Vincent Mailhol 2 participants 5 messages Nov 11 – Nov 12 Kbuild

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

cover kbuild: reduce single-build dependencies

Building a single .o file depends on the prepare target which contains a lot of dependencies. Some of those preparation dependencies are not needed for a single .o build.

This is a small annoyance because a modification of the compilation flags causes several prepare dependencies to rebuild. Consequently, single-build incurs an artificial penalty.

Move the generic preparation checks and scripts/mod out of the single-build preparation path. The first patch is a preparation change to make rq-offsets-file an explicit prepare dependency. The second addresses missing-syscalls check and atomic-checks. The third and last patch addresses scripts/mod.

== Illustration ==

These are the build steps triggered when changing the compilation flags (here: adding W=1).

Before this series...

  $ time make W=1 init/main.o
    CC      scripts/mod/empty.o
    MKELF   scripts/mod/elfconfig.h
    HOSTCC  scripts/mod/modpost.o
    CC      scripts/mod/devicetable-offsets.s
    HOSTCC  scripts/mod/file2alias.o
    HOSTCC  scripts/mod/sumversion.o
    HOSTCC  scripts/mod/symsearch.o
    HOSTLD  scripts/mod/modpost
    CC      kernel/bounds.s
    CC      arch/x86/kernel/asm-offsets.s
    CC      kernel/sched/rq-offsets.s
    CALL    scripts/checksyscalls.sh 
    DESCEND objtool
    CC      init/main.o

real 0m3.603s
user 0m2.839s
sys 0m0.959s

... and after:

  $ time make W=1 init/main.o
    CC      kernel/bounds.s
    CC      arch/x86/kernel/asm-offsets.s
    CC      kernel/sched/rq-offsets.s
    DESCEND objtool
    CC      init/main.o

real 0m2.060s
user 0m1.505s
sys 0m0.732s

(done on an x86_64 cpu).

---
Vincent Mailhol (3):
kbuild: make rq-offsets-file an explicit preparation dependency
kbuild: move preparation checks out of single-build
kbuild: move scripts/mod target out of single-build

 Kbuild   |  7 +++++--
 Makefile | 11 +++++++----
 2 files changed, 12 insertions(+), 6 deletions(-)

---
base-commit: 6628e7d4c6b25c8d18f1345e3a20d21829aae7e4
change-id: 20261111-reduce-single-build-dependencies-966646f25a0c

Best regards,

Signature · 1 line
Vincent Mailhol <[email protected]>
#1 lore raw

PATCH 1/3 kbuild: make rq-offsets-file an explicit preparation dependency

The generated rq-offsets.h header is included by linux/sched.h and must exist before compiling ordinary objects. Currently that header is prepared indirectly through the missing-syscalls target.

Make prepare depend directly on rq-offsets-file. Because rq-offsets-file depends on offsets-file, that other dependency can be removed.

Keep the rq-offsets-file dependency in missing-syscalls so that the check can still run independently.

---
 Kbuild | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Kbuild +1 -1
index a6a0192dea08..5a65be2d51b0 100644
@@ -87,7 +87,7 @@ $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/%  FORCE
 # A phony target that depends on all the preparation targets
 
 PHONY += prepare
-prepare: $(offsets-file) missing-syscalls $(atomic-checks)
+prepare: $(rq-offsets-file) missing-syscalls $(atomic-checks)
 	@:
 
 # Ordinary directory descending
Signature · 1 line
2.56.1
#2 lore raw

PATCH 2/3 kbuild: move preparation checks out of single-build

In reply to Vincent Mailhol

The missing-syscalls check and the atomic-checks are currently set as preparation dependencies. However, these are not needed to compile, for example, a single object.

Add a new prepare_checks target to group those checks and call it conditionally only if we are not doing a single-build so that the prepare target does not trigger that dependency anymore.

---
 Kbuild   | 7 +++++--
 Makefile | 2 +-
 2 files changed, 6 insertions(+), 3 deletions(-)
Kbuild +5 -2
index 5a65be2d51b0..5151e32985c6 100644
@@ -86,8 +86,11 @@ $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/%  FORCE
 
 # A phony target that depends on all the preparation targets
 
-PHONY += prepare
-prepare: $(rq-offsets-file) missing-syscalls $(atomic-checks)
+PHONY += prepare prepare_checks
+prepare: $(rq-offsets-file)
+	@:
+
+prepare_checks: missing-syscalls $(atomic-checks)
 	@:
 
 # Ordinary directory descending
Makefile +1 -1
index f895752d4b05..2c1289a44c56 100644
@@ -1413,8 +1413,8 @@ archprepare: outputmakefile archheaders archscripts scripts include/config/kerne
 	include/generated/rustc_cfg remove-stale-files
 
 prepare0: archprepare
+	$(Q)$(MAKE) $(build)=. prepare $(if $(single-build),,prepare_checks)
 	$(Q)$(MAKE) $(build)=scripts/mod
-	$(Q)$(MAKE) $(build)=. prepare
 
 # All the preparing..
 prepare: prepare0
Signature · 1 line
2.56.1
#3 lore raw

PATCH 3/3 kbuild: move scripts/mod target out of single-build

In reply to Vincent Mailhol

The scripts/mod target is currently set as a preparation dependency. However, this is not needed to compile, for example, a single object.

Add a new modpost_prepare target for scripts/mod and make in-tree modpost and modules_prepare depend on it so that the prepare target does not trigger that dependency anymore.

---
 Makefile | 9 ++++++---
 1 file changed, 6 insertions(+), 3 deletions(-)
Makefile +6 -3
index 2c1289a44c56..93236f310f08 100644
@@ -1405,7 +1405,7 @@ scripts: scripts_basic scripts_dtc
 # archprepare is used in arch Makefiles and when processed asm symlink,
 # version.h and scripts_basic is processed / created.
 
-PHONY += prepare archprepare
+PHONY += prepare archprepare modpost_prepare
 
 archprepare: outputmakefile archheaders archscripts scripts include/config/kernel.release \
 	asm-generic $(version_h) include/generated/utsrelease.h \
@@ -1414,6 +1414,8 @@ archprepare: outputmakefile archheaders archscripts scripts include/config/kerne
 
 prepare0: archprepare
 	$(Q)$(MAKE) $(build)=. prepare $(if $(single-build),,prepare_checks)
+
+modpost_prepare: archprepare
 	$(Q)$(MAKE) $(build)=scripts/mod
 
 # All the preparing..
@@ -1765,7 +1767,7 @@ endif
 modules: modules_prepare
 
 # Target to prepare building external modules
-modules_prepare: prepare
+modules_prepare: prepare modpost_prepare
 	$(Q)$(MAKE) $(build)=scripts scripts/module.lds
 
 endif # CONFIG_MODULES
@@ -2189,7 +2191,8 @@ KBUILD_MODULES :=
 endif # CONFIG_MODULES
 
 PHONY += modpost
-modpost: $(if $(single-build),, $(if $(KBUILD_BUILTIN), vmlinux.o)) \
+modpost: $(if $(KBUILD_EXTMOD),,modpost_prepare) \
+	 $(if $(single-build),, $(if $(KBUILD_BUILTIN), vmlinux.o)) \
 	 $(if $(KBUILD_MODULES), modules_check)
 	$(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost
 
Signature · 1 line
2.56.1
#4 lore raw

Re: [PATCH 3/3] kbuild: move scripts/mod target out of single-build

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Low] kbuild: unordered modpost_prepare and stack_protector_prepare mutates CFLAGS --

Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Low] kbuild: unordered modpost_prepare and stack_protector_prepare mutates CFLAGS
--

commit 08ffc13380249741520c2ea434688d701b5d0e9b
Author: Vincent Mailhol <[email protected]>

kbuild: move scripts/mod target out of single-build

This commit moves the scripts/mod build out of prepare0 into a new modpost_prepare target depended on by modules_prepare and modpost.

Quoted 14 lines
Makefile +1 -1
index 2c1289a44c560..93236f310f080 100644
@@ -1405,7 +1405,7 @@ scripts: scripts_basic scripts_dtc
 # archprepare is used in arch Makefiles and when processed asm symlink,
 # version.h and scripts_basic is processed / created.
 
-PHONY += prepare archprepare
+PHONY += prepare archprepare modpost_prepare
 
 archprepare: outputmakefile archheaders archscripts scripts include/config/kernel.release \
 	asm-generic $(version_h) include/generated/utsrelease.h \

Makefile archprepare: outputmakefile archheaders archscripts scripts include/config/kerne · line 1419

Quoted 6 lines
Makefile +2 -0
@@ -1414,6 +1414,8 @@ archprepare: outputmakefile archheaders archscripts scripts include/config/kerne
 
 prepare0: archprepare
 	$(Q)$(MAKE) $(build)=. prepare $(if $(single-build),,prepare_checks)
+
+modpost_prepare: archprepare
 	$(Q)$(MAKE) $(build)=scripts/mod

[Severity: Low]
Should modpost_prepare be ordered relative to stack_protector_prepare (or depend on prepare)?

On arm, arm64, powerpc, and riscv, prepare depends on stack_protector_prepare, which depends on prepare0 and mutates the exported KBUILD_CFLAGS variable at recipe execution time:

arch/arm64/Makefile:
prepare: stack_protector_prepare
stack_protector_prepare: prepare0

	$(eval KBUILD_CFLAGS += -mstack-protector-guard=sysreg		  \
				-mstack-protector-guard-reg=sp_el0	  \
				-mstack-protector-guard-offset=$(shell	  \
			awk '{if ($$2 == "TSK_STACK_CANARY") print $$3;}' \
				$(objtree)/include/generated/asm-offsets.h))

Previously, scripts/mod was built inside prepare0, so it always ran before stack_protector_prepare modified KBUILD_CFLAGS. With modpost_prepare only depending on archprepare, a parallel build runs modpost_prepare concurrently with prepare0 before stack_protector_prepare updates KBUILD_CFLAGS, whereas a serial build or make modules_prepare runs prepare first and passes the updated KBUILD_CFLAGS to scripts/mod.

Because scripts/mod/empty.o is compiled with KBUILD_CFLAGS, switching between parallel and serial or single-module builds changes its recorded command line, rebuilding empty.o, regenerating elfconfig.h (which uses if_changed), and forcing a full rebuild of modpost.

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/20261111-reduce-single-build-dependencies-v1-0-7196ca986a85@kernel.org?part=3