kbuild: support building external modules in a separate build directory
Subsystems (11)
v2 cover kbuild: support building external modules in a separate build directory
There has been a long-standing request to support building external modules in a separate build directory.
In v2, 05/11 introduces a 'srcroot' variable.
(I am not sure if this is the best name, though) $(srcroot) points to the top of the source directory.
- When M= is not given, it is the top of the kernel source
- When M= is given, it is the top of the external module source
Masahiro Yamada (11):
kbuild: replace two $(abs_objtree) with $(CURDIR) in top Makefile
kbuild: add $(objtree)/ prefix to some in-kernel build artifacts
kbuild: rename abs_objtree to abs_output
kbuild: use 'output' variable to create the output directory
kbuild: change working directory to external module directory with M=
kbuild: remove extmod_prefix, MODORDER, MODULES_NSDEPS variables
kbuild: support building external modules in a separate build
directory
kbuild: support -fmacro-prefix-map for external modules
kbuild: use absolute path in the generated wrapper Makefile
kbuild: make wrapper Makefile more convenient for external modules
kbuild: allow to start building external modules in any directory
Documentation/dev-tools/coccinelle.rst | 20 +-- Documentation/kbuild/kbuild.rst | 8 +- Documentation/kbuild/makefiles.rst | 14 ++ Documentation/kbuild/modules.rst | 29 ++++- Makefile | 173 ++++++++++++++++--------- arch/arm/Makefile | 4 +- arch/arm64/Makefile | 2 +- arch/powerpc/Makefile | 4 +- arch/riscv/Makefile | 2 +- rust/Makefile | 4 +- scripts/Kbuild.include | 2 +- scripts/Makefile.build | 6 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.host | 8 +- scripts/Makefile.lib | 2 - scripts/Makefile.modfinal | 20 +-- scripts/Makefile.modinst | 8 +- scripts/Makefile.modpost | 24 ++-- scripts/coccicheck | 6 +- scripts/depmod.sh | 4 +- scripts/nsdeps | 10 +- scripts/package/install-extmod-build | 7 + 23 files changed, 220 insertions(+), 141 deletions(-)
Signature · 1 line
2.43.0
PATCH v2 1/11 kbuild: replace two $(abs_objtree) with $(CURDIR) in top Makefile
Kbuild changes the working directory until it matches $(abs_objtree).
When $(need-sub-make) is empty, $(abs_objtree) is the same as $(CURDIR).
---
(no changes since v1)
Makefile | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Makefile +2 -2
@@ -228,12 +228,12 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(abs_objtree)) +ifeq ($(abs_srctree),$(CURDIR)) # building in the source tree srctree := . building_out_of_srctree := else - ifeq ($(abs_srctree)/,$(dir $(abs_objtree))) + ifeq ($(abs_srctree)/,$(dir $(CURDIR))) # building in a subdirectory of the source tree srctree := .. else
Signature · 1 line
2.43.0
PATCH v2 2/11 kbuild: add $(objtree)/ prefix to some in-kernel build artifacts
In reply to Masahiro Yamada$(objtree) refers to the top of the output directory of kernel builds.
This commit adds the explicit $(objtree)/ prefix to build artifacts needed for building external modules.
This change has no immediate impact, as the top-level Makefile currently defines:
objtree := .
This commit prepares for supporting the building of external modules in a different directory.
---
Changes in v2:
- Add more $(objtree)/ to arch/*/Makefile
Makefile | 10 +++++----- arch/arm/Makefile | 4 ++-- arch/arm64/Makefile | 2 +- arch/powerpc/Makefile | 4 ++-- arch/riscv/Makefile | 2 +- scripts/Kbuild.include | 2 +- scripts/Makefile.build | 4 ++-- scripts/Makefile.modfinal | 14 +++++++------- scripts/Makefile.modinst | 2 +- scripts/Makefile.modpost | 12 ++++++------ scripts/depmod.sh | 4 ++-- 11 files changed, 30 insertions(+), 30 deletions(-)
Makefile +5 -5
@@ -354,7 +354,7 @@ else # !mixed-build include $(srctree)/scripts/Kbuild.include # Read KERNELRELEASE from include/config/kernel.release (if it exists) -KERNELRELEASE = $(call read-file, include/config/kernel.release) +KERNELRELEASE = $(call read-file, $(objtree)/include/config/kernel.release) KERNELVERSION = $(VERSION)$(if $(PATCHLEVEL),.$(PATCHLEVEL)$(if $(SUBLEVEL),.$(SUBLEVEL)))$(EXTRAVERSION) export VERSION PATCHLEVEL SUBLEVEL KERNELRELEASE KERNELVERSION
@@ -720,7 +720,7 @@ endif export KBUILD_MODULES KBUILD_BUILTIN ifdef need-config -include include/config/auto.conf +include $(objtree)/include/config/auto.conf endif ifeq ($(KBUILD_EXTMOD),)
@@ -783,13 +783,13 @@ else # !may-sync-config # and include/config/auto.conf but do not care if they are up-to-date. # Use auto.conf to show the error message -checked-configs := include/generated/autoconf.h include/generated/rustc_cfg include/config/auto.conf +checked-configs := $(addprefix $(objtree)/, include/generated/autoconf.h include/generated/rustc_cfg include/config/auto.conf) missing-configs := $(filter-out $(wildcard $(checked-configs)), $(checked-configs)) ifdef missing-configs -PHONY += include/config/auto.conf +PHONY += $(objtree)/include/config/auto.conf -include/config/auto.conf: +$(objtree)/include/config/auto.conf: @echo >&2 '***' @echo >&2 '*** ERROR: Kernel configuration is invalid. The following files are missing:' @printf >&2 '*** - %s\n' $(missing-configs)
arch/arm/Makefile +2 -2
@@ -264,13 +264,13 @@ stack_protector_prepare: prepare0 -mstack-protector-guard=tls \ -mstack-protector-guard-offset=$(shell \ awk '{if ($$2 == "TSK_STACK_CANARY") print $$3;}'\ - include/generated/asm-offsets.h)) + $(objtree)/include/generated/asm-offsets.h)) else stack_protector_prepare: prepare0 $(eval SSP_PLUGIN_CFLAGS := \ -fplugin-arg-arm_ssp_per_task_plugin-offset=$(shell \ awk '{if ($$2 == "TSK_STACK_CANARY") print $$3;}'\ - include/generated/asm-offsets.h)) + $(objtree)/include/generated/asm-offsets.h)) $(eval KBUILD_CFLAGS += $(SSP_PLUGIN_CFLAGS)) $(eval GCC_PLUGINS_CFLAGS += $(SSP_PLUGIN_CFLAGS)) endif
arch/arm64/Makefile +1 -1
@@ -71,7 +71,7 @@ stack_protector_prepare: prepare0 -mstack-protector-guard-reg=sp_el0 \ -mstack-protector-guard-offset=$(shell \ awk '{if ($$2 == "TSK_STACK_CANARY") print $$3;}' \ - include/generated/asm-offsets.h)) + $(objtree)/include/generated/asm-offsets.h)) endif ifeq ($(CONFIG_ARM64_BTI_KERNEL),y)
arch/powerpc/Makefile +2 -2
@@ -402,9 +402,9 @@ prepare: stack_protector_prepare PHONY += stack_protector_prepare stack_protector_prepare: prepare0 ifdef CONFIG_PPC64 - $(eval KBUILD_CFLAGS += -mstack-protector-guard-offset=$(shell awk '{if ($$2 == "PACA_CANARY") print $$3;}' include/generated/asm-offsets.h)) + $(eval KBUILD_CFLAGS += -mstack-protector-guard-offset=$(shell awk '{if ($$2 == "PACA_CANARY") print $$3;}' $(objtree)/include/generated/asm-offsets.h)) else - $(eval KBUILD_CFLAGS += -mstack-protector-guard-offset=$(shell awk '{if ($$2 == "TASK_CANARY") print $$3;}' include/generated/asm-offsets.h)) + $(eval KBUILD_CFLAGS += -mstack-protector-guard-offset=$(shell awk '{if ($$2 == "TASK_CANARY") print $$3;}' $(objtree)/include/generated/asm-offsets.h)) endif endif
arch/riscv/Makefile +1 -1
@@ -129,7 +129,7 @@ stack_protector_prepare: prepare0 -mstack-protector-guard-reg=tp \ -mstack-protector-guard-offset=$(shell \ awk '{if ($$2 == "TSK_STACK_CANARY") print $$3;}' \ - include/generated/asm-offsets.h)) + $(objtree)/include/generated/asm-offsets.h)) endif # arch specific predefines for sparse
scripts/Kbuild.include +1 -1
@@ -205,7 +205,7 @@ if_changed_dep = $(if $(if-changed-cond),$(cmd_and_fixdep),@:) cmd_and_fixdep = \ $(cmd); \ - scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).cmd;\ + $(objtree)/scripts/basic/fixdep $(depfile) $@ '$(make-cmd)' > $(dot-target).cmd;\ rm -f $(depfile) # Usage: $(call if_changed_rule,foo)
scripts/Makefile.build +2 -2
@@ -34,7 +34,7 @@ subdir-asflags-y := subdir-ccflags-y := # Read auto.conf if it exists, otherwise ignore --include include/config/auto.conf +-include $(objtree)/include/config/auto.conf include $(srctree)/scripts/Kbuild.include include $(srctree)/scripts/Makefile.compiler
@@ -107,7 +107,7 @@ cmd_cpp_i_c = $(CPP) $(c_flags) -o $@ $< $(obj)/%.i: $(obj)/%.c FORCE $(call if_changed_dep,cpp_i_c) -genksyms = scripts/genksyms/genksyms \ +genksyms = $(objtree)/scripts/genksyms/genksyms \ $(if $(1), -T $(2)) \ $(if $(KBUILD_PRESERVE), -p) \ -r $(or $(wildcard $(2:.symtypes=.symref)), /dev/null)
scripts/Makefile.modfinal +7 -7
@@ -6,7 +6,7 @@ PHONY := __modfinal __modfinal: -include include/config/auto.conf +include $(objtree)/include/config/auto.conf include $(srctree)/scripts/Kbuild.include # for c_flags
@@ -37,15 +37,15 @@ quiet_cmd_ld_ko_o = LD [M] $@ cmd_ld_ko_o = \ $(LD) -r $(KBUILD_LDFLAGS) \ $(KBUILD_LDFLAGS_MODULE) $(LDFLAGS_MODULE) \ - -T scripts/module.lds -o $@ $(filter %.o, $^) + -T $(objtree)/scripts/module.lds -o $@ $(filter %.o, $^) quiet_cmd_btf_ko = BTF [M] $@ cmd_btf_ko = \ - if [ ! -f vmlinux ]; then \ + if [ ! -f $(objtree)/vmlinux ]; then \ printf "Skipping BTF generation for %s due to unavailability of vmlinux\n" $@ 1>&2; \ else \ - LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base vmlinux $@; \ - $(RESOLVE_BTFIDS) -b vmlinux $@; \ + LLVM_OBJCOPY="$(OBJCOPY)" $(PAHOLE) -J $(PAHOLE_FLAGS) $(MODULE_PAHOLE_FLAGS) --btf_base $(objtree)/vmlinux $@; \ + $(RESOLVE_BTFIDS) -b $(objtree)/vmlinux $@; \ fi; # Same as newer-prereqs, but allows to exclude specified extra dependencies
@@ -57,8 +57,8 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check), \ printf '%s\n' 'savedcmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:) # Re-generate module BTFs if either module's .ko or vmlinux changed -%.ko: %.o %.mod.o $(extmod_prefix).module-common.o scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),vmlinux) FORCE - +$(call if_changed_except,ld_ko_o,vmlinux) +%.ko: %.o %.mod.o $(extmod_prefix).module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE + +$(call if_changed_except,ld_ko_o,$(objtree)/vmlinux) ifdef CONFIG_DEBUG_INFO_BTF_MODULES +$(if $(newer-prereqs),$(call cmd,btf_ko)) endif
scripts/Makefile.modinst +1 -1
@@ -6,7 +6,7 @@ PHONY := __modinst __modinst: -include include/config/auto.conf +include $(objtree)/include/config/auto.conf include $(srctree)/scripts/Kbuild.include install-y :=
scripts/Makefile.modpost +6 -6
@@ -35,10 +35,10 @@ PHONY := __modpost __modpost: -include include/config/auto.conf +include $(objtree)/include/config/auto.conf include $(srctree)/scripts/Kbuild.include -MODPOST = scripts/mod/modpost +MODPOST = $(objtree)/scripts/mod/modpost modpost-args = \ $(if $(CONFIG_MODULES),-M) \
@@ -119,11 +119,11 @@ include $(kbuild-file) output-symdump := $(KBUILD_EXTMOD)/Module.symvers -ifeq ($(wildcard Module.symvers),) -missing-input := Module.symvers +ifeq ($(wildcard $(objtree)/Module.symvers),) +missing-input := $(objtree)/Module.symvers else -modpost-args += -i Module.symvers -modpost-deps += Module.symvers +modpost-args += -i $(objtree)/Module.symvers +modpost-deps += $(objtree)/Module.symvers endif modpost-args += -e $(addprefix -i , $(KBUILD_EXTRA_SYMBOLS))
scripts/depmod.sh +2 -2
@@ -12,7 +12,7 @@ KERNELRELEASE=$1 : ${DEPMOD:=depmod} -if ! test -r System.map ; then +if ! test -r "${objtree}/System.map" ; then echo "Warning: modules_install: missing 'System.map' file. Skipping depmod." >&2 exit 0 fi
@@ -25,7 +25,7 @@ if [ -z $(command -v $DEPMOD) ]; then exit 0 fi -set -- -ae -F System.map +set -- -ae -F "${objtree}/System.map" if test -n "$INSTALL_MOD_PATH"; then set -- "$@" -b "$INSTALL_MOD_PATH" fi
Signature · 1 line
2.43.0
PATCH v2 3/11 kbuild: rename abs_objtree to abs_output
In reply to Masahiro Yamada'objtree' refers to the top of the output directory of kernel builds.
Rename abs_objtree to a more generic name, to better reflect its use in the context of external module builds.
---
(no changes since v1)
Makefile | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
Makefile +5 -5
@@ -40,7 +40,7 @@ __all: this-makefile := $(lastword $(MAKEFILE_LIST)) abs_srctree := $(realpath $(dir $(this-makefile))) -abs_objtree := $(CURDIR) +abs_output := $(CURDIR) ifneq ($(sub_make_done),1)
@@ -185,8 +185,8 @@ ifneq ($(KBUILD_OUTPUT),) # $(realpath ...) gets empty if the path does not exist. Run 'mkdir -p' first. $(shell mkdir -p "$(KBUILD_OUTPUT)") # $(realpath ...) resolves symlinks -abs_objtree := $(realpath $(KBUILD_OUTPUT)) -$(if $(abs_objtree),,$(error failed to create output directory "$(KBUILD_OUTPUT)")) +abs_output := $(realpath $(KBUILD_OUTPUT)) +$(if $(abs_output),,$(error failed to create output directory "$(KBUILD_OUTPUT)")) endif # ifneq ($(KBUILD_OUTPUT),) ifneq ($(words $(subst :, ,$(abs_srctree))), 1)
@@ -197,7 +197,7 @@ export sub_make_done := 1 endif # sub_make_done -ifeq ($(abs_objtree),$(CURDIR)) +ifeq ($(abs_output),$(CURDIR)) # Suppress "Entering directory ..." if we are at the final work directory. no-print-directory := --no-print-directory else
@@ -221,7 +221,7 @@ $(filter-out $(this-makefile), $(MAKECMDGOALS)) __all: __sub-make # Invoke a second make in the output directory, passing relevant variables __sub-make: - $(Q)$(MAKE) $(no-print-directory) -C $(abs_objtree) \ + $(Q)$(MAKE) $(no-print-directory) -C $(abs_output) \ -f $(abs_srctree)/Makefile $(MAKECMDGOALS) else # need-sub-make
Signature · 1 line
2.43.0
PATCH v2 4/11 kbuild: use 'output' variable to create the output directory
In reply to Masahiro Yamada$(KBUILD_OUTPUT) specifies the output directory of kernel builds.
Use a more generic name, 'output', to better reflect this code hunk in the context of external module builds.
---
(no changes since v1)
Makefile | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-)
Makefile +8 -6
@@ -176,18 +176,20 @@ export KBUILD_EXTRA_WARN # The O= assignment takes precedence over the KBUILD_OUTPUT environment # variable. -# Do we want to change the working directory? ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -ifneq ($(KBUILD_OUTPUT),) +output := $(KBUILD_OUTPUT) + +# Do we want to change the working directory? +ifneq ($(output),) # $(realpath ...) gets empty if the path does not exist. Run 'mkdir -p' first. -$(shell mkdir -p "$(KBUILD_OUTPUT)") +$(shell mkdir -p "$(output)") # $(realpath ...) resolves symlinks -abs_output := $(realpath $(KBUILD_OUTPUT)) -$(if $(abs_output),,$(error failed to create output directory "$(KBUILD_OUTPUT)")) -endif # ifneq ($(KBUILD_OUTPUT),) +abs_output := $(realpath $(output)) +$(if $(abs_output),,$(error failed to create output directory "$(output)")) +endif ifneq ($(words $(subst :, ,$(abs_srctree))), 1) $(error source directory cannot contain spaces or colons)
Signature · 1 line
2.43.0
PATCH v2 5/11 kbuild: change working directory to external module directory with M=
In reply to Masahiro YamadaCurrently, Kbuild always operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.
This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module
make: Entering directory '/path/to/my/linux'
CC [M] /path/to/my/externel/module/helloworld.o
MODPOST /path/to/my/externel/module/Module.symvers
CC [M] /path/to/my/externel/module/helloworld.mod.o
CC [M] /path/to/my/externel/module/.module-common.o
LD [M] /path/to/my/externel/module/helloworld.ko
make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module
make: Entering directory '/path/to/my/linux'
make[1]: Entering directory '/path/to/my/externel/module'
CC [M] helloworld.o
MODPOST Module.symvers
CC [M] helloworld.mod.o
CC [M] .module-common.o
LD [M] helloworld.ko
make[1]: Leaving directory '/path/to/my/externel/module'
make: Leaving directory '/path/to/my/linux'Printing "Entering directory" twice is cumbersome. This will be addressed later.
---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- Reword Documentation/dev-tools/coccinelle.rst
Documentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)
Documentation/dev-tools/coccinelle.rst +6 -14
@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,
Documentation/kbuild/makefiles.rst +14 -0
@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, but + if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. + $(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any output
Makefile +48 -32
132 lines not on this page · raw message
rust/Makefile +2 -2
@@ -362,8 +362,8 @@ rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ - $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ - $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json + $(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \ + > rust-project.json redirect-intrinsics = \ __addsf3 __eqsf2 __extendsfdf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __truncdfsf2 __unordsf2 \
scripts/Makefile.build +1 -1
@@ -3,7 +3,7 @@ # Building # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := $(obj)/ $(obj)/:
scripts/Makefile.clean +1 -1
@@ -3,7 +3,7 @@ # Cleaning up # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := __clean __clean:
scripts/Makefile.compiler +1 -1
@@ -13,7 +13,7 @@ cc-cross-prefix = $(firstword $(foreach c, $(1), \ $(if $(shell command -v -- $(c)gcc 2>/dev/null), $(c)))) # output directory for tests below -TMPOUT = $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/).tmp_$$$$ +TMPOUT = .tmp_$$$$ # try-run # Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)
scripts/Makefile.modpost +3 -3
@@ -111,13 +111,13 @@ endif else # set src + obj - they may be used in the modules's Makefile -obj := $(KBUILD_EXTMOD) -src := $(if $(VPATH),$(VPATH)/)$(obj) +obj := . +src := $(srcroot) # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS include $(kbuild-file) -output-symdump := $(KBUILD_EXTMOD)/Module.symvers +output-symdump := Module.symvers ifeq ($(wildcard $(objtree)/Module.symvers),) missing-input := $(objtree)/Module.symvers
scripts/coccicheck +1 -5
@@ -80,11 +80,7 @@ command results in a shift count error.' NPROC=1 else ONLINE=0 - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" # Use only one thread per core by default if hyperthreading is enabled THREADS_PER_CORE=$(LANG=C lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")
scripts/nsdeps +1 -7
@@ -19,12 +19,6 @@ if ! { echo "$SPATCH_REQ_VERSION"; echo "$SPATCH_VERSION"; } | sort -CV ; then exit 1 fi -if [ "$KBUILD_EXTMOD" ]; then - src_prefix= -else - src_prefix=$srctree/ -fi - generate_deps_for_ns() { $SPATCH --very-quiet --in-place --sp-file \ $srctree/scripts/coccinelle/misc/add_namespace.cocci -D nsdeps -D ns=$1 $2
@@ -34,7 +28,7 @@ generate_deps() { local mod=${1%.ko:} shift local namespaces="$*" - local mod_source_files=$(sed "s|^\(.*\)\.o$|${src_prefix}\1.c|" $mod.mod) + local mod_source_files=$(sed "s|^\(.*\)\.o$|${srcroot}/\1.c|" $mod.mod) for ns in $namespaces; do echo "Adding namespace $ns to module $mod.ko."
scripts/package/install-extmod-build +7 -0
@@ -51,6 +51,13 @@ mkdir -p "${destdir}" if [ "${CC}" != "${HOSTCC}" ]; then echo "Rebuilding host programs with ${CC}..." + # This leverages external module building. + # - Clear sub_make_done to allow the top-level Makefile to redo sub-make. + # - Filter out --no-print-directory to print "Entering directory" logs + # when Make changes the working directory. + unset sub_make_done + MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//) + cat <<-'EOF' > "${destdir}/Kbuild" subdir-y := scripts EOF
Signature · 1 line
2.43.0
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 110 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +6
-14@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,Documentation/kbuild/makefiles.rst +4
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, but
Documentation/kbuild/makefiles.rst $(obj) · line 465
Quoted 10 lines
Documentation/kbuild/makefiles.rst +10
-0+ if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. +
Thanks, I think it's nice that there is now such a clear definition. $(srcroot) sounds fine to me.
Quoted 62 lines
Documentation/kbuild/makefiles.rst +0
-0$(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +27
-14@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute path before + # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory "$(KBUILD_EXTMOD)" does not exist)) +else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := else - ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1 endif -ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path.
Makefile else # need-sub-make · line 270
Quoted 10 lines
Makefile +7
-1+else ifeq ($(srcroot),$(CURDIR)) + # Building in the source. + srcroot := . +else ifeq ($(srcroot)/,$(dir $(CURDIR))) + # Building in a subdirectory of the source. + srcroot := .. endif -objtree := . +export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))
With this patch applied, the following breaks for me:
$ make O=build M=fs/btrfs CONFIG_BTRFS_FS=m
make[1]: Entering directory '/data/linux/kbuild-review/fs/btrfs'
CC [M] super.o
In file included from <command-line>:
/data/linux/kbuild-review/include/linux/compiler_types.h:89:10: fatal error: linux/compiler_attributes.h: No such file or directory
89 | #include <linux/compiler_attributes.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.Adding 'ccflags-y += -I$(srctree)/include' to fs/btrfs/Makefile breaks the build loudly. I could make it build again with
Makefile +1 -1
@@ -555,7 +555,7 @@ USERINCLUDE := \ LINUXINCLUDE := \ -I$(srctree)/arch/$(SRCARCH)/include \ -I$(objtree)/arch/$(SRCARCH)/include/generated \ - $(if $(building_out_of_srctree),-I$(srctree)/include) \ + $(if $(or $(building_out_of_srctree),$(filter $(srctree)/%, $(CURDIR))),-I$(srctree)/include) \ -I$(objtree)/include \ $(USERINCLUDE)
but this does not feel good. It building in-tree modules in this way a valid thing to do?
Kind regards,
Nicolas
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 226 lines · Nicolas Schier
Quoted 110 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +6
-14@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,Documentation/kbuild/makefiles.rst +4
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, butDocumentation/kbuild/makefiles.rst $(obj) · line 465
Quoted 10 lines
Documentation/kbuild/makefiles.rst +10
-0+ if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. +Thanks, I think it's nice that there is now such a clear definition. $(srcroot) sounds fine to me.
Quoted 62 lines
Documentation/kbuild/makefiles.rst +0
-0$(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +27
-14@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute path before + # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory "$(KBUILD_EXTMOD)" does not exist)) +else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := else - ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1 endif -ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path.Makefile else # need-sub-make · line 270
Quoted 10 lines
Makefile +7
-1+else ifeq ($(srcroot),$(CURDIR)) + # Building in the source. + srcroot := . +else ifeq ($(srcroot)/,$(dir $(CURDIR))) + # Building in a subdirectory of the source. + srcroot := .. endif -objtree := . +export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))With this patch applied, the following breaks for me:
$ make O=build M=fs/btrfs CONFIG_BTRFS_FS=m make[1]: Entering directory '/data/linux/kbuild-review/fs/btrfs' CC [M] super.o In file included from <command-line>: /data/linux/kbuild-review/include/linux/compiler_types.h:89:10: fatal error: linux/compiler_attributes.h: No such file or directory 89 | #include <linux/compiler_attributes.h> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.Adding 'ccflags-y += -I$(srctree)/include' to
fs/btrfs/Makefilebreaks the build loudly. I could make it build again withMakefile +1
-1@@ -555,7 +555,7 @@ USERINCLUDE := \ LINUXINCLUDE := \ -I$(srctree)/arch/$(SRCARCH)/include \ -I$(objtree)/arch/$(SRCARCH)/include/generated \ - $(if $(building_out_of_srctree),-I$(srctree)/include) \ + $(if $(or $(building_out_of_srctree),$(filter $(srctree)/%, $(CURDIR))),-I$(srctree)/include) \ -I$(objtree)/include \ $(USERINCLUDE)
Quoted 2 lines
but this does not feel good. It building in-tree modules in this way a valid thing to do?
Yes, it is a valid way.
This got broken with this commit, and fixed by the later commit, "kbuild: support building external modules in a separate build directory".
I will move the change for LINUXINCLUDE to this commit.
Thanks for comprehensive testing!
Signature · 2 lines
Best Regards Masahiro Yamada
Nicolas Schier
Reviewed-byRe: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 175 lines · Masahiro Yamada
Quoted 174 lines · Nicolas Schier
Quoted 110 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +6
-14@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,Documentation/kbuild/makefiles.rst +4
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, butDocumentation/kbuild/makefiles.rst $(obj) · line 465
Quoted 10 lines
Documentation/kbuild/makefiles.rst +10
-0+ if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. +Thanks, I think it's nice that there is now such a clear definition. $(srcroot) sounds fine to me.
Quoted 48 lines
Documentation/kbuild/makefiles.rst +0
-0$(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +24
-5@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute path before + # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory "$(KBUILD_EXTMOD)" does not exist)) +else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := else
Quoted 8 lines
Quoted 8 lines
Makefile else # need-sub-make · line 257
Quoted 8 lines
Makefile +1
-7- ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1
Nit: Adding a space between ':=1' for consistency?
Quoted 6 lines
Quoted 6 lines
Makefile else # need-sub-make · line 261
Quoted 6 lines
Makefile +2
-2endif -ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path.
Is this a superfluous "not"?
"Do nothing. Use the absolute path."?
Quoted 43 lines
Quoted 38 lines
Makefile else # need-sub-make · line 270
Quoted 10 lines
Makefile +7
-1+else ifeq ($(srcroot),$(CURDIR)) + # Building in the source. + srcroot := . +else ifeq ($(srcroot)/,$(dir $(CURDIR))) + # Building in a subdirectory of the source. + srcroot := .. endif -objtree := . +export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot))With this patch applied, the following breaks for me:
$ make O=build M=fs/btrfs CONFIG_BTRFS_FS=m make[1]: Entering directory '/data/linux/kbuild-review/fs/btrfs' CC [M] super.o In file included from <command-line>: /data/linux/kbuild-review/include/linux/compiler_types.h:89:10: fatal error: linux/compiler_attributes.h: No such file or directory 89 | #include <linux/compiler_attributes.h> | ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.Adding 'ccflags-y += -I$(srctree)/include' to
fs/btrfs/Makefilebreaks the build loudly. I could make it build again withMakefile +1
-1@@ -555,7 +555,7 @@ USERINCLUDE := \ LINUXINCLUDE := \ -I$(srctree)/arch/$(SRCARCH)/include \ -I$(objtree)/arch/$(SRCARCH)/include/generated \ - $(if $(building_out_of_srctree),-I$(srctree)/include) \ + $(if $(or $(building_out_of_srctree),$(filter $(srctree)/%, $(CURDIR))),-I$(srctree)/include) \ -I$(objtree)/include \ $(USERINCLUDE)Quoted 2 lines
but this does not feel good. It building in-tree modules in this way a valid thing to do?
Yes, it is a valid way.
Quoted 4 lines
This got broken with this commit, and fixed by the later commit, "kbuild: support building external modules in a separate build directory".
I will move the change for LINUXINCLUDE to this commit.
ah, thanks. I should have seen that.
Quoted 199 lines
Quoted 199 lines
Makefile +14
-17-VPATH := - -ifeq ($(KBUILD_EXTMOD),) ifdef building_out_of_srctree -VPATH := $(srctree) +export VPATH := $(srcroot) +else +VPATH := endif -endif - -export building_out_of_srctree srctree objtree VPATH # To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to scripts/kconfig/Makefile@@ -711,7 +726,7 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well -ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),) +ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMDGOALS)),) KBUILD_MODULES := 1 endif@@ -1107,7 +1122,7 @@ export MODLIB PHONY += prepare0 -export extmod_prefix = $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/) +export extmod_prefix = export MODORDER := $(extmod_prefix)modules.order export MODULES_NSDEPS := $(extmod_prefix)modules.nsdeps@@ -1799,14 +1814,10 @@ filechk_kernel.release = echo $(KERNELRELEASE) KBUILD_BUILTIN := KBUILD_MODULES := 1 -build-dir := $(KBUILD_EXTMOD) +build-dir := . -compile_commands.json: $(extmod_prefix)compile_commands.json -PHONY += compile_commands.json - -clean-dirs := $(KBUILD_EXTMOD) -clean: private rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \ - $(KBUILD_EXTMOD)/compile_commands.json +clean-dirs := . +clean: private rm-files := Module.symvers modules.nsdeps compile_commands.json PHONY += prepare # now expand this into a simple variable to reduce the cost of shell evaluations@@ -1948,7 +1959,7 @@ $(clean-dirs): clean: $(clean-dirs) $(call cmd,rmfiles) - @find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \ + @find . $(RCS_FIND_IGNORE) \ \( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \ -o -name '*.ko.*' \ -o -name '*.dtb' -o -name '*.dtbo' \@@ -1981,7 +1992,12 @@ tags TAGS cscope gtags: FORCE PHONY += rust-analyzer rust-analyzer: +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh +ifdef KBUILD_EXTMOD +# FIXME: external modules must not descend into a sub-directory of the kernel + $(Q)$(MAKE) $(build)=$(objtree)/rust src=$(srctree)/rust $@ +else $(Q)$(MAKE) $(build)=rust $@ +endif # Script to generate missing namespace dependencies # ---------------------------------------------------------------------------rust/Makefile +2
-2@@ -362,8 +362,8 @@ rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ - $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ - $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json + $(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \ + > rust-project.json redirect-intrinsics = \ __addsf3 __eqsf2 __extendsfdf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __truncdfsf2 __unordsf2 \scripts/Makefile.build +1
-1@@ -3,7 +3,7 @@ # Building # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := $(obj)/ $(obj)/:scripts/Makefile.clean +1
-1@@ -3,7 +3,7 @@ # Cleaning up # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := __clean __clean:scripts/Makefile.compiler +1
-1@@ -13,7 +13,7 @@ cc-cross-prefix = $(firstword $(foreach c, $(1), \ $(if $(shell command -v -- $(c)gcc 2>/dev/null), $(c)))) # output directory for tests below -TMPOUT = $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/).tmp_$$$$ +TMPOUT = .tmp_$$$$ # try-run # Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)scripts/Makefile.modpost +3
-3@@ -111,13 +111,13 @@ endif else # set src + obj - they may be used in the modules's Makefile -obj := $(KBUILD_EXTMOD) -src := $(if $(VPATH),$(VPATH)/)$(obj) +obj := . +src := $(srcroot) # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS include $(kbuild-file) -output-symdump := $(KBUILD_EXTMOD)/Module.symvers +output-symdump := Module.symvers ifeq ($(wildcard $(objtree)/Module.symvers),) missing-input := $(objtree)/Module.symversscripts/coccicheck +1
-5@@ -80,11 +80,7 @@ command results in a shift count error.' NPROC=1 else ONLINE=0 - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" # Use only one thread per core by default if hyperthreading is enabled THREADS_PER_CORE=$(LANG=C lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")scripts/nsdeps +1
-7@@ -19,12 +19,6 @@ if ! { echo "$SPATCH_REQ_VERSION"; echo "$SPATCH_VERSION"; } | sort -CV ; then exit 1 fi -if [ "$KBUILD_EXTMOD" ]; then - src_prefix= -else - src_prefix=$srctree/ -fi - generate_deps_for_ns() { $SPATCH --very-quiet --in-place --sp-file \ $srctree/scripts/coccinelle/misc/add_namespace.cocci -D nsdeps -D ns=$1 $2@@ -34,7 +28,7 @@ generate_deps() { local mod=${1%.ko:} shift local namespaces="$*" - local mod_source_files=$(sed "s|^\(.*\)\.o$|${src_prefix}\1.c|" $mod.mod) + local mod_source_files=$(sed "s|^\(.*\)\.o$|${srcroot}/\1.c|" $mod.mod) for ns in $namespaces; do echo "Adding namespace $ns to module $mod.ko."scripts/package/install-extmod-build +0
-0@@ -51,6 +51,13 @@ mkdir -p "${destdir}" if [ "${CC}" != "${HOSTCC}" ]; then echo "Rebuilding host programs with ${CC}..."
Quoted 12 lines
Quoted 12 lines
scripts/package/install-extmod-build +7
-0+ # This leverages external module building. + # - Clear sub_make_done to allow the top-level Makefile to redo sub-make. + # - Filter out --no-print-directory to print "Entering directory" logs + # when Make changes the working directory. + unset sub_make_done + MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//) + cat <<-'EOF' > "${destdir}/Kbuild" subdir-y := scripts EOFSignature · 1 line
2.43.0
Thanks, again!
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 190 lines · Nicolas Schier
Quoted 175 lines · Masahiro Yamada
Quoted 174 lines · Nicolas Schier
Quoted 110 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +6
-14@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,Documentation/kbuild/makefiles.rst +4
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, butDocumentation/kbuild/makefiles.rst $(obj) · line 465
Quoted 10 lines
Documentation/kbuild/makefiles.rst +10
-0+ if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. +Thanks, I think it's nice that there is now such a clear definition. $(srcroot) sounds fine to me.
Quoted 48 lines
Documentation/kbuild/makefiles.rst +0
-0$(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +24
-5@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute path before + # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory "$(KBUILD_EXTMOD)" does not exist)) +else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := elseQuoted 8 lines
Quoted 8 lines
Makefile else # need-sub-make · line 257
Quoted 8 lines
Makefile +1
-7- ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1Nit: Adding a space between ':=1' for consistency?
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines
Makefile +0
-0endif
Quoted 8 lines
Quoted 4 lines
Quoted 4 lines
Makefile else # need-sub-make · line 261
Quoted 4 lines
Makefile +2
-2-ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path.Is this a superfluous "not"?
"Do nothing. Use the absolute path."?
Thanks. I folded both.
Quoted 5 lines
Thanks, again!
Reviewed-by Nicolas Schier <[email protected]>
Thanks for the review!
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
In reply to Masahiro YamadaQuoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'
Quoted 5 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
With this in v6.13-rc1 the cxl-test module fails depmod. It causes depmod to be invoked from the incorrect place (or something doesn't respect INSTALL_MOD_PATH)
Is there something additional that this cxl-test module needs to do?
Repro:
/git/new$ make V=1 M=tools/testing/cxl INSTALL_MOD_PATH=qbuild/mkosi.extra/ modules_install
Outputs:
make -C /home/cxluser/git/new/tools/testing/cxl \
-f /home/cxluser/git/new/Makefile modules_install
make[1]: Entering directory '/home/cxluser/git/new/tools/testing/cxl' make --no-print-directory -C /home/cxluser/git/new/tools/testing/cxl \
-f /home/cxluser/git/new/Makefile modules_install
make -f /home/cxluser/git/new/scripts/Makefile.modinst \
sign-only=
# INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_acpi.ko
cp cxl_acpi.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_acpi.ko
# cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_acpi.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_acpi.ko : # INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko cp cxl_pmem.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko # cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko : # INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_port.ko cp cxl_port.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_port.ko # cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_port.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_port.ko : # INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_mem.ko cp cxl_mem.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_mem.ko # cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_mem.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_mem.ko : # INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_core.ko cp cxl_core.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_core.ko # cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_core.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_core.ko : # INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_test.ko cp test/cxl_test.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_test.ko # cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_test.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_test.ko : # INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock.ko cp test/cxl_mock.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock.ko # cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock.ko : # INSTALL qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock_mem.ko cp test/cxl_mock_mem.ko qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock_mem.ko # cmd_strip qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock_mem.ko : # cmd_sign qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock_mem.ko : # DEPMOD qbuild/mkosi.extra//lib/modules/6.13.0-rc1+ /home/cxluser/git/new/scripts/depmod.sh 6.13.0-rc1+ depmod: WARNING: could not open modules.order at /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+: No such file or directory depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/test/cxl_mock.ko needs unknown symbol nvdimm_bus_register depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol to_nvdimm depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol __nvdimm_create depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol nvdimm_bus_unregister depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol nvdimm_region_delete depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol nvdimm_delete depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol nvdimm_provider_data depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol nvdimm_pmem_region_create depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol nd_fletcher64 depmod: WARNING: /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+/updates/cxl_pmem.ko needs unknown symbol nvdimm_cmd_mask depmod: WARNING: could not open modules.builtin at /home/cxluser/git/new/tools/testing/cxl/qbuild/mkosi.extra//lib/modules/6.13.0-rc1+: No such file or directory make[1]: Leaving directory '/home/cxluser/git/new/tools/testing/cxl' cxluser@MySS:~/git/new$
Quoted 361 lines
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +6
-14@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,Documentation/kbuild/makefiles.rst +14
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, but + if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. + $(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +48
-32@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute path before + # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory "$(KBUILD_EXTMOD)" does not exist)) +else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := else - ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1 endif -ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path. +else ifeq ($(srcroot),$(CURDIR)) + # Building in the source. + srcroot := . +else ifeq ($(srcroot)/,$(dir $(CURDIR))) + # Building in a subdirectory of the source. + srcroot := .. endif -objtree := . +export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot)) -VPATH := - -ifeq ($(KBUILD_EXTMOD),) ifdef building_out_of_srctree -VPATH := $(srctree) +export VPATH := $(srcroot) +else +VPATH := endif -endif - -export building_out_of_srctree srctree objtree VPATH # To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to scripts/kconfig/Makefile@@ -711,7 +726,7 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well -ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),) +ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMDGOALS)),) KBUILD_MODULES := 1 endif@@ -1107,7 +1122,7 @@ export MODLIB PHONY += prepare0 -export extmod_prefix = $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/) +export extmod_prefix = export MODORDER := $(extmod_prefix)modules.order export MODULES_NSDEPS := $(extmod_prefix)modules.nsdeps@@ -1799,14 +1814,10 @@ filechk_kernel.release = echo $(KERNELRELEASE) KBUILD_BUILTIN := KBUILD_MODULES := 1 -build-dir := $(KBUILD_EXTMOD) +build-dir := . -compile_commands.json: $(extmod_prefix)compile_commands.json -PHONY += compile_commands.json - -clean-dirs := $(KBUILD_EXTMOD) -clean: private rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \ - $(KBUILD_EXTMOD)/compile_commands.json +clean-dirs := . +clean: private rm-files := Module.symvers modules.nsdeps compile_commands.json PHONY += prepare # now expand this into a simple variable to reduce the cost of shell evaluations@@ -1948,7 +1959,7 @@ $(clean-dirs): clean: $(clean-dirs) $(call cmd,rmfiles) - @find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \ + @find . $(RCS_FIND_IGNORE) \ \( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \ -o -name '*.ko.*' \ -o -name '*.dtb' -o -name '*.dtbo' \@@ -1981,7 +1992,12 @@ tags TAGS cscope gtags: FORCE PHONY += rust-analyzer rust-analyzer: +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh +ifdef KBUILD_EXTMOD +# FIXME: external modules must not descend into a sub-directory of the kernel + $(Q)$(MAKE) $(build)=$(objtree)/rust src=$(srctree)/rust $@ +else $(Q)$(MAKE) $(build)=rust $@ +endif # Script to generate missing namespace dependencies # ---------------------------------------------------------------------------rust/Makefile +2
-2@@ -362,8 +362,8 @@ rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ - $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ - $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json + $(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \ + > rust-project.json redirect-intrinsics = \ __addsf3 __eqsf2 __extendsfdf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __truncdfsf2 __unordsf2 \scripts/Makefile.build +1
-1@@ -3,7 +3,7 @@ # Building # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := $(obj)/ $(obj)/:scripts/Makefile.clean +1
-1@@ -3,7 +3,7 @@ # Cleaning up # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := __clean __clean:scripts/Makefile.compiler +1
-1@@ -13,7 +13,7 @@ cc-cross-prefix = $(firstword $(foreach c, $(1), \ $(if $(shell command -v -- $(c)gcc 2>/dev/null), $(c)))) # output directory for tests below -TMPOUT = $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/).tmp_$$$$ +TMPOUT = .tmp_$$$$ # try-run # Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)scripts/Makefile.modpost +3
-3@@ -111,13 +111,13 @@ endif else # set src + obj - they may be used in the modules's Makefile -obj := $(KBUILD_EXTMOD) -src := $(if $(VPATH),$(VPATH)/)$(obj) +obj := . +src := $(srcroot) # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS include $(kbuild-file) -output-symdump := $(KBUILD_EXTMOD)/Module.symvers +output-symdump := Module.symvers ifeq ($(wildcard $(objtree)/Module.symvers),) missing-input := $(objtree)/Module.symversscripts/coccicheck +1
-5@@ -80,11 +80,7 @@ command results in a shift count error.' NPROC=1 else ONLINE=0 - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" # Use only one thread per core by default if hyperthreading is enabled THREADS_PER_CORE=$(LANG=C lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")scripts/nsdeps +1
-7@@ -19,12 +19,6 @@ if ! { echo "$SPATCH_REQ_VERSION"; echo "$SPATCH_VERSION"; } | sort -CV ; then exit 1 fi -if [ "$KBUILD_EXTMOD" ]; then - src_prefix= -else - src_prefix=$srctree/ -fi - generate_deps_for_ns() { $SPATCH --very-quiet --in-place --sp-file \ $srctree/scripts/coccinelle/misc/add_namespace.cocci -D nsdeps -D ns=$1 $2@@ -34,7 +28,7 @@ generate_deps() { local mod=${1%.ko:} shift local namespaces="$*" - local mod_source_files=$(sed "s|^\(.*\)\.o$|${src_prefix}\1.c|" $mod.mod) + local mod_source_files=$(sed "s|^\(.*\)\.o$|${srcroot}/\1.c|" $mod.mod) for ns in $namespaces; do echo "Adding namespace $ns to module $mod.ko."scripts/package/install-extmod-build +7
-0@@ -51,6 +51,13 @@ mkdir -p "${destdir}" if [ "${CC}" != "${HOSTCC}" ]; then echo "Rebuilding host programs with ${CC}..." + # This leverages external module building. + # - Clear sub_make_done to allow the top-level Makefile to redo sub-make. + # - Filter out --no-print-directory to print "Entering directory" logs + # when Make changes the working directory. + unset sub_make_done + MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//) + cat <<-'EOF' > "${destdir}/Kbuild" subdir-y := scripts EOFSignature · 1 line
2.43.0
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 51 lines · Alison Schofield
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 5 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
With this in v6.13-rc1 the cxl-test module fails depmod. It causes depmod to be invoked from the incorrect place (or something doesn't respect
INSTALL_MOD_PATH)
Quoted 5 lines
Is there something additional that this cxl-test module needs to do?
Repro:
/git/new$ make V=1 M=tools/testing/cxl INSTALL_MOD_PATH=qbuild/mkosi.extra/modules_install
Kbuild changes the working directory, so you need to specify an absolute path for INSTALL_MOD_PATH.
INSTALL_MOD_PATH=$(realpath qbuild/mkosi.extra/)
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
In reply to Masahiro YamadaQuoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'
Quoted 2 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
This change has caused O=<relative directory> to fail.
For example:
make O=build defconfig
make -j$(nproc) V=1 O=build bindeb-pkg
outputs:
make ARCH=x86 KERNELRELEASE=6.13.0-rc1 KBUILD_BUILD_VERSION=3 run-command KBUILD_RUN_COMMAND='+$(srctree)/scripts/package/builddeb linux-libc-dev' dh_installchangelogs -plinux-image-6.13.0-rc1
../scripts/package/builddeb linux-headers-6.13.0-rc1dh_compress -plinux-image-6.13.0-rc1dh_fixperms -plinux-image-6.13.0-rc1dh_gencontrol -plinux-image-6.13.0-rc1 -- -fdebian/image.files
Rebuilding host programs with x86_64-linux-gnu-gcc...
make[6]: Entering directory '/scratch/kernels/linux/build'
/scratch/kernels/linux/Makefile:190: *** specified kernel directory "build" does not exist. Stop.
It is stepping into this directory and then trying to find the directory it just stepped into so $(realpath $(KBUILD_OUTPUT)) returns an empty string.
Using an absolute directory resolves this problem, but I believe it shouldn't be necessary.
- Charlie
Quoted 363 lines
Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +6
-14@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,Documentation/kbuild/makefiles.rst +14
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, but + if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. + $(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +48
-32@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute path before + # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory "$(KBUILD_EXTMOD)" does not exist)) +else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := else - ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1 endif -ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path. +else ifeq ($(srcroot),$(CURDIR)) + # Building in the source. + srcroot := . +else ifeq ($(srcroot)/,$(dir $(CURDIR))) + # Building in a subdirectory of the source. + srcroot := .. endif -objtree := . +export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot)) -VPATH := - -ifeq ($(KBUILD_EXTMOD),) ifdef building_out_of_srctree -VPATH := $(srctree) +export VPATH := $(srcroot) +else +VPATH := endif -endif - -export building_out_of_srctree srctree objtree VPATH # To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to scripts/kconfig/Makefile@@ -711,7 +726,7 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well -ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),) +ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMDGOALS)),) KBUILD_MODULES := 1 endif@@ -1107,7 +1122,7 @@ export MODLIB PHONY += prepare0 -export extmod_prefix = $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/) +export extmod_prefix = export MODORDER := $(extmod_prefix)modules.order export MODULES_NSDEPS := $(extmod_prefix)modules.nsdeps@@ -1799,14 +1814,10 @@ filechk_kernel.release = echo $(KERNELRELEASE) KBUILD_BUILTIN := KBUILD_MODULES := 1 -build-dir := $(KBUILD_EXTMOD) +build-dir := . -compile_commands.json: $(extmod_prefix)compile_commands.json -PHONY += compile_commands.json - -clean-dirs := $(KBUILD_EXTMOD) -clean: private rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \ - $(KBUILD_EXTMOD)/compile_commands.json +clean-dirs := . +clean: private rm-files := Module.symvers modules.nsdeps compile_commands.json PHONY += prepare # now expand this into a simple variable to reduce the cost of shell evaluations@@ -1948,7 +1959,7 @@ $(clean-dirs): clean: $(clean-dirs) $(call cmd,rmfiles) - @find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \ + @find . $(RCS_FIND_IGNORE) \ \( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \ -o -name '*.ko.*' \ -o -name '*.dtb' -o -name '*.dtbo' \@@ -1981,7 +1992,12 @@ tags TAGS cscope gtags: FORCE PHONY += rust-analyzer rust-analyzer: +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh +ifdef KBUILD_EXTMOD +# FIXME: external modules must not descend into a sub-directory of the kernel + $(Q)$(MAKE) $(build)=$(objtree)/rust src=$(srctree)/rust $@ +else $(Q)$(MAKE) $(build)=rust $@ +endif # Script to generate missing namespace dependencies # ---------------------------------------------------------------------------rust/Makefile +2
-2@@ -362,8 +362,8 @@ rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ - $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ - $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json + $(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \ + > rust-project.json redirect-intrinsics = \ __addsf3 __eqsf2 __extendsfdf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __truncdfsf2 __unordsf2 \scripts/Makefile.build +1
-1@@ -3,7 +3,7 @@ # Building # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := $(obj)/ $(obj)/:scripts/Makefile.clean +1
-1@@ -3,7 +3,7 @@ # Cleaning up # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := __clean __clean:scripts/Makefile.compiler +1
-1@@ -13,7 +13,7 @@ cc-cross-prefix = $(firstword $(foreach c, $(1), \ $(if $(shell command -v -- $(c)gcc 2>/dev/null), $(c)))) # output directory for tests below -TMPOUT = $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/).tmp_$$$$ +TMPOUT = .tmp_$$$$ # try-run # Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)scripts/Makefile.modpost +3
-3@@ -111,13 +111,13 @@ endif else # set src + obj - they may be used in the modules's Makefile -obj := $(KBUILD_EXTMOD) -src := $(if $(VPATH),$(VPATH)/)$(obj) +obj := . +src := $(srcroot) # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS include $(kbuild-file) -output-symdump := $(KBUILD_EXTMOD)/Module.symvers +output-symdump := Module.symvers ifeq ($(wildcard $(objtree)/Module.symvers),) missing-input := $(objtree)/Module.symversscripts/coccicheck +1
-5@@ -80,11 +80,7 @@ command results in a shift count error.' NPROC=1 else ONLINE=0 - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" # Use only one thread per core by default if hyperthreading is enabled THREADS_PER_CORE=$(LANG=C lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")scripts/nsdeps +1
-7@@ -19,12 +19,6 @@ if ! { echo "$SPATCH_REQ_VERSION"; echo "$SPATCH_VERSION"; } | sort -CV ; then exit 1 fi -if [ "$KBUILD_EXTMOD" ]; then - src_prefix= -else - src_prefix=$srctree/ -fi - generate_deps_for_ns() { $SPATCH --very-quiet --in-place --sp-file \ $srctree/scripts/coccinelle/misc/add_namespace.cocci -D nsdeps -D ns=$1 $2@@ -34,7 +28,7 @@ generate_deps() { local mod=${1%.ko:} shift local namespaces="$*" - local mod_source_files=$(sed "s|^\(.*\)\.o$|${src_prefix}\1.c|" $mod.mod) + local mod_source_files=$(sed "s|^\(.*\)\.o$|${srcroot}/\1.c|" $mod.mod) for ns in $namespaces; do echo "Adding namespace $ns to module $mod.ko."scripts/package/install-extmod-build +7
-0@@ -51,6 +51,13 @@ mkdir -p "${destdir}" if [ "${CC}" != "${HOSTCC}" ]; then echo "Rebuilding host programs with ${CC}..." + # This leverages external module building. + # - Clear sub_make_done to allow the top-level Makefile to redo sub-make. + # - Filter out --no-print-directory to print "Entering directory" logs + # when Make changes the working directory. + unset sub_make_done + MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//) + cat <<-'EOF' > "${destdir}/Kbuild" subdir-y := scripts EOFSignature · 1 line
2.43.0
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 63 lines · Charlie Jenkins
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 2 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
This change has caused O=<relative directory> to fail.
For example:
make O=build defconfig
make -j$(nproc) V=1 O=build bindeb-pkgoutputs:
make ARCH=x86 KERNELRELEASE=6.13.0-rc1 KBUILD_BUILD_VERSION=3 run-command KBUILD_RUN_COMMAND='+$(srctree)/scripts/package/builddeb linux-libc-dev'
dh_installchangelogs-plinux-image-6.13.0-rc1
../scripts/package/builddeb linux-headers-6.13.0-rc1dh_compress-plinux-image-6.13.0-rc1dh_fixperms-plinux-image-6.13.0-rc1dh_gencontrol-plinux-image-6.13.0-rc1 -- -fdebian/image.files
Rebuilding host programs with x86_64-linux-gnu-gcc...
make[6]: Entering directory '/scratch/kernels/linux/build'
/scratch/kernels/linux/Makefile:190: *** specified kernel directory "build" does not exist. Stop.
Quoted 6 lines
It is stepping into this directory and then trying to find the directory it just stepped into so $(realpath $(KBUILD_OUTPUT)) returns an empty string.
Using an absolute directory resolves this problem, but I believe it shouldn't be necessary.
Agree.
I will apply the following fixup unless I have a better idea.
(quoted diff) +0 -0
b/scripts/package/install-extmod-build
index 64d958ee45f3..85af1573db31 100755
scripts/package/install-extmod-build +2 -2
@@ -69,7 +69,7 @@ if [ "${CC}" != "${HOSTCC}" ]; then # # Use the single-target build to avoid the modpost invocation, which # would overwrite Module.symvers. - "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ + "${MAKE}" O=. HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ cat <<-'EOF' > "${destdir}/scripts/Kbuild" subdir-y := basic
@@ -78,7 +78,7 @@ if [ "${CC}" != "${HOSTCC}" ]; then EOF # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. - "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ + "${MAKE}" O=. HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" fi
--
Best Regards
Masahiro Yamada
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 95 lines · Masahiro Yamada
Quoted 63 lines · Charlie Jenkins
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 2 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
This change has caused O=<relative directory> to fail.
For example:
make O=build defconfig
make -j$(nproc) V=1 O=build bindeb-pkgoutputs:
make ARCH=x86 KERNELRELEASE=6.13.0-rc1 KBUILD_BUILD_VERSION=3 run-command KBUILD_RUN_COMMAND='+$(srctree)/scripts/package/builddeb linux-libc-dev'
dh_installchangelogs-plinux-image-6.13.0-rc1
../scripts/package/builddeb linux-headers-6.13.0-rc1dh_compress-plinux-image-6.13.0-rc1dh_fixperms-plinux-image-6.13.0-rc1dh_gencontrol-plinux-image-6.13.0-rc1 -- -fdebian/image.files
Rebuilding host programs with x86_64-linux-gnu-gcc...
make[6]: Entering directory '/scratch/kernels/linux/build'
/scratch/kernels/linux/Makefile:190: *** specified kernel directory "build" does not exist. Stop.Quoted 6 lines
It is stepping into this directory and then trying to find the directory it just stepped into so $(realpath $(KBUILD_OUTPUT)) returns an empty string.
Using an absolute directory resolves this problem, but I believe it shouldn't be necessary.
Agree.
I will apply the following fixup unless I have a better idea.
scripts/package/install-extmod-build +1
-1@@ -69,7 +69,7 @@ if [ "${CC}" != "${HOSTCC}" ]; then # # Use the single-target build to avoid the modpost invocation, which # would overwrite Module.symvers. - "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ + "${MAKE}" O=. HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ cat <<-'EOF' > "${destdir}/scripts/Kbuild" subdir-y := basic
scripts/package/install-extmod-build if [ "${CC}" != "${HOSTCC}" ]; then · line 84
Quoted 8 lines
scripts/package/install-extmod-build +1
-1@@ -78,7 +78,7 @@ if [ "${CC}" != "${HOSTCC}" ]; then EOF # Run once again to rebuild scripts/basic/ and scripts/mod/modpost. - "${MAKE}" HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ + "${MAKE}" O=. HOSTCC="${CC}" KBUILD_EXTMOD="${destdir}" scripts/ rm -f "${destdir}/Kbuild" "${destdir}/scripts/Kbuild" fi
Looks fine to me :)
If this is what you decide on, feel free to add my tags.
Quoted 5 lines
--
Best Regards
Masahiro Yamada
Re: [cocci] [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
In reply to Masahiro Yamada😊
Masahiro さんが Gmail
<https://www.google.com/gmail/about/?utm_source=gmail-in-product&utm_medium=et&utm_campaign=emojireactionemail#app>
でリアクションしました
2024年11月10日(日) 10:37 Masahiro Yamada <[email protected]>:
Quoted 447 lines
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +0
-0@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, ifused
Documentation/dev-tools/coccinelle.rst +0
-2-Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking upa
-.cocciconfig when using ``make coccicheck``.Documentation/dev-tools/coccinelle.rst +1
-7- ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For bothcases
-the spatch ``--dir`` argument is used, as such third rule applies when whether
-M= is used or not, and when M= is used the target directory can have its own
-.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called.
+Here, $srcroot refers to the source directory of the target: it points to the
+external module's source directory when M= used, and otherwise, to the kernel
+source directory. The third rule ensures the spatch reads the .cocciconfig from
+the target directory, allowing external modules to have their own .cocciconfig
+file.If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck
target,Documentation/kbuild/makefiles.rst +2
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which canbe
+ either the kernel source or the external modules source, depending on whether
+KBUILD_EXTMODis set. This can be either a relative or an absolute path, butDocumentation/kbuild/makefiles.rst +4
-0+ if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When buildingthe
Documentation/kbuild/makefiles.rst +4
-0+ kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.``when
Documentation/kbuild/makefiles.rst +2
-0+ building the kernel, but it is different when building external modules. + $(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +9
-1@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute pathbefore
Makefile +3
-0+ # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory"$(KBUILD_EXTMOD)" does not exist))
Makefile +6
-0+else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation ofmake
Makefile +20
-22-ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := else - ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1 endif -ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path. +else ifeq ($(srcroot),$(CURDIR)) + # Building in the source. + srcroot := . +else ifeq ($(srcroot)/,$(dir $(CURDIR))) + # Building in a subdirectory of the source. + srcroot := .. endif -objtree := . +export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot)) -VPATH := - -ifeq ($(KBUILD_EXTMOD),) ifdef building_out_of_srctree -VPATH := $(srctree) +export VPATH := $(srcroot) +else +VPATH := endif -endif - -export building_out_of_srctree srctree objtree VPATH # To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to scripts/kconfig/Makefile@@ -711,7 +726,7 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well -ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),) +ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMDGOALS)),) KBUILD_MODULES := 1endif
Makefile +4
-8@@ -1107,7 +1122,7 @@ export MODLIB PHONY += prepare0 -export extmod_prefix = $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/) +export extmod_prefix = export MODORDER := $(extmod_prefix)modules.order export MODULES_NSDEPS := $(extmod_prefix)modules.nsdeps@@ -1799,14 +1814,10 @@ filechk_kernel.release = echo $(KERNELRELEASE) KBUILD_BUILTIN := KBUILD_MODULES := 1 -build-dir := $(KBUILD_EXTMOD) +build-dir := . -compile_commands.json: $(extmod_prefix)compile_commands.json -PHONY += compile_commands.json - -clean-dirs := $(KBUILD_EXTMOD) -clean: private rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \ - $(KBUILD_EXTMOD)/compile_commands.json +clean-dirs := . +clean: private rm-files := Module.symvers modules.nsdepscompile_commands.json
PHONY += prepare
# now expand this into a simple variable to reduce the cost of shell evaluationsMakefile +3
-2@@ -1948,7 +1959,7 @@ $(clean-dirs): clean: $(clean-dirs) $(call cmd,rmfiles) - @find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \ + @find . $(RCS_FIND_IGNORE) \ \( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \ -o -name '*.ko.*' \ -o -name '*.dtb' -o -name '*.dtbo' \@@ -1981,7 +1992,12 @@ tags TAGS cscope gtags: FORCE PHONY += rust-analyzer rust-analyzer: +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh +ifdef KBUILD_EXTMOD +# FIXME: external modules must not descend into a sub-directory of thekernel
Makefile +3
-1+ $(Q)$(MAKE) $(build)=$(objtree)/rust src=$(srctree)/rust $@ +else $(Q)$(MAKE) $(build)=rust $@ +endif # Script to generate missing namespace dependencies # ---------------------------------------------------------------------------rust/Makefile +0
-2@@ -362,8 +362,8 @@ rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ - $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ - $(if$(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json
+ $(rustc_sysroot) $(RUST_LIB_SRC) $(if
$(KBUILD_EXTMOD),$(srcroot)) \
+ > rust-project.jsonredirect-intrinsics = \ __addsf3 __eqsf2 __extendsfdf2 __gesf2 __lesf2 __ltsf2 __mulsf3
__nesf2__truncdfsf2__unordsf2\scripts/Makefile.build +0
-0@@ -3,7 +3,7 @@ # Building #==========================================================================
scripts/Makefile.build +1
-1-src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := $(obj)/ $(obj)/:scripts/Makefile.clean +0
-0@@ -3,7 +3,7 @@ # Cleaning up #==========================================================================
scripts/Makefile.clean +1
-1-src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := __clean __clean:scripts/Makefile.compiler +1
-1@@ -13,7 +13,7 @@ cc-cross-prefix = $(firstword $(foreach c, $(1), \ $(if $(shell command -v -- $(c)gcc 2>/dev/null), $(c)))) # output directory for tests below -TMPOUT = $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/).tmp_$$$$ +TMPOUT = .tmp_$$$$ # try-run# Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)
scripts/Makefile.modpost +3
-3@@ -111,13 +111,13 @@ endif else # set src + obj - they may be used in the modules's Makefile -obj := $(KBUILD_EXTMOD) -src := $(if $(VPATH),$(VPATH)/)$(obj) +obj := . +src := $(srcroot) # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS include $(kbuild-file) -output-symdump := $(KBUILD_EXTMOD)/Module.symvers +output-symdump := Module.symvers ifeq ($(wildcard $(objtree)/Module.symvers),) missing-input := $(objtree)/Module.symversscripts/coccicheck +1
-6@@ -80,11 +80,7 @@ command results in a shift count error.' NPROC=1 else ONLINE=0 - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" # Use only one thread per core by default if hyperthreading is enabled THREADS_PER_CORE=$(LANG=C lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")scripts/nsdeps +0
-0@@ -19,12 +19,6 @@ if ! { echo "$SPATCH_REQ_VERSION"; echo"$SPATCH_VERSION"; } | sort -CV ; then
exit 1
fiscripts/nsdeps +0
-6-if [ "$KBUILD_EXTMOD" ]; then - src_prefix= -else - src_prefix=$srctree/ -fi - generate_deps_for_ns() { $SPATCH --very-quiet --in-place --sp-file \ $srctree/scripts/coccinelle/misc/add_namespace.cocci -Dnsdeps -D ns=$1 $2
scripts/nsdeps +1
-1@@ -34,7 +28,7 @@ generate_deps() { local mod=${1%.ko:} shift local namespaces="$*" - local mod_source_files=$(sed "s|^\(.*\)\.o$|${src_prefix}\1.c|" $mod.mod) + local mod_source_files=$(sed "s|^\(.*\)\.o$|${srcroot}/\1.c|" $mod.mod)for ns in $namespaces; do
echo "Adding namespace $ns to module $mod.ko."scripts/package/install-extmod-build +2
-0@@ -51,6 +51,13 @@ mkdir -p "${destdir}" if [ "${CC}" != "${HOSTCC}" ]; then echo "Rebuilding host programs with ${CC}..." + # This leverages external module building. + # - Clear sub_make_done to allow the top-level Makefile to redosub-make.
+ # - Filter out --no-print-directory to print "Entering directory"
logsscripts/package/install-extmod-build +4
-1+ # when Make changes the working directory. + unset sub_make_done + MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//) + cat <<-'EOF' > "${destdir}/Kbuild" subdir-y := scripts EOF --
Quoted 3 lines
2.43.0
{
"emoji": "😊",
"version": 1
}
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
In reply to Masahiro YamadaQuoted 40 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'
Quoted 5 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Hi Masahiro,
I get the following error since this patch is in master, but only when using COCCI= in combination with M=<relative or absolute path>.
It works when I either use COCCI= or M=, but not with both.
$ make coccicheck COCCI=scripts/coccinelle/misc/flexible_array.cocci M=arch/ make[1]: Entering directory '/home/fedora/linux/arch' You have not explicitly specified the mode to use. Using default "report" mode. Available modes are the following: patch, report, context, org, chain You can specify the mode with "make coccicheck MODE=<mode>" Note however that some modes are not implemented by some semantic patches.
Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.
grep: scripts/coccinelle/misc/flexible_array.cocci: No such file or directory
grep: scripts/coccinelle/misc/flexible_array.cocci: No such file or directory/usr/bin/spatch -D report --no-show-diff --very-quiet --cocci-file scripts/coccinelle/misc/flexible_array.cocci --patch /home/fedora/linux --dir . -I /home/fedora/linux/arch/arm64/include -I /home/fedora/linux/arch/arm64/include/generated -I /home/fedora/linux/include -I /home/fedora/linux/include -I /home/fedora/linux/arch/arm64/include/uapi -I /home/fedora/linux/arch/arm64/include/generated/uapi -I /home/fedora/linux/include/uapi -I /home/fedora/linux/include/generated/uapi --include /home/fedora/linux/include/linux/compiler-version.h --include /home/fedora/linux/include/linux/kconfig.h --jobs 4 --chunksize 1 Fatal error: exception Sys_error("scripts/coccinelle/misc/flexible_array.cocci: No such file or directory")
coccicheck failed
make[2]: *** [/home/fedora/linux/Makefile:2089: coccicheck] Error 2
make[1]: *** [/home/fedora/linux/Makefile:251: __sub-make] Error 2
make[1]: Leaving directory '/home/fedora/linux/arch'
make: *** [Makefile:251: __sub-make] Error 2
Thanks,
Thorsten
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 47 lines · Thorsten Blum
Quoted 40 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 5 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Quoted 6 lines
Hi Masahiro,
I get the following error since this patch is in master, but only when using COCCI= in combination with M=<relative or absolute path>.
It works when I either use COCCI= or M=, but not with both.
Using the absolute path of the cocci script fixes my problem, but this used to work with relative paths too.
$ make coccicheck COCCI=$(pwd)/scripts/coccinelle/misc/flexible_array.cocci M=arch/
Thanks,
Thorsten
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 54 lines · Thorsten Blum
Quoted 52 lines · Thorsten Blum
Quoted 40 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 5 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Hi Masahiro,
I get the following error since this patch is in master, but only when using COCCI= in combination with M=<relative or absolute path>.
Quoted 6 lines
Quoted 1 line
It works when I either use COCCI= or M=, but not with both.
Using the absolute path of the cocci script fixes my problem, but this used to work with relative paths too.
$ make coccicheck COCCI=$(pwd)/scripts/coccinelle/misc/flexible_array.cocci M=arch/
M= looks a bit weird for the upstream code, but I think using the absolute path is the right thing to do.
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 56 lines · Masahiro Yamada
Quoted 55 lines · Thorsten Blum
Quoted 53 lines · Thorsten Blum
Quoted 40 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 5 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Hi Masahiro,
I get the following error since this patch is in master, but only when using COCCI= in combination with M=<relative or absolute path>.
It works when I either use COCCI= or M=, but not with both.
Quoted 7 lines
Quoted 4 lines
Using the absolute path of the cocci script fixes my problem, but this used to work with relative paths too.
$ make coccicheck COCCI=$(pwd)/scripts/coccinelle/misc/flexible_array.cocci M=arch/M= looks a bit weird for the upstream code, but I think using the absolute path is the right thing to do.
The documentation[1] uses M= and also COCCI= with relative paths and some of the examples don't work anymore.
Could you try this one?
``` export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci make coccicheck DEBUG_FILE="err.log" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd ```
I get this:
$ export COCCI=scripts/coccinelle/misc/irqf_oneshot.cocci $ make coccicheck DEBUG_FILE="err.log" MODE=report SPFLAGS="--profile --show-trying" M=./drivers/mfd make[1]: Entering directory '/home/fedora/linux/drivers/mfd'
Please check for false positives in the output before submitting a patch. When using "patch" mode, carefully review the patch before submitting it.
grep: scripts/coccinelle/misc/irqf_oneshot.cocci: No such file or directory
grep: scripts/coccinelle/misc/irqf_oneshot.cocci: No such file or directory coccicheck failed
make[2]: *** [/home/fedora/linux/Makefile:2089: coccicheck] Error 2 make[1]: *** [/home/fedora/linux/Makefile:251: __sub-make] Error 2 make[1]: Leaving directory '/home/fedora/linux/drivers/mfd'
make: *** [Makefile:251: __sub-make] Error 2
Thanks,
Thorsten
[1] https://www.kernel.org/doc/html/latest/dev-tools/coccinelle.html
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 61 lines · Thorsten Blum
Quoted 59 lines · Masahiro Yamada
Quoted 58 lines · Thorsten Blum
Quoted 47 lines · Thorsten Blum
Quoted 40 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 5 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Quoted 6 lines
Hi Masahiro,
I get the following error since this patch is in master, but only when using COCCI= in combination with M=<relative or absolute path>.
It works when I either use COCCI= or M=, but not with both.
Using the absolute path of the cocci script fixes my problem, but this used to work with relative paths too.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
$ make coccicheck COCCI=$(pwd)/scripts/coccinelle/misc/flexible_array.cocci M=arch/M= looks a bit weird for the upstream code, but I think using the absolute path is the right thing to do.
The documentation[1] uses M= and also COCCI= with relative paths and some of the examples don't work anymore.
I know.
This is what cocci people hoped to work.
If something is broken in coccicheck, it must be fixed.
Signature · 2 lines
Best Regards Masahiro Yamada
Jon Hunter
maintainerRe: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
In reply to Masahiro YamadaHi Masahiro,
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'
Quoted 4 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>
Since this change I have been observing the following build error when building an external module ...
MODPOST Module.symvers
ERROR: modpost: drivers/gpu/host1x/host1x: 'host1x_device_init' exported
twice. Previous export was in drivers/gpu/host1x/host1x.ko
ERROR: modpost: drivers/gpu/host1x/host1x: 'host1x_device_exit' exported
twice. Previous export was in drivers/gpu/host1x/host1x.ko
Now host1x is an upstream driver, but I have a local copy that using to stage development changes (and avoid polluting the upstream driver). Plus I can swap between which version I am using on a live system.
What I noticed is that previously the Modules.symvers for the external module had the full path of the external module for the name. However, now the name is just the relative path and in this case 'drivers/gpu/host1x/host1x'. Hence, this clashes with the in-kernel driver and we get the 'exported twice' error.
I have been looking to see if there is a way to fix this because it has been a useful feature to override an upstream driver with a locally modified version.
Thanks
Jon
Signature · 1 line
nvpublic
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 68 lines · Jon Hunter
Hi Masahiro,
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 4 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>Since this change I have been observing the following build error when building an external module ...
MODPOST Module.symvers
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_init' exported
twice. Previous export was indrivers/gpu/host1x/host1x.ko
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_exit' exported
twice. Previous export was indrivers/gpu/host1x/host1x.koNow host1x is an upstream driver, but I have a local copy that using to stage development changes (and avoid polluting the upstream driver). Plus I can swap between which version I am using on a live system.
What I noticed is that previously the Modules.symvers for the external module had the full path of the external module for the name. However, now the name is just the relative path and in this case '
drivers/gpu/host1x/host1x'. Hence, this clashes with the in-kernel driver and we get the 'exported twice' error.
Quoted 3 lines
I have been looking to see if there is a way to fix this because it has been a useful feature to override an upstream driver with a locally modified version.
I do not know how to reproduce it.
if (s && (!external_module || s->module->is_vmlinux || s->module == mod)) {is not checking the module path at all.
I do not understand why it was affected.
Signature · 2 lines
Best Regards Masahiro Yamada
Jon Hunter
maintainerRe: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 74 lines · Masahiro Yamada
Quoted 68 lines · Jon Hunter
Hi Masahiro,
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 4 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>Since this change I have been observing the following build error when building an external module ...
MODPOST Module.symvers
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_init' exported
twice. Previous export was indrivers/gpu/host1x/host1x.ko
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_exit' exported
twice. Previous export was indrivers/gpu/host1x/host1x.koNow host1x is an upstream driver, but I have a local copy that using to stage development changes (and avoid polluting the upstream driver). Plus I can swap between which version I am using on a live system.
What I noticed is that previously the Modules.symvers for the external module had the full path of the external module for the name. However, now the name is just the relative path and in this case '
drivers/gpu/host1x/host1x'. Hence, this clashes with the in-kernel driver and we get the 'exported twice' error.Quoted 3 lines
I have been looking to see if there is a way to fix this because it has been a useful feature to override an upstream driver with a locally modified version.
Quoted 6 lines
I do not know how to reproduce it.
if (s && (!external_module || s->module->is_vmlinux || s->module == mod)) {is not checking the module path at all.
I do not understand why it was affected.
So this is not explicitly checking the path, but comparing the contents of the Module.symvers before and after this change for the external module I see ...
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init /absolute/path/to/drivers/gpu/host1x/host1x EXPORT_SYMBOL
And now I see ...
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init drivers/gpu/host1x/host1x EXPORT_SYMBOL
So the problem is that now there is no longer an absolute path in the external modules Module.symvers and so conflicts with the kernel's.
Does that make sense?
Jon
Signature · 1 line
nvpublic
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 94 lines · Jon Hunter
Quoted 74 lines · Masahiro Yamada
Quoted 68 lines · Jon Hunter
Hi Masahiro,
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 4 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>Since this change I have been observing the following build error when building an external module ...
MODPOST Module.symvers
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_init' exported
twice. Previous export was indrivers/gpu/host1x/host1x.ko
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_exit' exported
twice. Previous export was indrivers/gpu/host1x/host1x.koNow host1x is an upstream driver, but I have a local copy that using to stage development changes (and avoid polluting the upstream driver). Plus I can swap between which version I am using on a live system.
What I noticed is that previously the Modules.symvers for the external module had the full path of the external module for the name. However, now the name is just the relative path and in this case '
drivers/gpu/host1x/host1x'. Hence, this clashes with the in-kernel driver and we get the 'exported twice' error.Quoted 3 lines
I have been looking to see if there is a way to fix this because it has been a useful feature to override an upstream driver with a locally modified version.
Quoted 6 lines
I do not know how to reproduce it.
if (s && (!external_module || s->module->is_vmlinux || s->module == mod)) {is not checking the module path at all.
I do not understand why it was affected.So this is not explicitly checking the path, but comparing the contents of the Module.symvers before and after this change for the external module I see ...
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init /absolute/path/to/drivers/gpu/host1x/host1x EXPORT_SYMBOLAnd now I see ...
Quoted 7 lines
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init drivers/gpu/host1x/host1x EXPORT_SYMBOLSo the problem is that now there is no longer an absolute path in the external modules Module.symvers and so conflicts with the kernel's.
Does that make sense?
As I said, I do not know how to reproduce it.
Please provide the steps to reproduce it.
--
Best Regards
Masahiro Yamada
Jon Hunter
maintainerRe: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 101 lines · Masahiro Yamada
Quoted 100 lines · Jon Hunter
Quoted 74 lines · Masahiro Yamada
Quoted 68 lines · Jon Hunter
Hi Masahiro,
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 4 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>Since this change I have been observing the following build error when building an external module ...
MODPOST Module.symvers
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_init' exported
twice. Previous export was indrivers/gpu/host1x/host1x.ko
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_exit' exported
twice. Previous export was indrivers/gpu/host1x/host1x.koNow host1x is an upstream driver, but I have a local copy that using to stage development changes (and avoid polluting the upstream driver). Plus I can swap between which version I am using on a live system.
What I noticed is that previously the Modules.symvers for the external module had the full path of the external module for the name. However, now the name is just the relative path and in this case '
drivers/gpu/host1x/host1x'. Hence, this clashes with the in-kernel driver and we get the 'exported twice' error.Quoted 3 lines
I have been looking to see if there is a way to fix this because it has been a useful feature to override an upstream driver with a locally modified version.
Quoted 6 lines
I do not know how to reproduce it.
if (s && (!external_module || s->module->is_vmlinux || s->module == mod)) {is not checking the module path at all.
I do not understand why it was affected.So this is not explicitly checking the path, but comparing the contents of the Module.symvers before and after this change for the external module I see ...
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init /absolute/path/to/drivers/gpu/host1x/host1x EXPORT_SYMBOLAnd now I see ...
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init drivers/gpu/host1x/host1x EXPORT_SYMBOLSo the problem is that now there is no longer an absolute path in the external modules Module.symvers and so conflicts with the kernel's.
Quoted 6 lines
Quoted 1 line
Does that make sense?
As I said, I do not know how to reproduce it.
Please provide the steps to reproduce it.
Got it! The steps would be ...
1. Create an external module by copying using an existing upstream driver (such as host1x).
2. Create a new external module that uses the external module from step 1 and uses KBUILD_EXTRA_SYMBOLS to reference the Module.symvers for the driver in step 1.
Thanks!
Jon
Signature · 1 line
nvpublic
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
Quoted 113 lines · Jon Hunter
Quoted 101 lines · Masahiro Yamada
Quoted 100 lines · Jon Hunter
Quoted 74 lines · Masahiro Yamada
Quoted 68 lines · Jon Hunter
Hi Masahiro,
Quoted 39 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Quoted 4 lines
Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>Since this change I have been observing the following build error when building an external module ...
MODPOST Module.symvers
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_init' exported
twice. Previous export was indrivers/gpu/host1x/host1x.ko
ERROR: modpost:drivers/gpu/host1x/host1x: 'host1x_device_exit' exported
twice. Previous export was indrivers/gpu/host1x/host1x.koNow host1x is an upstream driver, but I have a local copy that using to stage development changes (and avoid polluting the upstream driver). Plus I can swap between which version I am using on a live system.
What I noticed is that previously the Modules.symvers for the external module had the full path of the external module for the name. However, now the name is just the relative path and in this case '
drivers/gpu/host1x/host1x'. Hence, this clashes with the in-kernel driver and we get the 'exported twice' error.Quoted 3 lines
I have been looking to see if there is a way to fix this because it has been a useful feature to override an upstream driver with a locally modified version.
Quoted 6 lines
I do not know how to reproduce it.
if (s && (!external_module || s->module->is_vmlinux || s->module == mod)) {is not checking the module path at all.
I do not understand why it was affected.So this is not explicitly checking the path, but comparing the contents of the Module.symvers before and after this change for the external module I see ...
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init /absolute/path/to/drivers/gpu/host1x/host1x EXPORT_SYMBOLAnd now I see ...
$ grep -r host1x_device_init Module.symvers 0x00000000 host1x_device_init drivers/gpu/host1x/host1x EXPORT_SYMBOLSo the problem is that now there is no longer an absolute path in the external modules Module.symvers and so conflicts with the kernel's.
Quoted 6 lines
Quoted 1 line
Does that make sense?
As I said, I do not know how to reproduce it.
Please provide the steps to reproduce it.
Got it! The steps would be ...
Quoted 8 lines
1. Create an external module by copying using an existing upstream driver (such as host1x).
2. Create a new external module that uses the external module from step 1 and usesKBUILD_EXTRA_SYMBOLSto reference the Module.symvers for the driver in step 1.Thanks!
Jon
OK, now I understand, and posted a patch.
Thanks.
Signature · 2 lines
Best Regards Masahiro Yamada
Re: [PATCH v2 05/11] kbuild: change working directory to external module directory with M=
In reply to Masahiro YamadaHi Masahiro-san,
Thanks for your contribution, but it looks like this is breaking quite some vim/nvim quick fixes.
If there is some compiling errors/warnings, the path will be the one inside the module directory, no matter if it's out-of-tree or upstreamed.
Is there anyway to get back the full filename so that vim/nvim quick fix can work again?
Thanks,
Qu
Quoted 402 lines · Masahiro Yamada
Currently,
Kbuildalways operates in the output directory of the kernel, even when building external modules. This increases the risk of external module Makefiles attempting to write to the kernel directory.This commit switches the working directory to the external module directory, allowing the removal of the $(KBUILD_EXTMOD)/ prefix from some build artifacts.
The command for building external modules maintains backward compatibility, but Makefiles that rely on working in the kernel directory may break. In such cases, $(objtree) and $(srctree) should be used to refer to the output and source directories of the kernel.
The appearance of the build log will change as follows:
[Before]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' CC [M] /path/to/my/externel/module/helloworld.o MODPOST /path/to/my/externel/module/Module.symvers CC [M] /path/to/my/externel/module/helloworld.mod.o CC [M] /path/to/my/externel/module/.module-common.o LD [M] /path/to/my/externel/module/helloworld.ko make: Leaving directory '/path/to/my/linux'[After]
$ make -C /path/to/my/linux M=/path/to/my/externel/module make: Entering directory '/path/to/my/linux' make[1]: Entering directory '/path/to/my/externel/module' CC [M] helloworld.o MODPOST Module.symvers CC [M] helloworld.mod.o CC [M] .module-common.o LD [M] helloworld.ko make[1]: Leaving directory '/path/to/my/externel/module' make: Leaving directory '/path/to/my/linux'Printing "Entering directory" twice is cumbersome. This will be addressed later.
Signed-off-by Masahiro Yamada <[email protected]>---
Changes in v2:
- Introduce a new 'srcroot' variable and clean-up code
- RewordDocumentation/dev-tools/coccinelle.rstDocumentation/dev-tools/coccinelle.rst | 20 ++----- Documentation/kbuild/makefiles.rst | 14 +++++ Makefile | 80 +++++++++++++++----------- rust/Makefile | 4 +- scripts/Makefile.build | 2 +- scripts/Makefile.clean | 2 +- scripts/Makefile.compiler | 2 +- scripts/Makefile.modpost | 6 +- scripts/coccicheck | 6 +- scripts/nsdeps | 8 +-- scripts/package/install-extmod-build | 7 +++ 11 files changed, 85 insertions(+), 66 deletions(-)Documentation/dev-tools/coccinelle.rst +6
-14@@ -250,25 +250,17 @@ variables for .cocciconfig is as follows: - Your directory from which spatch is called is processed next - The directory provided with the ``--dir`` option is processed last, if used -Since coccicheck runs through make, it naturally runs from the kernel -proper dir; as such the second rule above would be implied for picking up a -.cocciconfig when using ``make coccicheck``. - ``make coccicheck`` also supports using M= targets. If you do not supply any M= target, it is assumed you want to target the entire kernel. The kernel coccicheck script has:: - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" -KBUILD_EXTMOD is set when an explicit target with M= is used. For both cases -the spatch ``--dir`` argument is used, as such third rule applies when whether -M= is used or not, and when M= is used the target directory can have its own -.cocciconfig file. When M= is not passed as an argument to coccicheck the -target directory is the same as the directory from where spatch was called. +Here, $srcroot refers to the source directory of the target: it points to the +external module's source directory when M= used, and otherwise, to the kernel +source directory. The third rule ensures the spatch reads the .cocciconfig from +the target directory, allowing external modules to have their own .cocciconfig +file. If not using the kernel's coccicheck target, keep the above precedence order logic of .cocciconfig reading. If using the kernel's coccicheck target,Documentation/kbuild/makefiles.rst +14
-0@@ -449,6 +449,20 @@ $(obj) to prerequisites are referenced with $(src) (because they are not generated files). +$(srcroot) + $(srcroot) refers to the root of the source you are building, which can be + either the kernel source or the external modules source, depending on whether + KBUILD_EXTMOD is set. This can be either a relative or an absolute path, but + if KBUILD_ABS_SRCTREE=1 is set, it is always an absolute path. + +$(srctree) + $(srctree) refers to the root of the kernel source tree. When building the + kernel, this is the same as $(srcroot). + +$(objtree) + $(objtree) refers to the root of the kernel object tree. It is ``.`` when + building the kernel, but it is different when building external modules. + $(kecho) echoing information to user in a rule is often a good practice but when execution ``make -s`` one does not expect to see any outputMakefile +48
-32@@ -180,7 +180,24 @@ ifeq ("$(origin O)", "command line") KBUILD_OUTPUT := $(O) endif -output := $(KBUILD_OUTPUT) +ifdef KBUILD_EXTMOD + ifdef KBUILD_OUTPUT + objtree := $(realpath $(KBUILD_OUTPUT)) + $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) + else + objtree := $(CURDIR) + endif + output := $(KBUILD_EXTMOD) + # KBUILD_EXTMOD might be a relative path. Remember its absolute path before + # Make changes the working directory. + srcroot := $(realpath $(KBUILD_EXTMOD)) + $(if $(srcroot),,$(error specified external module directory "$(KBUILD_EXTMOD)" does not exist)) +else + objtree := . + output := $(KBUILD_OUTPUT) +endif + +export objtree srcroot # Do we want to change the working directory? ifneq ($(output),)@@ -230,35 +247,33 @@ else # need-sub-make # We process the rest of the Makefile if this is the final invocation of make -ifeq ($(abs_srctree),$(CURDIR)) - # building in the source tree - srctree := . - building_out_of_srctree := +ifndef KBUILD_EXTMOD +srcroot := $(abs_srctree) +endif + +ifeq ($(srcroot),$(CURDIR)) +building_out_of_srctree := else - ifeq ($(abs_srctree)/,$(dir $(CURDIR))) - # building in a subdirectory of the source tree - srctree := .. - else - srctree := $(abs_srctree) - endif - building_out_of_srctree := 1 +export building_out_of_srctree :=1 endif -ifneq ($(KBUILD_ABS_SRCTREE),) -srctree := $(abs_srctree) +ifdef KBUILD_ABS_SRCTREE + # Do not nothing. Use the absolute path. +else ifeq ($(srcroot),$(CURDIR)) + # Building in the source. + srcroot := . +else ifeq ($(srcroot)/,$(dir $(CURDIR))) + # Building in a subdirectory of the source. + srcroot := .. endif -objtree := . +export srctree := $(if $(KBUILD_EXTMOD),$(abs_srctree),$(srcroot)) -VPATH := - -ifeq ($(KBUILD_EXTMOD),) ifdef building_out_of_srctree -VPATH := $(srctree) +export VPATH := $(srcroot) +else +VPATH := endif -endif - -export building_out_of_srctree srctree objtree VPATH # To make sure we do not include .config for any of the *config targets # catch them early, and hand them over to scripts/kconfig/Makefile@@ -711,7 +726,7 @@ endif # in addition to whatever we do anyway. # Just "make" or "make all" shall build modules as well -ifneq ($(filter all modules nsdeps %compile_commands.json clang-%,$(MAKECMDGOALS)),) +ifneq ($(filter all modules nsdeps compile_commands.json clang-%,$(MAKECMDGOALS)),) KBUILD_MODULES := 1 endif@@ -1107,7 +1122,7 @@ export MODLIB PHONY += prepare0 -export extmod_prefix = $(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD)/) +export extmod_prefix = export MODORDER := $(extmod_prefix)modules.order export MODULES_NSDEPS := $(extmod_prefix)modules.nsdeps@@ -1799,14 +1814,10 @@ filechk_kernel.release = echo $(KERNELRELEASE) KBUILD_BUILTIN := KBUILD_MODULES := 1 -build-dir := $(KBUILD_EXTMOD) +build-dir := . -compile_commands.json: $(extmod_prefix)compile_commands.json -PHONY += compile_commands.json - -clean-dirs := $(KBUILD_EXTMOD) -clean: private rm-files := $(KBUILD_EXTMOD)/Module.symvers $(KBUILD_EXTMOD)/modules.nsdeps \ - $(KBUILD_EXTMOD)/compile_commands.json +clean-dirs := . +clean: private rm-files := Module.symvers modules.nsdeps compile_commands.json PHONY += prepare # now expand this into a simple variable to reduce the cost of shell evaluations@@ -1948,7 +1959,7 @@ $(clean-dirs): clean: $(clean-dirs) $(call cmd,rmfiles) - @find $(or $(KBUILD_EXTMOD), .) $(RCS_FIND_IGNORE) \ + @find . $(RCS_FIND_IGNORE) \ \( -name '*.[aios]' -o -name '*.rsi' -o -name '*.ko' -o -name '.*.cmd' \ -o -name '*.ko.*' \ -o -name '*.dtb' -o -name '*.dtbo' \@@ -1981,7 +1992,12 @@ tags TAGS cscope gtags: FORCE PHONY += rust-analyzer rust-analyzer: +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh +ifdef KBUILD_EXTMOD +# FIXME: external modules must not descend into a sub-directory of the kernel + $(Q)$(MAKE) $(build)=$(objtree)/rust src=$(srctree)/rust $@ +else $(Q)$(MAKE) $(build)=rust $@ +endif # Script to generate missing namespace dependencies # ---------------------------------------------------------------------------rust/Makefile +2
-2@@ -362,8 +362,8 @@ rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ - $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ - $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json + $(rustc_sysroot) $(RUST_LIB_SRC) $(if $(KBUILD_EXTMOD),$(srcroot)) \ + > rust-project.json redirect-intrinsics = \ __addsf3 __eqsf2 __extendsfdf2 __gesf2 __lesf2 __ltsf2 __mulsf3 __nesf2 __truncdfsf2 __unordsf2 \scripts/Makefile.build +1
-1@@ -3,7 +3,7 @@ # Building # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := $(obj)/ $(obj)/:scripts/Makefile.clean +1
-1@@ -3,7 +3,7 @@ # Cleaning up # ========================================================================== -src := $(if $(VPATH),$(VPATH)/)$(obj) +src := $(srcroot)/$(obj) PHONY := __clean __clean:scripts/Makefile.compiler +1
-1@@ -13,7 +13,7 @@ cc-cross-prefix = $(firstword $(foreach c, $(1), \ $(if $(shell command -v -- $(c)gcc 2>/dev/null), $(c)))) # output directory for tests below -TMPOUT = $(if $(KBUILD_EXTMOD),$(firstword $(KBUILD_EXTMOD))/).tmp_$$$$ +TMPOUT = .tmp_$$$$ # try-run # Usage: option = $(call try-run, $(CC)...-o "$$TMP",option-ok,otherwise)scripts/Makefile.modpost +3
-3@@ -111,13 +111,13 @@ endif else # set src + obj - they may be used in the modules's Makefile -obj := $(KBUILD_EXTMOD) -src := $(if $(VPATH),$(VPATH)/)$(obj) +obj := . +src := $(srcroot) # Include the module's Makefile to find KBUILD_EXTRA_SYMBOLS include $(kbuild-file) -output-symdump := $(KBUILD_EXTMOD)/Module.symvers +output-symdump := Module.symvers ifeq ($(wildcard $(objtree)/Module.symvers),) missing-input := $(objtree)/Module.symversscripts/coccicheck +1
-5@@ -80,11 +80,7 @@ command results in a shift count error.' NPROC=1 else ONLINE=0 - if [ "$KBUILD_EXTMOD" = "" ] ; then - OPTIONS="--dir $srctree $COCCIINCLUDE" - else - OPTIONS="--dir $KBUILD_EXTMOD $COCCIINCLUDE" - fi + OPTIONS="--dir $srcroot $COCCIINCLUDE" # Use only one thread per core by default if hyperthreading is enabled THREADS_PER_CORE=$(LANG=C lscpu | grep "Thread(s) per core: " | tr -cd "[:digit:]")scripts/nsdeps +1
-7@@ -19,12 +19,6 @@ if ! { echo "$SPATCH_REQ_VERSION"; echo "$SPATCH_VERSION"; } | sort -CV ; then exit 1 fi -if [ "$KBUILD_EXTMOD" ]; then - src_prefix= -else - src_prefix=$srctree/ -fi - generate_deps_for_ns() { $SPATCH --very-quiet --in-place --sp-file \ $srctree/scripts/coccinelle/misc/add_namespace.cocci -D nsdeps -D ns=$1 $2@@ -34,7 +28,7 @@ generate_deps() { local mod=${1%.ko:} shift local namespaces="$*" - local mod_source_files=$(sed "s|^\(.*\)\.o$|${src_prefix}\1.c|" $mod.mod) + local mod_source_files=$(sed "s|^\(.*\)\.o$|${srcroot}/\1.c|" $mod.mod) for ns in $namespaces; do echo "Adding namespace $ns to module $mod.ko."scripts/package/install-extmod-build +7
-0@@ -51,6 +51,13 @@ mkdir -p "${destdir}" if [ "${CC}" != "${HOSTCC}" ]; then echo "Rebuilding host programs with ${CC}..." + # This leverages external module building. + # - Clear sub_make_done to allow the top-level Makefile to redo sub-make. + # - Filter out --no-print-directory to print "Entering directory" logs + # when Make changes the working directory. + unset sub_make_done + MAKEFLAGS=$(echo "${MAKEFLAGS}" | sed s/--no-print-directory//) + cat <<-'EOF' > "${destdir}/Kbuild" subdir-y := scripts EOF
PATCH v2 6/11 kbuild: remove extmod_prefix, MODORDER, MODULES_NSDEPS variables
In reply to Masahiro YamadaWith the previous changes, $(extmod_prefix), $(MODORDER), and $(MODULES_NSDEPS) are constant. (empty, modules.order, and modules.nsdeps, respectively).
Remove these variables and hard-code their values.
---
Changes in v2:
- More cleanups
Makefile | 22 +++++++++------------- scripts/Makefile.modfinal | 8 ++++---- scripts/Makefile.modinst | 6 +++--- scripts/Makefile.modpost | 6 +++--- scripts/nsdeps | 2 +- 5 files changed, 20 insertions(+), 24 deletions(-)
Makefile +9 -13
@@ -1122,10 +1122,6 @@ export MODLIB PHONY += prepare0 -export extmod_prefix = -export MODORDER := $(extmod_prefix)modules.order -export MODULES_NSDEPS := $(extmod_prefix)modules.nsdeps - ifeq ($(KBUILD_EXTMOD),) build-dir := .
@@ -1876,7 +1872,7 @@ endif ifdef CONFIG_MODULES -$(MODORDER): $(build-dir) +modules.order: $(build-dir) @: # KBUILD_MODPOST_NOFINAL can be set to skip the final link of modules.
@@ -1887,7 +1883,7 @@ ifneq ($(KBUILD_MODPOST_NOFINAL),1) endif PHONY += modules_check -modules_check: $(MODORDER) +modules_check: modules.order $(Q)$(CONFIG_SHELL) $(srctree)/scripts/modules-check.sh $< else # CONFIG_MODULES
@@ -1928,15 +1924,15 @@ $(single-ko): single_modules $(single-no-ko): $(build-dir) @: -# Remove MODORDER when done because it is not the real one. +# Remove modules.order when done because it is not the real one. PHONY += single_modules single_modules: $(single-no-ko) modules_prepare - $(Q){ $(foreach m, $(single-ko), echo $(extmod_prefix)$(m:%.ko=%.o);) } > $(MODORDER) + $(Q){ $(foreach m, $(single-ko), echo $(m:%.ko=%.o);) } > modules.order $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modpost ifneq ($(KBUILD_MODPOST_NOFINAL),1) $(Q)$(MAKE) -f $(srctree)/scripts/Makefile.modfinal endif - $(Q)rm -f $(MODORDER) + $(Q)rm -f modules.order single-goals := $(addprefix $(build-dir)/, $(single-no-ko))
@@ -2013,12 +2009,12 @@ nsdeps: modules quiet_cmd_gen_compile_commands = GEN $@ cmd_gen_compile_commands = $(PYTHON3) $< -a $(AR) -o $@ $(filter-out $<, $(real-prereqs)) -$(extmod_prefix)compile_commands.json: $(srctree)/scripts/clang-tools/gen_compile_commands.py \ +compile_commands.json: $(srctree)/scripts/clang-tools/gen_compile_commands.py \ $(if $(KBUILD_EXTMOD),, vmlinux.a $(KBUILD_VMLINUX_LIBS)) \ - $(if $(CONFIG_MODULES), $(MODORDER)) FORCE + $(if $(CONFIG_MODULES), modules.order) FORCE $(call if_changed,gen_compile_commands) -targets += $(extmod_prefix)compile_commands.json +targets += compile_commands.json PHONY += clang-tidy clang-analyzer
@@ -2026,7 +2022,7 @@ ifdef CONFIG_CC_IS_CLANG quiet_cmd_clang_tools = CHECK $< cmd_clang_tools = $(PYTHON3) $(srctree)/scripts/clang-tools/run-clang-tools.py $@ $< -clang-tidy clang-analyzer: $(extmod_prefix)compile_commands.json +clang-tidy clang-analyzer: compile_commands.json $(call cmd,clang_tools) else clang-tidy clang-analyzer:
scripts/Makefile.modfinal +4 -4
@@ -13,7 +13,7 @@ include $(srctree)/scripts/Kbuild.include include $(srctree)/scripts/Makefile.lib # find all modules listed in modules.order -modules := $(call read-file, $(MODORDER)) +modules := $(call read-file, modules.order) __modfinal: $(modules:%.o=%.ko) @:
@@ -30,7 +30,7 @@ quiet_cmd_cc_o_c = CC [M] $@ %.mod.o: %.mod.c FORCE $(call if_changed_dep,cc_o_c) -$(extmod_prefix).module-common.o: $(srctree)/scripts/module-common.c FORCE +.module-common.o: $(srctree)/scripts/module-common.c FORCE $(call if_changed_dep,cc_o_c) quiet_cmd_ld_ko_o = LD [M] $@
@@ -57,13 +57,13 @@ if_changed_except = $(if $(call newer_prereqs_except,$(2))$(cmd-check), \ printf '%s\n' 'savedcmd_$@ := $(make-cmd)' > $(dot-target).cmd, @:) # Re-generate module BTFs if either module's .ko or vmlinux changed -%.ko: %.o %.mod.o $(extmod_prefix).module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE +%.ko: %.o %.mod.o .module-common.o $(objtree)/scripts/module.lds $(and $(CONFIG_DEBUG_INFO_BTF_MODULES),$(KBUILD_BUILTIN),$(objtree)/vmlinux) FORCE +$(call if_changed_except,ld_ko_o,$(objtree)/vmlinux) ifdef CONFIG_DEBUG_INFO_BTF_MODULES +$(if $(newer-prereqs),$(call cmd,btf_ko)) endif -targets += $(modules:%.o=%.ko) $(modules:%.o=%.mod.o) $(extmod_prefix).module-common.o +targets += $(modules:%.o=%.ko) $(modules:%.o=%.mod.o) .module-common.o # Add FORCE to the prerequisites of a target to force it to be always rebuilt. # ---------------------------------------------------------------------------
scripts/Makefile.modinst +3 -3
@@ -40,7 +40,7 @@ $(addprefix $(MODLIB)/, modules.builtin modules.builtin.modinfo modules.builtin. endif -modules := $(call read-file, $(MODORDER)) +modules := $(call read-file, modules.order) ifeq ($(KBUILD_EXTMOD),) dst := $(MODLIB)/kernel
@@ -59,7 +59,7 @@ suffix-$(CONFIG_MODULE_COMPRESS_XZ) := .xz suffix-$(CONFIG_MODULE_COMPRESS_ZSTD) := .zst endif -modules := $(patsubst $(extmod_prefix)%.o, $(dst)/%.ko$(suffix-y), $(modules)) +modules := $(patsubst %.o, $(dst)/%.ko$(suffix-y), $(modules)) install-$(CONFIG_MODULES) += $(modules) __modinst: $(install-y)
@@ -119,7 +119,7 @@ endif # Create necessary directories $(foreach dir, $(sort $(dir $(install-y))), $(shell mkdir -p $(dir))) -$(dst)/%.ko: $(extmod_prefix)%.ko FORCE +$(dst)/%.ko: %.ko FORCE $(call cmd,install) $(call cmd,strip) $(call cmd,sign)
scripts/Makefile.modpost +3 -3
@@ -46,7 +46,7 @@ modpost-args = \ $(if $(CONFIG_MODULE_SRCVERSION_ALL),-a) \ $(if $(CONFIG_SECTION_MISMATCH_WARN_ONLY),,-E) \ $(if $(KBUILD_MODPOST_WARN),-w) \ - $(if $(KBUILD_NSDEPS),-d $(MODULES_NSDEPS)) \ + $(if $(KBUILD_NSDEPS),-d modules.nsdeps) \ $(if $(CONFIG_MODULE_ALLOW_MISSING_NAMESPACE_IMPORTS)$(KBUILD_NSDEPS),-N) \ $(if $(findstring 1, $(KBUILD_EXTRA_WARN)),-W) \ -o $@
@@ -61,8 +61,8 @@ endif # Read out modules.order to pass in modpost. # Otherwise, allmodconfig would fail with "Argument list too long". ifdef KBUILD_MODULES -modpost-args += -T $(MODORDER) -modpost-deps += $(MODORDER) +modpost-args += -T modules.order +modpost-deps += modules.order endif ifeq ($(KBUILD_EXTMOD),)
scripts/nsdeps +1 -1
@@ -51,4 +51,4 @@ generate_deps() { while read line do generate_deps $line -done < $MODULES_NSDEPS +done < modules.nsdeps
Signature · 1 line
2.43.0
PATCH v2 7/11 kbuild: support building external modules in a separate build directory
In reply to Masahiro YamadaThere has been a long-standing request to support building external modules in a separate build directory.
This commit introduces a new environment variable, KBUILD_EXTMOD_OUTPUT, and its shorthand Make variable, MO.
A simple usage:
$ make -C <kernel-dir> M=<module-src-dir> MO=<module-build-dir>
---
Changes in v2:
- Add -I $(src) -I $(obj) for external modules as well
- Fix a typo 'Speficies' to 'Specifies'.
Documentation/kbuild/kbuild.rst | 8 +++++++- Documentation/kbuild/modules.rst | 8 +++++++- Makefile | 22 +++++++++++++++++++--- scripts/Makefile.host | 8 +++----- scripts/Makefile.lib | 2 -- 5 files changed, 36 insertions(+), 12 deletions(-)
Documentation/kbuild/kbuild.rst +7 -1
@@ -137,12 +137,18 @@ Specify the output directory when building the kernel. This variable can also be used to point to the kernel output directory when building external modules against a pre-built kernel in a separate build directory. Please note that this does NOT specify the output directory for the -external modules themselves. +external modules themselves. (Use KBUILD_EXTMOD_OUTPUT for that purpose.) The output directory can also be specified using "O=...". Setting "O=..." takes precedence over KBUILD_OUTPUT. +KBUILD_EXTMOD_OUTPUT +-------------------- +Specify the output directory for external modules. + +Setting "MO=..." takes precedence over KBUILD_EXTMOD_OUTPUT. + KBUILD_EXTRA_WARN ----------------- Specify the extra build checks. The same value can be assigned by passing
Documentation/kbuild/modules.rst +7 -1
@@ -66,7 +66,10 @@ Options of the kernel output directory if the kernel was built in a separate build directory.) - make -C $KDIR M=$PWD + You can optionally pass MO= option if you want to build the modules in + a separate directory. + + make -C $KDIR M=$PWD [MO=$BUILD_DIR] -C $KDIR The directory that contains the kernel and relevant build
@@ -80,6 +83,9 @@ Options directory where the external module (kbuild file) is located. + MO=$BUILD_DIR + Specifies a separate output directory for the external module. + Targets -------
Makefile +19 -3
@@ -134,6 +134,10 @@ ifeq ("$(origin M)", "command line") KBUILD_EXTMOD := $(M) endif +ifeq ("$(origin MO)", "command line") + KBUILD_EXTMOD_OUTPUT := $(MO) +endif + $(if $(word 2, $(KBUILD_EXTMOD)), \ $(error building multiple external modules is not supported))
@@ -187,7 +191,7 @@ ifdef KBUILD_EXTMOD else objtree := $(CURDIR) endif - output := $(KBUILD_EXTMOD) + output := $(or $(KBUILD_EXTMOD_OUTPUT),$(KBUILD_EXTMOD)) # KBUILD_EXTMOD might be a relative path. Remember its absolute path before # Make changes the working directory. srcroot := $(realpath $(KBUILD_EXTMOD))
@@ -555,7 +559,7 @@ USERINCLUDE := \ LINUXINCLUDE := \ -I$(srctree)/arch/$(SRCARCH)/include \ -I$(objtree)/arch/$(SRCARCH)/include/generated \ - $(if $(building_out_of_srctree),-I$(srctree)/include) \ + -I$(srctree)/include \ -I$(objtree)/include \ $(USERINCLUDE)
@@ -645,6 +649,7 @@ quiet_cmd_makefile = GEN Makefile } > Makefile outputmakefile: +ifeq ($(KBUILD_EXTMOD),) @if [ -f $(srctree)/.config -o \ -d $(srctree)/include/config -o \ -d $(srctree)/arch/$(SRCARCH)/include/generated ]; then \
@@ -654,7 +659,16 @@ outputmakefile: echo >&2 "***"; \ false; \ fi - $(Q)ln -fsn $(srctree) source +else + @if [ -f $(srcroot)/modules.order ]; then \ + echo >&2 "***"; \ + echo >&2 "*** The external module source tree is not clean."; \ + echo >&2 "*** Please run 'make -C $(abs_srctree) M=$(realpath $(srcroot)) clean'"; \ + echo >&2 "***"; \ + false; \ + fi +endif + $(Q)ln -fsn $(srcroot) source $(call cmd,makefile) $(Q)test -e .gitignore || \ { echo "# this is build directory, ignore it"; echo "*"; } > .gitignore
@@ -1940,6 +1954,8 @@ KBUILD_MODULES := 1 endif +prepare: outputmakefile + # Preset locale variables to speed up the build process. Limit locale # tweaks to this spot to avoid wrong language settings when running # make menuconfig etc.
scripts/Makefile.host +3 -5
@@ -96,12 +96,10 @@ hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem)) -# $(objtree)/$(obj) for including generated headers from checkin source files -ifeq ($(KBUILD_EXTMOD),) +# $(obj) for including generated headers from checkin source files ifdef building_out_of_srctree -hostc_flags += -I $(objtree)/$(obj) -hostcxx_flags += -I $(objtree)/$(obj) -endif +hostc_flags += -I $(obj) +hostcxx_flags += -I $(obj) endif #####
scripts/Makefile.lib +0 -2
@@ -213,13 +213,11 @@ endif # $(src) for including checkin headers from generated source files # $(obj) for including generated headers from checkin source files -ifeq ($(KBUILD_EXTMOD),) ifdef building_out_of_srctree _c_flags += $(addprefix -I, $(src) $(obj)) _a_flags += $(addprefix -I, $(src) $(obj)) _cpp_flags += $(addprefix -I, $(src) $(obj)) endif -endif # If $(is-kernel-object) is 'y', this object will be linked to vmlinux or modules is-kernel-object = $(or $(part-of-builtin),$(part-of-module))
Signature · 1 line
2.43.0
PATCH v2 8/11 kbuild: support -fmacro-prefix-map for external modules
In reply to Masahiro YamadaThis commit makes -fmacro-prefix-map work for external modules built in a separate output directory. It improves the reproducibility of external modules and provides the benefits described in commit a73619a845d5 ("kbuild: use -fmacro-prefix-map to make __FILE__ a relative path").
When building_out_of_srctree is not defined (e.g., when the kernel or external module is built in the source directory), this option is unnecessary.
---
Changes in v2:
- Check building_out_of_srctree instead of VPATH
Makefile | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
Makefile +4 -2
@@ -1041,8 +1041,10 @@ ifdef CONFIG_CC_IS_GCC KBUILD_CFLAGS += -fconserve-stack endif -# change __FILE__ to the relative path from the srctree -KBUILD_CPPFLAGS += $(call cc-option,-fmacro-prefix-map=$(srctree)/=) +# change __FILE__ to the relative path to the source directory +ifdef building_out_of_srctree +KBUILD_CPPFLAGS += $(call cc-option,-fmacro-prefix-map=$(srcroot)/=) +endif # include additional Makefiles when needed include-y := scripts/Makefile.extrawarn
Signature · 1 line
2.43.0
PATCH v2 9/11 kbuild: use absolute path in the generated wrapper Makefile
In reply to Masahiro YamadaKeep the consistent behavior when this Makefile is invoked from another directory.
---
(no changes since v1)
Makefile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
Makefile +3 -2
@@ -644,8 +644,9 @@ ifdef building_out_of_srctree quiet_cmd_makefile = GEN Makefile cmd_makefile = { \ - echo "\# Automatically generated by $(srctree)/Makefile: don't edit"; \ - echo "include $(srctree)/Makefile"; \ + echo "\# Automatically generated by $(abs_srctree)/Makefile: don't edit"; \ + echo "export KBUILD_OUTPUT = $(CURDIR)"; \ + echo "include $(abs_srctree)/Makefile"; \ } > Makefile outputmakefile:
Signature · 1 line
2.43.0
PATCH v2 10/11 kbuild: make wrapper Makefile more convenient for external modules
In reply to Masahiro YamadaWhen Kbuild starts building in a separate output directory, it generates a wrapper Makefile, allowing you to invoke 'make' from the output directory.
This commit makes it more convenient, so you can invoke 'make' without M= or MO=.
First, you need to build external modules in a separate directory:
$ make M=/path/to/module/source/dir MO=/path/to/module/build/dir
Once the wrapper Makefile is generated in /path/to/module/build/dir, you can proceed as follows:
$ cd /path/to/module/build/dir $ make
---
Changes in v2:
- use $(srcroot) instead of $(_vpath)
Makefile | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-)
Makefile +11 -1
@@ -642,10 +642,20 @@ ifdef building_out_of_srctree # At the same time when output Makefile generated, generate .gitignore to # ignore whole output directory +ifdef KBUILD_EXTMOD +print_env_for_makefile = \ + echo "export KBUILD_OUTPUT = $(objtree)"; \ + echo "export KBUILD_EXTMOD = $(realpath $(srcroot))" ; \ + echo "export KBUILD_EXTMOD_OUTPUT = $(CURDIR)" +else +print_env_for_makefile = \ + echo "export KBUILD_OUTPUT = $(CURDIR)" +endif + quiet_cmd_makefile = GEN Makefile cmd_makefile = { \ echo "\# Automatically generated by $(abs_srctree)/Makefile: don't edit"; \ - echo "export KBUILD_OUTPUT = $(CURDIR)"; \ + $(print_env_for_makefile); \ echo "include $(abs_srctree)/Makefile"; \ } > Makefile
Signature · 1 line
2.43.0
PATCH v2 11/11 kbuild: allow to start building external modules in any directory
In reply to Masahiro YamadaUnless an explicit O= option is provided, external module builds must start from the kernel directory.
This can be achieved by using the -C option:
$ make -C /path/to/kernel M=/path/to/external/module
This commit allows starting external module builds from any directory, so you can also do the following:
$ make -f /path/to/kernel/Makefile M=/path/to/external/module
The key difference is that the -C option changes the working directory and parses the Makefile located there, while the -f option only specifies the Makefile to use.
As shown in the examples in Documentation/kbuild/modules.rst, external modules usually have a wrapper Makefile that allows you to build them without specifying any make arguments. The Makefile typically contains a rule as follows:
KDIR ?= /path/to/kernel
default:
$(MAKE) -C $(KDIR) M=$(CURDIR) $(MAKECMDGOALS)
The log will appear as follows:
$ make
make -C /path/to/kernel M=/path/to/external/module
make[1]: Entering directory '/path/to/kernel'
make[2]: Entering directory '/path/to/external/module'
CC [M] helloworld.o
MODPOST Module.symvers
CC [M] helloworld.mod.o
CC [M] .module-common.o
LD [M] helloworld.ko
make[2]: Leaving directory '/path/to/external/module'
make[1]: Leaving directory '/path/to/kernel'This changes the working directory twice because the -C option first switches to the kernel directory, and then Kbuild internally recurses back to the external module directory.
With this commit, the wrapper Makefile can directly include the kernel Makefile:
KDIR ?= /path/to/kernel
export KBUILD_EXTMOD := $(realpath $(dir $(lastword $(MAKEFILE_LIST))))
include $(KDIR)/MakefileThis avoids unnecessary sub-make invocations:
$ make
CC [M] helloworld.o
MODPOST Module.symvers
CC [M] helloworld.mod.o
CC [M] .module-common.o
LD [M] helloworld.ko---
Changes in v2:
- Fix typos in the commit description (s/exernal/external/)
- Fix uname -r to $(shell uname -r) in Example 3
Documentation/kbuild/modules.rst | 21 +++++++++++++++++++++ Makefile | 8 ++++++-- 2 files changed, 27 insertions(+), 2 deletions(-)
Documentation/kbuild/modules.rst +21 -0
@@ -59,6 +59,12 @@ Command Syntax $ make -C /lib/modules/`uname -r`/build M=$PWD modules_install + Starting from Linux 6.13, you can use the -f option instead of -C. This + will avoid unnecessary change of the working directory. The external + module will be output to the directory where you invoke make. + + $ make -f /lib/modules/`uname -r`/build/Makefile M=$PWD + Options -------
@@ -221,6 +227,21 @@ Separate Kbuild File and Makefile consisting of several hundred lines, and here it really pays off to separate the kbuild part from the rest. + Linux 6.13 and later support another way. The external module Makefile + can include the kernel Makefile directly, rather than invoking sub Make. + + Example 3:: + + --> filename: Kbuild + obj-m := 8123.o + 8123-y := 8123_if.o 8123_pci.o + + --> filename: Makefile + KDIR ?= /lib/modules/$(shell uname -r)/build + export KBUILD_EXTMOD := $(realpath $(dir $(lastword $(MAKEFILE_LIST)))) + include $(KDIR)/Makefile + + Building Multiple Modules -------------------------
Makefile +6 -2
@@ -189,9 +189,13 @@ ifdef KBUILD_EXTMOD objtree := $(realpath $(KBUILD_OUTPUT)) $(if $(objtree),,$(error specified kernel directory "$(KBUILD_OUTPUT)" does not exist)) else - objtree := $(CURDIR) + objtree := $(abs_srctree) endif - output := $(or $(KBUILD_EXTMOD_OUTPUT),$(KBUILD_EXTMOD)) + # If Make is invoked from the kernel directory (either kernel + # source directory or kernel build directory), external modules + # are built in $(KBUILD_EXTMOD) for backward compatibility, + # otherwise, built in the current directory. + output := $(or $(KBUILD_EXTMOD_OUTPUT),$(if $(filter $(CURDIR),$(objtree) $(abs_srctree)),$(KBUILD_EXTMOD))) # KBUILD_EXTMOD might be a relative path. Remember its absolute path before # Make changes the working directory. srcroot := $(realpath $(KBUILD_EXTMOD))
Signature · 1 line
2.43.0