kbuild: lto: fix module versionings mismatch in incremental build
PATCH 1/1 kbuild: lto: fix module versionings mismatch in incremental build
When building modules(CONFIG_...=m), I found some of module versions are incorrect and set to 0.
This can be found in build log for first clean build which shows
WARNING: EXPORT symbol "XXXX" [drivers/XXX/XXX.ko] version generation failed, symbol will not be versioned.
But in second build(incremental build), the WARNING disappeared and the module version becomes valid CRC and make someone who want to change modules without updating kernel image can't insert their modules.
The problematic code is
+ $(foreach n, $(filter-out FORCE,$^), \
+ $(if $(wildcard $(n).symversions), \
+ ; cat $(n).symversions >> [email protected]))
For example:
rm -f fs/notify/built-in.a.symversions ; rm -f fs/notify/built-in.a; \
llvm-ar cDPrST fs/notify/built-in.a fs/notify/fsnotify.o \fs/notify/notification.o fs/notify/group.o ...
foreach n shows nothing to cat into $(n).symversions because if $(wildcard $(n).symversions) return nothing, but actually they do exist during this line was executed.
-rw-r--r-- 1 root root 168580 Jun 13 19:10 fs/notify/fsnotify.o
-rw-r--r-- 1 root root 111 Jun 13 19:10 fs/notify/fsnotify.o.symversions
The reason is the $(n).symversions are generated at runtime, but Makefile wildcard function expends and checks the file exist or not during parsing the Makefile.
Thus fix this by use test shell command to check the file existence in runtime.
--- scripts/Makefile.build | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
scripts/Makefile.build +1 -2
@@ -387,8 +387,7 @@ ifeq ($(CONFIG_LTO_CLANG) $(CONFIG_MODVERSIONS),y y) cmd_update_lto_symversions = \ rm -f [email protected] \ $(foreach n, $(filter-out FORCE,$^), \ - $(if $(wildcard $(n).symversions), \ - ; cat $(n).symversions >> [email protected])) + ; test -s $(n).symversions && cat $(n).symversions >> [email protected]) else cmd_update_lto_symversions = echo >/dev/null endif
Signature · 1 line
2.18.0
Andy Lavr <[email protected]> point out there is a build failed for
M] drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifog84.o
make[6]: /bin/sh: Argument list too long
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
make[6]: *** [/scripts/Makefile.build:463:drivers/gpu/drm/amd/amdgpu/amdgpu.o] Error 127
make[5]: *** [/scripts/Makefile.build:529: drivers/gpu/drm/amd/amdgpu] Error 2
make[5]: *** Waiting for unfinished jobs....
CC [M] drivers/gpu/drm/nouveau/nvkm/engine/fifo/gpfifogf100.o
I'll fix it in patch v2.
thanks Andy
Lecopzer
Quoted 5 lines · Lecopzer Chen
When building
modules(CONFIG_...=m), I found some of module versions are incorrect and set to 0.
This can be found in build log for first clean build which showsWARNING: EXPORT symbol "XXXX" [
drivers/XXX/XXX.ko] version generation failed, symbol will not be versioned.
I'm doing this, and I don't see the problem:
$ make LLVM=1 LLVM_IAS=1 distclean $ make LLVM=1 LLVM_IAS=1 menuconfig *enable LTO* *enable a module* $ make LLVM=1 LLVM_IAS=1 -j...
What series of commands (and .config) shows this for you?
Signature · 1 line
Kees Cook
Quoted 9 lines · Lecopzer Chen
Quoted 5 lines
When building
modules(CONFIG_...=m), I found some of module versions are incorrect and set to 0.
This can be found in build log for first clean build which showsWARNING: EXPORT symbol "XXXX" [
drivers/XXX/XXX.ko] version generation failed, symbol will not be versioned.I'm doing this, and I don't see the problem:
Quoted 7 lines
$ make LLVM=1 LLVM_IAS=1 distclean $ make LLVM=1 LLVM_IAS=1 menuconfig *enable LTO* *enable a module* $ make LLVM=1 LLVM_IAS=1 -j...What series of commands (and .config) shows this for you?
Hi Kees,
Thanks for you checking.
After double checking in clean android kernel build, this causes by make version.
(I have build failed in Linux LTO,
seems it's not well support in contract to android?)
I knew Google has LTO first in Android and upstream later, and most code are same as upstream, so the env here I use Android common kernel for easily testing.
Test env is android common kernel: android12-5.4 [1] with its latest code and it builds from build.sh[2]
$ BUILD_CONFIG=common/build.config.gki.aarch64 build/build.sh + make O=.... LLVM=1 LLVM_IAS=1 DEPMOD=depmod -j12 Image modules Image.lz4
With make set to v3.81, this can be reproduced with CONFIG_TEE=m. With version >= 4.2 this is not reproducible.
Our build env default set make to v3.81, although Android uses hermetic build and v4.3 now, but Linux doesn't have such things.
Maybe we can add build time checking or comment before CFI module versioning build rules to avoid anyone struggling with this again:).
[1] https://android.googlesource.com/kernel/common/+/refs/heads/android12-5.4
[2] https://android.googlesource.com/kernel/build/+/refs/heads/master
thanks,
Lecopzer
Quoted 31 lines · Lecopzer Chen
Quoted 9 lines · Lecopzer Chen
Quoted 5 lines
When building
modules(CONFIG_...=m), I found some of module versions are incorrect and set to 0.
This can be found in build log for first clean build which showsWARNING: EXPORT symbol "XXXX" [
drivers/XXX/XXX.ko] version generation failed, symbol will not be versioned.I'm doing this, and I don't see the problem:
Quoted 7 lines
$ make LLVM=1 LLVM_IAS=1 distclean $ make LLVM=1 LLVM_IAS=1 menuconfig *enable LTO* *enable a module* $ make LLVM=1 LLVM_IAS=1 -j...What series of commands (and .config) shows this for you?
Hi Kees,
Thanks for you checking.
After double checking in clean android kernel build, this causes by make version.
(I have build failed in Linux LTO,
seems it's not well support in contract to android?)I knew Google has LTO first in Android and upstream later, and most code are same as upstream, so the env here I use Android common kernel for easily testing.
Quoted 8 lines
Test env is android common kernel: android12-5.4 [1] with its latest code and it builds from build.sh[2]
$ BUILD_CONFIG=common/build.config.gki.aarch64 build/build.sh + make O=.... LLVM=1 LLVM_IAS=1 DEPMOD=depmod -j12 Image modules Image.lz4With make set to v3.81, this can be reproduced with
CONFIG_TEE=m. With version >= 4.2 this is not reproducible.
Ah, very interesting. While there are tests in Makefile for MAKE_VERSION, if we want to do this, it should likely be extended to Kconfig, as that's where the initial version tests for things happen. We could require MAKE_VERSION >= 4.2 for LTO?
-Kees
Quoted 15 lines
Our build env default set make to v3.81, although Android uses hermetic build and v4.3 now, but Linux doesn't have such things.
Maybe we can add build time checking or comment before CFI module versioning build rules to avoid anyone struggling with this again:).
[1] https://android.googlesource.com/kernel/common/+/refs/heads/android12-5.4
[2] https://android.googlesource.com/kernel/build/+/refs/heads/masterthanks,
Lecopzer
Signature · 1 line
Kees Cook
Quoted 41 lines · Lecopzer Chen
Quoted 31 lines
Quoted 9 lines · Lecopzer Chen
Quoted 5 lines
When building
modules(CONFIG_...=m), I found some of module versions are incorrect and set to 0.
This can be found in build log for first clean build which showsWARNING: EXPORT symbol "XXXX" [
drivers/XXX/XXX.ko] version generation failed, symbol will not be versioned.I'm doing this, and I don't see the problem:
Quoted 7 lines
$ make LLVM=1 LLVM_IAS=1 distclean $ make LLVM=1 LLVM_IAS=1 menuconfig *enable LTO* *enable a module* $ make LLVM=1 LLVM_IAS=1 -j...What series of commands (and .config) shows this for you?
Hi Kees,
Thanks for you checking.
After double checking in clean android kernel build, this causes by make version.
(I have build failed in Linux LTO,
seems it's not well support in contract to android?)I knew Google has LTO first in Android and upstream later, and most code are same as upstream, so the env here I use Android common kernel for easily testing.
Quoted 8 lines
Test env is android common kernel: android12-5.4 [1] with its latest code and it builds from build.sh[2]
$ BUILD_CONFIG=common/build.config.gki.aarch64 build/build.sh + make O=.... LLVM=1 LLVM_IAS=1 DEPMOD=depmod -j12 Image modules Image.lz4With make set to v3.81, this can be reproduced with
CONFIG_TEE=m. With version >= 4.2 this is not reproducible.
Quoted 6 lines
Ah, very interesting. While there are tests in
MakefileforMAKE_VERSION, if we want to do this, it should likely be extended toKconfig, as that's where the initial version tests for things happen. We could requireMAKE_VERSION>= 4.2 for LTO?-Kees
Yes, We can imitate how CLANG_VERSION was implemented in Kconfig.
Accroding to GNU make release page[1], I've only tested for 3.81, 4.2 and 4.3.
4.2 was released in 2016, I think it's fine for LTO lowest version.
[1] https://ftp.gnu.org/gnu/make/
thanks,
Lecopzer
Quoted 5 lines · Lecopzer Chen
Yes, We can imitate how
CLANG_VERSIONwas implemented inKconfig.Accroding to GNU make release page[1], I've only tested for 3.81, 4.2 and 4.3.
4.2 was released in 2016, I think it's fine for LTO lowest version.
Okay, sounds good. Are you able to build a patch for this?
Thanks for figuring it out!
Signature · 1 line
Kees Cook
Quoted 11 lines · Lecopzer Chen
Quoted 5 lines
Yes, We can imitate how
CLANG_VERSIONwas implemented inKconfig.Accroding to GNU make release page[1], I've only tested for 3.81, 4.2 and 4.3.
4.2 was released in 2016, I think it's fine for LTO lowest version.Okay, sounds good. Are you able to build a patch for this?
Thanks for figuring it out!
Okay, I'll send a patch in a couple of weeks.
Thanks,
Lecopzer