kbuild: rebuild GCC plugins when the compiler is upgraded
Subsystems (2)
PATCH 1/1 kbuild: rebuild GCC plugins when the compiler is upgraded
Linus reported a build error due to the GCC plugin incompatibility when the compiler is upgraded. [1]
cc1: error: incompatible gcc/plugin versions
cc1: error: failed to initialize plugin ./scripts/gcc-plugins/stackleak_plugin.so
GCC plugins are tied to a particular GCC version. So, they must be rebuilt when the compiler is upgraded.
This seems to be a long-standing flaw since the initial support of GCC plugins.
Extend commit 8b59cd81dc5e ("kbuild: ensure full rebuild when the compiler is updated"), so that GCC plugins are covered by the compiler upgrade detection.
[1]: https://lore.kernel.org/lkml/CAHk-=wieoN5ttOy7SnsGwZv+Fni3R6m-Ut=oxih6bbZ28G+4dw@mail.gmail.com/
---
Makefile | 1 + include/linux/compiler-version.h | 14 ++++++++++++++ include/linux/kconfig.h | 2 -- init/Kconfig | 8 ++++---- scripts/gcc-plugins/Makefile | 1 + 5 files changed, 20 insertions(+), 6 deletions(-) create mode 100644 include/linux/compiler-version.h
Makefile +1 -0
@@ -490,6 +490,7 @@ USERINCLUDE := \ -I$(objtree)/arch/$(SRCARCH)/include/generated/uapi \ -I$(srctree)/include/uapi \ -I$(objtree)/include/generated/uapi \ + -include $(srctree)/include/linux/compiler-version.h \ -include $(srctree)/include/linux/kconfig.h # Use LINUXINCLUDE when you must reference the include/ directory.
include/linux/compiler-version.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0-only */ + +#ifdef __LINUX_COMPILER_VERSION_H +#error "Please do not include <linux/compiler-version.h>. This is done by the build system." +#endif +#define __LINUX_COMPILER_VERSION_H + +/* +* This header exists to force full rebuild when the compiler is upgraded. +* +* When fixdep scans this, it will find this string "CONFIG_CC_VERSION_TEXT" +* and add dependency on include/config/cc/version/text.h, which is touched +* by Kconfig when the version string from the compiler changes. +*/
include/linux/kconfig.h +0 -2
@@ -2,8 +2,6 @@#ifndef __LINUX_KCONFIG_H#define __LINUX_KCONFIG_H -/* CONFIG_CC_VERSION_TEXT (Do not delete this comment. See help in Kconfig) */ -#include <generated/autoconf.h> #ifdef CONFIG_CPU_BIG_ENDIAN
init/Kconfig +4 -4
@@ -20,10 +20,10 @@ config CC_VERSION_TEXT When the compiler is updated, Kconfig will be invoked. - Ensure full rebuild when the compiler is updated - include/linux/kconfig.h contains this option in the comment line so - fixdep adds include/config/cc/version/text.h into the auto-generated - dependency. When the compiler is updated, syncconfig will touch it - and then every file will be rebuilt. + include/linux/compiler-version.h contains this option in the comment + line so fixdep adds include/config/cc/version/text.h into the + auto-generated dependency. When the compiler is updated, syncconfig + will touch it and then every file will be rebuilt. config CC_IS_GCC def_bool $(success,test "$(cc-name)" = GCC)
scripts/gcc-plugins/Makefile +1 -0
@@ -22,6 +22,7 @@ always-y += $(GCC_PLUGIN) GCC_PLUGINS_DIR = $(shell $(CC) -print-file-name=plugin) plugin_cxxflags = -Wp,-MMD,$(depfile) $(KBUILD_HOSTCXXFLAGS) -fPIC \ + -include $(srctree)/include/linux/compiler-version.h \ -I $(GCC_PLUGINS_DIR)/include -I $(obj) -std=gnu++11 \ -fno-rtti -fno-exceptions -fasynchronous-unwind-tables \ -ggdb -Wno-narrowing -Wno-unused-variable \
Signature · 1 line
2.27.0
Quoted 12 lines · Masahiro Yamada
Linus reported a build error due to the GCC plugin incompatibility when the compiler is upgraded. [1]
cc1: error: incompatible gcc/plugin versions
cc1: error: failed to initialize plugin./scripts/gcc-plugins/stackleak_plugin.soGCC plugins are tied to a particular GCC version. So, they must be rebuilt when the compiler is upgraded.
This seems to be a long-standing flaw since the initial support of GCC plugins.
Quoted 8 lines
Extend commit
8b59cd81dc5e("kbuild: ensure full rebuild when the compiler is updated"), so that GCC plugins are covered by the compiler upgrade detection.[1]: https://lore.kernel.org/lkml/CAHk-=wieoN5ttOy7SnsGwZv+Fni3R6m-Ut=oxih6bbZ28G+4dw@mail.gmail.com/
Reported-by Linus Torvalds <[email protected]>Signed-off-by Masahiro Yamada <[email protected]>
This seems fine to me, but I want to make sure Josh has somewhere to actually go with this. Josh, does this get you any closer? It sounds like the plugins need to move to another location for packaged kernels?
Signature · 1 line
Kees Cook
Quoted 4 lines · Kees Cook
This seems fine to me, but I want to make sure Josh has somewhere to actually go with this. Josh, does this get you any closer? It sounds like the plugins need to move to another location for packaged kernels?
Well, it might be worth extending the stuff that gets installed with /lib/modules/<kernel-version>/ with enough information and infrastruvcture to then build any external modules.
Yeah, yeah, there's the /boot/config-<kernelversion> thing too, but honestly, that probably shouldn't have been in /boot to begin with. That, and the plugins - and possibly any hashes needed for randstruct etc - might be good to squirrell away in a /lib/modules/<kernelversion>/build/ subdirectory?
I dunno. I obviously don't build external modules to begin with, much less with distro kernels...
Linus
Quoted 4 lines · Linus Torvalds
Quoted 3 lines · Kees Cook
This seems fine to me, but I want to make sure Josh has somewhere to actually go with this. Josh, does this get you any closer?
No, this doesn't seem to help me at all.
Quoted 6 lines
Quoted 2 lines
It sounds like the plugins need to move to another location for packaged kernels?
Well, it might be worth extending the stuff that gets installed with /lib/modules/<kernel-version>/ with enough information and infrastruvcture to then build any external modules.
The gcc plugins live in scripts/, which get installed by "make modules_install" already. So the plugins' source and makefiles are in /lib/modules/<kernel-version>/build/scripts/gcc-plugins.
So everything needed for building the plugins is already there. We just need the kernel makefiles to rebuild the plugins locally, when building an external module.
Signature · 1 line
Josh
Quoted 15 lines · Josh Poimboeuf
Quoted 4 lines · Linus Torvalds
Quoted 3 lines · Kees Cook
This seems fine to me, but I want to make sure Josh has somewhere to actually go with this. Josh, does this get you any closer?
No, this doesn't seem to help me at all.
Quoted 6 lines
Quoted 2 lines
It sounds like the plugins need to move to another location for packaged kernels?
Well, it might be worth extending the stuff that gets installed with /lib/modules/<kernel-version>/ with enough information and infrastruvcture to then build any external modules.
Quoted 7 lines
The gcc plugins live in
scripts/, which get installed by "makemodules_install" already. So the plugins' source and makefiles are in /lib/modules/<kernel-version>/build/scripts/gcc-plugins.So everything needed for building the plugins is already there. We just need the kernel makefiles to rebuild the plugins locally, when building an external module.
This seems to work with very limited testing... Based on top of Masahiro's recent patch:
https://lkml.kernel.org/r/CAK7LNARHoTnZ3gAvHgnYB4n-wYuboxC10A6zURh1ODGhxWd2yA@mail.gmail.com
From: Josh Poimboeuf <[email protected]>
Subject: [PATCH] gcc-plugins: Rebuild plugins in external module directory
When building external kernel modules, the build system doesn't require the GCC version to match the version used to build the original kernel.
In fact, most distros release the compiler and the kernel in separate packages, with separate release cadences. So it's not uncommon for mismatches to occur.
But with GCC plugins enabled, that's no longer allowed:
cc1: error: incompatible gcc/plugin versions
cc1: error: failed to initialize plugin ./scripts/gcc-plugins/structleak_plugin.so
That error comes from the plugin's call to plugin_default_version_check(), which strictly enforces the GCC version. The strict check makes sense, because there's nothing to prevent the GCC plugin ABI from changing, and it often does.
Since plugins are tightly tied to the compiler version, just rebuild them locally in the external module directory, and then use the local version in the external module build.
--- Makefile | 1 + scripts/Makefile.gcc-plugins | 2 +- scripts/gcc-plugins/Makefile | 8 ++++++-- 3 files changed, 8 insertions(+), 3 deletions(-)
Makefile +1 -0
@@ -1784,6 +1784,7 @@ prepare: echo " The kernel was built by: "$(CONFIG_CC_VERSION_TEXT); \ echo " You are using: $(CC_VERSION_TEXT)"; \ fi + $(Q)$(MAKE) $(build)=scripts/gcc-plugins PHONY += help help:
scripts/Makefile.gcc-plugins +1 -1
@@ -48,7 +48,7 @@ export DISABLE_ARM_SSP_PER_TASK_PLUGIN # All the plugin CFLAGS are collected here in case a build target needs to # filter them out of the KBUILD_CFLAGS. -GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) +GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD),$(objtree))/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) # The sancov_plugin.so is included via CFLAGS_KCOV, so it is removed here. GCC_PLUGINS_CFLAGS := $(filter-out %/sancov_plugin.so, $(GCC_PLUGINS_CFLAGS)) export GCC_PLUGINS_CFLAGS
scripts/gcc-plugins/Makefile +6 -2
@@ -1,10 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 -$(obj)/randomize_layout_plugin.so: $(objtree)/$(obj)/randomize_layout_seed.h +ifneq ($(KBUILD_EXTMOD),) +override obj := $(KBUILD_EXTMOD)/$(obj) +endif + +$(obj)/randomize_layout_plugin.so: $(objtree)/$(src)/randomize_layout_seed.h quiet_cmd_create_randomize_layout_seed = GENSEED $@ cmd_create_randomize_layout_seed = \ $(CONFIG_SHELL) $(srctree)/$(src)/gen-random-seed.sh $@ $(objtree)/include/generated/randomize_layout_hash.h -$(objtree)/$(obj)/randomize_layout_seed.h: FORCE +$(objtree)/$(src)/randomize_layout_seed.h: FORCE $(call if_changed,create_randomize_layout_seed) targets += randomize_layout_seed.h randomize_layout_hash.h
Signature · 1 line
2.29.2
Quoted 21 lines · Josh Poimboeuf
Quoted 19 lines · Josh Poimboeuf
Quoted 4 lines · Linus Torvalds
Quoted 3 lines · Kees Cook
This seems fine to me, but I want to make sure Josh has somewhere to actually go with this. Josh, does this get you any closer?
No, this doesn't seem to help me at all.
Quoted 6 lines
Quoted 2 lines
It sounds like the plugins need to move to another location for packaged kernels?
Well, it might be worth extending the stuff that gets installed with /lib/modules/<kernel-version>/ with enough information and infrastruvcture to then build any external modules.
The gcc plugins live in
scripts/, which get installed by "makemodules_install" already. So the plugins' source and makefiles are in /lib/modules/<kernel-version>/build/scripts/gcc-plugins.
Quoted 8 lines
Quoted 3 lines
So everything needed for building the plugins is already there. We just need the kernel makefiles to rebuild the plugins locally, when building an external module.
This seems to work with very limited testing... Based on top of Masahiro's recent patch:
https://lkml.kernel.org/r/CAK7LNARHoTnZ3gAvHgnYB4n-wYuboxC10A6zURh1ODGhxWd2yA@mail.gmail.com
Is this a bad coding contest?
I am not asking you to add ugly ifeq or whatever
hacks to say "this worked for me".
Please feel free to do this in the fedora kernel,
but do not send it to upstream.
Sorry, I really do not want to see hacks like this any more.
Remember, how badly objtool was integrated in the build system, and you even blocked me from fixing that.
Quoted 81 lines
From: Josh Poimboeuf <[email protected]>
Subject: [PATCH] gcc-plugins: Rebuild plugins in external module directoryWhen building external kernel modules, the build system doesn't require the GCC version to match the version used to build the original kernel.
In fact, most distros release the compiler and the kernel in separate packages, with separate release cadences. So it's not uncommon for mismatches to occur.
But with GCC plugins enabled, that's no longer allowed:
cc1: error: incompatible gcc/plugin versions
cc1: error: failed to initialize plugin./scripts/gcc-plugins/structleak_plugin.soThat error comes from the plugin's call to
plugin_default_version_check(), which strictly enforces the GCC version. The strict check makes sense, because there's nothing to prevent the GCC plugin ABI from changing, and it often does.Since plugins are tightly tied to the compiler version, just rebuild them locally in the external module directory, and then use the local version in the external module build.
Reported-by Ondrej Mosnacek <[email protected]>Signed-off-by Josh Poimboeuf <[email protected]>--- Makefile | 1 + scripts/Makefile.gcc-plugins | 2 +- scripts/gcc-plugins/Makefile | 8 ++++++-- 3 files changed, 8 insertions(+), 3 deletions(-)Makefile +1
-0@@ -1784,6 +1784,7 @@ prepare: echo " The kernel was built by: "$(CONFIG_CC_VERSION_TEXT); \ echo " You are using: $(CC_VERSION_TEXT)"; \ fi + $(Q)$(MAKE) $(build)=scripts/gcc-plugins PHONY += help help:scripts/Makefile.gcc-plugins +1
-1@@ -48,7 +48,7 @@ export DISABLE_ARM_SSP_PER_TASK_PLUGIN # All the plugin CFLAGS are collected here in case a build target needs to # filter them out of the KBUILD_CFLAGS. -GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) +GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(if $(KBUILD_EXTMOD),$(KBUILD_EXTMOD),$(objtree))/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) # The sancov_plugin.so is included via CFLAGS_KCOV, so it is removed here. GCC_PLUGINS_CFLAGS := $(filter-out %/sancov_plugin.so, $(GCC_PLUGINS_CFLAGS)) export GCC_PLUGINS_CFLAGSscripts/gcc-plugins/Makefile +6
-3@@ -1,10 +1,14 @@ # SPDX-License-Identifier: GPL-2.0 -$(obj)/randomize_layout_plugin.so: $(objtree)/$(obj)/randomize_layout_seed.h +ifneq ($(KBUILD_EXTMOD),) +override obj := $(KBUILD_EXTMOD)/$(obj) +endif + +$(obj)/randomize_layout_plugin.so: $(objtree)/$(src)/randomize_layout_seed.h quiet_cmd_create_randomize_layout_seed = GENSEED $@ cmd_create_randomize_layout_seed = \ $(CONFIG_SHELL) $(srctree)/$(src)/gen-random-seed.sh $@ $(objtree)/include/generated/randomize_layout_hash.h -$(objtree)/$(obj)/randomize_layout_seed.h: FORCE +$(objtree)/$(src)/randomize_layout_seed.h: FORCE $(call if_changed,create_randomize_layout_seed) targets += randomize_layout_seed.h randomize_layout_hash.h --2.29.2
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 33 lines · Masahiro Yamada
Quoted 21 lines · Josh Poimboeuf
Quoted 19 lines · Josh Poimboeuf
Quoted 4 lines · Linus Torvalds
Quoted 3 lines · Kees Cook
This seems fine to me, but I want to make sure Josh has somewhere to actually go with this. Josh, does this get you any closer?
No, this doesn't seem to help me at all.
Quoted 6 lines
Quoted 2 lines
It sounds like the plugins need to move to another location for packaged kernels?
Well, it might be worth extending the stuff that gets installed with /lib/modules/<kernel-version>/ with enough information and infrastruvcture to then build any external modules.
The gcc plugins live in
scripts/, which get installed by "makemodules_install" already. So the plugins' source and makefiles are in /lib/modules/<kernel-version>/build/scripts/gcc-plugins.Quoted 8 lines
Quoted 3 lines
So everything needed for building the plugins is already there. We just need the kernel makefiles to rebuild the plugins locally, when building an external module.
This seems to work with very limited testing... Based on top of Masahiro's recent patch:
https://lkml.kernel.org/r/CAK7LNARHoTnZ3gAvHgnYB4n-wYuboxC10A6zURh1ODGhxWd2yA@mail.gmail.com
Is this a bad coding contest?
Quoted 7 lines
I am not asking you to add ugly ifeq or whatever
hacks to say "this worked for me".Please feel free to do this in the fedora kernel,
but do not send it to upstream.Sorry, I really do not want to see hacks like this any more.
Geez, that's a bit harsh. I never claimed to be a brilliant makefile coder. Do you have any constructive feedback for improving the patch?
Quoted 2 lines
Remember, how badly objtool was integrated in the build system, and you even blocked me from fixing that.
I have no idea what you're talking about, nor how it would be relevant to this patch.
Signature · 1 line
Josh
Quoted 9 lines · Josh Poimboeuf
Quoted 9 lines
Is this a bad coding contest?
I am not asking you to add ugly ifeq or whatever
hacks to say "this worked for me".Please feel free to do this in the fedora kernel,
but do not send it to upstream.Sorry, I really do not want to see hacks like this any more.
Masahiro,
Ping. Do you have a better approach for building GCC plugins in the external module directory?
Signature · 1 line
Josh
Quoted 10 lines · Josh Poimboeuf
Quoted 8 lines · Josh Poimboeuf
Quoted 8 lines
Is this a bad coding contest?
I am not asking you to add ugly ifeq or whatever
hacks to say "this worked for me".Please feel free to do this in the fedora kernel,
but do not send it to upstream.
Quoted 6 lines
Quoted 1 line
Quoted 1 line
Sorry, I really do not want to see hacks like this any more.
Masahiro,
Ping. Do you have a better approach for building GCC plugins in the external module directory?
I am not sure if building GCC plugins in the external module directory is the right approach.
--
Best Regards
Masahiro Yamada
Quoted 8 lines · Masahiro Yamada
Quoted 4 lines
Masahiro,
Ping. Do you have a better approach for building GCC plugins in the external module directory?
I am not sure if building GCC plugins in the external module directory is the right approach.
I'm certainly open to any better ideas that would allow GCC plugins to be enabled in distro kernel builds.
Signature · 1 line
Josh