Detect changed compiler dependencies for full rebuild

PATCH Superseded Kees Cook 2 participants 15 messages May 2, 2025 – May 13, 2025

Subsystems (4)

Kbuild User-mode Linux (UML) Gcc plugins Ubsan

lore

Share & tools

lore

Kees Cook

author maintainer

2025-05-02 22:54 UTC #0 lore raw

v2 cover Detect changed compiler dependencies for full rebuild

v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/

Hi,

This is my attempt to introduce dependencies that track the various compiler behaviors that may globally change the build that aren't represented by either compiler flags nor the compiler version (CC_VERSION_TEXT). Namely, this is to detect when the contents of a file the compiler uses changes. We have 3 such situations currently in the tree:

- If any of the GCC plugins change, we need to rebuild everything that was built with them, as they may have changed their behavior and those behaviors may need to be synchronized across all translation units. (The most obvious of these is the randstruct GCC plugin, but is true for most of them.)

- If the randstruct seed itself changes (whether for GCC plugins or Clang), the entire tree needs to be rebuilt since the randomization of structures may change between compilation units if not.

- If the integer-wrap-ignore.scl file for Clang's integer wrapping sanitizer changes, a full rebuild is needed as the coverage for wrapping types may have changed, once again cause behavior differences between compilation units.

The best way I found to deal with this is to:
- Generate a .h file that is updated when the specific dependencies change. e.g.: randstruct_hash.h depends on randstruct.seed

- Add a -I argument globally to be able to locate the .h file. e.g.: -I$(objtree)/scripts/basic

- Add a conditional -D argument for each separate case e.g.: RANDSTRUCT_CFLAGS += -DRANDSTRUCT

- Include the .h file from compiler-version.h through an #ifdef for the define e.g.:
#ifdef RANDSTUCT
#include "randstruct_hash.h"
#endif

This means that all targets gain the dependency (via fixdep), but only when the defines are active, which means they are trivially controlled by the existing CFLAGS removal mechanisms that are already being used to turn off each of the above features.

-Kees

Kees Cook (3):
gcc-plugins: Force full rebuild when plugins change
randstruct: Force full rebuild when seed changes
integer-wrap: Force full rebuild when .scl file changes

 Makefile                         |  1 +
 arch/um/Makefile                 |  2 ++
 include/linux/compiler-version.h | 10 ++++++++++
 include/linux/vermagic.h         |  1 -
 scripts/Makefile.gcc-plugins     |  2 +-
 scripts/Makefile.ubsan           |  1 +
 scripts/basic/Makefile           | 20 +++++++++++++++-----
 scripts/gcc-plugins/Makefile     |  8 ++++++++
 8 files changed, 38 insertions(+), 7 deletions(-)

Signature · 1 line

2.34.1

Kees Cook

author maintainer

2025-05-02 22:54 UTC #1 lore raw

PATCH v2 1/3 gcc-plugins: Force full rebuild when plugins change

There was no dependency between the plugins changing and the rest of the kernel being built. Enforce this by including a synthetic header file when using plugins, that is regenerated any time the plugins are built.

This cannot be included via '-include ...' because Makefiles use the "filter-out" string function, which removes individual words. Removing all instances of "-include" from the CFLAGS will cause a lot of problems. :)

Instead, use -I to include the gcc-plugins directory, and depend on the new -DGCC_PLUGINS_ENABLED flag to include the generated header file via include/linux/compiler-version.h, which is already being used to control full rebuilds. The UM build requires that the -I be explicitly added.

Signed-off-by Kees Cook <[email protected]>

---
Cc: Masahiro Yamada <[email protected]>
Cc: Nathan Chancellor <[email protected]>
Cc: Nicolas Schier <[email protected]>
Cc: <[email protected]>
Cc: <[email protected]>

---
 arch/um/Makefile                 | 1 +
 include/linux/compiler-version.h | 4 ++++
 scripts/Makefile.gcc-plugins     | 2 +-
 scripts/gcc-plugins/Makefile     | 8 ++++++++
 4 files changed, 14 insertions(+), 1 deletion(-)

arch/um/Makefile +1 -0

index 1d36a613aad8..8cc0f22ebefa 100644
@@ -72,6 +72,7 @@ USER_CFLAGS = $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
 		$(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
 		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include \
 		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__ \
+		-I$(objtree)/scripts/gcc-plugins \
 		-include $(srctree)/include/linux/compiler-version.h \
 		-include $(srctree)/include/linux/kconfig.h

include/linux/compiler-version.h +4 -0

index 573fa85b6c0c..08943df04ebb 100644
@@ -12,3 +12,7 @@
  * and add dependency on include/config/CC_VERSION_TEXT, which is touched
  * by Kconfig when the version string from the compiler changes.
  */
+
+#ifdef GCC_PLUGINS_ENABLED
+#include "gcc-plugins-deps.h"
+#endif

scripts/Makefile.gcc-plugins +1 -1

index 5b8a8378ca8a..468bb8faa9d1 100644
@@ -38,7 +38,7 @@ export DISABLE_STACKLEAK_PLUGIN
 
 # All the plugin CFLAGS are collected here in case a build target needs to
 # filter them out of the KBUILD_CFLAGS.
-GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
+GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) -I$(objtree)/scripts/gcc-plugins -DGCC_PLUGINS_ENABLED
 export GCC_PLUGINS_CFLAGS
 
 # Add the flags to the build!

scripts/gcc-plugins/Makefile +8 -0

index 320afd3cf8e8..24671d39ec90 100644
@@ -66,3 +66,11 @@ quiet_cmd_plugin_cxx_o_c = HOSTCXX $@
 
 $(plugin-objs): $(obj)/%.o: $(src)/%.c FORCE
 	$(call if_changed_dep,plugin_cxx_o_c)
+
+quiet_cmd_gcc_plugins_updated = UPDATE  $@
+      cmd_gcc_plugins_updated = echo '/* $^ */' > $(obj)/gcc-plugins-deps.h
+
+$(obj)/gcc-plugins-deps.h: $(plugin-single) $(plugin-multi) FORCE
+	$(call if_changed,gcc_plugins_updated)
+
+always-y += gcc-plugins-deps.h

Signature · 1 line

2.34.1

Masahiro Yamada

2025-05-03 06:13 UTC #2 lore raw

Re: [PATCH v2 1/3] gcc-plugins: Force full rebuild when plugins change

Quoted 58 lines · Kees Cook

There was no dependency between the plugins changing and the rest of the kernel being built. Enforce this by including a synthetic header file when using plugins, that is regenerated any time the plugins are built.
This cannot be included via '-include ...' because Makefiles use the "filter-out" string function, which removes individual words. Removing all instances of "-include" from the CFLAGS will cause a lot of problems. :)
Instead, use -I to include the gcc-plugins directory, and depend on the new -DGCC_PLUGINS_ENABLED flag to include the generated header file via include/linux/compiler-version.h, which is already being used to control full rebuilds. The UM build requires that the -I be explicitly added.
Signed-off-by Kees Cook <[email protected]>
---
Cc: Masahiro Yamada <[email protected]>
Cc: Nathan Chancellor <[email protected]>
Cc: Nicolas Schier <[email protected]>
Cc: <[email protected]>
Cc: <[email protected]>
---
 arch/um/Makefile                 | 1 +
 include/linux/compiler-version.h | 4 ++++
 scripts/Makefile.gcc-plugins     | 2 +-
 scripts/gcc-plugins/Makefile     | 8 ++++++++
 4 files changed, 14 insertions(+), 1 deletion(-)
arch/um/Makefile +1 -0
index 1d36a613aad8..8cc0f22ebefa 100644
@@ -72,6 +72,7 @@ USER_CFLAGS = $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
                $(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
                -D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include \
                -idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__ \
+               -I$(objtree)/scripts/gcc-plugins \
                -include $(srctree)/include/linux/compiler-version.h \
                -include $(srctree)/include/linux/kconfig.h
 
include/linux/compiler-version.h +4 -0
index 573fa85b6c0c..08943df04ebb 100644
@@ -12,3 +12,7 @@
  * and add dependency on include/config/CC_VERSION_TEXT, which is touched
  * by Kconfig when the version string from the compiler changes.
  */
+
+#ifdef GCC_PLUGINS_ENABLED
+#include "gcc-plugins-deps.h"
+#endif
scripts/Makefile.gcc-plugins +0 -0
index 5b8a8378ca8a..468bb8faa9d1 100644

scripts/Makefile.gcc-plugins export DISABLE_STACKLEAK_PLUGIN · line 41

Quoted 5 lines

scripts/Makefile.gcc-plugins +1 -1

@@ -38,7 +38,7 @@ export DISABLE_STACKLEAK_PLUGIN
 
 # All the plugin CFLAGS are collected here in case a build target needs to
 # filter them out of the KBUILD_CFLAGS.
-GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y))
+GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) -I$(objtree)/scripts/gcc-plugins -DGCC_PLUGINS_ENABLED

This still relies on no-space after the -I option.

Quoted 8 lines

scripts/Makefile.gcc-plugins +0 -0
 export GCC_PLUGINS_CFLAGS
 
 # Add the flags to the build!
scripts/gcc-plugins/Makefile +0 -0
index 320afd3cf8e8..24671d39ec90 100644

scripts/gcc-plugins/Makefile quiet_cmd_plugin_cxx_o_c = HOSTCXX $@ · line 71

Quoted 6 lines

scripts/gcc-plugins/Makefile +3 -0

@@ -66,3 +66,11 @@ quiet_cmd_plugin_cxx_o_c = HOSTCXX $@
 
 $(plugin-objs): $(obj)/%.o: $(src)/%.c FORCE
        $(call if_changed_dep,plugin_cxx_o_c)
+
+quiet_cmd_gcc_plugins_updated = UPDATE  $@
+      cmd_gcc_plugins_updated = echo '/* $^ */' > $(obj)/gcc-plugins-deps.h

I think 'touch' should be enough.

If some plugins are disabled, it is detected by the normal if_changed rule.

Quoted 8 lines

scripts/gcc-plugins/Makefile +5 -1

+
+$(obj)/gcc-plugins-deps.h: $(plugin-single) $(plugin-multi) FORCE
+       $(call if_changed,gcc_plugins_updated)
+
+always-y += gcc-plugins-deps.h
--

2.34.1

I think it is simpler to place the header
in include/generated/.

I attached my suggestion below:

arch/um/Makefile +0 -0

index 8cc0f22ebefa..1d36a613aad8 100644
@@ -72,7 +72,6 @@ USER_CFLAGS = $(patsubst

$(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
                $(ARCH_INCLUDE) $(MODE_INCLUDE) $(filter -I%,$(CFLAGS)) \
                -D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include \
                -idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__ \
-               -I$(objtree)/scripts/gcc-plugins \
                -include $(srctree)/include/linux/compiler-version.h \
                -include $(srctree)/include/linux/kconfig.h

include/linux/compiler-version.h +1 -1

index 08943df04ebb..ea3d533dc04a 100644
@@ -14,5 +14,5 @@
  */
 
 #ifdef GCC_PLUGINS_ENABLED
-#include "gcc-plugins-deps.h"
+#include <generated/gcc-plugins-deps.h>
 #endif

scripts/Makefile.gcc-plugins +0 -2

index 67b045a66157..f9b51c2c2158 100644
@@ -44,7 +44,7 @@ export DISABLE_ARM_SSP_PER_TASK_PLUGIN
 
 # All the plugin CFLAGS are collected here in case a build target needs to
 # filter them out of the KBUILD_CFLAGS.
-GCC_PLUGINS_CFLAGS := $(strip $(addprefix
-fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y))

$(gcc-plugin-cflags-y)) -I$(objtree)/scripts/gcc-plugins -DGCC_PLUGINS_ENABLED
+GCC_PLUGINS_CFLAGS := $(strip $(addprefix -fplugin=$(objtree)/scripts/gcc-plugins/, $(gcc-plugin-y)) $(gcc-plugin-cflags-y)) -DGCC_PLUGINS_ENABLED
export GCC_PLUGINS_CFLAGS

# Add the flags to the build!

scripts/gcc-plugins/Makefile +3 -3

index 24671d39ec90..b354c0f9f66d 100644
@@ -67,10 +67,10 @@ quiet_cmd_plugin_cxx_o_c = HOSTCXX $@
 $(plugin-objs): $(obj)/%.o: $(src)/%.c FORCE
        $(call if_changed_dep,plugin_cxx_o_c)
 
-quiet_cmd_gcc_plugins_updated = UPDATE  $@
-      cmd_gcc_plugins_updated = echo '/* $^ */' > $(obj)/gcc-plugins-deps.h
+quiet_cmd_gcc_plugins_updated = TOUCH   $@
+      cmd_gcc_plugins_updated = touch $@
 
-$(obj)/gcc-plugins-deps.h: $(plugin-single) $(plugin-multi) FORCE
+$(obj)/../../include/generated/gcc-plugins-deps.h: $(plugin-single)

$(plugin-multi) FORCE

        $(call if_changed,gcc_plugins_updated)

-always-y += gcc-plugins-deps.h
+always-y += ../../include/generated/gcc-plugins-deps.h

--
Best Regards
Masahiro Yamada

Kees Cook

author maintainer

2025-05-03 17:25 UTC #3 lore raw

Re: [PATCH v2 1/3] gcc-plugins: Force full rebuild when plugins change

Quoted 7 lines · Masahiro Yamada

Review
Quoted 2 lines · Kees Cook
(quoted diff) +2 -0
+quiet_cmd_gcc_plugins_updated = UPDATE  $@
+      cmd_gcc_plugins_updated = echo '/* $^ */' > $(obj)/gcc-plugins-deps.h
I think 'touch' should be enough.
If some plugins are disabled, it is detected by the normal if_changed rule.

I kind of likely having the active plugins show up in there, but yes, "touch" is enough (it's what I started with originally).

Quoted 11 lines

Quoted 7 lines
+$(obj)/gcc-plugins-deps.h: $(plugin-single) $(plugin-multi) FORCE
(quoted diff) +3 -1
+       $(call if_changed,gcc_plugins_updated)
+
+always-y += gcc-plugins-deps.h
--
2.34.1
I think it is simpler to place the header
in include/generated/.

I couldn't figure out how to do this, but thankfully you did! :)

Quoted 12 lines

I attached my suggestion below:
[...]

scripts/gcc-plugins/Makefile +3 -3

-quiet_cmd_gcc_plugins_updated = UPDATE  $@
-      cmd_gcc_plugins_updated = echo '/* $^ */' > $(obj)/gcc-plugins-deps.h
+quiet_cmd_gcc_plugins_updated = TOUCH   $@
+      cmd_gcc_plugins_updated = touch $@
 
-$(obj)/gcc-plugins-deps.h: $(plugin-single) $(plugin-multi) FORCE
+$(obj)/../../include/generated/gcc-plugins-deps.h: $(plugin-single)

$(plugin-multi) FORCE

        $(call if_changed,gcc_plugins_updated)

Review

Quoted 2 lines

(quoted diff) +1 -1

-always-y += gcc-plugins-deps.h
+always-y += ../../include/generated/gcc-plugins-deps.h

Aaagh, thank you! I didn't even consider trying ".." for targets!

Yes, this is SO much better! I will read your other replies and work on v3...

Signature · 1 line

Kees Cook

Kees Cook

author maintainer

2025-05-02 22:54 UTC #4 lore raw

PATCH v2 2/3 randstruct: Force full rebuild when seed changes

In reply to Kees Cook

While the randstruct GCC plugin was being rebuilt if the randstruct seed changed, Clangs build did not notice the change. Include the hash header directly so that it becomes a universal build dependency and full rebuilds will happen if it changes.

Since we cannot use "-include ..." as the randstruct flags are removed via "filter-out" (which would cause all instances of "-include" to be removed), use the existing -DRANDSTRUCT to control the header inclusion via include/linux/compiler-version.h. Universally add a -I for the scripts/basic directory, where header exists. The UM build requires that the -I be explicitly added.

Signed-off-by Kees Cook <[email protected]>

---
Cc: Masahiro Yamada <[email protected]>
Cc: Nathan Chancellor <[email protected]>
Cc: Nicolas Schier <[email protected]>
Cc: Petr Pavlu <[email protected]>
Cc: Sebastian Andrzej Siewior <[email protected]>
Cc: <[email protected]>

---
 Makefile                         |  1 +
 arch/um/Makefile                 |  1 +
 include/linux/compiler-version.h |  3 +++
 include/linux/vermagic.h         |  1 -
 scripts/basic/Makefile           | 11 ++++++-----
 5 files changed, 11 insertions(+), 6 deletions(-)

Makefile +1 -0

index 5aa9ee52a765..cef652227843 100644
@@ -567,6 +567,7 @@ LINUXINCLUDE    := \
 		-I$(objtree)/arch/$(SRCARCH)/include/generated \
 		-I$(srctree)/include \
 		-I$(objtree)/include \
+		-I$(objtree)/scripts/basic \
 		$(USERINCLUDE)
 
 KBUILD_AFLAGS   := -D__ASSEMBLY__ -fno-PIE

arch/um/Makefile +1 -0

index 8cc0f22ebefa..38f6024e75d7 100644
@@ -73,6 +73,7 @@ USER_CFLAGS = $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
 		-D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include \
 		-idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__ \
 		-I$(objtree)/scripts/gcc-plugins \
+		-I$(objtree)/scripts/basic \
 		-include $(srctree)/include/linux/compiler-version.h \
 		-include $(srctree)/include/linux/kconfig.h

include/linux/compiler-version.h +3 -0

index 08943df04ebb..05d555320a0f 100644
@@ -16,3 +16,6 @@
 #ifdef GCC_PLUGINS_ENABLED
 #include "gcc-plugins-deps.h"
 #endif
+#ifdef RANDSTRUCT
+#include "randstruct_hash.h"
+#endif

include/linux/vermagic.h +0 -1

index 939ceabcaf06..335c360d4f9b 100644
@@ -33,7 +33,6 @@
 #define MODULE_VERMAGIC_MODVERSIONS ""
 #endif
 #ifdef RANDSTRUCT
-#include <generated/randstruct_hash.h>
 #define MODULE_RANDSTRUCT "RANDSTRUCT_" RANDSTRUCT_HASHED_SEED
 #else
 #define MODULE_RANDSTRUCT

scripts/basic/Makefile +6 -5

index dd289a6725ac..31637ce4dc5c 100644
@@ -8,9 +8,10 @@ hostprogs-always-y	+= fixdep
 # before running a Clang kernel build.
 gen-randstruct-seed	:= $(srctree)/scripts/gen-randstruct-seed.sh
 quiet_cmd_create_randstruct_seed = GENSEED $@
-cmd_create_randstruct_seed = \
-	$(CONFIG_SHELL) $(gen-randstruct-seed) \
-		$@ $(objtree)/include/generated/randstruct_hash.h
-$(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE
+      cmd_create_randstruct_seed = $(CONFIG_SHELL) $(gen-randstruct-seed) \
+		$(obj)/randstruct.seed $(obj)/randstruct_hash.h
+
+$(obj)/randstruct_hash.h $(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE
 	$(call if_changed,create_randstruct_seed)
-always-$(CONFIG_RANDSTRUCT) += randstruct.seed
+
+always-$(CONFIG_RANDSTRUCT) += randstruct.seed randstruct_hash.h

Signature · 1 line

2.34.1

Masahiro Yamada

2025-05-03 06:13 UTC #5 lore raw

Re: [PATCH v2 2/3] randstruct: Force full rebuild when seed changes

Quoted 34 lines · Kees Cook

While the randstruct GCC plugin was being rebuilt if the randstruct seed changed, Clangs build did not notice the change. Include the hash header directly so that it becomes a universal build dependency and full rebuilds will happen if it changes.
Since we cannot use "-include ..." as the randstruct flags are removed via "filter-out" (which would cause all instances of "-include" to be removed), use the existing -DRANDSTRUCT to control the header inclusion via include/linux/compiler-version.h. Universally add a -I for the scripts/basic directory, where header exists. The UM build requires that the -I be explicitly added.
Signed-off-by Kees Cook <[email protected]>
---
Cc: Masahiro Yamada <[email protected]>
Cc: Nathan Chancellor <[email protected]>
Cc: Nicolas Schier <[email protected]>
Cc: Petr Pavlu <[email protected]>
Cc: Sebastian Andrzej Siewior <[email protected]>
Cc: <[email protected]>
---
 Makefile                         |  1 +
 arch/um/Makefile                 |  1 +
 include/linux/compiler-version.h |  3 +++
 include/linux/vermagic.h         |  1 -
 scripts/basic/Makefile           | 11 ++++++-----
 5 files changed, 11 insertions(+), 6 deletions(-)
Makefile +0 -0
index 5aa9ee52a765..cef652227843 100644

Makefile LINUXINCLUDE := \ · line 570

Quoted 4 lines

Makefile +1 -0

@@ -567,6 +567,7 @@ LINUXINCLUDE    := \
                -I$(objtree)/arch/$(SRCARCH)/include/generated \
                -I$(srctree)/include \
                -I$(objtree)/include \
+               -I$(objtree)/scripts/basic \

Now you are adding random header search paths everywhere. This is very hacky.

I recommend keeping <generated/randstruct_hash.h>

Then, -I$(objtree)/scripts/basic is unneeded.

Quoted 45 lines

Makefile +0 -0

                $(USERINCLUDE)
 
 KBUILD_AFLAGS   := -D__ASSEMBLY__ -fno-PIE

arch/um/Makefile +1 -0

index 8cc0f22ebefa..38f6024e75d7 100644
@@ -73,6 +73,7 @@ USER_CFLAGS = $(patsubst $(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
                -D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include \
                -idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__ \
                -I$(objtree)/scripts/gcc-plugins \
+               -I$(objtree)/scripts/basic \
                -include $(srctree)/include/linux/compiler-version.h \
                -include $(srctree)/include/linux/kconfig.h

include/linux/compiler-version.h +3 -0

index 08943df04ebb..05d555320a0f 100644
@@ -16,3 +16,6 @@
 #ifdef GCC_PLUGINS_ENABLED
 #include "gcc-plugins-deps.h"
 #endif
+#ifdef RANDSTRUCT
+#include "randstruct_hash.h"
+#endif

include/linux/vermagic.h +0 -1

index 939ceabcaf06..335c360d4f9b 100644
@@ -33,7 +33,6 @@
 #define MODULE_VERMAGIC_MODVERSIONS ""
 #endif
 #ifdef RANDSTRUCT
-#include <generated/randstruct_hash.h>
 #define MODULE_RANDSTRUCT "RANDSTRUCT_" RANDSTRUCT_HASHED_SEED
 #else
 #define MODULE_RANDSTRUCT

scripts/basic/Makefile +0 -0

index dd289a6725ac..31637ce4dc5c 100644
@@ -8,9 +8,10 @@ hostprogs-always-y     += fixdep
 # before running a Clang kernel build.
 gen-randstruct-seed    := $(srctree)/scripts/gen-randstruct-seed.sh

scripts/basic/Makefile hostprogs-always-y += fixdep · line 15

Quoted 10 lines

scripts/basic/Makefile +4 -4

 quiet_cmd_create_randstruct_seed = GENSEED $@
-cmd_create_randstruct_seed = \
-       $(CONFIG_SHELL) $(gen-randstruct-seed) \
-               $@ $(objtree)/include/generated/randstruct_hash.h
-$(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE
+      cmd_create_randstruct_seed = $(CONFIG_SHELL) $(gen-randstruct-seed) \
+               $(obj)/randstruct.seed $(obj)/randstruct_hash.h
+
+$(obj)/randstruct_hash.h $(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE
        $(call if_changed,create_randstruct_seed)

This is wrong.

$(obj)/randstruct_hash.h $(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE

         $(call if_changed,create_randstruct_seed)

is equivalent to:

$(obj)/randstruct_hash.h: $(gen-randstruct-seed) FORCE

         $(call if_changed,create_randstruct_seed)

$(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE

         $(call if_changed,create_randstruct_seed)

So, this rule is executed twice; for randstruct_hash.h and for randstruct.seed

randstruct_hash.h and randstruct.seed will contain different hash values.

I recommend keeping the current code.

--
Best Regards
Masahiro Yamada

Kees Cook

author maintainer

2025-05-03 17:27 UTC #6 lore raw

Re: [PATCH v2 2/3] randstruct: Force full rebuild when seed changes

Quoted 9 lines · Masahiro Yamada

Quoted 2 lines · Kees Cook
+$(obj)/randstruct_hash.h $(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE
        $(call if_changed,create_randstruct_seed)
[...]
So, this rule is executed twice; for randstruct_hash.h and for randstruct.seed
randstruct_hash.h and randstruct.seed will contain different hash values.
I recommend keeping the current code.

Agreed -- getting stuff into generated is much preferred. Thank you!

Signature · 1 line

Kees Cook

Kees Cook

author maintainer

2025-05-02 22:54 UTC #7 lore raw

PATCH v2 3/3 integer-wrap: Force full rebuild when .scl file changes

In reply to Kees Cook

Since the integer wrapping sanitizer's behavior depends on its associated .scl file, we must force a full rebuild if the file changes. Universally include a synthetic header file that is rebuilt when the .scl file changes, via compiler-version.h, since using "-include ..." is not possible in the case of having compiler flags removed via "filter-out" (which would remove all instances of "-include").

Signed-off-by Kees Cook <[email protected]>

---
Cc: Masahiro Yamada <[email protected]>
Cc: Justin Stitt <[email protected]>
Cc: Nathan Chancellor <[email protected]>
Cc: Nicolas Schier <[email protected]>
Cc: Marco Elver <[email protected]>
Cc: Andrey Konovalov <[email protected]>
Cc: Andrey Ryabinin <[email protected]>
Cc: <[email protected]>
Cc: <[email protected]>
Cc: <[email protected]>

---
 include/linux/compiler-version.h | 3 +++
 scripts/Makefile.ubsan           | 1 +
 scripts/basic/Makefile           | 9 +++++++++
 3 files changed, 13 insertions(+)

include/linux/compiler-version.h +3 -0

index 05d555320a0f..9d6b1890ffc7 100644
@@ -19,3 +19,6 @@
 #ifdef RANDSTRUCT
 #include "randstruct_hash.h"
 #endif
+#ifdef INTEGER_WRAP
+#include "integer-wrap.h"
+#endif

scripts/Makefile.ubsan +1 -0

index 9e35198edbf0..653f7117819c 100644
@@ -15,6 +15,7 @@ ubsan-cflags-$(CONFIG_UBSAN_TRAP)		+= $(call cc-option,-fsanitize-trap=undefined
 export CFLAGS_UBSAN := $(ubsan-cflags-y)
 
 ubsan-integer-wrap-cflags-$(CONFIG_UBSAN_INTEGER_WRAP)     +=	\
+	-DINTEGER_WRAP						\
 	-fsanitize-undefined-ignore-overflow-pattern=all	\
 	-fsanitize=signed-integer-overflow			\
 	-fsanitize=unsigned-integer-overflow			\

scripts/basic/Makefile +9 -0

index 31637ce4dc5c..04f5620a3f8b 100644
@@ -15,3 +15,12 @@ $(obj)/randstruct_hash.h $(obj)/randstruct.seed: $(gen-randstruct-seed) FORCE
 	$(call if_changed,create_randstruct_seed)
 
 always-$(CONFIG_RANDSTRUCT) += randstruct.seed randstruct_hash.h
+
+# integer-wrap: if the .scl file changes, we need to do a full rebuild.
+quiet_cmd_integer_wrap_updated = UPDATE  $@
+      cmd_integer_wrap_updated = echo '/* $^ */' > $(obj)/integer-wrap.h
+
+$(obj)/integer-wrap.h: $(srctree)/scripts/integer-wrap-ignore.scl FORCE
+	$(call if_changed,integer_wrap_updated)
+
+always-$(CONFIG_UBSAN_INTEGER_WRAP) += integer-wrap.h

Signature · 1 line

2.34.1

Masahiro Yamada

2025-05-03 09:40 UTC #8 lore raw

In reply to Kees Cook

Quoted 4 lines · Kees Cook

v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/

What do you think of my patch as a prerequisite?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Perhaps, can you implement this series more simply?

My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.

When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal.

Signature · 2 lines

Best Regards
Masahiro Yamada

Kees Cook

author maintainer

2025-05-03 17:37 UTC #9 lore raw

Quoted 11 lines · Masahiro Yamada

Quoted 4 lines · Kees Cook
v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/
What do you think of my patch as a prerequisite?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Perhaps, can you implement this series more simply?

Quoted 6 lines

My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.
When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal.

This is roughly where I started when trying to implement this, but I didn't like the ergonomics of needing to scatter "touch" calls all over, which was especially difficult for targets that shared a build rule but may not all need to trigger a global rebuild. But what ultimately pushed me away from it was when I needed to notice if a non-built source file changed (the Clang .scl file), and I saw that I need to be dependency driven rather than target driven. (Though perhaps there is a way to address this with your global-rebuild.h?)

As far as doing a full rebuild, if it had been available last week, I probably would have used it, but now given the work that Nicolas, you, and I have put into this, we have a viable way (I think) to make this more specific. It does end up being a waste of time/resources to rebuild stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that does add up when I'm iterating on something that keeps triggering a full rebuild. We already have to do the argument filtering for targets that don't want randstruct, etc, so why not capitalize on that and make the rebuild avoid those files too?

So, I think the global-rebuild.h idea is a good one (though I think it should maybe be included in compiler-version.h just to avoid yet more compiler command line arguments), I'd really like to try to have the specific dependency-based way to get it done.

I'll send a v3, and see what you think?

-Kees

Signature · 1 line

Kees Cook

Masahiro Yamada

2025-05-08 16:44 UTC #10 lore raw

Quoted 29 lines · Kees Cook

Quoted 11 lines · Masahiro Yamada
Quoted 4 lines · Kees Cook
v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/
What do you think of my patch as a prerequisite?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Perhaps, can you implement this series more simply?
Quoted 6 lines
My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.
When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal.
This is roughly where I started when trying to implement this, but I didn't like the ergonomics of needing to scatter "touch" calls all over, which was especially difficult for targets that shared a build rule but may not all need to trigger a global rebuild. But what ultimately pushed me away from it was when I needed to notice if a non-built source file changed (the Clang .scl file), and I saw that I need to be dependency driven rather than target driven. (Though perhaps there is a way to address this with your global-rebuild.h?)

Quoted 9 lines

As far as doing a full rebuild, if it had been available last week, I probably would have used it, but now given the work that Nicolas, you, and I have put into this, we have a viable way (I think) to make this more specific. It does end up being a waste of time/resources to rebuild stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that does add up when I'm iterating on something that keeps triggering a full rebuild. We already have to do the argument filtering for targets that don't want randstruct, etc, so why not capitalize on that and make the rebuild avoid those files too?

efi-stub, vdso are very small.

Unless this turns out to be painful, I prefer
a simpler implementation.

You will see how .scl file is handled.

See the below code:

Kbuild +1 -0

index f327ca86990c..85747239314c 100644
@@ -67,10 +67,20 @@ targets += $(atomic-checks)
 $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/%  FORCE
        $(call if_changed,check_sha1)
 
+rebuild-$(CONFIG_GCC_PLUGINS)          += $(addprefix

scripts/gcc-plugins/, $(GCC_PLUGIN))
+rebuild-$(CONFIG_RANDSTRUCT) += include/generated/randstruct_hash.h
+rebuild-$(CONFIG_UBSAN_INTEGER_WRAP) += scripts/integer-wrap-ignore.scl
+
+quiet_cmd_touch = TOUCH $@
+ cmd_touch = touch $@
+
+include/generated/global-rebuild.h: $(rebuild-y)
+ $(call cmd,touch)
+
# A phony target that depends on all the preparation targets

PHONY += prepare
-prepare: $(offsets-file) missing-syscalls $(atomic-checks) +prepare: $(offsets-file) missing-syscalls $(atomic-checks) include/generated/global-rebuild.h

@:

# Ordinary directory descending

Makefile +8 -1

index b29cc321ffd9..f963a72b0761 100644
@@ -558,7 +558,8 @@ USERINCLUDE    := \
                -I$(srctree)/include/uapi \
                -I$(objtree)/include/generated/uapi \
                 -include $(srctree)/include/linux/compiler-version.h \
-                -include $(srctree)/include/linux/kconfig.h
+                -include $(srctree)/include/linux/kconfig.h \
+                -include $(objtree)/include/generated/global-rebuild.h
 
 # Use LINUXINCLUDE when you must reference the include/ directory.
 # Needed to be compatible with the O= option

@@ -1250,6 +1251,12 @@ endif
 include/config/kernel.release: FORCE
        $(call filechk,kernel.release)
 
+quiet_cmd_touch = TOUCH   $@
+      cmd_touch = touch $@
+
+include/generated/global-rebuild.h:
+       $(call cmd,touch)
+
 # Additional helpers built in scripts/
 # Carefully list dependencies so we do not try to build scripts twice
 # in parallel

@@ -1266,6 +1273,7 @@ scripts: scripts_basic scripts_dtc
 PHONY += prepare archprepare
 
 archprepare: outputmakefile archheaders archscripts scripts

include/config/kernel.release \

+       include/generated/global-rebuild.h \
        asm-generic $(version_h) include/generated/utsrelease.h \
        include/generated/compile.h include/generated/autoconf.h \
        include/generated/rustc_cfg remove-stale-files

arch/um/Makefile +0 -0

index 1d36a613aad8..f564a26c1364 100644
@@ -73,7 +73,8 @@ USER_CFLAGS = $(patsubst

$(KERNEL_DEFINES),,$(patsubst -I%,,$(KBUILD_CFLAGS))) \
                -D_FILE_OFFSET_BITS=64 -idirafter $(srctree)/include \
                -idirafter $(objtree)/include -D__KERNEL__ -D__UM_HOST__ \
                -include $(srctree)/include/linux/compiler-version.h \
-               -include $(srctree)/include/linux/kconfig.h
+               -include $(srctree)/include/linux/kconfig.h \
+               -include $(objtree)/include/generated/global-rebuild.h

#This will adjust *FLAGS accordingly to the platform. include $(srctree)/$(ARCH_DIR)/Makefile-os-Linux

Signature · 2 lines

Best Regards
Masahiro Yamada

Kees Cook

author maintainer

2025-05-08 16:56 UTC #11 lore raw

Quoted 56 lines · Masahiro Yamada

Quoted 29 lines · Kees Cook
Quoted 11 lines · Masahiro Yamada
Quoted 4 lines · Kees Cook
v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/
What do you think of my patch as a prerequisite?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Perhaps, can you implement this series more simply?
Quoted 6 lines
My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.
When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal.
This is roughly where I started when trying to implement this, but I didn't like the ergonomics of needing to scatter "touch" calls all over, which was especially difficult for targets that shared a build rule but may not all need to trigger a global rebuild. But what ultimately pushed me away from it was when I needed to notice if a non-built source file changed (the Clang .scl file), and I saw that I need to be dependency driven rather than target driven. (Though perhaps there is a way to address this with your global-rebuild.h?)
Quoted 9 lines
As far as doing a full rebuild, if it had been available last week, I probably would have used it, but now given the work that Nicolas, you, and I have put into this, we have a viable way (I think) to make this more specific. It does end up being a waste of time/resources to rebuild stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that does add up when I'm iterating on something that keeps triggering a full rebuild. We already have to do the argument filtering for targets that don't want randstruct, etc, so why not capitalize on that and make the rebuild avoid those files too?
efi-stub, vdso are very small.
Unless this turns out to be painful, I prefer
a simpler implementation.
You will see how .scl file is handled.
See the below code:
Kbuild +0 -0
index f327ca86990c..85747239314c 100644

Kbuild targets += $(atomic-checks) · line 72

Quoted 6 lines

Kbuild +2 -0

@@ -67,10 +67,20 @@ targets += $(atomic-checks)
 $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/%  FORCE
        $(call if_changed,check_sha1)
 
+rebuild-$(CONFIG_GCC_PLUGINS)          += $(addprefix
 scripts/gcc-plugins/, $(GCC_PLUGIN))
+rebuild-$(CONFIG_RANDSTRUCT)           += include/generated/randstruct_hash.h

These are in $(objtree)

Kbuild targets += $(atomic-checks) · line 73

Quoted 1 line

Kbuild +1 -0

+rebuild-$(CONFIG_UBSAN_INTEGER_WRAP)   += scripts/integer-wrap-ignore.scl

This is in $(srctree)

Kbuild targets += $(atomic-checks) · line 79

Quoted 6 lines

Kbuild +6 -0

+
+quiet_cmd_touch = TOUCH   $@
+      cmd_touch = touch $@
+
+include/generated/global-rebuild.h: $(rebuild-y)
+       $(call cmd,touch)

Is this rule going to find the right versions of the dependencies?

Makefile USERINCLUDE := \ · line 562

Quoted 9 lines

Makefile +2 -1

@@ -558,7 +558,8 @@ USERINCLUDE    := \
                -I$(srctree)/include/uapi \
                -I$(objtree)/include/generated/uapi \
                 -include $(srctree)/include/linux/compiler-version.h \
-                -include $(srctree)/include/linux/kconfig.h
+                -include $(srctree)/include/linux/kconfig.h \
+                -include $(objtree)/include/generated/global-rebuild.h

Instead of adding a new file, why not just touch compiler-version.h?

But whatever the case, sure, I can live with this. :)

-Kees

Signature · 1 line

Kees Cook

Masahiro Yamada

2025-05-08 23:13 UTC #12 lore raw

Quoted 61 lines · Kees Cook

Quoted 59 lines · Masahiro Yamada
Quoted 29 lines · Kees Cook
Quoted 11 lines · Masahiro Yamada
Quoted 4 lines · Kees Cook
v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/
What do you think of my patch as a prerequisite?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Perhaps, can you implement this series more simply?
Quoted 6 lines
My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.
When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal.
This is roughly where I started when trying to implement this, but I didn't like the ergonomics of needing to scatter "touch" calls all over, which was especially difficult for targets that shared a build rule but may not all need to trigger a global rebuild. But what ultimately pushed me away from it was when I needed to notice if a non-built source file changed (the Clang .scl file), and I saw that I need to be dependency driven rather than target driven. (Though perhaps there is a way to address this with your global-rebuild.h?)
Quoted 9 lines
As far as doing a full rebuild, if it had been available last week, I probably would have used it, but now given the work that Nicolas, you, and I have put into this, we have a viable way (I think) to make this more specific. It does end up being a waste of time/resources to rebuild stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that does add up when I'm iterating on something that keeps triggering a full rebuild. We already have to do the argument filtering for targets that don't want randstruct, etc, so why not capitalize on that and make the rebuild avoid those files too?
efi-stub, vdso are very small.
Unless this turns out to be painful, I prefer
a simpler implementation.
You will see how .scl file is handled.
See the below code:
Kbuild +0 -0
index f327ca86990c..85747239314c 100644
@@ -67,10 +67,20 @@ targets += $(atomic-checks)
 $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/%  FORCE
        $(call if_changed,check_sha1)
 

Quoted 5 lines

Kbuild targets += $(atomic-checks) · line 72
Quoted 3 lines
Kbuild +2 -0
+rebuild-$(CONFIG_GCC_PLUGINS)          += $(addprefix
 scripts/gcc-plugins/, $(GCC_PLUGIN))
+rebuild-$(CONFIG_RANDSTRUCT)           += include/generated/randstruct_hash.h
These are in $(objtree)

Yes.

Quoted 3 lines

Kbuild targets += $(atomic-checks) · line 73
Quoted 1 line
Kbuild +1 -0
+rebuild-$(CONFIG_UBSAN_INTEGER_WRAP)   += scripts/integer-wrap-ignore.scl
This is in $(srctree)

Yes.

Quoted 8 lines

Kbuild targets += $(atomic-checks) · line 79
Quoted 6 lines
Kbuild +6 -0
+
+quiet_cmd_touch = TOUCH   $@
+      cmd_touch = touch $@
+
+include/generated/global-rebuild.h: $(rebuild-y)
+       $(call cmd,touch)
Is this rule going to find the right versions of the dependencies?

I think so, but please test it.

Quoted 11 lines

Makefile USERINCLUDE := \ · line 562

Quoted 9 lines

Makefile +2 -1

@@ -558,7 +558,8 @@ USERINCLUDE    := \
                -I$(srctree)/include/uapi \
                -I$(objtree)/include/generated/uapi \
                 -include $(srctree)/include/linux/compiler-version.h \
-                -include $(srctree)/include/linux/kconfig.h
+                -include $(srctree)/include/linux/kconfig.h \
+                -include $(objtree)/include/generated/global-rebuild.h

Instead of adding a new file, why not just touch compiler-version.h?

Because the compiler-version.h is in $(srctree), which might be in the read-only file system.

Quoted 6 lines

But whatever the case, sure, I can live with this. :)
-Kees
--
Kees Cook

Signature · 2 lines

Best Regards
Masahiro Yamada

Kees Cook

author maintainer

2025-05-08 23:59 UTC #13 lore raw

Quoted 85 lines · Masahiro Yamada

Quoted 61 lines · Kees Cook
Quoted 59 lines · Masahiro Yamada
Quoted 29 lines · Kees Cook
Quoted 11 lines · Masahiro Yamada
Quoted 4 lines · Kees Cook
v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/
What do you think of my patch as a prerequisite?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Perhaps, can you implement this series more simply?
Quoted 6 lines
My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.
When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal.
This is roughly where I started when trying to implement this, but I didn't like the ergonomics of needing to scatter "touch" calls all over, which was especially difficult for targets that shared a build rule but may not all need to trigger a global rebuild. But what ultimately pushed me away from it was when I needed to notice if a non-built source file changed (the Clang .scl file), and I saw that I need to be dependency driven rather than target driven. (Though perhaps there is a way to address this with your global-rebuild.h?)
Quoted 9 lines
As far as doing a full rebuild, if it had been available last week, I probably would have used it, but now given the work that Nicolas, you, and I have put into this, we have a viable way (I think) to make this more specific. It does end up being a waste of time/resources to rebuild stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that does add up when I'm iterating on something that keeps triggering a full rebuild. We already have to do the argument filtering for targets that don't want randstruct, etc, so why not capitalize on that and make the rebuild avoid those files too?
efi-stub, vdso are very small.
Unless this turns out to be painful, I prefer
a simpler implementation.
You will see how .scl file is handled.
See the below code:
Kbuild +0 -0
index f327ca86990c..85747239314c 100644
@@ -67,10 +67,20 @@ targets += $(atomic-checks)
 $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/%  FORCE
        $(call if_changed,check_sha1)
 
Quoted 5 lines
Kbuild targets += $(atomic-checks) · line 72
Quoted 3 lines
Kbuild +2 -0
+rebuild-$(CONFIG_GCC_PLUGINS)          += $(addprefix
 scripts/gcc-plugins/, $(GCC_PLUGIN))
+rebuild-$(CONFIG_RANDSTRUCT)           += include/generated/randstruct_hash.h
These are in $(objtree)
Yes.
Quoted 3 lines
Kbuild targets += $(atomic-checks) · line 73
Quoted 1 line
Kbuild +1 -0
+rebuild-$(CONFIG_UBSAN_INTEGER_WRAP)   += scripts/integer-wrap-ignore.scl
This is in $(srctree)
Yes.
Quoted 8 lines
Kbuild targets += $(atomic-checks) · line 79
Quoted 6 lines
Kbuild +6 -0
+
+quiet_cmd_touch = TOUCH   $@
+      cmd_touch = touch $@
+
+include/generated/global-rebuild.h: $(rebuild-y)
+       $(call cmd,touch)
Is this rule going to find the right versions of the dependencies?

Quoted 1 line

I think so, but please test it.

The patch was white-space damaged and wrapped, but I rebuilt it manually and it mostly works. There still seems to be some ordering issues, as some stuff gets rebuilt on a record build:

# Clean the tree and pick an "everything" build

$ make O=gcc-test clean allmodconfig -s

# Make a target normally

$ make O=gcc-test kernel/seccomp.o -s

# Touch a gcc plugin that was in .config

$ touch scripts/gcc-plugins/stackleak_plugin.c

# Build and a full rebuild is triggered (good)

$ make O=gcc-test kernel/seccomp.o
make[1]: Entering directory '/srv/code/gcc-test'
  GEN     Makefile
  DESCEND objtool
  HOSTCXX scripts/gcc-plugins/stackleak_plugin.so
  INSTALL libsubcmd_headers
  TOUCH   include/generated/global-rebuild.h
  CC      kernel/bounds.s
  CC      arch/x86/kernel/asm-offsets.s
  CALL    ../scripts/checksyscalls.sh
  CC      kernel/seccomp.o
make[1]: Leaving directory '/srv/code/gcc-test'

# Build again, but more stuff gets built

$ make O=gcc-test kernel/seccomp.o
make[1]: Entering directory '/srv/code/gcc-test'
  GEN     Makefile
  DESCEND objtool
  CC      scripts/mod/empty.o
  CC      scripts/mod/devicetable-offsets.s
  INSTALL libsubcmd_headers
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTCC  scripts/mod/symsearch.o
  HOSTCC  scripts/mod/file2alias.o
  HOSTLD  scripts/mod/modpost
  CALL    ../scripts/checksyscalls.sh
make[1]: Leaving directory '/srv/code/gcc-test'

# Third time finally everything is stable

$ hmake O=gcc-test kernel/seccomp.o
make[1]: Entering directory '/srv/code/gcc-test'
  GEN     Makefile
  DESCEND objtool
  CALL    ../scripts/checksyscalls.sh
  INSTALL libsubcmd_headers
make[1]: Leaving directory '/srv/code/gcc-test'

Note that scripts/mod/* gets rebuilt on the second rebuild.

Signature · 1 line

Kees Cook

Masahiro Yamada

2025-05-13 14:53 UTC #14 lore raw

Quoted 143 lines · Kees Cook

Quoted 85 lines · Masahiro Yamada
Quoted 61 lines · Kees Cook
Quoted 59 lines · Masahiro Yamada
Quoted 29 lines · Kees Cook
Quoted 11 lines · Masahiro Yamada
Quoted 4 lines · Kees Cook
v2:
- switch from -include to -I with a -D gated include compiler-version.h
v1: https://lore.kernel.org/lkml/[email protected]/
What do you think of my patch as a prerequisite?
https://lore.kernel.org/linux-kbuild/[email protected]/T/#u
Perhaps, can you implement this series more simply?
Quoted 6 lines
My idea is to touch a single include/generated/global-rebuild.h rather than multiple files such as gcc-plugins-deps.h, integer-wrap.h, etc.
When the file is touched, the entire kernel source tree will be rebuilt. This may rebuild more than needed (e.g. vdso) but I do not think it is a big deal.
This is roughly where I started when trying to implement this, but I didn't like the ergonomics of needing to scatter "touch" calls all over, which was especially difficult for targets that shared a build rule but may not all need to trigger a global rebuild. But what ultimately pushed me away from it was when I needed to notice if a non-built source file changed (the Clang .scl file), and I saw that I need to be dependency driven rather than target driven. (Though perhaps there is a way to address this with your global-rebuild.h?)
Quoted 9 lines
As far as doing a full rebuild, if it had been available last week, I probably would have used it, but now given the work that Nicolas, you, and I have put into this, we have a viable way (I think) to make this more specific. It does end up being a waste of time/resources to rebuild stuff that doesn't need to be (efi-stub, vdso, boot code, etc), and that does add up when I'm iterating on something that keeps triggering a full rebuild. We already have to do the argument filtering for targets that don't want randstruct, etc, so why not capitalize on that and make the rebuild avoid those files too?
efi-stub, vdso are very small.
Unless this turns out to be painful, I prefer
a simpler implementation.
You will see how .scl file is handled.
See the below code:
Kbuild +0 -0
index f327ca86990c..85747239314c 100644
@@ -67,10 +67,20 @@ targets += $(atomic-checks)
 $(atomic-checks): $(obj)/.checked-%: include/linux/atomic/%  FORCE
        $(call if_changed,check_sha1)
 
Quoted 5 lines
Kbuild targets += $(atomic-checks) · line 72
Quoted 3 lines
Kbuild +2 -0
+rebuild-$(CONFIG_GCC_PLUGINS)          += $(addprefix
 scripts/gcc-plugins/, $(GCC_PLUGIN))
+rebuild-$(CONFIG_RANDSTRUCT)           += include/generated/randstruct_hash.h
These are in $(objtree)
Yes.
Quoted 3 lines
Kbuild targets += $(atomic-checks) · line 73
Quoted 1 line
Kbuild +1 -0
+rebuild-$(CONFIG_UBSAN_INTEGER_WRAP)   += scripts/integer-wrap-ignore.scl
This is in $(srctree)
Yes.
Quoted 8 lines
Kbuild targets += $(atomic-checks) · line 79
Quoted 6 lines
Kbuild +6 -0
+
+quiet_cmd_touch = TOUCH   $@
+      cmd_touch = touch $@
+
+include/generated/global-rebuild.h: $(rebuild-y)
+       $(call cmd,touch)
Is this rule going to find the right versions of the dependencies?
Quoted 1 line
I think so, but please test it.
The patch was white-space damaged and wrapped, but I rebuilt it manually and it mostly works. There still seems to be some ordering issues, as some stuff gets rebuilt on a record build:
# Clean the tree and pick an "everything" build
$ make O=gcc-test clean allmodconfig -s
# Make a target normally
$ make O=gcc-test kernel/seccomp.o -s
# Touch a gcc plugin that was in .config
$ touch scripts/gcc-plugins/stackleak_plugin.c
# Build and a full rebuild is triggered (good)
$ make O=gcc-test kernel/seccomp.o
make[1]: Entering directory '/srv/code/gcc-test'
  GEN     Makefile
  DESCEND objtool
  HOSTCXX scripts/gcc-plugins/stackleak_plugin.so
  INSTALL libsubcmd_headers
  TOUCH   include/generated/global-rebuild.h
  CC      kernel/bounds.s
  CC      arch/x86/kernel/asm-offsets.s
  CALL    ../scripts/checksyscalls.sh
  CC      kernel/seccomp.o
make[1]: Leaving directory '/srv/code/gcc-test'
# Build again, but more stuff gets built
$ make O=gcc-test kernel/seccomp.o
make[1]: Entering directory '/srv/code/gcc-test'
  GEN     Makefile
  DESCEND objtool
  CC      scripts/mod/empty.o
  CC      scripts/mod/devicetable-offsets.s
  INSTALL libsubcmd_headers
  MKELF   scripts/mod/elfconfig.h
  HOSTCC  scripts/mod/modpost.o
  HOSTCC  scripts/mod/sumversion.o
  HOSTCC  scripts/mod/symsearch.o
  HOSTCC  scripts/mod/file2alias.o
  HOSTLD  scripts/mod/modpost
  CALL    ../scripts/checksyscalls.sh
make[1]: Leaving directory '/srv/code/gcc-test'
# Third time finally everything is stable
$ hmake O=gcc-test kernel/seccomp.o
make[1]: Entering directory '/srv/code/gcc-test'
  GEN     Makefile
  DESCEND objtool
  CALL    ../scripts/checksyscalls.sh
  INSTALL libsubcmd_headers
make[1]: Leaving directory '/srv/code/gcc-test'

Quoted 1 line

Note that scripts/mod/* gets rebuilt on the second rebuild.

Hmm.
OK, my code did not work.

I accept your patch set (although I am not a big fan of the added complexity...)

Could you move the normalize_path macro
to scripts/Kbuild?

Signature · 2 lines

Best Regards
Masahiro Yamada