kbuild: Give jobserver-exec recipes the jobserver

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Reported-by Paul E. McKenney <[email protected]>

Closes https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ open here

Fixes 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")

Assisted-by LLM

Signed-off-by Kees Cook <[email protected]>

---
 arch/alpha/boot/Makefile             | 4 ++--
 arch/arc/boot/Makefile               | 2 +-
 arch/arm64/boot/Makefile             | 2 +-
 arch/microblaze/boot/Makefile        | 2 +-
 arch/mips/boot/Makefile              | 2 +-
 arch/nios2/boot/Makefile             | 2 +-
 arch/parisc/boot/compressed/Makefile | 2 +-
 arch/riscv/boot/Makefile             | 2 +-
 arch/s390/boot/Makefile              | 2 +-
 arch/sh/boot/Makefile                | 2 +-
 arch/sh/boot/compressed/Makefile     | 2 +-
 arch/sparc/boot/Makefile             | 4 ++--
 arch/x86/boot/compressed/Makefile    | 2 +-
 arch/xtensa/boot/Makefile            | 2 +-
 14 files changed, 16 insertions(+), 16 deletions(-)

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Reported-by Paul E. McKenney <[email protected]>

Closes https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ open here

Fixes 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")

Assisted-by LLM

Signed-off-by Kees Cook <[email protected]>

Quoted 19 lines · Kees Cook

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Quoted 5 lines

Reported-by Paul E. McKenney <[email protected]>

Closes https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ open here

Fixes 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")

Assisted-by LLM

Signed-off-by Kees Cook <[email protected]>

Nice find!

Acked-by Lorenzo Stoakes (ARM) <[email protected]>

Sorry for breaking your build Paul ;)

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Reported-by Paul E. McKenney <[email protected]>

Closes https://lore.kernel.org/all/0ddb5e5e-f801-444c-aa98-0cfe814a3026@paulmck-laptop/ open here

Fixes 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available")

Assisted-by LLM

Signed-off-by Kees Cook <[email protected]>

---
 arch/alpha/boot/Makefile             | 4 ++--
 arch/arc/boot/Makefile               | 2 +-
 arch/arm64/boot/Makefile             | 2 +-
 arch/microblaze/boot/Makefile        | 2 +-
 arch/mips/boot/Makefile              | 2 +-
 arch/nios2/boot/Makefile             | 2 +-
 arch/parisc/boot/compressed/Makefile | 2 +-
 arch/riscv/boot/Makefile             | 2 +-
 arch/s390/boot/Makefile              | 2 +-
 arch/sh/boot/Makefile                | 2 +-
 arch/sh/boot/compressed/Makefile     | 2 +-
 arch/sparc/boot/Makefile             | 4 ++--
 arch/x86/boot/compressed/Makefile    | 2 +-
 arch/xtensa/boot/Makefile            | 2 +-
 14 files changed, 16 insertions(+), 16 deletions(-)

arch/alpha/boot/Makefile +2 -2

index d8dba85e606c..b050753e9015 100644
@@ -40,7 +40,7 @@ endif
 
 # Compressed kernel image
 $(obj)/vmlinux.gz: $(obj)/vmlinux FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 	@echo '  Kernel $@ is ready'
 
 $(obj)/main.o: $(obj)/ksize.h

@@ -85,7 +85,7 @@ $(obj)/vmlinux.nh: vmlinux $(OBJSTRIP) FORCE
 	$(call if_changed,objstrip)
 
 $(obj)/vmlinux.nh.gz: $(obj)/vmlinux.nh FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 $(obj)/tools/lxboot: $(obj)/bootloader $(OBJSTRIP) FORCE
 	$(call if_changed,objstrip)

arch/arc/boot/Makefile +1 -1

index 5a8550124b73..8a44c3e7ff3e 100644
@@ -23,7 +23,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
 	$(call if_changed,lzma)

arch/arm64/boot/Makefile +1 -1

index b5a08333bc57..d42d07667822 100644
@@ -26,7 +26,7 @@ $(obj)/Image.bz2: $(obj)/Image FORCE
 	$(call if_changed,bzip2)
 
 $(obj)/Image.gz: $(obj)/Image FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 $(obj)/Image.lz4: $(obj)/Image FORCE
 	$(call if_changed,lz4)

arch/microblaze/boot/Makefile +1 -1

index 2b42c370d574..e3df3a64808a 100644
@@ -14,7 +14,7 @@ $(obj)/linux.bin.ub: $(obj)/linux.bin FORCE
 	$(call if_changed,uimage)
 
 $(obj)/linux.bin.gz: $(obj)/linux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 quiet_cmd_strip = STRIP   $< $@$2
 	cmd_strip = $(STRIP) -K microblaze_start -K _end -K __log_buf \

arch/mips/boot/Makefile +1 -1

index 8473c4671702..996bf41b78de 100644
@@ -63,7 +63,7 @@ $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
 	$(call if_changed,bzip2)
 
 $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE
 	$(call if_changed,lzma)

arch/nios2/boot/Makefile +1 -1

index 29c11a06b750..1381e6e428a8 100644
@@ -18,7 +18,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/vmlinux.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 $(obj)/vmImage: $(obj)/vmlinux.gz FORCE
 	$(call if_changed,uimage)

arch/parisc/boot/compressed/Makefile +1 -1

index 14eefb5ed5d1..a7fc7d424c00 100644
@@ -55,7 +55,7 @@ suffix-$(CONFIG_KERNEL_LZO)  := lzo
 suffix-$(CONFIG_KERNEL_XZ)  := xz
 
 $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
 	$(call if_changed,bzip2_with_size)
 $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE

arch/riscv/boot/Makefile +1 -1

index 4a2d75b7e42d..b80a077a8e8f 100644
@@ -23,7 +23,7 @@ $(obj)/Image: vmlinux FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/Image.gz: $(obj)/Image FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 $(obj)/loader.o: $(src)/loader.S $(obj)/Image
 

arch/s390/boot/Makefile +1 -1

index 10b75e053a6f..e8e43ae80dd5 100644
@@ -117,7 +117,7 @@ suffix-$(CONFIG_KERNEL_XZ)  := .xz
 suffix-$(CONFIG_KERNEL_ZSTD)  := .zst
 
 $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
 	$(call if_changed,bzip2_with_size)
 $(obj)/vmlinux.bin.lz4: $(obj)/vmlinux.bin FORCE

arch/sh/boot/Makefile +1 -1

index 1f5d2df3c7e0..cf63e319b786 100644
@@ -66,7 +66,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
 	$(call if_changed,bzip2)

arch/sh/boot/compressed/Makefile +1 -1

index 58df491778b2..b85aa5f0d616 100644
@@ -39,7 +39,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 $(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE
 	$(call if_changed,bzip2_with_size)
 $(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE

arch/sparc/boot/Makefile +2 -2

index 339c42d35089..b53cf596611e 100644
@@ -23,7 +23,7 @@ ifeq ($(CONFIG_SPARC64),y)
 # Actual linking
 
 $(obj)/zImage: $(obj)/image FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 	@$(kecho) 'Kernel: $@ is ready' '(#'$(or $(KBUILD_BUILD_VERSION),`cat .version`)')'
 
 $(obj)/vmlinux.aout: vmlinux FORCE

@@ -45,7 +45,7 @@ $(obj)/image.bin: $(obj)/image FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/image.gz: $(obj)/image.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 UIMAGE_LOADADDR = $(CONFIG_UBOOT_LOAD_ADDR)
 UIMAGE_ENTRYADDR = $(CONFIG_UBOOT_ENTRY_ADDR)

arch/x86/boot/compressed/Makefile +1 -1

index 6ec5e031db1d..680ad803cba7 100644
@@ -133,7 +133,7 @@ vmlinux.bin.all-y := $(obj)/vmlinux.bin
 vmlinux.bin.all-$(CONFIG_X86_NEED_RELOCS) += $(obj)/vmlinux.relocs
 
 $(obj)/vmlinux.bin.gz: $(vmlinux.bin.all-y) FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 $(obj)/vmlinux.bin.bz2: $(vmlinux.bin.all-y) FORCE
 	$(call if_changed,bzip2_with_size)
 $(obj)/vmlinux.bin.lzma: $(vmlinux.bin.all-y) FORCE

arch/xtensa/boot/Makefile +1 -1

index d8b0fadf429a..934f9015b15f 100644
@@ -36,7 +36,7 @@ $(obj)/vmlinux.bin: vmlinux FORCE
 	$(call if_changed,objcopy)
 
 $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE
-	$(call if_changed,gzip)
+	+$(call if_changed,gzip)
 
 boot-elf: $(obj)/vmlinux.bin
 boot-redboot: $(obj)/vmlinux.bin.gz

Signature · 1 line

2.55.0

Quoted 14 lines · Kees Cook

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Quoted 4 lines

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Thank you, Kees!

My initial test of this patch (using rcutorture) still showed the failure.

I am running GCC 11.5.0 and without any patches to "make". I take it that I need to find and apply CentOS Stream 9's make-4.3-cloexec.patch?

But if I use "make -j" instead of rcutorture, this patch does work fine. Even with "make -j632".

I put rcutorture in the background using nohup and full redirection, but "nohup make -j632 > /tmp/Make.out 2>&1 < /dev/null &" works just fine. And running rcutorture in the foreground still fails.

Very strange.

Quoted 15 lines · Paul E. McKenney

Quoted 14 lines · Kees Cook

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Quoted 8 lines

Quoted 4 lines

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Thank you, Kees!

My initial test of this patch (using rcutorture) still showed the failure.

Oh! So the patch didn't fix it for you?

Quoted 5 lines

I am running GCC 11.5.0 and without any patches to "make". I take it that I need to find and apply CentOS Stream 9's make-4.3-cloexec.patch?

But if I use "make -j" instead of rcutorture, this patch does work fine. Even with "make -j632".

Without rcutorture was "make -jNN" broken for you without this patch?

Quoted 5 lines

I put rcutorture in the background using nohup and full redirection, but "nohup make -j632 > /tmp/Make.out 2>&1 < /dev/null &" works just fine. And running rcutorture in the foreground still fails.

Very strange.

O_O It only breaks ... when backgrounded? Oh my. Okay, I will try to figure that out.

Quoted 35 lines · Kees Cook

Quoted 15 lines · Paul E. McKenney

Quoted 14 lines · Kees Cook

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Quoted 8 lines

Quoted 4 lines

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Thank you, Kees!

My initial test of this patch (using rcutorture) still showed the failure.

Oh! So the patch didn't fix it for you?

Quoted 5 lines

I am running GCC 11.5.0 and without any patches to "make". I take it that I need to find and apply CentOS Stream 9's make-4.3-cloexec.patch?

But if I use "make -j" instead of rcutorture, this patch does work fine. Even with "make -j632".

Without rcutorture was "make -jNN" broken for you without this patch?

Quoted 8 lines

Quoted 5 lines

I put rcutorture in the background using nohup and full redirection, but "nohup make -j632 > /tmp/Make.out 2>&1 < /dev/null &" works just fine. And running rcutorture in the foreground still fails.

Very strange.

O_O It only breaks ... when backgrounded? Oh my. Okay, I will try to figure that out.

Apologies, let me present my results in a more organized fashion:

			foreground	background
			-next	Fix	-next	Fix

	make -jN	Bug	Good	Bug	Good

	kvm.sh [1]	Bug	Bug	Bug	Bug

[1] tools/testing/selftests/rcutorture/bin/kvm.sh --build-only --allcpus --configs TREE02 --trust-make

Here, -next is next-20261008, and Fix is with your patch on it.

Just as mystifying to me, to be honest!

Quoted 58 lines · Paul E. McKenney

Quoted 35 lines · Kees Cook

Quoted 15 lines · Paul E. McKenney

Quoted 14 lines · Kees Cook

Since commit 48ebe1cbbeb51 ("kbuild: compress the kernel with pigz if available"), cmd_gzip runs pigz through scripts/jobserver-exec, which takes the job slots make has free. GNU Make passes the jobserver pipe only to recipes it treats as recursive, but leaves --jobserver-auth in MAKEFLAGS for all of them. The make 4.3 in CentOS Stream 9 also hides the pipe in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it, runs pigz on one thread, and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

Mark the gzip recipes as recursive with "+", as commit ecab4115c44cc ("kbuild: mark rustc (and others) invocations as recursive") did for rustc. As with those, "make -n" now runs them.

Quoted 8 lines

Quoted 4 lines

Tested ARCH=x86_64 and ARCH=arm64 defconfig with GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch, GCC 16.2.0, and -j64: without this change the warning appears and pigz runs with "-p 1"; with it there is no warning, and pigz runs with "-p 54" and "-p 64".

Thank you, Kees!

My initial test of this patch (using rcutorture) still showed the failure.

Oh! So the patch didn't fix it for you?

Quoted 5 lines

I am running GCC 11.5.0 and without any patches to "make". I take it that I need to find and apply CentOS Stream 9's make-4.3-cloexec.patch?

But if I use "make -j" instead of rcutorture, this patch does work fine. Even with "make -j632".

Without rcutorture was "make -jNN" broken for you without this patch?

Quoted 8 lines

Quoted 5 lines

I put rcutorture in the background using nohup and full redirection, but "nohup make -j632 > /tmp/Make.out 2>&1 < /dev/null &" works just fine. And running rcutorture in the foreground still fails.

Very strange.

O_O It only breaks ... when backgrounded? Oh my. Okay, I will try to figure that out.

Apologies, let me present my results in a more organized fashion:

			foreground	background
			-next	Fix	-next	Fix

	make -jN	Bug	Good	Bug	Good

	kvm.sh [1]	Bug	Bug	Bug	Bug

[1] tools/testing/selftests/rcutorture/bin/kvm.sh --build-only --allcpus --configs TREE02 --trust-make

Here, -next is next-20261008, and Fix is with your patch on it.

Quoted 1 line

Just as mystifying to me, to be honest!

Ah, I think I found it, but I'm not sure why kvm.sh tripped it but bare "make" didn't. I missed the non-literal forms:

	$(call if_changed,$(compress-y))

I'll send a v2 with docs updated too.

			foreground	background
			-next	Fix	-next	Fix

	make -jN	Bug	Good	Bug	Good

	kvm.sh [1]	Bug	Bug	Bug	Bug

[1] tools/testing/selftests/rcutorture/bin/kvm.sh --build-only --allcpus --configs TREE02 --trust-make

Quoted 7 lines · Paul E. McKenney

			foreground	background
			-next	Fix	-next	Fix

	make -jN	Bug	Good	Bug	Good

	kvm.sh [1]	Bug	Bug	Bug	Bug

Quoted 1 line

[1] tools/testing/selftests/rcutorture/bin/kvm.sh --build-only --allcpus --configs TREE02 --trust-make

Thanks for the details! IIUC, the difference is the config, not foreground vs background. kvm-build.sh adds CONFIG_INITRAMFS_SOURCE, which is compressed with gzip by default. The usr/Makefile target picks that compressor through a variable, so v1 missed it. Your plain "make -jN" builds probably had no built-in initramfs, so the kernel image was the only place where pigz ran, which v1 did cover.

Hopefully v2 has all the missed targets fixed now:
https://lore.kernel.org/lkml/[email protected]/T/#m3b466b33f835119bfbd0c1b3c48d2a8a87f31691

generate_initcall_order.pl, which orders initcalls for Clang LTO, forks a child per input file, up to the job count scripts/jobserver-exec passes it in PARALLELISM. It used to run from link-vmlinux.sh, which the top-level Makefile invokes as a recursive recipe, but commit 5d45950dfbb15 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o") moved it into a recipe without "+".

GNU Make passes the jobserver pipe only to recipes it treats as recursive, and the make 4.3 in CentOS Stream 9 also hides it in sub-makes (a backport of the fix for GNU Make bug 58232), so jobserver-exec cannot open it and prints:

WARNING: Unable to reopen jobserver read-side pipe: FileNotFoundError(2, 'No such file or directory')

PARALLELISM is then left unset, and the script caps its children at the number of online CPUs instead of the free job slots.

Mark the recipe as recursive again.

Tested ARCH=x86_64 defconfig with CONFIG_LTO_CLANG_THIN=y, LLVM=1 (Clang 24.0.0git), and GNU Make 4.3 plus CentOS Stream 9's make-4.3-cloexec.patch.

Fixes 5d45950dfbb15 ("kbuild: move vmlinux.o link to scripts/Makefile.vmlinux_o")

Assisted-by LLM

Signed-off-by Kees Cook <[email protected]>

---
 scripts/Makefile.vmlinux_o | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

scripts/Makefile.vmlinux_o +0 -0

index 24a3a4fd271c..0b37bd5dc3a4 100644