kbuild: modinst: remove the multithread option from zstd compression
PATCH 1/1 kbuild: modinst: remove the multithread option from zstd compression
Parallel execution is supported by GNU Make:
$ make -j<N> modules_install
It is questionable to enable multithreading within each zstd process by default.
If you still want to do it, you can use the environment variable:
$ ZSTD_NBTHREADS=<N> make modules_install
---
scripts/Makefile.modinst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
scripts/Makefile.modinst +1 -1
@@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@ quiet_cmd_xz = XZ $@ cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $< quiet_cmd_zstd = ZSTD $@ - cmd_zstd = $(ZSTD) -T0 --rm -f -q $< + cmd_zstd = $(ZSTD) --rm -f -q $< $(dst)/%.ko.gz: $(dst)/%.ko FORCE $(call cmd,gzip)
Signature · 1 line
2.43.0
Quoted 12 lines · Masahiro Yamada
Parallel execution is supported by GNU Make:
$ make -j<N> modules_installIt is questionable to enable multithreading within each zstd process by default.
If you still want to do it, you can use the environment variable:
$ ZSTD_NBTHREADS=<N> make modules_install
Hi Masahiro,
I have some understanding problems.
[ start-build.txt ]
dileks 24225 24217 0 17:55 tty2 00:00:00 /usr/bin/perf stat
make V=1 -k -j4 ARCH=x86_64 LLVM=1 LLVM=/opt/llvm/bin/ PAHOLE=/opt/pahole/bin/pahole KBUILD_BUILD_HOST=iniza [email protected] KBUILD_BUILD_TIMESTAMP=2024-08-03 KDEB_PKGVERSION=6.10.3-1~trixie+dileks1 LOCALVERSION=-1-amd64-clang18-kcfi olddefconfig bindeb-pkg
^^ How shall someone pass so that ... ZSTD_NBTHREADS=<N> make modules_install ... is used?
As far as I understood, each kernel-module file is taken - in the Debian build-process - sequentially file for file - ZSTD compressed and afterwards deleted.
Is there a benefit when 'make -j<N>' is used?
[ EXAMPLE - with my patch from [1] ]
zstd -T0 -19 --rm -f -q debian/linux-image-6.10.3-1-amd64-clang18-kcfi/lib/modules/6.10.3-1-amd64-clang18-kcfi/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko
^^ How should ZSTD_NBTHREADS=<N> replace 'zstd -T0'?
When I pass '-T0 -L19' to ZSTD two cores are used on my CPU and a better compress-level produces smaller kernel-module files. I would like to benefit from both.
Thanks.
Best regards,
-Sedat-
Quoted 23 lines
Signed-off-by Masahiro Yamada <[email protected]>---
scripts/Makefile.modinst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/Makefile.modinst +1
-2@@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@ quiet_cmd_xz = XZ $@ cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $< quiet_cmd_zstd = ZSTD $@ - cmd_zstd = $(ZSTD) -T0 --rm -f -q $< + cmd_zstd = $(ZSTD) --rm -f -q $< $(dst)/%.ko.gz: $(dst)/%.ko FORCE $(call cmd,gzip) --2.43.0
Quoted 28 lines · Sedat Dilek
Quoted 12 lines · Masahiro Yamada
Parallel execution is supported by GNU Make:
$ make -j<N> modules_installIt is questionable to enable multithreading within each zstd process by default.
If you still want to do it, you can use the environment variable:
$ ZSTD_NBTHREADS=<N> make modules_installHi Masahiro,
I have some understanding problems.
[ start-build.txt ]
dileks 24225 24217 0 17:55 tty2 00:00:00/usr/bin/perfstat
make V=1 -k -j4 ARCH=x86_64 LLVM=1 LLVM=/opt/llvm/bin/ PAHOLE=/opt/pahole/bin/pahole KBUILD_BUILD_HOST=iniza [email protected] KBUILD_BUILD_TIMESTAMP=2024-08-03 KDEB_PKGVERSION=6.10.3-1~trixie+dileks1 LOCALVERSION=-1-amd64-clang18-kcfi olddefconfig bindeb-pkg
Quoted 2 lines
^^ How shall someone pass so that ... ZSTD_NBTHREADS=<N> make
modules_install... is used?
Option 1
$ export ZSTD_NBTHREADS=<N> $ make V=1 -k [snip] olddefconfig bindeb-pkg
Option 2
$ ZSTD_NBTHREADS=<N> make V=1 -k [snip] olddefconfig bindeb-pkg
Quoted 5 lines
As far as I understood, each kernel-module file is taken - in the Debian build-process - sequentially file for file - ZSTD compressed and afterwards deleted.
Is there a benefit when 'make -j<N>' is used?
zstd --rm -f -q foo.ko
zstd --rm -f -q bar.ko
zstd --rm -f -q baz.ko
will run in parallel with Make's -j<N> option.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 33 lines · Masahiro Yamada
Parallel execution is supported by GNU Make:
$ make -j<N> modules_installIt is questionable to enable multithreading within each zstd process by default.
If you still want to do it, you can use the environment variable:
$ ZSTD_NBTHREADS=<N> make modules_installSigned-off-by Masahiro Yamada <[email protected]>---
scripts/Makefile.modinst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/Makefile.modinst +1
-1@@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@ quiet_cmd_xz = XZ $@ cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $< quiet_cmd_zstd = ZSTD $@ - cmd_zstd = $(ZSTD) -T0 --rm -f -q $< + cmd_zstd = $(ZSTD) --rm -f -q $< $(dst)/%.ko.gz: $(dst)/%.ko FORCE $(call cmd,gzip)Signature · 1 line
2.43.0
Sedat Dilek
Tested-byQuoted 30 lines · Nicolas Schier
Quoted 28 lines · Masahiro Yamada
Parallel execution is supported by GNU Make:
$ make -j<N> modules_installIt is questionable to enable multithreading within each zstd process by default.
If you still want to do it, you can use the environment variable:
$ ZSTD_NBTHREADS=<N> make modules_installSigned-off-by Masahiro Yamada <[email protected]>---
scripts/Makefile.modinst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)scripts/Makefile.modinst +1
-1@@ -146,7 +146,7 @@ quiet_cmd_gzip = GZIP $@ quiet_cmd_xz = XZ $@ cmd_xz = $(XZ) --check=crc32 --lzma2=dict=1MiB -f $< quiet_cmd_zstd = ZSTD $@ - cmd_zstd = $(ZSTD) -T0 --rm -f -q $< + cmd_zstd = $(ZSTD) --rm -f -q $<
Quoted 8 lines
Quoted 5 lines
scripts/Makefile.modinst +0
-1$(dst)/%.ko.gz: $(dst)/%.ko FORCE $(call cmd,gzip) --2.43.0
Reviewed-by Nicolas Schier <[email protected]>
-Sedat-