kbuild: Add CONFIG_LZ4_LEVEL for choosing compression level
PATCH v2 1/1 kbuild: Add CONFIG_LZ4_LEVEL for choosing compression level
Allow choosing the LZ4 compression level using the new config flag CONFIG_LZ4_LEVEL.
This will allow developers who want to quickly iterate with one-off kernel builds to speed up kernel compression at the cost of creating a larger kernel image.
LZ4 at level 9 (which the kernel uses by default) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine; LZ4 at level 1 just needs ~220ms wall clock time.
---
My patch provides kconfig for choosing compression level of LZ4; Tor Vic mentioned using kconfig for choosing the compression level of ZSTD.
Does my current approach of having a kconfig option specifically for LZ4 compression level look good (and we can add the same for ZSTD later if necessary); or should there be a single kconfig option that sets a compression level that is used across all compression algorithms? I guess doing it separately for different algorithms makes more sense because different compression levels might mean different things for different algorithms?
Do I need to somehow make it explicit that cmd_lz4 and cmd_lz4_with_size
rely on CONFIG_LZ4_LEVEL, or does the build system figure that out on
its own somehow? Experimentally it seems like the build system triggers
a rebuild when the config option changes, but I'm not sure how that
actually works.
---
Changes in v2:
- removed x86/EFI subsystems from recipients since I'm no longer
touching those
- rewrote patch to make LZ4 level configurable instead of offering
uncompressed builds on x86 (Ard)
- Link to v1: https://lore.kernel.org/r/[email protected]
--- init/Kconfig | 16 ++++++++++++++++ scripts/Makefile.lib | 4 ++-- 2 files changed, 18 insertions(+), 2 deletions(-)
init/Kconfig +16 -0
@@ -358,6 +358,9 @@ config KERNEL_LZ4 is about 8% bigger than LZO. But the decompression speed is faster than LZO. + If you are working on the kernel and recompiling frequently, you can + adjust the compression level with CONFIG_LZ4_LEVEL. + config KERNEL_ZSTD bool "ZSTD" depends on HAVE_KERNEL_ZSTD
@@ -380,6 +383,19 @@ config KERNEL_UNCOMPRESSED endchoice +config LZ4_LEVEL + int "LZ4 compression level" + range 1 12 + depends on KERNEL_LZ4 + default 9 + help + This option controls the compression level used for LZ4 invocations, + in particular for compressing the kernel image. + + If you are working on the kernel and recompiling frequently, you can + set this to a low value like 1 to speed up kernel builds at the cost + of using more disk space. + config DEFAULT_INIT string "Default init path" default ""
scripts/Makefile.lib +2 -2
@@ -425,10 +425,10 @@ quiet_cmd_lzo_with_size = LZO $@ cmd_lzo_with_size = { cat $(real-prereqs) | $(KLZOP) -9; $(size_append); } > $@ quiet_cmd_lz4 = LZ4 $@ - cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -9 - - > $@ + cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - - > $@ quiet_cmd_lz4_with_size = LZ4 $@ - cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -9 - -; \ + cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - -; \ $(size_append); } > $@ # U-Boot mkimage
---
base-commit: 95ec54a420b8f445e04a7ca0ea8deb72c51fe1d3
change-id: 20250121-kernel-compress-fast-350ce5801c28
Signature · 1 line
Jann Horn <[email protected]>
Quoted 16 lines · Jann Horn
Allow choosing the LZ4 compression level using the new config flag
CONFIG_LZ4_LEVEL.This will allow developers who want to quickly iterate with one-off kernel builds to speed up kernel compression at the cost of creating a larger kernel image.
LZ4 at level 9 (which the kernel uses by default) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine; LZ4 at level 1 just needs ~220ms wall clock time.
Signed-off-by Jann Horn <[email protected]>---
My patch provides kconfig for choosing compression level of LZ4; Tor Vic mentioned using kconfig for choosing the compression level of ZSTD.
Quoted 7 lines
Does my current approach of having a kconfig option specifically for LZ4 compression level look good (and we can add the same for ZSTD later if necessary); or should there be a single kconfig option that sets a compression level that is used across all compression algorithms? I guess doing it separately for different algorithms makes more sense because different compression levels might mean different things for different algorithms?
Hi Jann,
Yes, different algos have different ways of specifying compression levels. You cannot have one single option for all of the available compressors.
Quoted 6 lines
Do I need to somehow make it explicit that
cmd_lz4andcmd_lz4_with_sizerely onCONFIG_LZ4_LEVEL, or does the build system figure that out on its own somehow? Experimentally it seems like the build system triggers a rebuild when the config option changes, but I'm not sure how that actually works.
Here is my take for zstd which I've been using for a while now. (zstd levels above 19 need the '--ultra' switch, hence the honkytonk below).
--
From 2e9a0e058e10a428d2ee079b969627e475cc0f2e Mon Sep 17 00:00:00 2001
From: Tor Vic <[email protected]>
Date: Tue, 6 Aug 2024 11:24:32 +0200
Subject: [PATCH] Kbuild, zstd: Allow users to choose a compression level for
the kernel image
--- init/Kconfig | 13 +++++++++++++ scripts/Makefile.lib | 9 ++++++++- 2 files changed, 21 insertions(+), 1 deletion(-)
init/Kconfig +13 -0
@@ -380,6 +380,19 @@ config KERNEL_UNCOMPRESSED endchoice +menu "ZSTD compression options" + depends on KERNEL_ZSTD + +config ZSTD_COMPRESSION_LVL + int "Compression level (1-22)" + range 1 22 + default "22" + help + Choose a compression level for zstd kernel compression. + Default is 22, which is the maximum. + +endmenu + config DEFAULT_INIT string "Default init path" default ""
scripts/Makefile.lib +0 -0
@@ -516,6 +516,13 @@ quiet_cmd_xzmisc = XZMISC $@ # decompression is used, like initramfs decompression, zstd22 should
likely not
# be used because it would require zstd to allocate a 128 MB buffer.
+ifdef CONFIG_ZSTD_COMPRESSION_LVL
+zstd_comp_lvl := $(CONFIG_ZSTD_COMPRESSION_LVL)
+ifeq ($(call test-gt, $(zstd_comp_lvl), 19),y)
+zstd_comp_lvl += --ultra
+endif
+endif
+
quiet_cmd_zstd = ZSTD $@
cmd_zstd = cat $(real-prereqs) | $(ZSTD) -19 > $@
@@ -523,7 +530,7 @@ quiet_cmd_zstd22 = ZSTD22 $@
cmd_zstd22 = cat $(real-prereqs) | $(ZSTD) -22 --ultra > $@
quiet_cmd_zstd22_with_size = ZSTD22 $@
- cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -22 --ultra; $(size_append); } > $@
+ cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -T0 -$(zstd_comp_lvl); $(size_append); } > $@ # ASM offsets
#
---------------------------------------------------------------------------