Skip to contentSkip to conversation

kbuild: Add CONFIG_LZ4_LEVEL for choosing compression level

PATCH In review Jann Horn 2 participants 2 messages Jan 22, 2025 Kbuild

lore
Share & tools
lore

Jann Horn

author
#0 lore raw

PATCH v2 1/1 kbuild: Add CONFIG_LZ4_LEVEL for choosing compression level

Allow choosing the LZ4 compression level using the new config flag CONFIG_LZ4_LEVEL.

This will allow developers who want to quickly iterate with one-off kernel builds to speed up kernel compression at the cost of creating a larger kernel image.

LZ4 at level 9 (which the kernel uses by default) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine; LZ4 at level 1 just needs ~220ms wall clock time.

---
My patch provides kconfig for choosing compression level of LZ4; Tor Vic mentioned using kconfig for choosing the compression level of ZSTD.

Does my current approach of having a kconfig option specifically for LZ4 compression level look good (and we can add the same for ZSTD later if necessary); or should there be a single kconfig option that sets a compression level that is used across all compression algorithms? I guess doing it separately for different algorithms makes more sense because different compression levels might mean different things for different algorithms?

Do I need to somehow make it explicit that cmd_lz4 and cmd_lz4_with_size
rely on CONFIG_LZ4_LEVEL, or does the build system figure that out on
its own somehow? Experimentally it seems like the build system triggers
a rebuild when the config option changes, but I'm not sure how that
actually works.
---
Changes in v2:
- removed x86/EFI subsystems from recipients since I'm no longer
touching those
- rewrote patch to make LZ4 level configurable instead of offering
uncompressed builds on x86 (Ard)
- Link to v1: https://lore.kernel.org/r/[email protected]

---
 init/Kconfig         | 16 ++++++++++++++++
 scripts/Makefile.lib |  4 ++--
 2 files changed, 18 insertions(+), 2 deletions(-)
init/Kconfig +16 -0
index a20e6efd3f0fbdd7f0df2448854cc30734a0ee4f..3fd93bbe98390997e7e2d41426241b90d3b33053 100644
@@ -358,6 +358,9 @@ config KERNEL_LZ4
 	  is about 8% bigger than LZO. But the decompression speed is
 	  faster than LZO.
 
+	  If you are working on the kernel and recompiling frequently, you can
+	  adjust the compression level with CONFIG_LZ4_LEVEL.
+
 config KERNEL_ZSTD
 	bool "ZSTD"
 	depends on HAVE_KERNEL_ZSTD
@@ -380,6 +383,19 @@ config KERNEL_UNCOMPRESSED
 
 endchoice
 
+config LZ4_LEVEL
+	int "LZ4 compression level"
+	range 1 12
+	depends on KERNEL_LZ4
+	default 9
+	help
+	  This option controls the compression level used for LZ4 invocations,
+	  in particular for compressing the kernel image.
+
+	  If you are working on the kernel and recompiling frequently, you can
+	  set this to a low value like 1 to speed up kernel builds at the cost
+	  of using more disk space.
+
 config DEFAULT_INIT
 	string "Default init path"
 	default ""
scripts/Makefile.lib +2 -2
index 7395200538da89a2f6e6d21f8959f3f60d291d79..c3eedbc4acf133d5f9e7f13d637af09ad1968930 100644
@@ -425,10 +425,10 @@ quiet_cmd_lzo_with_size = LZO     $@
       cmd_lzo_with_size = { cat $(real-prereqs) | $(KLZOP) -9; $(size_append); } > $@
 
 quiet_cmd_lz4 = LZ4     $@
-      cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -9 - - > $@
+      cmd_lz4 = cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - - > $@
 
 quiet_cmd_lz4_with_size = LZ4     $@
-      cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -9 - -; \
+      cmd_lz4_with_size = { cat $(real-prereqs) | $(LZ4) -l -$(CONFIG_LZ4_LEVEL) - -; \
                   $(size_append); } > $@
 
 # U-Boot mkimage

---
base-commit: 95ec54a420b8f445e04a7ca0ea8deb72c51fe1d3
change-id: 20250121-kernel-compress-fast-350ce5801c28

Signature · 1 line
Jann Horn <[email protected]>
#1 lore raw
Quoted 16 lines · Jann Horn

Allow choosing the LZ4 compression level using the new config flag CONFIG_LZ4_LEVEL.

This will allow developers who want to quickly iterate with one-off kernel builds to speed up kernel compression at the cost of creating a larger kernel image.

LZ4 at level 9 (which the kernel uses by default) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine; LZ4 at level 1 just needs ~220ms wall clock time.

---
My patch provides kconfig for choosing compression level of LZ4; Tor Vic mentioned using kconfig for choosing the compression level of ZSTD.

Quoted 7 lines

Does my current approach of having a kconfig option specifically for LZ4 compression level look good (and we can add the same for ZSTD later if necessary); or should there be a single kconfig option that sets a compression level that is used across all compression algorithms? I guess doing it separately for different algorithms makes more sense because different compression levels might mean different things for different algorithms?

Hi Jann,

Yes, different algos have different ways of specifying compression levels. You cannot have one single option for all of the available compressors.

Quoted 6 lines

Do I need to somehow make it explicit that cmd_lz4 and cmd_lz4_with_size rely on CONFIG_LZ4_LEVEL, or does the build system figure that out on its own somehow? Experimentally it seems like the build system triggers a rebuild when the config option changes, but I'm not sure how that actually works.

Here is my take for zstd which I've been using for a while now. (zstd levels above 19 need the '--ultra' switch, hence the honkytonk below).

--
From 2e9a0e058e10a428d2ee079b969627e475cc0f2e Mon Sep 17 00:00:00 2001
From: Tor Vic <[email protected]>
Date: Tue, 6 Aug 2024 11:24:32 +0200
Subject: [PATCH] Kbuild, zstd: Allow users to choose a compression level for
the kernel image

---
 init/Kconfig         | 13 +++++++++++++
 scripts/Makefile.lib |  9 ++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)
init/Kconfig +13 -0
index 105cdd05db759..9c39bc17964de 100644
@@ -380,6 +380,19 @@ config KERNEL_UNCOMPRESSED
 
 endchoice
 
+menu "ZSTD compression options"
+	depends on KERNEL_ZSTD
+
+config ZSTD_COMPRESSION_LVL
+	int "Compression level (1-22)"
+	range 1 22
+	default "22"
+	help
+	  Choose a compression level for zstd kernel compression.
+	  Default is 22, which is the maximum.
+
+endmenu
+
 config DEFAULT_INIT
 	string "Default init path"
 	default ""
scripts/Makefile.lib +0 -0
index 7395200538da8..58884363ebc62 100644
@@ -516,6 +516,13 @@ quiet_cmd_xzmisc = XZMISC  $@
 # decompression is used, like initramfs decompression, zstd22 should 

likely not

 # be used because it would require zstd to allocate a 128 MB buffer.

+ifdef CONFIG_ZSTD_COMPRESSION_LVL
+zstd_comp_lvl := $(CONFIG_ZSTD_COMPRESSION_LVL)
+ifeq ($(call test-gt, $(zstd_comp_lvl), 19),y)
+zstd_comp_lvl += --ultra
+endif
+endif
+
quiet_cmd_zstd = ZSTD $@

       cmd_zstd = cat $(real-prereqs) | $(ZSTD) -19 > $@

@@ -523,7 +530,7 @@ quiet_cmd_zstd22 = ZSTD22 $@

       cmd_zstd22 = cat $(real-prereqs) | $(ZSTD) -22 --ultra > $@

quiet_cmd_zstd22_with_size = ZSTD22 $@

-      cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -22 --ultra; $(size_append); } > $@
+      cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -T0 -$(zstd_comp_lvl); $(size_append); } > $@

# ASM offsets
#
---------------------------------------------------------------------------