x86: Add CONFIG_KERNEL_UNCOMPRESSED support
Subsystems (4)
RFC PATCH 1/1 x86: Add CONFIG_KERNEL_UNCOMPRESSED support
Support storing the kernel uncompressed for developers who want to quickly iterate with one-off kernel builds.
Store it in the usual format with a 4-byte length suffix and keep this new codepath as close as possible to the normal path where decompression happens.
The other compression methods offered by the kernel take some time; even LZ4 (which the kernel uses at compression level 9) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine.
An alternate approach to this would be to offer customization of the LZ4 compression level through a kconfig variable; and yet another approach would be to abuse the existing gzip decompression logic by storing the kernel as "non-compressed" DEFLATE blocks, so that the decompression code will essentially end up just doing a bunch of memcpy() calls.
--- arch/x86/Kconfig | 1 + arch/x86/boot/compressed/Makefile | 3 +++ arch/x86/boot/compressed/misc.c | 4 ++++ arch/x86/boot/header.S | 2 ++ drivers/firmware/efi/libstub/zboot.c | 2 ++ lib/decompress_dummy.c | 15 +++++++++++++++ scripts/Makefile.lib | 3 +++ 7 files changed, 30 insertions(+)
arch/x86/Kconfig +1 -0
@@ -250,6 +250,7 @@ config X86 select HAVE_KERNEL_LZO select HAVE_KERNEL_XZ select HAVE_KERNEL_ZSTD + select HAVE_KERNEL_UNCOMPRESSED select HAVE_KPROBES select HAVE_KPROBES_ON_FTRACE select HAVE_FUNCTION_ERROR_INJECTION
arch/x86/boot/compressed/Makefile +3 -0
@@ -137,6 +137,8 @@ $(obj)/vmlinux.bin.lz4: $(vmlinux.bin.all-y) FORCE $(call if_changed,lz4_with_size) $(obj)/vmlinux.bin.zst: $(vmlinux.bin.all-y) FORCE $(call if_changed,zstd22_with_size) +$(obj)/vmlinux.bin.store: $(vmlinux.bin.all-y) FORCE + $(call if_changed,store_with_size) suffix-$(CONFIG_KERNEL_GZIP) := gz suffix-$(CONFIG_KERNEL_BZIP2) := bz2
@@ -145,6 +147,7 @@ suffix-$(CONFIG_KERNEL_XZ) := xz suffix-$(CONFIG_KERNEL_LZO) := lzo suffix-$(CONFIG_KERNEL_LZ4) := lz4 suffix-$(CONFIG_KERNEL_ZSTD) := zst +suffix-$(CONFIG_KERNEL_UNCOMPRESSED) := store quiet_cmd_mkpiggy = MKPIGGY $@ cmd_mkpiggy = $(obj)/mkpiggy $< > $@
arch/x86/boot/compressed/misc.c +4 -0
@@ -88,6 +88,10 @@ static int cols __section(".data");#ifdef CONFIG_KERNEL_ZSTD #include "../../../../lib/decompress_unzstd.c" #endif + +#ifdef CONFIG_KERNEL_UNCOMPRESSED +#include "../../../../lib/decompress_dummy.c" +#endif /* * NOTE: When adding a new decompressor, please update the analysis in * ../header.S.
arch/x86/boot/header.S +2 -0
@@ -483,6 +483,8 @@ pref_address: .quad LOAD_PHYSICAL_ADDR # preferred load addr # larger margin. # # extra_bytes = (uncompressed_size >> 8) + 131072 +# +# Uncompressed data does not grow. #define ZO_z_extra_bytes ((ZO_z_output_len >> 8) + 131072) #if ZO_z_output_len > ZO_z_input_len
drivers/firmware/efi/libstub/zboot.c +2 -0
@@ -27,6 +27,8 @@ static unsigned long free_mem_ptr, free_mem_end_ptr;#include "../../../../lib/decompress_unxz.c" #elif defined(CONFIG_KERNEL_ZSTD )#include "../../../../lib/decompress_unzstd.c" +#elif defined(CONFIG_KERNEL_UNCOMPRESSED ) +#include "../../../../lib/decompress_dummy.c" #endif extern char efi_zboot_header[];
lib/decompress_dummy.c new file +15 -0
@@ -0,0 +1,15 @@ +// SPDX-License-Identifier: GPL-2.0 +STATIC int INIT __decompress(unsigned char *buf, long len, +long (*fill)(void *,unsigned long ), +long (*flush)(void *,unsigned long ), +unsigned char *out_buf,long out_len, +long *pos, +void (*error)(char *x)) +{ +if (out_len < len-4 ) { +error ("output buffer too small" ); +return -1 ; + } +memcpy (out_buf, buf, len-4 ); +return 0 ; +}
scripts/Makefile.lib +3 -0
@@ -525,6 +525,9 @@ quiet_cmd_zstd22 = ZSTD22 $@ quiet_cmd_zstd22_with_size = ZSTD22 $@ cmd_zstd22_with_size = { cat $(real-prereqs) | $(ZSTD) -22 --ultra; $(size_append); } > $@ +quiet_cmd_store_with_size = STORE $@ + cmd_store_with_size = { cat $(real-prereqs); $(size_append); } > $@ + # ASM offsets # ---------------------------------------------------------------------------
---
base-commit: 95ec54a420b8f445e04a7ca0ea8deb72c51fe1d3
change-id: 20250121-kernel-compress-fast-350ce5801c28
Signature · 1 line
Jann Horn <[email protected]>
Ard Biesheuvel
maintainerHi Jann,
Quoted 11 lines · Jann Horn
Support storing the kernel uncompressed for developers who want to quickly iterate with one-off kernel builds.
Store it in the usual format with a 4-byte length suffix and keep this new codepath as close as possible to the normal path where decompression happens.The other compression methods offered by the kernel take some time; even LZ4 (which the kernel uses at compression level 9) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine.
Quoted 6 lines
An alternate approach to this would be to offer customization of the LZ4 compression level through a kconfig variable; and yet another approach would be to abuse the existing gzip decompression logic by storing the kernel as "non-compressed" DEFLATE blocks, so that the decompression code will essentially end up just doing a bunch of
memcpy()calls.
This all seems pretty complicated, and adding yet another (pseudo-)compression method is not great in terms of maintenance burden, especially because there are other consumers of the compressed images (both for bzImage and EFI zboot)
Did you try running gzip with -1 instead of -9? On my build machine, this reduces the compression time of a defconfig bzImage build from 4.3 seconds to 0.9 seconds.
Quoted 21 lines · Ard Biesheuvel
Hi Jann,
Quoted 11 lines · Jann Horn
Support storing the kernel uncompressed for developers who want to quickly iterate with one-off kernel builds.
Store it in the usual format with a 4-byte length suffix and keep this new codepath as close as possible to the normal path where decompression happens.The other compression methods offered by the kernel take some time; even LZ4 (which the kernel uses at compression level 9) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine.
Quoted 6 lines
An alternate approach to this would be to offer customization of the LZ4 compression level through a kconfig variable; and yet another approach would be to abuse the existing gzip decompression logic by storing the kernel as "non-compressed" DEFLATE blocks, so that the decompression code will essentially end up just doing a bunch of
memcpy()calls.
Quoted 8 lines
This all seems pretty complicated, and adding yet another (pseudo-)compression method is not great in terms of maintenance burden, especially because there are other consumers of the compressed images (both for bzImage and EFI zboot)
Did you try running gzip with -1 instead of -9? On my build machine, this reduces the compression time of a defconfig bzImage build from 4.3 seconds to 0.9 seconds.
I tried lz4 with -1; that is very fast (240ms wall clock time on my machine, and just 120ms user time):
$ ls -lh arch/x86/boot/compressed/vmlinux.bin -rwxr-x--- 1 [...] 110M Jan 22 00:01 arch/x86/boot/compressed/vmlinux.bin $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -9 - - | wc -c 2.86user 0.04system 0:02.96elapsed 97%CPU (0avgtext+0avgdata 15756maxresident)k 0inputs+0outputs (0major+220minor)pagefaults 0swaps 46309676 $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -1 - - | wc -c 0.12user 0.06system 0:00.24elapsed 75%CPU (0avgtext+0avgdata 15524maxresident)k 0inputs+0outputs (0major+94minor)pagefaults 0swaps 56029608
But I wasn't sure how to wire that up in a nice way. I guess the nicest option would be to create a separate kconfig variable for the compression level to use for any cmd_lz4/cmd_lz4_with_size invocations in the build process; and then maybe only make this option visible if LZ4 is selected as kernel compression method?
Another option would be to create a new option in the "Kernel compression mode" choice menu with a name like "LZ4 (fast)", turn CONFIG_KERNEL_LZ4 into an internal flag that is selected by both LZ4 variants shown in the choice menu, and duplicate some of the make rules, but that seems overly complicated.
Quoted 51 lines · Jann Horn
Quoted 21 lines · Ard Biesheuvel
Hi Jann,
Quoted 11 lines · Jann Horn
Support storing the kernel uncompressed for developers who want to quickly iterate with one-off kernel builds.
Store it in the usual format with a 4-byte length suffix and keep this new codepath as close as possible to the normal path where decompression happens.The other compression methods offered by the kernel take some time; even LZ4 (which the kernel uses at compression level 9) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine.
Quoted 6 lines
An alternate approach to this would be to offer customization of the LZ4 compression level through a kconfig variable; and yet another approach would be to abuse the existing gzip decompression logic by storing the kernel as "non-compressed" DEFLATE blocks, so that the decompression code will essentially end up just doing a bunch of
memcpy()calls.Quoted 8 lines
This all seems pretty complicated, and adding yet another (pseudo-)compression method is not great in terms of maintenance burden, especially because there are other consumers of the compressed images (both for bzImage and EFI zboot)
Did you try running gzip with -1 instead of -9? On my build machine, this reduces the compression time of a defconfig bzImage build from 4.3 seconds to 0.9 seconds.
I tried lz4 with -1; that is very fast (240ms wall clock time on my machine, and just 120ms user time):
$ ls -lh arch/x86/boot/compressed/vmlinux.bin -rwxr-x--- 1 [...] 110M Jan 22 00:01 arch/x86/boot/compressed/vmlinux.bin $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -9 - - | wc -c 2.86user 0.04system 0:02.96elapsed 97%CPU (0avgtext+0avgdata 15756maxresident)k 0inputs+0outputs (0major+220minor)pagefaults 0swaps 46309676 $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -1 - - | wc -c 0.12user 0.06system 0:00.24elapsed 75%CPU (0avgtext+0avgdata 15524maxresident)k 0inputs+0outputs (0major+94minor)pagefaults 0swaps 56029608But I wasn't sure how to wire that up in a nice way. I guess the nicest option would be to create a separate kconfig variable for the compression level to use for any cmd_lz4/cmd_lz4_with_size invocations in the build process; and then maybe only make this option visible if LZ4 is selected as kernel compression method?
Quoted 6 lines
Another option would be to create a new option in the "Kernel compression mode" choice menu with a name like "LZ4 (fast)", turn
CONFIG_KERNEL_LZ4into an internal flag that is selected by both LZ4 variants shown in the choice menu, and duplicate some of the make rules, but that seems overly complicated.
Hello,
In my opinion 'lz4 -9' doesn't make much sense.
It's terribly slow and the compression ratio is also not exactly good.
Instead, zstd seems to be a much better choice. Not quite as ultra fast as lz4 levels 1 to 3, but much better compression.
As an example, I compressed a kernel source tarball (zstd is multithreaded with 4 threads here, which are not fully used with small-ish files like vmlinux):
- zstd -3: from 1.32 GB to 199 MB in 5.23 seconds
- zstd -6: to 173 MB in 10.77 seconds - zstd -10: to 165 MB in 24.52 seconds - lz4 -1: to 373 MB in 1.60 seconds - lz4 -3: to 278 MB in 6.45 seconds - lz4 -9: to 266 MB in 22.03 seconds
And a vmlinux from my installed kernel 6.13:
- zstd -3: from 51.8 MB to 16.7 MB in 0.60 seconds
- zstd -6: to 15.8 MB in 1.39 seconds - zstd -10: to 15.3 MB in 3.54 seconds - lz4 -1: to 23.7 MB in 0.08 seconds - lz4 -3: to 20.2 MB in 0.51 seconds - lz4 -9: to 19.7 MB in 1.23 seconds
For my kernel, I use a Kconfig option to choose a zstd compression level. I could submit it if there is interest.
Tor
Quoted 59 lines · Tor Vic
Quoted 51 lines · Jann Horn
Quoted 21 lines · Ard Biesheuvel
Hi Jann,
Quoted 11 lines · Jann Horn
Support storing the kernel uncompressed for developers who want to quickly iterate with one-off kernel builds.
Store it in the usual format with a 4-byte length suffix and keep this new codepath as close as possible to the normal path where decompression happens.The other compression methods offered by the kernel take some time; even LZ4 (which the kernel uses at compression level 9) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine.
Quoted 6 lines
An alternate approach to this would be to offer customization of the LZ4 compression level through a kconfig variable; and yet another approach would be to abuse the existing gzip decompression logic by storing the kernel as "non-compressed" DEFLATE blocks, so that the decompression code will essentially end up just doing a bunch of
memcpy()calls.Quoted 8 lines
This all seems pretty complicated, and adding yet another (pseudo-)compression method is not great in terms of maintenance burden, especially because there are other consumers of the compressed images (both for bzImage and EFI zboot)
Did you try running gzip with -1 instead of -9? On my build machine, this reduces the compression time of a defconfig bzImage build from 4.3 seconds to 0.9 seconds.
I tried lz4 with -1; that is very fast (240ms wall clock time on my machine, and just 120ms user time):
$ ls -lh arch/x86/boot/compressed/vmlinux.bin -rwxr-x--- 1 [...] 110M Jan 22 00:01 arch/x86/boot/compressed/vmlinux.bin $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -9 - - | wc -c 2.86user 0.04system 0:02.96elapsed 97%CPU (0avgtext+0avgdata 15756maxresident)k 0inputs+0outputs (0major+220minor)pagefaults 0swaps 46309676 $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -1 - - | wc -c 0.12user 0.06system 0:00.24elapsed 75%CPU (0avgtext+0avgdata 15524maxresident)k 0inputs+0outputs (0major+94minor)pagefaults 0swaps 56029608But I wasn't sure how to wire that up in a nice way. I guess the nicest option would be to create a separate kconfig variable for the compression level to use for any cmd_lz4/cmd_lz4_with_size invocations in the build process; and then maybe only make this option visible if LZ4 is selected as kernel compression method?
Quoted 6 lines
Another option would be to create a new option in the "Kernel compression mode" choice menu with a name like "LZ4 (fast)", turn
CONFIG_KERNEL_LZ4into an internal flag that is selected by both LZ4 variants shown in the choice menu, and duplicate some of the make rules, but that seems overly complicated.
Quoted 7 lines
Hello,
In my opinion 'lz4 -9' doesn't make much sense.
It's terribly slow and the compression ratio is also not exactly good.Instead, zstd seems to be a much better choice. Not quite as ultra fast as lz4 levels 1 to 3, but much better compression.
I think you're describing a slightly different usecase.
My goal here is something I can use for when I build a kernel, boot it in QEMU, test something, and then immediately throw the kernel away - I don't care that much how much disk space the kernel image uses, and the goal I'm optimizing for is pretty much just the time needed for one build followed by one boot.
Quoted 9 lines · Jann Horn
Quoted 8 lines
Hello,
In my opinion 'lz4 -9' doesn't make much sense.
It's terribly slow and the compression ratio is also not exactly good.Instead, zstd seems to be a much better choice. Not quite as ultra fast as lz4 levels 1 to 3, but much better compression.
Quoted 7 lines
I think you're describing a slightly different usecase.
My goal here is something I can use for when I build a kernel, boot it in QEMU, test something, and then immediately throw the kernel away - I don't care that much how much disk space the kernel image uses, and the goal I'm optimizing for is pretty much just the time needed for one build followed by one boot.
Ah, yes, fair enough. In that case, 'lz4 -1' makes the most sense I guess. Sorry for the noise.
Tor
Ard Biesheuvel
maintainerQuoted 35 lines · Jann Horn
Quoted 21 lines · Ard Biesheuvel
Hi Jann,
Quoted 11 lines · Jann Horn
Support storing the kernel uncompressed for developers who want to quickly iterate with one-off kernel builds.
Store it in the usual format with a 4-byte length suffix and keep this new codepath as close as possible to the normal path where decompression happens.The other compression methods offered by the kernel take some time; even LZ4 (which the kernel uses at compression level 9) takes ~2.8 seconds to compress a 110M large vmlinux.bin on my machine.
Quoted 6 lines
An alternate approach to this would be to offer customization of the LZ4 compression level through a kconfig variable; and yet another approach would be to abuse the existing gzip decompression logic by storing the kernel as "non-compressed" DEFLATE blocks, so that the decompression code will essentially end up just doing a bunch of
memcpy()calls.Quoted 8 lines
This all seems pretty complicated, and adding yet another (pseudo-)compression method is not great in terms of maintenance burden, especially because there are other consumers of the compressed images (both for bzImage and EFI zboot)
Did you try running gzip with -1 instead of -9? On my build machine, this reduces the compression time of a defconfig bzImage build from 4.3 seconds to 0.9 seconds.
I tried lz4 with -1; that is very fast (240ms wall clock time on my machine, and just 120ms user time):
Quoted 11 lines
$ ls -lh arch/x86/boot/compressed/vmlinux.bin -rwxr-x--- 1 [...] 110M Jan 22 00:01 arch/x86/boot/compressed/vmlinux.bin $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -9 - - | wc -c 2.86user 0.04system 0:02.96elapsed 97%CPU (0avgtext+0avgdata 15756maxresident)k 0inputs+0outputs (0major+220minor)pagefaults 0swaps 46309676 $ cat arch/x86/boot/compressed/vmlinux.bin | time lz4 -l -1 - - | wc -c 0.12user 0.06system 0:00.24elapsed 75%CPU (0avgtext+0avgdata 15524maxresident)k 0inputs+0outputs (0major+94minor)pagefaults 0swaps 56029608
Excellent.
Quoted 6 lines
But I wasn't sure how to wire that up in a nice way. I guess the nicest option would be to create a separate kconfig variable for the compression level to use for any cmd_lz4/cmd_lz4_with_size invocations in the build process; and then maybe only make this option visible if LZ4 is selected as kernel compression method?
Quoted 6 lines
Another option would be to create a new option in the "Kernel compression mode" choice menu with a name like "LZ4 (fast)", turn
CONFIG_KERNEL_LZ4into an internal flag that is selected by both LZ4 variants shown in the choice menu, and duplicate some of the make rules, but that seems overly complicated.
I didn't realise that KERNEL_UNCOMPRESSED already exists and you are just wiring it up for x86. But I still think that we should avoid that, not only because it is yet another bzImage format but also because I still see a 3x size reduction even with the fastest setting.
I think adding one Kconfig symbol that depends on KERNEL_LZ4 and switches from -9 to -1 for LZ4 only is reasonable.
Quoted 13 lines · Ard Biesheuvel
Quoted 6 lines
But I wasn't sure how to wire that up in a nice way. I guess the nicest option would be to create a separate kconfig variable for the compression level to use for any cmd_lz4/cmd_lz4_with_size invocations in the build process; and then maybe only make this option visible if LZ4 is selected as kernel compression method?
Quoted 6 lines
Another option would be to create a new option in the "Kernel compression mode" choice menu with a name like "LZ4 (fast)", turn
CONFIG_KERNEL_LZ4into an internal flag that is selected by both LZ4 variants shown in the choice menu, and duplicate some of the make rules, but that seems overly complicated.
Quoted 8 lines
I didn't realise that
KERNEL_UNCOMPRESSEDalready exists and you are just wiring it up for x86. But I still think that we should avoid that, not only because it is yet another bzImage format but also because I still see a 3x size reduction even with the fastest setting.I think adding one
Kconfigsymbol that depends onKERNEL_LZ4and switches from -9 to -1 for LZ4 only is reasonable.
Maybe a CONFIG_COMPRESS_FAST option that maps to and enables the current fastest compressor? Then we can also add LZ4_FAST and map it to it. (And if a future fastest compressor is added it can change this mapping.) Or something like that?
And if there's still a measurable difference in total build time (say larger than 1%) from doing it all uncompressed, then I think we should go with Jann's original patch that hooks up zero-compression on x86. It doesn't look overly complicated.
Thanks,
Ingo