m68k: Add target for lz4 compressed vmlinux
PATCH 1/1 m68k: Add target for lz4 compressed vmlinux
Traditionally gzip (/bzip?) has been the compressed image format but its a pain to decompress because its very hard to load and decompress it in chunks which you need to do if you don't have enough memory to load the whole kernel image and decompress it somewhere else.
With lz4 you can set a block size, read it from the header, and then you only need memory for a single block and the decompressed kernel.
I use lz4 compressed images on 68000 with 8MB of ram and MVME147 with 16MB. I want to use lz4 in my fork of EMILE to boot m68k macs because streaming decompress of gzip is painful.
--- arch/m68k/Makefile | 11 +++++++++++ 1 file changed, 11 insertions(+)
arch/m68k/Makefile +11 -0
@@ -124,6 +124,17 @@ else $(KBZIP2) -1c vmlinux >vmlinux.bz2 endif +vmlinux.lz4: vmlinux + +ifndef CONFIG_KGDB + cp vmlinux vmlinux.tmp + $(STRIP) vmlinux.tmp + $(LZ4) -z9f vmlinux.tmp vmlinux.lz4 + rm vmlinux.tmp +else + $(LZ4) -z9f vmlinux vmlinux.lz4 +endif + CLEAN_FILES += vmlinux.gz vmlinux.bz2 archheaders:
Signature · 1 line
2.50.0
Quoted 27 lines · Daniel Palmer
Traditionally gzip (/bzip?) has been the compressed image format but its a pain to decompress because its very hard to load and decompress it in chunks which you need to do if you don't have enough memory to load the whole kernel image and decompress it somewhere else.
With lz4 you can set a block size, read it from the header, and then you only need memory for a single block and the decompressed kernel.
I use lz4 compressed images on 68000 with 8MB of ram and MVME147 with 16MB. I want to use lz4 in my fork of EMILE to boot m68k macs because streaming decompress of gzip is painful.
Signed-off-by Daniel Palmer <[email protected]>--- arch/m68k/Makefile | 11 +++++++++++ 1 file changed, 11 insertions(+)arch/m68k/Makefile +0
-0@@ -124,6 +124,17 @@ else $(KBZIP2) -1c vmlinux >vmlinux.bz2 endif
arch/m68k/Makefile else · line 136
Quoted 10 lines
arch/m68k/Makefile +10
-0+vmlinux.lz4: vmlinux + +ifndef CONFIG_KGDB + cp vmlinux vmlinux.tmp + $(STRIP) vmlinux.tmp + $(LZ4) -z9f vmlinux.tmp vmlinux.lz4 + rm vmlinux.tmp +else + $(LZ4) -z9f vmlinux vmlinux.lz4 +endif
Splitting the vmlinux.tmp creation into a dedicated target would make all the compressor targets simpler. It will need a bit more disk space, but there are a bunch of vmlinux copies already in any case.
arch/m68k/Makefile else · line 138
Quoted 2 lines
arch/m68k/Makefile +1
-0+ CLEAN_FILES += vmlinux.gz vmlinux.bz2
CLEAN_FILES also needs to be updated.
Quoted 4 lines
arch/m68k/Makefile +0
-0archheaders:Signature · 1 line
2.50.0
Hi Thomas,
Quoted 4 lines · Thomas Weißschuh
Splitting the vmlinux.tmp creation into a dedicated target would make all the compressor targets simpler. It will need a bit more disk space, but there are a bunch of vmlinux copies already in any case.
That's true. I think a target for vmlinux.stripped and then use that in the compressed image targets.
How about this?:
vmlinux.stripped: vmlinux
cp $< $@
$(STRIP) $@vmlinux.gz: vmlinux.stripped
ifndef CONFIG_KGDB
$(KGZIP) -9c vmlinux.stripped >vmlinux.gz
else
$(KGZIP) -9c vmlinux >vmlinux.gzendif
<snip>
Quoted 4 lines
Quoted 2 lines
+
CLEAN_FILES+= vmlinux.gz vmlinux.bz2
CLEAN_FILESalso needs to be updated.
Noted. Will fix for v2.
Thanks,
Daniel
Hi Daniel,
Quoted 17 lines · Daniel Palmer
Quoted 4 lines · Thomas Weißschuh
Splitting the vmlinux.tmp creation into a dedicated target would make all the compressor targets simpler. It will need a bit more disk space, but there are a bunch of vmlinux copies already in any case.
That's true. I think a target for vmlinux.stripped and then use that in the compressed image targets.
How about this?:
vmlinux.stripped: vmlinux
cp $< $@ $(STRIP) $@vmlinux.gz: vmlinux.stripped
Quoted 7 lines
ifndef
CONFIG_KGDB$(KGZIP) -9c vmlinux.stripped >vmlinux.gz else $(KGZIP) -9c vmlinux >vmlinux.gzendif
<snip>
I would continue with vmlinux.tmp. It might not actually be stripped.
quiet_cmd_precompress = PRECOMPRESS $@
ifndef CONFIG_KGDB
cmd_precompress = cp $< $@
else
cmd_precompress = $(STRIP) $< -o $@
endif
vmlinux.tmp: vmlinux FORCE
$(call if_changed,precompress)
targets += vmlinux.tmp
This will also correctly handle CONFIG_KGDB changing.
(Maybe the naming can be improved)
And for the compressor invocations we already have predefined commands:
vmlinux.gz: vmlinux.tmp FORCE
$(call if_changed,gzip)
targets += vmlinux.gz
Quoted 6 lines
Quoted 4 lines
Quoted 2 lines
+
CLEAN_FILES+= vmlinux.gz vmlinux.bz2
CLEAN_FILESalso needs to be updated.Noted. Will fix for v2.
If you use $(call if_changed) as explained above, the "targets += ..." will make the CLEAN_FILES unnecessary.
Hi Thomas,
Quoted 14 lines · Thomas Weißschuh
Hi Daniel,
Quoted 1 line · Daniel Palmer
On Sat, 9 Aug 2025 at 16:50, Thomas Weißschuh <[email protected]> wrote:
I would continue with vmlinux.tmp. It might not actually be stripped.
quiet_cmd_precompress= PRECOMPRESS $@
ifndefCONFIG_KGDB
cmd_precompress= cp $< $@
elsecmd_precompress = $(STRIP) $< -o $@endif
Quoted 8 lines
vmlinux.tmp: vmlinux FORCE
$(call if_changed,precompress)targets += vmlinux.tmp
This will also correctly handle
CONFIG_KGDBchanging.(Maybe the naming can be improved)
Nice, that's a lot better and I've reworked my patch to do that.
Quoted 4 lines
And for the compressor invocations we already have predefined commands:
vmlinux.gz: vmlinux.tmp FORCE
$(call if_changed,gzip)
I also did this for all of the targets and $(call if_changed,lz4) results in an lz4 file with the old header format[0] and 8MB block size which is not what I need.
So v2 will have a patch to add a helper to generate that style of lz4 in scripts/Makefile.lib
Thanks!,
Daniel
0 - https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#legacy-frame
Quoted 15 lines · Daniel Palmer
Quoted 14 lines · Thomas Weißschuh
Quoted 1 line · Daniel Palmer
On Sat, 9 Aug 2025 at 16:50, Thomas Weißschuh <[email protected]> wrote:
I would continue with vmlinux.tmp. It might not actually be stripped.
quiet_cmd_precompress= PRECOMPRESS $@
ifndefCONFIG_KGDB
cmd_precompress= cp $< $@
elsecmd_precompress = $(STRIP) $< -o $@endif
vmlinux.tmp: vmlinux FORCE
$(call if_changed,precompress)
Quoted 7 lines
Quoted 5 lines
targets += vmlinux.tmp
This will also correctly handle
CONFIG_KGDBchanging.(Maybe the naming can be improved)
Nice, that's a lot better and I've reworked my patch to do that.
If you also modify the existing code, you should split up the changes over multiple patches. First split out the rule for vmlinux.tmp, then switch over the compressors. Each patch should do a single step and explain why it is done.
And I am still not happy about the naming in my example :-/
Quoted 10 lines
Quoted 4 lines
And for the compressor invocations we already have predefined commands:
vmlinux.gz: vmlinux.tmp FORCE
$(call if_changed,gzip)I also did this for all of the targets and $(call if_changed,lz4) results in an lz4 file with the old header format[0] and 8MB block size which is not what I need.
So v2 will have a patch to add a helper to generate that style of lz4 inscripts/Makefile.lib
Modifying generic files in scripts/ will be more effort to argue for. Also that change should be its own patch again.
I am wondering a bit why the legacy header format does not work. The manpage of lz4 specifically mentions that it is commonly used for Linux kernel images. Which is exactly what we are doing here. Does your usecase work not work with the legacy header? If not, please mention this in the patches.
Thomas