Skip to contentSkip to conversation

m68k: Add target for lz4 compressed vmlinux

PATCH In review Daniel Palmer 2 participants 6 messages Aug 9, 2025 – Aug 10, 2025 m68k architecture

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 m68k: Add target for lz4 compressed vmlinux

Traditionally gzip (/bzip?) has been the compressed image format but its a pain to decompress because its very hard to load and decompress it in chunks which you need to do if you don't have enough memory to load the whole kernel image and decompress it somewhere else.

With lz4 you can set a block size, read it from the header, and then you only need memory for a single block and the decompressed kernel.

I use lz4 compressed images on 68000 with 8MB of ram and MVME147 with 16MB. I want to use lz4 in my fork of EMILE to boot m68k macs because streaming decompress of gzip is painful.

---
 arch/m68k/Makefile | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/m68k/Makefile +11 -0
index 0abcf994ce55..a56e853037c0 100644
@@ -124,6 +124,17 @@ else
 	$(KBZIP2) -1c vmlinux >vmlinux.bz2
 endif
 
+vmlinux.lz4: vmlinux
+
+ifndef CONFIG_KGDB
+	cp vmlinux vmlinux.tmp
+	$(STRIP) vmlinux.tmp
+	$(LZ4) -z9f vmlinux.tmp vmlinux.lz4
+	rm vmlinux.tmp
+else
+	$(LZ4) -z9f vmlinux vmlinux.lz4
+endif
+
 CLEAN_FILES += vmlinux.gz vmlinux.bz2
 
 archheaders:
Signature · 1 line
2.50.0
#1 lore raw
Quoted 27 lines · Daniel Palmer

Traditionally gzip (/bzip?) has been the compressed image format but its a pain to decompress because its very hard to load and decompress it in chunks which you need to do if you don't have enough memory to load the whole kernel image and decompress it somewhere else.

With lz4 you can set a block size, read it from the header, and then you only need memory for a single block and the decompressed kernel.

I use lz4 compressed images on 68000 with 8MB of ram and MVME147 with 16MB. I want to use lz4 in my fork of EMILE to boot m68k macs because streaming decompress of gzip is painful.

---
 arch/m68k/Makefile | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/m68k/Makefile +0 -0
index 0abcf994ce55..a56e853037c0 100644
@@ -124,6 +124,17 @@ else
 	$(KBZIP2) -1c vmlinux >vmlinux.bz2
 endif
 

arch/m68k/Makefile else · line 136

Quoted 10 lines
arch/m68k/Makefile +10 -0
+vmlinux.lz4: vmlinux
+
+ifndef CONFIG_KGDB
+	cp vmlinux vmlinux.tmp
+	$(STRIP) vmlinux.tmp
+	$(LZ4) -z9f vmlinux.tmp vmlinux.lz4
+	rm vmlinux.tmp
+else
+	$(LZ4) -z9f vmlinux vmlinux.lz4
+endif

Splitting the vmlinux.tmp creation into a dedicated target would make all the compressor targets simpler. It will need a bit more disk space, but there are a bunch of vmlinux copies already in any case.

arch/m68k/Makefile else · line 138

Quoted 2 lines
arch/m68k/Makefile +1 -0
+
 CLEAN_FILES += vmlinux.gz vmlinux.bz2

CLEAN_FILES also needs to be updated.

Quoted 4 lines
arch/m68k/Makefile +0 -0
 archheaders:
Signature · 1 line
2.50.0
#2 lore raw

Hi Thomas,

Quoted 4 lines · Thomas Weißschuh

Splitting the vmlinux.tmp creation into a dedicated target would make all the compressor targets simpler. It will need a bit more disk space, but there are a bunch of vmlinux copies already in any case.

That's true. I think a target for vmlinux.stripped and then use that in the compressed image targets.

How about this?:

vmlinux.stripped: vmlinux

       cp $< $@
       $(STRIP) $@

vmlinux.gz: vmlinux.stripped

ifndef CONFIG_KGDB

       $(KGZIP) -9c vmlinux.stripped >vmlinux.gz
else
       $(KGZIP) -9c vmlinux >vmlinux.gz

endif

<snip>

Quoted 4 lines
Quoted 2 lines

+
CLEAN_FILES += vmlinux.gz vmlinux.bz2

CLEAN_FILES also needs to be updated.

Noted. Will fix for v2.

Thanks,

Daniel

#3 lore raw

Hi Daniel,

Quoted 17 lines · Daniel Palmer
Quoted 4 lines · Thomas Weißschuh

Splitting the vmlinux.tmp creation into a dedicated target would make all the compressor targets simpler. It will need a bit more disk space, but there are a bunch of vmlinux copies already in any case.

That's true. I think a target for vmlinux.stripped and then use that in the compressed image targets.

How about this?:

vmlinux.stripped: vmlinux

       cp $< $@
       $(STRIP) $@

vmlinux.gz: vmlinux.stripped

Quoted 7 lines

ifndef CONFIG_KGDB

       $(KGZIP) -9c vmlinux.stripped >vmlinux.gz
else
       $(KGZIP) -9c vmlinux >vmlinux.gz

endif

<snip>

I would continue with vmlinux.tmp. It might not actually be stripped.

quiet_cmd_precompress = PRECOMPRESS $@
ifndef CONFIG_KGDB
cmd_precompress = cp $< $@
else

      cmd_precompress = $(STRIP) $< -o $@

endif

vmlinux.tmp: vmlinux FORCE

	$(call if_changed,precompress)

targets += vmlinux.tmp

This will also correctly handle CONFIG_KGDB changing.

(Maybe the naming can be improved)

And for the compressor invocations we already have predefined commands:

vmlinux.gz: vmlinux.tmp FORCE

	$(call if_changed,gzip)

targets += vmlinux.gz

Quoted 6 lines
Quoted 4 lines
Quoted 2 lines

+
CLEAN_FILES += vmlinux.gz vmlinux.bz2

CLEAN_FILES also needs to be updated.

Noted. Will fix for v2.

If you use $(call if_changed) as explained above, the "targets += ..." will make the CLEAN_FILES unnecessary.

#4 lore raw

Hi Thomas,

Quoted 14 lines · Thomas Weißschuh

Hi Daniel,

Quoted 1 line · Daniel Palmer

On Sat, 9 Aug 2025 at 16:50, Thomas Weißschuh <[email protected]> wrote:

I would continue with vmlinux.tmp. It might not actually be stripped.

quiet_cmd_precompress = PRECOMPRESS $@
ifndef CONFIG_KGDB
cmd_precompress = cp $< $@
else

      cmd_precompress = $(STRIP) $< -o $@

endif

Quoted 8 lines

vmlinux.tmp: vmlinux FORCE

        $(call if_changed,precompress)

targets += vmlinux.tmp

This will also correctly handle CONFIG_KGDB changing.

(Maybe the naming can be improved)

Nice, that's a lot better and I've reworked my patch to do that.

Quoted 4 lines

And for the compressor invocations we already have predefined commands:

vmlinux.gz: vmlinux.tmp FORCE

        $(call if_changed,gzip)

I also did this for all of the targets and $(call if_changed,lz4) results in an lz4 file with the old header format[0] and 8MB block size which is not what I need.
So v2 will have a patch to add a helper to generate that style of lz4 in scripts/Makefile.lib

Thanks!,

Daniel

0 - https://github.com/lz4/lz4/blob/dev/doc/lz4_Frame_format.md#legacy-frame

#5 lore raw
Quoted 15 lines · Daniel Palmer
Quoted 14 lines · Thomas Weißschuh
Quoted 1 line · Daniel Palmer

On Sat, 9 Aug 2025 at 16:50, Thomas Weißschuh <[email protected]> wrote:

I would continue with vmlinux.tmp. It might not actually be stripped.

quiet_cmd_precompress = PRECOMPRESS $@
ifndef CONFIG_KGDB
cmd_precompress = cp $< $@
else

      cmd_precompress = $(STRIP) $< -o $@

endif

vmlinux.tmp: vmlinux FORCE

        $(call if_changed,precompress)
Quoted 7 lines
Quoted 5 lines

targets += vmlinux.tmp

This will also correctly handle CONFIG_KGDB changing.

(Maybe the naming can be improved)

Nice, that's a lot better and I've reworked my patch to do that.

If you also modify the existing code, you should split up the changes over multiple patches. First split out the rule for vmlinux.tmp, then switch over the compressors. Each patch should do a single step and explain why it is done.

And I am still not happy about the naming in my example :-/

Quoted 10 lines
Quoted 4 lines

And for the compressor invocations we already have predefined commands:

vmlinux.gz: vmlinux.tmp FORCE

        $(call if_changed,gzip)

I also did this for all of the targets and $(call if_changed,lz4) results in an lz4 file with the old header format[0] and 8MB block size which is not what I need.
So v2 will have a patch to add a helper to generate that style of lz4 in scripts/Makefile.lib

Modifying generic files in scripts/ will be more effort to argue for. Also that change should be its own patch again.

I am wondering a bit why the legacy header format does not work. The manpage of lz4 specifically mentions that it is commonly used for Linux kernel images. Which is exactly what we are doing here. Does your usecase work not work with the legacy header? If not, please mention this in the patches.

Thomas