Skip to contentSkip to conversation

Easy trick to reduce kernel footprint

DISCUSSION Willy Tarreau 5 participants 13 messages Jun 5, 2005 – Jun 6, 2005

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi,

Here's a simple trick for all those who try to squeeze their kernels to the absolute smallest size.

I recently discovered p7zip which comes with the LZMA compression algorithm, which is somewhat better than gzip and bzip2 on most datasets, and I also noticed that this tool provides support for gzip and bzip2 outputs. So I tried to produce some of those standard outputs, and observed a slight gain compared to the default tools. The reason is that we can change the number of passes and the dictionnary size.

So as an experiment, I used it to compress my kernel+initramfs and I could gain about 2% (23 kB) which is not bad at all for embedded systems. Don't ask about the '.' after '-si' in the '7za' command line, it's just that the tool expects a file name, and I didn't managed to fix it, the source is some sort of obfuscated c++ code (who said "pleonasm" ?) ported from windows, but at least it works. Out of curiosity, I also tried both bzip2 and the LZMA algorithm, although that's not fair because the decompressor code would have to be changed to support them.

Size of the 2.6.12-rc4-mm2 vmlinux compiled without the initramfs image :

bash-3.00$ size vmlinux
text data bss dec hex filename
1024458 123712 73580 1221750 12a476 vmlinux

Size of the initramfs :
-rw-rw-r-- 1 willy users 1014272 2005-05-27 16:31 usr/initramfs_data.cpio

Now the resulting bzImage size with various compression methods

     size  command
  1197277  make
  1197275  make cmd_gzip="gzip -9c <\$< >\$@"
  1173550  make cmd_gzip="7za a -tgzip -mx9 -mpass=4 -so -si . <\$< >\$@"
  1207599  make cmd_gzip="bzip2 -c9 <\$< >\$@"
  1051705  make cmd_gzip="rm -f \$@;7za a -t7z -mx9 -md=64m -si \$@ < \$<"

Surprizingly, bzip2 makes a bigger kernel. The gzip implementation in 7za saves 23 kB (2%) on the overall image without touching any code. The LZMA implementation could save 145 kB (12%), but would require a different extraction code (I've already seen patches to bring LZMA support on 2.4).

Note: I don't know who implemented the if_changed macro in the makefile which allows us to pass cmd_gzip here, but it was an excellent idea.

Regards,
Willy

#1 lore raw
Quoted 5 lines · Willy Tarreau

I recently discovered p7zip which comes with the LZMA compression algorithm, which is somewhat better than gzip and bzip2 on most datasets, and I also noticed that this tool provides support for gzip and bzip2 outputs. So I tried to produce some of those standard outputs, and observed a slight gain compared to the default tools.

But the kernel gunzip can't decompress everything that the normal gzip could decompress, i found some options it didn't work with. (but this was long ago when there wasn't a linux version and i used wine for 7za, but i don't think anyone even thinks about fixing gunzip())

Quoted 4 lines

Surprizingly, bzip2 makes a bigger kernel. The gzip implementation in 7za saves 23 kB (2%) on the overall image without touching any code. The LZMA implementation could save 145 kB (12%), but would require a different extraction code (I've already seen patches to bring LZMA support on 2.4).

Somehow this is not so easy, i'll post some stuff in another mail, perhaps there is just a stupid tiny error.

But at least for initrds lzma works nicely, therefore a patch:

This patch adds the possibility to compress a initrd with lzma, you need ths CONFIG_BLK_DEV_RAM_LZMA build option for this. (Device Drivers -> Block devices)

For example the debian installer initrd:

-rw-r--r--  1 ijuz ijuz 2870355 Mar  5 20:00 initrd.gz
-rw-r--r--  1 ijuz ijuz 2158769 May  8 02:09 initrd.lzma

Building the lzma tool cut and paste style:

mkdir build-lzma
wget http://www.7-zip.org/dl/lzma417.tar.bz2
tar xjf lzma417.tar.bz2
cd SRC/7zip/Compress/LZMA_Alone/
make
cp lzma ../../../../../
cd ../../../../../

drivers/block/Kconfig +37 -0
diff -Nru linux-2.6.12-rc3.orig/drivers/block/Kconfig linux-2.6.12-rc3/drivers/block/Kconfig
@@ -398,6 +398,43 @@
 	  what are you doing. If you are using IBM S/390, then set this to
 	  8192.
 
+config BLK_DEV_RAM_GZ
+       bool "Gzip compressed ramdisk support"
+       depends on BLK_DEV_RAM=y || BLK_DEV_RAM=m
+       default y
+       help
+         This option enables support for gzip compressed ramdisk images.
+         An image can be loaded as initrd (see above) or as a normal
+         ramdisk at boot time.
+         Enabling this option is good for people who use an initrd or
+         a normal ramdisk and who are low on available disk space.
+         The decompression process at boot time needs about 2 MB additional
+         RAM to the space the unpacked ramdisk needs.
+
+         Due to the fact that this feature is normally included in the
+         original ramdisk, you should say Y to this option.
+
+config BLK_DEV_RAM_LZMA
+       bool "Lzma compressed ramdisk support (EXPERIMENTAL)"
+       depends on BLK_DEV_RAM=y || BLK_DEV_RAM=m && EXPERIMENTAL
+       default n
+       help
+         This option enables support for lzma compressed ramdisk images.
+         An image can be loaded as initrd (see above) or as a normal
+         ramdisk at boot time.
+         Enabling this option is good for people who use an initrd or
+         a normal ramdisk and who are very low on available disk space
+         (eg. for embedded systems with rom image and plenty of RAM).
+         This option was implemented, because lzma has better compression
+         than gzip. If you want to use this feature and disk space is a
+         matter, say N to gzip support and compress your ramdisk using
+         lzma.
+
+         This is an optional feature. Only enable this option if you need
+         lzma support for your ramdisk (e.g. for boot+root floppies). So
+         if you are unsure say N.
+
+
 config BLK_DEV_INITRD
 	bool "Initial RAM disk (initrd) support"
 	depends on BLK_DEV_RAM=y
init/do_mounts.h +32 -0
diff -Nru linux-2.6.12-rc3.orig/init/do_mounts.h linux-2.6.12-rc3/init/do_mounts.h
@@ -90,3 +90,35 @@
 static inline void md_run_setup(void) {}
 
 #endif
+
+
+/*
+ *
+ * The following lines define some constants to support different
+ * compressed ramdisk types.
+ * Each compressed ramdisk type is bound to a _negative_ value,
+ * since positive values give us the number of blocks.
+ *
+ */
+
+#define RD_ERROR       (-65535)
+#define CRAMDISK_LZMA   (-1)
+#define CRAMDISK_GZ    (-2)
+
+#if defined(CONFIG_BLK_DEV_RAM_GZ) || defined(CONFIG_BLK_DEV_RAM_LZMA)
+#      define BUILD_CRAMDISK
+#endif /* (CONFIG_BLK_DEV_RAM_GZ) || defined(CONFIG_BLK_DEV_RAM_BZ2) */
+
+#ifdef CONFIG_BLK_DEV_RAM_GZ
+int __init gz_load(int in_fd, int out_fd);
+#else
+static inline int __init gz_load(int in_fd, int out_fd) { return RD_ERROR; }
+#endif /* CONFIG_BLK_DEV_RAM_GZ */
+
+#ifdef CONFIG_BLK_DEV_RAM_LZMA
+int __init lzma_load(int in_fd, int out_fd);
+#else
+static inline int __init lzma_load(int in_fd, int out_fd) { return RD_ERROR; }
+#endif /* CONFIG_BLK_DEV_RAM_BZ2 */
+
+
init/do_mounts_rd.c +189 -29

282 lines not on this page · raw message

lib/LzmaDecode.c +586 -0

586 lines not on this page · raw message

lib/LzmaDecode.h +100 -0

100 lines not on this page · raw message

Signature · 1 line
http://www.nosoftwarepatents.com
#2 lore raw
In reply to Willy Tarreau
Quoted 9 lines · Willy Tarreau

Here's a simple trick for all those who try to squeeze their kernels to the absolute smallest size.

I recently discovered p7zip which comes with the LZMA compression algorithm, which is somewhat better than gzip and bzip2 on most datasets, and I also noticed that this tool provides support for gzip and bzip2 outputs. So I tried to produce some of those standard outputs, and observed a slight gain compared to the default tools. The reason is that we can change the number of passes and the dictionnary size.

Is it any smaller than a UPX'd kernel? (I think you need the beta version. I know the upx-ucl in debian won't compress but upx-ucl-beta will if you force). I got a significant reduction using it.

Signature · 1 line
 Lab tests show that use of micro$oft causes cancer in lab animals
#3 lore raw
Quoted 11 lines · Wakko Warner
Quoted 9 lines · Willy Tarreau

Here's a simple trick for all those who try to squeeze their kernels to the absolute smallest size.

I recently discovered p7zip which comes with the LZMA compression algorithm, which is somewhat better than gzip and bzip2 on most datasets, and I also noticed that this tool provides support for gzip and bzip2 outputs. So I tried to produce some of those standard outputs, and observed a slight gain compared to the default tools. The reason is that we can change the number of passes and the dictionnary size.

Quoted 3 lines

Is it any smaller than a UPX'd kernel? (I think you need the beta version. I know the upx-ucl in debian won't compress but upx-ucl-beta will if you force). I got a significant reduction using it.

It's not better at all, but unfortunately, UPX cannot compress a kernel which embeds a big initramfs. The problem is that the compressed initramfs is embedded into vmlinux, which is then compressed into bzImage, and UPX only replaces the bzImage compression.

May be it would work if we did not compress the initramfs before including it into the vmlinux.

Willy

#4 lore raw

Keep me CC.

Quoted 5 lines · Willy Tarreau
Quoted 3 lines · Wakko Warner

Is it any smaller than a UPX'd kernel? (I think you need the beta version. I know the upx-ucl in debian won't compress but upx-ucl-beta will if you force). I got a significant reduction using it.

Quoted 7 lines

It's not better at all, but unfortunately, UPX cannot compress a kernel which embeds a big initramfs. The problem is that the compressed initramfs is embedded into vmlinux, which is then compressed into bzImage, and UPX only replaces the bzImage compression.

May be it would work if we did not compress the initramfs before including it into the vmlinux.

My initramfs is passed via initrd so that I can change any aspect of it with out recompiling the kernel (or maybe i could use a better understanding of initramfs) I compared bzImage to bupxImage and the savings I got was around 50kb difference IIRC.

Signature · 1 line
 Lab tests show that use of micro$oft causes cancer in lab animals
#5 lore raw
Quoted 4 lines · Wakko Warner

My initramfs is passed via initrd so that I can change any aspect of it with out recompiling the kernel (or maybe i could use a better understanding of initramfs) I compared bzImage to bupxImage and the savings I got was around 50kb difference IIRC.

With what algo and what kernel size ?
With the close-source upx-1.93 linked with the NRV compression, I often observe an average 15-20% gain on the bzImage size if it does not embed an initramfs. I remember that the UCL library was not as good as the NRV, so I've stopped using it a long time ago.

Regards,
Willy

#6 lore raw
Quoted 7 lines · Willy Tarreau
Quoted 4 lines · Wakko Warner

My initramfs is passed via initrd so that I can change any aspect of it with out recompiling the kernel (or maybe i could use a better understanding of initramfs) I compared bzImage to bupxImage and the savings I got was around 50kb difference IIRC.

Quoted 5 lines

With what algo and what kernel size ?
With the close-source upx-1.93 linked with the NRV compression, I often observe an average 15-20% gain on the bzImage size if it does not embed an initramfs. I remember that the UCL library was not as good as the NRV, so I've stopped using it a long time ago.

I'm using upx-ucl-beta v1.91+0.20030910cvs-2 debian package. Command line:
upx-ucl-beta -9 -f bupxImage-2.6.12-rc6

-rw-r--r--  3 root root 618442 Jun  6 16:14 bupxImage-2.6.12-rc6
-rwxrwxrwx  1 root root 669106 Jun  6 16:14 bzImage-2.6.12-rc6*

50664 bytes savings.
the kernel I have there is 100% modular except that ramdisk is compiled in. It was also compiled with -Os (Optimize for size config option) using gcc 3.3.5.

Signature · 1 line
 Lab tests show that use of micro$oft causes cancer in lab animals
#7 lore raw
In reply to Willy Tarreau
Quoted 3 lines · Willy Tarreau

saves 23 kB (2%) on the overall image without touching any code. The LZMA implementation could save 145 kB (12%), but would require a different extraction code (I've already seen patches to bring LZMA support on 2.4).

The patch for 2.4 is here:
http://www.zelow.no/floppyfw/download/Development/Patches/kernel/040-lzma-vmlinuz.diff
and it actually works.

My not working patch is here:
http://debian.christian-leber.de/kernel_lzma/lzma_image.patch
you have to apply the patch from the other mail before.
(or here: http://debian.christian-leber.de/kernel_lzma/lzma_ramdisk.patch)

If you just want to see how it doesn't work, here is a kernel image to see it failing in qemu, basically it just decompresses again and again.
http://debian.christian-leber.de/kernel_lzma/bzImage

The decompression works correctly(checksum), i think the problem is the address where the decompressed kernel image should be, i just can't get out how this piece of code is supposed to work.

Therefore it could be a small problem only.

Another minor problem of lzma is that you need slightly more memory than with gzip. (at least when you can't have the output in a continuous piece of memory)
On the plus side is also that the lzma decompression code itself is smaller and doesn't have such a ugly "interface".

Christian Leber

Signature · 1 line
http://www.nosoftwarepatents.com
#8 lore raw
In reply to Willy Tarreau
Quoted 3 lines · Willy Tarreau

I recently discovered p7zip which comes with the LZMA compression algorithm, which is somewhat better than gzip and bzip2 on most datasets, [...]

Hmmm.

Citeseer has never heard of that algorithm, top 10 google hits for "LZMA compression algorithm" are completely uninformative. Does anyone actually know, what this algorithm is doing?

Jörn

Signature · 4 lines
Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it.
-- Brian W. Kernighan
#9 lore raw

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jörn Engel wrote:

| On Mon, 6 June 2005 00:35:28 +0200, Willy Tarreau wrote:
|
|>I recently discovered p7zip which comes with the LZMA compression
algorithm,
|>which is somewhat better than gzip and bzip2 on most datasets, [...]
|
|
| Hmmm.
|
| Citeseer has never heard of that algorithm, top 10 google hits for
| "LZMA compression algorithm" are completely uninformative.  Does
| anyone actually know, what this algorithm is doing?
|
| Jörn
|

Dear Jörn

i just googled it up, and the first link ive found is:

http://martinus.geekisp.com/rublog.cgi/Projects/LZMA

might be that one...

Greetings
Oliver Leitner
Technical Staff
http://www.shells.at
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (MingW32)

iD8DBQFCo+VdWvEVE8MtwbgRAoKEAJ4pbKdAYMCd1yJnZzCdp8EfZqfCZACglVOR Q9bUv3wMpj1xCkdZS3eAiDg=
=fD7R
-----END PGP SIGNATURE-----

#10 lore raw
In reply to Jörn Engel
Quoted 10 lines · J?rn Engel
Quoted 3 lines · Willy Tarreau

I recently discovered p7zip which comes with the LZMA compression algorithm, which is somewhat better than gzip and bzip2 on most datasets, [...]

Hmmm.

Citeseer has never heard of that algorithm, top 10 google hits for "LZMA compression algorithm" are completely uninformative. Does anyone actually know, what this algorithm is doing?

It's described here :
http://en.wikipedia.org/wiki/LZMA

implemented here :
http://martinus.geekisp.com/rublog.cgi/Projects/LZMA

and here :
http://www.7-zip.org/sdk.html

Willy

#11 lore raw
Quoted 6 lines · Willy Tarreau
Quoted 4 lines · J?rn Engel

Citeseer has never heard of that algorithm, top 10 google hits for "LZMA compression algorithm" are completely uninformative. Does anyone actually know, what this algorithm is doing?

Quoted 8 lines

It's described here :
http://en.wikipedia.org/wiki/LZMA

implemented here :
http://martinus.geekisp.com/rublog.cgi/Projects/LZMA

and here :

   http://www.7-zip.org/sdk.html

Thanks, but I already saw all three of those before I posted my reply. Guess the only way to tell is by reading the source...

Jörn

Signature · 4 lines
Correctness comes second.
Features come third.
Performance comes last.
Maintainability is needed for all of them.
#12 lore raw
In reply to Jörn Engel
Quoted 3 lines · J?rn Engel

Citeseer has never heard of that algorithm, top 10 google hits for "LZMA compression algorithm" are completely uninformative. Does anyone actually know, what this algorithm is doing?

Doesn't google have a searching howto or something?

First hit:
http://www.7-zip.org/sdk.html

Christian Leber

Signature · 1 line
http://www.nosoftwarepatents.com