Skip to contentSkip to conversation

Uncompressed kernel doesn't build on x86_64

PATCH In review Pavel Roskin 5 participants 15 messages Nov 13, 2013 – Nov 16, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

Hi Christian,

Your recent patch enables uncompressed kernels, but selecting that option (CONFIG_KERNEL_UNCOMPRESSED) fails on x86_86:

commit 69f0554ec261fd686ac7fa1c598cc9eb27b83a80

Author: Christian Ruppert <[email protected]>
Date:   Tue Nov 12 15:11:43 2013 -0800

init/Kconfig: add option to disable kernel compression

That's how it fails:

  LD      init/built-in.o
  KSYM    .tmp_kallsyms1.o
  KSYM    .tmp_kallsyms2.o
  LD      vmlinux
  SORTEX  vmlinux
  SYSMAP  System.map

VOFFSET arch/x86/boot/voffset.h
make[4]: *** No rule to make target
arch/x86/boot/compressed/vmlinux.bin.', needed by arch/x86/boot/compressed/piggy.S'. Stop. make[3]: *** [arch/x86/boot/compressed/vmlinux] Error 2 make[2]: *** [bzImage] Error 2 make[1]: *** [sub-make] Error 2 make: *** [all] Error 2

arch/x86/boot/compressed/Makefile has no provision for uncompressed files. There is no definition for suffix-$(CONFIG_KERNEL_UNCOMPRESSED) so $(suffix-y) evaluates to an empty string and there is no rule to make vmlinuz.bin.$(suffix-y)

I think the patch should be either reverted or fixed as soon as possible.

Any suggestions for uncompressed suffix? I'd go with "copy". Then the "compression" could be a symlink. Alternatively, the "." could be included in other suffixes, the uncompressed suffix would be empty and the compression would be an empty statement.

Signature · 2 lines
Regards,
Pavel Roskin
#1 lore raw
Quoted 6 lines · Pavel Roskin

Any suggestions for uncompressed suffix? I'd go with "copy". Then the "compression" could be a symlink. Alternatively, the "." could be included in other suffixes, the uncompressed suffix would be empty and the compression would be an empty statement.

The latter would be my preference.

-hpa

#2 lore raw
In reply to Pavel Roskin
Quoted 10 lines · Pavel Roskin

Hi Christian,

Your recent patch enables uncompressed kernels, but selecting that option (CONFIG_KERNEL_UNCOMPRESSED) fails on x86_86:
[...]
arch/x86/boot/compressed/Makefile has no provision for uncompressed files. There is no definition for suffix-$(CONFIG_KERNEL_UNCOMPRESSED) so $(suffix-y) evaluates to an empty string and there is no rule to make vmlinuz.bin.$(suffix-y)

Quoted 2 lines

I think the patch should be either reverted or fixed as soon as possible.

This is an interesting observation, thanks. Although this patch is rather intended for embedded architectures than x86, it highlights potential issues with self-decompressing images of all sorts. I have no way of testing all architectures and so I'm sending two patches in the follow-up:
1. A conservative version of the original patch which requires architectures to explicitly enable uncompresed kernel images if they support compression as well and falls back to previous behaviour if nothing is specified. I have copied Vineet Gupta (maintainer of the ARC platform) since I enabled GZIP and uncompressed kernels for ARC by default.
2. A patch to enable uncompressed x86 kernels. As stated above, I don't think this makes a lot of sense in itself but it might serve as an example for people working on other platforms with self-extracting kernels and the nozip not-decompression algorithm might be useful on those platforms as well. I only had a single x86-64 machine available to test this, however, so some more testing might be required.

Quoted 4 lines

Any suggestions for uncompressed suffix? I'd go with "copy". Then the "compression" could be a symlink. Alternatively, the "." could be included in other suffixes, the uncompressed suffix would be empty and the compression would be an empty statement.

I followed H. Peter's suggestion and chose option 2 in patch 2.

Greetings,
Christian

#3 lore raw

PATCH 1/2 init/Kconfig: add option to disable kernel compression

Some ARC users say they can boot faster with without kernel compression. This probably depends on things like the FLASH chip they use etc.

Until now, kernel compression can only be disabled by removing "select HAVE_<compression>" lines from the architecture Kconfig. So add the Kconfig logic to permit disabling of kernel compression.

---
 arch/arc/Kconfig |  2 ++
 init/Kconfig     | 12 +++++++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)
arch/arc/Kconfig +2 -0
index 91dbb27..3991f03 100644
@@ -21,6 +21,8 @@ config ARC
 	select HAVE_ARCH_KGDB
 	select HAVE_ARCH_TRACEHOOK
 	select HAVE_IOREMAP_PROT
+	select HAVE_KERNEL_UNCOMPRESSED
+	select HAVE_KERNEL_GZIP
 	select HAVE_KPROBES
 	select HAVE_KRETPROBES
 	select HAVE_MEMBLOCK
init/Kconfig +11 -1
index 3ecd8a1..b1a6f92 100644
@@ -97,6 +97,9 @@ config LOCALVERSION_AUTO
 
 	  which is done within the script "scripts/setlocalversion".)
 
+config HAVE_KERNEL_UNCOMPRESSED
+        bool
+
 config HAVE_KERNEL_GZIP
 	bool
 
@@ -118,7 +121,6 @@ config HAVE_KERNEL_LZ4
 choice
 	prompt "Kernel compression mode"
 	default KERNEL_GZIP
-	depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4
 	help
 	  The linux kernel is a kind of self-extracting executable.
 	  Several compression algorithms are available, which differ
@@ -137,6 +139,14 @@ choice
 
 	  If in doubt, select 'gzip'
 
+config KERNEL_UNCOMPRESSED
+	bool "No compression"
+	depends on HAVE_KERNEL_UNCOMPRESSED || ! ( HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 )
+	help
+	  No compression at all. The kernel is huge but the compression and
+	  decompression times are zero.
+	  This is usually not what you want.
+
 config KERNEL_GZIP
 	bool "Gzip"
 	depends on HAVE_KERNEL_GZIP
Signature · 1 line
1.8.4
#4 lore raw

Re: [PATCH 1/2] init/Kconfig: add option to disable kernel compression

+CC Sam for Kconfig wisdom

Quoted 18 lines · Christian Ruppert

Some ARC users say they can boot faster with without kernel compression. This probably depends on things like the FLASH chip they use etc.

Until now, kernel compression can only be disabled by removing "select HAVE_<compression>" lines from the architecture Kconfig. So add the Kconfig logic to permit disabling of kernel compression.

---
 arch/arc/Kconfig |  2 ++
 init/Kconfig     | 12 +++++++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)
arch/arc/Kconfig +0 -0
index 91dbb27..3991f03 100644

arch/arc/Kconfig config ARC · line 25

Quoted 5 lines
arch/arc/Kconfig +2 -0
@@ -21,6 +21,8 @@ config ARC
 	select HAVE_ARCH_KGDB
 	select HAVE_ARCH_TRACEHOOK
 	select HAVE_IOREMAP_PROT
+	select HAVE_KERNEL_UNCOMPRESSED
+	select HAVE_KERNEL_GZIP

Fine.

Quoted 8 lines
arch/arc/Kconfig +0 -0
 	select HAVE_KPROBES
 	select HAVE_KRETPROBES
 	select HAVE_MEMBLOCK
init/Kconfig +0 -0
index 3ecd8a1..b1a6f92 100644

init/Kconfig config LOCALVERSION_AUTO · line 102

Quoted 6 lines
init/Kconfig +3 -0
@@ -97,6 +97,9 @@ config LOCALVERSION_AUTO
 
 	  which is done within the script "scripts/setlocalversion".)
 
+config HAVE_KERNEL_UNCOMPRESSED
+        bool
+

This is good to avoid perturbing other arches.

Quoted 17 lines
init/Kconfig +1 -1
 config HAVE_KERNEL_GZIP
 	bool
 
@@ -118,7 +121,6 @@ config HAVE_KERNEL_LZ4
 choice
 	prompt "Kernel compression mode"
 	default KERNEL_GZIP
-	depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4
 	help
 	  The linux kernel is a kind of self-extracting executable.
 	  Several compression algorithms are available, which differ
@@ -137,6 +139,14 @@ choice
 
 	  If in doubt, select 'gzip'
 
+config KERNEL_UNCOMPRESSED

init/Kconfig choice · line 152

Quoted 10 lines
init/Kconfig +7 -0
+	bool "No compression"
+	depends on HAVE_KERNEL_UNCOMPRESSED || ! ( HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 )
+	help
+	  No compression at all. The kernel is huge but the compression and
+	  decompression times are zero.
+	  This is usually not what you want.
+
 config KERNEL_GZIP
 	bool "Gzip"
 	depends on HAVE_KERNEL_GZIP

How about doing this part slightly differently (simpler IMO).
We add uncompressed as just another category rather than being a special case.

Indicative diff against current mainline code

choice
prompt "Kernel compression mode"
default KERNEL_GZIP
- depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA ||
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4
+ depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA ||
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED

+config KERNEL_UNCOMPRESSED
+ bool "No compression"
+ depends on HAVE_KERNEL_UNCOMPRESSED
+

-Vineet

#5 lore raw

Re: [PATCH 1/2] init/Kconfig: add option to disable kernel compression

Quoted 88 lines · Vineet Gupta

+CC Sam for Kconfig wisdom

Quoted 18 lines · Christian Ruppert

Some ARC users say they can boot faster with without kernel compression. This probably depends on things like the FLASH chip they use etc.

Until now, kernel compression can only be disabled by removing "select HAVE_<compression>" lines from the architecture Kconfig. So add the Kconfig logic to permit disabling of kernel compression.

---
 arch/arc/Kconfig |  2 ++
 init/Kconfig     | 12 +++++++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)
arch/arc/Kconfig +0 -0
index 91dbb27..3991f03 100644

arch/arc/Kconfig config ARC · line 25

Quoted 5 lines
arch/arc/Kconfig +2 -0
@@ -21,6 +21,8 @@ config ARC
 	select HAVE_ARCH_KGDB
 	select HAVE_ARCH_TRACEHOOK
 	select HAVE_IOREMAP_PROT
+	select HAVE_KERNEL_UNCOMPRESSED
+	select HAVE_KERNEL_GZIP

Fine.

Quoted 8 lines
arch/arc/Kconfig +0 -0
 	select HAVE_KPROBES
 	select HAVE_KRETPROBES
 	select HAVE_MEMBLOCK
init/Kconfig +0 -0
index 3ecd8a1..b1a6f92 100644

init/Kconfig config LOCALVERSION_AUTO · line 102

Quoted 6 lines
init/Kconfig +3 -0
@@ -97,6 +97,9 @@ config LOCALVERSION_AUTO
 
 	  which is done within the script "scripts/setlocalversion".)
 
+config HAVE_KERNEL_UNCOMPRESSED
+        bool
+

This is good to avoid perturbing other arches.

Quoted 17 lines
init/Kconfig +1 -1
 config HAVE_KERNEL_GZIP
 	bool
 
@@ -118,7 +121,6 @@ config HAVE_KERNEL_LZ4
 choice
 	prompt "Kernel compression mode"
 	default KERNEL_GZIP
-	depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4
 	help
 	  The linux kernel is a kind of self-extracting executable.
 	  Several compression algorithms are available, which differ
@@ -137,6 +139,14 @@ choice
 
 	  If in doubt, select 'gzip'
 
+config KERNEL_UNCOMPRESSED

init/Kconfig choice · line 152

Quoted 10 lines
init/Kconfig +7 -0
+	bool "No compression"
+	depends on HAVE_KERNEL_UNCOMPRESSED || ! ( HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 )
+	help
+	  No compression at all. The kernel is huge but the compression and
+	  decompression times are zero.
+	  This is usually not what you want.
+
 config KERNEL_GZIP
 	bool "Gzip"
 	depends on HAVE_KERNEL_GZIP

How about doing this part slightly differently (simpler IMO).
We add uncompressed as just another category rather than being a special case.

Indicative diff against current mainline code

choice
prompt "Kernel compression mode"
default KERNEL_GZIP
- depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA ||
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4
+ depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA ||
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED

init/Kconfig choice · line 149

Quoted 4 lines
init/Kconfig +4 -0
+config KERNEL_UNCOMPRESSED
+       bool "No compression"
+       depends on HAVE_KERNEL_UNCOMPRESSED
+

Good Idea. I fixed this and sent the updated patch directly in reply to H. Peter's revert patch to avoid too much patch ping-pong:
http://www.spinics.net/lists/kernel/msg1636397.html

Greetings,
Christian

#6 lore raw

Re: [PATCH 1/2] init/Kconfig: add option to disable kernel compression

Quoted 89 lines · Christian Ruppert
Quoted 88 lines · Vineet Gupta

+CC Sam for Kconfig wisdom

Quoted 18 lines · Christian Ruppert

Some ARC users say they can boot faster with without kernel compression. This probably depends on things like the FLASH chip they use etc.

Until now, kernel compression can only be disabled by removing "select HAVE_<compression>" lines from the architecture Kconfig. So add the Kconfig logic to permit disabling of kernel compression.

---
 arch/arc/Kconfig |  2 ++
 init/Kconfig     | 12 +++++++++++-
 2 files changed, 13 insertions(+), 1 deletion(-)
arch/arc/Kconfig +0 -0
index 91dbb27..3991f03 100644

arch/arc/Kconfig config ARC · line 25

Quoted 5 lines
arch/arc/Kconfig +2 -0
@@ -21,6 +21,8 @@ config ARC
 	select HAVE_ARCH_KGDB
 	select HAVE_ARCH_TRACEHOOK
 	select HAVE_IOREMAP_PROT
+	select HAVE_KERNEL_UNCOMPRESSED
+	select HAVE_KERNEL_GZIP

Fine.

Quoted 8 lines
arch/arc/Kconfig +0 -0
 	select HAVE_KPROBES
 	select HAVE_KRETPROBES
 	select HAVE_MEMBLOCK
init/Kconfig +0 -0
index 3ecd8a1..b1a6f92 100644

init/Kconfig config LOCALVERSION_AUTO · line 102

Quoted 6 lines
init/Kconfig +3 -0
@@ -97,6 +97,9 @@ config LOCALVERSION_AUTO
 
 	  which is done within the script "scripts/setlocalversion".)
 
+config HAVE_KERNEL_UNCOMPRESSED
+        bool
+

This is good to avoid perturbing other arches.

Quoted 17 lines
init/Kconfig +1 -1
 config HAVE_KERNEL_GZIP
 	bool
 
@@ -118,7 +121,6 @@ config HAVE_KERNEL_LZ4
 choice
 	prompt "Kernel compression mode"
 	default KERNEL_GZIP
-	depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4
 	help
 	  The linux kernel is a kind of self-extracting executable.
 	  Several compression algorithms are available, which differ
@@ -137,6 +139,14 @@ choice
 
 	  If in doubt, select 'gzip'
 
+config KERNEL_UNCOMPRESSED

init/Kconfig choice · line 152

Quoted 10 lines
init/Kconfig +7 -0
+	bool "No compression"
+	depends on HAVE_KERNEL_UNCOMPRESSED || ! ( HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA || HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 )
+	help
+	  No compression at all. The kernel is huge but the compression and
+	  decompression times are zero.
+	  This is usually not what you want.
+
 config KERNEL_GZIP
 	bool "Gzip"
 	depends on HAVE_KERNEL_GZIP

How about doing this part slightly differently (simpler IMO).
We add uncompressed as just another category rather than being a special case.

Indicative diff against current mainline code

choice
prompt "Kernel compression mode"
default KERNEL_GZIP
- depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA ||
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4
+ depends on HAVE_KERNEL_GZIP || HAVE_KERNEL_BZIP2 || HAVE_KERNEL_LZMA ||
HAVE_KERNEL_XZ || HAVE_KERNEL_LZO || HAVE_KERNEL_LZ4 || HAVE_KERNEL_UNCOMPRESSED

Quoted 8 lines

init/Kconfig choice · line 149

Quoted 4 lines
init/Kconfig +4 -0
+config KERNEL_UNCOMPRESSED
+       bool "No compression"
+       depends on HAVE_KERNEL_UNCOMPRESSED
+

Good Idea. I fixed this and sent the updated patch directly in reply to H. Peter's revert patch to avoid too much patch ping-pong:
http://www.spinics.net/lists/kernel/msg1636397.html

I didn't see that patch in my mailbox. It needs an ACK from me anyways.

-Vineet

#7 lore raw

PATCH 2/2 x86: Add support for uncompressed kernel images

In reply to Christian Ruppert
---
 arch/x86/Kconfig                  |  1 +
 arch/x86/boot/compressed/Makefile | 14 ++++++------
 arch/x86/boot/compressed/misc.c   |  4 ++++
 lib/decompress_copy.c             | 47 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 59 insertions(+), 7 deletions(-)
 create mode 100644 lib/decompress_copy.c
arch/x86/Kconfig +1 -0
index f67e839..2a32df9 100644
@@ -61,6 +61,7 @@ config X86
 	select USER_STACKTRACE_SUPPORT
 	select HAVE_REGS_AND_STACK_ACCESS_API
 	select HAVE_DMA_API_DEBUG
+	select HAVE_KERNEL_UNCOMPRESSED
 	select HAVE_KERNEL_GZIP
 	select HAVE_KERNEL_BZIP2
 	select HAVE_KERNEL_LZMA
arch/x86/boot/compressed/Makefile +7 -7
index dcd90df..f65e444 100644
@@ -67,16 +67,16 @@ $(obj)/vmlinux.bin.lzo: $(vmlinux.bin.all-y) FORCE
 $(obj)/vmlinux.bin.lz4: $(vmlinux.bin.all-y) FORCE
 	$(call if_changed,lz4)
 
-suffix-$(CONFIG_KERNEL_GZIP)	:= gz
-suffix-$(CONFIG_KERNEL_BZIP2)	:= bz2
-suffix-$(CONFIG_KERNEL_LZMA)	:= lzma
-suffix-$(CONFIG_KERNEL_XZ)	:= xz
-suffix-$(CONFIG_KERNEL_LZO) 	:= lzo
-suffix-$(CONFIG_KERNEL_LZ4) 	:= lz4
+suffix-$(CONFIG_KERNEL_GZIP)	:= .gz
+suffix-$(CONFIG_KERNEL_BZIP2)	:= .bz2
+suffix-$(CONFIG_KERNEL_LZMA)	:= .lzma
+suffix-$(CONFIG_KERNEL_XZ)	:= .xz
+suffix-$(CONFIG_KERNEL_LZO) 	:= .lzo
+suffix-$(CONFIG_KERNEL_LZ4) 	:= .lz4
 
 quiet_cmd_mkpiggy = MKPIGGY $@
       cmd_mkpiggy = $(obj)/mkpiggy $< > $@ || ( rm -f $@ ; false )
 
 targets += piggy.S
-$(obj)/piggy.S: $(obj)/vmlinux.bin.$(suffix-y) $(obj)/mkpiggy FORCE
+$(obj)/piggy.S: $(obj)/vmlinux.bin$(suffix-y) $(obj)/mkpiggy FORCE
 	$(call if_changed,mkpiggy)
arch/x86/boot/compressed/misc.c +4 -0
index 434f077..c210314 100644
@@ -149,6 +149,10 @@ static int lines, cols;
 #include "../../../../lib/decompress_unlz4.c"
 #endif
 
+#ifdef CONFIG_KERNEL_UNCOMPRESSED
+#include "../../../../lib/decompress_copy.c"
+#endif
+
 static void scroll(void)
 {
 	int i;
lib/decompress_copy.c new file +47 -0

47 lines not on this page · raw message

Signature · 1 line
1.8.4
#8 lore raw

Re: [PATCH 2/2] x86: Add support for uncompressed kernel images

So it sounds like we're starting from the beginning - I presume that means the patch currently in the kernel should be reverted first?

-hpa

#9 lore raw

Re: [PATCH 2/2] x86: Add support for uncompressed kernel images

Quoted 2 lines · H. Peter Anvin

So it sounds like we're starting from the beginning - I presume that means the patch currently in the kernel should be reverted first?

Ack

#10 lore raw

Re: [PATCH 2/2] x86: Add support for uncompressed kernel images

In reply to Christian Ruppert
Quoted 30 lines · Christian Ruppert
lib/decompress_copy.c new file +24 -0
new file mode 100644
index 0000000..8a41090
@@ -0,0 +1,47 @@
+#include <linux/decompress/mm.h>
+
+#define NOZIP_BUFSZ (16 * 1024)
+STATIC int INIT nozip(unsigned char *buf, int len,
+			int(*fill)(void*, unsigned int),
+			int(*flush)(void*, unsigned int),
+			unsigned char *outbuf,
+			int *pos,
+			void(*error)(char *x))
+{
+	char *b;
+
+	if (buf)
+		b = buf;
+	else
+		b = malloc(NOZIP_BUFSZ);
+
+	if (!b) {
+		error("Out of memory while allocating buffer");
+		return -1;
+	}
+
+	if (flush) {
+		if (!len)

lib/decompress_copy.c line 34

Quoted 10 lines
lib/decompress_copy.c new file +10 -0
+			len = fill(b, NOZIP_BUFSZ);
+
+		len = flush(b, len);
+	} else {
+		if (!len)
+			len = fill(outbuf, NOZIP_BUFSZ);
+		else {
+			int i;
+			for (i = 0; i < len; i++)
+				outbuf[i] = b[i];

Clever way of doing memcpy(). I know some other compression implementations open-code memmove(), but using it for the entire data set in the most common case is atrocious. If we need to get a default implementation of memmove() we should just do that.

-hpa

#11 lore raw

Re: [PATCH 2/2] x86: Add support for uncompressed kernel images

Quoted 38 lines · H. Peter Anvin
Quoted 37 lines · Christian Ruppert
lib/decompress_copy.c new file +31 -0
new file mode 100644
index 0000000..8a41090
@@ -0,0 +1,47 @@
+#include <linux/decompress/mm.h>
+
+#define NOZIP_BUFSZ (16 * 1024)
+STATIC int INIT nozip(unsigned char *buf, int len,
+			int(*fill)(void*, unsigned int),
+			int(*flush)(void*, unsigned int),
+			unsigned char *outbuf,
+			int *pos,
+			void(*error)(char *x))
+{
+	char *b;
+
+	if (buf)
+		b = buf;
+	else
+		b = malloc(NOZIP_BUFSZ);
+
+	if (!b) {
+		error("Out of memory while allocating buffer");
+		return -1;
+	}
+
+	if (flush) {
+		if (!len)
+			len = fill(b, NOZIP_BUFSZ);
+
+		len = flush(b, len);
+	} else {
+		if (!len)
+			len = fill(outbuf, NOZIP_BUFSZ);
+		else {
Quoted 8 lines

lib/decompress_copy.c line 34

Quoted 3 lines
lib/decompress_copy.c new file +3 -0
+			int i;
+			for (i = 0; i < len; i++)
+				outbuf[i] = b[i];

Clever way of doing memcpy(). I know some other compression implementations open-code memmove(), but using it for the entire data set in the most common case is atrocious. If we need to get a default implementation of memmove() we should just do that.

Actually, the x86 architecture already contains a basic implementation of memcpy in its self-decompressor. Correct me if I'm wrong but I guess memmove is not required since I don't expect the output of a generic decompression function to overlap with its input.

We can simply use that function if we agree that
. either nozip is x86 specific
. or every architecture using this code must bring along its own memcpy
implementation in its self-decompressor.

Do you think that would be OK?

Greetings,
Christian

#12 lore raw

Re: [PATCH 2/2] x86: Add support for uncompressed kernel images

Quoted 6 lines · Christian Ruppert

Actually, the x86 architecture already contains a basic implementation of memcpy in its self-decompressor. Correct me if I'm wrong but I guess memmove is not required since I don't expect the output of a generic decompression function to overlap with its input.

Quoted 7 lines

We can simply use that function if we agree that
. either nozip is x86 specific
. or every architecture using this code must bring along its own memcpy
implementation in its self-decompressor.

Do you think that would be OK?

I think all of them have memcpy().

-hpa

#13 lore raw
In reply to Christian Ruppert
Quoted 6 lines · Christian Ruppert

2. A patch to enable uncompressed x86 kernels. As stated above, I don't think this makes a lot of sense in itself but it might serve as an example for people working on other platforms with self-extracting kernels and the nozip not-decompression algorithm might be useful on those platforms as well. I only had a single x86-64 machine available to test this, however, so some more testing might be required.

I disagree with the argument that an uncompressed x86 kernel doesn't make sense, If you have a very fast boot device, then it is fully conceivable that an uncompressed kernel could boot faster than a compressed one. I have seen a very large number of systems where the LZO compression boots at least twice as fast as gzip or bz2 (because the disks are fast enough that a few megabytes of size difference make much less of an impact than a slow decompressor).

#14 lore raw
Quoted 9 lines · On Thu, 14 Nov 2013 09:13:44 -0500 Austin S Hemmelgarn <[email protected]>…
Quoted 7 lines · Christian Ruppert

2. A patch to enable uncompressed x86 kernels. As stated above, I don't think this makes a lot of sense in itself but it might serve as an example for people working on other platforms with self-extracting kernels and the nozip not-decompression algorithm might be useful on those platforms as well. I only had a single x86-64 machine available to test this, however, so some more testing might be required.

Quoted 7 lines

I disagree with the argument that an uncompressed x86 kernel doesn't make sense, If you have a very fast boot device, then it is fully conceivable that an uncompressed kernel could boot faster than a compressed one. I have seen a very large number of systems where the LZO compression boots at least twice as fast as gzip or bz2 (because the disks are fast enough that a few megabytes of size difference make much less of an impact than a slow decompressor).

I concur, the uncompressed kernel certainly makes sense in some cases.

If the kernel runs in an emulator, decompression could be slow.

If the kernel runs in a virtual machine, the kernel would need to be decompressed separately for every virtual machine. Reading from the disk would be done only once if several virtual machines are started in a short period of time.

Signature · 2 lines
Regards,
Pavel Roskin