Skip to contentSkip to conversation

Status of bzip2 and lzma kernel compression for ARM?

DISCUSSION Michael Opdenacker 6 participants 11 messages Jun 24, 2009 – Jul 13, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi Alain,

I would like to test bzip2 and lzma compression on ARM.

Would you mind telling me what the status of your patches on ARM is, now that the x86 and architecture independent code has been merged? Have you or has anyone already updated them for 2.6.30, or shall I update them by myself?

Thank you in advance,

Cheers,

Michael.

Signature · 10 lines
Michael Opdenacker, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com


-------------------------------------------------------------------
List admin: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ:        http://www.arm.linux.org.uk/mailinglists/faq.php
Etiquette:  http://www.arm.linux.org.uk/mailinglists/etiquette.php
#1 lore raw

Hi Michael,

Quoted 8 lines · Michael Opdenacker

Hi Alain,

I would like to test bzip2 and lzma compression on ARM.

Would you mind telling me what the status of your patches on ARM is, now that the x86 and architecture independent code has been merged? Have you or has anyone already updated them for 2.6.30, or shall I update them by myself?

I'm not sure what exactly do you mean by "test bzip2 and lzma compression on ARM", but if you refer to internal initramfs compression, I can tell that I've used lzma compression on PXA270 and it works fine.

Quoted 6 lines

Thank you in advance,

Cheers,

Michael.

Signature · 2 lines
Sincerely yours,
Mike.
#2 lore raw

Hi Michael, Mike

Le Thursday 25 June 2009 10:56:45 Mike Rapoport, vous avez écrit :

Quoted 7 lines

Hi Michael,

Quoted 4 lines · Michael Opdenacker

Hi Alain,

I would like to test bzip2 and lzma compression on ARM.

Quoted 8 lines
Quoted 4 lines

Would you mind telling me what the status of your patches on ARM is, now that the x86 and architecture independent code has been merged? Have you or has anyone already updated them for 2.6.30, or shall I update them by myself?

I'm not sure what exactly do you mean by "test bzip2 and lzma compression on ARM", but if you refer to internal initramfs compression, I can tell that I've used lzma compression on PXA270 and it works fine.

I think Michael refers to the lzma decompressor which allows you to use a lzma-compressed kernel as a (b)zImage with its architecture specific decompressor piggy-backed in the (b)zImage.

Such thing would also be very useful on MIPS and PowerPC as well.

Signature · 4 lines
Best regards, Florian Fainelli
Email : [email protected]
http://openwrt.org
-------------------------------
#3 lore raw
Quoted 6 lines · Florian Fainelli

Le Thursday 25 June 2009 10:56:45 Mike Rapoport, vous avez écrit :

Quoted 4 lines

I'm not sure what exactly do you mean by "test bzip2 and lzma compression on ARM", but if you refer to internal initramfs compression, I can tell that I've used lzma compression on PXA270 and it works fine.

Quoted 6 lines

I think Michael refers to the lzma decompressor which allows you to use a lzma-compressed kernel as a (b)zImage with its architecture specific decompressor piggy-backed in the (b)zImage.

Such thing would also be very useful on MIPS and PowerPC as well.

Yes, that's what I meant: using a bzip2 or lzma compressed kernel. I would like to know the boot time impact on an ARM board.

If I don't get any answer from Alain in the next days, I will propose a patch update for ARM.

Cheers,

Michael.

Signature · 4 lines
Michael Opdenacker, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com
#4 lore raw
Quoted 11 lines · Michael Opdenacker
Quoted 10 lines · Florian Fainelli

Le Thursday 25 June 2009 10:56:45 Mike Rapoport, vous avez ?crit :

Quoted 4 lines

I'm not sure what exactly do you mean by "test bzip2 and lzma compression on ARM", but if you refer to internal initramfs compression, I can tell that I've used lzma compression on PXA270 and it works fine.

I think Michael refers to the lzma decompressor which allows you to use a lzma-compressed kernel as a (b)zImage with its architecture specific decompressor piggy-backed in the (b)zImage.

Quoted 8 lines
Quoted 2 lines

Such thing would also be very useful on MIPS and PowerPC as well.

Yes, that's what I meant: using a bzip2 or lzma compressed kernel. I would like to know the boot time impact on an ARM board.

If I don't get any answer from Alain in the next days, I will propose a patch update for ARM.

It's possible to use this generically now, yes. Anything building uImages today can basically wire it up in a similar fashion to how blackfin has and support it out of the box.

#5 lore raw
In reply to Michael Opdenacker

Attached is a version of the ARM patch for 2.6.30

There seems to be some issues with the include files (which did not pose any problems in 2.6.28...), but hopefully somebody more experienced with the ARM architecture (and with its recent development) than I can help out there.

Regards,

Alain

arch/arm/boot/compressed/Makefile +21 -4
diff -purN linux-2.6.30/arch/arm/boot/compressed/Makefile linux-2.6.30.arm/arch/arm/boot/compressed/Makefile
@@ -63,8 +63,15 @@ endif
 
 SEDFLAGS	= s/TEXT_START/$(ZTEXTADDR)/;s/BSS_START/$(ZBSSADDR)/
 
-targets       := vmlinux vmlinux.lds piggy.gz piggy.o font.o font.c \
-		 head.o misc.o $(OBJS)
+suffix_$(CONFIG_KERNEL_GZIP)  = gz
+suffix_$(CONFIG_KERNEL_BZIP2) = bz2
+suffix_$(CONFIG_KERNEL_LZMA)  = lzma
+
+targets       := vmlinux vmlinux.lds \
+		 piggy.gz piggy.gz.o \
+		 piggy.bz2 piggy.bz2.o \
+		 piggy.lzma piggy.lzma.o \
+		 font.o font.c head.o misc.o $(OBJS)
 
 ifeq ($(CONFIG_FUNCTION_TRACER),y)
 ORIG_CFLAGS := $(KBUILD_CFLAGS)
@@ -91,7 +98,7 @@ LDFLAGS_vmlinux += -p --no-undefined -X 
 # would otherwise mess up our GOT table
 CFLAGS_misc.o := -Dstatic=
 
-$(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.o \
+$(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.$(suffix_y).o \
 	 	$(addprefix $(obj)/, $(OBJS)) FORCE
 	$(call if_changed,ld)
 	@:
@@ -99,7 +106,17 @@ $(obj)/vmlinux: $(obj)/vmlinux.lds $(obj
 $(obj)/piggy.gz: $(obj)/../Image FORCE
 	$(call if_changed,gzip)
 
-$(obj)/piggy.o:  $(obj)/piggy.gz FORCE
+$(obj)/piggy.bz2: $(obj)/../Image FORCE
+	$(call if_changed,bzip2)
+
+$(obj)/piggy.lzma: $(obj)/../Image FORCE
+	$(call if_changed,lzma)
+
+$(obj)/piggy.gz.o:  $(obj)/piggy.gz FORCE
+
+$(obj)/piggy.bz2.o:  $(obj)/piggy.bz2 FORCE
+
+$(obj)/piggy.lzma.o:  $(obj)/piggy.lzma FORCE
 
 CFLAGS_font.o := -Dstatic=
 
arch/arm/boot/compressed/misc.c +33 -108

186 lines not on this page · raw message

arch/arm/boot/compressed/piggy.bz2.S +6 -0
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.bz2.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.bz2.S
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.bz2"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.gz.S +6 -0
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.gz.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.gz.S
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.gz"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.lzma.S +6 -0
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.lzma.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.lzma.S
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.lzma"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.S +0 -6
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.S
@@ -1,6 +0,0 @@
-	.section .piggydata,#alloc
-	.globl	input_data
-input_data:
-	.incbin	"arch/arm/boot/compressed/piggy.gz"
-	.globl	input_data_end
-input_data_end:
arch/arm/Kconfig +3 -0
diff -purN linux-2.6.30/arch/arm/Kconfig linux-2.6.30.arm/arch/arm/Kconfig
@@ -18,6 +18,9 @@ config ARM
 	select HAVE_KRETPROBES if (HAVE_KPROBES)
 	select HAVE_FUNCTION_TRACER if (!XIP_KERNEL)
 	select HAVE_GENERIC_DMA_COHERENT
+	select HAVE_KERNEL_GZIP
+	select HAVE_KERNEL_BZIP2
+	select HAVE_KERNEL_LZMA
 	help
 	  The ARM series is a line of low-power-consumption RISC chip designs
 	  licensed by ARM Ltd and targeted at embedded applications and
arch/arm/Kconfig~ +1436 -0

1,436 lines not on this page · raw message

include/linux/zutil.h +18 -0
diff -purN linux-2.6.30/include/linux/zutil.h linux-2.6.30.arm/include/linux/zutil.h
@@ -60,6 +60,19 @@ typedef uLong (*check_func) (uLong check
 #define DO8(buf,i)  DO4(buf,i); DO4(buf,i+4);
 #define DO16(buf)   DO8(buf,0); DO8(buf,8);
 
+#ifdef NO_DIVISION
+static inline unsigned long mod65521(unsigned long s)
+{
+	while (1) {
+		if (s < BASE)
+			return s;
+		if (s < 0x10000)
+			return s-BASE;
+		s = (s & 0xffff) + 15 * (s >> 16);
+	}
+}
+#endif
+
 /* ========================================================================= */
 /*
      Update a running Adler-32 checksum with the bytes buf[0..len-1] and
@@ -97,8 +110,13 @@ static inline uLong zlib_adler32(uLong a
             s1 += *buf++;
 	    s2 += s1;
         } while (--k);
+#ifdef NO_DIVISION
+	s1 = mod65521(s1);
+	s2 = mod65521(s2);
+#else
         s1 %= BASE;
         s2 %= BASE;
+#endif
     }
     return (s2 << 16) | s1;
 }
lib/zlib_inflate/inflate.c +23 -1
diff -purN linux-2.6.30/lib/zlib_inflate/inflate.c linux-2.6.30.arm/lib/zlib_inflate/inflate.c
@@ -247,6 +247,23 @@ static int zlib_inflateSyncPacket(z_stre
     ((((q) >> 24) & 0xff) + (((q) >> 8) & 0xff00) + \
      (((q) & 0xff00) << 8) + (((q) & 0xff) << 24))
 
+#ifdef NO_DIVISION
+/*
+  modulo 15 routine for those architectures that don't have division
+*/
+static inline unsigned long mod31(unsigned long s)
+{
+	while (1) {
+		if (s < 31)
+			return s;
+		if (s < 32)
+			return 0;
+		s = (s >> 5) + (s & 31);
+	}
+}
+#endif
+
+
 /*
    inflate() uses a state machine to process as much input data and generate as
    much output data as possible before returning.  The state machine is
@@ -370,7 +387,12 @@ int zlib_inflate(z_streamp strm, int flu
             }
             NEEDBITS(16);
             if (
-                ((BITS(8) << 8) + (hold >> 8)) % 31) {
+#ifdef NO_DIVISION
+		mod31((BITS(8) << 8) + (hold >> 8))
+#else
+		((BITS(8) << 8) + (hold >> 8)) % 31
+#endif
+		) {
                 strm->msg = (char *)"incorrect header check";
                 state->mode = BAD;
                 break;
#6 lore raw
Quoted 10 lines · Alain Knaff

Attached is a version of the ARM patch for 2.6.30

There seems to be some issues with the include files (which did not pose any problems in 2.6.28...), but hopefully somebody more experienced with the ARM architecture (and with its recent development) than I can help out there.

Regards,

Alain

Actually, the needed changes turned out to be easier than I initially feared. Attached a version that compiles all right.

Regards,

Alain

arch/arm/boot/compressed/Makefile +21 -4
diff -purN linux-2.6.30/arch/arm/boot/compressed/Makefile linux-2.6.30.arm/arch/arm/boot/compressed/Makefile
@@ -63,8 +63,15 @@ endif
 
 SEDFLAGS	= s/TEXT_START/$(ZTEXTADDR)/;s/BSS_START/$(ZBSSADDR)/
 
-targets       := vmlinux vmlinux.lds piggy.gz piggy.o font.o font.c \
-		 head.o misc.o $(OBJS)
+suffix_$(CONFIG_KERNEL_GZIP)  = gz
+suffix_$(CONFIG_KERNEL_BZIP2) = bz2
+suffix_$(CONFIG_KERNEL_LZMA)  = lzma
+
+targets       := vmlinux vmlinux.lds \
+		 piggy.gz piggy.gz.o \
+		 piggy.bz2 piggy.bz2.o \
+		 piggy.lzma piggy.lzma.o \
+		 font.o font.c head.o misc.o $(OBJS)
 
 ifeq ($(CONFIG_FUNCTION_TRACER),y)
 ORIG_CFLAGS := $(KBUILD_CFLAGS)
@@ -91,7 +98,7 @@ LDFLAGS_vmlinux += -p --no-undefined -X 
 # would otherwise mess up our GOT table
 CFLAGS_misc.o := -Dstatic=
 
-$(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.o \
+$(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.$(suffix_y).o \
 	 	$(addprefix $(obj)/, $(OBJS)) FORCE
 	$(call if_changed,ld)
 	@:
@@ -99,7 +106,17 @@ $(obj)/vmlinux: $(obj)/vmlinux.lds $(obj
 $(obj)/piggy.gz: $(obj)/../Image FORCE
 	$(call if_changed,gzip)
 
-$(obj)/piggy.o:  $(obj)/piggy.gz FORCE
+$(obj)/piggy.bz2: $(obj)/../Image FORCE
+	$(call if_changed,bzip2)
+
+$(obj)/piggy.lzma: $(obj)/../Image FORCE
+	$(call if_changed,lzma)
+
+$(obj)/piggy.gz.o:  $(obj)/piggy.gz FORCE
+
+$(obj)/piggy.bz2.o:  $(obj)/piggy.bz2 FORCE
+
+$(obj)/piggy.lzma.o:  $(obj)/piggy.lzma FORCE
 
 CFLAGS_font.o := -Dstatic=
 
arch/arm/boot/compressed/misc.c +36 -108

189 lines not on this page · raw message

arch/arm/boot/compressed/piggy.bz2.S +6 -0
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.bz2.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.bz2.S
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.bz2"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.gz.S +6 -0
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.gz.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.gz.S
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.gz"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.lzma.S +6 -0
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.lzma.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.lzma.S
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.lzma"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.S +0 -6
diff -purN linux-2.6.30/arch/arm/boot/compressed/piggy.S linux-2.6.30.arm/arch/arm/boot/compressed/piggy.S
@@ -1,6 +0,0 @@
-	.section .piggydata,#alloc
-	.globl	input_data
-input_data:
-	.incbin	"arch/arm/boot/compressed/piggy.gz"
-	.globl	input_data_end
-input_data_end:
arch/arm/Kconfig +3 -0
diff -purN linux-2.6.30/arch/arm/Kconfig linux-2.6.30.arm/arch/arm/Kconfig
@@ -18,6 +18,9 @@ config ARM
 	select HAVE_KRETPROBES if (HAVE_KPROBES)
 	select HAVE_FUNCTION_TRACER if (!XIP_KERNEL)
 	select HAVE_GENERIC_DMA_COHERENT
+	select HAVE_KERNEL_GZIP
+	select HAVE_KERNEL_BZIP2
+	select HAVE_KERNEL_LZMA
 	help
 	  The ARM series is a line of low-power-consumption RISC chip designs
 	  licensed by ARM Ltd and targeted at embedded applications and
include/linux/decompress/mm.h +2 -0
diff -purN linux-2.6.30/include/linux/decompress/mm.h linux-2.6.30.arm/include/linux/decompress/mm.h
@@ -64,6 +64,8 @@ static void free(void *where)
 #include <linux/string.h>
 #include <linux/vmalloc.h>
 
+#include <linux/slab.h>
+
 /* Use defines rather than static inline in order to avoid spurious
  * warnings when not needed (indeed large_malloc / large_free are not
  * needed by inflate */
include/linux/zutil.h +18 -0
diff -purN linux-2.6.30/include/linux/zutil.h linux-2.6.30.arm/include/linux/zutil.h
@@ -60,6 +60,19 @@ typedef uLong (*check_func) (uLong check
 #define DO8(buf,i)  DO4(buf,i); DO4(buf,i+4);
 #define DO16(buf)   DO8(buf,0); DO8(buf,8);
 
+#ifdef NO_DIVISION
+static inline unsigned long mod65521(unsigned long s)
+{
+	while (1) {
+		if (s < BASE)
+			return s;
+		if (s < 0x10000)
+			return s-BASE;
+		s = (s & 0xffff) + 15 * (s >> 16);
+	}
+}
+#endif
+
 /* ========================================================================= */
 /*
      Update a running Adler-32 checksum with the bytes buf[0..len-1] and
@@ -97,8 +110,13 @@ static inline uLong zlib_adler32(uLong a
             s1 += *buf++;
 	    s2 += s1;
         } while (--k);
+#ifdef NO_DIVISION
+	s1 = mod65521(s1);
+	s2 = mod65521(s2);
+#else
         s1 %= BASE;
         s2 %= BASE;
+#endif
     }
     return (s2 << 16) | s1;
 }
lib/decompress_bunzip2.c +0 -1
diff -purN linux-2.6.30/lib/decompress_bunzip2.c linux-2.6.30.arm/lib/decompress_bunzip2.c
@@ -50,7 +50,6 @@
 #endif /* !STATIC */
 
 #include <linux/decompress/mm.h>
-#include <linux/slab.h>
 
 #ifndef INT_MAX
 #define INT_MAX 0x7fffffff
lib/decompress_inflate.c +0 -1
diff -purN linux-2.6.30/lib/decompress_inflate.c linux-2.6.30.arm/lib/decompress_inflate.c
@@ -23,7 +23,6 @@
 #endif /* STATIC */
 
 #include <linux/decompress/mm.h>
-#include <linux/slab.h>
 
 #define INBUF_LEN (16*1024)
 
lib/decompress_unlzma.c +0 -1
diff -purN linux-2.6.30/lib/decompress_unlzma.c linux-2.6.30.arm/lib/decompress_unlzma.c
@@ -34,7 +34,6 @@
 #endif /* STATIC */
 
 #include <linux/decompress/mm.h>
-#include <linux/slab.h>
 
 #define	MIN(a, b) (((a) < (b)) ? (a) : (b))
 
lib/zlib_inflate/inflate.c +23 -1
diff -purN linux-2.6.30/lib/zlib_inflate/inflate.c linux-2.6.30.arm/lib/zlib_inflate/inflate.c
@@ -247,6 +247,23 @@ static int zlib_inflateSyncPacket(z_stre
     ((((q) >> 24) & 0xff) + (((q) >> 8) & 0xff00) + \
      (((q) & 0xff00) << 8) + (((q) & 0xff) << 24))
 
+#ifdef NO_DIVISION
+/*
+  modulo 15 routine for those architectures that don't have division
+*/
+static inline unsigned long mod31(unsigned long s)
+{
+	while (1) {
+		if (s < 31)
+			return s;
+		if (s < 32)
+			return 0;
+		s = (s >> 5) + (s & 31);
+	}
+}
+#endif
+
+
 /*
    inflate() uses a state machine to process as much input data and generate as
    much output data as possible before returning.  The state machine is
@@ -370,7 +387,12 @@ int zlib_inflate(z_streamp strm, int flu
             }
             NEEDBITS(16);
             if (
-                ((BITS(8) << 8) + (hold >> 8)) % 31) {
+#ifdef NO_DIVISION
+		mod31((BITS(8) << 8) + (hold >> 8))
+#else
+		((BITS(8) << 8) + (hold >> 8)) % 31
+#endif
+		) {
                 strm->msg = (char *)"incorrect header check";
                 state->mode = BAD;
                 break;
#7 lore raw
Quoted 8 lines · Alain Knaff
Quoted 7 lines · Alain Knaff

Attached is a version of the ARM patch for 2.6.30

There seems to be some issues with the include files (which did not pose any problems in 2.6.28...), but hopefully somebody more experienced with the ARM architecture (and with its recent development) than I can help out there.

Quoted 6 lines
Quoted 3 lines

Regards,

Alain

Actually, the needed changes turned out to be easier than I initially feared. Attached a version that compiles all right.

I gave it a quick test and it hangs before "Decompressing kernel" is output. I hope I'll have time this week to debug what's wrong there.

Quoted 4 lines

Regards,

Alain

Signature · 2 lines
Sincerely yours,
Mike.
#8 lore raw
Quoted 13 lines · Mike Rapoport
Quoted 11 lines · Alain Knaff
Quoted 9 lines · Alain Knaff

Attached is a version of the ARM patch for 2.6.30

There seems to be some issues with the include files (which did not pose any problems in 2.6.28...), but hopefully somebody more experienced with the ARM architecture (and with its recent development) than I can help out there.

Regards,

Quoted 9 lines
Quoted 5 lines
Quoted 2 lines

Alain

Actually, the needed changes turned out to be easier than I initially feared. Attached a version that compiles all right.

I gave it a quick test and it hangs before "Decompressing kernel" is output. I hope I'll have time this week to debug what's wrong there.

Same here. I compiled 3 kernels with each option with no problem, but none goes beyond the "Uncompressing Linux.." message. I am using a CALAO USB-A9263 board with the AT91SAM9263 chip.

This means that even zlib compression is broken with this patch.

I hope I can work on this too.

Thanks anyway!

Cheers,

Michael.

Signature · 4 lines
Michael Opdenacker, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com
#9 lore raw
In reply to Mike Rapoport
Quoted 13 lines · Mike Rapoport
Quoted 10 lines · Alain Knaff
Quoted 9 lines · Alain Knaff

Attached is a version of the ARM patch for 2.6.30

There seems to be some issues with the include files (which did not pose any problems in 2.6.28...), but hopefully somebody more experienced with the ARM architecture (and with its recent development) than I can help out there.

Regards,

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Alain

Actually, the needed changes turned out to be easier than I initially feared. Attached a version that compiles all right.

I gave it a quick test and it hangs before "Decompressing kernel" is output. I hope I'll have time this week to debug what's wrong there.

On my testing board, it hangs after "Decompressing kernel" is output. It seems that it's getting stuck in the following piece of code in lib/decompressor_lzma.c

while (get_pos(&wr) < header.dst_size) {
                int pos_state = get_pos(&wr) & pos_state_mask;
                uint16_t *prob = p + LZMA_IS_MATCH +
                        (cst.state << LZMA_NUM_POS_BITS_MAX) + pos_state;
                if (rc_is_bit_0(&rc, prob))
                        process_bit0(&wr, &rc, &cst, p, pos_state, prob,
                                     lc, literal_pos_mask);
                else {
                        process_bit1(&wr, &rc, &cst, p, pos_state, prob);
                        if (cst.rep0 == 0)
                                break;
                }
}

Based on some quick tests, it seems that header.dst_size is 0xFFFFFFFFFFFFFFFF, and if my understanding is correct, it's because LZMA doesn't write the file size in the header if it reads its input from a pipe, and piggy.lzma turns out to be created using a pipe.

Replacing cmd_lzma in scripts/Makefile.lib by the following snippet:

cmd_lzma = (lzma -9 -c $(filter-out FORCE, $^) > $@ && $(call size_append, $(filter-out FORCE,$^)) >> $@ ) || (rm -f $@ ; false)

does the trick, as lzma is able to write the file size in the header (and I think the second part is useless, as it's the filesize encoded in 32bits little-endian). However, it now hangs right after the 'done, booting the kernel.'

Hope that helps,

Signature · 9 lines
Albin Tonnerre, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com

-------------------------------------------------------------------
List admin: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ:        http://www.arm.linux.org.uk/mailinglists/faq.php
Etiquette:  http://www.arm.linux.org.uk/mailinglists/etiquette.php
#10 lore raw
Quoted 3 lines · Albin Tonnerre

However, it now hangs right after the 'done, booting the kernel.'

After some further work, here's an updated patch that works properly on ARM, at least for gzip and lzma (I'm getting an 'out of memory' error with bzip2).

As a side note, my previous remarks about the LZMA header not being created properly only apply to the git tree, as this is caused by commit d3dd3b5a29bb9582957451531fed461628dfc834. 2.6.30 and 2.6.30.1 don't have this issue

Regards,
Albin

arch/arm/Kconfig +3 -0
index 9d02cdb..5cde822 100644
@@ -18,6 +18,9 @@ config ARM
 	select HAVE_KRETPROBES if (HAVE_KPROBES)
 	select HAVE_FUNCTION_TRACER if (!XIP_KERNEL)
 	select HAVE_GENERIC_DMA_COHERENT
+	select HAVE_KERNEL_GZIP
+	select HAVE_KERNEL_BZIP2
+	select HAVE_KERNEL_LZMA
 	help
 	  The ARM series is a line of low-power-consumption RISC chip designs
 	  licensed by ARM Ltd and targeted at embedded applications and
arch/arm/boot/compressed/Makefile +21 -4
index fbe5eef..c29a88f 100644
@@ -63,8 +63,15 @@ endif
 
 SEDFLAGS	= s/TEXT_START/$(ZTEXTADDR)/;s/BSS_START/$(ZBSSADDR)/
 
-targets       := vmlinux vmlinux.lds piggy.gz piggy.o font.o font.c \
-		 head.o misc.o $(OBJS)
+suffix_$(CONFIG_KERNEL_GZIP)  = gz
+suffix_$(CONFIG_KERNEL_BZIP2) = bz2
+suffix_$(CONFIG_KERNEL_LZMA)  = lzma
+
+targets       := vmlinux vmlinux.lds \
+		 piggy.gz piggy.gz.o \
+		 piggy.bz2 piggy.bz2.o \
+		 piggy.lzma piggy.lzma.o \
+		 font.o font.c head.o misc.o $(OBJS)
 
 ifeq ($(CONFIG_FUNCTION_TRACER),y)
 ORIG_CFLAGS := $(KBUILD_CFLAGS)
@@ -91,7 +98,7 @@ LDFLAGS_vmlinux += -p --no-undefined -X \
 # would otherwise mess up our GOT table
 CFLAGS_misc.o := -Dstatic=
 
-$(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.o \
+$(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.$(suffix_y).o \
 	 	$(addprefix $(obj)/, $(OBJS)) FORCE
 	$(call if_changed,ld)
 	@:
@@ -99,7 +106,17 @@ $(obj)/vmlinux: $(obj)/vmlinux.lds $(obj)/$(HEAD) $(obj)/piggy.o \
 $(obj)/piggy.gz: $(obj)/../Image FORCE
 	$(call if_changed,gzip)
 
-$(obj)/piggy.o:  $(obj)/piggy.gz FORCE
+$(obj)/piggy.bz2: $(obj)/../Image FORCE
+	$(call if_changed,bzip2)
+
+$(obj)/piggy.lzma: $(obj)/../Image FORCE
+	$(call if_changed,lzma)
+
+$(obj)/piggy.gz.o:  $(obj)/piggy.gz FORCE
+
+$(obj)/piggy.bz2.o:  $(obj)/piggy.bz2 FORCE
+
+$(obj)/piggy.lzma.o:  $(obj)/piggy.lzma FORCE
 
 CFLAGS_font.o := -Dstatic=
 
arch/arm/boot/compressed/misc.c +43 -108

196 lines not on this page · raw message

arch/arm/boot/compressed/piggy.S deleted +0 -6
deleted file mode 100644
index 54c9518..0000000
@@ -1,6 +0,0 @@
-	.section .piggydata,#alloc
-	.globl	input_data
-input_data:
-	.incbin	"arch/arm/boot/compressed/piggy.gz"
-	.globl	input_data_end
-input_data_end:
arch/arm/boot/compressed/piggy.bz2.S new file +6 -0
new file mode 100644
index 0000000..c5ecbde
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.bz2"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.gz.S new file +6 -0
new file mode 100644
index 0000000..54c9518
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.gz"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/compressed/piggy.lzma.S new file +6 -0
new file mode 100644
index 0000000..d7e69cf
@@ -0,0 +1,6 @@
+	.section .piggydata,#alloc
+	.globl	input_data
+input_data:
+	.incbin	"arch/arm/boot/compressed/piggy.lzma"
+	.globl	input_data_end
+input_data_end:
arch/arm/boot/uImage.lzma new file binary +0 -0
new file mode 100644
index 0000000..4fb24eb
Binary files /dev/null and b/arch/arm/boot/uImage.lzma differ
include/linux/decompress/mm.h +2 -0
index 12ff8c3..8becfce 100644
@@ -64,6 +64,8 @@ static void free(void *where)
 #include <linux/string.h>
 #include <linux/vmalloc.h>
 
+#include <linux/slab.h>
+
 /* Use defines rather than static inline in order to avoid spurious
  * warnings when not needed (indeed large_malloc / large_free are not
  * needed by inflate */
include/linux/zutil.h +18 -0
index 6adfa9a..bedd56b 100644
@@ -60,6 +60,19 @@ typedef uLong (*check_func) (uLong check, const Byte *buf,
 #define DO8(buf,i)  DO4(buf,i); DO4(buf,i+4);
 #define DO16(buf)   DO8(buf,0); DO8(buf,8);
 
+#ifdef NO_DIVISION
+static inline unsigned long mod65521(unsigned long s)
+{
+	while (1) {
+		if (s < BASE)
+			return s;
+		if (s < 0x10000)
+			return s-BASE;
+		s = (s & 0xffff) + 15 * (s >> 16);
+	}
+}
+#endif
+
 /* ========================================================================= */
 /*
      Update a running Adler-32 checksum with the bytes buf[0..len-1] and
@@ -97,8 +110,13 @@ static inline uLong zlib_adler32(uLong adler,
             s1 += *buf++;
 	    s2 += s1;
         } while (--k);
+#ifdef NO_DIVISION
+	s1 = mod65521(s1);
+	s2 = mod65521(s2);
+#else
         s1 %= BASE;
         s2 %= BASE;
+#endif
     }
     return (s2 << 16) | s1;
 }
lib/decompress_bunzip2.c +0 -1
index 708e2a8..5d3ddb5 100644
@@ -50,7 +50,6 @@
 #endif /* !STATIC */
 
 #include <linux/decompress/mm.h>
-#include <linux/slab.h>
 
 #ifndef INT_MAX
 #define INT_MAX 0x7fffffff
lib/decompress_inflate.c +0 -1
index e36b296..839a329 100644
@@ -23,7 +23,6 @@
 #endif /* STATIC */
 
 #include <linux/decompress/mm.h>
-#include <linux/slab.h>
 
 #define INBUF_LEN (16*1024)
 
lib/decompress_unlzma.c +0 -1
index 32123a1..546f2f4 100644
@@ -34,7 +34,6 @@
 #endif /* STATIC */
 
 #include <linux/decompress/mm.h>
-#include <linux/slab.h>
 
 #define	MIN(a, b) (((a) < (b)) ? (a) : (b))
 
lib/zlib_inflate/inflate.c +23 -1
index f5ce87b..566bf1e 100644
@@ -247,6 +247,23 @@ static int zlib_inflateSyncPacket(z_streamp strm)
     ((((q) >> 24) & 0xff) + (((q) >> 8) & 0xff00) + \
      (((q) & 0xff00) << 8) + (((q) & 0xff) << 24))
 
+#ifdef NO_DIVISION
+/*
+  modulo 15 routine for those architectures that don't have division
+*/
+static inline unsigned long mod31(unsigned long s)
+{
+	while (1) {
+		if (s < 31)
+			return s;
+		if (s < 32)
+			return 0;
+		s = (s >> 5) + (s & 31);
+	}
+}
+#endif
+
+
 /*
    inflate() uses a state machine to process as much input data and generate as
    much output data as possible before returning.  The state machine is
@@ -370,7 +387,12 @@ int zlib_inflate(z_streamp strm, int flush)
             }
             NEEDBITS(16);
             if (
-                ((BITS(8) << 8) + (hold >> 8)) % 31) {
+#ifdef NO_DIVISION
+		mod31((BITS(8) << 8) + (hold >> 8))
+#else
+		((BITS(8) << 8) + (hold >> 8)) % 31
+#endif
+		) {
                 strm->msg = (char *)"incorrect header check";
                 state->mode = BAD;
                 break;
Signature · 4 lines
Albin Tonnerre, Free Electrons
Kernel, drivers and embedded Linux development,
consulting, training and support.
http://free-electrons.com