bzip2/lzma kernel compression
Subsystems (4)
H. Peter Anvin
reviewerHi Linus,
This tree contains the bzip2 and LZMA kernel compression work that Alain Knaff has done. Sending this as a separate pull request in case you think that it is too late or immature for this cycle.
The good part is that it is a highly "brittle" feature -- if it fails, it will fail noisily and obviously.
I have not included the ARM parts that Alain developed; I will leave those to be fed through rmk.
One of the main attractions of this patchset is that it uses the newer lib/zlib_inflate code even for the kernel decompressor. Once all architectures that use the older lib/inflate.c have been converted over, we can remove that code entirely.
-hpa
The following changes since commit c861ea2cb2c25c1698734d9b0540a09e253690a1:
Linus Torvalds (1):
Merge branch 'for-linus' of git://git.kernel.org/.../jmorris/security-testing-2.6
are available in the git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip.git x86/setup-lzma
Alain Knaff (5):
bzip2/lzma: library support for gzip, bzip2 and lzma decompression bzip2/lzma: config and initramfs support for bzip2/lzma decompression bzip2/lzma: x86 kernel compression support
bzip2/lzma: fix built-in initramfs vs CONFIG_RD_GZIP bzip2/lzma: fix decompress_inflate.c vs multi-block-with-embedded-filename
H. Peter Anvin (8):
bzip2/lzma: use a table to search for initramfs compression formats bzip2/lzma: handle failures from bzip2 and lzma correctly bzip2/lzma: make config machinery an arch configurable bzip2/lzma: proper Kconfig dependencies for the ramdisk options bzip2/lzma: DECOMPRESS_GZIP should select ZLIB_INFLATE bzip2/lzma: move initrd/ramfs options out of BLK_DEV bzip2/lzma: fix constant in decompress_inflate bzip2/lzma: centralize format detection
arch/x86/Kconfig | 3 + arch/x86/boot/compressed/Makefile | 21 +- arch/x86/boot/compressed/misc.c | 118 +----- arch/x86/include/asm/boot.h | 12 +- include/linux/decompress/bunzip2.h | 10 + include/linux/decompress/generic.h | 33 ++ include/linux/decompress/inflate.h | 13 + include/linux/decompress/mm.h | 87 +++++ include/linux/decompress/unlzma.h | 12 + init/Kconfig | 60 +++ init/do_mounts_rd.c | 174 +++------- init/initramfs.c | 101 +---- lib/Kconfig | 14 + lib/Makefile | 7 +- lib/decompress.c | 50 +++ lib/decompress_bunzip2.c | 735 ++++++++++++++++++++++++++++++++++++ lib/decompress_inflate.c | 167 ++++++++ lib/decompress_unlzma.c | 647 +++++++++++++++++++++++++++++++ lib/zlib_inflate/inflate.h | 4 + lib/zlib_inflate/inftrees.h | 4 + scripts/Makefile.lib | 14 + scripts/bin_size | 10 + scripts/gen_initramfs_list.sh | 17 +- usr/Kconfig | 27 ++ usr/Makefile | 42 ++- usr/initramfs_data.S | 2 +- usr/initramfs_data.bz2.S | 29 ++ usr/initramfs_data.gz.S | 29 ++ usr/initramfs_data.lzma.S | 29 ++ 29 files changed, 2133 insertions(+), 338 deletions(-) create mode 100644 include/linux/decompress/bunzip2.h create mode 100644 include/linux/decompress/generic.h create mode 100644 include/linux/decompress/inflate.h create mode 100644 include/linux/decompress/mm.h create mode 100644 include/linux/decompress/unlzma.h create mode 100644 lib/decompress.c create mode 100644 lib/decompress_bunzip2.c create mode 100644 lib/decompress_inflate.c create mode 100644 lib/decompress_unlzma.c create mode 100644 scripts/bin_size create mode 100644 usr/initramfs_data.bz2.S create mode 100644 usr/initramfs_data.gz.S create mode 100644 usr/initramfs_data.lzma.S
arch/x86/Kconfig +3 -0
@@ -39,6 +39,9 @@ config X86 select HAVE_GENERIC_DMA_COHERENT if X86_32 select HAVE_EFFICIENT_UNALIGNED_ACCESS select USER_STACKTRACE_SUPPORT + select HAVE_KERNEL_GZIP + select HAVE_KERNEL_BZIP2 + select HAVE_KERNEL_LZMA config ARCH_DEFCONFIG string
arch/x86/boot/compressed/Makefile +19 -2
@@ -4,7 +4,7 @@ # create a compressed vmlinux image from the original vmlinux # -targets := vmlinux vmlinux.bin vmlinux.bin.gz head_$(BITS).o misc.o piggy.o +targets := vmlinux vmlinux.bin vmlinux.bin.gz vmlinux.bin.bz2 vmlinux.bin.lzma head_$(BITS).o misc.o piggy.o KBUILD_CFLAGS := -m$(BITS) -D__KERNEL__ $(LINUX_INCLUDE) -O2 KBUILD_CFLAGS += -fno-strict-aliasing -fPIC
@@ -47,18 +47,35 @@ ifeq ($(CONFIG_X86_32),y) ifdef CONFIG_RELOCATABLE $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin.all FORCE $(call if_changed,gzip) +$(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin.all FORCE + $(call if_changed,bzip2) +$(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin.all FORCE + $(call if_changed,lzma) else $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE $(call if_changed,gzip) +$(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE + $(call if_changed,bzip2) +$(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE + $(call if_changed,lzma) endif LDFLAGS_piggy.o := -r --format binary --oformat elf32-i386 -T else + $(obj)/vmlinux.bin.gz: $(obj)/vmlinux.bin FORCE $(call if_changed,gzip) +$(obj)/vmlinux.bin.bz2: $(obj)/vmlinux.bin FORCE + $(call if_changed,bzip2) +$(obj)/vmlinux.bin.lzma: $(obj)/vmlinux.bin FORCE + $(call if_changed,lzma) LDFLAGS_piggy.o := -r --format binary --oformat elf64-x86-64 -T endif -$(obj)/piggy.o: $(obj)/vmlinux.scr $(obj)/vmlinux.bin.gz FORCE +suffix_$(CONFIG_KERNEL_GZIP) = gz +suffix_$(CONFIG_KERNEL_BZIP2) = bz2 +suffix_$(CONFIG_KERNEL_LZMA) = lzma + +$(obj)/piggy.o: $(obj)/vmlinux.scr $(obj)/vmlinux.bin.$(suffix_y) FORCE $(call if_changed,ld)
arch/x86/boot/compressed/misc.c +14 -104
172 lines not on this page · raw message
arch/x86/include/asm/boot.h +11 -1
@@ -15,11 +15,21 @@ + (CONFIG_PHYSICAL_ALIGN - 1)) \ & ~(CONFIG_PHYSICAL_ALIGN - 1)) +#if (definedCONFIG_KERNEL_BZIP2 ) +#define BOOT_HEAP_SIZE 0x400000 +#else +#ifdef CONFIG_X86_64 #define BOOT_HEAP_SIZE 0x7000 -#define BOOT_STACK_SIZE 0x4000#else #define BOOT_HEAP_SIZE 0x4000 +#endif + +#endif + +#ifdef CONFIG_X86_64 +#define BOOT_STACK_SIZE 0x4000 +#else #define BOOT_STACK_SIZE 0x1000#endif
include/linux/decompress/bunzip2.h new file +10 -0
@@ -0,0 +1,10 @@ +#ifndef DECOMPRESS_BUNZIP2_H +#define DECOMPRESS_BUNZIP2_H + +int bunzip2 (unsigned char *inbuf,int len, +int (*fill)(void *,unsigned int ), +int (*flush)(void *,unsigned int ), +unsigned char *output, +int *pos, +void (*error)(char *x)); +#endif
include/linux/decompress/generic.h new file +33 -0
@@ -0,0 +1,33 @@ +#ifndef DECOMPRESS_GENERIC_H +#define DECOMPRESS_GENERIC_H + +/* Minimal chunksize to be read. +*Bzip2 prefers at least 4096 +*Lzma prefers 0x10000 */ +#define COMPR_IOBUF_SIZE 4096 + +typedef int (*decompress_fn ) (unsigned char *inbuf,int len, +int (*fill)(void *,unsigned int ), +int (*writebb)(void *,unsigned int ), +unsigned char *output, +int *posp, +void (*error)(char *x)); + +/* inbuf - input buffer +*len - len of pre-read data in inbuf +*fill - function to fill inbuf if empty +*writebb - function to write out outbug +*posp - if non-null, input position (number of bytes read) will be +* returned here +* +*If len != 0, the inbuf is initialized (with as much data), and fill +*should not be called +*If len = 0, the inbuf is allocated, but empty. Its size is IOBUF_SIZE +*fill should be called (repeatedly...) to read data, at most IOBUF_SIZE +*/ + +/* Utility routine to detect the decompression method */ +decompress_fn decompress_method (const unsigned char *inbuf,int len, +const char **name); + +#endif
include/linux/decompress/inflate.h new file +13 -0
@@ -0,0 +1,13 @@ +#ifndef INFLATE_H +#define INFLATE_H + +/* Other housekeeping constants */ +#define INBUFSIZ 4096 + +int gunzip (unsigned char *inbuf,int len, +int (*fill)(void *,unsigned int ), +int (*flush)(void *,unsigned int ), +unsigned char *output, +int *pos, +void (*error_fn)(char *x)); +#endif
include/linux/decompress/mm.h new file +87 -0
87 lines not on this page · raw message
include/linux/decompress/unlzma.h new file +12 -0
@@ -0,0 +1,12 @@ +#ifndef DECOMPRESS_UNLZMA_H +#define DECOMPRESS_UNLZMA_H + +int unlzma (unsigned char *,int , +int (*fill)(void *,unsigned int ), +int (*flush)(void *,unsigned int ), +unsigned char *output, +int *posp, +void (*error)(char *x) + ); + +#endif
init/Kconfig +60 -0
66 lines not on this page · raw message
init/do_mounts_rd.c +43 -131
264 lines not on this page · raw message
init/initramfs.c +21 -80
138 lines not on this page · raw message
lib/Kconfig +14 -0
@@ -98,6 +98,20 @@ config LZO_DECOMPRESS tristate # +# These all provide a common interface (hence the apparent duplication with +# ZLIB_INFLATE; DECOMPRESS_GZIP is just a wrapper.) +# +config DECOMPRESS_GZIP + select ZLIB_INFLATE + tristate + +config DECOMPRESS_BZIP2 + tristate + +config DECOMPRESS_LZMA + tristate + +# # Generic allocator support is selected if needed # config GENERIC_ALLOCATOR
lib/Makefile +6 -1
@@ -11,7 +11,8 @@ lib-y := ctype.o string.o vsprintf.o cmdline.o \ rbtree.o radix-tree.o dump_stack.o \ idr.o int_sqrt.o extable.o prio_tree.o \ sha1.o irq_regs.o reciprocal_div.o argv_split.o \ - proportions.o prio_heap.o ratelimit.o show_mem.o is_single_threaded.o + proportions.o prio_heap.o ratelimit.o show_mem.o \ + is_single_threaded.o decompress.o lib-$(CONFIG_MMU) += ioremap.o lib-$(CONFIG_SMP) += cpumask.o
@@ -65,6 +66,10 @@ obj-$(CONFIG_REED_SOLOMON) += reed_solomon/ obj-$(CONFIG_LZO_COMPRESS) += lzo/ obj-$(CONFIG_LZO_DECOMPRESS) += lzo/ +lib-$(CONFIG_DECOMPRESS_GZIP) += decompress_inflate.o +lib-$(CONFIG_DECOMPRESS_BZIP2) += decompress_bunzip2.o +lib-$(CONFIG_DECOMPRESS_LZMA) += decompress_unlzma.o + obj-$(CONFIG_TEXTSEARCH) += textsearch.o obj-$(CONFIG_TEXTSEARCH_KMP) += ts_kmp.o obj-$(CONFIG_TEXTSEARCH_BM) += ts_bm.o
lib/decompress.c new file +50 -0
50 lines not on this page · raw message
lib/decompress_bunzip2.c new file +735 -0
735 lines not on this page · raw message
lib/decompress_inflate.c new file +167 -0
167 lines not on this page · raw message
lib/decompress_unlzma.c new file +647 -0
647 lines not on this page · raw message
lib/zlib_inflate/inflate.h +4 -0
@@ -1,3 +1,6 @@ +#ifndef INFLATE_H +#define INFLATE_H +/* inflate.h -- internal inflate state definition * Copyright (C) 1995-2004 Mark Adler * For conditions of distribution and use, see copyright notice in zlib.h
@@ -105,3 +108,4 @@ struct inflate_state {unsigned short work[288 ];/* work area for code table building */ code codes[ENOUGH]; /* space for code tables */ }; +#endif
lib/zlib_inflate/inftrees.h +4 -0
@@ -1,3 +1,6 @@ +#ifndef INFTREES_H +#define INFTREES_H +/* inftrees.h -- header to use inftrees.c * Copyright (C) 1995-2005 Mark Adler * For conditions of distribution and use, see copyright notice in zlib.h
@@ -53,3 +56,4 @@ typedef enum {extern int zlib_inflate_table (codetype type,unsigned short *lens,unsigned codes,code **table,unsigned *bits,unsigned short *work); +#endif
scripts/Makefile.lib +14 -0
@@ -186,3 +186,17 @@ quiet_cmd_gzip = GZIP $@ cmd_gzip = gzip -f -9 < $< > $@ +# Bzip2 +# --------------------------------------------------------------------------- + +# Bzip2 does not include size in file... so we have to fake that +size_append=$(CONFIG_SHELL) $(srctree)/scripts/bin_size + +quiet_cmd_bzip2 = BZIP2 $@ +cmd_bzip2 = (bzip2 -9 < $< && $(size_append) $<) > $@ || (rm -f $@ ; false) + +# Lzma +# --------------------------------------------------------------------------- + +quiet_cmd_lzma = LZMA $@ +cmd_lzma = (lzma -9 -c $< && $(size_append) $<) >$@ || (rm -f $@ ; false)
scripts/bin_size new file +10 -0
@@ -0,0 +1,10 @@ +#!/bin/sh + +if [ $# = 0 ] ; then + echo Usage: $0 file +fi + +size_dec=`stat -c "%s" $1` +size_hex_echo_string=`printf "%08x" $size_dec | + sed 's/\(..\)\(..\)\(..\)\(..\)/\\\\x\4\\\\x\3\\\\x\2\\\\x\1/g'` +/bin/echo -ne $size_hex_echo_string
scripts/gen_initramfs_list.sh +11 -6
@@ -5,7 +5,7 @@ # Released under the terms of the GNU GPL # # Generate a cpio packed initramfs. It uses gen_init_cpio to generate -# the cpio archive, and gzip to pack it. +# the cpio archive, and then compresses it. # The script may also be used to generate the inputfile used for gen_init_cpio # This script assumes that gen_init_cpio is located in usr/ directory
@@ -16,8 +16,8 @@ usage() { cat << EOF Usage: $0 [-o <file>] [-u <uid>] [-g <gid>] {-d | <cpio_source>} ... - -o <file> Create gzipped initramfs file named <file> using - gen_init_cpio and gzip + -o <file> Create compressed initramfs file named <file> using + gen_init_cpio and compressor depending on the extension -u <uid> User ID to map to user ID 0 (root). <uid> is only meaningful if <cpio_source> is a directory. "squash" forces all files to uid 0.
@@ -225,6 +225,7 @@ cpio_list= output="/dev/stdout" output_file="" is_cpio_compressed= +compr="gzip -9 -f" arg="$1" case "$arg" in
@@ -233,11 +234,15 @@ case "$arg" in echo "deps_initramfs := \\" shift ;; - "-o") # generate gzipped cpio image named $1 + "-o") # generate compressed cpio image named $1 shift output_file="$1" cpio_list="$(mktemp ${TMPDIR:-/tmp}/cpiolist.XXXXXX)" output=${cpio_list} + echo "$output_file" | grep -q "\.gz$" && compr="gzip -9 -f" + echo "$output_file" | grep -q "\.bz2$" && compr="bzip2 -9 -f" + echo "$output_file" | grep -q "\.lzma$" && compr="lzma -9 -f" + echo "$output_file" | grep -q "\.cpio$" && compr="cat" shift ;; esac
@@ -274,7 +279,7 @@ while [ $# -gt 0 ]; do esac done -# If output_file is set we will generate cpio archive and gzip it +# If output_file is set we will generate cpio archive and compress it # we are carefull to delete tmp files if [ ! -z ${output_file} ]; then if [ -z ${cpio_file} ]; then
@@ -287,7 +292,7 @@ if [ ! -z ${output_file} ]; then if [ "${is_cpio_compressed}" = "compressed" ]; then cat ${cpio_tfile} > ${output_file} else - cat ${cpio_tfile} | gzip -f -9 - > ${output_file} + cat ${cpio_tfile} | ${compr} - > ${output_file} fi [ -z ${cpio_file} ] && rm ${cpio_tfile} fi
usr/Kconfig +27 -0
@@ -44,3 +44,30 @@ config INITRAMFS_ROOT_GID owned by group root in the initial ramdisk image. If you are not sure, leave it set to "0". + +config RD_GZIP + bool "Initial ramdisk compressed using gzip" + default y + depends on BLK_DEV_INITRD=y + select DECOMPRESS_GZIP + help + Support loading of a gzip encoded initial ramdisk or cpio buffer. + If unsure, say Y. + +config RD_BZIP2 + bool "Initial ramdisk compressed using bzip2" + default n + depends on BLK_DEV_INITRD=y + select DECOMPRESS_BZIP2 + help + Support loading of a bzip2 encoded initial ramdisk or cpio buffer + If unsure, say N. + +config RD_LZMA + bool "Initial ramdisk compressed using lzma" + default n + depends on BLK_DEV_INITRD=y + select DECOMPRESS_LZMA + help + Support loading of a lzma encoded initial ramdisk or cpio buffer + If unsure, say N.
usr/Makefile +30 -12
72 lines not on this page · raw message
usr/initramfs_data.S +1 -1
@@ -26,5 +26,5 @@ SECTIONS */ .section .init.ramfs,"a" -.incbin "usr/initramfs_data.cpio.gz" +.incbin "usr/initramfs_data.cpio"
usr/initramfs_data.bz2.S new file +29 -0
@@ -0,0 +1,29 @@ +/* + initramfs_data includes the compressed binary that is the + filesystem used for early user space. + Note: Older versions of "as" (prior to binutils 2.11.90.0.23 + released on 2001-07-14) dit not support .incbin. + If you are forced to use older binutils than that then the + following trick can be applied to create the resulting binary: + + + ld -m elf_i386 --format binary --oformat elf32-i386 -r \ + -T initramfs_data.scr initramfs_data.cpio.gz -o initramfs_data.o + ld -m elf_i386 -r -o built-in.o initramfs_data.o + + initramfs_data.scr looks like this: +SECTIONS +{ + .init.ramfs : { *(.data) } +} + + The above example is for i386 - the parameters vary from architectures. + Eventually look up LDFLAGS_BLOB in an older version of the + arch/$(ARCH)/Makefile to see the flags used before .incbin was introduced. + + Using .incbin has the advantage over ld that the correct flags are set + in the ELF header, as required by certain architectures. +*/ + +.section .init.ramfs,"a" +.incbin "usr/initramfs_data.cpio.bz2"
usr/initramfs_data.gz.S new file +29 -0
@@ -0,0 +1,29 @@ +/* + initramfs_data includes the compressed binary that is the + filesystem used for early user space. + Note: Older versions of "as" (prior to binutils 2.11.90.0.23 + released on 2001-07-14) dit not support .incbin. + If you are forced to use older binutils than that then the + following trick can be applied to create the resulting binary: + + + ld -m elf_i386 --format binary --oformat elf32-i386 -r \ + -T initramfs_data.scr initramfs_data.cpio.gz -o initramfs_data.o + ld -m elf_i386 -r -o built-in.o initramfs_data.o + + initramfs_data.scr looks like this: +SECTIONS +{ + .init.ramfs : { *(.data) } +} + + The above example is for i386 - the parameters vary from architectures. + Eventually look up LDFLAGS_BLOB in an older version of the + arch/$(ARCH)/Makefile to see the flags used before .incbin was introduced. + + Using .incbin has the advantage over ld that the correct flags are set + in the ELF header, as required by certain architectures. +*/ + +.section .init.ramfs,"a" +.incbin "usr/initramfs_data.cpio.gz"
usr/initramfs_data.lzma.S new file +29 -0
@@ -0,0 +1,29 @@ +/* + initramfs_data includes the compressed binary that is the + filesystem used for early user space. + Note: Older versions of "as" (prior to binutils 2.11.90.0.23 + released on 2001-07-14) dit not support .incbin. + If you are forced to use older binutils than that then the + following trick can be applied to create the resulting binary: + + + ld -m elf_i386 --format binary --oformat elf32-i386 -r \ + -T initramfs_data.scr initramfs_data.cpio.gz -o initramfs_data.o + ld -m elf_i386 -r -o built-in.o initramfs_data.o + + initramfs_data.scr looks like this: +SECTIONS +{ + .init.ramfs : { *(.data) } +} + + The above example is for i386 - the parameters vary from architectures. + Eventually look up LDFLAGS_BLOB in an older version of the + arch/$(ARCH)/Makefile to see the flags used before .incbin was introduced. + + Using .incbin has the advantage over ld that the correct flags are set + in the ELF header, as required by certain architectures. +*/ + +.section .init.ramfs,"a" +.incbin "usr/initramfs_data.cpio.lzma"
Quoted 9 lines · H. Peter Anvin
Hi Linus,
This tree contains the bzip2 and LZMA kernel compression work that Alain Knaff has done. Sending this as a separate pull request in case you think that it is too late or immature for this cycle.
The good part is that it is a highly "brittle" feature -- if it fails, it will fail noisily and obviously.
Quoted 7 lines
I have not included the ARM parts that Alain developed; I will leave those to be fed through rmk.
One of the main attractions of this patchset is that it uses the newer
lib/zlib_inflatecode even for the kernel decompressor. Once all architectures that use the olderlib/inflate.chave been converted over, we can remove that code entirely.
it seems still have some problem
mydisk13_x86_64.lzma is by lzma -9
[ 47.404316] RAMDISK: lzma image found at block 0 [ 51.676838] RAMDISK: incomplete write (-28 != 130) [ 51.894794] calling 5_generic_delete_inode_async+0x0/0xa9 @ 2316 [ 51.900895] initcall 5_generic_delete_inode_async+0x0/0xa9 returned 0 after 3 usecs [ 51.901117] kjournald starting. Commit interval 5 seconds [ 51.901130] EXT3 FS on ram0, internal journal [ 51.901138] EXT3-fs: mounted filesystem with ordered data mode. [ 51.901151] VFS: Mounted root (ext3 filesystem) on device 1:0. [ 51.901168] async_waiting @ 1 [ 51.904191] calling 6_generic_delete_inode_async+0x0/0xa9 @ 2317 [ 51.907123] initcall 6_generic_delete_inode_async+0x0/0xa9 returned 0 after 2861 usecs [ 51.947093] async_continuing @ 1 after 44846 usec [ 51.951839] Freeing unused kernel memory: 448k freed [ 51.957245] init_special_inode: bogus i_mode (4467)
Quoted 21 lines · Yinghai Lu
Quoted 9 lines · H. Peter Anvin
Hi Linus,
This tree contains the bzip2 and LZMA kernel compression work that Alain Knaff has done. Sending this as a separate pull request in case you think that it is too late or immature for this cycle.
The good part is that it is a highly "brittle" feature -- if it fails, it will fail noisily and obviously.
Quoted 7 lines
I have not included the ARM parts that Alain developed; I will leave those to be fed through rmk.
One of the main attractions of this patchset is that it uses the newer
lib/zlib_inflatecode even for the kernel decompressor. Once all architectures that use the olderlib/inflate.chave been converted over, we can remove that code entirely.it seems still have some problem
mydisk13_x86_64.lzma is by lzma -9
Quoted 17 lines
[ 47.404316] RAMDISK: lzma image found at block 0 [ 51.676838] RAMDISK: incomplete write (-28 != 130) [ 51.894794] calling 5_generic_delete_inode_async+0x0/0xa9 @ 2316 [ 51.900895] initcall 5_generic_delete_inode_async+0x0/0xa9 returned 0 after 3 usecs [ 51.901117] kjournald starting. Commit interval 5 seconds [ 51.901130] EXT3 FS on ram0, internal journal [ 51.901138] EXT3-fs: mounted filesystem with ordered data mode. [ 51.901151] VFS: Mounted root (ext3 filesystem) on device 1:0. [ 51.901168] async_waiting @ 1 [ 51.904191] calling 6_generic_delete_inode_async+0x0/0xa9 @ 2317 [ 51.907123] initcall 6_generic_delete_inode_async+0x0/0xa9 returned 0 after 2861 usecs [ 51.947093] async_continuing @ 1 after 44846 usec [ 51.951839] Freeing unused kernel memory: 448k freed [ 51.957245] init_special_inode: bogus i_mode (4467)
the .lzma is corrupted. it is strange. i was compiling kernel when lzma it...
recreate that .lzma the kernel works well.
YH
Quoted 40 lines · Yinghai Lu
Quoted 21 lines · Yinghai Lu
Quoted 9 lines · H. Peter Anvin
Hi Linus,
This tree contains the bzip2 and LZMA kernel compression work that Alain Knaff has done. Sending this as a separate pull request in case you think that it is too late or immature for this cycle.
The good part is that it is a highly "brittle" feature -- if it fails, it will fail noisily and obviously.
Quoted 7 lines
I have not included the ARM parts that Alain developed; I will leave those to be fed through rmk.
One of the main attractions of this patchset is that it uses the newer
lib/zlib_inflatecode even for the kernel decompressor. Once all architectures that use the olderlib/inflate.chave been converted over, we can remove that code entirely.it seems still have some problem
mydisk13_x86_64.lzma is by lzma -9Quoted 17 lines
[ 47.404316] RAMDISK: lzma image found at block 0 [ 51.676838] RAMDISK: incomplete write (-28 != 130) [ 51.894794] calling 5_generic_delete_inode_async+0x0/0xa9 @ 2316 [ 51.900895] initcall 5_generic_delete_inode_async+0x0/0xa9 returned 0 after 3 usecs [ 51.901117] kjournald starting. Commit interval 5 seconds [ 51.901130] EXT3 FS on ram0, internal journal [ 51.901138] EXT3-fs: mounted filesystem with ordered data mode. [ 51.901151] VFS: Mounted root (ext3 filesystem) on device 1:0. [ 51.901168] async_waiting @ 1 [ 51.904191] calling 6_generic_delete_inode_async+0x0/0xa9 @ 2317 [ 51.907123] initcall 6_generic_delete_inode_async+0x0/0xa9 returned 0 after 2861 usecs [ 51.947093] async_continuing @ 1 after 44846 usec [ 51.951839] Freeing unused kernel memory: 448k freed [ 51.957245] init_special_inode: bogus i_mode (4467)
Quoted 4 lines
the .lzma is corrupted. it is strange. i was compiling kernel when lzma it...
recreate that .lzma the kernel works well.
was the linux/.lzma file corrupted, and once you removed the stale file it all works fine and the problem does not come back?
I.e. the tree as offered by hpa is fine?
Ingo
H. Peter Anvin
reviewerQuoted 6 lines · Ingo Molnar
was the linux/.lzma file corrupted, and once you removed the stale file it all works fine and the problem does not come back?
I.e. the tree as offered by hpa is fine?
That was the understanding I got.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 9 lines · H. Peter Anvin
Quoted 6 lines · Ingo Molnar
was the linux/.lzma file corrupted, and once you removed the stale file it all works fine and the problem does not come back?
I.e. the tree as offered by hpa is fine?
That was the understanding I got.
yes.
something wrong with lzma util somehow. when loading is high, it could produce corrupted result.
it seems only one window has problem.
it is opensuse 11.1 ...
YH
A message that is not in the archive (sent off-list or not yet received): [email protected]
H. Peter Anvin
reviewerQuoted 7 lines · Alain Knaff
But now I am curious how this will evolve from here. I suppose it will soon appear in one of the patch-2.6.28-gitxy.gz under ftp://ftp.kernel.org/pub/linux/kernel/v2.6/snapshots , and then in an 2.6.29-rcx etc.
Or are there some more other steps involved in between?
Well, Linus opted not to merge it for 2.6.29-rc1, which means it is dead for this merge cycle. This gives us a couple of options, with the aim to get it merged into 2.6.30:
- We can continue to carry it in the -tip tree, which also means it will be in the linux-next tree.
- We can push it to Andrew Morton for the -mm tree.
- Sam could take it in his kbuild tree.
Out of these, I think the kbuild tree is entirely inappropriate. The selection of the other two is mostly a matter of testing, and which way will be easier to add the ARM code and other arch support.
-hpa
Signature · 2 lines
H. Peter Anvin, Intel Open Source Technology Center I work for Intel. I don't speak on their behalf.
Quoted 10 lines · H. Peter Anvin
Quoted 6 lines · Alain Knaff
But now I am curious how this will evolve from here. I suppose it will soon appear in one of the patch-2.6.28-gitxy.gz under ftp://ftp.kernel.org/pub/linux/kernel/v2.6/snapshots , and then in an 2.6.29-rcx etc.
Or are there some more other steps involved in between?Well, Linus opted not to merge it for 2.6.29-rc1, which means it is dead for this merge cycle.
Too bad :(
For when is the next merge window scheduled (approximatively...)?
What impact does this have on procedure for supplying updates to it? Indeed, I've got a couple of new features in the pipeline that I'd like to add in the new future:
- centralizing the switch of kernel compression in a common place (will
make it easier to add new compressions once all architectures support the new scheme, without the need of touching all of them).
- support for new LZMA variant with "real" magic numbers
- support for "no kernel compression" option (people have asked me for
this for the case where they have a boot loader that already handles decompression)
Quoted 7 lines
This gives us a couple of options, with the aim
to get it merged into 2.6.30:- We can continue to carry it in the -tip tree, which also means it will be in the linux-next tree.
- We can push it to Andrew Morton for the -mm tree.
- Sam could take it in his kbuild tree.
What are the advantages and disadvantages of each of those? Personally, I'd prefer a choice that:
- allows the most lightweight procedure for updating it (i.e. allows to
supply incremental changes, rather than do a "full" release)
- is most visible (so that when people ask me for it, I can for example
tell them "it's already in the -mm tree, download it from xxx". Oh, and visibility will give it also more test exposure)
Quoted 3 lines
Out of these, I think the kbuild tree is entirely inappropriate. The selection of the other two is mostly a matter of testing, and which way will be easier to add the ARM code and other arch support.
Well ease of merging the ARM code in is obviously also a consideration to take into account.
Quoted 3 lines
-hpa
Regards,
Alain