LZO Compression Support
Subsystems (3)
cover LZO Compression Support
The following series implements LZO compression support to makedumpfile. LZO is as good as in size but by far better in speed than ZLIB, readucing down time during generation of crash dump and refiltering.
The RFC discussion:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
How to build:
1. Get lzo libraries: lzo, lzo-devel and lzo-minilzo from either of the following:
1) Original author's website:
http://www.oberhumer.com/opensource/lzo/
2) yum framework on fedora. Older releases don't have the packages.
2. Apply the patch set to makedumpfile v1.4.2.
3. Do make.
How to use:
Introduce new -l option. If a user specify this, makedumpfile generates dumpfile compressed by pages with lzo compression.
Example)
$ makedumpfile -l vmcore dumpfile
Performance evaluation:
- Kumagai-san's evaluation simulating actually working servers:
http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
- My evaluation focusing on the worst cases:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
LZO Support for crash:
I'll post LZO support patch for crash after makedumpfile merges these patches.
---
HATAYAMA Daisuke (2):
Add help and manual messages about LZO compression support Add LZO Support
Makefile | 2 +- diskdump_mod.h | 3 ++- makedumpfile.8 | 6 +++--- makedumpfile.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++--------- makedumpfile.h | 2 ++ print_info.c | 16 ++++++++-------- 6 files changed, 64 insertions(+), 22 deletions(-)
Signature · 6 lines
HATAYAMA Daisuke _______________________________________________ kexec mailing list [email protected] http://lists.infradead.org/mailman/listinfo/kexec
PATCH 1/2 Add LZO Support
Add -l option as one of the command-line options users can specify. If -l option is specified, then makedumpfile generates dumpfile in kdump-compressed format with lzo compression by each page.
---
Makefile | 2 +- diskdump_mod.h | 3 ++- makedumpfile.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++--------- makedumpfile.h | 2 ++ 4 files changed, 53 insertions(+), 11 deletions(-)
Makefile +1 -1
@@ -29,7 +29,7 @@ OBJ_PART = print_info.o dwarf_info.o elf_info.o erase_info.o sadump_info.o SRC_ARCH = arch/arm.c arch/x86.c arch/x86_64.c arch/ia64.c arch/ppc64.c arch/s390x.c OBJ_ARCH = arch/arm.o arch/x86.o arch/x86_64.o arch/ia64.o arch/ppc64.o arch/s390x.o -LIBS = -ldw -lbz2 -lebl -ldl -lelf -lz +LIBS = -ldw -lbz2 -lebl -ldl -lelf -lz -llzo2 ifneq ($(LINKTYPE), dynamic) LIBS := -static $(LIBS) endif
diskdump_mod.h +2 -1
@@ -78,7 +78,8 @@ struct kdump_sub_header { };/* page flags */ -#define DUMP_DH_COMPRESSED 0x1/* page is compressed */ +#define DUMP_DH_COMPRESSED_ZLIB 0x1/* page is compressed with zlib */ +#define DUMP_DH_COMPRESSED_LZO 0x2/* paged is compressed with lzo */ /* descriptor of each page for vmcore */ typedef struct page_desc {
makedumpfile.c +48 -9
121 lines not on this page · raw message
makedumpfile.h +2 -0
@@ -31,6 +31,7 @@#include <libelf.h> #include <byteswap.h> #include <getopt.h> +#include <lzo/lzo1x.h> #include "common.h" #include "dwarf_info.h" #include "diskdump_mod.h"
@@ -767,6 +768,7 @@ struct DumpInfo {int num_dump_level;/* number of dump level */ int array_dump_level[NUM_ARRAY_DUMP_LEVEL ];int flag_compress;/* flag of compression */ +int flag_lzo_support;/* flag of LZO compression support */ int flag_elf_dumpfile;/* flag of creating ELF dumpfile */ int flag_generate_vmcoreinfo;/* flag of generating vmcoreinfo file */ int flag_read_vmcoreinfo;/* flag of reading vmcoreinfo file */
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
PATCH 2/2 Add help and manual messages about LZO compression support
In reply to HATAYAMA Daisuke---
makedumpfile.8 | 6 +++--- print_info.c | 16 ++++++++-------- 2 files changed, 11 insertions(+), 11 deletions(-)
makedumpfile.8 +3 -3
@@ -121,8 +121,8 @@ configuration, you need to use --diskset option. .SH OPTIONS .TP -\fB\-c\fR -Compress dump data by each page. +\fB\-c,\-l\fR +Compress dump data by each page using zlib for -c option or lzo for -l option. .br A user cannot specify this option with \-E option, because the ELF format does not support compressed data.
@@ -196,7 +196,7 @@ by dump_level 11, makedumpfile retries it by dump_level 31. \fB\-E\fR Create \fIDUMPFILE\fR in the ELF format. .br -This option cannot be specified with \-c option, because the ELF format does not +This option cannot be specified with either of \-c option or \-l option, because the ELF format does not support compressed data. .br .B Example:
print_info.c +8 -8
@@ -35,15 +35,15 @@ print_usage(void)MSG ("\n" );MSG ("Usage:\n" );MSG (" Creating DUMPFILE:\n" ); -MSG (" # makedumpfile [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE\n" ); +MSG (" # makedumpfile [-c|-l|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE DUMPFILE\n" );MSG ("\n" );MSG (" Creating DUMPFILE with filtered kernel data specified through filter config\n" );MSG (" file:\n" ); -MSG (" # makedumpfile [-c|-E] [-d DL] -x VMLINUX --config FILTERCONFIGFILE VMCORE\n" ); +MSG (" # makedumpfile [-c|-l|-E] [-d DL] -x VMLINUX --config FILTERCONFIGFILE VMCORE\n" );MSG (" DUMPFILE\n" );MSG ("\n" );MSG (" Outputting the dump data in the flattened format to the standard output:\n" ); -MSG (" # makedumpfile -F [-c|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n" ); +MSG (" # makedumpfile -F [-c|-l|-E] [-d DL] [-x VMLINUX|-i VMCOREINFO] VMCORE\n" );MSG ("\n" );MSG (" Rearranging the dump data in the flattened format to a readable DUMPFILE:\n" );MSG (" # makedumpfile -R DUMPFILE\n" );
@@ -70,14 +70,14 @@ print_usage(void)MSG ("\n" );MSG ("\n" );MSG (" Creating DUMPFILE from multiple VMCOREs generated on sadump diskset configuration:\n" ); -MSG (" # makedumpfile [-c] [-d DL] -x VMLINUX --diskset=VMCORE1 --diskset=VMCORE2\n" ); +MSG (" # makedumpfile [-c|-l] [-d DL] -x VMLINUX --diskset=VMCORE1 --diskset=VMCORE2\n" );MSG (" [--diskset=VMCORE3 ..] DUMPFILE\n" );MSG ("\n" );MSG ("\n" );MSG ("Available options:\n" ); -MSG (" [-c]:\n" ); -MSG (" Compress dump data by each page.\n" ); -MSG (" A user cannot specify this option with -E option, because the ELF format\n" ); +MSG (" [-c|-l]:\n" ); +MSG (" Compress dump data by each page using zlib for -c option and lzo for -l option.\n" ); +MSG (" A user cannot specify either of these options with -E option, because the ELF format\n" );MSG (" does not support compressed data.\n" );MSG (" THIS IS ONLY FOR THE CRASH UTILITY.\n" );MSG ("\n" );
@@ -103,7 +103,7 @@ print_usage(void)MSG ("\n" );MSG (" [-E]:\n" );MSG (" Create DUMPFILE in the ELF format.\n" ); -MSG (" This option cannot be specified with -c option, because the ELF\n" ); +MSG (" This option cannot be specified with either of -c option or -l option, because the ELF\n" );MSG (" format does not support compressed data.\n" );MSG ("\n" );MSG (" [-x VMLINUX]:\n" );
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Hatayama-san,
Quoted 37 lines · On Mon, 20 Feb 2012 18:10:00 +0900 HATAYAMA Daisuke…
The following series implements LZO compression support to makedumpfile. LZO is as good as in size but by far better in speed than ZLIB, readucing down time during generation of crash dump and refiltering.
The RFC discussion:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
http://lists.infradead.org/pipermail/kexec/2011-December/005868.htmlHow to build:
1. Get lzo libraries: lzo, lzo-devel and lzo-minilzo from either of the following:
1) Original author's website:
http://www.oberhumer.com/opensource/lzo/2) yum framework on fedora. Older releases don't have the packages.
2. Apply the patch set to makedumpfile v1.4.2.
3. Do make.
How to use:
Introduce new -l option. If a user specify this, makedumpfile generates dumpfile compressed by pages with lzo compression.
Example)
$ makedumpfile -l vmcore dumpfilePerformance evaluation:
- Kumagai-san's evaluation simulating actually working servers:
http://lists.infradead.org/pipermail/kexec/2011-December/005868.html
Quoted 7 lines
- My evaluation focusing on the worst cases:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.htmlLZO Support for crash:
I'll post LZO support patch for crash after makedumpfile merges these patches.
Thank you for your patches.
I'll review and merge them into v1.4.4 with making LZO support optional. I'll post the patch to make it optional after v1.4.3 is released.
Please wait for a while.
Thanks
Atsushi Kumagai
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Kumagai-san,
From: Atsushi Kumagai <[email protected]>
Subject: Re: [PATCH makedumpfile 0/2] LZO Compression Support
Date: Tue, 21 Feb 2012 16:50:41 +0900
Quoted 52 lines
Hello Hatayama-san,
Quoted 37 lines · On Mon, 20 Feb 2012 18:10:00 +0900 HATAYAMA Daisuke…
The following series implements LZO compression support to makedumpfile. LZO is as good as in size but by far better in speed than ZLIB, readucing down time during generation of crash dump and refiltering.
The RFC discussion:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.html
http://lists.infradead.org/pipermail/kexec/2011-December/005868.htmlHow to build:
1. Get lzo libraries: lzo, lzo-devel and lzo-minilzo from either of the following:
1) Original author's website:
http://www.oberhumer.com/opensource/lzo/2) yum framework on fedora. Older releases don't have the packages.
2. Apply the patch set to makedumpfile v1.4.2.
3. Do make.
How to use:
Introduce new -l option. If a user specify this, makedumpfile generates dumpfile compressed by pages with lzo compression.
Example)
$ makedumpfile -l vmcore dumpfilePerformance evaluation:
- Kumagai-san's evaluation simulating actually working servers:
http://lists.infradead.org/pipermail/kexec/2011-December/005868.htmlQuoted 7 lines
- My evaluation focusing on the worst cases:
http://lists.infradead.org/pipermail/kexec/2011-November/005783.htmlLZO Support for crash:
I'll post LZO support patch for crash after makedumpfile merges these patches.
Thank you for your patches.
Quoted 8 lines
I'll review and merge them into v1.4.4 with making LZO support optional. I'll post the patch to make it optional after v1.4.3 is released.
Please wait for a while.
Thanks
Atsushi Kumagai
Thanks. But I'm OK if I'll repost the patch after you have decided how you change Makefile to support optional building. Or is it more effective for you to do them by yourself at the same time?
Thanks.
HATAYAMA, Daisuke
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
Hello Hatayama-san,
Quoted 7 lines · On Tue, 21 Feb 2012 17:01:25 +0900 ( ) HATAYAMA Daisuke…
Quoted 7 lines
Thank you for your patches.
I'll review and merge them into v1.4.4 with making LZO support optional. I'll post the patch to make it optional after v1.4.3 is released.
Please wait for a while.
Quoted 7 lines
Quoted 3 lines
Thanks
Atsushi KumagaiThanks. But I'm OK if I'll repost the patch after you have decided how you change
Makefileto support optional building. Or is it more effective for you to do them by yourself at the same time?
I wonder if you could repost the patch with implementing my idea. My idea is that using build option (like "LINKTYPE" at v1.4.2).
Example)
1. enable lzo support
$ make USELZO=on
2. disable lzo support (default)
$ make
In future, I would like to use automake but don't worry about it now.
Thanks
Atsushi Kumagai
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
From: Atsushi Kumagai <[email protected]>
Subject: Re: [PATCH makedumpfile 0/2] LZO Compression Support
Date: Wed, 22 Feb 2012 11:42:09 +0900
Quoted 25 lines
Hello Hatayama-san,
Quoted 7 lines · On Tue, 21 Feb 2012 17:01:25 +0900 ( ) HATAYAMA Daisuke…
Quoted 7 lines
Thank you for your patches.
I'll review and merge them into v1.4.4 with making LZO support optional. I'll post the patch to make it optional after v1.4.3 is released.
Please wait for a while.
Quoted 7 lines
Quoted 3 lines
Thanks
Atsushi KumagaiThanks. But I'm OK if I'll repost the patch after you have decided how you change
Makefileto support optional building. Or is it more effective for you to do them by yourself at the same time?I wonder if you could repost the patch with implementing my idea. My idea is that using build option (like "LINKTYPE" at v1.4.2).
Example)
1. enable lzo support
Quoted 7 lines
$ make USELZO=on2. disable lzo support (default)
$ makeIn future, I would like to use automake but don't worry about it now.
I see. I'll repost the patch soon.
Thanks.
HATAYAMA, Daisuke
_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec