Skip to contentSkip to conversation

makedumpfile-1.3.4: Support kdump-compressed format for input file.

PATCH In review Ken'ichi Ohmichi 2 participants 12 messages Oct 9, 2009 – Oct 19, 2009

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

Hi,

makedumpfile version 1.3.4 is released.
Your comments/patches are welcome.

Changelog:
o New feature
- Support kdump-compressed format for input file. (by Takao Indoh) A new makedumpfile can read dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.
- Add dump filtering on an x86_64 xen domain-0.
A new makedumpfile can exclude unnecessary pages (cache pages, user process data pages, and free pages) on on x86_64 xen domain-0.
- Save utsname data into disk_dump_header.
A new makedumpfile saves utsname data into disk_dump_header. The crash utility can output the data from a dumpfile by 'help -n'.
- Add linux-2.6.30 - 2.6.31 support.

o Bugfix
- Fix some lacks of free(). (by Minoru Usui)

o Code Cleanup
- Add some comments to code for the readability.
- Remove VERSION_LINUX_2_6_XX.
- Add paddr_to_pfn() and pfn_to_paddr() macros.
- Merge the version information of makedumpfile.
- Merge the PT_NOTE methods of both elf64 and elf32.
- Reduce some indents in exclude_unnecessary_pages().
- Remove unnecessary spaces.
- read_disk_dump_header() and read_kdump_sub_header().
- Use DISKDUMP_HDADER_BLOCKS for the readability.

Explanation of makedumpfile:
To shorten the size of the dumpfile and the time of creating the dumpfile, makedumpfile copies only the necessary pages for analysis to the dumpfile from /proc/vmcore. You can specify the kind of unnecessary pages with dump_level. If you want to shorten the size further, enable the compression of the page data.

Download:
You can download the latest makedumpfile from the following URL. Details of the change are written on the CVS page of the following site.
https://sourceforge.net/projects/makedumpfile/

Method of installation:
You can compile the makedumpfile command as follows;
1. "tar -zxvf makedumpfile-x.y.z.tar.gz"
2. "cd makedumpfile-x.y.z"
3. "make; make install"

Usage:
makedumpfile [-c] [-E] [-d dump_level] [-x vmlinux] dump_mem dump_file

Example:
If you want to exclude pages filled by zero, cache pages, user pages and free pages and to enable compression, please execute the following command.

  # makedumpfile -c -d 31 -x vmlinux /proc/vmcore dumpfile

Thanks
Ken'ichi Ohmichi

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#1 lore raw

Hi,

Quoted 4 lines · Ken'ichi Ohmichi

Download:
You can download the latest makedumpfile from the following URL. Details of the change are written on the CVS page of the following site.
https://sourceforge.net/projects/makedumpfile/

makedumpfile's CVS repository was imported into git, and we will use the git repository.

The git repository: http://makedumpfile.git.sourceforge.net/

Thanks
Ken'ichi Ohmichi

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#2 lore raw
In reply to Ken'ichi Ohmichi

Hallo Ken'ichi!

From: "Ken'ichi Ohmichi" <[email protected]>
Subject: makedumpfile-1.3.4: Support kdump-compressed format for input file.
Date: Fri, 09 Oct 2009 11:40:35 +0900

Quoted 11 lines

Hi,

makedumpfile version 1.3.4 is released.
Your comments/patches are welcome.

Changelog:
o New feature
- Support kdump-compressed format for input file. (by Takao Indoh) A new makedumpfile can read dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

Does it need to specify either a vmlinux or vmcoreinfo file? Got a failure if not. Note, file vmcore is a full VMCore generated from a 2.6.31 x86_64 kernel and kexec-tools-2.0.0.

# ./makedumpfile -c -d 3 vmcore vmcore.1
...
The dumpfile is saved to vmcore.1
...
# /makedumpfile -c -d 31 vmcore.1 vmcore.2
vmcore.1 doesn't contain vmcoreinfo.
Specify '-x' option or '-i' option.
Commandline parameter is invalid.
Try `makedumpfile --help' for more information.

makedumpfile Failed.

Thanks!
CAI Qian

Quoted 48 lines

- Add dump filtering on an x86_64 xen domain-0.
A new makedumpfile can exclude unnecessary pages (cache pages, user process data pages, and free pages) on on x86_64 xen domain-0.
- Save utsname data into disk_dump_header.
A new makedumpfile saves utsname data into disk_dump_header. The crash utility can output the data from a dumpfile by 'help -n'.
- Add linux-2.6.30 - 2.6.31 support.

o Bugfix
- Fix some lacks of free(). (by Minoru Usui)

o Code Cleanup
- Add some comments to code for the readability.
- Remove VERSION_LINUX_2_6_XX.
- Add paddr_to_pfn() and pfn_to_paddr() macros.
- Merge the version information of makedumpfile.
- Merge the PT_NOTE methods of both elf64 and elf32.
- Reduce some indents in exclude_unnecessary_pages().
- Remove unnecessary spaces.
- read_disk_dump_header() and read_kdump_sub_header().
- Use DISKDUMP_HDADER_BLOCKS for the readability.

Explanation of makedumpfile:
To shorten the size of the dumpfile and the time of creating the dumpfile, makedumpfile copies only the necessary pages for analysis to the dumpfile from /proc/vmcore. You can specify the kind of unnecessary pages with dump_level. If you want to shorten the size further, enable the compression of the page data.

Download:
You can download the latest makedumpfile from the following URL. Details of the change are written on the CVS page of the following site.
https://sourceforge.net/projects/makedumpfile/

Method of installation:
You can compile the makedumpfile command as follows;
1. "tar -zxvf makedumpfile-x.y.z.tar.gz"
2. "cd makedumpfile-x.y.z"
3. "make; make install"

Usage:
makedumpfile [-c] [-E] [-d dump_level] [-x vmlinux] dump_mem dump_file

Example:
If you want to exclude pages filled by zero, cache pages, user pages and free pages and to enable compression, please execute the following command.

Quoted 6 lines
  # makedumpfile -c -d 31 -x vmlinux /proc/vmcore dumpfile

Thanks
Ken'ichi Ohmichi

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#3 lore raw

Hi CAI,

Thank you for testing and a good point.

Quoted 10 lines · CAI Qian
Quoted 8 lines

makedumpfile version 1.3.4 is released.
Your comments/patches are welcome.

Changelog:
o New feature
- Support kdump-compressed format for input file. (by Takao Indoh) A new makedumpfile can read dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

Does it need to specify either a vmlinux or vmcoreinfo file?

Yes, we need to do it for re-filtering now.
But it is not difficult to implement a feature that it is not necessary to specify either a vmlinux or vmcoreinfo file for re-filtering. I will try to implement the feature.

Thanks
Ken'ichi Ohmichi

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#4 lore raw

cover make '-x'/'-i' option unnecessary for re-filtering.

Hi,

Quoted 13 lines · Ken'ichi Ohmichi

Hi CAI,

Thank you for testing and a good point.

Quoted 8 lines · CAI Qian
Quoted 8 lines

makedumpfile version 1.3.4 is released.
Your comments/patches are welcome.

Changelog:
o New feature
- Support kdump-compressed format for input file. (by Takao Indoh) A new makedumpfile can read dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

Quoted 6 lines
Quoted 1 line

Does it need to specify either a vmlinux or vmcoreinfo file?

Yes, we need to do it for re-filtering now.
But it is not difficult to implement a feature that it is not necessary to specify either a vmlinux or vmcoreinfo file for re-filtering. I will try to implement the feature.

I created the patchset for implementing the above feature. Any comments/patches are welcome.

---
[PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.

makedumpfile v1.3.4 can read a dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

When re-filtering, it is necessary to specify '-x'/'-i' option. If not specifying, makedumpfile fails like the following:

 # ./makedumpfile -c -d 3 vmcore vmcore.1
 ...
 The dumpfile is saved to vmcore.1
 ...
 #
 # /makedumpfile -c -d 31 vmcore.1 vmcore.2
 vmcore.1 doesn't contain vmcoreinfo.
 Specify '-x' option or '-i' option.
 Commandline parameter is invalid.
 Try `makedumpfile --help' for more information.
 #

For usability, this patchset makes '-x'/'-i' option unnecessary for re-filtering.

Thanks
Ken'ichi Ohmichi

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#5 lore raw

PATCH 1/5 Cleanup: Set a meaning value for KDUMP_SUB_HEADER_BLOCKS.

[PATCH 1/5] Cleanup: Set a meaning value for KDUMP_SUB_HEADER_BLOCKS.

---

diskdump_mod.h +0 -1
diff -rpuN a/diskdump_mod.h b/diskdump_mod.h
@@ -25,7 +25,6 @@
 #define KDUMP_SIGNATURE			"KDUMP   "
 #define SIG_LEN (sizeof(DUMP_PARTITION_SIGNATURE) - 1)
 #define DISKDUMP_HDADER_BLOCKS		(1)
-#define KDUMP_SUB_HEADER_BLOCKS		(1)
 
 #define DUMP_HEADER_COMPLETED	0
 #define DUMP_HEADER_INCOMPLETED 1
makedumpfile.c +1 -1
diff -rpuN a/makedumpfile.c b/makedumpfile.c
@@ -5450,7 +5450,7 @@ write_kdump_header(void)
 	strcpy(dh->signature, KDUMP_SIGNATURE);
 	dh->header_version = 2;
 	dh->block_size   = info->page_size;
-	dh->sub_hdr_size = KDUMP_SUB_HEADER_BLOCKS;
+	dh->sub_hdr_size = divideup(sizeof(sub_dump_header), dh->block_size);
 	dh->max_mapnr    = info->max_mapnr;
 	dh->nr_cpus      = 1;
 	dh->bitmap_blocks

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#6 lore raw

{PATCH 3/5] Cleanup: Shorten the declaration name of kdump_sub_header.

In reply to Ken'ichi Ohmichi

{PATCH 3/5] Cleanup: Shorten the declaration name of kdump_sub_header.

---

makedumpfile.c +9 -9
diff -rpuN a/makedumpfile.c b/makedumpfile.c
@@ -5439,7 +5439,7 @@ write_kdump_header(void)
 {
 	size_t size;
 	struct disk_dump_header *dh = info->dump_header;
-	struct kdump_sub_header sub_dump_header;
+	struct kdump_sub_header kh;
 
 	if (info->flag_elf_dumpfile)
 		return FALSE;
@@ -5450,7 +5450,7 @@ write_kdump_header(void)
 	strcpy(dh->signature, KDUMP_SIGNATURE);
 	dh->header_version = 2;
 	dh->block_size     = info->page_size;
-	dh->sub_hdr_size   = divideup(sizeof(sub_dump_header), dh->block_size);
+	dh->sub_hdr_size   = divideup(sizeof(kh), dh->block_size);
 	dh->max_mapnr      = info->max_mapnr;
 	dh->nr_cpus        = 1;
 	dh->bitmap_blocks  = divideup(info->len_bitmap, dh->block_size);
@@ -5465,15 +5465,15 @@ write_kdump_header(void)
 	 * Write sub header
 	 */
 	size = sizeof(struct kdump_sub_header);
-	memset(&sub_dump_header, 0, size);
-	sub_dump_header.phys_base  = info->phys_base;
-	sub_dump_header.dump_level = info->dump_level;
+	memset(&kh, 0, size);
+	kh.phys_base  = info->phys_base;
+	kh.dump_level = info->dump_level;
 	if (info->flag_split) {
-		sub_dump_header.split = 1;
-		sub_dump_header.start_pfn = info->split_start_pfn;
-		sub_dump_header.end_pfn   = info->split_end_pfn;
+		kh.split = 1;
+		kh.start_pfn = info->split_start_pfn;
+		kh.end_pfn   = info->split_end_pfn;
 	}
-	if (!write_buffer(info->fd_dumpfile, dh->block_size, &sub_dump_header,
+	if (!write_buffer(info->fd_dumpfile, dh->block_size, &kh,
 	    size, info->name_dumpfile))
 		return FALSE;
 

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#7 lore raw

PATCH 2/5 Cleanup: Fix some indents in write_kdump_header().

In reply to Ken'ichi Ohmichi

[PATCH 2/5] Cleanup: Fix some indents in write_kdump_header().

---

makedumpfile.c +5 -6
diff -rpuN a/makedumpfile.c b/makedumpfile.c
@@ -5449,12 +5449,11 @@ write_kdump_header(void)
 	 */
 	strcpy(dh->signature, KDUMP_SIGNATURE);
 	dh->header_version = 2;
-	dh->block_size   = info->page_size;
-	dh->sub_hdr_size = divideup(sizeof(sub_dump_header), dh->block_size);
-	dh->max_mapnr    = info->max_mapnr;
-	dh->nr_cpus      = 1;
-	dh->bitmap_blocks
-	    = divideup(info->len_bitmap, dh->block_size);
+	dh->block_size     = info->page_size;
+	dh->sub_hdr_size   = divideup(sizeof(sub_dump_header), dh->block_size);
+	dh->max_mapnr      = info->max_mapnr;
+	dh->nr_cpus        = 1;
+	dh->bitmap_blocks  = divideup(info->len_bitmap, dh->block_size);
 	memcpy(&dh->timestamp, &info->timestamp, sizeof(dh->timestamp));
 	memcpy(&dh->utsname, &info->system_utsname, sizeof(dh->utsname));
 

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#8 lore raw

PATCH 4/5 Store vmcoreinfo data into a kdump-compressed dumpfile.

In reply to Ken'ichi Ohmichi

[PATCH 4/5] Store vmcoreinfo data into a kdump-compressed dumpfile.

By applying this patch, makedumpfile takes vmcoreinfo data from /proc/vmcore and stores the data into a dumpfile in the kdump- compressed fomat. By the next patch, makedumpfile takes the data from a dumpfile in the kdump-compressed format and uses the data for re-filtering.

---

diskdump_mod.h +2 -0
diff -rpuN a/diskdump_mod.h b/diskdump_mod.h
@@ -71,6 +71,8 @@ struct kdump_sub_header {
 	int		split;		/* header_version 2 and later */
 	unsigned long	start_pfn;	/* header_version 2 and later */
 	unsigned long	end_pfn;	/* header_version 2 and later */
+	off_t		offset_vmcoreinfo;/* header_version 3 and later */
+	unsigned long	size_vmcoreinfo;  /* header_version 3 and later */
 };
 
 /* page flags */
makedumpfile.c +41 -5

73 lines not on this page · raw message

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#9 lore raw

PATCH 5/5 Read vmcoreinfo data from a kdump-compressed dumpfile.

In reply to Ken'ichi Ohmichi

[PATCH 5/5] Read vmcoreinfo data from a kdump-compressed dumpfile.

makedumpfile takes the data from a dumpfile in the kdump-compressed format and uses the data for re-filtering.

---

makedumpfile.c +5 -0
diff -rpuN a/makedumpfile.c b/makedumpfile.c
@@ -871,6 +871,11 @@ get_kdump_compressed_header_info(char *f
 	}
 	memcpy(info->kh_memory, &kh, sizeof(kh));
 
+	if (dh.header_version >= 3) {
+		/* A dumpfile contains vmcoreinfo data. */
+		info->offset_vmcoreinfo = kh.offset_vmcoreinfo;
+		info->size_vmcoreinfo   = kh.size_vmcoreinfo;
+	}
 	return TRUE;
 error:
 	free(info->dh_memory);

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#10 lore raw

Re: [PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.

In reply to Ken'ichi Ohmichi

Hello Ken'ichi!

This series of patches is working great here.

Thanks!
CAI Qian

From: "Ken'ichi Ohmichi" <[email protected]>
Subject: [PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.
Date: Tue, 13 Oct 2009 14:18:44 +0900

Quoted 52 lines

Hi,

Quoted 13 lines · Ken'ichi Ohmichi

Hi CAI,

Thank you for testing and a good point.

Quoted 8 lines · CAI Qian
Quoted 8 lines

makedumpfile version 1.3.4 is released.
Your comments/patches are welcome.

Changelog:
o New feature
- Support kdump-compressed format for input file. (by Takao Indoh) A new makedumpfile can read dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

Quoted 6 lines
Quoted 1 line

Does it need to specify either a vmlinux or vmcoreinfo file?

Yes, we need to do it for re-filtering now.
But it is not difficult to implement a feature that it is not necessary to specify either a vmlinux or vmcoreinfo file for re-filtering. I will try to implement the feature.

I created the patchset for implementing the above feature. Any comments/patches are welcome.

---
[PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.

makedumpfile v1.3.4 can read a dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

When re-filtering, it is necessary to specify '-x'/'-i' option. If not specifying, makedumpfile fails like the following:

 # ./makedumpfile -c -d 3 vmcore vmcore.1
 ...
 The dumpfile is saved to vmcore.1
 ...
 #
 # /makedumpfile -c -d 31 vmcore.1 vmcore.2
 vmcore.1 doesn't contain vmcoreinfo.
 Specify '-x' option or '-i' option.
 Commandline parameter is invalid.
 Try `makedumpfile --help' for more information.
 #

For usability, this patchset makes '-x'/'-i' option unnecessary for re-filtering.

Quoted 8 lines

Thanks
Ken'ichi Ohmichi

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#11 lore raw

Re: [PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.

Hi CAI,

Quoted 3 lines · CAI Qian

Hello Ken'ichi!

This series of patches is working great here.

I'm glad to hear that.
I will merge this patchset into the next release.

Thanks
Ken'ichi Ohmichi

Quoted 65 lines

Thanks!
CAI Qian

From: "Ken'ichi Ohmichi" <[email protected]>
Subject: [PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.
Date: Tue, 13 Oct 2009 14:18:44 +0900

Quoted 57 lines

Hi,

Quoted 13 lines · Ken'ichi Ohmichi

Hi CAI,

Thank you for testing and a good point.

Quoted 8 lines · CAI Qian
Quoted 8 lines

makedumpfile version 1.3.4 is released.
Your comments/patches are welcome.

Changelog:
o New feature
- Support kdump-compressed format for input file. (by Takao Indoh) A new makedumpfile can read dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

Quoted 5 lines
Quoted 1 line

Does it need to specify either a vmlinux or vmcoreinfo file?

Yes, we need to do it for re-filtering now.
But it is not difficult to implement a feature that it is not necessary to specify either a vmlinux or vmcoreinfo file for re-filtering. I will try to implement the feature.

I created the patchset for implementing the above feature. Any comments/patches are welcome.

---
[PATCH 0/5] make '-x'/'-i' option unnecessary for re-filtering.

makedumpfile v1.3.4 can read a dumpfile which is kdump-compressed format instead of /proc/vmcore and re-filter it.

When re-filtering, it is necessary to specify '-x'/'-i' option. If not specifying, makedumpfile fails like the following:

 # ./makedumpfile -c -d 3 vmcore vmcore.1
 ...
 The dumpfile is saved to vmcore.1
 ...
 #
 # /makedumpfile -c -d 31 vmcore.1 vmcore.2
 vmcore.1 doesn't contain vmcoreinfo.
 Specify '-x' option or '-i' option.
 Commandline parameter is invalid.
 Try `makedumpfile --help' for more information.
 #

For usability, this patchset makes '-x'/'-i' option unnecessary for re-filtering.

Thanks
Ken'ichi Ohmichi

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec