Skip to contentSkip to conversation

add a new interface to show the memory usage of 1st kernel

PATCH Superseded Baoquan He 3 participants 18 messages Aug 25, 2014 – Aug 29, 2014

Subsystems (2)

v4 of 6 Superseded Latest is v6 →

lore
Share & tools
lore

Baoquan He

author
#0 lore raw

v4 cover add a new interface to show the memory usage of 1st kernel

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patchset, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

Previouly, patchset v1 was posted. And patch 7/7 has a change in v2. So several changes are made in this v3 post per comments from Vivek and Atsushi.

[patch 2/8] functions to get crashkernel memory range v3->v4:
In old iomem_for_each_line(), it will call exit(1) if opening iomem failed. Atsushi suggested changing this to be consistent with other return value. So here change all the return value to be nr, then check it outside of iomem_for_each_line, and then decide how to act.

[patch 3/8] preparation functions for parsing vmcoreinfo
v1->v3:
Since get_kernel_version need be called to get page_offset before initial() in mem_usage code flow, and surely it will be called inside initial() again. Add a static variable to avoid this duplicate calling.
v3->v4:
Check the value of info->kernel_version, just return if it has been assigned to a value. This is suggested by Atsushi, far better than the static variable idea.
And replace replace get_versiondep_info_x86_64() with get_versiondep_info in get_page_offset().

[patch 4/8] set vmcoreinfo for kcore
v3->v4:
Change several places error messages which are the same in nearby handling.

[patch 5/8] prepare the dump loads for kcore analysis
v1->v3:
Fix the compiler warnings.
v3->v4:
Rename is_vmalloc_addr() to is_vmalloc_addr_x86_64() in arch/x86_64.c. And then define is_vmalloc_addr() for all other ARCHs, just set their value to be TRUE except of x86_64.

[patch 6/8] introduce-a-function-exclude_zero_pages_cyclic v3->v4:
Newly introduce a function exclude_zero_pages_cyclic(), and call it in get_num_dumpable_cyclic().

Besides, a strange thing happened when the new interface was tested on my local AMD machine. It always terminated and printed message:
"Program terminated with signal SIGKILL"
With gdb, it should be going in readmem() of exclude_zero_pages_cyclic, I still don't know why it happened.

[patch 7/8] implement a function to print the memory usage
v1->v3:
Adjust the printing content and format of dumpable page numbers per Vivek's comments.

[patch 8/8]
v1->v2:
Set info->dump_level=MAX_DUMP_LEVEL, with MAX_DUMP_LEVEL all kinds of memory can be calculated.
v2->v3:
Add the description of this feature into help message and man page.
v3->v4:
Continue adjusting the output message of show_mem_usage calling per Vivek's comments.

Baoquan He (8):
initialize pfn_memhole in get_num_dumpable_cyclic functions to get crashkernel memory range preparation functions for parsing vmcoreinfo
set vmcoreinfo for kcore
prepare the dump loads for kcore analysis introduce a function exclude_zero_pages_cyclic() implement a function to print the memory usage add a new interface to show the memory usage of 1st kernel

 arch/x86_64.c  |   6 +-
 elf_info.c     | 237 +++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |   3 +
 makedumpfile.8 |  17 ++++
 makedumpfile.c | 288 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |  17 ++++
 print_info.c   |   8 ++
 7 files changed, 573 insertions(+), 3 deletions(-)
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#1 lore raw

PATCH v4 1/8 initialize pfn_memhole in get_num_dumpable_cyclic

This is a code bug. In initialize_2nd_bitmap_cyclic pfn_memhole is calculated, however it's not initialized before that. If an available pfn_memhole is wanted after get_num_dumpable_cyclic invocation, initializing pfn_memhole in get_num_dumpable_cyclic is necessary.

---
 makedumpfile.c | 2 ++
 1 file changed, 2 insertions(+)
makedumpfile.c +2 -0
index b4b6eca..a01703e 100644
@@ -5651,6 +5651,8 @@ get_num_dumpable_cyclic(void)
 	mdf_pfn_t pfn, num_dumpable=0;
 	struct cycle cycle = {0};
 
+	pfn_memhole = info->max_mapnr;
+
 	for_each_cycle(0, info->max_mapnr, &cycle)
 	{
 		if (!exclude_unnecessary_pages_cyclic(&cycle))
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#2 lore raw

PATCH v4 2/8 functions to get crashkernel memory range

In reply to Baoquan He

These functions are used to parse /proc/iomem code and get memory ranges of specific type. They are implemented in kexec-tools and borrowed here to get the crashkernel memory range. Since crashkernel memory range should be excluded from dumpable memory ranges.

---
 makedumpfile.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |  7 +++++
 2 files changed, 89 insertions(+)
makedumpfile.c +82 -0
index a01703e..44cf26f 100644
@@ -9043,6 +9043,88 @@ calculate_cyclic_buffer_size(void) {
 	return TRUE;
 }
 
+
+
+//#define CRASH_RESERVED_MEM_NR   8
+struct memory_range crash_reserved_mem[CRASH_RESERVED_MEM_NR];
+int crash_reserved_mem_nr;
+
+/*
+ * iomem_for_each_line()
+ *
+ * Iterate over each line in the file returned by proc_iomem(). If match is
+ * NULL or if the line matches with our match-pattern then call the
+ * callback if non-NULL.
+ *
+ * Return the number of lines matched.
+ */
+int iomem_for_each_line(char *match,
+			      int (*callback)(void *data,
+					      int nr,
+					      char *str,
+					      unsigned long base,
+					      unsigned long length),
+			      void *data)
+{
+	const char iomem[] = "/proc/iomem";
+	char line[BUFSIZE_FGETS];
+	FILE *fp;
+	unsigned long long start, end, size;
+	char *str;
+	int consumed;
+	int count;
+	int nr = 0;
+
+	fp = fopen(iomem, "r");
+	if (!fp) {
+		ERRMSG("Cannot open %s\n", iomem);
+		return nr;
+	}
+
+	while(fgets(line, sizeof(line), fp) != 0) {
+		count = sscanf(line, "%Lx-%Lx : %n", &start, &end, &consumed);
+		if (count != 2)
+			continue;
+		str = line + consumed;
+		size = end - start + 1;
+		if (!match || memcmp(str, match, strlen(match)) == 0) {
+			if (callback
+			    && callback(data, nr, str, start, size) < 0) {
+				break;
+			}
+			nr++;
+		}
+	}
+
+	fclose(fp);
+
+	return nr;
+}
+
+static int crashkernel_mem_callback(void *data, int nr,
+                                          char *str,
+                                          unsigned long base,
+                                          unsigned long length)
+{
+        if (nr >= CRASH_RESERVED_MEM_NR)
+                return 1;
+
+        crash_reserved_mem[nr].start = base;
+        crash_reserved_mem[nr].end   = base + length - 1;
+        return 0;
+}
+
+int is_crashkernel_mem_reserved(void)
+{
+        int ret;
+
+        ret = iomem_for_each_line("Crash kernel\n",
+                                        crashkernel_mem_callback, NULL);
+        crash_reserved_mem_nr = ret;
+
+        return !!crash_reserved_mem_nr;
+}
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
makedumpfile.h +7 -0
index 9f90b53..ed584af 100644
@@ -1540,6 +1540,13 @@ extern struct array_table	array_table;
 extern struct number_table	number_table;
 extern struct srcfile_table	srcfile_table;
 
+struct memory_range {
+        unsigned long long start, end;
+};
+
+#define CRASH_RESERVED_MEM_NR   8
+struct memory_range crash_reserved_mem[CRASH_RESERVED_MEM_NR];
+int crash_reserved_mem_nr;
 
 int readmem(int type_addr, unsigned long long addr, void *bufptr, size_t size);
 int get_str_osrelease_from_vmlinux(void);
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#3 lore raw

PATCH v4 3/8 preparation functions for parsing vmcoreinfo

In reply to Baoquan He

Add 2 preparation functions get_elf_loads and get_page_offset, later they will be needed for parsing vmcoreinfo.

Meanwhile since get_kernel_version need be called to get page_offset before initial() in mem_usage code flow, and surely it will be called inside initial() again. Add a static variable to avoid this duplicate calling.

---
 elf_info.c     | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |  1 +
 makedumpfile.c | 18 ++++++++++++++++++
 3 files changed, 69 insertions(+)
elf_info.c +50 -0
index b277f69..69d3fdb 100644
@@ -681,6 +681,56 @@ get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr)
 	return TRUE;
 }
 
+int
+get_elf_loads(int fd, char *filename)
+{
+	int i, j, phnum, elf_format;
+	Elf64_Phdr phdr;
+
+	/*
+	 * Check ELF64 or ELF32.
+	 */
+	elf_format = check_elf_format(fd, filename, &phnum, &num_pt_loads);
+	if (elf_format == ELF64)
+		flags_memory |= MEMORY_ELF64;
+	else if (elf_format != ELF32)
+		return FALSE;
+
+	if (!num_pt_loads) {
+		ERRMSG("Can't get the number of PT_LOAD.\n");
+		return FALSE;
+	}
+
+	/*
+	 * The below file information will be used as /proc/vmcore.
+	 */
+	fd_memory   = fd;
+	name_memory = filename;
+
+	pt_loads = calloc(sizeof(struct pt_load_segment), num_pt_loads);
+	if (pt_loads == NULL) {
+		ERRMSG("Can't allocate memory for the PT_LOAD. %s\n",
+		    strerror(errno));
+		return FALSE;
+	}
+	for (i = 0, j = 0; i < phnum; i++) {
+		if (!get_phdr_memory(i, &phdr))
+			return FALSE;
+
+		if (phdr.p_type != PT_LOAD)
+			continue;
+
+		if (j >= num_pt_loads)
+			return FALSE;
+		if(!dump_Elf_load(&phdr, j))
+			return FALSE;
+		j++;
+	}
+
+	return TRUE;
+}
+
+
 /*
  * Get ELF information about /proc/vmcore.
  */
elf_info.h +1 -0
index 801faff..263d993 100644
@@ -44,6 +44,7 @@ int get_elf64_ehdr(int fd, char *filename, Elf64_Ehdr *ehdr);
 int get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr);
 int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
+int get_elf_loads(int fd, char *filename);
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.c +18 -0
index 44cf26f..8c8ca91 100644
@@ -682,6 +682,9 @@ get_kernel_version(char *release)
 	long maj, min, rel;
 	char *start, *end;
 
+	if (info->kernel_version)
+		return info->kernel_version;
+
 	/*
 	 * This method checks that vmlinux and vmcore are same kernel version.
 	 */
@@ -706,6 +709,7 @@ get_kernel_version(char *release)
 		MSG("The kernel version is not supported.\n");
 		MSG("The created dumpfile may be incomplete.\n");
 	}
+
 	return version;
 }
 
@@ -9125,6 +9129,20 @@ int is_crashkernel_mem_reserved(void)
         return !!crash_reserved_mem_nr;
 }
 
+static int get_page_offset()
+{
+	struct utsname utsname;
+	if (uname(&utsname)) {
+		ERRMSG("Cannot get name and information about current kernel : %s", strerror(errno));
+		return FALSE;
+	}
+
+	info->kernel_version = get_kernel_version(utsname.release);
+	get_versiondep_info();
+
+	return TRUE;
+}
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#4 lore raw

PATCH v5 3/8 preparation functions for parsing vmcoreinfo

Add 2 preparation functions get_elf_loads and get_page_offset, later they will be needed for parsing vmcoreinfo.

Meanwhile check if info->kernel_version is not zero. If it has been assigned to a value, just return directly.

---
 elf_info.c     | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |  1 +
 makedumpfile.c | 18 ++++++++++++++++++
 3 files changed, 69 insertions(+)
elf_info.c +50 -0
index b277f69..69d3fdb 100644
@@ -681,6 +681,56 @@ get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr)
 	return TRUE;
 }
 
+int
+get_elf_loads(int fd, char *filename)
+{
+	int i, j, phnum, elf_format;
+	Elf64_Phdr phdr;
+
+	/*
+	 * Check ELF64 or ELF32.
+	 */
+	elf_format = check_elf_format(fd, filename, &phnum, &num_pt_loads);
+	if (elf_format == ELF64)
+		flags_memory |= MEMORY_ELF64;
+	else if (elf_format != ELF32)
+		return FALSE;
+
+	if (!num_pt_loads) {
+		ERRMSG("Can't get the number of PT_LOAD.\n");
+		return FALSE;
+	}
+
+	/*
+	 * The below file information will be used as /proc/vmcore.
+	 */
+	fd_memory   = fd;
+	name_memory = filename;
+
+	pt_loads = calloc(sizeof(struct pt_load_segment), num_pt_loads);
+	if (pt_loads == NULL) {
+		ERRMSG("Can't allocate memory for the PT_LOAD. %s\n",
+		    strerror(errno));
+		return FALSE;
+	}
+	for (i = 0, j = 0; i < phnum; i++) {
+		if (!get_phdr_memory(i, &phdr))
+			return FALSE;
+
+		if (phdr.p_type != PT_LOAD)
+			continue;
+
+		if (j >= num_pt_loads)
+			return FALSE;
+		if(!dump_Elf_load(&phdr, j))
+			return FALSE;
+		j++;
+	}
+
+	return TRUE;
+}
+
+
 /*
  * Get ELF information about /proc/vmcore.
  */
elf_info.h +1 -0
index 801faff..263d993 100644
@@ -44,6 +44,7 @@ int get_elf64_ehdr(int fd, char *filename, Elf64_Ehdr *ehdr);
 int get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr);
 int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
+int get_elf_loads(int fd, char *filename);
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.c +18 -0
index 44cf26f..8c8ca91 100644
@@ -682,6 +682,9 @@ get_kernel_version(char *release)
 	long maj, min, rel;
 	char *start, *end;
 
+	if (info->kernel_version)
+		return info->kernel_version;
+
 	/*
 	 * This method checks that vmlinux and vmcore are same kernel version.
 	 */
@@ -706,6 +709,7 @@ get_kernel_version(char *release)
 		MSG("The kernel version is not supported.\n");
 		MSG("The created dumpfile may be incomplete.\n");
 	}
+
 	return version;
 }
 
@@ -9125,6 +9129,20 @@ int is_crashkernel_mem_reserved(void)
         return !!crash_reserved_mem_nr;
 }
 
+static int get_page_offset()
+{
+	struct utsname utsname;
+	if (uname(&utsname)) {
+		ERRMSG("Cannot get name and information about current kernel : %s", strerror(errno));
+		return FALSE;
+	}
+
+	info->kernel_version = get_kernel_version(utsname.release);
+	get_versiondep_info();
+
+	return TRUE;
+}
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#5 lore raw

Re: [PATCH v4 3/8] preparation functions for parsing vmcoreinfo

In reply to Baoquan He

Sorry, forgot updating the git log.
Has posted a new one, please review v5.

Quoted 134 lines · Baoquan He

Add 2 preparation functions get_elf_loads and get_page_offset, later they will be needed for parsing vmcoreinfo.

Meanwhile since get_kernel_version need be called to get page_offset before initial() in mem_usage code flow, and surely it will be called inside initial() again. Add a static variable to avoid this duplicate calling.

---
 elf_info.c     | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |  1 +
 makedumpfile.c | 18 ++++++++++++++++++
 3 files changed, 69 insertions(+)
elf_info.c +50 -0
index b277f69..69d3fdb 100644
@@ -681,6 +681,56 @@ get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr)
 	return TRUE;
 }
 
+int
+get_elf_loads(int fd, char *filename)
+{
+	int i, j, phnum, elf_format;
+	Elf64_Phdr phdr;
+
+	/*
+	 * Check ELF64 or ELF32.
+	 */
+	elf_format = check_elf_format(fd, filename, &phnum, &num_pt_loads);
+	if (elf_format == ELF64)
+		flags_memory |= MEMORY_ELF64;
+	else if (elf_format != ELF32)
+		return FALSE;
+
+	if (!num_pt_loads) {
+		ERRMSG("Can't get the number of PT_LOAD.\n");
+		return FALSE;
+	}
+
+	/*
+	 * The below file information will be used as /proc/vmcore.
+	 */
+	fd_memory   = fd;
+	name_memory = filename;
+
+	pt_loads = calloc(sizeof(struct pt_load_segment), num_pt_loads);
+	if (pt_loads == NULL) {
+		ERRMSG("Can't allocate memory for the PT_LOAD. %s\n",
+		    strerror(errno));
+		return FALSE;
+	}
+	for (i = 0, j = 0; i < phnum; i++) {
+		if (!get_phdr_memory(i, &phdr))
+			return FALSE;
+
+		if (phdr.p_type != PT_LOAD)
+			continue;
+
+		if (j >= num_pt_loads)
+			return FALSE;
+		if(!dump_Elf_load(&phdr, j))
+			return FALSE;
+		j++;
+	}
+
+	return TRUE;
+}
+
+
 /*
  * Get ELF information about /proc/vmcore.
  */
elf_info.h +1 -0
index 801faff..263d993 100644
@@ -44,6 +44,7 @@ int get_elf64_ehdr(int fd, char *filename, Elf64_Ehdr *ehdr);
 int get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr);
 int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
+int get_elf_loads(int fd, char *filename);
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.c +18 -0
index 44cf26f..8c8ca91 100644
@@ -682,6 +682,9 @@ get_kernel_version(char *release)
 	long maj, min, rel;
 	char *start, *end;
 
+	if (info->kernel_version)
+		return info->kernel_version;
+
 	/*
 	 * This method checks that vmlinux and vmcore are same kernel version.
 	 */
@@ -706,6 +709,7 @@ get_kernel_version(char *release)
 		MSG("The kernel version is not supported.\n");
 		MSG("The created dumpfile may be incomplete.\n");
 	}
+
 	return version;
 }
 
@@ -9125,6 +9129,20 @@ int is_crashkernel_mem_reserved(void)
         return !!crash_reserved_mem_nr;
 }
 
+static int get_page_offset()
+{
+	struct utsname utsname;
+	if (uname(&utsname)) {
+		ERRMSG("Cannot get name and information about current kernel : %s", strerror(errno));
+		return FALSE;
+	}
+
+	info->kernel_version = get_kernel_version(utsname.release);
+	get_versiondep_info();
+
+	return TRUE;
+}
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
Signature · 1 line
1.8.5.3

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#6 lore raw

PATCH v4 4/8 set vmcoreinfo for kcore

In reply to Baoquan He

In vmcore dumping, note program of vmcoreinfo is set in elf header of /proc/vmcore. In 1st kernel, the vmcoreinfo is also needed for kcore analyzing. So in this patch information of vmcoreinfo is parsed and set in offset_vmcoreinfo and size_vmcoreinfo.

---
 elf_info.c     | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |  1 +
 makedumpfile.c | 33 +++++++++++++++++++++++++++++++++
 3 files changed, 81 insertions(+)
elf_info.c +47 -0
index 69d3fdb..d50124d 100644
@@ -395,6 +395,53 @@ get_pt_note_info(void)
 	return TRUE;
 }
 
+#define UNINITIALIZED  ((ulong)(-1))
+int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len)
+{
+	int i;
+	ulong kvaddr;
+	off_t offset;
+	char note[MAX_SIZE_NHDR];
+	int size_desc;
+	off_t offset_desc;
+
+	offset = UNINITIALIZED;
+	kvaddr = (ulong)vmcoreinfo_addr | PAGE_OFFSET;
+
+	for (i = 0; i < num_pt_loads; ++i) {
+		struct pt_load_segment *p = &pt_loads[i];
+		if ((kvaddr >= p->virt_start) && (kvaddr < p->virt_end)) {
+			offset = (off_t)(kvaddr - p->virt_start) +
+			(off_t)p->file_offset;
+			break;
+		}
+	}
+
+	if (offset == UNINITIALIZED){
+		ERRMSG("Can't get the offset of VMCOREINFO(%s). %s\n",
+		    name_memory, strerror(errno));
+		return FALSE;
+	}
+
+        if (lseek(fd_memory, offset, SEEK_SET) != offset){
+		ERRMSG("Can't seek the dump memory(%s). %s\n",
+		    name_memory, strerror(errno));
+		return FALSE;
+	}
+
+	if (read(fd_memory, note, MAX_SIZE_NHDR) != MAX_SIZE_NHDR){
+		ERRMSG("Can't read the dump memory(%s). %s\n",
+		    name_memory, strerror(errno));
+		return FALSE;
+	}
+
+	size_desc   = note_descsz(note);
+	offset_desc = offset + offset_note_desc(note);
+
+	set_vmcoreinfo(offset_desc, size_desc);
+
+	return TRUE;
+}
 
 /*
  * External functions.
elf_info.h +1 -0
index 263d993..3ce0138 100644
@@ -45,6 +45,7 @@ int get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr);
 int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
 int get_elf_loads(int fd, char *filename);
+int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len);
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.c +33 -0
index 8c8ca91..d43d02d 100644
@@ -9143,6 +9143,39 @@ static int get_page_offset()
 	return TRUE;
 }
 
+
+/* Returns the physical address of start of crash notes buffer for a kernel. */
+static int get_sys_kernel_vmcoreinfo(uint64_t *addr, uint64_t *len)
+{
+	char line[BUFSIZE_FGETS];
+	int count;
+	FILE *fp;
+	unsigned long long temp, temp2;
+
+	*addr = 0;
+	*len = 0;
+
+	if (!(fp = fopen("/sys/kernel/vmcoreinfo", "r")))
+		return FALSE;
+
+	if (!fgets(line, sizeof(line), fp)) {
+		ERRMSG("Cannot parse %s: %s, fgets failed.\n", "/sys/kernel/vmcoreinfo", strerror(errno));
+		return FALSE;
+	}
+	count = sscanf(line, "%Lx %Lx", &temp, &temp2);
+	if (count != 2){
+		ERRMSG("Cannot parse %s: %s, sscanf failed.\n", "/sys/kernel/vmcoreinfo", strerror(errno));
+		return FALSE;
+	}
+
+	*addr = (uint64_t) temp;
+	*len = (uint64_t) temp2;
+
+	fclose(fp);
+	return TRUE;
+}
+
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#7 lore raw

PATCH v4 5/8 prepare the dump loads for kcore analysis

In reply to Baoquan He

In kcore, only "System Ram" and "kernel text" program segments are needed. And to be more precise, exclude the crashkernel memory range.

---
 arch/x86_64.c  |   6 +--
 elf_info.c     | 140 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |   1 +
 makedumpfile.h |   8 ++++
 4 files changed, 152 insertions(+), 3 deletions(-)
arch/x86_64.c +3 -3
index 771d457..4788f55 100644
@@ -20,7 +20,7 @@
 #include "../makedumpfile.h"
 
 int
-is_vmalloc_addr(ulong vaddr)
+is_vmalloc_addr_x86_64(ulong vaddr)
 {
 	/*
 	 *  vmalloc, virtual memmap, and module space as VMALLOC space.
@@ -56,7 +56,7 @@ get_phys_base_x86_64(void)
 
 	for (i = 0; get_pt_load(i, &phys_start, NULL, &virt_start, NULL); i++) {
 		if ((virt_start >= __START_KERNEL_map) &&
-		    !(is_vmalloc_addr(virt_start))) {
+		    !(is_vmalloc_addr_x86_64(virt_start))) {
 
 			info->phys_base = phys_start -
 			    (virt_start & ~(__START_KERNEL_map));
@@ -281,7 +281,7 @@ vaddr_to_paddr_x86_64(unsigned long vaddr)
 	else
 		phys_base = 0;
 
-	if (is_vmalloc_addr(vaddr)) {
+	if (is_vmalloc_addr_x86_64(vaddr)) {
 		if ((paddr = vtop4_x86_64(vaddr)) == NOT_PADDR) {
 			ERRMSG("Can't convert a virtual address(%lx) to " \
 			    "physical address.\n", vaddr);
elf_info.c +140 -0

146 lines not on this page · raw message

elf_info.h +1 -0
index 3ce0138..ba27fdf 100644
@@ -46,6 +46,7 @@ int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
 int get_elf_loads(int fd, char *filename);
 int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len);
+int get_kcore_dump_loads();
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.h +8 -0
index ed584af..e83bd95 100644
@@ -765,6 +765,7 @@ unsigned long long vaddr_to_paddr_arm(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif /* arm */
 
 #ifdef __x86__
@@ -775,9 +776,11 @@ unsigned long long vaddr_to_paddr_x86(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif /* x86 */
 
 #ifdef __x86_64__
+int is_vmalloc_addr_x86_64(ulong vaddr);
 int get_phys_base_x86_64(void);
 int get_machdep_info_x86_64(void);
 int get_versiondep_info_x86_64(void);
@@ -786,6 +789,7 @@ unsigned long long vaddr_to_paddr_x86_64(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
+#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -796,6 +800,7 @@ unsigned long long vaddr_to_paddr_ppc64(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -805,6 +810,7 @@ unsigned long long vaddr_to_paddr_ppc(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
@@ -814,6 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -825,6 +832,7 @@ unsigned long long vaddr_to_paddr_ia64(unsigned long vaddr);
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#8 lore raw

PATCH v4 6/8 introduce a function exclude_zero_pages_cyclic()

In reply to Baoquan He

Introduced a new function exclude_zero_pages_cyclic(), this will exclude and counting zero pages. Calling it in get_num_dumpable_cyclic can get the number of zero pages.

---
 makedumpfile.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
makedumpfile.c +42 -0
index d43d02d..a511179 100644
@@ -4582,6 +4582,45 @@ exclude_zero_pages(void)
 	return TRUE;
 }
 
+int
+exclude_zero_pages_cyclic(struct cycle *cycle)
+{
+	mdf_pfn_t pfn;
+	unsigned long long paddr;
+	unsigned char buf[info->page_size];
+
+	for (pfn = cycle->start_pfn, paddr = pfn_to_paddr(pfn); pfn < cycle->end_pfn;
+	    pfn++, paddr += info->page_size) {
+
+		if (!is_in_segs(paddr))
+			continue;
+
+		if (!is_dumpable_cyclic(info->partial_bitmap2, pfn, cycle))
+			continue;
+
+		if (is_xen_memory()) {
+			if (!readmem(MADDR_XEN, paddr, buf, info->page_size)) {
+				ERRMSG("Can't get the page data(pfn:%llx, max_mapnr:%llx).\n",
+				    pfn, info->max_mapnr);
+				return FALSE;
+			}
+		} else {
+			if (!readmem(PADDR, paddr, buf, info->page_size)) {
+				ERRMSG("Can't get the page data(pfn:%llx, max_mapnr:%llx).\n",
+				    pfn, info->max_mapnr);
+				return FALSE;
+			}
+		}
+		if (is_zero_page(buf, info->page_size)) {
+			if (clear_bit_on_2nd_bitmap(pfn, cycle))
+				pfn_zero++;
+		}
+	}
+
+	return TRUE;
+}
+
+
 static int
 initialize_2nd_bitmap_cyclic(struct cycle *cycle)
 {
@@ -5662,6 +5701,9 @@ get_num_dumpable_cyclic(void)
 		if (!exclude_unnecessary_pages_cyclic(&cycle))
 			return FALSE;
 
+		if (info->flag_mem_usage)
+			exclude_zero_pages_cyclic(&cycle);
+
 		for(pfn=cycle.start_pfn; pfn<cycle.end_pfn; pfn++)
 			if (is_dumpable_cyclic(info->partial_bitmap2, pfn, &cycle))
 				num_dumpable++;
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#9 lore raw

PATCH v4 7/8 implement a function to print the memory usage

In reply to Baoquan He

Introduce print_mem_usage to print the result of analysis of /proc/kcore. The page number of memory in different use are printed.

---
 makedumpfile.c | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
makedumpfile.c +35 -0
index a511179..c8d1299 100644
@@ -7939,6 +7939,41 @@ print_report(void)
 	REPORT_MSG("\n");
 }
 
+static void
+print_mem_usage(void)
+{
+	mdf_pfn_t pfn_original, pfn_excluded, shrinking;
+
+	/*
+	* /proc/vmcore doesn't contain the memory hole area.
+	*/
+	pfn_original = info->max_mapnr - pfn_memhole;
+
+	pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private
+	    + pfn_user + pfn_free + pfn_hwpoison;
+	shrinking = (pfn_original - pfn_excluded) * 100;
+	shrinking = shrinking / pfn_original;
+
+	MSG("\n");
+	MSG("Page number of memory in different use\n");
+	MSG("--------------------------------------------------\n");
+	MSG("TYPE		PAGES			EXCLUDABLE	DESCRIPTION\n");
+
+	MSG("ZERO		%-16llu	yes		Pages filled with zero\n", pfn_zero);
+	MSG("CACHE		%-16llu	yes		Cache pages\n", pfn_cache);
+	MSG("CACHE_PRIVATE	%-16llu	yes		Cache pages + private\n",
+	    pfn_cache_private);
+	MSG("USER		%-16llu	yes		User process pages\n", pfn_user);
+	MSG("FREE		%-16llu	yes		Free pages\n", pfn_free);
+	MSG("KERN_DATA	%-16llu	no		Dumpable kernel data \n",
+	    pfn_original - pfn_excluded);
+
+	MSG("\n");
+
+	MSG("Total pages on system:	%-16llu\n", pfn_original);
+	MSG("\n");
+}
+
 int
 writeout_dumpfile(void)
 {
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#10 lore raw

PATCH v4 8/8 add a new interface to show the memory usage of 1st kernel

In reply to Baoquan He

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patch, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

The print is like below:
->$ ./makedumpfile --mem-usage /proc/kcore
Excluding unnecessary pages : [100.0 %] |

Page number of memory in different use
--------------------------------------------------
TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 0 yes Pages filled with zero
CACHE 562006 yes Cache pages
CACHE_PRIVATE 353502 yes Cache pages + private
USER 225780 yes User process pages
FREE 2761884 yes Free pages
KERN_DATA 235873 no Dumpable kernel data

Total pages on system: 4139045

---
 makedumpfile.8 | 17 +++++++++++++
 makedumpfile.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |  2 ++
 print_info.c   |  8 +++++++
 4 files changed, 103 insertions(+)
makedumpfile.8 +17 -0
index 25fe74e..64abbc7 100644
@@ -532,6 +532,23 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
 # makedumpfile \-\-dump-dmesg -x vmlinux /proc/vmcore dmesgfile
 .br
 
+
+.TP
+\fB\-\-mem-usage\fR
+This option is used to show the page numbers of current system in different
+use. It should be executed in 1st kernel. By the help of this, user can know
+how many pages is dumpable when different dump_level is specified. It analyzes
+the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding
+the crashkernel range, then calculates the page number of different kind per
+vmcoreinfo. So currently /proc/kcore need be specified explicitly.
+
+.br
+.B Example:
+.br
+# makedumpfile \-\-mem-usage /proc/kcore
+.br
+
+
 .TP
 \fB\-\-diskset=VMCORE\fR
 Specify multiple \fIVMCORE\fRs created on sadump diskset configuration
makedumpfile.c +76 -0

112 lines not on this page · raw message

makedumpfile.h +2 -0
index e83bd95..6ee4cae 100644
@@ -978,6 +978,7 @@ struct DumpInfo {
 	int		flag_force;	     /* overwrite existing stuff */
 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
+	int             flag_mem_usage;  /*show the page number of memory in different use*/
 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
 	int		flag_nospace;	     /* the flag of "No space on device" error */
 	int		flag_vmemmap;        /* kernel supports vmemmap address space */
@@ -1867,6 +1868,7 @@ struct elf_prstatus {
 #define OPT_CYCLIC_BUFFER       OPT_START+11
 #define OPT_EPPIC               OPT_START+12
 #define OPT_NON_MMAP            OPT_START+13
+#define OPT_MEM_USAGE            OPT_START+14
 
 /*
  * Function Prototype.
print_info.c +8 -0
index 7592690..29db918 100644
@@ -264,6 +264,14 @@ print_usage(void)
 	MSG("      LOGFILE. If a VMCORE does not contain VMCOREINFO for dmesg, it is\n");
 	MSG("      necessary to specfiy [-x VMLINUX] or [-i VMCOREINFO].\n");
 	MSG("\n");
+	MSG("  [--mem-usage]:\n");
+	MSG("      This option is used to show the page numbers of current system in different\n");
+	MSG("      use. It should be executed in 1st kernel. By the help of this, user can know\n");
+	MSG("      how many pages is dumpable when different dump_level is specified. It analyzes\n");
+	MSG("      the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding\n");
+	MSG("      the crashkernel range, then calculates the page number of different kind per\n");
+	MSG("      vmcoreinfo. So currently /proc/kcore need be specified explicitly.\n");
+	MSG("\n");
 	MSG("  [-D]:\n");
 	MSG("      Print debugging message.\n");
 	MSG("\n");
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#11 lore raw

PATCH v5 8/8 add a new interface to show the memory usage of 1st kernel

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patch, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

The print is like below:

~$ ./makedumpfile --mem-usage /proc/kcore
The kernel version is not supported.
The created dumpfile may be incomplete.
Excluding unnecessary pages : [100.0 %] |

Page number of memory in different use
--------------------------------------------------
TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 29149 yes Pages filled with zero
CACHE 171288 yes Cache pages
CACHE_PRIVATE 12051 yes Cache pages + private
USER 31816 yes User process pages
FREE 3700059 yes Free pages
KERN_DATA 105305 no Dumpable kernel data

Total pages on system: 4049668

Showing page number of memory in different use successfully.

makedumpfile Completed.

---
 makedumpfile.8 | 17 +++++++++++++
 makedumpfile.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |  2 ++
 print_info.c   |  8 +++++++
 4 files changed, 103 insertions(+)
makedumpfile.8 +17 -0
index 25fe74e..64abbc7 100644
@@ -532,6 +532,23 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
 # makedumpfile \-\-dump-dmesg -x vmlinux /proc/vmcore dmesgfile
 .br
 
+
+.TP
+\fB\-\-mem-usage\fR
+This option is used to show the page numbers of current system in different
+use. It should be executed in 1st kernel. By the help of this, user can know
+how many pages is dumpable when different dump_level is specified. It analyzes
+the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding
+the crashkernel range, then calculates the page number of different kind per
+vmcoreinfo. So currently /proc/kcore need be specified explicitly.
+
+.br
+.B Example:
+.br
+# makedumpfile \-\-mem-usage /proc/kcore
+.br
+
+
 .TP
 \fB\-\-diskset=VMCORE\fR
 Specify multiple \fIVMCORE\fRs created on sadump diskset configuration
makedumpfile.c +76 -0

112 lines not on this page · raw message

makedumpfile.h +2 -0
index e83bd95..6ee4cae 100644
@@ -978,6 +978,7 @@ struct DumpInfo {
 	int		flag_force;	     /* overwrite existing stuff */
 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
+	int             flag_mem_usage;  /*show the page number of memory in different use*/
 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
 	int		flag_nospace;	     /* the flag of "No space on device" error */
 	int		flag_vmemmap;        /* kernel supports vmemmap address space */
@@ -1867,6 +1868,7 @@ struct elf_prstatus {
 #define OPT_CYCLIC_BUFFER       OPT_START+11
 #define OPT_EPPIC               OPT_START+12
 #define OPT_NON_MMAP            OPT_START+13
+#define OPT_MEM_USAGE            OPT_START+14
 
 /*
  * Function Prototype.
print_info.c +8 -0
index 7592690..29db918 100644
@@ -264,6 +264,14 @@ print_usage(void)
 	MSG("      LOGFILE. If a VMCORE does not contain VMCOREINFO for dmesg, it is\n");
 	MSG("      necessary to specfiy [-x VMLINUX] or [-i VMCOREINFO].\n");
 	MSG("\n");
+	MSG("  [--mem-usage]:\n");
+	MSG("      This option is used to show the page numbers of current system in different\n");
+	MSG("      use. It should be executed in 1st kernel. By the help of this, user can know\n");
+	MSG("      how many pages is dumpable when different dump_level is specified. It analyzes\n");
+	MSG("      the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding\n");
+	MSG("      the crashkernel range, then calculates the page number of different kind per\n");
+	MSG("      vmcoreinfo. So currently /proc/kcore need be specified explicitly.\n");
+	MSG("\n");
 	MSG("  [-D]:\n");
 	MSG("      Print debugging message.\n");
 	MSG("\n");
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#12 lore raw
Quoted 12 lines · Baoquan He

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patch, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

Quoted 5 lines

The print is like below:

~$ ./makedumpfile --mem-usage /proc/kcore
The kernel version is not supported.
The created dumpfile may be incomplete.

I still think that above messages should go. It does not make any sense with --mem-usage. No dumpfile is being created here.

Quoted 6 lines

Excluding unnecessary pages : [100.0 %] |

Page number of memory in different use
--------------------------------------------------

I think above header can completely go away. It kind of looks odd.

Quoted 11 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 29149 yes Pages filled with zero
CACHE 171288 yes Cache pages
CACHE_PRIVATE 12051 yes Cache pages + private
USER 31816 yes User process pages
FREE 3700059 yes Free pages
KERN_DATA 105305 no Dumpable kernel data

Total pages on system: 4049668

Showing page number of memory in different use successfully.

I think we don't need above line. I am not even sure what does it mean.

Quoted 2 lines

makedumpfile Completed.

We don't need above line either.

In output we should just show the actual table. If there is an error, we should output error and exit (no table).

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#13 lore raw
Quoted 10 lines · Vivek Goyal
Quoted 5 lines · Baoquan He

The print is like below:

~$ ./makedumpfile --mem-usage /proc/kcore
The kernel version is not supported.
The created dumpfile may be incomplete.

I still think that above messages should go. It does not make any sense with --mem-usage. No dumpfile is being created here.

This is printed by function get_kernel_version(), it will be used by other makedumpfile main code flows too. This message is used to warn users that users' using kernel may not be tested suffciently. I think those tests are mainly taken by Atsushi. When he finished sufficient tests on a new version of kernel, the LATEST_VERSION will be changed to be a larger value. I am fine with this message, since it won't occur on our distribution, anyway kernel for distribution are all tested. And if LATEST_VERSION is older than our distribution kernel, maintainer may be pushed to take tests and update this value. So don't worry about this message.

But I would like to change the message like below to clean up misunderstanding if Atsushi doesn't object, this mostly occur when people working on latest upstream kernel.
-------------------------------------------------
The kernel version is not supported.
The makedumpfile operation may not be successful.
--------------------------------------------------

Quoted 9 lines
Quoted 7 lines

Excluding unnecessary pages : [100.0 %] |

Page number of memory in different use
--------------------------------------------------

I think above header can completely go away. It kind of looks odd.

OK, will remove it.

Quoted 9 lines
Quoted 9 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 29149 yes Pages filled with zero
CACHE 171288 yes Cache pages
CACHE_PRIVATE 12051 yes Cache pages + private
USER 31816 yes User process pages
FREE 3700059 yes Free pages
KERN_DATA 105305 no Dumpable kernel data

Quoted 5 lines
Quoted 3 lines

Total pages on system: 4049668

Showing page number of memory in different use successfully.

I think we don't need above line. I am not even sure what does it mean.

Will remove it.

Quoted 4 lines
Quoted 2 lines

makedumpfile Completed.

We don't need above line either.

This is also a public message printing, means if makedumpfile operation is successful or not. Maybe I can add a check like below:

@@ -9546,7 +9553,7 @@ main(int argc, char *argv[])

        retcd = COMPLETED;
 out:
        MSG("\n");
-       if (retcd == COMPLETED)
+       if ((retcd == COMPLETED) && (!info->flag_mem_usage))
                MSG("makedumpfile Completed.\n");
        else
                MSG("makedumpfile Failed.\n");

Hi Atsushi,

How do you think about this? And for the excluding progress indication too, add a check that if it's mem-usage handling, not printing.

Thanks
Baoquan

Quoted 5 lines

In output we should just show the actual table. If there is an error, we should output error and exit (no table).

Quoted 7 lines

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#14 lore raw
Quoted 23 lines · Baoquan He
Quoted 10 lines · Vivek Goyal
Quoted 5 lines · Baoquan He

The print is like below:

~$ ./makedumpfile --mem-usage /proc/kcore
The kernel version is not supported.
The created dumpfile may be incomplete.

I still think that above messages should go. It does not make any sense with --mem-usage. No dumpfile is being created here.

This is printed by function get_kernel_version(), it will be used by other makedumpfile main code flows too. This message is used to warn users that users' using kernel may not be tested suffciently. I think those tests are mainly taken by Atsushi. When he finished sufficient tests on a new version of kernel, the LATEST_VERSION will be changed to be a larger value. I am fine with this message, since it won't occur on our distribution, anyway kernel for distribution are all tested. And if LATEST_VERSION is older than our distribution kernel, maintainer may be pushed to take tests and update this value. So don't worry about this message.

Quoted 7 lines

But I would like to change the message like below to clean up misunderstanding if Atsushi doesn't object, this mostly occur when people working on latest upstream kernel.
-------------------------------------------------
The kernel version is not supported.
The makedumpfile operation may not be successful.
--------------------------------------------------

I think you might have to create a gloabl variable somewhere which tells what mode makedumpfile is operating in. And messages might have to be changed accordingly.

In this case, I think a single line of following message might be sufficient.

"Warning: This kernel version is not supported."

[..]

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

makedumpfile Completed.

We don't need above line either.

This is also a public message printing, means if makedumpfile operation is successful or not. Maybe I can add a check like below:

Generally commands return code either "0" or "1" to designate success or failure. And they don't display to user whether they succeeded or failed.

Review main(int argc, char *argv[]) · line 9559

Quoted 10 lines
(quoted diff) +1 -1
@@ -9546,7 +9553,7 @@ main(int argc, char *argv[])
        retcd = COMPLETED;
 out:
        MSG("\n");
-       if (retcd == COMPLETED)
+       if ((retcd == COMPLETED) && (!info->flag_mem_usage))
                MSG("makedumpfile Completed.\n");
        else
                MSG("makedumpfile Failed.\n");

I think this will work. This will display failure message but in case of success only table is displayed.

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#15 lore raw
In reply to Baoquan He

Hello Baoquan,

Quoted 23 lines · Vivek Goyal
Quoted 10 lines · Baoquan He
Quoted 6 lines

The print is like below:

~$ ./makedumpfile --mem-usage /proc/kcore
The kernel version is not supported.
The created dumpfile may be incomplete.

I still think that above messages should go. It does not make any sense with --mem-usage. No dumpfile is being created here.

This is printed by function get_kernel_version(), it will be used by other makedumpfile main code flows too. This message is used to warn users that users' using kernel may not be tested suffciently. I think those tests are mainly taken by Atsushi. When he finished sufficient tests on a new version of kernel, the LATEST_VERSION will be changed to be a larger value. I am fine with this message, since it won't occur on our distribution, anyway kernel for distribution are all tested. And if LATEST_VERSION is older than our distribution kernel, maintainer may be pushed to take tests and update this value. So don't worry about this message.

Quoted 7 lines

But I would like to change the message like below to clean up misunderstanding if Atsushi doesn't object, this mostly occur when people working on latest upstream kernel.
-------------------------------------------------
The kernel version is not supported.
The makedumpfile operation may not be successful.
--------------------------------------------------

Good change, let's do it. --mem-usage doesn't create a dumpfile but the behavior still depend on the kernel version, this message is meaningful. Also, as Vivek said, the latter line can be changed based on the operating mode, but I don't think it's necessary.

Quoted 41 lines
Quoted 9 lines
Quoted 7 lines

Excluding unnecessary pages : [100.0 %] |

Page number of memory in different use
--------------------------------------------------

I think above header can completely go away. It kind of looks odd.

OK, will remove it.

Quoted 9 lines
Quoted 9 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 29149 yes Pages filled with zero
CACHE 171288 yes Cache pages
CACHE_PRIVATE 12051 yes Cache pages + private
USER 31816 yes User process pages
FREE 3700059 yes Free pages
KERN_DATA 105305 no Dumpable kernel data

Quoted 5 lines
Quoted 3 lines

Total pages on system: 4049668

Showing page number of memory in different use successfully.

I think we don't need above line. I am not even sure what does it mean.

Will remove it.

Quoted 4 lines
Quoted 2 lines

makedumpfile Completed.

We don't need above line either.

This is also a public message printing, means if makedumpfile operation is successful or not. Maybe I can add a check like below:

(quoted diff) +0 -0
@@ -9546,7 +9553,7 @@ main(int argc, char *argv[])
       retcd = COMPLETED;

out:

       MSG("\n");

Review

Quoted 5 lines
(quoted diff) +1 -1
-       if (retcd == COMPLETED)
+       if ((retcd == COMPLETED) && (!info->flag_mem_usage))
               MSG("makedumpfile Completed.\n");
       else
               MSG("makedumpfile Failed.\n");

This code always show "makedumpfile Failed" for --mem-usage :-)

Anyway,

Quoted 4 lines

Hi Atsushi,

How do you think about this? And for the excluding progress indication too, add a check that if it's mem-usage handling, not printing.

I don't think we need to change the both messages, they indicate the actual progress and result even in the mem-usage mode.

Thanks
Atsushi Kumagai

Quoted 15 lines

Thanks
Baoquan

Quoted 12 lines

In output we should just show the actual table. If there is an error, we should output error and exit (no table).

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#16 lore raw
Quoted 19 lines · Atsushi Kumagai
Quoted 11 lines

This is printed by function get_kernel_version(), it will be used by other makedumpfile main code flows too. This message is used to warn users that users' using kernel may not be tested suffciently. I think those tests are mainly taken by Atsushi. When he finished sufficient tests on a new version of kernel, the LATEST_VERSION will be changed to be a larger value. I am fine with this message, since it won't occur on our distribution, anyway kernel for distribution are all tested. And if LATEST_VERSION is older than our distribution kernel, maintainer may be pushed to take tests and update this value. So don't worry about this message.

Quoted 7 lines

But I would like to change the message like below to clean up misunderstanding if Atsushi doesn't object, this mostly occur when people working on latest upstream kernel.
-------------------------------------------------
The kernel version is not supported.
The makedumpfile operation may not be successful.
--------------------------------------------------

Quoted 5 lines

Good change, let's do it. --mem-usage doesn't create a dumpfile but the behavior still depend on the kernel version, this message is meaningful. Also, as Vivek said, the latter line can be changed based on the operating mode, but I don't think it's necessary.

OK, I would like to change the printing message for now. If necessary, I prefer to post a independent cleanup patch for message printing.

Quoted 20 lines
Quoted 10 lines
Quoted 2 lines

This is also a public message printing, means if makedumpfile operation is successful or not. Maybe I can add a check like below:

(quoted diff) +0 -0
@@ -9546,7 +9553,7 @@ main(int argc, char *argv[])
       retcd = COMPLETED;

out:

       MSG("\n");

Review

Quoted 5 lines
(quoted diff) +1 -1
-       if (retcd == COMPLETED)
+       if ((retcd == COMPLETED) && (!info->flag_mem_usage))
               MSG("makedumpfile Completed.\n");
       else
               MSG("makedumpfile Failed.\n");

This code always show "makedumpfile Failed" for --mem-usage :-)

Anyway,

Quoted 7 lines
Quoted 4 lines

Hi Atsushi,

How do you think about this? And for the excluding progress indication too, add a check that if it's mem-usage handling, not printing.

I don't think we need to change the both messages, they indicate the actual progress and result even in the mem-usage mode.

OK, so just keep it in this patchset.

Thanks for suggestion.

Quoted 1 line

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#17 lore raw
In reply to Baoquan He

Forgot updating the git log with v4 change.

Have posted new one, please review v5.

Quoted 228 lines · Baoquan He

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patch, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

The print is like below:
->$ ./makedumpfile --mem-usage /proc/kcore
Excluding unnecessary pages : [100.0 %] |

Page number of memory in different use
--------------------------------------------------
TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 0 yes Pages filled with zero
CACHE 562006 yes Cache pages
CACHE_PRIVATE 353502 yes Cache pages + private
USER 225780 yes User process pages
FREE 2761884 yes Free pages
KERN_DATA 235873 no Dumpable kernel data

Total pages on system: 4139045

---
 makedumpfile.8 | 17 +++++++++++++
 makedumpfile.c | 76 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |  2 ++
 print_info.c   |  8 +++++++
 4 files changed, 103 insertions(+)
makedumpfile.8 +17 -0
index 25fe74e..64abbc7 100644
@@ -532,6 +532,23 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
 # makedumpfile \-\-dump-dmesg -x vmlinux /proc/vmcore dmesgfile
 .br
 
+
+.TP
+\fB\-\-mem-usage\fR
+This option is used to show the page numbers of current system in different
+use. It should be executed in 1st kernel. By the help of this, user can know
+how many pages is dumpable when different dump_level is specified. It analyzes
+the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding
+the crashkernel range, then calculates the page number of different kind per
+vmcoreinfo. So currently /proc/kcore need be specified explicitly.
+
+.br
+.B Example:
+.br
+# makedumpfile \-\-mem-usage /proc/kcore
+.br
+
+
 .TP
 \fB\-\-diskset=VMCORE\fR
 Specify multiple \fIVMCORE\fRs created on sadump diskset configuration
makedumpfile.c +76 -0
index c8d1299..87cca71 100644
@@ -7955,6 +7955,7 @@ print_mem_usage(void)
 	shrinking = shrinking / pfn_original;
 
 	MSG("\n");
+	MSG("\n");
 	MSG("Page number of memory in different use\n");
 	MSG("--------------------------------------------------\n");
 	MSG("TYPE		PAGES			EXCLUDABLE	DESCRIPTION\n");
@@ -9008,6 +9009,13 @@ check_param_for_creating_dumpfile(int argc, char *argv[])
 		 */
 		info->name_memory   = argv[optind];
 
+	} else if ((argc == optind + 1) && info->flag_mem_usage) {
+		/*
+		* Parameter for showing the page number of memory
+		* in different use from.
+		*/
+		info->name_memory   = argv[optind];
+
 	} else
 		return FALSE;
 
@@ -9252,6 +9260,58 @@ static int get_sys_kernel_vmcoreinfo(uint64_t *addr, uint64_t *len)
 	return TRUE;
 }
 
+int show_mem_usage(void)
+{
+        uint64_t vmcoreinfo_addr, vmcoreinfo_len;
+
+        if (!is_crashkernel_mem_reserved()) {
+                ERRMSG("No memory is reserved for crashkenrel!\n");
+                return FALSE;
+        }
+
+
+        if (!info->flag_cyclic)
+                info->flag_cyclic = TRUE;
+
+	info->dump_level = MAX_DUMP_LEVEL;
+
+        if (!get_page_offset())
+                return FALSE;
+
+        if (!open_dump_memory())
+                return FALSE;
+
+        if (!get_elf_loads(info->fd_memory, info->name_memory))
+                return FALSE;
+
+        if (!get_sys_kernel_vmcoreinfo(&vmcoreinfo_addr, &vmcoreinfo_len))
+                return FALSE;
+
+        if (!set_kcore_vmcoreinfo(vmcoreinfo_addr, vmcoreinfo_len))
+                return FALSE;
+
+        if (!get_kcore_dump_loads())
+                return FALSE;
+
+        if (!initial())
+                return FALSE;
+
+
+        if (!prepare_bitmap2_buffer_cyclic())
+                return FALSE;
+
+        info->num_dumpable = get_num_dumpable_cyclic();
+
+	free_bitmap2_buffer_cyclic();
+
+        print_mem_usage();
+
+        if (!close_files_for_creating_dumpfile())
+                return FALSE;
+
+        return TRUE;
+}
+
 
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
@@ -9269,6 +9329,7 @@ static struct option longopts[] = {
 	{"cyclic-buffer", required_argument, NULL, OPT_CYCLIC_BUFFER},
 	{"eppic", required_argument, NULL, OPT_EPPIC},
 	{"non-mmap", no_argument, NULL, OPT_NON_MMAP},
+	{"mem-usage", no_argument, NULL, OPT_MEM_USAGE},
 	{0, 0, 0, 0}
 };
 
@@ -9360,6 +9421,9 @@ main(int argc, char *argv[])
 		case OPT_DUMP_DMESG:
 			info->flag_dmesg = 1;
 			break;
+		case OPT_MEM_USAGE:
+                       info->flag_mem_usage = 1;
+                       break;
 		case OPT_COMPRESS_SNAPPY:
 			info->flag_compress = DUMP_DH_COMPRESSED_SNAPPY;
 			break;
@@ -9500,6 +9564,18 @@ main(int argc, char *argv[])
 
 		MSG("\n");
 		MSG("The dmesg log is saved to %s.\n", info->name_dumpfile);
+	} else if (info->flag_mem_usage) {
+		if (!check_param_for_creating_dumpfile(argc, argv)) {
+			MSG("Commandline parameter is invalid.\n");
+			MSG("Try `makedumpfile --help' for more information.\n");
+			goto out;
+		}
+
+		if (!show_mem_usage())
+			goto out;
+
+		MSG("\n");
+		MSG("Showing page number of memory in different use successfully.\n");
 	} else {
 		if (!check_param_for_creating_dumpfile(argc, argv)) {
 			MSG("Commandline parameter is invalid.\n");
makedumpfile.h +2 -0
index e83bd95..6ee4cae 100644
@@ -978,6 +978,7 @@ struct DumpInfo {
 	int		flag_force;	     /* overwrite existing stuff */
 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
+	int             flag_mem_usage;  /*show the page number of memory in different use*/
 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
 	int		flag_nospace;	     /* the flag of "No space on device" error */
 	int		flag_vmemmap;        /* kernel supports vmemmap address space */
@@ -1867,6 +1868,7 @@ struct elf_prstatus {
 #define OPT_CYCLIC_BUFFER       OPT_START+11
 #define OPT_EPPIC               OPT_START+12
 #define OPT_NON_MMAP            OPT_START+13
+#define OPT_MEM_USAGE            OPT_START+14
 
 /*
  * Function Prototype.
print_info.c +8 -0
index 7592690..29db918 100644
@@ -264,6 +264,14 @@ print_usage(void)
 	MSG("      LOGFILE. If a VMCORE does not contain VMCOREINFO for dmesg, it is\n");
 	MSG("      necessary to specfiy [-x VMLINUX] or [-i VMCOREINFO].\n");
 	MSG("\n");
+	MSG("  [--mem-usage]:\n");
+	MSG("      This option is used to show the page numbers of current system in different\n");
+	MSG("      use. It should be executed in 1st kernel. By the help of this, user can know\n");
+	MSG("      how many pages is dumpable when different dump_level is specified. It analyzes\n");
+	MSG("      the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding\n");
+	MSG("      the crashkernel range, then calculates the page number of different kind per\n");
+	MSG("      vmcoreinfo. So currently /proc/kcore need be specified explicitly.\n");
+	MSG("\n");
 	MSG("  [-D]:\n");
 	MSG("      Print debugging message.\n");
 	MSG("\n");
Signature · 1 line
1.8.5.3

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec