Skip to contentSkip to conversation

add a new interface to show the memory usage of 1st kernel

PATCH In review Baoquan He 4 participants 47 messages Sep 1, 2014 – Oct 31, 2014

Subsystems (2)
lore
Share & tools
lore

Baoquan He

author
#0 lore raw

v6 cover add a new interface to show the memory usage of 1st kernel

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patchset, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

Previouly, patchset v1 was posted. And patch 7/7 has a change in v2. So several changes are made in this v3 post per comments from Vivek and Atsushi.

[patch 2/8] functions to get crashkernel memory range v3->v4:
In old iomem_for_each_line(), it will call exit(1) if opening iomem failed. Atsushi suggested changing this to be consistent with other return value. So here change all the return value to be nr, then check it outside of iomem_for_each_line, and then decide how to act.

[patch 3/8] preparation functions for parsing vmcoreinfo
v1->v3:
Since get_kernel_version need be called to get page_offset before initial() in mem_usage code flow, and surely it will be called inside initial() again. Add a static variable to avoid this duplicate calling.
v3->v4:
Check the value of info->kernel_version, just return if it has been assigned to a value. This is suggested by Atsushi, far better than the static variable idea.
And replace replace get_versiondep_info_x86_64() with get_versiondep_info in get_page_offset().
v4->v5:
Update the git log.

[patch 4/8] set vmcoreinfo for kcore
v3->v4:
Change several places error messages which are the same in nearby handling.

[patch 5/8] prepare the dump loads for kcore analysis
v1->v3:
Fix the compiler warnings.
v3->v4:
Rename is_vmalloc_addr() to is_vmalloc_addr_x86_64() in arch/x86_64.c. And then define is_vmalloc_addr() for all other ARCHs, just set their value to be TRUE except of x86_64.

[patch 6/8] introduce-a-function-exclude_zero_pages_cyclic v3->v4:
Newly introduce a function exclude_zero_pages_cyclic(), and call it in get_num_dumpable_cyclic().

Besides, a strange thing happened when the new interface was tested on my local AMD machine. It always terminated and printed message:
"Program terminated with signal SIGKILL"
With gdb, it should be going in readmem() of exclude_zero_pages_cyclic, I still don't know why it happened.

[patch 7/8] implement a function to print the memory usage
v1->v3:
Adjust the printing content and format of dumpable page numbers per Vivek's comments.
v3->v6:
Slightly adjust the printing format and content.

[patch 8/8]
v1->v2:
Set info->dump_level=MAX_DUMP_LEVEL, with MAX_DUMP_LEVEL all kinds of memory can be calculated.
v2->v3:
Add the description of this feature into help message and man page.
v3->v4:
Continue adjusting the output message of show_mem_usage calling per Vivek's comments.
v4->v5:
Update the git log.
v5->v6:
Adjust the printing format which is related to dump files previously.

Baoquan He (8):
initialize pfn_memhole in get_num_dumpable_cyclic functions to get crashkernel memory range preparation functions for parsing vmcoreinfo
set vmcoreinfo for kcore
prepare the dump loads for kcore analysis introduce a function exclude_zero_pages_cyclic() implement a function to print the memory usage add a new interface to show the memory usage of 1st kernel

 arch/x86_64.c  |   6 +-
 elf_info.c     | 237 +++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |   3 +
 makedumpfile.8 |  17 ++++
 makedumpfile.c | 285 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 makedumpfile.h |  17 ++++
 print_info.c   |   8 ++
 7 files changed, 569 insertions(+), 4 deletions(-)
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#1 lore raw

PATCH v6 1/8 initialize pfn_memhole in get_num_dumpable_cyclic

This is a code bug. In initialize_2nd_bitmap_cyclic pfn_memhole is calculated, however it's not initialized before that. If an available pfn_memhole is wanted after get_num_dumpable_cyclic invocation, initializing pfn_memhole in get_num_dumpable_cyclic is necessary.

---
 makedumpfile.c | 2 ++
 1 file changed, 2 insertions(+)
makedumpfile.c +2 -0
index b4b6eca..a01703e 100644
@@ -5651,6 +5651,8 @@ get_num_dumpable_cyclic(void)
 	mdf_pfn_t pfn, num_dumpable=0;
 	struct cycle cycle = {0};
 
+	pfn_memhole = info->max_mapnr;
+
 	for_each_cycle(0, info->max_mapnr, &cycle)
 	{
 		if (!exclude_unnecessary_pages_cyclic(&cycle))
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#2 lore raw

PATCH v6 2/8 functions to get crashkernel memory range

In reply to Baoquan He

These functions are used to parse /proc/iomem code and get memory ranges of specific type. They are implemented in kexec-tools and borrowed here to get the crashkernel memory range. Since crashkernel memory range should be excluded from dumpable memory ranges.

---
 makedumpfile.c | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 makedumpfile.h |  7 +++++
 2 files changed, 89 insertions(+)
makedumpfile.c +82 -0
index a01703e..44cf26f 100644
@@ -9043,6 +9043,88 @@ calculate_cyclic_buffer_size(void) {
 	return TRUE;
 }
 
+
+
+//#define CRASH_RESERVED_MEM_NR   8
+struct memory_range crash_reserved_mem[CRASH_RESERVED_MEM_NR];
+int crash_reserved_mem_nr;
+
+/*
+ * iomem_for_each_line()
+ *
+ * Iterate over each line in the file returned by proc_iomem(). If match is
+ * NULL or if the line matches with our match-pattern then call the
+ * callback if non-NULL.
+ *
+ * Return the number of lines matched.
+ */
+int iomem_for_each_line(char *match,
+			      int (*callback)(void *data,
+					      int nr,
+					      char *str,
+					      unsigned long base,
+					      unsigned long length),
+			      void *data)
+{
+	const char iomem[] = "/proc/iomem";
+	char line[BUFSIZE_FGETS];
+	FILE *fp;
+	unsigned long long start, end, size;
+	char *str;
+	int consumed;
+	int count;
+	int nr = 0;
+
+	fp = fopen(iomem, "r");
+	if (!fp) {
+		ERRMSG("Cannot open %s\n", iomem);
+		return nr;
+	}
+
+	while(fgets(line, sizeof(line), fp) != 0) {
+		count = sscanf(line, "%Lx-%Lx : %n", &start, &end, &consumed);
+		if (count != 2)
+			continue;
+		str = line + consumed;
+		size = end - start + 1;
+		if (!match || memcmp(str, match, strlen(match)) == 0) {
+			if (callback
+			    && callback(data, nr, str, start, size) < 0) {
+				break;
+			}
+			nr++;
+		}
+	}
+
+	fclose(fp);
+
+	return nr;
+}
+
+static int crashkernel_mem_callback(void *data, int nr,
+                                          char *str,
+                                          unsigned long base,
+                                          unsigned long length)
+{
+        if (nr >= CRASH_RESERVED_MEM_NR)
+                return 1;
+
+        crash_reserved_mem[nr].start = base;
+        crash_reserved_mem[nr].end   = base + length - 1;
+        return 0;
+}
+
+int is_crashkernel_mem_reserved(void)
+{
+        int ret;
+
+        ret = iomem_for_each_line("Crash kernel\n",
+                                        crashkernel_mem_callback, NULL);
+        crash_reserved_mem_nr = ret;
+
+        return !!crash_reserved_mem_nr;
+}
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
makedumpfile.h +7 -0
index 9f90b53..ed584af 100644
@@ -1540,6 +1540,13 @@ extern struct array_table	array_table;
 extern struct number_table	number_table;
 extern struct srcfile_table	srcfile_table;
 
+struct memory_range {
+        unsigned long long start, end;
+};
+
+#define CRASH_RESERVED_MEM_NR   8
+struct memory_range crash_reserved_mem[CRASH_RESERVED_MEM_NR];
+int crash_reserved_mem_nr;
 
 int readmem(int type_addr, unsigned long long addr, void *bufptr, size_t size);
 int get_str_osrelease_from_vmlinux(void);
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#3 lore raw

PATCH v6 3/8 preparation functions for parsing vmcoreinfo

In reply to Baoquan He

Add 2 preparation functions get_elf_loads and get_page_offset, later they will be needed for parsing vmcoreinfo.

Meanwhile since get_kernel_version need be called to get page_offset before initial() in mem_usage code flow, and surely it will be called inside initial() again. Add a static variable to avoid this duplicate calling.

---
 elf_info.c     | 50 ++++++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |  1 +
 makedumpfile.c | 18 ++++++++++++++++++
 3 files changed, 69 insertions(+)
elf_info.c +50 -0
index b277f69..69d3fdb 100644
@@ -681,6 +681,56 @@ get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr)
 	return TRUE;
 }
 
+int
+get_elf_loads(int fd, char *filename)
+{
+	int i, j, phnum, elf_format;
+	Elf64_Phdr phdr;
+
+	/*
+	 * Check ELF64 or ELF32.
+	 */
+	elf_format = check_elf_format(fd, filename, &phnum, &num_pt_loads);
+	if (elf_format == ELF64)
+		flags_memory |= MEMORY_ELF64;
+	else if (elf_format != ELF32)
+		return FALSE;
+
+	if (!num_pt_loads) {
+		ERRMSG("Can't get the number of PT_LOAD.\n");
+		return FALSE;
+	}
+
+	/*
+	 * The below file information will be used as /proc/vmcore.
+	 */
+	fd_memory   = fd;
+	name_memory = filename;
+
+	pt_loads = calloc(sizeof(struct pt_load_segment), num_pt_loads);
+	if (pt_loads == NULL) {
+		ERRMSG("Can't allocate memory for the PT_LOAD. %s\n",
+		    strerror(errno));
+		return FALSE;
+	}
+	for (i = 0, j = 0; i < phnum; i++) {
+		if (!get_phdr_memory(i, &phdr))
+			return FALSE;
+
+		if (phdr.p_type != PT_LOAD)
+			continue;
+
+		if (j >= num_pt_loads)
+			return FALSE;
+		if(!dump_Elf_load(&phdr, j))
+			return FALSE;
+		j++;
+	}
+
+	return TRUE;
+}
+
+
 /*
  * Get ELF information about /proc/vmcore.
  */
elf_info.h +1 -0
index 801faff..263d993 100644
@@ -44,6 +44,7 @@ int get_elf64_ehdr(int fd, char *filename, Elf64_Ehdr *ehdr);
 int get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr);
 int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
+int get_elf_loads(int fd, char *filename);
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.c +18 -0
index 44cf26f..8c8ca91 100644
@@ -682,6 +682,9 @@ get_kernel_version(char *release)
 	long maj, min, rel;
 	char *start, *end;
 
+	if (info->kernel_version)
+		return info->kernel_version;
+
 	/*
 	 * This method checks that vmlinux and vmcore are same kernel version.
 	 */
@@ -706,6 +709,7 @@ get_kernel_version(char *release)
 		MSG("The kernel version is not supported.\n");
 		MSG("The created dumpfile may be incomplete.\n");
 	}
+
 	return version;
 }
 
@@ -9125,6 +9129,20 @@ int is_crashkernel_mem_reserved(void)
         return !!crash_reserved_mem_nr;
 }
 
+static int get_page_offset()
+{
+	struct utsname utsname;
+	if (uname(&utsname)) {
+		ERRMSG("Cannot get name and information about current kernel : %s", strerror(errno));
+		return FALSE;
+	}
+
+	info->kernel_version = get_kernel_version(utsname.release);
+	get_versiondep_info();
+
+	return TRUE;
+}
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#4 lore raw

PATCH v6 4/8 set vmcoreinfo for kcore

In reply to Baoquan He

In vmcore dumping, note program of vmcoreinfo is set in elf header of /proc/vmcore. In 1st kernel, the vmcoreinfo is also needed for kcore analyzing. So in this patch information of vmcoreinfo is parsed and set in offset_vmcoreinfo and size_vmcoreinfo.

---
 elf_info.c     | 47 +++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |  1 +
 makedumpfile.c | 33 +++++++++++++++++++++++++++++++++
 3 files changed, 81 insertions(+)
elf_info.c +47 -0
index 69d3fdb..d50124d 100644
@@ -395,6 +395,53 @@ get_pt_note_info(void)
 	return TRUE;
 }
 
+#define UNINITIALIZED  ((ulong)(-1))
+int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len)
+{
+	int i;
+	ulong kvaddr;
+	off_t offset;
+	char note[MAX_SIZE_NHDR];
+	int size_desc;
+	off_t offset_desc;
+
+	offset = UNINITIALIZED;
+	kvaddr = (ulong)vmcoreinfo_addr | PAGE_OFFSET;
+
+	for (i = 0; i < num_pt_loads; ++i) {
+		struct pt_load_segment *p = &pt_loads[i];
+		if ((kvaddr >= p->virt_start) && (kvaddr < p->virt_end)) {
+			offset = (off_t)(kvaddr - p->virt_start) +
+			(off_t)p->file_offset;
+			break;
+		}
+	}
+
+	if (offset == UNINITIALIZED){
+		ERRMSG("Can't get the offset of VMCOREINFO(%s). %s\n",
+		    name_memory, strerror(errno));
+		return FALSE;
+	}
+
+        if (lseek(fd_memory, offset, SEEK_SET) != offset){
+		ERRMSG("Can't seek the dump memory(%s). %s\n",
+		    name_memory, strerror(errno));
+		return FALSE;
+	}
+
+	if (read(fd_memory, note, MAX_SIZE_NHDR) != MAX_SIZE_NHDR){
+		ERRMSG("Can't read the dump memory(%s). %s\n",
+		    name_memory, strerror(errno));
+		return FALSE;
+	}
+
+	size_desc   = note_descsz(note);
+	offset_desc = offset + offset_note_desc(note);
+
+	set_vmcoreinfo(offset_desc, size_desc);
+
+	return TRUE;
+}
 
 /*
  * External functions.
elf_info.h +1 -0
index 263d993..3ce0138 100644
@@ -45,6 +45,7 @@ int get_elf32_ehdr(int fd, char *filename, Elf32_Ehdr *ehdr);
 int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
 int get_elf_loads(int fd, char *filename);
+int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len);
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.c +33 -0
index 8c8ca91..d43d02d 100644
@@ -9143,6 +9143,39 @@ static int get_page_offset()
 	return TRUE;
 }
 
+
+/* Returns the physical address of start of crash notes buffer for a kernel. */
+static int get_sys_kernel_vmcoreinfo(uint64_t *addr, uint64_t *len)
+{
+	char line[BUFSIZE_FGETS];
+	int count;
+	FILE *fp;
+	unsigned long long temp, temp2;
+
+	*addr = 0;
+	*len = 0;
+
+	if (!(fp = fopen("/sys/kernel/vmcoreinfo", "r")))
+		return FALSE;
+
+	if (!fgets(line, sizeof(line), fp)) {
+		ERRMSG("Cannot parse %s: %s, fgets failed.\n", "/sys/kernel/vmcoreinfo", strerror(errno));
+		return FALSE;
+	}
+	count = sscanf(line, "%Lx %Lx", &temp, &temp2);
+	if (count != 2){
+		ERRMSG("Cannot parse %s: %s, sscanf failed.\n", "/sys/kernel/vmcoreinfo", strerror(errno));
+		return FALSE;
+	}
+
+	*addr = (uint64_t) temp;
+	*len = (uint64_t) temp2;
+
+	fclose(fp);
+	return TRUE;
+}
+
+
 static struct option longopts[] = {
 	{"split", no_argument, NULL, OPT_SPLIT},
 	{"reassemble", no_argument, NULL, OPT_REASSEMBLE},
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#5 lore raw

PATCH v6 5/8 prepare the dump loads for kcore analysis

In reply to Baoquan He

In kcore, only "System Ram" and "kernel text" program segments are needed. And to be more precise, exclude the crashkernel memory range.

---
 arch/x86_64.c  |   6 +--
 elf_info.c     | 140 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |   1 +
 makedumpfile.h |   8 ++++
 4 files changed, 152 insertions(+), 3 deletions(-)
arch/x86_64.c +3 -3
index 771d457..4788f55 100644
@@ -20,7 +20,7 @@
 #include "../makedumpfile.h"
 
 int
-is_vmalloc_addr(ulong vaddr)
+is_vmalloc_addr_x86_64(ulong vaddr)
 {
 	/*
 	 *  vmalloc, virtual memmap, and module space as VMALLOC space.
@@ -56,7 +56,7 @@ get_phys_base_x86_64(void)
 
 	for (i = 0; get_pt_load(i, &phys_start, NULL, &virt_start, NULL); i++) {
 		if ((virt_start >= __START_KERNEL_map) &&
-		    !(is_vmalloc_addr(virt_start))) {
+		    !(is_vmalloc_addr_x86_64(virt_start))) {
 
 			info->phys_base = phys_start -
 			    (virt_start & ~(__START_KERNEL_map));
@@ -281,7 +281,7 @@ vaddr_to_paddr_x86_64(unsigned long vaddr)
 	else
 		phys_base = 0;
 
-	if (is_vmalloc_addr(vaddr)) {
+	if (is_vmalloc_addr_x86_64(vaddr)) {
 		if ((paddr = vtop4_x86_64(vaddr)) == NOT_PADDR) {
 			ERRMSG("Can't convert a virtual address(%lx) to " \
 			    "physical address.\n", vaddr);
elf_info.c +140 -0

146 lines not on this page · raw message

elf_info.h +1 -0
index 3ce0138..ba27fdf 100644
@@ -46,6 +46,7 @@ int get_elf_info(int fd, char *filename);
 void free_elf_info(void);
 int get_elf_loads(int fd, char *filename);
 int set_kcore_vmcoreinfo(uint64_t vmcoreinfo_addr, uint64_t vmcoreinfo_len);
+int get_kcore_dump_loads();
 
 int is_elf64_memory(void);
 int is_xen_memory(void);
makedumpfile.h +8 -0
index ed584af..e83bd95 100644
@@ -765,6 +765,7 @@ unsigned long long vaddr_to_paddr_arm(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif /* arm */
 
 #ifdef __x86__
@@ -775,9 +776,11 @@ unsigned long long vaddr_to_paddr_x86(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif /* x86 */
 
 #ifdef __x86_64__
+int is_vmalloc_addr_x86_64(ulong vaddr);
 int get_phys_base_x86_64(void);
 int get_machdep_info_x86_64(void);
 int get_versiondep_info_x86_64(void);
@@ -786,6 +789,7 @@ unsigned long long vaddr_to_paddr_x86_64(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
+#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -796,6 +800,7 @@ unsigned long long vaddr_to_paddr_ppc64(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -805,6 +810,7 @@ unsigned long long vaddr_to_paddr_ppc(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
@@ -814,6 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -825,6 +832,7 @@ unsigned long long vaddr_to_paddr_ia64(unsigned long vaddr);
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
+#define is_vmalloc_addr(X)	TRUE
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#6 lore raw

PATCH v6 6/8 introduce a function exclude_zero_pages_cyclic()

In reply to Baoquan He

Introduced a new function exclude_zero_pages_cyclic(), this will exclude and counting zero pages. Calling it in get_num_dumpable_cyclic can get the number of zero pages.

---
 makedumpfile.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 42 insertions(+)
makedumpfile.c +42 -0
index d43d02d..a511179 100644
@@ -4582,6 +4582,45 @@ exclude_zero_pages(void)
 	return TRUE;
 }
 
+int
+exclude_zero_pages_cyclic(struct cycle *cycle)
+{
+	mdf_pfn_t pfn;
+	unsigned long long paddr;
+	unsigned char buf[info->page_size];
+
+	for (pfn = cycle->start_pfn, paddr = pfn_to_paddr(pfn); pfn < cycle->end_pfn;
+	    pfn++, paddr += info->page_size) {
+
+		if (!is_in_segs(paddr))
+			continue;
+
+		if (!is_dumpable_cyclic(info->partial_bitmap2, pfn, cycle))
+			continue;
+
+		if (is_xen_memory()) {
+			if (!readmem(MADDR_XEN, paddr, buf, info->page_size)) {
+				ERRMSG("Can't get the page data(pfn:%llx, max_mapnr:%llx).\n",
+				    pfn, info->max_mapnr);
+				return FALSE;
+			}
+		} else {
+			if (!readmem(PADDR, paddr, buf, info->page_size)) {
+				ERRMSG("Can't get the page data(pfn:%llx, max_mapnr:%llx).\n",
+				    pfn, info->max_mapnr);
+				return FALSE;
+			}
+		}
+		if (is_zero_page(buf, info->page_size)) {
+			if (clear_bit_on_2nd_bitmap(pfn, cycle))
+				pfn_zero++;
+		}
+	}
+
+	return TRUE;
+}
+
+
 static int
 initialize_2nd_bitmap_cyclic(struct cycle *cycle)
 {
@@ -5662,6 +5701,9 @@ get_num_dumpable_cyclic(void)
 		if (!exclude_unnecessary_pages_cyclic(&cycle))
 			return FALSE;
 
+		if (info->flag_mem_usage)
+			exclude_zero_pages_cyclic(&cycle);
+
 		for(pfn=cycle.start_pfn; pfn<cycle.end_pfn; pfn++)
 			if (is_dumpable_cyclic(info->partial_bitmap2, pfn, &cycle))
 				num_dumpable++;
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#7 lore raw

PATCH v6 7/8 implement a function to print the memory usage

In reply to Baoquan He

Introduce print_mem_usage to print the result of analysis of /proc/kcore. The page number of memory in different use are printed.

---
 makedumpfile.c | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)
makedumpfile.c +34 -0
index a511179..c3d45de 100644
@@ -7939,6 +7939,40 @@ print_report(void)
 	REPORT_MSG("\n");
 }
 
+static void
+print_mem_usage(void)
+{
+	mdf_pfn_t pfn_original, pfn_excluded, shrinking;
+
+	/*
+	* /proc/vmcore doesn't contain the memory hole area.
+	*/
+	pfn_original = info->max_mapnr - pfn_memhole;
+
+	pfn_excluded = pfn_zero + pfn_cache + pfn_cache_private
+	    + pfn_user + pfn_free + pfn_hwpoison;
+	shrinking = (pfn_original - pfn_excluded) * 100;
+	shrinking = shrinking / pfn_original;
+
+	MSG("\n");
+	MSG("\n");
+	MSG("----------------------------------------------------------------------\n");
+	MSG("TYPE		PAGES			EXCLUDABLE	DESCRIPTION\n");
+
+	MSG("ZERO		%-16llu	yes		Pages filled with zero\n", pfn_zero);
+	MSG("CACHE		%-16llu	yes		Cache pages\n", pfn_cache);
+	MSG("CACHE_PRIVATE	%-16llu	yes		Cache pages + private\n",
+	    pfn_cache_private);
+	MSG("USER		%-16llu	yes		User process pages\n", pfn_user);
+	MSG("FREE		%-16llu	yes		Free pages\n", pfn_free);
+	MSG("KERN_DATA	%-16llu	no		Dumpable kernel data \n",
+	    pfn_original - pfn_excluded);
+
+	MSG("\n");
+
+	MSG("Total pages on system:	%-16llu\n", pfn_original);
+}
+
 int
 writeout_dumpfile(void)
 {
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
author
#8 lore raw

PATCH v6 8/8 add a new interface to show the memory usage of 1st kernel

In reply to Baoquan He

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patch, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

The print is like below:

~$ ./makedumpfile --mem-usage /proc/kcore
The kernel version is not supported.
The makedumpfile operation may be incomplete.
Excluding unnecessary pages : [100.0 %] |

----------------------------------------------------------------------
TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 28823 yes Pages filled with zero
CACHE 197932 yes Cache pages
CACHE_PRIVATE 15862 yes Cache pages + private
USER 30778 yes User process pages
FREE 3671105 yes Free pages
KERN_DATA 105168 no Dumpable kernel data

Total pages on system: 4049668

makedumpfile Completed.

BTW, in formal version of kernel which maintainer declares to support after sufficient test, the warning message about kernel version won't occur.

---
 makedumpfile.8 | 17 ++++++++++++++
 makedumpfile.c | 74 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 makedumpfile.h |  2 ++
 print_info.c   |  8 +++++++
 4 files changed, 100 insertions(+), 1 deletion(-)
makedumpfile.8 +17 -0
index 25fe74e..64abbc7 100644
@@ -532,6 +532,23 @@ it is necessary to specfiy [\-x \fIVMLINUX\fR] or [\-i \fIVMCOREINFO\fR].
 # makedumpfile \-\-dump-dmesg -x vmlinux /proc/vmcore dmesgfile
 .br
 
+
+.TP
+\fB\-\-mem-usage\fR
+This option is used to show the page numbers of current system in different
+use. It should be executed in 1st kernel. By the help of this, user can know
+how many pages is dumpable when different dump_level is specified. It analyzes
+the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding
+the crashkernel range, then calculates the page number of different kind per
+vmcoreinfo. So currently /proc/kcore need be specified explicitly.
+
+.br
+.B Example:
+.br
+# makedumpfile \-\-mem-usage /proc/kcore
+.br
+
+
 .TP
 \fB\-\-diskset=VMCORE\fR
 Specify multiple \fIVMCORE\fRs created on sadump diskset configuration
makedumpfile.c +73 -1

110 lines not on this page · raw message

makedumpfile.h +2 -0
index e83bd95..6ee4cae 100644
@@ -978,6 +978,7 @@ struct DumpInfo {
 	int		flag_force;	     /* overwrite existing stuff */
 	int		flag_exclude_xen_dom;/* exclude Domain-U from xen-kdump */
 	int             flag_dmesg;          /* dump the dmesg log out of the vmcore file */
+	int             flag_mem_usage;  /*show the page number of memory in different use*/
 	int		flag_use_printk_log; /* did we read printk_log symbol name? */
 	int		flag_nospace;	     /* the flag of "No space on device" error */
 	int		flag_vmemmap;        /* kernel supports vmemmap address space */
@@ -1867,6 +1868,7 @@ struct elf_prstatus {
 #define OPT_CYCLIC_BUFFER       OPT_START+11
 #define OPT_EPPIC               OPT_START+12
 #define OPT_NON_MMAP            OPT_START+13
+#define OPT_MEM_USAGE            OPT_START+14
 
 /*
  * Function Prototype.
print_info.c +8 -0
index 7592690..29db918 100644
@@ -264,6 +264,14 @@ print_usage(void)
 	MSG("      LOGFILE. If a VMCORE does not contain VMCOREINFO for dmesg, it is\n");
 	MSG("      necessary to specfiy [-x VMLINUX] or [-i VMCOREINFO].\n");
 	MSG("\n");
+	MSG("  [--mem-usage]:\n");
+	MSG("      This option is used to show the page numbers of current system in different\n");
+	MSG("      use. It should be executed in 1st kernel. By the help of this, user can know\n");
+	MSG("      how many pages is dumpable when different dump_level is specified. It analyzes\n");
+	MSG("      the 'System Ram' and 'kernel text' program segment of /proc/kcore excluding\n");
+	MSG("      the crashkernel range, then calculates the page number of different kind per\n");
+	MSG("      vmcoreinfo. So currently /proc/kcore need be specified explicitly.\n");
+	MSG("\n");
 	MSG("  [-D]:\n");
 	MSG("      Print debugging message.\n");
 	MSG("\n");
Signature · 7 lines
1.8.5.3


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#9 lore raw
Quoted 12 lines · Baoquan He

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patch, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

Quoted 8 lines

The print is like below:

~$ ./makedumpfile --mem-usage /proc/kcore
The kernel version is not supported.
The makedumpfile operation may be incomplete.
Excluding unnecessary pages : [100.0 %] |

----------------------------------------------------------------------

^^^^^

Why this line is there?

Quoted 9 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 28823 yes Pages filled with zero
CACHE 197932 yes Cache pages
CACHE_PRIVATE 15862 yes Cache pages + private
USER 30778 yes User process pages
FREE 3671105 yes Free pages
KERN_DATA 105168 no Dumpable kernel data

Total pages on system: 4049668

So this is assuming page size 4K? If some arch is using 16K or 64K, will it still work correctly.

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#10 lore raw
Quoted 6 lines · Vivek Goyal
Quoted 1 line · Baoquan He

----------------------------------------------------------------------

^^^^^

Why this line is there?

Oh, I thought you just meant the header, not including this line.

Quoted 9 lines
Quoted 9 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 28823 yes Pages filled with zero
CACHE 197932 yes Cache pages
CACHE_PRIVATE 15862 yes Cache pages + private
USER 30778 yes User process pages
FREE 3671105 yes Free pages
KERN_DATA 105168 no Dumpable kernel data

Quoted 4 lines
Quoted 1 line

Total pages on system: 4049668

So this is assuming page size 4K? If some arch is using 16K or 64K, will it still work correctly.

This patchset is only for x86_64. Let's firstly introduce this new feature, if user like it, it can be extended to more ARCHs.

Quoted 8 lines

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#11 lore raw
Quoted 9 lines · Baoquan He
Quoted 6 lines · Vivek Goyal
Quoted 1 line · Baoquan He

----------------------------------------------------------------------

^^^^^

Why this line is there?

Oh, I thought you just meant the header, not including this line.

Since Atsushi has decided to merge it, I would like to adjust this later after v1.5.7.

Maybe we can talk about this and the message issue later.

Thanks
Baoquan

Quoted 27 lines
Quoted 10 lines
Quoted 10 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 28823 yes Pages filled with zero
CACHE 197932 yes Cache pages
CACHE_PRIVATE 15862 yes Cache pages + private
USER 30778 yes User process pages
FREE 3671105 yes Free pages
KERN_DATA 105168 no Dumpable kernel data

Quoted 4 lines
Quoted 1 line

Total pages on system: 4049668

So this is assuming page size 4K? If some arch is using 16K or 64K, will it still work correctly.

This patchset is only for x86_64. Let's firstly introduce this new feature, if user like it, it can be extended to more ARCHs.

Quoted 8 lines

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#12 lore raw
Quoted 7 lines · Baoquan He
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Baoquan He
Quoted 2 lines

----------------------------------------------------------------------

^^^^^

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Why this line is there?

Oh, I thought you just meant the header, not including this line.

Since Atsushi has decided to merge it, I would like to adjust this later after v1.5.7.

Maybe we can talk about this and the message issue later.

v1.5.7 will be released within a week, but I'm still receiving such small fix for it.

Quoted 19 lines

Thanks
Baoquan

Quoted 17 lines
Quoted 10 lines
Quoted 10 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 28823 yes Pages filled with zero
CACHE 197932 yes Cache pages
CACHE_PRIVATE 15862 yes Cache pages + private
USER 30778 yes User process pages
FREE 3671105 yes Free pages
KERN_DATA 105168 no Dumpable kernel data

Quoted 4 lines
Quoted 1 line

Total pages on system: 4049668

So this is assuming page size 4K? If some arch is using 16K or 64K, will it still work correctly.

This patchset is only for x86_64. Let's firstly introduce this new feature, if user like it, it can be extended to more ARCHs.

I've remembered that your patch set doesn't mention the target architecture, it's better to update the man page and the help message to note that this feature is only for x86_64 for now. So could you post an additional patch to do it? I want to merge it into v1.5.7 at least.

Thanks
Atsushi Kumagai

Quoted 10 lines
Quoted 10 lines
Quoted 9 lines

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Quoted 4 lines
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#13 lore raw
Quoted 36 lines · Atsushi Kumagai
Quoted 7 lines · Baoquan He
Quoted 6 lines · Vivek Goyal
Quoted 5 lines · Baoquan He
Quoted 2 lines

----------------------------------------------------------------------

^^^^^

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

Why this line is there?

Oh, I thought you just meant the header, not including this line.

Since Atsushi has decided to merge it, I would like to adjust this later after v1.5.7.

Maybe we can talk about this and the message issue later.

v1.5.7 will be released within a week, but I'm still receiving such small fix for it.

Quoted 17 lines

Thanks
Baoquan

Quoted 15 lines
Quoted 14 lines
Quoted 11 lines

TYPE PAGES EXCLUDABLE DESCRIPTION
ZERO 28823 yes Pages filled with zero
CACHE 197932 yes Cache pages
CACHE_PRIVATE 15862 yes Cache pages + private
USER 30778 yes User process pages
FREE 3671105 yes Free pages
KERN_DATA 105168 no Dumpable kernel data

Total pages on system: 4049668

So this is assuming page size 4K? If some arch is using 16K or 64K, will it still work correctly.

Quoted 8 lines
Quoted 2 lines
Quoted 2 lines

This patchset is only for x86_64. Let's firstly introduce this new feature, if user like it, it can be extended to more ARCHs.

I've remembered that your patch set doesn't mention the target architecture, it's better to update the man page and the help message to note that this feature is only for x86_64 for now. So could you post an additional patch to do it? I want to merge it into v1.5.7 at least.

Sure, right away. Will post soon.

Quoted 20 lines

Thanks
Atsushi Kumagai

Quoted 14 lines
Quoted 14 lines
Quoted 9 lines

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#14 lore raw
In reply to Baoquan He

Hello Baoquan,

Quoted 55 lines

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patchset, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

Previouly, patchset v1 was posted. And patch 7/7 has a change in v2. So several changes are made in this v3 post per comments from Vivek and Atsushi.

[patch 2/8] functions to get crashkernel memory range v3->v4:
In old iomem_for_each_line(), it will call exit(1) if opening iomem failed. Atsushi suggested changing this to be consistent with other return value. So here change all the return value to be nr, then check it outside of iomem_for_each_line, and then decide how to act.

[patch 3/8] preparation functions for parsing vmcoreinfo
v1->v3:
Since get_kernel_version need be called to get page_offset before initial() in mem_usage code flow, and surely it will be called inside initial() again. Add a static variable to avoid this duplicate calling.
v3->v4:
Check the value of info->kernel_version, just return if it has been assigned to a value. This is suggested by Atsushi, far better than the static variable idea.
And replace replace get_versiondep_info_x86_64() with get_versiondep_info in get_page_offset().
v4->v5:
Update the git log.

[patch 4/8] set vmcoreinfo for kcore
v3->v4:
Change several places error messages which are the same in nearby handling.

[patch 5/8] prepare the dump loads for kcore analysis
v1->v3:
Fix the compiler warnings.
v3->v4:
Rename is_vmalloc_addr() to is_vmalloc_addr_x86_64() in arch/x86_64.c. And then define is_vmalloc_addr() for all other ARCHs, just set their value to be TRUE except of x86_64.

[patch 6/8] introduce-a-function-exclude_zero_pages_cyclic v3->v4:
Newly introduce a function exclude_zero_pages_cyclic(), and call it in get_num_dumpable_cyclic().

Quoted 5 lines

Besides, a strange thing happened when the new interface was tested on my local AMD machine. It always terminated and printed message:
"Program terminated with signal SIGKILL"
With gdb, it should be going in readmem() of exclude_zero_pages_cyclic, I still don't know why it happened.

This issue is still under investigation, but it may be a kernel issue since you just read a page of System RAM via /proc/kcore. At least your patches look good to me, I'll merge them into v1.5.7. Good job!

Thanks
Atsushi Kumagai

Quoted 41 lines

[patch 7/8] implement a function to print the memory usage v1->v3:
Adjust the printing content and format of dumpable page numbers per Vivek's comments.
v3->v6:
Slightly adjust the printing format and content.

[patch 8/8]
v1->v2:
Set info->dump_level=MAX_DUMP_LEVEL, with MAX_DUMP_LEVEL all kinds of memory can be calculated.
v2->v3:
Add the description of this feature into help message and man page.
v3->v4:
Continue adjusting the output message of show_mem_usage calling per Vivek's comments.
v4->v5:
Update the git log.
v5->v6:
Adjust the printing format which is related to dump files previously.

Baoquan He (8):
initialize pfn_memhole in get_num_dumpable_cyclic functions to get crashkernel memory range preparation functions for parsing vmcoreinfo
set vmcoreinfo for kcore
prepare the dump loads for kcore analysis introduce a function exclude_zero_pages_cyclic() implement a function to print the memory usage add a new interface to show the memory usage of 1st kernel

arch/x86_64.c  |   6 +-
elf_info.c     | 237 +++++++++++++++++++++++++++++++++++++++++++++++
elf_info.h     |   3 +
makedumpfile.8 |  17 ++++
makedumpfile.c | 285 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
makedumpfile.h |  17 ++++
print_info.c   |   8 ++
7 files changed, 569 insertions(+), 4 deletions(-)
Quoted 2 lines

--
1.8.5.3

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#15 lore raw
Quoted 14 lines · Atsushi Kumagai

Hello Baoquan,

Quoted 10 lines

[patch 6/8] introduce-a-function-exclude_zero_pages_cyclic v3->v4:
Newly introduce a function exclude_zero_pages_cyclic(), and call it in get_num_dumpable_cyclic().

Besides, a strange thing happened when the new interface was tested on my local AMD machine. It always terminated and printed message:
"Program terminated with signal SIGKILL"
With gdb, it should be going in readmem() of exclude_zero_pages_cyclic, I still don't know why it happened.

Quoted 3 lines

This issue is still under investigation, but it may be a kernel issue since you just read a page of System RAM via /proc/kcore. At least your patches look good to me, I'll merge them into v1.5.7. Good job!

Hi Atsushi,

Thanks for help on this patchset.

For this issue happened on AMD machine, I will try to look into it deeper later. Saw the mail you mentioned that someone has reported the simillar bug, will check it.

Thanks
Baoquan

Quoted 6 lines

Thanks
Atsushi Kumagai

Quoted 1 line

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#16 lore raw

Add "--mem-usage" support for s390x

In reply to Baoquan He

Hello Baoquan,

I looked into your patches and tried to add s390x support.

My naive approach was to just enable the is_vmalloc_addr() for s390x:

makedumpfile.h +2 -1
@@ -814,13 +814,15 @@ unsigned long long vaddr_to_paddr_ppc(un
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
+int is_vmalloc_addr_s390x(ulong vaddr);
 int get_machdep_info_s390x(void);
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_phys_base()        TRUE
 #define get_machdep_info()     get_machdep_info_s390x()
 #define get_versiondep_info()  TRUE
 #define vaddr_to_paddr(X)      vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)     TRUE
+#define is_vmalloc_addr(X)     is_vmalloc_addr_s390x(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */

Unfortunately this does not work and makedumpfile loops. I assume the reason is that get_kcore_dump_loads() is called before get_machdep_info_s390x() which sets info->vmalloc_start.

I looked into the x86 code and for me it looks like it should have the same problem.

Have I overlooked something here? Perhaps you can give me a hint?

For testing on my 1G system I hardcoded is_vmalloc_addr(X) to:

#define is_vmalloc_addr(X) ((X) > (1024 * 1024 * 1024))

Then --mem-usage seems to work:

# makedumpfile --mem-usage /proc/kcore
TYPE            PAGES                   EXCLUDABLE      DESCRIPTION
----------------------------------------------------------------------
ZERO            32014                   yes             Pages filled with zero
CACHE           23264                   yes             Cache pages
CACHE_PRIVATE   11462                   yes             Cache pages + private
USER            5154                    yes             User process pages
FREE            20227                   yes             Free pages
KERN_DATA       36903                   no              Dumpable kernel data 
page size:              4096            
Total pages on system:  129024          
Total size on system:   528482304        Byte

Best Regards
Michael

Quoted 92 lines · On Mon, 1 Sep 2014 11:15:32 +0800 Baoquan He <[email protected]> wrote:

Recently people complained that they don't know how to decide how much disk size need be reserved for kdump. E.g there are lots of machines with different memory size, if the memory usage information of current system can be shown, that can help them to make an estimate how much storage space need be reserved.

In this patchset, a new interface is added into makedumpfile. By the help of this, people can know the page number of memory in different use. The implementation is analyzing the "System Ram" and "kernel text" program segment of /proc/kcore excluding the crashkernel range, then calculating the page number of different kind per vmcoreinfo.

Previouly, patchset v1 was posted. And patch 7/7 has a change in v2. So several changes are made in this v3 post per comments from Vivek and Atsushi.

[patch 2/8] functions to get crashkernel memory range v3->v4:
In old iomem_for_each_line(), it will call exit(1) if opening iomem failed. Atsushi suggested changing this to be consistent with other return value. So here change all the return value to be nr, then check it outside of iomem_for_each_line, and then decide how to act.

[patch 3/8] preparation functions for parsing vmcoreinfo
v1->v3:
Since get_kernel_version need be called to get page_offset before initial() in mem_usage code flow, and surely it will be called inside initial() again. Add a static variable to avoid this duplicate calling.
v3->v4:
Check the value of info->kernel_version, just return if it has been assigned to a value. This is suggested by Atsushi, far better than the static variable idea.
And replace replace get_versiondep_info_x86_64() with get_versiondep_info in get_page_offset().
v4->v5:
Update the git log.

[patch 4/8] set vmcoreinfo for kcore
v3->v4:
Change several places error messages which are the same in nearby handling.

[patch 5/8] prepare the dump loads for kcore analysis
v1->v3:
Fix the compiler warnings.
v3->v4:
Rename is_vmalloc_addr() to is_vmalloc_addr_x86_64() in arch/x86_64.c. And then define is_vmalloc_addr() for all other ARCHs, just set their value to be TRUE except of x86_64.

[patch 6/8] introduce-a-function-exclude_zero_pages_cyclic v3->v4:
Newly introduce a function exclude_zero_pages_cyclic(), and call it in get_num_dumpable_cyclic().

Besides, a strange thing happened when the new interface was tested on my local AMD machine. It always terminated and printed message:
"Program terminated with signal SIGKILL"
With gdb, it should be going in readmem() of exclude_zero_pages_cyclic, I still don't know why it happened.

[patch 7/8] implement a function to print the memory usage
v1->v3:
Adjust the printing content and format of dumpable page numbers per Vivek's comments.
v3->v6:
Slightly adjust the printing format and content.

[patch 8/8]
v1->v2:
Set info->dump_level=MAX_DUMP_LEVEL, with MAX_DUMP_LEVEL all kinds of memory can be calculated.
v2->v3:
Add the description of this feature into help message and man page.
v3->v4:
Continue adjusting the output message of show_mem_usage calling per Vivek's comments.
v4->v5:
Update the git log.
v5->v6:
Adjust the printing format which is related to dump files previously.

Baoquan He (8):
initialize pfn_memhole in get_num_dumpable_cyclic functions to get crashkernel memory range preparation functions for parsing vmcoreinfo
set vmcoreinfo for kcore
prepare the dump loads for kcore analysis introduce a function exclude_zero_pages_cyclic() implement a function to print the memory usage add a new interface to show the memory usage of 1st kernel

Quoted 9 lines
 arch/x86_64.c  |   6 +-
 elf_info.c     | 237 +++++++++++++++++++++++++++++++++++++++++++++++
 elf_info.h     |   3 +
 makedumpfile.8 |  17 ++++
 makedumpfile.c | 285 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
 makedumpfile.h |  17 ++++
 print_info.c   |   8 ++
 7 files changed, 569 insertions(+), 4 deletions(-)

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#17 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 15 lines · Michael Holzheu

Hello Baoquan,

I looked into your patches and tried to add s390x support.

My naive approach was to just enable the is_vmalloc_addr() for s390x:

makedumpfile.h +1 -0
@@ -814,13 +814,15 @@ unsigned long long vaddr_to_paddr_ppc(un
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
+int is_vmalloc_addr_s390x(ulong vaddr);
 int get_machdep_info_s390x(void);

makedumpfile.h unsigned long long vaddr_to_paddr_ppc(un · line 827

Quoted 10 lines
makedumpfile.h +1 -1
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_phys_base()        TRUE
 #define get_machdep_info()     get_machdep_info_s390x()
 #define get_versiondep_info()  TRUE
 #define vaddr_to_paddr(X)      vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)     TRUE
+#define is_vmalloc_addr(X)     is_vmalloc_addr_s390x(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */

Hi Michael,

Please alse provide a get_versiondep_info_s390x since page_offset is needed in set_kcore_vmcoreinfo() and other information need it too, such as VMALLOC_START/VMEMMAP_START/MODULES_VADDR, if you want to provide a is_vmalloc_addr_s390x before initial() is called.

Thanks
Baoquan

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#18 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 13 lines · Baoquan He
Quoted 12 lines · Michael Holzheu

Hello Baoquan,

I looked into your patches and tried to add s390x support.

My naive approach was to just enable the is_vmalloc_addr() for s390x:

makedumpfile.h +0 -0
@@ -814,13 +814,15 @@ unsigned long long vaddr_to_paddr_ppc(un
 #endif          /* powerpc32 */
 
Quoted 6 lines

makedumpfile.h unsigned long long vaddr_to_paddr_ppc(un · line 821

Quoted 6 lines
makedumpfile.h +1 -0
 #ifdef __s390x__ /* s390x */
+int is_vmalloc_addr_s390x(ulong vaddr);
 int get_machdep_info_s390x(void);
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_phys_base()        TRUE
 #define get_machdep_info()     get_machdep_info_s390x()

Well, seems you have added the get_machdep_info_s390x(). Could you post your complete patch? Because I didn't see it in current devel branch.

Quoted 16 lines

makedumpfile.h unsigned long long vaddr_to_paddr_ppc(un · line 827

Quoted 7 lines
makedumpfile.h +1 -1
 #define get_versiondep_info()  TRUE
 #define vaddr_to_paddr(X)      vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)     TRUE
+#define is_vmalloc_addr(X)     is_vmalloc_addr_s390x(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */

Hi Michael,

Please alse provide a get_versiondep_info_s390x since page_offset is needed in set_kcore_vmcoreinfo() and other information need it too, such as VMALLOC_START/VMEMMAP_START/MODULES_VADDR, if you want to provide a is_vmalloc_addr_s390x before initial() is called.

Quoted 8 lines

Thanks
Baoquan

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#19 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 14 lines · Baoquan He
Quoted 13 lines · Baoquan He
Quoted 12 lines · Michael Holzheu

Hello Baoquan,

I looked into your patches and tried to add s390x support.

My naive approach was to just enable the is_vmalloc_addr() for s390x:

makedumpfile.h +0 -0
@@ -814,13 +814,15 @@ unsigned long long vaddr_to_paddr_ppc(un
 #endif          /* powerpc32 */
 
Quoted 9 lines
Quoted 6 lines

makedumpfile.h unsigned long long vaddr_to_paddr_ppc(un · line 821

Quoted 6 lines
makedumpfile.h +1 -0
 #ifdef __s390x__ /* s390x */
+int is_vmalloc_addr_s390x(ulong vaddr);
 int get_machdep_info_s390x(void);
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_phys_base()        TRUE
 #define get_machdep_info()     get_machdep_info_s390x()

Well, seems you have added the get_machdep_info_s390x(). Could you post your complete patch? Because I didn't see it in current devel branch.

Well, I am looking in the wrong. What you need to added is get_versiondep_info_s390x() as I replied in last mail.

Sorry, it's not related to get_machdep_info().

Quoted 26 lines
Quoted 17 lines

makedumpfile.h unsigned long long vaddr_to_paddr_ppc(un · line 827

Quoted 8 lines
makedumpfile.h +1 -1
 #define get_versiondep_info()  TRUE
 #define vaddr_to_paddr(X)      vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)     TRUE
+#define is_vmalloc_addr(X)     is_vmalloc_addr_s390x(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */

Hi Michael,

Please alse provide a get_versiondep_info_s390x since page_offset is needed in set_kcore_vmcoreinfo() and other information need it too, such as VMALLOC_START/VMEMMAP_START/MODULES_VADDR, if you want to provide a is_vmalloc_addr_s390x before initial() is called.

Quoted 8 lines

Thanks
Baoquan

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#20 lore raw

Re: Add "--mem-usage" support for s390x

In reply to Baoquan He
Quoted 25 lines · On Tue, 23 Sep 2014 10:40:58 +0800 Baoquan He <[email protected]> wrote:
Quoted 24 lines · Michael Holzheu

Hello Baoquan,

I looked into your patches and tried to add s390x support.

My naive approach was to just enable the is_vmalloc_addr() for s390x:

makedumpfile.h +2 -1
@@ -814,13 +814,15 @@ unsigned long long vaddr_to_paddr_ppc(un
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
+int is_vmalloc_addr_s390x(ulong vaddr);
 int get_machdep_info_s390x(void);
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_phys_base()        TRUE
 #define get_machdep_info()     get_machdep_info_s390x()
 #define get_versiondep_info()  TRUE
 #define vaddr_to_paddr(X)      vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)     TRUE
+#define is_vmalloc_addr(X)     is_vmalloc_addr_s390x(X)
 #endif          /* s390x */
 
Quoted 8 lines

makedumpfile.h unsigned long long vaddr_to_paddr_ppc(un · line 827

Quoted 1 line
makedumpfile.h +0 -0
 #ifdef __ia64__ /* ia64 */

Hi Michael,

Please alse provide a get_versiondep_info_s390x since page_offset is needed in set_kcore_vmcoreinfo() and other information need it too, such as VMALLOC_START/VMEMMAP_START/MODULES_VADDR, if you want to provide a is_vmalloc_addr_s390x before initial() is called.

Hello Baoquan,

Thanks for the hint! I looked into the x86_64 implementation of get_versiondep_info() where version dependent constants are used for vmalloc_start and others.

For s390x this is not so easy because vmalloc_start is dependent on the memory size of the system (see setup_memory_end() in arch/s390/kernel/setup.c). Unfortunately "info->max_mapnr" is not set at that time.

Any ideas?

Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#21 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 39 lines · Michael Holzheu
Quoted 25 lines · On Tue, 23 Sep 2014 10:40:58 +0800 Baoquan He <[email protected]> wrote:
Quoted 24 lines · Michael Holzheu

Hello Baoquan,

I looked into your patches and tried to add s390x support.

My naive approach was to just enable the is_vmalloc_addr() for s390x:

makedumpfile.h +2 -1
@@ -814,13 +814,15 @@ unsigned long long vaddr_to_paddr_ppc(un
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
+int is_vmalloc_addr_s390x(ulong vaddr);
 int get_machdep_info_s390x(void);
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
 #define get_phys_base()        TRUE
 #define get_machdep_info()     get_machdep_info_s390x()
 #define get_versiondep_info()  TRUE
 #define vaddr_to_paddr(X)      vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)     TRUE
+#define is_vmalloc_addr(X)     is_vmalloc_addr_s390x(X)
 #endif          /* s390x */
 
Quoted 8 lines

makedumpfile.h unsigned long long vaddr_to_paddr_ppc(un · line 827

Quoted 1 line
makedumpfile.h +0 -0
 #ifdef __ia64__ /* ia64 */

Hi Michael,

Please alse provide a get_versiondep_info_s390x since page_offset is needed in set_kcore_vmcoreinfo() and other information need it too, such as VMALLOC_START/VMEMMAP_START/MODULES_VADDR, if you want to provide a is_vmalloc_addr_s390x before initial() is called.

Hello Baoquan,

Quoted 8 lines

Thanks for the hint! I looked into the x86_64 implementation of get_versiondep_info() where version dependent constants are used for vmalloc_start and others.

For s390x this is not so easy because vmalloc_start is dependent on the memory size of the system (see setup_memory_end() in arch/s390/kernel/setup.c). Unfortunately "info->max_mapnr" is not set at that time.

I am not aware of s390 arch and memory layout. But I can explain what those versiondep_info are used for, hope they can help. In fact in x86_64, page_offset is got for set_kcore_vmcoreinfo(), there the vmcoreinfo_addr need be converted to kvaddr. Since vmcoreinfo_addr is a physical addr, we can't use it directly. And VMALLOC_START/VMEMMAP_START/MODULES_VADDR are all used to filter this virtual addr space region since our vmcore only care about the physical ram addr region.

If you need get these before they are used for s390 arch. If necessary you can build a different code flow if you can achive the goal. All these are all used to get dumpable load segments from kcore.

Quoted 10 lines

Any ideas?

Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#22 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 3 lines · On Thu, 25 Sep 2014 17:44:12 +0800 Baoquan He <[email protected]> wrote:
Quoted 2 lines · Michael Holzheu

On Tue, 23 Sep 2014 10:40:58 +0800
Baoquan He <[email protected]> wrote:

[snip]

Quoted 28 lines
Quoted 10 lines
Quoted 7 lines

Hi Michael,

Please alse provide a get_versiondep_info_s390x since page_offset is needed in set_kcore_vmcoreinfo() and other information need it too, such as VMALLOC_START/VMEMMAP_START/MODULES_VADDR, if you want to provide a is_vmalloc_addr_s390x before initial() is called.

Hello Baoquan,

Quoted 8 lines

Thanks for the hint! I looked into the x86_64 implementation of get_versiondep_info() where version dependent constants are used for vmalloc_start and others.

For s390x this is not so easy because vmalloc_start is dependent on the memory size of the system (see setup_memory_end() in arch/s390/kernel/setup.c). Unfortunately "info->max_mapnr" is not set at that time.

I am not aware of s390 arch and memory layout. But I can explain what those versiondep_info are used for, hope they can help. In fact in x86_64, page_offset is got for set_kcore_vmcoreinfo(), there the vmcoreinfo_addr need be converted to kvaddr. Since vmcoreinfo_addr is a physical addr, we can't use it directly. And VMALLOC_START/VMEMMAP_START/MODULES_VADDR are all used to filter this virtual addr space region since our vmcore only care about the physical ram addr region.

Quoted 3 lines

If you need get these before they are used for s390 arch. If necessary you can build a different code flow if you can achive the goal. All these are all used to get dumpable load segments from kcore.

Isn't this a chicken-and-egg problem? In order to determine vmalloc start I have to be able to read memory. But in order to read memory I have to call get_kcore_dump_loads() first.

What about using /proc/iomem to find out if an address is a real address?

The following patch seems to work for me:

---
 elf_info.c     |    4 ++--
 makedumpfile.c |   26 ++++++++++++++++++++++++++
 makedumpfile.h |    1 +
 3 files changed, 29 insertions(+), 2 deletions(-)
elf_info.c +2 -2
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int real_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_real_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", real_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;
makedumpfile.h +1 -0
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_real_addr(unsigned long addr);
 
 
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#23 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 28 lines · Michael Holzheu
Quoted 3 lines · On Thu, 25 Sep 2014 17:44:12 +0800 Baoquan He <[email protected]> wrote:
Quoted 2 lines · Michael Holzheu

On Tue, 23 Sep 2014 10:40:58 +0800
Baoquan He <[email protected]> wrote:

[snip]

Quoted 14 lines
Quoted 4 lines

For s390x this is not so easy because vmalloc_start is dependent on the memory size of the system (see setup_memory_end() in arch/s390/kernel/setup.c). Unfortunately "info->max_mapnr" is not set at that time.

I am not aware of s390 arch and memory layout. But I can explain what those versiondep_info are used for, hope they can help. In fact in x86_64, page_offset is got for set_kcore_vmcoreinfo(), there the vmcoreinfo_addr need be converted to kvaddr. Since vmcoreinfo_addr is a physical addr, we can't use it directly. And VMALLOC_START/VMEMMAP_START/MODULES_VADDR are all used to filter this virtual addr space region since our vmcore only care about the physical ram addr region.

Quoted 3 lines

If you need get these before they are used for s390 arch. If necessary you can build a different code flow if you can achive the goal. All these are all used to get dumpable load segments from kcore.

Quoted 5 lines

Isn't this a chicken-and-egg problem? In order to determine vmalloc start I have to be able to read memory. But in order to read memory I have to call get_kcore_dump_loads() first.

What about using /proc/iomem to find out if an address is a real address?

Well, that's good it works for s390. Anyway in get_kcore_dump_loads() it just gets the physical ram region, and filter out the unwanted region, so your method is good. In x86_64, the is_vmalloc_addr_x86_64 is not only filtering the vmalloc, but vmmemmap and modules_vaadr region. For simplicity it's only named as is_vmalloc_addr. So I think you can wrap this implementation into arch/s390x.c and name it as is_vmalloc_addr_s390x just for being consistent with the x86_64 implementation.

Besides I doubt only below changes will work for you. I didn't find where you adapt the set_kcore_vmcoreinfo() to set the vmcoreinfo. Without vmcoreinfo, makedumpfile doesn't work.

Quoted 67 lines

The following patch seems to work for me:

---
 elf_info.c     |    4 ++--
 makedumpfile.c |   26 ++++++++++++++++++++++++++
 makedumpfile.h |    1 +
 3 files changed, 29 insertions(+), 2 deletions(-)
elf_info.c +2 -2
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int real_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_real_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", real_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;

makedumpfile.h int read_disk_dump_header(struct disk_du · line 1573

Quoted 7 lines
makedumpfile.h +1 -0
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_real_addr(unsigned long addr);
 
 
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#24 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 43 lines · Baoquan He
Quoted 28 lines · Michael Holzheu
Quoted 3 lines · On Thu, 25 Sep 2014 17:44:12 +0800 Baoquan He <[email protected]> wrote:
Quoted 2 lines · Michael Holzheu

On Tue, 23 Sep 2014 10:40:58 +0800
Baoquan He <[email protected]> wrote:

[snip]

Quoted 14 lines
Quoted 4 lines

For s390x this is not so easy because vmalloc_start is dependent on the memory size of the system (see setup_memory_end() in arch/s390/kernel/setup.c). Unfortunately "info->max_mapnr" is not set at that time.

I am not aware of s390 arch and memory layout. But I can explain what those versiondep_info are used for, hope they can help. In fact in x86_64, page_offset is got for set_kcore_vmcoreinfo(), there the vmcoreinfo_addr need be converted to kvaddr. Since vmcoreinfo_addr is a physical addr, we can't use it directly. And VMALLOC_START/VMEMMAP_START/MODULES_VADDR are all used to filter this virtual addr space region since our vmcore only care about the physical ram addr region.

Quoted 3 lines

If you need get these before they are used for s390 arch. If necessary you can build a different code flow if you can achive the goal. All these are all used to get dumpable load segments from kcore.

Quoted 5 lines

Isn't this a chicken-and-egg problem? In order to determine vmalloc start I have to be able to read memory. But in order to read memory I have to call get_kcore_dump_loads() first.

What about using /proc/iomem to find out if an address is a real address?

Well, that's good it works for s390. Anyway in get_kcore_dump_loads() it just gets the physical ram region, and filter out the unwanted region, so your method is good. In x86_64, the is_vmalloc_addr_x86_64 is not only filtering the vmalloc, but vmmemmap and modules_vaadr region. For simplicity it's only named as is_vmalloc_addr. So I think you can wrap this implementation into arch/s390x.c and name it as is_vmalloc_addr_s390x just for being consistent with the x86_64 implementation.

Quoted 3 lines

Besides I doubt only below changes will work for you. I didn't find where you adapt the set_kcore_vmcoreinfo() to set the vmcoreinfo. Without vmcoreinfo, makedumpfile doesn't work.

Well, seems s390x has PAGE_OFFSET as 0x0UL, then it's set as 0 by initialization defaultly. So no changed is needed for set_kcore_vmcoreinfo().

Quoted 3 lines
Quoted 3 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#25 lore raw

Re: Add "--mem-usage" support for s390x

In reply to Baoquan He
Quoted 33 lines · On Fri, 26 Sep 2014 16:55:46 +0800 Baoquan He <[email protected]> wrote:
Quoted 32 lines · Michael Holzheu
Quoted 3 lines · On Thu, 25 Sep 2014 17:44:12 +0800 Baoquan He <[email protected]> wrote:
Quoted 2 lines · Michael Holzheu

On Tue, 23 Sep 2014 10:40:58 +0800
Baoquan He <[email protected]> wrote:

[snip]

Quoted 14 lines
Quoted 4 lines

For s390x this is not so easy because vmalloc_start is dependent on the memory size of the system (see setup_memory_end() in arch/s390/kernel/setup.c). Unfortunately "info->max_mapnr" is not set at that time.

I am not aware of s390 arch and memory layout. But I can explain what those versiondep_info are used for, hope they can help. In fact in x86_64, page_offset is got for set_kcore_vmcoreinfo(), there the vmcoreinfo_addr need be converted to kvaddr. Since vmcoreinfo_addr is a physical addr, we can't use it directly. And VMALLOC_START/VMEMMAP_START/MODULES_VADDR are all used to filter this virtual addr space region since our vmcore only care about the physical ram addr region.

Quoted 3 lines

If you need get these before they are used for s390 arch. If necessary you can build a different code flow if you can achive the goal. All these are all used to get dumpable load segments from kcore.

Isn't this a chicken-and-egg problem? In order to determine vmalloc start I have to be able to read memory. But in order to read memory I have to call get_kcore_dump_loads() first.

Quoted 7 lines
Quoted 1 line

What about using /proc/iomem to find out if an address is a real address?

Well, that's good it works for s390. Anyway in get_kcore_dump_loads() it just gets the physical ram region, and filter out the unwanted region, so your method is good. In x86_64, the is_vmalloc_addr_x86_64 is not only filtering the vmalloc, but vmmemmap and modules_vaadr region. For simplicity it's only named as is_vmalloc_addr.

Not sure if I understood, why ths is_real_addr() function does not work for x86_64.

Also for x86 all three areas, vmalloc, vmemmap, and modules_vaddr, are virtual memory regions with addresses outside of the the memory ranges where /proc/iommem reports physical memory, right?

So the new is_real_addr() function should return false for that areas.

Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#26 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 14 lines · Michael Holzheu

On Fri, 26 Sep 2014 16:55:46 +0800

Quoted 11 lines
Quoted 5 lines

Isn't this a chicken-and-egg problem? In order to determine vmalloc start I have to be able to read memory. But in order to read memory I have to call get_kcore_dump_loads() first.

What about using /proc/iomem to find out if an address is a real address?

Well, that's good it works for s390. Anyway in get_kcore_dump_loads() it just gets the physical ram region, and filter out the unwanted region, so your method is good. In x86_64, the is_vmalloc_addr_x86_64 is not only filtering the vmalloc, but vmmemmap and modules_vaadr region. For simplicity it's only named as is_vmalloc_addr.

Quoted 8 lines

Not sure if I understood, why ths is_real_addr() function does not work for x86_64.

Also for x86 all three areas, vmalloc, vmemmap, and modules_vaddr, are virtual memory regions with addresses outside of the the memory ranges where /proc/iommem reports physical memory, right?

So the new is_real_addr() function should return false for that areas.

is_real_addr() should work for x86_64, this almost does the way kexec-tools is doing. Originally I just consider this for x86_64, skipped other ARCHs. From x86_64 point of view, processing kcore only need to pick up the program segments we want. If it's hard to handle it, I am fine with it that you use the is_real_addr to do it. But please don't use this name, it could be is_phy_addr() or somthing like that. Please post your patch and test x86_64 too and we can review it.

In fact the other way is wrapping your is_real_addr() to is_vmalloc_addr_s390(), and make it work for s390. If later other ARCH also has this requirements, raise it to be common function. Anyway it's fine to me if it can work.

Thanks
Baoquan

Quoted 8 lines

Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#27 lore raw

Re: Add "--mem-usage" support for s390x

Quoted 32 lines · On Mon, 29 Sep 2014 17:04:32 +0800 Baoquan He <[email protected]> wrote:
Quoted 14 lines · Michael Holzheu

On Fri, 26 Sep 2014 16:55:46 +0800

Quoted 11 lines
Quoted 5 lines

Isn't this a chicken-and-egg problem? In order to determine vmalloc start I have to be able to read memory. But in order to read memory I have to call get_kcore_dump_loads() first.

What about using /proc/iomem to find out if an address is a real address?

Well, that's good it works for s390. Anyway in get_kcore_dump_loads() it just gets the physical ram region, and filter out the unwanted region, so your method is good. In x86_64, the is_vmalloc_addr_x86_64 is not only filtering the vmalloc, but vmmemmap and modules_vaadr region. For simplicity it's only named as is_vmalloc_addr.

Quoted 8 lines

Not sure if I understood, why ths is_real_addr() function does not work for x86_64.

Also for x86 all three areas, vmalloc, vmemmap, and modules_vaddr, are virtual memory regions with addresses outside of the the memory ranges where /proc/iommem reports physical memory, right?

So the new is_real_addr() function should return false for that areas.

is_real_addr() should work for x86_64, this almost does the way kexec-tools is doing. Originally I just consider this for x86_64, skipped other ARCHs. From x86_64 point of view, processing kcore only need to pick up the program segments we want. If it's hard to handle it, I am fine with it that you use the is_real_addr to do it. But please don't use this name, it could be is_phy_addr() or somthing like that. Please post your patch and test x86_64 too and we can review it.

Quoted 4 lines

In fact the other way is wrapping your is_real_addr() to is_vmalloc_addr_s390(), and make it work for s390. If later other ARCH also has this requirements, raise it to be common function. Anyway it's fine to me if it can work.

Hi Baoquan,

Because I don't have the possibility to test on x86_64 I decided to make a s390x only patch. I will send it with the next note.

Thanks for your help!
Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#28 lore raw

PATCH 1/1 makedumpfile: Enable --mem-usage for s390x

In reply to Baoquan He

Implement is_vmalloc_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage".

---
 makedumpfile.c |   26 ++++++++++++++++++++++++++
 makedumpfile.h |    3 ++-
 2 files changed, 28 insertions(+), 1 deletion(-)
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;
makedumpfile.h +2 -1
@@ -820,7 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_vmalloc_addr(X)	(!is_iomem_phys_addr(X))
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_iomem_phys_addr(unsigned long addr);
 
 
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#29 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 4 lines · Michael Holzheu

Implement is_vmalloc_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage".

Hi Michael,

This idea looks good to me. One question, should it be put in arch/s390.c since this is only for s390? Then iomem_for_each_line() need be declared in makedumpfile.h .

If later it's needed by other arch, can be taken out to makedumpfile.c, that should be better. Surely this is only my personal concern, if Atsushi like to accept it, I am fine too.

Thanks
Baoquan

Quoted 61 lines
---
 makedumpfile.c |   26 ++++++++++++++++++++++++++
 makedumpfile.h |    3 ++-
 2 files changed, 28 insertions(+), 1 deletion(-)
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;
makedumpfile.h +2 -1
@@ -820,7 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_vmalloc_addr(X)	(!is_iomem_phys_addr(X))
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_iomem_phys_addr(unsigned long addr);
 
 
 /*
Quoted 4 lines

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#30 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 9 lines · On Tue, 30 Sep 2014 17:02:01 +0800 Baoquan He <[email protected]> wrote:
Quoted 4 lines · Michael Holzheu

Implement is_vmalloc_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage".

Hi Michael,

Quoted 7 lines

This idea looks good to me. One question, should it be put in arch/s390.c since this is only for s390? Then iomem_for_each_line() need be declared in makedumpfile.h .

If later it's needed by other arch, can be taken out to makedumpfile.c, that should be better. Surely this is only my personal concern, if Atsushi like to accept it, I am fine too.

Hello Atsushi,

What is your preference regarding this question?

Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#31 lore raw

RE: [PATCH] makedumpfile: Enable --mem-usage for s390x

Hello,

Quoted 20 lines
Quoted 9 lines · On Tue, 30 Sep 2014 17:02:01 +0800 Baoquan He <[email protected]> wrote:
Quoted 4 lines · Michael Holzheu

Implement is_vmalloc_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage".

Hi Michael,

Quoted 7 lines

This idea looks good to me. One question, should it be put in arch/s390.c since this is only for s390? Then iomem_for_each_line() need be declared in makedumpfile.h .

If later it's needed by other arch, can be taken out to makedumpfile.c, that should be better. Surely this is only my personal concern, if Atsushi like to accept it, I am fine too.

Quoted 5 lines

Hello Atsushi,

What is your preference regarding this question?

Michael

In the first place, this is_vmalloc_addr() for s390 isn't a good implementation because it works only on 1st kernel due to the dependence on /proc/iomem. is_vmalloc_addr_XXX() are general functions, they can be called from any path besides --mem-usage.

I think the Michael's first idea below is better since it implements is_real_addr() only for --mem-usage as a common function for all architectures, it's explicit design.

Review int get_kcore_dump_loads(void) · line 860

Quoted 9 lines
(quoted diff) +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		loads++;
 	}

However, this code will not work since the argument of is_real_addr() must be physical address. Even unluckily, /proc/kcore's PT_LOAD looks useless for VtoP converting because PhysAddr is always 0:

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x00000000000002a8 0x0000000000000000 0x0000000000000000

                 0x0000000000000a84 0x0000000000000000         0
  LOAD           0x00007fffff601000 0xffffffffff600000 0x0000000000000000
                 0x0000000000001000 0x0000000000001000  RWE    1000
  LOAD           0x00007fff81001000 0xffffffff81000000 0x0000000000000000
                 0x0000000000a1b000 0x0000000000a1b000  RWE    1000
  LOAD           0x0000490000001000 0xffffc90000000000 0x0000000000000000
                 0x00001fffffffffff 0x00001fffffffffff  RWE    1000
  LOAD           0x00007fffa0001000 0xffffffffa0000000 0x0000000000000000
                 0x000000005f000000 0x000000005f000000  RWE    1000

...

So the way using /proc/iomem seems inappropriate, we have to consider other approaches (but I still don't have any good ideas...)

Thanks,
Atsushi Kumagai

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#32 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 64 lines · On Thu, 9 Oct 2014 06:41:10 +0000 Atsushi Kumagai…

Hello,

Quoted 20 lines
Quoted 9 lines · On Tue, 30 Sep 2014 17:02:01 +0800 Baoquan He <[email protected]> wrote:
Quoted 4 lines · Michael Holzheu

Implement is_vmalloc_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage".

Hi Michael,

Quoted 7 lines

This idea looks good to me. One question, should it be put in arch/s390.c since this is only for s390? Then iomem_for_each_line() need be declared in makedumpfile.h .

If later it's needed by other arch, can be taken out to makedumpfile.c, that should be better. Surely this is only my personal concern, if Atsushi like to accept it, I am fine too.

Quoted 5 lines

Hello Atsushi,

What is your preference regarding this question?

Michael

In the first place, this is_vmalloc_addr() for s390 isn't a good implementation because it works only on 1st kernel due to the dependence on /proc/iomem. is_vmalloc_addr_XXX() are general functions, they can be called from any path besides --mem-usage.

I think the Michael's first idea below is better since it implements is_real_addr() only for --mem-usage as a common function for all architectures, it's explicit design.

Review int get_kcore_dump_loads(void) · line 860

Quoted 9 lines
(quoted diff) +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		loads++;
 	}

However, this code will not work since the argument of is_real_addr() must be physical address. Even unluckily, /proc/kcore's PT_LOAD looks useless for VtoP converting because PhysAddr is always 0:

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x00000000000002a8 0x0000000000000000 0x0000000000000000

                 0x0000000000000a84 0x0000000000000000         0
  LOAD           0x00007fffff601000 0xffffffffff600000 0x0000000000000000
                 0x0000000000001000 0x0000000000001000  RWE    1000
  LOAD           0x00007fff81001000 0xffffffff81000000 0x0000000000000000
                 0x0000000000a1b000 0x0000000000a1b000  RWE    1000
  LOAD           0x0000490000001000 0xffffc90000000000 0x0000000000000000
                 0x00001fffffffffff 0x00001fffffffffff  RWE    1000
  LOAD           0x00007fffa0001000 0xffffffffa0000000 0x0000000000000000
                 0x000000005f000000 0x000000005f000000  RWE    1000
Quoted 5 lines

...

So the way using /proc/iomem seems inappropriate, we have to consider other approaches (but I still don't have any good ideas...)

Hello Atsushi,

Hmmm ok, sure. For x86 using /proc/iomem does not work because there is no 1:1 mapping for the kernel address space. The kernel/real memory is mapped somewhere at the end, right?

For s390 we have a 1:1 mapping for the kernel physical memory that starts with zero. Therefore IMHO we could use /proc/iomem. For example, on my s390x system with 1GB memory and 256MB crashkernel:

$ cat /proc/iomem
00000000-2fffffff : System RAM
  00000000-007ddd4b : Kernel code
  007ddd4c-00bfcc5f : Kernel data
  00e18000-01c28b1f : Kernel bss
30000000-3fffffff : Crash kernel
$ objdump -h /proc/kcore
...
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  NOTE           0x0000000000000158 0x0000000000000000 0x0000000000000000
                 0x00000000000027fc 0x0000000000000000         0
  LOAD           0x000003e080003000 0x000003e080000000 0x0000000000000000
                 0x0000001f00000000 0x0000001f00000000  RWE    1000
  LOAD           0x000003ff80003000 0x000003ff80000000 0x0000000000000000
                 0x0000000080000000 0x0000000080000000  RWE    1000
  LOAD           0x0000000000003000 0x0000000000000000 0x0000000000000000
                 0x0000000030000000 0x0000000030000000  RWE    1000
  LOAD           0x000003d100003000 0x000003d100000000 0x0000000000000000
                 0x0000000000c00000 0x0000000000c00000  RWE    1000

So in that case every /proc/kcore load that is below 0x30000000 must be real memory.

Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#33 lore raw

RE: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 34 lines
Quoted 34 lines

I think the Michael's first idea below is better since it implements is_real_addr() only for --mem-usage as a common function for all architectures, it's explicit design.

Review int get_kcore_dump_loads(void) · line 860

Quoted 9 lines
(quoted diff) +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		loads++;
 	}

However, this code will not work since the argument of is_real_addr() must be physical address. Even unluckily, /proc/kcore's PT_LOAD looks useless for VtoP converting because PhysAddr is always 0:

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x00000000000002a8 0x0000000000000000 0x0000000000000000

                 0x0000000000000a84 0x0000000000000000         0
  LOAD           0x00007fffff601000 0xffffffffff600000 0x0000000000000000
                 0x0000000000001000 0x0000000000001000  RWE    1000
  LOAD           0x00007fff81001000 0xffffffff81000000 0x0000000000000000
                 0x0000000000a1b000 0x0000000000a1b000  RWE    1000
  LOAD           0x0000490000001000 0xffffc90000000000 0x0000000000000000
                 0x00001fffffffffff 0x00001fffffffffff  RWE    1000
  LOAD           0x00007fffa0001000 0xffffffffa0000000 0x0000000000000000
                 0x000000005f000000 0x000000005f000000  RWE    1000

...

Quoted 8 lines
Quoted 2 lines

So the way using /proc/iomem seems inappropriate, we have to consider other approaches (but I still don't have any good ideas...)

Hello Atsushi,

Hmmm ok, sure. For x86 using /proc/iomem does not work because there is no 1:1 mapping for the kernel address space. The kernel/real memory is mapped somewhere at the end, right?

Exactly.

Quoted 26 lines

For s390 we have a 1:1 mapping for the kernel physical memory that starts with zero. Therefore IMHO we could use /proc/iomem. For example, on my s390x system with 1GB memory and 256MB crashkernel:

$ cat /proc/iomem
00000000-2fffffff : System RAM
 00000000-007ddd4b : Kernel code
 007ddd4c-00bfcc5f : Kernel data
 00e18000-01c28b1f : Kernel bss
30000000-3fffffff : Crash kernel
$ objdump -h /proc/kcore
...
 Type           Offset             VirtAddr           PhysAddr
                FileSiz            MemSiz              Flags  Align
 NOTE           0x0000000000000158 0x0000000000000000 0x0000000000000000
                0x00000000000027fc 0x0000000000000000         0
 LOAD           0x000003e080003000 0x000003e080000000 0x0000000000000000
                0x0000001f00000000 0x0000001f00000000  RWE    1000
 LOAD           0x000003ff80003000 0x000003ff80000000 0x0000000000000000
                0x0000000080000000 0x0000000080000000  RWE    1000
 LOAD           0x0000000000003000 0x0000000000000000 0x0000000000000000
                0x0000000030000000 0x0000000030000000  RWE    1000
 LOAD           0x000003d100003000 0x000003d100000000 0x0000000000000000
                0x0000000000c00000 0x0000000000c00000  RWE    1000
Quoted 4 lines

So in that case every /proc/kcore load that is below 0x30000000 must be real memory.

Michael

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Thanks,
Atsushi Kumagai

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#34 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 74 lines · Atsushi Kumagai
Quoted 34 lines
Quoted 34 lines

I think the Michael's first idea below is better since it implements is_real_addr() only for --mem-usage as a common function for all architectures, it's explicit design.

Review int get_kcore_dump_loads(void) · line 860

Quoted 9 lines
(quoted diff) +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		loads++;
 	}

However, this code will not work since the argument of is_real_addr() must be physical address. Even unluckily, /proc/kcore's PT_LOAD looks useless for VtoP converting because PhysAddr is always 0:

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x00000000000002a8 0x0000000000000000 0x0000000000000000

                 0x0000000000000a84 0x0000000000000000         0
  LOAD           0x00007fffff601000 0xffffffffff600000 0x0000000000000000
                 0x0000000000001000 0x0000000000001000  RWE    1000
  LOAD           0x00007fff81001000 0xffffffff81000000 0x0000000000000000
                 0x0000000000a1b000 0x0000000000a1b000  RWE    1000
  LOAD           0x0000490000001000 0xffffc90000000000 0x0000000000000000
                 0x00001fffffffffff 0x00001fffffffffff  RWE    1000
  LOAD           0x00007fffa0001000 0xffffffffa0000000 0x0000000000000000
                 0x000000005f000000 0x000000005f000000  RWE    1000

...

Quoted 8 lines
Quoted 2 lines

So the way using /proc/iomem seems inappropriate, we have to consider other approaches (but I still don't have any good ideas...)

Hello Atsushi,

Hmmm ok, sure. For x86 using /proc/iomem does not work because there is no 1:1 mapping for the kernel address space. The kernel/real memory is mapped somewhere at the end, right?

Exactly.

Quoted 29 lines

For s390 we have a 1:1 mapping for the kernel physical memory that starts with zero. Therefore IMHO we could use /proc/iomem. For example, on my s390x system with 1GB memory and 256MB crashkernel:

$ cat /proc/iomem
00000000-2fffffff : System RAM
 00000000-007ddd4b : Kernel code
 007ddd4c-00bfcc5f : Kernel data
 00e18000-01c28b1f : Kernel bss
30000000-3fffffff : Crash kernel
$ objdump -h /proc/kcore
...
 Type           Offset             VirtAddr           PhysAddr
                FileSiz            MemSiz              Flags  Align
 NOTE           0x0000000000000158 0x0000000000000000 0x0000000000000000
                0x00000000000027fc 0x0000000000000000         0
 LOAD           0x000003e080003000 0x000003e080000000 0x0000000000000000
                0x0000001f00000000 0x0000001f00000000  RWE    1000
 LOAD           0x000003ff80003000 0x000003ff80000000 0x0000000000000000
                0x0000000080000000 0x0000000080000000  RWE    1000
 LOAD           0x0000000000003000 0x0000000000000000 0x0000000000000000
                0x0000000030000000 0x0000000030000000  RWE    1000
 LOAD           0x000003d100003000 0x000003d100000000 0x0000000000000000
                0x0000000000c00000 0x0000000000c00000  RWE    1000

So in that case every /proc/kcore load that is below 0x30000000 must be real memory.

Quoted 8 lines
Quoted 1 line

Michael

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Yeah, this is a working way. In fact I can change to use /proc/iomem to get phys mem region, then filter each program loads of kcore by these phys mem region. Since phys can be converted to virt by adding PAGE_OFFSET, for x86_64 and s390x PAGE_OFFSET is specific. But I am not sure if other ARCH have the same problem, e.g their PAGE_OFFSET is not a fixed value just as VMALLOC_START in s390x.

Anyway, I think Atsushi's suggestion is good.

Thanks
Baoquan

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#35 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 38 lines · Baoquan He
Quoted 37 lines · Atsushi Kumagai
Quoted 37 lines
Quoted 36 lines

I think the Michael's first idea below is better since it implements is_real_addr() only for --mem-usage as a common function for all architectures, it's explicit design.

Review int get_kcore_dump_loads(void) · line 860

Quoted 9 lines
(quoted diff) +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_real_addr(p->virt_start))
 			continue;
 		loads++;
 	}

However, this code will not work since the argument of is_real_addr() must be physical address. Even unluckily, /proc/kcore's PT_LOAD looks useless for VtoP converting because PhysAddr is always 0:

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x00000000000002a8 0x0000000000000000 0x0000000000000000

                 0x0000000000000a84 0x0000000000000000         0
  LOAD           0x00007fffff601000 0xffffffffff600000 0x0000000000000000
                 0x0000000000001000 0x0000000000001000  RWE    1000
  LOAD           0x00007fff81001000 0xffffffff81000000 0x0000000000000000
                 0x0000000000a1b000 0x0000000000a1b000  RWE    1000
  LOAD           0x0000490000001000 0xffffc90000000000 0x0000000000000000
                 0x00001fffffffffff 0x00001fffffffffff  RWE    1000
  LOAD           0x00007fffa0001000 0xffffffffa0000000 0x0000000000000000
                 0x000000005f000000 0x000000005f000000  RWE    1000

...

So the way using /proc/iomem seems inappropriate, we have to consider other approaches (but I still don't have any good ideas...)

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

Hello Atsushi,

Hmmm ok, sure. For x86 using /proc/iomem does not work because there is no 1:1 mapping for the kernel address space. The kernel/real memory is mapped somewhere at the end, right?

Exactly.

Well, forget to say. I think x86 and x86_64 can use this way either. Since they are also linear mapping though it's not 1:1 mapping. phys can be converted to vrit by adding page_offset directly.

E.g x86_64, the PAGE_OFFSET is 0xffff880000000000, then I can very easily pick the physical region just as be below the start marked.

Program Headers:
Type Offset VirtAddr PhysAddr
FileSiz MemSiz Flags Align
NOTE 0x0000000000000318 0x0000000000000000
0x0000000000000000

                 0x0000000000000b54 0x0000000000000000         0
  LOAD           0x00007fffff601000 0xffffffffff600000
0x0000000000000000
                 0x0000000000001000 0x0000000000001000  RWE    1000
  LOAD           0x00007fff81001000 0xffffffff81000000
0x0000000000000000
                 0x0000000001633000 0x0000000001633000  RWE    1000
  LOAD           0x0000490000001000 0xffffc90000000000
0x0000000000000000
                 0x00001fffffffffff 0x00001fffffffffff  RWE    1000
  LOAD           0x00007fffa0001000 0xffffffffa0000000
0x0000000000000000
                 0x000000005f000000 0x000000005f000000  RWE    1000

*******
LOAD 0x0000080000002000 0xffff880000001000
0x0000000000000000

                 0x000000000009c000 0x000000000009c000  RWE    1000

*******

LOAD 0x00006a0000001000 0xffffea0000000000
0x0000000000000000

                 0x0000000000003000 0x0000000000003000  RWE    1000

*******
LOAD 0x0000080000101000 0xffff880000100000
0x0000000000000000

                 0x000000007fef0000 0x000000007fef0000  RWE    1000

*******

LOAD 0x00006a0000005000 0xffffea0000004000
0x0000000000000000

                 0x0000000001ffc000 0x0000000001ffc000  RWE    1000

**********
LOAD 0x0000080080001000 0xffff880080000000
0x0000000000000000

                 0x000000004fee0000 0x000000004fee0000  RWE    1000

***********

LOAD 0x00006a0002001000 0xffffea0002000000
0x0000000000000000

                 0x00000000013fc000 0x00000000013fc000  RWE    1000

***********
LOAD 0x0000080100001000 0xffff880100000000
0x0000000000000000

                 0x0000000130000000 0x0000000130000000  RWE    1000

************

LOAD 0x00006a0004001000 0xffffea0004000000
0x0000000000000000

                 0x0000000004c00000 0x0000000004c00000  RWE    1000

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#36 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

In reply to Atsushi Kumagai

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 7 lines

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Hello Atsushi,

Great, so let's do that.

@Baoquan:
If you want to use the is_iomem_phys_addr() function also for x86, you could perhaps add an additional patch.

Here the updated patch:
---
[PATCH] makedumpfile: Enable --mem-usage for s390x

Replace is_vmalloc_addr() by is_phys_addr() and implement is_phys_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage" on s390x.

---
 elf_info.c     |    4 ++--
 makedumpfile.c |   26 ++++++++++++++++++++++++++
 makedumpfile.h |   15 ++++++++-------
 3 files changed, 36 insertions(+), 9 deletions(-)
elf_info.c +2 -2
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;
makedumpfile.h +8 -7
@@ -765,7 +765,7 @@ unsigned long long vaddr_to_paddr_arm(un
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* arm */
 
 #ifdef __x86__
@@ -776,7 +776,7 @@ unsigned long long vaddr_to_paddr_x86(un
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* x86 */
 
 #ifdef __x86_64__
@@ -789,7 +789,7 @@ unsigned long long vaddr_to_paddr_x86_64
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
-#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
+#define is_phys_addr(X)		(!is_vmalloc_addr_x86_64(X)
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -800,7 +800,7 @@ unsigned long long vaddr_to_paddr_ppc64(
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -810,7 +810,7 @@ unsigned long long vaddr_to_paddr_ppc(un
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
@@ -820,7 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		is_iomem_phys_addr(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -832,7 +832,7 @@ unsigned long long vaddr_to_paddr_ia64(u
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_iomem_phys_addr(unsigned long addr);
 
 
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#37 lore raw

RE: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 13 lines

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 7 lines

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Quoted 7 lines

Hello Atsushi,

Great, so let's do that.

@Baoquan:
If you want to use the is_iomem_phys_addr() function also for x86, you could perhaps add an additional patch.

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()
...

vaddr_to_paddr_XXX() does VtoP translation *properly*, so it needs several symbols. This design is OK since it's a general method. OTOH, we need a kludge which can be used under any situations and should use it for --mem-usage:

VtoP_kludge_s390x(unsigned long vaddr)

  {
	/* s390 has 1:1 VtoP mapping that starts with zero. */

return vaddr;
}

also x86_64's can be implemented like below:

VtoP_kludge_x86_64(unsigned long vaddr)

  {
	/* if the address is direct mapped, it's easy to translate. */
	unsigned long paddr = vaddr - PAGE_OFFSET;
	return paddr;

}

Quoted 18 lines

Here the updated patch:
---
[PATCH] makedumpfile: Enable --mem-usage for s390x

Replace is_vmalloc_addr() by is_phys_addr() and implement is_phys_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage" on s390x.

---

elf_info.c     |    4 ++--
makedumpfile.c |   26 ++++++++++++++++++++++++++
makedumpfile.h |   15 ++++++++-------
3 files changed, 36 insertions(+), 9 deletions(-)

elf_info.c int get_kcore_dump_loads(void) · line 857

Quoted 5 lines
elf_info.c +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))

This part implicitly does VtoP translation.
Actually it works for s390x but it's not suitable as a general code, so we should use VtoP_kludge(should be better name!) too. Then we can use is_iomem_phys_addr() also for other architecture.

Thanks,
Atsushi Kumagai

Quoted 113 lines
elf_info.c +1 -1
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;
makedumpfile.h +7 -7
@@ -765,7 +765,7 @@ unsigned long long vaddr_to_paddr_arm(un
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* arm */
 
 #ifdef __x86__
@@ -776,7 +776,7 @@ unsigned long long vaddr_to_paddr_x86(un
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* x86 */
 
 #ifdef __x86_64__
@@ -789,7 +789,7 @@ unsigned long long vaddr_to_paddr_x86_64
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
-#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
+#define is_phys_addr(X)		(!is_vmalloc_addr_x86_64(X)
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -800,7 +800,7 @@ unsigned long long vaddr_to_paddr_ppc64(
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -810,7 +810,7 @@ unsigned long long vaddr_to_paddr_ppc(un
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
@@ -820,7 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		is_iomem_phys_addr(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -832,7 +832,7 @@ unsigned long long vaddr_to_paddr_ia64(u
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;

makedumpfile.h int read_disk_dump_header(struct disk_du · line 1573

Quoted 7 lines
makedumpfile.h +1 -0
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_iomem_phys_addr(unsigned long addr);
 
 
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#38 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Hello Atsushi,

Quoted 2 lines · On Thu, 23 Oct 2014 06:56:47 +0000 Atsushi Kumagai…
Quoted 2 lines

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 10 lines

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

Quoted 8 lines

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()
...
vaddr_to_paddr_XXX() does VtoP translation *properly*, so it needs several symbols. This design is OK since it's a general method.

So the current implementation that uses vaddr_to_paddr_xxx() for --mem-usage *before* initial() is broken? Or does it currently works only by chance?

Quoted 9 lines

OTOH, we need a kludge which can be used under any situations and should use it for --mem-usage:

VtoP_kludge_s390x(unsigned long vaddr)

  {
	/* s390 has 1:1 VtoP mapping that starts with zero. */

return vaddr;
}

Quoted 8 lines

also x86_64's can be implemented like below:

VtoP_kludge_x86_64(unsigned long vaddr)

  {
	/* if the address is direct mapped, it's easy to translate. */
	unsigned long paddr = vaddr - PAGE_OFFSET;
	return paddr;

}

What about using the kernel name __pa(vaddr)?

[snip]

Quoted 4 lines
Quoted 4 lines
elf_info.c +0 -0
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
Quoted 9 lines

elf_info.c int get_kcore_dump_loads(void) · line 857

Quoted 4 lines
elf_info.c +1 -1
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))

This part implicitly does VtoP translation.
Actually it works for s390x but it's not suitable as a general code, so we should use VtoP_kludge(should be better name!) too. Then we can use is_iomem_phys_addr() also for other architecture.

So how exactly should the code look like for non s390x architectures?

Michael

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#39 lore raw

RE: [PATCH] makedumpfile: Enable --mem-usage for s390x

Hello Michael,

Quoted 29 lines

Hello Atsushi,

Quoted 2 lines · On Thu, 23 Oct 2014 06:56:47 +0000 Atsushi Kumagai…
Quoted 2 lines

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 10 lines

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

Quoted 8 lines

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()
...
vaddr_to_paddr_XXX() does VtoP translation *properly*, so it needs several symbols. This design is OK since it's a general method.

Quoted 3 lines

So the current implementation that uses vaddr_to_paddr_xxx() for --mem-usage *before* initial() is broken? Or does it currently works only by chance?

Luckily v1.5.7 works if kaslr is disabled, but it can be broken easily in the future due to dependence on other symbols. vaddr_to_paddr_xxx() should be called after initial() in principle.

Quoted 11 lines
Quoted 11 lines

OTOH, we need a kludge which can be used under any situations and should use it for --mem-usage:

VtoP_kludge_s390x(unsigned long vaddr)

  {
	/* s390 has 1:1 VtoP mapping that starts with zero. */

return vaddr;
}

also x86_64's can be implemented like below:

Quoted 8 lines
Quoted 6 lines

VtoP_kludge_x86_64(unsigned long vaddr)

  {
	/* if the address is direct mapped, it's easy to translate. */
	unsigned long paddr = vaddr - PAGE_OFFSET;
	return paddr;

}

What about using the kernel name __pa(vaddr)?

Sounds good, but some symbols are still necessary for __pa() in some architectures according to the kernel, it seems difficult to introduce __pa() as a general method. So now I think is_iomem_phys_addr() isn't suitable for other architectures since it depends on __pa().

Anyway, s390x has the simple __pa(), let's continue to use is_iomem_phys_addr() even though it's only for s390x.

Quoted 16 lines

[snip]

Quoted 4 lines
Quoted 4 lines
elf_info.c +0 -0
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
Quoted 9 lines

elf_info.c int get_kcore_dump_loads(void) · line 857

Quoted 4 lines
elf_info.c +1 -1
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))

This part implicitly does VtoP translation.
Actually it works for s390x but it's not suitable as a general code, so we should use VtoP_kludge(should be better name!) too. Then we can use is_iomem_phys_addr() also for other architecture.

Quoted 3 lines

So how exactly should the code look like for non s390x architectures?

Michael

By early vmcoreinfo initialization, x86_64 will be possible to use is_vmalloc_addr_x86_64() as is_phys_addr(), so your last patch has no problem except missing __pa() in is_iomem_phys_addr().

Now I don't plan to use is_iomem_phys_addr() for non s390x architectures, I prefer putting is_iomem_phys_addr() into arch/s390x.c as Baoquan commented before.

Could you modify the patch ? or any questions ?

Thanks,
Atsushi Kumagai

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#40 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Hello Atsushi,

On Thu, 30 Oct 2014 01:29:18 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 5 lines

Now I don't plan to use is_iomem_phys_addr() for non s390x architectures, I prefer putting is_iomem_phys_addr() into arch/s390x.c as Baoquan commented before.

Could you modify the patch ? or any questions ?

I modified the patch and moved the is_iomem_phys_addr() function to s390x.c. I did not add __pa() because currently I don't see the need for it (?).

The updated patch below applies on top of our "Compile warnings on archs without get_versiondep_info()" patch.

Michael
---
[PATCH] makedumpfile: Enable --mem-usage for s390x

Replace is_vmalloc_addr() by is_phys_addr() and implement is_phys_addr() on s390x using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage".

---
 arch/s390x.c   |   26 ++++++++++++++++++++++++++
 elf_info.c     |    4 ++--
 makedumpfile.h |   20 +++++++++++++-------
 3 files changed, 41 insertions(+), 9 deletions(-)
arch/s390x.c +26 -0
@@ -308,4 +308,30 @@ vaddr_to_paddr_s390x(unsigned long vaddr
 	return paddr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr_s390x(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 #endif /* __s390x__ */
elf_info.c +2 -2
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.h +13 -7
@@ -767,7 +767,7 @@ unsigned long long vaddr_to_paddr_arm(un
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif /* arm */
 
 #ifdef __x86__
@@ -778,7 +778,7 @@ unsigned long long vaddr_to_paddr_x86(un
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif /* x86 */
 
 #ifdef __x86_64__
@@ -791,7 +791,7 @@ unsigned long long vaddr_to_paddr_x86_64
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
-#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
+#define is_phys_addr(X)		(!is_vmalloc_addr_x86_64(X))
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -802,7 +802,7 @@ unsigned long long vaddr_to_paddr_ppc64(
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -812,17 +812,18 @@ unsigned long long vaddr_to_paddr_ppc(un
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
 int get_machdep_info_s390x(void);
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
+int is_iomem_phys_addr_s390x(unsigned long addr);
 #define get_phys_base()		stub_true()
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		is_iomem_phys_addr_s390x(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -834,7 +835,7 @@ unsigned long long vaddr_to_paddr_ia64(u
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;
@@ -1569,6 +1570,11 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int iomem_for_each_line(char *match, int (*callback)(void *data, int nr,
+						     char *str,
+						     unsigned long base,
+						     unsigned long length),
+			void *data);
 
 
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#41 lore raw

RE: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 11 lines

Hello Atsushi,

On Thu, 30 Oct 2014 01:29:18 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 4 lines

Now I don't plan to use is_iomem_phys_addr() for non s390x architectures, I prefer putting is_iomem_phys_addr() into arch/s390x.c as Baoquan commented before.

Quoted 5 lines
Quoted 1 line

Could you modify the patch ? or any questions ?

I modified the patch and moved the is_iomem_phys_addr() function to s390x.c. I did not add __pa() because currently I don't see the need for it (?).

OK, but I'll add a memorandum in case we export this code to other architectures like:

int is_iomem_phys_addr_s390x(unsigned long addr) {
+ /* Implicit VtoP conversion will be performed for addr here. */

          struct addr_check addr_check = {addr, 0};
Quoted 2 lines

The updated patch below applies on top of our "Compile warnings on archs without get_versiondep_info()" patch.

Thanks for your work, I'll merge it into v1.5.8.

Thanks,
Atsushi Kumagai

Quoted 138 lines

Michael
---
[PATCH] makedumpfile: Enable --mem-usage for s390x

Replace is_vmalloc_addr() by is_phys_addr() and implement is_phys_addr() on s390x using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage".

---

arch/s390x.c   |   26 ++++++++++++++++++++++++++
elf_info.c     |    4 ++--
makedumpfile.h |   20 +++++++++++++-------
3 files changed, 41 insertions(+), 9 deletions(-)
arch/s390x.c +26 -0
@@ -308,4 +308,30 @@ vaddr_to_paddr_s390x(unsigned long vaddr
 	return paddr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr_s390x(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 #endif /* __s390x__ */
elf_info.c +2 -2
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.h +8 -7
@@ -767,7 +767,7 @@ unsigned long long vaddr_to_paddr_arm(un
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif /* arm */
 
 #ifdef __x86__
@@ -778,7 +778,7 @@ unsigned long long vaddr_to_paddr_x86(un
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif /* x86 */
 
 #ifdef __x86_64__
@@ -791,7 +791,7 @@ unsigned long long vaddr_to_paddr_x86_64
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
-#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
+#define is_phys_addr(X)		(!is_vmalloc_addr_x86_64(X))
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -802,7 +802,7 @@ unsigned long long vaddr_to_paddr_ppc64(
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -812,17 +812,18 @@ unsigned long long vaddr_to_paddr_ppc(un
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
 int get_machdep_info_s390x(void);
 unsigned long long vaddr_to_paddr_s390x(unsigned long vaddr);
+int is_iomem_phys_addr_s390x(unsigned long addr);
 #define get_phys_base()		stub_true()
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		is_iomem_phys_addr_s390x(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -834,7 +835,7 @@ unsigned long long vaddr_to_paddr_ia64(u
 #define get_versiondep_info()	stub_true()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
-#define is_vmalloc_addr(X)	stub_true_ul(X)
+#define is_phys_addr(X)		stub_true_ul(X)
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;
@@ -1569,6 +1570,11 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);

makedumpfile.h int read_disk_dump_header(struct disk_du · line 1580

Quoted 10 lines
makedumpfile.h +5 -0
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int iomem_for_each_line(char *match, int (*callback)(void *data, int nr,
+						     char *str,
+						     unsigned long base,
+						     unsigned long length),
+			void *data);
 
 
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#42 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

In reply to Atsushi Kumagai
Quoted 28 lines · Atsushi Kumagai
Quoted 13 lines

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 7 lines

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Quoted 7 lines

Hello Atsushi,

Great, so let's do that.

@Baoquan:
If you want to use the is_iomem_phys_addr() function also for x86, you could perhaps add an additional patch.

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

Quoted 8 lines

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()

This is a bug. Seems that get_kcore_dump_loads() has to be called in initial(). Since dumped vmcore need contains kernel text segment. Without setting correct value for MODULES_VADDR, it will be equal to __START_KERNEL_map, then calling is_vmalloc_addr() will filter kernel text mapping segment too.

Now it seems only one way to solve this, that is moving get_kcore_dump_loads() into initial() just after get_value_for_old_linux() is called.

Quoted 174 lines

...

vaddr_to_paddr_XXX() does VtoP translation *properly*, so it needs several symbols. This design is OK since it's a general method. OTOH, we need a kludge which can be used under any situations and should use it for --mem-usage:

VtoP_kludge_s390x(unsigned long vaddr)

  {
	/* s390 has 1:1 VtoP mapping that starts with zero. */

return vaddr;
}

also x86_64's can be implemented like below:

VtoP_kludge_x86_64(unsigned long vaddr)

  {
	/* if the address is direct mapped, it's easy to translate. */
	unsigned long paddr = vaddr - PAGE_OFFSET;
	return paddr;

}

Quoted 18 lines

Here the updated patch:
---
[PATCH] makedumpfile: Enable --mem-usage for s390x

Replace is_vmalloc_addr() by is_phys_addr() and implement is_phys_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage" on s390x.

---

elf_info.c     |    4 ++--
makedumpfile.c |   26 ++++++++++++++++++++++++++
makedumpfile.h |   15 ++++++++-------
3 files changed, 36 insertions(+), 9 deletions(-)

elf_info.c int get_kcore_dump_loads(void) · line 857

Quoted 5 lines
elf_info.c +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))

This part implicitly does VtoP translation.
Actually it works for s390x but it's not suitable as a general code, so we should use VtoP_kludge(should be better name!) too. Then we can use is_iomem_phys_addr() also for other architecture.

Thanks,
Atsushi Kumagai

Quoted 119 lines
elf_info.c +1 -1
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;
makedumpfile.h +8 -7
@@ -765,7 +765,7 @@ unsigned long long vaddr_to_paddr_arm(un
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* arm */
 
 #ifdef __x86__
@@ -776,7 +776,7 @@ unsigned long long vaddr_to_paddr_x86(un
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* x86 */
 
 #ifdef __x86_64__
@@ -789,7 +789,7 @@ unsigned long long vaddr_to_paddr_x86_64
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
-#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
+#define is_phys_addr(X)		(!is_vmalloc_addr_x86_64(X)
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -800,7 +800,7 @@ unsigned long long vaddr_to_paddr_ppc64(
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -810,7 +810,7 @@ unsigned long long vaddr_to_paddr_ppc(un
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
@@ -820,7 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		is_iomem_phys_addr(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -832,7 +832,7 @@ unsigned long long vaddr_to_paddr_ia64(u
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_iomem_phys_addr(unsigned long addr);
 
 
Quoted 2 lines

makedumpfile.h int read_disk_dump_header(struct disk_du · line 1573

Quoted 1 line
makedumpfile.h +0 -0
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#43 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 44 lines · Baoquan He
Quoted 28 lines · Atsushi Kumagai
Quoted 13 lines

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 7 lines

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Quoted 7 lines

Hello Atsushi,

Great, so let's do that.

@Baoquan:
If you want to use the is_iomem_phys_addr() function also for x86, you could perhaps add an additional patch.

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

Quoted 8 lines

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()

This is a bug. Seems that get_kcore_dump_loads() has to be called in initial(). Since dumped vmcore need contains kernel text segment. Without setting correct value for MODULES_VADDR, it will be equal to __START_KERNEL_map, then calling is_vmalloc_addr() will filter kernel text mapping segment too.

Quoted 3 lines

Now it seems only one way to solve this, that is moving get_kcore_dump_loads() into initial() just after get_value_for_old_linux() is called.

Hi Atsushi,

Checking code again, I found with kaslr support we need move vmcoreinfo reading earlier to initialize the MODULES_VADDR, otherwise get_phys_base_x86_64() will be wrong. Since not initializing MODULES_VADDR explicitly its value is equal to __START_KERNEL_map, that means kernel text mapping segment is filtered by is_vmalloc_addr_x86_64().

So could we move below code earlier?

makedumpfile.c +17 -17
index b27ea1e..1dac19f 100644
@@ -3092,6 +3092,23 @@ initial(void)
 		return FALSE;
 	}
 
+	/*
+	 * Get the debug information from /proc/vmcore.
+	 * NOTE: Don't move this code to the above, because the debugging
+	 *       information token by -x/-i option is overwritten by vmcoreinfo
+	 *       in /proc/vmcore. vmcoreinfo in /proc/vmcore is more reliable
+	 *       than -x/-i option.
+	 */
+	if (has_vmcoreinfo()) {
+		get_vmcoreinfo(&offset, &size);
+		if (!read_vmcoreinfo_from_vmcore(offset, size, FALSE))
+			return FALSE;
+		debug_info = TRUE;
+	}
+
+	if (!get_value_for_old_linux())
+		return FALSE;
+
 	if (info->flag_refiltering) {
 		if (info->flag_elf_dumpfile) {
 			MSG("'-E' option is disable, ");
@@ -3189,23 +3206,6 @@ initial(void)
 		}
 	}
 
-	/*
-	 * Get the debug information from /proc/vmcore.
-	 * NOTE: Don't move this code to the above, because the debugging
-	 *       information token by -x/-i option is overwritten by vmcoreinfo
-	 *       in /proc/vmcore. vmcoreinfo in /proc/vmcore is more reliable
-	 *       than -x/-i option.
-	 */
-	if (has_vmcoreinfo()) {
-		get_vmcoreinfo(&offset, &size);
-		if (!read_vmcoreinfo_from_vmcore(offset, size, FALSE))
-			return FALSE;
-		debug_info = TRUE;
-	}
-
-	if (!get_value_for_old_linux())
-		return FALSE;
-
 out:
 	if (!info->page_size) {
 		/*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#44 lore raw

RE: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 52 lines · Baoquan He
Quoted 44 lines · Atsushi Kumagai
Quoted 28 lines
Quoted 13 lines

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 7 lines

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Quoted 7 lines

Hello Atsushi,

Great, so let's do that.

@Baoquan:
If you want to use the is_iomem_phys_addr() function also for x86, you could perhaps add an additional patch.

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

Quoted 8 lines

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()

This is a bug. Seems that get_kcore_dump_loads() has to be called in initial(). Since dumped vmcore need contains kernel text segment. Without setting correct value for MODULES_VADDR, it will be equal to __START_KERNEL_map, then calling is_vmalloc_addr() will filter kernel text mapping segment too.

Quoted 3 lines

Now it seems only one way to solve this, that is moving get_kcore_dump_loads() into initial() just after get_value_for_old_linux() is called.

Hi Atsushi,

Quoted 8 lines

Checking code again, I found with kaslr support we need move vmcoreinfo reading earlier to initialize the MODULES_VADDR, otherwise get_phys_base_x86_64() will be wrong. Since not initializing MODULES_VADDR explicitly its value is equal to __START_KERNEL_map, that means kernel text mapping segment is filtered by is_vmalloc_addr_x86_64().

So could we move below code earlier?

Good catch, but...

Quoted 5 lines
makedumpfile.c +0 -0
index b27ea1e..1dac19f 100644

makedumpfile.c initial(void) · line 3101

Quoted 10 lines
makedumpfile.c +7 -0
@@ -3092,6 +3092,23 @@ initial(void)
 		return FALSE;
 	}
 
+	/*
+	 * Get the debug information from /proc/vmcore.
+	 * NOTE: Don't move this code to the above, because the debugging
+	 *       information token by -x/-i option is overwritten by vmcoreinfo
+	 *       in /proc/vmcore. vmcoreinfo in /proc/vmcore is more reliable
+	 *       than -x/-i option.
+	 */

please see above, this note means we have to keep the reading order of vmcoreinfo, so the code will be like below.
I'll do it on the devel branch soon.

makedumpfile.c +54 -55
index b27ea1e..3075809 100644
@@ -3091,61 +3091,6 @@ initial(void)
 		MSG("Try `makedumpfile --help' for more information.\n");
 		return FALSE;
 	}
-
-	if (info->flag_refiltering) {
-		if (info->flag_elf_dumpfile) {
-			MSG("'-E' option is disable, ");
-			MSG("because %s is kdump compressed format.\n",
-							info->name_memory);
-			return FALSE;
-		}
-
-		if(info->flag_cyclic) {
-			info->flag_cyclic = FALSE;
-			MSG("Switched running mode from cyclic to non-cyclic,\n");
-			MSG("because the cyclic mode doesn't support refiltering\n");
-			MSG("kdump compressed format.\n");
-		}
-
-		info->phys_base = info->kh_memory->phys_base;
-		info->max_dump_level |= info->kh_memory->dump_level;
-
-		if (!initialize_bitmap_memory())
-			return FALSE;
-
-	} else if (info->flag_sadump) {
-		if (info->flag_elf_dumpfile) {
-			MSG("'-E' option is disable, ");
-			MSG("because %s is sadump %s format.\n",
-			    info->name_memory, sadump_format_type_name());
-			return FALSE;
-		}
-
-		if(info->flag_cyclic) {
-			info->flag_cyclic = FALSE;
-			MSG("Switched running mode from cyclic to non-cyclic,\n");
-			MSG("because the cyclic mode doesn't support sadump format.\n");
-		}
-
-		set_page_size(sadump_page_size());
-
-		if (!sadump_initialize_bitmap_memory())
-			return FALSE;
-
-		(void) sadump_set_timestamp(&info->timestamp);
-
-		/*
-		 * NOTE: phys_base is never saved by sadump and so
-		 * must be computed in some way. We here choose the
-		 * way of looking at linux_banner. See
-		 * sadump_virt_phys_base(). The processing is
-		 * postponed until debug information becomes
-		 * available.
-		 */
-
-	} else if (!get_phys_base())
-		return FALSE;
-
 	/*
 	 * Get the debug information for analysis from the vmcoreinfo file
 	 */
@@ -3206,6 +3151,60 @@ initial(void)
 	if (!get_value_for_old_linux())
 		return FALSE;
 
+	if (info->flag_refiltering) {
+		if (info->flag_elf_dumpfile) {
+			MSG("'-E' option is disable, ");
+			MSG("because %s is kdump compressed format.\n",
+							info->name_memory);
+			return FALSE;
+		}
+
+		if(info->flag_cyclic) {
+			info->flag_cyclic = FALSE;
+			MSG("Switched running mode from cyclic to non-cyclic,\n");
+			MSG("because the cyclic mode doesn't support refiltering\n");
+			MSG("kdump compressed format.\n");
+		}
+
+		info->phys_base = info->kh_memory->phys_base;
+		info->max_dump_level |= info->kh_memory->dump_level;
+
+		if (!initialize_bitmap_memory())
+			return FALSE;
+
+	} else if (info->flag_sadump) {
+		if (info->flag_elf_dumpfile) {
+			MSG("'-E' option is disable, ");
+			MSG("because %s is sadump %s format.\n",
+			    info->name_memory, sadump_format_type_name());
+			return FALSE;
+		}
+
+		if(info->flag_cyclic) {
+			info->flag_cyclic = FALSE;
+			MSG("Switched running mode from cyclic to non-cyclic,\n");
+			MSG("because the cyclic mode doesn't support sadump format.\n");
+		}
+
+		set_page_size(sadump_page_size());
+
+		if (!sadump_initialize_bitmap_memory())
+			return FALSE;
+
+		(void) sadump_set_timestamp(&info->timestamp);
+
+		/*
+		 * NOTE: phys_base is never saved by sadump and so
+		 * must be computed in some way. We here choose the
+		 * way of looking at linux_banner. See
+		 * sadump_virt_phys_base(). The processing is
+		 * postponed until debug information becomes
+		 * available.
+		 */
+
+	} else if (!get_phys_base())
+		return FALSE;
+
 out:
 	if (!info->page_size) {
 		/*

Thanks,
Atsushi Kumagai

Quoted 35 lines
makedumpfile.c +10 -17
+	if (has_vmcoreinfo()) {
+		get_vmcoreinfo(&offset, &size);
+		if (!read_vmcoreinfo_from_vmcore(offset, size, FALSE))
+			return FALSE;
+		debug_info = TRUE;
+	}
+
+	if (!get_value_for_old_linux())
+		return FALSE;
+
 	if (info->flag_refiltering) {
 		if (info->flag_elf_dumpfile) {
 			MSG("'-E' option is disable, ");
@@ -3189,23 +3206,6 @@ initial(void)
 		}
 	}
 
-	/*
-	 * Get the debug information from /proc/vmcore.
-	 * NOTE: Don't move this code to the above, because the debugging
-	 *       information token by -x/-i option is overwritten by vmcoreinfo
-	 *       in /proc/vmcore. vmcoreinfo in /proc/vmcore is more reliable
-	 *       than -x/-i option.
-	 */
-	if (has_vmcoreinfo()) {
-		get_vmcoreinfo(&offset, &size);
-		if (!read_vmcoreinfo_from_vmcore(offset, size, FALSE))
-			return FALSE;
-		debug_info = TRUE;
-	}
-
-	if (!get_value_for_old_linux())
-		return FALSE;
-

out:

Quoted 7 lines
	if (!info->page_size) {
		/*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#45 lore raw

RE: [PATCH] makedumpfile: Enable --mem-usage for s390x

In reply to [email protected]
Quoted 44 lines · Atsushi Kumagai
Quoted 28 lines
Quoted 13 lines

On Tue, 14 Oct 2014 07:19:13 +0000
Atsushi Kumagai <[email protected]> wrote:

[snip]

Quoted 7 lines

I understand why your patch works on s390x, so how about this way ?

1. Define "is_phys_addr()" for --mem-usage.
2. Implement is_phys_addr() using is_iomem_phys_addr() for s390x

    while x86_64 uses is_vmalloc_addr_x86_64().

3. Use is_phys_addr() instead of is_vmalloc_addr() in get_kcore_dump_loads().

Quoted 7 lines

Hello Atsushi,

Great, so let's do that.

@Baoquan:
If you want to use the is_iomem_phys_addr() function also for x86, you could perhaps add an additional patch.

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

Quoted 8 lines

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()

This is a bug. Seems that get_kcore_dump_loads() has to be called in initial(). Since dumped vmcore need contains kernel text segment. Without setting correct value for MODULES_VADDR, it will be equal to __START_KERNEL_map, then calling is_vmalloc_addr() will filter kernel text mapping segment too.

Quoted 3 lines

Now it seems only one way to solve this, that is moving get_kcore_dump_loads() into initial() just after get_value_for_old_linux() is called.

I agree, I'll do it, thanks.

Atsushi Kumagai

Quoted 174 lines
Quoted 174 lines

...

vaddr_to_paddr_XXX() does VtoP translation *properly*, so it needs several symbols. This design is OK since it's a general method. OTOH, we need a kludge which can be used under any situations and should use it for --mem-usage:

VtoP_kludge_s390x(unsigned long vaddr)

  {
	/* s390 has 1:1 VtoP mapping that starts with zero. */

return vaddr;
}

also x86_64's can be implemented like below:

VtoP_kludge_x86_64(unsigned long vaddr)

  {
	/* if the address is direct mapped, it's easy to translate. */
	unsigned long paddr = vaddr - PAGE_OFFSET;
	return paddr;

}

Quoted 18 lines

Here the updated patch:
---
[PATCH] makedumpfile: Enable --mem-usage for s390x

Replace is_vmalloc_addr() by is_phys_addr() and implement is_phys_addr() using /proc/iommem parsing to enable the new makedumpfile option "--mem-usage" on s390x.

---

elf_info.c     |    4 ++--
makedumpfile.c |   26 ++++++++++++++++++++++++++
makedumpfile.h |   15 ++++++++-------
3 files changed, 36 insertions(+), 9 deletions(-)

elf_info.c int get_kcore_dump_loads(void) · line 857

Quoted 5 lines
elf_info.c +1 -1
@@ -854,7 +854,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))

This part implicitly does VtoP translation.
Actually it works for s390x but it's not suitable as a general code, so we should use VtoP_kludge(should be better name!) too. Then we can use is_iomem_phys_addr() also for other architecture.

Thanks,
Atsushi Kumagai

Quoted 119 lines
elf_info.c +1 -1
 			continue;
 		loads++;
 	}
@@ -874,7 +874,7 @@ int get_kcore_dump_loads(void)
 
 	for (i = 0, j = 0; i < num_pt_loads; ++i) {
 		struct pt_load_segment *p = &pt_loads[i];
-		if (is_vmalloc_addr(p->virt_start))
+		if (!is_phys_addr(p->virt_start))
 			continue;
 		if (j >= loads)
 			return FALSE;
makedumpfile.c +26 -0
@@ -9227,6 +9227,32 @@ int is_crashkernel_mem_reserved(void)
 	return !!crash_reserved_mem_nr;
 }
 
+struct addr_check {
+	unsigned long addr;
+	int found;
+};
+
+static int phys_addr_callback(void *data, int nr, char *str,
+			      unsigned long base, unsigned long length)
+{
+	struct addr_check *addr_check = data;
+	unsigned long addr = addr_check->addr;
+
+	if (addr >= base && addr < base + length) {
+		addr_check->found = 1;
+		return -1;
+	}
+	return 0;
+}
+
+int is_iomem_phys_addr(unsigned long addr)
+{
+	struct addr_check addr_check = {addr, 0};
+
+	iomem_for_each_line("System RAM\n", phys_addr_callback, &addr_check);
+	return addr_check.found;
+}
+
 static int get_page_offset(void)
 {
 	struct utsname utsname;
makedumpfile.h +8 -7
@@ -765,7 +765,7 @@ unsigned long long vaddr_to_paddr_arm(un
 #define get_machdep_info()	get_machdep_info_arm()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_arm(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* arm */
 
 #ifdef __x86__
@@ -776,7 +776,7 @@ unsigned long long vaddr_to_paddr_x86(un
 #define get_machdep_info()	get_machdep_info_x86()
 #define get_versiondep_info()	get_versiondep_info_x86()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif /* x86 */
 
 #ifdef __x86_64__
@@ -789,7 +789,7 @@ unsigned long long vaddr_to_paddr_x86_64
 #define get_machdep_info()	get_machdep_info_x86_64()
 #define get_versiondep_info()	get_versiondep_info_x86_64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_x86_64(X)
-#define is_vmalloc_addr(X)	is_vmalloc_addr_x86_64(X)
+#define is_phys_addr(X)		(!is_vmalloc_addr_x86_64(X)
 #endif /* x86_64 */
 
 #ifdef __powerpc64__ /* powerpc64 */
@@ -800,7 +800,7 @@ unsigned long long vaddr_to_paddr_ppc64(
 #define get_machdep_info()	get_machdep_info_ppc64()
 #define get_versiondep_info()	get_versiondep_info_ppc64()
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc64(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc64 */
 
 #ifdef __powerpc32__ /* powerpc32 */
@@ -810,7 +810,7 @@ unsigned long long vaddr_to_paddr_ppc(un
 #define get_machdep_info()	get_machdep_info_ppc()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ppc(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* powerpc32 */
 
 #ifdef __s390x__ /* s390x */
@@ -820,7 +820,7 @@ unsigned long long vaddr_to_paddr_s390x(
 #define get_machdep_info()	get_machdep_info_s390x()
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_s390x(X)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		is_iomem_phys_addr(X)
 #endif          /* s390x */
 
 #ifdef __ia64__ /* ia64 */
@@ -832,7 +832,7 @@ unsigned long long vaddr_to_paddr_ia64(u
 #define get_versiondep_info()	TRUE
 #define vaddr_to_paddr(X)	vaddr_to_paddr_ia64(X)
 #define VADDR_REGION(X)		(((unsigned long)(X)) >> REGION_SHIFT)
-#define is_vmalloc_addr(X)	TRUE
+#define is_phys_addr(X)		TRUE
 #endif          /* ia64 */
 
 typedef unsigned long long mdf_pfn_t;
@@ -1567,6 +1567,7 @@ int read_disk_dump_header(struct disk_du
 int read_kdump_sub_header(struct kdump_sub_header *kh, char *filename);
 void close_vmcoreinfo(void);
 int close_files_for_creating_dumpfile(void);
+int is_iomem_phys_addr(unsigned long addr);
 
 
Quoted 2 lines
Quoted 2 lines

makedumpfile.h int read_disk_dump_header(struct disk_du · line 1573

Quoted 1 line
makedumpfile.h +0 -0
 /*

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#46 lore raw

Re: [PATCH] makedumpfile: Enable --mem-usage for s390x

Quoted 24 lines · Atsushi Kumagai
Quoted 24 lines · Atsushi Kumagai
Quoted 8 lines

I noticed that is_vmalloc_addr_x86_64() can't be used as is_phys_addr() due to the "kaslr issue". I fixed it in the common path as below, but --mem-usage still has the same issue since initial() will be invoked after get_kcore_dump_loads().

http://lists.infradead.org/pipermail/kexec/2014-October/012743.html

Quoted 8 lines

I know it's so late, but I began to think the current implementation that invokes vaddr_to_paddr_XXX() before initial() is bad:

show_mem_usage()
+ get_kcore_dump_loads()
+ process_dump_load()
+ vaddr_to_paddr_XXX()
+ initial()

This is a bug. Seems that get_kcore_dump_loads() has to be called in initial(). Since dumped vmcore need contains kernel text segment. Without setting correct value for MODULES_VADDR, it will be equal to __START_KERNEL_map, then calling is_vmalloc_addr() will filter kernel text mapping segment too.

Quoted 5 lines
Quoted 3 lines

Now it seems only one way to solve this, that is moving get_kcore_dump_loads() into initial() just after get_value_for_old_linux() is called.

I agree, I'll do it, thanks.

Actually hpa haven't merged my patchset. Seems he doesn't like current kaslr design. I am not sure if we have gone too far to adjust makedumpfile code to support kaslr. I'll ask him for comments or plans.

Thanks
Baoquan

Quoted 26 lines

Atsushi Kumagai

Quoted 22 lines
Quoted 22 lines

...

vaddr_to_paddr_XXX() does VtoP translation *properly*, so it needs several symbols. This design is OK since it's a general method. OTOH, we need a kludge which can be used under any situations and should use it for --mem-usage:

VtoP_kludge_s390x(unsigned long vaddr)

  {
	/* s390 has 1:1 VtoP mapping that starts with zero. */

return vaddr;
}

also x86_64's can be implemented like below:

VtoP_kludge_x86_64(unsigned long vaddr)

  {
	/* if the address is direct mapped, it's easy to translate. */
	unsigned long paddr = vaddr - PAGE_OFFSET;
	return paddr;

}

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec