Skip to contentSkip to conversation

kdump, vmcore: Map vmcore memory in direct mapping region

RFC PATCH In review HATAYAMA Daisuke 2 participants 8 messages Jan 17, 2013 – Jan 21, 2013

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover kdump, vmcore: Map vmcore memory in direct mapping region

Currently, kdump reads the 1st kernel's memory, called old memory in the source code, using ioremap per a single page. This causes big performance degradation since page tables modification and tlb flush happen each time the single page is read.

This issue turned out from Cliff's kernel-space filtering work.

To avoid calling ioremap, we map a whole 1st kernel's memory targeted as vmcore regions in direct mapping table. By this we got big performance improvement. See the following simple benchmark.

Machine spec:

| CPU    | Intel(R) Xeon(R) CPU E7- 4820  @ 2.00GHz (4 sockets, 8 cores) (*) |
| Memory | 32 GB                                                             |
| Kernel | 3.7 vanilla and with this patch set                               |

(*) only 1 cpu is used in the 2nd kenrel now.

Benchmark:

I executed the following commands on the 2nd kernel and recorded real time.

  $ time dd bs=$((4096 * n)) if=/proc/vmcore of=/dev/null

[3.7 vanilla]

| block size | time      | performance |
|       [KB] |           | [MB/sec]    |
|------------+-----------+-------------|
|          4 | 5m 46.97s | 93.56       |
|          8 | 4m 20.68s | 124.52      |
|         16 | 3m 37.85s | 149.01      |

[3.7 with this patch]

| block size | time   | performance |
|       [KB] |        |    [GB/sec] |
|------------+--------+-------------|
|          4 | 17.59s |        1.85 |
|          8 | 14.73s |        2.20 |
|         16 | 14.26s |        2.28 |
|         32 | 13.38s |        2.43 |
|         64 | 12.77s |        2.54 |
|        128 | 12.41s |        2.62 |
|        256 | 12.50s |        2.60 |
|        512 | 12.37s |        2.62 |
|       1024 | 12.30s |        2.65 |
|       2048 | 12.29s |        2.64 |
|       4096 | 12.32s |        2.63 |

[perf bench]

I also did perf bench mem memcpy -o on the 2nd kenrel like:

# /var/crash/perf bench mem memcpy -o -l 128MB
# Running mem/memcpy benchmark...
# Copying 128MB Bytes ...
       2.854337 GB/Sec (with prefault)

Several trials stably showed around 2.85 [GB/Sec].

Notes:

* Why direct mapping region

I chose direct mapping region because this address space has 64TB length to cover a whole physical memory while vmlloc-and-ioremap region has 16TB only. For some particular machine with huge memory, the latter is already problematic.

In the near future, machine with more than 64TB could occur, but then direct mapping space would also be extended to follow.

* Memory consumption issue on the 2nd kenrel

Typical reserved memory size for the 2nd kerne is 512MB. But if mapping tera-byte memory with 4kB pages, page table size amounts to more than giga bytes.

But direct mapping region is mapped using 1GB and 2MB pages. By this, memory consumption for page table is minimamized in most cases.

Boot debug message tells you how each map is mapped:

vmcore: [oldmem 0000000027000000-000000002708afff]
vmcore: [oldmem 0000000000100000-0000000026ffffff]
vmcore: [oldmem 0000000037000000-000000007b00cfff]
vmcore: [oldmem 0000000100000000-000000087fffffff]
[mem 0x27000000-0x2708afff] page 4k
[mem 0x00100000-0x001fffff] page 4k
[mem 0x00200000-0x26ffffff] page 2M
[mem 0x37000000-0x7affffff] page 2M
[mem 0x7b000000-0x7b00cfff] page 4k
[mem 0x100000000-0x87fffffff] page 1G

where each [oldmem <start>-<end>] is mapped region and I omited some other messages.

TODO:

* Use of init_memory_mapping

init_memory_mapping is used to map memory in direct mapping region both in boot time and memory hot-plug codes. This should be used here too, but just as I explain in the patch description, I faced some page-fault related bugs after it was called in the 2nd kernel boot. This means page table mapping is not done correctly.

As a workaround, I wrote the code constructing page table from scratch just like Cliff's patch, and it works well aparently now.

But ideally it's necessary to know why init_memory_mapping doesn't work well. I continue to debug this. Sugestion around this is very helpful. This issue comes purely from lack of my familiality around here (^^;

* Benchmark of Cliff's kernel-space filtering

He has attempted kernel-space filtering of makedumpfile for performance improvement. I noticed the ioremap issue through his this work.

I now think bad performance is mainly caused by the ioremap issue. I don't know how much filtering performance is improved by doing it in kernel-space. I guess there's just a similar improvement just like increasing block size just as the above benchmark.

Anyway, we need first to compare kernel-space filtering with user-space one.

Note that this work is orthogonal to kernel-space filtering, can be proceeded separately.

---

HATAYAMA Daisuke (3):
vmcore: read vmcore through direct mapping region
vmcore: map vmcore memory in direct mapping region
vmcore: Add function to merge memory mapping of vmcore

 fs/proc/vmcore.c |  420 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 419 insertions(+), 1 deletions(-)
Signature · 2 lines
Thanks.
HATAYAMA, Daisuke
#1 lore raw

RFC PATCH 1/3 vmcore: Add function to merge memory mapping of vmcore

vmcore_list has memory map information in the 1st kernel, each of which represents position and size of the objects like:

1) NT_PRSTATUS x the number of lcpus
2) VMCOREINFO
3) kernel code
4) copy of the first 640kB memory
5) System RAM entries

where in /proc/vmcore, 1) and 2) are visible as a single PT_NOTE entry, and 5) as PT_LOAD entries.

This mapping is never exclusive. For example, any of 1), 2) and 4) is always contained in one of the System RAM entries.

I add function oldmem_merge_vmcore_list that merges ranges represented by vmcore_list and makes a merged list in oldmem_list.

We'll remap ranges represented by oldmem_list in direct mapping region in the patch set that follows this patch.

---

 fs/proc/vmcore.c |   83 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 83 insertions(+), 0 deletions(-)
fs/proc/vmcore.c +83 -0
index 0d5071d..405b5e2 100644
@@ -27,6 +27,11 @@
  */
 static LIST_HEAD(vmcore_list);
 
+/* Remap chunks of contiguous memory represented by this list in
+ * direct mapping region.
+ */
+static LIST_HEAD(oldmem_list);
+
 /* Stores the pointer to the buffer containing kernel elf core headers. */
 static char *elfcorebuf;
 static size_t elfcorebuf_sz;
@@ -137,6 +142,84 @@ static u64 map_offset_to_paddr(loff_t offset, struct list_head *vc_list,
 	return 0;
 }
 
+static struct vmcore* __init get_new_element(void);
+
+static int
+oldmem_merge_vmcore_list_one(struct vmcore *r, struct list_head *new_list)
+{
+	unsigned long m_start, m_end, n_start, n_end;
+	struct vmcore _m, *m, *n, *new;
+
+	m = &_m;
+	m->paddr = r->paddr;
+	m->size = r->size;
+	m->offset = r->offset;
+
+retry:
+	list_for_each_entry(n, new_list, list) {
+
+		m_start = m->paddr;
+		m_end = m->paddr + m->size - 1;
+
+		n_start = n->paddr;
+		n_end = n->paddr + n->size - 1;
+
+		/* not mergeable */
+		if (((m_start < n_start) && (m_end < n_start))
+		    || ((n_start < m_start) && (n_end < m_start)))
+			continue;
+
+		/* merge n to m */
+		m->paddr = min(m->paddr, n->paddr);
+		m->size = max(m_end, n_end) - min(m_start, n_start) + 1;
+		m->offset = min(m->offset, n->offset);
+
+		/* n is no longer useful, delete it */
+		list_del(&n->list);
+		kfree(n);
+
+		goto retry;
+	}
+
+	/* there's no map in new_list to merge m, create new element */
+	new = get_new_element();
+	if (!new)
+		return -ENOMEM;
+
+	new->paddr = m->paddr;
+	new->size = m->size;
+	new->offset = m->offset;
+
+	list_add_tail(&new->list, new_list);
+
+	return 0;
+}
+
+static int
+oldmem_merge_vmcore_list(struct list_head *vc_list, struct list_head *om_list)
+{
+	struct vmcore *m;
+	int ret;
+
+	list_for_each_entry(m, vc_list, list) {
+		printk("vmcore: [mem %016llx-%016llx]\n",
+		       m->paddr, m->paddr + m->size - 1);
+	}
+
+	list_for_each_entry(m, vc_list, list) {
+		ret = oldmem_merge_vmcore_list_one(m, om_list);
+		if (ret < 0)
+			return ret;
+	}
+
+	list_for_each_entry(m, om_list, list) {
+		printk("vmcore: [oldmem %016llx-%016llx]\n",
+		       m->paddr, m->paddr + m->size - 1);
+	}
+
+	return 0;
+}
+
 /* Read from the ELF header and then the crash dump. On error, negative value is
  * returned otherwise number of bytes read are returned.
  */
#2 lore raw

RFC PATCH 2/3 vmcore: map vmcore memory in direct mapping region

In reply to HATAYAMA Daisuke

Map memory map regions represented by vmcore in direct mapping region, where as much memory as possible are mapped using 1G or 4M pages to reduce memory consumption for page tables.

I resued large part of init_memory_mapping. In fact, I first tried to use it but I have faced some page-fault related bug that seems to be caused by this additional mapping. I have not figured out the cause yet so I wrote part of making page tables from scratch like Cliff's patch.

---

 fs/proc/vmcore.c |  292 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 files changed, 292 insertions(+), 0 deletions(-)
fs/proc/vmcore.c +292 -0
index 405b5e2..aa14570 100644
@@ -21,6 +21,8 @@
 #include <linux/list.h>
 #include <asm/uaccess.h>
 #include <asm/io.h>
+#include <asm/tlbflush.h>
+#include <asm/pgalloc.h>
 
 /* List representing chunks of contiguous memory areas and their offsets in
  * vmcore file.
@@ -220,6 +222,290 @@ oldmem_merge_vmcore_list(struct list_head *vc_list, struct list_head *om_list)
 	return 0;
 }
 
+enum {
+        NR_RANGE_MR = 5,
+};
+
+struct map_range {
+	unsigned long start;
+	unsigned long end;
+	unsigned page_size_mask;
+};
+
+static int save_mr(struct map_range *mr, int nr_range,
+		   unsigned long start_pfn, unsigned long end_pfn,
+		   unsigned long page_size_mask)
+{
+	if (start_pfn < end_pfn) {
+		if (nr_range >= NR_RANGE_MR)
+			panic("run out of range for init_memory_mapping\n");
+		mr[nr_range].start = start_pfn<<PAGE_SHIFT;
+		mr[nr_range].end   = end_pfn<<PAGE_SHIFT;
+		mr[nr_range].page_size_mask = page_size_mask;
+		nr_range++;
+	}
+
+	return nr_range;
+}
+
+static int
+oldmem_align_maps_in_page_size(struct map_range *mr,
+			       unsigned long start,
+			       unsigned long end)
+{
+        unsigned long page_size_mask = 0;
+        unsigned long start_pfn, end_pfn;
+        unsigned long pos;
+        int use_pse, use_gbpages;
+        int i, nr_range;
+
+#if defined(CONFIG_DEBUG_PAGEALLOC) || defined(CONFIG_KMEMCHECK)
+	/*
+	 * For CONFIG_DEBUG_PAGEALLOC, identity mapping will use small pages.
+	 * This will simplify cpa(), which otherwise needs to support splitting
+	 * large pages into small in interrupt context, etc.
+	 */
+	use_pse = use_gbpages = 0;
+#else
+	use_pse = cpu_has_pse;
+	use_gbpages = direct_gbpages;
+#endif
+
+	/* Enable PSE if available */
+	if (cpu_has_pse)
+		set_in_cr4(X86_CR4_PSE);
+
+	/* Enable PGE if available */
+	if (cpu_has_pge) {
+		set_in_cr4(X86_CR4_PGE);
+		__supported_pte_mask |= _PAGE_GLOBAL;
+	}
+
+	if (use_gbpages)
+		page_size_mask |= 1 << PG_LEVEL_1G;
+	if (use_pse)
+		page_size_mask |= 1 << PG_LEVEL_2M;
+
+	memset(mr, 0, sizeof(mr));
+	nr_range = 0;
+
+	/* head if not big page alignment ? */
+	start_pfn = start >> PAGE_SHIFT;
+	pos = start_pfn << PAGE_SHIFT;
+#ifdef CONFIG_X86_32
+	/*
+	 * Don't use a large page for the first 2/4MB of memory
+	 * because there are often fixed size MTRRs in there
+	 * and overlapping MTRRs into large pages can cause
+	 * slowdowns.
+	 */
+	if (pos == 0)
+		end_pfn = 1<<(PMD_SHIFT - PAGE_SHIFT);
+	else
+		end_pfn = ((pos + (PMD_SIZE - 1))>>PMD_SHIFT)
+				 << (PMD_SHIFT - PAGE_SHIFT);
+#else /* CONFIG_X86_64 */
+	end_pfn = ((pos + (PMD_SIZE - 1)) >> PMD_SHIFT)
+			<< (PMD_SHIFT - PAGE_SHIFT);
+#endif
+	if (end_pfn > (end >> PAGE_SHIFT))
+		end_pfn = end >> PAGE_SHIFT;
+	if (start_pfn < end_pfn) {
+		nr_range = save_mr(mr, nr_range, start_pfn, end_pfn, 0);
+		pos = end_pfn << PAGE_SHIFT;
+	}
+
+	/* big page (2M) range */
+	start_pfn = ((pos + (PMD_SIZE - 1))>>PMD_SHIFT)
+			 << (PMD_SHIFT - PAGE_SHIFT);
+#ifdef CONFIG_X86_32
+	end_pfn = (end>>PMD_SHIFT) << (PMD_SHIFT - PAGE_SHIFT);
+#else /* CONFIG_X86_64 */
+	end_pfn = ((pos + (PUD_SIZE - 1))>>PUD_SHIFT)
+			 << (PUD_SHIFT - PAGE_SHIFT);
+	if (end_pfn > ((end>>PMD_SHIFT)<<(PMD_SHIFT - PAGE_SHIFT)))
+		end_pfn = ((end>>PMD_SHIFT)<<(PMD_SHIFT - PAGE_SHIFT));
+#endif
+
+	if (start_pfn < end_pfn) {
+		nr_range = save_mr(mr, nr_range, start_pfn, end_pfn,
+				page_size_mask & (1<<PG_LEVEL_2M));
+		pos = end_pfn << PAGE_SHIFT;
+	}
+
+#ifdef CONFIG_X86_64
+	/* big page (1G) range */
+	start_pfn = ((pos + (PUD_SIZE - 1))>>PUD_SHIFT)
+			 << (PUD_SHIFT - PAGE_SHIFT);
+	end_pfn = (end >> PUD_SHIFT) << (PUD_SHIFT - PAGE_SHIFT);
+	if (start_pfn < end_pfn) {
+		nr_range = save_mr(mr, nr_range, start_pfn, end_pfn,
+				page_size_mask &
+				 ((1<<PG_LEVEL_2M)|(1<<PG_LEVEL_1G)));
+		pos = end_pfn << PAGE_SHIFT;
+	}
+
+	/* tail is not big page (1G) alignment */
+	start_pfn = ((pos + (PMD_SIZE - 1))>>PMD_SHIFT)
+			 << (PMD_SHIFT - PAGE_SHIFT);
+	end_pfn = (end >> PMD_SHIFT) << (PMD_SHIFT - PAGE_SHIFT);
+	if (start_pfn < end_pfn) {
+		nr_range = save_mr(mr, nr_range, start_pfn, end_pfn,
+				page_size_mask & (1<<PG_LEVEL_2M));
+		pos = end_pfn << PAGE_SHIFT;
+	}
+#endif
+
+	/* tail is not big page (2M) alignment */
+	start_pfn = pos>>PAGE_SHIFT;
+	end_pfn = end>>PAGE_SHIFT;
+	nr_range = save_mr(mr, nr_range, start_pfn, end_pfn, 0);
+
+	/* try to merge same page size and continuous */
+	for (i = 0; nr_range > 1 && i < nr_range - 1; i++) {
+		unsigned long old_start;
+		if (mr[i].end != mr[i+1].start ||
+		    mr[i].page_size_mask != mr[i+1].page_size_mask)
+			continue;
+		/* move it */
+		old_start = mr[i].start;
+		memmove(&mr[i], &mr[i+1],
+			(nr_range - 1 - i) * sizeof(struct map_range));
+		mr[i--].start = old_start;
+		nr_range--;
+	}
+
+	return nr_range;
+}
+
+static int
+oldmem_physical_mapping_init(unsigned long start, unsigned long end,
+			     unsigned long page_size_mask)
+{
+	unsigned long paddr, vaddr, hpagesize;
+	pgd_t *pgd;
+	pud_t *pud;
+	pmd_t *pmd;
+	pte_t *pte;
+	pgprot_t prot;
+	unsigned long pages_1G, pages_2M, pages_4K;
+	unsigned long pages_PUD, pages_PMD, pages_PTE;
+
+	if (page_size_mask & (1 << PG_LEVEL_1G)) {
+		hpagesize = PUD_SIZE;
+		prot = PAGE_KERNEL_LARGE;
+	} else if (page_size_mask & (1 << PG_LEVEL_2M)) {
+		hpagesize = PMD_SIZE;
+		prot = PAGE_KERNEL_LARGE;
+	} else {
+		hpagesize = PAGE_SIZE;
+		prot = PAGE_KERNEL;
+	}
+
+	paddr = start;
+	vaddr = (unsigned long)__va(start);
+
+	pages_1G = 0;
+	pages_2M = 0;
+	pages_4K = 0;
+
+	pages_PUD = 0;
+	pages_PMD = 0;
+	pages_PTE = 0;
+
+	while (paddr < end) {
+		pgd = pgd_offset_k(vaddr);
+		if (!pgd_present(*pgd)) {
+			pud = pud_alloc_one(&init_mm, vaddr);
+			set_pgd(pgd, __pgd(__pa(pud) | _KERNPG_TABLE));
+			pages_PUD++;
+		}
+		pud = pud_offset(pgd, vaddr);
+		if (page_size_mask & (1 << PG_LEVEL_1G)) {
+			set_pud(pud, __pud(paddr | pgprot_val(prot)));
+			pages_1G++;
+		} else {
+			if (!pud_present(*pud)) {
+				pmd = pmd_alloc_one(&init_mm, vaddr);
+				set_pud(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
+				pages_PMD++;
+			}
+			pmd = pmd_offset(pud, vaddr);
+			if (page_size_mask & (1 << PG_LEVEL_2M)) {
+				set_pmd(pmd, __pmd(paddr | pgprot_val(prot)));
+				pages_2M++;
+			} else {
+				if (!pmd_present(*pmd)) {
+					pte = pte_alloc_one_kernel(&init_mm, vaddr);
+					set_pmd(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
+					pages_PTE++;
+				}
+				pte = pte_offset_kernel(pmd, vaddr);
+				set_pte(pte, __pte(paddr | pgprot_val(prot)));
+				pages_4K++;
+			}
+		}
+		if (end - paddr < hpagesize)
+			break;
+		paddr += hpagesize;
+		vaddr += hpagesize;
+	}
+
+	update_page_count(PG_LEVEL_1G, pages_1G);
+	update_page_count(PG_LEVEL_2M, pages_2M);
+	update_page_count(PG_LEVEL_4K, pages_4K);
+
+	printk("vmcore: PUD pages: %lu\n", pages_PUD);
+	printk("vmcore: PMD pages: %lu\n", pages_PMD);
+	printk("vmcore: PTE pages: %lu\n", pages_PTE);
+
+	__flush_tlb_all();
+
+	return 0;
+}
+
+static void init_old_memory_mapping(unsigned long start, unsigned long end)
+{
+	struct map_range mr[NR_RANGE_MR];
+	int i, ret, nr_range;
+
+	nr_range = oldmem_align_maps_in_page_size(mr, start, end);
+
+	for (i = 0; i < nr_range; i++)
+		printk(KERN_DEBUG " [mem %#010lx-%#010lx] page %s\n",
+		       mr[i].start, mr[i].end - 1,
+		       (mr[i].page_size_mask & (1<<PG_LEVEL_1G))?"1G":(
+			       (mr[i].page_size_mask & (1<<PG_LEVEL_2M))?"2M":"4k"));
+
+	for (i = 0; i < nr_range; i++)
+		ret = oldmem_physical_mapping_init(mr[i].start,
+						   mr[i].end,
+						   mr[i].page_size_mask);
+
+	__flush_tlb_all();
+}
+
+static int oldmem_init(struct list_head *vc_list, struct list_head *om_list)
+{
+	struct vmcore *m;
+	int ret;
+
+	ret = oldmem_merge_vmcore_list(vc_list, om_list);
+	if (ret < 0)
+		return ret;
+
+	list_for_each_entry(m, om_list, list) {
+		unsigned long start, end;
+
+		start = (m->paddr >> PAGE_SHIFT) << PAGE_SHIFT;
+		end = ((m->paddr + m->size + PAGE_SIZE - 1) >> PAGE_SHIFT) << PAGE_SHIFT;
+
+		init_old_memory_mapping(start, end);
+	}
+
+	return 0;
+}
+
 /* Read from the ELF header and then the crash dump. On error, negative value is
  * returned otherwise number of bytes read are returned.
  */
@@ -777,6 +1063,12 @@ static int __init vmcore_init(void)
 		return rc;
 	}
 
+	rc = oldmem_init(&vmcore_list, &oldmem_list);
+	if (rc) {
+		printk(KERN_WARNING "Kdump: failed to map vmcore\n");
+		return rc;
+	}
+
 	proc_vmcore = proc_create("vmcore", S_IRUSR, NULL, &proc_vmcore_operations);
 	if (proc_vmcore)
 		proc_vmcore->size = vmcore_size;

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#3 lore raw

RFC PATCH 3/3 vmcore: read vmcore through direct mapping region

In reply to HATAYAMA Daisuke

Now regions represented by vmcore are mapped through direct mapping region. We reads requested memory through direct mapping region instead of using ioremap.

Notice that we still keep read_from_oldmem that uses ioremap because we need to use it when reading elf headers to make vmcore_list in vmcore initialization.

---

 fs/proc/vmcore.c |   45 ++++++++++++++++++++++++++++++++++++++++++++-
 1 files changed, 44 insertions(+), 1 deletions(-)
fs/proc/vmcore.c +44 -1
index aa14570..1c6259e 100644
@@ -123,6 +123,49 @@ static ssize_t read_from_oldmem(char *buf, size_t count,
 	return read;
 }
 
+/* Reads a page from the oldmem device from given offset. */
+static ssize_t read_from_oldmem_noioremap(char *buf, size_t count,
+					  u64 *ppos, int userbuf)
+{
+        unsigned long pfn, offset;
+        size_t nr_bytes;
+        ssize_t read = 0;
+
+        if (!count)
+                return 0;
+
+        offset = (unsigned long)(*ppos % PAGE_SIZE);
+        pfn = (unsigned long)(*ppos / PAGE_SIZE);
+
+        do {
+                if (count > (PAGE_SIZE - offset))
+                        nr_bytes = PAGE_SIZE - offset;
+                else
+                        nr_bytes = count;
+
+                /* If pfn is not ram, return zeros for sparse dump files */
+                if (pfn_is_ram(pfn) == 0)
+                        memset(buf, 0, nr_bytes);
+                else {
+                        void *vaddr = pfn_to_kaddr(pfn);
+
+                        if (userbuf) {
+                                if (copy_to_user(buf, vaddr + offset, nr_bytes))
+                                        return -EFAULT;
+                        } else
+                                memcpy(buf, vaddr + offset, nr_bytes);
+                }
+                *ppos += nr_bytes;
+                count -= nr_bytes;
+                buf += nr_bytes;
+                read += nr_bytes;
+                ++pfn;
+                offset = 0;
+        } while (count);
+
+        return read;
+}
+
 /* Maps vmcore file offset to respective physical address in memroy. */
 static u64 map_offset_to_paddr(loff_t offset, struct list_head *vc_list,
 					struct vmcore **m_ptr)
@@ -553,7 +596,7 @@ static ssize_t read_vmcore(struct file *file, char __user *buffer,
 		tsz = nr_bytes;
 
 	while (buflen) {
-		tmp = read_from_oldmem(buffer, tsz, &start, 1);
+		tmp = read_from_oldmem_noioremap(buffer, tsz, &start, 1);
 		if (tmp < 0)
 			return tmp;
 		buflen -= tsz;

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#4 lore raw
In reply to HATAYAMA Daisuke
Quoted 37 lines · HATAYAMA Daisuke

Currently, kdump reads the 1st kernel's memory, called old memory in the source code, using ioremap per a single page. This causes big performance degradation since page tables modification and tlb flush happen each time the single page is read.

This issue turned out from Cliff's kernel-space filtering work.

To avoid calling ioremap, we map a whole 1st kernel's memory targeted as vmcore regions in direct mapping table. By this we got big performance improvement. See the following simple benchmark.

Machine spec:

| CPU    | Intel(R) Xeon(R) CPU E7- 4820  @ 2.00GHz (4 sockets, 8 cores) (*) |
| Memory | 32 GB                                                             |
| Kernel | 3.7 vanilla and with this patch set                               |

(*) only 1 cpu is used in the 2nd kenrel now.

Benchmark:

I executed the following commands on the 2nd kernel and recorded real time.

  $ time dd bs=$((4096 * n)) if=/proc/vmcore of=/dev/null

[3.7 vanilla]

| block size | time      | performance |
|       [KB] |           | [MB/sec]    |
|------------+-----------+-------------|
|          4 | 5m 46.97s | 93.56       |
|          8 | 4m 20.68s | 124.52      |
|         16 | 3m 37.85s | 149.01      |

[3.7 with this patch]

Quoted 15 lines
| block size | time   | performance |
|       [KB] |        |    [GB/sec] |
|------------+--------+-------------|
|          4 | 17.59s |        1.85 |
|          8 | 14.73s |        2.20 |
|         16 | 14.26s |        2.28 |
|         32 | 13.38s |        2.43 |
|         64 | 12.77s |        2.54 |
|        128 | 12.41s |        2.62 |
|        256 | 12.50s |        2.60 |
|        512 | 12.37s |        2.62 |
|       1024 | 12.30s |        2.65 |
|       2048 | 12.29s |        2.64 |
|       4096 | 12.32s |        2.63 |

These are impressive improvements. I missed the discussion on mmap(). So why couldn't we provide mmap() interface for /proc/vmcore. If that works then application can select to mmap/unmap bigger chunks of file (instead ioremap mapping/remapping a page at a time).

And if application controls the size of mapping, then it can vary the size of mapping based on available amount of free memory. That way if somebody reserves less amount of memory, we could still dump but with some time penalty.

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#5 lore raw

From: Vivek Goyal <[email protected]>
Subject: Re: [RFC PATCH v1 0/3] kdump, vmcore: Map vmcore memory in direct mapping region
Date: Thu, 17 Jan 2013 17:13:48 -0500

Quoted 59 lines · HATAYAMA Daisuke
Quoted 37 lines

Currently, kdump reads the 1st kernel's memory, called old memory in the source code, using ioremap per a single page. This causes big performance degradation since page tables modification and tlb flush happen each time the single page is read.

This issue turned out from Cliff's kernel-space filtering work.

To avoid calling ioremap, we map a whole 1st kernel's memory targeted as vmcore regions in direct mapping table. By this we got big performance improvement. See the following simple benchmark.

Machine spec:

| CPU    | Intel(R) Xeon(R) CPU E7- 4820  @ 2.00GHz (4 sockets, 8 cores) (*) |
| Memory | 32 GB                                                             |
| Kernel | 3.7 vanilla and with this patch set                               |

(*) only 1 cpu is used in the 2nd kenrel now.

Benchmark:

I executed the following commands on the 2nd kernel and recorded real time.

  $ time dd bs=$((4096 * n)) if=/proc/vmcore of=/dev/null

[3.7 vanilla]

| block size | time      | performance |
|       [KB] |           | [MB/sec]    |
|------------+-----------+-------------|
|          4 | 5m 46.97s | 93.56       |
|          8 | 4m 20.68s | 124.52      |
|         16 | 3m 37.85s | 149.01      |

[3.7 with this patch]

Quoted 15 lines
| block size | time   | performance |
|       [KB] |        |    [GB/sec] |
|------------+--------+-------------|
|          4 | 17.59s |        1.85 |
|          8 | 14.73s |        2.20 |
|         16 | 14.26s |        2.28 |
|         32 | 13.38s |        2.43 |
|         64 | 12.77s |        2.54 |
|        128 | 12.41s |        2.62 |
|        256 | 12.50s |        2.60 |
|        512 | 12.37s |        2.62 |
|       1024 | 12.30s |        2.65 |
|       2048 | 12.29s |        2.64 |
|       4096 | 12.32s |        2.63 |

These are impressive improvements. I missed the discussion on mmap(). So why couldn't we provide mmap() interface for /proc/vmcore. If that works then application can select to mmap/unmap bigger chunks of file (instead ioremap mapping/remapping a page at a time).

Quoted 5 lines

And if application controls the size of mapping, then it can vary the size of mapping based on available amount of free memory. That way if somebody reserves less amount of memory, we could still dump but with some time penalty.

mmap() needs user-space page table in addition to kernel-space's, and it looks that remap_pfn_range() that creates the user-space page table, doesn't support large pages, only 4KB pages. If mmaping small chunks only for small memory programming, then we would again face the same issue as with ioremap. I don't know whether hugetlbfs supports mmap and 1GB page now.

Another idea to reduce size of page table is to extend mapping ranges to cover a whole memory as many 1GB pages as possible. For example, supporse M is size of system memory, then total size of PGD and PUD pages to cover M is:

( 1 + roundup(M, 512GB) / 512GB ) * PAGE_SIZE

     ~     ~~~~~~~~~~~~~~~~~~~~~~~~~
     ^                 ^
     |                 |

PGD page PUD pages

Ideally, 2TB system can be covered with 20KB and 16TB with 132KB only.

So I first want to evaluate this logic. Although I've not seen actually yet, I expect most of memory maps on tera-byte memory machines consists of 1GB-aligned huge chunks.

Thanks.
HATAYAMA, Daisuke

#6 lore raw

On Fri, Jan 18, 2013 at 11:06:59PM +0900, HATAYAMA Daisuke wrote:

[..]

Quoted 5 lines
Quoted 5 lines

These are impressive improvements. I missed the discussion on mmap(). So why couldn't we provide mmap() interface for /proc/vmcore. If that works then application can select to mmap/unmap bigger chunks of file (instead ioremap mapping/remapping a page at a time).

Quoted 7 lines
Quoted 5 lines

And if application controls the size of mapping, then it can vary the size of mapping based on available amount of free memory. That way if somebody reserves less amount of memory, we could still dump but with some time penalty.

mmap() needs user-space page table in addition to kernel-space's,

[ CC Rik van Riel]

I was chatting with Rik and it does not look like that there is any fundamental requirement that range of pfn being mapped in user tables has to be mapped in kernel tables too. Did you run into specific issue.

Quoted 3 lines

and
it looks that remap_pfn_range() that creates the user-space page table, doesn't support large pages, only 4KB pages.

This indeed looks like the case. May be we can enahnce remap_pfn_range() to take an argument and create larger size mappings.

Quoted 3 lines

If mmaping small
chunks only for small memory programming, then we would again face the same issue as with ioremap.

Even if it is 4KB pages, I think it will still be faster than current interface. Because we will not be issuing these many tlb flushes. (Assuming makedumpfile has been modified to map/unap large areas of /proc/vmcore).

Thanks
Vivek

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#7 lore raw

From: Vivek Goyal <[email protected]>
Subject: Re: [RFC PATCH v1 0/3] kdump, vmcore: Map vmcore memory in direct mapping region
Date: Fri, 18 Jan 2013 15:54:13 -0500

Quoted 14 lines · HATAYAMA Daisuke

[..]

Quoted 11 lines
Quoted 10 lines

These are impressive improvements. I missed the discussion on mmap(). So why couldn't we provide mmap() interface for /proc/vmcore. If that works then application can select to mmap/unmap bigger chunks of file (instead ioremap mapping/remapping a page at a time).

And if application controls the size of mapping, then it can vary the size of mapping based on available amount of free memory. That way if somebody reserves less amount of memory, we could still dump but with some time penalty.

Quoted 8 lines
Quoted 1 line

mmap() needs user-space page table in addition to kernel-space's,

[ CC Rik van Riel]

I was chatting with Rik and it does not look like that there is any fundamental requirement that range of pfn being mapped in user tables has to be mapped in kernel tables too. Did you run into specific issue.

No, I was confused simply this around.

Quoted 7 lines
Quoted 3 lines

and
it looks that remap_pfn_range() that creates the user-space page table, doesn't support large pages, only 4KB pages.

This indeed looks like the case. May be we can enahnce remap_pfn_range() to take an argument and create larger size mappings.

Adding a new argument to remap_pfn_range would never easily be accepted because it changes signature of it. It is the function that is exported to modules.

As init_memory_mapping does, it should internally automatically divide a given ranges of kernel address space into properly aligned ones then remap them.

Also, if we extend this in the future, we need to have some feature for userland to know a given kernel can use 2MB/1GB pages for remapping. makedumpfile needs to estimate how much memory is required for the remapping.

Quoted 9 lines
Quoted 3 lines

If mmaping small
chunks only for small memory programming, then we would again face the same issue as with ioremap.

Even if it is 4KB pages, I think it will still be faster than current interface. Because we will not be issuing these many tlb flushes. (Assuming makedumpfile has been modified to map/unap large areas of /proc/vmcore).

OK, I'll go in this direction first. From my local investigation, I'm beginning with thinking that my idea to map a whole DIMM ranges in direct mapping region is difficult due to some memory hot-plug issues, and mmap interface is more useful than keeping page table handling in /proc/vmcore when we process /proc/vmcore in paralell where each process reads different range.

Assuming we can use 4KB pages only, if we use 1MB buffer for page table, we can cover about 500MB memory region. Then, remapping is done about 2000 times. On ioremap case, remapping is done 268435456 times. Peformacne should be improved so much. We should benchmark this first.

Thanks.
HATAYAMA, Daisuke