Skip to contentSkip to conversation

arm64: handle "reserved" entries in /proc/iomem

PATCH In review Masayoshi Mizuma 2 participants 5 messages Dec 18, 2019 – Dec 23, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v3 cover arm64: handle "reserved" entries in /proc/iomem

In recent arm64 kernels, /proc/iomem has an extended file format like:

40000000-5871ffff : System RAM
41800000-426affff : Kernel code
426b0000-42aaffff : reserved
42ab0000-42c64fff : Kernel data
54400000-583fffff : Crash kernel
58590000-585effff : reserved
58700000-5871ffff : reserved
58720000-58b5ffff : reserved
58b60000-5be3ffff : System RAM
58b61000-58b61fff : reserved

where "reserved" entries can be an ACPI table, UEFI related code or data. They can be corrupted and result in early failure in booting a new kernel. As an actual example, LPI pending table and LPI property table, which are pointed by a UEFI data, are sometimes destroyed.

They are expected to be preserved across kexec'ing.

Changelog:
v3: - Re-based to the latest commit (bd07796).
- Added Tested-by tag from Bhupesh and Masayoshi
- Added an error handling in case mem_regions_alloc_and_exclude() fails (0002 patch).

AKASHI Takahiro (3):
kexec: add variant helper functions for handling memory regions
arm64: kexec: allocate memory space avoiding reserved regions
arm64: kdump: deal with a lot of resource entries in /proc/iomem

 kexec/arch/arm64/crashdump-arm64.c |  25 ++---
 kexec/arch/arm64/kexec-arm64.c     | 153 ++++++++++++++++++-----------
 kexec/mem_regions.c                |  42 ++++++++
 kexec/mem_regions.h                |   7 ++
 4 files changed, 153 insertions(+), 74 deletions(-)
Signature · 7 lines
2.18.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#1 lore raw

PATCH v3 1/3 kexec: add variant helper functions for handling memory regions

From: AKASHI Takahiro <[email protected]>

mem_regions_alloc_and_add() and mem_regions_alloc_and_exclude() are functionally equivalent to, respectively, mem_regions_add() and mem_regions_exclude() except the formers will re-allocate memory dynamically when no more entries are available in 'ranges' array.

---
 kexec/mem_regions.c | 42 ++++++++++++++++++++++++++++++++++++++++++
 kexec/mem_regions.h |  7 +++++++
 2 files changed, 49 insertions(+)
kexec/mem_regions.c +42 -0
index 50c8abc..ad7d3f1 100644
@@ -125,3 +125,45 @@ int mem_regions_exclude(struct memory_ranges *ranges,
 	}
 	return 0;
 }
+
+#define KEXEC_MEMORY_RANGES 16
+
+int mem_regions_alloc_and_add(struct memory_ranges *ranges,
+			      unsigned long long base,
+			      unsigned long long length, int type)
+{
+	void *new_ranges;
+
+	if (ranges->size >= ranges->max_size) {
+		new_ranges = realloc(ranges->ranges,
+				sizeof(struct memory_range) *
+				(ranges->max_size + KEXEC_MEMORY_RANGES));
+		if (!new_ranges)
+			return -1;
+
+		ranges->ranges = new_ranges;
+		ranges->max_size += KEXEC_MEMORY_RANGES;
+	}
+
+	return mem_regions_add(ranges, base, length, type);
+}
+
+int mem_regions_alloc_and_exclude(struct memory_ranges *ranges,
+				  const struct memory_range *range)
+{
+	void *new_ranges;
+
+	/* for safety, we should have at least one free entry in ranges */
+	if (ranges->size >= ranges->max_size) {
+		new_ranges = realloc(ranges->ranges,
+				sizeof(struct memory_range) *
+				(ranges->max_size + KEXEC_MEMORY_RANGES));
+		if (!new_ranges)
+			return -1;
+
+		ranges->ranges = new_ranges;
+		ranges->max_size += KEXEC_MEMORY_RANGES;
+	}
+
+	return mem_regions_exclude(ranges, range);
+}
kexec/mem_regions.h +7 -0
index ae9e972..e306d67 100644
@@ -12,4 +12,11 @@ int mem_regions_exclude(struct memory_ranges *ranges,
 int mem_regions_add(struct memory_ranges *ranges, unsigned long long base,
                     unsigned long long length, int type);
 
+int mem_regions_alloc_and_exclude(struct memory_ranges *ranges,
+				  const struct memory_range *range);
+
+int mem_regions_alloc_and_add(struct memory_ranges *ranges,
+			      unsigned long long base,
+			      unsigned long long length, int type);
+
 #endif
Signature · 7 lines
2.18.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#2 lore raw

PATCH v3 2/3 arm64: kexec: allocate memory space avoiding reserved regions

In reply to Masayoshi Mizuma

From: AKASHI Takahiro <[email protected]>

On UEFI/ACPI-only system, some memory regions, including but not limited to UEFI memory map and ACPI tables, must be preserved across kexec'ing. Otherwise, they can be corrupted and result in early failure in booting a new kernel.

In recent kernels, /proc/iomem now has an extended file format like:

40000000-5871ffff : System RAM
41800000-426affff : Kernel code
426b0000-42aaffff : reserved
42ab0000-42c64fff : Kernel data
54400000-583fffff : Crash kernel
58590000-585effff : reserved
58700000-5871ffff : reserved
58720000-58b5ffff : reserved
58b60000-5be3ffff : System RAM
58b61000-58b61fff : reserved

where the "reserved" entries at the top level or under System RAM (and its descendant resources) are ones of such kind and should not be regarded as usable memory ranges where several free spaces for loading kexec data will be allocated.

With this patch, get_memory_ranges() will handle this format of file correctly. Note that, for safety, unknown regions, in addition to "reserved" ones, will also be excluded.

---
 kexec/arch/arm64/kexec-arm64.c | 153 ++++++++++++++++++++-------------
 1 file changed, 94 insertions(+), 59 deletions(-)
kexec/arch/arm64/kexec-arm64.c +94 -59
index 6ad3b0a..45ebc54 100644
@@ -10,7 +10,9 @@
 #include <inttypes.h>
 #include <libfdt.h>
 #include <limits.h>
+#include <stdio.h>
 #include <stdlib.h>
+#include <string.h>
 #include <sys/stat.h>
 #include <linux/elf-em.h>
 #include <elf.h>
@@ -29,6 +31,7 @@
 #include "fs2dt.h"
 #include "iomem.h"
 #include "kexec-syscall.h"
+#include "mem_regions.h"
 #include "arch/options.h"
 
 #define ROOT_NODE_ADDR_CELLS_DEFAULT 1
@@ -905,19 +908,33 @@ int get_phys_base_from_pt_load(unsigned long *phys_offset)
 	return 0;
 }
 
+static bool to_be_excluded(char *str)
+{
+	if (!strncmp(str, SYSTEM_RAM, strlen(SYSTEM_RAM)) ||
+	    !strncmp(str, KERNEL_CODE, strlen(KERNEL_CODE)) ||
+	    !strncmp(str, KERNEL_DATA, strlen(KERNEL_DATA)) ||
+	    !strncmp(str, CRASH_KERNEL, strlen(CRASH_KERNEL)))
+		return false;
+	else
+		return true;
+}
+
 /**
- * get_memory_ranges_iomem_cb - Helper for get_memory_ranges_iomem.
+ * get_memory_ranges - Try to get the memory ranges from
+ * /proc/iomem.
  */
-
-static int get_memory_ranges_iomem_cb(void *data, int nr, char *str,
-	unsigned long long base, unsigned long long length)
+int get_memory_ranges(struct memory_range **range, int *ranges,
+	unsigned long kexec_flags)
 {
-	int ret;
 	unsigned long phys_offset = UINT64_MAX;
-	struct memory_range *r;
-
-	if (nr >= KEXEC_SEGMENT_MAX)
-		return -1;
+	FILE *fp;
+	const char *iomem = proc_iomem();
+	char line[MAX_LINE], *str;
+	unsigned long long start, end;
+	int n, consumed;
+	struct memory_ranges memranges;
+	struct memory_range *last, excl_range;
+	int ret;
 
 	if (!try_read_phys_offset_from_kcore) {
 		/* Since kernel version 4.19, 'kcore' contains
@@ -951,17 +968,72 @@ static int get_memory_ranges_iomem_cb(void *data, int nr, char *str,
 		try_read_phys_offset_from_kcore = true;
 	}
 
-	r = (struct memory_range *)data + nr;
+	fp = fopen(iomem, "r");
+	if (!fp)
+		die("Cannot open %s\n", iomem);
+
+	memranges.ranges = NULL;
+	memranges.size = memranges.max_size  = 0;
+
+	while (fgets(line, sizeof(line), fp) != 0) {
+		n = sscanf(line, "%llx-%llx : %n", &start, &end, &consumed);
+		if (n != 2)
+			continue;
+		str = line + consumed;
+
+		if (!strncmp(str, SYSTEM_RAM, strlen(SYSTEM_RAM))) {
+			ret = mem_regions_alloc_and_add(&memranges,
+					start, end - start + 1, RANGE_RAM);
+			if (ret) {
+				fprintf(stderr,
+					"Cannot allocate memory for ranges\n");
+				fclose(fp);
+				return -ENOMEM;
+			}
 
-	if (!strncmp(str, SYSTEM_RAM, strlen(SYSTEM_RAM)))
-		r->type = RANGE_RAM;
-	else if (!strncmp(str, IOMEM_RESERVED, strlen(IOMEM_RESERVED)))
-		r->type = RANGE_RESERVED;
-	else
-		return 1;
+			dbgprintf("%s:+[%d] %016llx - %016llx\n", __func__,
+				memranges.size - 1,
+				memranges.ranges[memranges.size - 1].start,
+				memranges.ranges[memranges.size - 1].end);
+		} else if (to_be_excluded(str)) {
+			if (!memranges.size)
+				continue;
+
+			/*
+			 * Note: mem_regions_exclude() doesn't guarantee
+			 * that the ranges are sorted out, but as long as
+			 * we cope with /proc/iomem, we only operate on
+			 * the last entry and so it is safe.
+			 */
 
-	r->start = base;
-	r->end = base + length - 1;
+			/* The last System RAM range */
+			last = &memranges.ranges[memranges.size - 1];
+
+			if (last->end < start)
+				/* New resource outside of System RAM */
+				continue;
+			if (end < last->start)
+				/* Already excluded by parent resource */
+				continue;
+
+			excl_range.start = start;
+			excl_range.end = end;
+			ret = mem_regions_alloc_and_exclude(&memranges, &excl_range);
+			if (ret) {
+				fprintf(stderr,
+					"Cannot allocate memory for ranges (exclude)\n");
+				fclose(fp);
+				return -ENOMEM;
+			}
+			dbgprintf("%s:-      %016llx - %016llx\n",
+					__func__, start, end);
+		}
+	}
+
+	fclose(fp);
+
+	*range = memranges.ranges;
+	*ranges = memranges.size;
 
 	/* As a fallback option, we can try determining the PHYS_OFFSET
 	 * value from the '/proc/iomem' entries as well.
@@ -982,52 +1054,15 @@ static int get_memory_ranges_iomem_cb(void *data, int nr, char *str,
 	 * between the user-space and kernel space 'PHYS_OFFSET'
 	 * value.
 	 */
-	set_phys_offset(r->start, "iomem");
-
-	dbgprintf("%s: %016llx - %016llx : %s", __func__, r->start,
-		r->end, str);
-
-	return 0;
-}
-
-/**
- * get_memory_ranges_iomem - Try to get the memory ranges from
- * /proc/iomem.
- */
+	if (memranges.size)
+		set_phys_offset(memranges.ranges[0].start, "iomem");
 
-static int get_memory_ranges_iomem(struct memory_range *array,
-	unsigned int *count)
-{
-	*count = kexec_iomem_for_each_line(NULL,
-		get_memory_ranges_iomem_cb, array);
-
-	if (!*count) {
-		dbgprintf("%s: failed: No RAM found.\n", __func__);
-		return EFAILED;
-	}
+	dbgprint_mem_range("System RAM ranges;",
+				memranges.ranges, memranges.size);
 
 	return 0;
 }
 
-/**
- * get_memory_ranges - Try to get the memory ranges some how.
- */
-
-int get_memory_ranges(struct memory_range **range, int *ranges,
-	unsigned long kexec_flags)
-{
-	static struct memory_range array[KEXEC_SEGMENT_MAX];
-	unsigned int count;
-	int result;
-
-	result = get_memory_ranges_iomem(array, &count);
-
-	*range = result ? NULL : array;
-	*ranges = result ? 0 : count;
-
-	return result;
-}
-
 int arch_compat_trampoline(struct kexec_info *info)
 {
 	return 0;
Signature · 7 lines
2.18.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#3 lore raw

PATCH v3 3/3 arm64: kdump: deal with a lot of resource entries in /proc/iomem

In reply to Masayoshi Mizuma

From: AKASHI Takahiro <[email protected]>

As described in the commit ("arm64: kexec: allocate memory space avoiding reserved regions"), /proc/iomem now has a lot of "reserved" entries, and it's not just enough to have a fixed size of memory range array.

With this patch, kdump is allowed to handle arbitrary number of memory ranges, using mem_regions_alloc_and_xxx() functions.

---
 kexec/arch/arm64/crashdump-arm64.c | 25 ++++++++++---------------
 1 file changed, 10 insertions(+), 15 deletions(-)
kexec/arch/arm64/crashdump-arm64.c +10 -15
index 4fd7aa8..38d1a0f 100644
@@ -23,13 +23,8 @@
 #include "kexec-elf.h"
 #include "mem_regions.h"
 
-/* memory ranges on crashed kernel */
-static struct memory_range system_memory_ranges[CRASH_MAX_MEMORY_RANGES];
-static struct memory_ranges system_memory_rgns = {
-	.size = 0,
-	.max_size = CRASH_MAX_MEMORY_RANGES,
-	.ranges = system_memory_ranges,
-};
+/* memory ranges of crashed kernel */
+static struct memory_ranges system_memory_rgns;
 
 /* memory range reserved for crashkernel */
 struct memory_range crash_reserved_mem;
@@ -82,7 +77,7 @@ static uint64_t get_kernel_page_offset(void)
  *
  * This function is called once for each memory region found in /proc/iomem.
  * It locates system RAM and crashkernel reserved memory and places these to
- * variables, respectively, system_memory_ranges and crash_reserved_mem.
+ * variables, respectively, system_memory_rgns and usablemem_rgns.
  */
 
 static int iomem_range_callback(void *UNUSED(data), int UNUSED(nr),
@@ -90,11 +85,11 @@ static int iomem_range_callback(void *UNUSED(data), int UNUSED(nr),
 				unsigned long long length)
 {
 	if (strncmp(str, CRASH_KERNEL, strlen(CRASH_KERNEL)) == 0)
-		return mem_regions_add(&usablemem_rgns,
-				       base, length, RANGE_RAM);
+		return mem_regions_alloc_and_add(&usablemem_rgns,
+						base, length, RANGE_RAM);
 	else if (strncmp(str, SYSTEM_RAM, strlen(SYSTEM_RAM)) == 0)
-		return mem_regions_add(&system_memory_rgns,
-				       base, length, RANGE_RAM);
+		return mem_regions_alloc_and_add(&system_memory_rgns,
+						base, length, RANGE_RAM);
 	else if (strncmp(str, KERNEL_CODE, strlen(KERNEL_CODE)) == 0)
 		elf_info.kern_paddr_start = base;
 	else if (strncmp(str, KERNEL_DATA, strlen(KERNEL_DATA)) == 0)
@@ -135,9 +130,9 @@ static int crash_get_memory_ranges(void)
 
 	dbgprint_mem_range("Reserved memory range", &crash_reserved_mem, 1);
 
-	if (mem_regions_exclude(&system_memory_rgns, &crash_reserved_mem)) {
-		fprintf(stderr,
-			"Error: Number of crash memory ranges excedeed the max limit\n");
+	if (mem_regions_alloc_and_exclude(&system_memory_rgns,
+						&crash_reserved_mem)) {
+		fprintf(stderr, "Cannot allocate memory for ranges\n");
 		return -ENOMEM;
 	}
 
Signature · 7 lines
2.18.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#4 lore raw
In reply to Masayoshi Mizuma

Thanks for the patches Masa,

Quoted 32 lines · Masayoshi Mizuma

In recent arm64 kernels, /proc/iomem has an extended file format like:

40000000-5871ffff : System RAM
41800000-426affff : Kernel code
426b0000-42aaffff : reserved
42ab0000-42c64fff : Kernel data
54400000-583fffff : Crash kernel
58590000-585effff : reserved
58700000-5871ffff : reserved
58720000-58b5ffff : reserved
58b60000-5be3ffff : System RAM
58b61000-58b61fff : reserved

where "reserved" entries can be an ACPI table, UEFI related code or data. They can be corrupted and result in early failure in booting a new kernel. As an actual example, LPI pending table and LPI property table, which are pointed by a UEFI data, are sometimes destroyed.

They are expected to be preserved across kexec'ing.

Changelog:
v3: - Re-based to the latest commit (bd07796).
- Added Tested-by tag from Bhupesh and Masayoshi
- Added an error handling in case mem_regions_alloc_and_exclude() fails (0002 patch).

AKASHI Takahiro (3):
kexec: add variant helper functions for handling memory regions
arm64: kexec: allocate memory space avoiding reserved regions
arm64: kdump: deal with a lot of resource entries in /proc/iomem

Quoted 5 lines
 kexec/arch/arm64/crashdump-arm64.c |  25 ++---
 kexec/arch/arm64/kexec-arm64.c     | 153 ++++++++++++++++++-----------
 kexec/mem_regions.c                |  42 ++++++++
 kexec/mem_regions.h                |   7 ++
 4 files changed, 153 insertions(+), 74 deletions(-)

The changes look fine to me (the patches already have my Tested-by).

Hi Simon,

Can you please help pick these changes for upstream kexec-tools and arm64 kexec is broken on few machines in the absence of these changes. The corresponding linux change (see [0]) has already been accepted in linux-next tree.

[0]. ab0eb16205b4 ("efi/memreserve: Register reservations as 'reserved' in /proc/iomem")

Thanks,
Bhupesh

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec