Skip to contentSkip to conversation

x86/kdump: Fix 'kmem -s' reported an invalid freepointer when SME was active

PATCH Superseded Lianbo Jiang 3 participants 16 messages Oct 12, 2019 – Oct 16, 2019

Subsystems (3)

v3 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v3 cover x86/kdump: Fix 'kmem -s' reported an invalid freepointer when SME was active

In purgatory(), the main things are as below:

[1] verify sha256 hashes for various segments. Lets keep these codes, and do not touch the logic.

[2] copy the first 640k content to a backup region.
Lets safely remove it and clean all code related to backup region.

This patch series will remove the backup region, because the current handling of copying the first 640k runs into problems when SME is active.

The low 1MiB region will always be reserved when the crashkernel kernel command line option is specified. And this way makes it unnecessary to do anything with the low 1MiB region, because the memory allocated later won't fall into the low 1MiB area.

This series includes three patches:
[1] Fix 'kmem -s' reported an invalid freepointer when SME was active The low 1MiB region will always be reserved when the crashkernel kernel command line option is specified, which ensures that the memory allocated later won't fall into the low 1MiB area.

[2] x86/kdump cleanup: remove the unused crash_copy_backup_region() The crash_copy_backup_region() has never been used, so clean up the redundant code.

[3] x86/kdump: clean up all the code related to the backup region Remove the backup region and clean up.

Changes since v1:
[1] Add extra checking condition: when the crashkernel option is specified, reserve the low 640k area.

Changes since v2:
[1] Reserve the low 1MiB region when the crashkernel option is only specified.(Suggested by Eric)

[2] Remove the unused crash_copy_backup_region()

[3] Remove the backup region and clean up

[4] Split them into three patches

Lianbo Jiang (3):
x86/kdump: Fix 'kmem -s' reported an invalid freepointer when SME was
active
x86/kdump cleanup: remove the unused crash_copy_backup_region() x86/kdump: clean up all the code related to the backup region

 arch/x86/include/asm/crash.h       |  1 -
 arch/x86/include/asm/kexec.h       | 10 ----
 arch/x86/include/asm/purgatory.h   | 10 ----
 arch/x86/kernel/crash.c            | 91 ++++++------------------------
 arch/x86/kernel/machine_kexec_64.c | 47 ---------------
 arch/x86/purgatory/purgatory.c     | 19 -------
 arch/x86/realmode/init.c           | 11 ++++
 7 files changed, 27 insertions(+), 162 deletions(-)
Signature · 1 line
2.17.1
#1 lore raw

PATCH v3 1/3 x86/kdump: Fix 'kmem -s' reported an invalid freepointer when SME was active

Bugzilla: https://bugzilla.kernel.org/show_bug.cgi?id=204793

Kdump kernel will reuse the first 640k region because of some reasons, for example: the trampline and conventional PC system BIOS region may require to allocate memory in this area. Obviously, kdump kernel will also overwrite the first 640k region, therefore, kernel has to copy the contents of the first 640k area to a backup area, which is done in purgatory(), because vmcore may need the old memory. When vmcore is dumped, kdump kernel will read the old memory from the backup area of the first 640k area.

Basically, the main reason should be clear, kernel does not correctly handle the first 640k region when SME is active, which causes that kernel does not properly copy these old memory to the backup area in purgatory(). Therefore, kdump kernel reads out the incorrect contents from the backup area when dumping vmcore. Finally, the phenomenon is as follow:

[root linux]$ crash vmlinux /var/crash/127.0.0.1-2019-09-19-08\:31\:27/vmcore
WARNING: kernel relocated [240MB]: patching 97110 gdb minimal_symbol values

      KERNEL: /var/crash/127.0.0.1-2019-09-19-08:31:27/vmlinux
    DUMPFILE: /var/crash/127.0.0.1-2019-09-19-08:31:27/vmcore  [PARTIAL DUMP]
        CPUS: 128
        DATE: Thu Sep 19 08:31:18 2019
      UPTIME: 00:01:21
LOAD AVERAGE: 0.16, 0.07, 0.02
       TASKS: 1343
    NODENAME: amd-ethanol
     RELEASE: 5.3.0-rc7+
     VERSION: #4 SMP Thu Sep 19 08:14:00 EDT 2019
     MACHINE: x86_64  (2195 Mhz)
      MEMORY: 127.9 GB
       PANIC: "Kernel panic - not syncing: sysrq triggered crash"
         PID: 9789
     COMMAND: "bash"
        TASK: "ffff89711894ae80  [THREAD_INFO: ffff89711894ae80]"
         CPU: 83
       STATE: TASK_RUNNING (PANIC)

crash> kmem -s|grep -i invalid
kmem: dma-kmalloc-512: slab:ffffd77680001c00 invalid freepointer:a6086ac099f0c5a4
kmem: dma-kmalloc-512: slab:ffffd77680001c00 invalid freepointer:a6086ac099f0c5a4 crash>

BTW: I also tried to fix the above problem in purgatory(), but there are too many restricts in purgatory() context, for example: i can't allocate new memory to create the identity mapping page table for SME situation.

Currently, there are two places where the first 640k area is needed, the first one is in the find_trampoline_placement(), another one is in the reserve_real_mode(), and their content doesn't matter. To avoid the above error, when the crashkernel kernel command line option is specified, lets reserve the remain low 1MiB memory(after reserving real mode memroy) so that the allocated memory does not fall into the low 1MiB area, which makes us not to copy the first 640k content to a backup region in purgatory().

In addition, also need to clean all the code related to the backup region later.

---
 arch/x86/realmode/init.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
arch/x86/realmode/init.c +11 -0
index 7dce39c8c034..bf4c8ffc5ed9 100644
@@ -34,6 +34,17 @@ void __init reserve_real_mode(void)
 
 	memblock_reserve(mem, size);
 	set_real_mode_mem(mem);
+
+#ifdef CONFIG_KEXEC_CORE
+	/*
+	 * When the crashkernel option is specified, only use the low
+	 * 1MiB for the real mode trampoline.
+	 */
+	if (strstr(boot_command_line, "crashkernel=")) {
+		memblock_reserve(0, SZ_1M);
+		pr_info("Reserving low 1MiB of memory for crashkernel\n");
+	}
+#endif /* CONFIG_KEXEC_CORE */
 }
 
 static void __init setup_real_mode(void)
Signature · 7 lines
2.17.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#2 lore raw

PATCH v3 2/3 x86/kdump cleanup: remove the unused crash_copy_backup_region()

In reply to Lianbo Jiang

The crash_copy_backup_region() has never been used, so clean up the redundant code.

---
 arch/x86/include/asm/crash.h | 1 -
 1 file changed, 1 deletion(-)
arch/x86/include/asm/crash.h +0 -1
index 0acf5ee45a21..089b2850f9d1 100644
@@ -3,7 +3,6 @@
 #define _ASM_X86_CRASH_H
 
 int crash_load_segments(struct kimage *image);
-int crash_copy_backup_region(struct kimage *image);
 int crash_setup_memmap_entries(struct kimage *image,
 		struct boot_params *params);
 void crash_smp_send_stop(void);
Signature · 7 lines
2.17.1


_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec
#3 lore raw

PATCH v3 3/3 x86/kdump: clean up all the code related to the backup region

In reply to Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---
 arch/x86/include/asm/kexec.h       | 10 ----
 arch/x86/include/asm/purgatory.h   | 10 ----
 arch/x86/kernel/crash.c            | 91 ++++++------------------------
 arch/x86/kernel/machine_kexec_64.c | 47 ---------------
 arch/x86/purgatory/purgatory.c     | 19 -------
 5 files changed, 16 insertions(+), 161 deletions(-)
arch/x86/include/asm/kexec.h +0 -10
index 5e7d6b46de97..6802c59e8252 100644
@@ -66,10 +66,6 @@ struct kimage;
 # define KEXEC_ARCH KEXEC_ARCH_X86_64
 #endif
 
-/* Memory to backup during crash kdump */
-#define KEXEC_BACKUP_SRC_START	(0UL)
-#define KEXEC_BACKUP_SRC_END	(640 * 1024UL - 1)	/* 640K */
-
 /*
  * This function is responsible for capturing register states if coming
  * via panic otherwise just fix up the ss and sp if coming via kernel
@@ -154,12 +150,6 @@ struct kimage_arch {
 	pud_t *pud;
 	pmd_t *pmd;
 	pte_t *pte;
-	/* Details of backup region */
-	unsigned long backup_src_start;
-	unsigned long backup_src_sz;
-
-	/* Physical address of backup segment */
-	unsigned long backup_load_addr;
 
 	/* Core ELF header buffer */
 	void *elf_headers;
arch/x86/include/asm/purgatory.h +0 -10
index 92c34e517da1..5528e9325049 100644
@@ -6,16 +6,6 @@
 #include <linux/purgatory.h>
 
 extern void purgatory(void);
-/*
- * These forward declarations serve two purposes:
- *
- * 1) Make sparse happy when checking arch/purgatory
- * 2) Document that these are required to be global so the symbol
- *    lookup in kexec works
- */
-extern unsigned long purgatory_backup_dest;
-extern unsigned long purgatory_backup_src;
-extern unsigned long purgatory_backup_sz;
 #endif	/* __ASSEMBLY__ */
 
 #endif /* _ASM_PURGATORY_H */
arch/x86/kernel/crash.c +16 -75

144 lines not on this page · raw message

arch/x86/kernel/machine_kexec_64.c +0 -47

59 lines not on this page · raw message

arch/x86/purgatory/purgatory.c +0 -19
index 3b95410ff0f8..2961234d0795 100644
@@ -14,28 +14,10 @@
 
 #include "../boot/string.h"
 
-unsigned long purgatory_backup_dest __section(.kexec-purgatory);
-unsigned long purgatory_backup_src __section(.kexec-purgatory);
-unsigned long purgatory_backup_sz __section(.kexec-purgatory);
-
 u8 purgatory_sha256_digest[SHA256_DIGEST_SIZE] __section(.kexec-purgatory);
 
 struct kexec_sha_region purgatory_sha_regions[KEXEC_SEGMENT_MAX] __section(.kexec-purgatory);
 
-/*
- * On x86, second kernel requries first 640K of memory to boot. Copy
- * first 640K to a backup region in reserved memory range so that second
- * kernel can use first 640K.
- */
-static int copy_backup_region(void)
-{
-	if (purgatory_backup_dest) {
-		memcpy((void *)purgatory_backup_dest,
-		       (void *)purgatory_backup_src, purgatory_backup_sz);
-	}
-	return 0;
-}
-
 static int verify_sha256_digest(void)
 {
 	struct kexec_sha_region *ptr, *end;
@@ -66,7 +48,6 @@ void purgatory(void)
 		for (;;)
 			;
 	}
-	copy_backup_region();
 }
 
 /*
Signature · 1 line
2.17.1
#4 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 31 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -4
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 10 lines
arch/x86/kernel/crash.c +9 -1
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Quoted 9 lines
arch/x86/kernel/crash.c +0 -1
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
-	/* Add first 640K segment */

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 10 lines
arch/x86/kernel/crash.c +6 -2
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Is this needed because of your other special case above?

When SME is active and the crashkernel command line is enabled do we just want to leave the low 1MB unencrypted? So we don't need any special cases?

Eric

#5 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 9 lines
Quoted 9 lines
arch/x86/kernel/crash.c +0 -0
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
Quoted 14 lines

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 11 lines
arch/x86/kernel/crash.c +6 -3
-	/* Add first 640K segment */
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

Thanks
Dave

#6 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 82 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 9 lines
Quoted 9 lines
arch/x86/kernel/crash.c +0 -0
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
Quoted 14 lines

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 11 lines
arch/x86/kernel/crash.c +6 -3
-	/* Add first 640K segment */
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 3 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

But if we have stopped special casing the low 1M. Why do we need a special case here at all?

If you need the special case it is almost certainly wrong to say you have ram above 640KiB and below 1MiB. That is the legacy ROM and video MMIO area.

There is a reason the original code said 640KiB.

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#7 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 84 lines · Eric W. Biederman
Quoted 82 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 23 lines
Quoted 10 lines
arch/x86/kernel/crash.c +0 -1
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
-	/* Add first 640K segment */

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 10 lines
arch/x86/kernel/crash.c +6 -2
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 7 lines
Quoted 3 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

But if we have stopped special casing the low 1M. Why do we need a special case here at all?

Here, need to pass the low memory range to kdump kernel, which will guarantee the availability of low memory in kdump kernel, otherwise, kdump kernel won't use the low memory region.

Quoted 6 lines

If you need the special case it is almost certainly wrong to say you have ram above 640KiB and below 1MiB. That is the legacy ROM and video MMIO area.

There is a reason the original code said 640KiB.

Do you mean that the 640k region is good enough here instead of 1MiB?

Thanks.
Lianbo

Quoted 2 lines

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#8 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 96 lines · lijiang
Quoted 84 lines · Eric W. Biederman
Quoted 82 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 23 lines
Quoted 10 lines
arch/x86/kernel/crash.c +0 -1
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
-	/* Add first 640K segment */

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 10 lines
arch/x86/kernel/crash.c +6 -2
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 7 lines
Quoted 3 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

But if we have stopped special casing the low 1M. Why do we need a special case here at all?

Here, need to pass the low memory range to kdump kernel, which will guarantee the availability of low memory in kdump kernel, otherwise, kdump kernel won't use the low memory region.

Quoted 7 lines
Quoted 6 lines

If you need the special case it is almost certainly wrong to say you have ram above 640KiB and below 1MiB. That is the legacy ROM and video MMIO area.

There is a reason the original code said 640KiB.

Do you mean that the 640k region is good enough here instead of 1MiB?

Reading through the code of crash_setup_memap_entries I see that what the code is doing now. The code is repeating the e820 memory map with the memory areas that were not reserved for the crash kernel removed.

In which case what the code needs to be doing something like:

	cmd.type = E820_TYPE_RAM;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, 1024*1024, &cmd,
			       memmap_entry_callback);

Depending on which bugs exist it might make sense to limit this to the low 640KiB. But finding something the kernel already recognizes as RAM should prevent most of those problems already. Barring bugs I admit it doesn't make sense to repeat the work that someone else has already done.

This bit:

	/* Add e820 reserved ranges */
	cmd.type = E820_TYPE_RESERVED;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, -1, &cmd,
			   memmap_entry_callback);

Should probably start at 1MiB instead of 0. Just so we don't report the
memory below 1MiB as unconditionally reserved. I don't properly
understand the IORES_DESC_RESERVED flag, and how that differs from flags. So please test my suggestions to verify the code works as expected.

Eric

author
#9 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 130 lines · Eric W. Biederman
Quoted 96 lines · lijiang
Quoted 84 lines · Eric W. Biederman
Quoted 82 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 23 lines
Quoted 10 lines
arch/x86/kernel/crash.c +0 -1
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
-	/* Add first 640K segment */

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 10 lines
arch/x86/kernel/crash.c +6 -2
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 7 lines
Quoted 3 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

But if we have stopped special casing the low 1M. Why do we need a special case here at all?

Here, need to pass the low memory range to kdump kernel, which will guarantee the availability of low memory in kdump kernel, otherwise, kdump kernel won't use the low memory region.

Quoted 7 lines
Quoted 6 lines

If you need the special case it is almost certainly wrong to say you have ram above 640KiB and below 1MiB. That is the legacy ROM and video MMIO area.

There is a reason the original code said 640KiB.

Do you mean that the 640k region is good enough here instead of 1MiB?

Reading through the code of crash_setup_memap_entries I see that what the code is doing now. The code is repeating the e820 memory map with the memory areas that were not reserved for the crash kernel removed.

In which case what the code needs to be doing something like:

	cmd.type = E820_TYPE_RAM;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, 1024*1024, &cmd,
			       memmap_entry_callback);

Depending on which bugs exist it might make sense to limit this to the low 640KiB. But finding something the kernel already recognizes as RAM should prevent most of those problems already. Barring bugs I admit it doesn't make sense to repeat the work that someone else has already done.

This bit:

	/* Add e820 reserved ranges */
	cmd.type = E820_TYPE_RESERVED;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, -1, &cmd,
			   memmap_entry_callback);
Quoted 6 lines

Should probably start at 1MiB instead of 0. Just so we don't report the
memory below 1MiB as unconditionally reserved. I don't properly
understand the IORES_DESC_RESERVED flag, and how that differs from flags. So please test my suggestions to verify the code works as expected.

Thanks for your comment, Eric.

I will make a test based on your suggestions. But i need an SME machine, maybe i will reply later.

Thanks.
Lianbo

Quoted 2 lines

Eric

author
#10 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

In reply to Eric W. Biederman
Quoted 110 lines · Eric W. Biederman
Quoted 96 lines · lijiang
Quoted 84 lines · Eric W. Biederman
Quoted 82 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 23 lines
Quoted 10 lines
arch/x86/kernel/crash.c +0 -1
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
-	/* Add first 640K segment */

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 10 lines
arch/x86/kernel/crash.c +6 -2
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 7 lines
Quoted 3 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

But if we have stopped special casing the low 1M. Why do we need a special case here at all?

Here, need to pass the low memory range to kdump kernel, which will guarantee the availability of low memory in kdump kernel, otherwise, kdump kernel won't use the low memory region.

Quoted 7 lines
Quoted 6 lines

If you need the special case it is almost certainly wrong to say you have ram above 640KiB and below 1MiB. That is the legacy ROM and video MMIO area.

There is a reason the original code said 640KiB.

Do you mean that the 640k region is good enough here instead of 1MiB?

Reading through the code of crash_setup_memap_entries I see that what the code is doing now. The code is repeating the e820 memory map with the memory areas that were not reserved for the crash kernel removed.

Quoted 7 lines

In which case what the code needs to be doing something like:

	cmd.type = E820_TYPE_RAM;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, 1024*1024, &cmd,
			       memmap_entry_callback);

The above code does not get the results what we expected, it gets the reserved memory marked as 'IORES_DESC_RESERVED' in the low 1MiB range.

Finally, kdump kernel happened the panic as follow:
......

[    3.555662] Kernel panic - not syncing: Real mode trampoline was not allocated
[    3.556660] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.4.0-rc3+ #4
[    3.556660] Hardware name: AMD Corporation Speedway/Speedway, BIOS RSW1009C 07/27/2018
[    3.556660] Call Trace:
[    3.556660]  dump_stack+0x46/0x60
[    3.556660]  panic+0xfb/0x2d7
[    3.556660]  ? hv_init_spinlocks+0x7f/0x7f
[    3.556660]  init_real_mode+0x27/0x1fa
[    3.556660]  ? hv_init_spinlocks+0x7f/0x7f
[    3.556660]  ? do_one_initcall+0x46/0x1e4
[    3.556660]  ? proc_register+0xd0/0x130
[    3.556660]  ? kernel_init_freeable+0xe2/0x242
[    3.556660]  ? rest_init+0xaa/0xaa
[    3.556660]  ? kernel_init+0xa/0x106
[    3.556660]  ? ret_from_fork+0x22/0x40
[    3.556660] Rebooting in 10 seconds..
[    3.556660] ACPI MEMORY or I/O RESET_REG.

I modified the above code, and tested it. This can find out the system ram in the low 1MiB range. And it worked well.

@@ -356,11 +338,11 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)

        memset(&cmd, 0, sizeof(struct crash_memmap_data));
        cmd.params = params;
+       /* Add the low 1MiB */
+       cmd.type = E820_TYPE_RAM;
+       flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
+       walk_iomem_res_desc(IORES_DESC_NONE, flags, 0, 1024*1024 - 1, &cmd,

+ memmap_entry_callback);

Quoted 6 lines

Depending on which bugs exist it might make sense to limit this to the low 640KiB. But finding something the kernel already recognizes as RAM should prevent most of those problems already. Barring bugs I admit it doesn't make sense to repeat the work that someone else has already done.

Quoted 8 lines

This bit:

	/* Add e820 reserved ranges */
	cmd.type = E820_TYPE_RESERVED;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, -1, &cmd,
			   memmap_entry_callback);

Should probably start at 1MiB instead of 0. Just so we don't report the

If so, it can not find out the reserved memory marked as 'IORES_DESC_RESERVED' in the low 1MiB range, finally, it doesn't pass the reserved memory in the low 1MiB to kdump kernel, which could cause some problems, such as SME or PCI MMCONFIG issue.

Quoted 2 lines

memory below 1MiB as unconditionally reserved. I don't properly
understand the IORES_DESC_RESERVED flag, and how that differs from

I found three commits about 'IORES_DESC_RESERVED' flag, hope this helps. 1.ae9e13d621d6 ("x86/e820, ioport: Add a new I/O resource descriptor IORES_DESC_RESERVED") 2.5da04cc86d12 ("x86/mm: Rework ioremap resource mapping determination") 3.980621daf368 ("x86/crash: Add e820 reserved ranges to kdump kernel's e820 table")

Quoted 3 lines

flags. So please test my suggestions to verify the code works as expected.

I have tested the two changes that you mentioned, please refer to the reply above.

Thanks.
Lianbo

Quoted 2 lines

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#11 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 145 lines · lijiang
Quoted 110 lines · Eric W. Biederman
Quoted 96 lines · lijiang
Quoted 84 lines · Eric W. Biederman
Quoted 82 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 23 lines
Quoted 10 lines
arch/x86/kernel/crash.c +0 -1
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
-	/* Add first 640K segment */

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 10 lines
arch/x86/kernel/crash.c +6 -2
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 7 lines
Quoted 3 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

But if we have stopped special casing the low 1M. Why do we need a special case here at all?

Here, need to pass the low memory range to kdump kernel, which will guarantee the availability of low memory in kdump kernel, otherwise, kdump kernel won't use the low memory region.

Quoted 7 lines
Quoted 6 lines

If you need the special case it is almost certainly wrong to say you have ram above 640KiB and below 1MiB. That is the legacy ROM and video MMIO area.

There is a reason the original code said 640KiB.

Do you mean that the 640k region is good enough here instead of 1MiB?

Reading through the code of crash_setup_memap_entries I see that what the code is doing now. The code is repeating the e820 memory map with the memory areas that were not reserved for the crash kernel removed.

Quoted 7 lines

In which case what the code needs to be doing something like:

	cmd.type = E820_TYPE_RAM;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, 1024*1024, &cmd,
			       memmap_entry_callback);

The above code does not get the results what we expected, it gets the reserved memory marked as 'IORES_DESC_RESERVED' in the low 1MiB range.

Finally, kdump kernel happened the panic as follow:
......

[    3.555662] Kernel panic - not syncing: Real mode trampoline was not allocated
[    3.556660] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 5.4.0-rc3+ #4
[    3.556660] Hardware name: AMD Corporation Speedway/Speedway, BIOS RSW1009C 07/27/2018
[    3.556660] Call Trace:
[    3.556660]  dump_stack+0x46/0x60
[    3.556660]  panic+0xfb/0x2d7
[    3.556660]  ? hv_init_spinlocks+0x7f/0x7f
[    3.556660]  init_real_mode+0x27/0x1fa
[    3.556660]  ? hv_init_spinlocks+0x7f/0x7f
[    3.556660]  ? do_one_initcall+0x46/0x1e4
[    3.556660]  ? proc_register+0xd0/0x130
[    3.556660]  ? kernel_init_freeable+0xe2/0x242
[    3.556660]  ? rest_init+0xaa/0xaa
[    3.556660]  ? kernel_init+0xa/0x106
[    3.556660]  ? ret_from_fork+0x22/0x40
[    3.556660] Rebooting in 10 seconds..
[    3.556660] ACPI MEMORY or I/O RESET_REG.

I modified the above code, and tested it. This can find out the system ram in the low 1MiB range. And it worked well.

Review int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 346

Quoted 9 lines
(quoted diff) +5 -0
@@ -356,11 +338,11 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
        memset(&cmd, 0, sizeof(struct crash_memmap_data));
        cmd.params = params;
 
+       /* Add the low 1MiB */
+       cmd.type = E820_TYPE_RAM;
+       flags = IORESOURCE_SYSTEM_RAM | IORESOURCE_BUSY;
+       walk_iomem_res_desc(IORES_DESC_NONE, flags, 0, 1024*1024 - 1, &cmd,
+                       memmap_entry_callback);
 

That looks like a very reasonable fix.

Quoted 13 lines
Quoted 13 lines

Depending on which bugs exist it might make sense to limit this to the low 640KiB. But finding something the kernel already recognizes as RAM should prevent most of those problems already. Barring bugs I admit it doesn't make sense to repeat the work that someone else has already done.

This bit:

	/* Add e820 reserved ranges */
	cmd.type = E820_TYPE_RESERVED;
	flags = IORESOURCE_MEM;
	walk_iomem_res_desc(IORES_DESC_RESERVED, flags, 0, -1, &cmd,
			   memmap_entry_callback);
Quoted 4 lines
Quoted 1 line

Should probably start at 1MiB instead of 0. Just so we don't report the

If so, it can not find out the reserved memory marked as 'IORES_DESC_RESERVED' in the low 1MiB range, finally, it doesn't pass the reserved memory in the low 1MiB to kdump kernel, which could cause some problems, such as SME or PCI MMCONFIG issue.

Good point. For some reason I was thinking IORESOURCE_MEM and IORESOURCE_SYSTEM_RAM were the same thing. It has been way to long since I have been in that part of the code.

So yes let's leave that part alone.

Quoted 7 lines
Quoted 2 lines

memory below 1MiB as unconditionally reserved. I don't properly
understand the IORES_DESC_RESERVED flag, and how that differs from

I found three commits about 'IORES_DESC_RESERVED' flag, hope this helps. 1.ae9e13d621d6 ("x86/e820, ioport: Add a new I/O resource descriptor IORES_DESC_RESERVED") 2.5da04cc86d12 ("x86/mm: Rework ioremap resource mapping determination") 3.980621daf368 ("x86/crash: Add e820 reserved ranges to kdump kernel's e820 table")

Quoted 5 lines
Quoted 3 lines

flags. So please test my suggestions to verify the code works as expected.

I have tested the two changes that you mentioned, please refer to the reply above.

Thank you. It looks like you have figured out how these things should work.

Eric

#12 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

In reply to Eric W. Biederman
Quoted 84 lines · Eric W. Biederman
Quoted 82 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Quoted 23 lines
Quoted 10 lines
arch/x86/kernel/crash.c +0 -1
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
-	/* Add first 640K segment */

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 10 lines
arch/x86/kernel/crash.c +6 -2
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 6 lines
Quoted 3 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

But if we have stopped special casing the low 1M. Why do we need a special case here at all?

Seems both Lianbo and I do not understand the query. Let me try to explain it more.

The 2nd kernel still need use low memory for the trampoline use. So we have to let 2nd kernel access the low memory as system ram.

The original special case is far more than above we are doing, it does several things:
1. backup the low 640K into kdump reserved high memory
2. set the low 640K as System RAM in kdump kernel as we do in this patch.
3. in /proc/vmcore elf header, map the low 640K to the backup region so that /proc/vmcore can give right old memory for that region.

After the change we are doing in this series, we dropped the 1 and 3 but 2 is still needed because kdump kernel still need use low memory. But we do not care the vmcore part because nobody use the memory in old kernel, we already reserve it, and excluded the range in vmcore.

I think another thing you mentioned about some reserved memory under 1M, even if we set 0-1M as System RAM, we still keep all the reserved regions in /proc/iomem as identical between 1st and 2nd kernels, so it just works, see below about cat /proc/iomem in kdump kernel (dropped into a shell before copying the vmcore out):

kdump:/# cat /proc/iomem|less
00000000-00000fff : Reserved
00001000-0009ffff : System RAM
000a0000-000bffff : PCI Bus 0000:00
000f0000-000fffff : System ROM
30000000-3000006f : System RAM
30000070-39f5cfff : System RAM
  38600000-39001070 : Kernel code
  39001071-396ce5ff : Kernel data
  39acd000-39bfffff : Kernel bss

You can see 000a0000-000bffff : PCI Bus 0000:00 is same across the kdump reboot.

But maybe if it is not elegant enough with simply using 0 - 1M, maybe use 0 - 640K as Lianbo said in another reply?

-------------

BTW, we also discussed about compatibility issues, for kexec_file it just works because our change is in kernel. For kexec-tools part, we can just leave the userspace code as is, that means if one wants the SME case be fixed he needs an kernel update to reserve the low memory.

We can not drop the kexec-tools special case about 640K because if we drop it and people use old kernels which does not reserve low 1M then kdump can not work at all.

Thanks
Dave

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#13 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

In reply to Dave Young
Quoted 50 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Quoted 8 lines

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Thank you, Eric and Dave. I will add the code comment as below if it would be OK.

@@ -234,9 +232,20 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)

 {
        struct crash_mem *cmem = arg;
-       cmem->ranges[cmem->nr_ranges].start = res->start;
-       cmem->ranges[cmem->nr_ranges].end = res->end;
-       cmem->nr_ranges++;
+       /*
+        * Currently, pass the low 1MiB range to kdump kernel in e820
+        * as system ram so that kdump kernel can also use the low 1MiB
+        * memory due to the real mode trampoline code.
+        * And later, the low 1MiB range will be exclued from elf header,
+        * which will avoid remapping the 1MiB system ram when dumping
+        * vmcore.
+        */
+       if (res->start >= SZ_1M) {
+               cmem->ranges[cmem->nr_ranges].start = res->start;
+               cmem->ranges[cmem->nr_ranges].end = res->end;
+               cmem->nr_ranges++;
+       } else if (res->end > SZ_1M) {
+               cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+               cmem->ranges[cmem->nr_ranges].end = res->end;
+               cmem->nr_ranges++;

+ }

        return 0;

}

Quoted 24 lines
Quoted 9 lines
Quoted 9 lines
arch/x86/kernel/crash.c +0 -0
 	return 0;
 }
 
@@ -356,9 +337,12 @@ int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params)
 	memset(&cmd, 0, sizeof(struct crash_memmap_data));
 	cmd.params = params;
 
Quoted 14 lines

arch/x86/kernel/crash.c int crash_setup_memmap_entries(struct kimage *image, struct boot_params *params) · line 347

Quoted 11 lines
arch/x86/kernel/crash.c +6 -3
-	/* Add first 640K segment */
-	ei.addr = image->arch.backup_src_start;
-	ei.size = image->arch.backup_src_sz;
+	/*
+	 * Add the low memory range[0x1000, SZ_1M], skip
+	 * the first zero page.
+	 */
+	ei.addr = PAGE_SIZE;
+	ei.size = SZ_1M - PAGE_SIZE;
 	ei.type = E820_TYPE_RAM;
 	add_e820_entry(params, &ei);

Likewise here. Why do we need a special case?
Why the magic with PAGE_SIZE?

Quoted 6 lines

Good catch, the zero page part is useless, I think no other special reason, just assumed zero page is not usable, but it should be ok to remove the special handling, just pass 0 - 1M is good enough.

Quoted 1 line

Thanks

Dave

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

#14 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 78 lines · lijiang
Quoted 50 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Quoted 8 lines

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Thank you, Eric and Dave. I will add the code comment as below if it would be OK.

arch/x86/kernel/crash.c +10 -3
@@ -234,9 +232,20 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
        struct crash_mem *cmem = arg;
 
-       cmem->ranges[cmem->nr_ranges].start = res->start;
-       cmem->ranges[cmem->nr_ranges].end = res->end;
-       cmem->nr_ranges++;
+       /*
+        * Currently, pass the low 1MiB range to kdump kernel in e820
+        * as system ram so that kdump kernel can also use the low 1MiB
+        * memory due to the real mode trampoline code.
+        * And later, the low 1MiB range will be exclued from elf header,
+        * which will avoid remapping the 1MiB system ram when dumping
+        * vmcore.
+        */
+       if (res->start >= SZ_1M) {
+               cmem->ranges[cmem->nr_ranges].start = res->start;

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 246

Quoted 10 lines
arch/x86/kernel/crash.c +7 -0
+               cmem->ranges[cmem->nr_ranges].end = res->end;
+               cmem->nr_ranges++;
+       } else if (res->end > SZ_1M) {
+               cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+               cmem->ranges[cmem->nr_ranges].end = res->end;
+               cmem->nr_ranges++;
+       }
 
        return 0;
 }

I just read through the appropriate section of crash.c and the way things are structured doing this work in prepare_elf64_ram_headers_callback is wrong.

This can be done in a simpler manner in elf_header_exclude_ranges. Something like:

	/* The low 1MiB is always reserved */
	ret = crash_exclude_mem_range(cmem, 0, 1024*1024);
	if (ret)
		return ret;

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec

author
#15 lore raw

Re: [PATCH 3/3 v3] x86/kdump: clean up all the code related to the backup region

Quoted 95 lines · Eric W. Biederman
Quoted 78 lines · lijiang
Quoted 50 lines · Dave Young

Hi Eric,

Quoted 31 lines · Eric W. Biederman
Quoted 29 lines · Lianbo Jiang

When the crashkernel kernel command line option is specified, the low 1MiB memory will always be reserved, which makes that the memory allocated later won't fall into the low 1MiB area, thereby, it's not necessary to create a backup region and also no need to copy the first 640k content to a backup region.

Currently, the code related to the backup region can be safely removed, so lets clean up.

---

arch/x86/kernel/crash.c +0 -2
index eb651fbde92a..cc5774fc84c0 100644
@@ -173,8 +173,6 @@ void native_machine_crash_shutdown(struct pt_regs *regs)
 
 #ifdef CONFIG_KEXEC_FILE
 
-static unsigned long crash_zero_bytes;
-
 static int get_nr_ram_ranges_callback(struct resource *res, void *arg)
 {
 	unsigned int *nr_ranges = arg;
@@ -234,9 +232,15 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
 	struct crash_mem *cmem = arg;
 
Quoted 15 lines

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 243

Quoted 12 lines
arch/x86/kernel/crash.c +9 -3
-	cmem->ranges[cmem->nr_ranges].start = res->start;
-	cmem->ranges[cmem->nr_ranges].end = res->end;
-	cmem->nr_ranges++;
+	if (res->start >= SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = res->start;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	} else if (res->end > SZ_1M) {
+		cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+		cmem->ranges[cmem->nr_ranges].end = res->end;
+		cmem->nr_ranges++;
+	}

What is going on with this chunk? I can guess but this needs a clear comment.

Quoted 8 lines

Indeed it needs some code comment, this is based on some offline discussion. cat /proc/vmcore will give a warning because ioremap is mapping the system ram.

We pass the first 1M to kdump kernel in e820 as system ram so that 2nd kernel can use the low 1M memory because for example the trampoline code.

Thank you, Eric and Dave. I will add the code comment as below if it would be OK.

arch/x86/kernel/crash.c +10 -3
@@ -234,9 +232,20 @@ static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg)
 {
        struct crash_mem *cmem = arg;
 
-       cmem->ranges[cmem->nr_ranges].start = res->start;
-       cmem->ranges[cmem->nr_ranges].end = res->end;
-       cmem->nr_ranges++;
+       /*
+        * Currently, pass the low 1MiB range to kdump kernel in e820
+        * as system ram so that kdump kernel can also use the low 1MiB
+        * memory due to the real mode trampoline code.
+        * And later, the low 1MiB range will be exclued from elf header,
+        * which will avoid remapping the 1MiB system ram when dumping
+        * vmcore.
+        */
+       if (res->start >= SZ_1M) {
+               cmem->ranges[cmem->nr_ranges].start = res->start;

arch/x86/kernel/crash.c static int prepare_elf64_ram_headers_callback(struct resource *res, void *arg) · line 246

Quoted 10 lines
arch/x86/kernel/crash.c +7 -0
+               cmem->ranges[cmem->nr_ranges].end = res->end;
+               cmem->nr_ranges++;
+       } else if (res->end > SZ_1M) {
+               cmem->ranges[cmem->nr_ranges].start = SZ_1M;
+               cmem->ranges[cmem->nr_ranges].end = res->end;
+               cmem->nr_ranges++;
+       }
 
        return 0;
 }

I just read through the appropriate section of crash.c and the way things are structured doing this work in prepare_elf64_ram_headers_callback is wrong.

Quoted 3 lines

This can be done in a simpler manner in elf_header_exclude_ranges. Something like:

Thank you, Eric. It seems that here is a more reasonable place, i will make a test about it and improve it in next post.

Lianbo

Quoted 7 lines
	/* The low 1MiB is always reserved */
	ret = crash_exclude_mem_range(cmem, 0, 1024*1024);
	if (ret)
		return ret;

Eric

_______________________________________________
kexec mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/kexec