Skip to contentSkip to conversation

x86/setup: consolidate early memory reservations

PATCH Superseded Mike Rapoport 1 participant 4 messages Jan 28, 2021 – Feb 5, 2021

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover x86/setup: consolidate early memory reservations

From: Mike Rapoport <[email protected]>

Hi,

David noticed that we do some of memblock_reserve() calls after allocations are possible:

https://lore.kernel.org/lkml/[email protected]

For now there is no actual problem because in top-down mode we allocate from the end of the memory and in bottom-up mode we allocate above the kernel image. But there is a patch in the mm tree that allow bottom-up allocations below the kernel:

https://lore.kernel.org/lkml/[email protected]

and with this change we may get a memory corruption if an allocation steps on some of the firmware areas that are yet to be reserved.

The below patches consolidate early memory reservations done during setup_arch() so that memory used by firmware, bootloader, kernel text/data and the memory that should be excluded from the available memory for whatever other reason is reserved before memblock allocations are possible.

The patches are vs v5.11-rc3-mmots-2021-01-12-02-00 as I think they are prerequisite for the memblock bottom-up changes, but if needed I can rebase then on another tree.

v2:
* get rid of trim_platform_memory_ranges() and call trim_snb_memory() directly, per Boris comments
* massage changelog and comments to use passive voice, per Boris
* add Acked-by and Reviewed-by, thanks Boris and David

v1: https://lore.kernel.org/lkml/[email protected]

Mike Rapoport (2):
x86/setup: consolidate early memory reservations x86/setup: merge several reservations of start of the memory

 arch/x86/kernel/setup.c | 96 ++++++++++++++++++-----------------------
 1 file changed, 42 insertions(+), 54 deletions(-)
Signature · 1 line
2.28.0
#1 lore raw

PATCH v2 1/2 x86/setup: consolidate early memory reservations

From: Mike Rapoport <[email protected]>

The early reservations of memory areas used by the firmware, bootloader, kernel text and data are spread over setup_arch(). Moreover, some of them happen *after* memblock allocations, e.g trim_platform_memory_ranges() and trim_low_memory_range() are called after reserve_real_mode() that allocates memory.

There was no corruption of these memory regions because memblock always allocates memory either from the end of memory (in top-down mode) or above the kernel image (in bottom-up mode). However, the bottom up mode is going to be updated to span the entire memory [1] to avoid limitations caused by KASLR.

Consolidate early memory reservations in a dedicated function to improve robustness against future changes. Having the early reservations in one place also makes it clearer what memory must be reserved before we allow memblock allocations.

[1] https://lore.kernel.org/lkml/[email protected]

---
 arch/x86/kernel/setup.c | 92 ++++++++++++++++++++---------------------
 1 file changed, 44 insertions(+), 48 deletions(-)
arch/x86/kernel/setup.c +44 -48
index 3412c4595efd..3b582406363a 100644
@@ -646,18 +646,6 @@ static void __init trim_snb_memory(void)
 	}
 }
 
-/*
- * Here we put platform-specific memory range workarounds, i.e.
- * memory known to be corrupt or otherwise in need to be reserved on
- * specific platforms.
- *
- * If this gets used more widely it could use a real dispatch mechanism.
- */
-static void __init trim_platform_memory_ranges(void)
-{
-	trim_snb_memory();
-}
-
 static void __init trim_bios_range(void)
 {
 	/*
@@ -728,7 +716,38 @@ static void __init trim_low_memory_range(void)
 	 */
 	memblock_reserve(0, ALIGN(reserve_low, PAGE_SIZE));
 }
-	
+
+static void __init early_reserve_memory(void)
+{
+	/*
+	 * Reserve the memory occupied by the kernel between _text and
+	 * __end_of_kernel_reserve symbols. Any kernel sections after the
+	 * __end_of_kernel_reserve symbol must be explicitly reserved with a
+	 * separate memblock_reserve() or they will be discarded.
+	 */
+	memblock_reserve(__pa_symbol(_text),
+			 (unsigned long)__end_of_kernel_reserve - (unsigned long)_text);
+
+	/*
+	 * Make sure page 0 is always reserved because on systems with
+	 * L1TF its contents can be leaked to user processes.
+	 */
+	memblock_reserve(0, PAGE_SIZE);
+
+	early_reserve_initrd();
+
+	if (efi_enabled(EFI_BOOT))
+		efi_memblock_x86_reserve_range();
+
+	memblock_x86_reserve_range_setup_data();
+
+	reserve_ibft_region();
+	reserve_bios_regions();
+
+	trim_snb_memory();
+	trim_low_memory_range();
+}
+
 /*
  * Dump out kernel offset information on panic.
  */
@@ -763,29 +782,6 @@ dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
 
 void __init setup_arch(char **cmdline_p)
 {
-	/*
-	 * Reserve the memory occupied by the kernel between _text and
-	 * __end_of_kernel_reserve symbols. Any kernel sections after the
-	 * __end_of_kernel_reserve symbol must be explicitly reserved with a
-	 * separate memblock_reserve() or they will be discarded.
-	 */
-	memblock_reserve(__pa_symbol(_text),
-			 (unsigned long)__end_of_kernel_reserve - (unsigned long)_text);
-
-	/*
-	 * Make sure page 0 is always reserved because on systems with
-	 * L1TF its contents can be leaked to user processes.
-	 */
-	memblock_reserve(0, PAGE_SIZE);
-
-	early_reserve_initrd();
-
-	/*
-	 * At this point everything still needed from the boot loader
-	 * or BIOS or kernel text should be early reserved or marked not
-	 * RAM in e820. All other memory is free game.
-	 */
-
 #ifdef CONFIG_X86_32
 	memcpy(&boot_cpu_data, &new_cpu_data, sizeof(new_cpu_data));
 
@@ -909,8 +905,18 @@ void __init setup_arch(char **cmdline_p)
 
 	parse_early_param();
 
-	if (efi_enabled(EFI_BOOT))
-		efi_memblock_x86_reserve_range();
+	/*
+	 * Do some memory reservations *before* memory is added to
+	 * memblock, so memblock allocations won't overwrite it.
+	 * Do it after early param, so we could get (unlikely) panic from
+	 * serial.
+	 *
+	 * After this point everything still needed from the boot loader or
+	 * firmware or kernel text should be early reserved or marked not
+	 * RAM in e820. All other memory is free game.
+	 */
+	early_reserve_memory();
+
 #ifdef CONFIG_MEMORY_HOTPLUG
 	/*
 	 * Memory used by the kernel cannot be hot-removed because Linux
@@ -937,9 +943,6 @@ void __init setup_arch(char **cmdline_p)
 
 	x86_report_nx();
 
-	/* after early param, so could get panic from serial */
-	memblock_x86_reserve_range_setup_data();
-
 	if (acpi_mps_check()) {
 #ifdef CONFIG_X86_LOCAL_APIC
 		disable_apic = 1;
@@ -1031,8 +1034,6 @@ void __init setup_arch(char **cmdline_p)
 	 */
 	find_smp_config();
 
-	reserve_ibft_region();
-
 	early_alloc_pgt_buf();
 
 	/*
@@ -1053,8 +1054,6 @@ void __init setup_arch(char **cmdline_p)
 	 */
 	sev_setup_arch();
 
-	reserve_bios_regions();
-
 	efi_fake_memmap();
 	efi_find_mirror();
 	efi_esrt_init();
@@ -1080,9 +1079,6 @@ void __init setup_arch(char **cmdline_p)
 
 	reserve_real_mode();
 
-	trim_platform_memory_ranges();
-	trim_low_memory_range();
-
 	init_mem_mapping();
 
 	idt_setup_early_pf();
Signature · 1 line
2.28.0
#2 lore raw

PATCH v2 2/2 x86/setup: merge several reservations of start of the memory

In reply to Mike Rapoport

From: Mike Rapoport <[email protected]>

Currently the first several pages are reserved both to avoid leaking their contents on systems with L1TF and to avoid corrupting BIOS memory.

Merge the two memory reservations.

---
 arch/x86/kernel/setup.c | 28 ++++++++++------------------
 1 file changed, 10 insertions(+), 18 deletions(-)
arch/x86/kernel/setup.c +10 -18
index 3b582406363a..b36624e3dc9e 100644
@@ -703,20 +703,6 @@ static int __init parse_reservelow(char *p)
 
 early_param("reservelow", parse_reservelow);
 
-static void __init trim_low_memory_range(void)
-{
-	/*
-	 * A special case is the first 4Kb of memory;
-	 * This is a BIOS owned area, not kernel ram, but generally
-	 * not listed as such in the E820 table.
-	 *
-	 * This typically reserves additional memory (64KiB by default)
-	 * since some BIOSes are known to corrupt low memory.  See the
-	 * Kconfig help text for X86_RESERVE_LOW.
-	 */
-	memblock_reserve(0, ALIGN(reserve_low, PAGE_SIZE));
-}
-
 static void __init early_reserve_memory(void)
 {
 	/*
@@ -729,10 +715,17 @@ static void __init early_reserve_memory(void)
 			 (unsigned long)__end_of_kernel_reserve - (unsigned long)_text);
 
 	/*
-	 * Make sure page 0 is always reserved because on systems with
-	 * L1TF its contents can be leaked to user processes.
+	 * The first 4Kb of memory is a BIOS owned area, but generally it is
+	 * not listed as such in the E820 table.
+	 *
+	 * Reserve the first memory page and typically some additional
+	 * memory (64KiB by default) since some BIOSes are known to corrupt
+	 * low memory. See the Kconfig help text for X86_RESERVE_LOW.
+	 *
+	 * In addition, make sure page 0 is always reserved because on
+	 * systems with L1TF its contents can be leaked to user processes.
 	 */
-	memblock_reserve(0, PAGE_SIZE);
+	memblock_reserve(0, ALIGN(reserve_low, PAGE_SIZE));
 
 	early_reserve_initrd();
 
@@ -745,7 +738,6 @@ static void __init early_reserve_memory(void)
 	reserve_bios_regions();
 
 	trim_snb_memory();
-	trim_low_memory_range();
 }
 
 /*
Signature · 1 line
2.28.0
#3 lore raw
In reply to Mike Rapoport

Hi,

Any comments on this?

Quoted 46 lines · Mike Rapoport

From: Mike Rapoport <[email protected]>

Hi,

David noticed that we do some of memblock_reserve() calls after allocations are possible:

https://lore.kernel.org/lkml/[email protected]

For now there is no actual problem because in top-down mode we allocate from the end of the memory and in bottom-up mode we allocate above the kernel image. But there is a patch in the mm tree that allow bottom-up allocations below the kernel:

https://lore.kernel.org/lkml/[email protected]

and with this change we may get a memory corruption if an allocation steps on some of the firmware areas that are yet to be reserved.

The below patches consolidate early memory reservations done during setup_arch() so that memory used by firmware, bootloader, kernel text/data and the memory that should be excluded from the available memory for whatever other reason is reserved before memblock allocations are possible.

The patches are vs v5.11-rc3-mmots-2021-01-12-02-00 as I think they are prerequisite for the memblock bottom-up changes, but if needed I can rebase then on another tree.

v2:
* get rid of trim_platform_memory_ranges() and call trim_snb_memory() directly, per Boris comments
* massage changelog and comments to use passive voice, per Boris
* add Acked-by and Reviewed-by, thanks Boris and David

v1: https://lore.kernel.org/lkml/[email protected]

Mike Rapoport (2):
x86/setup: consolidate early memory reservations x86/setup: merge several reservations of start of the memory

 arch/x86/kernel/setup.c | 96 ++++++++++++++++++-----------------------
 1 file changed, 42 insertions(+), 54 deletions(-)
Signature · 1 line
2.28.0
Signature · 2 lines
Sincerely yours,
Mike.