efi/x86: Avoid the need to mangle the EFI memory map
v3 cover efi/x86: Avoid the need to mangle the EFI memory map
From: Ard Biesheuvel <[email protected]>
At boot, x86 uses E820 tables (3 different versions!), memblock tables and the EFI memory map to reason about which parts of system RAM are available to the OS, and which are reserved.
While other EFI architectures treat the EFI memory map as immutable, the x86 boot code modifies it to keep track of memory reservations of boot services data regions, in order to distinguish which parts have been memblock_reserve()'d permanently, and which ones have been reserved only temporarily to work around buggy implementations of the EFI runtime service [SetVirtualAddressMap()] that reconfigures the VA space of the runtime services themselves.
This method is mostly fine for marking entire regions as reserved, but it gets complicated when the code decides to split EFI memory map entries in order to mark some of it permanently reserved, and the rest of it temporarily reserved.
Let's clean this up, by
- marking permanent reservations of EFI boot services data memory as MEMBLOCK_RSRV_KERN
- taking this marking into account when deciding whether or not a EFI boot services data region can be freed
- dropping all of the EFI memory map insertion/splitting logic and the allocation/freeing logic, all of which have become redundant.
Changes since v2:
- Avoid relying on memblock tables after those may have been freed already (spotted by Sashiko). Instead, tweak the ranges_to_free code added recently by Mike so that the array can grow arbitrarily, and carry multiple entries per EFI boot services data region.
- Fix use of the memory attributes table after kexec too, which is now feasible given that memblock_reserve()'ing EFI boot services memory is no longer broken - this supersedes [0]
- Drop memblock changes that were merged into 7.1-rc0 (formerly #1-#2)
Changes since v1:
- Also get rid of all reallocation logic, and just reuse the initial allocation throughout, and keep track of the number of valid entries
- Drop abuse of the EFI_MEMORY_RUNTIME flag
- Add acks from Mike to #1-#2
[0] https://lore.kernel.org/all/[email protected]/
Ard Biesheuvel (17):
x86/efi: Omit redundant kernel image overlap check x86/efi: Drop redundant EFI_PARAVIRT check
x86/efi: Only merge EFI memory map entries on 32-bit systems x86/efi: Defer sub-1M check from unmap to free stage x86/efi: Simplify real mode trampoline allocation quirk x86/efi: Unmap kernel-reserved boot regions from EFI page tables x86/efi: Drop EFI_MEMORY_RUNTIME check from __ioremap_check_other() x86/efi: Allow ranges_to_free array to grow beyond initial size x86/efi: Intersect ranges_to_free with MEMBLOCK_RSRV_KERN regions x86/efi: Do not rely on EFI_MEMORY_RUNTIME bit and avoid entry
splitting
efi: Use nr_map not map_end to find the last valid memory map entry x86/efi: Clean the memory map using iterator and filter API x86/efi: Update the runtime map in place
x86/efi: Reuse memory map instead of reallocating it x86/efi: Merge two traversals of the memory map when freeing boot
regions
x86/efi: Avoid EFI_MEMORY_RUNTIME for early EFI boot memory
reservations
x86/efi: Drop kexec quirk for the EFI memory attributes table
arch/x86/include/asm/efi.h | 15 +- arch/x86/mm/ioremap.c | 8 +- arch/x86/platform/efi/Makefile | 2 +- arch/x86/platform/efi/efi.c | 167 +++-------- arch/x86/platform/efi/efi_32.c | 31 +++ arch/x86/platform/efi/memmap.c | 247 ----------------- arch/x86/platform/efi/quirks.c | 291 +++++++------------- arch/x86/platform/efi/runtime-map.c | 4 +- drivers/firmware/efi/arm-runtime.c | 2 +- drivers/firmware/efi/memmap.c | 8 +- drivers/firmware/efi/riscv-runtime.c | 2 +- include/linux/efi.h | 13 +- 12 files changed, 197 insertions(+), 593 deletions(-) delete mode 100644 arch/x86/platform/efi/memmap.c
base-commit: 2e68039281932e6dc37718a1ea7cbb8e2cda42e6
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 1/17 x86/efi: Omit redundant kernel image overlap check
From: Ard Biesheuvel <[email protected]>
The physical region covering the kernel's executable image is memblock_reserve()'d in early_mem_reserve(), and so it is guaranteed not to intersect with the regions passed to can_free_region(). So remove the pointless overlap check.
--- arch/x86/platform/efi/quirks.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)
arch/x86/platform/efi/quirks.c +4 -11
@@ -305,16 +305,11 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size)* can free regions in efi_free_boot_services(). * * Use this function to ensure we do not free regions owned by somebody -* else. We must only reserve (and then free) regions: -* -* - Not within any part of the kernel -* - Not the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) +* else. We must only reserve (and then free) regions that do not intersect +* with the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) */ static __init bool can_free_region (u64 start,u64 size) { -if (start + size >__pa_symbol (_text) && start <=__pa_symbol (_end)) -return false; -if (!e820__mapped_all (start, start+size,E820_TYPE_RAM ))return false;
@@ -343,10 +338,8 @@ void __init efi_reserve_boot_services(void)* Because the following memblock_reserve() is paired * with free_reserved_area() for this region in * efi_free_boot_services(), we must be extremely -* careful not to reserve, and subsequently free, -* critical regions of memory (like the kernel image) or -* those regions that somebody else has already -* reserved. +* careful not to reserve, and subsequently free, critical +* regions of memory that somebody else has already reserved. * * A good example of a critical region that must not be * freed is page zero (first 4Kb of memory), which may
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Re: [PATCH v3 01/17] x86/efi: Omit redundant kernel image overlap check
Quoted 8 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
The physical region covering the kernel's executable image is memblock_reserve()'d in
early_mem_reserve(), and so it is guaranteed not to intersect with the regions passed tocan_free_region(). So remove the pointless overlap check.Signed-off-by Ard Biesheuvel <[email protected]>
Quoted 43 lines
--- arch/x86/platform/efi/quirks.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)arch/x86/platform/efi/quirks.c +4
-11@@ -305,16 +305,11 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size)* can free regions in efi_free_boot_services(). * * Use this function to ensure we do not free regions owned by somebody -* else. We must only reserve (and then free) regions: -* -* - Not within any part of the kernel -* - Not the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) +* else. We must only reserve (and then free) regions that do not intersect +* with the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) */ static __init bool can_free_region (u64 start,u64 size) { -if (start + size >__pa_symbol (_text) && start <=__pa_symbol (_end)) -return false; -if (!e820__mapped_all (start, start+size,E820_TYPE_RAM ))return false;@@ -343,10 +338,8 @@ void __init efi_reserve_boot_services(void)* Because the following memblock_reserve() is paired * with free_reserved_area() for this region in * efi_free_boot_services(), we must be extremely -* careful not to reserve, and subsequently free, -* critical regions of memory (like the kernel image) or -* those regions that somebody else has already -* reserved. +* careful not to reserve, and subsequently free, critical +* regions of memory that somebody else has already reserved. * * A good example of a critical region that must not be * freed is page zero (first 4Kb of memory), which may Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 01/17] x86/efi: Omit redundant kernel image overlap check
In reply to Ard BiesheuvelQuoted 8 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
The physical region covering the kernel's executable image is memblock_reserve()'d in
early_mem_reserve(), and so it is guaranteed not to intersect with the regions passed tocan_free_region(). So remove the pointless overlap check.Signed-off-by Ard Biesheuvel <[email protected]>
Quoted 43 lines
--- arch/x86/platform/efi/quirks.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)arch/x86/platform/efi/quirks.c +4
-11@@ -305,16 +305,11 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size)* can free regions in efi_free_boot_services(). * * Use this function to ensure we do not free regions owned by somebody -* else. We must only reserve (and then free) regions: -* -* - Not within any part of the kernel -* - Not the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) +* else. We must only reserve (and then free) regions that do not intersect +* with the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) */ static __init bool can_free_region (u64 start,u64 size) { -if (start + size >__pa_symbol (_text) && start <=__pa_symbol (_end)) -return false; -if (!e820__mapped_all (start, start+size,E820_TYPE_RAM ))return false;@@ -343,10 +338,8 @@ void __init efi_reserve_boot_services(void)* Because the following memblock_reserve() is paired * with free_reserved_area() for this region in * efi_free_boot_services(), we must be extremely -* careful not to reserve, and subsequently free, -* critical regions of memory (like the kernel image) or -* those regions that somebody else has already -* reserved. +* careful not to reserve, and subsequently free, critical +* regions of memory that somebody else has already reserved. * * A good example of a critical region that must not be * freed is page zero (first 4Kb of memory), which may Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 2/17 x86/efi: Drop redundant EFI_PARAVIRT check
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
efi_memblock_x86_reserve_range() exits early if EFI_PARAVIRT is set, so there is no point in checking it a second time further down.
--- arch/x86/platform/efi/efi.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
arch/x86/platform/efi/efi.c +3 -5
@@ -211,11 +211,9 @@ int __init efi_memblock_x86_reserve_range(void) data.desc_size = e->efi_memdesc_size; data.desc_version = e->efi_memdesc_version; -if (!efi_enabled (EFI_PARAVIRT )) { - rv =efi_memmap_init_early (&data); -if (rv) -return rv; - } + rv =efi_memmap_init_early (&data); +if (rv) +return rv;if (add_efi_memmap ||do_efi_soft_reserve ())do_add_efi_memmap ();
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Re: [PATCH v3 02/17] x86/efi: Drop redundant EFI_PARAVIRT check
Quoted 6 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
efi_memblock_x86_reserve_range()exits early ifEFI_PARAVIRTis set, so there is no point in checking it a second time further down.Signed-off-by Ard Biesheuvel <[email protected]>
Quoted 26 lines
--- arch/x86/platform/efi/efi.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)arch/x86/platform/efi/efi.c +3
-5@@ -211,11 +211,9 @@ int __init efi_memblock_x86_reserve_range(void) data.desc_size = e->efi_memdesc_size; data.desc_version = e->efi_memdesc_version; -if (!efi_enabled (EFI_PARAVIRT )) { - rv =efi_memmap_init_early (&data); -if (rv) -return rv; - } + rv =efi_memmap_init_early (&data); +if (rv) +return rv;if (add_efi_memmap ||do_efi_soft_reserve ())do_add_efi_memmap ();Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Signature · 2 lines
Sincerely yours, Mike.
Re: [PATCH v3 02/17] x86/efi: Drop redundant EFI_PARAVIRT check
In reply to Ard BiesheuvelQuoted 6 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
efi_memblock_x86_reserve_range()exits early ifEFI_PARAVIRTis set, so there is no point in checking it a second time further down.Signed-off-by Ard Biesheuvel <[email protected]>
PATCH v3 3/17 x86/efi: Only merge EFI memory map entries on 32-bit systems
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Commit
202f9d0a4180 ("x86, efi: Merge contiguous memory regions of the same type and attribute")
introduced a pass over the EFI memory map, ensuring that contiguous regions of the same type and attribute are coalesced into a single entry. This was needed because relative references may exist between those regions, and so the virtual remapping needs to preserve the relative placement of these regions. This virtual remapping was based on ioremap() at the time, which does not guarantee that adjacent physical addresses are mapped adjacently in the virtual space.
Commit
d2f7cbe7b26a ("x86/efi: Runtime services virtual mapping")
introduced a new strategy for virtually remapping the EFI runtime services, which is now the only remaining one, and commit
a5caa209ba9c ("x86/efi: Fix boot crash by mapping EFI memmap entries bottom-up at runtime, instead of top-down")
tweaked the logic to ensure that the relative offset of adjacent regions of any type is preserved on 64-bit systems, by reversing the order in which the EFI memory map is traversed when choosing the virtual placement.
This means that merging regions is no longer needed on 64-bit, given that the relative placement of adjacent regions is guaranteed to be preserved in the virtual space. So make this hack 32-bit only.
--- arch/x86/include/asm/efi.h | 6 ++++ arch/x86/platform/efi/efi.c | 31 -------------------- arch/x86/platform/efi/efi_32.c | 31 ++++++++++++++++++++ 3 files changed, 37 insertions(+), 31 deletions(-)
arch/x86/include/asm/efi.h +6 -0
@@ -143,6 +143,12 @@ extern void efi_unmap_boot_services(void);void arch_efi_call_virt_setup (void );void arch_efi_call_virt_teardown (void ); +#ifdef CONFIG_X86_32 +void efi_merge_regions (void ); +#else +static inline void efi_merge_regions (void ) {} +#endif +extern u64 efi_setup;#ifdef CONFIG_EFI
arch/x86/platform/efi/efi.c +0 -31
@@ -502,37 +502,6 @@ void __init efi_init(void)efi_print_memmap (); } -/* Merge contiguous regions of the same type and attribute */ -static void __init efi_merge_regions (void ) -{ -efi_memory_desc_t *md, *prev_md =NULL ; - -for_each_efi_memory_desc (md) { -u64 prev_size; - -if (!prev_md) { - prev_md = md; -continue ; - } - -if (prev_md->type != md->type || - prev_md->attribute != md->attribute) { - prev_md = md; -continue ; - } - - prev_size = prev_md->num_pages <<EFI_PAGE_SHIFT ; - -if (md->phys_addr == (prev_md->phys_addr + prev_size)) { - prev_md->num_pages += md->num_pages; - md->type =EFI_RESERVED_TYPE ; - md->attribute =0 ; -continue ; - } - prev_md = md; - } -} -static void *realloc_pages (void *old_memmap,int old_shift) {void *ret;
arch/x86/platform/efi/efi_32.c +31 -0
@@ -152,3 +152,34 @@ void arch_efi_call_virt_teardown(void)firmware_restrict_branch_speculation_end ();efi_fpu_end (); } + +/* Merge contiguous regions of the same type and attribute */ +void __init efi_merge_regions (void ) +{ +efi_memory_desc_t *md, *prev_md =NULL ; + +for_each_efi_memory_desc (md) { +u64 prev_size; + +if (!prev_md) { + prev_md = md; +continue ; + } + +if (prev_md->type != md->type || + prev_md->attribute != md->attribute) { + prev_md = md; +continue ; + } + + prev_size = prev_md->num_pages <<EFI_PAGE_SHIFT ; + +if (md->phys_addr == (prev_md->phys_addr + prev_size)) { + prev_md->num_pages += md->num_pages; + md->type =EFI_RESERVED_TYPE ; + md->attribute =0 ; +continue ; + } + prev_md = md; + } +}
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Re: [PATCH v3 03/17] x86/efi: Only merge EFI memory map entries on 32-bit systems
Quoted 2 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
... snip ...
Quoted 6 lines
This means that merging regions is no longer needed on 64-bit, given that the relative placement of adjacent regions is guaranteed to be preserved in the virtual space. So make this hack 32-bit only.
Signed-off-by Ard Biesheuvel <[email protected]>
Will try to set some time aside to test this, but this may actually resolve some numa/efi quirks i've observed in some KVM/QEMU configs when trying out different memory layouts - so thank you for this.
In the meantime, diff seems reasonable as-is, will try to come by with a test tag.
~Gregory
PATCH v3 4/17 x86/efi: Defer sub-1M check from unmap to free stage
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
As a first step towards moving the free logic to a later stage altogether, and only keeping the unmap and the realmode trampoline hack during the early stage of freeing the boot service code and data regions, move the logic that avoids freeing memory below 1M to the later stage.
--- arch/x86/platform/efi/quirks.c | 28 +++++++++----------- 1 file changed, 12 insertions(+), 16 deletions(-)
arch/x86/platform/efi/quirks.c +12 -16
@@ -468,18 +468,6 @@ void __init efi_unmap_boot_services(void) size -= rm_size; } -/* -* Don't free memory under 1M for two reasons: -* - BIOS might clobber it -* - Crash kernel needs it to be reserved -*/ -if (start + size <SZ_1M ) -continue ; -if (start <SZ_1M ) { - size -= (SZ_1M - start); - start =SZ_1M ; - } -/* * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory * map are still not initialized and we can't reliably free
@@ -537,12 +525,20 @@ static int __init efi_free_boot_services(void)if (!ranges_to_free)return 0 ; -while (range->start) { -void *start =phys_to_virt (range->start); +while (range->start || range->end) { +/* +* Don't free memory under 1M for two reasons: +* - BIOS might clobber it +* - Crash kernel needs it to be reserved +*/ +unsigned long s =max (range->start,SZ_1M ); +void *start =phys_to_virt (s);void *end =phys_to_virt (range->end); -free_reserved_area (start, end,-1 ,NULL ); - freed += (end - start); +if (start < end) { +free_reserved_area (start, end,-1 ,NULL ); + freed += (end - start); + } range++; }kfree (ranges_to_free);
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Re: [PATCH v3 04/17] x86/efi: Defer sub-1M check from unmap to free stage
Quoted 9 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
As a first step towards moving the free logic to a later stage altogether, and only keeping the unmap and the realmode trampoline hack during the early stage of freeing the boot service code and data regions, move the logic that avoids freeing memory below 1M to the later stage.
Signed-off-by Ard Biesheuvel <[email protected]>
Quoted 3 lines
--- arch/x86/platform/efi/quirks.c | 28 +++++++++----------- 1 file changed, 12 insertions(+), 16 deletions(-)
Signature · 2 lines
Sincerely yours, Mike.
PATCH v3 5/17 x86/efi: Simplify real mode trampoline allocation quirk
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
To work around a common bug in EFI firmware for x86 systems, Linux reserves all EFI boot services code and data regions until after it has invoked the SetVirtualAddressMap() EFI runtime service. This is needed because those regions may still be accessed by the firmware during that call, even though the EFI spec says that they shouldn't.
This includes any boot services data regions below 1M, which might mean that by the time the real mode trampoline is being allocated, all memory below 1M is already exhausted.
Commit
5bc653b73182 ("x86/efi: Allocate a trampoline if needed in efi_free_boot_services()")
added a quirk to detect this condition, and to make another attempt at allocating the real mode trampoline when freeing those boot services regions again. This is a rather crude hack, which gets in the way of cleanup work on the EFI/x86 memory map handling code.
Given that
- the real mode trampoline is normally allocated soon after all EFI boot services regions are reserved temporarily,
- this allocation logic marks all memory below 1M as reserved,
- the trampoline memory is not actually populated until an early initcall,
there is actually no need to reserve any boot services regions below 1M, even if they are mapped into the EFI page tables during the call to SetVirtualAddressMap(). So cap the lower bound of the reserved regions to 1M, and fix up the size accordingly when making the reservation. This allows the additional quirk to be dropped entirely.
--- arch/x86/platform/efi/quirks.c | 34 ++++---------------- 1 file changed, 6 insertions(+), 28 deletions(-)
arch/x86/platform/efi/quirks.c +6 -28
@@ -324,10 +324,14 @@ void __init efi_reserve_boot_services(void)return ;for_each_efi_memory_desc (md) { -u64 start = md->phys_addr; -u64 size = md->num_pages <<EFI_PAGE_SHIFT ; +u64 start =max (md->phys_addr,SZ_1M ); +u64 end = md->phys_addr + (md->num_pages <<EFI_PAGE_SHIFT ); +u64 size = end - start;bool already_reserved; +if (end <= start) +continue ; +if (md->type !=EFI_BOOT_SERVICES_CODE && md->type !=EFI_BOOT_SERVICES_DATA )continue ;
@@ -340,11 +344,6 @@ void __init efi_reserve_boot_services(void)* efi_free_boot_services(), we must be extremely * careful not to reserve, and subsequently free, critical * regions of memory that somebody else has already reserved. -* -* A good example of a critical region that must not be -* freed is page zero (first 4Kb of memory), which may -* contain boot services code/data but is marked -* E820_TYPE_RESERVED by trim_bios_range(). */ if (!already_reserved) {memblock_reserve (start, size);
@@ -427,7 +426,6 @@ void __init efi_unmap_boot_services(void)for_each_efi_memory_desc (md) {unsigned long long start = md->phys_addr;unsigned long long size = md->num_pages <<EFI_PAGE_SHIFT ; -size_t rm_size;if (md->type !=EFI_BOOT_SERVICES_CODE && md->type !=EFI_BOOT_SERVICES_DATA ) {
@@ -448,26 +446,6 @@ void __init efi_unmap_boot_services(void)*/ efi_unmap_pages (md); -/* -* Nasty quirk: if all sub-1MB memory is used for boot -* services, we can get here without having allocated the -* real mode trampoline. It's too late to hand boot services -* memory back to the memblock allocator, so instead -* try to manually allocate the trampoline if needed. -* -* I've seen this on a Dell XPS 13 9350 with firmware -* 1.4.4 with SGX enabled booting Linux via Fedora 24's -* grub2-efi on a hard disk. (And no, I don't know why -* this happened, but Linux should still try to boot rather -* panicking early.) -*/ - rm_size =real_mode_size_needed (); -if (rm_size && (start + rm_size) < (1 <<20 ) && size >= rm_size) { -set_real_mode_mem (start); - start += rm_size; - size -= rm_size; - } -/* * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory * map are still not initialized and we can't reliably free
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Re: [PATCH v3 05/17] x86/efi: Simplify real mode trampoline allocation quirk
Quoted 45 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
To work around a common bug in EFI firmware for x86 systems, Linux reserves all EFI boot services code and data regions until after it has invoked the
SetVirtualAddressMap()EFI runtime service. This is needed because those regions may still be accessed by the firmware during that call, even though the EFI spec says that they shouldn't.This includes any boot services data regions below 1M, which might mean that by the time the real mode trampoline is being allocated, all memory below 1M is already exhausted.
Commit
5bc653b73182("x86/efi: Allocate a trampoline if needed inefi_free_boot_services()")added a quirk to detect this condition, and to make another attempt at allocating the real mode trampoline when freeing those boot services regions again. This is a rather crude hack, which gets in the way of cleanup work on the EFI/x86 memory map handling code.
Given that
- the real mode trampoline is normally allocated soon after all EFI boot services regions are reserved temporarily,
- this allocation logic marks all memory below 1M as reserved,
- the trampoline memory is not actually populated until an early initcall,there is actually no need to reserve any boot services regions below 1M, even if they are mapped into the EFI page tables during the call to
SetVirtualAddressMap(). So cap the lower bound of the reserved regions to 1M, and fix up the size accordingly when making the reservation. This allows the additional quirk to be dropped entirely.Signed-off-by Ard Biesheuvel <[email protected]>--- arch/x86/platform/efi/quirks.c | 34 ++++---------------- 1 file changed, 6 insertions(+), 28 deletions(-)arch/x86/platform/efi/quirks.c +0
-0
arch/x86/platform/efi/quirks.c void __init efi_reserve_boot_services(void) · line 327
Quoted 6 lines
arch/x86/platform/efi/quirks.c +1
-2@@ -324,10 +324,14 @@ void __init efi_reserve_boot_services(void)return ;for_each_efi_memory_desc (md) { -u64 start = md->phys_addr; -u64 size = md->num_pages <<EFI_PAGE_SHIFT ; +u64 start =max (md->phys_addr,SZ_1M );
A comment that says that we don't reserve regions below 1M because they are reserved elsewhere would be nice here. Other that that
Quoted 10 lines
arch/x86/platform/efi/quirks.c +5
-0+u64 end = md->phys_addr + (md->num_pages <<EFI_PAGE_SHIFT ); +u64 size = end - start;bool already_reserved; +if (end <= start) +continue ; +if (md->type !=EFI_BOOT_SERVICES_CODE && md->type !=EFI_BOOT_SERVICES_DATA )continue ;
Signature · 2 lines
Sincerely yours, Mike.
PATCH v3 6/17 x86/efi: Unmap kernel-reserved boot regions from EFI page tables
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Currently, the logic that unmaps boot services code and data regions that were mapped temporarily to work around firmware bugs disregards regions that have been marked as EFI_MEMORY_RUNTIME. However, such regions only have significance to the OS, and there is no reason to retain the mapping in the EFI page tables, given that the runtime firmware must never touch those regions.
So pull the unmap forward.
--- arch/x86/platform/efi/quirks.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
arch/x86/platform/efi/quirks.c +6 -6
@@ -433,12 +433,6 @@ void __init efi_unmap_boot_services(void)continue ; } -/* Do not free, someone else owns it: */ -if (md->attribute &EFI_MEMORY_RUNTIME ) { - num_entries++; -continue ; - } -/* * Before calling set_virtual_address_map(), EFI boot services * code/data regions were mapped as a quirk for buggy firmware.
@@ -446,6 +440,12 @@ void __init efi_unmap_boot_services(void)*/ efi_unmap_pages (md); +/* Do not free, someone else owns it: */ +if (md->attribute &EFI_MEMORY_RUNTIME ) { + num_entries++; +continue ; + } +/* * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory * map are still not initialized and we can't reliably free
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 7/17 x86/efi: Drop EFI_MEMORY_RUNTIME check from __ioremap_check_other()
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
__ioremap_check_other() is called when memremap() is used on memory that turns out to be reserved. This may be the case for ESRT or MOK tables that are reserved via efi_mem_reserve(), in which case they will be covered by EfiBootServicesData entries in the EFI memory map.
Such entries are created with the EFI_MEMORY_RUNTIME attribute set, to distinguish them from EfiBootServicesData entries that were reserved only temporarily, in order to work around firmware bugs.
However, given that
a) __ioremap_check_other() is only called for memory that could not be mapped using try_ram_remap(),
b) on x86, the EFI memory map only retains EfiBootServicesData entries that cover a permanent reservation,
the EFI_MEMORY_RUNTIME check is redundant, and can be dropped.
This removes the need to set this attribute in the first place, which is desirable as it results in considerable complexity in managing the EFI memory map on x86. This will be addressed in follow-up work.
While at it, use switch() rather than if() to avoid multiple calls to efi_mem_type(), which is backed by a hypervisor call in some cases.
--- arch/x86/mm/ioremap.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
arch/x86/mm/ioremap.c +5 -3
@@ -124,10 +124,12 @@ static void __ioremap_check_other(resource_size_t addr, struct ioremap_desc *desif (!IS_ENABLED (CONFIG_EFI ))return ; -if (efi_mem_type (addr) ==EFI_RUNTIME_SERVICES_DATA || - (efi_mem_type (addr) ==EFI_BOOT_SERVICES_DATA && -efi_mem_attributes (addr) &EFI_MEMORY_RUNTIME )) +switch (efi_mem_type (addr)) { +case EFI_RUNTIME_SERVICES_DATA : +case EFI_BOOT_SERVICES_DATA : desc->flags |=IORES_MAP_ENCRYPTED ; +break ; + } }static int __ioremap_collect_map_flags (struct resource *res,void *arg)
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 8/17 x86/efi: Allow ranges_to_free array to grow beyond initial size
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
In order to avoid the need to mangle the EFI memory map, which is being done to keep track of which boot services data regions are really really reserved, and which ones are only reserved temporarily, this information needs to be recorded in a different manner.
The temporary ranges_to_free array is a suitable candidate, as it is specifically intended to capture which boot services data regions should be handed back to the page allocator once deferred struct page initialization is done.
This requires that boot services data regions are intersected with the memblock reserved list, and this may result in more ranges_to_free elements than the current upper bound of the number of EFI memory map entries.
So reallocate the array when running out of slots.
--- arch/x86/platform/efi/quirks.c | 40 ++++++++++++++++---- 1 file changed, 32 insertions(+), 8 deletions(-)
arch/x86/platform/efi/quirks.c +32 -8
@@ -401,23 +401,46 @@ struct efi_freeable_range {u64 end; }; -static struct efi_freeable_range *ranges_to_free; +static struct efi_freeable_range *ranges_to_free__initdata ; +static int num_to_free__initdata ; + +static int __init efi_add_range_to_free (u64 range_start,u64 range_end) +{ +static int idx__initdata ; + + ranges_to_free[idx].start = range_start; + ranges_to_free[idx].end = range_end; + +if (++idx >= num_to_free) { + num_to_free *=2 ; + ranges_to_free =krealloc_array (ranges_to_free, + num_to_free, +sizeof (ranges_to_free[0 ]), +GFP_KERNEL ); +if (!ranges_to_free) +return -ENOMEM ; + } + +/* add a terminating entry at the end */ + ranges_to_free[idx].start = ranges_to_free[idx].end =0 ; + +return 0 ; +}void __init efi_unmap_boot_services (void ) {struct efi_memory_map_data data = {0 };efi_memory_desc_t *md;int num_entries =0 ; -int idx =0 ; -size_t sz;void *new, *new_md;/* Keep all regions for /sys/kernel/debug/efi */ if (efi_enabled (EFI_DBG ))return ; - sz =sizeof (*ranges_to_free) * (efi.memmap.nr_map +1 ); - ranges_to_free =kzalloc (sz,GFP_KERNEL ); + num_to_free = efi.memmap.nr_map; + ranges_to_free =kmalloc_array (num_to_free,sizeof (ranges_to_free[0 ]), +GFP_KERNEL );if (!ranges_to_free) {pr_err ("Failed to allocate storage for freeable EFI regions\n" );return ;
@@ -452,9 +475,10 @@ void __init efi_unmap_boot_services(void)* memory here. * Queue the ranges to free at a later point. */ - ranges_to_free[idx].start = start; - ranges_to_free[idx].end = start + size; - idx++; +if (efi_add_range_to_free (start, start + size)) { +pr_err ("Failed to reallocate storage for freeable EFI regions\n" ); +return ; + } }if (!num_entries)
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Re: [PATCH v3 08/17] x86/efi: Allow ranges_to_free array to grow beyond initial size
Quoted 4 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
In order to avoid the need to mangle the EFI memory map, which is being done to keep track of which boot services data regions are really reallyo
typo: really reallyo
Quoted 8 lines
reserved, and which ones are only reserved temporarily, this information needs to be recorded in a different manner.
The temporary
ranges_to_freearray is a suitable candidate, as it is specifically intended to capture which boot services data regions should be handed back to the page allocator once deferred struct page initialization is done.
Quoted 8 lines
This requires that boot services data regions are intersected with the memblock reserved list, and this may result in more
ranges_to_freeelements than the current upper bound of the number of EFI memory map entries.So reallocate the array when running out of slots.
Signed-off-by Ard Biesheuvel <[email protected]>
Quoted 3 lines
--- arch/x86/platform/efi/quirks.c | 40 ++++++++++++++++---- 1 file changed, 32 insertions(+), 8 deletions(-)
Signature · 2 lines
Sincerely yours, Mike.
PATCH v3 9/17 x86/efi: Intersect ranges_to_free with MEMBLOCK_RSRV_KERN regions
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
efi_mem_reserve() will mark reservations in memblock with the MEMBLOCK_RSRV_KERN attribute, so take this into account when building the ranges_to_free array. This removes the need to split EFI memory map entries and tag them with the EFI_MEMORY_RUNTIME attribute.
--- arch/x86/platform/efi/quirks.c | 44 +++++++++++++++----- 1 file changed, 33 insertions(+), 11 deletions(-)
arch/x86/platform/efi/quirks.c +33 -11
@@ -406,19 +406,41 @@ static int num_to_free __initdata;static int __init efi_add_range_to_free (u64 range_start,u64 range_end) { +struct memblock_region *region;static int idx__initdata ; - ranges_to_free[idx].start = range_start; - ranges_to_free[idx].end = range_end; - -if (++idx >= num_to_free) { - num_to_free *=2 ; - ranges_to_free =krealloc_array (ranges_to_free, - num_to_free, -sizeof (ranges_to_free[0 ]), -GFP_KERNEL ); -if (!ranges_to_free) -return -ENOMEM ; +for_each_reserved_mem_region (region) { +u64 region_end = region->base + region->size; +u64 start, end; + +/* memblock tables are sorted so no need to carry on */ +if (region->base >= range_end) +break ; + +if (region_end < range_start) +continue ; + +if (region->flags &MEMBLOCK_RSRV_KERN ) +continue ; + + start =PAGE_ALIGN (max (range_start, region->base)); + end =PAGE_ALIGN_DOWN (min (range_end, region_end)); + +if (start >= end) +continue ; + + ranges_to_free[idx].start = start; + ranges_to_free[idx].end = end; + +if (++idx >= num_to_free) { + num_to_free *=2 ; + ranges_to_free =krealloc_array (ranges_to_free, + num_to_free, +sizeof (ranges_to_free[0 ]), +GFP_KERNEL ); +if (!ranges_to_free) +return -ENOMEM ; + } }/* add a terminating entry at the end */
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
Re: [PATCH v3 09/17] x86/efi: Intersect ranges_to_free with MEMBLOCK_RSRV_KERN regions
Quoted 8 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
efi_mem_reserve()will mark reservations in memblock with theMEMBLOCK_RSRV_KERNattribute, so take this into account when building theranges_to_freearray. This removes the need to split EFI memory map entries and tag them with theEFI_MEMORY_RUNTIMEattribute.Signed-off-by Ard Biesheuvel <[email protected]>
Quoted 3 lines
--- arch/x86/platform/efi/quirks.c | 44 +++++++++++++++----- 1 file changed, 33 insertions(+), 11 deletions(-)
Signature · 2 lines
Sincerely yours, Mike.
PATCH v3 10/17 x86/efi: Do not rely on EFI_MEMORY_RUNTIME bit and avoid entry splitting
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Now that efi_mem_reserve() has been updated to rely on RSRV_KERN memblock reservations, it is no longer needed to mark memblock reserved boot services regions as EFI_MEMORY_RUNTIME. This means that it is no longer needed to split existing entries in the EFI memory map, removing the need to re-allocate/copy/remap the entire EFI memory map on every call to efi_mem_reserve().
So drop this functionality - it is no longer needed.
--- arch/x86/include/asm/efi.h | 4 - arch/x86/platform/efi/memmap.c | 138 -------------------- arch/x86/platform/efi/quirks.c | 77 +++-------- 3 files changed, 17 insertions(+), 202 deletions(-)
arch/x86/include/asm/efi.h +0 -4
@@ -402,10 +402,6 @@ extern int __init efi_memmap_alloc(unsigned int num_entries, struct efi_memory_map_data *data);extern int __init efi_memmap_install (struct efi_memory_map_data *data); -extern int __init efi_memmap_split_count (efi_memory_desc_t *md, -struct range *range); -extern void __init efi_memmap_insert (struct efi_memory_map *old_memmap, -void *buf,struct efi_mem_range *mem);enum efi_secureboot_mode __x86_efi_boot_mode (void );
arch/x86/platform/efi/memmap.c +0 -138
141 lines not on this page · raw message
arch/x86/platform/efi/quirks.c +17 -60
126 lines not on this page · raw message
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 11/17 efi: Use nr_map not map_end to find the last valid memory map entry
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Currently, the efi.memmap struct keeps track of the start and the end of the EFI memory map in memory, as well as the number of entries.
Let's repaint the nr_map field as the number of *valid* entries, and update all the iterators and other memory map traversal routines accordingly.
This allows pruning of invalid or unneeded entries by moving the remaining entries to the start of the map, without the need for freeing/reallocating or unmapping and remapping. Now that entries are never added, but only removed, it is possible to retain the same allocation throughout the boot process, and free the part that is no longer in use afterwards.
--- arch/x86/platform/efi/efi.c | 18 ++++++++++++------ arch/x86/platform/efi/memmap.c | 2 +- arch/x86/platform/efi/quirks.c | 2 +- arch/x86/platform/efi/runtime-map.c | 4 ++-- drivers/firmware/efi/arm-runtime.c | 2 +- drivers/firmware/efi/memmap.c | 8 +++----- drivers/firmware/efi/riscv-runtime.c | 2 +- include/linux/efi.h | 8 ++++---- 8 files changed, 25 insertions(+), 21 deletions(-)
arch/x86/platform/efi/efi.c +12 -6
@@ -222,7 +222,7 @@ int __init efi_memblock_x86_reserve_range(void)"Unexpected EFI_MEMORY_DESCRIPTOR version %ld" , efi.memmap.desc_version); -memblock_reserve (pmap, efi.memmap.nr_map * efi.memmap.desc_size); +memblock_reserve (pmap, efi.memmap.num_valid_entries * efi.memmap.desc_size);set_bit (EFI_PRESERVE_BS_REGIONS , &efi.flags);return 0 ;
@@ -289,7 +289,7 @@ static void __init efi_clean_memmap(void) .phys_map = efi.memmap.phys_map, .desc_version = efi.memmap.desc_version, .desc_size = efi.memmap.desc_size, - .size = efi.memmap.desc_size * (efi.memmap.nr_map - n_removal), + .size = efi.memmap.desc_size * (efi.memmap.num_valid_entries - n_removal), .flags = 0, };
@@ -533,7 +533,8 @@ static inline void *efi_map_next_entry_reverse(void *entry) {/* Initial call */ if (!entry) -return efi.memmap.map_end - efi.memmap.desc_size; +return efi_memdesc_ptr (efi.memmap.map, efi.memmap.desc_size, + efi.memmap.num_valid_entries -1 ); entry -= efi.memmap.desc_size;if (entry < efi.memmap.map)
@@ -555,6 +556,9 @@ static inline void *efi_map_next_entry_reverse(void *entry)*/ static void *efi_map_next_entry (void *entry) { +if (!efi.memmap.num_valid_entries) +return NULL ; +if (efi_enabled (EFI_64BIT )) {/* * Starting in UEFI v2.5 the EFI_PROPERTIES_TABLE
@@ -581,7 +585,9 @@ static void *efi_map_next_entry(void *entry)return efi.memmap.map; entry += efi.memmap.desc_size; -if (entry >= efi.memmap.map_end) +if (entry >= (void *)efi_memdesc_ptr (efi.memmap.map, + efi.memmap.desc_size, + efi.memmap.num_valid_entries))return NULL ;return entry;
@@ -712,13 +718,13 @@ static void __init kexec_enter_virtual_mode(void)efi_memmap_unmap ();if (efi_memmap_init_late (efi.memmap.phys_map, - efi.memmap.desc_size * efi.memmap.nr_map)) { + efi.memmap.desc_size * efi.memmap.num_valid_entries)) {pr_err ("Failed to remap late EFI memory map\n" );clear_bit (EFI_RUNTIME_SERVICES , &efi.flags);return ; } - num_pages =ALIGN (efi.memmap.nr_map * efi.memmap.desc_size,PAGE_SIZE ); + num_pages =ALIGN (efi.memmap.num_valid_entries * efi.memmap.desc_size,PAGE_SIZE ); num_pages >>=PAGE_SHIFT ;if (efi_setup_page_tables (efi.memmap.phys_map, num_pages)) {
arch/x86/platform/efi/memmap.c +1 -1
@@ -90,7 +90,7 @@ int __init efi_memmap_alloc(unsigned int num_entries,*/ int __init efi_memmap_install (struct efi_memory_map_data *data) { -unsigned long size = efi.memmap.desc_size * efi.memmap.nr_map; +unsigned long size = efi.memmap.map_end - efi.memmap.map;unsigned long flags = efi.memmap.flags;u64 phys = efi.memmap.phys_map;int ret;
arch/x86/platform/efi/quirks.c +1 -1
@@ -410,7 +410,7 @@ void __init efi_unmap_boot_services(void)if (efi_enabled (EFI_DBG ))return ; - num_to_free = efi.memmap.nr_map; + num_to_free = efi.memmap.num_valid_entries; ranges_to_free =kmalloc_array (num_to_free,sizeof (ranges_to_free[0 ]),GFP_KERNEL );if (!ranges_to_free) {
arch/x86/platform/efi/runtime-map.c +2 -2
@@ -138,7 +138,7 @@ add_sysfs_runtime_map_entry(struct kobject *kobj, int nr,int efi_get_runtime_map_size (void ) { -return efi.memmap.nr_map * efi.memmap.desc_size; +return efi.memmap.num_valid_entries * efi.memmap.desc_size; }int efi_get_runtime_map_desc_size (void )
@@ -166,7 +166,7 @@ static int __init efi_runtime_map_init(void)if (!efi_enabled (EFI_MEMMAP ) || !efi_kobj)return 0 ; - map_entries =kzalloc_objs (entry, efi.memmap.nr_map); + map_entries =kzalloc_objs (entry, efi.memmap.num_valid_entries);if (!map_entries) { ret = -ENOMEM ; goto out;
drivers/firmware/efi/arm-runtime.c +1 -1
@@ -96,7 +96,7 @@ static int __init arm_enable_runtime_services(void)efi_memmap_unmap (); - mapsize = efi.memmap.desc_size * efi.memmap.nr_map; + mapsize = efi.memmap.desc_size * efi.memmap.num_valid_entries;if (efi_memmap_init_late (efi.memmap.phys_map, mapsize)) {pr_err ("Failed to remap EFI memory map\n" );
drivers/firmware/efi/memmap.c +3 -5
@@ -49,7 +49,7 @@ int __init __efi_memmap_init(struct efi_memory_map_data *data) } map.phys_map = data->phys_map; - map.nr_map = data->size / data->desc_size; + map.num_valid_entries = data->size / data->desc_size; map.map_end = map.map + data->size; map.desc_version = data->desc_version;
@@ -87,10 +87,8 @@ void __init efi_memmap_unmap(void)return ;if (!(efi.memmap.flags &EFI_MEMMAP_LATE )) { -unsigned long size; - - size = efi.memmap.desc_size * efi.memmap.nr_map; -early_memunmap (efi.memmap.map, size); +early_memunmap (efi.memmap.map, + efi.memmap.map_end - efi.memmap.map); }else {memunmap (efi.memmap.map); }
drivers/firmware/efi/riscv-runtime.c +1 -1
@@ -66,7 +66,7 @@ static int __init riscv_enable_runtime_services(void)efi_memmap_unmap (); - mapsize = efi.memmap.desc_size * efi.memmap.nr_map; + mapsize = efi.memmap.desc_size * efi.memmap.num_valid_entries;if (efi_memmap_init_late (efi.memmap.phys_map, mapsize)) {pr_err ("Failed to remap EFI memory map\n" );
include/linux/efi.h +4 -4
@@ -568,7 +568,7 @@ struct efi_memory_map {phys_addr_t phys_map;void *map;void *map_end; -int nr_map; +int num_valid_entries;unsigned long desc_version;unsigned long desc_size;#define EFI_MEMMAP_LATE (1UL << 0)
@@ -803,9 +803,9 @@ extern int efi_memattr_apply_permissions(struct mm_struct *mm,/* Iterate through an efi_memory_map */ #define for_each_efi_memory_desc_in_map (m, md) \ - for ((md) = (m)->map; \ - (md) && ((void *)(md) + (m)->desc_size) <= (m)->map_end; \ - (md) = (void *)(md) + (m)->desc_size) + for (int __idx = 0; \ + (md) = efi_memdesc_ptr((m)->map, (m)->desc_size, __idx), \ + __idx < (m)->num_valid_entries; ++__idx)/** * for_each_efi_memory_desc - iterate over descriptors in efi.memmap
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 12/17 x86/efi: Clean the memory map using iterator and filter API
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Instead of open coding the iteration logic, use the existing iterator API to iterate over all valid entries in the EFI memory map.
In addition, break out the logic that iterates over and conditionally suppresses memory map entries so it can be reused later, as something similar is happening two more times during boot.
Note that actually reinstalling the EFI memory map, which involves unmapping and remapping it, is no longer needed, given that the number of valid entries can only go down. So omit efi_memmap_install() and just update the number of valid entries.
--- arch/x86/platform/efi/efi.c | 30 +++++++++----------- 1 file changed, 13 insertions(+), 17 deletions(-)
arch/x86/platform/efi/efi.c +13 -17
@@ -266,36 +266,32 @@ static bool __init efi_memmap_entry_valid(const efi_memory_desc_t *md, int i)return false; } -static void __init efi_clean_memmap (void ) +static int __init +efi_memmap_filter_entries (bool (*callback)(const efi_memory_desc_t *,int )) {efi_memory_desc_t *out = efi.memmap.map;const efi_memory_desc_t *in = out; -const efi_memory_desc_t *end = efi.memmap.map_end; -int i, n_removal; +int i =0 , filtered =0 ; -for (i = n_removal =0 ; in < end; i++) { -if (efi_memmap_entry_valid (in, i)) { +for_each_efi_memory_desc (in) { +if (callback (in, i++)) {if (out != in)memcpy (out, in, efi.memmap.desc_size); out = (void *)out + efi.memmap.desc_size; }else { - n_removal++; + filtered++; } - in = (void *)in + efi.memmap.desc_size; } + efi.memmap.num_valid_entries -= filtered; +return filtered; +} -if (n_removal >0 ) { -struct efi_memory_map_data data = { - .phys_map = efi.memmap.phys_map, - .desc_version = efi.memmap.desc_version, - .desc_size = efi.memmap.desc_size, - .size = efi.memmap.desc_size * (efi.memmap.num_valid_entries - n_removal), - .flags =0 , - }; +static void __init efi_clean_memmap (void ) +{ +int n_removal =efi_memmap_filter_entries (efi_memmap_entry_valid); +if (n_removal >0 )pr_warn ("Removing %d invalid memory map entries.\n" , n_removal); -efi_memmap_install (&data); - } }/*
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 13/17 x86/efi: Update the runtime map in place
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
When creating the EFI runtime map, a copy is created containing only the entries that will be mapped on behalf of the firmware, but the assignment of the virtual address field is applied to both copies.
Subsequently, the copy is installed as the new EFI memory map, and the old one is just leaked.
This means that there is no reason whatsoever to allocate and install the copy, and it is much easier to just update the existing memory map in place to set the virtual addresses and suppress unused entries.
So reuse the filter function used by efi_clean_memmap() to drop all entries that are irrelevant, and then apply the existing logic to assign the virtual addresses and create the mappings in the EFI page tables.
Note that x86_64 and i386 traverse the memory map in opposite order, so this part remains a separate pass as before. This logic will be further simplified in subsequent patch.
--- arch/x86/platform/efi/efi.c | 82 ++++---------------- 1 file changed, 16 insertions(+), 66 deletions(-)
arch/x86/platform/efi/efi.c +16 -66
@@ -498,27 +498,6 @@ void __init efi_init(void)efi_print_memmap (); } -static void *realloc_pages (void *old_memmap,int old_shift) -{ -void *ret; - - ret = (void *)__get_free_pages (GFP_KERNEL , old_shift +1 ); -if (!ret) - goto out; - -/* -* A first-time allocation doesn't have anything to copy. -*/ -if (!old_memmap) -return ret; - -memcpy (ret, old_memmap,PAGE_SIZE << old_shift); - -out: -free_pages ((unsigned long )old_memmap, old_shift); -return ret; -} -/* * Iterate the EFI memory map in reverse order because the regions * will be mapped top-down. The end result is the same as if we had
@@ -589,7 +568,7 @@ static void *efi_map_next_entry(void *entry)return entry; } -static bool should_map_region (efi_memory_desc_t *md) +static bool should_map_region (const efi_memory_desc_t *md,int unused) {/* * Runtime regions always require runtime mappings (obviously).
@@ -642,40 +621,14 @@ static bool should_map_region(efi_memory_desc_t *md)* Map the efi memory ranges of the runtime services and update new_mmap with * virtual addresses. */ -static void *__init efi_map_regions (int *count,int *pg_shift) +static void __init efi_map_regions (void ) { -void *p, *new_memmap =NULL ; -unsigned long left =0 ; -unsigned long desc_size; -efi_memory_desc_t *md; - - desc_size = efi.memmap.desc_size; - - p =NULL ; -while ((p =efi_map_next_entry (p))) { - md = p; +efi_memory_desc_t *md =NULL ; -if (!should_map_region (md)) -continue ; +efi_memmap_filter_entries (should_map_region); +while ((md =efi_map_next_entry (md)))efi_map_region (md); - -if (left < desc_size) { - new_memmap =realloc_pages (new_memmap, *pg_shift); -if (!new_memmap) -return NULL ; - - left +=PAGE_SIZE << *pg_shift; - (*pg_shift)++; - } - -memcpy (new_memmap + (*count * desc_size), md, desc_size); - - left -= desc_size; - (*count)++; - } - -return new_memmap; }static void __init kexec_enter_virtual_mode (void )
@@ -752,9 +705,8 @@ static void __init kexec_enter_virtual_mode(void)*/ static void __init __efi_enter_virtual_mode (void ) { -int count =0 , pg_shift =0 ; -void *new_memmap =NULL ;efi_status_t status; +unsigned long size;unsigned long pa;if (efi_alloc_page_tables ()) {
@@ -762,15 +714,6 @@ static void __init __efi_enter_virtual_mode(void) goto err; } -efi_merge_regions (); - new_memmap =efi_map_regions (&count, &pg_shift); -if (!new_memmap) { -pr_err ("Error reallocating memory, EFI runtime non-functional!\n" ); - goto err; - } - - pa =__pa (new_memmap); -/* * Unregister the early EFI memmap from efi_init() and install * the new EFI memory map that we are about to pass to the
@@ -778,22 +721,29 @@ static void __init __efi_enter_virtual_mode(void)*/ efi_memmap_unmap (); -if (efi_memmap_init_late (pa, efi.memmap.desc_size * count)) { + size = efi.memmap.desc_size * efi.memmap.num_valid_entries; +if (efi_memmap_init_late (efi.memmap.phys_map, size)) {pr_err ("Failed to remap late EFI memory map\n" ); goto err; } +efi_merge_regions (); +efi_map_regions (); +if (efi_enabled (EFI_DBG )) {pr_info ("EFI runtime memory map:\n" );efi_print_memmap (); } -if (efi_setup_page_tables (pa,1 << pg_shift)) + size = efi.memmap.desc_size * efi.memmap.num_valid_entries; +if (efi_setup_page_tables (efi.memmap.phys_map, +DIV_ROUND_UP (size,PAGE_SIZE ))) goto err;efi_sync_low_kernel_mappings (); - status = efi_set_virtual_address_map(efi.memmap.desc_size * count, + pa = efi.memmap.phys_map; + status = efi_set_virtual_address_map(size, efi.memmap.desc_size, efi.memmap.desc_version, (efi_memory_desc_t *)pa,
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 14/17 x86/efi: Reuse memory map instead of reallocating it
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
The EFI memory map consists of 10s to 100s of entries of around 40 bytes each. The initial version is allocated and populated by the EFI stub, but later on, after freeing the boot services data regions and pruning the associated entries, a new memory map is allocated with room for only the remaining entries, which are typically much fewer in number.
Given that the original allocation is never freed, this does not actually save any memory currently, and it is much simpler to just move the entries that need to be preserved to the beginning of the map, and truncate it. That way, a lot of the complicated memory map allocation and freeing code can simply be dropped.
--- arch/x86/include/asm/efi.h | 5 - arch/x86/platform/efi/Makefile | 2 +- arch/x86/platform/efi/memmap.c | 109 -------------------- arch/x86/platform/efi/quirks.c | 35 +------ include/linux/efi.h | 5 +- 5 files changed, 7 insertions(+), 149 deletions(-)
arch/x86/include/asm/efi.h +0 -5
@@ -398,11 +398,6 @@ static inline void efi_reserve_boot_services(void) }#endif /* CONFIG_EFI */ -extern int __init efi_memmap_alloc (unsigned int num_entries, -struct efi_memory_map_data *data); - -extern int __init efi_memmap_install (struct efi_memory_map_data *data); -enum efi_secureboot_mode __x86_efi_boot_mode (void );#define arch_efi_boot_mode __x86_efi_boot_mode()
arch/x86/platform/efi/Makefile +1 -1
@@ -2,7 +2,7 @@ KASAN_SANITIZE := n GCOV_PROFILE := n -obj-$(CONFIG_EFI) += memmap.o quirks.o efi.o efi_$(BITS).o \ +obj-$(CONFIG_EFI) += quirks.o efi.o efi_$(BITS).o \ efi_stub_$(BITS).o obj-$(CONFIG_EFI_MIXED) += efi_thunk_$(BITS).o obj-$(CONFIG_EFI_RUNTIME_MAP) += runtime-map.o
arch/x86/platform/efi/memmap.c deleted +0 -109
109 lines not on this page · raw message
arch/x86/platform/efi/quirks.c +5 -30
@@ -401,10 +401,8 @@ static int __init efi_add_range_to_free(u64 range_start, u64 range_end,void __init efi_unmap_boot_services (void ) { -struct efi_memory_map_data data = {0 };efi_memory_desc_t *md; -int num_entries =0 ; -void *new, *new_md; +void *new_md;/* Keep all regions for /sys/kernel/debug/efi */ if (efi_enabled (EFI_DBG ))
@@ -425,7 +423,6 @@ void __init efi_unmap_boot_services(void)if (md->type !=EFI_BOOT_SERVICES_CODE && md->type !=EFI_BOOT_SERVICES_DATA ) { - num_entries++;continue ; }
@@ -438,7 +435,6 @@ void __init efi_unmap_boot_services(void)/* Do not free, someone else owns it: */ if (md->attribute &EFI_MEMORY_RUNTIME ) { - num_entries++;continue ; }
@@ -452,23 +448,6 @@ void __init efi_unmap_boot_services(void)pr_err ("Failed to reallocate storage for freeable EFI regions\n" );return ; } - -if (has_reservations) - num_entries++; - } - -if (!num_entries) -return ; - -if (efi_memmap_alloc (num_entries, &data) !=0 ) { -pr_err ("Failed to allocate new EFI memmap\n" ); -return ; - } - - new =memremap (data.phys_map, data.size,MEMREMAP_WB ); -if (!new) { -pr_err ("Failed to map new EFI memmap\n" ); -return ; }/*
@@ -476,7 +455,7 @@ void __init efi_unmap_boot_services(void)* regions that are not tagged EFI_MEMORY_RUNTIME, since those * regions have now been freed. */ - new_md = new; + new_md = efi.memmap.map;for_each_efi_memory_desc (md) { if (!(md->attribute & EFI_MEMORY_RUNTIME) && (md->type == EFI_BOOT_SERVICES_CODE ||
@@ -486,16 +465,12 @@ void __init efi_unmap_boot_services(void)continue ; } -memcpy (new_md, md, efi.memmap.desc_size); +if (new_md != md) +memcpy (new_md, md, efi.memmap.desc_size); new_md += efi.memmap.desc_size; } -memunmap (new); - -if (efi_memmap_install (&data) !=0 ) { -pr_err ("Could not install new EFI memmap\n" ); -return ; - } + efi.memmap.num_valid_entries = (new_md - efi.memmap.map) / efi.memmap.desc_size; }static int __init efi_free_boot_services (void )
include/linux/efi.h +1 -4
@@ -553,8 +553,7 @@ struct efi_unaccepted_memory {/* * Architecture independent structure for describing a memory map for the -* benefit of efi_memmap_init_early(), and for passing context between -* efi_memmap_alloc() and efi_memmap_install(). +* benefit of efi_memmap_init_early(). */ struct efi_memory_map_data {phys_addr_t phys_map;
@@ -572,8 +571,6 @@ struct efi_memory_map {unsigned long desc_version;unsigned long desc_size;#define EFI_MEMMAP_LATE (1UL << 0) -#define EFI_MEMMAP_MEMBLOCK (1UL << 1) -#define EFI_MEMMAP_SLAB (1UL << 2)unsigned long flags; };
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 15/17 x86/efi: Merge two traversals of the memory map when freeing boot regions
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Combine two traversals of the EFI memory map, one that instantiates ranges_to_free array elements, and one that prunes EFI memory map entries that do not need to be preserved.
This will make it easier to determine whether or not a EFI boot services region was freed in its entirety, and this informs the decision whether an entry needs to be preserved in the EFI runtime map.
This will allow the distinction between early reservations of EFI boot services memory (marked with the EFI_MEMORY_RUNTIME attribute) and late ones (marked using the MEMBLOCK_RSRV_KERN attribute) to be dropped in a subsequent patch.
--- arch/x86/platform/efi/quirks.c | 58 +++++++++----------- 1 file changed, 27 insertions(+), 31 deletions(-)
arch/x86/platform/efi/quirks.c +27 -31
@@ -416,13 +416,23 @@ void __init efi_unmap_boot_services(void)return ; } + new_md = efi.memmap.map;for_each_efi_memory_desc (md) {unsigned long long start = md->phys_addr;unsigned long long size = md->num_pages <<EFI_PAGE_SHIFT ;bool has_reservations = false; +/* +* Build a new EFI memmap that excludes any boot services data +* regions that do not cover any reserved areas, since those +* regions are being freed. +*/ +if (new_md != md) +memcpy (new_md, md, efi.memmap.desc_size); +if (md->type !=EFI_BOOT_SERVICES_CODE && md->type !=EFI_BOOT_SERVICES_DATA ) { + new_md += efi.memmap.desc_size;continue ; }
@@ -433,40 +443,26 @@ void __init efi_unmap_boot_services(void)*/ efi_unmap_pages (md); -/* Do not free, someone else owns it: */ -if (md->attribute &EFI_MEMORY_RUNTIME ) { -continue ; - } - -/* -* With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory -* map are still not initialized and we can't reliably free -* memory here. -* Queue the ranges to free at a later point. -*/ -if (efi_add_range_to_free (start, start + size, &has_reservations)) { -pr_err ("Failed to reallocate storage for freeable EFI regions\n" ); -return ; - } - } +if (!(md->attribute &EFI_MEMORY_RUNTIME )) { +/* +* With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory +* map are still not initialized and we can't reliably free +* memory here. +* Queue the ranges to free at a later point. +*/ +if (efi_add_range_to_free (start, start + size, &has_reservations)) { +pr_err ("Failed to reallocate storage for freeable EFI regions\n" ); +clear_bit (EFI_MEMMAP , &efi.flags); +return ; + } + +/* Continue without advancing new_md so this region is omitted */ +if (!has_reservations) +continue ; -/* -* Build a new EFI memmap that excludes any boot services -* regions that are not tagged EFI_MEMORY_RUNTIME, since those -* regions have now been freed. -*/ - new_md = efi.memmap.map; -for_each_efi_memory_desc (md) { -if (!(md->attribute &EFI_MEMORY_RUNTIME ) && - (md->type ==EFI_BOOT_SERVICES_CODE || - md->type ==EFI_BOOT_SERVICES_DATA ) && -can_free_region (md->phys_addr, - md->num_pages <<EFI_PAGE_SHIFT )) { -continue ; } -if (new_md != md) -memcpy (new_md, md, efi.memmap.desc_size); +/* Advance new_md so this region is preserved in the EFI memory map */ new_md += efi.memmap.desc_size; }
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 16/17 x86/efi: Avoid EFI_MEMORY_RUNTIME for early EFI boot memory reservations
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Currently, memblock reservations of EFI boot services memory made before all EFI boot services memory is temporarily reserved are upgraded, by being marked with the EFI_MEMORY_RUNTIME bit, and this results in the entire region to remain reserved permanently, regardless of the size of the original memblock reservation that triggered this.
This is a hack, and may be quite inefficient in cases where the firmware does a good job of merging memory map entries.
So instead, rely on the MEMBLOCK_RSRV_KERN flag, by marking existing memblock reservations with this flag before creating the new, temporary ones with the flag cleared. This unifies the treatment of early vs late memblock reservations inside EFI boot services memory, and avoids clobbering the EFI memory map.
--- arch/x86/platform/efi/quirks.c | 66 ++++++-------------- 1 file changed, 18 insertions(+), 48 deletions(-)
arch/x86/platform/efi/quirks.c +18 -48
@@ -273,7 +273,6 @@ void __init efi_reserve_boot_services(void)u64 start =max (md->phys_addr,SZ_1M );u64 end = md->phys_addr + (md->num_pages <<EFI_PAGE_SHIFT );u64 size = end - start; -bool already_reserved;if (end <= start)continue ;
@@ -282,37 +281,11 @@ void __init efi_reserve_boot_services(void) md->type != EFI_BOOT_SERVICES_DATA)continue ; - already_reserved =memblock_is_region_reserved (start, size); +/* upgrade existing reservations to MEMBLOCK_RSRV_KERN */ +if (memblock_is_region_reserved (start, size)) +memblock_reserved_mark_kern (start, size); -/* -* Because the following memblock_reserve() is paired -* with free_reserved_area() for this region in -* efi_free_boot_services(), we must be extremely -* careful not to reserve, and subsequently free, critical -* regions of memory that somebody else has already reserved. -*/ -if (!already_reserved) { -memblock_reserve (start, size); - -/* -* If we are the first to reserve the region, no -* one else cares about it. We own it and can -* free it later. -*/ -if (can_free_region (start, size)) -continue ; - } - -/* -* We don't own the region. We must not free it. -* -* Setting this bit for a boot services region really -* doesn't make sense as far as the firmware is -* concerned, but it does provide us with a way to tag -* those regions that must not be paired with -* memblock_phys_free(). -*/ - md->attribute |=EFI_MEMORY_RUNTIME ; +memblock_reserve (start, size); } }
@@ -443,25 +416,22 @@ void __init efi_unmap_boot_services(void)*/ efi_unmap_pages (md); -if (!(md->attribute &EFI_MEMORY_RUNTIME )) { -/* -* With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory -* map are still not initialized and we can't reliably free -* memory here. -* Queue the ranges to free at a later point. -*/ -if (efi_add_range_to_free (start, start + size, &has_reservations)) { -pr_err ("Failed to reallocate storage for freeable EFI regions\n" ); -clear_bit (EFI_MEMMAP , &efi.flags); -return ; - } - -/* Continue without advancing new_md so this region is omitted */ -if (!has_reservations) -continue ; - +/* +* With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory +* map are still not initialized and we can't reliably free +* memory here. +* Queue the ranges to free at a later point. +*/ +if (efi_add_range_to_free (start, start + size, &has_reservations)) { +pr_err ("Failed to reallocate storage for freeable EFI regions\n" ); +clear_bit (EFI_MEMMAP , &efi.flags); +return ; } +/* Continue without advancing new_md so this region is omitted */ +if (!has_reservations) +continue ; +/* Advance new_md so this region is preserved in the EFI memory map */ new_md += efi.memmap.desc_size; }
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog
PATCH v3 17/17 x86/efi: Drop kexec quirk for the EFI memory attributes table
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Now that the EFI memory attributes table is preserved properly, and the quirk to detect corrupted tables has been updated not to result in false positives when the number of EFI memory map entries is low compared to the number of EFI memory attributes table entries, there is no longer a need to ignore the latter when doing a kexec boot. So drop the workaround.
This reverts commit
64b45dd46e15 ("x86/efi: skip memattr table on kexec boot")
--- arch/x86/platform/efi/quirks.c | 4 ---- 1 file changed, 4 deletions(-)
arch/x86/platform/efi/quirks.c +0 -4
@@ -522,10 +522,6 @@ int __init efi_reuse_config(u64 tables, int nr_tables)if (!efi_guidcmp (guid,SMBIOS_TABLE_GUID )) ((efi_config_table_64_t *)p)->table = data->smbios; -/* Do not bother to play with mem attr table across kexec */ -if (!efi_guidcmp (guid,EFI_MEMORY_ATTRIBUTES_TABLE_GUID )) - ((efi_config_table_64_t *)p)->table =EFI_INVALID_TABLE_ADDR ; - p += sz; }early_memunmap (tablep, nr_tables * sz);
Signature · 1 line
2.54.0.rc2.544.gc7ae2d5bb8-goog