x86/efi: Prepwork for memory map cleanup
cover x86/efi: Prepwork for memory map cleanup
From: Ard Biesheuvel <[email protected]>
This is a follow-up to [0], which had some fundamental issues, as spotted by Sashiko [1]. Disentangling the EFI memory map mess on x86 needs a rethink, but some of the prepwork is still worthwhile, and so it is presented here separately.
The end goal is still the same, i.e., to remove the need for the x86 boot code to modify the EFI memory map, or reallocate/copy/remap it numerous times.
[0] https://lore.kernel.org/all/[email protected]/
[1] https://sashiko.dev/#/patchset/20260319090529.1091660-21-ardb%2Bgit%40google.com
Ard Biesheuvel (7):
x86/efi: Omit redundant kernel image overlap check x86/efi: Drop redundant EFI_PARAVIRT check
x86/efi: Only merge EFI memory map entries on 32-bit systems x86/efi: Defer sub-1M check from unmap to free stage x86/efi: Simplify real mode trampoline allocation quirk x86/efi: Unmap kernel-reserved boot regions from EFI page tables x86/efi: Drop EFI_MEMORY_RUNTIME check from __ioremap_check_other()
arch/x86/include/asm/efi.h | 6 ++ arch/x86/mm/ioremap.c | 7 +- arch/x86/platform/efi/efi.c | 39 +--------- arch/x86/platform/efi/efi_32.c | 31 ++++++++ arch/x86/platform/efi/quirks.c | 78 +++++++------------- 5 files changed, 69 insertions(+), 92 deletions(-)
base-commit: 259e3e6f9382b6a9fe570313d97c59a233f7d72f
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 1/7 x86/efi: Omit redundant kernel image overlap check
From: Ard Biesheuvel <[email protected]>
The physical region covering the kernel's executable image is memblock_reserve()'d in early_mem_reserve(), and so it is guaranteed not to intersect with the regions passed to can_free_region(). So remove the pointless overlap check.
--- arch/x86/platform/efi/quirks.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)
arch/x86/platform/efi/quirks.c +4 -11
@@ -305,16 +305,11 @@ void __init efi_arch_mem_reserve(phys_addr_t addr, u64 size)* can free regions in efi_free_boot_services(). * * Use this function to ensure we do not free regions owned by somebody -* else. We must only reserve (and then free) regions: -* -* - Not within any part of the kernel -* - Not the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) +* else. We must only reserve (and then free) regions that do not intersect +* with the BIOS reserved area (E820_TYPE_RESERVED, E820_TYPE_NVS, etc) */ static __init bool can_free_region (u64 start,u64 size) { -if (start + size >__pa_symbol (_text) && start <=__pa_symbol (_end)) -return false; -if (!e820__mapped_all (start, start+size,E820_TYPE_RAM ))return false;
@@ -343,10 +338,8 @@ void __init efi_reserve_boot_services(void)* Because the following memblock_reserve() is paired * with free_reserved_area() for this region in * efi_free_boot_services(), we must be extremely -* careful not to reserve, and subsequently free, -* critical regions of memory (like the kernel image) or -* those regions that somebody else has already -* reserved. +* careful not to reserve, and subsequently free, critical +* regions of memory that somebody else has already reserved. * * A good example of a critical region that must not be * freed is page zero (first 4Kb of memory), which may
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 2/7 x86/efi: Drop redundant EFI_PARAVIRT check
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
efi_memblock_x86_reserve_range() exits early if EFI_PARAVIRT is set, so there is no point in checking it a second time further down.
--- arch/x86/platform/efi/efi.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)
arch/x86/platform/efi/efi.c +3 -5
@@ -211,11 +211,9 @@ int __init efi_memblock_x86_reserve_range(void) data.desc_size = e->efi_memdesc_size; data.desc_version = e->efi_memdesc_version; -if (!efi_enabled (EFI_PARAVIRT )) { - rv =efi_memmap_init_early (&data); -if (rv) -return rv; - } + rv =efi_memmap_init_early (&data); +if (rv) +return rv;if (add_efi_memmap ||do_efi_soft_reserve ())do_add_efi_memmap ();
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 3/7 x86/efi: Only merge EFI memory map entries on 32-bit systems
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Commit
202f9d0a4180 ("x86, efi: Merge contiguous memory regions of the same type and attribute")
introduced a pass over the EFI memory map, ensuring that contiguous regions of the same type and attribute are coalesced into a single entry. This was needed because relative references may exist between those regions, and so the virtual remapping needs to preserve the relative placement of these regions. This virtual remapping was based on ioremap() at the time, which does not guarantee that adjacent physical addresses are mapped adjacently in the virtual space.
Commit
d2f7cbe7b26a ("x86/efi: Runtime services virtual mapping")
introduced a new strategy for virtually remapping the EFI runtime services, which is now the only remaining one, and commit
a5caa209ba9c ("x86/efi: Fix boot crash by mapping EFI memmap entries bottom-up at runtime, instead of top-down")
tweaked the logic to ensure that the relative offset of adjacent regions of any type is preserved on 64-bit systems, by reversing the order in which the EFI memory map is traversed when choosing the virtual placement.
This means that merging regions is no longer needed on 64-bit, given that the relative placement of adjacent regions is guaranteed to be preserved in the virtual space. So make this hack 32-bit only.
--- arch/x86/include/asm/efi.h | 6 ++++ arch/x86/platform/efi/efi.c | 31 -------------------- arch/x86/platform/efi/efi_32.c | 31 ++++++++++++++++++++ 3 files changed, 37 insertions(+), 31 deletions(-)
arch/x86/include/asm/efi.h +6 -0
@@ -143,6 +143,12 @@ extern void efi_unmap_boot_services(void);void arch_efi_call_virt_setup (void );void arch_efi_call_virt_teardown (void ); +#ifdef CONFIG_X86_32 +void efi_merge_regions (void ); +#else +static inline void efi_merge_regions (void ) {} +#endif +extern u64 efi_setup;#ifdef CONFIG_EFI
arch/x86/platform/efi/efi.c +0 -31
@@ -502,37 +502,6 @@ void __init efi_init(void)efi_print_memmap (); } -/* Merge contiguous regions of the same type and attribute */ -static void __init efi_merge_regions (void ) -{ -efi_memory_desc_t *md, *prev_md =NULL ; - -for_each_efi_memory_desc (md) { -u64 prev_size; - -if (!prev_md) { - prev_md = md; -continue ; - } - -if (prev_md->type != md->type || - prev_md->attribute != md->attribute) { - prev_md = md; -continue ; - } - - prev_size = prev_md->num_pages <<EFI_PAGE_SHIFT ; - -if (md->phys_addr == (prev_md->phys_addr + prev_size)) { - prev_md->num_pages += md->num_pages; - md->type =EFI_RESERVED_TYPE ; - md->attribute =0 ; -continue ; - } - prev_md = md; - } -} -static void *realloc_pages (void *old_memmap,int old_shift) {void *ret;
arch/x86/platform/efi/efi_32.c +31 -0
@@ -152,3 +152,34 @@ void arch_efi_call_virt_teardown(void)firmware_restrict_branch_speculation_end ();efi_fpu_end (); } + +/* Merge contiguous regions of the same type and attribute */ +void __init efi_merge_regions (void ) +{ +efi_memory_desc_t *md, *prev_md =NULL ; + +for_each_efi_memory_desc (md) { +u64 prev_size; + +if (!prev_md) { + prev_md = md; +continue ; + } + +if (prev_md->type != md->type || + prev_md->attribute != md->attribute) { + prev_md = md; +continue ; + } + + prev_size = prev_md->num_pages <<EFI_PAGE_SHIFT ; + +if (md->phys_addr == (prev_md->phys_addr + prev_size)) { + prev_md->num_pages += md->num_pages; + md->type =EFI_RESERVED_TYPE ; + md->attribute =0 ; +continue ; + } + prev_md = md; + } +}
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 4/7 x86/efi: Defer sub-1M check from unmap to free stage
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
As a first step towards moving the free logic to a later stage altogether, and only keeping the unmap and the realmode trampoline hack during the early stage of freeing the boot service code and data regions, move the logic that avoids freeing memory below 1M to the later stage.
--- arch/x86/platform/efi/quirks.c | 28 +++++++++----------- 1 file changed, 12 insertions(+), 16 deletions(-)
arch/x86/platform/efi/quirks.c +12 -16
@@ -468,18 +468,6 @@ void __init efi_unmap_boot_services(void) size -= rm_size; } -/* -* Don't free memory under 1M for two reasons: -* - BIOS might clobber it -* - Crash kernel needs it to be reserved -*/ -if (start + size <SZ_1M ) -continue ; -if (start <SZ_1M ) { - size -= (SZ_1M - start); - start =SZ_1M ; - } -/* * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory * map are still not initialized and we can't reliably free
@@ -537,12 +525,20 @@ static int __init efi_free_boot_services(void)if (!ranges_to_free)return 0 ; -while (range->start) { -void *start =phys_to_virt (range->start); +while (range->start || range->end) { +/* +* Don't free memory under 1M for two reasons: +* - BIOS might clobber it +* - Crash kernel needs it to be reserved +*/ +unsigned long s =max (range->start,SZ_1M ); +void *start =phys_to_virt (s);void *end =phys_to_virt (range->end); -free_reserved_area (start, end,-1 ,NULL ); - freed += (end - start); +if (start < end) { +free_reserved_area (start, end,-1 ,NULL ); + freed += (end - start); + } range++; }kfree (ranges_to_free);
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 5/7 x86/efi: Simplify real mode trampoline allocation quirk
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
To work around a common bug in EFI firmware for x86 systems, Linux reserves all EFI boot services code and data regions until after it has invoked the SetVirtualAddressMap() EFI runtime service. This is needed because those regions may still be accessed by the firmware during that call, even though the EFI spec says that they shouldn't.
This includes any boot services data regions below 1M, which might mean that by the time the real mode trampoline is being allocated, all memory below 1M is already exhausted.
Commit
5bc653b73182 ("x86/efi: Allocate a trampoline if needed in efi_free_boot_services()")
added a quirk to detect this condition, and to make another attempt at allocating the real mode trampoline when freeing those boot services regions again. This is a rather crude hack, which gets in the way of cleanup work on the EFI/x86 memory map handling code.
Given that
- the real mode trampoline is normally allocated soon after all EFI boot services regions are reserved temporarily,
- this allocation logic marks all memory below 1M as reserved,
- the trampoline memory is not actually populated until an early initcall,
there is actually no need to reserve any boot services regions below 1M, even if they are mapped into the EFI page tables during the call to SetVirtualAddressMap(). So cap the lower bound of the reserved regions to 1M, and fix up the size accordingly when making the reservation. This allows the additional quirk to be dropped entirely.
--- arch/x86/platform/efi/quirks.c | 29 ++++---------------- 1 file changed, 6 insertions(+), 23 deletions(-)
arch/x86/platform/efi/quirks.c +6 -23
@@ -324,10 +324,14 @@ void __init efi_reserve_boot_services(void)return ;for_each_efi_memory_desc (md) { -u64 start = md->phys_addr; -u64 size = md->num_pages <<EFI_PAGE_SHIFT ; +u64 start =max (md->phys_addr,SZ_1M ); +u64 end = md->phys_addr + (md->num_pages <<EFI_PAGE_SHIFT ); +u64 size = end - start;bool already_reserved; +if (end <= start) +continue ; +if (md->type !=EFI_BOOT_SERVICES_CODE && md->type !=EFI_BOOT_SERVICES_DATA )continue ;
@@ -427,7 +431,6 @@ void __init efi_unmap_boot_services(void)for_each_efi_memory_desc (md) {unsigned long long start = md->phys_addr;unsigned long long size = md->num_pages <<EFI_PAGE_SHIFT ; -size_t rm_size;if (md->type !=EFI_BOOT_SERVICES_CODE && md->type !=EFI_BOOT_SERVICES_DATA ) {
@@ -448,26 +451,6 @@ void __init efi_unmap_boot_services(void)*/ efi_unmap_pages (md); -/* -* Nasty quirk: if all sub-1MB memory is used for boot -* services, we can get here without having allocated the -* real mode trampoline. It's too late to hand boot services -* memory back to the memblock allocator, so instead -* try to manually allocate the trampoline if needed. -* -* I've seen this on a Dell XPS 13 9350 with firmware -* 1.4.4 with SGX enabled booting Linux via Fedora 24's -* grub2-efi on a hard disk. (And no, I don't know why -* this happened, but Linux should still try to boot rather -* panicking early.) -*/ - rm_size =real_mode_size_needed (); -if (rm_size && (start + rm_size) < (1 <<20 ) && size >= rm_size) { -set_real_mode_mem (start); - start += rm_size; - size -= rm_size; - } -/* * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory * map are still not initialized and we can't reliably free
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 6/7 x86/efi: Unmap kernel-reserved boot regions from EFI page tables
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
Currently, the logic that unmaps boot services code and data regions that were mapped temporarily to work around firmware bugs disregards regions that have been marked as EFI_MEMORY_RUNTIME. However, such regions only have significance to the OS, and there is no reason to retain the mapping in the EFI page tables, given that the runtime firmware must never touch those regions.
So pull the unmap forward.
--- arch/x86/platform/efi/quirks.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
arch/x86/platform/efi/quirks.c +6 -6
@@ -438,12 +438,6 @@ void __init efi_unmap_boot_services(void)continue ; } -/* Do not free, someone else owns it: */ -if (md->attribute &EFI_MEMORY_RUNTIME ) { - num_entries++; -continue ; - } -/* * Before calling set_virtual_address_map(), EFI boot services * code/data regions were mapped as a quirk for buggy firmware.
@@ -451,6 +445,12 @@ void __init efi_unmap_boot_services(void)*/ efi_unmap_pages (md); +/* Do not free, someone else owns it: */ +if (md->attribute &EFI_MEMORY_RUNTIME ) { + num_entries++; +continue ; + } +/* * With CONFIG_DEFERRED_STRUCT_PAGE_INIT parts of the memory * map are still not initialized and we can't reliably free
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
PATCH 7/7 x86/efi: Drop EFI_MEMORY_RUNTIME check from __ioremap_check_other()
In reply to Ard BiesheuvelFrom: Ard Biesheuvel <[email protected]>
__ioremap_check_other() is called when memremap() is used on memory that turns out to be reserved. This may be the case for ESRT or MOK tables that are reserved via efi_mem_reserve(), in which case they will be covered by EfiBootServicesData entries in the EFI memory map.
Such entries are created with the EFI_MEMORY_RUNTIME attribute set, to distinguish them from EfiBootServicesData entries that were reserved only temporarily, in order to work around firmware bugs.
However, given that
a) __ioremap_check_other() is only called for memory that could not be mapped using try_ram_remap(),
b) on x86, the EFI memory map only retains EfiBootServicesData entries that cover a permanent reservation,
the EFI_MEMORY_RUNTIME check is redundant, and can be dropped.
This removes the need to set this attribute in the first place, which is desirable as it results in considerable complexity in managing the EFI memory map on x86. This will be addressed in follow-up work.
While at it, use switch() rather than if() to avoid multiple calls to efi_mem_type(), which is backed by a hypervisor call in some cases.
--- arch/x86/mm/ioremap.c | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-)
arch/x86/mm/ioremap.c +4 -3
@@ -124,10 +124,11 @@ static void __ioremap_check_other(resource_size_t addr, struct ioremap_desc *desif (!IS_ENABLED (CONFIG_EFI ))return ; -if (efi_mem_type (addr) ==EFI_RUNTIME_SERVICES_DATA || - (efi_mem_type (addr) ==EFI_BOOT_SERVICES_DATA && -efi_mem_attributes (addr) &EFI_MEMORY_RUNTIME )) +switch (efi_mem_type (addr)) { +case EFI_RUNTIME_SERVICES_DATA : +case EFI_BOOT_SERVICES_DATA : desc->flags |=IORES_MAP_ENCRYPTED ; + } }static int __ioremap_collect_map_flags (struct resource *res,void *arg)
Signature · 1 line
2.53.0.1213.gd9a14994de-goog
Re: [PATCH 2/7] x86/efi: Drop redundant EFI_PARAVIRT check
In reply to Ard BiesheuvelQuoted 16 lines · Ard Biesheuvel
From: Ard Biesheuvel <[email protected]>
efi_memblock_x86_reserve_range()exits early ifEFI_PARAVIRTis set, so there is no point in checking it a second time further down.Signed-off-by Ard Biesheuvel <[email protected]>--- arch/x86/platform/efi/efi.c | 8 +++----- 1 file changed, 3 insertions(+), 5 deletions(-)arch/x86/platform/efi/efi.c +0
-0@@ -211,11 +211,9 @@ int __init efi_memblock_x86_reserve_range(void) data.desc_size = e->efi_memdesc_size;
arch/x86/platform/efi/efi.c int __init efi_memblock_x86_reserve_range(void) · line 216
Quoted 10 lines
arch/x86/platform/efi/efi.c +3
-5data.desc_version = e->efi_memdesc_version; -if (!efi_enabled (EFI_PARAVIRT )) { - rv =efi_memmap_init_early (&data); -if (rv) -return rv; - } + rv =efi_memmap_init_early (&data); +if (rv) +return rv;
Maybe add:
architectures")
?
Anyway,