Skip to contentSkip to conversation

EFI page table isolation

GIT PULL In review Matt Fleming 2 participants 11 messages Nov 23, 2015 – Nov 27, 2015

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

Folks,

This patch series is a response to the report that the EFI region mappings trigger warnings when booting with CONFIG_DEBUG_WX enabled. They allocate a new page table structure and ensure that all the mappings we require during EFI runtime calls are only setup there.

It turns out that it still makes sense to share some page table entries with 'swapper_pg_dir', just not the entries where we need to allow security lax permissions. Sharing entries is useful for memory hotplug, for example.

When writing this series I discovered a number of bugs in the existing code that only became apparent when we stopped using 'trampoline_pgd' which already mapped a bunch of things for us. I've put those bug fixes at the start of the series.

Further testing would be very much appreciated as this is a notoriously funky area of the EFI code.

Changes in v3:

- PFN_ALIGN() _text and _end when calculating npages to map

- Drop the hunk that mapped the stack in efi_setup_page_tables(), now that we're mapping RAM we know the stack is already mapped.

- Update the wording in Documentation/<..>/mm.txt to make it clear we carve out a 64Gb *size* of virtual address space, we don't use the first 64Gb virtual addresses.

Changes in v2:

- Folded PATCH 1 and 2 together because they both fall under the umbrella of "making sure cpa->pfn is really a page frame number".

- Fixed some checkpatch warnings about mixing spaces and tabs and made some stylistic changes per Borislav's comments.

- Moved efi_alloc_page_tables() earlier in __efi_enter_virtual_mode() so that we fail early if we can't allocate memory for the page tables.

The following changes since commit 2c66e24d75d424919c42288b418d2e593fa818b1:

x86/efi: Fix kernel panic when CONFIG_DEBUG_VIRTUAL is enabled (2015-10-25 10:22:25 +0000)

are available in the git repository at:

git://git.kernel.org/pub/scm/linux/kernel/git/mfleming/efi.git tags/efi-next

for you to fetch changes up to fac3e3c52017a5d974c7d8168e8e43c8f68af82a:

Documentation/x86: Update EFI memory region description (2015-11-23 12:33:09 +0000)

----------------------------------------------------------------
* Use completely separate page tables for EFI runtime service calls so that the security-lax mapping permissions (RWX) do not leak into the standard kernel page tables and trigger warnings when CONFIG_DEBUG_WX is enabled.

----------------------------------------------------------------
Matt Fleming (6):
x86/efi: PFN_ALIGN() _text and _end when calculating number of pages x86/mm/pageattr: Ensure cpa->pfn only contains page frame numbers x86/efi: Map RAM into the identity page table for mixed mode x86/efi: Hoist page table switching code into efi_call_virt() x86/efi: Build our own page table structures Documentation/x86: Update EFI memory region description

 Documentation/x86/x86_64/mm.txt     |  12 +--
 arch/x86/include/asm/efi.h          |  26 ++++++
 arch/x86/mm/pageattr.c              |  17 ++--
 arch/x86/platform/efi/efi.c         |  39 ++++-----
 arch/x86/platform/efi/efi_32.c      |   5 ++
 arch/x86/platform/efi/efi_64.c      | 159 ++++++++++++++++++++++++++++--------
 arch/x86/platform/efi/efi_stub_64.S |  43 ----------
 7 files changed, 181 insertions(+), 120 deletions(-)
#1 lore raw

PATCH 1/6 x86/efi: PFN_ALIGN() _text and _end when calculating number of pages

While _text is currently aligned to PAGE_SIZE in the vmlinux linker script because it's based on CONFIG_PHYSICAL_ALIGN, it's always better to be explicit about these things to be sure no alignment bugs are lurking. There's no analogous enforcement for _end.

Dave provided an example of why the 'npages' calculation is wrong,

"Just for fun, imagine that _end=0xfff and _text=0x1001. npages
would be 0."

Use PFN_ALIGN() to be sure the calculation is correctly aligned to PAGE_SIZE.

---
 arch/x86/platform/efi/efi_64.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/platform/efi/efi_64.c +1 -1
index a0ac0f9c307f..3a90eb72d153 100644
@@ -183,7 +183,7 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	efi_scratch.phys_stack = virt_to_phys(page_address(page));
 	efi_scratch.phys_stack += PAGE_SIZE; /* stack grows down */
 
-	npages = (_end - _text) >> PAGE_SHIFT;
+	npages = (PFN_ALIGN(_end) - PFN_ALIGN(_text)) >> PAGE_SHIFT;
 	text = __pa(_text);
 
 	if (kernel_map_pages_in_pgd(pgd, text >> PAGE_SHIFT, text, npages, 0)) {
Signature · 1 line
2.6.2
#2 lore raw

Re: [PATCH 1/6] x86/efi: PFN_ALIGN() _text and _end when calculating number of pages

Quoted 29 lines · Matt Fleming

While _text is currently aligned to PAGE_SIZE in the vmlinux linker script because it's based on CONFIG_PHYSICAL_ALIGN, it's always better to be explicit about these things to be sure no alignment bugs are lurking. There's no analogous enforcement for _end.

Dave provided an example of why the 'npages' calculation is wrong,

"Just for fun, imagine that _end=0xfff and _text=0x1001. npages
would be 0."

Use PFN_ALIGN() to be sure the calculation is correctly aligned to PAGE_SIZE.

---
 arch/x86/platform/efi/efi_64.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/platform/efi/efi_64.c +0 -0
index a0ac0f9c307f..3a90eb72d153 100644

arch/x86/platform/efi/efi_64.c int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages) · line 187

Quoted 6 lines
arch/x86/platform/efi/efi_64.c +1 -1
@@ -183,7 +183,7 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	efi_scratch.phys_stack = virt_to_phys(page_address(page));
 	efi_scratch.phys_stack += PAGE_SIZE; /* stack grows down */
 
-	npages = (_end - _text) >> PAGE_SHIFT;
+	npages = (PFN_ALIGN(_end) - PFN_ALIGN(_text)) >> PAGE_SHIFT;
 	text = __pa(_text);

Didn't we want to do the _end alignment linker script fix instead?

Alignment assumptions are easy to make when symbols are well aligned typically (as in this case), so we should guarantee the alignment property instead of complicating the code.

Thanks,

Ingo

#3 lore raw

Re: [PATCH 1/6] x86/efi: PFN_ALIGN() _text and _end when calculating number of pages

Quoted 2 lines · Ingo Molnar

Didn't we want to do the _end alignment linker script fix instead?

I think we should do both. This patch is tagged for stable because it fixes a bug in the existing code. It's obvious and it's explicit and it's much easier to know when someone might want to backport it.

Changing the linker script which indirectly fixes the above bug is a much more subtle solution, with much larger potential for fallout because it affects multiple chunks of kernel code.

Quoted 3 lines

Alignment assumptions are easy to make when symbols are well aligned typically (as in this case), so we should guarantee the alignment property instead of complicating the code.

I don't agree that sprinkling PFN_ALIGN() complicates the code, it's a minimal change with a well known kernel idiom. But yes, aligning these symbols in the linker script is generally a good idea.

The two patches are worthwhile, for different reasons; let's do both.

#4 lore raw

Re: [PATCH 1/6] x86/efi: PFN_ALIGN() _text and _end when calculating number of pages

Quoted 16 lines · Matt Fleming
Quoted 2 lines · Ingo Molnar

Didn't we want to do the _end alignment linker script fix instead?

I think we should do both. This patch is tagged for stable because it fixes a bug in the existing code. It's obvious and it's explicit and it's much easier to know when someone might want to backport it.

Changing the linker script which indirectly fixes the above bug is a much more subtle solution, with much larger potential for fallout because it affects multiple chunks of kernel code.

Quoted 3 lines

Alignment assumptions are easy to make when symbols are well aligned typically (as in this case), so we should guarantee the alignment property instead of complicating the code.

Quoted 5 lines

I don't agree that sprinkling PFN_ALIGN() complicates the code, it's a minimal change with a well known kernel idiom. But yes, aligning these symbols in the linker script is generally a good idea.

The two patches are worthwhile, for different reasons; let's do both.

I disagree, this form:

        npages = (_end - _text) >> PAGE_SHIFT;

is a lot clearer to read than:

        npages = (PFN_ALIGN(_end) - PFN_ALIGN(_text)) >> PAGE_SHIFT;

especially once we ensure that _end and _text are page aligned. The latter form will only result in cargo-cult carrying over of unnecessary PFN_ALIGN() operations.

Section boundaries of the kernel should generally be page aligned, this is useful for a number of other reasons as well.

As far as backporting goes, it would generally be _safer_ to backport the linker script fix, in case there are other unrealized alignment bugs in the kernel. Especially if upstream does the same.

Thanks,

Ingo

#5 lore raw

Re: [PATCH 1/6] x86/efi: PFN_ALIGN() _text and _end when calculating number of pages

Quoted 4 lines · Ingo Molnar

As far as backporting goes, it would generally be _safer_ to backport the linker script fix, in case there are other unrealized alignment bugs in the kernel. Especially if upstream does the same.

OK, I'll respin this series and replace this patch with the linker script fix.

#6 lore raw

PATCH v3 2/6 x86/mm/pageattr: Ensure cpa->pfn only contains page frame numbers

In reply to Matt Fleming

The x86 pageattr code is confused about the data that is stored in cpa->pfn, sometimes it's treated as a page frame number, sometimes it's treated as an unshifted physical address, and in one place it's treated as a pte.

The result of this is that the mapping functions do not map the intended physical address.

This isn't a problem in practice because most of the addresses we're mapping in the EFI code paths are already mapped in 'trampoline_pgd' and so the pageattr mapping functions don't actually do anything in this case. But when we move to using a separate page table for the EFI runtime this will be an issue.

---

Changes in v3:
- Dropped the superfluous hunk that mapped the stack.

Changes in v2:
- Folded the deletion of the _PAGE_NX code into this patch.

 arch/x86/mm/pageattr.c         | 17 ++++++-----------
 arch/x86/platform/efi/efi_64.c | 16 ++++++++++------
 2 files changed, 16 insertions(+), 17 deletions(-)
arch/x86/mm/pageattr.c +6 -11
index 9abe0c9b1098..d5240be55915 100644
@@ -885,15 +885,10 @@ static void populate_pte(struct cpa_data *cpa,
 	pte = pte_offset_kernel(pmd, start);
 
 	while (num_pages-- && start < end) {
-
-		/* deal with the NX bit */
-		if (!(pgprot_val(pgprot) & _PAGE_NX))
-			cpa->pfn &= ~_PAGE_NX;
-
-		set_pte(pte, pfn_pte(cpa->pfn >> PAGE_SHIFT, pgprot));
+		set_pte(pte, pfn_pte(cpa->pfn, pgprot));
 
 		start	 += PAGE_SIZE;
-		cpa->pfn += PAGE_SIZE;
+		cpa->pfn++;
 		pte++;
 	}
 }
@@ -949,11 +944,11 @@ static int populate_pmd(struct cpa_data *cpa,
 
 		pmd = pmd_offset(pud, start);
 
-		set_pmd(pmd, __pmd(cpa->pfn | _PAGE_PSE |
+		set_pmd(pmd, __pmd(cpa->pfn << PAGE_SHIFT | _PAGE_PSE |
 				   massage_pgprot(pmd_pgprot)));
 
 		start	  += PMD_SIZE;
-		cpa->pfn  += PMD_SIZE;
+		cpa->pfn  += PMD_SIZE >> PAGE_SHIFT;
 		cur_pages += PMD_SIZE >> PAGE_SHIFT;
 	}
 
@@ -1022,11 +1017,11 @@ static int populate_pud(struct cpa_data *cpa, unsigned long start, pgd_t *pgd,
 	 * Map everything starting from the Gb boundary, possibly with 1G pages
 	 */
 	while (end - start >= PUD_SIZE) {
-		set_pud(pud, __pud(cpa->pfn | _PAGE_PSE |
+		set_pud(pud, __pud(cpa->pfn << PAGE_SHIFT | _PAGE_PSE |
 				   massage_pgprot(pud_pgprot)));
 
 		start	  += PUD_SIZE;
-		cpa->pfn  += PUD_SIZE;
+		cpa->pfn  += PUD_SIZE >> PAGE_SHIFT;
 		cur_pages += PUD_SIZE >> PAGE_SHIFT;
 		pud++;
 	}
arch/x86/platform/efi/efi_64.c +10 -6
index 3a90eb72d153..4fb0ba21d168 100644
@@ -143,7 +143,7 @@ void efi_sync_low_kernel_mappings(void)
 
 int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 {
-	unsigned long text;
+	unsigned long pfn, text;
 	struct page *page;
 	unsigned npages;
 	pgd_t *pgd;
@@ -160,7 +160,8 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	 * and ident-map those pages containing the map before calling
 	 * phys_efi_set_virtual_address_map().
 	 */
-	if (kernel_map_pages_in_pgd(pgd, pa_memmap, pa_memmap, num_pages, _PAGE_NX)) {
+	pfn = pa_memmap >> PAGE_SHIFT;
+	if (kernel_map_pages_in_pgd(pgd, pfn, pa_memmap, num_pages, _PAGE_NX)) {
 		pr_err("Error ident-mapping new memmap (0x%lx)!\n", pa_memmap);
 		return 1;
 	}
@@ -185,8 +186,9 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 
 	npages = (PFN_ALIGN(_end) - PFN_ALIGN(_text)) >> PAGE_SHIFT;
 	text = __pa(_text);
+	pfn = text >> PAGE_SHIFT;
 
-	if (kernel_map_pages_in_pgd(pgd, text >> PAGE_SHIFT, text, npages, 0)) {
+	if (kernel_map_pages_in_pgd(pgd, pfn, text, npages, 0)) {
 		pr_err("Failed to map kernel text 1:1\n");
 		return 1;
 	}
@@ -204,12 +206,14 @@ void __init efi_cleanup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 static void __init __map_region(efi_memory_desc_t *md, u64 va)
 {
 	pgd_t *pgd = (pgd_t *)__va(real_mode_header->trampoline_pgd);
-	unsigned long pf = 0;
+	unsigned long flags = 0;
+	unsigned long pfn;
 
 	if (!(md->attribute & EFI_MEMORY_WB))
-		pf |= _PAGE_PCD;
+		flags |= _PAGE_PCD;
 
-	if (kernel_map_pages_in_pgd(pgd, md->phys_addr, va, md->num_pages, pf))
+	pfn = md->phys_addr >> PAGE_SHIFT;
+	if (kernel_map_pages_in_pgd(pgd, pfn, va, md->num_pages, flags))
 		pr_warn("Error mapping PA 0x%llx -> VA 0x%llx!\n",
 			   md->phys_addr, va);
 }
Signature · 1 line
2.6.2
#7 lore raw

PATCH 3/6 x86/efi: Map RAM into the identity page table for mixed mode

In reply to Matt Fleming

We are relying on the pre-existing mappings in 'trampoline_pgd' when accessing function arguments in the EFI mixed mode thunking code.

Instead let's map memory explicitly so that things will continue to work when we move to a separate page table in the future.

---
 arch/x86/platform/efi/efi_64.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
arch/x86/platform/efi/efi_64.c +20 -0
index 4fb0ba21d168..be1eeae668e7 100644
@@ -144,6 +144,7 @@ void efi_sync_low_kernel_mappings(void)
 int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 {
 	unsigned long pfn, text;
+	efi_memory_desc_t *md;
 	struct page *page;
 	unsigned npages;
 	pgd_t *pgd;
@@ -177,6 +178,25 @@ int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
 	if (!IS_ENABLED(CONFIG_EFI_MIXED))
 		return 0;
 
+	/*
+	 * Map all of RAM so that we can access arguments in the 1:1
+	 * mapping when making EFI runtime calls.
+	 */
+	for_each_efi_memory_desc(&memmap, md) {
+		if (md->type != EFI_CONVENTIONAL_MEMORY &&
+		    md->type != EFI_LOADER_DATA &&
+		    md->type != EFI_LOADER_CODE)
+			continue;
+
+		pfn = md->phys_addr >> PAGE_SHIFT;
+		npages = md->num_pages;
+
+		if (kernel_map_pages_in_pgd(pgd, pfn, md->phys_addr, npages, 0)) {
+			pr_err("Failed to map 1:1 memory\n");
+			return 1;
+		}
+	}
+
 	page = alloc_page(GFP_KERNEL|__GFP_DMA32);
 	if (!page)
 		panic("Unable to allocate EFI runtime stack < 4GB\n");
Signature · 1 line
2.6.2
#8 lore raw

PATCH v2 4/6 x86/efi: Hoist page table switching code into efi_call_virt()

In reply to Matt Fleming

This change is a prerequisite for pending patches that switch to a dedicated EFI page table, instead of using 'trampoline_pgd' which shares PGD entries with 'swapper_pg_dir'. The pending patches make it impossible to dereference the runtime service function pointer without first switching %cr3.

It's true that we now have duplicated switching code in efi_call_virt() and efi_call_phys_{prolog,epilog}() but we are sacrificing code duplication for a little more clarity and the ease of writing the page table switching code in C instead of asm.

Cc Sai Praneeth Prakhya <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Linus Torvalds <torvalds-de/[email protected]>
Cc Dave Jones <davej-rdkfGonbjUTCLXcRTR1eJlpr/1R2p/[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Andrew Morton <akpm-de/[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Denys Vlasenko <[email protected]>,
Cc Stephen Smalley <[email protected]>

---

Changes in v2:
- Fixed up some checkpatch warnings about mixing tabs and spaces.

 arch/x86/include/asm/efi.h          | 25 +++++++++++++++++++++
 arch/x86/platform/efi/efi_64.c      | 24 ++++++++++-----------
 arch/x86/platform/efi/efi_stub_64.S | 43 -------------------------------------
 3 files changed, 36 insertions(+), 56 deletions(-)
arch/x86/include/asm/efi.h +25 -0
index cfee9d4b02af..0eb45a187339 100644
@@ -3,6 +3,7 @@
 
 #include <asm/fpu/api.h>
 #include <asm/pgtable.h>
+#include <asm/tlb.h>
 
 /*
  * We map the EFI regions needed for runtime services non-contiguously,
@@ -64,6 +65,17 @@ extern u64 asmlinkage efi_call(void *fp, ...);
 
 #define efi_call_phys(f, args...)		efi_call((f), args)
 
+/*
+ * Scratch space used for switching the pagetable in the EFI stub
+ */
+struct efi_scratch {
+	u64 r15;
+	u64 prev_cr3;
+	pgd_t *efi_pgt;
+	bool use_pgd;
+	u64 phys_stack;
+} __packed;
+
 #define efi_call_virt(f, ...)						\
 ({									\
 	efi_status_t __s;						\
@@ -71,7 +83,20 @@ extern u64 asmlinkage efi_call(void *fp, ...);
 	efi_sync_low_kernel_mappings();					\
 	preempt_disable();						\
 	__kernel_fpu_begin();						\
+									\
+	if (efi_scratch.use_pgd) {					\
+		efi_scratch.prev_cr3 = read_cr3();			\
+		write_cr3((unsigned long)efi_scratch.efi_pgt);		\
+		__flush_tlb_all();					\
+	}								\
+									\
 	__s = efi_call((void *)efi.systab->runtime->f, __VA_ARGS__);	\
+									\
+	if (efi_scratch.use_pgd) {					\
+		write_cr3(efi_scratch.prev_cr3);			\
+		__flush_tlb_all();					\
+	}								\
+									\
 	__kernel_fpu_end();						\
 	preempt_enable();						\
 	__s;								\
arch/x86/platform/efi/efi_64.c +11 -13
index be1eeae668e7..530e8cdc1764 100644
@@ -47,16 +47,7 @@
  */
 static u64 efi_va = EFI_VA_START;
 
-/*
- * Scratch space used for switching the pagetable in the EFI stub
- */
-struct efi_scratch {
-	u64 r15;
-	u64 prev_cr3;
-	pgd_t *efi_pgt;
-	bool use_pgd;
-	u64 phys_stack;
-} __packed;
+struct efi_scratch efi_scratch;
 
 static void __init early_code_mapping_set_exec(int executable)
 {
@@ -83,8 +74,11 @@ pgd_t * __init efi_call_phys_prolog(void)
 	int pgd;
 	int n_pgds;
 
-	if (!efi_enabled(EFI_OLD_MEMMAP))
-		return NULL;
+	if (!efi_enabled(EFI_OLD_MEMMAP)) {
+		save_pgd = (pgd_t *)read_cr3();
+		write_cr3((unsigned long)efi_scratch.efi_pgt);
+		goto out;
+	}
 
 	early_code_mapping_set_exec(1);
 
@@ -96,6 +90,7 @@ pgd_t * __init efi_call_phys_prolog(void)
 		vaddress = (unsigned long)__va(pgd * PGDIR_SIZE);
 		set_pgd(pgd_offset_k(pgd * PGDIR_SIZE), *pgd_offset_k(vaddress));
 	}
+out:
 	__flush_tlb_all();
 
 	return save_pgd;
@@ -109,8 +104,11 @@ void __init efi_call_phys_epilog(pgd_t *save_pgd)
 	int pgd_idx;
 	int nr_pgds;
 
-	if (!save_pgd)
+	if (!efi_enabled(EFI_OLD_MEMMAP)) {
+		write_cr3((unsigned long)save_pgd);
+		__flush_tlb_all();
 		return;
+	}
 
 	nr_pgds = DIV_ROUND_UP((max_pfn << PAGE_SHIFT) , PGDIR_SIZE);
 
arch/x86/platform/efi/efi_stub_64.S +0 -43

57 lines not on this page · raw message

Signature · 1 line
2.6.2
#9 lore raw

PATCH v2 5/6 x86/efi: Build our own page table structures

In reply to Matt Fleming

With commit e1a58320a38d ("x86/mm: Warn on W^X mappings") all users booting on 64-bit UEFI machines see the following warning,

  ------------[ cut here ]------------
  WARNING: CPU: 7 PID: 1 at arch/x86/mm/dump_pagetables.c:225 note_page+0x5dc/0x780()
  x86/mm: Found insecure W+X mapping at address ffff88000005f000/0xffff88000005f000
  ...
  x86/mm: Checked W+X mappings: FAILED, 165660 W+X pages found.
  ...

This is caused by mapping EFI regions with RWX permissions. There isn't much we can do to restrict the permissions for these regions due to the way the firmware toolchains mix code and data, but we can at least isolate these mappings so that they do not appear in the regular kernel page tables.

In commit d2f7cbe7b26a ("x86/efi: Runtime services virtual mapping") we started using 'trampoline_pgd' to map the EFI regions because there was an existing identity mapping there which we use during the SetVirtualAddressMap() call and for broken firmware that accesses those addresses.

But 'trampoline_pgd' shares some PGD entries with 'swapper_pg_dir' and does not provide the isolation we require. Notably the virtual address for __START_KERNEL_map and MODULES_START are mapped by the same PGD entry so we need to be more careful when copying changes over in efi_sync_low_kernel_mappings().

This patch doesn't go the full mile, we still want to share some PGD entries with 'swapper_pg_dir'. Having completely separate page tables brings its own issues such as synchronising new mappings after memory hotplug and module loading. Sharing also keeps memory usage down.

Cc Sai Praneeth Prakhya <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Linus Torvalds <torvalds-de/[email protected]>
Cc Dave Jones <davej-rdkfGonbjUTCLXcRTR1eJlpr/1R2p/[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Andrew Morton <akpm-de/[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Denys Vlasenko <[email protected]>,
Cc Stephen Smalley <[email protected]>

---

Changes in v2:
- Move efi_alloc_page_tables() earlier in __efi_enter_virtual_mode() so we return early if we fail to allocate memory.

- More checkpatch warning fixes.

- Ident the BUILD_BUG_ON() asserts according to Boris' request

 arch/x86/include/asm/efi.h     |  1 +
 arch/x86/platform/efi/efi.c    | 39 ++++++-----------
 arch/x86/platform/efi/efi_32.c |  5 +++
 arch/x86/platform/efi/efi_64.c | 97 +++++++++++++++++++++++++++++++++++-------
 4 files changed, 102 insertions(+), 40 deletions(-)
arch/x86/include/asm/efi.h +1 -0
index 0eb45a187339..9be9f4963070 100644
@@ -124,6 +124,7 @@ extern void __init efi_memory_uc(u64 addr, unsigned long size);
 extern void __init efi_map_region(efi_memory_desc_t *md);
 extern void __init efi_map_region_fixed(efi_memory_desc_t *md);
 extern void efi_sync_low_kernel_mappings(void);
+extern int __init efi_alloc_page_tables(void);
 extern int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages);
 extern void __init efi_cleanup_page_tables(unsigned long pa_memmap, unsigned num_pages);
 extern void __init old_map_region(efi_memory_desc_t *md);
arch/x86/platform/efi/efi.c +14 -25
index c69e58fb7f19..e1f7c7f79ec5 100644
@@ -804,7 +804,7 @@ static void __init kexec_enter_virtual_mode(void)
  * This function will switch the EFI runtime services to virtual mode.
  * Essentially, we look through the EFI memmap and map every region that
  * has the runtime attribute bit set in its memory descriptor into the
- * ->trampoline_pgd page table using a top-down VA allocation scheme.
+ * efi_pgd page table.
  *
  * The old method which used to update that memory descriptor with the
  * virtual address obtained from ioremap() is still supported when the
@@ -814,8 +814,8 @@ static void __init kexec_enter_virtual_mode(void)
  *
  * The new method does a pagetable switch in a preemption-safe manner
  * so that we're in a different address space when calling a runtime
- * function. For function arguments passing we do copy the PGDs of the
- * kernel page table into ->trampoline_pgd prior to each call.
+ * function. For function arguments passing we do copy the PUDs of the
+ * kernel page table into efi_pgd prior to each call.
  *
  * Specially for kexec boot, efi runtime maps in previous kernel should
  * be passed in via setup_data. In that case runtime ranges will be mapped
@@ -830,6 +830,12 @@ static void __init __efi_enter_virtual_mode(void)
 
 	efi.systab = NULL;
 
+	if (efi_alloc_page_tables()) {
+		pr_err("Failed to allocate EFI page tables\n");
+		clear_bit(EFI_RUNTIME_SERVICES, &efi.flags);
+		return;
+	}
+
 	efi_merge_regions();
 	new_memmap = efi_map_regions(&count, &pg_shift);
 	if (!new_memmap) {
@@ -889,28 +895,11 @@ static void __init __efi_enter_virtual_mode(void)
 	efi_runtime_mkexec();
 
 	/*
-	 * We mapped the descriptor array into the EFI pagetable above but we're
-	 * not unmapping it here. Here's why:
-	 *
-	 * We're copying select PGDs from the kernel page table to the EFI page
-	 * table and when we do so and make changes to those PGDs like unmapping
-	 * stuff from them, those changes appear in the kernel page table and we
-	 * go boom.
-	 *
-	 * From setup_real_mode():
-	 *
-	 * ...
-	 * trampoline_pgd[0] = init_level4_pgt[pgd_index(__PAGE_OFFSET)].pgd;
-	 *
-	 * In this particular case, our allocation is in PGD 0 of the EFI page
-	 * table but we've copied that PGD from PGD[272] of the EFI page table:
-	 *
-	 *	pgd_index(__PAGE_OFFSET = 0xffff880000000000) = 272
-	 *
-	 * where the direct memory mapping in kernel space is.
-	 *
-	 * new_memmap's VA comes from that direct mapping and thus clearing it,
-	 * it would get cleared in the kernel page table too.
+	 * We mapped the descriptor array into the EFI pagetable above
+	 * but we're not unmapping it here because if we're running in
+	 * EFI mixed mode we need all of memory to be accessible when
+	 * we pass parameters to the EFI runtime services in the
+	 * thunking code.
 	 *
 	 * efi_cleanup_page_tables(__pa(new_memmap), 1 << pg_shift);
 	 */
arch/x86/platform/efi/efi_32.c +5 -0
index ed5b67338294..58d669bc8250 100644
@@ -38,6 +38,11 @@
  * say 0 - 3G.
  */
 
+int __init efi_alloc_page_tables(void)
+{
+	return 0;
+}
+
 void efi_sync_low_kernel_mappings(void) {}
 void __init efi_dump_pagetable(void) {}
 int __init efi_setup_page_tables(unsigned long pa_memmap, unsigned num_pages)
arch/x86/platform/efi/efi_64.c +82 -15

143 lines not on this page · raw message

Signature · 1 line
2.6.2
#10 lore raw

PATCH v3 6/6 Documentation/x86: Update EFI memory region description

In reply to Matt Fleming

Make it clear that the EFI page tables are only available during EFI runtime calls since that subject has come up a fair numbers of times in the past.

Additionally, add the EFI region start and end addresses to the table so that it's possible to see at a glance where they fall in relation to other regions.

Cc Sai Praneeth Prakhya <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Linus Torvalds <[email protected]>
Cc Dave Jones <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc H. Peter Anvin <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Denys Vlasenko <[email protected]>,
Cc Stephen Smalley <[email protected]>

---

Changes in v3:
- Make it clear that we use a 64GB chunk of virtual address space, not the first 64Gb addresses.

 Documentation/x86/x86_64/mm.txt | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
Documentation/x86/x86_64/mm.txt +6 -6
index 05712ac83e38..c518dce7da4d 100644
@@ -16,6 +16,8 @@ ffffec0000000000 - fffffc0000000000 (=44 bits) kasan shadow memory (16TB)
 ... unused hole ...
 ffffff0000000000 - ffffff7fffffffff (=39 bits) %esp fixup stacks
 ... unused hole ...
+ffffffef00000000 - ffffffff00000000 (=64 GB) EFI region mapping space
+... unused hole ...
 ffffffff80000000 - ffffffffa0000000 (=512 MB)  kernel text mapping, from phys 0
 ffffffffa0000000 - ffffffffff5fffff (=1525 MB) module mapping space
 ffffffffff600000 - ffffffffffdfffff (=8 MB) vsyscalls
@@ -32,11 +34,9 @@ reference.
 Current X86-64 implementations only support 40 bits of address space,
 but we support up to 46 bits. This expands into MBZ space in the page tables.
 
-->trampoline_pgd:
-
-We map EFI runtime services in the aforementioned PGD in the virtual
-range of 64Gb (arbitrarily set, can be raised if needed)
-
-0xffffffef00000000 - 0xffffffff00000000
+We map EFI runtime services in the 'efi_pgd' PGD in a 64Gb large virtual
+memory window (this size is arbitrary, it can be raised later if needed).
+The mappings are not part of any other kernel PGD and are only available
+during EFI runtime calls.
 
 -Andi Kleen, Jul 2004
Signature · 1 line
2.6.2