EFI runtime services virtual mapping, vN++
Subsystems (4)
Borislav Petkov
maintainercover EFI runtime services virtual mapping, vN++
From: Borislav Petkov <[email protected]>
Hi all,
here's maybe the final version of the patchset, no major changes since the last time but cosmetic and cleanups and tidying as requested by Matt and others.
Patches also at:
git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git#efi
The previous announcement:
here's finally a new version of the runtime services VA mapping patchset which hopefully implements hpa's idea of statically mapping EFI runtime regions in a top-down manner starting at -4Gb virtual.
We're also using a different pagetable so as not to pollute kernel address space. For that, we switch to that table before doing an EFI call, and afterwards we switch back to the previous one.
To the patches:
1-2 are simple cleanups which Matt probably can take now
3-10 add the machinery to map regions into an arbitrary PGD. Those I've split deliberately into very small bites so that they can be reviewed more thoroughly and easily for my pagetable skills are pretty basic.
11 is the actual patch which implements that mapping so that we can use runtime services in kexec (which is the whole reason for this fuss :))
So please take a long hard look at those, hammer on them on your boxes and let me know. They boot fine on my Dell UEFI box and in OVMF (obviously :)).
Borislav Petkov (11):
efi: Simplify EFI_DEBUG
x86, pageattr: Lookup address in an arbitrary PGD x86, pageattr: Add a PGD pagetable populating function x86, pageattr: Add a PUD pagetable populating function x86, pageattr: Add a PMD pagetable populating function x86, pageattr: Add a PTE pagetable populating function x86, pageattr: Add a PUD error unwinding path x86, pageattr: Add last levels of error path x86, cpa: Map in an arbitrary pgd
EFI: Runtime services virtual mapping
efi: Check krealloc return value
Documentation/kernel-parameters.txt | 6 + Documentation/x86/x86_64/mm.txt | 7 + arch/x86/include/asm/efi.h | 64 +++-- arch/x86/include/asm/pgtable_types.h | 3 +- arch/x86/mm/pageattr.c | 461 +++++++++++++++++++++++++++++++++-- arch/x86/platform/efi/efi.c | 111 ++++++--- arch/x86/platform/efi/efi_32.c | 9 +- arch/x86/platform/efi/efi_64.c | 109 +++++++++ arch/x86/platform/efi/efi_stub_64.S | 54 ++++ include/linux/efi.h | 1 + 10 files changed, 755 insertions(+), 70 deletions(-)
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 1/11 efi: Simplify EFI_DEBUG
From: Borislav Petkov <[email protected]>
... and lose one #ifdef .. #endif sandwich.
--- arch/x86/platform/efi/efi.c | 9 +++------ 1 file changed, 3 insertions(+), 6 deletions(-)
arch/x86/platform/efi/efi.c +3 -6
@@ -51,7 +51,7 @@#include <asm/x86_init.h> #include <asm/rtc.h> -#define EFI_DEBUG 1 +#define EFI_DEBUG #define EFI_MIN_RESERVE 5120
@@ -398,9 +398,9 @@ int __init efi_memblock_x86_reserve_range(void)return 0 ; } -#if EFI_DEBUG static void __init print_efi_memmap (void ) { +#ifdef EFI_DEBUG efi_memory_desc_t *md;void *p;int i;
@@ -415,8 +415,8 @@ static void __init print_efi_memmap(void) md->phys_addr + (md->num_pages <<EFI_PAGE_SHIFT ), (md->num_pages >> (20 - EFI_PAGE_SHIFT))); } -}#endif /* EFI_DEBUG */ +}void __init efi_reserve_boot_services (void ) {
@@ -696,10 +696,7 @@ void __init efi_init(void) x86_platform.set_wallclock = efi_set_rtc_mmss; }#endif - -#if EFI_DEBUG print_efi_memmap (); -#endif }void __init efi_late_init (void )
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 2/11 x86, pageattr: Lookup address in an arbitrary PGD
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
This is preparatory work in order to be able to map pages into a specified PGD and not implicitly and only into init_mm.
--- arch/x86/mm/pageattr.c | 36 ++++++++++++++++++++++++++---------- 1 file changed, 26 insertions(+), 10 deletions(-)
arch/x86/mm/pageattr.c +26 -10
@@ -30,6 +30,7 @@*/ struct cpa_data {unsigned long *vaddr; +pgd_t *pgd;pgprot_t mask_set;pgprot_t mask_clr;int numpages;
@@ -322,17 +323,9 @@ static inline pgprot_t static_protections(pgprot_t prot, unsigned long address,return prot; } -/* -* Lookup the page table entry for a virtual address. Return a pointer -* to the entry and the level of the mapping. -* -* Note: We return pud and pmd either when the entry is marked large -* or when the present bit is not set. Otherwise we would return a -* pointer to a nonexisting mapping. -*/ -pte_t *lookup_address (unsigned long address,unsigned int *level) +static pte_t *__lookup_address_in_pgd (pgd_t *pgd,unsigned long address, +unsigned int *level) { -pgd_t *pgd =pgd_offset_k (address);pud_t *pud;pmd_t *pmd;
@@ -361,8 +354,31 @@ pte_t *lookup_address(unsigned long address, unsigned int *level)return pte_offset_kernel (pmd, address); } + +/* +* Lookup the page table entry for a virtual address. Return a pointer +* to the entry and the level of the mapping. +* +* Note: We return pud and pmd either when the entry is marked large +* or when the present bit is not set. Otherwise we would return a +* pointer to a nonexisting mapping. +*/ +pte_t *lookup_address (unsigned long address,unsigned int *level) +{ +return __lookup_address_in_pgd (pgd_offset_k (address), address, level); +}EXPORT_SYMBOL_GPL (lookup_address); +static pte_t *_lookup_address_cpa (struct cpa_data *cpa,unsigned long address, +unsigned int *level) +{ +if (cpa->pgd) +return __lookup_address_in_pgd (cpa->pgd +pgd_index (address), + address, level); + +return lookup_address (address, level); +} +/* * This is necessary because __pa() does not work on some * kinds of memory, like vmalloc() or the alloc_remap()
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 3/11 x86, pageattr: Add a PGD pagetable populating function
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
This allocates, if necessary, and populates the corresponding PGD entry with a PUD page. The next population level is a dummy macro which will be removed by the next patch and it is added here to keep the patch small and easily reviewable but not break bisection, at the same time.
--- arch/x86/mm/pageattr.c | 39 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+)
arch/x86/mm/pageattr.c +39 -0
@@ -666,6 +666,45 @@ static int split_large_page(pte_t *kpte, unsigned long address)return 0 ; } +#define populate_pud (cpa, addr, pgd, pgprot) (-1) + +/* +* Restrictions for kernel page table do not necessarily apply when mapping in +* an alternate PGD. +*/ +static int populate_pgd (struct cpa_data *cpa,unsigned long addr) +{ +pgprot_t pgprot =__pgprot (_KERNPG_TABLE); +bool allocd_pgd = false; +pgd_t *pgd_entry; +pud_t *pud =NULL ;/* shut up gcc */ +int ret; + + pgd_entry = cpa->pgd +pgd_index (addr); + +/* +* Allocate a PUD page and hand it down for mapping. +*/ +if (pgd_none (*pgd_entry)) { + pud = (pud_t *)get_zeroed_page (GFP_KERNEL | __GFP_NOTRACK); +if (!pud) +return -1 ; + +set_pgd (pgd_entry,__pgd (__pa (pud) | _KERNPG_TABLE)); + allocd_pgd = true; + } + +pgprot_val (pgprot) &= ~pgprot_val (cpa->mask_clr); +pgprot_val (pgprot) |=pgprot_val (cpa->mask_set); + + ret =populate_pud (cpa, addr, pgd_entry, pgprot); +if (ret <0 ) +return ret; + + cpa->numpages = ret; +return 0 ; +} +static int __cpa_process_fault (struct cpa_data *cpa,unsigned long vaddr,int primary) {
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 4/11 x86, pageattr: Add a PUD pagetable populating function
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Add the next level of the pagetable populating function, we handle chunks around a 1G boundary by mapping them with the lower level functions - otherwise we use 1G pages for the mappings, thus using as less amount of pagetable pages as possible.
--- arch/x86/mm/pageattr.c | 87 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 86 insertions(+), 1 deletion(-)
arch/x86/mm/pageattr.c +86 -1
@@ -666,7 +666,92 @@ static int split_large_page(pte_t *kpte, unsigned long address)return 0 ; } -#define populate_pud (cpa, addr, pgd, pgprot) (-1) +static int alloc_pmd_page (pud_t *pud) +{ +pmd_t *pmd = (pmd_t *)get_zeroed_page (GFP_KERNEL | __GFP_NOTRACK); +if (!pmd) +return -1 ; + +set_pud (pud,__pud (__pa (pmd) | _KERNPG_TABLE)); +return 0 ; +} + +#define populate_pmd (cpa, start, end, pages, pud, pgprot) (-1) + +static int populate_pud (struct cpa_data *cpa,unsigned long start,pgd_t *pgd, +pgprot_t pgprot) +{ +pud_t *pud; +unsigned long end; +int cur_pages =0 ; + + end = start + (cpa->numpages <<PAGE_SHIFT ); + +/* +* Not on a Gb page boundary? => map everything up to it with +* smaller pages. +*/ +if (start & (PUD_SIZE -1 )) { +unsigned long pre_end; +unsigned long next_page = (start +PUD_SIZE ) &PUD_MASK ; + + pre_end = min_t(unsigned long, end, next_page); + cur_pages = (pre_end - start) >>PAGE_SHIFT ; + cur_pages =min_t (int, (int )cpa->numpages, cur_pages); + + pud =pud_offset (pgd, start); + +/* +* Need a PMD page? +*/ +if (pud_none (*pud)) +if (alloc_pmd_page (pud)) +return -1 ; + + cur_pages =populate_pmd (cpa, start, pre_end, cur_pages, + pud, pgprot); +if (cur_pages <0 ) +return cur_pages; + + start = pre_end; + } + +/* We mapped them all? */ +if (cpa->numpages == cur_pages) +return cur_pages; + + pud =pud_offset (pgd, start); + +/* +* Map everything starting from the Gb boundary, possibly with 1G pages +*/ +while (end - start >=PUD_SIZE ) { +set_pud (pud,__pud (cpa->pfn | _PAGE_PSE |massage_pgprot (pgprot))); + + start +=PUD_SIZE ; + cpa->pfn +=PUD_SIZE ; + cur_pages +=PUD_SIZE >>PAGE_SHIFT ; + pud++; + } + +/* Map trailing leftover */ +if (start < end) { +int tmp; + + pud =pud_offset (pgd, start); +if (pud_none (*pud)) +if (alloc_pmd_page (pud)) +return -1 ; + + tmp =populate_pmd (cpa, start, end, cpa->numpages - cur_pages, + pud, pgprot); +if (tmp <0 ) +return cur_pages; + + cur_pages += tmp; + } +return cur_pages; +}/* * Restrictions for kernel page table do not necessarily apply when mapping in
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 5/11 x86, pageattr: Add a PMD pagetable populating function
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Handle PMD-level mappings the same as PUD ones.
--- arch/x86/mm/pageattr.c | 82 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 81 insertions(+), 1 deletion(-)
arch/x86/mm/pageattr.c +81 -1
@@ -666,6 +666,16 @@ static int split_large_page(pte_t *kpte, unsigned long address)return 0 ; } +static int alloc_pte_page (pmd_t *pmd) +{ +pte_t *pte = (pte_t *)get_zeroed_page (GFP_KERNEL | __GFP_NOTRACK); +if (!pte) +return -1 ; + +set_pmd (pmd,__pmd (__pa (pte) | _KERNPG_TABLE)); +return 0 ; +} +static int alloc_pmd_page (pud_t *pud) {pmd_t *pmd = (pmd_t *)get_zeroed_page (GFP_KERNEL | __GFP_NOTRACK);
@@ -676,7 +686,77 @@ static int alloc_pmd_page(pud_t *pud)return 0 ; } -#define populate_pmd (cpa, start, end, pages, pud, pgprot) (-1) +#define populate_pte (cpa, start, end, pages, pmd, pgprot) do {} while (0) + +static int populate_pmd (struct cpa_data *cpa, +unsigned long start,unsigned long end, +unsigned num_pages,pud_t *pud,pgprot_t pgprot) +{ +unsigned int cur_pages =0 ; +pmd_t *pmd; + +/* +* Not on a 2M boundary? +*/ +if (start & (PMD_SIZE -1 )) { +unsigned long pre_end = start + (num_pages <<PAGE_SHIFT ); +unsigned long next_page = (start +PMD_SIZE ) &PMD_MASK ; + + pre_end = min_t(unsigned long, pre_end, next_page); + cur_pages = (pre_end - start) >>PAGE_SHIFT ; + cur_pages = min_t(unsigned int, num_pages, cur_pages); + +/* +* Need a PTE page? +*/ + pmd =pmd_offset (pud, start); +if (pmd_none (*pmd)) +if (alloc_pte_page (pmd)) +return -1 ; + +populate_pte (cpa, start, pre_end, cur_pages, pmd, pgprot); + + start = pre_end; + } + +/* +* We mapped them all? +*/ +if (num_pages == cur_pages) +return cur_pages; + +while (end - start >=PMD_SIZE ) { + +/* +* We cannot use a 1G page so allocate a PMD page if needed. +*/ +if (pud_none (*pud)) +if (alloc_pmd_page (pud)) +return -1 ; + + pmd =pmd_offset (pud, start); + +set_pmd (pmd,__pmd (cpa->pfn | _PAGE_PSE |massage_pgprot (pgprot))); + + start +=PMD_SIZE ; + cpa->pfn +=PMD_SIZE ; + cur_pages +=PMD_SIZE >>PAGE_SHIFT ; + } + +/* +* Map trailing 4K pages. +*/ +if (start < end) { + pmd =pmd_offset (pud, start); +if (pmd_none (*pmd)) +if (alloc_pte_page (pmd)) +return -1 ; + +populate_pte (cpa, start, end, num_pages - cur_pages, + pmd, pgprot); + } +return num_pages; +}static int populate_pud (struct cpa_data *cpa,unsigned long start,pgd_t *pgd,pgprot_t pgprot)
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 6/11 x86, pageattr: Add a PTE pagetable populating function
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Handle last level by unconditionally writing the PTEs into the PTE page while paying attention to the NX bit.
--- arch/x86/mm/pageattr.c | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-)
arch/x86/mm/pageattr.c +21 -1
@@ -686,7 +686,27 @@ static int alloc_pmd_page(pud_t *pud)return 0 ; } -#define populate_pte (cpa, start, end, pages, pmd, pgprot) do {} while (0) +static void populate_pte (struct cpa_data *cpa, +unsigned long start,unsigned long end, +unsigned num_pages,pmd_t *pmd,pgprot_t pgprot) +{ +pte_t *pte; + + pte =pte_offset_kernel (pmd, start); + +while (num_pages-- && start < end) { + +/* deal with the NX bit */ +if (!(pgprot_val (pgprot) & _PAGE_NX)) + cpa->pfn &= ~_PAGE_NX; + +set_pte (pte,pfn_pte (cpa->pfn >>PAGE_SHIFT , pgprot)); + + start +=PAGE_SIZE ; + cpa->pfn +=PAGE_SIZE ; + pte++; + } +} static int populate_pmd(struct cpa_data *cpa, unsigned long start, unsigned long end,
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 7/11 x86, pageattr: Add a PUD error unwinding path
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
In case we encounter an error during the mapping of a region, we want to unwind what we've established so far exactly the way we did the mapping. This is the PUD part kept deliberately small for easier review.
--- arch/x86/mm/pageattr.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 58 insertions(+), 2 deletions(-)
arch/x86/mm/pageattr.c +58 -2
@@ -666,6 +666,51 @@ static int split_large_page(pte_t *kpte, unsigned long address)return 0 ; } +#define unmap_pmd_range (pud, start, pre_end) do {} while (0) + +static void unmap_pud_range (pgd_t *pgd,unsigned long start,unsigned long end) +{ +pud_t *pud =pud_offset (pgd, start); + +/* +* Not on a GB page boundary? +*/ +if (start & (PUD_SIZE -1 )) { +unsigned long next_page = (start +PUD_SIZE ) &PUD_MASK ; + unsigned long pre_end = min_t(unsigned long, end, next_page); + +unmap_pmd_range (pud, start, pre_end); + + start = pre_end; + pud++; + } + +/* +* Try to unmap in 1G chunks? +*/ +while (end - start >=PUD_SIZE ) { + +if (pud_large (*pud)) +pud_clear (pud); +else +unmap_pmd_range (pud, start, start +PUD_SIZE ); + + start +=PUD_SIZE ; + pud++; + } + +/* +* 2M leftovers? +*/ +if (start < end) +unmap_pmd_range (pud, start, end); + +/* +* No need to try to free the PUD page because we'll free it in +* populate_pgd's error path +*/ +} +static int alloc_pte_page (pmd_t *pmd) {pte_t *pte = (pte_t *)get_zeroed_page (GFP_KERNEL | __GFP_NOTRACK);
@@ -883,9 +928,20 @@ static int populate_pgd(struct cpa_data *cpa, unsigned long addr)pgprot_val (pgprot) |=pgprot_val (cpa->mask_set); ret =populate_pud (cpa, addr, pgd_entry, pgprot); -if (ret <0 ) -return ret; +if (ret <0 ) { +unmap_pud_range (pgd_entry, addr, + addr + (cpa->numpages <<PAGE_SHIFT )); +if (allocd_pgd) { +/* +* If I allocated this PUD page, I can just as well +* free it in this error path. +*/ +pgd_clear (pgd_entry); +free_page ((unsigned long )pud); + } +return ret; + } cpa->numpages = ret;return 0 ; }
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 8/11 x86, pageattr: Add last levels of error path
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
We try to free the pagetable pages once we've unmapped our portion.
--- arch/x86/mm/pageattr.c | 94 +++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 93 insertions(+), 1 deletion(-)
arch/x86/mm/pageattr.c +93 -1
@@ -666,7 +666,99 @@ static int split_large_page(pte_t *kpte, unsigned long address)return 0 ; } -#define unmap_pmd_range (pud, start, pre_end) do {} while (0) +static bool try_to_free_pte_page (pte_t *pte) +{ +int i; + +for (i =0 ; i <PTRS_PER_PTE ; i++) +if (!pte_none (pte[i])) +return false; + +free_page ((unsigned long )pte); +return true; +} + +static bool try_to_free_pmd_page (pmd_t *pmd) +{ +int i; + +for (i =0 ; i <PTRS_PER_PMD ; i++) +if (!pmd_none (pmd[i])) +return false; + +free_page ((unsigned long )pmd); +return true; +} + +static bool unmap_pte_range (pmd_t *pmd,unsigned long start,unsigned long end) +{ +pte_t *pte =pte_offset_kernel (pmd, start); + +while (start < end) { +set_pte (pte,__pte (0 )); + + start +=PAGE_SIZE ; + pte++; + } + +if (try_to_free_pte_page ((pte_t *)pmd_page_vaddr (*pmd))) { +pmd_clear (pmd); +return true; + } +return false; +} + +static void __unmap_pmd_range (pud_t *pud,pmd_t *pmd, +unsigned long start,unsigned long end) +{ +if (unmap_pte_range (pmd, start, end)) +if (try_to_free_pmd_page ((pmd_t *)pud_page_vaddr (*pud))) +pud_clear (pud); +} + +static void unmap_pmd_range (pud_t *pud,unsigned long start,unsigned long end) +{ +pmd_t *pmd =pmd_offset (pud, start); + +/* +* Not on a 2MB page boundary? +*/ +if (start & (PMD_SIZE -1 )) { +unsigned long next_page = (start +PMD_SIZE ) &PMD_MASK ; + unsigned long pre_end = min_t(unsigned long, end, next_page); + +__unmap_pmd_range (pud, pmd, start, pre_end); + + start = pre_end; + pmd++; + } + +/* +* Try to unmap in 2M chunks. +*/ +while (end - start >=PMD_SIZE ) { +if (pmd_large (*pmd)) +pmd_clear (pmd); +else +__unmap_pmd_range (pud, pmd, start, start +PMD_SIZE ); + + start +=PMD_SIZE ; + pmd++; + } + +/* +* 4K leftovers? +*/ +if (start < end) +return __unmap_pmd_range (pud, pmd, start, end); + +/* +* Try again to free the PMD page if haven't succeeded above. +*/ +if (!pud_none (*pud)) +if (try_to_free_pmd_page ((pmd_t *)pud_page_vaddr (*pud))) +pud_clear (pud); +}static void unmap_pud_range (pgd_t *pgd,unsigned long start,unsigned long end) {
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 9/11 x86, cpa: Map in an arbitrary pgd
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Add the ability to map pages in an arbitrary pgd. This wires in the remaining stuff so that there's a new interface with which you can map a region into an arbitrary PGD.
--- arch/x86/mm/pageattr.c | 53 +++++++++++++++++++++++++++++++++++++++++++------- 1 file changed, 46 insertions(+), 7 deletions(-)
arch/x86/mm/pageattr.c +46 -7
@@ -453,7 +453,7 @@ try_preserve_large_page(pte_t *kpte, unsigned long address,* Check for races, another CPU might have split this page * up already: */ - tmp =lookup_address (address, &level); + tmp =_lookup_address_cpa (cpa, address, &level);if (tmp != kpte) goto out_unlock;
@@ -559,7 +559,8 @@ out_unlock: }static int -__split_large_page (pte_t *kpte,unsigned long address,struct page *base) +__split_large_page (struct cpa_data *cpa,pte_t *kpte,unsigned long address, +struct page *base) {pte_t *pbase = (pte_t *)page_address (base);unsigned long pfn, pfninc =1 ;
@@ -572,7 +573,7 @@ __split_large_page(pte_t *kpte, unsigned long address, struct page *base)* Check for races, another CPU might have split this page * up for us already: */ - tmp =lookup_address (address, &level); + tmp =_lookup_address_cpa (cpa, address, &level);if (tmp != kpte) {spin_unlock (&pgd_lock);return 1 ;
@@ -648,7 +649,8 @@ __split_large_page(pte_t *kpte, unsigned long address, struct page *base)return 0 ; } -static int split_large_page (pte_t *kpte,unsigned long address) +static int split_large_page (struct cpa_data *cpa,pte_t *kpte, +unsigned long address) {struct page *base;
@@ -660,7 +662,7 @@ static int split_large_page(pte_t *kpte, unsigned long address)if (!base)return -ENOMEM ; -if (__split_large_page (kpte, address, base)) +if (__split_large_page (cpa, kpte, address, base))__free_page (base);return 0 ;
@@ -1041,6 +1043,9 @@ static int populate_pgd(struct cpa_data *cpa, unsigned long addr)static int __cpa_process_fault (struct cpa_data *cpa,unsigned long vaddr,int primary) { +if (cpa->pgd) +return populate_pgd (cpa, vaddr); +/* * Ignore all non primary paths. */
@@ -1085,7 +1090,7 @@ static int __change_page_attr(struct cpa_data *cpa, int primary)else address = *cpa->vaddr; repeat: - kpte =lookup_address (address, &level); + kpte =_lookup_address_cpa (cpa, address, &level);if (!kpte)return __cpa_process_fault (cpa, address, primary);
@@ -1149,7 +1154,7 @@ repeat:/* * We have to split the large page: */ - err =split_large_page (kpte, address); + err =split_large_page (cpa, kpte, address);if (!err) {/* * Do a global flush tlb after splitting the large page
@@ -1298,6 +1303,8 @@ static int change_page_attr_set_clr(unsigned long *addr, int numpages,int ret, cache, checkalias;unsigned long baddr =0 ; +memset (&cpa,0 ,sizeof (cpa)); +/* * Check, if we are requested to change a not supported * feature:
@@ -1744,6 +1751,7 @@ static int __set_pages_p(struct page *page, int numpages) {unsigned long tempaddr = (unsigned long )page_address (page);struct cpa_data cpa = { .vaddr = &tempaddr, + .pgd =NULL , .numpages = numpages, .mask_set =__pgprot (_PAGE_PRESENT | _PAGE_RW), .mask_clr =__pgprot (0 ),
@@ -1762,6 +1770,7 @@ static int __set_pages_np(struct page *page, int numpages) {unsigned long tempaddr = (unsigned long )page_address (page);struct cpa_data cpa = { .vaddr = &tempaddr, + .pgd =NULL , .numpages = numpages, .mask_set =__pgprot (0 ), .mask_clr =__pgprot (_PAGE_PRESENT | _PAGE_RW),
@@ -1822,6 +1831,36 @@ bool kernel_page_present(struct page *page)#endif /* CONFIG_DEBUG_PAGEALLOC */ +int kernel_map_pages_in_pgd (pgd_t *pgd,u64 pfn,unsigned long address, +unsigned numpages,unsigned long page_flags) +{ +int retval = -EINVAL ; + +struct cpa_data cpa = { + .vaddr = &address, + .pfn = pfn, + .pgd = pgd, + .numpages = numpages, + .mask_set =__pgprot (0 ), + .mask_clr =__pgprot (0 ), + .flags =0 , + }; + +if (!(__supported_pte_mask & _PAGE_NX)) + goto out; + +if (!(page_flags & _PAGE_NX)) + cpa.mask_clr =__pgprot (_PAGE_NX); + + cpa.mask_set =__pgprot (_PAGE_PRESENT | page_flags); + + retval =__change_page_attr_set_clr (&cpa,0 ); +__flush_tlb_all (); + +out: +return retval; +} +/* * The testcases use internal knowledge of the implementation that shouldn't * be exposed to the rest of the kernel. Include these directly here.
Signature · 1 line
1.8.4
Borislav Petkov
maintainerPATCH 10/11 EFI: Runtime services virtual mapping
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
We map the EFI regions needed for runtime services non-contiguously, with preserved alignment on virtual addresses starting from -4G down for a total max space of 64G. This way, we provide for stable runtime services addresses across kernels so that a kexec'd kernel can still use them.
Thus, they're mapped in a separate pagetable so that we don't pollute the kernel namespace.
Add an efi= kernel command line parameter for passing miscellaneous options and chicken bits from the command line.
While at it, add a chicken bit called "efi=old_map" which can be used as a fallback to the old runtime services mapping method in case there's some b0rkage with a particular EFI implementation (haha, it is hard to hold up the sarcasm here...).
Also, add the UEFI RT VA space to Documentation/x86/x86_64/mm.txt.
--- Documentation/kernel-parameters.txt | 6 ++ Documentation/x86/x86_64/mm.txt | 7 +++ arch/x86/include/asm/efi.h | 64 ++++++++++++++------ arch/x86/include/asm/pgtable_types.h | 3 +- arch/x86/platform/efi/efi.c | 94 +++++++++++++++++++++--------- arch/x86/platform/efi/efi_32.c | 9 ++- arch/x86/platform/efi/efi_64.c | 109 +++++++++++++++++++++++++++++++++++ arch/x86/platform/efi/efi_stub_64.S | 54 +++++++++++++++++ include/linux/efi.h | 1 + 9 files changed, 300 insertions(+), 47 deletions(-)
Documentation/kernel-parameters.txt +6 -0
@@ -833,6 +833,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. edd= [EDD] Format: {"off" | "on" | "skip[mbr]"} + efi= [EFI] + Format: { "old_map" } + old_map [X86-64]: switch to the old ioremap-based EFI + runtime services mapping. 32-bit still uses this one by + default. + efi_no_storage_paranoia [EFI; X86] Using this parameter you can use more than 50% of your efi variable storage. Use this parameter only if
Documentation/x86/x86_64/mm.txt +7 -0
@@ -28,4 +28,11 @@ reference. Current X86-64 implementations only support 40 bits of address space, but we support up to 46 bits. This expands into MBZ space in the page tables. +->trampoline_pgd: + +We map EFI runtime services in the aforementioned PGD in the virtual +range of 64Gb (arbitrarily set, can be raised if needed) + +0xffffffef00000000 - 0xffffffff00000000 + -Andi Kleen, Jul 2004
arch/x86/include/asm/efi.h +47 -17
89 lines not on this page · raw message
arch/x86/include/asm/pgtable_types.h +2 -1
@@ -379,7 +379,8 @@ static inline void update_page_count(int level, unsigned long pages) { }*/ extern pte_t *lookup_address (unsigned long address,unsigned int *level);extern phys_addr_t slow_virt_to_phys (void *__address); - +extern int kernel_map_pages_in_pgd (pgd_t *pgd,u64 pfn,unsigned long address, +unsigned numpages,unsigned long page_flags);#endif /* !__ASSEMBLY__ */ #endif /* _ASM_X86_PGTABLE_DEFS_H */
arch/x86/platform/efi/efi.c +66 -28
155 lines not on this page · raw message
arch/x86/platform/efi/efi_32.c +8 -1
@@ -37,9 +37,16 @@* claim EFI runtime service handler exclusively and to duplicate a memory in * low memory space say 0 - 3G. */ -static unsigned long efi_rt_eflags; +void efi_sync_low_kernel_mappings (void ) {} +void efi_setup_page_tables (void ) {} + +void __init efi_map_region (efi_memory_desc_t *md) +{ +old_map_region (md); +} +void efi_call_phys_prelog (void ) {struct desc_ptr gdt_descr;
arch/x86/platform/efi/efi_64.c +109 -0
137 lines not on this page · raw message
arch/x86/platform/efi/efi_stub_64.S +54 -0
107 lines not on this page · raw message
include/linux/efi.h +1 -0
@@ -653,6 +653,7 @@ extern int __init efi_setup_pcdp_console(char *);#define EFI_RUNTIME_SERVICES 3/* Can we use runtime services? */ #define EFI_MEMMAP 4/* Can we use EFI memory map? */ #define EFI_64BIT 5/* Is the firmware 64-bit? */ +#define EFI_ARCH_1 6/* First arch-specific bit */ #ifdef CONFIG_EFI # ifdef CONFIG_X86
Signature · 1 line
1.8.4
Re: [PATCH 10/11] EFI: Runtime services virtual mapping
Quoted 22 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
We map the EFI regions needed for runtime services non-contiguously, with preserved alignment on virtual addresses starting from -4G down for a total max space of 64G. This way, we provide for stable runtime services addresses across kernels so that a kexec'd kernel can still use them.
Thus, they're mapped in a separate pagetable so that we don't pollute the kernel namespace.
Add an efi= kernel command line parameter for passing miscellaneous options and chicken bits from the command line.
While at it, add a chicken bit called "efi=old_map" which can be used as a fallback to the old runtime services mapping method in case there's some b0rkage with a particular EFI implementation (haha, it is hard to hold up the sarcasm here...).
Also, add the UEFI RT VA space to
Documentation/x86/x86_64/mm.txt.Signed-off-by Borislav Petkov <[email protected]>
Quoted 12 lines
--- Documentation/kernel-parameters.txt | 6 ++ Documentation/x86/x86_64/mm.txt | 7 +++ arch/x86/include/asm/efi.h | 64 ++++++++++++++------ arch/x86/include/asm/pgtable_types.h | 3 +- arch/x86/platform/efi/efi.c | 94 +++++++++++++++++++++--------- arch/x86/platform/efi/efi_32.c | 9 ++- arch/x86/platform/efi/efi_64.c | 109 +++++++++++++++++++++++++++++++++++ arch/x86/platform/efi/efi_stub_64.S | 54 +++++++++++++++++ include/linux/efi.h | 1 + 9 files changed, 300 insertions(+), 47 deletions(-)
Hi,
This one still does not apply:
[dave@darkstar linux-2.6]$ git am ~/Mail/bp/new/*
Applying: efi: Simplify EFI_DEBUG
Applying: x86, pageattr: Lookup address in an arbitrary PGD
Applying: x86, pageattr: Add a PGD pagetable populating function
Applying: x86, pageattr: Add a PUD pagetable populating function
Applying: x86, pageattr: Add a PMD pagetable populating function
Applying: x86, pageattr: Add a PTE pagetable populating function
Applying: x86, pageattr: Add a PUD error unwinding path
Applying: x86, pageattr: Add last levels of error path
Applying: x86, cpa: Map in an arbitrary pgd
Applying: EFI: Runtime services virtual mapping
error: patch failed: arch/x86/platform/efi/efi.c:808
error: arch/x86/platform/efi/efi.c: patch does not apply Patch failed at 0010 EFI: Runtime services virtual mapping The copy of the patch that failed is found in:
/home/dave/git/linux-2.6/.git/rebase-apply/patch
When you have resolved this problem, run "git am --resolved". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".
I found the reason, here is the diff of the efi.c between (linus tree + matt next) and the one in matt next:
efi.c.matt-next renamed from efi.c.linus-tree +4 -7
@@ -806,13 +806,10 @@ void __init efi_enter_virtual_mode(void) for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) { md = p; - if (!(md->attribute & EFI_MEMORY_RUNTIME)) { -#ifdef CONFIG_X86_64 - if (md->type != EFI_BOOT_SERVICES_CODE && - md->type != EFI_BOOT_SERVICES_DATA) -#endif - continue; - } + if (!(md->attribute & EFI_MEMORY_RUNTIME) && + md->type != EFI_BOOT_SERVICES_CODE && + md->type != EFI_BOOT_SERVICES_DATA) + continue; size = md->num_pages << EFI_PAGE_SHIFT; end = md->phys_addr + size;
The conflict is caused by missing one commit in matt's efi next tree:
commit 700870119f49084da004ab588ea2b799689efaf7
Author: Josh Boyer <[email protected]> Date: Thu Apr 18 07:51:34 2013 -0700
x86, efi: Don't map Boot Services on i386
Add patch to fix 32bit EFI service mapping (rhbz 726701)
Multiple people are reporting hitting the following WARNING on i386,
WARNING: at arch/x86/mm/ioremap.c:102 __ioremap_caller+0x3d3/0x440() Modules linked in:
Pid: 0, comm: swapper Not tainted 3.9.0-rc7+ #95
--
Thanks
Dave
Re: [PATCH 10/11] EFI: Runtime services virtual mapping
Quoted 92 lines · Dave Young
Quoted 22 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
We map the EFI regions needed for runtime services non-contiguously, with preserved alignment on virtual addresses starting from -4G down for a total max space of 64G. This way, we provide for stable runtime services addresses across kernels so that a kexec'd kernel can still use them.
Thus, they're mapped in a separate pagetable so that we don't pollute the kernel namespace.
Add an efi= kernel command line parameter for passing miscellaneous options and chicken bits from the command line.
While at it, add a chicken bit called "efi=old_map" which can be used as a fallback to the old runtime services mapping method in case there's some b0rkage with a particular EFI implementation (haha, it is hard to hold up the sarcasm here...).
Also, add the UEFI RT VA space to
Documentation/x86/x86_64/mm.txt.Signed-off-by Borislav Petkov <[email protected]>Quoted 12 lines
--- Documentation/kernel-parameters.txt | 6 ++ Documentation/x86/x86_64/mm.txt | 7 +++ arch/x86/include/asm/efi.h | 64 ++++++++++++++------ arch/x86/include/asm/pgtable_types.h | 3 +- arch/x86/platform/efi/efi.c | 94 +++++++++++++++++++++--------- arch/x86/platform/efi/efi_32.c | 9 ++- arch/x86/platform/efi/efi_64.c | 109 +++++++++++++++++++++++++++++++++++ arch/x86/platform/efi/efi_stub_64.S | 54 +++++++++++++++++ include/linux/efi.h | 1 + 9 files changed, 300 insertions(+), 47 deletions(-)Hi,
This one still does not apply:
[dave@darkstar linux-2.6]$ git am ~/Mail/bp/new/*
Applying: efi: SimplifyEFI_DEBUG
Applying: x86, pageattr: Lookup address in an arbitrary PGD
Applying: x86, pageattr: Add a PGD pagetable populating function
Applying: x86, pageattr: Add a PUD pagetable populating function
Applying: x86, pageattr: Add a PMD pagetable populating function
Applying: x86, pageattr: Add a PTE pagetable populating function
Applying: x86, pageattr: Add a PUD error unwinding path
Applying: x86, pageattr: Add last levels of error path
Applying: x86, cpa: Map in an arbitrary pgd
Applying: EFI: Runtime services virtual mapping
error: patch failed:arch/x86/platform/efi/efi.c:808
error:arch/x86/platform/efi/efi.c: patch does not apply Patch failed at 0010 EFI: Runtime services virtual mapping The copy of the patch that failed is found in:/home/dave/git/linux-2.6/.git/rebase-apply/patchWhen you have resolved this problem, run "git am --resolved". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".
I found the reason, here is the diff of the
efi.cbetween (linus tree + matt next) and the one in matt next:efi.c.matt-next renamed from efi.c.linus-tree +4
-7@@ -806,13 +806,10 @@ void __init efi_enter_virtual_mode(void) for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) { md = p; - if (!(md->attribute & EFI_MEMORY_RUNTIME)) { -#ifdef CONFIG_X86_64 - if (md->type != EFI_BOOT_SERVICES_CODE && - md->type != EFI_BOOT_SERVICES_DATA) -#endif - continue; - } + if (!(md->attribute & EFI_MEMORY_RUNTIME) && + md->type != EFI_BOOT_SERVICES_CODE && + md->type != EFI_BOOT_SERVICES_DATA) + continue; size = md->num_pages << EFI_PAGE_SHIFT; end = md->phys_addr + size;The conflict is caused by missing one commit in matt's efi next tree:
commit
700870119f49084da004ab588ea2b799689efaf7Author: Josh Boyer <[email protected]> Date: Thu Apr 18 07:51:34 2013 -0700x86, efi: Don't map Boot Services on i386
Quoted 7 lines
Add patch to fix 32bit EFI service mapping (rhbz 726701)
Multiple people are reporting hitting the following WARNING on i386,
WARNING: at
arch/x86/mm/ioremap.c:102__ioremap_caller+0x3d3/0x440() Modules linked in:
Pid: 0, comm: swapper Not tainted 3.9.0-rc7+ #95
Rethinking about above commit, it is actually a workaround for mapping boot service, I suspect the problem connect to the off-by-one problem in boot service reserve function. Matt, idea?
--
Thanks
Dave
Re: [PATCH 10/11] EFI: Runtime services virtual mapping
Quoted 12 lines · Dave Young
Quoted 12 lines
The conflict is caused by missing one commit in matt's efi next tree:
commit
700870119f49084da004ab588ea2b799689efaf7Author: Josh Boyer <[email protected]> Date: Thu Apr 18 07:51:34 2013 -0700x86, efi: Don't map Boot Services on i386
Add patch to fix 32bit EFI service mapping (rhbz 726701)
Multiple people are reporting hitting the following WARNING on i386,
Quoted 7 lines
Quoted 3 lines
WARNING: at arch/x86/mm/ioremap.c:102 __ioremap_caller+0x3d3/0x440() Modules linked in: Pid: 0, comm: swapper Not tainted 3.9.0-rc7+ #95Rethinking about above commit, it is actually a workaround for mapping boot service, I suspect the problem connect to the off-by-one problem in boot service reserve function. Matt, idea?
No, it's not the same problem. And just to make absolutely sure I reverted the above commit and applied your patch, but I still see the WARN_ON() described above.
Signature · 1 line
Matt Fleming, Intel Open Source Technology Center
Re: [PATCH 10/11] EFI: Runtime services virtual mapping
Quoted 17 lines · Matt Fleming
Quoted 16 lines · Dave Young
Quoted 15 lines
The conflict is caused by missing one commit in matt's efi next tree:
commit
700870119f49084da004ab588ea2b799689efaf7Author: Josh Boyer <[email protected]> Date: Thu Apr 18 07:51:34 2013 -0700x86, efi: Don't map Boot Services on i386
Add patch to fix 32bit EFI service mapping (rhbz 726701)
Multiple people are reporting hitting the following WARNING on i386,
WARNING: at
arch/x86/mm/ioremap.c:102__ioremap_caller+0x3d3/0x440() Modules linked in:
Pid: 0, comm: swapper Not tainted 3.9.0-rc7+ #95
Quoted 7 lines
Quoted 3 lines
Rethinking about above commit, it is actually a workaround for mapping boot service, I suspect the problem connect to the off-by-one problem in boot service reserve function. Matt, idea?
No, it's not the same problem. And just to make absolutely sure I reverted the above commit and applied your patch, but I still see the
WARN_ON()described above.
Ok, I'm wrong then, thanks for verifying..
Re: [PATCH 10/11] EFI: Runtime services virtual mapping
In reply to Dave YoungQuoted 92 lines · Dave Young
Quoted 22 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
We map the EFI regions needed for runtime services non-contiguously, with preserved alignment on virtual addresses starting from -4G down for a total max space of 64G. This way, we provide for stable runtime services addresses across kernels so that a kexec'd kernel can still use them.
Thus, they're mapped in a separate pagetable so that we don't pollute the kernel namespace.
Add an efi= kernel command line parameter for passing miscellaneous options and chicken bits from the command line.
While at it, add a chicken bit called "efi=old_map" which can be used as a fallback to the old runtime services mapping method in case there's some b0rkage with a particular EFI implementation (haha, it is hard to hold up the sarcasm here...).
Also, add the UEFI RT VA space to
Documentation/x86/x86_64/mm.txt.Signed-off-by Borislav Petkov <[email protected]>Quoted 12 lines
--- Documentation/kernel-parameters.txt | 6 ++ Documentation/x86/x86_64/mm.txt | 7 +++ arch/x86/include/asm/efi.h | 64 ++++++++++++++------ arch/x86/include/asm/pgtable_types.h | 3 +- arch/x86/platform/efi/efi.c | 94 +++++++++++++++++++++--------- arch/x86/platform/efi/efi_32.c | 9 ++- arch/x86/platform/efi/efi_64.c | 109 +++++++++++++++++++++++++++++++++++ arch/x86/platform/efi/efi_stub_64.S | 54 +++++++++++++++++ include/linux/efi.h | 1 + 9 files changed, 300 insertions(+), 47 deletions(-)Hi,
This one still does not apply:
[dave@darkstar linux-2.6]$ git am ~/Mail/bp/new/*
Applying: efi: SimplifyEFI_DEBUG
Applying: x86, pageattr: Lookup address in an arbitrary PGD
Applying: x86, pageattr: Add a PGD pagetable populating function
Applying: x86, pageattr: Add a PUD pagetable populating function
Applying: x86, pageattr: Add a PMD pagetable populating function
Applying: x86, pageattr: Add a PTE pagetable populating function
Applying: x86, pageattr: Add a PUD error unwinding path
Applying: x86, pageattr: Add last levels of error path
Applying: x86, cpa: Map in an arbitrary pgd
Applying: EFI: Runtime services virtual mapping
error: patch failed:arch/x86/platform/efi/efi.c:808
error:arch/x86/platform/efi/efi.c: patch does not apply Patch failed at 0010 EFI: Runtime services virtual mapping The copy of the patch that failed is found in:/home/dave/git/linux-2.6/.git/rebase-apply/patchWhen you have resolved this problem, run "git am --resolved". If you prefer to skip this patch, run "git am --skip" instead. To restore the original branch and stop patching, run "git am --abort".
I found the reason, here is the diff of the
efi.cbetween (linus tree + matt next) and the one in matt next:efi.c.matt-next renamed from efi.c.linus-tree +4
-7@@ -806,13 +806,10 @@ void __init efi_enter_virtual_mode(void) for (p = memmap.map; p < memmap.map_end; p += memmap.desc_size) { md = p; - if (!(md->attribute & EFI_MEMORY_RUNTIME)) { -#ifdef CONFIG_X86_64 - if (md->type != EFI_BOOT_SERVICES_CODE && - md->type != EFI_BOOT_SERVICES_DATA) -#endif - continue; - } + if (!(md->attribute & EFI_MEMORY_RUNTIME) && + md->type != EFI_BOOT_SERVICES_CODE && + md->type != EFI_BOOT_SERVICES_DATA) + continue; size = md->num_pages << EFI_PAGE_SHIFT; end = md->phys_addr + size;The conflict is caused by missing one commit in matt's efi next tree:
commit
700870119f49084da004ab588ea2b799689efaf7Author: Josh Boyer <[email protected]> Date: Thu Apr 18 07:51:34 2013 -0700x86, efi: Don't map Boot Services on i386
Quoted 7 lines
Add patch to fix 32bit EFI service mapping (rhbz 726701)
Multiple people are reporting hitting the following WARNING on i386,
WARNING: at
arch/x86/mm/ioremap.c:102__ioremap_caller+0x3d3/0x440() Modules linked in:
Pid: 0, comm: swapper Not tainted 3.9.0-rc7+ #95
Hi, Matt
How should I handle this problem, would you take above commit or rebase efi next branch?
--
Thanks
Dave
Re: [PATCH 10/11] EFI: Runtime services virtual mapping
Quoted 4 lines · Dave Young
Hi, Matt
How should I handle this problem, would you take above commit or rebase efi next branch?
Dave, could you base your patches against the EFI 'master' branch? I've rebased that branch on v3.12, which includes commit 700870119f49 ("x86,
efi: Don't map Boot Services on i386"), and merged everything from 'next'. Just note in the patch description which branch you've based your work on and why in case I forget ;-)
Let me know if this causes any issues.
Signature · 1 line
Matt Fleming, Intel Open Source Technology Center
Borislav Petkov
maintainerPATCH 11/11 efi: Check krealloc return value
In reply to Borislav PetkovFrom: Borislav Petkov <[email protected]>
Check it just in case. We might just as well panic there because runtime won't be functioning anyway.
--- arch/x86/platform/efi/efi.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/platform/efi/efi.c +8 -0
@@ -860,6 +860,9 @@ void __init efi_enter_virtual_mode(void) new_memmap =krealloc (new_memmap, (count +1 ) * memmap.desc_size,GFP_KERNEL ); +if (!new_memmap) + goto err_out; +memcpy (new_memmap + (count * memmap.desc_size), md, memmap.desc_size); count++;
@@ -914,6 +917,11 @@ void __init efi_enter_virtual_mode(void)EFI_VARIABLE_BOOTSERVICE_ACCESS |EFI_VARIABLE_RUNTIME_ACCESS , 0, NULL); + +return ; + + err_out: +pr_err ("Error reallocating memory, EFI runtime non-functional!\n" ); }/*
Signature · 1 line
1.8.4
Quoted 11 lines · Borislav Petkov
From: Borislav Petkov <[email protected]>
Hi all,
here's maybe the final version of the patchset, no major changes since the last time but cosmetic and cleanups and tidying as requested by Matt and others.
Patches also at:
git://git.kernel.org/pub/scm/linux/kernel/git/bp/bp.git#efi
Thanks Borislav, I've merged this version in the EFI 'next' branch to queue up for v3.14.
Good work!
Signature · 1 line
Matt Fleming, Intel Open Source Technology Center