ASI - Part II (Decorated Page-Table)
Subsystems (3)
v4 cover ASI - Part II (Decorated Page-Table)
This is part II of ASI RFC v4. Please refer to the cover letter of part I for an overview the ASI RFC.
https://lore.kernel.org/lkml/[email protected]/
This part introduces decorated page-table which encapsulate native page table (e.g. a PGD) in order to provide convenient page-table management functions, such as tracking address range mapped in a page-table or safely handling references to another page-table.
Decorated page-table can then be used to easily create and manage page tables to be used with ASI. It will be used by the ASI test driver (see part III) and later by KVM ASI.
Decorated page-table is independent of ASI, and can potentially be used anywhere a page-table is needed.
Thanks,
alex.
-----
Alexandre Chartre (13):
mm/x86: Introduce decorated page-table (dpt)
mm/dpt: Track buffers allocated for a decorated page-table mm/dpt: Add decorated page-table entry offset functions mm/dpt: Add decorated page-table entry allocation functions mm/dpt: Add decorated page-table entry set functions mm/dpt: Functions to populate a decorated page-table from a VA range mm/dpt: Helper functions to map module into a decorated page-table mm/dpt: Keep track of VA ranges mapped in a decorated page-table mm/dpt: Functions to clear decorated page-table entries for a VA range mm/dpt: Function to copy page-table entries for percpu buffer mm/dpt: Add decorated page-table remap function
mm/dpt: Handle decorated page-table mapped range leaks and overlaps mm/asi: Function to init decorated page-table with ASI core mappings
arch/x86/include/asm/asi.h | 2 + arch/x86/include/asm/dpt.h | 89 +++ arch/x86/mm/Makefile | 2 +- arch/x86/mm/asi.c | 57 ++ arch/x86/mm/dpt.c | 1051 ++++++++++++++++++++++++++++++++++++ 5 files changed, 1200 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/asm/dpt.h create mode 100644 arch/x86/mm/dpt.c
Signature · 1 line
2.18.2
RFC PATCH v4 1/13 mm/x86: Introduce decorated page-table (dpt)
A decorated page-table (dpt) encapsulates a native page-table (e.g. a PGD) and maintain additional attributes related to this page-table. It aims to be the base structure for providing useful functions to manage a page-table, such as tracking VA range mapped in a page-table or safely handling references to another page-table.
--- arch/x86/include/asm/dpt.h | 23 +++++++++++++ arch/x86/mm/Makefile | 2 +- arch/x86/mm/dpt.c | 67 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 91 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/asm/dpt.h create mode 100644 arch/x86/mm/dpt.c
arch/x86/include/asm/dpt.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef ARCH_X86_MM_DPT_H +#define ARCH_X86_MM_DPT_H + +#include <linux/spinlock.h> + +#include <asm/pgtable.h> + +/* +* A decorated page-table (dpt) encapsulates a native page-table (e.g. +* a PGD) and maintain additional attributes related to this page-table. +*/ +struct dpt { +spinlock_t lock;/* protect all attributes */ +pgd_t *pagetable;/* the actual page-table */ +unsigned int alignment;/* page-table alignment */ + +}; + +extern struct dpt *dpt_create (unsigned int pgt_alignment); +extern void dpt_destroy (struct dpt *dpt); + +#endif
arch/x86/mm/Makefile +1 -1
@@ -48,7 +48,7 @@ obj-$(CONFIG_NUMA_EMU) += numa_emulation.o obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS) += pkeys.o obj-$(CONFIG_RANDOMIZE_MEMORY) += kaslr.o obj-$(CONFIG_PAGE_TABLE_ISOLATION) += pti.o -obj-$(CONFIG_ADDRESS_SPACE_ISOLATION) += asi.o +obj-$(CONFIG_ADDRESS_SPACE_ISOLATION) += asi.o dpt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt.o obj-$(CONFIG_AMD_MEM_ENCRYPT) += mem_encrypt_identity.o
arch/x86/mm/dpt.c new file +67 -0
67 lines not on this page · raw message
Signature · 1 line
2.18.2
RFC PATCH v4 2/13 mm/dpt: Track buffers allocated for a decorated page-table
In reply to Alexandre ChartreAdd functions to track buffers allocated for a decorated page-table. A page-table can have direct references to the kernel page table, at different levels (PGD, P4D, PUD, PMD). When freeing a page-table, we should make sure that we free parts actually allocated for the decorated page-table, and not parts of the kernel page table referenced from the page-table. To do so, we will keep track of buffers when building the page-table.
--- arch/x86/include/asm/dpt.h | 21 ++++++++++ arch/x86/mm/dpt.c | 82 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 103 insertions(+)
arch/x86/include/asm/dpt.h +21 -0
@@ -3,9 +3,18 @@#define ARCH_X86_MM_DPT_H #include <linux/spinlock.h> +#include <linux/xarray.h> #include <asm/pgtable.h> +enum page_table_level { +PGT_LEVEL_PTE , +PGT_LEVEL_PMD , +PGT_LEVEL_PUD , +PGT_LEVEL_P4D , +PGT_LEVEL_PGD +}; +/* * A decorated page-table (dpt) encapsulates a native page-table (e.g. * a PGD) and maintain additional attributes related to this page-table.
@@ -15,6 +24,18 @@ struct dpt {pgd_t *pagetable;/* the actual page-table */ unsigned int alignment;/* page-table alignment */ +/* +* A page-table can have direct references to another page-table, +* at different levels (PGD, P4D, PUD, PMD). When freeing or +* modifying a page-table, we should make sure that we free/modify +* parts effectively allocated to the actual page-table, and not +* parts of another page-table referenced from this page-table. +* +* To do so, the backend_pages XArray is used to keep track of pages +* used for this page-table. +*/ +struct xarray backend_pages;/* page-table pages */ +unsigned long backend_pages_count;/* pages count */ };extern struct dpt *dpt_create (unsigned int pgt_alignment);
arch/x86/mm/dpt.c +82 -0
104 lines not on this page · raw message
Signature · 1 line
2.18.2
RFC PATCH v4 3/13 mm/dpt: Add decorated page-table entry offset functions
In reply to Alexandre ChartreAdd wrappers around the p4d/pud/pmd/pte offset kernel functions which ensure that page-table pointers are in the specified decorated page-table.
--- arch/x86/mm/dpt.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 66 insertions(+)
arch/x86/mm/dpt.c +66 -0
@@ -82,6 +82,72 @@ static bool dpt_valid_offset(struct dpt *dpt, void *offset)return valid; } +/* +* dpt_pXX_offset() functions are equivalent to kernel pXX_offset() +* functions but, in addition, they ensure that page table pointers +* are in the specified decorated page table. Otherwise an error is +* returned. +*/ + +static pte_t *dpt_pte_offset (struct dpt *dpt, +pmd_t *pmd,unsigned long addr) +{ +pte_t *pte; + + pte =pte_offset_map (pmd, addr); +if (!dpt_valid_offset (dpt, pte)) { +pr_err ("DPT %p: PTE %px not found\n" , dpt, pte); +return ERR_PTR (-EINVAL ); + } + +return pte; +} + +static pmd_t *dpt_pmd_offset (struct dpt *dpt, +pud_t *pud,unsigned long addr) +{ +pmd_t *pmd; + + pmd =pmd_offset (pud, addr); +if (!dpt_valid_offset (dpt, pmd)) { +pr_err ("DPT %p: PMD %px not found\n" , dpt, pmd); +return ERR_PTR (-EINVAL ); + } + +return pmd; +} + +static pud_t *dpt_pud_offset (struct dpt *dpt, +p4d_t *p4d,unsigned long addr) +{ +pud_t *pud; + + pud =pud_offset (p4d, addr); +if (!dpt_valid_offset (dpt, pud)) { +pr_err ("DPT %p: PUD %px not found\n" , dpt, pud); +return ERR_PTR (-EINVAL ); + } + +return pud; +} + +static p4d_t *dpt_p4d_offset (struct dpt *dpt, +pgd_t *pgd,unsigned long addr) +{ +p4d_t *p4d; + + p4d =p4d_offset (pgd, addr); +/* +* p4d is the same has pgd if we don't have a 5-level page table. +*/ +if ((p4d != (p4d_t *)pgd) && !dpt_valid_offset (dpt, p4d)) { +pr_err ("DPT %p: P4D %px not found\n" , dpt, p4d); +return ERR_PTR (-EINVAL ); + } + +return p4d; +} +/* * dpt_create - allocate a page-table and create a corresponding * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
RFC PATCH v4 4/13 mm/dpt: Add decorated page-table entry allocation functions
In reply to Alexandre ChartreAdd functions to allocate p4d/pud/pmd/pte pages for an decorated page-table and keep track of them.
--- arch/x86/mm/dpt.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+)
arch/x86/mm/dpt.c +110 -0
@@ -4,6 +4,7 @@* */ +#include <linux/mm.h> #include <linux/slab.h> #include <asm/dpt.h>
@@ -148,6 +149,115 @@ static p4d_t *dpt_p4d_offset(struct dpt *dpt,return p4d; } +/* +* dpt_pXX_alloc() functions are equivalent to kernel pXX_alloc() functions +* but, in addition, they keep track of new pages allocated for the specified +* decorated page-table. +*/ + +static pte_t *dpt_pte_alloc (struct dpt *dpt,pmd_t *pmd,unsigned long addr) +{ +struct page *page; +pte_t *pte; +int err; + +if (pmd_none (*pmd)) { + page =alloc_page (GFP_KERNEL | __GFP_ZERO); +if (!page) +return ERR_PTR (-ENOMEM ); + pte = (pte_t *)page_address (page); + err =dpt_add_backend_page (dpt, pte,PGT_LEVEL_PTE ); +if (err) { +free_page ((unsigned long )pte); +return ERR_PTR (err); + } +set_pmd_safe (pmd,__pmd (__pa (pte) | _KERNPG_TABLE)); + pte =pte_offset_map (pmd, addr); + }else { + pte =dpt_pte_offset (dpt, pmd, addr); + } + +return pte; +} + +static pmd_t *dpt_pmd_alloc (struct dpt *dpt,pud_t *pud,unsigned long addr) +{ +struct page *page; +pmd_t *pmd; +int err; + +if (pud_none (*pud)) { + page =alloc_page (GFP_KERNEL | __GFP_ZERO); +if (!page) +return ERR_PTR (-ENOMEM ); + pmd = (pmd_t *)page_address (page); + err =dpt_add_backend_page (dpt, pmd,PGT_LEVEL_PMD ); +if (err) { +free_page ((unsigned long )pmd); +return ERR_PTR (err); + } +set_pud_safe (pud,__pud (__pa (pmd) | _KERNPG_TABLE)); + pmd =pmd_offset (pud, addr); + }else { + pmd =dpt_pmd_offset (dpt, pud, addr); + } + +return pmd; +} + +static pud_t *dpt_pud_alloc (struct dpt *dpt,p4d_t *p4d,unsigned long addr) +{ +struct page *page; +pud_t *pud; +int err; + +if (p4d_none (*p4d)) { + page =alloc_page (GFP_KERNEL | __GFP_ZERO); +if (!page) +return ERR_PTR (-ENOMEM ); + pud = (pud_t *)page_address (page); + err =dpt_add_backend_page (dpt, pud,PGT_LEVEL_PUD ); +if (err) { +free_page ((unsigned long )pud); +return ERR_PTR (err); + } +set_p4d_safe (p4d,__p4d (__pa (pud) | _KERNPG_TABLE)); + pud =pud_offset (p4d, addr); + }else { + pud =dpt_pud_offset (dpt, p4d, addr); + } + +return pud; +} + +static p4d_t *dpt_p4d_alloc (struct dpt *dpt,pgd_t *pgd,unsigned long addr) +{ +struct page *page; +p4d_t *p4d; +int err; + +if (!pgtable_l5_enabled ()) +return (p4d_t *)pgd; + +if (pgd_none (*pgd)) { + page =alloc_page (GFP_KERNEL | __GFP_ZERO); +if (!page) +return ERR_PTR (-ENOMEM ); + p4d = (p4d_t *)page_address (page); + err =dpt_add_backend_page (dpt, p4d,PGT_LEVEL_P4D ); +if (err) { +free_page ((unsigned long )p4d); +return ERR_PTR (err); + } +set_pgd_safe (pgd,__pgd (__pa (p4d) | _KERNPG_TABLE)); + p4d =p4d_offset (pgd, addr); + }else { + p4d =dpt_p4d_offset (dpt, pgd, addr); + } + +return p4d; +} +/* * dpt_create - allocate a page-table and create a corresponding * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
RFC PATCH v4 5/13 mm/dpt: Add decorated page-table entry set functions
In reply to Alexandre ChartreAdd wrappers around the page table entry (pgd/p4d/pud/pmd) set functions which check that an existing entry is not being overwritten.
--- arch/x86/mm/dpt.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 126 insertions(+)
arch/x86/mm/dpt.c +126 -0
@@ -258,6 +258,132 @@ static p4d_t *dpt_p4d_alloc(struct dpt *dpt, pgd_t *pgd, unsigned long addr)return p4d; } +/* +* dpt_set_pXX() functions are equivalent to kernel set_pXX() functions +* but, in addition, they ensure that they are not overwriting an already +* existing reference in the decorated page table. Otherwise an error is +* returned. +*/ + +static int dpt_set_pte (struct dpt *dpt,pte_t *pte,pte_t pte_value) +{ +#ifdef DEBUG +/* +* The pte pointer should come from dpt_pte_alloc() or dpt_pte_offset() +* both of which check if the pointer is in the decorated page table. +* So this is a paranoid check to ensure the pointer is really in the +* decorated page table. +*/ +if (!dpt_valid_offset (dpt, pte)) { +pr_err ("DPT %p: PTE %px not found\n" , dpt, pte); +return -EINVAL ; + } +#endif +set_pte (pte, pte_value); + +return 0 ; +} + +static int dpt_set_pmd (struct dpt *dpt,pmd_t *pmd,pmd_t pmd_value) +{ +#ifdef DEBUG +/* +* The pmd pointer should come from dpt_pmd_alloc() or dpt_pmd_offset() +* both of which check if the pointer is in the decorated page table. +* So this is a paranoid check to ensure the pointer is really in the +* decorated page table. +*/ +if (!dpt_valid_offset (dpt, pmd)) { +pr_err ("DPT %p: PMD %px not found\n" , dpt, pmd); +return -EINVAL ; + } +#endif +if (pmd_val (*pmd) ==pmd_val (pmd_value)) +return 0 ; + +if (!pmd_none (*pmd)) { +pr_err ("DPT %p: PMD %px overwriting %lx with %lx\n" , + dpt, pmd,pmd_val (*pmd),pmd_val (pmd_value)); +return -EBUSY ; + } + +set_pmd (pmd, pmd_value); + +return 0 ; +} + +static int dpt_set_pud (struct dpt *dpt,pud_t *pud,pud_t pud_value) +{ +#ifdef DEBUG +/* +* The pud pointer should come from dpt_pud_alloc() or dpt_pud_offset() +* both of which check if the pointer is in the decorated page table. +* So this is a paranoid check to ensure the pointer is really in the +* decorated page table. +*/ +if (!dpt_valid_offset (dpt, pud)) { +pr_err ("DPT %p: PUD %px not found\n" , dpt, pud); +return -EINVAL ; + } +#endif +if (pud_val (*pud) ==pud_val (pud_value)) +return 0 ; + +if (!pud_none (*pud)) { +pr_err ("DPT %p: PUD %px overwriting %lx with %lx\n" , + dpt, pud,pud_val (*pud),pud_val (pud_value)); +return -EBUSY ; + } + +set_pud (pud, pud_value); + +return 0 ; +} + +static int dpt_set_p4d (struct dpt *dpt,p4d_t *p4d,p4d_t p4d_value) +{ +#ifdef DEBUG +/* +* The p4d pointer should come from dpt_p4d_alloc() or dpt_p4d_offset() +* both of which check if the pointer is in the decorated page table. +* So this is a paranoid check to ensure the pointer is really in the +* decorated page table. +*/ +if (!dpt_valid_offset (dpt, p4d)) { +pr_err ("DPT %p: P4D %px not found\n" , dpt, p4d); +return -EINVAL ; + } +#endif +if (p4d_val (*p4d) ==p4d_val (p4d_value)) +return 0 ; + +if (!p4d_none (*p4d)) { +pr_err ("DPT %p: P4D %px overwriting %lx with %lx\n" , + dpt, p4d,p4d_val (*p4d),p4d_val (p4d_value)); +return -EBUSY ; + } + +set_p4d (p4d, p4d_value); + +return 0 ; +} + +static int dpt_set_pgd (struct dpt *dpt,pgd_t *pgd,pgd_t pgd_value) +{ +if (pgd_val (*pgd) ==pgd_val (pgd_value)) +return 0 ; + +if (!pgd_none (*pgd)) { +pr_err ("DPT %p: PGD %px overwriting %lx with %lx\n" , + dpt, pgd,pgd_val (*pgd),pgd_val (pgd_value)); +return -EBUSY ; + } + +set_pgd (pgd, pgd_value); + +return 0 ; +} +/* * dpt_create - allocate a page-table and create a corresponding * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
RFC PATCH v4 6/13 mm/dpt: Functions to populate a decorated page-table from a VA range
In reply to Alexandre ChartreProvide functions to copy page-table entries from the kernel page-table to a decorated page-table for a specified VA range. These functions are based on the copy_pxx_range() functions defined in mm/memory.c. A first difference is that a level parameter can be specified to indicate the page-table level (PGD, P4D, PUD PMD, PTE) at which the copy should be done. Also functions don't rely on mm or vma, and they don't alter the source page-table even if an entry is bad. Finally, the VA range start and size don't need to be page-aligned.
--- arch/x86/include/asm/dpt.h | 3 + arch/x86/mm/dpt.c | 205 +++++++++++++++++++++++++++++++++++++ 2 files changed, 208 insertions(+)
arch/x86/include/asm/dpt.h +3 -0
@@ -40,5 +40,8 @@ struct dpt {extern struct dpt *dpt_create (unsigned int pgt_alignment);extern void dpt_destroy (struct dpt *dpt); +extern int dpt_map_range (struct dpt *dpt,void *ptr,size_t size, +enum page_table_level level); +extern int dpt_map (struct dpt *dpt,void *ptr,unsigned long size);#endif
arch/x86/mm/dpt.c +205 -0
211 lines not on this page · raw message
Signature · 1 line
2.18.2
RFC PATCH v4 7/13 mm/dpt: Helper functions to map module into a decorated page-table
In reply to Alexandre ChartreAdd helper functions to easily map a module into a decorated page-table.
--- arch/x86/include/asm/dpt.h | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+)
arch/x86/include/asm/dpt.h +21 -0
@@ -2,6 +2,7 @@#ifndef ARCH_X86_MM_DPT_H #define ARCH_X86_MM_DPT_H +#include <linux/module.h> #include <linux/spinlock.h> #include <linux/xarray.h>
@@ -44,4 +45,24 @@ extern int dpt_map_range(struct dpt *dpt, void *ptr, size_t size, enum page_table_level level);extern int dpt_map (struct dpt *dpt,void *ptr,unsigned long size); +static inline int dpt_map_module (struct dpt *dpt,char *module_name) +{ +struct module *module; + + module =find_module (module_name); +if (!module) +return -ESRCH ; + +return dpt_map (dpt, module->core_layout.base, module->core_layout.size); +} + +/* +* Copy the memory mapping for the current module. This is defined as a +* macro to ensure it is expanded in the module making the call so that +* THIS_MODULE has the correct value. +*/ +#define DPT_MAP_THIS_MODULE (dpt) \ + (dpt_map(dpt, THIS_MODULE->core_layout.base, \ + THIS_MODULE->core_layout.size)) +#endif
Signature · 1 line
2.18.2
RFC PATCH v4 8/13 mm/dpt: Keep track of VA ranges mapped in a decorated page-table
In reply to Alexandre ChartreAdd functions to keep track of VA ranges mapped in a decorated page-table. This will be used when unmapping to ensure the same range is unmapped, at the same page-table level. This will also be used to handle mapping and unmapping of overlapping VA ranges.
--- arch/x86/include/asm/dpt.h | 12 ++++++++ arch/x86/mm/dpt.c | 60 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 72 insertions(+)
arch/x86/include/asm/dpt.h +12 -0
@@ -16,6 +16,17 @@ enum page_table_level { PGT_LEVEL_PGD }; +/* +* Structure to keep track of address ranges mapped into a decorated +* page-table. +*/ +struct dpt_range_mapping { +struct list_head list; +void *ptr;/* range start address */ +size_t size;/* range size */ +enum page_table_level level;/* mapping level */ +}; +/* * A decorated page-table (dpt) encapsulates a native page-table (e.g. * a PGD) and maintain additional attributes related to this page-table.
@@ -24,6 +35,7 @@ struct dpt {spinlock_t lock;/* protect all attributes */ pgd_t *pagetable;/* the actual page-table */ unsigned int alignment;/* page-table alignment */ +struct list_head mapping_list;/* list of VA range mapping */ /* * A page-table can have direct references to another page-table,
arch/x86/mm/dpt.c +60 -0
98 lines not on this page · raw message
Signature · 1 line
2.18.2
RFC PATCH v4 9/13 mm/dpt: Functions to clear decorated page-table entries for a VA range
In reply to Alexandre ChartreProvide functions to clear page-table entries in a decorated page-table for a specified VA range. Functions also check that the clearing effectively happens in the decorated page-table and there is no crossing of the decorated page-table boundary (through references to another page table), so that another page table is not modified by mistake.
As information (address, size, page-table level) about VA ranges mapped to the decorated page-table is tracked, clearing is done with just specifying the start address of the range.
--- arch/x86/include/asm/dpt.h | 1 + arch/x86/mm/dpt.c | 135 +++++++++++++++++++++++++++++++++++++ 2 files changed, 136 insertions(+)
arch/x86/include/asm/dpt.h +1 -0
@@ -56,6 +56,7 @@ extern void dpt_destroy(struct dpt *dpt);extern int dpt_map_range (struct dpt *dpt,void *ptr,size_t size,enum page_table_level level);extern int dpt_map (struct dpt *dpt,void *ptr,unsigned long size); +extern void dpt_unmap (struct dpt *dpt,void *ptr);static inline int dpt_map_module (struct dpt *dpt,char *module_name) {
arch/x86/mm/dpt.c +135 -0
141 lines not on this page · raw message
Signature · 1 line
2.18.2
RFC PATCH v4 10/13 mm/dpt: Function to copy page-table entries for percpu buffer
In reply to Alexandre ChartreProvide functions to copy page-table entries from the kernel page-table to a decorated page-table for a percpu buffer. A percpu buffer have a different VA range for each cpu and all them have to be copied.
--- arch/x86/include/asm/dpt.h | 6 ++++++ arch/x86/mm/dpt.c | 38 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 44 insertions(+)
arch/x86/include/asm/dpt.h +6 -0
@@ -78,4 +78,10 @@ static inline int dpt_map_module(struct dpt *dpt, char *module_name) (dpt_map (dpt,THIS_MODULE ->core_layout.base, \ THIS_MODULE->core_layout.size)) +extern int dpt_map_percpu (struct dpt *dpt,void *percpu_ptr,size_t size); +extern void dpt_unmap_percpu (struct dpt *dpt,void *percpu_ptr); + +#define DPT_MAP_CPUVAR (dpt, cpuvar) \ + dpt_map_percpu(dpt, &cpuvar, sizeof(cpuvar)) +#endif
arch/x86/mm/dpt.c +38 -0
@@ -771,6 +771,44 @@ void dpt_unmap(struct dpt *dpt, void *ptr) }EXPORT_SYMBOL (dpt_unmap); +void dpt_unmap_percpu (struct dpt *dpt,void *percpu_ptr) +{ +void *ptr; +int cpu; + +pr_debug ("DPT %p: UNMAP PERCPU %px\n" , dpt, percpu_ptr); +for_each_possible_cpu (cpu) { + ptr =per_cpu_ptr (percpu_ptr, cpu); +pr_debug ("DPT %p: UNMAP PERCPU%d %px\n" , dpt, cpu, ptr); +dpt_unmap (dpt, ptr); + } +} +EXPORT_SYMBOL (dpt_unmap_percpu); + +int dpt_map_percpu (struct dpt *dpt,void *percpu_ptr,size_t size) +{ +int cpu, err; +void *ptr; + +pr_debug ("DPT %p: MAP PERCPU %px\n" , dpt, percpu_ptr); +for_each_possible_cpu (cpu) { + ptr =per_cpu_ptr (percpu_ptr, cpu); +pr_debug ("DPT %p: MAP PERCPU%d %px\n" , dpt, cpu, ptr); + err =dpt_map (dpt, ptr, size); +if (err) { +/* +* Need to unmap any percpu mapping which has +* succeeded before the failure. +*/ +dpt_unmap_percpu (dpt, percpu_ptr); +return err; + } + } + +return 0 ; +} +EXPORT_SYMBOL (dpt_map_percpu); +/* * dpt_create - allocate a page-table and create a corresponding * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
RFC PATCH v4 11/13 mm/dpt: Add decorated page-table remap function
In reply to Alexandre ChartreAdd a function to remap an already mapped buffer with a new address in a decorated page-table: the already mapped buffer is unmapped, and a new mapping is added for the specified new address.
This is useful to track and remap a buffer which can be freed and then reallocated.
--- arch/x86/include/asm/dpt.h | 1 + arch/x86/mm/dpt.c | 25 +++++++++++++++++++++++++ 2 files changed, 26 insertions(+)
arch/x86/include/asm/dpt.h +1 -0
@@ -57,6 +57,7 @@ extern int dpt_map_range(struct dpt *dpt, void *ptr, size_t size, enum page_table_level level);extern int dpt_map (struct dpt *dpt,void *ptr,unsigned long size);extern void dpt_unmap (struct dpt *dpt,void *ptr); +extern int dpt_remap (struct dpt *dpt,void **mapping,void *ptr,size_t size);static inline int dpt_map_module (struct dpt *dpt,char *module_name) {
arch/x86/mm/dpt.c +25 -0
@@ -809,6 +809,31 @@ int dpt_map_percpu(struct dpt *dpt, void *percpu_ptr, size_t size) }EXPORT_SYMBOL (dpt_map_percpu); +int dpt_remap (struct dpt *dpt,void **current_ptrp,void *new_ptr,size_t size) +{ +void *current_ptr = *current_ptrp; +int err; + +if (current_ptr == new_ptr) { +/* no change, already mapped */ +return 0 ; + } + +if (current_ptr) { +dpt_unmap (dpt, current_ptr); + *current_ptrp =NULL ; + } + + err =dpt_map (dpt, new_ptr, size); +if (err) +return err; + + *current_ptrp = new_ptr; + +return 0 ; +} +EXPORT_SYMBOL (dpt_remap); +/* * dpt_create - allocate a page-table and create a corresponding * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
RFC PATCH v4 12/13 mm/dpt: Handle decorated page-table mapped range leaks and overlaps
In reply to Alexandre ChartreWhen mapping a buffer in a decorated page-table, data around the buffer can also be mapped if the entire buffer is not aligned with the page directory size used for the mapping. So, data can potentially leak into the decorated page-table. In such a case, print a warning that data are leaking.
Also data effectively mapped can overlap with an already mapped buffer. This is not an issue when mapping data but, when unmapping, make sure data from another buffer don't get unmapped as a side effect.
--- arch/x86/include/asm/dpt.h | 1 + arch/x86/mm/dpt.c | 197 +++++++++++++++++++++++++++++++------ 2 files changed, 168 insertions(+), 30 deletions(-)
arch/x86/include/asm/dpt.h +1 -0
@@ -25,6 +25,7 @@ struct dpt_range_mapping {void *ptr;/* range start address */ size_t size;/* range size */ enum page_table_level level;/* mapping level */ +int refcnt;/* reference count (for overlap) */ };/*
arch/x86/mm/dpt.c +167 -30
252 lines not on this page · raw message
Signature · 1 line
2.18.2
RFC PATCH v4 13/13 mm/asi: Function to init decorated page-table with ASI core mappings
In reply to Alexandre ChartreCore mappings are the minimal mappings we need to be able to enter isolation and handle an isolation abort or exit. This includes the kernel code, the GDT and the percpu ASI sessions. We also need a stack so we map the current stack when entering isolation and unmap it on exit/abort.
--- arch/x86/include/asm/asi.h | 2 ++ arch/x86/mm/asi.c | 57 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+)
arch/x86/include/asm/asi.h +2 -0
@@ -45,6 +45,7 @@#include <linux/export.h> #include <asm/asi_session.h> +#include <asm/dpt.h> /* * ASI_NR_DYN_ASIDS is the same as TLB_NR_DYN_ASIDS. We can't directly
@@ -150,6 +151,7 @@ extern void asi_destroy(struct asi *asi);extern void asi_set_pagetable (struct asi *asi,pgd_t *pagetable);extern int asi_enter (struct asi *asi);extern void asi_exit (struct asi *asi); +extern int asi_init_dpt (struct dpt *dpt);#ifdef CONFIG_PAGE_TABLE_ISOLATION DECLARE_ASI_TYPE (user);
arch/x86/mm/asi.c +57 -0
63 lines not on this page · raw message
Signature · 1 line
2.18.2
Hello Alexandre,
Quoted 10 lines · Alexandre Chartre
This is part II of ASI RFC v4. Please refer to the cover letter of part I for an overview the ASI RFC.
https://lore.kernel.org/lkml/[email protected]/
This part introduces decorated page-table which encapsulate native page table (e.g. a PGD) in order to provide convenient page-table management functions, such as tracking address range mapped in a page-table or safely handling references to another page-table.
Quoted 6 lines
Decorated page-table can then be used to easily create and manage page tables to be used with ASI. It will be used by the ASI test driver (see part III) and later by KVM ASI.
Decorated page-table is independent of ASI, and can potentially be used anywhere a page-table is needed.
This is very impressive work!
I wonder why did you decide to make dpt x86-specific? Unless I've missed simething, the dpt implementation does not rely on anything architecture specific and can go straight to linux/mm.
Another thing that comes to mind is that we already have a very decorated page table, which is mm_struct. I admit that my attempt to split out the core page table bits from the mm_struct [1] didn't went far, but I still think we need a first class abstraction for the page table that will be used by both user memory management and the management of the reduced kernel address spaces.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=pg_table/v0.0
Quoted 33 lines
Thanks,
alex.
-----
Alexandre Chartre (13):
mm/x86: Introduce decorated page-table (dpt)
mm/dpt: Track buffers allocated for a decorated page-table mm/dpt: Add decorated page-table entry offset functions mm/dpt: Add decorated page-table entry allocation functions mm/dpt: Add decorated page-table entry set functions mm/dpt: Functions to populate a decorated page-table from a VA range mm/dpt: Helper functions to map module into a decorated page-table mm/dpt: Keep track of VA ranges mapped in a decorated page-table mm/dpt: Functions to clear decorated page-table entries for a VA range mm/dpt: Function to copy page-table entries for percpu buffer mm/dpt: Add decorated page-table remap function
mm/dpt: Handle decorated page-table mapped range leaks and overlaps mm/asi: Function to init decorated page-table with ASI core mappingsarch/x86/include/asm/asi.h | 2 + arch/x86/include/asm/dpt.h | 89 +++ arch/x86/mm/Makefile | 2 +- arch/x86/mm/asi.c | 57 ++ arch/x86/mm/dpt.c | 1051 ++++++++++++++++++++++++++++++++++++ 5 files changed, 1200 insertions(+), 1 deletion(-) create mode 100644 arch/x86/include/asm/dpt.h create mode 100644 arch/x86/mm/dpt.cSignature · 1 line
2.18.2
Signature · 2 lines
Sincerely yours, Mike.
Quoted 19 lines · Mike Rapoport
Hello Alexandre,
Quoted 16 lines · Alexandre Chartre
This is part II of ASI RFC v4. Please refer to the cover letter of part I for an overview the ASI RFC.
https://lore.kernel.org/lkml/[email protected]/This part introduces decorated page-table which encapsulate native page table (e.g. a PGD) in order to provide convenient page-table management functions, such as tracking address range mapped in a page-table or safely handling references to another page-table.
Decorated page-table can then be used to easily create and manage page tables to be used with ASI. It will be used by the ASI test driver (see part III) and later by KVM ASI.
Decorated page-table is independent of ASI, and can potentially be used anywhere a page-table is needed.
Hi Mike,
Quoted 5 lines
This is very impressive work!
I wonder why did you decide to make dpt x86-specific? Unless I've missed simething, the dpt implementation does not rely on anything architecture specific and can go straight to linux/mm.
Correct, this is not x86 specific. I put it in arch/x86 because that's currently the only place were I use it, but it can be moved to linux/mm.
Quoted 2 lines
Another thing that comes to mind is that we already have a very decorated page table, which is
mm_struct.
mm_struct doesn't define a generic page-table encapsulation. mm_struct references a page table (i.e. PGD) and adds all kind of attributes needed for mm management but not necessarily related to page-table.
Quoted 5 lines
I admit that my attempt to
split out the core page table bits from themm_struct[1] didn't went far, but I still think we need a first class abstraction for the page table that will be used by both user memory management and the management of the reduced kernel address spaces.
Agree. I remember your attempt to extract the page-table from mm_struct; this is not a simple work! For ASI, I didn't need mm, so it was simpler to built a simple decorated page-table without attempting to use with mm (at least for now).
Thanks,
alex.
PS: if you want to play with dpt, there's a bug in dpt_destroy(), patch 08 adds a a double free of dpt->backend_pages pages.
Quoted 37 lines
[1] https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=pg_table/v0.0
Quoted 33 lines
Thanks,
alex.
-----
Alexandre Chartre (13):
mm/x86: Introduce decorated page-table (dpt)
mm/dpt: Track buffers allocated for a decorated page-table mm/dpt: Add decorated page-table entry offset functions mm/dpt: Add decorated page-table entry allocation functions mm/dpt: Add decorated page-table entry set functions mm/dpt: Functions to populate a decorated page-table from a VA range mm/dpt: Helper functions to map module into a decorated page-table mm/dpt: Keep track of VA ranges mapped in a decorated page-table mm/dpt: Functions to clear decorated page-table entries for a VA range mm/dpt: Function to copy page-table entries for percpu buffer mm/dpt: Add decorated page-table remap function
mm/dpt: Handle decorated page-table mapped range leaks and overlaps mm/asi: Function to init decorated page-table with ASI core mappingsarch/x86/include/asm/asi.h | 2 + arch/x86/include/asm/dpt.h | 89 +++ arch/x86/mm/Makefile | 2 +- arch/x86/mm/asi.c | 57 ++ arch/x86/mm/dpt.c | 1051 ++++++++++++++++++++++++++++++++++++ 5 files changed, 1200 insertions(+), 1 deletion(-)create mode 100644
arch/x86/include/asm/dpt.h
create mode 100644arch/x86/mm/dpt.cSignature · 1 line
2.18.2