Skip to contentSkip to conversation

ASI - Part II (Decorated Page-Table)

RFC PATCH In review Alexandre Chartre 2 participants 16 messages May 4, 2020 – May 14, 2020

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

v4 cover ASI - Part II (Decorated Page-Table)

This is part II of ASI RFC v4. Please refer to the cover letter of part I for an overview the ASI RFC.

https://lore.kernel.org/lkml/[email protected]/

This part introduces decorated page-table which encapsulate native page table (e.g. a PGD) in order to provide convenient page-table management functions, such as tracking address range mapped in a page-table or safely handling references to another page-table.

Decorated page-table can then be used to easily create and manage page tables to be used with ASI. It will be used by the ASI test driver (see part III) and later by KVM ASI.

Decorated page-table is independent of ASI, and can potentially be used anywhere a page-table is needed.

Thanks,

alex.

-----

Alexandre Chartre (13):
mm/x86: Introduce decorated page-table (dpt)
mm/dpt: Track buffers allocated for a decorated page-table mm/dpt: Add decorated page-table entry offset functions mm/dpt: Add decorated page-table entry allocation functions mm/dpt: Add decorated page-table entry set functions mm/dpt: Functions to populate a decorated page-table from a VA range mm/dpt: Helper functions to map module into a decorated page-table mm/dpt: Keep track of VA ranges mapped in a decorated page-table mm/dpt: Functions to clear decorated page-table entries for a VA range mm/dpt: Function to copy page-table entries for percpu buffer mm/dpt: Add decorated page-table remap function
mm/dpt: Handle decorated page-table mapped range leaks and overlaps mm/asi: Function to init decorated page-table with ASI core mappings

 arch/x86/include/asm/asi.h |    2 +
 arch/x86/include/asm/dpt.h |   89 +++
 arch/x86/mm/Makefile       |    2 +-
 arch/x86/mm/asi.c          |   57 ++
 arch/x86/mm/dpt.c          | 1051 ++++++++++++++++++++++++++++++++++++
 5 files changed, 1200 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/include/asm/dpt.h
 create mode 100644 arch/x86/mm/dpt.c
Signature · 1 line
2.18.2
#1 lore raw

RFC PATCH v4 1/13 mm/x86: Introduce decorated page-table (dpt)

A decorated page-table (dpt) encapsulates a native page-table (e.g. a PGD) and maintain additional attributes related to this page-table. It aims to be the base structure for providing useful functions to manage a page-table, such as tracking VA range mapped in a page-table or safely handling references to another page-table.

---
 arch/x86/include/asm/dpt.h | 23 +++++++++++++
 arch/x86/mm/Makefile       |  2 +-
 arch/x86/mm/dpt.c          | 67 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 91 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/include/asm/dpt.h
 create mode 100644 arch/x86/mm/dpt.c
arch/x86/include/asm/dpt.h new file +23 -0
new file mode 100644
index 000000000000..1da4d43d5e94
@@ -0,0 +1,23 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef ARCH_X86_MM_DPT_H
+#define ARCH_X86_MM_DPT_H
+
+#include <linux/spinlock.h>
+
+#include <asm/pgtable.h>
+
+/*
+ * A decorated page-table (dpt) encapsulates a native page-table (e.g.
+ * a PGD) and maintain additional attributes related to this page-table.
+ */
+struct dpt {
+	spinlock_t		lock;		/* protect all attributes */
+	pgd_t			*pagetable;	/* the actual page-table */
+	unsigned int		alignment;	/* page-table alignment */
+
+};
+
+extern struct dpt *dpt_create(unsigned int pgt_alignment);
+extern void dpt_destroy(struct dpt *dpt);
+
+#endif
arch/x86/mm/Makefile +1 -1
index e57af263e870..5b52d854a030 100644
@@ -48,7 +48,7 @@ obj-$(CONFIG_NUMA_EMU)		+= numa_emulation.o
 obj-$(CONFIG_X86_INTEL_MEMORY_PROTECTION_KEYS)	+= pkeys.o
 obj-$(CONFIG_RANDOMIZE_MEMORY)			+= kaslr.o
 obj-$(CONFIG_PAGE_TABLE_ISOLATION)		+= pti.o
-obj-$(CONFIG_ADDRESS_SPACE_ISOLATION)		+= asi.o
+obj-$(CONFIG_ADDRESS_SPACE_ISOLATION)		+= asi.o dpt.o
 
 obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt.o
 obj-$(CONFIG_AMD_MEM_ENCRYPT)	+= mem_encrypt_identity.o
arch/x86/mm/dpt.c new file +67 -0

67 lines not on this page · raw message

Signature · 1 line
2.18.2
#2 lore raw

RFC PATCH v4 2/13 mm/dpt: Track buffers allocated for a decorated page-table

In reply to Alexandre Chartre

Add functions to track buffers allocated for a decorated page-table. A page-table can have direct references to the kernel page table, at different levels (PGD, P4D, PUD, PMD). When freeing a page-table, we should make sure that we free parts actually allocated for the decorated page-table, and not parts of the kernel page table referenced from the page-table. To do so, we will keep track of buffers when building the page-table.

---
 arch/x86/include/asm/dpt.h | 21 ++++++++++
 arch/x86/mm/dpt.c          | 82 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 103 insertions(+)
arch/x86/include/asm/dpt.h +21 -0
index 1da4d43d5e94..b9cba051ebf2 100644
@@ -3,9 +3,18 @@
 #define ARCH_X86_MM_DPT_H
 
 #include <linux/spinlock.h>
+#include <linux/xarray.h>
 
 #include <asm/pgtable.h>
 
+enum page_table_level {
+	PGT_LEVEL_PTE,
+	PGT_LEVEL_PMD,
+	PGT_LEVEL_PUD,
+	PGT_LEVEL_P4D,
+	PGT_LEVEL_PGD
+};
+
 /*
  * A decorated page-table (dpt) encapsulates a native page-table (e.g.
  * a PGD) and maintain additional attributes related to this page-table.
@@ -15,6 +24,18 @@ struct dpt {
 	pgd_t			*pagetable;	/* the actual page-table */
 	unsigned int		alignment;	/* page-table alignment */
 
+	/*
+	 * A page-table can have direct references to another page-table,
+	 * at different levels (PGD, P4D, PUD, PMD). When freeing or
+	 * modifying a page-table, we should make sure that we free/modify
+	 * parts effectively allocated to the actual page-table, and not
+	 * parts of another page-table referenced from this page-table.
+	 *
+	 * To do so, the backend_pages XArray is used to keep track of pages
+	 * used for this page-table.
+	 */
+	struct xarray		backend_pages;		/* page-table pages */
+	unsigned long		backend_pages_count;	/* pages count */
 };
 
 extern struct dpt *dpt_create(unsigned int pgt_alignment);
arch/x86/mm/dpt.c +82 -0

104 lines not on this page · raw message

Signature · 1 line
2.18.2
#3 lore raw

RFC PATCH v4 3/13 mm/dpt: Add decorated page-table entry offset functions

In reply to Alexandre Chartre

Add wrappers around the p4d/pud/pmd/pte offset kernel functions which ensure that page-table pointers are in the specified decorated page-table.

---
 arch/x86/mm/dpt.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 66 insertions(+)
arch/x86/mm/dpt.c +66 -0
index 6df2d4fde8ec..44aad99bc3dc 100644
@@ -82,6 +82,72 @@ static bool dpt_valid_offset(struct dpt *dpt, void *offset)
 	return valid;
 }
 
+/*
+ * dpt_pXX_offset() functions are equivalent to kernel pXX_offset()
+ * functions but, in addition, they ensure that page table pointers
+ * are in the specified decorated page table. Otherwise an error is
+ * returned.
+ */
+
+static pte_t *dpt_pte_offset(struct dpt *dpt,
+			     pmd_t *pmd, unsigned long addr)
+{
+	pte_t *pte;
+
+	pte = pte_offset_map(pmd, addr);
+	if (!dpt_valid_offset(dpt, pte)) {
+		pr_err("DPT %p: PTE %px not found\n", dpt, pte);
+		return ERR_PTR(-EINVAL);
+	}
+
+	return pte;
+}
+
+static pmd_t *dpt_pmd_offset(struct dpt *dpt,
+			     pud_t *pud, unsigned long addr)
+{
+	pmd_t *pmd;
+
+	pmd = pmd_offset(pud, addr);
+	if (!dpt_valid_offset(dpt, pmd)) {
+		pr_err("DPT %p: PMD %px not found\n", dpt, pmd);
+		return ERR_PTR(-EINVAL);
+	}
+
+	return pmd;
+}
+
+static pud_t *dpt_pud_offset(struct dpt *dpt,
+			     p4d_t *p4d, unsigned long addr)
+{
+	pud_t *pud;
+
+	pud = pud_offset(p4d, addr);
+	if (!dpt_valid_offset(dpt, pud)) {
+		pr_err("DPT %p: PUD %px not found\n", dpt, pud);
+		return ERR_PTR(-EINVAL);
+	}
+
+	return pud;
+}
+
+static p4d_t *dpt_p4d_offset(struct dpt *dpt,
+			     pgd_t *pgd, unsigned long addr)
+{
+	p4d_t *p4d;
+
+	p4d = p4d_offset(pgd, addr);
+	/*
+	 * p4d is the same has pgd if we don't have a 5-level page table.
+	 */
+	if ((p4d != (p4d_t *)pgd) && !dpt_valid_offset(dpt, p4d)) {
+		pr_err("DPT %p: P4D %px not found\n", dpt, p4d);
+		return ERR_PTR(-EINVAL);
+	}
+
+	return p4d;
+}
+
 /*
  * dpt_create - allocate a page-table and create a corresponding
  * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
#4 lore raw

RFC PATCH v4 4/13 mm/dpt: Add decorated page-table entry allocation functions

In reply to Alexandre Chartre

Add functions to allocate p4d/pud/pmd/pte pages for an decorated page-table and keep track of them.

---
 arch/x86/mm/dpt.c | 110 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 110 insertions(+)
arch/x86/mm/dpt.c +110 -0
index 44aad99bc3dc..a2f54ba00255 100644
@@ -4,6 +4,7 @@
  *
  */
 
+#include <linux/mm.h>
 #include <linux/slab.h>
 
 #include <asm/dpt.h>
@@ -148,6 +149,115 @@ static p4d_t *dpt_p4d_offset(struct dpt *dpt,
 	return p4d;
 }
 
+/*
+ * dpt_pXX_alloc() functions are equivalent to kernel pXX_alloc() functions
+ * but, in addition, they keep track of new pages allocated for the specified
+ * decorated page-table.
+ */
+
+static pte_t *dpt_pte_alloc(struct dpt *dpt, pmd_t *pmd, unsigned long addr)
+{
+	struct page *page;
+	pte_t *pte;
+	int err;
+
+	if (pmd_none(*pmd)) {
+		page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+		if (!page)
+			return ERR_PTR(-ENOMEM);
+		pte = (pte_t *)page_address(page);
+		err = dpt_add_backend_page(dpt, pte, PGT_LEVEL_PTE);
+		if (err) {
+			free_page((unsigned long)pte);
+			return ERR_PTR(err);
+		}
+		set_pmd_safe(pmd, __pmd(__pa(pte) | _KERNPG_TABLE));
+		pte = pte_offset_map(pmd, addr);
+	} else {
+		pte = dpt_pte_offset(dpt, pmd,  addr);
+	}
+
+	return pte;
+}
+
+static pmd_t *dpt_pmd_alloc(struct dpt *dpt, pud_t *pud, unsigned long addr)
+{
+	struct page *page;
+	pmd_t *pmd;
+	int err;
+
+	if (pud_none(*pud)) {
+		page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+		if (!page)
+			return ERR_PTR(-ENOMEM);
+		pmd = (pmd_t *)page_address(page);
+		err = dpt_add_backend_page(dpt, pmd, PGT_LEVEL_PMD);
+		if (err) {
+			free_page((unsigned long)pmd);
+			return ERR_PTR(err);
+		}
+		set_pud_safe(pud, __pud(__pa(pmd) | _KERNPG_TABLE));
+		pmd = pmd_offset(pud, addr);
+	} else {
+		pmd = dpt_pmd_offset(dpt, pud, addr);
+	}
+
+	return pmd;
+}
+
+static pud_t *dpt_pud_alloc(struct dpt *dpt, p4d_t *p4d, unsigned long addr)
+{
+	struct page *page;
+	pud_t *pud;
+	int err;
+
+	if (p4d_none(*p4d)) {
+		page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+		if (!page)
+			return ERR_PTR(-ENOMEM);
+		pud = (pud_t *)page_address(page);
+		err = dpt_add_backend_page(dpt, pud, PGT_LEVEL_PUD);
+		if (err) {
+			free_page((unsigned long)pud);
+			return ERR_PTR(err);
+		}
+		set_p4d_safe(p4d, __p4d(__pa(pud) | _KERNPG_TABLE));
+		pud = pud_offset(p4d, addr);
+	} else {
+		pud = dpt_pud_offset(dpt, p4d, addr);
+	}
+
+	return pud;
+}
+
+static p4d_t *dpt_p4d_alloc(struct dpt *dpt, pgd_t *pgd, unsigned long addr)
+{
+	struct page *page;
+	p4d_t *p4d;
+	int err;
+
+	if (!pgtable_l5_enabled())
+		return (p4d_t *)pgd;
+
+	if (pgd_none(*pgd)) {
+		page = alloc_page(GFP_KERNEL | __GFP_ZERO);
+		if (!page)
+			return ERR_PTR(-ENOMEM);
+		p4d = (p4d_t *)page_address(page);
+		err = dpt_add_backend_page(dpt, p4d, PGT_LEVEL_P4D);
+		if (err) {
+			free_page((unsigned long)p4d);
+			return ERR_PTR(err);
+		}
+		set_pgd_safe(pgd, __pgd(__pa(p4d) | _KERNPG_TABLE));
+		p4d = p4d_offset(pgd, addr);
+	} else {
+		p4d = dpt_p4d_offset(dpt, pgd, addr);
+	}
+
+	return p4d;
+}
+
 /*
  * dpt_create - allocate a page-table and create a corresponding
  * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
#5 lore raw

RFC PATCH v4 5/13 mm/dpt: Add decorated page-table entry set functions

In reply to Alexandre Chartre

Add wrappers around the page table entry (pgd/p4d/pud/pmd) set functions which check that an existing entry is not being overwritten.

---
 arch/x86/mm/dpt.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 126 insertions(+)
arch/x86/mm/dpt.c +126 -0
index a2f54ba00255..7a1b4cd53b03 100644
@@ -258,6 +258,132 @@ static p4d_t *dpt_p4d_alloc(struct dpt *dpt, pgd_t *pgd, unsigned long addr)
 	return p4d;
 }
 
+/*
+ * dpt_set_pXX() functions are equivalent to kernel set_pXX() functions
+ * but, in addition, they ensure that they are not overwriting an already
+ * existing reference in the decorated page table. Otherwise an error is
+ * returned.
+ */
+
+static int dpt_set_pte(struct dpt *dpt, pte_t *pte, pte_t pte_value)
+{
+#ifdef DEBUG
+	/*
+	 * The pte pointer should come from dpt_pte_alloc() or dpt_pte_offset()
+	 * both of which check if the pointer is in the decorated page table.
+	 * So this is a paranoid check to ensure the pointer is really in the
+	 * decorated page table.
+	 */
+	if (!dpt_valid_offset(dpt, pte)) {
+		pr_err("DPT %p: PTE %px not found\n", dpt, pte);
+		return -EINVAL;
+	}
+#endif
+	set_pte(pte, pte_value);
+
+	return 0;
+}
+
+static int dpt_set_pmd(struct dpt *dpt, pmd_t *pmd, pmd_t pmd_value)
+{
+#ifdef DEBUG
+	/*
+	 * The pmd pointer should come from dpt_pmd_alloc() or dpt_pmd_offset()
+	 * both of which check if the pointer is in the decorated page table.
+	 * So this is a paranoid check to ensure the pointer is really in the
+	 * decorated page table.
+	 */
+	if (!dpt_valid_offset(dpt, pmd)) {
+		pr_err("DPT %p: PMD %px not found\n", dpt, pmd);
+		return -EINVAL;
+	}
+#endif
+	if (pmd_val(*pmd) == pmd_val(pmd_value))
+		return 0;
+
+	if (!pmd_none(*pmd)) {
+		pr_err("DPT %p: PMD %px overwriting %lx with %lx\n",
+		       dpt, pmd, pmd_val(*pmd), pmd_val(pmd_value));
+		return -EBUSY;
+	}
+
+	set_pmd(pmd, pmd_value);
+
+	return 0;
+}
+
+static int dpt_set_pud(struct dpt *dpt, pud_t *pud, pud_t pud_value)
+{
+#ifdef DEBUG
+	/*
+	 * The pud pointer should come from dpt_pud_alloc() or dpt_pud_offset()
+	 * both of which check if the pointer is in the decorated page table.
+	 * So this is a paranoid check to ensure the pointer is really in the
+	 * decorated page table.
+	 */
+	if (!dpt_valid_offset(dpt, pud)) {
+		pr_err("DPT %p: PUD %px not found\n", dpt, pud);
+		return -EINVAL;
+	}
+#endif
+	if (pud_val(*pud) == pud_val(pud_value))
+		return 0;
+
+	if (!pud_none(*pud)) {
+		pr_err("DPT %p: PUD %px overwriting %lx with %lx\n",
+		       dpt, pud, pud_val(*pud), pud_val(pud_value));
+		return -EBUSY;
+	}
+
+	set_pud(pud, pud_value);
+
+	return 0;
+}
+
+static int dpt_set_p4d(struct dpt *dpt, p4d_t *p4d, p4d_t p4d_value)
+{
+#ifdef DEBUG
+	/*
+	 * The p4d pointer should come from dpt_p4d_alloc() or dpt_p4d_offset()
+	 * both of which check if the pointer is in the decorated page table.
+	 * So this is a paranoid check to ensure the pointer is really in the
+	 * decorated page table.
+	 */
+	if (!dpt_valid_offset(dpt, p4d)) {
+		pr_err("DPT %p: P4D %px not found\n", dpt, p4d);
+		return -EINVAL;
+	}
+#endif
+	if (p4d_val(*p4d) == p4d_val(p4d_value))
+		return 0;
+
+	if (!p4d_none(*p4d)) {
+		pr_err("DPT %p: P4D %px overwriting %lx with %lx\n",
+		       dpt, p4d, p4d_val(*p4d), p4d_val(p4d_value));
+		return -EBUSY;
+	}
+
+	set_p4d(p4d, p4d_value);
+
+	return 0;
+}
+
+static int dpt_set_pgd(struct dpt *dpt, pgd_t *pgd, pgd_t pgd_value)
+{
+	if (pgd_val(*pgd) == pgd_val(pgd_value))
+		return 0;
+
+	if (!pgd_none(*pgd)) {
+		pr_err("DPT %p: PGD %px overwriting %lx with %lx\n",
+		       dpt, pgd, pgd_val(*pgd), pgd_val(pgd_value));
+		return -EBUSY;
+	}
+
+	set_pgd(pgd, pgd_value);
+
+	return 0;
+}
+
 /*
  * dpt_create - allocate a page-table and create a corresponding
  * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
#6 lore raw

RFC PATCH v4 6/13 mm/dpt: Functions to populate a decorated page-table from a VA range

In reply to Alexandre Chartre

Provide functions to copy page-table entries from the kernel page-table to a decorated page-table for a specified VA range. These functions are based on the copy_pxx_range() functions defined in mm/memory.c. A first difference is that a level parameter can be specified to indicate the page-table level (PGD, P4D, PUD PMD, PTE) at which the copy should be done. Also functions don't rely on mm or vma, and they don't alter the source page-table even if an entry is bad. Finally, the VA range start and size don't need to be page-aligned.

---
 arch/x86/include/asm/dpt.h |   3 +
 arch/x86/mm/dpt.c          | 205 +++++++++++++++++++++++++++++++++++++
 2 files changed, 208 insertions(+)
arch/x86/include/asm/dpt.h +3 -0
index b9cba051ebf2..85d2c5051acb 100644
@@ -40,5 +40,8 @@ struct dpt {
 
 extern struct dpt *dpt_create(unsigned int pgt_alignment);
 extern void dpt_destroy(struct dpt *dpt);
+extern int dpt_map_range(struct dpt *dpt, void *ptr, size_t size,
+			 enum page_table_level level);
+extern int dpt_map(struct dpt *dpt, void *ptr, unsigned long size);
 
 #endif
arch/x86/mm/dpt.c +205 -0

211 lines not on this page · raw message

Signature · 1 line
2.18.2
#7 lore raw

RFC PATCH v4 7/13 mm/dpt: Helper functions to map module into a decorated page-table

In reply to Alexandre Chartre

Add helper functions to easily map a module into a decorated page-table.

---
 arch/x86/include/asm/dpt.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
arch/x86/include/asm/dpt.h +21 -0
index 85d2c5051acb..5a38d97a70a8 100644
@@ -2,6 +2,7 @@
 #ifndef ARCH_X86_MM_DPT_H
 #define ARCH_X86_MM_DPT_H
 
+#include <linux/module.h>
 #include <linux/spinlock.h>
 #include <linux/xarray.h>
 
@@ -44,4 +45,24 @@ extern int dpt_map_range(struct dpt *dpt, void *ptr, size_t size,
 			 enum page_table_level level);
 extern int dpt_map(struct dpt *dpt, void *ptr, unsigned long size);
 
+static inline int dpt_map_module(struct dpt *dpt, char *module_name)
+{
+	struct module *module;
+
+	module = find_module(module_name);
+	if (!module)
+		return -ESRCH;
+
+	return dpt_map(dpt, module->core_layout.base, module->core_layout.size);
+}
+
+/*
+ * Copy the memory mapping for the current module. This is defined as a
+ * macro to ensure it is expanded in the module making the call so that
+ * THIS_MODULE has the correct value.
+ */
+#define DPT_MAP_THIS_MODULE(dpt)			\
+	(dpt_map(dpt, THIS_MODULE->core_layout.base,	\
+		 THIS_MODULE->core_layout.size))
+
 #endif
Signature · 1 line
2.18.2
#8 lore raw

RFC PATCH v4 8/13 mm/dpt: Keep track of VA ranges mapped in a decorated page-table

In reply to Alexandre Chartre

Add functions to keep track of VA ranges mapped in a decorated page-table. This will be used when unmapping to ensure the same range is unmapped, at the same page-table level. This will also be used to handle mapping and unmapping of overlapping VA ranges.

---
 arch/x86/include/asm/dpt.h | 12 ++++++++
 arch/x86/mm/dpt.c          | 60 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 72 insertions(+)
arch/x86/include/asm/dpt.h +12 -0
index 5a38d97a70a8..0d74afb10141 100644
@@ -16,6 +16,17 @@ enum page_table_level {
 	PGT_LEVEL_PGD
 };
 
+/*
+ * Structure to keep track of address ranges mapped into a decorated
+ * page-table.
+ */
+struct dpt_range_mapping {
+	struct list_head list;
+	void *ptr;			/* range start address */
+	size_t size;			/* range size */
+	enum page_table_level level;	/* mapping level */
+};
+
 /*
  * A decorated page-table (dpt) encapsulates a native page-table (e.g.
  * a PGD) and maintain additional attributes related to this page-table.
@@ -24,6 +35,7 @@ struct dpt {
 	spinlock_t		lock;		/* protect all attributes */
 	pgd_t			*pagetable;	/* the actual page-table */
 	unsigned int		alignment;	/* page-table alignment */
+	struct list_head	mapping_list;	/* list of VA range mapping */
 
 	/*
 	 * A page-table can have direct references to another page-table,
arch/x86/mm/dpt.c +60 -0

98 lines not on this page · raw message

Signature · 1 line
2.18.2
#9 lore raw

RFC PATCH v4 9/13 mm/dpt: Functions to clear decorated page-table entries for a VA range

In reply to Alexandre Chartre

Provide functions to clear page-table entries in a decorated page-table for a specified VA range. Functions also check that the clearing effectively happens in the decorated page-table and there is no crossing of the decorated page-table boundary (through references to another page table), so that another page table is not modified by mistake.

As information (address, size, page-table level) about VA ranges mapped to the decorated page-table is tracked, clearing is done with just specifying the start address of the range.

---
 arch/x86/include/asm/dpt.h |   1 +
 arch/x86/mm/dpt.c          | 135 +++++++++++++++++++++++++++++++++++++
 2 files changed, 136 insertions(+)
arch/x86/include/asm/dpt.h +1 -0
index 0d74afb10141..01727ef0577e 100644
@@ -56,6 +56,7 @@ extern void dpt_destroy(struct dpt *dpt);
 extern int dpt_map_range(struct dpt *dpt, void *ptr, size_t size,
 			 enum page_table_level level);
 extern int dpt_map(struct dpt *dpt, void *ptr, unsigned long size);
+extern void dpt_unmap(struct dpt *dpt, void *ptr);
 
 static inline int dpt_map_module(struct dpt *dpt, char *module_name)
 {
arch/x86/mm/dpt.c +135 -0

141 lines not on this page · raw message

Signature · 1 line
2.18.2
#10 lore raw

RFC PATCH v4 10/13 mm/dpt: Function to copy page-table entries for percpu buffer

In reply to Alexandre Chartre

Provide functions to copy page-table entries from the kernel page-table to a decorated page-table for a percpu buffer. A percpu buffer have a different VA range for each cpu and all them have to be copied.

---
 arch/x86/include/asm/dpt.h |  6 ++++++
 arch/x86/mm/dpt.c          | 38 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 44 insertions(+)
arch/x86/include/asm/dpt.h +6 -0
index 01727ef0577e..fd8c1b84ffe2 100644
@@ -78,4 +78,10 @@ static inline int dpt_map_module(struct dpt *dpt, char *module_name)
 	(dpt_map(dpt, THIS_MODULE->core_layout.base,	\
 		 THIS_MODULE->core_layout.size))
 
+extern int dpt_map_percpu(struct dpt *dpt, void *percpu_ptr, size_t size);
+extern void dpt_unmap_percpu(struct dpt *dpt, void *percpu_ptr);
+
+#define	DPT_MAP_CPUVAR(dpt, cpuvar)			\
+	dpt_map_percpu(dpt, &cpuvar, sizeof(cpuvar))
+
 #endif
arch/x86/mm/dpt.c +38 -0
index c495c9b59b3e..adc59f9ed876 100644
@@ -771,6 +771,44 @@ void dpt_unmap(struct dpt *dpt, void *ptr)
 }
 EXPORT_SYMBOL(dpt_unmap);
 
+void dpt_unmap_percpu(struct dpt *dpt, void *percpu_ptr)
+{
+	void *ptr;
+	int cpu;
+
+	pr_debug("DPT %p: UNMAP PERCPU %px\n", dpt, percpu_ptr);
+	for_each_possible_cpu(cpu) {
+		ptr = per_cpu_ptr(percpu_ptr, cpu);
+		pr_debug("DPT %p: UNMAP PERCPU%d %px\n", dpt, cpu, ptr);
+		dpt_unmap(dpt, ptr);
+	}
+}
+EXPORT_SYMBOL(dpt_unmap_percpu);
+
+int dpt_map_percpu(struct dpt *dpt, void *percpu_ptr, size_t size)
+{
+	int cpu, err;
+	void *ptr;
+
+	pr_debug("DPT %p: MAP PERCPU %px\n", dpt, percpu_ptr);
+	for_each_possible_cpu(cpu) {
+		ptr = per_cpu_ptr(percpu_ptr, cpu);
+		pr_debug("DPT %p: MAP PERCPU%d %px\n", dpt, cpu, ptr);
+		err = dpt_map(dpt, ptr, size);
+		if (err) {
+			/*
+			 * Need to unmap any percpu mapping which has
+			 * succeeded before the failure.
+			 */
+			dpt_unmap_percpu(dpt, percpu_ptr);
+			return err;
+		}
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL(dpt_map_percpu);
+
 /*
  * dpt_create - allocate a page-table and create a corresponding
  * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
#11 lore raw

RFC PATCH v4 11/13 mm/dpt: Add decorated page-table remap function

In reply to Alexandre Chartre

Add a function to remap an already mapped buffer with a new address in a decorated page-table: the already mapped buffer is unmapped, and a new mapping is added for the specified new address.

This is useful to track and remap a buffer which can be freed and then reallocated.

---
 arch/x86/include/asm/dpt.h |  1 +
 arch/x86/mm/dpt.c          | 25 +++++++++++++++++++++++++
 2 files changed, 26 insertions(+)
arch/x86/include/asm/dpt.h +1 -0
index fd8c1b84ffe2..3234ba968d80 100644
@@ -57,6 +57,7 @@ extern int dpt_map_range(struct dpt *dpt, void *ptr, size_t size,
 			 enum page_table_level level);
 extern int dpt_map(struct dpt *dpt, void *ptr, unsigned long size);
 extern void dpt_unmap(struct dpt *dpt, void *ptr);
+extern int dpt_remap(struct dpt *dpt, void **mapping, void *ptr, size_t size);
 
 static inline int dpt_map_module(struct dpt *dpt, char *module_name)
 {
arch/x86/mm/dpt.c +25 -0
index adc59f9ed876..9517e3081716 100644
@@ -809,6 +809,31 @@ int dpt_map_percpu(struct dpt *dpt, void *percpu_ptr, size_t size)
 }
 EXPORT_SYMBOL(dpt_map_percpu);
 
+int dpt_remap(struct dpt *dpt, void **current_ptrp, void *new_ptr, size_t size)
+{
+	void *current_ptr = *current_ptrp;
+	int err;
+
+	if (current_ptr == new_ptr) {
+		/* no change, already mapped */
+		return 0;
+	}
+
+	if (current_ptr) {
+		dpt_unmap(dpt, current_ptr);
+		*current_ptrp = NULL;
+	}
+
+	err = dpt_map(dpt, new_ptr, size);
+	if (err)
+		return err;
+
+	*current_ptrp = new_ptr;
+
+	return 0;
+}
+EXPORT_SYMBOL(dpt_remap);
+
 /*
  * dpt_create - allocate a page-table and create a corresponding
  * decorated page-table. The page-table is allocated and aligned
Signature · 1 line
2.18.2
#12 lore raw

RFC PATCH v4 12/13 mm/dpt: Handle decorated page-table mapped range leaks and overlaps

In reply to Alexandre Chartre

When mapping a buffer in a decorated page-table, data around the buffer can also be mapped if the entire buffer is not aligned with the page directory size used for the mapping. So, data can potentially leak into the decorated page-table. In such a case, print a warning that data are leaking.

Also data effectively mapped can overlap with an already mapped buffer. This is not an issue when mapping data but, when unmapping, make sure data from another buffer don't get unmapped as a side effect.

---
 arch/x86/include/asm/dpt.h |   1 +
 arch/x86/mm/dpt.c          | 197 +++++++++++++++++++++++++++++++------
 2 files changed, 168 insertions(+), 30 deletions(-)
arch/x86/include/asm/dpt.h +1 -0
index 3234ba968d80..e0adbf69dadf 100644
@@ -25,6 +25,7 @@ struct dpt_range_mapping {
 	void *ptr;			/* range start address */
 	size_t size;			/* range size */
 	enum page_table_level level;	/* mapping level */
+	int refcnt;			/* reference count (for overlap) */
 };
 
 /*
arch/x86/mm/dpt.c +167 -30

252 lines not on this page · raw message

Signature · 1 line
2.18.2
#13 lore raw

RFC PATCH v4 13/13 mm/asi: Function to init decorated page-table with ASI core mappings

In reply to Alexandre Chartre

Core mappings are the minimal mappings we need to be able to enter isolation and handle an isolation abort or exit. This includes the kernel code, the GDT and the percpu ASI sessions. We also need a stack so we map the current stack when entering isolation and unmap it on exit/abort.

---
 arch/x86/include/asm/asi.h |  2 ++
 arch/x86/mm/asi.c          | 57 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 59 insertions(+)
arch/x86/include/asm/asi.h +2 -0
index ac0594d4f549..eafed750e07f 100644
@@ -45,6 +45,7 @@
 #include <linux/export.h>
 
 #include <asm/asi_session.h>
+#include <asm/dpt.h>
 
 /*
  * ASI_NR_DYN_ASIDS is the same as TLB_NR_DYN_ASIDS. We can't directly
@@ -150,6 +151,7 @@ extern void asi_destroy(struct asi *asi);
 extern void asi_set_pagetable(struct asi *asi, pgd_t *pagetable);
 extern int asi_enter(struct asi *asi);
 extern void asi_exit(struct asi *asi);
+extern int asi_init_dpt(struct dpt *dpt);
 
 #ifdef CONFIG_PAGE_TABLE_ISOLATION
 DECLARE_ASI_TYPE(user);
arch/x86/mm/asi.c +57 -0

63 lines not on this page · raw message

Signature · 1 line
2.18.2
#14 lore raw
In reply to Alexandre Chartre

Hello Alexandre,

Quoted 10 lines · Alexandre Chartre

This is part II of ASI RFC v4. Please refer to the cover letter of part I for an overview the ASI RFC.

https://lore.kernel.org/lkml/[email protected]/

This part introduces decorated page-table which encapsulate native page table (e.g. a PGD) in order to provide convenient page-table management functions, such as tracking address range mapped in a page-table or safely handling references to another page-table.

Quoted 6 lines

Decorated page-table can then be used to easily create and manage page tables to be used with ASI. It will be used by the ASI test driver (see part III) and later by KVM ASI.

Decorated page-table is independent of ASI, and can potentially be used anywhere a page-table is needed.

This is very impressive work!

I wonder why did you decide to make dpt x86-specific? Unless I've missed simething, the dpt implementation does not rely on anything architecture specific and can go straight to linux/mm.

Another thing that comes to mind is that we already have a very decorated page table, which is mm_struct. I admit that my attempt to split out the core page table bits from the mm_struct [1] didn't went far, but I still think we need a first class abstraction for the page table that will be used by both user memory management and the management of the reduced kernel address spaces.

[1] https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=pg_table/v0.0

Quoted 33 lines

Thanks,

alex.

-----

Alexandre Chartre (13):
mm/x86: Introduce decorated page-table (dpt)
mm/dpt: Track buffers allocated for a decorated page-table mm/dpt: Add decorated page-table entry offset functions mm/dpt: Add decorated page-table entry allocation functions mm/dpt: Add decorated page-table entry set functions mm/dpt: Functions to populate a decorated page-table from a VA range mm/dpt: Helper functions to map module into a decorated page-table mm/dpt: Keep track of VA ranges mapped in a decorated page-table mm/dpt: Functions to clear decorated page-table entries for a VA range mm/dpt: Function to copy page-table entries for percpu buffer mm/dpt: Add decorated page-table remap function
mm/dpt: Handle decorated page-table mapped range leaks and overlaps mm/asi: Function to init decorated page-table with ASI core mappings

 arch/x86/include/asm/asi.h |    2 +
 arch/x86/include/asm/dpt.h |   89 +++
 arch/x86/mm/Makefile       |    2 +-
 arch/x86/mm/asi.c          |   57 ++
 arch/x86/mm/dpt.c          | 1051 ++++++++++++++++++++++++++++++++++++
 5 files changed, 1200 insertions(+), 1 deletion(-)
 create mode 100644 arch/x86/include/asm/dpt.h
 create mode 100644 arch/x86/mm/dpt.c
Signature · 1 line
2.18.2
Signature · 2 lines
Sincerely yours,
Mike.
#15 lore raw
Quoted 19 lines · Mike Rapoport

Hello Alexandre,

Quoted 16 lines · Alexandre Chartre

This is part II of ASI RFC v4. Please refer to the cover letter of part I for an overview the ASI RFC.

   https://lore.kernel.org/lkml/[email protected]/

This part introduces decorated page-table which encapsulate native page table (e.g. a PGD) in order to provide convenient page-table management functions, such as tracking address range mapped in a page-table or safely handling references to another page-table.

Decorated page-table can then be used to easily create and manage page tables to be used with ASI. It will be used by the ASI test driver (see part III) and later by KVM ASI.

Decorated page-table is independent of ASI, and can potentially be used anywhere a page-table is needed.

Hi Mike,

Quoted 5 lines

This is very impressive work!

I wonder why did you decide to make dpt x86-specific? Unless I've missed simething, the dpt implementation does not rely on anything architecture specific and can go straight to linux/mm.

Correct, this is not x86 specific. I put it in arch/x86 because that's currently the only place were I use it, but it can be moved to linux/mm.

Quoted 2 lines

Another thing that comes to mind is that we already have a very decorated page table, which is mm_struct.

mm_struct doesn't define a generic page-table encapsulation. mm_struct references a page table (i.e. PGD) and adds all kind of attributes needed for mm management but not necessarily related to page-table.

Quoted 5 lines

I admit that my attempt to
split out the core page table bits from the mm_struct [1] didn't went far, but I still think we need a first class abstraction for the page table that will be used by both user memory management and the management of the reduced kernel address spaces.

Agree. I remember your attempt to extract the page-table from mm_struct; this is not a simple work! For ASI, I didn't need mm, so it was simpler to built a simple decorated page-table without attempting to use with mm (at least for now).

Thanks,

alex.

PS: if you want to play with dpt, there's a bug in dpt_destroy(), patch 08 adds a a double free of dpt->backend_pages pages.

Quoted 37 lines

[1] https://git.kernel.org/pub/scm/linux/kernel/git/rppt/linux.git/log/?h=pg_table/v0.0

Quoted 33 lines

Thanks,

alex.

-----

Alexandre Chartre (13):
mm/x86: Introduce decorated page-table (dpt)
mm/dpt: Track buffers allocated for a decorated page-table mm/dpt: Add decorated page-table entry offset functions mm/dpt: Add decorated page-table entry allocation functions mm/dpt: Add decorated page-table entry set functions mm/dpt: Functions to populate a decorated page-table from a VA range mm/dpt: Helper functions to map module into a decorated page-table mm/dpt: Keep track of VA ranges mapped in a decorated page-table mm/dpt: Functions to clear decorated page-table entries for a VA range mm/dpt: Function to copy page-table entries for percpu buffer mm/dpt: Add decorated page-table remap function
mm/dpt: Handle decorated page-table mapped range leaks and overlaps mm/asi: Function to init decorated page-table with ASI core mappings

  arch/x86/include/asm/asi.h |    2 +
  arch/x86/include/asm/dpt.h |   89 +++
  arch/x86/mm/Makefile       |    2 +-
  arch/x86/mm/asi.c          |   57 ++
  arch/x86/mm/dpt.c          | 1051 ++++++++++++++++++++++++++++++++++++
  5 files changed, 1200 insertions(+), 1 deletion(-)

create mode 100644 arch/x86/include/asm/dpt.h
create mode 100644 arch/x86/mm/dpt.c

Signature · 1 line
2.18.2