Skip to contentSkip to conversation

Prepare memregion handling for future isolation primitives

PATCH New Gregor Haas 2 participants 13 messages Jul 31, 2024 – Sep 20, 2024 1 message not in the archive Sophgo devicetrees and drivers

lore
Share & tools
lore
#0 lore raw

cover Prepare memregion handling for future isolation primitives

The existing memregion handling code deeply embeds the assumption that we will be isolating domains using PMP/SMEPMP rather than another isolation primitive. I am currently working on a series of patches that will enable SMMTT [1] support in both OpenSBI and QEMU. SMMTT is a much more granular, table-based isolation primitive which does not have the same alignment/sizing requirements as PMP.

This patch series lays some of the groundwork for landing SMMTT support in OpenSBI. Perhaps the most significant change here is that memregions now keep track of their raw sizes (rather than power-of-two rounded orders) throughout most of their lifetime. Then, when OpenSBI is about to apply its isolation configuration at the end of the coldboot/warmboot paths, the memregions for each domain are "sanitized" according to isolation primitive-specific rules.

The first two patches in this series are optional to apply, although I've found them useful when developing SMMTT. The first moves the memregion-related functionality out of sbi_domain.c into a new sbi_memregion.c. The second renames struct sbi_domain_memregion to struct sbi_memregion, while also renaming all of the memregion permission macros similarly. This makes the code somewhat more readable, especially since domain-handling and memregion-handling become more decoupled throughout this patch series.

[1] https://github.com/riscv/riscv-smmtt

Gregor Haas (8):
lib: sbi: Separate domain-handling code from memregion-handling code
lib: sbi: Create sbi_domain_dump_memregions()
lib: sbi: Rename domain_memregion -> memregion
lib: sbi: memregion: Introduce memregion_start/end macros
lib: sbi: memregion: Make memregions keep track of raw size rather
than order
lib: sbi: Do a merging pass on memregions whenever a domain is
sanitized
lib: sbi: Remove PMP assumptions from memregion
lib: sbi: memregion: Move memregion_sanitize_pmp() closer to its call
site

 docs/domain_support.md                   |  45 ++-
 include/sbi/sbi_domain.h                 | 192 +---------
 include/sbi/sbi_hart.h                   |   2 +-
 include/sbi/sbi_memregion.h              | 215 +++++++++++
 include/sbi/sbi_platform.h               |   2 +-
 lib/sbi/objects.mk                       |   1 +
 lib/sbi/sbi_domain.c                     | 444 +++-------------------
 lib/sbi/sbi_domain_context.c             |   2 +-
 lib/sbi/sbi_hart.c                       |  91 +++--
 lib/sbi/sbi_init.c                       |   6 +-
 lib/sbi/sbi_memregion.c                  | 459 +++++++++++++++++++++++
 lib/utils/fdt/fdt_domain.c               |  53 +--
 lib/utils/fdt/fdt_fixup.c                |  20 +-
 lib/utils/ipi/aclint_mswi.c              |  12 +-
 lib/utils/ipi/andes_plicsw.c             |   6 +-
 lib/utils/irqchip/aplic.c                |  12 +-
 lib/utils/irqchip/imsic.c                |  13 +-
 lib/utils/irqchip/plic.c                 |   4 +-
 lib/utils/regmap/fdt_regmap_syscon.c     |   4 +-
 lib/utils/serial/cadence-uart.c          |   4 +-
 lib/utils/serial/fdt_serial_htif.c       |   4 +-
 lib/utils/serial/uart8250.c              |   4 +-
 lib/utils/timer/aclint_mtimer.c          |  24 +-
 lib/utils/timer/andes_plmt.c             |   6 +-
 platform/generic/renesas/rzfive/rzfive.c |   2 +-
 platform/generic/sophgo/sg2042.c         |   6 +-
 26 files changed, 910 insertions(+), 723 deletions(-)
 create mode 100644 include/sbi/sbi_memregion.h
 create mode 100644 lib/sbi/sbi_memregion.c
Signature · 1 line
2.45.2
#1 lore raw

PATCH 1/8 lib: sbi: Separate domain-handling code from memregion-handling code

The next changes in this patch series will add a large amount of functionality to the domain memregion abstraction. Split out memregions from domains, since sbi_domain.c is already almost 900 lines long. This commit verbatim moves code without modifying it, except to add new includes and integrate sbi_memregion.c into the build system.

---
 include/sbi/sbi_domain.h    | 184 +---------------------
 include/sbi/sbi_memregion.h | 199 ++++++++++++++++++++++++
 lib/sbi/objects.mk          |   1 +
 lib/sbi/sbi_domain.c        | 296 +----------------------------------
 lib/sbi/sbi_memregion.c     | 301 ++++++++++++++++++++++++++++++++++++
 5 files changed, 507 insertions(+), 474 deletions(-)
 create mode 100644 include/sbi/sbi_memregion.h
 create mode 100644 lib/sbi/sbi_memregion.c
include/sbi/sbi_domain.h +1 -183
index a6e99c6..596bab3 100644
@@ -14,150 +14,10 @@
 #include <sbi/sbi_types.h>
 #include <sbi/sbi_hartmask.h>
 #include <sbi/sbi_domain_context.h>
+#include <sbi/sbi_memregion.h>
 
 struct sbi_scratch;
 
-/** Domain access types */
-enum sbi_domain_access {
-	SBI_DOMAIN_READ = (1UL << 0),
-	SBI_DOMAIN_WRITE = (1UL << 1),
-	SBI_DOMAIN_EXECUTE = (1UL << 2),
-	SBI_DOMAIN_MMIO = (1UL << 3)
-};
-
-/** Representation of OpenSBI domain memory region */
-struct sbi_domain_memregion {
-	/**
-	 * Size of memory region as power of 2
-	 * It has to be minimum 3 and maximum __riscv_xlen
-	 */
-	unsigned long order;
-	/**
-	 * Base address of memory region
-	 * It must be 2^order aligned address
-	 */
-	unsigned long base;
-	/** Flags representing memory region attributes */
-#define SBI_DOMAIN_MEMREGION_M_READABLE		(1UL << 0)
-#define SBI_DOMAIN_MEMREGION_M_WRITABLE		(1UL << 1)
-#define SBI_DOMAIN_MEMREGION_M_EXECUTABLE	(1UL << 2)
-#define SBI_DOMAIN_MEMREGION_SU_READABLE	(1UL << 3)
-#define SBI_DOMAIN_MEMREGION_SU_WRITABLE	(1UL << 4)
-#define SBI_DOMAIN_MEMREGION_SU_EXECUTABLE	(1UL << 5)
-
-#define SBI_DOMAIN_MEMREGION_ACCESS_MASK	(0x3fUL)
-#define SBI_DOMAIN_MEMREGION_M_ACCESS_MASK	(0x7UL)
-#define SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK	(0x38UL)
-
-#define SBI_DOMAIN_MEMREGION_SU_ACCESS_SHIFT	(3)
-
-#define SBI_DOMAIN_MEMREGION_SHARED_RDONLY		\
-		(SBI_DOMAIN_MEMREGION_M_READABLE |	\
-		 SBI_DOMAIN_MEMREGION_SU_READABLE)
-
-#define SBI_DOMAIN_MEMREGION_SHARED_SUX_MRX		\
-		(SBI_DOMAIN_MEMREGION_M_READABLE   |	\
-		 SBI_DOMAIN_MEMREGION_M_EXECUTABLE |	\
-		 SBI_DOMAIN_MEMREGION_SU_EXECUTABLE)
-
-#define SBI_DOMAIN_MEMREGION_SHARED_SUX_MX		\
-		(SBI_DOMAIN_MEMREGION_M_EXECUTABLE |	\
-		 SBI_DOMAIN_MEMREGION_SU_EXECUTABLE)
-
-#define SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW		\
-		(SBI_DOMAIN_MEMREGION_M_READABLE |	\
-		 SBI_DOMAIN_MEMREGION_M_WRITABLE |	\
-		 SBI_DOMAIN_MEMREGION_SU_READABLE|	\
-		 SBI_DOMAIN_MEMREGION_SU_WRITABLE)
-
-#define SBI_DOMAIN_MEMREGION_SHARED_SUR_MRW		\
-		(SBI_DOMAIN_MEMREGION_M_READABLE |	\
-		 SBI_DOMAIN_MEMREGION_M_WRITABLE |	\
-		 SBI_DOMAIN_MEMREGION_SU_READABLE)
-
-	/* Shared read-only region between M and SU mode */
-#define SBI_DOMAIN_MEMREGION_IS_SUR_MR(__flags)			 \
-		((__flags & SBI_DOMAIN_MEMREGION_ACCESS_MASK) == \
-		 SBI_DOMAIN_MEMREGION_SHARED_RDONLY)
-
-	/* Shared region: SU execute-only and M read/execute */
-#define SBI_DOMAIN_MEMREGION_IS_SUX_MRX(__flags)		 \
-		((__flags & SBI_DOMAIN_MEMREGION_ACCESS_MASK) == \
-		 SBI_DOMAIN_MEMREGION_SHARED_SUX_MRX)
-
-	/* Shared region: SU and M execute-only */
-#define SBI_DOMAIN_MEMREGION_IS_SUX_MX(__flags)			 \
-		((__flags & SBI_DOMAIN_MEMREGION_ACCESS_MASK) == \
-		 SBI_DOMAIN_MEMREGION_SHARED_SUX_MX)
-
-	/* Shared region: SU and M read/write */
-#define SBI_DOMAIN_MEMREGION_IS_SURW_MRW(__flags)		 \
-		((__flags & SBI_DOMAIN_MEMREGION_ACCESS_MASK) == \
-		 SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW)
-
-	/* Shared region: SU read-only and M read/write */
-#define SBI_DOMAIN_MEMREGION_IS_SUR_MRW(__flags)		 \
-		((__flags & SBI_DOMAIN_MEMREGION_ACCESS_MASK) == \
-		 SBI_DOMAIN_MEMREGION_SHARED_SUR_MRW)
-
-	/*
-	 * Check if region flags match with any of the above
-	 * mentioned shared region type
-	 */
-#define SBI_DOMAIN_MEMREGION_IS_SHARED(_flags)			\
-		(SBI_DOMAIN_MEMREGION_IS_SUR_MR(_flags)  ||	\
-		 SBI_DOMAIN_MEMREGION_IS_SUX_MRX(_flags) ||	\
-		 SBI_DOMAIN_MEMREGION_IS_SUX_MX(_flags)  ||	\
-		 SBI_DOMAIN_MEMREGION_IS_SURW_MRW(_flags)||	\
-		 SBI_DOMAIN_MEMREGION_IS_SUR_MRW(_flags))
-
-#define SBI_DOMAIN_MEMREGION_M_ONLY_ACCESS(__flags)			\
-		((__flags & SBI_DOMAIN_MEMREGION_M_ACCESS_MASK) &&	\
-		 !(__flags & SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK))
-
-#define SBI_DOMAIN_MEMREGION_SU_ONLY_ACCESS(__flags)			\
-		((__flags & SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK)  &&	\
-		 !(__flags & SBI_DOMAIN_MEMREGION_M_ACCESS_MASK))
-
-/** Bit to control if permissions are enforced on all modes */
-#define SBI_DOMAIN_MEMREGION_ENF_PERMISSIONS	(1UL << 6)
-
-#define SBI_DOMAIN_MEMREGION_M_RWX		\
-				(SBI_DOMAIN_MEMREGION_M_READABLE | \
-				 SBI_DOMAIN_MEMREGION_M_WRITABLE | \
-				 SBI_DOMAIN_MEMREGION_M_EXECUTABLE)
-
-#define SBI_DOMAIN_MEMREGION_SU_RWX		\
-				(SBI_DOMAIN_MEMREGION_SU_READABLE | \
-				 SBI_DOMAIN_MEMREGION_SU_WRITABLE | \
-				 SBI_DOMAIN_MEMREGION_SU_EXECUTABLE)
-
-/* Unrestricted M-mode accesses but enfoced on SU-mode */
-#define SBI_DOMAIN_MEMREGION_READABLE		\
-				(SBI_DOMAIN_MEMREGION_SU_READABLE | \
-				 SBI_DOMAIN_MEMREGION_M_RWX)
-#define SBI_DOMAIN_MEMREGION_WRITEABLE		\
-				(SBI_DOMAIN_MEMREGION_SU_WRITABLE | \
-				 SBI_DOMAIN_MEMREGION_M_RWX)
-#define SBI_DOMAIN_MEMREGION_EXECUTABLE		\
-				(SBI_DOMAIN_MEMREGION_SU_EXECUTABLE | \
-				 SBI_DOMAIN_MEMREGION_M_RWX)
-
-/* Enforced accesses across all modes */
-#define SBI_DOMAIN_MEMREGION_ENF_READABLE	\
-				(SBI_DOMAIN_MEMREGION_SU_READABLE | \
-				 SBI_DOMAIN_MEMREGION_M_READABLE)
-#define SBI_DOMAIN_MEMREGION_ENF_WRITABLE	\
-				(SBI_DOMAIN_MEMREGION_SU_WRITABLE | \
-				 SBI_DOMAIN_MEMREGION_M_WRITABLE)
-#define SBI_DOMAIN_MEMREGION_ENF_EXECUTABLE	\
-				(SBI_DOMAIN_MEMREGION_SU_EXECUTABLE | \
-				 SBI_DOMAIN_MEMREGION_M_EXECUTABLE)
-
-#define SBI_DOMAIN_MEMREGION_MMIO		(1UL << 31)
-	unsigned long flags;
-};
-
 /** Maximum number of domains */
 #define SBI_DOMAIN_MAX_INDEX			32
 
@@ -246,48 +106,6 @@ bool sbi_domain_is_assigned_hart(const struct sbi_domain *dom, u32 hartid);
 ulong sbi_domain_get_assigned_hartmask(const struct sbi_domain *dom,
 				       ulong hbase);
 
-/**
- * Initialize a domain memory region based on it's physical
- * address and size.
- *
- * @param addr start physical address of memory region
- * @param size physical size of memory region
- * @param flags memory region flags
- * @param reg pointer to memory region being initialized
- */
-void sbi_domain_memregion_init(unsigned long addr,
-				unsigned long size,
-				unsigned long flags,
-				struct sbi_domain_memregion *reg);
-
-/**
- * Check whether we can access specified address for given mode and
- * memory region flags under a domain
- * @param dom pointer to domain
- * @param addr the address to be checked
- * @param mode the privilege mode of access
- * @param access_flags bitmask of domain access types (enum sbi_domain_access)
- * @return true if access allowed otherwise false
- */
-bool sbi_domain_check_addr(const struct sbi_domain *dom,
-			   unsigned long addr, unsigned long mode,
-			   unsigned long access_flags);
-
-/**
- * Check whether we can access specified address range for given mode and
- * memory region flags under a domain
- * @param dom pointer to domain
- * @param addr the start of the address range to be checked
- * @param size the size of the address range to be checked
- * @param mode the privilege mode of access
- * @param access_flags bitmask of domain access types (enum sbi_domain_access)
- * @return TRUE if access allowed otherwise FALSE
- */
-bool sbi_domain_check_addr_range(const struct sbi_domain *dom,
-				 unsigned long addr, unsigned long size,
-				 unsigned long mode,
-				 unsigned long access_flags);
-
 /** Dump domain details on the console */
 void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix);
 
include/sbi/sbi_memregion.h new file +199 -0

200 lines not on this page · raw message

lib/sbi/objects.mk +1 -0
index 535aa70..d80d6cc 100644
@@ -66,6 +66,7 @@ libsbi-objs-y += sbi_bitops.o
 libsbi-objs-y += sbi_console.o
 libsbi-objs-y += sbi_domain_context.o
 libsbi-objs-y += sbi_domain.o
+libsbi-objs-y += sbi_memregion.o
 libsbi-objs-y += sbi_emulate_csr.o
 libsbi-objs-y += sbi_fifo.o
 libsbi-objs-y += sbi_fwft.o
lib/sbi/sbi_domain.c +5 -291

323 lines not on this page · raw message

lib/sbi/sbi_memregion.c new file +301 -0

302 lines not on this page · raw message

Signature · 1 line
2.45.2
#2 lore raw

PATCH 2/8 lib: sbi: Create sbi_domain_dump_memregions()

In reply to Gregor Haas

Move the memregion-dumping code out of sbi_domain.c and into sbi_memregion.c

---
 include/sbi/sbi_memregion.h |  3 +++
 lib/sbi/sbi_domain.c        | 37 +--------------------------------
 lib/sbi/sbi_memregion.c     | 41 +++++++++++++++++++++++++++++++++++++
 3 files changed, 45 insertions(+), 36 deletions(-)
include/sbi/sbi_memregion.h +3 -0
index 8a62fc0..828d556 100644
@@ -196,4 +196,7 @@ bool sbi_domain_check_addr_range(const struct sbi_domain *dom,
 				 unsigned long mode,
 				 unsigned long access_flags);
 
+/** Dump domain memregion details on the console */
+void sbi_domain_dump_memregions(const struct sbi_domain *dom, const char *suffix);
+
 #endif // __SBI_MEMREGION_H__
\ No newline at end of file
lib/sbi/sbi_domain.c +1 -36
index 00ba870..868704e 100644
@@ -155,8 +155,6 @@ static int sanitize_domain(struct sbi_domain *dom)
 void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix)
 {
 	u32 i, j, k;
-	unsigned long rstart, rend;
-	struct sbi_domain_memregion *reg;
 
 	sbi_printf("Domain%d Name        %s: %s\n",
 		   dom->index, suffix, dom->name);
@@ -173,40 +171,7 @@ void sbi_domain_dump(const struct sbi_domain *dom, const char *suffix)
 	}
 	sbi_printf("\n");
 
-	i = 0;
-	sbi_domain_for_each_memregion(dom, reg) {
-		rstart = reg->base;
-		rend = (reg->order < __riscv_xlen) ?
-			rstart + ((1UL << reg->order) - 1) : -1UL;
-
-		sbi_printf("Domain%d Region%02d    %s: 0x%" PRILX "-0x%" PRILX " ",
-			   dom->index, i, suffix, rstart, rend);
-
-		k = 0;
-
-		sbi_printf("M: ");
-		if (reg->flags & SBI_DOMAIN_MEMREGION_MMIO)
-			sbi_printf("%cI", (k++) ? ',' : '(');
-		if (reg->flags & SBI_DOMAIN_MEMREGION_M_READABLE)
-			sbi_printf("%cR", (k++) ? ',' : '(');
-		if (reg->flags & SBI_DOMAIN_MEMREGION_M_WRITABLE)
-			sbi_printf("%cW", (k++) ? ',' : '(');
-		if (reg->flags & SBI_DOMAIN_MEMREGION_M_EXECUTABLE)
-			sbi_printf("%cX", (k++) ? ',' : '(');
-		sbi_printf("%s ", (k++) ? ")" : "()");
-
-		k = 0;
-		sbi_printf("S/U: ");
-		if (reg->flags & SBI_DOMAIN_MEMREGION_SU_READABLE)
-			sbi_printf("%cR", (k++) ? ',' : '(');
-		if (reg->flags & SBI_DOMAIN_MEMREGION_SU_WRITABLE)
-			sbi_printf("%cW", (k++) ? ',' : '(');
-		if (reg->flags & SBI_DOMAIN_MEMREGION_SU_EXECUTABLE)
-			sbi_printf("%cX", (k++) ? ',' : '(');
-		sbi_printf("%s\n", (k++) ? ")" : "()");
-
-		i++;
-	}
+	sbi_domain_dump_memregions(dom, suffix);
 
 	sbi_printf("Domain%d Next Address%s: 0x%" PRILX "\n",
 		   dom->index, suffix, dom->next_addr);
lib/sbi/sbi_memregion.c +41 -0

46 lines not on this page · raw message

Signature · 1 line
2.45.2
#3 lore raw

PATCH 3/8 lib: sbi: Rename domain_memregion -> memregion

In reply to Gregor Haas

Now that domains and memregions are separated into their own source files, rename all instances of struct sbi_domain_memregion to just struct sbi_memregion. Also, rename all of the memregion permission defined from SBI_DOMAIN_MEMREGION_* to SBI_MEMREGION_*. This makes the code a little less clunky and verbose.

---
 docs/domain_support.md                   |   4 +-
 include/sbi/sbi_domain.h                 |   4 +-
 include/sbi/sbi_memregion.h              | 196 +++++++++++------------
 include/sbi/sbi_platform.h               |   2 +-
 lib/sbi/sbi_domain.c                     |  40 ++---
 lib/sbi/sbi_hart.c                       |  52 +++---
 lib/sbi/sbi_memregion.c                  |  76 ++++-----
 lib/utils/fdt/fdt_domain.c               |  24 +--
 lib/utils/fdt/fdt_fixup.c                |  10 +-
 lib/utils/ipi/aclint_mswi.c              |  12 +-
 lib/utils/ipi/andes_plicsw.c             |   6 +-
 lib/utils/irqchip/aplic.c                |  12 +-
 lib/utils/irqchip/imsic.c                |  13 +-
 lib/utils/irqchip/plic.c                 |   4 +-
 lib/utils/regmap/fdt_regmap_syscon.c     |   4 +-
 lib/utils/serial/cadence-uart.c          |   4 +-
 lib/utils/serial/fdt_serial_htif.c       |   4 +-
 lib/utils/serial/uart8250.c              |   4 +-
 lib/utils/timer/aclint_mtimer.c          |  24 +--
 lib/utils/timer/andes_plmt.c             |   6 +-
 platform/generic/renesas/rzfive/rzfive.c |   2 +-
 platform/generic/sophgo/sg2042.c         |   6 +-
 22 files changed, 254 insertions(+), 255 deletions(-)
docs/domain_support.md +2 -2
index b34e43a..91677d3 100644
@@ -8,7 +8,7 @@ features such as PMP, ePMP, IOPMP, SiFive Shield, etc.
 
 Important entities which help implement OpenSBI domain support are:
 
-* **struct sbi_domain_memregion** - Representation of a domain memory region
+* **struct sbi_memregion** - Representation of a domain memory region
 * **struct sbi_hartmask** - Representation of domain HART set
 * **struct sbi_domain** - Representation of a domain instance
 
@@ -21,7 +21,7 @@ it is not mandatory for the OpenSBI platform support to populate domains.
 Domain Memory Region
 --------------------
 
-A domain memory region is represented by **struct sbi_domain_memregion** in
+A domain memory region is represented by **struct sbi_memregion** in
 OpenSBI and has following details:
 
 * **order** - The size of a memory region is **2 ^ order** where **order**
include/sbi/sbi_domain.h +2 -2
index 596bab3..a712504 100644
@@ -43,7 +43,7 @@ struct sbi_domain {
 	/** Contexts for possible HARTs indexed by hartindex */
 	struct sbi_context *hartindex_to_context_table[SBI_HARTMASK_MAX_BITS];
 	/** Array of memory regions terminated by a region with order zero */
-	struct sbi_domain_memregion *regions;
+	struct sbi_memregion *regions;
 	/** HART id of the HART booting this domain */
 	u32 boot_hartid;
 	/** Arg1 (or 'a1' register) of next booting stage for this domain */
@@ -130,7 +130,7 @@ int sbi_domain_register(struct sbi_domain *dom,
  * @return SBI_EALREADY if memory region conflicts with the existing one
  * @return SBI_EINVAL otherwise
  */
-int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg);
+int sbi_domain_root_add_memregion(const struct sbi_memregion *reg);
 
 /**
  * Add a memory range with its flags to the root domain
include/sbi/sbi_memregion.h +98 -98

245 lines not on this page · raw message

include/sbi/sbi_platform.h +1 -1
index 7b3ac4b..1814dbd 100644
@@ -50,7 +50,7 @@
 #include <sbi/sbi_version.h>
 #include <sbi/sbi_trap_ldst.h>
 
-struct sbi_domain_memregion;
+struct sbi_memregion;
 struct sbi_ecall_return;
 struct sbi_trap_regs;
 struct sbi_hart_features;
lib/sbi/sbi_domain.c +20 -20
index 868704e..22d5d8b 100644
@@ -115,7 +115,7 @@ static int sanitize_domain(struct sbi_domain *dom)
 		}
 	}
 
-	rc = sbi_domain_memregions_sanitize(dom);
+	rc = sbi_memregion_sanitize(dom);
 	if (rc) {
 		sbi_printf("%s: %s has unsanitizable regions\n",
 			   __func__, dom->name);
@@ -287,11 +287,11 @@ int sbi_domain_register(struct sbi_domain *dom,
 	return 0;
 }
 
-int sbi_domain_root_add_memregion(const struct sbi_domain_memregion *reg)
+int sbi_domain_root_add_memregion(const struct sbi_memregion *reg)
 {
 	int rc;
 	bool reg_merged;
-	struct sbi_domain_memregion *nreg, *nreg1, *nreg2;
+	struct sbi_memregion *nreg, *nreg1, *nreg2;
 
 	/* Sanity checks */
 	if (!reg || domain_finalized || !root.regions ||
@@ -346,7 +346,7 @@ int sbi_domain_root_add_memrange(unsigned long addr, unsigned long size,
 {
 	int rc;
 	unsigned long pos, end, rsize;
-	struct sbi_domain_memregion reg;
+	struct sbi_memregion reg;
 
 	pos = addr;
 	end = addr + size;
@@ -358,7 +358,7 @@ int sbi_domain_root_add_memrange(unsigned long addr, unsigned long size,
 			rsize = ((end - pos) < align) ?
 				(end - pos) : align;
 
-		sbi_domain_memregion_init(pos, rsize, region_flags, &reg);
+		sbi_memregion_init(pos, rsize, region_flags, &reg);
 		rc = sbi_domain_root_add_memregion(&reg);
 		if (rc)
 			return rc;
@@ -441,7 +441,7 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid)
 	u32 i;
 	int rc;
 	struct sbi_hartmask *root_hmask;
-	struct sbi_domain_memregion *root_memregs;
+	struct sbi_memregion *root_memregs;
 	const struct sbi_platform *plat = sbi_platform_ptr(scratch);
 
 	if (scratch->fw_rw_offset == 0 ||
@@ -478,16 +478,16 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid)
 	root.possible_harts = root_hmask;
 
 	/* Root domain firmware memory region */
-	sbi_domain_memregion_init(scratch->fw_start, scratch->fw_rw_offset,
-				  (SBI_DOMAIN_MEMREGION_M_READABLE |
-				   SBI_DOMAIN_MEMREGION_M_EXECUTABLE),
-				  &root_memregs[root_memregs_count++]);
+	sbi_memregion_init(scratch->fw_start, scratch->fw_rw_offset,
+			   (SBI_MEMREGION_M_READABLE |
+			    SBI_MEMREGION_M_EXECUTABLE),
+			   &root_memregs[root_memregs_count++]);
 
-	sbi_domain_memregion_init((scratch->fw_start + scratch->fw_rw_offset),
-				  (scratch->fw_size - scratch->fw_rw_offset),
-				  (SBI_DOMAIN_MEMREGION_M_READABLE |
-				   SBI_DOMAIN_MEMREGION_M_WRITABLE),
-				  &root_memregs[root_memregs_count++]);
+	sbi_memregion_init((scratch->fw_start + scratch->fw_rw_offset),
+			   (scratch->fw_size - scratch->fw_rw_offset),
+			   (SBI_MEMREGION_M_READABLE |
+			    SBI_MEMREGION_M_WRITABLE),
+			   &root_memregs[root_memregs_count++]);
 
 	root.fw_region_inited = true;
 
@@ -498,11 +498,11 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid)
 	 * have access to SU region while previous entries will allow
 	 * access to M-mode regions.
 	 */
-	sbi_domain_memregion_init(0, ~0UL,
-				  (SBI_DOMAIN_MEMREGION_SU_READABLE |
-				   SBI_DOMAIN_MEMREGION_SU_WRITABLE |
-				   SBI_DOMAIN_MEMREGION_SU_EXECUTABLE),
-				  &root_memregs[root_memregs_count++]);
+	sbi_memregion_init(0, ~0UL,
+			   (SBI_MEMREGION_SU_READABLE |
+			    SBI_MEMREGION_SU_WRITABLE |
+			    SBI_MEMREGION_SU_EXECUTABLE),
+			   &root_memregs[root_memregs_count++]);
 
 	/* Root domain memory region end */
 	root_memregs[root_memregs_count].order = 0;
lib/sbi/sbi_hart.c +26 -26

131 lines not on this page · raw message

lib/sbi/sbi_memregion.c +38 -38

205 lines not on this page · raw message

lib/utils/fdt/fdt_domain.c +12 -12
index 3623c6b..779acec 100644
@@ -103,9 +103,9 @@ static int __fixup_find_domain_offset(void *fdt, int doff, void *p)
 	return 0;
 }
 
-#define DISABLE_DEVICES_MASK	(SBI_DOMAIN_MEMREGION_READABLE | \
-				 SBI_DOMAIN_MEMREGION_WRITEABLE | \
-				 SBI_DOMAIN_MEMREGION_EXECUTABLE)
+#define DISABLE_DEVICES_MASK	(SBI_MEMREGION_READABLE | \
+				 SBI_MEMREGION_WRITEABLE | \
+				 SBI_MEMREGION_EXECUTABLE)
 
 static int __fixup_count_disable_devices(void *fdt, int doff, int roff,
 					 u32 perm, void *p)
@@ -237,7 +237,7 @@ static int __fdt_parse_region(void *fdt, int domain_offset,
 	u64 val64;
 	const u32 *val;
 	struct parse_region_data *preg = opaque;
-	struct sbi_domain_memregion *region;
+	struct sbi_memregion *region;
 
 	/*
 	 * Non-root domains cannot add a region with only M-mode
@@ -247,8 +247,8 @@ static int __fdt_parse_region(void *fdt, int domain_offset,
 	 * SU permission bits can't be all zeroes when M-mode permission
 	 * bits have at least one bit set.
 	 */
-	if (!(region_access & SBI_DOMAIN_MEMREGION_SU_ACCESS_MASK)
-	    && (region_access & SBI_DOMAIN_MEMREGION_M_ACCESS_MASK))
+	if (!(region_access & SBI_MEMREGION_SU_ACCESS_MASK)
+	    && (region_access & SBI_MEMREGION_M_ACCESS_MASK))
 		return SBI_EINVAL;
 
 	/* Find next region of the domain */
@@ -274,9 +274,9 @@ static int __fdt_parse_region(void *fdt, int domain_offset,
 	region->order = val32;
 
 	/* Read "mmio" DT property */
-	region->flags = region_access & SBI_DOMAIN_MEMREGION_ACCESS_MASK;
+	region->flags = region_access & SBI_MEMREGION_ACCESS_MASK;
 	if (fdt_get_property(fdt, region_offset, "mmio", NULL))
-		region->flags |= SBI_DOMAIN_MEMREGION_MMIO;
+		region->flags |= SBI_MEMREGION_MMIO;
 
 	preg->region_count++;
 
@@ -293,7 +293,7 @@ static int __fdt_parse_domain(void *fdt, int domain_offset, void *opaque)
 	struct sbi_hartmask assign_mask;
 	struct parse_region_data preg;
 	int *cold_domain_offset = opaque;
-	struct sbi_domain_memregion *reg;
+	struct sbi_memregion *reg;
 	int i, err = 0, len, cpus_offset, cpu_offset, doffset;
 
 	dom = sbi_zalloc(sizeof(*dom));
@@ -362,9 +362,9 @@ static int __fdt_parse_domain(void *fdt, int domain_offset, void *opaque)
 	 * 2) mmio regions protecting M-mode only mmio devices
 	 */
 	sbi_domain_for_each_memregion(&root, reg) {
-		if ((reg->flags & SBI_DOMAIN_MEMREGION_SU_READABLE) ||
-		    (reg->flags & SBI_DOMAIN_MEMREGION_SU_WRITABLE) ||
-		    (reg->flags & SBI_DOMAIN_MEMREGION_SU_EXECUTABLE))
+		if ((reg->flags & SBI_MEMREGION_SU_READABLE) ||
+		    (reg->flags & SBI_MEMREGION_SU_WRITABLE) ||
+		    (reg->flags & SBI_MEMREGION_SU_EXECUTABLE))
 			continue;
 		if (preg.max_regions <= preg.region_count) {
 			err = SBI_EINVAL;
lib/utils/fdt/fdt_fixup.c +5 -5
index 974b1b8..fd45763 100644
@@ -282,7 +282,7 @@ static int fdt_resv_memory_update_node(void *fdt, unsigned long addr,
  */
 int fdt_reserved_memory_fixup(void *fdt)
 {
-	struct sbi_domain_memregion *reg;
+	struct sbi_memregion *reg;
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	unsigned long filtered_base[PMP_COUNT] = { 0 };
 	unsigned char filtered_order[PMP_COUNT] = { 0 };
@@ -343,13 +343,13 @@ int fdt_reserved_memory_fixup(void *fdt)
 	i = 0;
 	sbi_domain_for_each_memregion(dom, reg) {
 		/* Ignore MMIO or READABLE or WRITABLE or EXECUTABLE regions */
-		if (reg->flags & SBI_DOMAIN_MEMREGION_MMIO)
+		if (reg->flags & SBI_MEMREGION_MMIO)
 			continue;
-		if (reg->flags & SBI_DOMAIN_MEMREGION_SU_READABLE)
+		if (reg->flags & SBI_MEMREGION_SU_READABLE)
 			continue;
-		if (reg->flags & SBI_DOMAIN_MEMREGION_SU_WRITABLE)
+		if (reg->flags & SBI_MEMREGION_SU_WRITABLE)
 			continue;
-		if (reg->flags & SBI_DOMAIN_MEMREGION_SU_EXECUTABLE)
+		if (reg->flags & SBI_MEMREGION_SU_EXECUTABLE)
 			continue;
 
 		if (i >= PMP_COUNT) {
lib/utils/ipi/aclint_mswi.c +6 -6
index 4ae6bb1..231a1d6 100644
@@ -85,7 +85,7 @@ int aclint_mswi_cold_init(struct aclint_mswi_data *mswi)
 	int rc;
 	struct sbi_scratch *scratch;
 	unsigned long pos, region_size;
-	struct sbi_domain_memregion reg;
+	struct sbi_memregion reg;
 
 	/* Sanity checks */
 	if (!mswi || (mswi->addr & (ACLINT_MSWI_ALIGN - 1)) ||
@@ -117,11 +117,11 @@ int aclint_mswi_cold_init(struct aclint_mswi_data *mswi)
 	for (pos = 0; pos < mswi->size; pos += ACLINT_MSWI_ALIGN) {
 		region_size = ((mswi->size - pos) < ACLINT_MSWI_ALIGN) ?
 			      (mswi->size - pos) : ACLINT_MSWI_ALIGN;
-		sbi_domain_memregion_init(mswi->addr + pos, region_size,
-					  (SBI_DOMAIN_MEMREGION_MMIO |
-					   SBI_DOMAIN_MEMREGION_M_READABLE |
-					   SBI_DOMAIN_MEMREGION_M_WRITABLE),
-					  &reg);
+		sbi_memregion_init(mswi->addr + pos, region_size,
+				   (SBI_MEMREGION_MMIO |
+				    SBI_MEMREGION_M_READABLE |
+				    SBI_MEMREGION_M_WRITABLE),
+				   &reg);
 		rc = sbi_domain_root_add_memregion(&reg);
 		if (rc)
 			return rc;
lib/utils/ipi/andes_plicsw.c +3 -3
index 413ac20..5ce64fc 100644
@@ -103,9 +103,9 @@ int plicsw_cold_ipi_init(struct plicsw_data *plicsw)
 	/* Add PLICSW region to the root domain */
 	rc = sbi_domain_root_add_memrange(plicsw->addr, plicsw->size,
 					  PLICSW_REGION_ALIGN,
-					  SBI_DOMAIN_MEMREGION_MMIO |
-					  SBI_DOMAIN_MEMREGION_M_READABLE |
-					  SBI_DOMAIN_MEMREGION_M_WRITABLE);
+					  SBI_MEMREGION_MMIO |
+					  SBI_MEMREGION_M_READABLE |
+					  SBI_MEMREGION_M_WRITABLE);
 	if (rc)
 		return rc;
 
lib/utils/irqchip/aplic.c +6 -6
index 10236e7..84cdae0 100644
@@ -169,7 +169,7 @@ int aplic_cold_irqchip_init(struct aplic_data *aplic)
 {
 	int rc;
 	u32 i, j, tmp;
-	struct sbi_domain_memregion reg;
+	struct sbi_memregion reg;
 	struct aplic_delegate_data *deleg;
 	u32 first_deleg_irq, last_deleg_irq;
 
@@ -268,11 +268,11 @@ int aplic_cold_irqchip_init(struct aplic_data *aplic)
 	    ((first_deleg_irq < last_deleg_irq) &&
 	    (last_deleg_irq == aplic->num_source) &&
 	    (first_deleg_irq == 1))) {
-		sbi_domain_memregion_init(aplic->addr, aplic->size,
-					  (SBI_DOMAIN_MEMREGION_MMIO |
-					   SBI_DOMAIN_MEMREGION_M_READABLE |
-					   SBI_DOMAIN_MEMREGION_M_WRITABLE),
-					  &reg);
+		sbi_memregion_init(aplic->addr, aplic->size,
+				   (SBI_MEMREGION_MMIO |
+				    SBI_MEMREGION_M_READABLE |
+				    SBI_MEMREGION_M_WRITABLE),
+				   &reg);
 		rc = sbi_domain_root_add_memregion(&reg);
 		if (rc)
 			return rc;
lib/utils/irqchip/imsic.c +6 -7
index 7d6993e..d0f2e95 100644
@@ -348,7 +348,7 @@ int imsic_data_check(struct imsic_data *imsic)
 int imsic_cold_irqchip_init(struct imsic_data *imsic)
 {
 	int i, rc;
-	struct sbi_domain_memregion reg;
+	struct sbi_memregion reg;
 
 	/* Sanity checks */
 	rc = imsic_data_check(imsic);
@@ -378,12 +378,11 @@ int imsic_cold_irqchip_init(struct imsic_data *imsic)
 
 	/* Add IMSIC regions to the root domain */
 	for (i = 0; i < IMSIC_MAX_REGS && imsic->regs[i].size; i++) {
-		sbi_domain_memregion_init(imsic->regs[i].addr,
-					  imsic->regs[i].size,
-					  (SBI_DOMAIN_MEMREGION_MMIO |
-					   SBI_DOMAIN_MEMREGION_M_READABLE |
-					   SBI_DOMAIN_MEMREGION_M_WRITABLE),
-					  &reg);
+		sbi_memregion_init(imsic->regs[i].addr, imsic->regs[i].size,
+				   (SBI_MEMREGION_MMIO |
+				    SBI_MEMREGION_M_READABLE |
+				    SBI_MEMREGION_M_WRITABLE),
+				   &reg);
 		rc = sbi_domain_root_add_memregion(&reg);
 		if (rc)
 			return rc;
lib/utils/irqchip/plic.c +2 -2
index 193e320..b1b3ba2 100644
@@ -174,6 +174,6 @@ int plic_cold_irqchip_init(const struct plic_data *plic)
 		plic_set_priority(plic, i, 0);
 
 	return sbi_domain_root_add_memrange(plic->addr, plic->size, BIT(20),
-					(SBI_DOMAIN_MEMREGION_MMIO |
-					 SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW));
+					(SBI_MEMREGION_MMIO |
+					 SBI_MEMREGION_SHARED_SURW_MRW));
 }
lib/utils/regmap/fdt_regmap_syscon.c +2 -2
index d87b6e0..bd031f5 100644
@@ -242,8 +242,8 @@ static int regmap_syscon_init(void *fdt, int nodeoff, u32 phandle,
 	}
 
 	rc = sbi_domain_root_add_memrange(addr, size, PAGE_SIZE,
-				(SBI_DOMAIN_MEMREGION_MMIO |
-				 SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW));
+				(SBI_MEMREGION_MMIO |
+				 SBI_MEMREGION_SHARED_SURW_MRW));
 	if (rc)
 		goto fail_free_syscon;
 
lib/utils/serial/cadence-uart.c +2 -2
index 30fbb95..a9e6ea9 100644
@@ -127,6 +127,6 @@ int cadence_uart_init(unsigned long base, u32 in_freq, u32 baudrate)
 	sbi_console_set_device(&cadence_console);
 
 	return sbi_domain_root_add_memrange(base, PAGE_SIZE, PAGE_SIZE,
-					    (SBI_DOMAIN_MEMREGION_MMIO |
-					    SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW));
+					    (SBI_MEMREGION_MMIO |
+					    SBI_MEMREGION_SHARED_SURW_MRW));
 }
lib/utils/serial/fdt_serial_htif.c +2 -2
index 7d88e7a..d113cb5 100644
@@ -33,8 +33,8 @@ static int serial_htif_init(void *fdt, int nodeoff,
 	fdt_get_node_addr_size(fdt, nodeoff, 1, &tohost_addr, NULL);
 
 	rc = sbi_domain_root_add_memrange(fromhost_addr, PAGE_SIZE, PAGE_SIZE,
-					  (SBI_DOMAIN_MEMREGION_MMIO |
-					   SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW));
+					  (SBI_MEMREGION_MMIO |
+					   SBI_MEMREGION_SHARED_SURW_MRW));
 	if (rc)
 		return rc;
 
lib/utils/serial/uart8250.c +2 -2
index 1fe053f..aec9456 100644
@@ -136,6 +136,6 @@ int uart8250_init(unsigned long base, u32 in_freq, u32 baudrate, u32 reg_shift,
 	sbi_console_set_device(&uart8250_console);
 
 	return sbi_domain_root_add_memrange(base, PAGE_SIZE, PAGE_SIZE,
-					    (SBI_DOMAIN_MEMREGION_MMIO |
-					    SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW));
+					    (SBI_MEMREGION_MMIO |
+					    SBI_MEMREGION_SHARED_SURW_MRW));
 }
lib/utils/timer/aclint_mtimer.c +12 -12
index 9acb26e..41dbfb0 100644
@@ -227,34 +227,34 @@ int aclint_mtimer_cold_init(struct aclint_mtimer_data *mt,
 		rc = sbi_domain_root_add_memrange(mt->mtimecmp_addr,
 					mt->mtime_size + mt->mtimecmp_size,
 					MTIMER_REGION_ALIGN,
-					(SBI_DOMAIN_MEMREGION_MMIO |
-					 SBI_DOMAIN_MEMREGION_M_READABLE |
-					 SBI_DOMAIN_MEMREGION_M_WRITABLE));
+					(SBI_MEMREGION_MMIO |
+					 SBI_MEMREGION_M_READABLE |
+					 SBI_MEMREGION_M_WRITABLE));
 		if (rc)
 			return rc;
 	} else if (mt->mtimecmp_addr == (mt->mtime_addr + mt->mtime_size)) {
 		rc = sbi_domain_root_add_memrange(mt->mtime_addr,
 					mt->mtime_size + mt->mtimecmp_size,
 					MTIMER_REGION_ALIGN,
-					(SBI_DOMAIN_MEMREGION_MMIO |
-					 SBI_DOMAIN_MEMREGION_M_READABLE |
-					 SBI_DOMAIN_MEMREGION_M_WRITABLE));
+					(SBI_MEMREGION_MMIO |
+					 SBI_MEMREGION_M_READABLE |
+					 SBI_MEMREGION_M_WRITABLE));
 		if (rc)
 			return rc;
 	} else {
 		rc = sbi_domain_root_add_memrange(mt->mtime_addr,
 						mt->mtime_size, MTIMER_REGION_ALIGN,
-						(SBI_DOMAIN_MEMREGION_MMIO |
-						 SBI_DOMAIN_MEMREGION_M_READABLE |
-						 SBI_DOMAIN_MEMREGION_M_WRITABLE));
+						(SBI_MEMREGION_MMIO |
+						 SBI_MEMREGION_M_READABLE |
+						 SBI_MEMREGION_M_WRITABLE));
 		if (rc)
 			return rc;
 
 		rc = sbi_domain_root_add_memrange(mt->mtimecmp_addr,
 						mt->mtimecmp_size, MTIMER_REGION_ALIGN,
-						(SBI_DOMAIN_MEMREGION_MMIO |
-						 SBI_DOMAIN_MEMREGION_M_READABLE |
-						 SBI_DOMAIN_MEMREGION_M_WRITABLE));
+						(SBI_MEMREGION_MMIO |
+						 SBI_MEMREGION_M_READABLE |
+						 SBI_MEMREGION_M_WRITABLE));
 		if (rc)
 			return rc;
 	}
lib/utils/timer/andes_plmt.c +3 -3
index 6e4bfaf..71be37f 100644
@@ -83,9 +83,9 @@ int plmt_cold_timer_init(struct plmt_data *plmt)
 	rc = sbi_domain_root_add_memrange(
 		(unsigned long)plmt->time_val, plmt->size,
 		PLMT_REGION_ALIGN,
-		SBI_DOMAIN_MEMREGION_MMIO |
-		SBI_DOMAIN_MEMREGION_M_READABLE |
-		SBI_DOMAIN_MEMREGION_M_WRITABLE);
+		SBI_MEMREGION_MMIO |
+		SBI_MEMREGION_M_READABLE |
+		SBI_MEMREGION_M_WRITABLE);
 	if (rc)
 		return rc;
 
platform/generic/renesas/rzfive/rzfive.c +1 -1
index 515bcd7..e13224d 100644
@@ -45,7 +45,7 @@ static int renesas_rzfive_early_init(bool cold_boot, const struct fdt_match *mat
 	 * we grant full access.
 	 */
 	return sbi_domain_root_add_memrange(0x30000, 0x20000, 0x1000,
-					    SBI_DOMAIN_MEMREGION_M_RWX);
+					    SBI_MEMREGION_M_RWX);
 }
 
 static const struct fdt_match renesas_rzfive_match[] = {
platform/generic/sophgo/sg2042.c +3 -3
index ae2d702..6a2736d 100644
@@ -36,9 +36,9 @@ static int sophgo_sg2042_early_init(bool cold_boot,
 					SOPHGO_SG2042_TIMER_SIZE *
 					SOPHGO_SG2042_TIMER_NUM,
 					MTIMER_REGION_ALIGN,
-					(SBI_DOMAIN_MEMREGION_MMIO |
-					 SBI_DOMAIN_MEMREGION_M_READABLE |
-					 SBI_DOMAIN_MEMREGION_M_WRITABLE));
+					(SBI_MEMREGION_MMIO |
+					 SBI_MEMREGION_M_READABLE |
+					 SBI_MEMREGION_M_WRITABLE));
 
 
 	return 0;
Signature · 1 line
2.45.2
#4 lore raw

PATCH 4/8 lib: sbi: memregion: Introduce memregion_start/end macros

In reply to Gregor Haas

Various disparate parts of the memregion handling code calculate the base and bounds of memregions. Standardize these calculations into two new macros:
memregion_start() and memregion_end(). This will reduce the amount of code changes later, when we make memregions keep track of raw sizes rather than rounded orders.

---
 include/sbi/sbi_memregion.h |  6 ++++++
 lib/sbi/sbi_memregion.c     | 23 ++++++++++-------------
 2 files changed, 16 insertions(+), 13 deletions(-)
include/sbi/sbi_memregion.h +6 -0
index 0aa9a44..6e1b771 100644
@@ -145,6 +145,12 @@ struct sbi_memregion {
 	unsigned long flags;
 };
 
+#define memregion_start(reg) \
+	((reg)->base)
+
+#define memregion_end(reg) \
+	((reg)->order < __riscv_xlen) ? \
+	      (reg)->base + ((1UL << (reg)->order) - 1) : -1UL;
 /**
  * Initialize a domain memory region based on it's physical
  * address and size.
lib/sbi/sbi_memregion.c +10 -13
index 4abd20a..8978793 100644
@@ -37,10 +37,10 @@ void sbi_memregion_init(unsigned long addr,
 static bool is_region_subset(const struct sbi_memregion *regA,
 			     const struct sbi_memregion *regB)
 {
-	ulong regA_start = regA->base;
-	ulong regA_end = regA->base + (BIT(regA->order) - 1);
-	ulong regB_start = regB->base;
-	ulong regB_end = regB->base + (BIT(regB->order) - 1);
+	ulong regA_start = memregion_start(regA);
+	ulong regA_end = memregion_end(regA);
+	ulong regB_start = memregion_start(regB);
+	ulong regB_end = memregion_end(regB);
 
 	if ((regB_start <= regA_start) &&
 	    (regA_start < regB_end) &&
@@ -217,9 +217,8 @@ bool sbi_domain_check_addr(const struct sbi_domain *dom,
 					(rflags & SBI_MEMREGION_SU_ACCESS_MASK)
 						>> SBI_MEMREGION_SU_ACCESS_SHIFT);
 
-		rstart = reg->base;
-		rend = (reg->order < __riscv_xlen) ?
-						     rstart + ((1UL << reg->order) - 1) : -1UL;
+		rstart = memregion_start(reg);
+		rend = memregion_end(reg);
 		if (rstart <= addr && addr <= rend) {
 			rmmio = (rflags & SBI_MEMREGION_MMIO) ? true : false;
 			if (mmio != rmmio)
@@ -239,9 +238,8 @@ static const struct sbi_memregion *find_region(
 	struct sbi_memregion *reg;
 
 	sbi_domain_for_each_memregion(dom, reg) {
-		rstart = reg->base;
-		rend = (reg->order < __riscv_xlen) ?
-						     rstart + ((1UL << reg->order) - 1) : -1UL;
+		rstart = memregion_start(reg);
+		rend = memregion_end(reg);
 		if (rstart <= addr && addr <= rend)
 			return reg;
 	}
@@ -307,9 +305,8 @@ void sbi_domain_dump_memregions(const struct sbi_domain *dom, const char *suffix
 	int i = 0, k;
 
 	sbi_domain_for_each_memregion(dom, reg) {
-		rstart = reg->base;
-		rend = (reg->order < __riscv_xlen) ?
-		        rstart + ((1UL << reg->order) - 1) : -1UL;
+		rstart = memregion_start(reg);
+		rend = memregion_end(reg);
 
 		sbi_printf("Domain%d Region%02d    %s: 0x%" PRILX "-0x%" PRILX " ",
 			   dom->index, i, suffix, rstart, rend);
Signature · 1 line
2.45.2
#5 lore raw

PATCH 5/8 lib: sbi: memregion: Make memregions keep track of raw size rather than order

In reply to Gregor Haas

In order to accomodate isolation primitives other than PMP (such as the forthcoming SMMTT implementation), make memregions keep track of their raw size rather than prerounding the size to a power-of-two order. Not all isolation primitives have the same stringent naturally-aligned power-of-two requirements as PMP, and so we need a layer of abstraction between memregion information as it is discovered (from device trees or otherwise) and isolation information as applied using a specific primitive. This commit lays the groundwork for such an abstraction.

---
 docs/domain_support.md      | 41 +++++++++++++++++++++++++------------
 include/sbi/sbi_domain.h    |  2 +-
 include/sbi/sbi_memregion.h | 10 ++++-----
 lib/sbi/sbi_domain.c        | 18 ++++++++--------
 lib/sbi/sbi_hart.c          | 23 +++++++++++++++------
 lib/sbi/sbi_memregion.c     | 30 +++++++++++++++++----------
 lib/utils/fdt/fdt_domain.c  | 29 ++++++++++++++++++--------
 lib/utils/fdt/fdt_fixup.c   | 10 ++++-----
 8 files changed, 102 insertions(+), 61 deletions(-)
docs/domain_support.md +28 -13
index 91677d3..33bd8d6 100644
@@ -24,10 +24,9 @@ Domain Memory Region
 A domain memory region is represented by **struct sbi_memregion** in
 OpenSBI and has following details:
 
-* **order** - The size of a memory region is **2 ^ order** where **order**
-  must be **3 <= order <= __riscv_xlen**
-* **base** - The base address of a memory region is **2 ^ order**
-  aligned start address
+* **size** - The size of a memory region, where the maximum size is encoded
+  as -1UL.
+* **base** - The base address of a memory region.
 * **flags** - The flags of a memory region represent memory type (i.e.
   RAM or MMIO) and allowed accesses (i.e. READ, WRITE, EXECUTE, etc.)
 
@@ -56,10 +55,16 @@ has following details:
 * **system_suspend_allowed** - Is domain allowed to suspend the system?
 
 The memory regions represented by **regions** in **struct sbi_domain** have
-following additional constraints to align with RISC-V PMP requirements:
+following additional constraints:
 
 * A memory region to protect OpenSBI firmware from S-mode and U-mode
   should always be present
+
+When applying the specific isolation primitive to the domain, there may be
+other specific constraints on memory regions. These are listed below.
+
+To align with RISC-V PMP requirements:
+
 * For two overlapping memory regions, one should be sub-region of another
 * Two overlapping memory regions should not be of same size
 * Two overlapping memory regions cannot have same flags
@@ -138,17 +143,26 @@ The DT properties of a domain memory region DT node are as follows:
 
 * **compatible** (Mandatory) - The compatible string of the domain memory
   region. This DT property should have value *"opensbi,domain,memregion"*
-* **base** (Mandatory) - The base address of the domain memory region. This
-  DT property should have a **2 ^ order** aligned 64 bit address (i.e. two
-  DT cells).
-* **order** (Mandatory) - The order of the domain memory region. This DT
-  property should have a 32 bit value (i.e. one DT cell) in the range
-  **3 <= order <= __riscv_xlen**.
+* **base** (Mandatory) - The base address of the domain memory region. If
+  **order** is specified (see below), this DT property should have a
+  **2 ^ order** aligned 64 bit address (i.e. two DT cells). Otherwise, if
+  **size** is specified (see below), this DT property should have a 64 bit
+  address (i.e. two DT cells).
 * **mmio** (Optional) - A boolean flag representing whether the domain
   memory region is a memory-mapped I/O (MMIO) region.
 * **devices** (Optional) - The list of device DT node phandles for devices
   which fall under this domain memory region.
 
+Additionally, **one** of the two DT properties **must** be specified:
+
+* **size** - The size of the domain memory region. This DT property should
+  have a 64 bit value (i.e. two DT cells). The maximum possible size should
+  be encoded as `<0xFFFFFFFF 0xFFFFFFFF>`.
+* **order** - The order of the domain memory region. This DT property should
+  have a 32 bit value (i.e. one DT cell) in the range **3 <= order <= __riscv_xlen**.
+  This property is kept for compatibility purposes; new device trees should
+  use the **size** property instead.
+
 ### Domain Instance Node
 
 The domain instance DT node describes set of possible HARTs, set of memory
@@ -237,6 +251,7 @@ be done:
         opensbi-domains {
             compatible = "opensbi,domain,config";
 
+            /* Region using "order" property rather than size */
             tmem: tmem {
                 compatible = "opensbi,domain,memregion";
                 base = <0x0 0x80100000>;
@@ -246,7 +261,7 @@ be done:
             tuart: tuart {
                 compatible = "opensbi,domain,memregion";
                 base = <0x0 0x10011000>;
-                order = <12>;
+                size = <0x0 0x1000>;
                 mmio;
                 devices = <&uart1>;
             };
@@ -254,7 +269,7 @@ be done:
             allmem: allmem {
                 compatible = "opensbi,domain,memregion";
                 base = <0x0 0x0>;
-                order = <64>;
+                size = <0xFFFFFFFF 0xFFFFFFFF>;
             };
 
             tdomain: trusted-domain {
include/sbi/sbi_domain.h +1 -1
index a712504..74e3ea9 100644
@@ -86,7 +86,7 @@ extern struct sbi_domain *domidx_to_domain_table[];
 
 /** Iterate over each memory region of a domain */
 #define sbi_domain_for_each_memregion(__d, __r) \
-	for ((__r) = (__d)->regions; (__r)->order; (__r)++)
+	for ((__r) = (__d)->regions; (__r)->size; (__r)++)
 
 /**
  * Check whether given HART is assigned to specified domain
include/sbi/sbi_memregion.h +4 -6
index 6e1b771..99faba7 100644
@@ -15,13 +15,11 @@ enum sbi_domain_access {
 /** Representation of OpenSBI domain memory region */
 struct sbi_memregion {
 	/**
-	 * Size of memory region as power of 2
-	 * It has to be minimum 3 and maximum __riscv_xlen
+	 * Size of memory region. The maximum value is encoded as -1UL
 	 */
-	unsigned long order;
+	unsigned long size;
 	/**
 	 * Base address of memory region
-	 * It must be 2^order aligned address
 	 */
 	unsigned long base;
 	/** Flags representing memory region attributes */
@@ -149,8 +147,8 @@ struct sbi_memregion {
 	((reg)->base)
 
 #define memregion_end(reg) \
-	((reg)->order < __riscv_xlen) ? \
-	      (reg)->base + ((1UL << (reg)->order) - 1) : -1UL;
+	(((reg)->size == -1UL) ? -1UL : (reg)->base + (reg)->size - 1)
+
 /**
  * Initialize a domain memory region based on it's physical
  * address and size.
lib/sbi/sbi_domain.c +8 -10
index 22d5d8b..e272d81 100644
@@ -98,7 +98,7 @@ ulong sbi_domain_get_assigned_hartmask(const struct sbi_domain *dom,
 
 static int sanitize_domain(struct sbi_domain *dom)
 {
-	u32 i, rc;
+	int i, rc;
 
 	/* Check possible HARTs */
 	if (!dom->possible_harts) {
@@ -302,7 +302,7 @@ int sbi_domain_root_add_memregion(const struct sbi_memregion *reg)
 	nreg = &root.regions[root_memregs_count];
 	sbi_memcpy(nreg, reg, sizeof(*reg));
 	root_memregs_count++;
-	root.regions[root_memregs_count].order = 0;
+       root.regions[root_memregs_count].size = 0;
 
 	/* Sort and optimize root regions */
 	do {
@@ -315,19 +315,17 @@ int sbi_domain_root_add_memregion(const struct sbi_memregion *reg)
 			return rc;
 		}
 
-		/* Merge consecutive memregions with same order and flags */
+		/* Merge consecutive memregions with same flags */
 		reg_merged = false;
 		sbi_domain_for_each_memregion(&root, nreg) {
 			nreg1 = nreg + 1;
-			if (!nreg1->order)
+			if (!nreg1->size)
 				continue;
 
-			if (!(nreg->base & (BIT(nreg->order + 1) - 1)) &&
-			    (nreg->base + BIT(nreg->order)) == nreg1->base &&
-			    nreg->order == nreg1->order &&
+			if ((nreg->base + nreg->size) == nreg1->base &&
 			    nreg->flags == nreg1->flags) {
-				nreg->order++;
-				while (nreg1->order) {
+				nreg->size += nreg1->size;
+				while (nreg1->size) {
 					nreg2 = nreg1 + 1;
 					sbi_memcpy(nreg1, nreg2, sizeof(*nreg1));
 					nreg1++;
@@ -505,7 +503,7 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid)
 			   &root_memregs[root_memregs_count++]);
 
 	/* Root domain memory region end */
-	root_memregs[root_memregs_count].order = 0;
+	root_memregs[root_memregs_count].size = 0;
 
 	/* Root domain boot HART id is same as coldboot HART id */
 	root.boot_hartid = cold_hartid;
lib/sbi/sbi_hart.c +17 -6
index 4dd1a25..e82045c 100644
@@ -346,6 +346,11 @@ static unsigned int sbi_hart_get_smepmp_flags(struct sbi_scratch *scratch,
 	return pmp_flags;
 }
 
+static inline bool pmp_is_power_of_two(unsigned long order, unsigned long size)
+{
+	return order == __riscv_xlen ? true : BIT(order) == size;
+}
+
 static void sbi_hart_smepmp_set(struct sbi_scratch *scratch,
 				struct sbi_domain *dom,
 				struct sbi_memregion *reg,
@@ -355,14 +360,16 @@ static void sbi_hart_smepmp_set(struct sbi_scratch *scratch,
 				unsigned long pmp_addr_max)
 {
 	unsigned long pmp_addr = reg->base >> PMP_SHIFT;
+	unsigned long order = log2roundup(reg->size);
 
-	if (pmp_log2gran <= reg->order && pmp_addr < pmp_addr_max) {
-		pmp_set(pmp_idx, pmp_flags, reg->base, reg->order);
+	if (pmp_is_power_of_two(order, reg->size) &&
+	    pmp_log2gran <= order && pmp_addr < pmp_addr_max) {
+		pmp_set(pmp_idx, pmp_flags, reg->base, order);
 	} else {
 		sbi_printf("Can not configure pmp for domain %s because"
 			   " memory region address 0x%lx or size 0x%lx "
 			   "is not in range.\n", dom->name, reg->base,
-			   reg->order);
+			   reg->size);
 	}
 }
 
@@ -451,6 +458,7 @@ static int sbi_hart_oldpmp_configure(struct sbi_scratch *scratch,
 	unsigned int pmp_idx = 0;
 	unsigned int pmp_flags;
 	unsigned long pmp_addr;
+	unsigned long order;
 
 	sbi_domain_for_each_memregion(dom, reg) {
 		if (pmp_count <= pmp_idx)
@@ -473,13 +481,16 @@ static int sbi_hart_oldpmp_configure(struct sbi_scratch *scratch,
 			pmp_flags |= PMP_X;
 
 		pmp_addr = reg->base >> PMP_SHIFT;
-		if (pmp_log2gran <= reg->order && pmp_addr < pmp_addr_max) {
-			pmp_set(pmp_idx++, pmp_flags, reg->base, reg->order);
+		order = log2roundup(reg->size);
+
+		if (pmp_is_power_of_two(order, reg->size) &&
+		    pmp_log2gran <= order && pmp_addr < pmp_addr_max) {
+			pmp_set(pmp_idx++, pmp_flags, reg->base, order);
 		} else {
 			sbi_printf("Can not configure pmp for domain %s because"
 				   " memory region address 0x%lx or size 0x%lx "
 				   "is not in range.\n", dom->name, reg->base,
-				   reg->order);
+				   reg->size);
 		}
 	}
 
lib/sbi/sbi_memregion.c +19 -11
index 8978793..56ee649 100644
@@ -28,7 +28,7 @@ void sbi_memregion_init(unsigned long addr,
 
 	if (reg) {
 		reg->base = base;
-		reg->order = order;
+		reg->size = (order == __riscv_xlen) ? -1UL : BIT(order);
 		reg->flags = flags;
 	}
 }
@@ -64,13 +64,14 @@ static bool is_region_compatible(const struct sbi_memregion *regA,
 /* Check if region complies with constraints */
 static bool is_region_valid(const struct sbi_memregion *reg)
 {
-	if (reg->order < 3 || __riscv_xlen < reg->order)
+	unsigned int order = log2roundup(reg->size);
+	if (order < 3 || __riscv_xlen < order)
 		return false;
 
-	if (reg->order == __riscv_xlen && reg->base != 0)
+	if (order == __riscv_xlen && reg->base != 0)
 		return false;
 
-	if (reg->order < __riscv_xlen && (reg->base & (BIT(reg->order) - 1)))
+	if (order < __riscv_xlen && (reg->base & (BIT(order) - 1)))
 		return false;
 
 	return true;
@@ -80,10 +81,17 @@ static bool is_region_valid(const struct sbi_memregion *reg)
 static bool is_region_before(const struct sbi_memregion *regA,
 			     const struct sbi_memregion *regB)
 {
-	if (regA->order < regB->order)
+	// Sentinel region always goes last
+	if (!regA->size)
+		return false;
+
+	if (!regB->size)
+		return true;
+
+	if (regA->size < regB->size)
 		return true;
 
-	if ((regA->order == regB->order) &&
+	if ((regA->size == regB->size) &&
 	    (regA->base < regB->base))
 		return true;
 
@@ -121,8 +129,8 @@ int sbi_memregion_sanitize(struct sbi_domain *dom)
 	sbi_domain_for_each_memregion(dom, reg) {
 		if (!is_region_valid(reg)) {
 			sbi_printf("%s: %s has invalid region base=0x%lx "
-				   "order=%lu flags=0x%lx\n", __func__,
-				   dom->name, reg->base, reg->order,
+				   "size=0x%lx flags=0x%lx\n", __func__,
+				   dom->name, reg->base, reg->size,
 				   reg->flags);
 			return SBI_EINVAL;
 		}
@@ -260,7 +268,7 @@ static const struct sbi_memregion *find_next_subset_region(
 			continue;
 
 		if (!ret || (sreg->base < ret->base) ||
-		    ((sreg->base == ret->base) && (sreg->order < ret->order)))
+		    ((sreg->base == ret->base) && (sreg->size < ret->size)))
 			ret = sreg;
 	}
 
@@ -289,8 +297,8 @@ bool sbi_domain_check_addr_range(const struct sbi_domain *dom,
 		sreg = find_next_subset_region(dom, reg, addr);
 		if (sreg)
 			addr = sreg->base;
-		else if (reg->order < __riscv_xlen)
-			addr = reg->base + (1UL << reg->order);
+		else if (reg->size != -1UL)
+			addr = reg->base + reg->size;
 		else
 			break;
 	}
lib/utils/fdt/fdt_domain.c +20 -9
index 779acec..51d9e9d 100644
@@ -233,7 +233,6 @@ static int __fdt_parse_region(void *fdt, int domain_offset,
 			      void *opaque)
 {
 	int len;
-	u32 val32;
 	u64 val64;
 	const u32 *val;
 	struct parse_region_data *preg = opaque;
@@ -264,14 +263,26 @@ static int __fdt_parse_region(void *fdt, int domain_offset,
 	val64 = (val64 << 32) | fdt32_to_cpu(val[1]);
 	region->base = val64;
 
-	/* Read "order" DT property */
-	val = fdt_getprop(fdt, region_offset, "order", &len);
-	if (!val || len != 4)
-		return SBI_EINVAL;
-	val32 = fdt32_to_cpu(*val);
-	if (val32 < 3 || __riscv_xlen < val32)
-		return SBI_EINVAL;
-	region->order = val32;
+	/* Read "size" DT property */
+	val = fdt_getprop(fdt, region_offset, "size", &len);
+	if (!val || len != 8) {
+		// Check for older "order" property
+		val = fdt_getprop(fdt, region_offset, "order", &len);
+		if (!val || len != 4) {
+			return SBI_EINVAL;
+		}
+
+		val64 = fdt32_to_cpu(*val);
+		if (val64 < 3 || __riscv_xlen < val64)
+			return SBI_EINVAL;
+
+		val64 = BIT(val64);
+	} else {
+		val64 = ((uint64_t) fdt32_to_cpu(val[0])) << 32;
+		val64 |= fdt32_to_cpu(val[1]);
+	}
+
+	region->size = val64;
 
 	/* Read "mmio" DT property */
 	region->flags = region_access & SBI_MEMREGION_ACCESS_MASK;
lib/utils/fdt/fdt_fixup.c +5 -5
index fd45763..20819f9 100644
@@ -285,7 +285,7 @@ int fdt_reserved_memory_fixup(void *fdt)
 	struct sbi_memregion *reg;
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	unsigned long filtered_base[PMP_COUNT] = { 0 };
-	unsigned char filtered_order[PMP_COUNT] = { 0 };
+	unsigned long filtered_size[PMP_COUNT] = { 0 };
 	unsigned long addr, size;
 	int err, parent, i, j;
 	int na = fdt_address_cells(fdt, 0);
@@ -362,23 +362,23 @@ int fdt_reserved_memory_fixup(void *fdt)
 		addr = reg->base;
 		for (j = 0; j < i; j++) {
 			if (addr == filtered_base[j]
-			    && filtered_order[j] < reg->order) {
+			    && filtered_size[j] < reg->size) {
 				overlap = true;
-				filtered_order[j] = reg->order;
+				filtered_size[j] = reg->size;
 				break;
 			}
 		}
 
 		if (!overlap) {
 			filtered_base[i] = reg->base;
-			filtered_order[i] = reg->order;
+			filtered_size[i] = reg->size;
 			i++;
 		}
 	}
 
 	for (j = 0; j < i; j++) {
 		addr = filtered_base[j];
-		size = 1UL << filtered_order[j];
+		size = filtered_size[j];
 		fdt_resv_memory_update_node(fdt, addr, size, j, parent);
 	}
 
Signature · 1 line
2.45.2
#6 lore raw

PATCH 6/8 lib: sbi: Do a merging pass on memregions whenever a domain is sanitized

In reply to Gregor Haas

Currently, memregions are only merged into each other (if consecutive) when adding a region to the root domain. This is also useful functionality for other secondary domains, so move this functionality into the memregion sanitization path. We also now manually need to recount root_memregs_count after calling sbi_memregion_sanitize(), since this was done in the merging pass before.

---
 lib/sbi/sbi_domain.c    |  54 ++++++++-------------
 lib/sbi/sbi_memregion.c | 105 ++++++++++++++++++++++++++++------------
 2 files changed, 94 insertions(+), 65 deletions(-)
lib/sbi/sbi_domain.c +20 -34
index e272d81..6b7bcc5 100644
@@ -99,6 +99,7 @@ ulong sbi_domain_get_assigned_hartmask(const struct sbi_domain *dom,
 static int sanitize_domain(struct sbi_domain *dom)
 {
 	int i, rc;
+	struct sbi_memregion *reg;
 
 	/* Check possible HARTs */
 	if (!dom->possible_harts) {
@@ -122,6 +123,15 @@ static int sanitize_domain(struct sbi_domain *dom)
 		return rc;
 	}
 
+	/* Recount root memregions since the above call may have consolidated
+	 * or otherwise pruned memregions away */
+	if (dom == &root) {
+		root_memregs_count = 0;
+		sbi_domain_for_each_memregion(dom, reg) {
+			root_memregs_count++;
+		}
+	}
+
 	/*
 	 * We don't need to check boot HART id of domain because if boot
 	 * HART id is not possible/assigned to this domain then it won't
@@ -290,8 +300,7 @@ int sbi_domain_register(struct sbi_domain *dom,
 int sbi_domain_root_add_memregion(const struct sbi_memregion *reg)
 {
 	int rc;
-	bool reg_merged;
-	struct sbi_memregion *nreg, *nreg1, *nreg2;
+	struct sbi_memregion *nreg;
 
 	/* Sanity checks */
 	if (!reg || domain_finalized || !root.regions ||
@@ -302,39 +311,16 @@ int sbi_domain_root_add_memregion(const struct sbi_memregion *reg)
 	nreg = &root.regions[root_memregs_count];
 	sbi_memcpy(nreg, reg, sizeof(*reg));
 	root_memregs_count++;
-       root.regions[root_memregs_count].size = 0;
-
-	/* Sort and optimize root regions */
-	do {
-		/* Sanitize the root domain so that memregions are sorted */
-		rc = sanitize_domain(&root);
-		if (rc) {
-			sbi_printf("%s: sanity checks failed for"
-				   " %s (error %d)\n", __func__,
-				   root.name, rc);
-			return rc;
-		}
+	root.regions[root_memregs_count].size = 0;
 
-		/* Merge consecutive memregions with same flags */
-		reg_merged = false;
-		sbi_domain_for_each_memregion(&root, nreg) {
-			nreg1 = nreg + 1;
-			if (!nreg1->size)
-				continue;
-
-			if ((nreg->base + nreg->size) == nreg1->base &&
-			    nreg->flags == nreg1->flags) {
-				nreg->size += nreg1->size;
-				while (nreg1->size) {
-					nreg2 = nreg1 + 1;
-					sbi_memcpy(nreg1, nreg2, sizeof(*nreg1));
-					nreg1++;
-				}
-				reg_merged = true;
-				root_memregs_count--;
-			}
-		}
-	} while (reg_merged);
+	/* Sanitize the root domain so that memregions are sorted */
+	rc = sanitize_domain(&root);
+	if (rc) {
+		sbi_printf("%s: sanity checks failed for"
+			   " %s (error %d)\n", __func__,
+			   root.name, rc);
+		return rc;
+	}
 
 	return 0;
 }
lib/sbi/sbi_memregion.c +74 -31

126 lines not on this page · raw message

Signature · 1 line
2.45.2
#7 lore raw

PATCH 7/8 lib: sbi: Remove PMP assumptions from memregion

In reply to Gregor Haas

Move the application of PMP assumptions (naturally-aligned power-of-two) from memregion initialization time to isolation primitive application time. We introduce a new parameter to sbi_memregion_sanitize(), indicating for which isolation primitive to sanitize the domain's memregions. During memregion discoverty time, this parameter is set to SBI_ISOLATION_UNKNOWN which simply keeps the memregions as-is. Then, later, when applying an isolation primitive (currently either PMP or SMEPMP), we call sbi_memregion_sanitize() again with SBI_ISOLATION_{PMP,SMEPMP} in order to finalize the memregions for application.

---
 include/sbi/sbi_domain.h     |   2 +
 include/sbi/sbi_hart.h       |   2 +-
 include/sbi/sbi_memregion.h  |  19 ++++--
 lib/sbi/sbi_domain.c         |  27 ++-------
 lib/sbi/sbi_domain_context.c |   2 +-
 lib/sbi/sbi_hart.c           |  16 ++++-
 lib/sbi/sbi_init.c           |   6 +-
 lib/sbi/sbi_memregion.c      | 109 ++++++++++++++++++++++++++++-------
 8 files changed, 128 insertions(+), 55 deletions(-)
include/sbi/sbi_domain.h +2 -0
index 74e3ea9..8fbd081 100644
@@ -44,6 +44,8 @@ struct sbi_domain {
 	struct sbi_context *hartindex_to_context_table[SBI_HARTMASK_MAX_BITS];
 	/** Array of memory regions terminated by a region with order zero */
 	struct sbi_memregion *regions;
+	/** Current isolation mode */
+	enum sbi_isolation_method isol_mode;
 	/** HART id of the HART booting this domain */
 	u32 boot_hartid;
 	/** Arg1 (or 'a1' register) of next booting stage for this domain */
include/sbi/sbi_hart.h +1 -1
index 81ec061..81b08ab 100644
@@ -123,7 +123,7 @@ unsigned int sbi_hart_pmp_count(struct sbi_scratch *scratch);
 unsigned int sbi_hart_pmp_log2gran(struct sbi_scratch *scratch);
 unsigned int sbi_hart_pmp_addrbits(struct sbi_scratch *scratch);
 unsigned int sbi_hart_mhpm_bits(struct sbi_scratch *scratch);
-int sbi_hart_pmp_configure(struct sbi_scratch *scratch);
+int sbi_hart_isolation_configure(struct sbi_scratch *scratch);
 int sbi_hart_map_saddr(unsigned long base, unsigned long size);
 int sbi_hart_unmap_saddr(void);
 int sbi_hart_priv_version(struct sbi_scratch *scratch);
include/sbi/sbi_memregion.h +14 -5
index 99faba7..4f62a9a 100644
@@ -2,6 +2,13 @@
 #ifndef __SBI_MEMREGION_H__
 #define __SBI_MEMREGION_H__
 
+/** Domain isolation types */
+enum sbi_isolation_method {
+	SBI_ISOLATION_UNKNOWN = 0,
+	SBI_ISOLATION_PMP,
+	SBI_ISOLATION_SMEPMP,
+};
+
 #include <sbi/sbi_domain.h>
 
 /** Domain access types */
@@ -164,13 +171,15 @@ void sbi_memregion_init(unsigned long addr,
 			struct sbi_memregion *reg);
 
 /**
+ * Sanitize a domain's memory regions based on the selected isolation
+ * type. This function should encode any constraints on region
+ * information, such as address alignment or sizing requirements.
  *
- * Traverse all of a domain's memory regions and sanitize
- * them, while making sure they are formatted properly
- *
- * @param dom the domain for which to sanitize regions
+ * @param dom the domain to sanitize
+ * @param type the isolation type to apply
  */
-int sbi_memregion_sanitize(struct sbi_domain *dom);
+int sbi_memregion_sanitize(struct sbi_domain *dom,
+			   enum sbi_isolation_method type);
 
 /**
  * Check whether we can access specified address for given mode and
lib/sbi/sbi_domain.c +4 -23
index 6b7bcc5..638ffa9 100644
@@ -116,7 +116,7 @@ static int sanitize_domain(struct sbi_domain *dom)
 		}
 	}
 
-	rc = sbi_memregion_sanitize(dom);
+	rc = sbi_memregion_sanitize(dom, SBI_ISOLATION_UNKNOWN);
 	if (rc) {
 		sbi_printf("%s: %s has unsanitizable regions\n",
 			   __func__, dom->name);
@@ -328,28 +328,9 @@ int sbi_domain_root_add_memregion(const struct sbi_memregion *reg)
 int sbi_domain_root_add_memrange(unsigned long addr, unsigned long size,
 			   unsigned long align, unsigned long region_flags)
 {
-	int rc;
-	unsigned long pos, end, rsize;
 	struct sbi_memregion reg;
-
-	pos = addr;
-	end = addr + size;
-	while (pos < end) {
-		rsize = pos & (align - 1);
-		if (rsize)
-			rsize = 1UL << sbi_ffs(pos);
-		else
-			rsize = ((end - pos) < align) ?
-				(end - pos) : align;
-
-		sbi_memregion_init(pos, rsize, region_flags, &reg);
-		rc = sbi_domain_root_add_memregion(&reg);
-		if (rc)
-			return rc;
-		pos += rsize;
-	}
-
-	return 0;
+	sbi_memregion_init(addr, size, region_flags, &reg);
+	return sbi_domain_root_add_memregion(&reg);
 }
 
 int sbi_domain_finalize(struct sbi_scratch *scratch, u32 cold_hartid)
@@ -372,7 +353,7 @@ int sbi_domain_finalize(struct sbi_scratch *scratch, u32 cold_hartid)
 		/* Domain boot HART index */
 		dhart = sbi_hartid_to_hartindex(dom->boot_hartid);
 
-		/* Ignore of boot HART is off limits */
+		/* Ignore if boot HART is off limits */
 		if (!sbi_hartindex_valid(dhart))
 			continue;
 
lib/sbi/sbi_domain_context.c +1 -1
index 49a2f76..750f016 100755
@@ -48,7 +48,7 @@ static void switch_to_next_domain_context(struct sbi_context *ctx,
 	for (int i = 0; i < pmp_count; i++) {
 		pmp_disable(i);
 	}
-	sbi_hart_pmp_configure(scratch);
+	sbi_hart_isolation_configure(scratch);
 
 	/* Save current CSR context and restore target domain's CSR context */
 	ctx->sstatus	= csr_swap(CSR_SSTATUS, dom_ctx->sstatus);
lib/sbi/sbi_hart.c +14 -2
index e82045c..df7ee98 100644
@@ -378,10 +378,16 @@ static int sbi_hart_smepmp_configure(struct sbi_scratch *scratch,
 				     unsigned int pmp_log2gran,
 				     unsigned long pmp_addr_max)
 {
+	int rc;
 	struct sbi_memregion *reg;
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	unsigned int pmp_idx, pmp_flags;
 
+	rc = sbi_memregion_sanitize(dom, SBI_ISOLATION_SMEPMP);
+	if (rc < 0) {
+		return rc;
+	}
+
 	/*
 	 * Set the RLB so that, we can write to PMP entries without
 	 * enforcement even if some entries are locked.
@@ -453,6 +459,7 @@ static int sbi_hart_oldpmp_configure(struct sbi_scratch *scratch,
 				     unsigned int pmp_log2gran,
 				     unsigned long pmp_addr_max)
 {
+	int rc;
 	struct sbi_memregion *reg;
 	struct sbi_domain *dom = sbi_domain_thishart_ptr();
 	unsigned int pmp_idx = 0;
@@ -460,6 +467,11 @@ static int sbi_hart_oldpmp_configure(struct sbi_scratch *scratch,
 	unsigned long pmp_addr;
 	unsigned long order;
 
+	rc = sbi_memregion_sanitize(dom, SBI_ISOLATION_PMP);
+	if (rc < 0) {
+		return rc;
+	}
+
 	sbi_domain_for_each_memregion(dom, reg) {
 		if (pmp_count <= pmp_idx)
 			break;
@@ -540,7 +552,7 @@ int sbi_hart_unmap_saddr(void)
 	return pmp_disable(SBI_SMEPMP_RESV_ENTRY);
 }
 
-int sbi_hart_pmp_configure(struct sbi_scratch *scratch)
+int sbi_hart_isolation_configure(struct sbi_scratch *scratch)
 {
 	int rc;
 	unsigned int pmp_bits, pmp_log2gran;
@@ -564,7 +576,7 @@ int sbi_hart_pmp_configure(struct sbi_scratch *scratch)
 	/*
 	 * As per section 3.7.2 of privileged specification v1.12,
 	 * virtual address translations can be speculatively performed
-	 * (even before actual access). These, along with PMP traslations,
+	 * (even before actual access). These, along with PMP translations,
 	 * can be cached. This can pose a problem with CPU hotplug
 	 * and non-retentive suspend scenario because PMP states are
 	 * not preserved.
lib/sbi/sbi_init.c +3 -3
index d80efe9..34970b5 100644
@@ -359,7 +359,7 @@ static void __noreturn init_coldboot(struct sbi_scratch *scratch, u32 hartid)
 	 * Configure PMP at last because if SMEPMP is detected,
 	 * M-mode access to the S/U space will be rescinded.
 	 */
-	rc = sbi_hart_pmp_configure(scratch);
+	rc = sbi_hart_isolation_configure(scratch);
 	if (rc) {
 		sbi_printf("%s: PMP configure failed (error %d)\n",
 			   __func__, rc);
@@ -438,7 +438,7 @@ static void __noreturn init_warm_startup(struct sbi_scratch *scratch,
 	 * Configure PMP@last because if SMEPMP is detected,
 	 * M-mode access to the S/U space will be rescinded.
 	 */
-	rc = sbi_hart_pmp_configure(scratch);
+	rc = sbi_hart_isolation_configure(scratch);
 	if (rc)
 		sbi_hart_hang();
 
@@ -459,7 +459,7 @@ static void __noreturn init_warm_resume(struct sbi_scratch *scratch,
 	if (rc)
 		sbi_hart_hang();
 
-	rc = sbi_hart_pmp_configure(scratch);
+	rc = sbi_hart_isolation_configure(scratch);
 	if (rc)
 		sbi_hart_hang();
 
lib/sbi/sbi_memregion.c +89 -20

161 lines not on this page · raw message

Signature · 1 line
2.45.2
#8 lore raw

PATCH 8/8 lib: sbi: memregion: Move memregion_sanitize_pmp() closer to its call site

In reply to Gregor Haas

This is a code movement change to move the previously created memregion_sanitize_pmp() function closer to its call site.

---
 lib/sbi/sbi_memregion.c | 44 ++++++++++++++++++++---------------------
 1 file changed, 22 insertions(+), 22 deletions(-)
lib/sbi/sbi_memregion.c +22 -22
index 3783a0a..7c5a977 100644
@@ -4,28 +4,6 @@
 #include <sbi/sbi_error.h>
 #include <sbi/sbi_string.h>
 
-static void memregion_sanitize_pmp(struct sbi_memregion *reg)
-{
-	unsigned long base = 0, order;
-	unsigned long addr = reg->base, size = reg->size;
-	for (order = log2roundup(size) ; order <= __riscv_xlen; order++) {
-		if (order < __riscv_xlen) {
-			base = addr & ~((1UL << order) - 1UL);
-			if ((base <= addr) &&
-			    (addr < (base + (1UL << order))) &&
-			    (base <= (addr + size - 1UL)) &&
-			    ((addr + size - 1UL) < (base + (1UL << order))))
-				break;
-		} else {
-			base = 0;
-			break;
-		}
-	}
-
-	reg->base = base;
-	reg->size = (order == __riscv_xlen) ? -1UL : BIT(order);
-}
-
 void sbi_memregion_init(unsigned long addr,
 			       unsigned long size,
 			       unsigned long flags,
@@ -212,6 +190,28 @@ static void merge_memregions(struct sbi_domain *dom, int *nmerged)
 	}
 }
 
+static void memregion_sanitize_pmp(struct sbi_memregion *reg)
+{
+	unsigned long base = 0, order;
+	unsigned long addr = reg->base, size = reg->size;
+	for (order = log2roundup(size) ; order <= __riscv_xlen; order++) {
+		if (order < __riscv_xlen) {
+			base = addr & ~((1UL << order) - 1UL);
+			if ((base <= addr) &&
+			    (addr < (base + (1UL << order))) &&
+			    (base <= (addr + size - 1UL)) &&
+			    ((addr + size - 1UL) < (base + (1UL << order))))
+				break;
+		} else {
+			base = 0;
+			break;
+		}
+	}
+
+	reg->base = base;
+	reg->size = (order == __riscv_xlen) ? -1UL : BIT(order);
+}
+
 static int memregion_sanitize(struct sbi_domain *dom,
 			      struct sbi_memregion *reg,
 			      enum sbi_isolation_method type)
Signature · 1 line
2.45.2
#9 lore raw

cover Prepare memregion handling for future isolation primitives

In reply to Gregor Haas
Quoted 22 lines · On Wed, Jul 31, 2024 at 11:46?PM Gregor Haas <[email protected]> wrote:

The existing memregion handling code deeply embeds the assumption that we will be isolating domains using PMP/SMEPMP rather than another isolation primitive. I am currently working on a series of patches that will enable SMMTT [1] support in both OpenSBI and QEMU. SMMTT is a much more granular, table-based isolation primitive which does not have the same alignment/sizing requirements as PMP.

This patch series lays some of the groundwork for landing SMMTT support in OpenSBI. Perhaps the most significant change here is that memregions now keep track of their raw sizes (rather than power-of-two rounded orders) throughout most of their lifetime. Then, when OpenSBI is about to apply its isolation configuration at the end of the coldboot/warmboot paths, the memregions for each domain are "sanitized" according to isolation primitive-specific rules.

The first two patches in this series are optional to apply, although I've found them useful when developing SMMTT. The first moves the memregion-related functionality out of sbi_domain.c into a new sbi_memregion.c. The second renames struct sbi_domain_memregion to struct sbi_memregion, while also renaming all of the memregion permission macros similarly. This makes the code somewhat more readable, especially since domain-handling and memregion-handling become more decoupled throughout this patch series.

Quoted 6 lines

[1] https://github.com/riscv/riscv-smmtt

Gregor Haas (8):
lib: sbi: Separate domain-handling code from memregion-handling code
lib: sbi: Create sbi_domain_dump_memregions()
lib: sbi: Rename domain_memregion -> memregion

I think it's better to keep the sbi_domain_ prefix for memregion in the filename as well as functions/structures for consistency. The sbi_domain_context also preserves the sbi_domain_ prefix

Quoted 3 lines

lib: sbi: memregion: Introduce memregion_start/end macros
lib: sbi: memregion: Make memregions keep track of raw size rather
than order

We will be using multiple HW isolation mechanisms in the long run. Please see more comments below.

Converting range based memregions (start, end) into order-of-2 overlapping memregions on-the-fly for PMP and ePMP will be painful for domain context switching. It's better to stay with order-of-2 overlapping memregions and translate these to Smmtt table mappings.

Quoted 5 lines

lib: sbi: Do a merging pass on memregions whenever a domain is
sanitized
lib: sbi: Remove PMP assumptions from memregion
lib: sbi: memregion: Move memregion_sanitize_pmp() closer to its call
site

The Smmtt only allows specifying permissions for S-mode domains but we still need mechanisms (such as PMP, ePMP) to specify permissions for M-mode firmware. This is for restricting M-mode access to only specific parts of memory and IO. This means, we will be having multiple HW insolation mechanisms which co-exists in M-mode.

My suggestion is the following:
1) Introduce sbi_domain_isolation instance (or something similar) which represents a HW isolation mechanism. The isolation mechanism will have its own capabilities such as what kind of permission checks it can enforce.
2) There can be multiple sbi_domain_isolation instances (e.g. PMP, ePMP, Smmtt, IOPMP, World Guard, etc). Some of these isolation mechanisms are per-hart so need to be context switched when we switch domain while other isolation mechanisms will be only for IOs.
3) Convert existing PMP and ePMP implementation into a per-hart sbi_domain_isolation instance.
4) Implement Smmtt as yet another per-hart sbi_domain_isolation instance with different capabilities compared to PMP and ePMP. This Smmtt isolation mechanism will translate order-of-2 regions to mappings in the Smmtt table. Each domain will have a separate smmtt table so for domain context switch, we only need to switch the Smmtt table pointer.

Quoted 31 lines
 docs/domain_support.md                   |  45 ++-
 include/sbi/sbi_domain.h                 | 192 +---------
 include/sbi/sbi_hart.h                   |   2 +-
 include/sbi/sbi_memregion.h              | 215 +++++++++++
 include/sbi/sbi_platform.h               |   2 +-
 lib/sbi/objects.mk                       |   1 +
 lib/sbi/sbi_domain.c                     | 444 +++-------------------
 lib/sbi/sbi_domain_context.c             |   2 +-
 lib/sbi/sbi_hart.c                       |  91 +++--
 lib/sbi/sbi_init.c                       |   6 +-
 lib/sbi/sbi_memregion.c                  | 459 +++++++++++++++++++++++
 lib/utils/fdt/fdt_domain.c               |  53 +--
 lib/utils/fdt/fdt_fixup.c                |  20 +-
 lib/utils/ipi/aclint_mswi.c              |  12 +-
 lib/utils/ipi/andes_plicsw.c             |   6 +-
 lib/utils/irqchip/aplic.c                |  12 +-
 lib/utils/irqchip/imsic.c                |  13 +-
 lib/utils/irqchip/plic.c                 |   4 +-
 lib/utils/regmap/fdt_regmap_syscon.c     |   4 +-
 lib/utils/serial/cadence-uart.c          |   4 +-
 lib/utils/serial/fdt_serial_htif.c       |   4 +-
 lib/utils/serial/uart8250.c              |   4 +-
 lib/utils/timer/aclint_mtimer.c          |  24 +-
 lib/utils/timer/andes_plmt.c             |   6 +-
 platform/generic/renesas/rzfive/rzfive.c |   2 +-
 platform/generic/sophgo/sg2042.c         |   6 +-
 26 files changed, 910 insertions(+), 723 deletions(-)
 create mode 100644 include/sbi/sbi_memregion.h
 create mode 100644 lib/sbi/sbi_memregion.c
Quoted 8 lines

--
2.45.2

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup

#10 lore raw

cover Prepare memregion handling for future isolation primitives

(Resending because inadvertent HTML in my last response)

Hi Anup,

Thanks for your comments!

Quoted 26 lines · On Wed, Sep 18, 2024 at 11:16?PM Anup Patel <[email protected]> wrote:
Quoted 24 lines · On Wed, Jul 31, 2024 at 11:46?PM Gregor Haas <[email protected]> wrote:

The existing memregion handling code deeply embeds the assumption that we will be isolating domains using PMP/SMEPMP rather than another isolation primitive. I am currently working on a series of patches that will enable SMMTT [1] support in both OpenSBI and QEMU. SMMTT is a much more granular, table-based isolation primitive which does not have the same alignment/sizing requirements as PMP.

This patch series lays some of the groundwork for landing SMMTT support in OpenSBI. Perhaps the most significant change here is that memregions now keep track of their raw sizes (rather than power-of-two rounded orders) throughout most of their lifetime. Then, when OpenSBI is about to apply its isolation configuration at the end of the coldboot/warmboot paths, the memregions for each domain are "sanitized" according to isolation primitive-specific rules.

The first two patches in this series are optional to apply, although I've found them useful when developing SMMTT. The first moves the memregion-related functionality out of sbi_domain.c into a new sbi_memregion.c. The second renames struct sbi_domain_memregion to struct sbi_memregion, while also renaming all of the memregion permission macros similarly. This makes the code somewhat more readable, especially since domain-handling and memregion-handling become more decoupled throughout this patch series.

[1] https://github.com/riscv/riscv-smmtt

Quoted 8 lines
Quoted 4 lines

Gregor Haas (8):
lib: sbi: Separate domain-handling code from memregion-handling code
lib: sbi: Create sbi_domain_dump_memregions()
lib: sbi: Rename domain_memregion -> memregion

I think it's better to keep the sbi_domain_ prefix for memregion in the filename as well as functions/structures for consistency. The sbi_domain_context also preserves the sbi_domain_ prefix

Sounds good, I'll drop this change from my next revision. To clarify, are you okay with the first optional patch in this series
(Separate domain-handling code from memregion-handling code)?

Quoted 10 lines
Quoted 3 lines

lib: sbi: memregion: Introduce memregion_start/end macros
lib: sbi: memregion: Make memregions keep track of raw size rather
than order

We will be using multiple HW isolation mechanisms in the long run. Please see more comments below.

Converting range based memregions (start, end) into order-of-2 overlapping memregions on-the-fly for PMP and ePMP will be painful for domain context switching.

In my current patches, this conversion takes place in sbi_memregion_sanitize() which is called both in sbi_domain_sanitize() (although no conversion takes place here) and in sbi_hart_{sme}pmp_configure(), where conversion does happen since we now know which isolation primitive will be used. Currently, memregions are converted in-place. I can add some functionality here to detect whether the memregions have already been converted, and to return early if so. This would keep conversion out of the domain context-switching path, except for the first switch.

Quoted 2 lines

It's better to stay with order-of-2 overlapping memregions and translate these to Smmtt table mappings.

My main concern with this approach is that this conversion process is lossy -- when we round into NAPOT, we often end up not only modifying the base address but also increasing the region's size. In SMMTT, this can have the effect of significantly increasing table memory utilization. Storing the region's full base+bounds information leads to more efficient resource utilization down the line.

Quoted 6 lines
Quoted 5 lines

lib: sbi: Do a merging pass on memregions whenever a domain is
sanitized
lib: sbi: Remove PMP assumptions from memregion
lib: sbi: memregion: Move memregion_sanitize_pmp() closer to its call
site

Quoted 6 lines

The Smmtt only allows specifying permissions for S-mode domains but we still need mechanisms (such as PMP, ePMP) to specify permissions for M-mode firmware. This is for restricting M-mode access to only specific parts of memory and IO. This means, we will be having multiple HW insolation mechanisms which co-exists in M-mode.

I admit that I did not see this case :) it seems that we will need to, for each memregion, decide which isolation primitive should be used depending on the region's permissions and the available primitives. This is not something my patches currently do.

Quoted 10 lines

My suggestion is the following:
1) Introduce sbi_domain_isolation instance (or something similar) which represents a HW isolation mechanism. The isolation mechanism will have its own capabilities such as what kind of permission checks it can enforce.
2) There can be multiple sbi_domain_isolation instances (e.g. PMP, ePMP, Smmtt, IOPMP, World Guard, etc). Some of these isolation mechanisms are per-hart so need to be context switched when we switch domain while other isolation mechanisms will be only for IOs.
3) Convert existing PMP and ePMP implementation into a per-hart sbi_domain_isolation instance.

Something like:

struct sbi_domain_isolation {
    unsigned long supported_flags;
    bool per_hart;
    int (*sanitize)(struct sbi_domain_memregion *);
    bool (*region_valid)(struct sbi_domain_memregion *);
};

And then use the existing carray machinery to separate out isolation implementations into individual source files?

Quoted 6 lines

4) Implement Smmtt as yet another per-hart sbi_domain_isolation instance with different capabilities compared to PMP and ePMP. This Smmtt isolation mechanism will translate order-of-2 regions to mappings in the Smmtt table. Each domain will have a separate smmtt table so for domain context switch, we only need to switch the Smmtt table pointer.

This makes sense to me, but again -- I think it would be worth it to carry around
the unrounded memregion information until this point so that we can use table memory as efficiently as possible.

Quoted 35 lines
Quoted 35 lines
 docs/domain_support.md                   |  45 ++-
 include/sbi/sbi_domain.h                 | 192 +---------
 include/sbi/sbi_hart.h                   |   2 +-
 include/sbi/sbi_memregion.h              | 215 +++++++++++
 include/sbi/sbi_platform.h               |   2 +-
 lib/sbi/objects.mk                       |   1 +
 lib/sbi/sbi_domain.c                     | 444 +++-------------------
 lib/sbi/sbi_domain_context.c             |   2 +-
 lib/sbi/sbi_hart.c                       |  91 +++--
 lib/sbi/sbi_init.c                       |   6 +-
 lib/sbi/sbi_memregion.c                  | 459 +++++++++++++++++++++++
 lib/utils/fdt/fdt_domain.c               |  53 +--
 lib/utils/fdt/fdt_fixup.c                |  20 +-
 lib/utils/ipi/aclint_mswi.c              |  12 +-
 lib/utils/ipi/andes_plicsw.c             |   6 +-
 lib/utils/irqchip/aplic.c                |  12 +-
 lib/utils/irqchip/imsic.c                |  13 +-
 lib/utils/irqchip/plic.c                 |   4 +-
 lib/utils/regmap/fdt_regmap_syscon.c     |   4 +-
 lib/utils/serial/cadence-uart.c          |   4 +-
 lib/utils/serial/fdt_serial_htif.c       |   4 +-
 lib/utils/serial/uart8250.c              |   4 +-
 lib/utils/timer/aclint_mtimer.c          |  24 +-
 lib/utils/timer/andes_plmt.c             |   6 +-
 platform/generic/renesas/rzfive/rzfive.c |   2 +-
 platform/generic/sophgo/sg2042.c         |   6 +-
 26 files changed, 910 insertions(+), 723 deletions(-)
 create mode 100644 include/sbi/sbi_memregion.h
 create mode 100644 lib/sbi/sbi_memregion.c

--
2.45.2

Quoted 7 lines
Quoted 4 lines

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Regards,
Anup

Looking forward to your response!

Thanks,
Gregor Haas

A message that is not in the archive (sent off-list or not yet received): CAMqWt3qi+ZXRGNH=u4rR0X9Tqv8wt+5zUpoCfvjFkP1n8Rj5zA@mail.gmail.com

#12 lore raw

cover Prepare memregion handling for future isolation primitives

Quoted 37 lines · On Fri, Sep 20, 2024 at 1:32?AM Gregor Haas <[email protected]> wrote:

Hi Anup,

Thanks for your comments!

Quoted 31 lines · On Wed, Sep 18, 2024 at 11:16?PM Anup Patel <[email protected]> wrote:
Quoted 28 lines · On Wed, Jul 31, 2024 at 11:46?PM Gregor Haas <[email protected]> wrote:

The existing memregion handling code deeply embeds the assumption that we will be isolating domains using PMP/SMEPMP rather than another isolation primitive. I am currently working on a series of patches that will enable SMMTT [1] support in both OpenSBI and QEMU. SMMTT is a much more granular, table-based isolation primitive which does not have the same alignment/sizing requirements as PMP.

This patch series lays some of the groundwork for landing SMMTT support in OpenSBI. Perhaps the most significant change here is that memregions now keep track of their raw sizes (rather than power-of-two rounded orders) throughout most of their lifetime. Then, when OpenSBI is about to apply its isolation configuration at the end of the coldboot/warmboot paths, the memregions for each domain are "sanitized" according to isolation primitive-specific rules.

The first two patches in this series are optional to apply, although I've found them useful when developing SMMTT. The first moves the memregion-related functionality out of sbi_domain.c into a new sbi_memregion.c. The second renames struct sbi_domain_memregion to struct sbi_memregion, while also renaming all of the memregion permission macros similarly. This makes the code somewhat more readable, especially since domain-handling and memregion-handling become more decoupled throughout this patch series.

[1] https://github.com/riscv/riscv-smmtt

Gregor Haas (8):
lib: sbi: Separate domain-handling code from memregion-handling code
lib: sbi: Create sbi_domain_dump_memregions()
lib: sbi: Rename domain_memregion -> memregion

Quoted 8 lines
Quoted 3 lines

I think it's better to keep the sbi_domain_ prefix for memregion in the filename as well as functions/structures for consistency. The sbi_domain_context also preserves the sbi_domain_ prefix

Sounds good, I'll drop this change from my next revision. To clarify, are you okay with the first optional patch in this series
(Separate domain-handling code from memregion-handling code)?

Yes, factoring-out sbi_domain_memregion to separate sources is fine.

Quoted 14 lines
Quoted 6 lines
Quoted 5 lines

lib: sbi: memregion: Introduce memregion_start/end macros
lib: sbi: memregion: Make memregions keep track of raw size rather
than order

Quoted 6 lines

We will be using multiple HW isolation mechanisms in the long run. Please see more comments below.

Converting range based memregions (start, end) into order-of-2 overlapping memregions on-the-fly for PMP and ePMP will be painful for domain context switching.

Quoted 8 lines

In my current patches, this conversion takes place in sbi_memregion_sanitize() which is called both in sbi_domain_sanitize() (although no conversion takes place here) and in sbi_hart_{sme}pmp_configure(), where conversion does happen since we now know which isolation primitive will be used. Currently, memregions are converted in-place. I can add some functionality here to detect whether the memregions have already been converted, and to return early if so. This would keep conversion out of the domain context-switching path, except for the first switch.

Instead of memregion conversion business, it's better to keep both the NAPOT order and actual range in the memregion. Both these attributes can be populated at the time of memregion registration. The HW isolation mechanism can either pick the NAPOT order of the memregion or actual range.

I would also insist that we preserve the merging business in sbi_domain_sanitize() because the number of PMP regions on a CPU are generally limited.

Quoted 7 lines
Quoted 5 lines

It's better to stay with order-of-2 overlapping memregions and translate these

to Smmtt table mappings.

Quoted 6 lines

My main concern with this approach is that this conversion process is lossy -- when we round into NAPOT, we often end up not only modifying the base address but also increasing the region's size. In SMMTT, this can have the effect of significantly increasing table memory utilization. Storing the region's full base+bounds information leads to more efficient resource utilization down the line.

On the contrary, I think it will benefit Smmtt without much impact on table memory utilization because it is easy to map NAPOT regions to higher order mappings in the Smmtt table.

Probably, there is no right answer here and it is better to keep both information at the time of registering the memregions.

Quoted 44 lines
Quoted 8 lines
Quoted 7 lines

lib: sbi: Do a merging pass on memregions whenever a domain is
sanitized
lib: sbi: Remove PMP assumptions from memregion
lib: sbi: memregion: Move memregion_sanitize_pmp() closer to its call
site

Quoted 6 lines

The Smmtt only allows specifying permissions for S-mode domains but we still need mechanisms (such as PMP, ePMP) to specify permissions for M-mode firmware. This is for restricting M-mode access to only specific parts of memory and IO. This means, we will be having multiple HW insolation mechanisms which co-exists in M-mode.

I admit that I did not see this case :) it seems that we will need to, for each memregion, decide which isolation primitive should be used depending on the region's permissions and the available primitives. This is not something my patches currently do.

Quoted 6 lines

My suggestion is the following:
1) Introduce sbi_domain_isolation instance (or something similar) which represents a HW isolation mechanism. The isolation mechanism will have its own capabilities such as what kind of permission checks it can enforce.

Quoted 6 lines

2) There can be multiple sbi_domain_isolation instances (e.g. PMP, ePMP, Smmtt, IOPMP, World Guard, etc). Some of these isolation mechanisms are per-hart so need to be context switched when we switch domain while other isolation mechanisms will be only for IOs.
3) Convert existing PMP and ePMP implementation into a per-hart sbi_domain_isolation instance.

Something like:

struct sbi_domain_isolation {
    unsigned long supported_flags;
    bool per_hart;
    int (*sanitize)(struct sbi_domain_memregion *);
    bool (*region_valid)(struct sbi_domain_memregion *);
};
Quoted 2 lines

And then use the existing carray machinery to separate out isolation implementations into individual source files?

The carray machinery might not be required if we initially focus only on PMP/ePMP and Smmtt because both are ISA features which can be detected at boot-time. For example, if PMP/ePMP is available then sbi_hart_init() can register PMP/ePMP isolation in the coldboot init path.

I am thinking something like below ...

struct sbi_domain_isolation {
struct sbi_dlist head; /* For adding to list of registered
isolation mech. */
const char *name: /* For boot-time advertising registered isolation mech. */ bool per_hart;
unsigned long supported_flags; /* Capabilities */
int (*setup)(struct sbi_domain *dom); /* Called once when a domain
is registered */
void (*cleanup)(struct sbi_domain *dom); /* Called once when a
domain is unregis. */

    int (*switch)(struct sbi_domain *next, struct sbi_domain *prev);
                                  /* prev = NULL means we are
configuring at boot-time */
    int (*map_smode)(unsigned long base, unsigned long size);
    int (*unmap_smode(void);
         /* map_smode()/unmap_smode() is used for temporary mapping

some s-mode area */
};

Quoted 10 lines
Quoted 8 lines

4) Implement Smmtt as yet another per-hart sbi_domain_isolation instance with different capabilities compared to PMP and ePMP. This Smmtt isolation mechanism will translate order-of-2 regions to mappings in the Smmtt table. Each domain will have a separate smmtt table so for domain context switch, we only need to switch the Smmtt table pointer.

Quoted 3 lines

This makes sense to me, but again -- I think it would be worth it to carry around the unrounded memregion information until this point so that we can use table memory as efficiently as possible.

I sort of agree but see above comments.

Quoted 42 lines
Quoted 42 lines
Quoted 41 lines


 docs/domain_support.md                   |  45 ++-
 include/sbi/sbi_domain.h                 | 192 +---------
 include/sbi/sbi_hart.h                   |   2 +-
 include/sbi/sbi_memregion.h              | 215 +++++++++++
 include/sbi/sbi_platform.h               |   2 +-
 lib/sbi/objects.mk                       |   1 +
 lib/sbi/sbi_domain.c                     | 444 +++-------------------
 lib/sbi/sbi_domain_context.c             |   2 +-
 lib/sbi/sbi_hart.c                       |  91 +++--
 lib/sbi/sbi_init.c                       |   6 +-
 lib/sbi/sbi_memregion.c                  | 459 +++++++++++++++++++++++
 lib/utils/fdt/fdt_domain.c               |  53 +--
 lib/utils/fdt/fdt_fixup.c                |  20 +-
 lib/utils/ipi/aclint_mswi.c              |  12 +-
 lib/utils/ipi/andes_plicsw.c             |   6 +-
 lib/utils/irqchip/aplic.c                |  12 +-
 lib/utils/irqchip/imsic.c                |  13 +-
 lib/utils/irqchip/plic.c                 |   4 +-
 lib/utils/regmap/fdt_regmap_syscon.c     |   4 +-
 lib/utils/serial/cadence-uart.c          |   4 +-
 lib/utils/serial/fdt_serial_htif.c       |   4 +-
 lib/utils/serial/uart8250.c              |   4 +-
 lib/utils/timer/aclint_mtimer.c          |  24 +-
 lib/utils/timer/andes_plmt.c             |   6 +-
 platform/generic/renesas/rzfive/rzfive.c |   2 +-
 platform/generic/sophgo/sg2042.c         |   6 +-
 26 files changed, 910 insertions(+), 723 deletions(-)
 create mode 100644 include/sbi/sbi_memregion.h
 create mode 100644 lib/sbi/sbi_memregion.c

--
2.45.2

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Quoted 6 lines
Quoted 2 lines

Regards,
Anup

Looking forward to your response!

Thinking about this more, we will also need some infrastructure in sbi_domain to allow different parts of OpenSBI to save per-domain data. The HW isolation mechanisms might also want to save stuff on a per-domain basis. I will try to send this as a separate series.

Regards,
Anup

#13 lore raw

cover Prepare memregion handling for future isolation primitives

Quoted 97 lines · On Fri, Sep 20, 2024 at 11:01?AM Anup Patel <[email protected]> wrote:
Quoted 37 lines · On Fri, Sep 20, 2024 at 1:32?AM Gregor Haas <[email protected]> wrote:

Hi Anup,

Thanks for your comments!

Quoted 31 lines · On Wed, Sep 18, 2024 at 11:16?PM Anup Patel <[email protected]> wrote:
Quoted 28 lines · On Wed, Jul 31, 2024 at 11:46?PM Gregor Haas <[email protected]> wrote:

The existing memregion handling code deeply embeds the assumption that we will be isolating domains using PMP/SMEPMP rather than another isolation primitive. I am currently working on a series of patches that will enable SMMTT [1] support in both OpenSBI and QEMU. SMMTT is a much more granular, table-based isolation primitive which does not have the same alignment/sizing requirements as PMP.

This patch series lays some of the groundwork for landing SMMTT support in OpenSBI. Perhaps the most significant change here is that memregions now keep track of their raw sizes (rather than power-of-two rounded orders) throughout most of their lifetime. Then, when OpenSBI is about to apply its isolation configuration at the end of the coldboot/warmboot paths, the memregions for each domain are "sanitized" according to isolation primitive-specific rules.

The first two patches in this series are optional to apply, although I've found them useful when developing SMMTT. The first moves the memregion-related functionality out of sbi_domain.c into a new sbi_memregion.c. The second renames struct sbi_domain_memregion to struct sbi_memregion, while also renaming all of the memregion permission macros similarly. This makes the code somewhat more readable, especially since domain-handling and memregion-handling become more decoupled throughout this patch series.

[1] https://github.com/riscv/riscv-smmtt

Gregor Haas (8):
lib: sbi: Separate domain-handling code from memregion-handling code
lib: sbi: Create sbi_domain_dump_memregions()
lib: sbi: Rename domain_memregion -> memregion

Quoted 8 lines
Quoted 3 lines

I think it's better to keep the sbi_domain_ prefix for memregion in the filename as well as functions/structures for consistency. The sbi_domain_context also preserves the sbi_domain_ prefix

Sounds good, I'll drop this change from my next revision. To clarify, are you okay with the first optional patch in this series
(Separate domain-handling code from memregion-handling code)?

Yes, factoring-out sbi_domain_memregion to separate sources is fine.

Quoted 14 lines
Quoted 6 lines
Quoted 5 lines

lib: sbi: memregion: Introduce memregion_start/end macros
lib: sbi: memregion: Make memregions keep track of raw size rather
than order

Quoted 6 lines

We will be using multiple HW isolation mechanisms in the long run. Please see more comments below.

Converting range based memregions (start, end) into order-of-2 overlapping memregions on-the-fly for PMP and ePMP will be painful for domain context switching.

Quoted 8 lines

In my current patches, this conversion takes place in sbi_memregion_sanitize() which is called both in sbi_domain_sanitize() (although no conversion takes place here) and in sbi_hart_{sme}pmp_configure(), where conversion does happen since we now know which isolation primitive will be used. Currently, memregions are converted in-place. I can add some functionality here to detect whether the memregions have already been converted, and to return early if so. This would keep conversion out of the domain context-switching path, except for the first switch.

Instead of memregion conversion business, it's better to keep both the NAPOT order and actual range in the memregion. Both these attributes can be populated at the time of memregion registration. The HW isolation mechanism can either pick the NAPOT order of the memregion or actual range.

I would also insist that we preserve the merging business in sbi_domain_sanitize() because the number of PMP regions on a CPU are generally limited.

Quoted 7 lines
Quoted 5 lines

It's better to stay with order-of-2 overlapping memregions and translate these

to Smmtt table mappings.

Quoted 6 lines

My main concern with this approach is that this conversion process is lossy -- when we round into NAPOT, we often end up not only modifying the base address but also increasing the region's size. In SMMTT, this can have the effect of significantly increasing table memory utilization. Storing the region's full base+bounds information leads to more efficient resource utilization down the line.

Quoted 6 lines

On the contrary, I think it will benefit Smmtt without much impact on table memory utilization because it is easy to map NAPOT regions to higher order mappings in the Smmtt table.

Probably, there is no right answer here and it is better to keep both information at the time of registering the memregions.

I suggest that for now we stay with NAPOT regions until we have fully implemented Smmtt with multiple HW isolation mechanisms coexisting. We can re-evaluate the benefits of linear regions after that.

Regards,
Anup

Quoted 146 lines
Quoted 45 lines
Quoted 9 lines
Quoted 8 lines

lib: sbi: Do a merging pass on memregions whenever a domain is
sanitized
lib: sbi: Remove PMP assumptions from memregion
lib: sbi: memregion: Move memregion_sanitize_pmp() closer to its call
site

Quoted 6 lines

The Smmtt only allows specifying permissions for S-mode domains but we still need mechanisms (such as PMP, ePMP) to specify permissions for M-mode firmware. This is for restricting M-mode access to only specific parts of memory and IO. This means, we will be having multiple HW insolation mechanisms which co-exists in M-mode.

I admit that I did not see this case :) it seems that we will need to, for each memregion, decide which isolation primitive should be used depending on the region's permissions and the available primitives. This is not something my patches currently do.

Quoted 6 lines

My suggestion is the following:
1) Introduce sbi_domain_isolation instance (or something similar) which represents a HW isolation mechanism. The isolation mechanism will have its own capabilities such as what kind of permission checks it can enforce.

Quoted 6 lines

2) There can be multiple sbi_domain_isolation instances (e.g. PMP, ePMP, Smmtt, IOPMP, World Guard, etc). Some of these isolation mechanisms are per-hart so need to be context switched when we switch domain while other isolation mechanisms will be only for IOs.
3) Convert existing PMP and ePMP implementation into a per-hart sbi_domain_isolation instance.

Something like:

struct sbi_domain_isolation {
    unsigned long supported_flags;
    bool per_hart;
    int (*sanitize)(struct sbi_domain_memregion *);
    bool (*region_valid)(struct sbi_domain_memregion *);
};
Quoted 2 lines

And then use the existing carray machinery to separate out isolation implementations into individual source files?

The carray machinery might not be required if we initially focus only on PMP/ePMP and Smmtt because both are ISA features which can be detected at boot-time. For example, if PMP/ePMP is available then sbi_hart_init() can register PMP/ePMP isolation in the coldboot init path.

I am thinking something like below ...

struct sbi_domain_isolation {
struct sbi_dlist head; /* For adding to list of registered
isolation mech. */
const char *name: /* For boot-time advertising registered isolation mech. */ bool per_hart;
unsigned long supported_flags; /* Capabilities */
int (*setup)(struct sbi_domain *dom); /* Called once when a domain
is registered */
void (*cleanup)(struct sbi_domain *dom); /* Called once when a
domain is unregis. */

    int (*switch)(struct sbi_domain *next, struct sbi_domain *prev);
                                  /* prev = NULL means we are
configuring at boot-time */
    int (*map_smode)(unsigned long base, unsigned long size);
    int (*unmap_smode(void);
         /* map_smode()/unmap_smode() is used for temporary mapping

some s-mode area */
};

Quoted 10 lines
Quoted 8 lines

4) Implement Smmtt as yet another per-hart sbi_domain_isolation instance with different capabilities compared to PMP and ePMP. This Smmtt isolation mechanism will translate order-of-2 regions to mappings in the Smmtt table. Each domain will have a separate smmtt table so for domain context switch, we only need to switch the Smmtt table pointer.

Quoted 3 lines

This makes sense to me, but again -- I think it would be worth it to carry around the unrounded memregion information until this point so that we can use table memory as efficiently as possible.

I sort of agree but see above comments.

Quoted 42 lines
Quoted 42 lines
Quoted 41 lines


 docs/domain_support.md                   |  45 ++-
 include/sbi/sbi_domain.h                 | 192 +---------
 include/sbi/sbi_hart.h                   |   2 +-
 include/sbi/sbi_memregion.h              | 215 +++++++++++
 include/sbi/sbi_platform.h               |   2 +-
 lib/sbi/objects.mk                       |   1 +
 lib/sbi/sbi_domain.c                     | 444 +++-------------------
 lib/sbi/sbi_domain_context.c             |   2 +-
 lib/sbi/sbi_hart.c                       |  91 +++--
 lib/sbi/sbi_init.c                       |   6 +-
 lib/sbi/sbi_memregion.c                  | 459 +++++++++++++++++++++++
 lib/utils/fdt/fdt_domain.c               |  53 +--
 lib/utils/fdt/fdt_fixup.c                |  20 +-
 lib/utils/ipi/aclint_mswi.c              |  12 +-
 lib/utils/ipi/andes_plicsw.c             |   6 +-
 lib/utils/irqchip/aplic.c                |  12 +-
 lib/utils/irqchip/imsic.c                |  13 +-
 lib/utils/irqchip/plic.c                 |   4 +-
 lib/utils/regmap/fdt_regmap_syscon.c     |   4 +-
 lib/utils/serial/cadence-uart.c          |   4 +-
 lib/utils/serial/fdt_serial_htif.c       |   4 +-
 lib/utils/serial/uart8250.c              |   4 +-
 lib/utils/timer/aclint_mtimer.c          |  24 +-
 lib/utils/timer/andes_plmt.c             |   6 +-
 platform/generic/renesas/rzfive/rzfive.c |   2 +-
 platform/generic/sophgo/sg2042.c         |   6 +-
 26 files changed, 910 insertions(+), 723 deletions(-)
 create mode 100644 include/sbi/sbi_memregion.h
 create mode 100644 lib/sbi/sbi_memregion.c

--
2.45.2

--
opensbi mailing list
opensbi at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/opensbi

Quoted 6 lines
Quoted 2 lines

Regards,
Anup

Looking forward to your response!

Thinking about this more, we will also need some infrastructure in sbi_domain to allow different parts of OpenSBI to save per-domain data. The HW isolation mechanisms might also want to save stuff on a per-domain basis. I will try to send this as a separate series.

Regards,
Anup