Initial ESWIN/EIC7700 support
v3 cover Initial ESWIN/EIC7700 support
EIC7700 is the SoC used in HiFive P550 and Milk-V Megrez. This SoC is currently one of the only off-the-shelf board/chips that support H extension, although it's v0.6.1. It also supports pre-ratified N-trace. Add support for it so people can benefit from latest OpenSBI features.
The device-tree of HiFive P550 has been upstreamed to Linux:
https://lore.kernel.org/all/[email protected]/
However U-boot is not, and there are bugs in vendor U-boot device-tree, and also inconsistencies between the two. Thus, this patch is coded with the upstreamed device-tree as the reference, but tested with the patched vendor U-boot device tree as FW_FDT_PATH. The patched vendor U-boot is hosted here: https://github.com/ganboing/u-boot-eic7x/tree/eic7x-dt-fix
Refer to the last PATCH for the instructions on building the firmware blob and launch it through UART boot.
The major complication of this chip is that it requires certain memory regions to be blocked with PMP entries to prevent speculative execution or HW prefetcher from touching the data-cacheable regions within to avoid bus errors. Due to the fact that this SoC handles cache incoherent DMA by mapping memory twice, one as cached, and the other as uncached, we also need an extra PMP to protect the OpenSBI in the uncached portion in address space. The PMP handling is tricky, so I documented it very extensively for people to reason about it. I managed to get it done with only NAPOT PMP entries and still got 1 free PMP for root harts for die 0 (No free PMP for die 1 root harts). This even permits a udomain/tdomain like partitioning, so we can even try out TEEs. Sample boot log:
OpenSBI v1.7-73-g5c235e5d
____ _____ ____ _____
/ __ \ / ____| _ \_ _|
| | | |_ __ ___ _ __ | (___ | |_) || |
| | | | '_ \ / _ \ '_ \ \___ \| _ < | |
| |__| | |_) | __/ | | |____) | |_) || |_
\____/| .__/ \___|_| |_|_____/|____/_____|
| |
|_|Platform Name : SiFive HiFive Premier P550
Platform Features : medeleg
Platform HART Count : 4
Platform IPI Device : aclint-mswi
Platform Timer Device : aclint-mtimer @ 1000000Hz
Platform Console Device : uart8250
Platform HSM Device : ---
Platform PMU Device : ---
Platform Reboot Device : eic770x_reset
Platform Shutdown Device : ---
Platform Suspend Device : ---
Platform CPPC Device : ---
Firmware Base : 0x80000000
Firmware Size : 613 KB
Firmware RW Offset : 0x80000
Firmware RW Size : 101 KB
Firmware Heap Offset : 0x8d000
Firmware Heap Size : 49 KB (total), 1 KB (reserved), 14 KB (used), 33 KB (free)
Firmware Scratch Size : 4096 B (total), 424 B (used), 3672 B (free)
Runtime SBI Version : 3.0
Standard SBI Extensions : time,rfnc,ipi,base,hsm,srst,pmu,dbcn,fwft,legacy,dbtr,sse
Experimental SBI Extensions : none
Domain0 Name : root
Domain0 Boot HART : 0
Domain0 HARTs : 0*,1*,2*,3*
Domain0 Region00 : 0x0000000080000000-0x00000000800fffff M: (F,R,W,X) S/U: ()
Domain0 Region01 : 0x000000c000000000-0x000000c0000fffff M: (F,R,W,X) S/U: ()
Domain0 Region02 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain0 Region03 : 0x0000000000000000-0x000000007fffffff M: (I,R,W) S/U: (R,W)
Domain0 Region04 : 0x0000000000000000-0x0000000fffffffff M: (R,W) S/U: (R,W,X)
Domain0 Region05 : 0x0000000000000000-0x0000007fffffffff M: (I) S/U: ()
Domain0 Region06 : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X)
Domain0 Next Address : 0x0000000080200000
Domain0 Next Arg1 : 0x00000000f8000000
Domain0 Next Mode : S-mode
Domain0 SysReset : yes
Domain0 SysSuspend : yes
Domain1 Name : trusted-domain
Domain1 Boot HART : -1
Domain1 HARTs : Domain1 Region00 : 0x0000000080000000-0x00000000800fffff M: (F,R,W,X) S/U: () Domain1 Region01 : 0x000000c000000000-0x000000c0000fffff M: (F,R,W,X) S/U: () Domain1 Region02 : 0x0000000050910000-0x0000000050910fff M: (I,R,W) S/U: (R,W) Domain1 Region03 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: () Domain1 Region04 : 0x000000047ff00000-0x000000047fffffff M: () S/U: (R,W,X) Domain1 Region05 : 0x0000000000000000-0x000000007fffffff M: (I,R,W) S/U: () Domain1 Region06 : 0x0000000000000000-0x0000007fffffffff M: (I) S/U: () Domain1 Next Address : 0x000000047ff00000 Domain1 Next Arg1 : 0x0000000000000000 Domain1 Next Mode : S-mode Domain1 SysReset : no Domain1 SysSuspend : no
Domain2 Name : untrusted-domain
Domain2 Boot HART : -1
Domain2 HARTs : Domain2 Region00 : 0x0000000080000000-0x00000000800fffff M: (F,R,W,X) S/U: () Domain2 Region01 : 0x000000c000000000-0x000000c0000fffff M: (F,R,W,X) S/U: () Domain2 Region02 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: () Domain2 Region03 : 0x000000047ff00000-0x000000047fffffff M: () S/U: () Domain2 Region04 : 0x0000000000000000-0x0000000fffffffff M: (R,W) S/U: (R,W,X) Domain2 Region05 : 0x0000000000000000-0x0000007fffffffff M: (I) S/U: () Domain2 Region06 : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X) Domain2 Next Address : 0x0000000080000000 Domain2 Next Arg1 : 0x00000000f8000000 Domain2 Next Mode : S-mode Domain2 SysReset : no Domain2 SysSuspend : no
Boot HART ID : 0
Boot HART Domain : root
Boot HART Priv Version : v1.11
Boot HART Base ISA : rv64imafdchx
Boot HART ISA Extensions : sscofpmf,zihpm,sdtrig
Boot HART PMP Count : 8
Boot HART PMP Granularity : 12 bits
Boot HART PMP Address Bits : 39
Boot HART MHPM Info : 4 (0x00000078)
Boot HART Debug Triggers : 4 triggers
Boot HART MIDELEG : 0x0000000000002666
Boot HART MEDELEG : 0x0000000000f0b509
---
Changes in v3:
- Figure out the cause behind bus error, and document it properly
- Drop the consolidation logic and let the lib/memregion logic to optimize out unnecessary regions -- simplifies many things.
- Better and more comprehensive comments in source code.
- Support tdomain/udomain like use cases on die 0.
Changes in v2:
- Major enhancement of PMP consolidation logic. Also fixed a Linux Panic bug due to the mismatch between PMP settings and reserved memory regions passed to Linux via FDT.
- Also protects the OpenSBI firmware in uncached memory portion of address space.
- More detailed documentation on EIC770X/P550
---
Bo Gan (4):
lib: sbi: allow platform to override PMP (un)configuration
lib: sbi: give platform choice of using single memregion to cover
OpenSBI
include: sbi_domain: make is_region_subset public
platform: generic: eswin: add EIC7700
include/sbi/sbi_domain.h | 29 ++ include/sbi/sbi_hart.h | 3 + include/sbi/sbi_platform.h | 74 +++++ lib/sbi/sbi_domain.c | 80 +++-- lib/sbi/sbi_domain_context.c | 11 +- lib/sbi/sbi_hart.c | 45 ++- platform/generic/Kconfig | 5 + platform/generic/configs/defconfig | 1 + platform/generic/eswin/Kconfig | 29 ++ platform/generic/eswin/eic770x.c | 385 +++++++++++++++++++++++ platform/generic/eswin/objects.mk | 11 + platform/generic/include/eswin/eic770x.h | 73 +++++ 12 files changed, 687 insertions(+), 59 deletions(-) create mode 100644 platform/generic/eswin/Kconfig create mode 100644 platform/generic/eswin/eic770x.c create mode 100644 platform/generic/eswin/objects.mk create mode 100644 platform/generic/include/eswin/eic770x.h
Signature · 7 lines
2.34.1 -- opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
PATCH v3 1/4 lib: sbi: allow platform to override PMP (un)configuration
Platform sometimes wants to override the entire PMP (un)config phase, not just performing additional work for each individual PMP entry. This allows platforms to insert SoC/core specific PMP entries in a way that works together with the existing ones set by lib/ code, not conflicting. platform can also choose to merge or skip memory regions in a reasonable way in case there's a shortage of PMP entries.
In addition, logic for determining flags in sbi_hart_oldpmp_configure is abstracted out as sbi_domain_get_oldpmp_flags, analogous to the sbi_domain_get_smepmp_flags for smepmp. so platform with traditional PMP can leverage it in its own pmp_configure
--- include/sbi/sbi_domain.h | 7 +++++ include/sbi/sbi_hart.h | 3 ++ include/sbi/sbi_platform.h | 53 ++++++++++++++++++++++++++++++++++++ lib/sbi/sbi_domain.c | 21 ++++++++++++++ lib/sbi/sbi_domain_context.c | 11 +------- lib/sbi/sbi_hart.c | 45 ++++++++++++++++++------------ 6 files changed, 113 insertions(+), 27 deletions(-)
include/sbi/sbi_domain.h +7 -0
@@ -253,6 +253,13 @@ void sbi_domain_memregion_init(unsigned long addr,unsigned long flags, struct sbi_domain_memregion *reg); +/** +* Return the oldpmp pmpcfg LRWX encoding for the flags in @reg. +* +* @param reg pointer to memory region; its flags field encodes permissions. +*/ +unsigned int sbi_domain_get_oldpmp_flags (struct sbi_domain_memregion *reg); +/** * Return the Smepmp pmpcfg LRWX encoding for the flags in @reg. *
include/sbi/sbi_hart.h +3 -0
@@ -133,6 +133,8 @@ struct sbi_hart_features { };struct sbi_scratch ; +struct sbi_domain ; +struct sbi_domain_memregion ;int sbi_hart_reinit (struct sbi_scratch *scratch);int sbi_hart_init (struct sbi_scratch *scratch,bool cold_boot);
@@ -147,6 +149,7 @@ unsigned int sbi_hart_pmp_log2gran(struct sbi_scratch *scratch);unsigned int sbi_hart_pmp_addrbits (struct sbi_scratch *scratch);unsigned int sbi_hart_mhpm_bits (struct sbi_scratch *scratch);bool sbi_hart_smepmp_is_fw_region (unsigned int pmp_idx); +void sbi_hart_pmp_unconfigure (struct sbi_scratch *scratch);int sbi_hart_pmp_configure (struct sbi_scratch *scratch);int sbi_hart_map_saddr (unsigned long base,unsigned long size);int sbi_hart_unmap_saddr (void );
include/sbi/sbi_platform.h +53 -0
65 lines not on this page · raw message
lib/sbi/sbi_domain.c +21 -0
@@ -122,6 +122,27 @@ void sbi_domain_memregion_init(unsigned long addr, } } +unsigned int sbi_domain_get_oldpmp_flags (struct sbi_domain_memregion *reg) +{ +unsigned int pmp_flags =0 ; + +/* +* If permissions are to be enforced for all modes on +* this region, the lock bit should be set. +*/ +if (reg->flags &SBI_DOMAIN_MEMREGION_ENF_PERMISSIONS ) + pmp_flags |=PMP_L ; + +if (reg->flags &SBI_DOMAIN_MEMREGION_SU_READABLE ) + pmp_flags |=PMP_R ; +if (reg->flags &SBI_DOMAIN_MEMREGION_SU_WRITABLE ) + pmp_flags |=PMP_W ; +if (reg->flags &SBI_DOMAIN_MEMREGION_SU_EXECUTABLE ) + pmp_flags |=PMP_X ; + +return pmp_flags; +} +unsigned int sbi_domain_get_smepmp_flags (struct sbi_domain_memregion *reg) {unsigned int pmp_flags =0 ;
lib/sbi/sbi_domain_context.c +1 -10
@@ -102,7 +102,6 @@ static int switch_to_next_domain_context(struct hart_context *ctx,struct sbi_trap_context *trap_ctx;struct sbi_domain *current_dom, *target_dom;struct sbi_scratch *scratch =sbi_scratch_thishart_ptr (); -unsigned int pmp_count =sbi_hart_pmp_count (scratch);if (!ctx || !dom_ctx || ctx == dom_ctx)return SBI_EINVAL ;
@@ -120,15 +119,7 @@ static int switch_to_next_domain_context(struct hart_context *ctx,sbi_hartmask_set_hartindex (hartindex, &target_dom->assigned_harts);spin_unlock (&target_dom->assigned_harts_lock); -/* Reconfigure PMP settings for the new domain */ -for (int i =0 ; i < pmp_count; i++) { -/* Don't revoke firmware access permissions */ -if (sbi_hart_smepmp_is_fw_region (i)) -continue ; - -sbi_platform_pmp_disable (sbi_platform_thishart_ptr (), i); -pmp_disable (i); - } +sbi_hart_pmp_unconfigure (scratch);sbi_hart_pmp_configure (scratch);/* Save current CSR context and restore target domain's CSR context */
lib/sbi/sbi_hart.c +28 -17
70 lines not on this page · raw message
Signature · 7 lines
2.34.1 -- opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
PATCH v3 2/4 lib: sbi: give platform choice of using single memregion to cover OpenSBI
In reply to Bo GanBy default the OpenSBI itself is covered by 2 memregions for RX/RW sections. This is required by platforms with Smepmp to enforce proper permissions in M mode. Note: M-mode only regions can't have RWX permissions with Smepmp. Platforms with traditional PMPs won't be able to benefit from it, as both regions are effectively RWX in M mode, but usually it's harmless to so. Now we provide these platforms with an option to disable this logic. It saves 1 PMP entry. For platforms really in short of PMPs, it does make a difference.
Note: Platform requesting single OpenSBI memregion must be using
traditional (old) PMP. We expect the platform code to do the right thing.
--- include/sbi/sbi_platform.h | 21 +++++++++++++++++++++ lib/sbi/sbi_domain.c | 36 ++++++++++++++++++++++++------------ 2 files changed, 45 insertions(+), 12 deletions(-)
include/sbi/sbi_platform.h +21 -0
@@ -76,6 +76,9 @@ struct sbi_platform_operations {/* Check if specified HART is allowed to do cold boot */ bool (*cold_boot_allowed)(u32 hartid); +/* Check if platform requires single firmware region */ +bool (*single_fw_region)(void ); +/* Platform nascent initialization */ int (*nascent_init)(void );
@@ -359,6 +362,24 @@ static inline bool sbi_platform_cold_boot_allowed(return true; } +/** +* Check whether platform requires single firmware region +* +* Note: Single firmware region only works with legacy PMP because with +* Smepmp M-mode only regions can't have RWX permissions. +* +* @param plat pointer to struct sbi_platform +* +* @return true if single firmware region required and false otherwise +*/ +static inline bool sbi_platform_single_fw_region ( +const struct sbi_platform *plat) +{ +if (plat &&sbi_platform_ops (plat)->single_fw_region) +return sbi_platform_ops (plat)->single_fw_region (); +return false; +} +/** * Nascent (very early) initialization for current HART *
lib/sbi/sbi_domain.c +24 -12
@@ -925,18 +925,30 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid) root.possible_harts = root_hmask;/* Root domain firmware memory region */ -sbi_domain_memregion_init (scratch->fw_start, scratch->fw_rw_offset, - (SBI_DOMAIN_MEMREGION_M_READABLE | -SBI_DOMAIN_MEMREGION_M_EXECUTABLE | -SBI_DOMAIN_MEMREGION_FW ), - &root_memregs[root_memregs_count++]); - -sbi_domain_memregion_init ((scratch->fw_start + scratch->fw_rw_offset), - (scratch->fw_size - scratch->fw_rw_offset), - (SBI_DOMAIN_MEMREGION_M_READABLE | -SBI_DOMAIN_MEMREGION_M_WRITABLE | -SBI_DOMAIN_MEMREGION_FW ), - &root_memregs[root_memregs_count++]); +if (sbi_platform_single_fw_region (sbi_platform_ptr (scratch))) { +sbi_domain_memregion_init (scratch->fw_start, scratch->fw_size, + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_M_EXECUTABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + }else { +sbi_domain_memregion_init (scratch->fw_start, + scratch->fw_rw_offset, + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_EXECUTABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + +sbi_domain_memregion_init ((scratch->fw_start + + scratch->fw_rw_offset), + (scratch->fw_size - + scratch->fw_rw_offset), + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + } root.fw_region_inited = true;
Signature · 7 lines
2.34.1 -- opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Re: [PATCH v3 2/4] lib: sbi: give platform choice of using single memregion to cover OpenSBI
Hi Bo,
Maybe use a FW option to enable single FW region at compile or runtime options.
compile time [1]:
make PLATFORM=generic FW_OPTIONS=0x4
runtime [2]:
Use fw dynamic and pass 0x4 via ((struct fw_dynamic_info *)$a2)->options
[1] https://github.com/riscv-software-src/opensbi/blob/master/docs/firmware/fw.md#options-for-opensbi-firmware-behaviors
[2] https://github.com/riscv-software-src/opensbi/blob/825d0e918a9e41cc57097a8cb913f26550699911/docs/firmware/fw_dynamic.md?plain=1#L10
include/sbi/sbi_scratch.h +0 -0
@@ -115,6 +115,13 @@ enum sbi_scratch_options {
SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0),
/** Enable runtime debug prints */ SBI_SCRATCH_DEBUG_PRINTS = (1 << 1),
+ /**
+ * Use single PMP entry covering entire firmware region. + * Saves one PMP entry on non-Smepmp platforms at the cost + * of coarser PMP granularity.
+ * Not compatible with Smepmp (M-mode RWX is not supported). + */
+ SBI_SCRATCH_SINGLE_FW_PMP = (1 << 2),
};
/** Get pointer to sbi_scratch for current HART */
lib/sbi/sbi_domain.c +0 -0
@@ -904,18 +904,27 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid)
root.possible_harts = root_hmask;
/* Root domain firmware memory region */
- sbi_domain_memregion_init(scratch->fw_start, scratch->fw_rw_offset, - (SBI_DOMAIN_MEMREGION_M_READABLE |
- SBI_DOMAIN_MEMREGION_M_EXECUTABLE |
- SBI_DOMAIN_MEMREGION_FW),
- &root_memregs[root_memregs_count++]);
-
- sbi_domain_memregion_init((scratch->fw_start + scratch->fw_rw_offset), - (scratch->fw_size - scratch->fw_rw_offset), - (SBI_DOMAIN_MEMREGION_M_READABLE |
- SBI_DOMAIN_MEMREGION_M_WRITABLE |
- SBI_DOMAIN_MEMREGION_FW),
- &root_memregs[root_memregs_count++]);
+ if (scratch->options & SBI_SCRATCH_SINGLE_FW_PMP) {
+ sbi_domain_memregion_init(scratch->fw_start, scratch->fw_size,
+ (SBI_DOMAIN_MEMREGION_M_READABLE |
+ SBI_DOMAIN_MEMREGION_M_WRITABLE |
+ SBI_DOMAIN_MEMREGION_M_EXECUTABLE |
+ SBI_DOMAIN_MEMREGION_FW),
+ &root_memregs[root_memregs_count++]);
+ } else {
+ sbi_domain_memregion_init(scratch->fw_start, scratch->fw_rw_offset,
+ (SBI_DOMAIN_MEMREGION_M_READABLE |
+ SBI_DOMAIN_MEMREGION_M_EXECUTABLE |
+ SBI_DOMAIN_MEMREGION_FW),
+ &root_memregs[root_memregs_count++]);
+
+ sbi_domain_memregion_init((scratch->fw_start + scratch->fw_rw_offset), + (scratch->fw_size - scratch->fw_rw_offset),
+ (SBI_DOMAIN_MEMREGION_M_READABLE |
+ SBI_DOMAIN_MEMREGION_M_WRITABLE |
+ SBI_DOMAIN_MEMREGION_FW),
+ &root_memregs[root_memregs_count++]);
+ }
root.fw_region_inited = true;
Best regards,
Peter Lin
Quoted 106 lines · Bo Gan
By default the OpenSBI itself is covered by 2 memregions for RX/RW sections. This is required by platforms with Smepmp to enforce proper permissions in M mode. Note: M-mode only regions can't have RWX permissions with Smepmp. Platforms with traditional PMPs won't be able to benefit from it, as both regions are effectively RWX in M mode, but usually it's harmless to so. Now we provide these platforms with an option to disable this logic. It saves 1 PMP entry. For platforms really in short of PMPs, it does make a difference.
Note: Platform requesting single OpenSBI memregion must be using
traditional (old) PMP. We expect the platform code to do the right thing.Signed-off-by Bo Gan <[email protected]>--- include/sbi/sbi_platform.h | 21 +++++++++++++++++++++ lib/sbi/sbi_domain.c | 36 ++++++++++++++++++++++++------------ 2 files changed, 45 insertions(+), 12 deletions(-)include/sbi/sbi_platform.h +21
-0@@ -76,6 +76,9 @@ struct sbi_platform_operations {/* Check if specified HART is allowed to do cold boot */ bool (*cold_boot_allowed)(u32 hartid); +/* Check if platform requires single firmware region */ +bool (*single_fw_region)(void ); +/* Platform nascent initialization */ int (*nascent_init)(void );@@ -359,6 +362,24 @@ static inline bool sbi_platform_cold_boot_allowed(return true; } +/** +* Check whether platform requires single firmware region +* +* Note: Single firmware region only works with legacy PMP because with +* Smepmp M-mode only regions can't have RWX permissions. +* +* @param plat pointer to struct sbi_platform +* +* @return true if single firmware region required and false otherwise +*/ +static inline bool sbi_platform_single_fw_region ( +const struct sbi_platform *plat) +{ +if (plat &&sbi_platform_ops (plat)->single_fw_region) +return sbi_platform_ops (plat)->single_fw_region (); +return false; +} +/** * Nascent (very early) initialization for current HART * lib/sbi/sbi_domain.c +24
-12@@ -925,18 +925,30 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid) root.possible_harts = root_hmask;/* Root domain firmware memory region */ -sbi_domain_memregion_init (scratch->fw_start, scratch->fw_rw_offset, - (SBI_DOMAIN_MEMREGION_M_READABLE | -SBI_DOMAIN_MEMREGION_M_EXECUTABLE | -SBI_DOMAIN_MEMREGION_FW ), - &root_memregs[root_memregs_count++]); - -sbi_domain_memregion_init ((scratch->fw_start + scratch->fw_rw_offset), - (scratch->fw_size - scratch->fw_rw_offset), - (SBI_DOMAIN_MEMREGION_M_READABLE | -SBI_DOMAIN_MEMREGION_M_WRITABLE | -SBI_DOMAIN_MEMREGION_FW ), - &root_memregs[root_memregs_count++]); +if (sbi_platform_single_fw_region (sbi_platform_ptr (scratch))) { +sbi_domain_memregion_init (scratch->fw_start, scratch->fw_size, + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_M_EXECUTABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + }else { +sbi_domain_memregion_init (scratch->fw_start, + scratch->fw_rw_offset, + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_EXECUTABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + +sbi_domain_memregion_init ((scratch->fw_start + + scratch->fw_rw_offset), + (scratch->fw_size - + scratch->fw_rw_offset), + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + } root.fw_region_inited = true;
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Re: [PATCH v3 2/4] lib: sbi: give platform choice of using single memregion to cover OpenSBI
Hi Peter,
Quoted 11 lines · Yu-Chien Peter Lin
Hi Bo,
Maybe use a FW option to enable single FW region at compile or runtime options.
compile time [1]:
make PLATFORM=generic FW_OPTIONS=0x4runtime [2]:
Use fw dynamic and pass0x4via ((structfw_dynamic_info*)$a2)->options
I could be wrong, but I don't think it's the right approach. The strategy of covering OpenSBI itself with single memregion is something internal to OpenSBI, and shouldn't be allowed to be controlled by previous stage boot- loader. Even if we allows it, previous stage bootloader will likely still use options=0, and expect it to just boot, not to mention EIC7700 doesn't have a previous stage bootloader. (It directly boots with OpenSBI) This means we'll have to specify at compile time, not so different than my previous version that got NAK'ed by Anup.
Bo
Quoted 70 lines
[1] https://github.com/riscv-software-src/opensbi/blob/master/docs/firmware/fw.md#options-for-opensbi-firmware-behaviors
[2] https://github.com/riscv-software-src/opensbi/blob/825d0e918a9e41cc57097a8cb913f26550699911/docs/firmware/fw_dynamic.md?plain=1#L10include/sbi/sbi_scratch.h +7
-0@@ -115,6 +115,13 @@ enum sbi_scratch_options { SBI_SCRATCH_NO_BOOT_PRINTS = (1 << 0), /** Enable runtime debug prints */ SBI_SCRATCH_DEBUG_PRINTS = (1 << 1), + /** + * Use single PMP entry covering entire firmware region. + * Saves one PMP entry on non-Smepmp platforms at the cost + * of coarser PMP granularity. + * Not compatible with Smepmp (M-mode RWX is not supported). + */ + SBI_SCRATCH_SINGLE_FW_PMP = (1 << 2), }; /** Get pointer to sbi_scratch for current HART */lib/sbi/sbi_domain.c +21
-12@@ -904,18 +904,27 @@ int sbi_domain_init(struct sbi_scratch *scratch, u32 cold_hartid) root.possible_harts = root_hmask; /* Root domain firmware memory region */ - sbi_domain_memregion_init(scratch->fw_start, scratch->fw_rw_offset, - (SBI_DOMAIN_MEMREGION_M_READABLE | -SBI_DOMAIN_MEMREGION_M_EXECUTABLE | - SBI_DOMAIN_MEMREGION_FW), - &root_memregs[root_memregs_count++]); - - sbi_domain_memregion_init((scratch->fw_start + scratch->fw_rw_offset), - (scratch->fw_size - scratch->fw_rw_offset), - (SBI_DOMAIN_MEMREGION_M_READABLE | - SBI_DOMAIN_MEMREGION_M_WRITABLE | - SBI_DOMAIN_MEMREGION_FW), - &root_memregs[root_memregs_count++]); + if (scratch->options & SBI_SCRATCH_SINGLE_FW_PMP) { + sbi_domain_memregion_init(scratch->fw_start, scratch->fw_size, + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_M_EXECUTABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + } else { + sbi_domain_memregion_init(scratch->fw_start, scratch->fw_rw_offset, + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_EXECUTABLE | +SBI_DOMAIN_MEMREGION_FW ), + &root_memregs[root_memregs_count++]); + + sbi_domain_memregion_init((scratch->fw_start + scratch->fw_rw_offset), + (scratch->fw_size - scratch->fw_rw_offset), + (SBI_DOMAIN_MEMREGION_M_READABLE | + SBI_DOMAIN_MEMREGION_M_WRITABLE | + SBI_DOMAIN_MEMREGION_FW), + &root_memregs[root_memregs_count++]); + } root.fw_region_inited = true;Best regards,
Peter LinQuoted 1 line · Bo Gan
...
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
PATCH v3 3/4 include: sbi_domain: make is_region_subset public
In reply to Bo GanThe helper function is renamed as sbi_domain_memregion_is_subset, and made public in header file.
--- include/sbi/sbi_domain.h | 22 ++++++++++++++++++++++ lib/sbi/sbi_domain.c | 23 +++-------------------- 2 files changed, 25 insertions(+), 20 deletions(-)
include/sbi/sbi_domain.h +22 -0
@@ -164,6 +164,25 @@ struct sbi_domain_memregion {unsigned long flags; }; +/** Check if regionA is sub-region of regionB */ +static inline bool sbi_domain_memregion_is_subset ( +const struct sbi_domain_memregion *regA, +const struct sbi_domain_memregion *regB) +{ +ulong regA_start = regA->base; +ulong regA_end = regA->base + (BIT (regA->order) -1 ); +ulong regB_start = regB->base; +ulong regB_end = regB->base + (BIT (regB->order) -1 ); + +if ((regB_start <= regA_start) && + (regA_start < regB_end) && + (regB_start < regA_end) && + (regA_end <= regB_end)) +return true; + +return false; +} +/** Representation of OpenSBI domain */ struct sbi_domain {/** Node in linked list of domains */
@@ -222,6 +241,9 @@ extern struct sbi_dlist domain_list;#define sbi_domain_for_each_memregion (__d, __r) \ for ((__r) = (__d)->regions; (__r)->order; (__r)++) +#define sbi_domain_for_each_memregion_idx (__d, __r, __i) \ + for ((__r) = (__d)->regions, (__i) = 0; (__r)->order; (__r)++, (__i)++) +/** * Check whether given HART is assigned to specified domain * @param dom pointer to domain
lib/sbi/sbi_domain.c +3 -20
@@ -281,29 +281,12 @@ static bool is_region_valid(const struct sbi_domain_memregion *reg)return true; } -/** Check if regionA is sub-region of regionB */ -static bool is_region_subset (const struct sbi_domain_memregion *regA, -const struct sbi_domain_memregion *regB) -{ -ulong regA_start = regA->base; -ulong regA_end = regA->base + (BIT (regA->order) -1 ); -ulong regB_start = regB->base; -ulong regB_end = regB->base + (BIT (regB->order) -1 ); - -if ((regB_start <= regA_start) && - (regA_start < regB_end) && - (regB_start < regA_end) && - (regA_end <= regB_end)) -return true; - -return false; -} -/** Check if regionA can be replaced by regionB */ static bool is_region_compatible (const struct sbi_domain_memregion *regA,const struct sbi_domain_memregion *regB) { -if (is_region_subset (regA, regB) && regA->flags == regB->flags) +if (sbi_domain_memregion_is_subset (regA, regB) && + regA->flags == regB->flags)return true;return false;
@@ -363,7 +346,7 @@ static const struct sbi_domain_memregion *find_next_subset_region(sbi_domain_for_each_memregion (dom, sreg) {if (sreg == reg || (sreg->base <= addr) || - !is_region_subset (sreg, reg)) + !sbi_domain_memregion_is_subset (sreg, reg))continue ; if (!ret || (sreg->base < ret->base) ||
Signature · 7 lines
2.34.1 -- opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
PATCH v3 4/4 platform: generic: eswin: add EIC7700
In reply to Bo GanInitial platform support for ESWIN Computing EIC7700 based on public SoC datasheet[1] and tested on HiFive Premier P550. Vendor U-boot/Linux boots fine, and I've tested Geekbench 6.5.0 Preview and got scores on par with the vendor OpenSBI. System shutdown is not implemented due to the lack of public doc regarding the UART communication protocol between the SoC and the onboard BMC, which controls ATX power on HiFive P550 [2].
The files and functions are intentionally named as eic770x in many places for future enhancements to support the 2 die version of the same SoC, namely EIC7702, seen on DC-ROMA AI PC FML13V03 [3]. This patch set only deals with the single die version, and the assumption is we can be either die with id=0 or id=1, but there's only a single die in the system, or we are only using a single die out of 2. However, the way the SoC handles 2- die greatly affects how we configure it in a 1-die setup. EIC770X address map has die 0/1 memory regions interleaved (see comments in eic770x.c). If only 1 die is connected or active, it creates holes in the address map for those regions corresponding to the remote die. When speculative- execution or HW prefetcher touches data-cacheable regions that happen to fall into those holes, it can trigger bus error. Specifically:
- Remote (non-existent) die L3 zero device
- Remote (non-existent) die cached memory region
- Other holes in Memory Port
To make matters worse, EIC770X doesn't have cache coherent DMA, and due to the fact that the P550 core lacks Svpbmt, the SoC maps main memory twice as different regions, so it can bypass cache and fetch the data directly from memory. In address space, we have two memory regions, one as cached, the other as uncached. Thus, we also need an extra PMP entry to protect OpenSBI blob from the uncached window. To do this, platform code requires single_fw_region, otherwise, we'll run out of PMP entries.
EIC770X also have several feature disable/enable CSRs accessible in M mode. By default many core features such as speculation and HW prefetch are disabled, and M mode software is responsible of enabling. Hence, introduce 4 new build time tunable parameters to Kconfig, which reflects the values get updated to those CSRs:
- ESWIN_EIC770X_FEAT0_CFG
- ESWIN_EIC770X_FEAT1_CFG
- ESWIN_EIC770X_L1_HWPF_CFG
- ESWIN_EIC770X_L2_HWPF_CFG
The default values are somewhat optimal for generic workloads. They are dumped when running SiFive's vendor OpenSBI, and in addition, with my own tuning to address the perf regression reported by drmpeg [4]
To build the firmware+u-boot blob, Use the following, and docs [5] for testing it with UART boot without flashing:
make FW_TEXT_START=0x80000000 \
FW_PAYLOAD_OFFSET=0x200000 \
FW_PAYLOAD_PATH=u-boot-nodtb.bin \
FW_PAYLOAD_FDT_ADDR=0xf8000000 \
FW_FDT_PATH=u-boot.dtb[1] https://github.com/eswincomputing/EIC7700X-SoC-Technical-Reference-Manual
[2] https://www.sifive.com/boards/hifive-premier-p550#documentation
[3] https://github.com/geerlingguy/sbc-reviews/issues/82
[4] https://forums.sifive.com/t/low-1-core-stream-bandwidth/7274/15
[5] https://github.com/ganboing/EIC770x-Docs/blob/main/p550/bootchain/UART-Boot.md
--- platform/generic/Kconfig | 5 + platform/generic/configs/defconfig | 1 + platform/generic/eswin/Kconfig | 29 ++ platform/generic/eswin/eic770x.c | 385 +++++++++++++++++++++++ platform/generic/eswin/objects.mk | 11 + platform/generic/include/eswin/eic770x.h | 73 +++++ 6 files changed, 504 insertions(+) create mode 100644 platform/generic/eswin/Kconfig create mode 100644 platform/generic/eswin/eic770x.c create mode 100644 platform/generic/eswin/objects.mk create mode 100644 platform/generic/include/eswin/eic770x.h
platform/generic/Kconfig +5 -0
@@ -88,7 +88,12 @@ config PLATFORM_SPACEMIT_K1 select FDT_HSM_SPACEMIT default n +config PLATFORM_ESWIN_EIC770X + bool "ESWIN EIC770X support" + default n + source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig" +source "$(OPENSBI_SRC_DIR)/platform/generic/eswin/Kconfig" source "$(OPENSBI_SRC_DIR)/platform/generic/thead/Kconfig" endif
platform/generic/configs/defconfig +1 -0
@@ -10,6 +10,7 @@ CONFIG_PLATFORM_STARFIVE_JH7110=y CONFIG_PLATFORM_THEAD=y CONFIG_PLATFORM_MIPS_P8700=y CONFIG_PLATFORM_SPACEMIT_K1=y +CONFIG_PLATFORM_ESWIN_EIC770X=y CONFIG_FDT_CACHE=y CONFIG_FDT_CACHE_SIFIVE_CCACHE=y CONFIG_FDT_CPPC=y
platform/generic/eswin/Kconfig new file +29 -0
@@ -0,0 +1,29 @@ +# SPDX-License-Identifier: BSD-2-Clause + +config ESWIN_EIC770X_FEAT0_CFG + int "ESWIN EIC7700X Feature Disable 0 CSR Configuration" + default 0x4000 + help + CSR Value to initialize EIC770X_FEAT0 (0x7c1) with. + Refer to EIC770X SoC TRM for recommendations. + +config ESWIN_EIC770X_FEAT1_CFG + int "ESWIN EIC7700X Feature Disable 1 CSR Configuration" + default 0x80 + help + CSR Value to initialize EIC770X_FEAT1 (0x7c2) with. + Refer to EIC770X SoC TRM for recommendations. + +config ESWIN_EIC770X_L1_HWPF_CFG + int "ESWIN EIC7700X L1 HW Prefetcher CSR Configuration" + default 0x1005c1be649 + help + CSR Value to initialize EIC770X_L1_HWPF (0x7c3) with. + Refer to EIC770X SoC TRM for recommendations. + +config ESWIN_EIC770X_L2_HWPF_CFG + int "ESWIN EIC7700X L2 HW Prefetcher CSR Configuration" + default 0x929f + help + CSR Value to initialize EIC770X_L2_HWPF (0x7c4) with. + Refer to EIC770X SoC TRM for recommendations.
platform/generic/eswin/eic770x.c new file +385 -0
385 lines not on this page · raw message
platform/generic/eswin/objects.mk new file +11 -0
@@ -0,0 +1,11 @@ +# +# SPDX-License-Identifier: BSD-2-Clause +# +# Copyright (C) 2025 Bo Gan <[email protected]> +# + +carray-platform_override_modules-$(CONFIG_PLATFORM_ESWIN_EIC770X) += eswin_eic7700 +platform-objs-$(CONFIG_PLATFORM_ESWIN_EIC770X) += eswin/eic770x.o + +FW_PAYLOAD=y +FW_PAYLOAD_OFFSET=0x200000
platform/generic/include/eswin/eic770x.h new file +73 -0
73 lines not on this page · raw message
Signature · 7 lines
2.34.1 -- opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Re: [PATCH v3 4/4] platform: generic: eswin: add EIC7700
Hi Bo,
Quoted 661 lines · Bo Gan
Initial platform support for ESWIN Computing EIC7700 based on public SoC datasheet[1] and tested on HiFive Premier P550. Vendor U-boot/Linux boots fine, and I've tested Geekbench 6.5.0 Preview and got scores on par with the vendor OpenSBI. System shutdown is not implemented due to the lack of public doc regarding the UART communication protocol between the SoC and the onboard BMC, which controls ATX power on HiFive P550 [2].
The files and functions are intentionally named as eic770x in many places for future enhancements to support the 2 die version of the same SoC, namely EIC7702, seen on DC-ROMA AI PC FML13V03 [3]. This patch set only deals with the single die version, and the assumption is we can be either die with id=0 or id=1, but there's only a single die in the system, or we are only using a single die out of 2. However, the way the SoC handles 2- die greatly affects how we configure it in a 1-die setup. EIC770X address map has die 0/1 memory regions interleaved (see comments in
eic770x.c). If only 1 die is connected or active, it creates holes in the address map for those regions corresponding to the remote die. When speculative- execution or HW prefetcher touches data-cacheable regions that happen to fall into those holes, it can trigger bus error. Specifically:- Remote (non-existent) die L3 zero device
- Remote (non-existent) die cached memory region
- Other holes in Memory PortTo make matters worse, EIC770X doesn't have cache coherent DMA, and due to the fact that the P550 core lacks Svpbmt, the SoC maps main memory twice as different regions, so it can bypass cache and fetch the data directly from memory. In address space, we have two memory regions, one as cached, the other as uncached. Thus, we also need an extra PMP entry to protect OpenSBI blob from the uncached window. To do this, platform code requires
single_fw_region, otherwise, we'll run out of PMP entries.EIC770X also have several feature disable/enable CSRs accessible in M mode. By default many core features such as speculation and HW prefetch are disabled, and M mode software is responsible of enabling. Hence, introduce 4 new build time tunable parameters to
Kconfig, which reflects the values get updated to those CSRs:
-ESWIN_EIC770X_FEAT0_CFG
-ESWIN_EIC770X_FEAT1_CFG
-ESWIN_EIC770X_L1_HWPF_CFG
-ESWIN_EIC770X_L2_HWPF_CFGThe default values are somewhat optimal for generic workloads. They are dumped when running SiFive's vendor OpenSBI, and in addition, with my own tuning to address the perf regression reported by drmpeg [4]
To build the firmware+u-boot blob, Use the following, and docs [5] for testing it with UART boot without flashing:
make FW_TEXT_START=0x80000000 \ FW_PAYLOAD_OFFSET=0x200000 \ FW_PAYLOAD_PATH=u-boot-nodtb.bin \ FW_PAYLOAD_FDT_ADDR=0xf8000000 \ FW_FDT_PATH=u-boot.dtb[1] https://github.com/eswincomputing/EIC7700X-SoC-Technical-Reference-Manual
[2] https://www.sifive.com/boards/hifive-premier-p550#documentation
[3] https://github.com/geerlingguy/sbc-reviews/issues/82
[4] https://forums.sifive.com/t/low-1-core-stream-bandwidth/7274/15
[5] https://github.com/ganboing/EIC770x-Docs/blob/main/p550/bootchain/UART-Boot.mdSigned-off-by Bo Gan <[email protected]>--- platform/generic/Kconfig | 5 + platform/generic/configs/defconfig | 1 + platform/generic/eswin/Kconfig | 29 ++ platform/generic/eswin/eic770x.c | 385 +++++++++++++++++++++++ platform/generic/eswin/objects.mk | 11 + platform/generic/include/eswin/eic770x.h | 73 +++++ 6 files changed, 504 insertions(+)create mode 100644 platform/generic/eswin/Kconfig create mode 100644
platform/generic/eswin/eic770x.ccreate mode 100644platform/generic/eswin/objects.mkcreate mode 100644platform/generic/include/eswin/eic770x.hplatform/generic/Kconfig +5
-0@@ -88,7 +88,12 @@ config PLATFORM_SPACEMIT_K1 select FDT_HSM_SPACEMIT default n +config PLATFORM_ESWIN_EIC770X + bool "ESWIN EIC770X support" + default n + source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig" +source "$(OPENSBI_SRC_DIR)/platform/generic/eswin/Kconfig" source "$(OPENSBI_SRC_DIR)/platform/generic/thead/Kconfig" endifplatform/generic/configs/defconfig +1
-0@@ -10,6 +10,7 @@ CONFIG_PLATFORM_STARFIVE_JH7110=y CONFIG_PLATFORM_THEAD=y CONFIG_PLATFORM_MIPS_P8700=y CONFIG_PLATFORM_SPACEMIT_K1=y +CONFIG_PLATFORM_ESWIN_EIC770X=y CONFIG_FDT_CACHE=y CONFIG_FDT_CACHE_SIFIVE_CCACHE=y CONFIG_FDT_CPPC=yplatform/generic/eswin/Kconfig new file +29
-0@@ -0,0 +1,29 @@ +# SPDX-License-Identifier: BSD-2-Clause + +config ESWIN_EIC770X_FEAT0_CFG + int "ESWIN EIC7700X Feature Disable 0 CSR Configuration" + default 0x4000 + help + CSR Value to initialize EIC770X_FEAT0 (0x7c1) with. + Refer to EIC770X SoC TRM for recommendations. + +config ESWIN_EIC770X_FEAT1_CFG + int "ESWIN EIC7700X Feature Disable 1 CSR Configuration" + default 0x80 + help + CSR Value to initialize EIC770X_FEAT1 (0x7c2) with. + Refer to EIC770X SoC TRM for recommendations. + +config ESWIN_EIC770X_L1_HWPF_CFG + int "ESWIN EIC7700X L1 HW Prefetcher CSR Configuration" + default 0x1005c1be649 + help + CSR Value to initialize EIC770X_L1_HWPF (0x7c3) with. + Refer to EIC770X SoC TRM for recommendations. + +config ESWIN_EIC770X_L2_HWPF_CFG + int "ESWIN EIC7700X L2 HW Prefetcher CSR Configuration" + default 0x929f + help + CSR Value to initialize EIC770X_L2_HWPF (0x7c4) with. + Refer to EIC770X SoC TRM for recommendations.platform/generic/eswin/eic770x.c new file +88
-0@@ -0,0 +1,385 @@ +/* +* SPDX-License-Identifier: BSD-2-Clause +* +* Copyright (c) 2025 Bo Gan <[email protected]> +* +*/ + +#include <platform_override.h> +#include <sbi/riscv_io.h> +#include <sbi/sbi_console.h> +#include <sbi/sbi_system.h> +#include <sbi/sbi_math.h> +#include <eswin/eic770x.h> + +static int eic770x_system_reset_check (u32 type,u32 reason) +{ +switch (type) { +case SBI_SRST_RESET_TYPE_COLD_REBOOT : +case SBI_SRST_RESET_TYPE_WARM_REBOOT : +return 1 ; +default : +return 0 ; + } +} + +static void eic770x_system_reset (u32 type,u32 reason) +{ +switch (type) { +case SBI_SRST_RESET_TYPE_COLD_REBOOT : +case SBI_SRST_RESET_TYPE_WARM_REBOOT : +writel (EIC770X_SYSCRG_RST_VAL , (void *)EIC770X_SYSCRG_RST ); + } + +sbi_printf ("%s: Unable to reset system\n" , __func__); +sbi_hart_hang (); +} + +static struct sbi_system_reset_device eic770x_reset = { + .name ="eic770x_reset" , + .system_reset_check = eic770x_system_reset_check, + .system_reset = eic770x_system_reset +}; + +#define add_root_mem_chk (...) do { \ + rc = sbi_domain_root_add_memrange(__VA_ARGS__); \ + if (rc) \ + return rc; \ +} while (0) + +/** +* EIC7700 special arrangement of PMP entries: +* +* We have to use extra PMPs to block data cacheable regions that +* that doesn't belong to the current hart's die in order to prevent +* speculative accesses or HW prefetcher from generating bus error: +* +* bus error of cause event: 9, accrued: 0x220, +* physical address: 0x24ffffffa0 +* +* The data cacheable regions (per datasheet) include: +* +* - [0x1a000000, 0x1a400000) -- Die 0 L3 zero device +* - [0x3a000000, 0x3a400000) -- Die 1 L3 zero device +* - [0x80000000, 0x80_00000000) -- memory port +* +* To make the blocker effective for M mode too, the extra PMPs need +* LOCK bit to be set, and once set, we can't change them later. +* We also have to to use 1 extra PMP to protect OpenSBI in uncached +* memory. EIC770X maps main memory (DRAM) twice -- one in memory +* port (cached), the other in system port (uncached). P550 doesn't +* support Svpbmt, so EIC770X use the uncached window to handle DMA +* that are cache incoherent -- pretty much all peripherals +* +* Final PMP configuration: +* +* From die 0 point of view, block +* - [0x3a000000, 0x3a400000) -- Die 1 L3 zero device +* - [0x10_00000000, 0x80_00000000) -- Die 1 cached mem + holes +* +* Root domain Harts: +* PMP[0]: [ 0x80000000, 0x80080000) ____ Firmware in cached mem +* PMP[1]: [0xc0_00000000, 0xc0_00080000) ____ Firmware in uncached +* PMP[2]: [ 0x3a000000, 0x3a400000) L___ Die 1 L3 zero device +* PMP[3]: [ 0x2000000 0x2010000) ____ CLINT +* PMP[4]: [ 0x0, 0x10_00000000) _RWX P550/System/Die 0 cached mem +* PMP[5]: <Free> +* PMP[6]: [ 0x0, 0x80_00000000) L___ P550/System/Memory Port +* PMP[7]: [ 0x0, 0xffffffffffffffff] _RWX Everything I noticed that some PMP entries such as PMP[2] and PMP[6] are configured with pmpcfg.L set. During domain context switch, all PMP entries managed by OpenSBI domain will be disabled [1], so locked entries will remain active, and if a hole entry not using identical PMP id across domains, other regions that program the locked PMP id cannot be applied. This will block the use of e.g. OP-TEE which relies on domain context switch mechanism.
I am proposing reserved PMP allocator [2]. Hopefully, it can address this requirement. By specifying reserved 3 entries on P550 platform, the PMP usage should become:
PMP[0]: Reserved NAPOT [
0x3a000000,0x3a400000)L___Die 1 L3 zero device
PMP[1]: Reserved TOR1 [0x10_00000000,L___Die 1 cached mem + holes
PMP[2]: Reserved TOR2 0x80_00000000)L___Die 1 cached mem + holes
PMP[3]: Domain0Region0 [0x80000000,0x80080000) ____ Firmware in cached mem
PMP[4]: Domain0Region1 [0xc0_00000000, 0xc0_00080000) ____ Firmware in uncached
PMP[5]: Domain0Region2 [0x20000000x2010000) ____ CLINT
PMP[6]: Domain0Region3 [0x0,0xffffffffffffffff]_RWXEverything
PMP[7]: unusedOne aspect that requires careful attention is CPU suspend with power gating scenarios. When reserved PMP states can be removed, additional reserved PMP management (save/restore operations for PMPs that protect holes) must be included in the suspend/resume flow.
[1] https://github.com/riscv-software-src/opensbi/blob/825d0e918a9e41cc57097a8cb913f26550699911/lib/sbi/sbi_domain_context.c#L123-L131
[2] https://lore.kernel.org/all/[email protected]/+ *
+ * From die 1 point of view, block+ * - [0x1a000000, 0x1a400000) -- Die 0 L3 zero device + * - [0x80000000, 0x20_00000000) -- Die 0 cached mem + holes + * - [0x30_00000000, 0x80_00000000) -- other holes in Memory port + * + * Root domain Harts: + * PMP[0]: [0x20_00000000, 0x20_00080000) ____ Firmware in cached mem + * PMP[1]: [0xe0_00000000, 0xe0_00080000) ____ Firmware in uncached + * PMP[2]: [ 0x1a000000, 0x1a400000) L___ Die 0 L3 zero dev + * PMP[3]: [ 0x22000000 0x22010000) ____ CLINT + * PMP[4]: [ 0x0, 0x80000000) _RWX Die 0/1 P550 internal + * PMP[5]: [0x20_00000000, 0x30_00000000) _RWX Die 1 cached memory + * PMP[6]: [ 0x0, 0x80_00000000) L___ P550/System/Memory Port + * PMP[7]: [ 0x0, 0xffffffffffffffff] _RWX Everything + * + * EIC770X memory port map: + * P550 Internal + * ├─ 0x0000_0000 - 0x2000_0000 die 0 internal + * └─ 0x2000_0000 - 0x4000_0000 die 1 internal + * System Port 0 + * ├─ 0x4000_0000 - 0x6000_0000 die 0 low MMIO + * └─ 0x6000_0000 - 0x8000_0000 die 1 low MMIO + * Memory Port + * ├─ 0x8000_0000 - 0x10_8000_0000 die 0 memory (cached) + * ├─ 0x20_0000_0000 - 0x30_0000_0000 die 1 memory (cached) + * └─ 0x40_0000_0000 - 0x60_0000_0000 interleaved memory (cached) + * System Port 1 + * ├─ 0x80_0000_0000 - 0xa0_0000_0000 die 0 high MMIO + * ├─ 0xa0_0000_0000 - 0xc0_0000_0000 die 1 high MMIO + * ├─ 0xc0_0000_0000 - 0xd0_0000_0000 die 0 memory (uncached) + * ├─ 0xe0_0000_0000 - 0xf0_0000_0000 die 1 memory (uncached) + * ├─ 0x100_0000_0000 - 0x120_0000_0000 interleaved memory (uncached) + * └─ ...+ *
+ * Inearly_init, add memory regions such thatlib/code has the knowledge
+ * of blocked ranges. When the driver code inserts new regions,lib/code
+ * can optimize away unnecessary ones. Next, infinal_init, we program the
+ * PMPs to a default state that'll keep ourselves functional (CLINT/...
+ * accessible). Later, inpmp_configure, do the actual configuration of
+ * PMP, using domain memory regions and permissions.
+ */platform/generic/eswin/eic770x.c new file +255
-0+ +static int eswin_eic7700_early_init (bool cold_boot) +{ +struct sbi_scratch *scratch =sbi_scratch_thishart_ptr (); +int rc; + +if (!cold_boot) +return generic_early_init (cold_boot); + +/* Enable bus blocker */ +writel (1 , (void *)EIC770X_TL64D2D_OUT ); +writel (1 , (void *)EIC770X_TL256D2D_OUT ); +writel (1 , (void *)EIC770X_TL256D2D_IN ); + asmvolatile ("fence o, rw" ); + +/* Block firmware in uncached memory */ +add_root_mem_chk (EIC770X_TO_UNCACHED ( + scratch->fw_start), +1UL <<log2roundup (scratch->fw_size), +1UL <<log2roundup (scratch->fw_size), + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_M_EXECUTABLE | +SBI_DOMAIN_MEMREGION_FW )); + +/* Allow SURW of P550 + System Port */ +add_root_mem_chk (0 , +EIC770X_MEMPORT_BASE , +EIC770X_MEMPORT_BASE , + (SBI_DOMAIN_MEMREGION_MMIO | +SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW )); + +if (current_hart_die ()) { +/* Allow SURWX of die 1 cached memory */ +add_root_mem_chk (EIC770X_D1_MEM_BASE , +EIC770X_D1_MEM_SIZE , +EIC770X_D1_MEM_SIZE , + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_SU_RWX )); + }else { +/* Allow SURWX of P550 + System Port + die 0 cached memory */ +add_root_mem_chk (0 , +EIC770X_D0_MEM_LIMIT , +EIC770X_D0_MEM_LIMIT , + (SBI_DOMAIN_MEMREGION_M_READABLE | +SBI_DOMAIN_MEMREGION_M_WRITABLE | +SBI_DOMAIN_MEMREGION_SU_RWX )); + } + +/* Block P550 + System Port 0 + Memory Port (enforced) */ +add_root_mem_chk (0 , +EIC770X_MEMPORT_LIMIT , +EIC770X_MEMPORT_LIMIT , + (SBI_DOMAIN_MEMREGION_MMIO | +SBI_DOMAIN_MEMREGION_ENF_PERMISSIONS )); + +return generic_early_init (cold_boot); +} + +#define PMP_FW_START 0 +#define PMP_FW_COUNT 2 +#define PMP_RESERVED_A 2 +#define PMP_FREE_A_START 3 +#define PMP_FREE_A_COUNT 3 +#define PMP_RESERVED_B 6 +#define PMP_FREE_B_START 7 +#define PMP_FREE_B_COUNT 1 + +static int eswin_eic7700_final_init (bool cold_boot) +{ +/** +* For both dies after final_init: +* +* PMP[0]: Protect OpenSBI in cached memory +* PMP[1]: Protect OpenSBI in uncached memory +* PMP[2]: Block remote die P550 L3 Zero Device +* PMP[3-5]: <Free ranges A> +* PMP[5]: Temporary enable P550 + System Port +* PMP[6]: Block all P550 + System + Memory Port +* PMP[7-7]: <Free ranges B> +*/ +struct sbi_domain_memregion *reg; +unsigned int pmp_idx =PMP_FW_START , + pmp_max =PMP_FW_START +PMP_FW_COUNT ; +int rc; + +if (cold_boot) +sbi_system_reset_add_device (&eic770x_reset); + +/** +* Do generic_final_init stuff first, because it touchs FDT. +* After final_init, we'll block entire memory port with the +* LOCK bit set, which means we can't access memory outside +* of [fw_start, fw_start + fw_size). The FDT could very well +* reside outside of firmware region. Later, pmp_configure() +* may unblock it with some preceding entries for root domain +* harts. It may not unblock it, however, for non-root harts. +*/ + rc =generic_final_init (cold_boot); +if (rc) +return rc; + + +/* Process firmware regions */ +sbi_domain_for_each_memregion (&root, reg) { +if (!SBI_DOMAIN_MEMREGION_IS_FIRMWARE (reg->flags)) +continue ; + +if (pmp_idx >= pmp_max) { +sbi_printf ("%s: insufficient FW PMP entries\n" , + __func__); +return SBI_EFAIL ; + } +pmp_set (pmp_idx++,sbi_domain_get_oldpmp_flags (reg), + reg->base, reg->order); + } + +pmp_set (PMP_RESERVED_A ,PMP_L ,EIC770X_L3_ZERO_REMOTE , +log2roundup (EIC770X_L3_ZERO_SIZE )); +/** +* Enable P550 internal + System Port, so OpenSBI can access +* CLINT/PLIC/UART. Might be overwritten in pmp_configure. +*/ +pmp_set (PMP_FREE_A_START +PMP_FREE_A_COUNT -1 ,0 ,0 , +log2roundup (EIC770X_MEMPORT_BASE )); + +pmp_set (PMP_RESERVED_B ,PMP_L ,0 , +log2roundup (EIC770X_MEMPORT_LIMIT )); + +/** +* These must come after the setup of PMP, as we are about to +* enable speculation and HW prefetcher bits +*/ +csr_write (EIC770X_CSR_FEAT0 ,CONFIG_ESWIN_EIC770X_FEAT0_CFG ); +csr_write (EIC770X_CSR_FEAT1 ,CONFIG_ESWIN_EIC770X_FEAT1_CFG ); +csr_write (EIC770X_CSR_L1_HWPF ,CONFIG_ESWIN_EIC770X_L1_HWPF_CFG ); +csr_write (EIC770X_CSR_L2_HWPF ,CONFIG_ESWIN_EIC770X_L2_HWPF_CFG ); + +return 0 ; +} + +static int eswin_eic7700_pmp_configure (unsigned int pmp_count, +unsigned int pmp_log2gran, +unsigned long pmp_addr_max) +{ +struct sbi_domain *dom =sbi_domain_thishart_ptr (); +struct sbi_domain_memregion *reg, *prev =NULL ; +unsigned int pmp_idx, pmp_max; +unsigned int i, j; + +/* Process the first free range A [3-5] */ + pmp_idx =PMP_FREE_A_START , + pmp_max =PMP_FREE_A_START +PMP_FREE_A_COUNT ; + +sbi_domain_for_each_memregion_idx (dom, reg, i) { +if (SBI_DOMAIN_MEMREGION_IS_FIRMWARE (reg->flags)) +continue ; + +/** +* This must be the one blocking P550 + System Port + +* Memory Port we setup in early_init, or a superset +* of it. If seen, break, and program the rest in +* free range B. +*/ +if (reg->base ==0 && + reg->order >=log2roundup (EIC770X_MEMPORT_LIMIT )) +break ; + +/** +* Relaxation: +* Treat a previous region with SURW as SURWX if the +* current has SURWX, and current region with MMIO +* if previous has MMIO, and see if it can be merged. +* This saves 1 PMP entry on die 0/ +*/ +if (prev &&sbi_domain_memregion_is_subset (prev, reg) && + (reg->flags |SBI_DOMAIN_MEMREGION_MMIO ) == + (prev->flags |SBI_DOMAIN_MEMREGION_SU_EXECUTABLE )) + pmp_idx--; + +if (pmp_idx >= pmp_max) + goto no_more_pmp; + +pmp_set (pmp_idx++,sbi_domain_get_oldpmp_flags (reg), + reg->base, reg->order); + prev = reg; + } +/* Disable the rest */ +while (pmp_idx < pmp_max) +pmp_disable (pmp_idx++); + +/* Process the second free range B [7-7] */ + pmp_idx =PMP_FREE_B_START , + pmp_max =PMP_FREE_B_START +PMP_FREE_B_COUNT ; + +sbi_domain_for_each_memregion_idx (dom, reg, j) { +if (i >= j) +continue ; + +if (pmp_idx >= pmp_max) + goto no_more_pmp; + +pmp_set (pmp_idx++,sbi_domain_get_oldpmp_flags (reg), + reg->base, reg->order); + } +/* Disable the rest */ +while (pmp_idx < pmp_max) +pmp_disable (pmp_idx++); + +return 0 ; +no_more_pmp: +sbi_printf ("%s: insufficient PMP entries\n" , __func__); +return SBI_EFAIL ; +} + +static void eswin_eic7700_pmp_unconfigure (void ) +{ +/* Enable P550 internal + System Port */ +pmp_set (PMP_FREE_A_START +PMP_FREE_A_COUNT -1 ,0 ,0 , +log2roundup (EIC770X_MEMPORT_BASE )); + +for (unsigned int i =0 ; i <PMP_FREE_A_COUNT -1 ; i++) +pmp_disable (i +PMP_FREE_A_START ); + +for (unsigned int i =0 ; i <PMP_FREE_B_COUNT ; i++) +pmp_disable (i +PMP_FREE_B_START ); +} + +static bool eswin_eic7700_single_fw_region (void ) +{ +return true; +} + +static int eswin_eic7700_platform_init (const void *fdt,int nodeoff, +const struct fdt_match *match) +{ + generic_platform_ops.early_init = eswin_eic7700_early_init; + generic_platform_ops.final_init = eswin_eic7700_final_init; + generic_platform_ops.pmp_configure = eswin_eic7700_pmp_configure; + generic_platform_ops.pmp_unconfigure = eswin_eic7700_pmp_unconfigure; + generic_platform_ops.single_fw_region = eswin_eic7700_single_fw_region; + +return 0 ; +} + +static const struct fdt_match eswin_eic7700_match[] = { + { .compatible ="eswin,eic7700" }, + { }, +}; + +const struct fdt_driver eswin_eic7700 = { + .match_table = eswin_eic7700_match, + .init = eswin_eic7700_platform_init, +};platform/generic/eswin/objects.mk new file +11
-0@@ -0,0 +1,11 @@ +# +# SPDX-License-Identifier: BSD-2-Clause +# +# Copyright (C) 2025 Bo Gan <[email protected]> +# + +carray-platform_override_modules-$(CONFIG_PLATFORM_ESWIN_EIC770X) += eswin_eic7700 +platform-objs-$(CONFIG_PLATFORM_ESWIN_EIC770X) += eswin/eic770x.o + +FW_PAYLOAD=y +FW_PAYLOAD_OFFSET=0x200000platform/generic/include/eswin/eic770x.h new file +56
-0@@ -0,0 +1,73 @@ +/* +* SPDX-License-Identifier: BSD-2-Clause +* +* Copyright (c) 2025 Bo Gan <[email protected]> +* +*/ + +#ifndef __EIC770X_H__ +#define __EIC770X_H__ + +/* CSRs */ +#define EIC770X_CSR_BRPREDICT 0x7c0 +#define EIC770X_CSR_FEAT0 0x7c1 +#define EIC770X_CSR_FEAT1 0x7c2 +#define EIC770X_CSR_L1_HWPF 0x7c3 +#define EIC770X_CSR_L2_HWPF 0x7c4 + +/* Hart ID to core/die conversion */ +#define CPU_CORE_BITS 2 +#define CPU_CORE_MASK ((1 << CPU_CORE_BITS) - 1) +#define CPU_DIE_SHIFT CPU_CORE_BITS +#define CPU_DIE_BITS 1 +#define CPU_DIE_MASK ((1 << CPU_DIE_SHIFT) - 1) + +#define hart_core (i) ((i) & CPU_CORE_MASK) +#define hart_die (i) (((i) >> CPU_DIE_SHIFT) & CPU_DIE_MASK) +#define current_hart_core () hart_core(current_hartid()) +#define current_hart_die () hart_die(current_hartid()) + +/* P550 Internal and System Port 0 */ +#define EIC770X_P550INT_SIZE 0x20000000UL +#define EIC770X_P550INT_BASE (d) (0UL + EIC770X_P550INT_SIZE * (d)) +#define EIC770X_P550INT_LOCAL EIC770X_P550INT_BASE(current_hart_die()) +#define EIC770X_TL64D2D_OUT (EIC770X_P550INT_LOCAL + 0x200000) +#define EIC770X_TL256D2D_OUT (EIC770X_P550INT_LOCAL + 0x202000) +#define EIC770X_TL256D2D_IN (EIC770X_P550INT_LOCAL + 0x204000) +#define EIC770X_L3_ZERO_SIZE 0x400000UL +#define EIC770X_L3_ZERO_BASE (d) (EIC770X_P550INT_BASE(d) + 0x1a000000) +#define EIC770X_L3_ZERO_LOCAL EIC770X_L3_ZERO_BASE(current_hart_die()) +#define EIC770X_L3_ZERO_REMOTE EIC770X_L3_ZERO_BASE(1 - current_hart_die()) + +#define EIC770X_SYSPORT_SIZE 0x20000000UL +#define EIC770X_SYSPORT_BASE (d) (0x40000000UL + EIC770X_SYSPORT_SIZE * (d)) +#define EIC770X_SYSPORT_LOCAL EIC770X_SYSPORT_BASE(current_hart_die()) +#define EIC770X_SYSCRG (EIC770X_SYSPORT_LOCAL + 0x11828000UL) +#define EIC770X_SYSCRG_RST (EIC770X_SYSCRG + 0x300UL) +#define EIC770X_SYSCRG_RST_VAL 0x1AC0FFE6UL + +/* Memory Ports */ +#define EIC770X_MEMPORT_BASE 0x0080000000UL // 2G +#define EIC770X_MEMPORT_SIZE 0x7f80000000UL // +510G +#define EIC770X_MEMPORT_LIMIT (EIC770X_MEMPORT_BASE + EIC770X_MEMPORT_SIZE) +#define EIC770X_D0_MEM_BASE 0x0080000000UL // 2G +#define EIC770X_D0_MEM_SIZE 0x0f80000000UL // +62G +#define EIC770X_D0_MEM_LIMIT (EIC770X_D0_MEM_BASE + EIC770X_D0_MEM_SIZE) +#define EIC770X_D1_MEM_BASE 0x2000000000UL // 128GThese definitions break RV32 builds; please ensure the generic platform compiles cleanly on both RV32 and RV64 toolchains.
Best regards,
Peter Linplatform/generic/include/eswin/eic770x.h new file +17
-0+#define EIC770X_D1_MEM_SIZE 0x1000000000UL // +64G +#define EIC770X_D1_MEM_LIMIT (EIC770X_D1_MEM_BASE + EIC770X_D1_MEM_SIZE) +#define EIC770X_CACHED_BASE (current_hart_die() ? \ + EIC770X_D1_MEM_BASE : \ + EIC770X_D0_MEM_BASE) + +/* Uncached memory mapped in System Port 1 */ +#define EIC770X_D0_UC_BASE 0xc000000000UL +#define EIC770X_D1_UC_BASE 0xe000000000UL +#define EIC770X_UNCACHED_BASE (current_hart_die() ? \ + EIC770X_D1_UC_BASE : \ + EIC770X_D0_UC_BASE) + +#define EIC770X_TO_UNCACHED (x) ((x) - EIC770X_CACHED_BASE + \ + EIC770X_UNCACHED_BASE) + +#endif
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Re: [PATCH v3 4/4] platform: generic: eswin: add EIC7700
Hi Peter,
Thanks for the review. I saw your other reply, but let me still reply here, while trying to fix the formatting:
Quoted 44 lines · Yu-Chien Peter Lin
Hi Bo,
Quoted 32 lines · Bo Gan
...
+/**
+ * EIC7700 special arrangement of PMP entries:
+ *
+ * We have to use extra PMPs to block data cacheable regions that
+ * that doesn't belong to the current hart's die in order to prevent
+ * speculative accesses or HW prefetcher from generating bus error:
+ *
+ * bus error of cause event: 9, accrued:0x220,
+ * physical address:0x24ffffffa0
+ *
+ * The data cacheable regions (per datasheet) include:
+ *+ * - [0x1a000000, 0x1a400000) -- Die 0 L3 zero device + * - [0x3a000000, 0x3a400000) -- Die 1 L3 zero device + * - [0x80000000, 0x80_00000000) -- memory port+ *
+ * To make the blocker effective for M mode too, the extra PMPs need
+ * LOCK bit to be set, and once set, we can't change them later.
+ * We also have to to use 1 extra PMP to protect OpenSBI in uncached
+ * memory. EIC770X maps main memory (DRAM) twice -- one in memory
+ * port (cached), the other in system port (uncached). P550 doesn't
+ * support Svpbmt, so EIC770X use the uncached window to handle DMA
+ * that are cache incoherent -- pretty much all peripherals
+ *
+ * Final PMP configuration:
+ *
+ * From die 0 point of view, block
+ * - [0x3a000000,0x3a400000) -- Die 1 L3 zero device
+ * - [0x10_00000000, 0x80_00000000) -- Die 1 cached mem + holes
+ *
+ * Root domain Harts:Quoted 9 lines
+ * PMP[0]: [ 0x80000000, 0x80080000) ____ Firmware in cached mem + * PMP[1]: [0xc0_00000000, 0xc0_00080000) ____ Firmware in uncached + * PMP[2]: [ 0x3a000000, 0x3a400000) L___ Die 1 L3 zero device + * PMP[3]: [ 0x2000000 0x2010000) ____ CLINT + * PMP[4]: [ 0x0, 0x10_00000000) _RWX P550/System/Die 0 cached mem + * PMP[5]: <Free> + * PMP[6]: [ 0x0, 0x80_00000000) L___ P550/System/Memory Port + * PMP[7]: [ 0x0, 0xffffffffffffffff] _RWX Everything
Quoted 6 lines
I noticed that some PMP entries such as PMP[2] and PMP[6] are configured with pmpcfg.L set. During domain context switch, all PMP entries managed by OpenSBI domain will be disabled [1], so locked entries will remain active, and if a hole entry not using identical PMP id across domains, other regions that program the locked PMP id cannot be applied. This will block the use of e.g. OP-TEE which relies on domain context switch mechanism.
The non-root domains shouldn't have ENF_PERMISSIONS (locked PMP entries). I'll add docs for EIC7700 in subsequent patchsets. In short, it's a user error to do that in EIC7700's case, or perhaps even applies for all plat OP-TEE is supported (to some extent), and the boot log from cover letter demonstrates a t/u-domain like setup. It, however, does require hacks to existing code by reverting this patch from Himanshu Chauhan
lib: utils: Disallow non-root domains from adding M-mode regions
Thus, I left it for future work. FYI, the domains are:
opensbi-domains {
compatible = "opensbi,domain,config";
tmem: tmem {
compatible = "opensbi,domain,memregion";
base = <0x4 0x7ff00000>;
order = <20>;
};
tuart: tuart {
compatible = "opensbi,domain,memregion";
base = <0x0 0x50910000>;
order = <12>;
mmio;
};
p550_sys: p550_sys {
compatible = "opensbi,domain,memregion";
base = <0x0 0x0>;
order = <31>;
mmio;
};
p550_sys_mem: p550_sys_mem {
compatible = "opensbi,domain,memregion";
base = <0x0 0x0>;
order = <36>;
};
allmem: allmem {
compatible = "opensbi,domain,memregion";
base = <0x0 0x0>;
order = <64>;
};
tdomain: trusted-domain {
compatible = "opensbi,domain,instance";
possible-harts = <>;
regions = <&tuart (MEM_MRW | MEM_SURW)>,
<&p550_sys MEM_MRW>,
<&tmem MEM_SURWX>;
next-addr = <0x4 0x7ff00000>;
next-arg1 = <0x0 0x0>;
next-mode = <0x1>;
}; udomain: untrusted-domain {
compatible = "opensbi,domain,instance";
possible-harts = <>;
regions = <&p550_sys_mem (MEM_MRW | MEM_SURWX)>,
<&tmem 0>,
<&allmem MEM_SURWX>; };
};
In short, there's no need to match the PMP entry id from non-root domains The pmp_configure will figure out the order of programming PMPs, and the pmp_unconfigure will reset everything to a state where OpenSBI can still function before the next pmp_configure(). The memory region config in the domain instance has indeed became a little bit tedious, where it needs to explicitly allow certain regions, but I hope it's acceptable.
Quoted 4 lines
I am proposing reserved PMP allocator [2]. Hopefully, it can address this requirement. By specifying reserved 3 entries on P550 platform, the PMP usage should become:
Quoted 9 lines
PMP[0]: Reserved NAPOT [
0x3a000000,0x3a400000)L___Die 1 L3 zero device
PMP[1]: Reserved TOR1 [0x10_00000000,L___Die 1 cached mem + holes
PMP[2]: Reserved TOR2 0x80_00000000)L___Die 1 cached mem + holes
PMP[3]: Domain0Region0 [0x80000000,0x80080000) ____ Firmware in cached mem
PMP[4]: Domain0Region1 [0xc0_00000000, 0xc0_00080000) ____ Firmware in uncached
PMP[5]: Domain0Region2 [0x20000000x2010000) ____ CLINT
PMP[6]: Domain0Region3 [0x0,0xffffffffffffffff]_RWXEverything
PMP[7]: unused
Sounds reasonable to me, but there are 2 concerns.
1. My v2 change was using PMP TOR, and it's NAK'ed by Anup. Perhaps I should wait and see his feedback on this one, and your other change, before moving forward.
2. From die 1 POV, it's not easy to block those "hole" ranges using PMP at the beginning. Even with TOR, we'll run out of PMPs, because die 1 needs to have 2 ranges in memory port blocked. It'll inevitably be done with reserved PMP and domain PMP interleaved, like what I have here.
Quoted 7 lines
One aspect that requires careful attention is CPU suspend with power gating scenarios. When reserved PMP states can be removed, additional reserved PMP management (save/restore operations for PMPs that protect holes) must be included in the suspend/resume flow.
[1] https://github.com/riscv-software-src/opensbi/blob/825d0e918a9e41cc57097a8cb913f26550699911/lib/sbi/sbi_domain_context.c#L123-L131
[2] https://lore.kernel.org/all/[email protected]/
This can be enhanced in future pathsets. My plan is to have a HSM that restores the PMP blocking those "holes". Actually it also needs to restore those feature enable/disable CSRs. This patchset is the "initial support" for EIC7700, and I'm not precluding future improvements.
Quoted 1 line
Quoted 1 line
...
I'll fix the RV32 build issue for sure. Thanks again.
Bo.
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Quoted 140 lines · Bo Gan
EIC7700 is the SoC used in HiFive P550 and Milk-V Megrez. This SoC is currently one of the only off-the-shelf board/chips that support H extension, although it's v0.6.1. It also supports pre-ratified N-trace. Add support for it so people can benefit from latest OpenSBI features.
The device-tree of HiFive P550 has been upstreamed to Linux:
https://lore.kernel.org/all/[email protected]/
However U-boot is not, and there are bugs in vendor U-boot device-tree, and also inconsistencies between the two. Thus, this patch is coded with the upstreamed device-tree as the reference, but tested with the patched vendor U-boot device tree asFW_FDT_PATH. The patched vendor U-boot is hosted here: https://github.com/ganboing/u-boot-eic7x/tree/eic7x-dt-fix
Refer to the last PATCH for the instructions on building the firmware blob and launch it through UART boot.The major complication of this chip is that it requires certain memory regions to be blocked with PMP entries to prevent speculative execution or HW prefetcher from touching the data-cacheable regions within to avoid bus errors. Due to the fact that this SoC handles cache incoherent DMA by mapping memory twice, one as cached, and the other as uncached, we also need an extra PMP to protect the OpenSBI in the uncached portion in address space. The PMP handling is tricky, so I documented it very extensively for people to reason about it. I managed to get it done with only NAPOT PMP entries and still got 1 free PMP for root harts for die 0 (No free PMP for die 1 root harts). This even permits a udomain/tdomain like partitioning, so we can even try out TEEs. Sample boot log:
OpenSBI v1.7-73-g5c235e5d
____ _____ ____ _____/ __ \ / ____| _ \_ _| | | | |_ __ ___ _ __ | (___ | |_) || | | | | | '_ \ / _ \ '_ \ \___ \| _ < | | | |__| | |_) | __/ | | |____) | |_) || |_ \____/| .__/ \___|_| |_|_____/|____/_____| | | |_|Platform Name : SiFive HiFive Premier P550
Platform Features : medeleg
Platform HART Count : 4
Platform IPI Device : aclint-mswi
Platform Timer Device : aclint-mtimer @ 1000000Hz
Platform Console Device : uart8250
Platform HSM Device : ---
Platform PMU Device : ---
Platform Reboot Device :eic770x_reset
Platform Shutdown Device : ---
Platform Suspend Device : ---
Platform CPPC Device : ---
Firmware Base :0x80000000
Firmware Size : 613 KB
Firmware RW Offset :0x80000
Firmware RW Size : 101 KB
Firmware Heap Offset :0x8d000
Firmware Heap Size : 49 KB (total), 1 KB (reserved), 14 KB (used), 33 KB (free)
Firmware Scratch Size : 4096 B (total), 424 B (used), 3672 B (free)
Runtime SBI Version : 3.0
Standard SBI Extensions : time,rfnc,ipi,base,hsm,srst,pmu,dbcn,fwft,legacy,dbtr,sse
Experimental SBI Extensions : noneDomain0 Name : root
Domain0 Boot HART : 0
Domain0 HARTs : 0*,1*,2*,3*
Domain0 Region00 : 0x0000000080000000-0x00000000800fffff M: (F,R,W,X) S/U: ()
Domain0 Region01 : 0x000000c000000000-0x000000c0000fffff M: (F,R,W,X) S/U: ()
Domain0 Region02 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: ()
Domain0 Region03 : 0x0000000000000000-0x000000007fffffff M: (I,R,W) S/U: (R,W)
Domain0 Region04 : 0x0000000000000000-0x0000000fffffffff M: (R,W) S/U: (R,W,X)
Domain0 Region05 : 0x0000000000000000-0x0000007fffffffff M: (I) S/U: ()
Domain0 Region06 : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X)
Domain0 Next Address :0x0000000080200000
Domain0 Next Arg1 :0x00000000f8000000
Domain0 Next Mode : S-mode
Domain0 SysReset : yes
Domain0 SysSuspend : yesDomain1 Name : trusted-domain
Domain1 Boot HART : -1Domain1 HARTs : Domain1 Region00 : 0x0000000080000000-0x00000000800fffff M: (F,R,W,X) S/U: () Domain1 Region01 : 0x000000c000000000-0x000000c0000fffff M: (F,R,W,X) S/U: () Domain1 Region02 : 0x0000000050910000-0x0000000050910fff M: (I,R,W) S/U: (R,W) Domain1 Region03 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: () Domain1 Region04 : 0x000000047ff00000-0x000000047fffffff M: () S/U: (R,W,X) Domain1 Region05 : 0x0000000000000000-0x000000007fffffff M: (I,R,W) S/U: () Domain1 Region06 : 0x0000000000000000-0x0000007fffffffff M: (I) S/U: () Domain1 Next Address : 0x000000047ff00000 Domain1 Next Arg1 : 0x0000000000000000 Domain1 Next Mode : S-mode Domain1 SysReset : no Domain1 SysSuspend : noDomain2 Name : untrusted-domain
Domain2 Boot HART : -1Domain2 HARTs : Domain2 Region00 : 0x0000000080000000-0x00000000800fffff M: (F,R,W,X) S/U: () Domain2 Region01 : 0x000000c000000000-0x000000c0000fffff M: (F,R,W,X) S/U: () Domain2 Region02 : 0x0000000002000000-0x000000000200ffff M: (I,R,W) S/U: () Domain2 Region03 : 0x000000047ff00000-0x000000047fffffff M: () S/U: () Domain2 Region04 : 0x0000000000000000-0x0000000fffffffff M: (R,W) S/U: (R,W,X) Domain2 Region05 : 0x0000000000000000-0x0000007fffffffff M: (I) S/U: () Domain2 Region06 : 0x0000000000000000-0xffffffffffffffff M: () S/U: (R,W,X) Domain2 Next Address : 0x0000000080000000 Domain2 Next Arg1 : 0x00000000f8000000 Domain2 Next Mode : S-mode Domain2 SysReset : no Domain2 SysSuspend : noBoot HART ID : 0
Boot HART Domain : root
Boot HART Priv Version : v1.11
Boot HART Base ISA : rv64imafdchx
Boot HART ISA Extensions : sscofpmf,zihpm,sdtrig
Boot HART PMP Count : 8
Boot HART PMP Granularity : 12 bits
Boot HART PMP Address Bits : 39
Boot HART MHPM Info : 4 (0x00000078)
Boot HART Debug Triggers : 4 triggers
Boot HART MIDELEG :0x0000000000002666
Boot HART MEDELEG :0x0000000000f0b509Signed-off-by Bo Gan <[email protected]>---
Changes in v3:
- Figure out the cause behind bus error, and document it properly
- Drop the consolidation logic and let the lib/memregion logic to optimize out unnecessary regions -- simplifies many things.
- Better and more comprehensive comments in source code.
- Support tdomain/udomain like use cases on die 0.Changes in v2:
- Major enhancement of PMP consolidation logic. Also fixed a Linux Panic bug due to the mismatch between PMP settings and reserved memory regions passed to Linux via FDT.
- Also protects the OpenSBI firmware in uncached memory portion of address space.
- More detailed documentation on EIC770X/P550
Quoted 3 lines
---
Bo Gan (4):
lib: sbi: allow platform to override PMP (un)configuration
Instead of platform specific PMP (un)configuration override, you can rebase this series upon "OpenSBI hart protection abstraction" series and implement EIC770X specific hart protection using PMP.
Quoted 23 lines
lib: sbi: give platform choice of using single memregion to cover
OpenSBI
include:sbi_domain: makeis_region_subsetpublic
platform: generic: eswin: add EIC7700include/sbi/sbi_domain.h | 29 ++ include/sbi/sbi_hart.h | 3 + include/sbi/sbi_platform.h | 74 +++++ lib/sbi/sbi_domain.c | 80 +++-- lib/sbi/sbi_domain_context.c | 11 +- lib/sbi/sbi_hart.c | 45 ++- platform/generic/Kconfig | 5 + platform/generic/configs/defconfig | 1 + platform/generic/eswin/Kconfig | 29 ++ platform/generic/eswin/eic770x.c | 385 +++++++++++++++++++++++ platform/generic/eswin/objects.mk | 11 + platform/generic/include/eswin/eic770x.h | 73 +++++ 12 files changed, 687 insertions(+), 59 deletions(-) create mode 100644 platform/generic/eswin/Kconfig create mode 100644 platform/generic/eswin/eic770x.c create mode 100644 platform/generic/eswin/objects.mk create mode 100644 platform/generic/include/eswin/eic770x.h
Quoted 8 lines
--
2.34.1--
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
Regards,
Anup
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Hi Anup,
Quoted 7 lines · Anup Patel
Quoted 1 line · Bo Gan
...
Instead of platform specific PMP (un)configuration override, you can rebase this series upon "OpenSBI hart protection abstraction" series and implement EIC770X specific hart protection using PMP.
Thanks, Anup. Will rebase on hart prot abstraction change. I also found another issue regarding the memory region covering the uncached range of firmware, where it should use the MMIO flag to not expose the it through FDT to S mode. MMIO flag also better matches the System Port PMA in HW. Let me know if you found any other issues, and I can fix altogether.
Quoted 8 lines
Quoted 5 lines
lib: sbi: give platform choice of using single memregion to cover
OpenSBI
include:sbi_domain: makeis_region_subsetpublic
platform: generic: eswin: add EIC7700
...Regards,
Anup
Bo
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Hi Anup,
Is the hart protection abstraction going to be merged soon? In v1.8? I'm making a v4 which is rebased on your change. Let me know if there's any other change to the hart protection abstraction, so I can rebase again before sending out. Also any feedback on Peter Lin's comments?
I hope I can make it for v1.8. Thanks again.
Bo
Quoted 28 lines · Bo Gan
Hi Anup,
Quoted 7 lines · Anup Patel
Quoted 1 line · Bo Gan
...
Instead of platform specific PMP (un)configuration override, you can rebase this series upon "OpenSBI hart protection abstraction" series and implement EIC770X specific hart protection using PMP.
Thanks, Anup. Will rebase on hart prot abstraction change. I also found another issue regarding the memory region covering the uncached range of firmware, where it should use the MMIO flag to not expose the it through FDT to S mode. MMIO flag also better matches the System Port PMA in HW. Let me know if you found any other issues, and I can fix altogether.
Quoted 8 lines
Quoted 5 lines
lib: sbi: give platform choice of using single memregion to cover OpenSBI
include:sbi_domain: makeis_region_subsetpublic
platform: generic: eswin: add EIC7700
...Regards,
AnupBo
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi
Quoted 4 lines · Bo Gan
Hi Anup,
Is the hart protection abstraction going to be merged soon? In v1.8?
Yes, I plan to include hart protection abstraction for v1.8.
Quoted 2 lines
I'm making a v4 which is rebased on your change. Let me know if there's any other change to the hart protection abstraction, so I can rebase
There won't be any major changes to hart protection abstraction for v1.8. I expect hart protection abstraction to evolve over time but we don't need to do everything in the initial implementation.
Quoted 1 line
again before sending out. Also any feedback on Peter Lin's comments?
Yes, I will look at it pretty soon.
Regards,
Anup
Signature · 3 lines
opensbi mailing list [email protected] http://lists.infradead.org/mailman/listinfo/opensbi