Skip to contentSkip to conversation

platform: generic: Tenstorrent Atlantis support

PATCH Superseded Nicholas Piggin 2 participants 9 messages Mar 10 – Apr 23 RISC-V

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/4 platform: generic: Tenstorrent Atlantis support

Add the Tenstorrent Atlantis as a generic-platform. This initial support enables the single_fw_region option, and verifies and prints HART PMA CSR configuration.

---
Note the Atlantis hardware is not yet released, and QEMU models are in the process of being upstreamed, but at the moment not complete. So for now this can not really be tested with a public implementation, but we plan to get implementation specific IOMMU MMRs and CPU CSRs into QEMU that can meaningfully exercise this platform.

There will be some more Atlantis platform pieces to come, but we want to get started with upstreaming and these are some relatively small and stable pieces.

The IOMMU support depends on a proposed change to device-tree binding for the Tenstorrent specific riscv iommu which I posted for RFC here:

https://lore.kernel.org/linux-riscv/[email protected]/T/#u

Thanks,
Nick

---
 docs/platform/generic.md                      |   2 +
 docs/platform/tt-atlantis.md                  |  35 +++++
 platform/generic/Kconfig                      |   6 +
 platform/generic/configs/defconfig            |   1 +
 .../generic/include/tenstorrent/ascalon.h     |  12 ++
 platform/generic/include/tenstorrent/pma.h    |  18 +++
 platform/generic/tenstorrent/Kconfig          |   5 +
 platform/generic/tenstorrent/ascalon.c        |  53 +++++++
 platform/generic/tenstorrent/atlantis.c       |  50 +++++++
 platform/generic/tenstorrent/objects.mk       |  10 ++
 platform/generic/tenstorrent/pma.c            | 138 ++++++++++++++++++
 11 files changed, 330 insertions(+)
 create mode 100644 docs/platform/tt-atlantis.md
 create mode 100644 platform/generic/include/tenstorrent/ascalon.h
 create mode 100644 platform/generic/include/tenstorrent/pma.h
 create mode 100644 platform/generic/tenstorrent/Kconfig
 create mode 100644 platform/generic/tenstorrent/ascalon.c
 create mode 100644 platform/generic/tenstorrent/atlantis.c
 create mode 100644 platform/generic/tenstorrent/objects.mk
 create mode 100644 platform/generic/tenstorrent/pma.c
docs/platform/generic.md +2 -0
index c48d6a9a..0b896ede 100644
@@ -47,6 +47,7 @@ RISC-V Platforms Using Generic Platform
 * **SiFive HiFive Unleashed** (*[sifive_fu540.md]*)
 * **Spike** (*[spike.md]*)
 * **T-HEAD C9xx series Processors** (*[thead-c9xx.md]*)
+* **Tenstorrent Atlantis Platform** (*[tt-atlantis.md]*)
 * **OpenPiton FPGA SoC** (*[fpga-openpiton.md]*)
 * **Ariane FPGA SoC** (*[fpga-ariane.md]*)
 
@@ -57,5 +58,6 @@ RISC-V Platforms Using Generic Platform
 [sifive_fu540.md]: sifive_fu540.md
 [spike.md]: spike.md
 [thead-c9xx.md]: thead-c9xx.md
+[tt-atlantis.md]: tt-atlantis.md
 [fpga-openpiton.md]: fpga-openpiton.md
 [fpga-ariane.md]: fpga-ariane.md
docs/platform/tt-atlantis.md new file +35 -0
new file mode 100644
index 00000000..b9bdd238
@@ -0,0 +1,35 @@
+Tenstorrent Atlantis Platform
+=============================
+
+The Tenstorrent Atlantis is an SoC and development board from
+Tenstorrent in partnership with CoreLab Technology. It contains 8 RISC-V
+RVA23 compliant Tenstorrent Ascalon cores with RISC-V AIA, RISC-V IOMMU,
+and a range of devices and IO connectivity.
+
+To build the platform-specific library and firmware images, provide the
+*PLATFORM=generic* parameter to the top level `make` command.
+
+Platform Options
+----------------
+
+The *Tenstorrent Atlantis* platform does not have any platform-specific
+options.
+
+Building Tenstorrent Atlantis Platform
+--------------------------------------
+
+The Atlantis Platform is still under development. This section will be
+expanded as firmware and support become available.
+
+QEMU support is currently being developed and initial support has been
+proposed for upstream. To run QEMU that is patched with 'tt-atlantis'
+machine support, run:
+
+```
+qemu-system-riscv64 -M tt-atlantis -nographic \
+	-bios build/platform/generic/firmware/fw_payload.bin \
+    -kernel <linux_build_dir>/Image
+```
+
+Recent (6.18) Linux/riscv 64-bit defconfig kernels should run the QEMU
+tt-atlantis machine.
platform/generic/Kconfig +6 -0
index 2627998a..7de5e5a6 100644
@@ -85,6 +85,11 @@ config PLATFORM_STARFIVE_JH7110
 	bool "StarFive JH7110 support"
 	default n
 
+config PLATFORM_TENSTORRENT_ATLANTIS
+	bool "Tenstorrent Atlantis support"
+	select CPU_TENSTORRENT_ASCALON
+	default n
+
 config PLATFORM_THEAD
 	bool "THEAD C9xx support"
 	select THEAD_C9XX_ERRATA
@@ -110,6 +115,7 @@ config PLATFORM_MIPS_P8700_BOSTON
 
 source "$(OPENSBI_SRC_DIR)/platform/generic/andes/Kconfig"
 source "$(OPENSBI_SRC_DIR)/platform/generic/eswin/Kconfig"
+source "$(OPENSBI_SRC_DIR)/platform/generic/tenstorrent/Kconfig"
 source "$(OPENSBI_SRC_DIR)/platform/generic/thead/Kconfig"
 
 endif
platform/generic/configs/defconfig +1 -0
index 346058f5..153a3aa0 100644
@@ -9,6 +9,7 @@ CONFIG_PLATFORM_SIFIVE_FU540=y
 CONFIG_PLATFORM_SIFIVE_FU740=y
 CONFIG_PLATFORM_SOPHGO_SG2042=y
 CONFIG_PLATFORM_STARFIVE_JH7110=y
+CONFIG_PLATFORM_TENSTORRENT_ATLANTIS=y
 CONFIG_PLATFORM_THEAD=y
 CONFIG_PLATFORM_MIPS_P8700_EYEQ7H=y
 CONFIG_PLATFORM_MIPS_P8700_BOSTON=y
platform/generic/include/tenstorrent/ascalon.h new file +12 -0
new file mode 100644
index 00000000..5d7b7635
@@ -0,0 +1,12 @@
+/*
+ * SPDX-FileCopyrightText: (c) 2025-2026 Tenstorrent USA, Inc.
+ * SPDX-License-Identifier: BSD-2-Clause
+ */
+
+#ifndef __TENSTORRENT_ASCALON_H__
+#define __TENSTORRENT_ASCALON_H__
+
+void tt_ascalon_discover_pmas_from_boot_hart(void);
+void tt_ascalon_verify_pmas_nonboot_hart(void);
+
+#endif
platform/generic/include/tenstorrent/pma.h new file +18 -0
new file mode 100644
index 00000000..051764ee
@@ -0,0 +1,18 @@
+/*
+ * SPDX-FileCopyrightText: (c) 2025-2026 Tenstorrent USA, Inc.
+ * SPDX-License-Identifier: BSD-2-Clause
+ */
+
+#ifndef __TENSTORRENT_PMA_H__
+#define __TENSTORRENT_PMA_H__
+
+/* Max number of PMAs for devices (CPU, IOMMU) for Tenstorrent platforms. */
+#define TT_MAX_PMAS	32
+
+u64 tt_pma_get(unsigned int n);
+void tt_pma_set(unsigned int n, u64 pma);
+bool tt_pma_validate(unsigned int i, u64 pma);
+void tt_pma_print(unsigned int i, u64 pma);
+
+#endif
+
platform/generic/tenstorrent/Kconfig new file +5 -0
new file mode 100644
index 00000000..76c7fb32
@@ -0,0 +1,5 @@
+# SPDX-FileCopyrightText: (c) 2025-2026 Tenstorrent USA, Inc.
+# SPDX-License-Identifier: BSD-2-Clause
+
+config CPU_TENSTORRENT_ASCALON
+	bool
platform/generic/tenstorrent/ascalon.c new file +53 -0

53 lines not on this page · raw message

platform/generic/tenstorrent/atlantis.c new file +50 -0

50 lines not on this page · raw message

platform/generic/tenstorrent/objects.mk new file +10 -0
new file mode 100644
index 00000000..1e4aeb2d
@@ -0,0 +1,10 @@
+#
+# SPDX-FileCopyrightText: (c) 2025-2026 Tenstorrent USA, Inc.
+# SPDX-License-Identifier: BSD-2-Clause
+#
+
+platform-objs-y += tenstorrent/pma.o
+platform-objs-$(CONFIG_CPU_TENSTORRENT_ASCALON) += tenstorrent/ascalon.o
+
+carray-platform_override_modules-$(CONFIG_PLATFORM_TENSTORRENT_ATLANTIS) += tenstorrent_atlantis
+platform-objs-$(CONFIG_PLATFORM_TENSTORRENT_ATLANTIS) += tenstorrent/atlantis.o
platform/generic/tenstorrent/pma.c new file +138 -0

138 lines not on this page · raw message

Signature · 7 lines
2.51.0


-- 
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#1 lore raw

PATCH 2/4 lib: sbi: Move PMP encoding into a new file

The Tenstorrent RISC-V IOMMU PMP MMRs use the same encoding as PMP CSRs. In preparation to support it, move the non hart-specific PMP operations into their own file where they will also be used to build the IOMMU PMPs.

---
 include/sbi/riscv_asm.h |   2 +-
 include/sbi/sbi_pmp.h   |  24 +++++++
 lib/sbi/objects.mk      |   1 +
 lib/sbi/riscv_asm.c     | 156 ++++++++++++----------------------------
 lib/sbi/sbi_pmp.c       |  91 +++++++++++++++++++++++
 5 files changed, 164 insertions(+), 110 deletions(-)
 create mode 100644 include/sbi/sbi_pmp.h
 create mode 100644 lib/sbi/sbi_pmp.c
include/sbi/riscv_asm.h +1 -1
index 0cf3fc37..b97e1880 100644
@@ -213,7 +213,7 @@ void misa_string(int xlen, char *out, unsigned int out_sz);
 int pmp_disable(unsigned int n);
 
 /* Check if the matching field is set */
-int is_pmp_entry_mapped(unsigned long entry);
+int is_pmp_entry_mapped(unsigned int entry);
 
 int pmp_set(unsigned int n, unsigned long prot, unsigned long addr,
 	    unsigned long log2len);
include/sbi/sbi_pmp.h new file +24 -0
new file mode 100644
index 00000000..66664bb0
@@ -0,0 +1,24 @@
+/*
+ * SPDX-FileCopyrightText: (c) 2025-2026 Tenstorrent USA, Inc.
+ * SPDX-License-Identifier: BSD-2-Clause
+ */
+
+#ifndef __SBI_PMP_H__
+#define __SBI_PMP_H__
+
+#include <sbi/sbi_types.h>
+
+struct pmp {
+	u8	cfg;
+	u64	addr;
+};
+typedef struct pmp pmp_t;
+
+bool pmp_enabled(pmp_t *pmp);
+
+int pmp_create(pmp_t *pmp, unsigned long prot, unsigned long addr,
+	    unsigned long log2len);
+int pmp_decode(pmp_t *pmp, unsigned long *prot, unsigned long *addr,
+	    unsigned long *log2len);
+
+#endif
lib/sbi/objects.mk +1 -0
index 07d13229..0e29a277 100644
@@ -87,6 +87,7 @@ libsbi-objs-y += sbi_init.o
 libsbi-objs-y += sbi_ipi.o
 libsbi-objs-y += sbi_irqchip.o
 libsbi-objs-y += sbi_platform.o
+libsbi-objs-y += sbi_pmp.o
 libsbi-objs-y += sbi_pmu.o
 libsbi-objs-y += sbi_dbtr.o
 libsbi-objs-y += sbi_mpxy.o
lib/sbi/riscv_asm.c +47 -109

203 lines not on this page · raw message

lib/sbi/sbi_pmp.c new file +91 -0

91 lines not on this page · raw message

Signature · 7 lines
2.51.0


-- 
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#2 lore raw

Re: [PATCH 2/4] lib: sbi: Move PMP encoding into a new file

Quoted 10 lines · Nicholas Piggin

The Tenstorrent RISC-V IOMMU PMP MMRs use the same encoding as PMP CSRs. In preparation to support it, move the non hart-specific PMP operations into their own file where they will also be used to build the IOMMU PMPs.

lib/sbi/riscv_asm.c int pmp_set(unsigned int n, unsigned long prot, unsigned long addr, · line 343

Quoted 7 lines
lib/sbi/riscv_asm.c +5 -0
 
+int is_pmp_entry_mapped(unsigned int entry)
+{
+       pmp_t pmp;
 
+       if (hart_pmp_read(entry, &pmp))
+               return pmp_enabled(&pmp);

The diff is a bit all over the place so I may have confused myself. The patch keeps is_pmp_entry_mapped but changes the body from

    if (pmp_get(entry, &prot, &addr, &log2len) != 0)
        return false;

to

    if (hart_pmp_read(entry, &pmp))
        return pmp_enabled(&pmp);

but hart_pmp_read returns SBI_OK (0) on success. So we should keep the !=0?

lib/sbi/riscv_asm.c int pmp_set(unsigned int n, unsigned long prot, unsigned long addr, · line 429

Quoted 8 lines
lib/sbi/riscv_asm.c +5 -2
+int pmp_get(unsigned int n, unsigned long *prot_out, unsigned long *addr_out,
+           unsigned long *log2len)
+{
+       pmp_t pmp;
+       int rc;
 
-       /* return details */
-       *prot_out    = prot;

It looks like prot_out is no longer written to after your patch. This hunk never made it into the new pmp_decode:

       /* decode PMP config */
       cfgmask = (0xffUL << pmpcfg_shift);
       pmpcfg  = csr_read_num(pmpcfg_csr) & cfgmask;
       prot    = pmpcfg >> pmpcfg_shift;

lib/sbi/sbi_pmp.c line 28

Quoted 12 lines
lib/sbi/sbi_pmp.c new file +8 -0
@@ -0,0 +1,91 @@
 
+int pmp_create(pmp_t *pmp, unsigned long prot, unsigned long addr,
+           unsigned long log2len)
+{
+       unsigned long addrmask, pmpaddr;
+
+       /* check parameters */
+       if (log2len > __riscv_xlen || log2len < PMP_SHIFT)
+               return SBI_EINVAL;

Here we check that log2len <=__riscv_xlen, but in pmp_decode below we set log2len = __riscv_xlen + 3. Is that just how it works, or should pmp_decode be able to decode what pmp_create does?

Quoted 51 lines
lib/sbi/sbi_pmp.c new file +51 -0
+
+       /* encode PMP config */
+       prot &= ~PMP_A;
+       prot |= (log2len == PMP_SHIFT) ? PMP_A_NA4 : PMP_A_NAPOT;
+
+       /* encode PMP address */
+       if (log2len == PMP_SHIFT) {
+               pmpaddr = (addr >> PMP_SHIFT);
+       } else {
+               if (log2len == __riscv_xlen) {
+                       pmpaddr = -1UL;
+               } else {
+                       addrmask = (1UL << (log2len - PMP_SHIFT)) - 1;
+                       pmpaddr  = ((addr >> PMP_SHIFT) & ~addrmask);
+                       pmpaddr |= (addrmask >> 1);
+               }
+       }
+
+       pmp->cfg = prot;
+       pmp->addr = pmpaddr;
+
+       return SBI_OK;
+}
+
+int pmp_decode(pmp_t *pmp, unsigned long *prot_out, unsigned long *addr_out,
+           unsigned long *log2len)
+{
+       /* check parameters */
+       if (!pmp || !prot_out || !addr_out || !log2len)
+               return SBI_EINVAL;
+
+       if (!pmp_enabled(pmp))
+               return SBI_EINVAL;
+
+       /* decode PMP address */
+       if ((pmp->cfg & PMP_A) == PMP_A_NAPOT) {
+               if (pmp->addr == -1UL) {
+                       *addr_out = 0;
+                       *log2len = __riscv_xlen + 3;
+               } else {
+                       unsigned long mask = ~((1UL << cto(pmp->addr)) - 1);
+                       *addr_out = (pmp->addr & mask) << PMP_SHIFT;
+                       *log2len = (cto(pmp->addr) + PMP_SHIFT + 1);
+               }
+       } else {
+               *addr_out = pmp->addr << PMP_SHIFT;
+               *log2len = PMP_SHIFT;
+       }
+
+       return SBI_OK;
+}
Signature · 3 lines
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#3 lore raw

Re: [PATCH 2/4] lib: sbi: Move PMP encoding into a new file

Quoted 25 lines · Joel Stanley
Quoted 10 lines · Nicholas Piggin

The Tenstorrent RISC-V IOMMU PMP MMRs use the same encoding as PMP CSRs. In preparation to support it, move the non hart-specific PMP operations into their own file where they will also be used to build the IOMMU PMPs.

lib/sbi/riscv_asm.c int pmp_set(unsigned int n, unsigned long prot, unsigned long addr, · line 343

Quoted 7 lines
lib/sbi/riscv_asm.c +5 -0
 
+int is_pmp_entry_mapped(unsigned int entry)
+{
+       pmp_t pmp;
 
+       if (hart_pmp_read(entry, &pmp))
+               return pmp_enabled(&pmp);

The diff is a bit all over the place so I may have confused myself. The patch keeps is_pmp_entry_mapped but changes the body from

    if (pmp_get(entry, &prot, &addr, &log2len) != 0)
        return false;
Quoted 6 lines

to

    if (hart_pmp_read(entry, &pmp))
        return pmp_enabled(&pmp);

but hart_pmp_read returns SBI_OK (0) on success. So we should keep the !=0?

Yes, you're right.

Quoted 13 lines

lib/sbi/riscv_asm.c int pmp_set(unsigned int n, unsigned long prot, unsigned long addr, · line 429

Quoted 9 lines
lib/sbi/riscv_asm.c +5 -2
+int pmp_get(unsigned int n, unsigned long *prot_out, unsigned long *addr_out,
+           unsigned long *log2len)
+{
+       pmp_t pmp;
+       int rc;
 
-       /* return details */
-       *prot_out    = prot;

It looks like prot_out is no longer written to after your patch. This hunk never made it into the new pmp_decode:

Quoted 4 lines
       /* decode PMP config */
       cfgmask = (0xffUL << pmpcfg_shift);
       pmpcfg  = csr_read_num(pmpcfg_csr) & cfgmask;
       prot    = pmpcfg >> pmpcfg_shift;

Good catch.

Quoted 4 lines
Quoted 3 lines
lib/sbi/sbi_pmp.c new file +0 -0
@@ -0,0 +1,91 @@
Quoted 12 lines

lib/sbi/sbi_pmp.c line 28

Quoted 8 lines
lib/sbi/sbi_pmp.c new file +8 -0
+int pmp_create(pmp_t *pmp, unsigned long prot, unsigned long addr,
+           unsigned long log2len)
+{
+       unsigned long addrmask, pmpaddr;
+
+       /* check parameters */
+       if (log2len > __riscv_xlen || log2len < PMP_SHIFT)
+               return SBI_EINVAL;

Here we check that log2len <=__riscv_xlen, but in pmp_decode below we set log2len = __riscv_xlen + 3. Is that just how it works, or should pmp_decode be able to decode what pmp_create does?

That + 3 looks wrong, I'll have to dig a bit more into it. I think I need to simplify and/or split the patch up so it's a bit more feasbile to review (and hopefully less buggy).

Thanks,
Nick

Signature · 3 lines
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#4 lore raw

PATCH 3/4 lib: sbi: Add hart_ prefix to PMP functions

In reply to Nicholas Piggin

Give PMP functions that deal with HART CSRs a hart_ prefix, to help distinguish from general PMP encoding functions that will be shared by the Tenstorrent IOMMU.

---
 include/sbi/riscv_asm.h          |  8 +++----
 lib/sbi/riscv_asm.c              | 10 ++++-----
 lib/sbi/sbi_hart_pmp.c           | 14 ++++++-------
 platform/generic/eswin/eic770x.c | 36 ++++++++++++++++----------------
 4 files changed, 34 insertions(+), 34 deletions(-)
include/sbi/riscv_asm.h +4 -4
index b97e1880..14bc35e5 100644
@@ -210,15 +210,15 @@ int misa_xlen(void);
 void misa_string(int xlen, char *out, unsigned int out_sz);
 
 /* Disable pmp entry at a given index */
-int pmp_disable(unsigned int n);
+int hart_pmp_disable(unsigned int n);
 
 /* Check if the matching field is set */
-int is_pmp_entry_mapped(unsigned int entry);
+int hart_is_pmp_enabled(unsigned int n);
 
-int pmp_set(unsigned int n, unsigned long prot, unsigned long addr,
+int hart_pmp_set(unsigned int n, unsigned long prot, unsigned long addr,
 	    unsigned long log2len);
 
-int pmp_get(unsigned int n, unsigned long *prot_out, unsigned long *addr_out,
+int hart_pmp_get(unsigned int n, unsigned long *prot_out, unsigned long *addr_out,
 	    unsigned long *log2len);
 
 #endif /* !__ASSEMBLER__ */
lib/sbi/riscv_asm.c +5 -5
index c11abb1a..03c8e4d7 100644
@@ -329,23 +329,23 @@ int hart_pmp_write(unsigned int n, pmp_t *pmp)
 	return SBI_OK;
 }
 
-int pmp_disable(unsigned int n)
+int hart_pmp_disable(unsigned int n)
 {
 	struct pmp pmp = { .cfg = 0, .addr = 0 };
 	return hart_pmp_write(n, &pmp);
 }
 
-int is_pmp_entry_mapped(unsigned int entry)
+int hart_is_pmp_enabled(unsigned int n)
 {
 	pmp_t pmp;
 
-	if (hart_pmp_read(entry, &pmp))
+	if (hart_pmp_read(n, &pmp))
 		return pmp_enabled(&pmp);
 
 	return false;
 }
 
-int pmp_set(unsigned int n, unsigned long prot, unsigned long addr,
+int hart_pmp_set(unsigned int n, unsigned long prot, unsigned long addr,
 	    unsigned long log2len)
 {
 	pmp_t pmp;
@@ -358,7 +358,7 @@ int pmp_set(unsigned int n, unsigned long prot, unsigned long addr,
 	return hart_pmp_write(n, &pmp);
 }
 
-int pmp_get(unsigned int n, unsigned long *prot_out, unsigned long *addr_out,
+int hart_pmp_get(unsigned int n, unsigned long *prot_out, unsigned long *addr_out,
 	    unsigned long *log2len)
 {
 	pmp_t pmp;
lib/sbi/sbi_hart_pmp.c +7 -7
index 02a3b3c4..0aa4752f 100644
@@ -100,7 +100,7 @@ static void sbi_hart_smepmp_set(struct sbi_scratch *scratch,
 		sbi_platform_pmp_set(sbi_platform_ptr(scratch),
 				     pmp_idx, reg->flags, pmp_flags,
 				     reg->base, reg->order);
-		pmp_set(pmp_idx, pmp_flags, reg->base, reg->order);
+		hart_pmp_set(pmp_idx, pmp_flags, reg->base, reg->order);
 	} else {
 		sbi_printf("Can not configure pmp for domain %s because"
 			   " memory region address 0x%lx or size 0x%lx "
@@ -139,7 +139,7 @@ static int sbi_hart_smepmp_configure(struct sbi_scratch *scratch)
 	csr_set(CSR_MSECCFG, MSECCFG_RLB);
 
 	/* Disable the reserved entry */
-	pmp_disable(SBI_SMEPMP_RESV_ENTRY);
+	hart_pmp_disable(SBI_SMEPMP_RESV_ENTRY);
 
 	/* Program M-only regions when MML is not set. */
 	pmp_idx = 0;
@@ -224,7 +224,7 @@ static int sbi_hart_smepmp_map_range(struct sbi_scratch *scratch,
 	unsigned int pmp_flags = (PMP_W | PMP_X);
 	unsigned long order, base = 0;
 
-	if (is_pmp_entry_mapped(SBI_SMEPMP_RESV_ENTRY))
+	if (hart_is_pmp_enabled(SBI_SMEPMP_RESV_ENTRY))
 		return SBI_ENOSPC;
 
 	for (order = MAX(sbi_hart_pmp_log2gran(scratch), log2roundup(size));
@@ -244,7 +244,7 @@ static int sbi_hart_smepmp_map_range(struct sbi_scratch *scratch,
 	sbi_platform_pmp_set(sbi_platform_ptr(scratch), SBI_SMEPMP_RESV_ENTRY,
 			     SBI_DOMAIN_MEMREGION_SHARED_SURW_MRW,
 			     pmp_flags, base, order);
-	pmp_set(SBI_SMEPMP_RESV_ENTRY, pmp_flags, base, order);
+	hart_pmp_set(SBI_SMEPMP_RESV_ENTRY, pmp_flags, base, order);
 
 	return SBI_OK;
 }
@@ -253,7 +253,7 @@ static int sbi_hart_smepmp_unmap_range(struct sbi_scratch *scratch,
 				       unsigned long addr, unsigned long size)
 {
 	sbi_platform_pmp_disable(sbi_platform_ptr(scratch), SBI_SMEPMP_RESV_ENTRY);
-	return pmp_disable(SBI_SMEPMP_RESV_ENTRY);
+	return hart_pmp_disable(SBI_SMEPMP_RESV_ENTRY);
 }
 
 static int sbi_hart_oldpmp_configure(struct sbi_scratch *scratch)
@@ -281,7 +281,7 @@ static int sbi_hart_oldpmp_configure(struct sbi_scratch *scratch)
 			sbi_platform_pmp_set(sbi_platform_ptr(scratch),
 					     pmp_idx, reg->flags, pmp_flags,
 					     reg->base, reg->order);
-			pmp_set(pmp_idx++, pmp_flags, reg->base, reg->order);
+			hart_pmp_set(pmp_idx++, pmp_flags, reg->base, reg->order);
 		} else {
 			sbi_printf("Can not configure pmp for domain %s because"
 				   " memory region address 0x%lx or size 0x%lx "
@@ -307,7 +307,7 @@ static void sbi_hart_pmp_unconfigure(struct sbi_scratch *scratch)
 			continue;
 
 		sbi_platform_pmp_disable(sbi_platform_ptr(scratch), i);
-		pmp_disable(i);
+		hart_pmp_disable(i);
 	}
 }
 
platform/generic/eswin/eic770x.c +18 -18
index 7330df9f..dfca0c41 100644
@@ -254,21 +254,21 @@ static int eswin_eic7700_final_init(bool cold_boot)
 				   __func__);
 			return SBI_EFAIL;
 		}
-		pmp_set(pmp_idx++, sbi_domain_get_oldpmp_flags(reg),
-			reg->base, reg->order);
+		hart_pmp_set(pmp_idx++, sbi_domain_get_oldpmp_flags(reg),
+			     reg->base, reg->order);
 	}
 
-	pmp_set(PMP_RESERVED_A, PMP_L, EIC770X_L3_ZERO_REMOTE,
-			   log2roundup(EIC770X_L3_ZERO_SIZE));
+	hart_pmp_set(PMP_RESERVED_A, PMP_L, EIC770X_L3_ZERO_REMOTE,
+		     log2roundup(EIC770X_L3_ZERO_SIZE));
 	/**
 	 * Enable P550 internal + System Port, so OpenSBI can access
 	 * CLINT/PLIC/UART. Might be overwritten in pmp_configure.
 	 */
-	pmp_set(PMP_FREE_A_START + PMP_FREE_A_COUNT - 1, 0, 0,
-		log2roundup(EIC770X_MEMPORT_BASE));
+	hart_pmp_set(PMP_FREE_A_START + PMP_FREE_A_COUNT - 1, 0, 0,
+		     log2roundup(EIC770X_MEMPORT_BASE));
 
-	pmp_set(PMP_RESERVED_B, PMP_L, 0,
-		log2roundup(EIC770X_MEMPORT_LIMIT));
+	hart_pmp_set(PMP_RESERVED_B, PMP_L, 0,
+		     log2roundup(EIC770X_MEMPORT_LIMIT));
 	/**
 	 * These must come after the setup of PMP, as we are about to
 	 * enable speculation and HW prefetcher bits
@@ -321,13 +321,13 @@ static int eswin_eic7700_pmp_configure(struct sbi_scratch *scratch)
 		if (pmp_idx >= pmp_max)
 			goto no_more_pmp;
 
-		pmp_set(pmp_idx++, sbi_domain_get_oldpmp_flags(reg),
-			reg->base, reg->order);
+		hart_pmp_set(pmp_idx++, sbi_domain_get_oldpmp_flags(reg),
+			     reg->base, reg->order);
 		prev = reg;
 	}
 	/* Disable the rest */
 	while (pmp_idx < pmp_max)
-		pmp_disable(pmp_idx++);
+		hart_pmp_disable(pmp_idx++);
 
 	/* Process the second free range B [7-7] */
 	pmp_idx = PMP_FREE_B_START,
@@ -340,12 +340,12 @@ static int eswin_eic7700_pmp_configure(struct sbi_scratch *scratch)
 		if (pmp_idx >= pmp_max)
 			goto no_more_pmp;
 
-		pmp_set(pmp_idx++, sbi_domain_get_oldpmp_flags(reg),
-			reg->base, reg->order);
+		hart_pmp_set(pmp_idx++, sbi_domain_get_oldpmp_flags(reg),
+			     reg->base, reg->order);
 	}
 	/* Disable the rest */
 	while (pmp_idx < pmp_max)
-		pmp_disable(pmp_idx++);
+		hart_pmp_disable(pmp_idx++);
 
 	sbi_hart_pmp_fence();
 	return 0;
@@ -357,14 +357,14 @@ no_more_pmp:
 static void eswin_eic7700_pmp_unconfigure(struct sbi_scratch *scratch)
 {
 	/* Enable P550 internal + System Port */
-	pmp_set(PMP_FREE_A_START + PMP_FREE_A_COUNT - 1, 0, 0,
-		log2roundup(EIC770X_MEMPORT_BASE));
+	hart_pmp_set(PMP_FREE_A_START + PMP_FREE_A_COUNT - 1, 0, 0,
+		     log2roundup(EIC770X_MEMPORT_BASE));
 
 	for (unsigned int i = 0; i < PMP_FREE_A_COUNT - 1; i++)
-		pmp_disable(i + PMP_FREE_A_START);
+		hart_pmp_disable(i + PMP_FREE_A_START);
 
 	for (unsigned int i = 0; i < PMP_FREE_B_COUNT; i++)
-		pmp_disable(i + PMP_FREE_B_START);
+		hart_pmp_disable(i + PMP_FREE_B_START);
 }
 
 static struct sbi_hart_protection eswin_eic7700_pmp_protection = {
Signature · 7 lines
2.51.0


-- 
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#5 lore raw

Re: [PATCH 3/4] lib: sbi: Add hart_ prefix to PMP functions

Quoted 10 lines · Nicholas Piggin

Give PMP functions that deal with HART CSRs a hart_ prefix, to help distinguish from general PMP encoding functions that will be shared by the Tenstorrent IOMMU.

---
 include/sbi/riscv_asm.h          |  8 +++----
 lib/sbi/riscv_asm.c              | 10 ++++-----
 lib/sbi/sbi_hart_pmp.c           | 14 ++++++-------

Some changes have been made to master since you sent your patch. This fixup is required for sbi_hart_pmp.c :

lib/sbi/sbi_hart_pmp.c +0 -0
@@ -206,7 +206,7 @@ static int sbi_hart_smepmp_configure(struct

sbi_scratch *scratch)

        }
        /* Disable remaining PMP entries */
        for(; pmp_idx < pmp_count; pmp_idx++)

- pmp_disable(pmp_idx);
+ hart_pmp_disable(pmp_idx);

/*
* All entries are programmed.
@@ -291,7 +291,7 @@ static int sbi_hart_oldpmp_configure(struct sbi_scratch *scratch)

        }
        /* Disable remaining PMP entries */
        for(; pmp_idx < pmp_count; pmp_idx++)

- pmp_disable(pmp_idx);
+ hart_pmp_disable(pmp_idx);

        sbi_hart_pmp_fence();
        return 0;
Signature · 3 lines
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#6 lore raw

PATCH 4/4 platform: generic: tenstorrent: Add RISC-V IOMMU support

In reply to Nicholas Piggin

Add support for the Tenstorrent RISC-V IOMMU, and enable it for the Atlantis platform. The IOMMU must have PMA and PMP registers set up. These are implemented as MMRs with the same format as the corresponding HART CSRs, making it possible to reuse much existing PMP code. PMAs are copied from HART CSRs directly because those are set up by a prior boot stage.

---
 platform/generic/Kconfig                     |   1 +
 platform/generic/include/tenstorrent/iommu.h |  12 +
 platform/generic/tenstorrent/Kconfig         |   3 +
 platform/generic/tenstorrent/atlantis.c      |  18 ++
 platform/generic/tenstorrent/iommu.c         | 245 +++++++++++++++++++
 platform/generic/tenstorrent/objects.mk      |   1 +
 platform/generic/tenstorrent/pma.c           |   3 +-
 7 files changed, 282 insertions(+), 1 deletion(-)
 create mode 100644 platform/generic/include/tenstorrent/iommu.h
 create mode 100644 platform/generic/tenstorrent/iommu.c
platform/generic/Kconfig +1 -0
index 7de5e5a6..ba2cfa38 100644
@@ -88,6 +88,7 @@ config PLATFORM_STARFIVE_JH7110
 config PLATFORM_TENSTORRENT_ATLANTIS
 	bool "Tenstorrent Atlantis support"
 	select CPU_TENSTORRENT_ASCALON
+	select CPU_TENSTORRENT_IOMMU
 	default n
 
 config PLATFORM_THEAD
platform/generic/include/tenstorrent/iommu.h new file +12 -0
new file mode 100644
index 00000000..45231dfa
@@ -0,0 +1,12 @@
+/*
+ * SPDX-FileCopyrightText: (c) 2025-2026 Tenstorrent USA, Inc.
+ * SPDX-License-Identifier: BSD-2-Clause
+ */
+
+#ifndef __TENSTORRENT_IOMMU_H__
+#define __TENSTORRENT_IOMMU_H__
+
+int tt_iommu_configure(unsigned long iommu_m_regs);
+int tt_iommu_fdt_configure(const void *fdt);
+
+#endif
platform/generic/tenstorrent/Kconfig +3 -0
index 76c7fb32..5c524b33 100644
@@ -3,3 +3,6 @@
 
 config CPU_TENSTORRENT_ASCALON
 	bool
+
+config CPU_TENSTORRENT_IOMMU
+	bool
platform/generic/tenstorrent/atlantis.c +18 -0
index 4c312f7e..59bbd0eb 100644
@@ -12,12 +12,30 @@
 
 #include <tenstorrent/ascalon.h>
 #include <tenstorrent/pma.h>
+#include <tenstorrent/iommu.h>
 
 static int tt_atlantis_final_init(bool cold_boot)
 {
 	if (cold_boot) {
+		int rc;
+
 		/* Boot firmware sets HART PMAs. Read and verify them. */
 		tt_ascalon_discover_pmas_from_boot_hart();
+
+		/*
+		 * IOMMU must be configured at platform final_init time, to get
+		 * the right PMP settings, see init_coldboot() comment. IOMMU
+		 * is also configured with PMAs discovered from the boot HART,
+		 * above.
+		 */
+		rc = tt_iommu_fdt_configure(fdt_get_address());
+		if (rc) {
+			if (rc == SBI_ENOTSUPP)
+				sbi_printf("Tenstorrent Atlantis: No IOMMU "
+					   "in device tree, continuing.\n");
+			else
+				return rc;
+		}
 	} else {
 		/* Verify nonboot HARTs have PMAs matching boot HART */
 		tt_ascalon_verify_pmas_nonboot_hart();
platform/generic/tenstorrent/iommu.c new file +245 -0

245 lines not on this page · raw message

platform/generic/tenstorrent/objects.mk +1 -0
index 1e4aeb2d..0bf9c69b 100644
@@ -5,6 +5,7 @@
 
 platform-objs-y += tenstorrent/pma.o
 platform-objs-$(CONFIG_CPU_TENSTORRENT_ASCALON) += tenstorrent/ascalon.o
+platform-objs-$(CONFIG_CPU_TENSTORRENT_IOMMU) += tenstorrent/iommu.o
 
 carray-platform_override_modules-$(CONFIG_PLATFORM_TENSTORRENT_ATLANTIS) += tenstorrent_atlantis
 platform-objs-$(CONFIG_PLATFORM_TENSTORRENT_ATLANTIS) += tenstorrent/atlantis.o
platform/generic/tenstorrent/pma.c +2 -1
index daf60192..1556707c 100644
@@ -13,7 +13,8 @@
 
 /*
  * All PMAs in the system should be the same (after boot). The init code
- * must have set PMAs for all HARTs.
+ * must have set PMAs for all HARTs. IOMMU init programs IOMMU PMAs to
+ * match the HARTs.
  */
 
 /*
Signature · 7 lines
2.51.0


-- 
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#7 lore raw

Re: [PATCH 4/4] platform: generic: tenstorrent: Add RISC-V IOMMU support

Quoted 20 lines · Nicholas Piggin
platform/generic/tenstorrent/iommu.c new file +17 -0
 
+int tt_iommu_fdt_configure(const void *fdt)
+{
+       const char *compatible = "tenstorrent,riscv-iommu";
+       bool found_dt = false;
+       int offset = -1;
+
+       for (;;) {
+               uint64_t addr, size;
+               int rc;
+
+               offset = fdt_node_offset_by_compatible(fdt, offset, compatible);
+               if (offset < 0)
+                       break;
+
+               rc = fdt_get_node_addr_size_by_name(fdt, offset, "machine",
+                                                   &addr, &size);
+               if (rc < 0 || !size) {

platform/generic/tenstorrent/iommu.c line 230

Quoted 10 lines
platform/generic/tenstorrent/iommu.c new file +10 -0
+                       sbi_printf("tenstorrent,riscv-iommu did not find "
+                                       "machine regs\n");
+                       continue;
+               }
+
+               if (size < RISCV_IOMMU_REG_MACHINE_SIZE) {
+                       sbi_printf("tenstorrent,riscv-iommu@0x%016lx "
+                                  " machine regs region too small\n", addr);
+                       continue;
+               }

should this set found_dt = true here?

platform/generic/tenstorrent/iommu.c line 242

Quoted 12 lines
platform/generic/tenstorrent/iommu.c new file +12 -0
+
+               sbi_dprintf("tenstorrent,riscv-iommu found machine regs "
+                           "0x%016lx-0x%016lx\n", addr, addr + size - 1);
+
+               rc = tt_iommu_configure(addr);
+               if (rc)
+                       sbi_printf("tenstorrent,riscv-iommu@0x%016lx "
+                                  "init failed\n", addr);
+       }
+
+       if (!found_dt)
+               return SBI_ENOTSUPP;

because found_dt never gets modified.

Quoted 3 lines
platform/generic/tenstorrent/iommu.c new file +3 -0
+
+       return SBI_OK;
+}
Signature · 3 lines
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi
#8 lore raw

Re: [PATCH 4/4] platform: generic: tenstorrent: Add RISC-V IOMMU support

Quoted 26 lines · Joel Stanley
Quoted 24 lines · Nicholas Piggin
platform/generic/tenstorrent/iommu.c new file +21 -0
 
+int tt_iommu_fdt_configure(const void *fdt)
+{
+       const char *compatible = "tenstorrent,riscv-iommu";
+       bool found_dt = false;
+       int offset = -1;
+
+       for (;;) {
+               uint64_t addr, size;
+               int rc;
+
+               offset = fdt_node_offset_by_compatible(fdt, offset, compatible);
+               if (offset < 0)
+                       break;
+
+               rc = fdt_get_node_addr_size_by_name(fdt, offset, "machine",
+                                                   &addr, &size);
+               if (rc < 0 || !size) {
+                       sbi_printf("tenstorrent,riscv-iommu did not find "
+                                       "machine regs\n");
+                       continue;
+               }
Quoted 8 lines

platform/generic/tenstorrent/iommu.c line 230

Quoted 6 lines
platform/generic/tenstorrent/iommu.c new file +6 -0
+
+               if (size < RISCV_IOMMU_REG_MACHINE_SIZE) {
+                       sbi_printf("tenstorrent,riscv-iommu@0x%016lx "
+                                  " machine regs region too small\n", addr);
+                       continue;
+               }

should this set found_dt = true here?

Yes it should.

Thanks,
Nick

Signature · 3 lines
opensbi mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/opensbi