Add support for Broadcom BCM2712 IOMMU driver (Raspberry Pi 5)

Hi,

This series adds a driver for the Broadcom BCM2712 IOMMU found on Raspberry Pi 5, and hooks up the display controller IOMMU for efficient management of graphics memory. This is adapted from the downstream driver from Raspberry Pi (original author Nick Hollinghurst), with main changes:
- In page mapping mode, operate the translation aperture from address 0, with no bypass window. More like a IOMMU and less like a GART.
- Implement the page table management using generic_pt
- Support 4MB hugepage mapping in top level page table
- Implement brcm,iommu-cache as standards-compliant DT property
- Drop the dma-iova-offset hack, used to work around some issue seen with dma-ranges. This will need to be investigated separately and solved properly. (It's not needed for display controller iommu support included here.)
- Misc simplifications/standardisations/cleanups

The IOMMU works strictly with 4KB pages. This means that unfortunately when the kernel is compiled with PAGE_SIZE=16KB (the Raspberry Pi 5 kernel default), 12KB is wasted in each page that is allocated for page tables. I plan to address this in followup work.

It has been tested on Raspberry Pi 5 using a 3D-accelerated graphical environment which causes plenty of IOMMU maps & unmaps.

Thanks for the review comments and discussion. Further feedback and testing welcome!

Changes in v7:
- Add latest Reviewed-by / Acked-by
- Otherwise no changes, resending due to email delivery issue
- Link to v6: https://lore.kernel.org/r/[email protected]

Changes in v6:
- Add MAINTAINERS entry
- Drop unnecessary vc4 iommus property (it only needs to sit on the
underlying hvs device)
- Add dt-bindings doc for hvs iommus property
- Link to v5: https://lore.kernel.org/r/[email protected]

Changes in v5:
- Drop blocked_domain support as hardware always translates a minimum
256MB aperture when enabled
- Restrict Kconfig dependency to ARCH_BCM2835, matching other RPi drivers
- Improved error handling in device attach
- Add missing iommu_put_pages_list() in iotlb sync
- Minor readability improvements
- Link to v4: https://lore.kernel.org/r/[email protected]

Changes in v4:
- Add Jim Quinlan and Florian Fainelli as bindings maintainers
- Replace MMU register access macros with static inline functions
- Clear illegal address register and flush caches on blocking domain attach
- Add __must_hold annotations in TLB helpers
- Use BIT() macro for MMUC_CONTROL definitions
- Link to v3: https://lore.kernel.org/r/[email protected]

Changes in v3:
- Switch from a aperture and bypass window GART-like approach
to IOMMU-like 0-based translation
- Add 3 SW bits which seem unused (per the chipset docs); drop patch
for NO_SW_BIT handling. This reduces the claimed address space, so I had
to tweak a iommu_pt test accordingly.
- Disallow domain sharing across multiple hardware IOMMU instances
- Clear and flush local MMU TLB during domain attach
- Handle unattached domains safely during TLB sync
- Set cache platform driver data late during probe to prevent race condition
- Improved handling of cache flush timeouts
- Use format string for device name in iommu_device_sysfs_add()
- Use cleanup.h helpers for scope management
- Expand DT binding descriptions for IOMMU and IOMMUC
- Add iommus property to HVS display node in DT
- Drop iommuc back-compat with current RPi DT
- Link to v2: https://lore.kernel.org/r/[email protected]

Changes in v2:
- Use iommu_fwspec in xlate & probe_device
- Add blocked_domain support
- Simplify page size configuration between iommu driver and format
- Introduce PT_FEAT_NO_SW_BIT flag to allow formats to explicitly opt-in to
a defensive flushing algorithm when no sw bit is available
- Implement 4MB pages at Level 1; this is more efficient and means the
"allow full-table contiguous leaves in unit tests" commit has been
dropped because it is no longer needed.
- Hardcode the aperture in an unused part of the address space, removing
it from DT (it's a software-specific decision, not an attribute of the
hardware), adding more explanatory comments
- Use pt_full_va_prefix to have the PT format operate within the aperture
space, enabling use of IOMMU_PT_DOMAIN_OPS
- Add more dt-bindings explanation to brcm,iommu-cache and drop docs
around the cache property used in RPi firmware (the driver still supports
this however, for fw compat reasons)
- Correct iommu device address in bcm2712.dtsi
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Daniel Drake (6):
iommu_pt: Fix test_pgsize_boundary() failure on narrow-OA formats
dt-bindings: iommu: Add Broadcom BCM2712 IOMMU
dt-bindings: display: brcm,bcm2835-hvs: Document iommus property iommu/generic_pt: Add Broadcom BCM2712 page table format
iommu: Add Broadcom BCM2712 IOMMU driver
arm64: dts: broadcom: bcm2712: Add GPU IOMMU and IOMMU cache nodes

 .../bindings/display/brcm,bcm2835-hvs.yaml         |   3 +
 .../bindings/iommu/brcm,bcm2712-iommu.yaml         |  54 ++
 .../bindings/iommu/brcm,bcm2712-iommuc.yaml        |  40 ++
 MAINTAINERS                                        |  12 +
 arch/arm64/boot/dts/broadcom/bcm2712.dtsi          |  14 +
 drivers/iommu/Kconfig                              |  15 +
 drivers/iommu/Makefile                             |   1 +
 drivers/iommu/bcm2712-iommu-cache.c                |  84 ++++
 drivers/iommu/bcm2712-iommu-cache.h                |   9 +
 drivers/iommu/bcm2712-iommu.c                      | 550 +++++++++++++++++++++
 drivers/iommu/generic_pt/.kunitconfig              |   1 +
 drivers/iommu/generic_pt/Kconfig                   |  10 +
 drivers/iommu/generic_pt/fmt/Makefile              |   2 +
 drivers/iommu/generic_pt/fmt/bcm2712.h             | 288 +++++++++++
 drivers/iommu/generic_pt/fmt/defs_bcm2712.h        |  18 +
 drivers/iommu/generic_pt/fmt/iommu_bcm2712.c       |   6 +
 drivers/iommu/generic_pt/kunit_iommu_pt.h          |   2 +-
 include/linux/generic_pt/common.h                  |   6 +
 include/linux/generic_pt/iommu.h                   |  12 +
 19 files changed, 1126 insertions(+), 1 deletion(-)

---
base-commit: 28924df2a08f440c73991b83028032c901de2ae4
change-id: 20260712-bcm2712-iommu-submit-2e09899e65c4

Best regards,

Daniel Drake <[email protected]>

test_pgsize_boundary() tests boundary mapping with a hardcoded physical address that requires 38 bits of physical address space. However, the upcoming page table format for the Broadcom BCM2712 IOMMU only supports 37 bits of output address space.

This causes a test failure as iommu_map() rejects the large address with -ERANGE.

Drop the high bits of the address (they are not relevant to the boundary calculations being tested) so that it fits into 32 bits.

---
 drivers/iommu/generic_pt/kunit_iommu_pt.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

index ece1c9b8c55d..9fc71db74234 100644
@@ -403,7 +403,7 @@ static void test_pgsize_boundary(struct kunit *test)
 	    priv->smallest_pgsz != SZ_4K)
 		kunit_skip(test, "Format does not have the required range");
 
-	do_map(test, 0xfef80000, 0x208b95d000, 0xfef9ffff - 0xfef80000 + 1);
+	do_map(test, 0xfef80000, 0x8b95d000, 0xfef9ffff - 0xfef80000 + 1);
 }
 
 /* See https://lore.kernel.org/r/[email protected] */

2.55.0

Add bindings for the Broadcom BCM2712 IOMMUs and their shared TLB cache.

---
 .../bindings/iommu/brcm,bcm2712-iommu.yaml         | 54 ++++++++++++++++++++++
 .../bindings/iommu/brcm,bcm2712-iommuc.yaml        | 40 ++++++++++++++++
 2 files changed, 94 insertions(+)

new file mode 100644
index 000000000000..5ca8ce469ec3
@@ -0,0 +1,54 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iommu/brcm,bcm2712-iommu.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom BCM2712 IOMMU
+
+maintainers:
+  - Daniel Drake <[email protected]>
+  - Florian Fainelli <[email protected]>
+  - Jim Quinlan <[email protected]>
+
+description:
+  The BCM2712 SoC features multiple independent IOMMU instances providing
+  address translation for multimedia hardware blocks, such as the VC6 display
+  pipeline, camera receivers, and ISP. Each IOMMU has its own page tables,
+  control registers, and local TLB, and is additionally linked to a centralized
+  L2 TLB (IOMMUC).
+
+properties:
+  compatible:
+    const: brcm,bcm2712-iommu
+
+  reg:
+    maxItems: 1
+
+  '#iommu-cells':
+    const: 0
+
+  brcm,iommu-cache:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description:
+      Phandle to the shared IOMMU cache (IOMMUC). Each IOMMU instance caches
+      translated descriptors in this centralized L2 TLB cache block and must
+      trigger cache invalidation commands on it when updating or unmapping page
+      table entries.
+
+required:
+  - compatible
+  - reg
+  - '#iommu-cells'
+  - brcm,iommu-cache
+
+additionalProperties: false
+
+examples:
+  - |
+    iommu@5200 {
+        compatible = "brcm,bcm2712-iommu";
+        reg = <0x5200 0x80>;
+        brcm,iommu-cache = <&iommuc>;
+        #iommu-cells = <0>;
+    };

new file mode 100644
index 000000000000..47c584ad667a
@@ -0,0 +1,40 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iommu/brcm,bcm2712-iommuc.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Broadcom BCM2712 Shared IOMMU Cache (IOMMUC)
+
+maintainers:
+  - Daniel Drake <[email protected]>
+  - Florian Fainelli <[email protected]>
+  - Jim Quinlan <[email protected]>
+
+description:
+  The BCM2712 IOMMUC is a centralized Translation Lookaside Buffer (TLB) cache
+  which accelerates address translation across the SoC's IOMMU devices. If an
+  address mapping is not found in the IOMMU's local TLB cache, then this
+  IOMMUC is consulted as a L2 TLB. The SoC includes a single IOMMUC which is
+  shared between all IOMMUs, and it must be explicitly invalidated when
+  modifying or unmapping IOMMU page tables.
+
+properties:
+  compatible:
+    const: brcm,bcm2712-iommuc
+
+  reg:
+    maxItems: 1
+
+required:
+  - compatible
+  - reg
+
+additionalProperties: false
+
+examples:
+  - |
+    iommuc@5b00 {
+        compatible = "brcm,bcm2712-iommuc";
+        reg = <0x5b00 0x80>;
+    };

2.55.0

On BCM2712, the Hardware Video Scaler (HVS) is connected behind an IOMMU. Add the optional iommus property to allow describing this connection.

---
 Documentation/devicetree/bindings/display/brcm,bcm2835-hvs.yaml | 3 +++
 1 file changed, 3 insertions(+)

index 9aca38a58a16..8f8aae9f3788 100644
@@ -35,6 +35,9 @@ properties:
     minItems: 1
     maxItems: 2
 
+  iommus:
+    maxItems: 1
+
 required:
   - compatible
   - reg

2.55.0

Implement generic_pt support for the 2-level page table format used by the BCM2712 IOMMUs.

Adapted from Raspberry Pi's downstream bcm2712-iommu driver (original author Nick Hollinghurst).

---
 drivers/iommu/generic_pt/.kunitconfig        |   1 +
 drivers/iommu/generic_pt/Kconfig             |  10 +
 drivers/iommu/generic_pt/fmt/Makefile        |   2 +
 drivers/iommu/generic_pt/fmt/bcm2712.h       | 288 +++++++++++++++++++++++++++
 drivers/iommu/generic_pt/fmt/defs_bcm2712.h  |  18 ++
 drivers/iommu/generic_pt/fmt/iommu_bcm2712.c |   6 +
 include/linux/generic_pt/common.h            |   6 +
 include/linux/generic_pt/iommu.h             |  12 ++
 8 files changed, 343 insertions(+)

index 0bb98fe581fe..a3da74328f12 100644
@@ -7,6 +7,7 @@ CONFIG_IOMMU_PT_AMDV1=y
 CONFIG_IOMMU_PT_VTDSS=y
 CONFIG_IOMMU_PT_RISCV64=y
 CONFIG_IOMMU_PT_X86_64=y
+CONFIG_IOMMU_PT_BCM2712=y
 CONFIG_IOMMU_PT_KUNIT_TEST=y
 
 CONFIG_IOMMUFD=y

index f4ed1add58b7..2a81dac960ac 100644
@@ -72,6 +72,15 @@ config IOMMU_PT_X86_64
 
 	  Selected automatically by an IOMMU driver that uses this format.
 
+config IOMMU_PT_BCM2712
+	tristate "IOMMU page table for Broadcom BCM2712"
+	help
+	  iommu_domain implementation for the Broadcom BCM2712 IOMMU found on
+	  Raspberry Pi 5. It supports 4K page sizes over a 2-level page table
+	  format.
+
+	  Selected automatically by an IOMMU driver that uses this format.
+
 config IOMMU_PT_KUNIT_TEST
 	tristate "IOMMU Page Table KUnit Test" if !KUNIT_ALL_TESTS
 	depends on KUNIT

@@ -79,6 +88,7 @@ config IOMMU_PT_KUNIT_TEST
 	depends on IOMMU_PT_RISCV64 || !IOMMU_PT_RISCV64
 	depends on IOMMU_PT_X86_64 || !IOMMU_PT_X86_64
 	depends on IOMMU_PT_VTDSS || !IOMMU_PT_VTDSS
+	depends on IOMMU_PT_BCM2712 || !IOMMU_PT_BCM2712
 	default KUNIT_ALL_TESTS
 	help
 	  Enable kunit tests for GENERIC_PT and IOMMU_PT that covers all the

index ea024d582594..82ca3c36823d 100644
@@ -9,6 +9,8 @@ iommu_pt_fmt-$(CONFIG_IOMMU_PT_RISCV64) += riscv64
 
 iommu_pt_fmt-$(CONFIG_IOMMU_PT_X86_64) += x86_64
 
+iommu_pt_fmt-$(CONFIG_IOMMU_PT_BCM2712) += bcm2712
+
 IOMMU_PT_KUNIT_TEST :=
 define create_format
 obj-$(2) += iommu_$(1).o

new file mode 100644
index 000000000000..f1d983be1716
@@ -0,0 +1,18 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef __GENERIC_PT_FMT_DEFS_BCM2712_H
+#define __GENERIC_PT_FMT_DEFS_BCM2712_H
+
+#include <linux/generic_pt/common.h>
+#include <linux/types.h>
+
+typedef u32 pt_bcm2712_entry_t;
+typedef u64 pt_vaddr_t;
+typedef u64 pt_oaddr_t;
+
+struct bcm2712pt_write_attrs {
+	pt_bcm2712_entry_t descriptor_bits;
+	gfp_t gfp;
+};
+#define pt_write_attrs bcm2712pt_write_attrs
+
+#endif

new file mode 100644
index 000000000000..43fbc3cf3adc
@@ -0,0 +1,6 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#define PT_FMT bcm2712
+#define PT_SUPPORTED_FEATURES BIT(PT_FEAT_DMA_INCOHERENT)
+
+#include <linux/generic_pt/iommu.h>
+#include "iommu_template.h"

index 07ef1c8341a4..1b39b27f0cfa 100644
@@ -213,4 +213,10 @@ enum {
 	PT_FEAT_X86_64_AMD_ENCRYPT_TABLES = PT_FEAT_FMT_START,
 };
 
+struct pt_bcm2712 {
+	struct pt_common common;
+	u8 bigpage_lg2;
+	u8 superpage_lg2;
+};
+
 #endif

index dd0edd02a48a..13ebb72a67de 100644
@@ -346,6 +346,18 @@ struct pt_iommu_x86_64_hw_info {
 
 IOMMU_FORMAT(x86_64, x86_64_pt);
 
+struct pt_iommu_bcm2712_cfg {
+	struct pt_iommu_cfg common;
+	u8 bigpage_lg2;
+	u8 superpage_lg2;
+};
+
+struct pt_iommu_bcm2712_hw_info {
+	phys_addr_t pt_base;
+};
+
+IOMMU_FORMAT(bcm2712, bcm2712pt);
+
 #undef IOMMU_PROTOTYPES
 #undef IOMMU_FORMAT
 #endif

2.55.0

This IOMMU translates memory access requests for the VC6 display pipeline and various multimedia devices in the Broadcom BCM2712 SoC used on Raspberry Pi 5.

The driver uses the generic_pt framework to manage the 2-level page tables. It also provides support for the centralized L2 TLB (IOMMUC), which is shared across the SoC's IOMMU instances and must be flushed when modifying or unmapping page table entries.

Adapted from Raspberry Pi's downstream bcm2712-iommu driver (original author Nick Hollinghurst).

---
 MAINTAINERS                         |  12 +
 drivers/iommu/Kconfig               |  15 +
 drivers/iommu/Makefile              |   1 +
 drivers/iommu/bcm2712-iommu-cache.c |  84 ++++++
 drivers/iommu/bcm2712-iommu-cache.h |   9 +
 drivers/iommu/bcm2712-iommu.c       | 550 ++++++++++++++++++++++++++++++++++++
 6 files changed, 671 insertions(+)

index 6215fcb07770..032401692efb 100644
@@ -5166,6 +5166,18 @@ N:	bcm2712
 N:	bcm283*
 N:	raspberrypi
 
+BROADCOM BCM2712 IOMMU DRIVER
+M:	Daniel Drake <[email protected]>
+M:	Florian Fainelli <[email protected]>
+M:	Jim Quinlan <[email protected]>
+R:	Broadcom internal kernel review list <[email protected]>
+L:	[email protected]
+L:	[email protected] (moderated for non-subscribers)
+S:	Maintained
+F:	Documentation/devicetree/bindings/iommu/brcm,bcm2712-iommu*
+F:	drivers/iommu/bcm2712-iommu*
+F:	drivers/iommu/generic_pt/fmt/*bcm2712*
+
 BROADCOM BCM281XX/BCM11XXX/BCM216XX ARM ARCHITECTURE
 M:	Florian Fainelli <[email protected]>
 M:	Ray Jui <[email protected]>

index 6e07bd69467a..de85fba9c0a7 100644
@@ -386,6 +386,21 @@ config VSI_IOMMU
 	  Say Y here if you want to use this IOMMU in front of these
 	  hardware blocks.
 
+config BCM2712_IOMMU
+	bool "BCM2712 IOMMU driver"
+	depends on (ARCH_BCM2835 && ARM64) || COMPILE_TEST
+	select IOMMU_API
+	select GENERIC_PT
+	select IOMMU_PT
+	select IOMMU_PT_BCM2712
+	help
+	  Support for IOMMU on BCM2712 SoC. This IOMMU can be used by the
+	  display controller and various multimedia devices to perform
+	  efficient memory management.
+
+	  Say Y here if you want to use this IOMMU in front of these
+	  hardware blocks.
+
 config IOMMU_DEBUG_PAGEALLOC
 	bool "Debug IOMMU mappings against page allocations"
 	depends on DEBUG_PAGEALLOC && IOMMU_API && PAGE_EXTENSION

index 2f05725eaab1..29a26d2a3af1 100644
@@ -37,4 +37,5 @@ obj-$(CONFIG_IOMMU_IOPF) += io-pgfault.o
 obj-$(CONFIG_SPRD_IOMMU) += sprd-iommu.o
 obj-$(CONFIG_APPLE_DART) += apple-dart.o
 obj-$(CONFIG_VSI_IOMMU) += vsi-iommu.o
+obj-$(CONFIG_BCM2712_IOMMU) += bcm2712-iommu.o bcm2712-iommu-cache.o
 obj-$(CONFIG_IOMMU_DEBUG_PAGEALLOC) += iommu-debug-pagealloc.o

new file mode 100644
index 000000000000..d2f7851831cc
@@ -0,0 +1,9 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef _BCM2712_IOMMU_CACHE_H
+#define _BCM2712_IOMMU_CACHE_H
+
+struct bcm2712_iommu_cache;
+
+void bcm2712_iommu_cache_flush(struct bcm2712_iommu_cache *cache);
+
+#endif

2.55.0

Define the IOMMU and IOMMUC nodes for the Broadcom BCM2712 SoC found in the Raspberry Pi 5. Enable use of the IOMMU for the HVS device that handles memory allocations for the display block.

---
 arch/arm64/boot/dts/broadcom/bcm2712.dtsi | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

index 45aa8ec5e2b1..aca9e0e3a874 100644
@@ -626,6 +626,19 @@ pcie2: pcie@1000120000 {
 			status = "disabled";
 		};
 
+		iommu4: iommu@1000005200 {
+			/* IOMMU for VC4 (HVS & MPL/TXP), Unicam, PISP-FE, MiniBVN */
+			compatible = "brcm,bcm2712-iommu";
+			reg = <0x10 0x5200 0x0 0x80>;
+			#iommu-cells = <0>;
+			brcm,iommu-cache = <&iommuc>;
+		};
+
+		iommuc: iommuc@1000005b00 {
+			compatible = "brcm,bcm2712-iommuc";
+			reg = <0x10 0x5b00 0x0 0x80>;
+		};
+
 		mip0: msi-controller@1000130000 {
 			compatible = "brcm,bcm2712-mip";
 			reg = <0x10 0x00130000 0x00 0xc0>,

@@ -701,5 +714,6 @@ hvs: hvs@107c580000 {
 		interrupt-parent = <&disp_intr>;
 		interrupts = <2>, <9>, <16>;
 		interrupt-names = "ch0-eof", "ch1-eof", "ch2-eof";
+		iommus = <&iommu4>;
 	};
 };

2.55.0

Applied patches 1-5, thanks.

And applied patch 6 to devicetree-arm64/next, thanks!

Florian