Skip to contentSkip to conversation

Apple M1 DART IOMMU driver

PATCH Applied Sven Peter 2 participants 5 messages Aug 3, 2021 – Aug 9, 2021

Subsystems (4)
lore
Share & tools
lore

Sven Peter

author
#0 lore raw

v5 cover Apple M1 DART IOMMU driver

Hi,

This is v5 of my Apple M1 DART IOMMU driver series as a follow up to the previous versions [1][2][3][7].

Short summary: this series adds support for the iommu found in Apple's new M1 SoC which is required to use DMA on most peripherals like the display controller, the USB ports or the internal PCIe bus (which is used for WiFi, Ethernet and more USB ports).
So far this code has been tested by multiple people with dwc3 in host and device mode (which both only require changes to the device tree after this patchset) and PCIe (using a yet to be finalized patchset).

Note that this version has to be applied on top of iommu/core or iommu/next since it now uses the new map_pages/unmap_pages API.

== Testing this patchset with the USB-C controller ==

The two USB-C ports on the M1 machines are exposed as two separate dwc3 controllers which are behind a DART. Now that my USB phy bringup code has been merged into our bootloader m1n1 you can easily test this patchset yourself:

1) Follow the instructions at [4] to setup our bootloader m1n1 on your M1 machine which will allow you to boot kernels using a normal USB cable. Note that you'll still need a special setup to expose the UART for very low-level debugging.

2) Apply this patchset and add the DART and dwc3 nodes as done in e.g. [5].

3) Boot the kernel through our bootloader m1n1. You'll need a version after commit [6] which enables the USB PHY and the USB PD chip.

Note that the dwc3 controller has a quirk where each root port can only be used once right now. The most stable way to test is to already connected the USB device(s) before booting the kernel.

It's also possible to test the PCIe bus but this requires a more complex setup for now. I can write a quick howto if anyone is interested though. (tl;dr: Mark Kettenis has a u-boot fork that includes PCIe bringup code and Marc Zyngier has a WIP patchset to add a PCIe driver)

== Project Blurb ==

Asahi Linux is an open community project dedicated to developing and maintaining mainline support for Apple Silicon on Linux. Feel free to drop by #asahi and #asahi-dev on OFTC to chat with us, or check our website for more information on the project:

https://asahilinux.org/

== Changes ==

Changes for v5:
- Added reviewed-by and tested-by tags (thanks!)
- Rebased on top of iommu/core and replaced map/unmap with map_pages/unmap_pages
- Removed software bypass hacks: I've tried a few different variants now and they all have drawbacks or incompatibilities that I am not comfortable with. This means that PCIe devices (for which there is no kernel support yet anyway) will not work correctly for now on 4K kernels. I plan to address this in a follow-up series where I want to modify the dma-iommu layer to support pagesize mismatches.
- Removed reference to ARM from the constants for the io-pgtable code
- Addressed the following comments by Robin Murphy, which resulted in some major changes to apple-dart.c
- Correctly assign iommu_groups in apple_dart_device_group
- Get rid of the fwspec-inspired of_xlate linked lists and replaced them with a simple static array with a streamid bitmap covering all known cases
- Relax locking: Now only a single spinlock around TLB flushes and a mutex around domain initialization are required. attach_dev/detach_dev uses atomic64_t.
- Set .suppress_bind_attrs to prevent manual unbinding
- Get rid of .shutdown since there's no real need to clean anything up
- Manage interrupts manually instead of using devm_* to prevent situations where a shared interrupt could trigger while clocks are disabled
- apple_dart_irq now prints "unknown" when more than a single error bit has been set.
- Use DL_FLAG_AUTOREMOVE_SUPPLIER so that there's no need to keep track of the pointer
- Ignore any unknown protection flags instead of failing
- Use dev_err_ratelimited and clk_bulk_disable_unprepare instead of open-coding them
- Correctly set and clear iommu_bus_ops
- Removed unhelpful WARN_ONs and duplicate sanity checks
- Removed unnecessary identity stream map reset code
- Renamed apple-dart-iommu.c to apple-dart.c
- Fixed commit style to use the correct subsystem style
- Possibly some smaller fixes I forgot about

Changes for v4:
- Addressed Rob Herring's remark about the incorrect phandles in the device tree binding example and added his reviewed-by tag
- Take the software linear mapping range from the bus instead of hardcoding it in the driver
- Use def_domain_type to force bypass mode if there's a pagesize mismatch between the DART (hardwired to 16KB) and the kernel (may use 4K)
- Added lockdep_assert_held instead of comments as suggested by Rouven Czerwinski
- rebased on 5.13-rc7

Changes for v3:
- fixed name of the iommu node in the device tree binding example pointed out by Arnd Bergmann
- remove hardware specific checks from io-pgtable.c as pointed out by Will Deacon
- introduced a fake bypass mode by programming static linear pagetables if the DART does not support regular bypass mode as proposed by Alex Graf
- added checks to enforce bypass mode if there is a pagesize mismatch between the DART HW and the CPU.
- fixed usage of GFP_KERNEL during a held spinlock found by Julia Lawall
- rebased on v5.13-rc3

Changes for v2:
- fixed devicetree binding linting issues pointed out by Rob Herring and reworked that file.
- made DART-specific code in io-pgtable.c unconditional and removed flag from Kconfig as proposed by Robin Murphy.
- allowed multiple DART nodes in the "iommus" property as proposed by Rob Herring and Robin Murphy. this resulted in significant changes to apple-iommu-dart.c.
- the domain aperture is now forced to 32bit if translation is enabled after the original suggestion to limit the aperture by Mark Kettenis and the follow-up discussion and investigation with Mark Kettenis, Arnd Bergmann, Robin Murphy and Rob Herring. This change also simplified the code in io-pgtable.c and made some of the improvements suggested during review not apply anymore.
- added support for bypassed and isolated domain modes.
- reject IOMMU_MMIO and IOMMU_NOEXEC since it's unknown how to set these up for now or if the hardware even supports these flags.
- renamed some registers to be less confusing (mainly s/DOMAIN/STREAM/ to prevent confusion with linux's iommu domain concept).

[1] https://lore.kernel.org/linux-iommu/[email protected]/
[2] https://lore.kernel.org/linux-iommu/[email protected]/
[3] https://lore.kernel.org/linux-iommu/[email protected]/
[4] https://github.com/AsahiLinux/docs/wiki/Developer-Quickstart
[5] https://github.com/AsahiLinux/linux/commit/7d4ebb0b22e9bfec849e2af86ddeb46ec29d7feb
[6] https://github.com/AsahiLinux/m1n1/commit/9529ec2b4fd6550f9cfd66d9f2448b90804699a1
[7] https://lore.kernel.org/linux-iommu/[email protected]/

Sven Peter (3):
iommu/io-pgtable: Add DART pagetable format
dt-bindings: iommu: add DART iommu bindings
iommu/dart: Add DART iommu driver

 .../devicetree/bindings/iommu/apple,dart.yaml |  81 ++
 MAINTAINERS                                   |   7 +
 drivers/iommu/Kconfig                         |  14 +
 drivers/iommu/Makefile                        |   1 +
 drivers/iommu/apple-dart.c                    | 923 ++++++++++++++++++
 drivers/iommu/io-pgtable-arm.c                |  63 ++
 drivers/iommu/io-pgtable.c                    |   1 +
 include/linux/io-pgtable.h                    |   7 +
 8 files changed, 1097 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/iommu/apple,dart.yaml
 create mode 100644 drivers/iommu/apple-dart.c
Signature · 1 line
2.25.1
author
#1 lore raw

PATCH v5 1/3 iommu/io-pgtable: Add DART pagetable format

Apple's DART iommu uses a pagetable format that shares some similarities with the ones already implemented by io-pgtable.c. Add a new format variant to support the required differences so that we don't have to duplicate the pagetable handling code.

---
 drivers/iommu/io-pgtable-arm.c | 63 ++++++++++++++++++++++++++++++++++
 drivers/iommu/io-pgtable.c     |  1 +
 include/linux/io-pgtable.h     |  7 ++++
 3 files changed, 71 insertions(+)
drivers/iommu/io-pgtable-arm.c +63 -0
index 053df4048a29..0779eb96bd29 100644
@@ -130,6 +130,9 @@
 #define ARM_MALI_LPAE_MEMATTR_IMP_DEF	0x88ULL
 #define ARM_MALI_LPAE_MEMATTR_WRITE_ALLOC 0x8DULL
 
+#define APPLE_DART_PTE_PROT_NO_WRITE (1<<7)
+#define APPLE_DART_PTE_PROT_NO_READ (1<<8)
+
 /* IOPTE accessors */
 #define iopte_deref(pte,d) __va(iopte_to_paddr(pte, d))
 
@@ -402,6 +405,15 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
 {
 	arm_lpae_iopte pte;
 
+	if (data->iop.fmt == APPLE_DART) {
+		pte = 0;
+		if (!(prot & IOMMU_WRITE))
+			pte |= APPLE_DART_PTE_PROT_NO_WRITE;
+		if (!(prot & IOMMU_READ))
+			pte |= APPLE_DART_PTE_PROT_NO_READ;
+		return pte;
+	}
+
 	if (data->iop.fmt == ARM_64_LPAE_S1 ||
 	    data->iop.fmt == ARM_32_LPAE_S1) {
 		pte = ARM_LPAE_PTE_nG;
@@ -1102,6 +1114,52 @@ arm_mali_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie)
 	return NULL;
 }
 
+static struct io_pgtable *
+apple_dart_alloc_pgtable(struct io_pgtable_cfg *cfg, void *cookie)
+{
+	struct arm_lpae_io_pgtable *data;
+	int i;
+
+	if (cfg->oas > 36)
+		return NULL;
+
+	data = arm_lpae_alloc_pgtable(cfg);
+	if (!data)
+		return NULL;
+
+	/*
+	 * The table format itself always uses two levels, but the total VA
+	 * space is mapped by four separate tables, making the MMIO registers
+	 * an effective "level 1". For simplicity, though, we treat this
+	 * equivalently to LPAE stage 2 concatenation at level 2, with the
+	 * additional TTBRs each just pointing at consecutive pages.
+	 */
+	if (data->start_level < 1)
+		goto out_free_data;
+	if (data->start_level == 1 && data->pgd_bits > 2)
+		goto out_free_data;
+	if (data->start_level > 1)
+		data->pgd_bits = 0;
+	data->start_level = 2;
+	cfg->apple_dart_cfg.n_ttbrs = 1 << data->pgd_bits;
+	data->pgd_bits += data->bits_per_level;
+
+	data->pgd = __arm_lpae_alloc_pages(ARM_LPAE_PGD_SIZE(data), GFP_KERNEL,
+					   cfg);
+	if (!data->pgd)
+		goto out_free_data;
+
+	for (i = 0; i < cfg->apple_dart_cfg.n_ttbrs; ++i)
+		cfg->apple_dart_cfg.ttbr[i] =
+			virt_to_phys(data->pgd + i * ARM_LPAE_GRANULE(data));
+
+	return &data->iop;
+
+out_free_data:
+	kfree(data);
+	return NULL;
+}
+
 struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s1_init_fns = {
 	.alloc	= arm_64_lpae_alloc_pgtable_s1,
 	.free	= arm_lpae_free_pgtable,
@@ -1127,6 +1185,11 @@ struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns = {
 	.free	= arm_lpae_free_pgtable,
 };
 
+struct io_pgtable_init_fns io_pgtable_apple_dart_init_fns = {
+	.alloc	= apple_dart_alloc_pgtable,
+	.free	= arm_lpae_free_pgtable,
+};
+
 #ifdef CONFIG_IOMMU_IO_PGTABLE_LPAE_SELFTEST
 
 static struct io_pgtable_cfg *cfg_cookie __initdata;
drivers/iommu/io-pgtable.c +1 -0
index 6e9917ce980f..f4bfcef98297 100644
@@ -20,6 +20,7 @@ io_pgtable_init_table[IO_PGTABLE_NUM_FMTS] = {
 	[ARM_64_LPAE_S1] = &io_pgtable_arm_64_lpae_s1_init_fns,
 	[ARM_64_LPAE_S2] = &io_pgtable_arm_64_lpae_s2_init_fns,
 	[ARM_MALI_LPAE] = &io_pgtable_arm_mali_lpae_init_fns,
+	[APPLE_DART] = &io_pgtable_apple_dart_init_fns,
 #endif
 #ifdef CONFIG_IOMMU_IO_PGTABLE_ARMV7S
 	[ARM_V7S] = &io_pgtable_arm_v7s_init_fns,
include/linux/io-pgtable.h +7 -0
index c43f3b899d2a..a738483fb4da 100644
@@ -16,6 +16,7 @@ enum io_pgtable_fmt {
 	ARM_V7S,
 	ARM_MALI_LPAE,
 	AMD_IOMMU_V1,
+	APPLE_DART,
 	IO_PGTABLE_NUM_FMTS,
 };
 
@@ -136,6 +137,11 @@ struct io_pgtable_cfg {
 			u64	transtab;
 			u64	memattr;
 		} arm_mali_lpae_cfg;
+
+		struct {
+			u64 ttbr[4];
+			u32 n_ttbrs;
+		} apple_dart_cfg;
 	};
 };
 
@@ -254,5 +260,6 @@ extern struct io_pgtable_init_fns io_pgtable_arm_64_lpae_s2_init_fns;
 extern struct io_pgtable_init_fns io_pgtable_arm_v7s_init_fns;
 extern struct io_pgtable_init_fns io_pgtable_arm_mali_lpae_init_fns;
 extern struct io_pgtable_init_fns io_pgtable_amd_iommu_v1_init_fns;
+extern struct io_pgtable_init_fns io_pgtable_apple_dart_init_fns;
 
 #endif /* __IO_PGTABLE_H */
Signature · 1 line
2.25.1
author
#2 lore raw

PATCH v5 2/3 dt-bindings: iommu: add DART iommu bindings

In reply to Sven Peter

DART (Device Address Resolution Table) is the iommu found on Apple ARM SoCs such as the M1.

---
 .../devicetree/bindings/iommu/apple,dart.yaml | 81 +++++++++++++++++++
 MAINTAINERS                                   |  6 ++
 2 files changed, 87 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/iommu/apple,dart.yaml
Documentation/devicetree/bindings/iommu/apple,dart.yaml new file +81 -0
new file mode 100644
index 000000000000..94aa9e9afa59
@@ -0,0 +1,81 @@
+# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/iommu/apple,dart.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Apple DART IOMMU
+
+maintainers:
+  - Sven Peter <[email protected]>
+
+description: |+
+  Apple SoCs may contain an implementation of their Device Address
+  Resolution Table which provides a mandatory layer of address
+  translations for various masters.
+
+  Each DART instance is capable of handling up to 16 different streams
+  with individual pagetables and page-level read/write protection flags.
+
+  This DART IOMMU also raises interrupts in response to various
+  fault conditions.
+
+properties:
+  compatible:
+    const: apple,t8103-dart
+
+  reg:
+    maxItems: 1
+
+  interrupts:
+    maxItems: 1
+
+  clocks:
+    description:
+      Reference to the gate clock phandle if required for this IOMMU.
+      Optional since not all IOMMUs are attached to a clock gate.
+
+  '#iommu-cells':
+    const: 1
+    description:
+      Has to be one. The single cell describes the stream id emitted by
+      a master to the IOMMU.
+
+required:
+  - compatible
+  - reg
+  - '#iommu-cells'
+  - interrupts
+
+additionalProperties: false
+
+examples:
+  - |+
+    dart1: iommu@82f80000 {
+      compatible = "apple,t8103-dart";
+      reg = <0x82f80000 0x4000>;
+      interrupts = <1 781 4>;
+      #iommu-cells = <1>;
+    };
+
+    master1 {
+      iommus = <&dart1 0>;
+    };
+
+  - |+
+    dart2a: iommu@82f00000 {
+      compatible = "apple,t8103-dart";
+      reg = <0x82f00000 0x4000>;
+      interrupts = <1 781 4>;
+      #iommu-cells = <1>;
+    };
+    dart2b: iommu@82f80000 {
+      compatible = "apple,t8103-dart";
+      reg = <0x82f80000 0x4000>;
+      interrupts = <1 781 4>;
+      #iommu-cells = <1>;
+    };
+
+    master2 {
+      iommus = <&dart2a 0>, <&dart2b 1>;
+    };
MAINTAINERS +6 -0
index c9467d2839f5..0f450f7d5336 100644
@@ -1262,6 +1262,12 @@ L:	[email protected]
 S:	Odd fixes
 F:	drivers/input/mouse/bcm5974.c
 
+APPLE DART IOMMU DRIVER
+M:	Sven Peter <[email protected]>
+L:	[email protected]
+S:	Maintained
+F:	Documentation/devicetree/bindings/iommu/apple,dart.yaml
+
 APPLE SMC DRIVER
 M:	Henrik Rydberg <[email protected]>
 L:	[email protected]
Signature · 1 line
2.25.1
author
#3 lore raw

PATCH v5 3/3 iommu/dart: Add DART iommu driver

In reply to Sven Peter

Apple's new SoCs use iommus for almost all peripherals. These Device Address Resolution Tables must be setup before these peripherals can act as DMA masters.

---
 MAINTAINERS                |   1 +
 drivers/iommu/Kconfig      |  14 +
 drivers/iommu/Makefile     |   1 +
 drivers/iommu/apple-dart.c | 923 +++++++++++++++++++++++++++++++++++++
 4 files changed, 939 insertions(+)
 create mode 100644 drivers/iommu/apple-dart.c
MAINTAINERS +1 -0
index 0f450f7d5336..5f3ef4298594 100644
@@ -1267,6 +1267,7 @@ M:	Sven Peter <[email protected]>
 L:	[email protected]
 S:	Maintained
 F:	Documentation/devicetree/bindings/iommu/apple,dart.yaml
+F:	drivers/iommu/apple-dart.c
 
 APPLE SMC DRIVER
 M:	Henrik Rydberg <[email protected]>
drivers/iommu/Kconfig +14 -0
index c84da8205be7..dfe81da483e9 100644
@@ -290,6 +290,20 @@ config SPAPR_TCE_IOMMU
 	  Enables bits of IOMMU API required by VFIO. The iommu_ops
 	  is not implemented as it is not necessary for VFIO.
 
+config APPLE_DART
+	tristate "Apple DART IOMMU Support"
+	depends on ARM64 || (COMPILE_TEST && !GENERIC_ATOMIC64)
+	select IOMMU_API
+	select IOMMU_IO_PGTABLE_LPAE
+	default ARCH_APPLE
+	help
+	  Support for Apple DART (Device Address Resolution Table) IOMMUs
+	  found in Apple ARM SoCs like the M1.
+	  This IOMMU is required for most peripherals using DMA to access
+	  the main memory.
+
+	  Say Y here if you are using an Apple SoC.
+
 # ARM IOMMU support
 config ARM_SMMU
 	tristate "ARM Ltd. System MMU (SMMU) Support"
drivers/iommu/Makefile +1 -0
index c0fb0ba88143..bc7f730edbb0 100644
@@ -29,3 +29,4 @@ obj-$(CONFIG_HYPERV_IOMMU) += hyperv-iommu.o
 obj-$(CONFIG_VIRTIO_IOMMU) += virtio-iommu.o
 obj-$(CONFIG_IOMMU_SVA_LIB) += iommu-sva-lib.o io-pgfault.o
 obj-$(CONFIG_SPRD_IOMMU) += sprd-iommu.o
+obj-$(CONFIG_APPLE_DART) += apple-dart.o
drivers/iommu/apple-dart.c new file +923 -0

923 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
maintainer Applied
#4 lore raw
In reply to Sven Peter
Quoted 4 lines · Sven Peter

Sven Peter (3):
iommu/io-pgtable: Add DART pagetable format
dt-bindings: iommu: add DART iommu bindings
iommu/dart: Add DART iommu driver

Applied, thanks. This driver now lives in the apple/dart branch of the IOMMU tree.