Skip to contentSkip to conversation

arm64: tegra: Prevent early SMMU faults

PATCH Superseded Thierry Reding 4 participants 27 messages Apr 20, 2021 – Jun 2, 2021

Subsystems (6)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

v2 cover arm64: tegra: Prevent early SMMU faults

From: Thierry Reding <[email protected]>

Hi,

this is a set of patches that is the result of earlier discussions regarding early identity mappings that are needed to avoid SMMU faults during early boot.

The goal here is to avoid early identity mappings altogether and instead postpone the need for the identity mappings to when devices are attached to the SMMU. This works by making the SMMU driver coordinate with the memory controller driver on when to start enforcing SMMU translations. This makes Tegra behave in a more standard way and pushes the code to deal with the Tegra-specific programming into the NVIDIA SMMU implementation.

Compared to the original version of these patches, I've split the preparatory work into a separate patch series because it became very large and will be mostly uninteresting for this audience.

Patch 1 provides a mechanism to program SID overrides at runtime. Patch 2 updates the ARM SMMU device tree bindings to include the Tegra186 compatible string as suggested by Robin during review.

Patches 3 and 4 create the fundamentals in the SMMU driver to support this and also make this functionality available on Tegra186. Patch 5 hooks the ARM SMMU up to the memory controller so that the memory client stream ID overrides can be programmed at the right time.

Patch 6 extends this mechanism to Tegra186 and patches 7-9 enable all of this through device tree updates. Patch 10 is included here to show how SMMU will be enabled for display controllers. However, it cannot be applied yet because the code to create identity mappings for potentially live framebuffers hasn't been merged yet.

The end result is that various peripherals will have SMMU enabled, while the display controllers will keep using passthrough, as initially set up by firmware. Once the device tree bindings have been accepted and the SMMU driver has been updated to create identity mappings for the display controllers, they can be hooked up to the SMMU and the code in this series will automatically program the SID overrides to enable SMMU translations at the right time.

Note that the series creates a compile time dependency between the memory controller and IOMMU trees. If it helps I can provide a branch for each tree, modelling the dependency, once the series has been reviewed.

Changes in v2:
- split off the preparatory work into a separate series (that needs to be applied first)
- address review comments by Robin

Thierry

Thierry Reding (10):
memory: tegra: Implement SID override programming
dt-bindings: arm-smmu: Add Tegra186 compatible string iommu/arm-smmu: Implement ->probe_finalize() iommu/arm-smmu: tegra: Detect number of instances at runtime iommu/arm-smmu: tegra: Implement SID override programming iommu/arm-smmu: Use Tegra implementation on Tegra186
arm64: tegra: Use correct compatible string for Tegra186 SMMU
arm64: tegra: Hook up memory controller to SMMU on Tegra186
arm64: tegra: Enable SMMU support on Tegra194
arm64: tegra: Enable SMMU support for display on Tegra194

 .../devicetree/bindings/iommu/arm,smmu.yaml   |  11 +-
 arch/arm64/boot/dts/nvidia/tegra186.dtsi      |   4 +-
 arch/arm64/boot/dts/nvidia/tegra194.dtsi      | 166 ++++++++++++++++++
 drivers/iommu/arm/arm-smmu/arm-smmu-impl.c    |   3 +-
 drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c  |  90 ++++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c         |  13 ++
 drivers/iommu/arm/arm-smmu/arm-smmu.h         |   1 +
 drivers/memory/tegra/mc.c                     |   9 +
 drivers/memory/tegra/tegra186.c               |  72 ++++++++
 include/soc/tegra/mc.h                        |   3 +
 10 files changed, 349 insertions(+), 23 deletions(-)
Signature · 1 line
2.30.2
#1 lore raw

PATCH v2 1/10 memory: tegra: Implement SID override programming

From: Thierry Reding <[email protected]>

Instead of programming all SID overrides during early boot, perform the operation on-demand after the SMMU translations have been set up for a device. This reuses data from device tree to match memory clients for a device and programs the SID specified in device tree, which corresponds to the SID used for the SMMU context banks for the device.

---
 drivers/memory/tegra/mc.c       |  9 +++++
 drivers/memory/tegra/tegra186.c | 72 +++++++++++++++++++++++++++++++++
 include/soc/tegra/mc.h          |  3 ++
 3 files changed, 84 insertions(+)
drivers/memory/tegra/mc.c +9 -0
index c854639cf30c..bace5ecfe770 100644
@@ -97,6 +97,15 @@ struct tegra_mc *devm_tegra_memory_controller_get(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(devm_tegra_memory_controller_get);
 
+int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev)
+{
+	if (mc->soc->ops && mc->soc->ops->probe_device)
+		return mc->soc->ops->probe_device(mc, dev);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(tegra_mc_probe_device);
+
 static int tegra_mc_block_dma_common(struct tegra_mc *mc,
 				     const struct tegra_mc_reset *rst)
 {
drivers/memory/tegra/tegra186.c +72 -0

94 lines not on this page · raw message

include/soc/tegra/mc.h +3 -0
index 1387747d574b..bbad6330008b 100644
@@ -176,6 +176,7 @@ struct tegra_mc_ops {
 	int (*suspend)(struct tegra_mc *mc);
 	int (*resume)(struct tegra_mc *mc);
 	irqreturn_t (*handle_irq)(int irq, void *data);
+	int (*probe_device)(struct tegra_mc *mc, struct device *dev);
 };
 
 struct tegra_mc_soc {
@@ -240,4 +241,6 @@ devm_tegra_memory_controller_get(struct device *dev)
 }
 #endif
 
+int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev);
+
 #endif /* __SOC_TEGRA_MC_H__ */
Signature · 1 line
2.30.2
#2 lore raw

Re: [PATCH v2 01/10] memory: tegra: Implement SID override programming

Quoted 93 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Instead of programming all SID overrides during early boot, perform the operation on-demand after the SMMU translations have been set up for a device. This reuses data from device tree to match memory clients for a device and programs the SID specified in device tree, which corresponds to the SID used for the SMMU context banks for the device.

---
 drivers/memory/tegra/mc.c       |  9 +++++
 drivers/memory/tegra/tegra186.c | 72 +++++++++++++++++++++++++++++++++
 include/soc/tegra/mc.h          |  3 ++
 3 files changed, 84 insertions(+)
drivers/memory/tegra/mc.c +9 -0
index c854639cf30c..bace5ecfe770 100644
@@ -97,6 +97,15 @@ struct tegra_mc *devm_tegra_memory_controller_get(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(devm_tegra_memory_controller_get);
 
+int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev)
+{
+	if (mc->soc->ops && mc->soc->ops->probe_device)
+		return mc->soc->ops->probe_device(mc, dev);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(tegra_mc_probe_device);
+
 static int tegra_mc_block_dma_common(struct tegra_mc *mc,
 				     const struct tegra_mc_reset *rst)
 {
drivers/memory/tegra/tegra186.c +36 -0
index 1f87915ccd62..e65eac5764d4 100644
@@ -4,6 +4,7 @@
  */
 
 #include <linux/io.h>
+#include <linux/iommu.h>
 #include <linux/module.h>
 #include <linux/mod_devicetable.h>
 #include <linux/of_device.h>
@@ -15,6 +16,10 @@
 #include <dt-bindings/memory/tegra186-mc.h>
 #endif
 
+#define MC_SID_STREAMID_OVERRIDE_MASK GENMASK(7, 0)
+#define MC_SID_STREAMID_SECURITY_WRITE_ACCESS_DISABLED BIT(16)
+#define MC_SID_STREAMID_SECURITY_OVERRIDE BIT(8)
+
 static void tegra186_mc_program_sid(struct tegra_mc *mc)
 {
 	unsigned int i;
@@ -66,10 +71,77 @@ static int tegra186_mc_resume(struct tegra_mc *mc)
 	return 0;
 }
 
+static void tegra186_mc_client_sid_override(struct tegra_mc *mc,
+					    const struct tegra_mc_client *client,
+					    unsigned int sid)
+{
+	u32 value, old;
+
+	value = readl(mc->regs + client->regs.sid.security);
+	if ((value & MC_SID_STREAMID_SECURITY_OVERRIDE) == 0) {
+		/*
+		 * If the secure firmware has locked this down the override
+		 * for this memory client, there's nothing we can do here.
+		 */
+		if (value & MC_SID_STREAMID_SECURITY_WRITE_ACCESS_DISABLED)
+			return;
+
+		/*
+		 * Otherwise, try to set the override itself. Typically the
+		 * secure firmware will never have set this configuration.
+		 * Instead, it will either have disabled write access to
+		 * this field, or it will already have set an explicit
+		 * override itself.
+		 */
+		WARN_ON((value & MC_SID_STREAMID_SECURITY_OVERRIDE) == 0);
+
+		value |= MC_SID_STREAMID_SECURITY_OVERRIDE;
+		writel(value, mc->regs + client->regs.sid.security);
+	}
+
+	value = readl(mc->regs + client->regs.sid.override);
+	old = value & MC_SID_STREAMID_OVERRIDE_MASK;
+

drivers/memory/tegra/tegra186.c static int tegra186_mc_resume(struct tegra_mc *mc) · line 114

Quoted 10 lines
drivers/memory/tegra/tegra186.c +10 -0
+	if (old != sid) {
+		dev_dbg(mc->dev, "overriding SID %x for %s with %x\n", old,
+			client->name, sid);
+		writel(sid, mc->regs + client->regs.sid.override);
+	}
+}
+
+static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev)
+{
+#if IS_ENABLED(CONFIG_IOMMU_API)

Is this part really build-time dependent? I don't see here any uses of IOMMU specific fields, so maybe this should be runtime choice based on enabled interconnect devices?

Quoted 46 lines
drivers/memory/tegra/tegra186.c +26 -0
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct of_phandle_args args;
+	unsigned int i, index = 0;
+
+	while (!of_parse_phandle_with_args(dev->of_node, "interconnects", "#interconnect-cells",
+					   index, &args)) {
+		if (args.np == mc->dev->of_node && args.args_count != 0) {
+			for (i = 0; i < mc->soc->num_clients; i++) {
+				const struct tegra_mc_client *client = &mc->soc->clients[i];
+
+				if (client->id == args.args[0]) {
+					u32 sid = fwspec->ids[0] & MC_SID_STREAMID_OVERRIDE_MASK;
+
+					tegra186_mc_client_sid_override(mc, client, sid);
+				}
+			}
+		}
+
+		index++;
+	}
+#endif
+
+	return 0;
+}
+
 const struct tegra_mc_ops tegra186_mc_ops = {
 	.probe = tegra186_mc_probe,
 	.remove = tegra186_mc_remove,
 	.resume = tegra186_mc_resume,
+	.probe_device = tegra186_mc_probe_device,
 };
 
 #if defined(CONFIG_ARCH_TEGRA_186_SOC)
include/soc/tegra/mc.h +1 -0
index 1387747d574b..bbad6330008b 100644
@@ -176,6 +176,7 @@ struct tegra_mc_ops {
 	int (*suspend)(struct tegra_mc *mc);
 	int (*resume)(struct tegra_mc *mc);
 	irqreturn_t (*handle_irq)(int irq, void *data);
+	int (*probe_device)(struct tegra_mc *mc, struct device *dev);
 };
 
 struct tegra_mc_soc {

include/soc/tegra/mc.h devm_tegra_memory_controller_get(struct device *dev) · line 245

Quoted 5 lines
include/soc/tegra/mc.h +2 -0
@@ -240,4 +241,6 @@ devm_tegra_memory_controller_get(struct device *dev)
 }
 #endif
 
+int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev);
+

What about !CONFIG_TEGRA_MC? I think arm-smmmu will fail.

include/soc/tegra/mc.h devm_tegra_memory_controller_get(struct device *dev) · line 246

Quoted 2 lines
include/soc/tegra/mc.h +0 -0
 #endif /* __SOC_TEGRA_MC_H__ */

Best regards,
Krzysztof

#3 lore raw

Re: [PATCH v2 01/10] memory: tegra: Implement SID override programming

Quoted 100 lines · Krzysztof Kozlowski
Quoted 99 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Instead of programming all SID overrides during early boot, perform the operation on-demand after the SMMU translations have been set up for a device. This reuses data from device tree to match memory clients for a device and programs the SID specified in device tree, which corresponds to the SID used for the SMMU context banks for the device.

---
 drivers/memory/tegra/mc.c       |  9 +++++
 drivers/memory/tegra/tegra186.c | 72 +++++++++++++++++++++++++++++++++
 include/soc/tegra/mc.h          |  3 ++
 3 files changed, 84 insertions(+)
drivers/memory/tegra/mc.c +9 -0
index c854639cf30c..bace5ecfe770 100644
@@ -97,6 +97,15 @@ struct tegra_mc *devm_tegra_memory_controller_get(struct device *dev)
 }
 EXPORT_SYMBOL_GPL(devm_tegra_memory_controller_get);
 
+int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev)
+{
+	if (mc->soc->ops && mc->soc->ops->probe_device)
+		return mc->soc->ops->probe_device(mc, dev);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(tegra_mc_probe_device);
+
 static int tegra_mc_block_dma_common(struct tegra_mc *mc,
 				     const struct tegra_mc_reset *rst)
 {
drivers/memory/tegra/tegra186.c +42 -0
index 1f87915ccd62..e65eac5764d4 100644
@@ -4,6 +4,7 @@
  */
 
 #include <linux/io.h>
+#include <linux/iommu.h>
 #include <linux/module.h>
 #include <linux/mod_devicetable.h>
 #include <linux/of_device.h>
@@ -15,6 +16,10 @@
 #include <dt-bindings/memory/tegra186-mc.h>
 #endif
 
+#define MC_SID_STREAMID_OVERRIDE_MASK GENMASK(7, 0)
+#define MC_SID_STREAMID_SECURITY_WRITE_ACCESS_DISABLED BIT(16)
+#define MC_SID_STREAMID_SECURITY_OVERRIDE BIT(8)
+
 static void tegra186_mc_program_sid(struct tegra_mc *mc)
 {
 	unsigned int i;
@@ -66,10 +71,77 @@ static int tegra186_mc_resume(struct tegra_mc *mc)
 	return 0;
 }
 
+static void tegra186_mc_client_sid_override(struct tegra_mc *mc,
+					    const struct tegra_mc_client *client,
+					    unsigned int sid)
+{
+	u32 value, old;
+
+	value = readl(mc->regs + client->regs.sid.security);
+	if ((value & MC_SID_STREAMID_SECURITY_OVERRIDE) == 0) {
+		/*
+		 * If the secure firmware has locked this down the override
+		 * for this memory client, there's nothing we can do here.
+		 */
+		if (value & MC_SID_STREAMID_SECURITY_WRITE_ACCESS_DISABLED)
+			return;
+
+		/*
+		 * Otherwise, try to set the override itself. Typically the
+		 * secure firmware will never have set this configuration.
+		 * Instead, it will either have disabled write access to
+		 * this field, or it will already have set an explicit
+		 * override itself.
+		 */
+		WARN_ON((value & MC_SID_STREAMID_SECURITY_OVERRIDE) == 0);
+
+		value |= MC_SID_STREAMID_SECURITY_OVERRIDE;
+		writel(value, mc->regs + client->regs.sid.security);
+	}
+
+	value = readl(mc->regs + client->regs.sid.override);
+	old = value & MC_SID_STREAMID_OVERRIDE_MASK;
+
+	if (old != sid) {
+		dev_dbg(mc->dev, "overriding SID %x for %s with %x\n", old,
+			client->name, sid);
+		writel(sid, mc->regs + client->regs.sid.override);
+	}
+}
Quoted 8 lines

drivers/memory/tegra/tegra186.c static int tegra186_mc_resume(struct tegra_mc *mc) · line 114

Quoted 4 lines
drivers/memory/tegra/tegra186.c +4 -0
+
+static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev)
+{
+#if IS_ENABLED(CONFIG_IOMMU_API)

Is this part really build-time dependent? I don't see here any uses of IOMMU specific fields, so maybe this should be runtime choice based on enabled interconnect devices?

Unfortunately it is. struct iommu_fwspec is an empty structure for !CONFIG_IOMMU_API, so the code below that tries to access fwspec->ids fails for !CONFIG_IOMMU_API configurations if we don't protect this with the preprocessor guard.

Quoted 50 lines
Quoted 50 lines
drivers/memory/tegra/tegra186.c +26 -0
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct of_phandle_args args;
+	unsigned int i, index = 0;
+
+	while (!of_parse_phandle_with_args(dev->of_node, "interconnects", "#interconnect-cells",
+					   index, &args)) {
+		if (args.np == mc->dev->of_node && args.args_count != 0) {
+			for (i = 0; i < mc->soc->num_clients; i++) {
+				const struct tegra_mc_client *client = &mc->soc->clients[i];
+
+				if (client->id == args.args[0]) {
+					u32 sid = fwspec->ids[0] & MC_SID_STREAMID_OVERRIDE_MASK;
+
+					tegra186_mc_client_sid_override(mc, client, sid);
+				}
+			}
+		}
+
+		index++;
+	}
+#endif
+
+	return 0;
+}
+
 const struct tegra_mc_ops tegra186_mc_ops = {
 	.probe = tegra186_mc_probe,
 	.remove = tegra186_mc_remove,
 	.resume = tegra186_mc_resume,
+	.probe_device = tegra186_mc_probe_device,
 };
 
 #if defined(CONFIG_ARCH_TEGRA_186_SOC)
include/soc/tegra/mc.h +1 -0
index 1387747d574b..bbad6330008b 100644
@@ -176,6 +176,7 @@ struct tegra_mc_ops {
 	int (*suspend)(struct tegra_mc *mc);
 	int (*resume)(struct tegra_mc *mc);
 	irqreturn_t (*handle_irq)(int irq, void *data);
+	int (*probe_device)(struct tegra_mc *mc, struct device *dev);
 };
 
 struct tegra_mc_soc {
@@ -240,4 +241,6 @@ devm_tegra_memory_controller_get(struct device *dev)
 }
 #endif
 
Quoted 4 lines

include/soc/tegra/mc.h devm_tegra_memory_controller_get(struct device *dev) · line 245

Quoted 2 lines
include/soc/tegra/mc.h +2 -0
+int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev);
+

What about !CONFIG_TEGRA_MC? I think arm-smmmu will fail.

I think it doesn't fail because for !CONFIG_TEGRA_MC it basically throws away most of nvidia_smmu_impl_init() already because ERR_PTR(-ENODEV) is returned by devm_tegra_memory_controller_get() and so it unconditionally fails early on already.

I say I /think/ that happens because I can't reproduce a build failure even if I manually tweak the .config such that ARM_SMMU is enabled and TEGRA_MC is disabled. But I can't say I fully understand why it's working, because, yes, the symbol definitely doesn't exist. But again, if the compiler is clever enough to figure out that that function can't be called anyway and doesn't even want it, why bother making it more complicated than it has to be?

Thierry

#4 lore raw

Re: [PATCH v2 01/10] memory: tegra: Implement SID override programming

Quoted 1 line · Thierry Reding

On Mon, Apr 26, 2021 at 10:28:43AM +0200, Krzysztof Kozlowski wrote:

(...)

Quoted 19 lines
Quoted 10 lines
Quoted 10 lines
drivers/memory/tegra/tegra186.c +10 -0
+
+	value = readl(mc->regs + client->regs.sid.override);
+	old = value & MC_SID_STREAMID_OVERRIDE_MASK;
+
+	if (old != sid) {
+		dev_dbg(mc->dev, "overriding SID %x for %s with %x\n", old,
+			client->name, sid);
+		writel(sid, mc->regs + client->regs.sid.override);
+	}
+}
Quoted 8 lines

drivers/memory/tegra/tegra186.c static int tegra186_mc_resume(struct tegra_mc *mc) · line 114

Quoted 4 lines
drivers/memory/tegra/tegra186.c +4 -0
+
+static int tegra186_mc_probe_device(struct tegra_mc *mc, struct device *dev)
+{
+#if IS_ENABLED(CONFIG_IOMMU_API)

Is this part really build-time dependent? I don't see here any uses of IOMMU specific fields, so maybe this should be runtime choice based on enabled interconnect devices?

Quoted 4 lines

Unfortunately it is. struct iommu_fwspec is an empty structure for !CONFIG_IOMMU_API, so the code below that tries to access fwspec->ids fails for !CONFIG_IOMMU_API configurations if we don't protect this with the preprocessor guard.

OK, thanks.

Quoted 61 lines
Quoted 51 lines
Quoted 51 lines
drivers/memory/tegra/tegra186.c +26 -0
+	struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
+	struct of_phandle_args args;
+	unsigned int i, index = 0;
+
+	while (!of_parse_phandle_with_args(dev->of_node, "interconnects", "#interconnect-cells",
+					   index, &args)) {
+		if (args.np == mc->dev->of_node && args.args_count != 0) {
+			for (i = 0; i < mc->soc->num_clients; i++) {
+				const struct tegra_mc_client *client = &mc->soc->clients[i];
+
+				if (client->id == args.args[0]) {
+					u32 sid = fwspec->ids[0] & MC_SID_STREAMID_OVERRIDE_MASK;
+
+					tegra186_mc_client_sid_override(mc, client, sid);
+				}
+			}
+		}
+
+		index++;
+	}
+#endif
+
+	return 0;
+}
+
 const struct tegra_mc_ops tegra186_mc_ops = {
 	.probe = tegra186_mc_probe,
 	.remove = tegra186_mc_remove,
 	.resume = tegra186_mc_resume,
+	.probe_device = tegra186_mc_probe_device,
 };
 
 #if defined(CONFIG_ARCH_TEGRA_186_SOC)
include/soc/tegra/mc.h +1 -0
index 1387747d574b..bbad6330008b 100644
@@ -176,6 +176,7 @@ struct tegra_mc_ops {
 	int (*suspend)(struct tegra_mc *mc);
 	int (*resume)(struct tegra_mc *mc);
 	irqreturn_t (*handle_irq)(int irq, void *data);
+	int (*probe_device)(struct tegra_mc *mc, struct device *dev);
 };
 
 struct tegra_mc_soc {
@@ -240,4 +241,6 @@ devm_tegra_memory_controller_get(struct device *dev)
 }
 #endif
 
Quoted 4 lines

include/soc/tegra/mc.h devm_tegra_memory_controller_get(struct device *dev) · line 245

Quoted 2 lines
include/soc/tegra/mc.h +2 -0
+int tegra_mc_probe_device(struct tegra_mc *mc, struct device *dev);
+

What about !CONFIG_TEGRA_MC? I think arm-smmmu will fail.

I think it doesn't fail because for !CONFIG_TEGRA_MC it basically throws away most of nvidia_smmu_impl_init() already because ERR_PTR(-ENODEV) is returned by devm_tegra_memory_controller_get() and so it unconditionally fails early on already.

Quoted 7 lines

I say I /think/ that happens because I can't reproduce a build failure even if I manually tweak the .config such that ARM_SMMU is enabled and TEGRA_MC is disabled. But I can't say I fully understand why it's working, because, yes, the symbol definitely doesn't exist. But again, if the compiler is clever enough to figure out that that function can't be called anyway and doesn't even want it, why bother making it more complicated than it has to be?

Since you tested that case, it's fine.

Best regards,
Krzysztof

#5 lore raw

PATCH v2 2/10 dt-bindings: arm-smmu: Add Tegra186 compatible string

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

The ARM SMMU instantiations found on Tegra186 and later need inter- operation with the memory controller in order to correctly program stream ID overrides.

Furthermore, on Tegra194 multiple instances of the SMMU can gang up to achieve higher throughput. In order to do this, they have to be programmed identically so that the memory controller can interleave memory accesses between them.

Add the Tegra186 compatible string to make sure the interoperation with the memory controller can be enabled on that SoC generation.

---
 Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +9 -2
index 9d27aa5111d4..1181b590db71 100644
@@ -54,8 +54,14 @@ properties:
           - const: arm,mmu-500
       - description: NVIDIA SoCs that program two ARM MMU-500s identically
         items:
+      - description: NVIDIA SoCs that require memory controller interaction
+          and may program multiple ARM MMU-500s identically with the memory
+          controller interleaving translations between multiple instances
+          for improved performance.
+        items:
           - enum:
-              - nvidia,tegra194-smmu
+              - const: nvidia,tegra194-smmu
+              - const: nvidia,tegra186-smmu
           - const: nvidia,smmu-500
       - items:
           - const: arm,mmu-500
@@ -165,10 +171,11 @@ allOf:
           contains:
             enum:
               - nvidia,tegra194-smmu
+              - nvidia,tegra186-smmu
     then:
       properties:
         reg:
-          minItems: 2
+          minItems: 1
           maxItems: 2
     else:
       properties:
Signature · 1 line
2.30.2
#6 lore raw

PATCH v2 3/10 iommu/arm-smmu: Implement ->probe_finalize()

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

Implement a ->probe_finalize() callback that can be used by vendor implementations to perform extra programming necessary after devices have been attached to the SMMU.

---
Changes in v2:
-remove unnecessarily paranoid check

---
 drivers/iommu/arm/arm-smmu/arm-smmu.c | 13 +++++++++++++
 drivers/iommu/arm/arm-smmu/arm-smmu.h |  1 +
 2 files changed, 14 insertions(+)
drivers/iommu/arm/arm-smmu/arm-smmu.c +13 -0
index 6f72c4d208ca..d20ce4d57df2 100644
@@ -1450,6 +1450,18 @@ static void arm_smmu_release_device(struct device *dev)
 	iommu_fwspec_free(dev);
 }
 
+static void arm_smmu_probe_finalize(struct device *dev)
+{
+	struct arm_smmu_master_cfg *cfg;
+	struct arm_smmu_device *smmu;
+
+	cfg = dev_iommu_priv_get(dev);
+	smmu = cfg->smmu;
+
+	if (smmu->impl->probe_finalize)
+		smmu->impl->probe_finalize(smmu, dev);
+}
+
 static struct iommu_group *arm_smmu_device_group(struct device *dev)
 {
 	struct arm_smmu_master_cfg *cfg = dev_iommu_priv_get(dev);
@@ -1569,6 +1581,7 @@ static struct iommu_ops arm_smmu_ops = {
 	.iova_to_phys		= arm_smmu_iova_to_phys,
 	.probe_device		= arm_smmu_probe_device,
 	.release_device		= arm_smmu_release_device,
+	.probe_finalize		= arm_smmu_probe_finalize,
 	.device_group		= arm_smmu_device_group,
 	.enable_nesting		= arm_smmu_enable_nesting,
 	.set_pgtable_quirks	= arm_smmu_set_pgtable_quirks,
drivers/iommu/arm/arm-smmu/arm-smmu.h +1 -0
index c31a59d35c64..147c95e7c59c 100644
@@ -439,6 +439,7 @@ struct arm_smmu_impl {
 				  struct device *dev, int start);
 	void (*write_s2cr)(struct arm_smmu_device *smmu, int idx);
 	void (*write_sctlr)(struct arm_smmu_device *smmu, int idx, u32 reg);
+	void (*probe_finalize)(struct arm_smmu_device *smmu, struct device *dev);
 };
 
 #define INVALID_SMENDX			-1
Signature · 1 line
2.30.2
#7 lore raw

PATCH v2 4/10 iommu/arm-smmu: tegra: Detect number of instances at runtime

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

Parse the reg property in device tree and detect the number of instances represented by a device tree node. This is subsequently needed in order to support single-instance SMMUs with the Tegra implementation because additional programming is needed to properly configure the SID override registers in the memory controller.

---
Changes in v2:
- provide a separate implementation to simplify single instances

---
 drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c | 58 ++++++++++++++------
 1 file changed, 42 insertions(+), 16 deletions(-)
drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c +42 -16
index 29117444e5a0..0e547b29143d 100644
@@ -20,13 +20,19 @@
  * The third instance usage is through standard arm-smmu driver itself and
  * is out of scope of this implementation.
  */
-#define NUM_SMMU_INSTANCES 2
+#define MAX_SMMU_INSTANCES 2
 
 struct nvidia_smmu {
-	struct arm_smmu_device	smmu;
-	void __iomem		*bases[NUM_SMMU_INSTANCES];
+	struct arm_smmu_device smmu;
+	void __iomem *bases[MAX_SMMU_INSTANCES];
+	unsigned int num_instances;
 };
 
+static inline struct nvidia_smmu *to_nvidia_smmu(struct arm_smmu_device *smmu)
+{
+	return container_of(smmu, struct nvidia_smmu, smmu);
+}
+
 static inline void __iomem *nvidia_smmu_page(struct arm_smmu_device *smmu,
 					     unsigned int inst, int page)
 {
@@ -47,9 +53,10 @@ static u32 nvidia_smmu_read_reg(struct arm_smmu_device *smmu,
 static void nvidia_smmu_write_reg(struct arm_smmu_device *smmu,
 				  int page, int offset, u32 val)
 {
+	struct nvidia_smmu *nvidia = to_nvidia_smmu(smmu);
 	unsigned int i;
 
-	for (i = 0; i < NUM_SMMU_INSTANCES; i++) {
+	for (i = 0; i < nvidia->num_instances; i++) {
 		void __iomem *reg = nvidia_smmu_page(smmu, i, page) + offset;
 
 		writel_relaxed(val, reg);
@@ -67,9 +74,10 @@ static u64 nvidia_smmu_read_reg64(struct arm_smmu_device *smmu,
 static void nvidia_smmu_write_reg64(struct arm_smmu_device *smmu,
 				    int page, int offset, u64 val)
 {
+	struct nvidia_smmu *nvidia = to_nvidia_smmu(smmu);
 	unsigned int i;
 
-	for (i = 0; i < NUM_SMMU_INSTANCES; i++) {
+	for (i = 0; i < nvidia->num_instances; i++) {
 		void __iomem *reg = nvidia_smmu_page(smmu, i, page) + offset;
 
 		writeq_relaxed(val, reg);
@@ -79,6 +87,7 @@ static void nvidia_smmu_write_reg64(struct arm_smmu_device *smmu,
 static void nvidia_smmu_tlb_sync(struct arm_smmu_device *smmu, int page,
 				 int sync, int status)
 {
+	struct nvidia_smmu *nvidia = to_nvidia_smmu(smmu);
 	unsigned int delay;
 
 	arm_smmu_writel(smmu, page, sync, 0);
@@ -90,7 +99,7 @@ static void nvidia_smmu_tlb_sync(struct arm_smmu_device *smmu, int page,
 			u32 val = 0;
 			unsigned int i;
 
-			for (i = 0; i < NUM_SMMU_INSTANCES; i++) {
+			for (i = 0; i < nvidia->num_instances; i++) {
 				void __iomem *reg;
 
 				reg = nvidia_smmu_page(smmu, i, page) + status;
@@ -112,9 +121,10 @@ static void nvidia_smmu_tlb_sync(struct arm_smmu_device *smmu, int page,
 
 static int nvidia_smmu_reset(struct arm_smmu_device *smmu)
 {
+	struct nvidia_smmu *nvidia = to_nvidia_smmu(smmu);
 	unsigned int i;
 
-	for (i = 0; i < NUM_SMMU_INSTANCES; i++) {
+	for (i = 0; i < nvidia->num_instances; i++) {
 		u32 val;
 		void __iomem *reg = nvidia_smmu_page(smmu, i, ARM_SMMU_GR0) +
 				    ARM_SMMU_GR0_sGFSR;
@@ -157,8 +167,9 @@ static irqreturn_t nvidia_smmu_global_fault(int irq, void *dev)
 	unsigned int inst;
 	irqreturn_t ret = IRQ_NONE;
 	struct arm_smmu_device *smmu = dev;
+	struct nvidia_smmu *nvidia = to_nvidia_smmu(smmu);
 
-	for (inst = 0; inst < NUM_SMMU_INSTANCES; inst++) {
+	for (inst = 0; inst < nvidia->num_instances; inst++) {
 		irqreturn_t irq_ret;
 
 		irq_ret = nvidia_smmu_global_fault_inst(irq, smmu, inst);
@@ -202,11 +213,13 @@ static irqreturn_t nvidia_smmu_context_fault(int irq, void *dev)
 	struct arm_smmu_device *smmu;
 	struct iommu_domain *domain = dev;
 	struct arm_smmu_domain *smmu_domain;
+	struct nvidia_smmu *nvidia;
 
 	smmu_domain = container_of(domain, struct arm_smmu_domain, domain);
 	smmu = smmu_domain->smmu;
+	nvidia = to_nvidia_smmu(smmu);
 
-	for (inst = 0; inst < NUM_SMMU_INSTANCES; inst++) {
+	for (inst = 0; inst < nvidia->num_instances; inst++) {
 		irqreturn_t irq_ret;
 
 		/*
@@ -235,12 +248,17 @@ static const struct arm_smmu_impl nvidia_smmu_impl = {
 	.context_fault = nvidia_smmu_context_fault,
 };
 
+static const struct arm_smmu_impl nvidia_smmu_single_impl = {
+	.probe_finalize = nvidia_smmu_probe_finalize,
+};
+
 struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu)
 {
 	struct resource *res;
 	struct device *dev = smmu->dev;
 	struct nvidia_smmu *nvidia_smmu;
 	struct platform_device *pdev = to_platform_device(dev);
+	unsigned int i;
 
 	nvidia_smmu = devm_krealloc(dev, smmu, sizeof(*nvidia_smmu), GFP_KERNEL);
 	if (!nvidia_smmu)
@@ -248,16 +266,24 @@ struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu)
 
 	/* Instance 0 is ioremapped by arm-smmu.c. */
 	nvidia_smmu->bases[0] = smmu->base;
+	nvidia_smmu->num_instances++;
 
-	res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
-	if (!res)
-		return ERR_PTR(-ENODEV);
+	for (i = 1; i < MAX_SMMU_INSTANCES; i++) {
+		res = platform_get_resource(pdev, IORESOURCE_MEM, i);
+		if (!res)
+			break;
 
-	nvidia_smmu->bases[1] = devm_ioremap_resource(dev, res);
-	if (IS_ERR(nvidia_smmu->bases[1]))
-		return ERR_CAST(nvidia_smmu->bases[1]);
+		nvidia_smmu->bases[i] = devm_ioremap_resource(dev, res);
+		if (IS_ERR(nvidia_smmu->bases[i]))
+			return ERR_CAST(nvidia_smmu->bases[i]);
+
+		nvidia_smmu->num_instances++;
+	}
 
-	nvidia_smmu->smmu.impl = &nvidia_smmu_impl;
+	if (nvidia_smmu->num_instances == 1)
+		nvidia_smmu->smmu.impl = &nvidia_smmu_single_impl;
+	else
+		nvidia_smmu->smmu.impl = &nvidia_smmu_impl;
 
 	return &nvidia_smmu->smmu;
 }
Signature · 1 line
2.30.2
reviewer
#8 lore raw

RE: [PATCH v2 04/10] iommu/arm-smmu: tegra: Detect number of instances at runtime

drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c static const struct arm_smmu_impl nvidia_smmu_impl = { · line 253

Quoted 3 lines
drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c +3 -0
+static const struct arm_smmu_impl nvidia_smmu_single_impl = {
+       .probe_finalize = nvidia_smmu_probe_finalize,
+};

nvidia_smmu_probe_finalize is used before it is defined. It is defined in patch 5.

-KR
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#9 lore raw

PATCH v2 5/10 iommu/arm-smmu: tegra: Implement SID override programming

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

The secure firmware keeps some SID override registers set as passthrough in order to allow devices such as the display controller to operate with no knowledge of SMMU translations until an operating system driver takes over. This is needed in order to seamlessly transition from the firmware framebuffer to the OS framebuffer.

Upon successfully attaching a device to the SMMU and in the process creating identity mappings for memory regions that are being accessed, the Tegra implementation will call into the memory controller driver to program the override SIDs appropriately.

---
 drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c | 32 ++++++++++++++++++--
 1 file changed, 29 insertions(+), 3 deletions(-)
drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c +29 -3
index 0e547b29143d..01e9b50b10a1 100644
@@ -7,6 +7,8 @@
 #include <linux/platform_device.h>
 #include <linux/slab.h>
 
+#include <soc/tegra/mc.h>
+
 #include "arm-smmu.h"
 
 /*
@@ -15,10 +17,17 @@
  * interleaved IOVA accesses across them and translates accesses from
  * non-isochronous HW devices.
  * Third one is used for translating accesses from isochronous HW devices.
+ *
+ * In addition, the SMMU driver needs to coordinate with the memory controller
+ * driver to ensure that the right SID override is programmed for any given
+ * memory client. This is necessary to allow for use-case such as seamlessly
+ * handing over the display controller configuration from the firmware to the
+ * kernel.
+ *
  * This implementation supports programming of the two instances that must
- * be programmed identically.
- * The third instance usage is through standard arm-smmu driver itself and
- * is out of scope of this implementation.
+ * be programmed identically and takes care of invoking the memory controller
+ * driver for SID override programming after devices have been attached to an
+ * SMMU instance.
  */
 #define MAX_SMMU_INSTANCES 2
 
@@ -26,6 +35,7 @@ struct nvidia_smmu {
 	struct arm_smmu_device smmu;
 	void __iomem *bases[MAX_SMMU_INSTANCES];
 	unsigned int num_instances;
+	struct tegra_mc *mc;
 };
 
 static inline struct nvidia_smmu *to_nvidia_smmu(struct arm_smmu_device *smmu)
@@ -237,6 +247,17 @@ static irqreturn_t nvidia_smmu_context_fault(int irq, void *dev)
 	return ret;
 }
 
+static void nvidia_smmu_probe_finalize(struct arm_smmu_device *smmu, struct device *dev)
+{
+	struct nvidia_smmu *nvidia = to_nvidia_smmu(smmu);
+	int err;
+
+	err = tegra_mc_probe_device(nvidia->mc, dev);
+	if (err < 0)
+		dev_err(smmu->dev, "memory controller probe failed for %s: %d\n",
+			dev_name(dev), err);
+}
+
 static const struct arm_smmu_impl nvidia_smmu_impl = {
 	.read_reg = nvidia_smmu_read_reg,
 	.write_reg = nvidia_smmu_write_reg,
@@ -246,6 +267,7 @@ static const struct arm_smmu_impl nvidia_smmu_impl = {
 	.tlb_sync = nvidia_smmu_tlb_sync,
 	.global_fault = nvidia_smmu_global_fault,
 	.context_fault = nvidia_smmu_context_fault,
+	.probe_finalize = nvidia_smmu_probe_finalize,
 };
 
 static const struct arm_smmu_impl nvidia_smmu_single_impl = {
@@ -264,6 +286,10 @@ struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu)
 	if (!nvidia_smmu)
 		return ERR_PTR(-ENOMEM);
 
+	nvidia_smmu->mc = devm_tegra_memory_controller_get(dev);
+	if (IS_ERR(nvidia_smmu->mc))
+		return ERR_CAST(nvidia_smmu->mc);
+
 	/* Instance 0 is ioremapped by arm-smmu.c. */
 	nvidia_smmu->bases[0] = smmu->base;
 	nvidia_smmu->num_instances++;
Signature · 1 line
2.30.2
#10 lore raw

PATCH v2 6/10 iommu/arm-smmu: Use Tegra implementation on Tegra186

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

Tegra186 requires the same SID override programming as Tegra194 in order to seamlessly transition from the firmware framebuffer to the Linux framebuffer, so the Tegra implementation needs to be used on Tegra186 devices as well.

---
 drivers/iommu/arm/arm-smmu/arm-smmu-impl.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/iommu/arm/arm-smmu/arm-smmu-impl.c +2 -1
index 136872e77195..9f465e146799 100644
@@ -211,7 +211,8 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)
 	if (of_property_read_bool(np, "calxeda,smmu-secure-config-access"))
 		smmu->impl = &calxeda_impl;
 
-	if (of_device_is_compatible(np, "nvidia,tegra194-smmu"))
+	if (of_device_is_compatible(np, "nvidia,tegra194-smmu") ||
+	    of_device_is_compatible(np, "nvidia,tegra186-smmu"))
 		return nvidia_smmu_impl_init(smmu);
 
 	smmu = qcom_smmu_impl_init(smmu);
Signature · 1 line
2.30.2
#11 lore raw

PATCH v2 7/10 arm64: tegra: Use correct compatible string for Tegra186 SMMU

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

The SMMU found on Tegra186 requires interoperation with the memory controller in order to program stream ID overrides. The generic ARM SMMU 500 compatible is therefore inaccurate. Replace it with a more correct, SoC-specific compatible string.

---
 arch/arm64/boot/dts/nvidia/tegra186.dtsi | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/boot/dts/nvidia/tegra186.dtsi +1 -1
index 9f75bbf00cf7..a173f40256ae 100644
@@ -1082,7 +1082,7 @@ pci@3,0 {
 	};
 
 	smmu: iommu@12000000 {
-		compatible = "arm,mmu-500";
+		compatible = "nvidia,tegra186-smmu", "nvidia,smmu-500";
 		reg = <0 0x12000000 0 0x800000>;
 		interrupts = <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
 			     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
Signature · 1 line
2.30.2
#12 lore raw

PATCH v2 8/10 arm64: tegra: Hook up memory controller to SMMU on Tegra186

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

On Tegra186 and later, the memory controller needs to be programmed in coordination with any of the ARM SMMU instances to configure the stream ID used for each memory client.

To support this, add a phandle reference to the memory controller to the SMMU device tree node.

---
 arch/arm64/boot/dts/nvidia/tegra186.dtsi | 2 ++
 1 file changed, 2 insertions(+)
arch/arm64/boot/dts/nvidia/tegra186.dtsi +2 -0
index a173f40256ae..d02f6bf3e2ca 100644
@@ -1152,6 +1152,8 @@ smmu: iommu@12000000 {
 		stream-match-mask = <0x7f80>;
 		#global-interrupts = <1>;
 		#iommu-cells = <1>;
+
+		nvidia,memory-controller = <&mc>;
 	};
 
 	host1x@13e00000 {
Signature · 1 line
2.30.2
#13 lore raw

PATCH v2 9/10 arm64: tegra: Enable SMMU support on Tegra194

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

Add the device tree node for the dual-SMMU found on Tegra194 and hook up peripherals such as host1x, BPMP, HDA, SDMMC, EQOS and VIC.

---
 arch/arm64/boot/dts/nvidia/tegra194.dtsi | 86 ++++++++++++++++++++++++
 1 file changed, 86 insertions(+)
arch/arm64/boot/dts/nvidia/tegra194.dtsi +86 -0
index 4d37ee0ea4d1..6ed296e27158 100644
@@ -62,6 +62,7 @@ ethernet@2490000 {
 			interconnects = <&mc TEGRA194_MEMORY_CLIENT_EQOSR &emc>,
 					<&mc TEGRA194_MEMORY_CLIENT_EQOSW &emc>;
 			interconnect-names = "dma-mem", "write";
+			iommus = <&smmu TEGRA194_SID_EQOS>;
 			status = "disabled";
 
 			snps,write-requests = <1>;
@@ -733,6 +734,7 @@ sdmmc1: mmc@3400000 {
 			interconnects = <&mc TEGRA194_MEMORY_CLIENT_SDMMCRA &emc>,
 					<&mc TEGRA194_MEMORY_CLIENT_SDMMCWA &emc>;
 			interconnect-names = "dma-mem", "write";
+			iommus = <&smmu TEGRA194_SID_SDMMC1>;
 			nvidia,pad-autocal-pull-up-offset-3v3-timeout =
 									<0x07>;
 			nvidia,pad-autocal-pull-down-offset-3v3-timeout =
@@ -759,6 +761,7 @@ sdmmc3: mmc@3440000 {
 			interconnects = <&mc TEGRA194_MEMORY_CLIENT_SDMMCR &emc>,
 					<&mc TEGRA194_MEMORY_CLIENT_SDMMCW &emc>;
 			interconnect-names = "dma-mem", "write";
+			iommus = <&smmu TEGRA194_SID_SDMMC3>;
 			nvidia,pad-autocal-pull-up-offset-1v8 = <0x00>;
 			nvidia,pad-autocal-pull-down-offset-1v8 = <0x7a>;
 			nvidia,pad-autocal-pull-up-offset-3v3-timeout = <0x07>;
@@ -790,6 +793,7 @@ sdmmc4: mmc@3460000 {
 			interconnects = <&mc TEGRA194_MEMORY_CLIENT_SDMMCRAB &emc>,
 					<&mc TEGRA194_MEMORY_CLIENT_SDMMCWAB &emc>;
 			interconnect-names = "dma-mem", "write";
+			iommus = <&smmu TEGRA194_SID_SDMMC4>;
 			nvidia,pad-autocal-pull-up-offset-hs400 = <0x00>;
 			nvidia,pad-autocal-pull-down-offset-hs400 = <0x00>;
 			nvidia,pad-autocal-pull-up-offset-1v8-timeout = <0x0a>;
@@ -821,6 +825,7 @@ hda@3510000 {
 			interconnects = <&mc TEGRA194_MEMORY_CLIENT_HDAR &emc>,
 					<&mc TEGRA194_MEMORY_CLIENT_HDAW &emc>;
 			interconnect-names = "dma-mem", "write";
+			iommus = <&smmu TEGRA194_SID_HDA>;
 			status = "disabled";
 		};
 
@@ -1300,6 +1305,84 @@ pmc: pmc@c360000 {
 			interrupt-controller;
 		};
 
+		smmu: iommu@12000000 {
+			compatible = "nvidia,tegra194-smmu", "nvidia,smmu-500";
+			reg = <0x12000000 0x800000>,
+			      <0x11000000 0x800000>;
+			interrupts = <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 232 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 170 IRQ_TYPE_LEVEL_HIGH>;
+			stream-match-mask = <0x7f80>;
+			#global-interrupts = <2>;
+			#iommu-cells = <1>;
+
+			nvidia,memory-controller = <&mc>;
+			status = "okay";
+		};
+
 		host1x@13e00000 {
 			compatible = "nvidia,tegra194-host1x";
 			reg = <0x13e00000 0x10000>,
@@ -1319,6 +1402,7 @@ host1x@13e00000 {
 			ranges = <0x15000000 0x15000000 0x01000000>;
 			interconnects = <&mc TEGRA194_MEMORY_CLIENT_HOST1XDMAR &emc>;
 			interconnect-names = "dma-mem";
+			iommus = <&smmu TEGRA194_SID_HOST1X>;
 
 			display-hub@15200000 {
 				compatible = "nvidia,tegra194-display";
@@ -1430,6 +1514,7 @@ vic@15340000 {
 				interconnects = <&mc TEGRA194_MEMORY_CLIENT_VICSRD &emc>,
 						<&mc TEGRA194_MEMORY_CLIENT_VICSWR &emc>;
 				interconnect-names = "dma-mem", "write";
+				iommus = <&smmu TEGRA194_SID_VIC>;
 			};
 
 			dpaux0: dpaux@155c0000 {
@@ -2136,6 +2221,7 @@ bpmp: bpmp {
 				<&mc TEGRA194_MEMORY_CLIENT_BPMPDMAR &emc>,
 				<&mc TEGRA194_MEMORY_CLIENT_BPMPDMAW &emc>;
 		interconnect-names = "read", "write", "dma-mem", "dma-write";
+		iommus = <&smmu TEGRA194_SID_BPMP>;
 
 		bpmp_i2c: i2c {
 			compatible = "nvidia,tegra186-bpmp-i2c";
Signature · 1 line
2.30.2
#14 lore raw

PATCH v2 10/10 arm64: tegra: Enable SMMU support for display on Tegra194

In reply to Thierry Reding

From: Thierry Reding <[email protected]>

The display controllers are attached to a separate ARM SMMU instance that is dedicated to servicing isochronous memory clients. Add this ISO instance of the ARM SMMU to device tree and attach all four display controllers to it.

---
 arch/arm64/boot/dts/nvidia/tegra194.dtsi | 80 ++++++++++++++++++++++++
 1 file changed, 80 insertions(+)
arch/arm64/boot/dts/nvidia/tegra194.dtsi +80 -0
index 6ed296e27158..00f8248f216e 100644
@@ -1305,6 +1305,82 @@ pmc: pmc@c360000 {
 			interrupt-controller;
 		};
 
+		smmu_iso: iommu@10000000 {
+			compatible = "nvidia,tegra194-smmu", "nvidia,smmu-500";
+			reg = <0x10000000 0x800000>;
+			interrupts = <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>,
+				     <GIC_SPI 240 IRQ_TYPE_LEVEL_HIGH>;
+			stream-match-mask = <0x7f80>;
+			#global-interrupts = <1>;
+			#iommu-cells = <1>;
+
+			nvidia,memory-controller = <&mc>;
+			status = "okay";
+		};
+
 		smmu: iommu@12000000 {
 			compatible = "nvidia,tegra194-smmu", "nvidia,smmu-500";
 			reg = <0x12000000 0x800000>,
@@ -1441,6 +1517,7 @@ display@15200000 {
 					interconnects = <&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR &emc>,
 							<&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR1 &emc>;
 					interconnect-names = "dma-mem", "read-1";
+					iommus = <&smmu_iso TEGRA194_SID_NVDISPLAY>;
 
 					nvidia,outputs = <&sor0 &sor1 &sor2 &sor3>;
 					nvidia,head = <0>;
@@ -1459,6 +1536,7 @@ display@15210000 {
 					interconnects = <&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR &emc>,
 							<&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR1 &emc>;
 					interconnect-names = "dma-mem", "read-1";
+					iommus = <&smmu_iso TEGRA194_SID_NVDISPLAY>;
 
 					nvidia,outputs = <&sor0 &sor1 &sor2 &sor3>;
 					nvidia,head = <1>;
@@ -1477,6 +1555,7 @@ display@15220000 {
 					interconnects = <&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR &emc>,
 							<&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR1 &emc>;
 					interconnect-names = "dma-mem", "read-1";
+					iommus = <&smmu_iso TEGRA194_SID_NVDISPLAY>;
 
 					nvidia,outputs = <&sor0 &sor1 &sor2 &sor3>;
 					nvidia,head = <2>;
@@ -1495,6 +1574,7 @@ display@15230000 {
 					interconnects = <&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR &emc>,
 							<&mc TEGRA194_MEMORY_CLIENT_NVDISPLAYR1 &emc>;
 					interconnect-names = "dma-mem", "read-1";
+					iommus = <&smmu_iso TEGRA194_SID_NVDISPLAY>;
 
 					nvidia,outputs = <&sor0 &sor1 &sor2 &sor3>;
 					nvidia,head = <3>;
Signature · 1 line
2.30.2
#15 lore raw
In reply to Thierry Reding
Quoted 67 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Hi,

this is a set of patches that is the result of earlier discussions regarding early identity mappings that are needed to avoid SMMU faults during early boot.

The goal here is to avoid early identity mappings altogether and instead postpone the need for the identity mappings to when devices are attached to the SMMU. This works by making the SMMU driver coordinate with the memory controller driver on when to start enforcing SMMU translations. This makes Tegra behave in a more standard way and pushes the code to deal with the Tegra-specific programming into the NVIDIA SMMU implementation.

Compared to the original version of these patches, I've split the preparatory work into a separate patch series because it became very large and will be mostly uninteresting for this audience.

Patch 1 provides a mechanism to program SID overrides at runtime. Patch 2 updates the ARM SMMU device tree bindings to include the Tegra186 compatible string as suggested by Robin during review.

Patches 3 and 4 create the fundamentals in the SMMU driver to support this and also make this functionality available on Tegra186. Patch 5 hooks the ARM SMMU up to the memory controller so that the memory client stream ID overrides can be programmed at the right time.

Patch 6 extends this mechanism to Tegra186 and patches 7-9 enable all of this through device tree updates. Patch 10 is included here to show how SMMU will be enabled for display controllers. However, it cannot be applied yet because the code to create identity mappings for potentially live framebuffers hasn't been merged yet.

The end result is that various peripherals will have SMMU enabled, while the display controllers will keep using passthrough, as initially set up by firmware. Once the device tree bindings have been accepted and the SMMU driver has been updated to create identity mappings for the display controllers, they can be hooked up to the SMMU and the code in this series will automatically program the SID overrides to enable SMMU translations at the right time.

Note that the series creates a compile time dependency between the memory controller and IOMMU trees. If it helps I can provide a branch for each tree, modelling the dependency, once the series has been reviewed.

Changes in v2:
- split off the preparatory work into a separate series (that needs to be applied first)
- address review comments by Robin

Thierry

Thierry Reding (10):
memory: tegra: Implement SID override programming
dt-bindings: arm-smmu: Add Tegra186 compatible string iommu/arm-smmu: Implement ->probe_finalize() iommu/arm-smmu: tegra: Detect number of instances at runtime iommu/arm-smmu: tegra: Implement SID override programming iommu/arm-smmu: Use Tegra implementation on Tegra186
arm64: tegra: Use correct compatible string for Tegra186 SMMU
arm64: tegra: Hook up memory controller to SMMU on Tegra186
arm64: tegra: Enable SMMU support on Tegra194
arm64: tegra: Enable SMMU support for display on Tegra194

Quoted 11 lines
 .../devicetree/bindings/iommu/arm,smmu.yaml   |  11 +-
 arch/arm64/boot/dts/nvidia/tegra186.dtsi      |   4 +-
 arch/arm64/boot/dts/nvidia/tegra194.dtsi      | 166 ++++++++++++++++++
 drivers/iommu/arm/arm-smmu/arm-smmu-impl.c    |   3 +-
 drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c  |  90 ++++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c         |  13 ++
 drivers/iommu/arm/arm-smmu/arm-smmu.h         |   1 +
 drivers/memory/tegra/mc.c                     |   9 +
 drivers/memory/tegra/tegra186.c               |  72 ++++++++
 include/soc/tegra/mc.h                        |   3 +
 10 files changed, 349 insertions(+), 23 deletions(-)

Will, Robin,

do you have any more comments on the ARM SMMU bits of this series? If not, can you guys provide an Acked-by so that Krzysztof can pick this (modulo the DT patches) up into the memory-controller tree for v5.14?

I'll send out a v3 with the bisectibilitiy fix that Krishna pointed out.

Thanks,
Thierry

maintainer
#16 lore raw
Quoted 80 lines · Thierry Reding
Quoted 67 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Hi,

this is a set of patches that is the result of earlier discussions regarding early identity mappings that are needed to avoid SMMU faults during early boot.

The goal here is to avoid early identity mappings altogether and instead postpone the need for the identity mappings to when devices are attached to the SMMU. This works by making the SMMU driver coordinate with the memory controller driver on when to start enforcing SMMU translations. This makes Tegra behave in a more standard way and pushes the code to deal with the Tegra-specific programming into the NVIDIA SMMU implementation.

Compared to the original version of these patches, I've split the preparatory work into a separate patch series because it became very large and will be mostly uninteresting for this audience.

Patch 1 provides a mechanism to program SID overrides at runtime. Patch 2 updates the ARM SMMU device tree bindings to include the Tegra186 compatible string as suggested by Robin during review.

Patches 3 and 4 create the fundamentals in the SMMU driver to support this and also make this functionality available on Tegra186. Patch 5 hooks the ARM SMMU up to the memory controller so that the memory client stream ID overrides can be programmed at the right time.

Patch 6 extends this mechanism to Tegra186 and patches 7-9 enable all of this through device tree updates. Patch 10 is included here to show how SMMU will be enabled for display controllers. However, it cannot be applied yet because the code to create identity mappings for potentially live framebuffers hasn't been merged yet.

The end result is that various peripherals will have SMMU enabled, while the display controllers will keep using passthrough, as initially set up by firmware. Once the device tree bindings have been accepted and the SMMU driver has been updated to create identity mappings for the display controllers, they can be hooked up to the SMMU and the code in this series will automatically program the SID overrides to enable SMMU translations at the right time.

Note that the series creates a compile time dependency between the memory controller and IOMMU trees. If it helps I can provide a branch for each tree, modelling the dependency, once the series has been reviewed.

Changes in v2:
- split off the preparatory work into a separate series (that needs to be applied first)
- address review comments by Robin

Thierry

Thierry Reding (10):
memory: tegra: Implement SID override programming
dt-bindings: arm-smmu: Add Tegra186 compatible string iommu/arm-smmu: Implement ->probe_finalize() iommu/arm-smmu: tegra: Detect number of instances at runtime iommu/arm-smmu: tegra: Implement SID override programming iommu/arm-smmu: Use Tegra implementation on Tegra186
arm64: tegra: Use correct compatible string for Tegra186 SMMU
arm64: tegra: Hook up memory controller to SMMU on Tegra186
arm64: tegra: Enable SMMU support on Tegra194
arm64: tegra: Enable SMMU support for display on Tegra194

Quoted 11 lines
 .../devicetree/bindings/iommu/arm,smmu.yaml   |  11 +-
 arch/arm64/boot/dts/nvidia/tegra186.dtsi      |   4 +-
 arch/arm64/boot/dts/nvidia/tegra194.dtsi      | 166 ++++++++++++++++++
 drivers/iommu/arm/arm-smmu/arm-smmu-impl.c    |   3 +-
 drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c  |  90 ++++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c         |  13 ++
 drivers/iommu/arm/arm-smmu/arm-smmu.h         |   1 +
 drivers/memory/tegra/mc.c                     |   9 +
 drivers/memory/tegra/tegra186.c               |  72 ++++++++
 include/soc/tegra/mc.h                        |   3 +
 10 files changed, 349 insertions(+), 23 deletions(-)
Quoted 8 lines

Will, Robin,

do you have any more comments on the ARM SMMU bits of this series? If not, can you guys provide an Acked-by so that Krzysztof can pick this (modulo the DT patches) up into the memory-controller tree for v5.14?

I'll send out a v3 with the bisectibilitiy fix that Krishna pointed out.

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Will

#17 lore raw
Quoted 87 lines · Will Deacon
Quoted 86 lines · Thierry Reding
Quoted 67 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Hi,

this is a set of patches that is the result of earlier discussions regarding early identity mappings that are needed to avoid SMMU faults during early boot.

The goal here is to avoid early identity mappings altogether and instead postpone the need for the identity mappings to when devices are attached to the SMMU. This works by making the SMMU driver coordinate with the memory controller driver on when to start enforcing SMMU translations. This makes Tegra behave in a more standard way and pushes the code to deal with the Tegra-specific programming into the NVIDIA SMMU implementation.

Compared to the original version of these patches, I've split the preparatory work into a separate patch series because it became very large and will be mostly uninteresting for this audience.

Patch 1 provides a mechanism to program SID overrides at runtime. Patch 2 updates the ARM SMMU device tree bindings to include the Tegra186 compatible string as suggested by Robin during review.

Patches 3 and 4 create the fundamentals in the SMMU driver to support this and also make this functionality available on Tegra186. Patch 5 hooks the ARM SMMU up to the memory controller so that the memory client stream ID overrides can be programmed at the right time.

Patch 6 extends this mechanism to Tegra186 and patches 7-9 enable all of this through device tree updates. Patch 10 is included here to show how SMMU will be enabled for display controllers. However, it cannot be applied yet because the code to create identity mappings for potentially live framebuffers hasn't been merged yet.

The end result is that various peripherals will have SMMU enabled, while the display controllers will keep using passthrough, as initially set up by firmware. Once the device tree bindings have been accepted and the SMMU driver has been updated to create identity mappings for the display controllers, they can be hooked up to the SMMU and the code in this series will automatically program the SID overrides to enable SMMU translations at the right time.

Note that the series creates a compile time dependency between the memory controller and IOMMU trees. If it helps I can provide a branch for each tree, modelling the dependency, once the series has been reviewed.

Changes in v2:
- split off the preparatory work into a separate series (that needs to be applied first)
- address review comments by Robin

Thierry

Thierry Reding (10):
memory: tegra: Implement SID override programming
dt-bindings: arm-smmu: Add Tegra186 compatible string iommu/arm-smmu: Implement ->probe_finalize() iommu/arm-smmu: tegra: Detect number of instances at runtime iommu/arm-smmu: tegra: Implement SID override programming iommu/arm-smmu: Use Tegra implementation on Tegra186
arm64: tegra: Use correct compatible string for Tegra186 SMMU
arm64: tegra: Hook up memory controller to SMMU on Tegra186
arm64: tegra: Enable SMMU support on Tegra194
arm64: tegra: Enable SMMU support for display on Tegra194

Quoted 11 lines
 .../devicetree/bindings/iommu/arm,smmu.yaml   |  11 +-
 arch/arm64/boot/dts/nvidia/tegra186.dtsi      |   4 +-
 arch/arm64/boot/dts/nvidia/tegra194.dtsi      | 166 ++++++++++++++++++
 drivers/iommu/arm/arm-smmu/arm-smmu-impl.c    |   3 +-
 drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c  |  90 ++++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c         |  13 ++
 drivers/iommu/arm/arm-smmu/arm-smmu.h         |   1 +
 drivers/memory/tegra/mc.c                     |   9 +
 drivers/memory/tegra/tegra186.c               |  72 ++++++++
 include/soc/tegra/mc.h                        |   3 +
 10 files changed, 349 insertions(+), 23 deletions(-)

Will, Robin,

do you have any more comments on the ARM SMMU bits of this series? If not, can you guys provide an Acked-by so that Krzysztof can pick this (modulo the DT patches) up into the memory-controller tree for v5.14?

Quoted 5 lines
Quoted 2 lines

I'll send out a v3 with the bisectibilitiy fix that Krishna pointed out.

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

Alternatively I can set this all up on stable branches and send out pull requests for both you and Krzysztof to merge. Or if this is all too complicated and you'd just prefer to ack the patches I could also take this through ARM SoC via the Tegra tree.

Thierry

#18 lore raw
Quoted 91 lines · Thierry Reding
Quoted 90 lines · Will Deacon
Quoted 88 lines · Thierry Reding
Quoted 67 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Hi,

this is a set of patches that is the result of earlier discussions regarding early identity mappings that are needed to avoid SMMU faults during early boot.

The goal here is to avoid early identity mappings altogether and instead postpone the need for the identity mappings to when devices are attached to the SMMU. This works by making the SMMU driver coordinate with the memory controller driver on when to start enforcing SMMU translations. This makes Tegra behave in a more standard way and pushes the code to deal with the Tegra-specific programming into the NVIDIA SMMU implementation.

Compared to the original version of these patches, I've split the preparatory work into a separate patch series because it became very large and will be mostly uninteresting for this audience.

Patch 1 provides a mechanism to program SID overrides at runtime. Patch 2 updates the ARM SMMU device tree bindings to include the Tegra186 compatible string as suggested by Robin during review.

Patches 3 and 4 create the fundamentals in the SMMU driver to support this and also make this functionality available on Tegra186. Patch 5 hooks the ARM SMMU up to the memory controller so that the memory client stream ID overrides can be programmed at the right time.

Patch 6 extends this mechanism to Tegra186 and patches 7-9 enable all of this through device tree updates. Patch 10 is included here to show how SMMU will be enabled for display controllers. However, it cannot be applied yet because the code to create identity mappings for potentially live framebuffers hasn't been merged yet.

The end result is that various peripherals will have SMMU enabled, while the display controllers will keep using passthrough, as initially set up by firmware. Once the device tree bindings have been accepted and the SMMU driver has been updated to create identity mappings for the display controllers, they can be hooked up to the SMMU and the code in this series will automatically program the SID overrides to enable SMMU translations at the right time.

Note that the series creates a compile time dependency between the memory controller and IOMMU trees. If it helps I can provide a branch for each tree, modelling the dependency, once the series has been reviewed.

Changes in v2:
- split off the preparatory work into a separate series (that needs to be applied first)
- address review comments by Robin

Thierry

Thierry Reding (10):
memory: tegra: Implement SID override programming
dt-bindings: arm-smmu: Add Tegra186 compatible string iommu/arm-smmu: Implement ->probe_finalize() iommu/arm-smmu: tegra: Detect number of instances at runtime iommu/arm-smmu: tegra: Implement SID override programming iommu/arm-smmu: Use Tegra implementation on Tegra186
arm64: tegra: Use correct compatible string for Tegra186 SMMU
arm64: tegra: Hook up memory controller to SMMU on Tegra186
arm64: tegra: Enable SMMU support on Tegra194
arm64: tegra: Enable SMMU support for display on Tegra194

Quoted 11 lines
 .../devicetree/bindings/iommu/arm,smmu.yaml   |  11 +-
 arch/arm64/boot/dts/nvidia/tegra186.dtsi      |   4 +-
 arch/arm64/boot/dts/nvidia/tegra194.dtsi      | 166 ++++++++++++++++++
 drivers/iommu/arm/arm-smmu/arm-smmu-impl.c    |   3 +-
 drivers/iommu/arm/arm-smmu/arm-smmu-nvidia.c  |  90 ++++++++--
 drivers/iommu/arm/arm-smmu/arm-smmu.c         |  13 ++
 drivers/iommu/arm/arm-smmu/arm-smmu.h         |   1 +
 drivers/memory/tegra/mc.c                     |   9 +
 drivers/memory/tegra/tegra186.c               |  72 ++++++++
 include/soc/tegra/mc.h                        |   3 +
 10 files changed, 349 insertions(+), 23 deletions(-)

Will, Robin,

do you have any more comments on the ARM SMMU bits of this series? If not, can you guys provide an Acked-by so that Krzysztof can pick this (modulo the DT patches) up into the memory-controller tree for v5.14?

I'll send out a v3 with the bisectibilitiy fix that Krishna pointed out.

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

Quoted 8 lines

Alternatively I can set this all up on stable branches and send out pull requests for both you and Krzysztof to merge. Or if this is all too complicated and you'd just prefer to ack the patches I could also take this through ARM SoC via the Tegra tree.

Thierry

Best regards,
Krzysztof

#19 lore raw
Quoted 15 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

Quoted 7 lines

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Best regards,
Krzysztof
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#20 lore raw
Quoted 20 lines · Krzysztof Kozlowski
Quoted 19 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Quoted 6 lines
Quoted 3 lines

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Yes, that's the one. That series is fairly self-contained, but Dmitry's power management set has dependencies that pull in the regulator, clock and ARM SoC trees.

I did a test merge of the driver unification series with a branch that has Dmitry's patches and all the dependencies and there are no conflicts so that, fortunately, doesn't further complicates things.

Do you want me to send you a pull request with Dmitry's memory controller changes? You could then apply the unification series on top, which should allow this SMMU series to apply cleanly on top of that.

I can also carry all these changes in the Tegra tree and send a PR in a few days once this has seen a bit more testing in linux-next, which also makes sure it's got a bit more testing in our internal test farm.

Thierry

#21 lore raw
Quoted 32 lines · Thierry Reding
Quoted 20 lines · Krzysztof Kozlowski
Quoted 19 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Quoted 6 lines
Quoted 3 lines

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Yes, that's the one. That series is fairly self-contained, but Dmitry's power management set has dependencies that pull in the regulator, clock and ARM SoC trees.

Quoted 7 lines

I did a test merge of the driver unification series with a branch that has Dmitry's patches and all the dependencies and there are no conflicts so that, fortunately, doesn't further complicates things.

Do you want me to send you a pull request with Dmitry's memory controller changes? You could then apply the unification series on top, which should allow this SMMU series to apply cleanly on top of that.

Makes sense and it looks quite bulletproof for future changes. Let's do like this. I will apply your patch 1/10 from this v2 on top of it and driver unification later.

Quoted 4 lines

I can also carry all these changes in the Tegra tree and send a PR in a few days once this has seen a bit more testing in linux-next, which also makes sure it's got a bit more testing in our internal test farm.

Best regards,
Krzysztof

maintainer
#22 lore raw
Quoted 37 lines · Krzysztof Kozlowski
Quoted 36 lines · Thierry Reding
Quoted 20 lines · Krzysztof Kozlowski
Quoted 19 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Quoted 6 lines
Quoted 3 lines

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Yes, that's the one. That series is fairly self-contained, but Dmitry's power management set has dependencies that pull in the regulator, clock and ARM SoC trees.

I did a test merge of the driver unification series with a branch that has Dmitry's patches and all the dependencies and there are no conflicts so that, fortunately, doesn't further complicates things.

Quoted 7 lines
Quoted 3 lines

Do you want me to send you a pull request with Dmitry's memory controller changes? You could then apply the unification series on top, which should allow this SMMU series to apply cleanly on top of that.

Makes sense and it looks quite bulletproof for future changes. Let's do like this. I will apply your patch 1/10 from this v2 on top of it and driver unification later.

Okey doke. Thierry -- please let me know which patches I can queue. Having the SMMU driver changes in the IOMMU tree really helps in case of conflicts with other SMMU changes. As I say, I can put stuff on a separate branch for you if it helps.

Will

#23 lore raw
Quoted 42 lines · Will Deacon
Quoted 41 lines · Krzysztof Kozlowski
Quoted 39 lines · Thierry Reding
Quoted 20 lines · Krzysztof Kozlowski
Quoted 19 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Quoted 6 lines
Quoted 3 lines

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Yes, that's the one. That series is fairly self-contained, but Dmitry's power management set has dependencies that pull in the regulator, clock and ARM SoC trees.

I did a test merge of the driver unification series with a branch that has Dmitry's patches and all the dependencies and there are no conflicts so that, fortunately, doesn't further complicates things.

Do you want me to send you a pull request with Dmitry's memory controller changes? You could then apply the unification series on top, which should allow this SMMU series to apply cleanly on top of that.

Quoted 8 lines
Quoted 3 lines

Makes sense and it looks quite bulletproof for future changes. Let's do like this. I will apply your patch 1/10 from this v2 on top of it and driver unification later.

Okey doke. Thierry -- please let me know which patches I can queue. Having the SMMU driver changes in the IOMMU tree really helps in case of conflicts with other SMMU changes. As I say, I can put stuff on a separate branch for you if it helps.

Given that the SMMU patches have a build-time dependency on the first patch in the series, and the series depends on the unification series, I think Krzysztof would have to pull the memory controller branch that I have with Dmitry's work, apply the unification series on top and then take patch 1 of this series on top of that. That's the state that you'd have to pull into the SMMU tree to get the right dependencies.

The SMMU pieces are the leaf of the dependency tree, so technically no separate branch is needed, because I don't think anybody would have to pull from it. However, a separate branch might make it easier to back any of this work out if we have to.

Krzysztof, I do plan on sending out a v3 of the unification series to address the two cleanups that Dmitry and you have pointed out. After that I can send out v3 of this series to fix the ordering issue that Krishna had mentioned. After all of those are applied, would you be able to provide a stable branch for Will's SMMU tree?

Thierry

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#24 lore raw
Quoted 64 lines · Thierry Reding
Quoted 42 lines · Will Deacon
Quoted 41 lines · Krzysztof Kozlowski
Quoted 39 lines · Thierry Reding
Quoted 20 lines · Krzysztof Kozlowski
Quoted 19 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Quoted 6 lines
Quoted 3 lines

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Yes, that's the one. That series is fairly self-contained, but Dmitry's power management set has dependencies that pull in the regulator, clock and ARM SoC trees.

I did a test merge of the driver unification series with a branch that has Dmitry's patches and all the dependencies and there are no conflicts so that, fortunately, doesn't further complicates things.

Do you want me to send you a pull request with Dmitry's memory controller changes? You could then apply the unification series on top, which should allow this SMMU series to apply cleanly on top of that.

Quoted 8 lines
Quoted 3 lines

Makes sense and it looks quite bulletproof for future changes. Let's do like this. I will apply your patch 1/10 from this v2 on top of it and driver unification later.

Okey doke. Thierry -- please let me know which patches I can queue. Having the SMMU driver changes in the IOMMU tree really helps in case of conflicts with other SMMU changes. As I say, I can put stuff on a separate branch for you if it helps.

Given that the SMMU patches have a build-time dependency on the first patch in the series, and the series depends on the unification series, I think Krzysztof would have to pull the memory controller branch that I have with Dmitry's work, apply the unification series on top and then take patch 1 of this series on top of that. That's the state that you'd have to pull into the SMMU tree to get the right dependencies.

The SMMU pieces are the leaf of the dependency tree, so technically no separate branch is needed, because I don't think anybody would have to pull from it. However, a separate branch might make it easier to back any of this work out if we have to.

Quoted 5 lines

Krzysztof, I do plan on sending out a v3 of the unification series to address the two cleanups that Dmitry and you have pointed out. After that I can send out v3 of this series to fix the ordering issue that Krishna had mentioned. After all of those are applied, would you be able to provide a stable branch for Will's SMMU tree?

Yes, I will provide a stable branch/tag.

Best regards,
Krzysztof

#25 lore raw
In reply to Krzysztof Kozlowski
Quoted 37 lines · Krzysztof Kozlowski
Quoted 36 lines · Thierry Reding
Quoted 20 lines · Krzysztof Kozlowski
Quoted 19 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Quoted 6 lines
Quoted 3 lines

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Yes, that's the one. That series is fairly self-contained, but Dmitry's power management set has dependencies that pull in the regulator, clock and ARM SoC trees.

I did a test merge of the driver unification series with a branch that has Dmitry's patches and all the dependencies and there are no conflicts so that, fortunately, doesn't further complicates things.

Quoted 7 lines
Quoted 3 lines

Do you want me to send you a pull request with Dmitry's memory controller changes? You could then apply the unification series on top, which should allow this SMMU series to apply cleanly on top of that.

Makes sense and it looks quite bulletproof for future changes. Let's do like this. I will apply your patch 1/10 from this v2 on top of it and driver unification later.

The SMMU series here depends on the unification series, so the unification series needs to go first. It'd be a fair bit of work to reverse that because the ->probe_device() callback implemented by the first patch of this SMMU series is part of the tegra_mc_ops structure that's introduced in the unification series.

Thierry

#26 lore raw
Quoted 46 lines · Thierry Reding
Quoted 37 lines · Krzysztof Kozlowski
Quoted 36 lines · Thierry Reding
Quoted 20 lines · Krzysztof Kozlowski
Quoted 19 lines · Krzysztof Kozlowski
Quoted 5 lines · Thierry Reding
Quoted 4 lines · Will Deacon
Quoted 3 lines · Thierry Reding
Quoted 2 lines · Thierry Reding

From: Thierry Reding <[email protected]>

Quoted 8 lines
Quoted 2 lines

Probably best if I queue 3-6 on a separate branch once you send a v3, then Krzysztof can pull that in if he needs it.

Patch 5 has a build-time dependency on patch 1, so they need to go in together. The reason why I suggested Krzysztof pick these up is because there is a restructuring series that this depends on, which will go into Krzysztof's tree. So in order to pull in 3-6, you'd get a bunch of other and mostly unrelated stuff as well.

I missed that part... what other series are needed for this one? Except Dmitry's power management set I do not have anything in my sight for Tegras memory controllers.

Quoted 6 lines
Quoted 3 lines

Anyway, I can take the memory bits and provide a stable tag with these. Recently there was quite a lot work around Tegra memory controllers, so this makes especially sense if new patches appear.

OK, I think I have now the patchset you talked about - "memory: tegra:
Driver unification" v2, right?

Yes, that's the one. That series is fairly self-contained, but Dmitry's power management set has dependencies that pull in the regulator, clock and ARM SoC trees.

I did a test merge of the driver unification series with a branch that has Dmitry's patches and all the dependencies and there are no conflicts so that, fortunately, doesn't further complicates things.

Quoted 7 lines
Quoted 3 lines

Do you want me to send you a pull request with Dmitry's memory controller changes? You could then apply the unification series on top, which should allow this SMMU series to apply cleanly on top of that.

Makes sense and it looks quite bulletproof for future changes. Let's do like this. I will apply your patch 1/10 from this v2 on top of it and driver unification later.

Quoted 5 lines

The SMMU series here depends on the unification series, so the unification series needs to go first. It'd be a fair bit of work to reverse that because the ->probe_device() callback implemented by the first patch of this SMMU series is part of the tegra_mc_ops structure that's introduced in the unification series.

Right, you already wrote it in the first email in this thread, I just reversed words in my head... Then as you wrote - take Dmitry's changes and share them via pull to me. I'll put on top the unification series, then #1 from this SMMU series and finally I'll provide a pull request for Will so his SMMU can go on.

Best regards,
Krzysztof