Skip to contentSkip to conversation

Nvidia Arm SMMUv2 Implementation

PATCH Superseded Krishna Reddy 4 participants 11 messages Jun 29, 2020 – Jun 30, 2020

Subsystems (4)

v7 of 11 Superseded Latest is v11 →

lore
Share & tools
lore
#0 lore raw

v7 cover Nvidia Arm SMMUv2 Implementation

Changes in v7:
Incorporated the review feedback from Nicolin Chen, Robin Murphy and Thierry Reding. Rebased and validated patches on top of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git next

v6- https://lkml.org/lkml/2020/6/4/1018
v5 - https://lkml.org/lkml/2020/5/21/1114
v4 - https://lkml.org/lkml/2019/10/30/1054
v3 - https://lkml.org/lkml/2019/10/18/1601
v2 - https://lkml.org/lkml/2019/9/2/980
v1 - https://lkml.org/lkml/2019/8/29/1588

Krishna Reddy (3):
iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
dt-bindings: arm-smmu: Add binding for Tegra194 SMMU iommu/arm-smmu: Add global/context fault implementation hooks

 .../devicetree/bindings/iommu/arm,smmu.yaml   |   5 +
 MAINTAINERS                                   |   2 +
 drivers/iommu/Makefile                        |   2 +-
 drivers/iommu/arm-smmu-impl.c                 |   3 +
 drivers/iommu/arm-smmu-nvidia.c               | 294 ++++++++++++++++++
 drivers/iommu/arm-smmu.c                      |  17 +-
 drivers/iommu/arm-smmu.h                      |   4 +
 7 files changed, 324 insertions(+), 3 deletions(-)
 create mode 100644 drivers/iommu/arm-smmu-nvidia.c

base-commit: 48f0bcfb7aad2c6eb4c1e66476b58475aa14393e

Signature · 1 line
2.26.2
#1 lore raw

PATCH v7 1/3 iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage

NVIDIA's Tegra194 SoC uses two ARM MMU-500s together to interleave IOVA accesses across them.
Add NVIDIA implementation for dual ARM MMU-500s and add new compatible string for Tegra194 SoC SMMU topology.

---
 MAINTAINERS                     |   2 +
 drivers/iommu/Makefile          |   2 +-
 drivers/iommu/arm-smmu-impl.c   |   3 +
 drivers/iommu/arm-smmu-nvidia.c | 195 ++++++++++++++++++++++++++++++++
 drivers/iommu/arm-smmu.h        |   1 +
 5 files changed, 202 insertions(+), 1 deletion(-)
 create mode 100644 drivers/iommu/arm-smmu-nvidia.c
MAINTAINERS +2 -0
index 7b5ffd646c6b9..64c37dbdd4426 100644
@@ -16808,8 +16808,10 @@ F:	drivers/i2c/busses/i2c-tegra.c
 
 TEGRA IOMMU DRIVERS
 M:	Thierry Reding <[email protected]>
+R:	Krishna Reddy <[email protected]>
 L:	[email protected]
 S:	Supported
+F:	drivers/iommu/arm-smmu-nvidia.c
 F:	drivers/iommu/tegra*
 
 TEGRA KBC DRIVER
drivers/iommu/Makefile +1 -1
index 342190196dfb0..2b8203db73ec3 100644
@@ -15,7 +15,7 @@ obj-$(CONFIG_AMD_IOMMU) += amd/iommu.o amd/init.o amd/quirks.o
 obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += amd/debugfs.o
 obj-$(CONFIG_AMD_IOMMU_V2) += amd/iommu_v2.o
 obj-$(CONFIG_ARM_SMMU) += arm_smmu.o
-arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o
+arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-nvidia.o arm-smmu-qcom.o
 obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o
 obj-$(CONFIG_DMAR_TABLE) += intel/dmar.o
 obj-$(CONFIG_INTEL_IOMMU) += intel/iommu.o intel/pasid.o
drivers/iommu/arm-smmu-impl.c +3 -0
index c75b9d957b702..70f7318017617 100644
@@ -171,6 +171,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)
 	if (of_property_read_bool(np, "calxeda,smmu-secure-config-access"))
 		smmu->impl = &calxeda_impl;
 
+	if (of_device_is_compatible(smmu->dev->of_node, "nvidia,tegra194-smmu"))
+		return nvidia_smmu_impl_init(smmu);
+
 	if (of_device_is_compatible(np, "qcom,sdm845-smmu-500") ||
 	    of_device_is_compatible(np, "qcom,sc7180-smmu-500"))
 		return qcom_smmu_impl_init(smmu);
drivers/iommu/arm-smmu-nvidia.c new file +195 -0

195 lines not on this page · raw message

drivers/iommu/arm-smmu.h +1 -0
index d172c024be618..8cf1511ed9874 100644
@@ -450,6 +450,7 @@ static inline void arm_smmu_writeq(struct arm_smmu_device *smmu, int page,
 	arm_smmu_writeq((s), ARM_SMMU_CB((s), (n)), (o), (v))
 
 struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu);
+struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu);
 struct arm_smmu_device *qcom_smmu_impl_init(struct arm_smmu_device *smmu);
 
 int arm_mmu500_reset(struct arm_smmu_device *smmu);
Signature · 1 line
2.26.2
Reviewed-by on v7 1/3
#2 lore raw

Re: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage

Quoted 7 lines · Krishna Reddy

NVIDIA's Tegra194 SoC uses two ARM MMU-500s together to interleave IOVA accesses across them.
Add NVIDIA implementation for dual ARM MMU-500s and add new compatible string for Tegra194 SoC SMMU topology.

drivers/iommu/arm-smmu-nvidia.c line 54

Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c new file +10 -0
+static inline void __iomem *nvidia_smmu_page(struct arm_smmu_device *smmu,
+			       unsigned int inst, int page)
+{
+	struct nvidia_smmu *nvidia_smmu = to_nvidia_smmu(smmu);
+
+	if (!nvidia_smmu->bases[0])
+		nvidia_smmu->bases[0] = smmu->base;
+
+	return nvidia_smmu->bases[inst] + (page << smmu->pgshift);
+}

Not critical -- just a nit: why not put the bases[0] in init()?

Everything else looks good to me:

#3 lore raw

RE: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage

Quoted 6 lines

drivers/iommu/arm-smmu-nvidia.c

Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4 -0
+     if (!nvidia_smmu->bases[0])
+             nvidia_smmu->bases[0] = smmu->base;
+
+     return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }

Not critical -- just a nit: why not put the bases[0] in init()?

smmu->base is not available during nvidia_smmu_impl_init() call. It is set afterwards in arm-smmu.c. It can't be avoided without changing the devm_ioremap() and impl_init() call order in arm-smmu.c.

-KR

#4 lore raw

Re: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage

Quoted 9 lines · Krishna Reddy
Quoted 6 lines

drivers/iommu/arm-smmu-nvidia.c

Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4 -0
+     if (!nvidia_smmu->bases[0])
+             nvidia_smmu->bases[0] = smmu->base;
+
+     return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }

Not critical -- just a nit: why not put the bases[0] in init()?

smmu->base is not available during nvidia_smmu_impl_init() call. It is set afterwards in arm-smmu.c. It can't be avoided without changing the devm_ioremap() and impl_init() call order in arm-smmu.c.

I see...just checked arm_ssmu_impl_init().

maintainer
#5 lore raw

Re: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage

In reply to Krishna Reddy
Quoted 9 lines · Krishna Reddy
Quoted 6 lines

drivers/iommu/arm-smmu-nvidia.c

Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4 -0
+     if (!nvidia_smmu->bases[0])
+             nvidia_smmu->bases[0] = smmu->base;
+
+     return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }

Not critical -- just a nit: why not put the bases[0] in init()?

smmu->base is not available during nvidia_smmu_impl_init() call. It is set afterwards in arm-smmu.c. It can't be avoided without changing the devm_ioremap() and impl_init() call order in arm-smmu.c.

Why don't we move the call to devm_ioremap_resource() to before arm_smmu_impl_init() in arm_smmu_device_probe()? From a quick look I don't see why we cannot do this and seems better than what we are currently doing which is quite confusing and hard to understand.

Jon

Signature · 1 line
nvpublic
reviewer
#6 lore raw

Re: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage

Quoted 9 lines · Jon Hunter
Quoted 7 lines · Krishna Reddy
Quoted 6 lines

drivers/iommu/arm-smmu-nvidia.c

Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4 -0
+     if (!nvidia_smmu->bases[0])
+             nvidia_smmu->bases[0] = smmu->base;
+
+     return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }

Not critical -- just a nit: why not put the bases[0] in init()?

Quoted 8 lines
Quoted 2 lines

smmu->base is not available during nvidia_smmu_impl_init() call. It is set afterwards in arm-smmu.c. It can't be avoided without changing the devm_ioremap() and impl_init() call order in arm-smmu.c.

Why don't we move the call to devm_ioremap_resource() to before arm_smmu_impl_init() in arm_smmu_device_probe()? From a quick look I don't see why we cannot do this and seems better than what we are currently doing which is quite confusing and hard to understand.

Yeah, I don't see any problem with adding a patch to do that. impl_init() does need to happen before generic probe starts touching any registers, but it wouldn't have any business overriding the platform resources or anything that would affect the ioremap itself. Plus it's reasonable that some theoretical future impl_init() might want to check registers for, say, a particular hardware revision, so having smmmu->base mapped and valid at that point would be no bad thing.

Robin.

#7 lore raw

PATCH v7 2/3 dt-bindings: arm-smmu: Add binding for Tegra194 SMMU

In reply to Krishna Reddy

Add binding for NVIDIA's Tegra194 SoC SMMU topology that is based on ARM MMU-500.

---
 Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 5 +++++
 1 file changed, 5 insertions(+)
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +5 -0
index d7ceb4c34423b..5b2586ac715ed 100644
@@ -38,6 +38,11 @@ properties:
               - qcom,sc7180-smmu-500
               - qcom,sdm845-smmu-500
           - const: arm,mmu-500
+      - description: NVIDIA SoCs that use more than one "arm,mmu-500"
+        items:
+          - enum:
+              - nvdia,tegra194-smmu
+          - const: arm,mmu-500
       - items:
           - const: arm,mmu-500
           - const: arm,smmu-v2
Signature · 1 line
2.26.2
#8 lore raw

PATCH v7 3/3 iommu/arm-smmu: Add global/context fault implementation hooks

In reply to Krishna Reddy

Add global/context fault hooks to allow NVIDIA SMMU implementation handle faults across multiple SMMUs.

---
 drivers/iommu/arm-smmu-nvidia.c | 101 +++++++++++++++++++++++++++++++-
 drivers/iommu/arm-smmu.c        |  17 +++++-
 drivers/iommu/arm-smmu.h        |   3 +
 3 files changed, 118 insertions(+), 3 deletions(-)
drivers/iommu/arm-smmu-nvidia.c +100 -1
index b73c483fa3376..7276bb203ae79 100644
@@ -147,6 +147,102 @@ static int nvidia_smmu_reset(struct arm_smmu_device *smmu)
 	return 0;
 }
 
+static struct arm_smmu_domain *to_smmu_domain(struct iommu_domain *dom)
+{
+	return container_of(dom, struct arm_smmu_domain, domain);
+}
+
+static irqreturn_t nvidia_smmu_global_fault_inst(int irq,
+					       struct arm_smmu_device *smmu,
+					       int inst)
+{
+	u32 gfsr, gfsynr0, gfsynr1, gfsynr2;
+	void __iomem *gr0_base = nvidia_smmu_page(smmu, inst, 0);
+
+	gfsr = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSR);
+	gfsynr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR0);
+	gfsynr1 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR1);
+	gfsynr2 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR2);
+
+	if (!gfsr)
+		return IRQ_NONE;
+
+	dev_err_ratelimited(smmu->dev,
+		"Unexpected global fault, this could be serious\n");
+	dev_err_ratelimited(smmu->dev,
+		"\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n",
+		gfsr, gfsynr0, gfsynr1, gfsynr2);
+
+	writel_relaxed(gfsr, gr0_base + ARM_SMMU_GR0_sGFSR);
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t nvidia_smmu_global_fault(int irq, void *dev)
+{
+	int inst;
+	irqreturn_t irq_ret = IRQ_NONE;
+	struct arm_smmu_device *smmu = dev;
+	struct nvidia_smmu *nvidia_smmu = to_nvidia_smmu(smmu);
+
+	for (inst = 0; inst < nvidia_smmu->num_inst; inst++) {
+		irq_ret = nvidia_smmu_global_fault_inst(irq, smmu, inst);
+		if (irq_ret == IRQ_HANDLED)
+			return irq_ret;
+	}
+
+	return irq_ret;
+}
+
+static irqreturn_t nvidia_smmu_context_fault_bank(int irq,
+					    struct arm_smmu_device *smmu,
+					    int idx, int inst)
+{
+	u32 fsr, fsynr, cbfrsynra;
+	unsigned long iova;
+	void __iomem *gr1_base = nvidia_smmu_page(smmu, inst, 1);
+	void __iomem *cb_base = nvidia_smmu_page(smmu, inst, smmu->numpage + idx);
+
+	fsr = arm_smmu_cb_read(smmu, idx, ARM_SMMU_CB_FSR);
+	if (!(fsr & ARM_SMMU_FSR_FAULT))
+		return IRQ_NONE;
+
+	fsynr = readl_relaxed(cb_base + ARM_SMMU_CB_FSYNR0);
+	iova = readq_relaxed(cb_base + ARM_SMMU_CB_FAR);
+	cbfrsynra = readl_relaxed(gr1_base + ARM_SMMU_GR1_CBFRSYNRA(idx));
+
+	dev_err_ratelimited(smmu->dev,
+	"Unhandled context fault: fsr=0x%x, iova=0x%08lx, fsynr=0x%x, cbfrsynra=0x%x, cb=%d\n",
+			    fsr, iova, fsynr, cbfrsynra, idx);
+
+	writel_relaxed(fsr, cb_base + ARM_SMMU_CB_FSR);
+	return IRQ_HANDLED;
+}
+
+static irqreturn_t nvidia_smmu_context_fault(int irq, void *dev)
+{
+	int inst, idx;
+	irqreturn_t irq_ret = IRQ_NONE;
+	struct iommu_domain *domain = dev;
+	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+	struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+	for (inst = 0; inst < to_nvidia_smmu(smmu)->num_inst; inst++) {
+		/*
+		 * Interrupt line shared between all context faults.
+		 * Check for faults across all contexts.
+		 */
+		for (idx = 0; idx < smmu->num_context_banks; idx++) {
+			irq_ret = nvidia_smmu_context_fault_bank(irq, smmu,
+								 idx, inst);
+
+			if (irq_ret == IRQ_HANDLED)
+				return irq_ret;
+		}
+	}
+
+	return irq_ret;
+}
+
 static const struct arm_smmu_impl nvidia_smmu_impl = {
 	.read_reg = nvidia_smmu_read_reg,
 	.write_reg = nvidia_smmu_write_reg,
@@ -154,6 +250,8 @@ static const struct arm_smmu_impl nvidia_smmu_impl = {
 	.write_reg64 = nvidia_smmu_write_reg64,
 	.reset = nvidia_smmu_reset,
 	.tlb_sync = nvidia_smmu_tlb_sync,
+	.global_fault = nvidia_smmu_global_fault,
+	.context_fault = nvidia_smmu_context_fault,
 };
 
 struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu)
@@ -185,7 +283,8 @@ struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu)
 	}
 
 	nvidia_smmu->smmu.impl = &nvidia_smmu_impl;
-	/* Free the arm_smmu_device struct allocated in arm-smmu.c.
+	/*
+	 * Free the arm_smmu_device struct allocated in arm-smmu.c.
 	 * Once this function returns, arm-smmu.c would use arm_smmu_device
 	 * allocated as part of nvidia_smmu struct.
 	 */
drivers/iommu/arm-smmu.c +15 -2
index 243bc4cb2705b..3bb0aba15a356 100644
@@ -673,6 +673,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,
 	enum io_pgtable_fmt fmt;
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
 	struct arm_smmu_cfg *cfg = &smmu_domain->cfg;
+	irqreturn_t (*context_fault)(int irq, void *dev);
 
 	mutex_lock(&smmu_domain->init_mutex);
 	if (smmu_domain->smmu)
@@ -835,7 +836,13 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,
 	 * handler seeing a half-initialised domain state.
 	 */
 	irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx];
-	ret = devm_request_irq(smmu->dev, irq, arm_smmu_context_fault,
+
+	if (smmu->impl && smmu->impl->context_fault)
+		context_fault = smmu->impl->context_fault;
+	else
+		context_fault = arm_smmu_context_fault;
+
+	ret = devm_request_irq(smmu->dev, irq, context_fault,
 			       IRQF_SHARED, "arm-smmu-context-fault", domain);
 	if (ret < 0) {
 		dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",
@@ -2107,6 +2114,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 	struct arm_smmu_device *smmu;
 	struct device *dev = &pdev->dev;
 	int num_irqs, i, err;
+	irqreturn_t (*global_fault)(int irq, void *dev);
 
 	smmu = devm_kzalloc(dev, sizeof(*smmu), GFP_KERNEL);
 	if (!smmu) {
@@ -2193,9 +2201,14 @@ static int arm_smmu_device_probe(struct platform_device *pdev)
 		smmu->num_context_irqs = smmu->num_context_banks;
 	}
 
+	if (smmu->impl && smmu->impl->global_fault)
+		global_fault = smmu->impl->global_fault;
+	else
+		global_fault = arm_smmu_global_fault;
+
 	for (i = 0; i < smmu->num_global_irqs; ++i) {
 		err = devm_request_irq(smmu->dev, smmu->irqs[i],
-				       arm_smmu_global_fault,
+				       global_fault,
 				       IRQF_SHARED,
 				       "arm-smmu global fault",
 				       smmu);
drivers/iommu/arm-smmu.h +3 -0
index 8cf1511ed9874..8b330076ff2af 100644
@@ -18,6 +18,7 @@
 #include <linux/io-64-nonatomic-hi-lo.h>
 #include <linux/io-pgtable.h>
 #include <linux/iommu.h>
+#include <linux/irqreturn.h>
 #include <linux/mutex.h>
 #include <linux/spinlock.h>
 #include <linux/types.h>
@@ -387,6 +388,8 @@ struct arm_smmu_impl {
 	void (*tlb_sync)(struct arm_smmu_device *smmu, int page, int sync,
 			 int status);
 	int (*def_domain_type)(struct device *dev);
+	irqreturn_t (*global_fault)(int irq, void *dev);
+	irqreturn_t (*context_fault)(int irq, void *dev);
 };
 
 static inline void __iomem *arm_smmu_page(struct arm_smmu_device *smmu, int n)
Signature · 1 line
2.26.2
#9 lore raw

Re: [PATCH v7 3/3] iommu/arm-smmu: Add global/context fault implementation hooks

Quoted 9 lines · Krishna Reddy

Add global/context fault hooks to allow NVIDIA SMMU implementation handle faults across multiple SMMUs.

drivers/iommu/arm-smmu-nvidia.c +4 -0
+static irqreturn_t nvidia_smmu_global_fault_inst(int irq,
+					       struct arm_smmu_device *smmu,
+					       int inst)
+{

drivers/iommu/arm-smmu-nvidia.c static int nvidia_smmu_reset(struct arm_smmu_device *smmu) · line 168

Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c +10 -0
+	u32 gfsr, gfsynr0, gfsynr1, gfsynr2;
+	void __iomem *gr0_base = nvidia_smmu_page(smmu, inst, 0);
+
+	gfsr = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSR);
+	gfsynr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR0);
+	gfsynr1 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR1);
+	gfsynr2 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR2);
+
+	if (!gfsr)
+		return IRQ_NONE;

Could move this before gfsynr readings to save some readl() for !gfsr cases?

Quoted 3 lines
drivers/iommu/arm-smmu-nvidia.c +2 -0
+static irqreturn_t nvidia_smmu_context_fault_bank(int irq,
 
+	void __iomem *cb_base = nvidia_smmu_page(smmu, inst, smmu->numpage + idx);

[...]

Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c +1 -0
+	fsr = arm_smmu_cb_read(smmu, idx, ARM_SMMU_CB_FSR);

[...]

drivers/iommu/arm-smmu-nvidia.c static int nvidia_smmu_reset(struct arm_smmu_device *smmu) · line 217

Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c +1 -0
+	writel_relaxed(fsr, cb_base + ARM_SMMU_CB_FSR);

It reads FSR of the default inst (1st), but clears the FSR of corresponding inst -- just want to make sure that this is okay and intended.

drivers/iommu/arm-smmu-nvidia.c struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu) · line 290

Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c +2 -1
@@ -185,7 +283,8 @@ struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu)
 	}
 
 	nvidia_smmu->smmu.impl = &nvidia_smmu_impl;
-	/* Free the arm_smmu_device struct allocated in arm-smmu.c.
+	/*
+	 * Free the arm_smmu_device struct allocated in arm-smmu.c.
 	 * Once this function returns, arm-smmu.c would use arm_smmu_device
 	 * allocated as part of nvidia_smmu struct.
 	 */

Hmm, this coding style fix should be probably squashed into PATCH-1?

#10 lore raw

RE: [PATCH v7 3/3] iommu/arm-smmu: Add global/context fault implementation hooks

Quoted 4 lines
Quoted 4 lines
(quoted diff) +3 -0
+static irqreturn_t nvidia_smmu_context_fault_bank(int irq,
 
+     void __iomem *cb_base = nvidia_smmu_page(smmu, inst, 
+ smmu->numpage + idx);

[...]

Quoted 1 line
Quoted 1 line

+ fsr = arm_smmu_cb_read(smmu, idx, ARM_SMMU_CB_FSR);

[...]

Quoted 3 lines
Quoted 1 line

+ writel_relaxed(fsr, cb_base + ARM_SMMU_CB_FSR);

It reads FSR of the default inst (1st), but clears the FSR of corresponding inst -- just want to make sure that this is okay and intended.

FSR should be read from corresponding inst. Not from instance 0. Let me post updated patch.

-KR