Nvidia Arm SMMUv2 Implementation
Subsystems (4)
v7 cover Nvidia Arm SMMUv2 Implementation
Changes in v7:
Incorporated the review feedback from Nicolin Chen, Robin Murphy and Thierry Reding. Rebased and validated patches on top of git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git next
v6- https://lkml.org/lkml/2020/6/4/1018
v5 - https://lkml.org/lkml/2020/5/21/1114
v4 - https://lkml.org/lkml/2019/10/30/1054
v3 - https://lkml.org/lkml/2019/10/18/1601
v2 - https://lkml.org/lkml/2019/9/2/980
v1 - https://lkml.org/lkml/2019/8/29/1588
Krishna Reddy (3):
iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
dt-bindings: arm-smmu: Add binding for Tegra194 SMMU iommu/arm-smmu: Add global/context fault implementation hooks
.../devicetree/bindings/iommu/arm,smmu.yaml | 5 + MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 294 ++++++++++++++++++ drivers/iommu/arm-smmu.c | 17 +- drivers/iommu/arm-smmu.h | 4 + 7 files changed, 324 insertions(+), 3 deletions(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.c
base-commit: 48f0bcfb7aad2c6eb4c1e66476b58475aa14393e
Signature · 1 line
2.26.2
PATCH v7 1/3 iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
NVIDIA's Tegra194 SoC uses two ARM MMU-500s together to interleave IOVA accesses across them.
Add NVIDIA implementation for dual ARM MMU-500s and add new compatible string for Tegra194 SoC SMMU topology.
--- MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 195 ++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.h | 1 + 5 files changed, 202 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.c
MAINTAINERS +2 -0
@@ -16808,8 +16808,10 @@ F: drivers/i2c/busses/i2c-tegra.c TEGRA IOMMU DRIVERS M: Thierry Reding <[email protected]> +R: Krishna Reddy <[email protected]> L: [email protected] S: Supported +F: drivers/iommu/arm-smmu-nvidia.c F: drivers/iommu/tegra* TEGRA KBC DRIVER
drivers/iommu/Makefile +1 -1
@@ -15,7 +15,7 @@ obj-$(CONFIG_AMD_IOMMU) += amd/iommu.o amd/init.o amd/quirks.o obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += amd/debugfs.o obj-$(CONFIG_AMD_IOMMU_V2) += amd/iommu_v2.o obj-$(CONFIG_ARM_SMMU) += arm_smmu.o -arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o +arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-nvidia.o arm-smmu-qcom.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += intel/dmar.o obj-$(CONFIG_INTEL_IOMMU) += intel/iommu.o intel/pasid.o
drivers/iommu/arm-smmu-impl.c +3 -0
@@ -171,6 +171,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)if (of_property_read_bool (np,"calxeda,smmu-secure-config-access" )) smmu->impl = &calxeda_impl; +if (of_device_is_compatible (smmu->dev->of_node,"nvidia,tegra194-smmu" )) +return nvidia_smmu_impl_init (smmu); +if (of_device_is_compatible (np,"qcom,sdm845-smmu-500" ) ||of_device_is_compatible (np,"qcom,sc7180-smmu-500" ))return qcom_smmu_impl_init (smmu);
drivers/iommu/arm-smmu-nvidia.c new file +195 -0
195 lines not on this page · raw message
drivers/iommu/arm-smmu.h +1 -0
@@ -450,6 +450,7 @@ static inline void arm_smmu_writeq(struct arm_smmu_device *smmu, int page, arm_smmu_writeq((s), ARM_SMMU_CB((s), (n)), (o), (v))struct arm_smmu_device *arm_smmu_impl_init (struct arm_smmu_device *smmu); +struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu);struct arm_smmu_device *qcom_smmu_impl_init (struct arm_smmu_device *smmu);int arm_mmu500_reset (struct arm_smmu_device *smmu);
Signature · 1 line
2.26.2
Re: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
Quoted 7 lines · Krishna Reddy
NVIDIA's Tegra194 SoC uses two ARM MMU-500s together to interleave IOVA accesses across them.
Add NVIDIA implementation for dual ARM MMU-500s and add new compatible string for Tegra194 SoC SMMU topology.Signed-off-by Krishna Reddy <[email protected]>
drivers/iommu/arm-smmu-nvidia.c line 54
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c new file +10
-0+static inline void __iomem *nvidia_smmu_page (struct arm_smmu_device *smmu, +unsigned int inst,int page) +{ +struct nvidia_smmu *nvidia_smmu =to_nvidia_smmu (smmu); + +if (!nvidia_smmu->bases[0 ]) + nvidia_smmu->bases[0 ] = smmu->base; + +return nvidia_smmu->bases[inst] + (page << smmu->pgshift); +}
Not critical -- just a nit: why not put the bases[0] in init()?
Everything else looks good to me:
RE: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
Quoted 6 lines
drivers/iommu/arm-smmu-nvidia.c
Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4
-0+if (!nvidia_smmu->bases[0 ]) + nvidia_smmu->bases[0 ] = smmu->base; + +return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }Not critical -- just a nit: why not put the bases[0] in
init()?
smmu->base is not available during nvidia_smmu_impl_init() call. It is set afterwards in arm-smmu.c. It can't be avoided without changing the devm_ioremap() and impl_init() call order in arm-smmu.c.
-KR
Re: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
Quoted 9 lines · Krishna Reddy
Quoted 6 lines
drivers/iommu/arm-smmu-nvidia.c
Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4
-0+if (!nvidia_smmu->bases[0 ]) + nvidia_smmu->bases[0 ] = smmu->base; + +return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }Not critical -- just a nit: why not put the bases[0] in
init()?
smmu->baseis not available duringnvidia_smmu_impl_init()call. It is set afterwards inarm-smmu.c. It can't be avoided without changing thedevm_ioremap()andimpl_init()call order inarm-smmu.c.
I see...just checked arm_ssmu_impl_init().
Jon Hunter
maintainerRe: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
In reply to Krishna ReddyQuoted 9 lines · Krishna Reddy
Quoted 6 lines
drivers/iommu/arm-smmu-nvidia.c
Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4
-0+if (!nvidia_smmu->bases[0 ]) + nvidia_smmu->bases[0 ] = smmu->base; + +return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }Not critical -- just a nit: why not put the bases[0] in
init()?
smmu->baseis not available duringnvidia_smmu_impl_init()call. It is set afterwards inarm-smmu.c. It can't be avoided without changing thedevm_ioremap()andimpl_init()call order inarm-smmu.c.
Why don't we move the call to devm_ioremap_resource() to before arm_smmu_impl_init() in arm_smmu_device_probe()? From a quick look I don't see why we cannot do this and seems better than what we are currently doing which is quite confusing and hard to understand.
Jon
Signature · 1 line
nvpublic
Robin Murphy
reviewerRe: [PATCH v7 1/3] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
Quoted 9 lines · Jon Hunter
Quoted 7 lines · Krishna Reddy
Quoted 6 lines
drivers/iommu/arm-smmu-nvidia.c
Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4
-0+if (!nvidia_smmu->bases[0 ]) + nvidia_smmu->bases[0 ] = smmu->base; + +return nvidia_smmu->bases[inst] + (page << smmu->pgshift); }Not critical -- just a nit: why not put the bases[0] in
init()?
Quoted 8 lines
Quoted 2 lines
smmu->baseis not available duringnvidia_smmu_impl_init()call. It is set afterwards inarm-smmu.c. It can't be avoided without changing thedevm_ioremap()andimpl_init()call order inarm-smmu.c.Why don't we move the call to
devm_ioremap_resource()to beforearm_smmu_impl_init()inarm_smmu_device_probe()? From a quick look I don't see why we cannot do this and seems better than what we are currently doing which is quite confusing and hard to understand.
Yeah, I don't see any problem with adding a patch to do that. impl_init() does need to happen before generic probe starts touching any registers, but it wouldn't have any business overriding the platform resources or anything that would affect the ioremap itself. Plus it's reasonable that some theoretical future impl_init() might want to check registers for, say, a particular hardware revision, so having smmmu->base mapped and valid at that point would be no bad thing.
Robin.
PATCH v7 2/3 dt-bindings: arm-smmu: Add binding for Tegra194 SMMU
In reply to Krishna ReddyAdd binding for NVIDIA's Tegra194 SoC SMMU topology that is based on ARM MMU-500.
--- Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 5 +++++ 1 file changed, 5 insertions(+)
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +5 -0
@@ -38,6 +38,11 @@ properties: - qcom,sc7180-smmu-500 - qcom,sdm845-smmu-500 - const: arm,mmu-500 + - description: NVIDIA SoCs that use more than one "arm,mmu-500" + items: + - enum: + - nvdia,tegra194-smmu + - const: arm,mmu-500 - items: - const: arm,mmu-500 - const: arm,smmu-v2
Signature · 1 line
2.26.2
PATCH v7 3/3 iommu/arm-smmu: Add global/context fault implementation hooks
In reply to Krishna ReddyAdd global/context fault hooks to allow NVIDIA SMMU implementation handle faults across multiple SMMUs.
--- drivers/iommu/arm-smmu-nvidia.c | 101 +++++++++++++++++++++++++++++++- drivers/iommu/arm-smmu.c | 17 +++++- drivers/iommu/arm-smmu.h | 3 + 3 files changed, 118 insertions(+), 3 deletions(-)
drivers/iommu/arm-smmu-nvidia.c +100 -1
@@ -147,6 +147,102 @@ static int nvidia_smmu_reset(struct arm_smmu_device *smmu)return 0 ; } +static struct arm_smmu_domain *to_smmu_domain (struct iommu_domain *dom) +{ + return container_of(dom, struct arm_smmu_domain, domain); +} + +static irqreturn_t nvidia_smmu_global_fault_inst (int irq, +struct arm_smmu_device *smmu, +int inst) +{ +u32 gfsr, gfsynr0, gfsynr1, gfsynr2; +void __iomem *gr0_base =nvidia_smmu_page (smmu, inst,0 ); + + gfsr =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSR); + gfsynr0 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR0); + gfsynr1 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR1); + gfsynr2 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR2); + +if (!gfsr) +return IRQ_NONE ; + +dev_err_ratelimited (smmu->dev, +"Unexpected global fault, this could be serious\n" ); +dev_err_ratelimited (smmu->dev, +"\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n" , + gfsr, gfsynr0, gfsynr1, gfsynr2); + +writel_relaxed (gfsr, gr0_base + ARM_SMMU_GR0_sGFSR); +return IRQ_HANDLED ; +} + +static irqreturn_t nvidia_smmu_global_fault (int irq,void *dev) +{ +int inst; +irqreturn_t irq_ret =IRQ_NONE ; +struct arm_smmu_device *smmu = dev; +struct nvidia_smmu *nvidia_smmu =to_nvidia_smmu (smmu); + +for (inst =0 ; inst < nvidia_smmu->num_inst; inst++) { + irq_ret =nvidia_smmu_global_fault_inst (irq, smmu, inst); +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + +return irq_ret; +} + +static irqreturn_t nvidia_smmu_context_fault_bank (int irq, +struct arm_smmu_device *smmu, +int idx,int inst) +{ +u32 fsr, fsynr, cbfrsynra; +unsigned long iova; +void __iomem *gr1_base =nvidia_smmu_page (smmu, inst,1 ); +void __iomem *cb_base =nvidia_smmu_page (smmu, inst, smmu->numpage + idx); + + fsr =arm_smmu_cb_read (smmu, idx,ARM_SMMU_CB_FSR ); +if (!(fsr &ARM_SMMU_FSR_FAULT )) +return IRQ_NONE ; + + fsynr =readl_relaxed (cb_base +ARM_SMMU_CB_FSYNR0 ); + iova =readq_relaxed (cb_base +ARM_SMMU_CB_FAR ); + cbfrsynra =readl_relaxed (gr1_base +ARM_SMMU_GR1_CBFRSYNRA (idx)); + +dev_err_ratelimited (smmu->dev, +"Unhandled context fault: fsr=0x%x, iova=0x%08lx, fsynr=0x%x, cbfrsynra=0x%x, cb=%d\n" , + fsr, iova, fsynr, cbfrsynra, idx); + +writel_relaxed (fsr, cb_base +ARM_SMMU_CB_FSR ); +return IRQ_HANDLED ; +} + +static irqreturn_t nvidia_smmu_context_fault (int irq,void *dev) +{ +int inst, idx; +irqreturn_t irq_ret =IRQ_NONE ; +struct iommu_domain *domain = dev; +struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain); +struct arm_smmu_device *smmu = smmu_domain->smmu; + +for (inst =0 ; inst <to_nvidia_smmu (smmu)->num_inst; inst++) { +/* +* Interrupt line shared between all context faults. +* Check for faults across all contexts. +*/ +for (idx =0 ; idx < smmu->num_context_banks; idx++) { + irq_ret =nvidia_smmu_context_fault_bank (irq, smmu, + idx, inst); + +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + } + +return irq_ret; +} +static const struct arm_smmu_impl nvidia_smmu_impl = { .read_reg = nvidia_smmu_read_reg, .write_reg = nvidia_smmu_write_reg,
@@ -154,6 +250,8 @@ static const struct arm_smmu_impl nvidia_smmu_impl = { .write_reg64 = nvidia_smmu_write_reg64, .reset = nvidia_smmu_reset, .tlb_sync = nvidia_smmu_tlb_sync, + .global_fault = nvidia_smmu_global_fault, + .context_fault = nvidia_smmu_context_fault, };struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu)
@@ -185,7 +283,8 @@ struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu) } nvidia_smmu->smmu.impl = &nvidia_smmu_impl; -/* Free the arm_smmu_device struct allocated in arm-smmu.c. +/* +* Free the arm_smmu_device struct allocated in arm-smmu.c. * Once this function returns, arm-smmu.c would use arm_smmu_device * allocated as part of nvidia_smmu struct. */
drivers/iommu/arm-smmu.c +15 -2
@@ -673,6 +673,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,enum io_pgtable_fmt fmt;struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain);struct arm_smmu_cfg *cfg = &smmu_domain->cfg; +irqreturn_t (*context_fault)(int irq,void *dev);mutex_lock (&smmu_domain->init_mutex);if (smmu_domain->smmu)
@@ -835,7 +836,13 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,* handler seeing a half-initialised domain state. */ irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; - ret =devm_request_irq (smmu->dev, irq, arm_smmu_context_fault, + +if (smmu->impl && smmu->impl->context_fault) + context_fault = smmu->impl->context_fault; +else + context_fault = arm_smmu_context_fault; + + ret =devm_request_irq (smmu->dev, irq, context_fault,IRQF_SHARED ,"arm-smmu-context-fault" , domain);if (ret <0 ) { dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",
@@ -2107,6 +2114,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)struct arm_smmu_device *smmu;struct device *dev = &pdev->dev;int num_irqs, i, err; +irqreturn_t (*global_fault)(int irq,void *dev); smmu =devm_kzalloc (dev,sizeof (*smmu),GFP_KERNEL );if (!smmu) {
@@ -2193,9 +2201,14 @@ static int arm_smmu_device_probe(struct platform_device *pdev) smmu->num_context_irqs = smmu->num_context_banks; } +if (smmu->impl && smmu->impl->global_fault) + global_fault = smmu->impl->global_fault; +else + global_fault = arm_smmu_global_fault; +for (i =0 ; i < smmu->num_global_irqs; ++i) { err =devm_request_irq (smmu->dev, smmu->irqs[i], - arm_smmu_global_fault, + global_fault,IRQF_SHARED ,"arm-smmu global fault" , smmu);
drivers/iommu/arm-smmu.h +3 -0
@@ -18,6 +18,7 @@#include <linux/io-64-nonatomic-hi-lo.h> #include <linux/io-pgtable.h> #include <linux/iommu.h> +#include <linux/irqreturn.h> #include <linux/mutex.h> #include <linux/spinlock.h> #include <linux/types.h>
@@ -387,6 +388,8 @@ struct arm_smmu_impl {void (*tlb_sync)(struct arm_smmu_device *smmu,int page,int sync,int status);int (*def_domain_type)(struct device *dev); +irqreturn_t (*global_fault)(int irq,void *dev); +irqreturn_t (*context_fault)(int irq,void *dev); };static inline void __iomem *arm_smmu_page (struct arm_smmu_device *smmu,int n)
Signature · 1 line
2.26.2
Re: [PATCH v7 3/3] iommu/arm-smmu: Add global/context fault implementation hooks
Quoted 9 lines · Krishna Reddy
Add global/context fault hooks to allow NVIDIA SMMU implementation handle faults across multiple SMMUs.
Signed-off-by Krishna Reddy <[email protected]>drivers/iommu/arm-smmu-nvidia.c +4
-0+static irqreturn_t nvidia_smmu_global_fault_inst (int irq, +struct arm_smmu_device *smmu, +int inst) +{
drivers/iommu/arm-smmu-nvidia.c static int nvidia_smmu_reset(struct arm_smmu_device *smmu) · line 168
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c +10
-0+u32 gfsr, gfsynr0, gfsynr1, gfsynr2; +void __iomem *gr0_base =nvidia_smmu_page (smmu, inst,0 ); + + gfsr =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSR); + gfsynr0 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR0); + gfsynr1 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR1); + gfsynr2 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR2); + +if (!gfsr) +return IRQ_NONE ;
Could move this before gfsynr readings to save some readl() for !gfsr cases?
Quoted 3 lines
drivers/iommu/arm-smmu-nvidia.c +2
-0+static irqreturn_t nvidia_smmu_context_fault_bank (int irq, + void __iomem *cb_base = nvidia_smmu_page(smmu, inst, smmu->numpage + idx);
[...]
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c +1
-0+ fsr =arm_smmu_cb_read (smmu, idx,ARM_SMMU_CB_FSR );
[...]
drivers/iommu/arm-smmu-nvidia.c static int nvidia_smmu_reset(struct arm_smmu_device *smmu) · line 217
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c +1
-0+writel_relaxed (fsr, cb_base +ARM_SMMU_CB_FSR );
It reads FSR of the default inst (1st), but clears the FSR of corresponding inst -- just want to make sure that this is okay and intended.
drivers/iommu/arm-smmu-nvidia.c struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu) · line 290
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c +2
-1@@ -185,7 +283,8 @@ struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu) } nvidia_smmu->smmu.impl = &nvidia_smmu_impl; -/* Free the arm_smmu_device struct allocated in arm-smmu.c. +/* +* Free the arm_smmu_device struct allocated in arm-smmu.c. * Once this function returns, arm-smmu.c would use arm_smmu_device * allocated as part of nvidia_smmu struct. */
Hmm, this coding style fix should be probably squashed into PATCH-1?
RE: [PATCH v7 3/3] iommu/arm-smmu: Add global/context fault implementation hooks
Quoted 4 lines
Quoted 4 lines
(quoted diff) +3
-0+static irqreturn_t nvidia_smmu_context_fault_bank(int irq, + void __iomem *cb_base = nvidia_smmu_page(smmu, inst, + smmu->numpage + idx);
[...]
Quoted 1 line
Quoted 1 line
+ fsr =
arm_smmu_cb_read(smmu, idx, ARM_SMMU_CB_FSR);
[...]
Quoted 3 lines
Quoted 1 line
+
writel_relaxed(fsr, cb_base + ARM_SMMU_CB_FSR);It reads FSR of the default inst (1st), but clears the FSR of corresponding inst -- just want to make sure that this is okay and intended.
FSR should be read from corresponding inst. Not from instance 0. Let me post updated patch.
-KR