NVIDIA ARM SMMUv2 Implementation
Subsystems (4)
v9 cover NVIDIA ARM SMMUv2 Implementation
Changes in v9:
Move TLB Timeout and spin count macros to arm-smmu.h header to share with implementation. Set minItems and maxItems for reg property when compatible contains nvidia,tegra194-smmu. Update commit message for NVIDIA implementation patch.
Fail single SMMU instance usage through NVIDIA implementation to limit the usage to two or three instances. Fix checkpatch warnings with --strict checking.
v8 - https://lkml.org/lkml/2020/6/29/2385
v7 - https://lkml.org/lkml/2020/6/28/347
v6 - https://lkml.org/lkml/2020/6/4/1018
v5 - https://lkml.org/lkml/2020/5/21/1114
v4 - https://lkml.org/lkml/2019/10/30/1054
v3 - https://lkml.org/lkml/2019/10/18/1601
v2 - https://lkml.org/lkml/2019/9/2/980
v1 - https://lkml.org/lkml/2019/8/29/1588
Krishna Reddy (4):
iommu/arm-smmu: move TLB timeout and spin count macros iommu/arm-smmu: add NVIDIA implementation for ARM MMU-500 usage
dt-bindings: arm-smmu: add binding for Tegra194 SMMU iommu/arm-smmu: Add global/context fault implementation hooks
.../devicetree/bindings/iommu/arm,smmu.yaml | 18 ++ MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 304 ++++++++++++++++++ drivers/iommu/arm-smmu.c | 20 +- drivers/iommu/arm-smmu.h | 6 + 7 files changed, 349 insertions(+), 6 deletions(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.c
base-commit: 48f0bcfb7aad2c6eb4c1e66476b58475aa14393e
Signature · 1 line
2.26.2
PATCH v9 1/4 iommu/arm-smmu: move TLB timeout and spin count macros
Move TLB timeout and spin count macros to header file to allow using the same values from vendor specific implementations.
--- drivers/iommu/arm-smmu.c | 3 --- drivers/iommu/arm-smmu.h | 2 ++ 2 files changed, 2 insertions(+), 3 deletions(-)
drivers/iommu/arm-smmu.c +0 -3
@@ -52,9 +52,6 @@*/ #define QCOM_DUMMY_VAL -1 -#define TLB_LOOP_TIMEOUT 1000000/* 1s! */ -#define TLB_SPIN_COUNT 10 -#define MSI_IOVA_BASE 0x8000000#define MSI_IOVA_LENGTH 0x100000
drivers/iommu/arm-smmu.h +2 -0
@@ -236,6 +236,8 @@ enum arm_smmu_cbar_type {/* Maximum number of context banks per SMMU */ #define ARM_SMMU_MAX_CBS 128 +#define TLB_LOOP_TIMEOUT 1000000/* 1s! */ +#define TLB_SPIN_COUNT 10/* Shared driver definitions */ enum arm_smmu_arch_version {
Signature · 1 line
2.26.2
PATCH v9 2/4 iommu/arm-smmu: add NVIDIA implementation for ARM MMU-500 usage
In reply to Krishna ReddyNVIDIA's Tegra194 SoC has three ARM MMU-500 instances. It uses two of ARM MMU-500s together to interleave IOVA accesses across them and must be programmed identically.
The third SMMU instance is used as a regular ARM MMU-500 and it can either be programmed independently or identical to other two ARM MMU-500s.
This implementation supports programming two or three ARM MMU-500s identically as per DT config.
--- MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 206 ++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.h | 1 + 5 files changed, 213 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.c
MAINTAINERS +2 -0
@@ -16808,8 +16808,10 @@ F: drivers/i2c/busses/i2c-tegra.c TEGRA IOMMU DRIVERS M: Thierry Reding <[email protected]> +R: Krishna Reddy <[email protected]> L: [email protected] S: Supported +F: drivers/iommu/arm-smmu-nvidia.c F: drivers/iommu/tegra* TEGRA KBC DRIVER
drivers/iommu/Makefile +1 -1
@@ -15,7 +15,7 @@ obj-$(CONFIG_AMD_IOMMU) += amd/iommu.o amd/init.o amd/quirks.o obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += amd/debugfs.o obj-$(CONFIG_AMD_IOMMU_V2) += amd/iommu_v2.o obj-$(CONFIG_ARM_SMMU) += arm_smmu.o -arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o +arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-nvidia.o arm-smmu-qcom.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += intel/dmar.o obj-$(CONFIG_INTEL_IOMMU) += intel/iommu.o intel/pasid.o
drivers/iommu/arm-smmu-impl.c +3 -0
@@ -171,6 +171,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)if (of_property_read_bool (np,"calxeda,smmu-secure-config-access" )) smmu->impl = &calxeda_impl; +if (of_device_is_compatible (np,"nvidia,tegra194-smmu" )) +return nvidia_smmu_impl_init (smmu); +if (of_device_is_compatible (np,"qcom,sdm845-smmu-500" ) ||of_device_is_compatible (np,"qcom,sc7180-smmu-500" ))return qcom_smmu_impl_init (smmu);
drivers/iommu/arm-smmu-nvidia.c new file +206 -0
206 lines not on this page · raw message
drivers/iommu/arm-smmu.h +1 -0
@@ -452,6 +452,7 @@ static inline void arm_smmu_writeq(struct arm_smmu_device *smmu, int page, arm_smmu_writeq((s), ARM_SMMU_CB((s), (n)), (o), (v))struct arm_smmu_device *arm_smmu_impl_init (struct arm_smmu_device *smmu); +struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu);struct arm_smmu_device *qcom_smmu_impl_init (struct arm_smmu_device *smmu);int arm_mmu500_reset (struct arm_smmu_device *smmu);
Signature · 1 line
2.26.2
PATCH v9 3/4 dt-bindings: arm-smmu: add binding for Tegra194 SMMU
In reply to Krishna ReddyAdd binding for NVIDIA's Tegra194 SoC SMMU topology that is based on ARM MMU-500.
--- .../devicetree/bindings/iommu/arm,smmu.yaml | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +18 -0
@@ -38,6 +38,11 @@ properties: - qcom,sc7180-smmu-500 - qcom,sdm845-smmu-500 - const: arm,mmu-500 + - description: NVIDIA SoCs that use more than one "arm,mmu-500" + items: + - enum: + - nvidia,tegra194-smmu + - const: arm,mmu-500 - items: - const: arm,mmu-500 - const: arm,smmu-v2
@@ -138,6 +143,19 @@ required: additionalProperties: false +allOf: + - if: + properties: + compatible: + contains: + enum: + - nvidia,tegra194-smmu + then: + properties: + reg: + minItems: 2 + maxItems: 3 + examples: - |+ /* SMMU with stream matching or stream indexing */
Signature · 1 line
2.26.2
PATCH v9 4/4 iommu/arm-smmu: add global/context fault implementation hooks
In reply to Krishna ReddyAdd global/context fault hooks to allow NVIDIA SMMU implementation handle faults across multiple SMMUs.
--- drivers/iommu/arm-smmu-nvidia.c | 98 +++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.c | 17 +++++- drivers/iommu/arm-smmu.h | 3 + 3 files changed, 116 insertions(+), 2 deletions(-)
drivers/iommu/arm-smmu-nvidia.c +98 -0
@@ -144,6 +144,102 @@ static int nvidia_smmu_reset(struct arm_smmu_device *smmu)return 0 ; } +static struct arm_smmu_domain *to_smmu_domain (struct iommu_domain *dom) +{ + return container_of(dom, struct arm_smmu_domain, domain); +} + +static irqreturn_t nvidia_smmu_global_fault_inst (int irq, +struct arm_smmu_device *smmu, +int inst) +{ +u32 gfsr, gfsynr0, gfsynr1, gfsynr2; +void __iomem *gr0_base =nvidia_smmu_page (smmu, inst,0 ); + + gfsr =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSR); +if (!gfsr) +return IRQ_NONE ; + + gfsynr0 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR0); + gfsynr1 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR1); + gfsynr2 =readl_relaxed (gr0_base + ARM_SMMU_GR0_sGFSYNR2); + +dev_err_ratelimited (smmu->dev, +"Unexpected global fault, this could be serious\n" ); +dev_err_ratelimited (smmu->dev, +"\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n" , + gfsr, gfsynr0, gfsynr1, gfsynr2); + +writel_relaxed (gfsr, gr0_base + ARM_SMMU_GR0_sGFSR); +return IRQ_HANDLED ; +} + +static irqreturn_t nvidia_smmu_global_fault (int irq,void *dev) +{ +int inst; +irqreturn_t irq_ret =IRQ_NONE ; +struct arm_smmu_device *smmu = dev; +struct nvidia_smmu *nvidia_smmu =to_nvidia_smmu (smmu); + +for (inst =0 ; inst < nvidia_smmu->num_inst; inst++) { + irq_ret =nvidia_smmu_global_fault_inst (irq, smmu, inst); +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + +return irq_ret; +} + +static irqreturn_t nvidia_smmu_context_fault_bank (int irq, +struct arm_smmu_device *smmu, +int idx,int inst) +{ +u32 fsr, fsynr, cbfrsynra; +unsigned long iova; +void __iomem *gr1_base =nvidia_smmu_page (smmu, inst,1 ); +void __iomem *cb_base =nvidia_smmu_page (smmu, inst, smmu->numpage + idx); + + fsr =readl_relaxed (cb_base +ARM_SMMU_CB_FSR ); +if (!(fsr &ARM_SMMU_FSR_FAULT )) +return IRQ_NONE ; + + fsynr =readl_relaxed (cb_base +ARM_SMMU_CB_FSYNR0 ); + iova =readq_relaxed (cb_base +ARM_SMMU_CB_FAR ); + cbfrsynra =readl_relaxed (gr1_base +ARM_SMMU_GR1_CBFRSYNRA (idx)); + +dev_err_ratelimited (smmu->dev, +"Unhandled context fault: fsr=0x%x, iova=0x%08lx, fsynr=0x%x, cbfrsynra=0x%x, cb=%d\n" , + fsr, iova, fsynr, cbfrsynra, idx); + +writel_relaxed (fsr, cb_base +ARM_SMMU_CB_FSR ); +return IRQ_HANDLED ; +} + +static irqreturn_t nvidia_smmu_context_fault (int irq,void *dev) +{ +int inst, idx; +irqreturn_t irq_ret =IRQ_NONE ; +struct iommu_domain *domain = dev; +struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain); +struct arm_smmu_device *smmu = smmu_domain->smmu; + +for (inst =0 ; inst <to_nvidia_smmu (smmu)->num_inst; inst++) { +/* +* Interrupt line is shared between all contexts. +* Check for faults across all contexts. +*/ +for (idx =0 ; idx < smmu->num_context_banks; idx++) { + irq_ret =nvidia_smmu_context_fault_bank (irq, smmu, + idx, inst); + +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + } + +return irq_ret; +} +static const struct arm_smmu_impl nvidia_smmu_impl = { .read_reg = nvidia_smmu_read_reg, .write_reg = nvidia_smmu_write_reg,
@@ -151,6 +247,8 @@ static const struct arm_smmu_impl nvidia_smmu_impl = { .write_reg64 = nvidia_smmu_write_reg64, .reset = nvidia_smmu_reset, .tlb_sync = nvidia_smmu_tlb_sync, + .global_fault = nvidia_smmu_global_fault, + .context_fault = nvidia_smmu_context_fault, };struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu)
drivers/iommu/arm-smmu.c +15 -2
@@ -670,6 +670,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,enum io_pgtable_fmt fmt;struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain);struct arm_smmu_cfg *cfg = &smmu_domain->cfg; +irqreturn_t (*context_fault)(int irq,void *dev);mutex_lock (&smmu_domain->init_mutex);if (smmu_domain->smmu)
@@ -832,7 +833,13 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,* handler seeing a half-initialised domain state. */ irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; - ret =devm_request_irq (smmu->dev, irq, arm_smmu_context_fault, + +if (smmu->impl && smmu->impl->context_fault) + context_fault = smmu->impl->context_fault; +else + context_fault = arm_smmu_context_fault; + + ret =devm_request_irq (smmu->dev, irq, context_fault,IRQF_SHARED ,"arm-smmu-context-fault" , domain);if (ret <0 ) { dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",
@@ -2104,6 +2111,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)struct arm_smmu_device *smmu;struct device *dev = &pdev->dev;int num_irqs, i, err; +irqreturn_t (*global_fault)(int irq,void *dev); smmu =devm_kzalloc (dev,sizeof (*smmu),GFP_KERNEL );if (!smmu) {
@@ -2190,9 +2198,14 @@ static int arm_smmu_device_probe(struct platform_device *pdev) smmu->num_context_irqs = smmu->num_context_banks; } +if (smmu->impl && smmu->impl->global_fault) + global_fault = smmu->impl->global_fault; +else + global_fault = arm_smmu_global_fault; +for (i =0 ; i < smmu->num_global_irqs; ++i) { err =devm_request_irq (smmu->dev, smmu->irqs[i], - arm_smmu_global_fault, + global_fault,IRQF_SHARED ,"arm-smmu global fault" , smmu);
drivers/iommu/arm-smmu.h +3 -0
@@ -18,6 +18,7 @@#include <linux/io-64-nonatomic-hi-lo.h> #include <linux/io-pgtable.h> #include <linux/iommu.h> +#include <linux/irqreturn.h> #include <linux/mutex.h> #include <linux/spinlock.h> #include <linux/types.h>
@@ -389,6 +390,8 @@ struct arm_smmu_impl {void (*tlb_sync)(struct arm_smmu_device *smmu,int page,int sync,int status);int (*def_domain_type)(struct device *dev); +irqreturn_t (*global_fault)(int irq,void *dev); +irqreturn_t (*context_fault)(int irq,void *dev); };static inline void __iomem *arm_smmu_page (struct arm_smmu_device *smmu,int n)
Signature · 1 line
2.26.2
Jon Hunter
maintainerRe: [PATCH v9 2/4] iommu/arm-smmu: add NVIDIA implementation for ARM MMU-500 usage
In reply to Krishna ReddyQuoted 86 lines · Krishna Reddy
NVIDIA's Tegra194 SoC has three ARM MMU-500 instances. It uses two of ARM MMU-500s together to interleave IOVA accesses across them and must be programmed identically.
The third SMMU instance is used as a regular ARM MMU-500 and it can either be programmed independently or identical to other two ARM MMU-500s.This implementation supports programming two or three ARM MMU-500s identically as per DT config.
Signed-off-by Krishna Reddy <[email protected]>--- MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 206 ++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.h | 1 + 5 files changed, 213 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.cMAINTAINERS +2
-0@@ -16808,8 +16808,10 @@ F: drivers/i2c/busses/i2c-tegra.c TEGRA IOMMU DRIVERS M: Thierry Reding <[email protected]> +R: Krishna Reddy <[email protected]> L: [email protected] S: Supported +F: drivers/iommu/arm-smmu-nvidia.c F: drivers/iommu/tegra* TEGRA KBC DRIVERdrivers/iommu/Makefile +1
-1@@ -15,7 +15,7 @@ obj-$(CONFIG_AMD_IOMMU) += amd/iommu.o amd/init.o amd/quirks.o obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += amd/debugfs.o obj-$(CONFIG_AMD_IOMMU_V2) += amd/iommu_v2.o obj-$(CONFIG_ARM_SMMU) += arm_smmu.o -arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o +arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-nvidia.o arm-smmu-qcom.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += intel/dmar.o obj-$(CONFIG_INTEL_IOMMU) += intel/iommu.o intel/pasid.odrivers/iommu/arm-smmu-impl.c +3
-0@@ -171,6 +171,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)if (of_property_read_bool (np,"calxeda,smmu-secure-config-access" )) smmu->impl = &calxeda_impl; +if (of_device_is_compatible (np,"nvidia,tegra194-smmu" )) +return nvidia_smmu_impl_init (smmu); +if (of_device_is_compatible (np,"qcom,sdm845-smmu-500" ) ||of_device_is_compatible (np,"qcom,sc7180-smmu-500" ))return qcom_smmu_impl_init (smmu);drivers/iommu/arm-smmu-nvidia.c new file +18
-0@@ -0,0 +1,206 @@ +// SPDX-License-Identifier: GPL-2.0-only +// NVIDIA ARM SMMU v2 implementation quirks +// Copyright (C) 2019-2020 NVIDIA CORPORATION. All rights reserved. + +#include <linux/bitfield.h> +#include <linux/delay.h> +#include <linux/of.h> +#include <linux/platform_device.h> +#include <linux/slab.h> + +#include "arm-smmu.h" + +/* +* Tegra194 has three ARM MMU-500 Instances. +* Two of them are used together for interleaved IOVA accesses and +* used by non-isochronous HW devices for SMMU translations. +* Third one is used for SMMU translations from isochronous HW devices. +* It is possible to use this implementation to program either
drivers/iommu/arm-smmu-nvidia.c line 28
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c new file +10
-0+* all three or two of the instances identically as desired through +* DT node. +* +* Programming all the three instances identically comes with redundant TLB +* invalidations as all three never need to be TLB invalidated for a HW device. +* +* When Linux kernel supports multiple SMMU devices, the SMMU device used for +* isochornous HW devices should be added as a separate ARM MMU-500 device +* in DT and be programmed independently for efficient TLB invalidates. +*/
We should address Robin's comment about the 'When' above.
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c new file +10
-0+#define MAX_SMMU_INSTANCES 3 + +struct nvidia_smmu { +struct arm_smmu_device smmu; +unsigned int num_inst; +void __iomem *bases[MAX_SMMU_INSTANCES ]; +}; + +static inline struct nvidia_smmu *to_nvidia_smmu (struct arm_smmu_device *smmu) +{
drivers/iommu/arm-smmu-nvidia.c line 48
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c new file +10
-0+ return container_of(smmu, struct nvidia_smmu, smmu); +} + +static inline void __iomem *nvidia_smmu_page (struct arm_smmu_device *smmu, +unsigned int inst,int page) +{ +struct nvidia_smmu *nvidia_smmu =to_nvidia_smmu (smmu); + +if (!nvidia_smmu->bases[0 ]) + nvidia_smmu->bases[0 ] = smmu->base;
Robin said that he would accept a patch to move the devm_ioremap_resource() call in arm_smmu_device_probe() so that we do not need to do this. See the V7 series. I think that this would be a good improvement so that we could avoid doing the above.
Cheers
Jon
Signature · 1 line
nvpublic
Jon Hunter
maintainerRe: [PATCH v9 3/4] dt-bindings: arm-smmu: add binding for Tegra194 SMMU
In reply to Krishna ReddyQuoted 13 lines · Krishna Reddy
Add binding for NVIDIA's Tegra194 SoC SMMU topology that is based on ARM MMU-500.
Signed-off-by Krishna Reddy <[email protected]>--- .../devicetree/bindings/iommu/arm,smmu.yaml | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)Documentation/devicetree/bindings/iommu/arm,smmu.yaml +0
-0
Documentation/devicetree/bindings/iommu/arm,smmu.yaml properties: · line 45
Quoted 8 lines
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +5
-0@@ -38,6 +38,11 @@ properties: - qcom,sc7180-smmu-500 - qcom,sdm845-smmu-500 - const: arm,mmu-500 + - description: NVIDIA SoCs that use more than one "arm,mmu-500" + items: + - enum: + - nvidia,tegra194-smmu + - const: arm,mmu-500
We should also address Robin's comments here. I don't think that we ever want to fail back to just the regular 'arm,mmu-500' and should be should probably just drop this part.
Jon
Signature · 1 line
nvpublic