Nvidia Arm SMMUv2 Implementation
Subsystems (4)
v6 cover Nvidia Arm SMMUv2 Implementation
Changes in v6:
Restricted the patch set to driver specific patches.
Fixed the cast warning reported by kbuild test robot.
Rebased on git://git.kernel.org/pub/scm/linux/kernel/git/joro/iommu.git next
v5 - https://lkml.org/lkml/2020/5/21/1114
v4 - https://lkml.org/lkml/2019/10/30/1054
v3 - https://lkml.org/lkml/2019/10/18/1601
v2 - https://lkml.org/lkml/2019/9/2/980
v1 - https://lkml.org/lkml/2019/8/29/1588
Krishna Reddy (4):
iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
dt-bindings: arm-smmu: Add binding for Tegra194 SMMU iommu/arm-smmu: Add global/context fault implementation hooks iommu/arm-smmu-nvidia: fix the warning reported by kbuild test robot
.../devicetree/bindings/iommu/arm,smmu.yaml | 5 + MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 261 ++++++++++++++++++ drivers/iommu/arm-smmu.c | 11 +- drivers/iommu/arm-smmu.h | 4 + 7 files changed, 285 insertions(+), 3 deletions(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.c
base-commit: 431275afdc7155415254aef4bd3816a1b8a2ead0
Signature · 1 line
2.26.2
PATCH v6 1/4 iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
NVIDIA's Tegra194 soc uses two ARM MMU-500s together to interleave IOVA accesses across them.
Add NVIDIA implementation for dual ARM MMU-500s and add new compatible string for Tegra194 soc.
--- MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 161 ++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.h | 1 + 5 files changed, 168 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.c
MAINTAINERS +2 -0
@@ -16572,9 +16572,11 @@ F: drivers/i2c/busses/i2c-tegra.c TEGRA IOMMU DRIVERS M: Thierry Reding <[email protected]> +R: Krishna Reddy <[email protected]> L: [email protected] S: Supported F: drivers/iommu/tegra* +F: drivers/iommu/arm-smmu-nvidia.c TEGRA KBC DRIVER M: Laxman Dewangan <[email protected]>
drivers/iommu/Makefile +1 -1
@@ -15,7 +15,7 @@ obj-$(CONFIG_AMD_IOMMU) += amd_iommu.o amd_iommu_init.o amd_iommu_quirks.o obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += amd_iommu_debugfs.o obj-$(CONFIG_AMD_IOMMU_V2) += amd_iommu_v2.o obj-$(CONFIG_ARM_SMMU) += arm_smmu.o -arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o +arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o arm-smmu-nvidia.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += dmar.o obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o intel-pasid.o
drivers/iommu/arm-smmu-impl.c +3 -0
@@ -160,6 +160,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)*/ switch (smmu->model) {case ARM_MMU500 : +if (of_device_is_compatible (smmu->dev->of_node, +"nvidia,tegra194-smmu-500" )) +return nvidia_smmu_impl_init (smmu); smmu->impl = &arm_mmu500_impl;break ;case CAVIUM_SMMUV2 :
drivers/iommu/arm-smmu-nvidia.c new file +161 -0
161 lines not on this page · raw message
drivers/iommu/arm-smmu.h +1 -0
@@ -451,6 +451,7 @@ static inline void arm_smmu_writeq(struct arm_smmu_device *smmu, int page,struct arm_smmu_device *arm_smmu_impl_init (struct arm_smmu_device *smmu);struct arm_smmu_device *qcom_smmu_impl_init (struct arm_smmu_device *smmu); +struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu);int arm_mmu500_reset (struct arm_smmu_device *smmu);
Signature · 1 line
2.26.2
Re: [PATCH v6 1/4] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
Quoted 1 line · Krishna Reddy
NVIDIA's Tegra194 soc uses two ARM MMU-500s together to interleave
s/soc/SoC/
Quoted 3 lines
IOVA accesses across them.
Add NVIDIA implementation for dual ARM MMU-500s and add new compatible string for Tegra194 soc.
Same here.
Quoted 45 lines
Signed-off-by Krishna Reddy <[email protected]>--- MAINTAINERS | 2 + drivers/iommu/Makefile | 2 +- drivers/iommu/arm-smmu-impl.c | 3 + drivers/iommu/arm-smmu-nvidia.c | 161 ++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.h | 1 + 5 files changed, 168 insertions(+), 1 deletion(-) create mode 100644 drivers/iommu/arm-smmu-nvidia.cMAINTAINERS +2
-0@@ -16572,9 +16572,11 @@ F: drivers/i2c/busses/i2c-tegra.c TEGRA IOMMU DRIVERS M: Thierry Reding <[email protected]> +R: Krishna Reddy <[email protected]> L: [email protected] S: Supported F: drivers/iommu/tegra* +F: drivers/iommu/arm-smmu-nvidia.c TEGRA KBC DRIVER M: Laxman Dewangan <[email protected]>drivers/iommu/Makefile +1
-1@@ -15,7 +15,7 @@ obj-$(CONFIG_AMD_IOMMU) += amd_iommu.o amd_iommu_init.o amd_iommu_quirks.o obj-$(CONFIG_AMD_IOMMU_DEBUGFS) += amd_iommu_debugfs.o obj-$(CONFIG_AMD_IOMMU_V2) += amd_iommu_v2.o obj-$(CONFIG_ARM_SMMU) += arm_smmu.o -arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o +arm_smmu-objs += arm-smmu.o arm-smmu-impl.o arm-smmu-qcom.o arm-smmu-nvidia.o obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o obj-$(CONFIG_DMAR_TABLE) += dmar.o obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o intel-pasid.odrivers/iommu/arm-smmu-impl.c +0
-0
drivers/iommu/arm-smmu-impl.c struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu) · line 165
Quoted 6 lines
drivers/iommu/arm-smmu-impl.c +3
-0@@ -160,6 +160,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)*/ switch (smmu->model) {case ARM_MMU500 : +if (of_device_is_compatible (smmu->dev->of_node, +"nvidia,tegra194-smmu-500" )) +return nvidia_smmu_impl_init (smmu);
Should NVIDIA_TEGRA194_SMMU be a separate value for smmu->model, perhaps? That way we avoid this somewhat odd check here.
drivers/iommu/arm-smmu-nvidia.c
Quoted 4 lines
drivers/iommu/arm-smmu-impl.c +0
-0smmu->impl = &arm_mmu500_impl;break ;case CAVIUM_SMMUV2 :drivers/iommu/arm-smmu-nvidia.c +0
-0
I wonder if it would be better to name this arm-smmu-tegra.c to make it clearer that this is for a Tegra chip. We do have regular expressions in MAINTAINERS that catch anything with "tegra" in it to make this easier.
drivers/iommu/arm-smmu-nvidia.c line 2
Quoted 7 lines
new file mode 100644
index 0000000000000..dafc293a45217drivers/iommu/arm-smmu-nvidia.c new file +2
-0@@ -0,0 +1,161 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Nvidia ARM SMMU v2 implementation quirks
s/Nvidia/NVIDIA/
drivers/iommu/arm-smmu-nvidia.c line 3
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+// Copyright (C) 2019 NVIDIA CORPORATION. All rights reserved.
I suppose this should now also include 2020.
drivers/iommu/arm-smmu-nvidia.c line 5
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c new file +2
-0+ +#define pr_fmt (fmt) "nvidia-smmu: " fmt
Same here. Might be worth making this "tegra-smmu: " for consistency.
drivers/iommu/arm-smmu-nvidia.c line 17
Quoted 12 lines
drivers/iommu/arm-smmu-nvidia.c new file +12
-0+ +#include <linux/bitfield.h> +#include <linux/delay.h> +#include <linux/of.h> +#include <linux/platform_device.h> +#include <linux/slab.h> + +#include "arm-smmu.h" + +/* Tegra194 has three ARM MMU-500 Instances. +* Two of them are used together for Interleaved IOVA accesses and +* used by Non-Isochronous Hw devices for SMMU translations.
"non-isochronous", s/Hw/HW/
drivers/iommu/arm-smmu-nvidia.c line 18
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+* Third one is used for SMMU translations from Isochronous HW devices.
"isochronous"
drivers/iommu/arm-smmu-nvidia.c line 19
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+* It is possible to use this Implementation to program either
"implementation"
drivers/iommu/arm-smmu-nvidia.c line 23
Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4
-0+* all three or two of the instances identically as desired through +* DT node. +* +* Programming all the three instances identically comes with redundant tlb
s/tlb/TLB/
drivers/iommu/arm-smmu-nvidia.c line 24
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+* invalidations as all three never need to be tlb invalidated for a HW device.
Same here.
drivers/iommu/arm-smmu-nvidia.c line 26
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c new file +2
-0+* +* When Linux Kernel supports multiple SMMU devices, The SMMU device used for
"kernel" and "..., the SMMU device"
drivers/iommu/arm-smmu-nvidia.c line 27
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+* Isochornous HW devices should be added as a separate ARM MMU-500 device
"isochronous"
drivers/iommu/arm-smmu-nvidia.c line 28
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+* in DT and be programmed independently for efficient tlb invalidates.
"TLB"
drivers/iommu/arm-smmu-nvidia.c line 33
Quoted 5 lines
drivers/iommu/arm-smmu-nvidia.c new file +5
-0+* +*/ +#define MAX_SMMU_INSTANCES 3 + +#define TLB_LOOP_TIMEOUT 1000000/* 1s! */
USEC_PER_SEC?
drivers/iommu/arm-smmu-nvidia.c line 42
Quoted 9 lines
drivers/iommu/arm-smmu-nvidia.c new file +9
-0+#define TLB_SPIN_COUNT 10 + +struct nvidia_smmu { +struct arm_smmu_device smmu; +unsigned int num_inst; +void __iomem *bases[MAX_SMMU_INSTANCES ]; +}; + +#define to_nvidia_smmu (s) container_of(s, struct nvidia_smmu, smmu)
Making this static inline can make error messages more readable.
drivers/iommu/arm-smmu-nvidia.c line 46
Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4
-0+ +#define nsmmu_page (smmu, inst, page) \ + (((inst) ? to_nvidia_smmu(smmu)->bases[(inst)] : smmu->base) + \ + ((page) << smmu->pgshift))
Can we simply define to_nvidia_smmu(smmu)->bases[0] = smmu->base in nvidia_smmu_impl_init()? Then this would become just:
to_nvidia_smmu(smmu)->bases[inst] + ((page) << (smmu)->pgshift)
Also, the nsmmu_ prefix looks somewhat odd here. You already use struct nvidia_smmu as the name of the structure, so why not be consistent and continue to use nvidia_smmu_ as the prefix for function names?
Or perhaps even use tegra_smmu_ as the prefix to match the filename change I suggested earlier.
Quoted 25 lines
drivers/iommu/arm-smmu-nvidia.c new file +25
-0+ +static u32 nsmmu_read_reg (struct arm_smmu_device *smmu, +int page,int offset) +{ +return readl_relaxed (nsmmu_page (smmu,0 , page) + offset); +} + +static void nsmmu_write_reg (struct arm_smmu_device *smmu, +int page,int offset,u32 val) +{ +unsigned int i; + +for (i =0 ; i <to_nvidia_smmu (smmu)->num_inst; i++) +writel_relaxed (val,nsmmu_page (smmu, i, page) + offset); +} + +static u64 nsmmu_read_reg64 (struct arm_smmu_device *smmu, +int page,int offset) +{ +return readq_relaxed (nsmmu_page (smmu,0 , page) + offset); +} + +static void nsmmu_write_reg64 (struct arm_smmu_device *smmu, +int page,int offset,u64 val) +{
drivers/iommu/arm-smmu-nvidia.c line 81
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c new file +10
-0+unsigned int i; + +for (i =0 ; i <to_nvidia_smmu (smmu)->num_inst; i++) +writeq_relaxed (val,nsmmu_page (smmu, i, page) + offset); +} + +static void nsmmu_tlb_sync (struct arm_smmu_device *smmu,int page, +int sync,int status) +{ +u32 reg;
I see this is being used to store the value read from a register. I find it clearer to call this "value" or "val" (or in this case perhaps even "status") because whenever I read "reg" I immediately think this is meant to be a register offset, which can then be confusing when I see it used in I/O accessors because it is in the wrong position.
But anyway, that's just my opinion and this is a bit subjective, so feel free to ignore.
drivers/iommu/arm-smmu-nvidia.c line 89
Quoted 8 lines
drivers/iommu/arm-smmu-nvidia.c new file +8
-0+unsigned int i; +unsigned int spin_cnt, delay; + +arm_smmu_writel (smmu, page, sync,0 ); + +for (delay =1 ; delay <TLB_LOOP_TIMEOUT ; delay *=2 ) { +for (spin_cnt =TLB_SPIN_COUNT ; spin_cnt >0 ; spin_cnt--) { + reg =0 ;
You may want to declare the variable at this scope since you never need it outside. Also, use a space between variable initialization and the for block below for better readability.
drivers/iommu/arm-smmu-nvidia.c line 93
Quoted 4 lines
drivers/iommu/arm-smmu-nvidia.c new file +4
-0+for (i =0 ; i <to_nvidia_smmu (smmu)->num_inst; i++) { + reg |=readl_relaxed ( +nsmmu_page (smmu, i, page) + status); + }
Maybe add a separate variable for the page address so this can be a bit uncluttered. Also, I'd prefer a blank line after the block for readability.
drivers/iommu/arm-smmu-nvidia.c line 96
Quoted 3 lines
drivers/iommu/arm-smmu-nvidia.c new file +3
-0+if (!(reg & ARM_SMMU_sTLBGSTATUS_GSACTIVE)) +return ; +cpu_relax ();
Blank line above cpu_relax() for readability.
drivers/iommu/arm-smmu-nvidia.c line 98
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c new file +2
-0+ } +udelay (delay);
Again, a blank line between blocks and subsequent statements can help readability.
drivers/iommu/arm-smmu-nvidia.c line 101
Quoted 3 lines
drivers/iommu/arm-smmu-nvidia.c new file +3
-0+ } +dev_err_ratelimited (smmu->dev, +"TLB sync timed out -- SMMU may be deadlocked\n" );
Same here.
Also, is there anything we can do when this happens?
Quoted 32 lines
drivers/iommu/arm-smmu-nvidia.c new file +32
-0+} + +static int nsmmu_reset (struct arm_smmu_device *smmu) +{ +u32 reg; +unsigned int i; + +for (i =0 ; i <to_nvidia_smmu (smmu)->num_inst; i++) { +/* clear global FSR */ + reg =readl_relaxed (nsmmu_page (smmu, i,ARM_SMMU_GR0 ) + + ARM_SMMU_GR0_sGFSR); +writel_relaxed (reg,nsmmu_page (smmu, i,ARM_SMMU_GR0 ) + + ARM_SMMU_GR0_sGFSR); + } + +return 0 ; +} + +static const struct arm_smmu_impl nvidia_smmu_impl = { + .read_reg = nsmmu_read_reg, + .write_reg = nsmmu_write_reg, + .read_reg64 = nsmmu_read_reg64, + .write_reg64 = nsmmu_write_reg64, + .reset = nsmmu_reset, + .tlb_sync = nsmmu_tlb_sync, +}; + +struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu) +{ +unsigned int i; +struct nvidia_smmu *nsmmu; +struct resource *res;
drivers/iommu/arm-smmu-nvidia.c line 143
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c new file +10
-0+struct device *dev = smmu->dev; +struct platform_device *pdev =to_platform_device (smmu->dev); + + nsmmu =devm_kzalloc (smmu->dev,sizeof (*nsmmu),GFP_KERNEL ); +if (!nsmmu) +return ERR_PTR (-ENOMEM ); + + nsmmu->smmu = *smmu; +/* Instance 0 is ioremapped by arm-smmu.c */ + nsmmu->num_inst =1 ;
Maybe add this here to simplify the nsmmu_page() macro above:
nsmmu->bases[0] = smmu->base;
drivers/iommu/arm-smmu-nvidia.c line 151
Quoted 8 lines
drivers/iommu/arm-smmu-nvidia.c new file +8
-0+ +for (i =1 ; i <MAX_SMMU_INSTANCES ; i++) { + res =platform_get_resource (pdev,IORESOURCE_MEM , i); +if (!res) +break ; + nsmmu->bases[i] =devm_ioremap_resource (dev, res); +if (IS_ERR (nsmmu->bases[i])) +return (struct arm_smmu_device *)nsmmu->bases[i];
ERR_CAST()?
drivers/iommu/arm-smmu-nvidia.c line 153
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c new file +2
-0+ nsmmu->num_inst++; + }
More blank lines would make this much easier to read, in my opinion.
drivers/iommu/arm-smmu-nvidia.c line 156
Quoted 3 lines
drivers/iommu/arm-smmu-nvidia.c new file +3
-0+ + nsmmu->smmu.impl = &nvidia_smmu_impl; +devm_kfree (smmu->dev, smmu);
Maybe a comment here would be useful for readers to immediately understand why you're doing this here.
drivers/iommu/arm-smmu-nvidia.c line 158
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c new file +2
-0+pr_info ("NVIDIA ARM SMMU Implementation, Instances=%d\n" , + nsmmu->num_inst);
I think I'd just omit this. In general you should only let the user know when things go wrong, but the above is printed when everything goes as expected.
Thierry
Robin Murphy
reviewerRe: [PATCH v6 1/4] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
On 2020-06-23 11:29, Thierry Reding wrote:
[...]
Quoted 6 lines
Quoted 6 lines
drivers/iommu/arm-smmu-impl.c +0
-0@@ -160,6 +160,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)*/
Quoted 8 lines
drivers/iommu/arm-smmu-impl.c struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu) · line 165
Quoted 5 lines
drivers/iommu/arm-smmu-impl.c +3
-0switch (smmu->model) {case ARM_MMU500 : +if (of_device_is_compatible (smmu->dev->of_node, +"nvidia,tegra194-smmu-500" )) +return nvidia_smmu_impl_init (smmu);Should
NVIDIA_TEGRA194_SMMUbe a separate value forsmmu->model, perhaps? That way we avoid this somewhat odd check here.
No, this is simply in the wrong place. The design here is that we pick up anything related to the basic SMMU IP (model) first, then make any platform-specific integration checks. That way a platform-specific init function can see the model impl set and subclass it if necessary (although nobody's actually done that yet). The setup for Cavium is just a short-cut since their model is unique to their integration, so the lines get a bit blurred and there's little benefit to trying to separate it out.
In short, put this down below with the other of_device_is_compatible() checks.
Quoted 8 lines
drivers/iommu/arm-smmu-nvidia.c
Quoted 4 lines
drivers/iommu/arm-smmu-impl.c +0
-0smmu->impl = &arm_mmu500_impl;break ;case CAVIUM_SMMUV2 :drivers/iommu/arm-smmu-nvidia.c +0
-0I wonder if it would be better to name this
arm-smmu-tegra.cto make it clearer that this is for a Tegra chip. We do have regular expressions inMAINTAINERSthat catch anything with "tegra" in it to make this easier.
There was a notion that these would be grouped by vendor, but if there's a strong preference for all NVIDIA-SoC-related stuff to be named "Tegra" then I'm not going to complain too much.
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c line 2
Quoted 7 lines
new file mode 100644
index 0000000000000..dafc293a45217drivers/iommu/arm-smmu-nvidia.c new file +2
-0@@ -0,0 +1,161 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Nvidia ARM SMMU v2 implementation quirks s/Nvidia/NVIDIA/
Quoted 8 lines
drivers/iommu/arm-smmu-nvidia.c line 3
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+// Copyright (C) 2019 NVIDIA CORPORATION. All rights reserved. I suppose this should now also include 2020.
drivers/iommu/arm-smmu-nvidia.c line 5
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c new file +2
-0+ +#define pr_fmt (fmt) "nvidia-smmu: " fmtSame here. Might be worth making this "tegra-smmu: " for consistency.
On the other hand, a log prefix that is literally the name of a completely unrelated driver seems more confusing to users than useful. Same for the function naming - the tegra_smmu_* namespace is already owned by that driver.
Robin.
Re: [PATCH v6 1/4] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
Quoted 37 lines · Robin Murphy
On 2020-06-23 11:29, Thierry Reding wrote:
[...]Quoted 6 lines
Quoted 6 lines
drivers/iommu/arm-smmu-impl.c +0
-0@@ -160,6 +160,9 @@ struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu)*/ Quoted 8 lines
drivers/iommu/arm-smmu-impl.c struct arm_smmu_device *arm_smmu_impl_init(struct arm_smmu_device *smmu) · line 165
Quoted 5 lines
drivers/iommu/arm-smmu-impl.c +3
-0switch (smmu->model) {case ARM_MMU500 : +if (of_device_is_compatible (smmu->dev->of_node, +"nvidia,tegra194-smmu-500" )) +return nvidia_smmu_impl_init (smmu);Should
NVIDIA_TEGRA194_SMMUbe a separate value forsmmu->model, perhaps? That way we avoid this somewhat odd check here.No, this is simply in the wrong place. The design here is that we pick up anything related to the basic SMMU IP (model) first, then make any platform-specific integration checks. That way a platform-specific init function can see the model impl set and subclass it if necessary (although nobody's actually done that yet). The setup for Cavium is just a short-cut since their model is unique to their integration, so the lines get a bit blurred and there's little benefit to trying to separate it out.
In short, put this down below with the other
of_device_is_compatible()checks.Quoted 8 lines
drivers/iommu/arm-smmu-nvidia.c
Quoted 4 lines
drivers/iommu/arm-smmu-impl.c +0
-0smmu->impl = &arm_mmu500_impl;break ;case CAVIUM_SMMUV2 :drivers/iommu/arm-smmu-nvidia.c +0
-0I wonder if it would be better to name this
arm-smmu-tegra.cto make it clearer that this is for a Tegra chip. We do have regular expressions inMAINTAINERSthat catch anything with "tegra" in it to make this easier.
Quoted 3 lines
There was a notion that these would be grouped by vendor, but if there's a strong preference for all NVIDIA-SoC-related stuff to be named "Tegra" then I'm not going to complain too much.
Maybe I was being overly cautious. I was just trying to avoid adding something called nvidia-arm-smmu which might eventually turn out to be ambiguous if there was ever a non-Tegra chip and the ARM SMMU implementation was not compatible with the one instantiated on Tegra.
Note that I have no knowledge of such a chip being designed, so this may never actually become an issue.
In either case, the compatible string already identifies this as Tegra- specific, so we could always change the driver name later on if we have to.
Quoted 19 lines
Quoted 10 lines
drivers/iommu/arm-smmu-nvidia.c line 2
Quoted 7 lines
new file mode 100644
index 0000000000000..dafc293a45217drivers/iommu/arm-smmu-nvidia.c new file +2
-0@@ -0,0 +1,161 @@ +// SPDX-License-Identifier: GPL-2.0-only +// Nvidia ARM SMMU v2 implementation quirks s/Nvidia/NVIDIA/
Quoted 8 lines
drivers/iommu/arm-smmu-nvidia.c line 3
Quoted 1 line
drivers/iommu/arm-smmu-nvidia.c new file +1
-0+// Copyright (C) 2019 NVIDIA CORPORATION. All rights reserved. I suppose this should now also include 2020.
drivers/iommu/arm-smmu-nvidia.c line 5
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c new file +2
-0+ +#define pr_fmt (fmt) "nvidia-smmu: " fmtSame here. Might be worth making this "tegra-smmu: " for consistency.
Quoted 4 lines
On the other hand, a log prefix that is literally the name of a completely unrelated driver seems more confusing to users than useful. Same for the function naming - the tegra_smmu_* namespace is already owned by that driver.
The ARM SMMU replaced the Tegra SMMU on Tegra186 and later, so both drivers are never going to run concurrently. The only "problem" might be that both drivers have symbols with the same prefix, but since they don't export any of those symbols I don't see how that would become a real issue.
But then again, the Tegra SMMU is also technically an NVIDIA SMMU, so sticking with the current name might also be confusing.
Perhaps a good compromise would be to use a "tegra194{-,_}smmu" prefix instead, which would make this fairly specific to just Tegra194 (and compatible chips). That's a fairly common pattern we've been following on Tegra, as you can for example see in drivers/gpio/gpio-tegra186.c, drivers/dma/tegra210-adma.c, drivers/memory/tegra/tegra186-emc.c, etc.
Thierry
RE: [PATCH v6 1/4] iommu/arm-smmu: add NVIDIA implementation for dual ARM MMU-500 usage
In reply to Thierry RedingQuoted 1 line
Should
NVIDIA_TEGRA194_SMMUbe a separate value forsmmu->model, perhaps? That way we avoid this somewhat odd check here.
NVIDIA haven't made any changes to arm,mmu-500. It is only used in different topology. New model would be mis-leading here. As suggested by Robin, It can just be moved to end of function.
Quoted 5 lines
drivers/iommu/arm-smmu-nvidia.c
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c +0
-0I wonder if it would be better to name this
arm-smmu-tegra.cto make it clearer that this is for a Tegra chip. We do have regular expressions inMAINTAINERSthat catch anything with "tegra" in it to make this easier. Also, thensmmu_prefix looks somewhat odd here. You already use structnvidia_smmuas the name of the structure, so why not be consistent and continue to usenvidia_smmu_as the prefix for function names? Or perhaps even usetegra_smmu_as the prefix to match the filename change I suggested earlier.
Prefix can be updated to nvidia_smmu as we seem to be okay for now to keep file name as arm-smmu-nvidia.c after the vendor name.
Quoted 2 lines
Quoted 1 line
+#define
TLB_LOOP_TIMEOUT1000000 /* 1s! */
USEC_PER_SEC?
It is not meant for a conversion. Reused Timeout variable from arm-smmu.c for tlb_sync implementation. Can rename it to TLB_LOOP_TIMEOUT_IN_US.
Quoted 5 lines
drivers/iommu/arm-smmu-nvidia.c line 101
Quoted 3 lines
drivers/iommu/arm-smmu-nvidia.c +3
-0+ } +dev_err_ratelimited (smmu->dev, +"TLB sync timed out -- SMMU may be deadlocked\n" );Same here.
Also, is there anything we can do when this happens?
This is never expected to happen on Silicon. This code and message is reused from arm-smmu.c.
Quoted 9 lines
drivers/iommu/arm-smmu-nvidia.c line 46
Quoted 3 lines
drivers/iommu/arm-smmu-nvidia.c +3
-0+#define nsmmu_page (smmu, inst, page) \ + (((inst) ? to_nvidia_smmu(smmu)->bases[(inst)] : smmu->base) + \ + ((page) << smmu->pgshift))Can we simply define to_nvidia_smmu(smmu)->bases[0] =
smmu->baseinnvidia_smmu_impl_init()? Then this would become just:to_nvidia_smmu(smmu)->bases[inst] + ((page) << (smmu)->pgshift) + Maybe add this here to simplify the nsmmu_page() macro above: nsmmu->bases[0] = smmu->base;
This preferred to avoid the check in nsmmu_page(). But, smmu->base is not yet populated when nvidia_smmu_impl_init() is called. Let me look at the alternative place to set it.
-KR
PATCH v6 2/4 dt-bindings: arm-smmu: Add binding for Tegra194 SMMU
In reply to Krishna ReddyAdd binding for NVIDIA's Tegra194 Soc SMMU that is based on ARM MMU-500.
--- Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 5 +++++ 1 file changed, 5 insertions(+)
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +5 -0
@@ -37,6 +37,11 @@ properties: - qcom,sc7180-smmu-500 - qcom,sdm845-smmu-500 - const: arm,mmu-500 + - description: NVIDIA SoCs that use more than one "arm,mmu-500" + items: + - enum: + - nvdia,tegra194-smmu-500 + - const: arm,mmu-500 - items: - const: arm,mmu-500 - const: arm,smmu-v2
Signature · 1 line
2.26.2
Re: [PATCH v6 2/4] dt-bindings: arm-smmu: Add binding for Tegra194 SMMU
Quoted 13 lines · Krishna Reddy
Add binding for NVIDIA's Tegra194 Soc SMMU that is based on ARM MMU-500.
Signed-off-by Krishna Reddy <[email protected]>--- Documentation/devicetree/bindings/iommu/arm,smmu.yaml | 5 +++++ 1 file changed, 5 insertions(+)Documentation/devicetree/bindings/iommu/arm,smmu.yaml +0
-0
Documentation/devicetree/bindings/iommu/arm,smmu.yaml properties: · line 43
Quoted 7 lines
Documentation/devicetree/bindings/iommu/arm,smmu.yaml +4
-0@@ -37,6 +37,11 @@ properties: - qcom,sc7180-smmu-500 - qcom,sdm845-smmu-500 - const: arm,mmu-500 + - description: NVIDIA SoCs that use more than one "arm,mmu-500" + items: + - enum: + - nvdia,tegra194-smmu-500
The -500 suffix here seems a bit redundant since there's no other type of SMMU in Tegra194, correct?
Thierry
RE: [PATCH v6 2/4] dt-bindings: arm-smmu: Add binding for Tegra194 SMMU
Quoted 2 lines
Quoted 1 line
+ - nvdia,tegra194-smmu-500
The -500 suffix here seems a bit redundant since there's no other type of SMMU in Tegra194, correct?
Yeah, there is only one type of SMMU supported in T194. It was added to be synonymous with mmu-500. Can be removed.
-KR
PATCH v6 3/4 iommu/arm-smmu: Add global/context fault implementation hooks
In reply to Krishna ReddyAdd global/context fault hooks to allow NVIDIA SMMU implementation handle faults across multiple SMMUs.
--- drivers/iommu/arm-smmu-nvidia.c | 100 ++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.c | 11 +++- drivers/iommu/arm-smmu.h | 3 + 3 files changed, 112 insertions(+), 2 deletions(-)
drivers/iommu/arm-smmu-nvidia.c +100 -0
@@ -117,6 +117,104 @@ static int nsmmu_reset(struct arm_smmu_device *smmu)return 0 ; } +static struct arm_smmu_domain *to_smmu_domain (struct iommu_domain *dom) +{ + return container_of(dom, struct arm_smmu_domain, domain); +} + +static irqreturn_t nsmmu_global_fault_inst (int irq, +struct arm_smmu_device *smmu, +int inst) +{ +u32 gfsr, gfsynr0, gfsynr1, gfsynr2; + + gfsr =readl_relaxed (nsmmu_page (smmu, inst,0 ) + ARM_SMMU_GR0_sGFSR); + gfsynr0 =readl_relaxed (nsmmu_page (smmu, inst,0 ) + + ARM_SMMU_GR0_sGFSYNR0); + gfsynr1 =readl_relaxed (nsmmu_page (smmu, inst,0 ) + + ARM_SMMU_GR0_sGFSYNR1); + gfsynr2 =readl_relaxed (nsmmu_page (smmu, inst,0 ) + + ARM_SMMU_GR0_sGFSYNR2); + +if (!gfsr) +return IRQ_NONE ; + +dev_err_ratelimited (smmu->dev, +"Unexpected global fault, this could be serious\n" ); +dev_err_ratelimited (smmu->dev, +"\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n" , + gfsr, gfsynr0, gfsynr1, gfsynr2); + +writel_relaxed (gfsr,nsmmu_page (smmu, inst,0 ) + ARM_SMMU_GR0_sGFSR); +return IRQ_HANDLED ; +} + +static irqreturn_t nsmmu_global_fault (int irq,void *dev) +{ +int inst; +irqreturn_t irq_ret =IRQ_NONE ; +struct arm_smmu_device *smmu = dev; + +for (inst =0 ; inst <to_nvidia_smmu (smmu)->num_inst; inst++) { + irq_ret =nsmmu_global_fault_inst (irq, smmu, inst); +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + +return irq_ret; +} + +static irqreturn_t nsmmu_context_fault_bank (int irq, +struct arm_smmu_device *smmu, +int idx,int inst) +{ +u32 fsr, fsynr, cbfrsynra; +unsigned long iova; + + fsr =arm_smmu_cb_read (smmu, idx,ARM_SMMU_CB_FSR ); +if (!(fsr &ARM_SMMU_FSR_FAULT )) +return IRQ_NONE ; + + fsynr =readl_relaxed (nsmmu_page (smmu, inst, smmu->numpage + idx) + +ARM_SMMU_CB_FSYNR0 ); + iova =readq_relaxed (nsmmu_page (smmu, inst, smmu->numpage + idx) + +ARM_SMMU_CB_FAR ); + cbfrsynra =readl_relaxed (nsmmu_page (smmu, inst,1 ) + +ARM_SMMU_GR1_CBFRSYNRA (idx)); + +dev_err_ratelimited (smmu->dev, +"Unhandled context fault: fsr=0x%x, iova=0x%08lx, fsynr=0x%x, cbfrsynra=0x%x, cb=%d\n" , + fsr, iova, fsynr, cbfrsynra, idx); + +writel_relaxed (fsr,nsmmu_page (smmu, inst, smmu->numpage + idx) + +ARM_SMMU_CB_FSR ); +return IRQ_HANDLED ; +} + +static irqreturn_t nsmmu_context_fault (int irq,void *dev) +{ +int inst, idx; +irqreturn_t irq_ret =IRQ_NONE ; +struct iommu_domain *domain = dev; +struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain); +struct arm_smmu_device *smmu = smmu_domain->smmu; + +for (inst =0 ; inst <to_nvidia_smmu (smmu)->num_inst; inst++) { +/* Interrupt line shared between all context faults. +* Check for faults across all contexts. +*/ +for (idx =0 ; idx < smmu->num_context_banks; idx++) { + irq_ret =nsmmu_context_fault_bank (irq, smmu, + idx, inst); + +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + } + +return irq_ret; +} +static const struct arm_smmu_impl nvidia_smmu_impl = { .read_reg = nsmmu_read_reg, .write_reg = nsmmu_write_reg,
@@ -124,6 +222,8 @@ static const struct arm_smmu_impl nvidia_smmu_impl = { .write_reg64 = nsmmu_write_reg64, .reset = nsmmu_reset, .tlb_sync = nsmmu_tlb_sync, + .global_fault = nsmmu_global_fault, + .context_fault = nsmmu_context_fault, };struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu)
drivers/iommu/arm-smmu.c +9 -2
@@ -673,6 +673,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,enum io_pgtable_fmt fmt;struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain);struct arm_smmu_cfg *cfg = &smmu_domain->cfg; +irqreturn_t (*context_fault)(int irq,void *dev);mutex_lock (&smmu_domain->init_mutex);if (smmu_domain->smmu)
@@ -835,7 +836,9 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,* handler seeing a half-initialised domain state. */ irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; - ret =devm_request_irq (smmu->dev, irq, arm_smmu_context_fault, + context_fault = (smmu->impl && smmu->impl->context_fault) ? + smmu->impl->context_fault : arm_smmu_context_fault; + ret =devm_request_irq (smmu->dev, irq, context_fault,IRQF_SHARED ,"arm-smmu-context-fault" , domain);if (ret <0 ) { dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",
@@ -2107,6 +2110,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)struct arm_smmu_device *smmu;struct device *dev = &pdev->dev;int num_irqs, i, err; +irqreturn_t (*global_fault)(int irq,void *dev); smmu =devm_kzalloc (dev,sizeof (*smmu),GFP_KERNEL );if (!smmu) {
@@ -2193,9 +2197,12 @@ static int arm_smmu_device_probe(struct platform_device *pdev) smmu->num_context_irqs = smmu->num_context_banks; } + global_fault = (smmu->impl && smmu->impl->global_fault) ? + smmu->impl->global_fault : arm_smmu_global_fault; +for (i =0 ; i < smmu->num_global_irqs; ++i) { err =devm_request_irq (smmu->dev, smmu->irqs[i], - arm_smmu_global_fault, + global_fault,IRQF_SHARED ,"arm-smmu global fault" , smmu);
drivers/iommu/arm-smmu.h +3 -0
@@ -18,6 +18,7 @@#include <linux/io-64-nonatomic-hi-lo.h> #include <linux/io-pgtable.h> #include <linux/iommu.h> +#include <linux/irqreturn.h> #include <linux/mutex.h> #include <linux/spinlock.h> #include <linux/types.h>
@@ -387,6 +388,8 @@ struct arm_smmu_impl {void (*tlb_sync)(struct arm_smmu_device *smmu,int page,int sync,int status);int (*def_domain_type)(struct device *dev); +irqreturn_t (*global_fault)(int irq,void *dev); +irqreturn_t (*context_fault)(int irq,void *dev); };static inline void __iomem *arm_smmu_page (struct arm_smmu_device *smmu,int n)
Signature · 1 line
2.26.2
Re: [PATCH v6 3/4] iommu/arm-smmu: Add global/context fault implementation hooks
Quoted 141 lines · Krishna Reddy
Add global/context fault hooks to allow NVIDIA SMMU implementation handle faults across multiple SMMUs.
Signed-off-by Krishna Reddy <[email protected]>--- drivers/iommu/arm-smmu-nvidia.c | 100 ++++++++++++++++++++++++++++++++ drivers/iommu/arm-smmu.c | 11 +++- drivers/iommu/arm-smmu.h | 3 + 3 files changed, 112 insertions(+), 2 deletions(-)drivers/iommu/arm-smmu-nvidia.c +100
-0@@ -117,6 +117,104 @@ static int nsmmu_reset(struct arm_smmu_device *smmu)return 0 ; } +static struct arm_smmu_domain *to_smmu_domain (struct iommu_domain *dom) +{ + return container_of(dom, struct arm_smmu_domain, domain); +} + +static irqreturn_t nsmmu_global_fault_inst (int irq, +struct arm_smmu_device *smmu, +int inst) +{ +u32 gfsr, gfsynr0, gfsynr1, gfsynr2; + + gfsr =readl_relaxed (nsmmu_page (smmu, inst,0 ) + ARM_SMMU_GR0_sGFSR); + gfsynr0 =readl_relaxed (nsmmu_page (smmu, inst,0 ) + + ARM_SMMU_GR0_sGFSYNR0); + gfsynr1 =readl_relaxed (nsmmu_page (smmu, inst,0 ) + + ARM_SMMU_GR0_sGFSYNR1); + gfsynr2 =readl_relaxed (nsmmu_page (smmu, inst,0 ) + + ARM_SMMU_GR0_sGFSYNR2); + +if (!gfsr) +return IRQ_NONE ; + +dev_err_ratelimited (smmu->dev, +"Unexpected global fault, this could be serious\n" ); +dev_err_ratelimited (smmu->dev, +"\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n" , + gfsr, gfsynr0, gfsynr1, gfsynr2); + +writel_relaxed (gfsr,nsmmu_page (smmu, inst,0 ) + ARM_SMMU_GR0_sGFSR); +return IRQ_HANDLED ; +} + +static irqreturn_t nsmmu_global_fault (int irq,void *dev) +{ +int inst; +irqreturn_t irq_ret =IRQ_NONE ; +struct arm_smmu_device *smmu = dev; + +for (inst =0 ; inst <to_nvidia_smmu (smmu)->num_inst; inst++) { + irq_ret =nsmmu_global_fault_inst (irq, smmu, inst); +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + +return irq_ret; +} + +static irqreturn_t nsmmu_context_fault_bank (int irq, +struct arm_smmu_device *smmu, +int idx,int inst) +{ +u32 fsr, fsynr, cbfrsynra; +unsigned long iova; + + fsr =arm_smmu_cb_read (smmu, idx,ARM_SMMU_CB_FSR ); +if (!(fsr &ARM_SMMU_FSR_FAULT )) +return IRQ_NONE ; + + fsynr =readl_relaxed (nsmmu_page (smmu, inst, smmu->numpage + idx) + +ARM_SMMU_CB_FSYNR0 ); + iova =readq_relaxed (nsmmu_page (smmu, inst, smmu->numpage + idx) + +ARM_SMMU_CB_FAR ); + cbfrsynra =readl_relaxed (nsmmu_page (smmu, inst,1 ) + +ARM_SMMU_GR1_CBFRSYNRA (idx)); + +dev_err_ratelimited (smmu->dev, +"Unhandled context fault: fsr=0x%x, iova=0x%08lx, fsynr=0x%x, cbfrsynra=0x%x, cb=%d\n" , + fsr, iova, fsynr, cbfrsynra, idx); + +writel_relaxed (fsr,nsmmu_page (smmu, inst, smmu->numpage + idx) + +ARM_SMMU_CB_FSR ); +return IRQ_HANDLED ; +} + +static irqreturn_t nsmmu_context_fault (int irq,void *dev) +{ +int inst, idx; +irqreturn_t irq_ret =IRQ_NONE ; +struct iommu_domain *domain = dev; +struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain); +struct arm_smmu_device *smmu = smmu_domain->smmu; + +for (inst =0 ; inst <to_nvidia_smmu (smmu)->num_inst; inst++) { +/* Interrupt line shared between all context faults. +* Check for faults across all contexts. +*/ +for (idx =0 ; idx < smmu->num_context_banks; idx++) { + irq_ret =nsmmu_context_fault_bank (irq, smmu, + idx, inst); + +if (irq_ret ==IRQ_HANDLED ) +return irq_ret; + } + } + +return irq_ret; +} +static const struct arm_smmu_impl nvidia_smmu_impl = { .read_reg = nsmmu_read_reg, .write_reg = nsmmu_write_reg,@@ -124,6 +222,8 @@ static const struct arm_smmu_impl nvidia_smmu_impl = { .write_reg64 = nsmmu_write_reg64, .reset = nsmmu_reset, .tlb_sync = nsmmu_tlb_sync, + .global_fault = nsmmu_global_fault, + .context_fault = nsmmu_context_fault, };struct arm_smmu_device *nvidia_smmu_impl_init (struct arm_smmu_device *smmu)drivers/iommu/arm-smmu.c +1
-0@@ -673,6 +673,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,enum io_pgtable_fmt fmt;struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain);struct arm_smmu_cfg *cfg = &smmu_domain->cfg; +irqreturn_t (*context_fault)(int irq,void *dev);mutex_lock (&smmu_domain->init_mutex);if (smmu_domain->smmu)
drivers/iommu/arm-smmu.c static int arm_smmu_init_domain_context(struct iommu_domain *domain, · line 840
Quoted 6 lines
drivers/iommu/arm-smmu.c +2
-1@@ -835,7 +836,9 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,* handler seeing a half-initialised domain state. */ irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; - ret = devm_request_irq(smmu->dev, irq, arm_smmu_context_fault, + context_fault = (smmu->impl && smmu->impl->context_fault) ? + smmu->impl->context_fault : arm_smmu_context_fault;
A simpler way might have been to assign arm_smmu_context_fault to all implementations. That way we wouldn't have to perform this check here and instead just always using smmu->impl->context_fault.
Quoted 13 lines
drivers/iommu/arm-smmu.c +2
-0+ ret =devm_request_irq (smmu->dev, irq, context_fault,IRQF_SHARED ,"arm-smmu-context-fault" , domain);if (ret <0 ) { dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",@@ -2107,6 +2110,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)struct arm_smmu_device *smmu;struct device *dev = &pdev->dev;int num_irqs, i, err; +irqreturn_t (*global_fault)(int irq,void *dev); smmu =devm_kzalloc (dev,sizeof (*smmu),GFP_KERNEL );if (!smmu) {
drivers/iommu/arm-smmu.c static int arm_smmu_device_probe(struct platform_device *pdev) · line 2202
Quoted 6 lines
drivers/iommu/arm-smmu.c +3
-0@@ -2193,9 +2197,12 @@ static int arm_smmu_device_probe(struct platform_device *pdev) smmu->num_context_irqs = smmu->num_context_banks; } + global_fault = (smmu->impl && smmu->impl->global_fault) ? + smmu->impl->global_fault : arm_smmu_global_fault; +
Same as above.
Thierry
Robin Murphy
reviewerRe: [PATCH v6 3/4] iommu/arm-smmu: Add global/context fault implementation hooks
On 2020-06-23 09:36, Thierry Reding wrote:
[...]
Quoted 10 lines
Quoted 10 lines
drivers/iommu/arm-smmu.c +1
-0@@ -673,6 +673,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,enum io_pgtable_fmt fmt;struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain);struct arm_smmu_cfg *cfg = &smmu_domain->cfg; +irqreturn_t (*context_fault)(int irq,void *dev);
Quoted 13 lines
drivers/iommu/arm-smmu.c static int arm_smmu_init_domain_context(struct iommu_domain *domain, · line 840
Quoted 9 lines
drivers/iommu/arm-smmu.c +2
-1mutex_lock (&smmu_domain->init_mutex);if (smmu_domain->smmu)@@ -835,7 +836,9 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,* handler seeing a half-initialised domain state. */ irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; - ret = devm_request_irq(smmu->dev, irq, arm_smmu_context_fault, + context_fault = (smmu->impl && smmu->impl->context_fault) ? + smmu->impl->context_fault : arm_smmu_context_fault;A simpler way might have been to assign
arm_smmu_context_faultto all implementations. That way we wouldn't have to perform this check here and instead just always usingsmmu->impl->context_fault.
But smmu->impl can still be NULL...
Everything in impl, including the presence of impl itself, is optional, so the notion of overriding a default with the same default doesn't really make much sense, and would go against the pattern everywhere else.
Robin.
Quoted 24 lines
Quoted 13 lines
drivers/iommu/arm-smmu.c +2
-0+ ret =devm_request_irq (smmu->dev, irq, context_fault,IRQF_SHARED ,"arm-smmu-context-fault" , domain);if (ret <0 ) { dev_err(smmu->dev, "failed to request context IRQ %d (%u)\n",@@ -2107,6 +2110,7 @@ static int arm_smmu_device_probe(struct platform_device *pdev)struct arm_smmu_device *smmu;struct device *dev = &pdev->dev;int num_irqs, i, err; +irqreturn_t (*global_fault)(int irq,void *dev); smmu =devm_kzalloc (dev,sizeof (*smmu),GFP_KERNEL );if (!smmu) {drivers/iommu/arm-smmu.c static int arm_smmu_device_probe(struct platform_device *pdev) · line 2202
Quoted 6 lines
drivers/iommu/arm-smmu.c +3
-0@@ -2193,9 +2197,12 @@ static int arm_smmu_device_probe(struct platform_device *pdev) smmu->num_context_irqs = smmu->num_context_banks; } + global_fault = (smmu->impl && smmu->impl->global_fault) ? + smmu->impl->global_fault : arm_smmu_global_fault; +Same as above.
Thierry
Re: [PATCH v6 3/4] iommu/arm-smmu: Add global/context fault implementation hooks
Quoted 25 lines · Robin Murphy
On 2020-06-23 09:36, Thierry Reding wrote:
[...]Quoted 14 lines
Quoted 14 lines
drivers/iommu/arm-smmu.c +1
-0@@ -673,6 +673,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,enum io_pgtable_fmt fmt;struct arm_smmu_domain *smmu_domain =to_smmu_domain (domain);struct arm_smmu_cfg *cfg = &smmu_domain->cfg; +irqreturn_t (*context_fault)(int irq,void *dev);mutex_lock (&smmu_domain->init_mutex);if (smmu_domain->smmu)@@ -835,7 +836,9 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,* handler seeing a half-initialised domain state. */ Quoted 8 lines
drivers/iommu/arm-smmu.c static int arm_smmu_init_domain_context(struct iommu_domain *domain, · line 840
Quoted 4 lines
drivers/iommu/arm-smmu.c +2
-1irq = smmu->irqs[smmu->num_global_irqs + cfg->irptndx]; - ret = devm_request_irq(smmu->dev, irq, arm_smmu_context_fault, + context_fault = (smmu->impl && smmu->impl->context_fault) ? + smmu->impl->context_fault : arm_smmu_context_fault;A simpler way might have been to assign
arm_smmu_context_faultto all implementations. That way we wouldn't have to perform this check here and instead just always usingsmmu->impl->context_fault.
Quoted 5 lines
But
smmu->implcan still be NULL...Everything in impl, including the presence of impl itself, is optional, so the notion of overriding a default with the same default doesn't really make much sense, and would go against the pattern everywhere else.
True. I had assumed that every implementation would set smmu->impl anyway, in which case there'd be little reason to use these default fallbacks since each implementation could simply directly refer to the exact implementation that it wants.
Perhaps the above could be made a bit more palatable by using a standard if/else rather than the ternary operator? That would also more closely match the pattern elsewhere.
Thierry
PATCH v6 4/4 iommu/arm-smmu-nvidia: fix the warning reported by kbuild test robot
In reply to Krishna ReddyQuoted 2 lines
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c:151:33: sparse: sparse: cast removes address space '<asn:2>' of expression
--- drivers/iommu/arm-smmu-nvidia.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/arm-smmu-nvidia.c +1 -1
@@ -248,7 +248,7 @@ struct arm_smmu_device *nvidia_smmu_impl_init(struct arm_smmu_device *smmu)break ; nsmmu->bases[i] =devm_ioremap_resource (dev, res);if (IS_ERR (nsmmu->bases[i])) -return (struct arm_smmu_device *)nsmmu->bases[i]; +return ERR_CAST (nsmmu->bases[i]); nsmmu->num_inst++; }
Signature · 1 line
2.26.2
Re: [PATCH v6 4/4] iommu/arm-smmu-nvidia: fix the warning reported by kbuild test robot
Quoted 8 lines · Krishna Reddy
Quoted 2 lines
Quoted 2 lines
drivers/iommu/arm-smmu-nvidia.c:151:33: sparse: sparse: cast removes address space '<asn:2>' of expressionReported-by kbuild test robot <[email protected]>Signed-off-by Krishna Reddy <[email protected]>--- drivers/iommu/arm-smmu-nvidia.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
This should be folded into the patch that introduced this error.
Thierry