Skip to contentSkip to conversation

arm-smmu: Misc changes/Calxeda ECX-2000 support

PATCH New Andreas Herrmann 3 participants 43 messages Sep 26, 2013 – Oct 8, 2013

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

cover arm-smmu: Misc changes/Calxeda ECX-2000 support

Hi,

Following a new patch set to fix some issues, that I've seen when using arm-smmu driver with MMU-400 and to enable support for SMMUs on Calxeda ECX-2000.

Patches are rebased on Joerg's latest joro/next branch (as of today). I hope that I have addressed all (most) comments from the last submission for following patches:

[PATCH 1/9] iommu/arm-smmu: Switch to arch_initcall for driver ... [PATCH 2/9] iommu/arm-smmu: Calculate SMMU_CB_BASE from smmu ... [PATCH 3/9] ARM: dma-mapping: Always pass proper prot flags to ... [PATCH 4/9] iommu/arm-smmu: Check for num_context_irqs > 0 to avoid ... [PATCH 5/9] iommu/arm-smmu: Clear global and context bank fault ...

This time I've added patches for SMMU support on Calxeda ECX-2000. Unfortunately this requires a quirk in the arm-smmu driver (and that is the ugly part):

[PATCH 6/9] iommu/arm-smmu: Support buggy implemenations where all
config accesses are secure

I still kept the patch (slightly modified) to introduce a function to isolate devices. I also merged the command line stuff into that patch because it mainly belongs to it:

[PATCH 7/9] iommu/arm-smmu: Add function that conditionally isolates
all masters of all SMMUs

(Maybe it's better to use a bus notifier to attach/detach devices instead. But I just started to look into this and code is not yet ready+tested.)

Also I think we need a default handler for context faults Otherwise we won't notice when something goes wrong:

[PATCH 8/9] iommu/arm-smmu: Introduce a default fault handler

And finally here's the dts-adaption to enable SMMUs

[PATCH 9/9] ARM: dts: Add nodes for SMMUs on Calxeda ECX-2000

Would be nice if the noncontroversial patches (1-5?) could be applied. (Esp. in case there are longer discussions about the rest.)

Comments welcome.

Thanks,

Andreas

#1 lore raw

PATCH 1/9 iommu/arm-smmu: Switch to arch_initcall for driver registration

This should ensure that arm-smmu is initialized before other drivers start handling devices that propably need smmu support.

Also remove module_exit function as we most likely never want to unload this driver.

---
 drivers/iommu/arm-smmu.c |    8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +1 -7
index 181c9ba..6808577 100644
@@ -1976,13 +1976,7 @@ static int __init arm_smmu_init(void)
 	return 0;
 }
 
-static void __exit arm_smmu_exit(void)
-{
-	return platform_driver_unregister(&arm_smmu_driver);
-}
-
-module_init(arm_smmu_init);
-module_exit(arm_smmu_exit);
+arch_initcall(arm_smmu_init);
 
 MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
 MODULE_AUTHOR("Will Deacon <[email protected]>");
Signature · 1 line
1.7.9.5
#2 lore raw

PATCH 1/9 iommu/arm-smmu: Switch to arch_initcall for driver registration

Quoted 5 lines · Andreas Herrmann

This should ensure that arm-smmu is initialized before other drivers start handling devices that propably need smmu support.

Also remove module_exit function as we most likely never want to unload this driver.

Doesn't hurt to leave the exit function there though, right?

Quoted 11 lines
---
 drivers/iommu/arm-smmu.c |    8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +0 -0
index 181c9ba..6808577 100644
@@ -1976,13 +1976,7 @@ static int __init arm_smmu_init(void)
 	return 0;

drivers/iommu/arm-smmu.c static int __init arm_smmu_init(void) · line 1979

Quoted 10 lines
drivers/iommu/arm-smmu.c +1 -7
 }
 
-static void __exit arm_smmu_exit(void)
-{
-	return platform_driver_unregister(&arm_smmu_driver);
-}
-
-module_init(arm_smmu_init);
-module_exit(arm_smmu_exit);
+arch_initcall(arm_smmu_init);

Why not subsys_initcall, like the other ARM IOMMUs?

Will

#3 lore raw

PATCH 1/9 iommu/arm-smmu: Switch to arch_initcall for driver registration

Quoted 8 lines · Will Deacon
Quoted 5 lines · Andreas Herrmann

This should ensure that arm-smmu is initialized before other drivers start handling devices that propably need smmu support.

Also remove module_exit function as we most likely never want to unload this driver.

Doesn't hurt to leave the exit function there though, right?

Yes.

Quoted 13 lines
Quoted 13 lines
---
 drivers/iommu/arm-smmu.c |    8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +0 -0
index 181c9ba..6808577 100644
@@ -1976,13 +1976,7 @@ static int __init arm_smmu_init(void)
 	return 0;
 }
 
Quoted 10 lines

drivers/iommu/arm-smmu.c static int __init arm_smmu_init(void) · line 1979

Quoted 8 lines
drivers/iommu/arm-smmu.c +1 -7
-static void __exit arm_smmu_exit(void)
-{
-	return platform_driver_unregister(&arm_smmu_driver);
-}
-
-module_init(arm_smmu_init);
-module_exit(arm_smmu_exit);
+arch_initcall(arm_smmu_init);

Why not subsys_initcall, like the other ARM IOMMUs?

No specific reason. I just tried to load the driver at the earliest point during boot.

I just retested successful with subsys_initcall. The initialization sequence looks fine (and yes, arch_initcall is overkill).

Will adapt this.

Andreas

#4 lore raw

PATCH 1/1 iommu/arm-smmu: Switch to subsys_initcall for driver registration

This should ensure that arm-smmu is initialized before other drivers start handling devices that propably need smmu support.

---
 drivers/iommu/arm-smmu.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/arm-smmu.c +1 -1
index 181c9ba..548f5d1 100644
@@ -1981,7 +1981,7 @@ static void __exit arm_smmu_exit(void)
 	return platform_driver_unregister(&arm_smmu_driver);
 }
 
-module_init(arm_smmu_init);
+subsys_initcall(arm_smmu_init);
 module_exit(arm_smmu_exit);
 
 MODULE_DESCRIPTION("IOMMU API for ARM architected SMMU implementations");
Signature · 1 line
1.7.9.5
#5 lore raw

PATCH 2/9 iommu/arm-smmu: Calculate SMMU_CB_BASE from smmu register values

In reply to Andreas Herrmann

Currently it is derived from smmu resource size. In case of a mismatchin between the two calculations trust DT more than register values and overwrite cb_base.

---
 drivers/iommu/arm-smmu.c |   17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
drivers/iommu/arm-smmu.c +15 -2
index 6808577..4307fbc 100644
@@ -207,7 +207,7 @@
 #define CBA2R_RW64_64BIT		(1 << 0)
 
 /* Translation context bank */
-#define ARM_SMMU_CB_BASE(smmu)		((smmu)->base + ((smmu)->size >> 1))
+#define ARM_SMMU_CB_BASE(smmu)		((smmu)->cb_base)
 #define ARM_SMMU_CB(smmu, n)		((n) * (smmu)->pagesize)
 
 #define ARM_SMMU_CB_SCTLR		0x0
@@ -339,6 +339,7 @@ struct arm_smmu_device {
 	struct device_node		*parent_of_node;
 
 	void __iomem			*base;
+	void __iomem			*cb_base;
 	unsigned long			size;
 	unsigned long			pagesize;
 
@@ -1622,6 +1623,7 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	unsigned long size;
 	void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
 	u32 id;
+	unsigned long t;
 
 	dev_notice(smmu->dev, "probing hardware configuration...\n");
 
@@ -1702,12 +1704,23 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 
 	/* Check that we ioremapped enough */
 	size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
-	size *= (smmu->pagesize << 1);
+	size *= smmu->pagesize;
+	smmu->cb_base = smmu->base + size;
+	size *= 2;
+
 	if (smmu->size < size)
 		dev_warn(smmu->dev,
 			 "device is 0x%lx bytes but only mapped 0x%lx!\n",
 			 size, smmu->size);
 
+	t = (unsigned long) smmu->base + (smmu->size >> 1);
+	if ((unsigned long)smmu->cb_base != t) {
+		dev_warn(smmu->dev, "address space mismatch, "
+			"overwriting cb_base (old: 0x%lx, new: 0x%lx)\n",
+			(unsigned long) smmu->cb_base, t);
+		smmu->cb_base = (void *) t;
+	}
+
 	smmu->num_s2_context_banks = (id >> ID1_NUMS2CB_SHIFT) &
 				      ID1_NUMS2CB_MASK;
 	smmu->num_context_banks = (id >> ID1_NUMCB_SHIFT) & ID1_NUMCB_MASK;
Signature · 1 line
1.7.9.5
#6 lore raw

PATCH 2/9 iommu/arm-smmu: Calculate SMMU_CB_BASE from smmu register values

Hi Andreas,

Quoted 3 lines · Andreas Herrmann

Currently it is derived from smmu resource size. In case of a mismatchin between the two calculations trust DT more than register values and overwrite cb_base.

I thought the driver already favoured the DT?

Quoted 12 lines
drivers/iommu/arm-smmu.c +4 -1
@@ -1702,12 +1704,23 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 
 	/* Check that we ioremapped enough */
 	size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
-	size *= (smmu->pagesize << 1);
+	size *= smmu->pagesize;
+	smmu->cb_base = smmu->base + size;
+	size *= 2;
+
 	if (smmu->size < size)
 		dev_warn(smmu->dev,
 			 "device is 0x%lx bytes but only mapped 0x%lx!\n",

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1723

Quoted 10 lines
drivers/iommu/arm-smmu.c +8 -0
 			 size, smmu->size);
 
+	t = (unsigned long) smmu->base + (smmu->size >> 1);
+	if ((unsigned long)smmu->cb_base != t) {
+		dev_warn(smmu->dev, "address space mismatch, "
+			"overwriting cb_base (old: 0x%lx, new: 0x%lx)\n",
+			(unsigned long) smmu->cb_base, t);
+		smmu->cb_base = (void *) t;
+	}
+

I expect I'm just being slow here (only one coffee in), but I can't see what this gets us over the current use of resource_size (which goes and uses the DT).

Will

#7 lore raw

PATCH 2/9 iommu/arm-smmu: Calculate SMMU_CB_BASE from smmu register values

Quoted 23 lines · Will Deacon

Hi Andreas,

Quoted 3 lines · Andreas Herrmann

Currently it is derived from smmu resource size. In case of a mismatchin between the two calculations trust DT more than register values and overwrite cb_base.

I thought the driver already favoured the DT?

Quoted 14 lines
drivers/iommu/arm-smmu.c +4 -1
@@ -1702,12 +1704,23 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 
 	/* Check that we ioremapped enough */
 	size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
-	size *= (smmu->pagesize << 1);
+	size *= smmu->pagesize;
+	smmu->cb_base = smmu->base + size;
+	size *= 2;
+
 	if (smmu->size < size)
 		dev_warn(smmu->dev,
 			 "device is 0x%lx bytes but only mapped 0x%lx!\n",
 			 size, smmu->size);
 
Quoted 12 lines

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1723

Quoted 8 lines
drivers/iommu/arm-smmu.c +8 -0
+	t = (unsigned long) smmu->base + (smmu->size >> 1);
+	if ((unsigned long)smmu->cb_base != t) {
+		dev_warn(smmu->dev, "address space mismatch, "
+			"overwriting cb_base (old: 0x%lx, new: 0x%lx)\n",
+			(unsigned long) smmu->cb_base, t);
+		smmu->cb_base = (void *) t;
+	}
+

I expect I'm just being slow here (only one coffee in), but I can't see what this gets us over the current use of resource_size (which goes and uses the DT).

On balance it adds a warning if there is an inconsistency between the resource size and the relevant registers describing the SMMU address space.

If there is no such warning you might use a broken DT (with wrong resource size) causing weird malfunction (because cb_base might be wrong) and it will take some effort to root cause this.

But if you see such a warning it's clear what to check first.

I admit the commit message is misleading (somehow inherited from the previous patch version) and maybe this kind of consistency check should not be combined with cb_base calculation but moved into a separate function called from arm_smmu_device_cfg_probe.

Andreas

#8 lore raw

PATCH 2/9 iommu/arm-smmu: Calculate SMMU_CB_BASE from smmu register values

Quoted 37 lines · Andreas Herrmann
Quoted 23 lines · Will Deacon

Hi Andreas,

Quoted 3 lines · Andreas Herrmann

Currently it is derived from smmu resource size. In case of a mismatchin between the two calculations trust DT more than register values and overwrite cb_base.

I thought the driver already favoured the DT?

Quoted 14 lines
drivers/iommu/arm-smmu.c +4 -1
@@ -1702,12 +1704,23 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 
 	/* Check that we ioremapped enough */
 	size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
-	size *= (smmu->pagesize << 1);
+	size *= smmu->pagesize;
+	smmu->cb_base = smmu->base + size;
+	size *= 2;
+
 	if (smmu->size < size)
 		dev_warn(smmu->dev,
 			 "device is 0x%lx bytes but only mapped 0x%lx!\n",
 			 size, smmu->size);
 
Quoted 12 lines

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1723

Quoted 8 lines
drivers/iommu/arm-smmu.c +8 -0
+	t = (unsigned long) smmu->base + (smmu->size >> 1);
+	if ((unsigned long)smmu->cb_base != t) {
+		dev_warn(smmu->dev, "address space mismatch, "
+			"overwriting cb_base (old: 0x%lx, new: 0x%lx)\n",
+			(unsigned long) smmu->cb_base, t);
+		smmu->cb_base = (void *) t;
+	}
+

I expect I'm just being slow here (only one coffee in), but I can't see what this gets us over the current use of resource_size (which goes and uses the DT).

Quoted 3 lines

On balance it adds a warning if there is an inconsistency between the resource size and the relevant registers describing the SMMU address space.

Well, we should already print the "device is 0x%lx bytes but only mapped 0x%lx!" message, which I think is enough to go and figure out what happened.

Will

#9 lore raw

PATCH 2/9 iommu/arm-smmu: Calculate SMMU_CB_BASE from smmu register values

Quoted 38 lines · Will Deacon
Quoted 37 lines · Andreas Herrmann
Quoted 35 lines · Will Deacon

Hi Andreas,

Quoted 3 lines · Andreas Herrmann

Currently it is derived from smmu resource size. In case of a mismatchin between the two calculations trust DT more than register values and overwrite cb_base.

I thought the driver already favoured the DT?

Quoted 12 lines
drivers/iommu/arm-smmu.c +4 -1
@@ -1702,12 +1704,23 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 
 	/* Check that we ioremapped enough */
 	size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
-	size *= (smmu->pagesize << 1);
+	size *= smmu->pagesize;
+	smmu->cb_base = smmu->base + size;
+	size *= 2;
+
 	if (smmu->size < size)
 		dev_warn(smmu->dev,
 			 "device is 0x%lx bytes but only mapped 0x%lx!\n",

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1723

Quoted 10 lines
drivers/iommu/arm-smmu.c +8 -0
 			 size, smmu->size);
 
+	t = (unsigned long) smmu->base + (smmu->size >> 1);
+	if ((unsigned long)smmu->cb_base != t) {
+		dev_warn(smmu->dev, "address space mismatch, "
+			"overwriting cb_base (old: 0x%lx, new: 0x%lx)\n",
+			(unsigned long) smmu->cb_base, t);
+		smmu->cb_base = (void *) t;
+	}
+

I expect I'm just being slow here (only one coffee in), but I can't see what this gets us over the current use of resource_size (which goes and uses the DT).

Quoted 6 lines
Quoted 3 lines

On balance it adds a warning if there is an inconsistency between the resource size and the relevant registers describing the SMMU address space.

Well, we should already print the "device is 0x%lx bytes but only mapped 0x%lx!" message, which I think is enough to go and figure out what happened.

No, you can map a larger region and still wrongly calculate cb_base w/o this warning.
So the required check is something like

         /* Check for mismatch between SMMU address space size and size of mapped region */
         size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
         size *= (smmu->pagesize << 1);
         if (smmu->size != size)
                 dev_warn(smmu->dev,
                          "SMMU_GLOBAL_SIZE (0x%lx) differs from mapped "
			  region size (0x%lx)!\n", size, smmu->size);

Andreas

#10 lore raw

PATCH 2/9 iommu/arm-smmu: Calculate SMMU_CB_BASE from smmu register values

Quoted 6 lines · Andreas Herrmann
Quoted 2 lines · Will Deacon

Well, we should already print the "device is 0x%lx bytes but only mapped 0x%lx!" message, which I think is enough to go and figure out what happened.

No, you can map a larger region and still wrongly calculate cb_base w/o this warning.

Gotcha.

Quoted 9 lines

So the required check is something like

         /* Check for mismatch between SMMU address space size and size of mapped region */
         size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
         size *= (smmu->pagesize << 1);
         if (smmu->size != size)
                 dev_warn(smmu->dev,
                          "SMMU_GLOBAL_SIZE (0x%lx) differs from mapped "
			  region size (0x%lx)!\n", size, smmu->size);

Sure, tightening up that check sounds like a good idea. Looking forward to the next version of the patches!

(I need some more time to think about that secure config access patch :).

Will

#11 lore raw

PATCH 1/1 iommu/arm-smmu: Refine check for proper size of mapped region

There is already a check to print a warning if the size of SMMU address space (calculated from SMMU register values) is greater than the size of the mapped memory region (e.g. passed via DT to the driver).

Adapt this check to print also a warning in case the mapped region is larger than the SMMU address space.

Such a mismatch could be intentional (to fix wrong register values). If its not intentional (e.g. due to wrong DT information) this will very likely cause a malfunction of the driver as SMMU_CB_BASE is derived from the size of the mapped region. The warning helps to identify the root cause in this case.

---
 drivers/iommu/arm-smmu.c |    9 ++++-----
 1 file changed, 4 insertions(+), 5 deletions(-)

FYI, posted the patch to the mail thread for reference. I'll send updated patches as a series asap.

Andreas

drivers/iommu/arm-smmu.c +4 -5
index 548f5d1..d19676c 100644
@@ -1700,13 +1700,12 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	id = readl_relaxed(gr0_base + ARM_SMMU_GR0_ID1);
 	smmu->pagesize = (id & ID1_PAGESIZE) ? SZ_64K : SZ_4K;
 
-	/* Check that we ioremapped enough */
+	/* Check for size mismatch of SMMU address space from mapped region */
 	size = 1 << (((id >> ID1_NUMPAGENDXB_SHIFT) & ID1_NUMPAGENDXB_MASK) + 1);
 	size *= (smmu->pagesize << 1);
-	if (smmu->size < size)
-		dev_warn(smmu->dev,
-			 "device is 0x%lx bytes but only mapped 0x%lx!\n",
-			 size, smmu->size);
+	if (smmu->size != size)
+		dev_warn(smmu->dev, "SMMU address space size (0x%lx) differs "
+			"from mapped region size (0x%lx)!\n", size, smmu->size);
 
 	smmu->num_s2_context_banks = (id >> ID1_NUMS2CB_SHIFT) &
 				      ID1_NUMS2CB_MASK;
Signature · 1 line
1.7.9.5
#12 lore raw

PATCH 3/9 ARM: dma-mapping: Always pass proper prot flags to iommu_map()

In reply to Andreas Herrmann

... otherwise it is impossible for the low level iommu driver to figure out which pte flags should be used.

In __map_sg_chunk we can derive the flags from dma_data_direction.

In __iommu_create_mapping we should treat the memory like DMA_BIDIRECTIONAL and pass both IOMMU_READ and IOMMU_WRITE to iommu_map.
__iommu_create_mapping is used during dma_alloc_coherent (via arm_iommu_alloc_attrs). AFAIK dma_alloc_coherent is responsible for allocation _and_ mapping. I think this implies that access to the mapped pages should be allowed.

---
 arch/arm/mm/dma-mapping.c |   43 ++++++++++++++++++++++++++++---------------
 1 file changed, 28 insertions(+), 15 deletions(-)
arch/arm/mm/dma-mapping.c +28 -15
index f5e1a84..1272ed2 100644
@@ -1232,7 +1232,8 @@ __iommu_create_mapping(struct device *dev, struct page **pages, size_t size)
 				break;
 
 		len = (j - i) << PAGE_SHIFT;
-		ret = iommu_map(mapping->domain, iova, phys, len, 0);
+		ret = iommu_map(mapping->domain, iova, phys, len,
+				IOMMU_READ|IOMMU_WRITE);
 		if (ret < 0)
 			goto fail;
 		iova += len;
@@ -1431,6 +1432,27 @@ static int arm_iommu_get_sgtable(struct device *dev, struct sg_table *sgt,
 					 GFP_KERNEL);
 }
 
+static int __dma_direction_to_prot(enum dma_data_direction dir)
+{
+	int prot;
+
+	switch (dir) {
+	case DMA_BIDIRECTIONAL:
+		prot = IOMMU_READ | IOMMU_WRITE;
+		break;
+	case DMA_TO_DEVICE:
+		prot = IOMMU_READ;
+		break;
+	case DMA_FROM_DEVICE:
+		prot = IOMMU_WRITE;
+		break;
+	default:
+		prot = 0;
+	}
+
+	return prot;
+}
+
 /*
  * Map a part of the scatter-gather list into contiguous io address space
  */
@@ -1444,6 +1466,7 @@ static int __map_sg_chunk(struct device *dev, struct scatterlist *sg,
 	int ret = 0;
 	unsigned int count;
 	struct scatterlist *s;
+	int prot;
 
 	size = PAGE_ALIGN(size);
 	*handle = DMA_ERROR_CODE;
@@ -1460,7 +1483,9 @@ static int __map_sg_chunk(struct device *dev, struct scatterlist *sg,
 			!dma_get_attr(DMA_ATTR_SKIP_CPU_SYNC, attrs))
 			__dma_page_cpu_to_dev(sg_page(s), s->offset, s->length, dir);
 
-		ret = iommu_map(mapping->domain, iova, phys, len, 0);
+		prot = __dma_direction_to_prot(dir);
+
+		ret = iommu_map(mapping->domain, iova, phys, len, prot);
 		if (ret < 0)
 			goto fail;
 		count += len >> PAGE_SHIFT;
@@ -1665,19 +1690,7 @@ static dma_addr_t arm_coherent_iommu_map_page(struct device *dev, struct page *p
 	if (dma_addr == DMA_ERROR_CODE)
 		return dma_addr;
 
-	switch (dir) {
-	case DMA_BIDIRECTIONAL:
-		prot = IOMMU_READ | IOMMU_WRITE;
-		break;
-	case DMA_TO_DEVICE:
-		prot = IOMMU_READ;
-		break;
-	case DMA_FROM_DEVICE:
-		prot = IOMMU_WRITE;
-		break;
-	default:
-		prot = 0;
-	}
+	prot = __dma_direction_to_prot(dir);
 
 	ret = iommu_map(mapping->domain, dma_addr, page_to_phys(page), len, prot);
 	if (ret < 0)
Signature · 1 line
1.7.9.5
#13 lore raw

PATCH 3/9 ARM: dma-mapping: Always pass proper prot flags to iommu_map()

Quoted 13 lines · Andreas Herrmann

... otherwise it is impossible for the low level iommu driver to figure out which pte flags should be used.

In __map_sg_chunk we can derive the flags from dma_data_direction.

In __iommu_create_mapping we should treat the memory like DMA_BIDIRECTIONAL and pass both IOMMU_READ and IOMMU_WRITE to iommu_map.
__iommu_create_mapping is used during dma_alloc_coherent (via arm_iommu_alloc_attrs). AFAIK dma_alloc_coherent is responsible for allocation _and_ mapping. I think this implies that access to the mapped pages should be allowed.

Quoted 2 lines

This one should go via the dma-mapping tree.

Will

#14 lore raw

PATCH 3/9 ARM: dma-mapping: Always pass proper prot flags to iommu_map()

In reply to Andreas Herrmann

Hello,

Quoted 13 lines · Andreas Herrmann

... otherwise it is impossible for the low level iommu driver to figure out which pte flags should be used.

In __map_sg_chunk we can derive the flags from dma_data_direction.

In __iommu_create_mapping we should treat the memory like DMA_BIDIRECTIONAL and pass both IOMMU_READ and IOMMU_WRITE to iommu_map.
__iommu_create_mapping is used during dma_alloc_coherent (via arm_iommu_alloc_attrs). AFAIK dma_alloc_coherent is responsible for allocation _and_ mapping. I think this implies that access to the mapped pages should be allowed.

Quoted 2 lines

Thanks pointing the issue and preparing the patch. I will push it to the dma-mapping fixes branch.

Quoted 76 lines
---
  arch/arm/mm/dma-mapping.c |   43 ++++++++++++++++++++++++++++---------------
  1 file changed, 28 insertions(+), 15 deletions(-)
arch/arm/mm/dma-mapping.c +27 -9
index f5e1a84..1272ed2 100644
@@ -1232,7 +1232,8 @@ __iommu_create_mapping(struct device *dev, struct page **pages, size_t size)
  				break;
 
  		len = (j - i) << PAGE_SHIFT;
-		ret = iommu_map(mapping->domain, iova, phys, len, 0);
+		ret = iommu_map(mapping->domain, iova, phys, len,
+				IOMMU_READ|IOMMU_WRITE);
  		if (ret < 0)
  			goto fail;
  		iova += len;
@@ -1431,6 +1432,27 @@ static int arm_iommu_get_sgtable(struct device *dev, struct sg_table *sgt,
  					 GFP_KERNEL);
  }
 
+static int __dma_direction_to_prot(enum dma_data_direction dir)
+{
+	int prot;
+
+	switch (dir) {
+	case DMA_BIDIRECTIONAL:
+		prot = IOMMU_READ | IOMMU_WRITE;
+		break;
+	case DMA_TO_DEVICE:
+		prot = IOMMU_READ;
+		break;
+	case DMA_FROM_DEVICE:
+		prot = IOMMU_WRITE;
+		break;
+	default:
+		prot = 0;
+	}
+
+	return prot;
+}
+
  /*
   * Map a part of the scatter-gather list into contiguous io address space
   */
@@ -1444,6 +1466,7 @@ static int __map_sg_chunk(struct device *dev, struct scatterlist *sg,
  	int ret = 0;
  	unsigned int count;
  	struct scatterlist *s;
+	int prot;
 
  	size = PAGE_ALIGN(size);
  	*handle = DMA_ERROR_CODE;
@@ -1460,7 +1483,9 @@ static int __map_sg_chunk(struct device *dev, struct scatterlist *sg,
  			!dma_get_attr(DMA_ATTR_SKIP_CPU_SYNC, attrs))
  			__dma_page_cpu_to_dev(sg_page(s), s->offset, s->length, dir);
 
-		ret = iommu_map(mapping->domain, iova, phys, len, 0);
+		prot = __dma_direction_to_prot(dir);
+
+		ret = iommu_map(mapping->domain, iova, phys, len, prot);
  		if (ret < 0)
  			goto fail;
  		count += len >> PAGE_SHIFT;
@@ -1665,19 +1690,7 @@ static dma_addr_t arm_coherent_iommu_map_page(struct device *dev, struct page *p
  	if (dma_addr == DMA_ERROR_CODE)
  		return dma_addr;
 
-	switch (dir) {
-	case DMA_BIDIRECTIONAL:
-		prot = IOMMU_READ | IOMMU_WRITE;
-		break;
-	case DMA_TO_DEVICE:
-		prot = IOMMU_READ;
-		break;

arch/arm/mm/dma-mapping.c static dma_addr_t arm_coherent_iommu_map_page(struct device *dev, struct page *p · line 1696

Quoted 10 lines
arch/arm/mm/dma-mapping.c +1 -6
-	case DMA_FROM_DEVICE:
-		prot = IOMMU_WRITE;
-		break;
-	default:
-		prot = 0;
-	}
+	prot = __dma_direction_to_prot(dir);
 
  	ret = iommu_map(mapping->domain, dma_addr, page_to_phys(page), len, prot);
  	if (ret < 0)

Best regards

Signature · 2 lines
Marek Szyprowski
Samsung R&D Institute Poland
#15 lore raw

PATCH 4/9 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

In reply to Andreas Herrmann

With the right (or wrong;-) definition of v1 SMMU node in DTB it is possible to trigger a division by zero in arm_smmu_init_domain_context (if number of context irqs is 0):

       if (smmu->version == 1) {
               root_cfg->irptndx = atomic_inc_return(&smmu->irptndx);
 =>            root_cfg->irptndx %= smmu->num_context_irqs;
       } else {

Avoid this by checking for num_context_irqs > 0 when probing for SMMU devices.

Rationale: Assuming that at least one context bank for non-secure usage is provided per SMMU, it follows (from ARM SMMU Architecture Spec) that at least one context interrupt must be available.

Also remove the line of code that derived num_context_irqs from num_irqs and num_global_irqs. If DT is wrong and interrupt property contains less interrupts than num_global_irqs this would set num_context_irqs to a big u32 value which most likely causes trouble in other parts of the driver.

---
 drivers/iommu/arm-smmu.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/iommu/arm-smmu.c +5 -1
index 4307fbc..de9dd60 100644
@@ -1822,7 +1822,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 			 num_irqs, smmu->num_global_irqs);
 		smmu->num_global_irqs = num_irqs;
 	}
-	smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;
+
+	if (!smmu->num_context_irqs) {
+		dev_err(dev, "no context interrupt specified in DT\n");
+		return -ENODEV;
+	}
 
 	smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
 				  GFP_KERNEL);
Signature · 1 line
1.7.9.5
#16 lore raw

PATCH 4/9 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

Quoted 9 lines · Andreas Herrmann

With the right (or wrong;-) definition of v1 SMMU node in DTB it is possible to trigger a division by zero in arm_smmu_init_domain_context (if number of context irqs is 0):

       if (smmu->version == 1) {
               root_cfg->irptndx = atomic_inc_return(&smmu->irptndx);
 =>            root_cfg->irptndx %= smmu->num_context_irqs;
       } else {
Quoted 6 lines

Avoid this by checking for num_context_irqs > 0 when probing for SMMU devices.

Rationale: Assuming that at least one context bank for non-secure usage is provided per SMMU, it follows (from ARM SMMU Architecture Spec) that at least one context interrupt must be available.

One problem with this reasoning is that the interrupt line might just not be wired up to the GIC, despite existing on the SMMU. Still, we needn't solve that now (let's wait for somebody to build it first...).

Quoted 16 lines

Also remove the line of code that derived num_context_irqs from num_irqs and num_global_irqs. If DT is wrong and interrupt property contains less interrupts than num_global_irqs this would set num_context_irqs to a big u32 value which most likely causes trouble in other parts of the driver.

---
 drivers/iommu/arm-smmu.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/iommu/arm-smmu.c +0 -0
index 4307fbc..de9dd60 100644

drivers/iommu/arm-smmu.c static int arm_smmu_device_dt_probe(struct platform_device *pdev) · line 1825

Quoted 4 lines
drivers/iommu/arm-smmu.c +0 -1
@@ -1822,7 +1822,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 			 num_irqs, smmu->num_global_irqs);
 		smmu->num_global_irqs = num_irqs;
 	}
-	smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;

Why are you deleting this line?

drivers/iommu/arm-smmu.c static int arm_smmu_device_dt_probe(struct platform_device *pdev) · line 1827

Quoted 3 lines
drivers/iommu/arm-smmu.c +3 -0
+
+	if (!smmu->num_context_irqs) {
+		dev_err(dev, "no context interrupt specified in DT\n");

I'd avoid mentioning "DT" in the log message, just in case this ever starts probing from something else.

Will

#17 lore raw

PATCH 4/9 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

Quoted 32 lines · Will Deacon
Quoted 9 lines · Andreas Herrmann

With the right (or wrong;-) definition of v1 SMMU node in DTB it is possible to trigger a division by zero in arm_smmu_init_domain_context (if number of context irqs is 0):

       if (smmu->version == 1) {
               root_cfg->irptndx = atomic_inc_return(&smmu->irptndx);
 =>            root_cfg->irptndx %= smmu->num_context_irqs;
       } else {
Quoted 6 lines

Avoid this by checking for num_context_irqs > 0 when probing for SMMU devices.

Rationale: Assuming that at least one context bank for non-secure usage is provided per SMMU, it follows (from ARM SMMU Architecture Spec) that at least one context interrupt must be available.

One problem with this reasoning is that the interrupt line might just not be wired up to the GIC, despite existing on the SMMU. Still, we needn't solve that now (let's wait for somebody to build it first...).

Quoted 11 lines

Also remove the line of code that derived num_context_irqs from num_irqs and num_global_irqs. If DT is wrong and interrupt property contains less interrupts than num_global_irqs this would set num_context_irqs to a big u32 value which most likely causes trouble in other parts of the driver.

---
 drivers/iommu/arm-smmu.c |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
Quoted 11 lines

drivers/iommu/arm-smmu.c static int arm_smmu_device_dt_probe(struct platform_device *pdev) · line 1825

Quoted 9 lines
drivers/iommu/arm-smmu.c +0 -1
index 4307fbc..de9dd60 100644
@@ -1822,7 +1822,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 			 num_irqs, smmu->num_global_irqs);
 		smmu->num_global_irqs = num_irqs;
 	}
-	smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;

Why are you deleting this line?

Because I felt it's redundant in some cases and erroneously I thought it could be bogus if num_irqs < num_global_irqs.
Of course the latter is wrong, as num_global_irqs is corrected two lines above.

Now I think it's always redundant. num_context_irqs is only incremented here

               if (num_irqs > smmu->num_global_irqs)
                        smmu->num_context_irqs++;

So either it is still 0 (and no fixup required for num_irqs < num_global_irqs) or it contains already a positive value based on (num_irqs - num_global_irqs).

But maybe I've missed something.
(At least I need to fix the commit message wrt to this removal.)

Quoted 6 lines

drivers/iommu/arm-smmu.c static int arm_smmu_device_dt_probe(struct platform_device *pdev) · line 1827

Quoted 3 lines
drivers/iommu/arm-smmu.c +3 -0
+
+	if (!smmu->num_context_irqs) {
+		dev_err(dev, "no context interrupt specified in DT\n");

I'd avoid mentioning "DT" in the log message, just in case this ever starts probing from something else.

Ok, will fix this.

Andreas

#18 lore raw

PATCH 4/9 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

Quoted 19 lines · Andreas Herrmann
Quoted 12 lines · Will Deacon

drivers/iommu/arm-smmu.c static int arm_smmu_device_dt_probe(struct platform_device *pdev) · line 1825

Quoted 9 lines · Andreas Herrmann
drivers/iommu/arm-smmu.c +0 -1
index 4307fbc..de9dd60 100644
@@ -1822,7 +1822,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 			 num_irqs, smmu->num_global_irqs);
 		smmu->num_global_irqs = num_irqs;
 	}
-	smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;

Why are you deleting this line?

Because I felt it's redundant in some cases and erroneously I thought it could be bogus if num_irqs < num_global_irqs.
Of course the latter is wrong, as num_global_irqs is corrected two lines above.

Quoted 5 lines

Now I think it's always redundant. num_context_irqs is only incremented here

               if (num_irqs > smmu->num_global_irqs)
                        smmu->num_context_irqs++;

Yes, I think you're right. This leaves us in a situation where getting the #global-interrupts property wrong will trigger a warning followed immediately by a failure to probe. That means we could simply augment the current check and make it fatal instead:

--->8

drivers/iommu/arm-smmu.c +3 -4
index 2931921..03ffbae 100644
@@ -1799,12 +1799,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
                        smmu->num_context_irqs++;
        }
 
-       if (num_irqs < smmu->num_global_irqs) {
+       if (num_irqs <= smmu->num_global_irqs) {
                dev_warn(dev, "found %d interrupts but expected at least %d\n",
-                        num_irqs, smmu->num_global_irqs);
-               smmu->num_global_irqs = num_irqs;
+                        num_irqs, smmu->num_global_irqs + 1);
+               return -ENODEV;
        }
-       smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;
 
        smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
                                  GFP_KERNEL);

8<---

What do you think?

Will

#19 lore raw

PATCH 4/9 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

Quoted 38 lines · Will Deacon
Quoted 19 lines · Andreas Herrmann
Quoted 12 lines · Will Deacon

drivers/iommu/arm-smmu.c static int arm_smmu_device_dt_probe(struct platform_device *pdev) · line 1825

Quoted 9 lines · Andreas Herrmann
drivers/iommu/arm-smmu.c +0 -1
index 4307fbc..de9dd60 100644
@@ -1822,7 +1822,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 			 num_irqs, smmu->num_global_irqs);
 		smmu->num_global_irqs = num_irqs;
 	}
-	smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;

Why are you deleting this line?

Because I felt it's redundant in some cases and erroneously I thought it could be bogus if num_irqs < num_global_irqs.
Of course the latter is wrong, as num_global_irqs is corrected two lines above.

Quoted 5 lines

Now I think it's always redundant. num_context_irqs is only incremented here

               if (num_irqs > smmu->num_global_irqs)
                        smmu->num_context_irqs++;

Yes, I think you're right. This leaves us in a situation where getting the #global-interrupts property wrong will trigger a warning followed immediately by a failure to probe. That means we could simply augment the current check and make it fatal instead:

--->8

drivers/iommu/arm-smmu.c +0 -0
index 2931921..03ffbae 100644

drivers/iommu/arm-smmu.c static int arm_smmu_device_dt_probe(struct platform_device *pdev) · line 1802

Quoted 4 lines
drivers/iommu/arm-smmu.c +0 -1
@@ -1799,12 +1799,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
                        smmu->num_context_irqs++;
        }
 
-       if (num_irqs < smmu->num_global_irqs) {

+ /* expect num_global_irqs plus at least 1 context interrupt */

Quoted 12 lines
drivers/iommu/arm-smmu.c +3 -3
+       if (num_irqs <= smmu->num_global_irqs) {
                dev_warn(dev, "found %d interrupts but expected at least %d\n",
-                        num_irqs, smmu->num_global_irqs);
-               smmu->num_global_irqs = num_irqs;
+                        num_irqs, smmu->num_global_irqs + 1);
+               return -ENODEV;
        }
-       smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;
 
        smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
                                  GFP_KERNEL);
Quoted 3 lines

8<---

What do you think?

Yes, that should suffice.

I know it's clear for us but what about a short comment to emphasize that we expect to find at least one context irq?

-       if (num_irqs < smmu->num_global_irqs) {
+       /* expect num_global_irqs plus at least one context irq */
+       if (num_irqs <= smmu->num_global_irqs) {

which can be translated to

- if (num_irqs < smmu->num_global_irqs) {
+ if (!smmu->num_context_irqs) {

I don't care -- every version is fine (at least I know what to look for, if this warning message shows up). Most important thing is that a wrong DT should not cause a division by zero in the driver.

Andreas

#20 lore raw

PATCH 4/9 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

Quoted 14 lines · Andreas Herrmann
Quoted 1 line · Will Deacon

What do you think?

Yes, that should suffice.

I know it's clear for us but what about a short comment to emphasize that we expect to find at least one context irq?

drivers/iommu/arm-smmu.c +2 -1
-       if (num_irqs < smmu->num_global_irqs) {
+       /* expect num_global_irqs plus at least one context irq */
+       if (num_irqs <= smmu->num_global_irqs) {

which can be translated to

Review

Quoted 2 lines
(quoted diff) +1 -1
-       if (num_irqs < smmu->num_global_irqs) {
+       if (!smmu->num_context_irqs) {

Even better! Do you want to send a patch, or shall I just create one myself?

Will

#21 lore raw

PATCH 4/9 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

Quoted 13 lines · Will Deacon
Quoted 12 lines · Andreas Herrmann
Quoted 1 line · Will Deacon

What do you think?

Yes, that should suffice.

I know it's clear for us but what about a short comment to emphasize that we expect to find at least one context irq?

drivers/iommu/arm-smmu.c +2 -1
-       if (num_irqs < smmu->num_global_irqs) {
+       /* expect num_global_irqs plus at least one context irq */
+       if (num_irqs <= smmu->num_global_irqs) {
Quoted 6 lines

Review

Quoted 4 lines

which can be translated to

(quoted diff) +1 -1
-       if (num_irqs < smmu->num_global_irqs) {
+       if (!smmu->num_context_irqs) {

Even better! Do you want to send a patch, or shall I just create one myself?

I can do it.
(Have to update other patches anyway.)

Andreas

#22 lore raw

PATCH 1/1 iommu/arm-smmu: Check for num_context_irqs > 0 to avoid divide by zero exception

With the right (or wrong;-) definition of v1 SMMU node in DTB it is possible to trigger a division by zero in arm_smmu_init_domain_context (if number of context irqs is 0):

       if (smmu->version == 1) {
               root_cfg->irptndx = atomic_inc_return(&smmu->irptndx);
 =>            root_cfg->irptndx %= smmu->num_context_irqs;
       } else {

Avoid this by checking for num_context_irqs > 0 when probing for SMMU devices.

---
 drivers/iommu/arm-smmu.c |    7 +++----
 1 file changed, 3 insertions(+), 4 deletions(-)
drivers/iommu/arm-smmu.c +3 -4
index d19676c..7d07561 100644
@@ -1803,12 +1803,11 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 			smmu->num_context_irqs++;
 	}
 
-	if (num_irqs < smmu->num_global_irqs) {
+	if (!smmu->num_context_irqs) {
 		dev_warn(dev, "found %d interrupts but expected at least %d\n",
-			 num_irqs, smmu->num_global_irqs);
-		smmu->num_global_irqs = num_irqs;
+			 num_irqs, smmu->num_global_irqs + 1);
+		return -ENODEV;
 	}
-	smmu->num_context_irqs = num_irqs - smmu->num_global_irqs;
 
 	smmu->irqs = devm_kzalloc(dev, sizeof(*smmu->irqs) * num_irqs,
 				  GFP_KERNEL);
Signature · 1 line
1.7.9.5
#23 lore raw

PATCH 5/9 iommu/arm-smmu: Clear global and context bank fault status registers

In reply to Andreas Herrmann

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |    6 ++++++
 1 file changed, 6 insertions(+)
drivers/iommu/arm-smmu.c +6 -0
index de9dd60..9d31ad9 100644
@@ -642,6 +642,9 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 	stage1 = root_cfg->cbar != CBAR_TYPE_S2_TRANS;
 	cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, root_cfg->cbndx);
 
+	/* clear fsr */
+	writel_relaxed(0xffffffff, cb_base + ARM_SMMU_CB_FSR);
+
 	/* CBAR */
 	reg = root_cfg->cbar;
 	if (smmu->version == 1)
@@ -1564,6 +1567,9 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	int i = 0;
 	u32 scr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
 
+	/* clear global FSRs */
+	writel(0xffffffff, gr0_base + ARM_SMMU_GR0_sGFSR);
+
 	/* Mark all SMRn as invalid and all S2CRn as bypass */
 	for (i = 0; i < smmu->num_mapping_groups; ++i) {
 		writel_relaxed(~SMR_VALID, gr0_base + ARM_SMMU_GR0_SMR(i));
Signature · 1 line
1.7.9.5
#24 lore raw

PATCH 5/9 iommu/arm-smmu: Clear global and context bank fault status registers

Quoted 8 lines · Andreas Herrmann

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |    6 ++++++
 1 file changed, 6 insertions(+)

<pedantry> Your capitalisation of fsr vs FSR is inconsistent </pedantry>

drivers/iommu/arm-smmu.c static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain) · line 646

Quoted 10 lines
drivers/iommu/arm-smmu.c +2 -0
index de9dd60..9d31ad9 100644
@@ -642,6 +642,9 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 	stage1 = root_cfg->cbar != CBAR_TYPE_S2_TRANS;
 	cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, root_cfg->cbndx);
 
+	/* clear fsr */
+	writel_relaxed(0xffffffff, cb_base + ARM_SMMU_CB_FSR);

I think this is too late, since we've already requested the IRQ line for this context bank by the time we get here. There are two options (afaict):

(1) Clear the CB FSRs during probe *and* during domain destruction

(2) Delay request_irq until after the context bank has been initialised

Also, rather than 0xffffffff, FSR_IGN | FSR_FAULT is probably clearer.

drivers/iommu/arm-smmu.c static void arm_smmu_device_reset(struct arm_smmu_device *smmu) · line 1571

Quoted 9 lines
drivers/iommu/arm-smmu.c +2 -0
 	/* CBAR */
 	reg = root_cfg->cbar;
 	if (smmu->version == 1)
@@ -1564,6 +1567,9 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	int i = 0;
 	u32 scr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
 
+	/* clear global FSRs */
+	writel(0xffffffff, gr0_base + ARM_SMMU_GR0_sGFSR);

For this guy, a read of the register and a write back is probably better than writing 1s to all the reserved bits.

Will

#25 lore raw

PATCH 5/9 iommu/arm-smmu: Clear global and context bank fault status registers

Quoted 26 lines · Will Deacon
Quoted 8 lines · Andreas Herrmann

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |    6 ++++++
 1 file changed, 6 insertions(+)

<pedantry> Your capitalisation of fsr vs FSR is inconsistent </pedantry>

drivers/iommu/arm-smmu.c static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain) · line 646

Quoted 10 lines
drivers/iommu/arm-smmu.c +2 -0
index de9dd60..9d31ad9 100644
@@ -642,6 +642,9 @@ static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 	stage1 = root_cfg->cbar != CBAR_TYPE_S2_TRANS;
 	cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, root_cfg->cbndx);
 
+	/* clear fsr */
+	writel_relaxed(0xffffffff, cb_base + ARM_SMMU_CB_FSR);

I think this is too late, since we've already requested the IRQ line for this context bank by the time we get here. There are two options (afaict):

Quoted 3 lines

(1) Clear the CB FSRs during probe *and* during domain destruction

(2) Delay request_irq until after the context bank has been initialised

I'd do kind of (2) and clear the FSR of that context bank before request_irq is called.

Quoted 1 line

Also, rather than 0xffffffff, FSR_IGN | FSR_FAULT is probably clearer.

Yep.

Quoted 7 lines
Quoted 7 lines
drivers/iommu/arm-smmu.c +0 -0
 	/* CBAR */
 	reg = root_cfg->cbar;
 	if (smmu->version == 1)
@@ -1564,6 +1567,9 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	int i = 0;
 	u32 scr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
 
Quoted 5 lines

drivers/iommu/arm-smmu.c static void arm_smmu_device_reset(struct arm_smmu_device *smmu) · line 1571

Quoted 2 lines
drivers/iommu/arm-smmu.c +2 -0
+	/* clear global FSRs */
+	writel(0xffffffff, gr0_base + ARM_SMMU_GR0_sGFSR);

For this guy, a read of the register and a write back is probably better than writing 1s to all the reserved bits.

Ok.

Andreas

#26 lore raw

PATCH 1/1 iommu/arm-smmu: Clear global and context bank fault status registers

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |   27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +20 -7
index 579b6f8..cbbf597 100644
@@ -631,6 +631,12 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
 	return IRQ_HANDLED;
 }
 
+static void arm_smmu_clear_cb_fsr(struct arm_smmu_device *smmu, u8 cbndx)
+{
+	void __iomem *cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cbndx);
+	writel_relaxed(FSR_FAULT, cb_base + ARM_SMMU_CB_FSR);
+}
+
 static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 {
 	u32 reg;
@@ -838,6 +844,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,
 		root_cfg->irptndx = root_cfg->cbndx;
 	}
 
+	arm_smmu_clear_cb_fsr(smmu, root_cfg->cbndx);
 	irq = smmu->irqs[smmu->num_global_irqs + root_cfg->irptndx];
 	ret = request_irq(irq, arm_smmu_context_fault, IRQF_SHARED,
 			  "arm-smmu-context-fault", domain);
@@ -1564,7 +1571,11 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
 	void __iomem *sctlr_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB_SCTLR;
 	int i = 0;
-	u32 scr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
+	u32 val;
+
+	/* clear global FSR */
+	val = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSR);
+	writel(val, gr0_base + ARM_SMMU_GR0_sGFSR);
 
 	/* Mark all SMRn as invalid and all S2CRn as bypass */
 	for (i = 0; i < smmu->num_mapping_groups; ++i) {
@@ -1581,24 +1592,26 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	writel_relaxed(0, gr0_base + ARM_SMMU_GR0_TLBIALLH);
 	writel_relaxed(0, gr0_base + ARM_SMMU_GR0_TLBIALLNSNH);
 
+	val = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
+
 	/* Enable fault reporting */
-	scr0 |= (sCR0_GFRE | sCR0_GFIE | sCR0_GCFGFRE | sCR0_GCFGFIE);
+	val |= (sCR0_GFRE | sCR0_GFIE | sCR0_GCFGFRE | sCR0_GCFGFIE);
 
 	/* Disable TLB broadcasting. */
-	scr0 |= (sCR0_VMIDPNE | sCR0_PTM);
+	val |= (sCR0_VMIDPNE | sCR0_PTM);
 
 	/* Enable client access, but bypass when no mapping is found */
-	scr0 &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
+	val &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
 
 	/* Disable forced broadcasting */
-	scr0 &= ~sCR0_FB;
+	val &= ~sCR0_FB;
 
 	/* Don't upgrade barriers */
-	scr0 &= ~(sCR0_BSU_MASK << sCR0_BSU_SHIFT);
+	val &= ~(sCR0_BSU_MASK << sCR0_BSU_SHIFT);
 
 	/* Push the button */
 	arm_smmu_tlb_sync(smmu);
-	writel(scr0, gr0_base + ARM_SMMU_GR0_sCR0);
+	writel(val, gr0_base + ARM_SMMU_GR0_sCR0);
 }
 
 static int arm_smmu_id_size_to_bits(int size)
Signature · 1 line
1.7.9.5
#27 lore raw

PATCH 1/1 iommu/arm-smmu: Clear global and context bank fault status registers

Quoted 15 lines · Andreas Herrmann

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |   27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +0 -0
index 579b6f8..cbbf597 100644

drivers/iommu/arm-smmu.c static irqreturn_t arm_smmu_global_fault(int irq, void *dev) · line 638

Quoted 8 lines
drivers/iommu/arm-smmu.c +5 -0
@@ -631,6 +631,12 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
 	return IRQ_HANDLED;
 }
 
+static void arm_smmu_clear_cb_fsr(struct arm_smmu_device *smmu, u8 cbndx)
+{
+	void __iomem *cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cbndx);
+	writel_relaxed(FSR_FAULT, cb_base + ARM_SMMU_CB_FSR);
+}

Hmm, why not just stick this in arm_smmu_init_context_bank...

drivers/iommu/arm-smmu.c static int arm_smmu_init_domain_context(struct iommu_domain *domain, · line 847

Quoted 8 lines
drivers/iommu/arm-smmu.c +1 -0
 static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 {
 	u32 reg;
@@ -838,6 +844,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,
 		root_cfg->irptndx = root_cfg->cbndx;
 	}
 
+	arm_smmu_clear_cb_fsr(smmu, root_cfg->cbndx);

... then move that function call here instead of where it currently is?

drivers/iommu/arm-smmu.c static void arm_smmu_device_reset(struct arm_smmu_device *smmu) · line 1574

Quoted 9 lines
drivers/iommu/arm-smmu.c +1 -1
 	irq = smmu->irqs[smmu->num_global_irqs + root_cfg->irptndx];
 	ret = request_irq(irq, arm_smmu_context_fault, IRQF_SHARED,
 			  "arm-smmu-context-fault", domain);
@@ -1564,7 +1571,11 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
 	void __iomem *sctlr_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB_SCTLR;
 	int i = 0;
-	u32 scr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
+	u32 val;

I guess we have to dilute the meaning of the variable, but perhaps reg' is slightly more indicative than val'?

Do you plan to post all your patches as a new series (it's hard to keep track of what to pick)?

Cheers,

Will

#28 lore raw

PATCH 1/1 iommu/arm-smmu: Clear global and context bank fault status registers

Quoted 19 lines · Will Deacon
Quoted 18 lines · Andreas Herrmann

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |   27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +0 -0
index 579b6f8..cbbf597 100644
@@ -631,6 +631,12 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
 	return IRQ_HANDLED;
 }
 
Quoted 7 lines

drivers/iommu/arm-smmu.c static irqreturn_t arm_smmu_global_fault(int irq, void *dev) · line 638

Quoted 5 lines
drivers/iommu/arm-smmu.c +5 -0
+static void arm_smmu_clear_cb_fsr(struct arm_smmu_device *smmu, u8 cbndx)
+{
+	void __iomem *cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cbndx);
+	writel_relaxed(FSR_FAULT, cb_base + ARM_SMMU_CB_FSR);
+}

Hmm, why not just stick this in arm_smmu_init_context_bank...

Because we should clear the FSR before we call request_irq. Otherwise we might handle interrupts although the context bank is not enabled.

Moving request_irq after arm_smmu_init_context_bank is not optimal either. (We should have configured the context interrupt before translation is enabled. Otherwise it's possible to miss a fault.)

Quoted 10 lines

drivers/iommu/arm-smmu.c static int arm_smmu_init_domain_context(struct iommu_domain *domain, · line 847

Quoted 8 lines
drivers/iommu/arm-smmu.c +1 -0
 static void arm_smmu_init_context_bank(struct arm_smmu_domain *smmu_domain)
 {
 	u32 reg;
@@ -838,6 +844,7 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,
 		root_cfg->irptndx = root_cfg->cbndx;
 	}
 
+	arm_smmu_clear_cb_fsr(smmu, root_cfg->cbndx);

... then move that function call here instead of where it currently is?

See above.

Quoted 4 lines
Quoted 4 lines
drivers/iommu/arm-smmu.c +0 -0
 	irq = smmu->irqs[smmu->num_global_irqs + root_cfg->irptndx];
 	ret = request_irq(irq, arm_smmu_context_fault, IRQF_SHARED,
 			  "arm-smmu-context-fault", domain);
@@ -1564,7 +1571,11 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
Quoted 8 lines

drivers/iommu/arm-smmu.c static void arm_smmu_device_reset(struct arm_smmu_device *smmu) · line 1574

Quoted 5 lines
drivers/iommu/arm-smmu.c +1 -1
 	void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
 	void __iomem *sctlr_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB_SCTLR;
 	int i = 0;
-	u32 scr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
+	u32 val;

I guess we have to dilute the meaning of the variable, but perhaps reg' is slightly more indicative than val'?

Will change that.

Quoted 2 lines

Do you plan to post all your patches as a new series (it's hard to keep track of what to pick)?

Yes.

Andreas

#29 lore raw

PATCH 1/1 iommu/arm-smmu: Clear global and context bank fault status registers

Quoted 28 lines · Andreas Herrmann
Quoted 19 lines · Will Deacon
Quoted 18 lines · Andreas Herrmann

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |   27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +0 -0
index 579b6f8..cbbf597 100644
@@ -631,6 +631,12 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
 	return IRQ_HANDLED;
 }
 
Quoted 7 lines

drivers/iommu/arm-smmu.c static irqreturn_t arm_smmu_global_fault(int irq, void *dev) · line 638

Quoted 5 lines
drivers/iommu/arm-smmu.c +5 -0
+static void arm_smmu_clear_cb_fsr(struct arm_smmu_device *smmu, u8 cbndx)
+{
+	void __iomem *cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cbndx);
+	writel_relaxed(FSR_FAULT, cb_base + ARM_SMMU_CB_FSR);
+}

Hmm, why not just stick this in arm_smmu_init_context_bank...

Quoted 7 lines

Because we should clear the FSR before we call request_irq. Otherwise we might handle interrupts although the context bank is not enabled.

Moving request_irq after arm_smmu_init_context_bank is not optimal either. (We should have configured the context interrupt before translation is enabled. Otherwise it's possible to miss a fault.)

How would you miss a fault? If the device can start issuing transactions before the SMMU has set up the mapping, then there's a race in the caller code which we shouldn't attempt to resolve here.

Will

#30 lore raw

PATCH 1/1 iommu/arm-smmu: Clear global and context bank fault status registers

Quoted 37 lines · Will Deacon
Quoted 28 lines · Andreas Herrmann
Quoted 19 lines · Will Deacon
Quoted 18 lines · Andreas Herrmann

After reset these registers have unknown values.
This might cause problems when evaluating SMMU_GFSR and/or SMMU_CB_FSR in handlers for combined interrupts.

---
 drivers/iommu/arm-smmu.c |   27 ++++++++++++++++++++-------
 1 file changed, 20 insertions(+), 7 deletions(-)
drivers/iommu/arm-smmu.c +0 -0
index 579b6f8..cbbf597 100644
@@ -631,6 +631,12 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
 	return IRQ_HANDLED;
 }
 
Quoted 7 lines

drivers/iommu/arm-smmu.c static irqreturn_t arm_smmu_global_fault(int irq, void *dev) · line 638

Quoted 5 lines
drivers/iommu/arm-smmu.c +5 -0
+static void arm_smmu_clear_cb_fsr(struct arm_smmu_device *smmu, u8 cbndx)
+{
+	void __iomem *cb_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB(smmu, cbndx);
+	writel_relaxed(FSR_FAULT, cb_base + ARM_SMMU_CB_FSR);
+}

Hmm, why not just stick this in arm_smmu_init_context_bank...

Quoted 7 lines

Because we should clear the FSR before we call request_irq. Otherwise we might handle interrupts although the context bank is not enabled.

Moving request_irq after arm_smmu_init_context_bank is not optimal either. (We should have configured the context interrupt before translation is enabled. Otherwise it's possible to miss a fault.)

Quoted 5 lines

How would you miss a fault?

If the device can start issuing transactions before the SMMU has set up the mapping, then there's a race in the caller code which we shouldn't attempt to resolve here.

Broken device, broken driver code (maybe violating dma-api) whatsoever. All that's needed is a device that is already doing DMA when we enable SMMU handling for its transactions.
Yes, normally this should not happen. But if it happens we get a fault and better should handle it.

I think, it's somehow logical to have fault handling set up before fault reporting is switched on for a context bank.

Andreas

#31 lore raw

PATCH 6/9 iommu/arm-smmu: Support buggy implemenations where all config accesses are secure

In reply to Andreas Herrmann

In such a case we have to use secure aliases of some non-secure registers.

This behaviour is controlled via a flag in smmu->bugs. It is set based on DT information when probing an SMMU device.

---
 drivers/iommu/arm-smmu.c |   64 +++++++++++++++++++++++++++++++++++-----------
 1 file changed, 49 insertions(+), 15 deletions(-)
drivers/iommu/arm-smmu.c +49 -15
index 9d31ad9..5fa34f9 100644
@@ -159,6 +159,12 @@
 #define PIDR2_ARCH_SHIFT		4
 #define PIDR2_ARCH_MASK			0xf
 
+/* secure aliases for non-secure registers */
+#define ARM_SMMU_GR0_nsCR0		0x400
+#define ARM_SMMU_GR0_nsGFSR		0x448
+#define ARM_SMMU_GR0_nsGFSYNR0		0x450
+#define ARM_SMMU_GR0_nsGFSYNR1		0x454
+
 /* Global TLB invalidation */
 #define ARM_SMMU_GR0_STLBIALL		0x60
 #define ARM_SMMU_GR0_TLBIVMID		0x64
@@ -349,6 +355,8 @@ struct arm_smmu_device {
 #define ARM_SMMU_FEAT_TRANS_S2		(1 << 3)
 #define ARM_SMMU_FEAT_TRANS_NESTED	(1 << 4)
 	u32				features;
+#define ARM_SMMU_BUG_SECURE_CFG_ACCESS	(1 << 0)
+	u32				bugs;
 	int				version;
 
 	u32				num_context_banks;
@@ -611,21 +619,32 @@ static irqreturn_t arm_smmu_global_fault(int irq, void *dev)
 	struct arm_smmu_device *smmu = dev;
 	void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
 
-	gfsr = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSR);
+	if (smmu->bugs & ARM_SMMU_BUG_SECURE_CFG_ACCESS) {
+		gfsr = readl_relaxed(gr0_base + ARM_SMMU_GR0_nsGFSR);
+		gfsynr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_nsGFSYNR0);
+		gfsynr1 = readl_relaxed(gr0_base + ARM_SMMU_GR0_nsGFSYNR1);
+		gfsynr2 = 0;
+	} else {
+		gfsr = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSR);
+		gfsynr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR0);
+		gfsynr1 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR1);
+		gfsynr2 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR2);
+	}
+
 	if (!gfsr)
 		return IRQ_NONE;
 
-	gfsynr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR0);
-	gfsynr1 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR1);
-	gfsynr2 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sGFSYNR2);
-
 	dev_err_ratelimited(smmu->dev,
 		"Unexpected global fault, this could be serious\n");
 	dev_err_ratelimited(smmu->dev,
 		"\tGFSR 0x%08x, GFSYNR0 0x%08x, GFSYNR1 0x%08x, GFSYNR2 0x%08x\n",
 		gfsr, gfsynr0, gfsynr1, gfsynr2);
 
-	writel(gfsr, gr0_base + ARM_SMMU_GR0_sGFSR);
+	if (smmu->bugs & ARM_SMMU_BUG_SECURE_CFG_ACCESS)
+		writel(gfsr, gr0_base + ARM_SMMU_GR0_nsGFSR);
+	else
+		writel(gfsr, gr0_base + ARM_SMMU_GR0_sGFSR);
+
 	return IRQ_HANDLED;
 }
 
@@ -1565,10 +1584,16 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	void __iomem *gr0_base = ARM_SMMU_GR0(smmu);
 	void __iomem *sctlr_base = ARM_SMMU_CB_BASE(smmu) + ARM_SMMU_CB_SCTLR;
 	int i = 0;
-	u32 scr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
+	u32 cr0;
 
 	/* clear global FSRs */
-	writel(0xffffffff, gr0_base + ARM_SMMU_GR0_sGFSR);
+	if (smmu->bugs & ARM_SMMU_BUG_SECURE_CFG_ACCESS) {
+		writel(0xffffffff, gr0_base + ARM_SMMU_GR0_nsGFSR);
+		cr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_nsCR0);
+	} else {
+		writel(0xffffffff, gr0_base + ARM_SMMU_GR0_sGFSR);
+		cr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
+	}
 
 	/* Mark all SMRn as invalid and all S2CRn as bypass */
 	for (i = 0; i < smmu->num_mapping_groups; ++i) {
@@ -1586,23 +1611,26 @@ static void arm_smmu_device_reset(struct arm_smmu_device *smmu)
 	writel_relaxed(0, gr0_base + ARM_SMMU_GR0_TLBIALLNSNH);
 
 	/* Enable fault reporting */
-	scr0 |= (sCR0_GFRE | sCR0_GFIE | sCR0_GCFGFRE | sCR0_GCFGFIE);
+	cr0 |= (sCR0_GFRE | sCR0_GFIE | sCR0_GCFGFRE | sCR0_GCFGFIE);
 
 	/* Disable TLB broadcasting. */
-	scr0 |= (sCR0_VMIDPNE | sCR0_PTM);
+	cr0 |= (sCR0_VMIDPNE | sCR0_PTM);
 
 	/* Enable client access, but bypass when no mapping is found */
-	scr0 &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
+	cr0 &= ~(sCR0_CLIENTPD | sCR0_USFCFG);
 
 	/* Disable forced broadcasting */
-	scr0 &= ~sCR0_FB;
+	cr0 &= ~sCR0_FB;
 
 	/* Don't upgrade barriers */
-	scr0 &= ~(sCR0_BSU_MASK << sCR0_BSU_SHIFT);
+	cr0 &= ~(sCR0_BSU_MASK << sCR0_BSU_SHIFT);
 
 	/* Push the button */
 	arm_smmu_tlb_sync(smmu);
-	writel(scr0, gr0_base + ARM_SMMU_GR0_sCR0);
+	if (smmu->bugs & ARM_SMMU_BUG_SECURE_CFG_ACCESS)
+		writel(cr0, gr0_base + ARM_SMMU_GR0_nsCR0);
+	else
+		writel(cr0, gr0_base + ARM_SMMU_GR0_sCR0);
 }
 
 static int arm_smmu_id_size_to_bits(int size)
@@ -1894,6 +1922,9 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 		}
 	}
 
+	if (of_property_read_bool(dev->of_node, "calxeda,smmu-secure-cfg-access"))
+		smmu->bugs |= ARM_SMMU_BUG_SECURE_CFG_ACCESS;
+
 	INIT_LIST_HEAD(&smmu->list);
 	spin_lock(&arm_smmu_devices_lock);
 	list_add(&smmu->list, &arm_smmu_devices);
@@ -1956,7 +1987,10 @@ static int arm_smmu_device_remove(struct platform_device *pdev)
 		free_irq(smmu->irqs[i], smmu);
 
 	/* Turn the thing off */
-	writel(sCR0_CLIENTPD, ARM_SMMU_GR0(smmu) + ARM_SMMU_GR0_sCR0);
+	if (smmu->bugs & ARM_SMMU_BUG_SECURE_CFG_ACCESS)
+		writel(sCR0_CLIENTPD, ARM_SMMU_GR0(smmu) + ARM_SMMU_GR0_nsCR0);
+	else
+		writel(sCR0_CLIENTPD, ARM_SMMU_GR0(smmu) + ARM_SMMU_GR0_sCR0);
 	return 0;
 }
 
Signature · 1 line
1.7.9.5
#32 lore raw

PATCH 6/9 iommu/arm-smmu: Support buggy implemenations where all config accesses are secure

Quoted 5 lines · Andreas Herrmann

In such a case we have to use secure aliases of some non-secure registers.

This behaviour is controlled via a flag in smmu->bugs. It is set based on DT information when probing an SMMU device.

:(

I guess my main comment is just some understanding on how things are wired. It's *just* the configuration access that is secure, right? Device transactions are non-secure, so the existing TLB invalidation code will work correctly?

Will

#33 lore raw

PATCH 6/9 iommu/arm-smmu: Support buggy implemenations where all config accesses are secure

Quoted 11 lines · Will Deacon
Quoted 5 lines · Andreas Herrmann

In such a case we have to use secure aliases of some non-secure registers.

This behaviour is controlled via a flag in smmu->bugs. It is set based on DT information when probing an SMMU device.

:(

I guess my main comment is just some understanding on how things are wired. It's *just* the configuration access that is secure, right?

Correct.

Quoted 2 lines

Device transactions are non-secure, so the existing TLB invalidation code will work correctly?

Yes, transactions are non-secure.

The TLB invalidation code (using SMMU_TLBIVMID for stage2) also seems too work -- no problems triggered so far (neither with sata nor with xgmac).

Andreas

#34 lore raw

PATCH 7/9 iommu/arm-smmu: Add function that conditionally isolates all masters of all SMMUs

In reply to Andreas Herrmann

(Depending on DT information and module parameters) each device is put into its own protection domain (if possible). For configuration with one or just a few masters per SMMU that is easy to achieve.

In case of many devices per SMMU (e.g. MMU-500 with it's distributed translation support) isolation of each device might not be possible -- depending on number of available SMR groups and/or context banks.

Default is that device isolation is contolled per SMMU with SMMU node property "linux,arm-smmu-isolate-devices" in a DT. If the property is set for an SMMU node device isolation is performed.

Also introduce a module parameter:

arm-smmu=off|force_isolation|disable_isolation

arm-smmu= arm-smmu driver option

off Disable arm-smmu driver (ie. ignore available SMMUs)

force_isolation
Try to attach each master device on every SMMU to a separate iommu_domain.
(Even for SMMUs that don't have the property "linux,arm-smmu-isolate-devices" set in device tree.)

disable_isolation
Disable device_isolation globally. (ie. no isolation even for SMMUs for which "linux,arm-smmu-isolate-devices" is set.)

W/o device isolation the driver detects SMMUs but no translation is configured (transactions just bypass translation process).

Note that for device isolation dma_base and size are derived from (coherent_)dma_mask of the master device.

---
 drivers/iommu/arm-smmu.c |  101 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 101 insertions(+)
drivers/iommu/arm-smmu.c +101 -0
index 5fa34f9..48f3bfb 100644
@@ -46,6 +46,7 @@
 #include <linux/amba/bus.h>
 
 #include <asm/pgalloc.h>
+#include <asm/dma-iommu.h>
 
 /* Maximum number of stream IDs assigned to a single device */
 #define MAX_MASTER_STREAMIDS		8
@@ -354,6 +355,7 @@ struct arm_smmu_device {
 #define ARM_SMMU_FEAT_TRANS_S1		(1 << 2)
 #define ARM_SMMU_FEAT_TRANS_S2		(1 << 3)
 #define ARM_SMMU_FEAT_TRANS_NESTED	(1 << 4)
+#define ARM_SMMU_FEAT_ISOLATE_DEVICES	(1 << 5)
 	u32				features;
 #define ARM_SMMU_BUG_SECURE_CFG_ACCESS	(1 << 0)
 	u32				bugs;
@@ -407,6 +409,10 @@ struct arm_smmu_domain {
 static DEFINE_SPINLOCK(arm_smmu_devices_lock);
 static LIST_HEAD(arm_smmu_devices);
 
+static bool arm_smmu_disabled;
+static bool arm_smmu_force_isolation;
+static bool arm_smmu_disable_isolation;
+
 static struct arm_smmu_master *find_smmu_master(struct arm_smmu_device *smmu,
 						struct device_node *dev_node)
 {
@@ -1810,6 +1816,74 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	return 0;
 }
 
+extern struct platform_device *of_find_device_by_node(struct device_node *np);
+
+static int arm_smmu_isolate_devices(void)
+{
+	struct dma_iommu_mapping *mapping;
+	struct arm_smmu_device *smmu;
+	struct rb_node *rbn;
+	struct arm_smmu_master *master;
+	struct platform_device *pdev;
+	struct device *dev;
+	void __iomem *gr0_base;
+	u32 cr0;
+	int ret = 0;
+	size_t size;
+
+	list_for_each_entry(smmu, &arm_smmu_devices, list) {
+		if (arm_smmu_disable_isolation ||
+			(!(smmu->features & ARM_SMMU_FEAT_ISOLATE_DEVICES)
+				&& !arm_smmu_force_isolation))
+			continue;
+		rbn = rb_first(&smmu->masters);
+		while (rbn) {
+			master = container_of(rbn, struct arm_smmu_master, node);
+			pdev = of_find_device_by_node(master->of_node);
+			if (!pdev)
+				break;
+			dev = &pdev->dev;
+
+			size = (size_t) dev->coherent_dma_mask;
+			size = size ? : (unsigned long) dev->dma_mask;
+			if (!size) {
+				dev_warn(dev, "(coherent_)dma_mask not set\n");
+				continue;
+			}
+
+			mapping = arm_iommu_create_mapping(&platform_bus_type,
+							0, size, 0);
+			if (IS_ERR(mapping)) {
+				ret = PTR_ERR(mapping);
+				dev_info(dev, "arm_iommu_create_mapping failed\n");
+				goto out;
+			}
+
+			ret = arm_iommu_attach_device(dev, mapping);
+			if (ret < 0) {
+				dev_info(dev, "arm_iommu_attach_device failed\n");
+				arm_iommu_release_mapping(mapping);
+			}
+			rbn = rb_next(rbn);
+		}
+
+		gr0_base = ARM_SMMU_GR0(smmu);
+		if (smmu->bugs & ARM_SMMU_BUG_SECURE_CFG_ACCESS) {
+			cr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_nsCR0);
+			cr0 |= sCR0_USFCFG;
+			writel(cr0, gr0_base + ARM_SMMU_GR0_nsCR0);
+		} else {
+			cr0 = readl_relaxed(gr0_base + ARM_SMMU_GR0_sCR0);
+			cr0 |= sCR0_USFCFG;
+			writel(cr0, gr0_base + ARM_SMMU_GR0_sCR0);
+		}
+	}
+
+out:
+	return ret;
+}
+
+
 static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 {
 	struct resource *res;
@@ -1820,6 +1894,9 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 	struct of_phandle_args masterspec;
 	int num_irqs, i, err;
 
+	if (arm_smmu_disabled)
+		return -ENODEV;
+
 	smmu = devm_kzalloc(dev, sizeof(*smmu), GFP_KERNEL);
 	if (!smmu) {
 		dev_err(dev, "failed to allocate arm_smmu_device\n");
@@ -1925,6 +2002,9 @@ static int arm_smmu_device_dt_probe(struct platform_device *pdev)
 	if (of_property_read_bool(dev->of_node, "calxeda,smmu-secure-cfg-access"))
 		smmu->bugs |= ARM_SMMU_BUG_SECURE_CFG_ACCESS;
 
+	if (of_property_read_bool(dev->of_node, "linux,arm-smmu-isolate-devices"))
+		smmu->features |= ARM_SMMU_FEAT_ISOLATE_DEVICES;
+
 	INIT_LIST_HEAD(&smmu->list);
 	spin_lock(&arm_smmu_devices_lock);
 	list_add(&smmu->list, &arm_smmu_devices);
@@ -2015,6 +2095,25 @@ static struct platform_driver arm_smmu_driver = {
 	.remove	= arm_smmu_device_remove,
 };
 
+static int __init arm_smmu_parse_options(char *str)
+{
+	if (*str) {
+		str++;
+		if (!strncmp(str, "off", 3))
+			arm_smmu_disabled = true;
+		else if(!strncmp(str, "force_isolation", 15))
+			arm_smmu_force_isolation = true;
+		else if(!strncmp(str, "disable_isolation", 17))
+			arm_smmu_disable_isolation = true;
+		else {
+			pr_warn("arm_smmu: invalid parameter (\"%s\")\n", str);
+			return 0;
+		}
+	}
+	return 1;
+}
+__setup("arm-smmu", arm_smmu_parse_options);
+
 static int __init arm_smmu_init(void)
 {
 	int ret;
@@ -2030,6 +2129,8 @@ static int __init arm_smmu_init(void)
 	if (!iommu_present(&amba_bustype))
 		bus_set_iommu(&amba_bustype, &arm_smmu_ops);
 
+	arm_smmu_isolate_devices();
+
 	return 0;
 }
 
Signature · 1 line
1.7.9.5
#35 lore raw

PATCH 7/9 iommu/arm-smmu: Add function that conditionally isolates all masters of all SMMUs

Hi Andreas,

Quoted 8 lines · Andreas Herrmann

(Depending on DT information and module parameters) each device is put into its own protection domain (if possible). For configuration with one or just a few masters per SMMU that is easy to achieve.

In case of many devices per SMMU (e.g. MMU-500 with it's distributed translation support) isolation of each device might not be possible -- depending on number of available SMR groups and/or context banks.

Quoted 5 lines

Default is that device isolation is contolled per SMMU with SMMU node property "linux,arm-smmu-isolate-devices" in a DT. If the property is set for an SMMU node device isolation is performed.

Also introduce a module parameter:

Actually, I think I'd rather do away with the module paramater / command line option altogether in favour of DT.

Quoted 20 lines
drivers/iommu/arm-smmu.c +20 -0
+extern struct platform_device *of_find_device_by_node(struct device_node *np);
+
+static int arm_smmu_isolate_devices(void)
+{
+	struct dma_iommu_mapping *mapping;
+	struct arm_smmu_device *smmu;
+	struct rb_node *rbn;
+	struct arm_smmu_master *master;
+	struct platform_device *pdev;
+	struct device *dev;
+	void __iomem *gr0_base;
+	u32 cr0;
+	int ret = 0;
+	size_t size;
+
+	list_for_each_entry(smmu, &arm_smmu_devices, list) {
+		if (arm_smmu_disable_isolation ||
+			(!(smmu->features & ARM_SMMU_FEAT_ISOLATE_DEVICES)
+				&& !arm_smmu_force_isolation))
+			continue;

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1848

Quoted 10 lines
drivers/iommu/arm-smmu.c +10 -0
+		rbn = rb_first(&smmu->masters);
+		while (rbn) {
+			master = container_of(rbn, struct arm_smmu_master, node);
+			pdev = of_find_device_by_node(master->of_node);
+			if (!pdev)
+				break;
+			dev = &pdev->dev;
+
+			size = (size_t) dev->coherent_dma_mask;
+			size = size ? : (unsigned long) dev->dma_mask;

Hmm, this could be *huge* with 64-bit capable DMA controllers (think LPAE). Russell also has some pending dma mask cleanup, which might break some assumptions here:

http://lists.infradead.org/pipermail/linux-arm-kernel/2013-September/199397.html

(namely that we're offsetting everything from zero).

Will

#36 lore raw

PATCH 7/9 iommu/arm-smmu: Add function that conditionally isolates all masters of all SMMUs

Quoted 44 lines · Will Deacon

Hi Andreas,

Quoted 8 lines · Andreas Herrmann

(Depending on DT information and module parameters) each device is put into its own protection domain (if possible). For configuration with one or just a few masters per SMMU that is easy to achieve.

In case of many devices per SMMU (e.g. MMU-500 with it's distributed translation support) isolation of each device might not be possible -- depending on number of available SMR groups and/or context banks.

Quoted 5 lines

Default is that device isolation is contolled per SMMU with SMMU node property "linux,arm-smmu-isolate-devices" in a DT. If the property is set for an SMMU node device isolation is performed.

Also introduce a module parameter:

Actually, I think I'd rather do away with the module paramater / command line option altogether in favour of DT.

Quoted 24 lines
drivers/iommu/arm-smmu.c +24 -0
+extern struct platform_device *of_find_device_by_node(struct device_node *np);
+
+static int arm_smmu_isolate_devices(void)
+{
+	struct dma_iommu_mapping *mapping;
+	struct arm_smmu_device *smmu;
+	struct rb_node *rbn;
+	struct arm_smmu_master *master;
+	struct platform_device *pdev;
+	struct device *dev;
+	void __iomem *gr0_base;
+	u32 cr0;
+	int ret = 0;
+	size_t size;
+
+	list_for_each_entry(smmu, &arm_smmu_devices, list) {
+		if (arm_smmu_disable_isolation ||
+			(!(smmu->features & ARM_SMMU_FEAT_ISOLATE_DEVICES)
+				&& !arm_smmu_force_isolation))
+			continue;
+		rbn = rb_first(&smmu->masters);
+		while (rbn) {
+			master = container_of(rbn, struct arm_smmu_master, node);
+			pdev = of_find_device_by_node(master->of_node);
Quoted 8 lines

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1848

Quoted 6 lines
drivers/iommu/arm-smmu.c +6 -0
+			if (!pdev)
+				break;
+			dev = &pdev->dev;
+
+			size = (size_t) dev->coherent_dma_mask;
+			size = size ? : (unsigned long) dev->dma_mask;

Hmm, this could be *huge* with 64-bit capable DMA controllers (think LPAE).

Yes, agreed.
(And even for 32-bit DMA this requires a large bitmap_size for the mapping.)

Quoted 6 lines

Russell also has some pending dma mask cleanup, which might break some assumptions here:

http://lists.infradead.org/pipermail/linux-arm-kernel/2013-September/199397.html

(namely that we're offsetting everything from zero).

What do you think is a reasonable general value for the size of a mapping? (Do we need a DT property to specify this?)

What about a size of 128 MB -- if I'm not mistaken this requires a bitmap_size of 4K.

Andreas

#37 lore raw

PATCH 7/9 iommu/arm-smmu: Add function that conditionally isolates all masters of all SMMUs

Quoted 29 lines · Andreas Herrmann
Quoted 10 lines · Will Deacon
Quoted 9 lines · Andreas Herrmann
drivers/iommu/arm-smmu.c +9 -0
+	list_for_each_entry(smmu, &arm_smmu_devices, list) {
+		if (arm_smmu_disable_isolation ||
+			(!(smmu->features & ARM_SMMU_FEAT_ISOLATE_DEVICES)
+				&& !arm_smmu_force_isolation))
+			continue;
+		rbn = rb_first(&smmu->masters);
+		while (rbn) {
+			master = container_of(rbn, struct arm_smmu_master, node);
+			pdev = of_find_device_by_node(master->of_node);
Quoted 8 lines

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1848

Quoted 6 lines
drivers/iommu/arm-smmu.c +6 -0
+			if (!pdev)
+				break;
+			dev = &pdev->dev;
+
+			size = (size_t) dev->coherent_dma_mask;
+			size = size ? : (unsigned long) dev->dma_mask;

Hmm, this could be *huge* with 64-bit capable DMA controllers (think LPAE).

Yes, agreed.
(And even for 32-bit DMA this requires a large bitmap_size for the mapping.)

Quoted 5 lines

Russell also has some pending dma mask cleanup, which might break some assumptions here:

http://lists.infradead.org/pipermail/linux-arm-kernel/2013-September/199397.html

Quoted 7 lines
Quoted 1 line

(namely that we're offsetting everything from zero).

What do you think is a reasonable general value for the size of a mapping? (Do we need a DT property to specify this?)

What about a size of 128 MB -- if I'm not mistaken this requires a bitmap_size of 4K.

I don't think we can reasonably pick a sane number for this. We *could* try doing it lazily (i.e. driving the mapping off a fault handler) but that's pretty scary and probably doesn't interact well with endpoints incompatible with the stall model (e.g. PCIe).

So the right solution is probably to get this information from the device drivers, but I don't know what the interaction is with the dma_mask... You'll need to do some digging.

Will

#38 lore raw

PATCH 8/9 iommu/arm-smmu: Introduce a default fault handler

In reply to Andreas Herrmann

And register the default handler for domains that are created during device isolation.

---
 drivers/iommu/arm-smmu.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/iommu/arm-smmu.c +10 -0
index 48f3bfb..380c2a0 100644
@@ -1816,6 +1816,13 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	return 0;
 }
 
+static int arm_smmu_context_fault_handler(struct iommu_domain *domain,
+		struct device *dev, unsigned long iova, int flags, void *arg)
+{
+	dev_warn(dev, "context fault: iova=0x%08lx, flags=0x%x\n", iova, flags);
+	return 0;
+}
+
 extern struct platform_device *of_find_device_by_node(struct device_node *np);
 
 static int arm_smmu_isolate_devices(void)
@@ -1859,6 +1866,9 @@ static int arm_smmu_isolate_devices(void)
 				goto out;
 			}
 
+			iommu_set_fault_handler(mapping->domain,
+					arm_smmu_context_fault_handler, dev);
+
 			ret = arm_iommu_attach_device(dev, mapping);
 			if (ret < 0) {
 				dev_info(dev, "arm_iommu_attach_device failed\n");
Signature · 1 line
1.7.9.5
#39 lore raw

PATCH 8/9 iommu/arm-smmu: Introduce a default fault handler

Quoted 13 lines · Andreas Herrmann

And register the default handler for domains that are created during device isolation.

---
 drivers/iommu/arm-smmu.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/iommu/arm-smmu.c +0 -0
index 48f3bfb..380c2a0 100644

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1823

Quoted 8 lines
drivers/iommu/arm-smmu.c +5 -0
@@ -1816,6 +1816,13 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	return 0;
 }
 
+static int arm_smmu_context_fault_handler(struct iommu_domain *domain,
+		struct device *dev, unsigned long iova, int flags, void *arg)
+{
+	dev_warn(dev, "context fault: iova=0x%08lx, flags=0x%x\n", iova, flags);
+	return 0;

Maybe we're better off sandwiching this into arm_smmu_context_fault when report_iommu_fault doesn't handle the fault.

Will

#40 lore raw

PATCH 8/9 iommu/arm-smmu: Introduce a default fault handler

Quoted 17 lines · Will Deacon
Quoted 16 lines · Andreas Herrmann

And register the default handler for domains that are created during device isolation.

---
 drivers/iommu/arm-smmu.c |   10 ++++++++++
 1 file changed, 10 insertions(+)
drivers/iommu/arm-smmu.c +0 -0
index 48f3bfb..380c2a0 100644
@@ -1816,6 +1816,13 @@ static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu)
 	return 0;
 }
 
Quoted 8 lines

drivers/iommu/arm-smmu.c static int arm_smmu_device_cfg_probe(struct arm_smmu_device *smmu) · line 1823

Quoted 5 lines
drivers/iommu/arm-smmu.c +5 -0
+static int arm_smmu_context_fault_handler(struct iommu_domain *domain,
+		struct device *dev, unsigned long iova, int flags, void *arg)
+{
+	dev_warn(dev, "context fault: iova=0x%08lx, flags=0x%x\n", iova, flags);
+	return 0;

Maybe we're better off sandwiching this into arm_smmu_context_fault when report_iommu_fault doesn't handle the fault.

... which would allow us to dump the entire cb fault syndrome register. (which has more info than what we get from above flags)

Fine with me.

Andreas

#41 lore raw

PATCH 1/1 iommu/arm-smmu: Print context fault information

Print context fault information when the fault was not handled by report_iommu_fault.

---
 drivers/iommu/arm-smmu.c |    3 +++
 1 file changed, 3 insertions(+)
drivers/iommu/arm-smmu.c +3 -0
index 2a9acc1..5e1dad1 100644
@@ -590,6 +590,9 @@ static irqreturn_t arm_smmu_context_fault(int irq, void *dev)
 		ret = IRQ_HANDLED;
 		resume = RESUME_RETRY;
 	} else {
+		dev_err_ratelimited(smmu->dev, "context fault: iova=0x%08lx, "
+				"fsynr=0x%x, cb=%d\n",
+				iova, fsynr, root_cfg->cbndx);
 		ret = IRQ_NONE;
 		resume = RESUME_TERMINATE;
 	}
Signature · 1 line
1.7.9.5
#42 lore raw

PATCH 9/9 ARM: dts: Add nodes for SMMUs on Calxeda ECX-2000

In reply to Andreas Herrmann
---
 arch/arm/boot/dts/ecx-2000.dts    |   44 +++++++++++++++++++++++++++++++++++--
 arch/arm/boot/dts/ecx-common.dtsi |    9 +++++---
 2 files changed, 48 insertions(+), 5 deletions(-)
arch/arm/boot/dts/ecx-2000.dts +42 -2
index 139b40c..d60beb0 100644
@@ -76,10 +76,11 @@
 	};
 
 	soc {
-		ranges = <0x00000000 0x00000000 0x00000000 0xffffffff>;
+		ranges = <0x0 0x0 0x0 0xffffffff>;
 
 		timer {
-			compatible = "arm,cortex-a15-timer", "arm,armv7-timer"; 			interrupts = <1 13 0xf08>,
+			compatible = "arm,cortex-a15-timer", "arm,armv7-timer";
+			interrupts = <1 13 0xf08>,
 				<1 14 0xf08>,
 				<1 11 0xf08>,
 				<1 10 0xf08>;
@@ -103,6 +104,45 @@
 			interrupts = <0 76 4  0 75 4  0 74 4  0 73 4>;
 		};
 	};
+
+	soc at 920000000 {
+		ranges = <0x9 0x20000000 0x9 0x20000000 0x290000>;
+		#address-cells = <2>;
+		#size-cells = <1>;
+		compatible = "simple-bus";
+		interrupt-parent = <&intc>;
+
+		smmu_mac0: smmu at 920000000 {
+			compatible = "arm,mmu-400";
+			reg = <0x9 0x20000000 0x10000>;
+			#global-interrupts = <1>;
+			interrupts = <0 106 4 0 106 4>;
+			mmu-masters = <&mac0 0x0 0x1>;
+			calxeda,smmu-secure-cfg-access;
+			linux,arm-smmu-isolate-devices;
+		};
+
+		smmu_mac1: smmu at 920080000 {
+			compatible = "arm,mmu-400";
+			reg = <0x9 0x20080000 0x10000>;
+			#global-interrupts = <1>;
+			interrupts = <0 108 4 0 108 4>;
+			mmu-masters = <&mac1 0x0 0x1>;
+			calxeda,smmu-secure-cfg-access;
+			linux,arm-smmu-isolate-devices;
+		};
+
+		smmu_sata: smmu at 920180000 {
+			compatible = "arm,mmu-400";
+			reg = <0x00000009 0x20180000 0x10000>;
+			mmu-masters = <&sata 0x0 0x1>;
+			#global-interrupts = <1>;
+			interrupts = <0 114 4 0 114 4>;
+			calxeda,smmu-secure-cfg-access;
+			linux,arm-smmu-isolate-devices;
+		};
+	};
+
 };
 
 /include/ "ecx-common.dtsi"
arch/arm/boot/dts/ecx-common.dtsi +6 -3
index e8559b7..6658710 100644
@@ -25,11 +25,12 @@
 		compatible = "simple-bus";
 		interrupt-parent = <&intc>;
 
-		sata at ffe08000 {
+		sata: sata at ffe08000 {
 			compatible = "calxeda,hb-ahci";
 			reg = <0xffe08000 0x10000>;
 			interrupts = <0 83 4>;
 			dma-coherent;
+			#stream-id-cells = <2>;
 			calxeda,port-phys = <&combophy5 0 &combophy0 0
 					     &combophy0 1 &combophy0 2
 					     &combophy0 3>;
@@ -208,18 +209,20 @@
 			clock-names = "apb_pclk";
 		};
 
-		ethernet at fff50000 {
+		mac0: ethernet at fff50000 {
 			compatible = "calxeda,hb-xgmac";
 			reg = <0xfff50000 0x1000>;
 			interrupts = <0 77 4  0 78 4  0 79 4>;
 			dma-coherent;
+			#stream-id-cells = <2>;
 		};
 
-		ethernet at fff51000 {
+		mac1: ethernet at fff51000 {
 			compatible = "calxeda,hb-xgmac";
 			reg = <0xfff51000 0x1000>;
 			interrupts = <0 80 4  0 81 4  0 82 4>;
 			dma-coherent;
+			#stream-id-cells = <2>;
 		};
 
 		combophy0: combo-phy at fff58000 {
Signature · 1 line
1.7.9.5