Implement SMMU passthrough using the default domain

The ARM SMMU drivers provide a DOMAIN_ATTR_NESTING domain attribute, which allows callers of the IOMMU API to request that the page table for a domain is installed at stage-2, if supported by the hardware.

Since setting this attribute only makes sense for UNMANAGED domains, this patch returns -ENODEV if the domain_{get,set}_attr operations are called on other domain types.

Signed-off-by Will Deacon <[email protected]>

---
 drivers/iommu/arm-smmu-v3.c | 6 ++++++
 drivers/iommu/arm-smmu.c    | 6 ++++++
 2 files changed, 12 insertions(+)

drivers/iommu/arm-smmu-v3.c +6 -0

index 4d6ec444a9d6..c254325b0c7a 100644
@@ -1839,6 +1839,9 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain,
 {
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
 
+	if (domain->type != IOMMU_DOMAIN_UNMANAGED)
+		return -ENODEV;
+
 	switch (attr) {
 	case DOMAIN_ATTR_NESTING:
 		*(int *)data = (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED);

@@ -1854,6 +1857,9 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain,
 	int ret = 0;
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
 
+	if (domain->type != IOMMU_DOMAIN_UNMANAGED)
+		return -ENODEV;
+
 	mutex_lock(&smmu_domain->init_mutex);
 
 	switch (attr) {

drivers/iommu/arm-smmu.c +6 -0

index a60cded8a6ed..a328ffb75509 100644
@@ -1497,6 +1497,9 @@ static int arm_smmu_domain_get_attr(struct iommu_domain *domain,
 {
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
 
+	if (domain->type != IOMMU_DOMAIN_UNMANAGED)
+		return -ENODEV;
+
 	switch (attr) {
 	case DOMAIN_ATTR_NESTING:
 		*(int *)data = (smmu_domain->stage == ARM_SMMU_DOMAIN_NESTED);

@@ -1512,6 +1515,9 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain,
 	int ret = 0;
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
 
+	if (domain->type != IOMMU_DOMAIN_UNMANAGED)
+		return -ENODEV;
+
 	mutex_lock(&smmu_domain->init_mutex);
 
 	switch (attr) {

For the sake of discussion, would it make sense to enforce this in domain_set_attr() itself? The intersection of drivers providing these callbacks and drivers supporting anything other than unmanaged domains is currently these two below, so it clearly wouldn't break anything to put this check in core code today. Looking forward, is there likely to be any plausible situation where users of a managed domain would be legitimate in mucking about with its attrs, on any platform?

The ARM SMMU drivers provide a DOMAIN_ATTR_NESTING domain attribute, which allows callers of the IOMMU API to request that the page table for a domain is installed at stage-2, if supported by the hardware.

Since setting this attribute only makes sense for UNMANAGED domains, this patch returns -ENODEV if the domain_{get,set}_attr operations are called on other domain types.

Quoted 7 lines · Will Deacon

The ARM SMMU drivers provide a DOMAIN_ATTR_NESTING domain attribute, which allows callers of the IOMMU API to request that the page table for a domain is installed at stage-2, if supported by the hardware.

Since setting this attribute only makes sense for UNMANAGED domains, this patch returns -ENODEV if the domain_{get,set}_attr operations are called on other domain types.

Isn't -EINVAL more suitable here? In the end the domain passed in is invalid because it does not support attributes, no?

In preparation for allowing the default domain type to be overridden, this patch adds support for IOMMU_DOMAIN_IDENTITY domains to the ARM SMMU driver.

An identity domain is created by placing the corresponding S2CR registers into "bypass" mode, which allows transactions to flow through the SMMU without any translation.

Signed-off-by Will Deacon <[email protected]>

---
 drivers/iommu/arm-smmu.c | 20 +++++++++++++++++---
 1 file changed, 17 insertions(+), 3 deletions(-)

drivers/iommu/arm-smmu.c +17 -3

index a328ffb75509..0f5e42a719e5 100644
@@ -404,6 +404,7 @@ enum arm_smmu_domain_stage {
 	ARM_SMMU_DOMAIN_S1 = 0,
 	ARM_SMMU_DOMAIN_S2,
 	ARM_SMMU_DOMAIN_NESTED,
+	ARM_SMMU_DOMAIN_BYPASS,
 };
 
 struct arm_smmu_domain {

@@ -824,6 +825,12 @@ static int arm_smmu_init_domain_context(struct iommu_domain *domain,
 	if (smmu_domain->smmu)
 		goto out_unlock;
 
+	if (domain->type == IOMMU_DOMAIN_IDENTITY) {
+		smmu_domain->stage = ARM_SMMU_DOMAIN_BYPASS;
+		smmu_domain->smmu = smmu;
+		goto out_unlock;
+	}
+
 	/*
 	 * Mapping the requested stage onto what we support is surprisingly
 	 * complicated, mainly because the spec allows S1+S2 SMMUs without

@@ -984,7 +991,7 @@ static void arm_smmu_destroy_domain_context(struct iommu_domain *domain)
 	void __iomem *cb_base;
 	int irq;
 
-	if (!smmu)
+	if (!smmu || domain->type == IOMMU_DOMAIN_IDENTITY)
 		return;
 
 	/*

@@ -1007,7 +1014,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type)
 {
 	struct arm_smmu_domain *smmu_domain;
 
-	if (type != IOMMU_DOMAIN_UNMANAGED && type != IOMMU_DOMAIN_DMA)
+	if (type != IOMMU_DOMAIN_UNMANAGED &&
+	    type != IOMMU_DOMAIN_DMA &&
+	    type != IOMMU_DOMAIN_IDENTITY)
 		return NULL;
 	/*
 	 * Allocate the domain and initialise some of its data structures.

@@ -1205,10 +1214,15 @@ static int arm_smmu_domain_add_master(struct arm_smmu_domain *smmu_domain,
 {
 	struct arm_smmu_device *smmu = smmu_domain->smmu;
 	struct arm_smmu_s2cr *s2cr = smmu->s2crs;
-	enum arm_smmu_s2cr_type type = S2CR_TYPE_TRANS;
 	u8 cbndx = smmu_domain->cfg.cbndx;
+	enum arm_smmu_s2cr_type type;
 	int i, idx;
 
+	if (smmu_domain->stage == ARM_SMMU_DOMAIN_BYPASS)
+		type = S2CR_TYPE_BYPASS;
+	else
+		type = S2CR_TYPE_TRANS;
+
 	for_each_cfg_sme(fwspec, i, idx) {
 		if (type == s2cr[idx].type && cbndx == s2cr[idx].cbndx)
 			continue;

In preparation for allowing the default domain type to be overridden, this patch adds support for IOMMU_DOMAIN_IDENTITY domains to the ARM SMMUv3 driver.

An identity domain is created by placing the corresponding stream table entries into "bypass" mode, which allows transactions to flow through the SMMU without any translation.

Signed-off-by Will Deacon <[email protected]>

---
 drivers/iommu/arm-smmu-v3.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

drivers/iommu/arm-smmu-v3.c +9 -1

index c254325b0c7a..d33291274455 100644
@@ -629,6 +629,7 @@ enum arm_smmu_domain_stage {
 	ARM_SMMU_DOMAIN_S1 = 0,
 	ARM_SMMU_DOMAIN_S2,
 	ARM_SMMU_DOMAIN_NESTED,
+	ARM_SMMU_DOMAIN_BYPASS,
 };
 
 struct arm_smmu_domain {

@@ -1385,7 +1386,9 @@ static struct iommu_domain *arm_smmu_domain_alloc(unsigned type)
 {
 	struct arm_smmu_domain *smmu_domain;
 
-	if (type != IOMMU_DOMAIN_UNMANAGED && type != IOMMU_DOMAIN_DMA)
+	if (type != IOMMU_DOMAIN_UNMANAGED &&
+	    type != IOMMU_DOMAIN_DMA &&
+	    type != IOMMU_DOMAIN_IDENTITY)
 		return NULL;
 
 	/*

@@ -1516,6 +1519,11 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain)
 	struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
 	struct arm_smmu_device *smmu = smmu_domain->smmu;
 
+	if (domain->type == IOMMU_DOMAIN_IDENTITY) {
+		smmu_domain->stage = ARM_SMMU_DOMAIN_BYPASS;
+		return 0;
+	}
+
 	/* Restrict the stage to what we can actually support */
 	if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S1))
 		smmu_domain->stage = ARM_SMMU_DOMAIN_S2;

drivers/iommu/arm-smmu-v3.c +1 -0

+	} else if (smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
 		ste->s1_cfg = &smmu_domain->s1_cfg;
 		ste->s2_cfg = NULL;
 		arm_smmu_write_ctx_desc(smmu, ste->s1_cfg);

The arm64 DMA-mapping implementation sets the DMA ops to the IOMMU DMA ops if we detect that an IOMMU is present for the master and the DMA ranges are valid.

In the case when the IOMMU domain for the device is not of type IOMMU_DOMAIN_DMA, then we have no business swizzling the ops, since we're not in control of the underlying address space. This patch leaves the DMA ops alone for masters attached to non-DMA IOMMU domains.

Signed-off-by Will Deacon <[email protected]>

---
 arch/arm64/mm/dma-mapping.c | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

arch/arm64/mm/dma-mapping.c +12 -5

index e04082700bb1..5d3c6ad621e8 100644
@@ -831,14 +831,21 @@ static bool do_iommu_attach(struct device *dev, const struct iommu_ops *ops,
 	 * then the IOMMU core will have already configured a group for this
 	 * device, and allocated the default domain for that group.
 	 */
-	if (!domain || iommu_dma_init_domain(domain, dma_base, size, dev)) {
-		pr_warn("Failed to set up IOMMU for device %s; retaining platform DMA ops\n",
-			dev_name(dev));
-		return false;
+	if (!domain)
+		goto out_err;
+
+	if (domain->type == IOMMU_DOMAIN_DMA) {
+		if (iommu_dma_init_domain(domain, dma_base, size, dev))
+			goto out_err;
+
+		dev->archdata.dma_ops = &iommu_dma_ops;
 	}
 
-	dev->archdata.dma_ops = &iommu_dma_ops;
 	return true;
+out_err:
+	pr_warn("Failed to set up IOMMU for device %s; retaining platform DMA ops\n",
+		 dev_name(dev));
+	return false;
 }
 
 static void queue_iommu_attach(struct device *dev, const struct iommu_ops *ops,

Quoted 8 lines · Will Deacon

The arm64 DMA-mapping implementation sets the DMA ops to the IOMMU DMA ops if we detect that an IOMMU is present for the master and the DMA ranges are valid.

In the case when the IOMMU domain for the device is not of type IOMMU_DOMAIN_DMA, then we have no business swizzling the ops, since we're not in control of the underlying address space. This patch leaves the DMA ops alone for masters attached to non-DMA IOMMU domains.

In fact, I don't think there would be any harm in taking this one through arm64 straight away. The DMA ops can't be expected to work successfully on any old domain, so it's a reasonable sanity check regardless.

Reviewed-by Robin Murphy <[email protected]>

Rather than modify each IOMMU driver to provide different semantics for DMA domains, instead we introduce a command line parameter that can be used to change the type of the default domain. Passthrough can then be specified using "iommu.default_domain=identity" on the kernel command line.

Quoted 5 lines · Will Deacon

Rather than modify each IOMMU driver to provide different semantics for DMA domains, instead we introduce a command line parameter that can be used to change the type of the default domain. Passthrough can then be specified using "iommu.default_domain=identity" on the kernel command line.

I like the general idea of this, but the above is a terrible name for a kernel commandline-parameter. The x86 iommus support iommu=pt which is pretty much the same as this patch does.

How about something like "iommu.passthrough=0/1"? And please add the parameter to the kernel documentation too.

Joerg

Quoted 5 lines · Will Deacon

Rather than modify each IOMMU driver to provide different semantics for DMA domains, instead we introduce a command line parameter that can be used to change the type of the default domain. Passthrough can then be specified using "iommu.default_domain=identity" on the kernel command line.

I like the general idea of this, but the above is a terrible name for a kernel commandline-parameter. The x86 iommus support iommu=pt which is pretty much the same as this patch does.

How about something like "iommu.passthrough=0/1"? And please add the parameter to the kernel documentation too.

Hi all,

A number of people have expressed interest in having the SMMU come up in a passthrough configuration, and then allow subsequent translation for things such as VFIO. Rather than do this in each SMMU driver, it's much cleaner to allow the default domain to be configured to be something other than DMA.

This patch series implements a command-line option to configure the default domain type. Currently, it supports "dma" and "identity" which is sufficient for the passthrough use-case.

Tested on an ARM fastmodel.

All feedback welcome,

Will

--->8

Will Deacon (5):
iommu/arm-smmu: Restrict domain attributes to UNMANAGED domains iommu/arm-smmu: Install bypass S2CRs for IOMMU_DOMAIN_IDENTITY domains iommu/arm-smmu-v3: Install bypass STEs for IOMMU_DOMAIN_IDENTITY
domains
arm64: dma-mapping: Only swizzle DMA ops for IOMMU_DOMAIN_DMA
iommu: Allow default domain type to be set on the kernel command line

arch/arm64/mm/dma-mapping.c | 17 ++++++++++++-----
drivers/iommu/arm-smmu-v3.c | 20 ++++++++++++++++++--
drivers/iommu/arm-smmu.c    | 26 +++++++++++++++++++++++---
drivers/iommu/iommu.c       | 19 +++++++++++++++++--
4 files changed, 70 insertions(+), 12 deletions(-)

--
2.1.4

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between process pagetables. GPU driver was not using DMA domain and the GPU's firmware was always expecting to run out of contextbank
'0' (although not correct) , which was not the case after the DMA domain
was made default as '0' was getting allocated for DMA domain and there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Quoted 11 lines · Sricharan

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between process pagetables. GPU driver was not using DMA domain and the GPU's firmware was always expecting to run out of contextbank
'0' (although not correct) , which was not the case after the DMA domain
was made default as '0' was getting allocated for DMA domain and there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Note that iommu-drivers can request direct-mapping for any given device on its initializtion. This is used on x86 for devices that need a 1-1 mapping for some reason.

Also device drivers can use the iommu-api and assign their own domain to a device, which allows them to manage the dma address space on their own.

Quoted 13 lines · Joerg Roedel

Quoted 11 lines · Sricharan

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between process pagetables. GPU driver was not using DMA domain and the GPU's firmware was always expecting to run out of contextbank
'0' (although not correct) , which was not the case after the DMA domain
was made default as '0' was getting allocated for DMA domain and there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Quoted 7 lines

Note that iommu-drivers can request direct-mapping for any given device on its initializtion. This is used on x86 for devices that need a 1-1 mapping for some reason.

Also device drivers can use the iommu-api and assign their own domain to a device, which allows them to manage the dma address space on their own.

Part of the problem is that dev->archdata.dma_ops gets wired up to iommu_dma_ops. Which isn't so bad on it's own, except that cache ops are not exposed to drivers, forcing us to use dma-mapping API (dma_map_sg, etc) for cache operations.

Possibly we should just expose cache op's to drivers bypass this abuse of dma-mapping.

btw, Will, we definitely want this to *not* rely on kcmdline for the gpu with it's own private iommu case..

Quoted 27 lines · Rob Clark

Quoted 13 lines · Joerg Roedel

Quoted 11 lines · Sricharan

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between process pagetables. GPU driver was not using DMA domain and the GPU's firmware was always expecting to run out of contextbank
'0' (although not correct) , which was not the case after the DMA domain
was made default as '0' was getting allocated for DMA domain and there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Quoted 7 lines

Note that iommu-drivers can request direct-mapping for any given device on its initializtion. This is used on x86 for devices that need a 1-1 mapping for some reason.

Also device drivers can use the iommu-api and assign their own domain to a device, which allows them to manage the dma address space on their own.

Part of the problem is that dev->archdata.dma_ops gets wired up to iommu_dma_ops. Which isn't so bad on it's own, except that cache ops are not exposed to drivers, forcing us to use dma-mapping API (dma_map_sg, etc) for cache operations.

Quoted 5 lines

Possibly we should just expose cache op's to drivers bypass this abuse of dma-mapping.

btw, Will, we definitely want this to *not* rely on kcmdline for the gpu with it's own private iommu case..

I still need to understand the unmanaged domain case, but I don't really see why that's related to this series to be honest.

-----Original Message-----
From: linux-arm-kernel [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Rob Clark
Sent: Thursday, February 02, 2017 8:33 PM
To: Joerg Roedel <[email protected]>
Cc: Will Deacon <[email protected]>; iommu at lists.linux-foundation.org; Sricharan <[email protected]>; linux-arm- kernel at lists.infradead.org
Subject: Re: [PATCH 0/5] Implement SMMU passthrough using the default domain

Quoted 13 lines · Joerg Roedel

Quoted 11 lines · Sricharan

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between process pagetables. GPU driver was not using DMA domain and the GPU's firmware was always expecting to run out of contextbank
'0' (although not correct) , which was not the case after the DMA domain
was made default as '0' was getting allocated for DMA domain and there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Quoted 7 lines

Note that iommu-drivers can request direct-mapping for any given device on its initializtion. This is used on x86 for devices that need a 1-1 mapping for some reason.

Also device drivers can use the iommu-api and assign their own domain to a device, which allows them to manage the dma address space on their own.

Part of the problem is that dev->archdata.dma_ops gets wired up to iommu_dma_ops. Which isn't so bad on it's own, except that cache ops are not exposed to drivers, forcing us to use dma-mapping API (dma_map_sg, etc) for cache operations.

Possibly we should just expose cache op's to drivers bypass this abuse of dma-mapping.

btw, Will, we definitely want this to *not* rely on kcmdline for the gpu with it's own private iommu case..

BR,
-R

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel at lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Hi Rob,

Quoted 30 lines

-----Original Message-----
From: linux-arm-kernel [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Rob Clark
Sent: Thursday, February 02, 2017 8:33 PM
To: Joerg Roedel <[email protected]>
Cc: Will Deacon <[email protected]>; iommu at lists.linux-foundation.org; Sricharan <[email protected]>; linux-arm- kernel at lists.infradead.org
Subject: Re: [PATCH 0/5] Implement SMMU passthrough using the default domain

Quoted 13 lines · Joerg Roedel

Quoted 11 lines · Sricharan

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between process pagetables. GPU driver was not using DMA domain and the GPU's firmware was always expecting to run out of contextbank
'0' (although not correct) , which was not the case after the DMA domain
was made default as '0' was getting allocated for DMA domain and there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Quoted 7 lines

Note that iommu-drivers can request direct-mapping for any given device on its initializtion. This is used on x86 for devices that need a 1-1 mapping for some reason.

Also device drivers can use the iommu-api and assign their own domain to a device, which allows them to manage the dma address space on their own.

Quoted 8 lines

Part of the problem is that dev->archdata.dma_ops gets wired up to iommu_dma_ops. Which isn't so bad on it's own, except that cache ops are not exposed to drivers, forcing us to use dma-mapping API (dma_map_sg, etc) for cache operations.

Possibly we should just expose cache op's to drivers bypass this abuse of dma-mapping.

[1], with this, when the default domain in not DOMAIN_DMA, then dev->archdata.dma_ops is not set to iommu_dma_ops , instead remains to be swiotlb_ops. Is that not correct for gpu's unmanaged domain case ?

https://www.spinics.net/lists/arm-kernel/msg556209.html

Quoted 2 lines

btw, Will, we definitely want this to *not* rely on kcmdline for the gpu with it's own private iommu case..

Ya, that changes behavior for all devices and some might want DMA_DOMAIN and some UNMANAGED.

-----Original Message-----
From: linux-arm-kernel [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Will Deacon
Sent: Thursday, February 02, 2017 9:41 PM
To: Sricharan <[email protected]>
Cc: linux-arm-kernel at lists.infradead.org; 'Rob Clark' <[email protected]>; 'Joerg Roedel' <[email protected]>; iommu at lists.linux- foundation.org
Subject: Re: [PATCH 0/5] Implement SMMU passthrough using the default domain

Quoted 49 lines · Sricharan

Hi Rob,

Quoted 30 lines

-----Original Message-----
From: linux-arm-kernel [mailto:linux-arm-kernel-bounces at lists.infradead.org] On Behalf Of Rob Clark
Sent: Thursday, February 02, 2017 8:33 PM
To: Joerg Roedel <[email protected]>
Cc: Will Deacon <[email protected]>; iommu at lists.linux-foundation.org; Sricharan <[email protected]>; linux-arm- kernel at lists.infradead.org
Subject: Re: [PATCH 0/5] Implement SMMU passthrough using the default domain

Quoted 13 lines · Joerg Roedel

Quoted 11 lines · Sricharan

Thanks for this series. We had a case with the GPU.
The GPU's iommu was setup by kernel and the GPU
also does dynamic updates for on-the-fly switching between process pagetables. GPU driver was not using DMA domain and the GPU's firmware was always expecting to run out of contextbank
'0' (although not correct) , which was not the case after the DMA domain
was made default as '0' was getting allocated for DMA domain and there were concerns about reusing the DMA domain as well.
Now with this series, looks there is an way out of that that can be tried.

So should the default domain not be per device specific selectable ?

Quoted 7 lines

Note that iommu-drivers can request direct-mapping for any given device on its initializtion. This is used on x86 for devices that need a 1-1 mapping for some reason.

Also device drivers can use the iommu-api and assign their own domain to a device, which allows them to manage the dma address space on their own.

Quoted 8 lines

Part of the problem is that dev->archdata.dma_ops gets wired up to iommu_dma_ops. Which isn't so bad on it's own, except that cache ops are not exposed to drivers, forcing us to use dma-mapping API (dma_map_sg, etc) for cache operations.

Possibly we should just expose cache op's to drivers bypass this abuse of dma-mapping.

[1], with this, when the default domain in not DOMAIN_DMA, then dev->archdata.dma_ops is not set to iommu_dma_ops , instead remains to be swiotlb_ops. Is that not correct for gpu's unmanaged domain case ?

https://www.spinics.net/lists/arm-kernel/msg556209.html

Quoted 2 lines

btw, Will, we definitely want this to *not* rely on kcmdline for the gpu with it's own private iommu case..

Quoted 2 lines

Ya, that changes behavior for all devices and some might want DMA_DOMAIN and some UNMANAGED.

My patch changes the *default* domain. When would this ever be UNMANAGED?

If you're using an UNMANAGED domain, I think you need to take care of the DMA ops yourself. There are likely missing functions to do that, but they should be added in a separate series.