Skip to contentSkip to conversation

iommu: Make core iommu-groups code more generic

PATCH In review Joerg Roedel 3 participants 15 messages Oct 21, 2015 – Dec 1, 2015 1 message not in the archive IOMMU

lore
Share & tools
lore

Joerg Roedel

author maintainer
#0 lore raw

cover iommu: Make core iommu-groups code more generic

Hi,

this patch-set makes the core code for managing iommu-groups more generic by lifting its dependencies on PCI. The core function iommu_group_get_for_dev() had a hard dev_is_pci() check in it, followed by PCI specific handling.

This check is removed in favour of the the revived device_group() iommu-ops call-back. With this call-back an IOMMU driver can define how the devices it manages are grouped together. Two functions, one generic and one for PCI devices, are provided that can be uses as a device_group() call-back or might be used in such a call-back.

The existing drivers making use of the iommu_group_get_for_dev() function are converted to this call-back.

Joerg

Joerg Roedel (8):
iommu: Revive device_group iommu-ops call-back
iommu: Export and rename iommu_group_get_for_pci_dev()
iommu: Add generic_device_group() function
iommu: Add device_group call-back to x86 iommu drivers iommu/fsl: Convert to device_group call-back iommu/arm-smmu: Switch to device_group call-back
iommu: Remove is_pci_dev() fall-back from iommu_group_get_for_dev
iommu: Move default domain allocation to iommu_group_get_for_dev()

 drivers/iommu/amd_iommu.c       |  1 +
 drivers/iommu/arm-smmu-v3.c     |  1 +
 drivers/iommu/arm-smmu.c        | 77 ++++++++++++++++++++++++-----------------
 drivers/iommu/fsl_pamu_domain.c | 41 +++++++++-------------
 drivers/iommu/intel-iommu.c     |  1 +
 drivers/iommu/iommu.c           | 46 +++++++++++++++++-------
 include/linux/iommu.h           |  7 +++-
 7 files changed, 105 insertions(+), 69 deletions(-)
Signature · 1 line
1.9.1
author maintainer
#1 lore raw

PATCH 1/8 iommu: Revive device_group iommu-ops call-back

From: Joerg Roedel <[email protected]>

That call-back is currently unused, change it into a call-back function for finding the right IOMMU group for a device.
This is a first step to remove the hard-coded PCI dependency in the iommu-group code.

---
 drivers/iommu/iommu.c | 9 ++++++---
 include/linux/iommu.h | 2 +-
 2 files changed, 7 insertions(+), 4 deletions(-)
drivers/iommu/iommu.c +6 -3
index 049df49..5637463 100644
@@ -814,6 +814,7 @@ static struct iommu_group *iommu_group_get_for_pci_dev(struct pci_dev *pdev)
  */
 struct iommu_group *iommu_group_get_for_dev(struct device *dev)
 {
+	const struct iommu_ops *ops = dev->bus->iommu_ops;
 	struct iommu_group *group;
 	int ret;
 
@@ -821,10 +822,12 @@ struct iommu_group *iommu_group_get_for_dev(struct device *dev)
 	if (group)
 		return group;
 
-	if (!dev_is_pci(dev))
-		return ERR_PTR(-EINVAL);
+	group = ERR_PTR(-EINVAL);
 
-	group = iommu_group_get_for_pci_dev(to_pci_dev(dev));
+	if (ops && ops->device_group)
+		group = ops->device_group(dev);
+	else if (dev_is_pci(dev))
+		group = iommu_group_get_for_pci_dev(to_pci_dev(dev));
 
 	if (IS_ERR(group))
 		return group;
include/linux/iommu.h +1 -1
index f9c1b6d..5a19952 100644
@@ -167,7 +167,7 @@ struct iommu_ops {
 	phys_addr_t (*iova_to_phys)(struct iommu_domain *domain, dma_addr_t iova);
 	int (*add_device)(struct device *dev);
 	void (*remove_device)(struct device *dev);
-	int (*device_group)(struct device *dev, unsigned int *groupid);
+	struct iommu_group *(*device_group)(struct device *dev);
 	int (*domain_get_attr)(struct iommu_domain *domain,
 			       enum iommu_attr attr, void *data);
 	int (*domain_set_attr)(struct iommu_domain *domain,
Signature · 1 line
1.9.1
author maintainer
#2 lore raw

PATCH 2/8 iommu: Export and rename iommu_group_get_for_pci_dev()

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

Rename that function to pci_device_group() and export it, so that IOMMU drivers can use it as their device_group call-back.

---
 drivers/iommu/iommu.c | 8 ++++++--
 include/linux/iommu.h | 3 +++
 2 files changed, 9 insertions(+), 2 deletions(-)
drivers/iommu/iommu.c +6 -2
index 5637463..fdea700 100644
@@ -731,13 +731,17 @@ static int get_pci_alias_or_group(struct pci_dev *pdev, u16 alias, void *opaque)
  * Use standard PCI bus topology, isolation features, and DMA alias quirks
  * to find or create an IOMMU group for a device.
  */
-static struct iommu_group *iommu_group_get_for_pci_dev(struct pci_dev *pdev)
+struct iommu_group *pci_device_group(struct device *dev)
 {
+	struct pci_dev *pdev = to_pci_dev(dev);
 	struct group_for_pci_data data;
 	struct pci_bus *bus;
 	struct iommu_group *group = NULL;
 	u64 devfns[4] = { 0 };
 
+	if (WARN_ON(!dev_is_pci(dev)))
+		return ERR_PTR(-EINVAL);
+
 	/*
 	 * Find the upstream DMA alias for the device.  A device must not
 	 * be aliased due to topology in order to have its own IOMMU group.
@@ -827,7 +831,7 @@ struct iommu_group *iommu_group_get_for_dev(struct device *dev)
 	if (ops && ops->device_group)
 		group = ops->device_group(dev);
 	else if (dev_is_pci(dev))
-		group = iommu_group_get_for_pci_dev(to_pci_dev(dev));
+		group = pci_device_group(dev);
 
 	if (IS_ERR(group))
 		return group;
include/linux/iommu.h +3 -0
index 5a19952..8026e6a 100644
@@ -316,6 +316,9 @@ static inline size_t iommu_map_sg(struct iommu_domain *domain,
 	return domain->ops->map_sg(domain, iova, sg, nents, prot);
 }
 
+/* PCI device grouping function */
+extern struct iommu_group *pci_device_group(struct device *dev);
+
 #else /* CONFIG_IOMMU_API */
 
 struct iommu_ops {};
Signature · 1 line
1.9.1
author maintainer
#3 lore raw

PATCH 3/8 iommu: Add generic_device_group() function

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

This function can be used as a device_group call-back and just allocates one iommu-group per device.

---
 drivers/iommu/iommu.c | 15 +++++++++++++++
 include/linux/iommu.h |  2 ++
 2 files changed, 17 insertions(+)
drivers/iommu/iommu.c +15 -0
index fdea700..a80c9c5 100644
@@ -728,6 +728,21 @@ static int get_pci_alias_or_group(struct pci_dev *pdev, u16 alias, void *opaque)
 }
 
 /*
+ * Generic device_group call-back function. It just allocates one
+ * iommu-group per device.
+ */
+struct iommu_group *generic_device_group(struct device *dev)
+{
+	struct iommu_group *group;
+
+	group = iommu_group_alloc();
+	if (IS_ERR(group))
+		return NULL;
+
+	return group;
+}
+
+/*
  * Use standard PCI bus topology, isolation features, and DMA alias quirks
  * to find or create an IOMMU group for a device.
  */
include/linux/iommu.h +2 -0
index 8026e6a..c73824f 100644
@@ -318,6 +318,8 @@ static inline size_t iommu_map_sg(struct iommu_domain *domain,
 
 /* PCI device grouping function */
 extern struct iommu_group *pci_device_group(struct device *dev);
+/* Generic device grouping function */
+extern struct iommu_group *generic_device_group(struct device *dev);
 
 #else /* CONFIG_IOMMU_API */
 
Signature · 1 line
1.9.1
author maintainer
#4 lore raw

PATCH 4/8 iommu: Add device_group call-back to x86 iommu drivers

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

Set the device_group call-back to pci_device_group() for the Intel VT-d and the AMD IOMMU driver.

---
 drivers/iommu/amd_iommu.c   | 1 +
 drivers/iommu/intel-iommu.c | 1 +
 2 files changed, 2 insertions(+)
drivers/iommu/amd_iommu.c +1 -0
index 08d2775..a76a3e5 100644
@@ -3198,6 +3198,7 @@ static const struct iommu_ops amd_iommu_ops = {
 	.iova_to_phys = amd_iommu_iova_to_phys,
 	.add_device = amd_iommu_add_device,
 	.remove_device = amd_iommu_remove_device,
+	.device_group = pci_device_group,
 	.get_dm_regions = amd_iommu_get_dm_regions,
 	.put_dm_regions = amd_iommu_put_dm_regions,
 	.pgsize_bitmap	= AMD_IOMMU_PGSIZES,
drivers/iommu/intel-iommu.c +1 -0
index 35365f0..f2b276d 100644
@@ -4873,6 +4873,7 @@ static const struct iommu_ops intel_iommu_ops = {
 	.iova_to_phys	= intel_iommu_iova_to_phys,
 	.add_device	= intel_iommu_add_device,
 	.remove_device	= intel_iommu_remove_device,
+	.device_group   = pci_device_group,
 	.pgsize_bitmap	= INTEL_IOMMU_PGSIZES,
 };
 
Signature · 1 line
1.9.1
author maintainer
#5 lore raw

PATCH 5/8 iommu/fsl: Convert to device_group call-back

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

Convert the fsl pamu driver to make use of the new device_group call-back.

---
 drivers/iommu/fsl_pamu_domain.c | 41 ++++++++++++++++-------------------------
 1 file changed, 16 insertions(+), 25 deletions(-)
drivers/iommu/fsl_pamu_domain.c +16 -25
index 1d45293..da0e1e3 100644
@@ -923,7 +923,7 @@ static struct iommu_group *get_pci_device_group(struct pci_dev *pdev)
 	pci_endpt_partioning = check_pci_ctl_endpt_part(pci_ctl);
 	/* We can partition PCIe devices so assign device group to the device */
 	if (pci_endpt_partioning) {
-		group = iommu_group_get_for_dev(&pdev->dev);
+		group = pci_device_group(&pdev->dev);
 
 		/*
 		 * PCIe controller is not a paritionable entity
@@ -956,44 +956,34 @@ static struct iommu_group *get_pci_device_group(struct pci_dev *pdev)
 	return group;
 }
 
-static int fsl_pamu_add_device(struct device *dev)
+static struct iommu_group *fsl_pamu_device_group(struct device *dev)
 {
 	struct iommu_group *group = ERR_PTR(-ENODEV);
-	struct pci_dev *pdev;
-	const u32 *prop;
-	int ret = 0, len;
+	int len;
 
 	/*
 	 * For platform devices we allocate a separate group for
 	 * each of the devices.
 	 */
-	if (dev_is_pci(dev)) {
-		pdev = to_pci_dev(dev);
-		/* Don't create device groups for virtual PCI bridges */
-		if (pdev->subordinate)
-			return 0;
+	if (dev_is_pci(dev))
+		group = get_pci_device_group(to_pci_dev(dev));
+	else if (of_get_property(dev->of_node, "fsl,liodn", &len))
+		group = get_device_iommu_group(dev);
 
-		group = get_pci_device_group(pdev);
+	return group;
+}
 
-	} else {
-		prop = of_get_property(dev->of_node, "fsl,liodn", &len);
-		if (prop)
-			group = get_device_iommu_group(dev);
-	}
+static int fsl_pamu_add_device(struct device *dev)
+{
+	struct iommu_group *group;
 
+	group = iommu_group_get_for_dev(dev);
 	if (IS_ERR(group))
 		return PTR_ERR(group);
 
-	/*
-	 * Check if device has already been added to an iommu group.
-	 * Group could have already been created for a PCI device in
-	 * the iommu_group_get_for_dev path.
-	 */
-	if (!dev->iommu_group)
-		ret = iommu_group_add_device(group, dev);
-
 	iommu_group_put(group);
-	return ret;
+
+	return 0;
 }
 
 static void fsl_pamu_remove_device(struct device *dev)
@@ -1072,6 +1062,7 @@ static const struct iommu_ops fsl_pamu_ops = {
 	.domain_get_attr = fsl_pamu_get_domain_attr,
 	.add_device	= fsl_pamu_add_device,
 	.remove_device	= fsl_pamu_remove_device,
+	.device_group   = fsl_pamu_device_group,
 };
 
 int __init pamu_domain_init(void)
Signature · 1 line
1.9.1
author maintainer
#6 lore raw

PATCH 6/8 iommu/arm-smmu: Switch to device_group call-back

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

This converts the ARM SMMU and the SMMUv3 driver to use the new device_group call-back.

---
 drivers/iommu/arm-smmu-v3.c |  1 +
 drivers/iommu/arm-smmu.c    | 77 +++++++++++++++++++++++++++------------------
 2 files changed, 47 insertions(+), 31 deletions(-)
drivers/iommu/arm-smmu-v3.c +1 -0
index 286e890..e9e591c 100644
@@ -1902,6 +1902,7 @@ static struct iommu_ops arm_smmu_ops = {
 	.iova_to_phys		= arm_smmu_iova_to_phys,
 	.add_device		= arm_smmu_add_device,
 	.remove_device		= arm_smmu_remove_device,
+	.device_group		= pci_device_group,
 	.domain_get_attr	= arm_smmu_domain_get_attr,
 	.domain_set_attr	= arm_smmu_domain_set_attr,
 	.pgsize_bitmap		= -1UL, /* Restricted during device attach */
drivers/iommu/arm-smmu.c +46 -31

128 lines not on this page · raw message

Signature · 1 line
1.9.1
author maintainer
#7 lore raw

PATCH 7/8 iommu: Remove is_pci_dev() fall-back from iommu_group_get_for_dev

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

All callers of iommu_group_get_for_dev() provide a device_group call-back now, so this fall-back is no longer needed.

---
 drivers/iommu/iommu.c | 2 --
 1 file changed, 2 deletions(-)
drivers/iommu/iommu.c +0 -2
index a80c9c5..e2b5526 100644
@@ -845,8 +845,6 @@ struct iommu_group *iommu_group_get_for_dev(struct device *dev)
 
 	if (ops && ops->device_group)
 		group = ops->device_group(dev);
-	else if (dev_is_pci(dev))
-		group = pci_device_group(dev);
 
 	if (IS_ERR(group))
 		return group;
Signature · 1 line
1.9.1
author maintainer
#8 lore raw

PATCH 8/8 iommu: Move default domain allocation to iommu_group_get_for_dev()

In reply to Joerg Roedel

From: Joerg Roedel <[email protected]>

Now that the iommu core support for iommu groups is not pci-centric anymore, we can move default domain allocation to the bus independent iommu_group_get_for_dev() function.

---
 drivers/iommu/iommu.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +10 -8
index e2b5526..abae363 100644
@@ -810,14 +810,6 @@ struct iommu_group *pci_device_group(struct device *dev)
 	if (IS_ERR(group))
 		return NULL;
 
-	/*
-	 * Try to allocate a default domain - needs support from the
-	 * IOMMU driver.
-	 */
-	group->default_domain = __iommu_domain_alloc(pdev->dev.bus,
-						     IOMMU_DOMAIN_DMA);
-	group->domain = group->default_domain;
-
 	return group;
 }
 
@@ -849,6 +841,16 @@ struct iommu_group *iommu_group_get_for_dev(struct device *dev)
 	if (IS_ERR(group))
 		return group;
 
+	/*
+	 * Try to allocate a default domain - needs support from the
+	 * IOMMU driver.
+	 */
+	if (!group->default_domain) {
+		group->default_domain = __iommu_domain_alloc(dev->bus,
+							     IOMMU_DOMAIN_DMA);
+		group->domain = group->default_domain;
+	}
+
 	ret = iommu_group_add_device(group, dev);
 	if (ret) {
 		iommu_group_put(group);
Signature · 1 line
1.9.1
#9 lore raw

Re: [PATCH 8/8] iommu: Move default domain allocation to iommu_group_get_for_dev()

Hi Joerg,

Looking at this from an SMMU perspective, I think there's an issue with attaching to the default domain like this.

Quoted 37 lines · Joerg Roedel

From: Joerg Roedel <[email protected]>

Now that the iommu core support for iommu groups is not pci-centric anymore, we can move default domain allocation to the bus independent iommu_group_get_for_dev() function.

---
 drivers/iommu/iommu.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/iommu/iommu.c +3 -8
index e2b5526..abae363 100644
@@ -810,14 +810,6 @@ struct iommu_group *pci_device_group(struct device *dev)
 	if (IS_ERR(group))
 		return NULL;
 
-	/*
-	 * Try to allocate a default domain - needs support from the
-	 * IOMMU driver.
-	 */
-	group->default_domain = __iommu_domain_alloc(pdev->dev.bus,
-						     IOMMU_DOMAIN_DMA);
-	group->domain = group->default_domain;
-
 	return group;
 }
 
@@ -849,6 +841,16 @@ struct iommu_group *iommu_group_get_for_dev(struct device *dev)
 	if (IS_ERR(group))
 		return group;
 
+	/*
+	 * Try to allocate a default domain - needs support from the
+	 * IOMMU driver.

drivers/iommu/iommu.c struct iommu_group *iommu_group_get_for_dev(struct device *dev) · line 856

Quoted 10 lines
drivers/iommu/iommu.c +7 -0
+	 */
+	if (!group->default_domain) {
+		group->default_domain = __iommu_domain_alloc(dev->bus,
+							     IOMMU_DOMAIN_DMA);
+		group->domain = group->default_domain;
+	}
+
 	ret = iommu_group_add_device(group, dev);
 	if (ret) {
 		iommu_group_put(group);

The call to iommu_group_get_for_dev in arm_smmu_add_device will end up calling __iommu_attach_device, since group->domain will now be initialised by the code above. This means the SMMU driver will see an ->attach_dev call for a device that is part-way through an ->add_device callback and will be missing the initialisation necessary for us to idenfity the SMMU instance to which is corresponds. In fact, the iommudata for the group won't be initialised at all, so the whole thing will fail afaict.

Note that I haven't actually taken this for a spin, so I could be missing something.

Will

author maintainer
#10 lore raw

Re: [PATCH 8/8] iommu: Move default domain allocation to iommu_group_get_for_dev()

Hi Will,

Quoted 10 lines · Will Deacon

The call to iommu_group_get_for_dev in arm_smmu_add_device will end up calling __iommu_attach_device, since group->domain will now be initialised by the code above. This means the SMMU driver will see an ->attach_dev call for a device that is part-way through an ->add_device callback and will be missing the initialisation necessary for us to idenfity the SMMU instance to which is corresponds. In fact, the iommudata for the group won't be initialised at all, so the whole thing will fail afaict.

Note that I haven't actually taken this for a spin, so I could be missing something.

Yeah, I havn't looked at how to convert the ARM-SMMU drivers to default domains yet, so the issue you describe above is totally possible.

But there is no way to trigger it yet, because your domain_alloc function can not yet allocate IOMMU_DOMAIN_DMA domains. While converting the issue must be fixed, of course.

I tested this patch-set on an AMD Seattle system and it worked fine there.

Joerg

#11 lore raw
In reply to Joerg Roedel
Quoted 14 lines · Joerg Roedel

Hi,

this patch-set makes the core code for managing iommu-groups more generic by lifting its dependencies on PCI. The core function iommu_group_get_for_dev() had a hard dev_is_pci() check in it, followed by PCI specific handling.

This check is removed in favour of the the revived device_group() iommu-ops call-back. With this call-back an IOMMU driver can define how the devices it manages are grouped together. Two functions, one generic and one for PCI devices, are provided that can be uses as a device_group() call-back or might be used in such a call-back.

Quoted 6 lines

The existing drivers making use of the iommu_group_get_for_dev() function are converted to this call-back.

Joerg

(Resend since some mail servers reject this.)

Hi Joerg,
Thanks for this patch-set. With this patch-set we can implement our
owner device_group(), then we could achieve all the iommu devices into the same m4u iommu-group easily.

This patch-set have been merged into v4.4-rc1 where I will get a
WARN while I test iommu_detach_device, below is the warning log:

(151119_13:39:37.472)WARNING:
at /proj/mtk40525/upstreamdev/v4.4/kernel/mediatek/drivers/iommu/iommu.c:1154 (151119_13:39:37.472)Modules linked in:
(151119_13:39:37.472)CPU: 1 PID: 731 Comm: sh Not tainted 4.4.0-rc1+ #37
(151119_13:39:37.472)Hardware name: MediaTek MT8173 evaluation board (DT)
(151119_13:39:37.472)task: ffffffc076bb4d00 ti: ffffffc076bdc000 task.ti: ffffffc076bdc000
(151119_13:39:37.472)PC is at iommu_detach_device+0x5c/0xb0 (151119_13:39:37.472)LR is at iommu_detach_device+0x30/0xb0
xxx

Quoted 1 line

From the code, there are 2 places confuse me:

1. It seems that the iommu core has supposed that there is only one device in each a group. In the iommu_detach_device/iommu_attach_device there is a checking like this:

        if (iommu_group_device_count(group) != 1) {
                WARN_ON(1);
                goto out_unlock;
        }

Then for our case(there are many devices in a group), Could we delete
this checking?

2. iommu_detach_device will call __iommu_detach_group in which it will return directly if group->iommu==group->default_iommu, Unfortunately both will be the same if we call iommu_group_get_for_dev(). then iommu_detach_device cann't finish the detach operation.
And if there are many devices in a group, It seems that we can not
detach whole the group in the iommu_detach_device.

/* ============================= */
Below is our add_device() and device_group(), if we call the wrong iommu interface, please also tell me. Thanks.

static int mtk_iommu_add_device(struct device *dev)

{
        struct iommu_group *group;

        if (!dev->archdata.iommu) /* Not a iommu client device */
                return -ENODEV;

        group = iommu_group_get_for_dev(dev);
        if (IS_ERR(group))
                return PTR_ERR(group);

        iommu_group_put(group);
        return 0;
}

static struct iommu_group *mtk_iommu_device_group(struct device *dev)

{
        struct mtk_iommu_data *data;
        struct mtk_iommu_client_priv *priv;

        priv = dev->archdata.iommu;
        if (!priv)
                return ERR_PTR(-ENODEV);

        /* All the client devices are in the same m4u iommu-group */
        data = dev_get_drvdata(priv->m4udev);
        if (!data->m4ugroup) {
                data->m4ugroup = iommu_group_alloc();
                if (IS_ERR(data->m4ugroup))
                        dev_err(dev, "Failed to allocate M4U IOMMU group
\n");
        }
        return data->m4ugroup;
}

/* ============================= */

A message that is not in the archive (sent off-list or not yet received): 1447920801.27650.49.camel@mhfsdcap03

author maintainer
#13 lore raw

Hi Yong Wu,

Quoted 4 lines · Yong Wu

(151119_13:39:37.472)WARNING:
at /proj/mtk40525/upstreamdev/v4.4/kernel/mediatek/drivers/iommu/iommu.c:1154 (151119_13:39:37.472)Modules linked in:
(151119_13:39:37.472)CPU: 1 PID: 731 Comm: sh Not tainted 4.4.0-rc1+

Quoted 8 lines

#37
(151119_13:39:37.472)Hardware name: MediaTek MT8173 evaluation board (DT)
(151119_13:39:37.472)task: ffffffc076bb4d00 ti: ffffffc076bdc000 task.ti: ffffffc076bdc000
(151119_13:39:37.472)PC is at iommu_detach_device+0x5c/0xb0 (151119_13:39:37.472)LR is at iommu_detach_device+0x30/0xb0 xxx

This warning triggers when you use iommu_detach_device() with a device that is in a group with other devices. For those devices you must attach/detach the whole group of devices.

For that you have to use iommu_detach_group() and iommu_attach_group().

Joerg

#14 lore raw
Quoted 16 lines · Joerg Roedel

Hi Yong Wu,

Quoted 4 lines · Yong Wu

(151119_13:39:37.472)WARNING:
at /proj/mtk40525/upstreamdev/v4.4/kernel/mediatek/drivers/iommu/iommu.c:1154 (151119_13:39:37.472)Modules linked in:
(151119_13:39:37.472)CPU: 1 PID: 731 Comm: sh Not tainted 4.4.0-rc1+

Quoted 8 lines

#37
(151119_13:39:37.472)Hardware name: MediaTek MT8173 evaluation board (DT)
(151119_13:39:37.472)task: ffffffc076bb4d00 ti: ffffffc076bdc000 task.ti: ffffffc076bdc000
(151119_13:39:37.472)PC is at iommu_detach_device+0x5c/0xb0 (151119_13:39:37.472)LR is at iommu_detach_device+0x30/0xb0 xxx

Quoted 9 lines

This warning triggers when you use iommu_detach_device() with a device that is in a group with other devices. For those devices you must attach/detach the whole group of devices.

For that you have to use iommu_detach_group() and iommu_attach_group().

Joerg

Currently We have to use iommu_detach_group and iommu_attach_group.
Is there any plan for adding detach/attach the device within a iommu group?

Why I ask this is because that:

There are 2 interfaces in MTK-SMI patch, named by
mtk_smi_larb_get/mtk_smi_larb_put which help enable/disable the power-domain and clocks for each a device(local arbiter). Take a example, VIDEO have to call mtk_smi_larb_get to enable its power and clocks before it works and call mtk_smi_larb_put to disable them after it work done.

If the iommu support attach/detach the device in a iommu-group, then
we could request VIDEO to call iommu_attach_device before it works(the mtk_iommu_attach_device will call the smi interface to enable its power, clocks and enable iommu).and call iommu_detach_device after it work done. then the VIDEO could call iommu_attach_device/iommu_detach_device instead of talking with SMI directly.

author maintainer
#15 lore raw

Hi Yong Wu,

Quoted 3 lines · Yong Wu

Currently We have to use iommu_detach_group and iommu_attach_group.
Is there any plan for adding detach/attach the device within a iommu group?

No, there is no such plan. The reason is using attach/detach_device always also affects the other devices in the iommu-group. Therefore attach/detach_device was limited to devices which have no others in their group.
Having attach/detach_group makes it more obvious to the user that the call affects more than one device.

Joerg