Skip to contentSkip to conversation

iommu: Expose an API to set identity domain

PATCH In review Pranjal Shrivastava 4 participants 5 messages Jul 7, 2023 – Jul 10, 2023 1 message not in the archive IOMMU

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 iommu: Expose an API to set identity domain

The iommu driver has a function iommu_change_dev_def_domain that allows the user to change iommu domain of a group through a sysfs entry. Currently, the following steps are required to change the iommu domain via sysfs:

1. Unbind the driver
2. echo domain_type > /sys/kernel/iommu_groups/<grp_id>/type
3. Re-bind the driver

This change introduces a wrapper function around it which allows drivers to change their iommu domain to identity during the driver probe. This faciitates bypassing the iommu for trusted IPs/devices for performance without needing to unbind and re-bind the driver from the user space. Additionally, it helps in achieving this functionality when the sysfs is disabled for a system.

---
 drivers/iommu/iommu.c | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)
drivers/iommu/iommu.c +21 -0
index f1dcfa3f1a1b..36f158621dc5 100644
@@ -13,6 +13,7 @@
 #include <linux/bug.h>
 #include <linux/types.h>
 #include <linux/init.h>
+#include <linux/dma-map-ops.h>
 #include <linux/export.h>
 #include <linux/slab.h>
 #include <linux/errno.h>
@@ -2908,6 +2909,10 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
 	if (ret)
 		goto free_new_domain;
 
+	/* Make sure dma_ops are appropriatley set */
+	if (type == IOMMU_DOMAIN_IDENTITY)
+		set_dma_ops(dev, NULL);
+
 	iommu_domain_free(prev_dom);
 
 	return 0;
@@ -2921,6 +2926,22 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
 	return ret;
 }
 
+/*
+ * Sets the default domain to identity of an iommu group
+ *
+ * @group: The group for which the default domain should be changed
+ * @dev: The first device in the group
+ *
+ * Returns 0 on success and error code on failure
+ * Note:Presently, this function shall be called only if the driver requests to
+ * change the group's default domain type to identity during the driver's probe.
+ */
+int iommu_set_def_identity_domain(struct device *dev, struct iommu_group *group)
+{
+	return iommu_change_dev_def_domain(group, dev, IOMMU_DOMAIN_IDENTITY);
+}
+EXPORT_SYMBOL_GPL(iommu_set_def_identity_domain);
+
 /*
  * Changing the default domain through sysfs requires the users to unbind the
  * drivers from the devices in the iommu group, except for a DMA -> DMA-FQ
Signature · 1 line
2.41.0.255.g8b1d071c50-goog
reviewer
#1 lore raw
Quoted 5 lines · Pranjal Shrivastava

The iommu driver has a function iommu_change_dev_def_domain that allows the user to change iommu domain of a group through a sysfs entry. Currently, the following steps are required to change the iommu domain via sysfs:

Quoted 8 lines

1. Unbind the driver
2. echo domain_type > /sys/kernel/iommu_groups/<grp_id>/type
3. Re-bind the driver

This change introduces a wrapper function around it which allows drivers to change their iommu domain to identity during the driver probe. This faciitates bypassing the iommu for trusted IPs/devices for performance without needing to unbind and re-bind the driver from the user space.

So a malicious endpoint now gets to bypass user policy and pwn the system by simply spoofing any endpoint ID whose driver has decided on the user's behalf that the user trusts it? Brilliant!

Sure, this is probably possible to an extent already with the IOMMU-driver-level quirks for integrated devices, since we're currently not considering the topology of exactly *where* we find those and whether it looks right or not, but c'mon, let's at least not *intentionally* smash a big hole in the trust model and leave an open invitation next to it.

Thanks,
Robin.

Quoted 54 lines

Additionally, it helps in achieving this functionality when the sysfs is disabled for a system.

---
  drivers/iommu/iommu.c | 21 +++++++++++++++++++++
  1 file changed, 21 insertions(+)
drivers/iommu/iommu.c +21 -0
index f1dcfa3f1a1b..36f158621dc5 100644
@@ -13,6 +13,7 @@
  #include <linux/bug.h>
  #include <linux/types.h>
  #include <linux/init.h>
+#include <linux/dma-map-ops.h>
  #include <linux/export.h>
  #include <linux/slab.h>
  #include <linux/errno.h>
@@ -2908,6 +2909,10 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
  	if (ret)
  		goto free_new_domain;
 
+	/* Make sure dma_ops are appropriatley set */
+	if (type == IOMMU_DOMAIN_IDENTITY)
+		set_dma_ops(dev, NULL);
+
  	iommu_domain_free(prev_dom);
 
  	return 0;
@@ -2921,6 +2926,22 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
  	return ret;
  }
 
+/*
+ * Sets the default domain to identity of an iommu group
+ *
+ * @group: The group for which the default domain should be changed
+ * @dev: The first device in the group
+ *
+ * Returns 0 on success and error code on failure
+ * Note:Presently, this function shall be called only if the driver requests to
+ * change the group's default domain type to identity during the driver's probe.
+ */
+int iommu_set_def_identity_domain(struct device *dev, struct iommu_group *group)
+{
+	return iommu_change_dev_def_domain(group, dev, IOMMU_DOMAIN_IDENTITY);
+}
+EXPORT_SYMBOL_GPL(iommu_set_def_identity_domain);
+
  /*
   * Changing the default domain through sysfs requires the users to unbind the
   * drivers from the devices in the iommu group, except for a DMA -> DMA-FQ
#3 lore raw
In reply to a message not in the archive

Hey Robin,

Agreed. It's a security v/s performance trade off. The main issue seems to be when a PCIe endpoint maliciously spoofs its ID. However, I guess we can avoid that by making the API platform device specific, in that case only the device which is a platform device can use the API to say "Hey, I know what I am doing, bypass the SMMU for me". Will that be acceptable?

Please let us know if there are any other ways/suggestions that allow us to install bypass for a single iommu group sitting in front of a platform device (This device is not removable/hot-pluggable and always sits on the system).

We're trying to achieve something similar to
https://lore.kernel.org/linux-iommu/[email protected]/T/#u
and trying to make it generic in the iommu layer and not specific to a certain IOMMU hardware.

Thanks, Pranjal

Quoted 87 lines · Robin Murphy
Quoted 84 lines · Pranjal Shrivastava
Quoted 5 lines

The iommu driver has a function iommu_change_dev_def_domain that allows the user to change iommu domain of a group through a sysfs entry. Currently, the following steps are required to change the iommu domain via sysfs:

Quoted 8 lines

1. Unbind the driver
2. echo domain_type > /sys/kernel/iommu_groups/<grp_id>/type
3. Re-bind the driver

This change introduces a wrapper function around it which allows drivers to change their iommu domain to identity during the driver probe. This faciitates bypassing the iommu for trusted IPs/devices for performance without needing to unbind and re-bind the driver from the user space.

So a malicious endpoint now gets to bypass user policy and pwn the system by simply spoofing any endpoint ID whose driver has decided on the user's behalf that the user trusts it? Brilliant!

Sure, this is probably possible to an extent already with the IOMMU-driver-level quirks for integrated devices, since we're currently not considering the topology of exactly *where* we find those and whether it looks right or not, but c'mon, let's at least not *intentionally* smash a big hole in the trust model and leave an open invitation next to it.

Thanks,
Robin.

Quoted 54 lines

Additionally, it helps in achieving this functionality when the sysfs is disabled for a system.

---
  drivers/iommu/iommu.c | 21 +++++++++++++++++++++
  1 file changed, 21 insertions(+)
drivers/iommu/iommu.c +21 -0
index f1dcfa3f1a1b..36f158621dc5 100644
@@ -13,6 +13,7 @@
  #include <linux/bug.h>
  #include <linux/types.h>
  #include <linux/init.h>
+#include <linux/dma-map-ops.h>
  #include <linux/export.h>
  #include <linux/slab.h>
  #include <linux/errno.h>
@@ -2908,6 +2909,10 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
      if (ret)
              goto free_new_domain;
 
+     /* Make sure dma_ops are appropriatley set */
+     if (type == IOMMU_DOMAIN_IDENTITY)
+             set_dma_ops(dev, NULL);
+
      iommu_domain_free(prev_dom);
 
      return 0;
@@ -2921,6 +2926,22 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
      return ret;
  }
 
+/*
+ * Sets the default domain to identity of an iommu group
+ *
+ * @group: The group for which the default domain should be changed
+ * @dev: The first device in the group
+ *
+ * Returns 0 on success and error code on failure
+ * Note:Presently, this function shall be called only if the driver requests to
+ * change the group's default domain type to identity during the driver's probe.
+ */
+int iommu_set_def_identity_domain(struct device *dev, struct iommu_group *group)
+{
+     return iommu_change_dev_def_domain(group, dev, IOMMU_DOMAIN_IDENTITY);
+}
+EXPORT_SYMBOL_GPL(iommu_set_def_identity_domain);
+
  /*
   * Changing the default domain through sysfs requires the users to unbind the
   * drivers from the devices in the iommu group, except for a DMA -> DMA-FQ
maintainer
#4 lore raw

Hi Pranjal,

Please try to avoid top-posting when you reply to the mailing list, as it breaks the flow of conversation. See:

Documentation/process/submitting-patches.rst

Quoted 6 lines · Pranjal Shrivastava

Agreed. It's a security v/s performance trade off. The main issue seems to be when a PCIe endpoint maliciously spoofs its ID. However, I guess we can avoid that by making the API platform device specific, in that case only the device which is a platform device can use the API to say "Hey, I know what I am doing, bypass the SMMU for me". Will that be acceptable?

I think that's effectively already the case for the sysfs interface, no? Untrusted PCI devices are forced into a default domain of IOMMU_DOMAIN_DMA and you can't change that.

Quoted 4 lines

Please let us know if there are any other ways/suggestions that allow us to install bypass for a single iommu group sitting in front of a platform device (This device is not removable/hot-pluggable and always sits on the system).

I think there are a few issues here:

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/iommu/iommu.c +0 -0
index f1dcfa3f1a1b..36f158621dc5 100644
@@ -13,6 +13,7 @@
  #include <linux/bug.h>
  #include <linux/types.h>
  #include <linux/init.h>
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

drivers/iommu/iommu.c static int iommu_change_dev_def_domain(struct iommu_group *group, · line 2911

Quoted 8 lines
drivers/iommu/iommu.c +1 -0
+#include <linux/dma-map-ops.h>
  #include <linux/export.h>
  #include <linux/slab.h>
  #include <linux/errno.h>
@@ -2908,6 +2909,10 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
      if (ret)
              goto free_new_domain;
 

This code (iommu_change_dev_def_domain()) has changed significantly in the recent merge window, so this doesn't apply.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

drivers/iommu/iommu.c static int iommu_change_dev_def_domain(struct iommu_group *group, · line 2915

Quoted 4 lines
drivers/iommu/iommu.c +4 -0
+     /* Make sure dma_ops are appropriatley set */
+     if (type == IOMMU_DOMAIN_IDENTITY)
+             set_dma_ops(dev, NULL);
+

This is usually handled on arm64 by the unbind operation, which will tear down the old domain and clear the function pointer.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/iommu/iommu.c +0 -0
      iommu_domain_free(prev_dom);
 
      return 0;
@@ -2921,6 +2926,22 @@ static int iommu_change_dev_def_domain(struct iommu_group *group,
      return ret;
  }
 
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
drivers/iommu/iommu.c +7 -0
+/*
+ * Sets the default domain to identity of an iommu group
+ *
+ * @group: The group for which the default domain should be changed
+ * @dev: The first device in the group
+ *
+ * Returns 0 on success and error code on failure
Quoted 8 lines
Quoted 8 lines

drivers/iommu/iommu.c static int iommu_change_dev_def_domain(struct iommu_group *group, · line 2943

Quoted 8 lines
drivers/iommu/iommu.c +8 -0
+ * Note:Presently, this function shall be called only if the driver requests to
+ * change the group's default domain type to identity during the driver's probe.
+ */
+int iommu_set_def_identity_domain(struct device *dev, struct iommu_group *group)
+{
+     return iommu_change_dev_def_domain(group, dev, IOMMU_DOMAIN_IDENTITY);
+}
+EXPORT_SYMBOL_GPL(iommu_set_def_identity_domain);

An exported symbol with no user in the tree is very unlikely to be accepted...

Why can't you do what you need from userspace using the existing interface?

If a new in-kernel interface is required, then I think it (a) needs opt-in from userspace and (b) needs an in-tree user.

Thanks,

Will

#5 lore raw
In reply to Pranjal Shrivastava
Quoted 4 lines · Pranjal Shrivastava

Please let us know if there are any other ways/suggestions that allow us to install bypass for a single iommu group sitting in front of a platform device (This device is not removable/hot-pluggable and always sits on the system).

Maybe the FW should pass in this request that the device is trusted and should get some performance DMA?

Jason