Skip to contentSkip to conversation

iommu: Clean up cookie and sw_msi in struct iommu_domain

PATCH Superseded Nicolin Chen 5 participants 16 messages Mar 3, 2025 – Mar 6, 2025

Subsystems (3)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover iommu: Clean up cookie and sw_msi in struct iommu_domain

A few things in iommu_domain structure (such as iova_cookie, iommufd_hwpt, fault handler, and etc) are exclusive to certain types of iommu_domains. So, all of them can be seen as domain cookies in different cookie types.

Furthermore, iova_cookie is shared with the msi_cookie use case. It could be cleaner by decoupling these too.

Define an enum iommu_domain_cookie_type to fit all those cookies in.

Since there are only two sw_msi implementations, check domain->cookie_type in iommu_dma_prepare_msi() to call the corresponding sw_msi function.

This is a clean-up series for the previous sw_msi Part-1 core series. It's on github:
https://github.com/nicolinc/iommufd/commits/iommufd_msi_cleanup-v3

Changelog
v3
* Integrate Robin's patch
https://lore.kernel.org/linux-iommu/4e68d5820be06adc1b34fc0d1c9399481151daee.1740742271.git.robin.murphy@arm.com/
replacing private_data_owner with the new cookie_type
* Move iommu_sw_msi stub from the public header to iommu-priv.h
v2
https://lore.kernel.org/all/[email protected]/
* Drop sw_msi function pointer
* Add a new private_data_owner tag in iommu_domain
* Let iommu core call the two sw_msi implementations directly
v1
https://lore.kernel.org/all/[email protected]/

Thanks
Nicolin

Nicolin Chen (2):
iommufd: Move iommufd_sw_msi and related functions to driver.c
iommu: Drop sw_msi from iommu_domain

Robin Murphy (1):
iommu: Sort out domain user data

 drivers/iommu/dma-iommu.h               |  14 ++
 drivers/iommu/iommu-priv.h              |  13 ++
 drivers/iommu/iommufd/iommufd_private.h |   7 +-
 include/linux/iommu.h                   |  35 ++--
 drivers/iommu/dma-iommu.c               | 207 ++++++++++++------------
 drivers/iommu/iommu-sva.c               |   1 +
 drivers/iommu/iommu.c                   |  34 +++-
 drivers/iommu/iommufd/device.c          | 127 +--------------
 drivers/iommu/iommufd/driver.c          | 125 ++++++++++++++
 drivers/iommu/iommufd/hw_pagetable.c    |   4 +-
 10 files changed, 313 insertions(+), 254 deletions(-)

base-commit: 5e9f822c9c683ae884fa5e71df41d1647b2876c6

Signature · 1 line
2.43.0
#1 lore raw

PATCH v3 1/3 iommu: Sort out domain user data

From: Robin Murphy <[email protected]>

When DMA/MSI cookies were made first-class citizens back in commit 46983fcd67ac ("iommu: Pull IOVA cookie management into the core"), there was no real need to further expose the two different cookie types. However, now that IOMMUFD wants to add a third type of MSI-mapping cookie, we do have a nicely compelling reason to properly dismabiguate things at the domain level beyond just vaguely guessing from the domain type.

Meanwhile, we also effectively have another "cookie" in the form of the anonymous union for other user data, which isn't much better in terms of being vague and unenforced. The fact is that all these cookie types are mutually exclusive, in the sense that combining them makes zero sense and/or would be catastrophic (iommu_set_fault_handler() on an SVA domain, anyone?) - the only combination which *might* be reasonable is perhaps a fault handler and an MSI cookie, but nobody's doing that at the moment, so let's rule it out as well for the sake of being clear and robust. To that end, we pull DMA and MSI cookies apart a little more, mostly to clear up the ambiguity at domain teardown, then for clarity (and to save a little space), move them into the union, whose ownership we can then properly describe and enforce entirely unambiguously.

[nicolinc: rebase on latest tree; use prefix IOMMU_COOKIE_; merge unions
in iommu_domain; add IOMMU_COOKIE_IOMMUFD for iommufd_hwpt]

---
 drivers/iommu/dma-iommu.h            |   5 +
 include/linux/iommu.h                |  20 ++-
 drivers/iommu/dma-iommu.c            | 193 ++++++++++++++-------------
 drivers/iommu/iommu-sva.c            |   1 +
 drivers/iommu/iommu.c                |  17 ++-
 drivers/iommu/iommufd/hw_pagetable.c |   1 +
 6 files changed, 139 insertions(+), 98 deletions(-)
drivers/iommu/dma-iommu.h +5 -0
index c12d63457c76..9cca11806e5d 100644
@@ -13,6 +13,7 @@ void iommu_setup_dma_ops(struct device *dev);
 
 int iommu_get_dma_cookie(struct iommu_domain *domain);
 void iommu_put_dma_cookie(struct iommu_domain *domain);
+void iommu_put_msi_cookie(struct iommu_domain *domain);
 
 int iommu_dma_init_fq(struct iommu_domain *domain);
 
@@ -40,6 +41,10 @@ static inline void iommu_put_dma_cookie(struct iommu_domain *domain)
 {
 }
 
+static inline void iommu_put_msi_cookie(struct iommu_domain *domain)
+{
+}
+
 static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list)
 {
 }
include/linux/iommu.h +15 -5
index e93d2e918599..06cc14e9993d 100644
@@ -41,6 +41,7 @@ struct iommu_dirty_ops;
 struct notifier_block;
 struct iommu_sva;
 struct iommu_dma_cookie;
+struct iommu_dma_msi_cookie;
 struct iommu_fault_param;
 struct iommufd_ctx;
 struct iommufd_viommu;
@@ -165,6 +166,15 @@ struct iommu_domain_geometry {
 	bool force_aperture;       /* DMA only allowed in mappable range? */
 };
 
+enum iommu_domain_cookie_type {
+	IOMMU_COOKIE_NONE,
+	IOMMU_COOKIE_DMA_IOVA,
+	IOMMU_COOKIE_DMA_MSI,
+	IOMMU_COOKIE_FAULT_HANDLER,
+	IOMMU_COOKIE_SVA,
+	IOMMU_COOKIE_IOMMUFD,
+};
+
 /* Domain feature flags */
 #define __IOMMU_DOMAIN_PAGING	(1U << 0)  /* Support for iommu_map/unmap */
 #define __IOMMU_DOMAIN_DMA_API	(1U << 1)  /* Domain for use in DMA-API
@@ -211,12 +221,12 @@ struct iommu_domain_geometry {
 
 struct iommu_domain {
 	unsigned type;
+	enum iommu_domain_cookie_type cookie_type;
 	const struct iommu_domain_ops *ops;
 	const struct iommu_dirty_ops *dirty_ops;
 	const struct iommu_ops *owner; /* Whose domain_alloc we came from */
 	unsigned long pgsize_bitmap;	/* Bitmap of page sizes in use */
 	struct iommu_domain_geometry geometry;
-	struct iommu_dma_cookie *iova_cookie;
 	int (*iopf_handler)(struct iopf_group *group);
 
 #if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
@@ -224,10 +234,10 @@ struct iommu_domain {
 		      phys_addr_t msi_addr);
 #endif
 
-	union { /* Pointer usable by owner of the domain */
-		struct iommufd_hw_pagetable *iommufd_hwpt; /* iommufd */
-	};
-	union { /* Fault handler */
+	union { /* cookie */
+		struct iommu_dma_cookie *iova_cookie;
+		struct iommu_dma_msi_cookie *msi_cookie;
+		struct iommufd_hw_pagetable *iommufd_hwpt;
 		struct {
 			iommu_fault_handler_t handler;
 			void *handler_token;
drivers/iommu/dma-iommu.c +103 -90

342 lines not on this page · raw message

drivers/iommu/iommu-sva.c +1 -0
index 503c5d23c1ea..ab18bc494eef 100644
@@ -310,6 +310,7 @@ static struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
 	}
 
 	domain->type = IOMMU_DOMAIN_SVA;
+	domain->cookie_type = IOMMU_COOKIE_SVA;
 	mmgrab(mm);
 	domain->mm = mm;
 	domain->owner = ops;
drivers/iommu/iommu.c +14 -3
index 0ee17893810f..8b9423340221 100644
@@ -1953,8 +1953,10 @@ void iommu_set_fault_handler(struct iommu_domain *domain,
 					iommu_fault_handler_t handler,
 					void *token)
 {
-	BUG_ON(!domain);
+	if (WARN_ON(!domain || domain->cookie_type != IOMMU_COOKIE_NONE))
+		return;
 
+	domain->cookie_type = IOMMU_COOKIE_FAULT_HANDLER;
 	domain->handler = handler;
 	domain->handler_token = token;
 }
@@ -2024,9 +2026,18 @@ EXPORT_SYMBOL_GPL(iommu_paging_domain_alloc_flags);
 
 void iommu_domain_free(struct iommu_domain *domain)
 {
-	if (domain->type == IOMMU_DOMAIN_SVA)
+	switch (domain->cookie_type) {
+	case IOMMU_COOKIE_DMA_IOVA:
+		iommu_put_dma_cookie(domain);
+		break;
+	case IOMMU_COOKIE_DMA_MSI:
+		iommu_put_msi_cookie(domain);
+		break;
+	case IOMMU_COOKIE_SVA:
 		mmdrop(domain->mm);
-	iommu_put_dma_cookie(domain);
+		break;
+	default:
+	}
 	if (domain->ops->free)
 		domain->ops->free(domain);
 }
drivers/iommu/iommufd/hw_pagetable.c +1 -0
index 7de6e914232e..227514030655 100644
@@ -413,6 +413,7 @@ int iommufd_hwpt_alloc(struct iommufd_ucmd *ucmd)
 		iommufd_put_object(ucmd->ictx, &fault->obj);
 	}
 	hwpt->domain->iommufd_hwpt = hwpt;
+	hwpt->domain->cookie_type = IOMMU_COOKIE_IOMMUFD;
 
 	cmd->out_hwpt_id = hwpt->obj.id;
 	rc = iommufd_ucmd_respond(ucmd, sizeof(*cmd));
Signature · 1 line
2.43.0
#10 lore raw

PATCH v3 2/3 iommufd: Move iommufd_sw_msi and related functions to driver.c

In reply to Nicolin Chen

To provide the iommufd_sw_msi() to the iommu core that is under a different Kconfig, move it and its related functions to driver.c. Then, stub it into the iommu-priv header. The iommufd_sw_msi_install() continues to be used by iommufd internal, so put it in the private header.

---
 drivers/iommu/iommu-priv.h              |  13 +++
 drivers/iommu/iommufd/iommufd_private.h |   7 +-
 drivers/iommu/iommufd/device.c          | 127 +-----------------------
 drivers/iommu/iommufd/driver.c          | 125 +++++++++++++++++++++++
 4 files changed, 148 insertions(+), 124 deletions(-)
drivers/iommu/iommu-priv.h +13 -0
index b4508423e13b..c74fff25be78 100644
@@ -5,6 +5,7 @@
 #define __LINUX_IOMMU_PRIV_H
 
 #include <linux/iommu.h>
+#include <linux/msi.h>
 
 static inline const struct iommu_ops *dev_iommu_ops(struct device *dev)
 {
@@ -43,4 +44,16 @@ void iommu_detach_group_handle(struct iommu_domain *domain,
 int iommu_replace_group_handle(struct iommu_group *group,
 			       struct iommu_domain *new_domain,
 			       struct iommu_attach_handle *handle);
+
+#if IS_ENABLED(CONFIG_IOMMUFD_DRIVER_CORE) && IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
+int iommufd_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
+		   phys_addr_t msi_addr);
+#else /* !CONFIG_IOMMUFD_DRIVER_CORE || !CONFIG_IRQ_MSI_IOMMU */
+static inline int iommufd_sw_msi(struct iommu_domain *domain,
+				 struct msi_desc *desc, phys_addr_t msi_addr)
+{
+	return -EOPNOTSUPP;
+}
+#endif /* CONFIG_IOMMUFD_DRIVER_CORE && CONFIG_IRQ_MSI_IOMMU */
+
 #endif /* __LINUX_IOMMU_PRIV_H */
drivers/iommu/iommufd/iommufd_private.h +5 -2
index 246297452a44..b6897a9991ee 100644
@@ -32,8 +32,11 @@ struct iommufd_sw_msi_maps {
 	DECLARE_BITMAP(bitmap, 64);
 };
 
-int iommufd_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
-		   phys_addr_t msi_addr);
+#ifdef CONFIG_IRQ_MSI_IOMMU
+int iommufd_sw_msi_install(struct iommufd_ctx *ictx,
+			   struct iommufd_hwpt_paging *hwpt_paging,
+			   struct iommufd_sw_msi_map *msi_map);
+#endif
 
 struct iommufd_ctx {
 	struct file *file;
drivers/iommu/iommufd/device.c +5 -122

161 lines not on this page · raw message

drivers/iommu/iommufd/driver.c +125 -0

130 lines not on this page · raw message

Signature · 1 line
2.43.0
#13 lore raw

PATCH v3 3/3 iommu: Drop sw_msi from iommu_domain

In reply to Nicolin Chen

There are only two sw_msi implementations in the entire system, thus it's not very necessary to have an sw_msi pointer.

Instead, check domain->private_data_owner to call the two implementations directly from the core code.

---
 drivers/iommu/dma-iommu.h            |  9 +++++++++
 include/linux/iommu.h                | 15 ---------------
 drivers/iommu/dma-iommu.c            | 14 ++------------
 drivers/iommu/iommu.c                | 17 +++++++++++++++--
 drivers/iommu/iommufd/hw_pagetable.c |  3 ---
 5 files changed, 26 insertions(+), 32 deletions(-)
drivers/iommu/dma-iommu.h +9 -0
index 9cca11806e5d..eca201c1f963 100644
@@ -19,6 +19,9 @@ int iommu_dma_init_fq(struct iommu_domain *domain);
 
 void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list);
 
+int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
+		     phys_addr_t msi_addr);
+
 extern bool iommu_dma_forcedac;
 
 #else /* CONFIG_IOMMU_DMA */
@@ -49,5 +52,11 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
 {
 }
 
+static inline int iommu_dma_sw_msi(struct iommu_domain *domain,
+				   struct msi_desc *desc, phys_addr_t msi_addr)
+{
+	return -ENODEV;
+}
+
 #endif	/* CONFIG_IOMMU_DMA */
 #endif	/* __DMA_IOMMU_H */
include/linux/iommu.h +0 -15
index 06cc14e9993d..e01c855ae8a7 100644
@@ -229,11 +229,6 @@ struct iommu_domain {
 	struct iommu_domain_geometry geometry;
 	int (*iopf_handler)(struct iopf_group *group);
 
-#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
-	int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc,
-		      phys_addr_t msi_addr);
-#endif
-
 	union { /* cookie */
 		struct iommu_dma_cookie *iova_cookie;
 		struct iommu_dma_msi_cookie *msi_cookie;
@@ -254,16 +249,6 @@ struct iommu_domain {
 	};
 };
 
-static inline void iommu_domain_set_sw_msi(
-	struct iommu_domain *domain,
-	int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc,
-		      phys_addr_t msi_addr))
-{
-#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
-	domain->sw_msi = sw_msi;
-#endif
-}
-
 static inline bool iommu_is_dma_domain(struct iommu_domain *domain)
 {
 	return domain->type & __IOMMU_DOMAIN_DMA_API;
drivers/iommu/dma-iommu.c +2 -12
index bc9bb9cb70c8..96e9f8d0a4f6 100644
@@ -94,9 +94,6 @@ static int __init iommu_dma_forcedac_setup(char *str)
 }
 early_param("iommu.forcedac", iommu_dma_forcedac_setup);
 
-static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
-			    phys_addr_t msi_addr);
-
 /* Number of entries per flush queue */
 #define IOVA_DEFAULT_FQ_SIZE	256
 #define IOVA_SINGLE_FQ_SIZE	32768
@@ -377,7 +374,6 @@ int iommu_get_dma_cookie(struct iommu_domain *domain)
 
 	mutex_init(&cookie->mutex);
 	INIT_LIST_HEAD(&cookie->msi_page_list);
-	iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi);
 	domain->cookie_type = IOMMU_COOKIE_DMA_IOVA;
 	domain->iova_cookie = cookie;
 	return 0;
@@ -411,7 +407,6 @@ int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base)
 
 	cookie->msi_iova = base;
 	INIT_LIST_HEAD(&cookie->msi_page_list);
-	iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi);
 	domain->cookie_type = IOMMU_COOKIE_DMA_MSI;
 	domain->msi_cookie = cookie;
 	return 0;
@@ -427,11 +422,6 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
 	struct iommu_dma_cookie *cookie = domain->iova_cookie;
 	struct iommu_dma_msi_page *msi, *tmp;
 
-#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
-	if (domain->sw_msi != iommu_dma_sw_msi)
-		return;
-#endif
-
 	if (cookie->iovad.granule) {
 		iommu_dma_free_fq(cookie);
 		put_iova_domain(&cookie->iovad);
@@ -1825,8 +1815,8 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev,
 	return NULL;
 }
 
-static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
-			    phys_addr_t msi_addr)
+int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
+		     phys_addr_t msi_addr)
 {
 	struct device *dev = msi_desc_to_dev(desc);
 	const struct iommu_dma_msi_page *msi_page;
drivers/iommu/iommu.c +15 -2
index 8b9423340221..c1c0656f41ef 100644
@@ -18,6 +18,7 @@
 #include <linux/errno.h>
 #include <linux/host1x_context_bus.h>
 #include <linux/iommu.h>
+#include <linux/iommufd.h>
 #include <linux/idr.h>
 #include <linux/err.h>
 #include <linux/pci.h>
@@ -3649,8 +3650,20 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr)
 		return 0;
 
 	mutex_lock(&group->mutex);
-	if (group->domain && group->domain->sw_msi)
-		ret = group->domain->sw_msi(group->domain, desc, msi_addr);
+	if (group->domain) {
+		switch (group->domain->cookie_type) {
+		case IOMMU_COOKIE_DMA_MSI:
+		case IOMMU_COOKIE_DMA_IOVA:
+			ret = iommu_dma_sw_msi(group->domain, desc, msi_addr);
+			break;
+		case IOMMU_COOKIE_IOMMUFD:
+			ret = iommufd_sw_msi(group->domain, desc, msi_addr);
+			break;
+		default:
+			ret = -EOPNOTSUPP;
+			break;
+		}
+	}
 	mutex_unlock(&group->mutex);
 	return ret;
 }
drivers/iommu/iommufd/hw_pagetable.c +0 -3
index 227514030655..98aecf904902 100644
@@ -156,7 +156,6 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas,
 			goto out_abort;
 		}
 	}
-	iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi);
 
 	/*
 	 * Set the coherency mode before we do iopt_table_add_domain() as some
@@ -252,7 +251,6 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx,
 		goto out_abort;
 	}
 	hwpt->domain->owner = ops;
-	iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi);
 
 	if (WARN_ON_ONCE(hwpt->domain->type != IOMMU_DOMAIN_NESTED)) {
 		rc = -EINVAL;
@@ -309,7 +307,6 @@ iommufd_viommu_alloc_hwpt_nested(struct iommufd_viommu *viommu, u32 flags,
 		goto out_abort;
 	}
 	hwpt->domain->owner = viommu->iommu_dev->ops;
-	iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi);
 
 	if (WARN_ON_ONCE(hwpt->domain->type != IOMMU_DOMAIN_NESTED)) {
 		rc = -EINVAL;
Signature · 1 line
2.43.0
maintainer Reviewed-by on v3 3/3
#14 lore raw

Re: [PATCH v3 3/3] iommu: Drop sw_msi from iommu_domain

Quoted 5 lines · Nicolin Chen

There are only two sw_msi implementations in the entire system, thus it's not very necessary to have an sw_msi pointer.

Instead, check domain->private_data_owner to call the two implementations directly from the core code.

FWIW I still wish this was squashed into the pending patches so we're not adding stuff we immediately remove again, but oh well, what's done is done.

(assuming this isn't also queued already given that once again I've had the audacity to not look at my inbox until after 2PM the day after it was posted...)

Quoted 185 lines
---
  drivers/iommu/dma-iommu.h            |  9 +++++++++
  include/linux/iommu.h                | 15 ---------------
  drivers/iommu/dma-iommu.c            | 14 ++------------
  drivers/iommu/iommu.c                | 17 +++++++++++++++--
  drivers/iommu/iommufd/hw_pagetable.c |  3 ---
  5 files changed, 26 insertions(+), 32 deletions(-)
drivers/iommu/dma-iommu.h +9 -0
index 9cca11806e5d..eca201c1f963 100644
@@ -19,6 +19,9 @@ int iommu_dma_init_fq(struct iommu_domain *domain);
 
  void iommu_dma_get_resv_regions(struct device *dev, struct list_head *list);
 
+int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
+		     phys_addr_t msi_addr);
+
  extern bool iommu_dma_forcedac;
 
  #else /* CONFIG_IOMMU_DMA */
@@ -49,5 +52,11 @@ static inline void iommu_dma_get_resv_regions(struct device *dev, struct list_he
  {
  }
 
+static inline int iommu_dma_sw_msi(struct iommu_domain *domain,
+				   struct msi_desc *desc, phys_addr_t msi_addr)
+{
+	return -ENODEV;
+}
+
  #endif	/* CONFIG_IOMMU_DMA */
  #endif	/* __DMA_IOMMU_H */
include/linux/iommu.h +0 -15
index 06cc14e9993d..e01c855ae8a7 100644
@@ -229,11 +229,6 @@ struct iommu_domain {
  	struct iommu_domain_geometry geometry;
  	int (*iopf_handler)(struct iopf_group *group);
 
-#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
-	int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc,
-		      phys_addr_t msi_addr);
-#endif
-
  	union { /* cookie */
  		struct iommu_dma_cookie *iova_cookie;
  		struct iommu_dma_msi_cookie *msi_cookie;
@@ -254,16 +249,6 @@ struct iommu_domain {
  	};
  };
 
-static inline void iommu_domain_set_sw_msi(
-	struct iommu_domain *domain,
-	int (*sw_msi)(struct iommu_domain *domain, struct msi_desc *desc,
-		      phys_addr_t msi_addr))
-{
-#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
-	domain->sw_msi = sw_msi;
-#endif
-}
-
  static inline bool iommu_is_dma_domain(struct iommu_domain *domain)
  {
  	return domain->type & __IOMMU_DOMAIN_DMA_API;
drivers/iommu/dma-iommu.c +2 -12
index bc9bb9cb70c8..96e9f8d0a4f6 100644
@@ -94,9 +94,6 @@ static int __init iommu_dma_forcedac_setup(char *str)
  }
  early_param("iommu.forcedac", iommu_dma_forcedac_setup);
 
-static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
-			    phys_addr_t msi_addr);
-
  /* Number of entries per flush queue */
  #define IOVA_DEFAULT_FQ_SIZE	256
  #define IOVA_SINGLE_FQ_SIZE	32768
@@ -377,7 +374,6 @@ int iommu_get_dma_cookie(struct iommu_domain *domain)
 
  	mutex_init(&cookie->mutex);
  	INIT_LIST_HEAD(&cookie->msi_page_list);
-	iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi);
  	domain->cookie_type = IOMMU_COOKIE_DMA_IOVA;
  	domain->iova_cookie = cookie;
  	return 0;
@@ -411,7 +407,6 @@ int iommu_get_msi_cookie(struct iommu_domain *domain, dma_addr_t base)
 
  	cookie->msi_iova = base;
  	INIT_LIST_HEAD(&cookie->msi_page_list);
-	iommu_domain_set_sw_msi(domain, iommu_dma_sw_msi);
  	domain->cookie_type = IOMMU_COOKIE_DMA_MSI;
  	domain->msi_cookie = cookie;
  	return 0;
@@ -427,11 +422,6 @@ void iommu_put_dma_cookie(struct iommu_domain *domain)
  	struct iommu_dma_cookie *cookie = domain->iova_cookie;
  	struct iommu_dma_msi_page *msi, *tmp;
 
-#if IS_ENABLED(CONFIG_IRQ_MSI_IOMMU)
-	if (domain->sw_msi != iommu_dma_sw_msi)
-		return;
-#endif
-
  	if (cookie->iovad.granule) {
  		iommu_dma_free_fq(cookie);
  		put_iova_domain(&cookie->iovad);
@@ -1825,8 +1815,8 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev,
  	return NULL;
  }
 
-static int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
-			    phys_addr_t msi_addr)
+int iommu_dma_sw_msi(struct iommu_domain *domain, struct msi_desc *desc,
+		     phys_addr_t msi_addr)
  {
  	struct device *dev = msi_desc_to_dev(desc);
  	const struct iommu_dma_msi_page *msi_page;
drivers/iommu/iommu.c +15 -2
index 8b9423340221..c1c0656f41ef 100644
@@ -18,6 +18,7 @@
  #include <linux/errno.h>
  #include <linux/host1x_context_bus.h>
  #include <linux/iommu.h>
+#include <linux/iommufd.h>
  #include <linux/idr.h>
  #include <linux/err.h>
  #include <linux/pci.h>
@@ -3649,8 +3650,20 @@ int iommu_dma_prepare_msi(struct msi_desc *desc, phys_addr_t msi_addr)
  		return 0;
 
  	mutex_lock(&group->mutex);
-	if (group->domain && group->domain->sw_msi)
-		ret = group->domain->sw_msi(group->domain, desc, msi_addr);
+	if (group->domain) {
+		switch (group->domain->cookie_type) {
+		case IOMMU_COOKIE_DMA_MSI:
+		case IOMMU_COOKIE_DMA_IOVA:
+			ret = iommu_dma_sw_msi(group->domain, desc, msi_addr);
+			break;
+		case IOMMU_COOKIE_IOMMUFD:
+			ret = iommufd_sw_msi(group->domain, desc, msi_addr);
+			break;
+		default:
+			ret = -EOPNOTSUPP;
+			break;
+		}
+	}
  	mutex_unlock(&group->mutex);
  	return ret;
  }
drivers/iommu/iommufd/hw_pagetable.c +0 -3
index 227514030655..98aecf904902 100644
@@ -156,7 +156,6 @@ iommufd_hwpt_paging_alloc(struct iommufd_ctx *ictx, struct iommufd_ioas *ioas,
  			goto out_abort;
  		}
  	}
-	iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi);
 
  	/*
  	 * Set the coherency mode before we do iopt_table_add_domain() as some
@@ -252,7 +251,6 @@ iommufd_hwpt_nested_alloc(struct iommufd_ctx *ictx,
  		goto out_abort;
  	}
  	hwpt->domain->owner = ops;
-	iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi);
 
  	if (WARN_ON_ONCE(hwpt->domain->type != IOMMU_DOMAIN_NESTED)) {
  		rc = -EINVAL;
@@ -309,7 +307,6 @@ iommufd_viommu_alloc_hwpt_nested(struct iommufd_viommu *viommu, u32 flags,
  		goto out_abort;
  	}
  	hwpt->domain->owner = viommu->iommu_dev->ops;
-	iommu_domain_set_sw_msi(hwpt->domain, iommufd_sw_msi);
 
  	if (WARN_ON_ONCE(hwpt->domain->type != IOMMU_DOMAIN_NESTED)) {
  		rc = -EINVAL;
#2 lore raw

Re: [PATCH v3 1/3] iommu: Sort out domain user data

In reply to Nicolin Chen
kernel test robot noticed the following build warnings:

Hi Nicolin,

kernel test robot noticed the following build warnings:

[auto build test WARNING on 5e9f822c9c683ae884fa5e71df41d1647b2876c6]

url:    https://github.com/intel-lab-lkp/linux/commits/Nicolin-Chen/iommu-Sort-out-domain-user-data/20250304-045529
base:   5e9f822c9c683ae884fa5e71df41d1647b2876c6
patch link:    https://lore.kernel.org/r/74a3bf91b52563faaf7ef7de065931e4df47e63f.1741034886.git.nicolinc%40nvidia.com
patch subject: [PATCH v3 1/3] iommu: Sort out domain user data
config: hexagon-randconfig-002-20250305 (https://download.01.org/0day-ci/archive/20250305/[email protected]/config)
compiler: clang version 18.1.8 (https://github.com/llvm/llvm-project 3b5b5c1ec4a3095ab096dd780e84d7ab81f3d7ff)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250305/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> drivers/iommu/iommu.c:2040:2: warning: label at end of compound statement is a C23 extension [-Wc23-extensions]
    2040 |         }
         |         ^

1 warning generated.

vim +2040 drivers/iommu/iommu.c

2026
2027 void iommu_domain_free(struct iommu_domain *domain)
2028 {
2029 switch (domain->cookie_type) {
2030 case IOMMU_COOKIE_DMA_IOVA:
2031 iommu_put_dma_cookie(domain);
2032 break;
2033 case IOMMU_COOKIE_DMA_MSI:
2034 iommu_put_msi_cookie(domain);
2035 break;
2036 case IOMMU_COOKIE_SVA:
2037 mmdrop(domain->mm);
2038 break;
2039 default:

> 2040		}

2041 if (domain->ops->free)
2042 domain->ops->free(domain);
2043 }
2044 EXPORT_SYMBOL_GPL(iommu_domain_free);
2045

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#3 lore raw

Re: [PATCH v3 1/3] iommu: Sort out domain user data

In reply to Nicolin Chen
kernel test robot noticed the following build errors:

Hi Nicolin,

kernel test robot noticed the following build errors:

[auto build test ERROR on 5e9f822c9c683ae884fa5e71df41d1647b2876c6]

url:    https://github.com/intel-lab-lkp/linux/commits/Nicolin-Chen/iommu-Sort-out-domain-user-data/20250304-045529
base:   5e9f822c9c683ae884fa5e71df41d1647b2876c6
patch link:    https://lore.kernel.org/r/74a3bf91b52563faaf7ef7de065931e4df47e63f.1741034886.git.nicolinc%40nvidia.com
patch subject: [PATCH v3 1/3] iommu: Sort out domain user data
config: mips-randconfig-r071-20250305 (https://download.01.org/0day-ci/archive/20250305/[email protected]/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250305/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

>> drivers/iommu/iommu.c:2039:10: error: label at end of compound statement: expected statement
           default:
                   ^
                    ;

1 error generated.

vim +2039 drivers/iommu/iommu.c

2026
2027 void iommu_domain_free(struct iommu_domain *domain)
2028 {
2029 switch (domain->cookie_type) {
2030 case IOMMU_COOKIE_DMA_IOVA:
2031 iommu_put_dma_cookie(domain);
2032 break;
2033 case IOMMU_COOKIE_DMA_MSI:
2034 iommu_put_msi_cookie(domain);
2035 break;
2036 case IOMMU_COOKIE_SVA:
2037 mmdrop(domain->mm);
2038 break;

> 2039		default:

2040 }
2041 if (domain->ops->free)
2042 domain->ops->free(domain);
2043 }
2044 EXPORT_SYMBOL_GPL(iommu_domain_free);
2045

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
maintainer
#4 lore raw

Re: [PATCH v3 1/3] iommu: Sort out domain user data

Quoted 19 lines · kernel test robot

Hi Nicolin,

kernel test robot noticed the following build errors:

[auto build test ERROR on 5e9f822c9c683ae884fa5e71df41d1647b2876c6]

url:    https://github.com/intel-lab-lkp/linux/commits/Nicolin-Chen/iommu-Sort-out-domain-user-data/20250304-045529
base:   5e9f822c9c683ae884fa5e71df41d1647b2876c6
patch link:    https://lore.kernel.org/r/74a3bf91b52563faaf7ef7de065931e4df47e63f.1741034886.git.nicolinc%40nvidia.com
patch subject: [PATCH v3 1/3] iommu: Sort out domain user data
config: mips-randconfig-r071-20250305 (https://download.01.org/0day-ci/archive/20250305/[email protected]/config)
compiler: clang version 15.0.7 (https://github.com/llvm/llvm-project 8dfdcc7b7bf66834a761bd8de445840ef68e4d1a)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250305/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

Quoted 7 lines

All errors (new ones prefixed by >>):

Quoted 1 line
Quoted 1 line

drivers/iommu/iommu.c:2039:10: error: label at end of compound statement: expected statement

            default:
                    ^
                     ;
    1 error generated.

Oops, my bad... omitting an extra break here was a semi-conscious brainfart. Weird that my AArch64 GCC doesn't complain about this construct though, even with "-ansi -Wall" - I have to go all the way to -Wpedantic before it notices :/

Thanks,
Robin.

Quoted 25 lines

vim +2039 drivers/iommu/iommu.c

   2026	
   2027	void iommu_domain_free(struct iommu_domain *domain)
   2028	{
   2029		switch (domain->cookie_type) {
   2030		case IOMMU_COOKIE_DMA_IOVA:
   2031			iommu_put_dma_cookie(domain);
   2032			break;
   2033		case IOMMU_COOKIE_DMA_MSI:
   2034			iommu_put_msi_cookie(domain);
   2035			break;
   2036		case IOMMU_COOKIE_SVA:
   2037			mmdrop(domain->mm);
   2038			break;
Quoted 1 line

2039 default:

   2040		}
   2041		if (domain->ops->free)
   2042			domain->ops->free(domain);
   2043	}
   2044	EXPORT_SYMBOL_GPL(iommu_domain_free);
   2045	
#6 lore raw

Re: [PATCH v3 1/3] iommu: Sort out domain user data

In reply to Nicolin Chen
kernel test robot noticed the following build warnings:

Hi Nicolin,

kernel test robot noticed the following build warnings:

[auto build test WARNING on 5e9f822c9c683ae884fa5e71df41d1647b2876c6]

url:    https://github.com/intel-lab-lkp/linux/commits/Nicolin-Chen/iommu-Sort-out-domain-user-data/20250304-045529
base:   5e9f822c9c683ae884fa5e71df41d1647b2876c6
patch link:    https://lore.kernel.org/r/74a3bf91b52563faaf7ef7de065931e4df47e63f.1741034886.git.nicolinc%40nvidia.com
patch subject: [PATCH v3 1/3] iommu: Sort out domain user data
config: powerpc-randconfig-001-20250305 (https://download.01.org/0day-ci/archive/20250306/[email protected]/config)
compiler: clang version 17.0.6 (https://github.com/llvm/llvm-project 6009708b4367171ccdbf4b5905cb6a803753fe18)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250306/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All warnings (new ones prefixed by >>):

>> drivers/iommu/iommu.c:2040:2: warning: label at end of compound statement is a C2x extension [-Wc2x-extensions]
    2040 |         }
         |         ^

1 warning generated.

vim +2040 drivers/iommu/iommu.c

2026
2027 void iommu_domain_free(struct iommu_domain *domain)
2028 {
2029 switch (domain->cookie_type) {
2030 case IOMMU_COOKIE_DMA_IOVA:
2031 iommu_put_dma_cookie(domain);
2032 break;
2033 case IOMMU_COOKIE_DMA_MSI:
2034 iommu_put_msi_cookie(domain);
2035 break;
2036 case IOMMU_COOKIE_SVA:
2037 mmdrop(domain->mm);
2038 break;
2039 default:

> 2040		}

2041 if (domain->ops->free)
2042 domain->ops->free(domain);
2043 }
2044 EXPORT_SYMBOL_GPL(iommu_domain_free);
2045

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#7 lore raw

Re: [PATCH v3 1/3] iommu: Sort out domain user data

In reply to Nicolin Chen
kernel test robot noticed the following build warnings:

Hi Nicolin,

kernel test robot noticed the following build warnings:

[auto build test WARNING on 5e9f822c9c683ae884fa5e71df41d1647b2876c6]

url:    https://github.com/intel-lab-lkp/linux/commits/Nicolin-Chen/iommu-Sort-out-domain-user-data/20250304-045529
base:   5e9f822c9c683ae884fa5e71df41d1647b2876c6
patch link:    https://lore.kernel.org/r/74a3bf91b52563faaf7ef7de065931e4df47e63f.1741034886.git.nicolinc%40nvidia.com
patch subject: [PATCH v3 1/3] iommu: Sort out domain user data
config: csky-randconfig-r111-20250305 (https://download.01.org/0day-ci/archive/20250306/[email protected]/config)
compiler: csky-linux-gcc (GCC) 14.2.0
reproduce: (https://download.01.org/0day-ci/archive/20250306/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

sparse warnings: (new ones prefixed by >>)

>> drivers/iommu/iommu.c:2040:9: sparse: sparse: statement expected after case label

drivers/iommu/iommu.c: note: in included file (through include/linux/mutex.h, include/linux/notifier.h, include/linux/clk.h, ...):
include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true include/linux/list.h:83:21: sparse: sparse: self-comparison always evaluates to true

vim +2040 drivers/iommu/iommu.c

2026
2027 void iommu_domain_free(struct iommu_domain *domain)
2028 {
2029 switch (domain->cookie_type) {
2030 case IOMMU_COOKIE_DMA_IOVA:
2031 iommu_put_dma_cookie(domain);
2032 break;
2033 case IOMMU_COOKIE_DMA_MSI:
2034 iommu_put_msi_cookie(domain);
2035 break;
2036 case IOMMU_COOKIE_SVA:
2037 mmdrop(domain->mm);
2038 break;
2039 default:

> 2040		}

2041 if (domain->ops->free)
2042 domain->ops->free(domain);
2043 }
2044 EXPORT_SYMBOL_GPL(iommu_domain_free);
2045

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
#5 lore raw

Re: [PATCH v3 1/3] iommu: Sort out domain user data

In reply to Robin Murphy
Quoted 8 lines · Robin Murphy
Quoted 7 lines

All errors (new ones prefixed by >>):

Quoted 1 line
Quoted 1 line

drivers/iommu/iommu.c:2039:10: error: label at end of compound statement: expected statement

            default:
                    ^
                     ;
    1 error generated.
Quoted 4 lines

Oops, my bad... omitting an extra break here was a semi-conscious brainfart. Weird that my AArch64 GCC doesn't complain about this construct though, even with "-ansi -Wall" - I have to go all the way to -Wpedantic before it notices :/

Mine doesn't complain about this either.

I will add a break and respin.

Thanks
Nicolin

maintainer
#11 lore raw

Re: [PATCH v3 2/3] iommufd: Move iommufd_sw_msi and related functions to driver.c

In reply to Nicolin Chen
Quoted 17 lines · Nicolin Chen
drivers/iommu/iommufd/device.c +2 -0
@@ -458,6 +337,7 @@ iommufd_device_attach_reserved_iova(struct iommufd_device *idev,
 	if (rc)
 		return rc;
 
+#ifdef CONFIG_IRQ_MSI_IOMMU
 	if (list_empty(&idev->igroup->device_list)) {
 		rc = iommufd_group_setup_msi(idev->igroup, hwpt_paging);
 		if (rc) {
@@ -466,6 +346,7 @@ iommufd_device_attach_reserved_iova(struct iommufd_device *idev,
 			return rc;
 		}
 	}
+#endif
 	return 0;
 }
 

drivers/iommu/iommufd/device.c iommufd_group_do_replace_reserved_iova(struct iommufd_group *igroup, · line 571

Quoted 9 lines
drivers/iommu/iommufd/device.c +2 -0
@@ -682,9 +563,11 @@ iommufd_group_do_replace_reserved_iova(struct iommufd_group *igroup,
 		}
 	}
 
+#ifdef CONFIG_IRQ_MSI_IOMMU
 	rc = iommufd_group_setup_msi(igroup, hwpt_paging);
 	if (rc)
 		goto err_unresv;
+#endif
 	return 0;

Lets try to avoid these ifdefs, maybe a empty inline, or maybe just need to put a if (!IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) in the right spots..

I'd also include the size measurement that you did in the commit message

Jason

#12 lore raw

Re: [PATCH v3 2/3] iommufd: Move iommufd_sw_msi and related functions to driver.c

Quoted 32 lines · Jason Gunthorpe
Quoted 17 lines · Nicolin Chen
drivers/iommu/iommufd/device.c +2 -0
@@ -458,6 +337,7 @@ iommufd_device_attach_reserved_iova(struct iommufd_device *idev,
 	if (rc)
 		return rc;
 
+#ifdef CONFIG_IRQ_MSI_IOMMU
 	if (list_empty(&idev->igroup->device_list)) {
 		rc = iommufd_group_setup_msi(idev->igroup, hwpt_paging);
 		if (rc) {
@@ -466,6 +346,7 @@ iommufd_device_attach_reserved_iova(struct iommufd_device *idev,
 			return rc;
 		}
 	}
+#endif
 	return 0;
 }
 

drivers/iommu/iommufd/device.c iommufd_group_do_replace_reserved_iova(struct iommufd_group *igroup, · line 571

Quoted 9 lines
drivers/iommu/iommufd/device.c +2 -0
@@ -682,9 +563,11 @@ iommufd_group_do_replace_reserved_iova(struct iommufd_group *igroup,
 		}
 	}
 
+#ifdef CONFIG_IRQ_MSI_IOMMU
 	rc = iommufd_group_setup_msi(igroup, hwpt_paging);
 	if (rc)
 		goto err_unresv;
+#endif
 	return 0;

Lets try to avoid these ifdefs, maybe a empty inline, or maybe just need to put a if (!IS_ENABLED(CONFIG_IRQ_MSI_IOMMU) in the right spots..

Quoted 2 lines

I'd also include the size measurement that you did in the commit message

Ack. Will rework.

Thanks
Nicolin

maintainer Reviewed-by on v3 1/3
#8 lore raw

RE: [PATCH v3 1/3] iommu: Sort out domain user data

In reply to Nicolin Chen
Quoted 26 lines

From: Nicolin Chen <[email protected]>
Sent: Tuesday, March 4, 2025 4:53 AM

From: Robin Murphy <[email protected]>

When DMA/MSI cookies were made first-class citizens back in commit 46983fcd67ac ("iommu: Pull IOVA cookie management into the core"), there was no real need to further expose the two different cookie types. However, now that IOMMUFD wants to add a third type of MSI-mapping cookie, we do have a nicely compelling reason to properly dismabiguate things at the domain level beyond just vaguely guessing from the domain type.

Meanwhile, we also effectively have another "cookie" in the form of the anonymous union for other user data, which isn't much better in terms of being vague and unenforced. The fact is that all these cookie types are mutually exclusive, in the sense that combining them makes zero sense and/or would be catastrophic (iommu_set_fault_handler() on an SVA domain, anyone?) - the only combination which *might* be reasonable is perhaps a fault handler and an MSI cookie, but nobody's doing that at the moment, so let's rule it out as well for the sake of being clear and robust. To that end, we pull DMA and MSI cookies apart a little more, mostly to clear up the ambiguity at domain teardown, then for clarity (and to save a little space), move them into the union, whose ownership we can then properly describe and enforce entirely unambiguously.

Quoted 5 lines

[nicolinc: rebase on latest tree; use prefix IOMMU_COOKIE_; merge unions
in iommu_domain; add IOMMU_COOKIE_IOMMUFD for
iommufd_hwpt]

with a nit:

drivers/iommu/dma-iommu.c void iommu_setup_dma_ops(struct device *dev) · line 1802

Quoted 6 lines
drivers/iommu/dma-iommu.c +1 -1
 	msi_addr &= ~(phys_addr_t)(size - 1);
-	list_for_each_entry(msi_page, &cookie->msi_page_list, list)
+	list_for_each_entry(msi_page, cookie_msi_pages(domain), list)
 		if (msi_page->phys == msi_addr)
 			return msi_page;

Above checks cookie type in every iteration. Better save the list pointer to a local variable.

maintainer Reviewed-by on v3 3/3
#15 lore raw

RE: [PATCH v3 3/3] iommu: Drop sw_msi from iommu_domain

In reply to Nicolin Chen
Quoted 8 lines

From: Nicolin Chen <[email protected]>
Sent: Tuesday, March 4, 2025 4:53 AM

There are only two sw_msi implementations in the entire system, thus it's not very necessary to have an sw_msi pointer.

Instead, check domain->private_data_owner to call the two implementations directly from the core code.

s/private_data_owner/cookie_type/

Quoted 3 lines
#9 lore raw

Re: [PATCH v3 1/3] iommu: Sort out domain user data

In reply to Tian, Kevin
Quoted 36 lines · Kevin
Quoted 26 lines

From: Nicolin Chen <[email protected]>
Sent: Tuesday, March 4, 2025 4:53 AM

From: Robin Murphy <[email protected]>

When DMA/MSI cookies were made first-class citizens back in commit 46983fcd67ac ("iommu: Pull IOVA cookie management into the core"), there was no real need to further expose the two different cookie types. However, now that IOMMUFD wants to add a third type of MSI-mapping cookie, we do have a nicely compelling reason to properly dismabiguate things at the domain level beyond just vaguely guessing from the domain type.

Meanwhile, we also effectively have another "cookie" in the form of the anonymous union for other user data, which isn't much better in terms of being vague and unenforced. The fact is that all these cookie types are mutually exclusive, in the sense that combining them makes zero sense and/or would be catastrophic (iommu_set_fault_handler() on an SVA domain, anyone?) - the only combination which *might* be reasonable is perhaps a fault handler and an MSI cookie, but nobody's doing that at the moment, so let's rule it out as well for the sake of being clear and robust. To that end, we pull DMA and MSI cookies apart a little more, mostly to clear up the ambiguity at domain teardown, then for clarity (and to save a little space), move them into the union, whose ownership we can then properly describe and enforce entirely unambiguously.

Quoted 5 lines

[nicolinc: rebase on latest tree; use prefix IOMMU_COOKIE_; merge unions
in iommu_domain; add IOMMU_COOKIE_IOMMUFD for
iommufd_hwpt]

with a nit:

Quoted 9 lines

drivers/iommu/dma-iommu.c void iommu_setup_dma_ops(struct device *dev) · line 1802

Quoted 6 lines
drivers/iommu/dma-iommu.c +1 -1
 	msi_addr &= ~(phys_addr_t)(size - 1);
-	list_for_each_entry(msi_page, &cookie->msi_page_list, list)
+	list_for_each_entry(msi_page, cookie_msi_pages(domain), list)
 		if (msi_page->phys == msi_addr)
 			return msi_page;

Above checks cookie type in every iteration. Better save the list pointer to a local variable.

Ack.

--------------------------------------------------------------
@@ -1793,2 +1793,3 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev,
{

+       struct list_head *msi_page_list = cookie_msi_pages(domain);
        struct iommu_dma_msi_page *msi_page;
@@ -1799,3 +1800,3 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev,
        msi_addr &= ~(phys_addr_t)(size - 1);
-       list_for_each_entry(msi_page, cookie_msi_pages(domain), list)
+       list_for_each_entry(msi_page, msi_page_list, list)
                if (msi_page->phys == msi_addr)
@@ -1817,3 +1818,3 @@ static struct iommu_dma_msi_page *iommu_dma_get_msi_page(struct device *dev,
        msi_page->iova = iova;
-       list_add(&msi_page->list, cookie_msi_pages(domain));
+       list_add(&msi_page->list, msi_page_list);
        return msi_page;

--------------------------------------------------------------

Thanks
Nicolin