Skip to contentSkip to conversation

Re-enable IDXD kernel workqueue under DMA API

PATCH Superseded Jacob Pan 3 participants 27 messages Jul 12, 2023 – Jul 19, 2023

Subsystems (6)

v10 of 12 Superseded Latest is v12 →

lore
Share & tools
lore

Jacob Pan

author
#0 lore raw

v10 cover Re-enable IDXD kernel workqueue under DMA API

Hi Joerg and all,

IDXD kernel work queues were disabled due to the flawed use of kernel VA and SVA API.

The solution is to enable it under DMA API where IDXD shared workqueue users can use ENQCMDS to submit work on buffers mapped by DMA API.

This patchset adds support for attaching PASID to the device's default domain and the ability to allocate global PASIDs from IOMMU APIs. IDXD driver can then re-enable the kernel work queues and use them under DMA API.

This depends on the IOASID removal series. (merged)
https://lore.kernel.org/all/[email protected]/

Thanks,

Jacob

---
Changelog:
v10:
- Fix global PASID alloc function with device's max_pasid=0
v9:
- Fix an IDXD driver issue where user interrupt enable bit got cleared during device enable/disable cycle. Reported and tested by Tony Zhu <[email protected]>
- Rebased to v6.4-rc7
v8:
- further vt-d driver refactoring (3-6) around set/remove device PASID (Baolu)
- make consistent use of NO_PASID in SMMU code (Jean)
- fix off-by-one error in max PASID check (Kevin)
v7:
- renamed IOMMU_DEF_RID_PASID to be IOMMU_NO_PASID to be more generic (Jean)
- simplify range checking for sva PASID (Baolu)
v6:
- use a simplified version of vt-d driver change for set_device_pasid from Baolu.
- check and rename global PASID allocation base
v5:
- exclude two patches related to supervisor mode, taken by VT-d maintainer Baolu.
- move PASID range check into allocation API so that device drivers only need to pass in struct device*. (Kevin)
- factor out helper functions in device-domain attach (Baolu)
- make explicit use of RID_PASID across architectures
v4:
- move dummy functions outside ifdef CONFIG_IOMMU_SVA (Baolu)
- dropped domain type check while disabling idxd system PASID (Baolu)

v3:
- moved global PASID allocation API from SVA to IOMMU (Kevin)
- remove #ifdef around global PASID reservation during boot (Baolu)
- remove restriction on PASID 0 allocation (Baolu)
- fix a bug in sysfs domain change when attaching devices
- clear idxd user interrupt enable bit after disabling device( Fenghua)
v2:
- refactored device PASID attach domain ops based on Baolu's early patch
- addressed TLB flush gap
- explicitly reserve RID_PASID from SVA PASID number space
- get dma domain directly, avoid checking domain types

Jacob Pan (3):
iommu: Generalize PASID 0 for normal DMA w/o PASID
iommu: Move global PASID allocation from SVA to core dmaengine/idxd: Re-enable kernel workqueue under DMA API

Lu Baolu (4):
iommu/vt-d: Add domain_flush_pasid_iotlb() iommu/vt-d: Remove pasid_mutex
iommu/vt-d: Make prq draining code generic iommu/vt-d: Add set_dev_pasid callback for dma domain

 drivers/dma/idxd/device.c                     |  39 ++---
 drivers/dma/idxd/dma.c                        |   5 +-
 drivers/dma/idxd/idxd.h                       |   9 +
 drivers/dma/idxd/init.c                       |  54 +++++-
 drivers/dma/idxd/sysfs.c                      |   7 -
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |   2 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   |  16 +-
 drivers/iommu/intel/iommu.c                   | 161 +++++++++++++++---
 drivers/iommu/intel/iommu.h                   |   9 +
 drivers/iommu/intel/pasid.c                   |   2 +-
 drivers/iommu/intel/pasid.h                   |   2 -
 drivers/iommu/intel/svm.c                     |  53 +-----
 drivers/iommu/iommu-sva.c                     |  28 ++-
 drivers/iommu/iommu.c                         |  28 +++
 include/linux/iommu.h                         |  11 ++
 15 files changed, 291 insertions(+), 135 deletions(-)
Signature · 1 line
2.25.1
author
#1 lore raw

PATCH v10 1/7 iommu: Generalize PASID 0 for normal DMA w/o PASID

PCIe Process address space ID (PASID) is used to tag DMA traffic, it provides finer grained isolation than requester ID (RID).

For each device/RID, 0 is a special PASID for the normal DMA (no PASID). This is universal across all architectures that supports PASID, therefore warranted to be reserved globally and declared in the common header. Consequently, we can avoid the conflict between different PASID use cases in the generic code. e.g. SVA and DMA API with PASIDs.

This paved away for device drivers to choose global PASID policy while continue doing normal DMA.

Noting that VT-d could support none-zero RID/NO_PASID, but currently not used.

---
v8:
- make consistent use of NO_PASID in SMMU code
- remove PASID_MIN
v7:
- renamed IOMMU_DEF_RID_PASID to be IOMMU_NO_PASID to be more generic
v6:
- let SMMU code use the common RID_PASID macro

---
 .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c   |  2 +-
 drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c   | 16 ++++++-------
 drivers/iommu/intel/iommu.c                   | 24 +++++++++----------
 drivers/iommu/intel/pasid.c                   |  2 +-
 drivers/iommu/intel/pasid.h                   |  2 --
 include/linux/iommu.h                         |  1 +
 6 files changed, 23 insertions(+), 24 deletions(-)
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c +1 -1
index a5a63b1c947e..5e6b39881c04 100644
@@ -80,7 +80,7 @@ arm_smmu_share_asid(struct mm_struct *mm, u16 asid)
 	 * be some overlap between use of both ASIDs, until we invalidate the
 	 * TLB.
 	 */
-	arm_smmu_write_ctx_desc(smmu_domain, 0, cd);
+	arm_smmu_write_ctx_desc(smmu_domain, IOMMU_NO_PASID, cd);
 
 	/* Invalidate TLB entries previously associated with that context */
 	arm_smmu_tlb_inv_asid(smmu, asid);
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +8 -8
index 9b0dc3505601..ee70687f060b 100644
@@ -1059,7 +1059,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
 	/*
 	 * This function handles the following cases:
 	 *
-	 * (1) Install primary CD, for normal DMA traffic (SSID = 0).
+	 * (1) Install primary CD, for normal DMA traffic (SSID = IOMMU_NO_PASID = 0).
 	 * (2) Install a secondary CD, for SID+SSID traffic.
 	 * (3) Update ASID of a CD. Atomically write the first 64 bits of the
 	 *     CD, then invalidate the old entry and mappings.
@@ -1607,7 +1607,7 @@ static void arm_smmu_handle_ppr(struct arm_smmu_device *smmu, u64 *evt)
 
 	sid = FIELD_GET(PRIQ_0_SID, evt[0]);
 	ssv = FIELD_GET(PRIQ_0_SSID_V, evt[0]);
-	ssid = ssv ? FIELD_GET(PRIQ_0_SSID, evt[0]) : 0;
+	ssid = ssv ? FIELD_GET(PRIQ_0_SSID, evt[0]) : IOMMU_NO_PASID;
 	last = FIELD_GET(PRIQ_0_PRG_LAST, evt[0]);
 	grpid = FIELD_GET(PRIQ_1_PRG_IDX, evt[1]);
 
@@ -1748,7 +1748,7 @@ arm_smmu_atc_inv_to_cmd(int ssid, unsigned long iova, size_t size,
 	 */
 	*cmd = (struct arm_smmu_cmdq_ent) {
 		.opcode			= CMDQ_OP_ATC_INV,
-		.substream_valid	= !!ssid,
+		.substream_valid	= (ssid != IOMMU_NO_PASID),
 		.atc.ssid		= ssid,
 	};
 
@@ -1795,7 +1795,7 @@ static int arm_smmu_atc_inv_master(struct arm_smmu_master *master)
 	struct arm_smmu_cmdq_ent cmd;
 	struct arm_smmu_cmdq_batch cmds;
 
-	arm_smmu_atc_inv_to_cmd(0, 0, 0, &cmd);
+	arm_smmu_atc_inv_to_cmd(IOMMU_NO_PASID, 0, 0, &cmd);
 
 	cmds.num = 0;
 	for (i = 0; i < master->num_streams; i++) {
@@ -1875,7 +1875,7 @@ static void arm_smmu_tlb_inv_context(void *cookie)
 		cmd.tlbi.vmid	= smmu_domain->s2_cfg.vmid;
 		arm_smmu_cmdq_issue_cmd_with_sync(smmu, &cmd);
 	}
-	arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0);
+	arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, 0, 0);
 }
 
 static void __arm_smmu_tlb_inv_range(struct arm_smmu_cmdq_ent *cmd,
@@ -1968,7 +1968,7 @@ static void arm_smmu_tlb_inv_range_domain(unsigned long iova, size_t size,
 	 * Unfortunately, this can't be leaf-only since we may have
 	 * zapped an entire table.
 	 */
-	arm_smmu_atc_inv_domain(smmu_domain, 0, iova, size);
+	arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, iova, size);
 }
 
 void arm_smmu_tlb_inv_range_asid(unsigned long iova, size_t size, int asid,
@@ -2142,7 +2142,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,
 	 * the master has been added to the devices list for this domain.
 	 * This isn't an issue because the STE hasn't been installed yet.
 	 */
-	ret = arm_smmu_write_ctx_desc(smmu_domain, 0, &cfg->cd);
+	ret = arm_smmu_write_ctx_desc(smmu_domain, IOMMU_NO_PASID, &cfg->cd);
 	if (ret)
 		goto out_free_cd_tables;
 
@@ -2328,7 +2328,7 @@ static void arm_smmu_enable_ats(struct arm_smmu_master *master)
 	pdev = to_pci_dev(master->dev);
 
 	atomic_inc(&smmu_domain->nr_ats_masters);
-	arm_smmu_atc_inv_domain(smmu_domain, 0, 0, 0);
+	arm_smmu_atc_inv_domain(smmu_domain, IOMMU_NO_PASID, 0, 0);
 	if (pci_enable_ats(pdev, stu))
 		dev_err(master->dev, "Failed to enable ATS (STU %zu)\n", stu);
 }
drivers/iommu/intel/iommu.c +12 -12
index 5c8c5cdc36cf..ddff43def3ab 100644
@@ -877,7 +877,7 @@ void dmar_fault_dump_ptes(struct intel_iommu *iommu, u16 source_id,
 	}
 	/* For request-without-pasid, get the pasid from context entry */
 	if (intel_iommu_sm && pasid == IOMMU_PASID_INVALID)
-		pasid = PASID_RID2PASID;
+		pasid = IOMMU_NO_PASID;
 
 	dir_index = pasid >> PASID_PDE_SHIFT;
 	pde = &dir[dir_index];
@@ -1449,7 +1449,7 @@ static void __iommu_flush_dev_iotlb(struct device_domain_info *info,
 	qdep = info->ats_qdep;
 	qi_flush_dev_iotlb(info->iommu, sid, info->pfsid,
 			   qdep, addr, mask);
-	quirk_extra_dev_tlb_flush(info, addr, mask, PASID_RID2PASID, qdep);
+	quirk_extra_dev_tlb_flush(info, addr, mask, IOMMU_NO_PASID, qdep);
 }
 
 static void iommu_flush_dev_iotlb(struct dmar_domain *domain,
@@ -1484,7 +1484,7 @@ static void iommu_flush_iotlb_psi(struct intel_iommu *iommu,
 		ih = 1 << 6;
 
 	if (domain->use_first_level) {
-		qi_flush_piotlb(iommu, did, PASID_RID2PASID, addr, pages, ih);
+		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, pages, ih);
 	} else {
 		unsigned long bitmask = aligned_pages - 1;
 
@@ -1554,7 +1554,7 @@ static void intel_flush_iotlb_all(struct iommu_domain *domain)
 		u16 did = domain_id_iommu(dmar_domain, iommu);
 
 		if (dmar_domain->use_first_level)
-			qi_flush_piotlb(iommu, did, PASID_RID2PASID, 0, -1, 0);
+			qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, 0, -1, 0);
 		else
 			iommu->flush.flush_iotlb(iommu, did, 0, 0,
 						 DMA_TLB_DSI_FLUSH);
@@ -1940,7 +1940,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 				context_pdts(pds);
 
 		/* Setup the RID_PASID field: */
-		context_set_sm_rid2pasid(context, PASID_RID2PASID);
+		context_set_sm_rid2pasid(context, IOMMU_NO_PASID);
 
 		/*
 		 * Setup the Device-TLB enable bit and Page request
@@ -2420,13 +2420,13 @@ static int dmar_domain_attach_device(struct dmar_domain *domain,
 		/* Setup the PASID entry for requests without PASID: */
 		if (hw_pass_through && domain_type_is_si(domain))
 			ret = intel_pasid_setup_pass_through(iommu, domain,
-					dev, PASID_RID2PASID);
+					dev, IOMMU_NO_PASID);
 		else if (domain->use_first_level)
 			ret = domain_setup_first_level(iommu, domain, dev,
-					PASID_RID2PASID);
+					IOMMU_NO_PASID);
 		else
 			ret = intel_pasid_setup_second_level(iommu, domain,
-					dev, PASID_RID2PASID);
+					dev, IOMMU_NO_PASID);
 		if (ret) {
 			dev_err(dev, "Setup RID2PASID failed\n");
 			device_block_translation(dev);
@@ -3968,7 +3968,7 @@ static void dmar_remove_one_dev_info(struct device *dev)
 	if (!dev_is_real_dma_subdevice(info->dev)) {
 		if (dev_is_pci(info->dev) && sm_supported(iommu))
 			intel_pasid_tear_down_entry(iommu, info->dev,
-					PASID_RID2PASID, false);
+					IOMMU_NO_PASID, false);
 
 		iommu_disable_pci_caps(info);
 		domain_context_clear(info);
@@ -3997,7 +3997,7 @@ static void device_block_translation(struct device *dev)
 	if (!dev_is_real_dma_subdevice(dev)) {
 		if (sm_supported(iommu))
 			intel_pasid_tear_down_entry(iommu, dev,
-						    PASID_RID2PASID, false);
+						    IOMMU_NO_PASID, false);
 		else
 			domain_context_clear(info);
 	}
@@ -4331,7 +4331,7 @@ static void domain_set_force_snooping(struct dmar_domain *domain)
 
 	list_for_each_entry(info, &domain->devices, link)
 		intel_pasid_setup_page_snoop_control(info->iommu, info->dev,
-						     PASID_RID2PASID);
+						     IOMMU_NO_PASID);
 }
 
 static bool intel_iommu_enforce_cache_coherency(struct iommu_domain *domain)
@@ -4987,7 +4987,7 @@ void quirk_extra_dev_tlb_flush(struct device_domain_info *info,
 		return;
 
 	sid = PCI_DEVID(info->bus, info->devfn);
-	if (pasid == PASID_RID2PASID) {
+	if (pasid == IOMMU_NO_PASID) {
 		qi_flush_dev_iotlb(info->iommu, sid, info->pfsid,
 				   qdep, address, mask);
 	} else {
drivers/iommu/intel/pasid.c +1 -1
index c5d479770e12..23dca3bc319d 100644
@@ -438,7 +438,7 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu,
 	 * SVA usage, device could do DMA with multiple PASIDs. It is more
 	 * efficient to flush devTLB specific to the PASID.
 	 */
-	if (pasid == PASID_RID2PASID)
+	if (pasid == IOMMU_NO_PASID)
 		qi_flush_dev_iotlb(iommu, sid, pfsid, qdep, 0, 64 - VTD_PAGE_SHIFT);
 	else
 		qi_flush_dev_iotlb_pasid(iommu, sid, pfsid, pasid, qdep, 0, 64 - VTD_PAGE_SHIFT);
drivers/iommu/intel/pasid.h +0 -2
index d6b7d21244b1..4e9e68c3c388 100644
@@ -10,8 +10,6 @@
 #ifndef __INTEL_PASID_H
 #define __INTEL_PASID_H
 
-#define PASID_RID2PASID			0x0
-#define PASID_MIN			0x1
 #define PASID_MAX			0x100000
 #define PASID_PTE_MASK			0x3F
 #define PASID_PTE_PRESENT		1
include/linux/iommu.h +1 -0
index d31642596675..2870bc29d456 100644
@@ -196,6 +196,7 @@ enum iommu_dev_features {
 	IOMMU_DEV_FEAT_IOPF,
 };
 
+#define IOMMU_NO_PASID	(0U) /* Reserved for DMA w/o PASID */
 #define IOMMU_PASID_INVALID	(-1U)
 typedef unsigned int ioasid_t;
 
Signature · 1 line
2.25.1
author
#2 lore raw

PATCH v10 2/7 iommu: Move global PASID allocation from SVA to core

In reply to Jacob Pan

Global PASID can be used beyond SVA. For example, drivers that use Intel ENQCMD to submit work must use global PASIDs in that PASID is stored in a per CPU MSR. When such device need to submit work for in-kernel DMA with PASID, it must allocate PASIDs from the same global number space to avoid conflict.

This patch moves global PASID allocation APIs from SVA to IOMMU APIs. Reserved PASIDs, currently only RID_PASID, are excluded from the global PASID allocation.

It is expected that device drivers will use the allocated PASIDs to attach to appropriate IOMMU domains for use.

---
v8: fix off-by-one in max_pasid check
v7: simplify range check (Baolu)
v6: explicitly exclude reserved a range from SVA PASID allocation
check mm PASID compatibility with device
v5: move PASID range check inside API so that device drivers only pass
in struct device* (Kevin)
v4: move dummy functions outside ifdef CONFIG_IOMMU_SVA (Baolu)

---
 drivers/iommu/iommu-sva.c | 28 ++++++++++------------------
 drivers/iommu/iommu.c     | 28 ++++++++++++++++++++++++++++
 include/linux/iommu.h     | 10 ++++++++++
 3 files changed, 48 insertions(+), 18 deletions(-)
drivers/iommu/iommu-sva.c +10 -18
index 3ebd4b6586b3..8627acc6e60c 100644
@@ -10,33 +10,30 @@
 #include "iommu-sva.h"
 
 static DEFINE_MUTEX(iommu_sva_lock);
-static DEFINE_IDA(iommu_global_pasid_ida);
 
 /* Allocate a PASID for the mm within range (inclusive) */
-static int iommu_sva_alloc_pasid(struct mm_struct *mm, ioasid_t min, ioasid_t max)
+static int iommu_sva_alloc_pasid(struct mm_struct *mm, struct device *dev)
 {
+	ioasid_t pasid;
 	int ret = 0;
 
-	if (min == IOMMU_PASID_INVALID ||
-	    max == IOMMU_PASID_INVALID ||
-	    min == 0 || max < min)
-		return -EINVAL;
-
 	if (!arch_pgtable_dma_compat(mm))
 		return -EBUSY;
 
 	mutex_lock(&iommu_sva_lock);
 	/* Is a PASID already associated with this mm? */
 	if (mm_valid_pasid(mm)) {
-		if (mm->pasid < min || mm->pasid > max)
+		if (mm->pasid >= dev->iommu->max_pasids)
 			ret = -EOVERFLOW;
 		goto out;
 	}
 
-	ret = ida_alloc_range(&iommu_global_pasid_ida, min, max, GFP_KERNEL);
-	if (ret < min)
+	pasid = iommu_alloc_global_pasid_dev(dev);
+	if (pasid == IOMMU_PASID_INVALID) {
+		ret = -ENOSPC;
 		goto out;
-	mm->pasid = ret;
+	}
+	mm->pasid = pasid;
 	ret = 0;
 out:
 	mutex_unlock(&iommu_sva_lock);
@@ -63,15 +60,10 @@ struct iommu_sva *iommu_sva_bind_device(struct device *dev, struct mm_struct *mm
 {
 	struct iommu_domain *domain;
 	struct iommu_sva *handle;
-	ioasid_t max_pasids;
 	int ret;
 
-	max_pasids = dev->iommu->max_pasids;
-	if (!max_pasids)
-		return ERR_PTR(-EOPNOTSUPP);
-
 	/* Allocate mm->pasid if necessary. */
-	ret = iommu_sva_alloc_pasid(mm, 1, max_pasids - 1);
+	ret = iommu_sva_alloc_pasid(mm, dev);
 	if (ret)
 		return ERR_PTR(ret);
 
@@ -216,5 +208,5 @@ void mm_pasid_drop(struct mm_struct *mm)
 	if (likely(!mm_valid_pasid(mm)))
 		return;
 
-	ida_free(&iommu_global_pasid_ida, mm->pasid);
+	iommu_free_global_pasid(mm->pasid);
 }
drivers/iommu/iommu.c +28 -0
index da340f11c5f5..37d5e1b8ed01 100644
@@ -39,6 +39,7 @@
 
 static struct kset *iommu_group_kset;
 static DEFINE_IDA(iommu_group_ida);
+static DEFINE_IDA(iommu_global_pasid_ida);
 
 static unsigned int iommu_def_domain_type __read_mostly;
 static bool iommu_dma_strict __read_mostly = IS_ENABLED(CONFIG_IOMMU_DEFAULT_DMA_STRICT);
@@ -3399,3 +3400,30 @@ struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
 
 	return domain;
 }
+
+ioasid_t iommu_alloc_global_pasid_dev(struct device *dev)
+{
+	int ret;
+
+	/* max_pasids == 0 means that the device does not support PASID */
+	if (!dev->iommu->max_pasids)
+		return IOMMU_PASID_INVALID;
+
+	/*
+	 * max_pasids is set up by vendor driver based on number of PASID bits
+	 * supported but the IDA allocation is inclusive.
+	 */
+	ret = ida_alloc_range(&iommu_global_pasid_ida, IOMMU_FIRST_GLOBAL_PASID,
+			      dev->iommu->max_pasids - 1, GFP_KERNEL);
+	return ret < 0 ? IOMMU_PASID_INVALID : ret;
+}
+EXPORT_SYMBOL_GPL(iommu_alloc_global_pasid_dev);
+
+void iommu_free_global_pasid(ioasid_t pasid)
+{
+	if (WARN_ON(pasid == IOMMU_PASID_INVALID))
+		return;
+
+	ida_free(&iommu_global_pasid_ida, pasid);
+}
+EXPORT_SYMBOL_GPL(iommu_free_global_pasid);
include/linux/iommu.h +10 -0
index 2870bc29d456..e97148f31038 100644
@@ -197,6 +197,7 @@ enum iommu_dev_features {
 };
 
 #define IOMMU_NO_PASID	(0U) /* Reserved for DMA w/o PASID */
+#define IOMMU_FIRST_GLOBAL_PASID	(1U) /*starting range for allocation */
 #define IOMMU_PASID_INVALID	(-1U)
 typedef unsigned int ioasid_t;
 
@@ -728,6 +729,8 @@ void iommu_detach_device_pasid(struct iommu_domain *domain,
 struct iommu_domain *
 iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid,
 			       unsigned int type);
+ioasid_t iommu_alloc_global_pasid_dev(struct device *dev);
+void iommu_free_global_pasid(ioasid_t pasid);
 #else /* CONFIG_IOMMU_API */
 
 struct iommu_ops {};
@@ -1089,6 +1092,13 @@ iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid,
 {
 	return NULL;
 }
+
+static inline ioasid_t iommu_alloc_global_pasid_dev(struct device *dev)
+{
+	return IOMMU_PASID_INVALID;
+}
+
+static inline void iommu_free_global_pasid(ioasid_t pasid) {}
 #endif /* CONFIG_IOMMU_API */
 
 /**
Signature · 1 line
2.25.1
#3 lore raw

RE: [PATCH v10 2/7] iommu: Move global PASID allocation from SVA to core

Quoted 20 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

}

drivers/iommu/iommu.c +11 -0
+
+ioasid_t iommu_alloc_global_pasid_dev(struct device *dev)
+{
+	int ret;
+
+	/* max_pasids == 0 means that the device does not support PASID */
+	if (!dev->iommu->max_pasids)
+		return IOMMU_PASID_INVALID;
+
+	/*
+	 * max_pasids is set up by vendor driver based on number of PASID

bits

drivers/iommu/iommu.c +3 -0
+	 * supported but the IDA allocation is inclusive.
+	 */
+	ret = ida_alloc_range(&iommu_global_pasid_ida,

IOMMU_FIRST_GLOBAL_PASID,

drivers/iommu/iommu.c struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, · line 3420

Quoted 4 lines
drivers/iommu/iommu.c +4 -0
+			      dev->iommu->max_pasids - 1, GFP_KERNEL);
+	return ret < 0 ? IOMMU_PASID_INVALID : ret;
+}
+EXPORT_SYMBOL_GPL(iommu_alloc_global_pasid_dev);

nit. Just call it iommu_alloc_global_pasid.

author
#4 lore raw

Re: [PATCH v10 2/7] iommu: Move global PASID allocation from SVA to core

Hi Kevin,

Quoted 19 lines · Kevin
Quoted 19 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

}

drivers/iommu/iommu.c +6 -0
+
+ioasid_t iommu_alloc_global_pasid_dev(struct device *dev)
+{
+	int ret;
+
+	/* max_pasids == 0 means that the device does not support

PASID */

drivers/iommu/iommu.c +5 -0
+	if (!dev->iommu->max_pasids)
+		return IOMMU_PASID_INVALID;
+
+	/*
+	 * max_pasids is set up by vendor driver based on number of

PASID bits

drivers/iommu/iommu.c +2 -0
+	 * supported but the IDA allocation is inclusive.
+	 */
Quoted 8 lines

drivers/iommu/iommu.c struct iommu_domain *iommu_sva_domain_alloc(struct device *dev, · line 3420

Quoted 6 lines
(quoted diff) +1 -0
+	ret = ida_alloc_range(&iommu_global_pasid_ida,

IOMMU_FIRST_GLOBAL_PASID,

drivers/iommu/iommu.c +4 -0
+			      dev->iommu->max_pasids - 1, GFP_KERNEL);
+	return ret < 0 ? IOMMU_PASID_INVALID : ret;
+}
+EXPORT_SYMBOL_GPL(iommu_alloc_global_pasid_dev);  

nit. Just call it iommu_alloc_global_pasid.

will do. it will be symmetric.

Thanks,

Jacob

author
#5 lore raw

PATCH v10 3/7 iommu/vt-d: Add domain_flush_pasid_iotlb()

In reply to Jacob Pan

From: Lu Baolu <[email protected]>

The VT-d spec requires to use PASID-based-IOTLB invalidation descriptor to invalidate IOTLB and the paging-structure caches for a first-stage page table. Add a generic helper to do this.

RID2PASID is used if the domain has been attached to a physical device, otherwise real PASIDs that the domain has been attached to will be used.

---
 drivers/iommu/intel/iommu.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
drivers/iommu/intel/iommu.c +20 -2
index ddff43def3ab..40685cbfaf0e 100644
@@ -1467,6 +1467,24 @@ static void iommu_flush_dev_iotlb(struct dmar_domain *domain,
 	spin_unlock_irqrestore(&domain->lock, flags);
 }
 
+/*
+ * The VT-d spec requires to use PASID-based-IOTLB Invalidation to
+ * invalidate IOTLB and the paging-structure-caches for a first-stage
+ * page table.
+ */
+static void domain_flush_pasid_iotlb(struct intel_iommu *iommu,
+				     struct dmar_domain *domain, u64 addr,
+				     unsigned long npages, bool ih)
+{
+	u16 did = domain_id_iommu(domain, iommu);
+	unsigned long flags;
+
+	spin_lock_irqsave(&domain->lock, flags);
+	if (!list_empty(&domain->devices))
+		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih);
+	spin_unlock_irqrestore(&domain->lock, flags);
+}
+
 static void iommu_flush_iotlb_psi(struct intel_iommu *iommu,
 				  struct dmar_domain *domain,
 				  unsigned long pfn, unsigned int pages,
@@ -1484,7 +1502,7 @@ static void iommu_flush_iotlb_psi(struct intel_iommu *iommu,
 		ih = 1 << 6;
 
 	if (domain->use_first_level) {
-		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, pages, ih);
+		domain_flush_pasid_iotlb(iommu, domain, addr, pages, ih);
 	} else {
 		unsigned long bitmask = aligned_pages - 1;
 
@@ -1554,7 +1572,7 @@ static void intel_flush_iotlb_all(struct iommu_domain *domain)
 		u16 did = domain_id_iommu(dmar_domain, iommu);
 
 		if (dmar_domain->use_first_level)
-			qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, 0, -1, 0);
+			domain_flush_pasid_iotlb(iommu, dmar_domain, 0, -1, 0);
 		else
 			iommu->flush.flush_iotlb(iommu, did, 0, 0,
 						 DMA_TLB_DSI_FLUSH);
Signature · 1 line
2.25.1
Reviewed-by on v10 3/7
#6 lore raw

RE: [PATCH v10 3/7] iommu/vt-d: Add domain_flush_pasid_iotlb()

Quoted 11 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

From: Lu Baolu <[email protected]>

The VT-d spec requires to use PASID-based-IOTLB invalidation descriptor to invalidate IOTLB and the paging-structure caches for a first-stage page table. Add a generic helper to do this.

RID2PASID is used if the domain has been attached to a physical device, otherwise real PASIDs that the domain has been attached to will be used.

this should mention that 'real' PASID attach is not handled in this patch. Otherwise it's confusing to connect this description to the new helper.

Quoted 12 lines
---
 drivers/iommu/intel/iommu.c | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
drivers/iommu/intel/iommu.c +0 -0
index ddff43def3ab..40685cbfaf0e 100644

drivers/iommu/intel/iommu.c static void iommu_flush_dev_iotlb(struct · line 1475

Quoted 9 lines
drivers/iommu/intel/iommu.c +5 -0
@@ -1467,6 +1467,24 @@ static void iommu_flush_dev_iotlb(struct
 dmar_domain *domain,
 	spin_unlock_irqrestore(&domain->lock, flags);
 }
 
+/*
+ * The VT-d spec requires to use PASID-based-IOTLB Invalidation to
+ * invalidate IOTLB and the paging-structure-caches for a first-stage
+ * page table.
+ */

this comment is better placed in the calling point. This function has nothing explicitly connected to first-stage.

with that fixed,

#7 lore raw

RE: [PATCH v10 3/7] iommu/vt-d: Add domain_flush_pasid_iotlb()

In reply to Jacob Pan
Quoted 13 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +10 -0
+static void domain_flush_pasid_iotlb(struct intel_iommu *iommu,
+				     struct dmar_domain *domain, u64 addr,
+				     unsigned long npages, bool ih)
+{
+	u16 did = domain_id_iommu(domain, iommu);
+	unsigned long flags;
+
+	spin_lock_irqsave(&domain->lock, flags);
+	if (!list_empty(&domain->devices))
+		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr,

npages, ih);

drivers/iommu/intel/iommu.c static void iommu_flush_dev_iotlb(struct dmar_domain *domain, · line 1485

Quoted 1 line
drivers/iommu/intel/iommu.c +1 -0
+	spin_unlock_irqrestore(&domain->lock, flags);

btw I gave a comment before that the check of list_empty() changes the semantics instead of just creating a helper.

If it's the right thing to do please split it into a separate fix patch.

maintainer
#8 lore raw

Re: [PATCH v10 3/7] iommu/vt-d: Add domain_flush_pasid_iotlb()

Quoted 18 lines · Kevin
Quoted 13 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +10 -0
+static void domain_flush_pasid_iotlb(struct intel_iommu *iommu,
+				     struct dmar_domain *domain, u64 addr,
+				     unsigned long npages, bool ih)
+{
+	u16 did = domain_id_iommu(domain, iommu);
+	unsigned long flags;
+
+	spin_lock_irqsave(&domain->lock, flags);
+	if (!list_empty(&domain->devices))
+		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr,

npages, ih);

drivers/iommu/intel/iommu.c static void iommu_flush_dev_iotlb(struct dmar_domain *domain, · line 1485

Quoted 1 line
drivers/iommu/intel/iommu.c +1 -0
+	spin_unlock_irqrestore(&domain->lock, flags);

btw I gave a comment before that the check of list_empty() changes the semantics instead of just creating a helper.

Quoted 1 line

If it's the right thing to do please split it into a separate fix patch.

Perhaps move it into patch 6?

Best regards,
baolu

#9 lore raw

RE: [PATCH v10 3/7] iommu/vt-d: Add domain_flush_pasid_iotlb()

Quoted 22 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:37 AM

Quoted 18 lines · Kevin
Quoted 13 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +10 -0
+static void domain_flush_pasid_iotlb(struct intel_iommu *iommu,
+				     struct dmar_domain *domain, u64 addr,
+				     unsigned long npages, bool ih)
+{
+	u16 did = domain_id_iommu(domain, iommu);
+	unsigned long flags;
+
+	spin_lock_irqsave(&domain->lock, flags);
+	if (!list_empty(&domain->devices))
+		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr,

npages, ih);

drivers/iommu/intel/iommu.c static void iommu_flush_dev_iotlb(struct dmar_domain *domain, · line 1485

Quoted 1 line
drivers/iommu/intel/iommu.c +1 -0
+	spin_unlock_irqrestore(&domain->lock, flags);

btw I gave a comment before that the check of list_empty() changes the semantics instead of just creating a helper.

Quoted 4 lines
Quoted 1 line

If it's the right thing to do please split it into a separate fix patch.

Perhaps move it into patch 6?

I still prefer to putting it in a separate patch since it changes the behavior in existing path. It's not really about dev_pasid which patch6 is trying to support.

maintainer
#10 lore raw

Re: [PATCH v10 3/7] iommu/vt-d: Add domain_flush_pasid_iotlb()

Quoted 22 lines · Kevin
Quoted 22 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:37 AM

Quoted 18 lines · Kevin
Quoted 13 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +10 -0
+static void domain_flush_pasid_iotlb(struct intel_iommu *iommu,
+				     struct dmar_domain *domain, u64 addr,
+				     unsigned long npages, bool ih)
+{
+	u16 did = domain_id_iommu(domain, iommu);
+	unsigned long flags;
+
+	spin_lock_irqsave(&domain->lock, flags);
+	if (!list_empty(&domain->devices))
+		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr,

npages, ih);

drivers/iommu/intel/iommu.c static void iommu_flush_dev_iotlb(struct dmar_domain *domain, · line 1485

Quoted 1 line
drivers/iommu/intel/iommu.c +1 -0
+	spin_unlock_irqrestore(&domain->lock, flags);

btw I gave a comment before that the check of list_empty() changes the semantics instead of just creating a helper.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

If it's the right thing to do please split it into a separate fix patch.

Perhaps move it into patch 6?

I still prefer to putting it in a separate patch since it changes the behavior in existing path. It's not really about dev_pasid which patch6 is trying to support.

Okay, that will also be fine to me.

Best regards,
baolu

author
#11 lore raw

PATCH v10 4/7 iommu/vt-d: Remove pasid_mutex

In reply to Jacob Pan

From: Lu Baolu <[email protected]>

The pasid_mutex was used to protect the paths of set/remove_dev_pasid(). It's duplicate with iommu_sva_lock. Remove it to avoid duplicate code.

---
 drivers/iommu/intel/svm.c | 45 +++++----------------------------------
 1 file changed, 5 insertions(+), 40 deletions(-)
drivers/iommu/intel/svm.c +5 -40
index e95b339e9cdc..2a82864e9d57 100644
@@ -259,8 +259,6 @@ static const struct mmu_notifier_ops intel_mmuops = {
 	.invalidate_range = intel_invalidate_range,
 };
 
-static DEFINE_MUTEX(pasid_mutex);
-
 static int pasid_to_svm_sdev(struct device *dev, unsigned int pasid,
 			     struct intel_svm **rsvm,
 			     struct intel_svm_dev **rsdev)
@@ -268,10 +266,6 @@ static int pasid_to_svm_sdev(struct device *dev, unsigned int pasid,
 	struct intel_svm_dev *sdev = NULL;
 	struct intel_svm *svm;
 
-	/* The caller should hold the pasid_mutex lock */
-	if (WARN_ON(!mutex_is_locked(&pasid_mutex)))
-		return -EINVAL;
-
 	if (pasid == IOMMU_PASID_INVALID || pasid >= PASID_MAX)
 		return -EINVAL;
 
@@ -371,22 +365,19 @@ static int intel_svm_bind_mm(struct intel_iommu *iommu, struct device *dev,
 	return ret;
 }
 
-/* Caller must hold pasid_mutex */
-static int intel_svm_unbind_mm(struct device *dev, u32 pasid)
+void intel_svm_remove_dev_pasid(struct device *dev, u32 pasid)
 {
 	struct intel_svm_dev *sdev;
 	struct intel_iommu *iommu;
 	struct intel_svm *svm;
 	struct mm_struct *mm;
-	int ret = -EINVAL;
 
 	iommu = device_to_iommu(dev, NULL, NULL);
 	if (!iommu)
-		goto out;
+		return;
 
-	ret = pasid_to_svm_sdev(dev, pasid, &svm, &sdev);
-	if (ret)
-		goto out;
+	if (pasid_to_svm_sdev(dev, pasid, &svm, &sdev))
+		return;
 	mm = svm->mm;
 
 	if (sdev) {
@@ -418,8 +409,6 @@ static int intel_svm_unbind_mm(struct device *dev, u32 pasid)
 			kfree(svm);
 		}
 	}
-out:
-	return ret;
 }
 
 /* Page request queue descriptor */
@@ -520,19 +509,7 @@ static void intel_svm_drain_prq(struct device *dev, u32 pasid)
 		goto prq_retry;
 	}
 
-	/*
-	 * A work in IO page fault workqueue may try to lock pasid_mutex now.
-	 * Holding pasid_mutex while waiting in iopf_queue_flush_dev() for
-	 * all works in the workqueue to finish may cause deadlock.
-	 *
-	 * It's unnecessary to hold pasid_mutex in iopf_queue_flush_dev().
-	 * Unlock it to allow the works to be handled while waiting for
-	 * them to finish.
-	 */
-	lockdep_assert_held(&pasid_mutex);
-	mutex_unlock(&pasid_mutex);
 	iopf_queue_flush_dev(dev);
-	mutex_lock(&pasid_mutex);
 
 	/*
 	 * Perform steps described in VT-d spec CH7.10 to drain page
@@ -827,26 +804,14 @@ int intel_svm_page_response(struct device *dev,
 	return ret;
 }
 
-void intel_svm_remove_dev_pasid(struct device *dev, ioasid_t pasid)
-{
-	mutex_lock(&pasid_mutex);
-	intel_svm_unbind_mm(dev, pasid);
-	mutex_unlock(&pasid_mutex);
-}
-
 static int intel_svm_set_dev_pasid(struct iommu_domain *domain,
 				   struct device *dev, ioasid_t pasid)
 {
 	struct device_domain_info *info = dev_iommu_priv_get(dev);
 	struct intel_iommu *iommu = info->iommu;
 	struct mm_struct *mm = domain->mm;
-	int ret;
 
-	mutex_lock(&pasid_mutex);
-	ret = intel_svm_bind_mm(iommu, dev, mm);
-	mutex_unlock(&pasid_mutex);
-
-	return ret;
+	return intel_svm_bind_mm(iommu, dev, mm);
 }
 
 static void intel_svm_domain_free(struct iommu_domain *domain)
Signature · 1 line
2.25.1
Reviewed-by on v10 4/7
#12 lore raw

RE: [PATCH v10 4/7] iommu/vt-d: Remove pasid_mutex

Quoted 10 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

From: Lu Baolu <[email protected]>

The pasid_mutex was used to protect the paths of set/remove_dev_pasid(). It's duplicate with iommu_sva_lock. Remove it to avoid duplicate code.

author
#13 lore raw

PATCH v10 5/7 iommu/vt-d: Make prq draining code generic

In reply to Jacob Pan

From: Lu Baolu <[email protected]>

Currently draining page requests and responses for a pasid is part of SVA implementation. This is because the driver only supports attaching an SVA domain to a device pasid. As we are about to support attaching other types of domains to a device pasid, the prq draining code becomes generic.

---
 drivers/iommu/intel/iommu.c | 30 +++++++++++++++++++-----------
 drivers/iommu/intel/iommu.h |  2 ++
 drivers/iommu/intel/svm.c   |  8 ++------
 3 files changed, 23 insertions(+), 17 deletions(-)
drivers/iommu/intel/iommu.c +19 -11
index 40685cbfaf0e..4b54a56831b4 100644
@@ -4733,21 +4733,29 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
 	struct intel_iommu *iommu = device_to_iommu(dev, NULL, NULL);
 	struct iommu_domain *domain;
 
-	/* Domain type specific cleanup: */
 	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
-	if (domain) {
-		switch (domain->type) {
-		case IOMMU_DOMAIN_SVA:
-			intel_svm_remove_dev_pasid(dev, pasid);
-			break;
-		default:
-			/* should never reach here */
-			WARN_ON(1);
-			break;
-		}
+	if (!domain)
+		goto out_tear_down;
+
+	/*
+	 * The SVA implementation needs to handle its own stuffs like the mm
+	 * notification. Before consolidating that code into iommu core, let
+	 * the intel sva code handle it.
+	 */
+	if (domain->type == IOMMU_DOMAIN_SVA) {
+		intel_svm_remove_dev_pasid(dev, pasid);
+		goto out_tear_down;
 	}
 
+	/*
+	 * Should never reach here until we add support for attaching
+	 * non-SVA domain to a pasid.
+	 */
+	WARN_ON(1);
+
+out_tear_down:
 	intel_pasid_tear_down_entry(iommu, dev, pasid, false);
+	intel_drain_pasid_prq(dev, pasid);
 }
 
 const struct iommu_ops intel_iommu_ops = {
drivers/iommu/intel/iommu.h +2 -0
index 1c5e1d88862b..6d94a29f5d52 100644
@@ -844,6 +844,7 @@ int intel_svm_page_response(struct device *dev, struct iommu_fault_event *evt,
 			    struct iommu_page_response *msg);
 struct iommu_domain *intel_svm_domain_alloc(void);
 void intel_svm_remove_dev_pasid(struct device *dev, ioasid_t pasid);
+void intel_drain_pasid_prq(struct device *dev, u32 pasid);
 
 struct intel_svm_dev {
 	struct list_head list;
@@ -862,6 +863,7 @@ struct intel_svm {
 };
 #else
 static inline void intel_svm_check(struct intel_iommu *iommu) {}
+static inline void intel_drain_pasid_prq(struct device *dev, u32 pasid) {}
 static inline struct iommu_domain *intel_svm_domain_alloc(void)
 {
 	return NULL;
drivers/iommu/intel/svm.c +2 -6
index 2a82864e9d57..588367a9e9b5 100644
@@ -26,8 +26,6 @@
 #include "trace.h"
 
 static irqreturn_t prq_event_thread(int irq, void *d);
-static void intel_svm_drain_prq(struct device *dev, u32 pasid);
-#define to_intel_svm_dev(handle) container_of(handle, struct intel_svm_dev, sva)
 
 static DEFINE_XARRAY_ALLOC(pasid_private_array);
 static int pasid_private_add(ioasid_t pasid, void *priv)
@@ -391,8 +389,6 @@ void intel_svm_remove_dev_pasid(struct device *dev, u32 pasid)
 		 * large and has to be physically contiguous. So it's
 		 * hard to be as defensive as we might like.
 		 */
-		intel_pasid_tear_down_entry(iommu, dev, svm->pasid, false);
-		intel_svm_drain_prq(dev, svm->pasid);
 		kfree_rcu(sdev, rcu);
 
 		if (list_empty(&svm->devs)) {
@@ -449,7 +445,7 @@ static bool is_canonical_address(u64 addr)
 }
 
 /**
- * intel_svm_drain_prq - Drain page requests and responses for a pasid
+ * intel_drain_pasid_prq - Drain page requests and responses for a pasid
  * @dev: target device
  * @pasid: pasid for draining
  *
@@ -463,7 +459,7 @@ static bool is_canonical_address(u64 addr)
  * described in VT-d spec CH7.10 to drain all page requests and page
  * responses pending in the hardware.
  */
-static void intel_svm_drain_prq(struct device *dev, u32 pasid)
+void intel_drain_pasid_prq(struct device *dev, u32 pasid)
 {
 	struct device_domain_info *info;
 	struct dmar_domain *domain;
Signature · 1 line
2.25.1
#14 lore raw

RE: [PATCH v10 5/7] iommu/vt-d: Make prq draining code generic

Quoted 7 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +0 -3
-	/* Domain type specific cleanup: */
 	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
-	if (domain) {
-		switch (domain->type) {

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4738

Quoted 10 lines
drivers/iommu/intel/iommu.c +2 -8
-		case IOMMU_DOMAIN_SVA:
-			intel_svm_remove_dev_pasid(dev, pasid);
-			break;
-		default:
-			/* should never reach here */
-			WARN_ON(1);
-			break;
-		}
+	if (!domain)
+		goto out_tear_down;

WARN_ON()

drivers/iommu/intel/svm.c void intel_svm_remove_dev_pasid(struct device *dev, u32 pasid) · line 395

Quoted 6 lines
drivers/iommu/intel/svm.c +0 -1
 		 * large and has to be physically contiguous. So it's
 		 * hard to be as defensive as we might like.
 		 */
-		intel_pasid_tear_down_entry(iommu, dev, svm->pasid,

false);

drivers/iommu/intel/svm.c +0 -1
-		intel_svm_drain_prq(dev, svm->pasid);

after removing the 2 lines the comment above becomes stale.

drivers/iommu/intel/svm.c static bool is_canonical_address(u64 addr) · line 465

Quoted 5 lines
drivers/iommu/intel/svm.c +1 -1
-static void intel_svm_drain_prq(struct device *dev, u32 pasid)
+void intel_drain_pasid_prq(struct device *dev, u32 pasid)
 {
 	struct device_domain_info *info;
 	struct dmar_domain *domain;

later we should move generic prq handling logic out of svm.c into a new prq.c

maintainer
#15 lore raw

Re: [PATCH v10 5/7] iommu/vt-d: Make prq draining code generic

Quoted 19 lines · Kevin
Quoted 7 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +0 -3
-	/* Domain type specific cleanup: */
  	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
-	if (domain) {
-		switch (domain->type) {

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4738

Quoted 10 lines
drivers/iommu/intel/iommu.c +2 -8
-		case IOMMU_DOMAIN_SVA:
-			intel_svm_remove_dev_pasid(dev, pasid);
-			break;
-		default:
-			/* should never reach here */
-			WARN_ON(1);
-			break;
-		}
+	if (!domain)
+		goto out_tear_down;

WARN_ON()

Why?

My understanding is that remve_device_pasid could be call in any context including no domain attached.

Quoted 9 lines

drivers/iommu/intel/svm.c void intel_svm_remove_dev_pasid(struct device *dev, u32 pasid) · line 395

Quoted 7 lines

* large and has to be physically contiguous. So it's
* hard to be as defensive as we might like. */

(quoted diff) +0 -1
-		intel_pasid_tear_down_entry(iommu, dev, svm->pasid,

false);

drivers/iommu/intel/svm.c +0 -1
-		intel_svm_drain_prq(dev, svm->pasid);

after removing the 2 lines the comment above becomes stale.

Yes.

Quoted 9 lines

drivers/iommu/intel/svm.c static bool is_canonical_address(u64 addr) · line 465

Quoted 6 lines
drivers/iommu/intel/svm.c +1 -1
-static void intel_svm_drain_prq(struct device *dev, u32 pasid)
+void intel_drain_pasid_prq(struct device *dev, u32 pasid)
  {
  	struct device_domain_info *info;
  	struct dmar_domain *domain;

later we should move generic prq handling logic out of svm.c into a new prq.c

Yes.

Best regards,
baolu

#16 lore raw

RE: [PATCH v10 5/7] iommu/vt-d: Make prq draining code generic

Quoted 24 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:28 AM

Quoted 19 lines · Kevin
Quoted 7 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +0 -3
-	/* Domain type specific cleanup: */
  	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
-	if (domain) {
-		switch (domain->type) {

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4738

Quoted 10 lines
drivers/iommu/intel/iommu.c +2 -8
-		case IOMMU_DOMAIN_SVA:
-			intel_svm_remove_dev_pasid(dev, pasid);
-			break;
-		default:
-			/* should never reach here */
-			WARN_ON(1);
-			break;
-		}
+	if (!domain)
+		goto out_tear_down;

WARN_ON()

Quoted 5 lines

Why?

My understanding is that remve_device_pasid could be call in any context including no domain attached.

oh I'm not aware of that. Can you elaborate the usage which uses a pasid w/o domain? pasid needs to point to a page table. Presumably every page table should be wrapped by a iommu domain...

maintainer
#17 lore raw

Re: [PATCH v10 5/7] iommu/vt-d: Make prq draining code generic

Quoted 26 lines · Kevin
Quoted 26 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:28 AM

Quoted 19 lines · Kevin
Quoted 7 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +0 -3
-	/* Domain type specific cleanup: */
   	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
-	if (domain) {
-		switch (domain->type) {

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4738

Quoted 10 lines
drivers/iommu/intel/iommu.c +2 -8
-		case IOMMU_DOMAIN_SVA:
-			intel_svm_remove_dev_pasid(dev, pasid);
-			break;
-		default:
-			/* should never reach here */
-			WARN_ON(1);
-			break;
-		}
+	if (!domain)
+		goto out_tear_down;

WARN_ON()

Why?

Quoted 7 lines
Quoted 3 lines

My understanding is that remve_device_pasid could be call in any context including no domain attached.

oh I'm not aware of that. Can you elaborate the usage which uses a pasid w/o domain? pasid needs to point to a page table. Presumably every page table should be wrapped by a iommu domain...

A case I can think of is error rewinding. A domain is being attached to multiple pasids. When one of them is failed, remove_device_pasid should be called on all pasids so that they are parked at a determinant state.

On the other hand, I don't want the remove_device_pasid to be the counterpart of attach_dev_pasid. remove_device_pasid simply denotes:

- The pasid will be parked in blocking state;
- If any domain that has been attached to this pasid, stop reference to it any more. Otherwise, there might be use-after-free issues.

Hence, remove_device_pasid should never fail.

Best regards,
baolu

#18 lore raw

RE: [PATCH v10 5/7] iommu/vt-d: Make prq draining code generic

Quoted 34 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 1:47 PM

Quoted 30 lines · Kevin
Quoted 29 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:28 AM

Quoted 19 lines · Kevin
Quoted 7 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +0 -3
-	/* Domain type specific cleanup: */
   	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
-	if (domain) {
-		switch (domain->type) {

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4738

Quoted 10 lines
drivers/iommu/intel/iommu.c +2 -8
-		case IOMMU_DOMAIN_SVA:
-			intel_svm_remove_dev_pasid(dev, pasid);
-			break;
-		default:
-			/* should never reach here */
-			WARN_ON(1);
-			break;
-		}
+	if (!domain)
+		goto out_tear_down;

WARN_ON()

Why?

My understanding is that remve_device_pasid could be call in any context including no domain attached.

Quoted 7 lines
Quoted 3 lines

oh I'm not aware of that. Can you elaborate the usage which uses a pasid w/o domain? pasid needs to point to a page table. Presumably every page table should be wrapped by a iommu domain...

A case I can think of is error rewinding. A domain is being attached to multiple pasids. When one of them is failed, remove_device_pasid should be called on all pasids so that they are parked at a determinant state.

Can you elaborate what is the association among those pasid's so failing one would lead to failing all?

Just like a domain can be attached to multiple devices. I don't think there is an unwinding policy forcing to detach all devices just because there is a failure attaching the domain to a new one.

Quoted 10 lines

On the other hand, I don't want the remove_device_pasid to be the counterpart of attach_dev_pasid. remove_device_pasid simply denotes:

- The pasid will be parked in blocking state;
- If any domain that has been attached to this pasid, stop reference to it any more. Otherwise, there might be use-after-free issues.

Hence, remove_device_pasid should never fail.

It should never fail. But could warn if there is a condition which shouldn't be hit. 😊

maintainer
#19 lore raw

Re: [PATCH v10 5/7] iommu/vt-d: Make prq draining code generic

Quoted 42 lines · Kevin
Quoted 34 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 1:47 PM

Quoted 30 lines · Kevin
Quoted 29 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:28 AM

Quoted 19 lines · Kevin
Quoted 7 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +0 -3
-	/* Domain type specific cleanup: */
    	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
-	if (domain) {
-		switch (domain->type) {

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4738

Quoted 10 lines
drivers/iommu/intel/iommu.c +2 -8
-		case IOMMU_DOMAIN_SVA:
-			intel_svm_remove_dev_pasid(dev, pasid);
-			break;
-		default:
-			/* should never reach here */
-			WARN_ON(1);
-			break;
-		}
+	if (!domain)
+		goto out_tear_down;

WARN_ON()

Why?

My understanding is that remve_device_pasid could be call in any context including no domain attached.

Quoted 7 lines
Quoted 3 lines

oh I'm not aware of that. Can you elaborate the usage which uses a pasid w/o domain? pasid needs to point to a page table. Presumably every page table should be wrapped by a iommu domain...

A case I can think of is error rewinding. A domain is being attached to multiple pasids. When one of them is failed, remove_device_pasid should be called on all pasids so that they are parked at a determinant state.

Quoted 6 lines

Can you elaborate what is the association among those pasid's so failing one would lead to failing all?

Just like a domain can be attached to multiple devices. I don't think there is an unwinding policy forcing to detach all devices just because there is a failure attaching the domain to a new one.

It's fine to add a check here if there's no real case. I was just thinking that remove_device_pasid could also be invoked when there's no domain attached.

Quoted 8 lines
Quoted 8 lines

On the other hand, I don't want the remove_device_pasid to be the counterpart of attach_dev_pasid. remove_device_pasid simply denotes:

- The pasid will be parked in blocking state;
- If any domain that has been attached to this pasid, stop reference to it any more. Otherwise, there might be use-after-free issues.

Quoted 5 lines
Quoted 2 lines

Hence, remove_device_pasid should never fail.

It should never fail. But could warn if there is a condition which shouldn't be hit. 😊

Okay, let's add a check here. And we can loose it later if any real use case comes.

Best regards,
baolu

author
#20 lore raw

PATCH v10 6/7 iommu/vt-d: Add set_dev_pasid callback for dma domain

In reply to Jacob Pan

From: Lu Baolu <[email protected]>

This allows the upper layers to set a domain to a PASID of a device if the PASID feature is supported by the IOMMU hardware. The typical use cases are, for example, kernel DMA with PASID and hardware assisted mediated device drivers.

The attaching device and pasid information is tracked in a per-domain list and is used for IOTLB and devTLB invalidation.

---
 drivers/iommu/intel/iommu.c | 103 ++++++++++++++++++++++++++++++++++--
 drivers/iommu/intel/iommu.h |   7 +++
 2 files changed, 105 insertions(+), 5 deletions(-)
drivers/iommu/intel/iommu.c +98 -5
index 4b54a56831b4..f710e724c51a 100644
@@ -1359,6 +1359,7 @@ domain_lookup_dev_info(struct dmar_domain *domain,
 
 static void domain_update_iotlb(struct dmar_domain *domain)
 {
+	struct dev_pasid_info *dev_pasid;
 	struct device_domain_info *info;
 	bool has_iotlb_device = false;
 	unsigned long flags;
@@ -1370,6 +1371,14 @@ static void domain_update_iotlb(struct dmar_domain *domain)
 			break;
 		}
 	}
+
+	list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) {
+		info = dev_iommu_priv_get(dev_pasid->dev);
+		if (info->ats_enabled) {
+			has_iotlb_device = true;
+			break;
+		}
+	}
 	domain->has_iotlb_device = has_iotlb_device;
 	spin_unlock_irqrestore(&domain->lock, flags);
 }
@@ -1455,6 +1464,7 @@ static void __iommu_flush_dev_iotlb(struct device_domain_info *info,
 static void iommu_flush_dev_iotlb(struct dmar_domain *domain,
 				  u64 addr, unsigned mask)
 {
+	struct dev_pasid_info *dev_pasid;
 	struct device_domain_info *info;
 	unsigned long flags;
 
@@ -1464,6 +1474,19 @@ static void iommu_flush_dev_iotlb(struct dmar_domain *domain,
 	spin_lock_irqsave(&domain->lock, flags);
 	list_for_each_entry(info, &domain->devices, link)
 		__iommu_flush_dev_iotlb(info, addr, mask);
+
+	list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) {
+		info = dev_iommu_priv_get(dev_pasid->dev);
+
+		if (!info->ats_enabled)
+			continue;
+
+		qi_flush_dev_iotlb_pasid(info->iommu,
+					 PCI_DEVID(info->bus, info->devfn),
+					 info->pfsid, dev_pasid->pasid,
+					 info->ats_qdep, addr,
+					 mask);
+	}
 	spin_unlock_irqrestore(&domain->lock, flags);
 }
 
@@ -1477,9 +1500,13 @@ static void domain_flush_pasid_iotlb(struct intel_iommu *iommu,
 				     unsigned long npages, bool ih)
 {
 	u16 did = domain_id_iommu(domain, iommu);
+	struct dev_pasid_info *dev_pasid;
 	unsigned long flags;
 
 	spin_lock_irqsave(&domain->lock, flags);
+	list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain)
+		qi_flush_piotlb(iommu, did, dev_pasid->pasid, addr, npages, ih);
+
 	if (!list_empty(&domain->devices))
 		qi_flush_piotlb(iommu, did, IOMMU_NO_PASID, addr, npages, ih);
 	spin_unlock_irqrestore(&domain->lock, flags);
@@ -1744,6 +1771,7 @@ static struct dmar_domain *alloc_domain(unsigned int type)
 		domain->use_first_level = true;
 	domain->has_iotlb_device = false;
 	INIT_LIST_HEAD(&domain->devices);
+	INIT_LIST_HEAD(&domain->dev_pasids);
 	spin_lock_init(&domain->lock);
 	xa_init(&domain->iommu_array);
 
@@ -4731,7 +4759,10 @@ static void intel_iommu_iotlb_sync_map(struct iommu_domain *domain,
 static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
 {
 	struct intel_iommu *iommu = device_to_iommu(dev, NULL, NULL);
+	struct dev_pasid_info *curr, *dev_pasid = NULL;
+	struct dmar_domain *dmar_domain;
 	struct iommu_domain *domain;
+	unsigned long flags;
 
 	domain = iommu_get_domain_for_dev_pasid(dev, pasid, 0);
 	if (!domain)
@@ -4747,17 +4778,78 @@ static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid)
 		goto out_tear_down;
 	}
 
-	/*
-	 * Should never reach here until we add support for attaching
-	 * non-SVA domain to a pasid.
-	 */
-	WARN_ON(1);
+	dmar_domain = to_dmar_domain(domain);
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain) {
+		if (curr->dev == dev && curr->pasid == pasid) {
+			list_del(&curr->link_domain);
+			dev_pasid = curr;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);
 
+	domain_detach_iommu(dmar_domain, iommu);
+	kfree(dev_pasid);
 out_tear_down:
 	intel_pasid_tear_down_entry(iommu, dev, pasid, false);
 	intel_drain_pasid_prq(dev, pasid);
 }
 
+static int intel_iommu_set_dev_pasid(struct iommu_domain *domain,
+				     struct device *dev, ioasid_t pasid)
+{
+	struct device_domain_info *info = dev_iommu_priv_get(dev);
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct intel_iommu *iommu = info->iommu;
+	struct dev_pasid_info *dev_pasid;
+	unsigned long flags;
+	int ret;
+
+	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
+		return -EOPNOTSUPP;
+
+	if (context_copied(iommu, info->bus, info->devfn))
+		return -EBUSY;
+
+	ret = prepare_domain_attach_device(domain, dev);
+	if (ret)
+		return ret;
+
+	dev_pasid = kzalloc(sizeof(*dev_pasid), GFP_KERNEL);
+	if (!dev_pasid)
+		return -ENOMEM;
+
+	ret = domain_attach_iommu(dmar_domain, iommu);
+	if (ret)
+		goto out_free;
+
+	if (domain_type_is_si(dmar_domain))
+		ret = intel_pasid_setup_pass_through(iommu, dmar_domain,
+						     dev, pasid);
+	else if (dmar_domain->use_first_level)
+		ret = domain_setup_first_level(iommu, dmar_domain,
+					       dev, pasid);
+	else
+		ret = intel_pasid_setup_second_level(iommu, dmar_domain,
+						     dev, pasid);
+	if (ret)
+		goto out_detach_iommu;
+
+	dev_pasid->dev = dev;
+	dev_pasid->pasid = pasid;
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_add(&dev_pasid->link_domain, &dmar_domain->dev_pasids);
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);
+
+	return 0;
+out_detach_iommu:
+	domain_detach_iommu(dmar_domain, iommu);
+out_free:
+	kfree(dev_pasid);
+	return ret;
+}
+
 const struct iommu_ops intel_iommu_ops = {
 	.capable		= intel_iommu_capable,
 	.domain_alloc		= intel_iommu_domain_alloc,
@@ -4777,6 +4869,7 @@ const struct iommu_ops intel_iommu_ops = {
 #endif
 	.default_domain_ops = &(const struct iommu_domain_ops) {
 		.attach_dev		= intel_iommu_attach_device,
+		.set_dev_pasid		= intel_iommu_set_dev_pasid,
 		.map_pages		= intel_iommu_map_pages,
 		.unmap_pages		= intel_iommu_unmap_pages,
 		.iotlb_sync_map		= intel_iommu_iotlb_sync_map,
drivers/iommu/intel/iommu.h +7 -0
index 6d94a29f5d52..68bb7cdf5543 100644
@@ -595,6 +595,7 @@ struct dmar_domain {
 
 	spinlock_t lock;		/* Protect device tracking lists */
 	struct list_head devices;	/* all devices' list */
+	struct list_head dev_pasids;	/* all attached pasids */
 
 	struct dma_pte	*pgd;		/* virtual address */
 	int		gaw;		/* max guest address width */
@@ -717,6 +718,12 @@ struct device_domain_info {
 	struct pasid_table *pasid_table; /* pasid table */
 };
 
+struct dev_pasid_info {
+	struct list_head link_domain;	/* link to domain siblings */
+	struct device *dev;		/* the physical device */
+	ioasid_t pasid;			/* PASID of the physical device */
+};
+
 static inline void __iommu_flush_cache(
 	struct intel_iommu *iommu, void *addr, int size)
 {
Signature · 1 line
2.25.1
#21 lore raw

RE: [PATCH v10 6/7] iommu/vt-d: Add set_dev_pasid callback for dma domain

Quoted 14 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +3 -5
-	/*
-	 * Should never reach here until we add support for attaching
-	 * non-SVA domain to a pasid.
-	 */
-	WARN_ON(1);
+	dmar_domain = to_dmar_domain(domain);
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain)

{

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4790

Quoted 7 lines
drivers/iommu/intel/iommu.c +7 -0
+		if (curr->dev == dev && curr->pasid == pasid) {
+			list_del(&curr->link_domain);
+			dev_pasid = curr;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);

what about no matching dev_pasid is find?

Quoted 86 lines
drivers/iommu/intel/iommu.c +57 -0
+	domain_detach_iommu(dmar_domain, iommu);
+	kfree(dev_pasid);
 out_tear_down:
 	intel_pasid_tear_down_entry(iommu, dev, pasid, false);
 	intel_drain_pasid_prq(dev, pasid);
 }
 
+static int intel_iommu_set_dev_pasid(struct iommu_domain *domain,
+				     struct device *dev, ioasid_t pasid)
+{
+	struct device_domain_info *info = dev_iommu_priv_get(dev);
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct intel_iommu *iommu = info->iommu;
+	struct dev_pasid_info *dev_pasid;
+	unsigned long flags;
+	int ret;
+
+	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
+		return -EOPNOTSUPP;
+
+	if (context_copied(iommu, info->bus, info->devfn))
+		return -EBUSY;
+
+	ret = prepare_domain_attach_device(domain, dev);
+	if (ret)
+		return ret;
+
+	dev_pasid = kzalloc(sizeof(*dev_pasid), GFP_KERNEL);
+	if (!dev_pasid)
+		return -ENOMEM;
+
+	ret = domain_attach_iommu(dmar_domain, iommu);
+	if (ret)
+		goto out_free;
+
+	if (domain_type_is_si(dmar_domain))
+		ret = intel_pasid_setup_pass_through(iommu, dmar_domain,
+						     dev, pasid);
+	else if (dmar_domain->use_first_level)
+		ret = domain_setup_first_level(iommu, dmar_domain,
+					       dev, pasid);
+	else
+		ret = intel_pasid_setup_second_level(iommu, dmar_domain,
+						     dev, pasid);
+	if (ret)
+		goto out_detach_iommu;
+
+	dev_pasid->dev = dev;
+	dev_pasid->pasid = pasid;
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_add(&dev_pasid->link_domain, &dmar_domain->dev_pasids);
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);
+
+	return 0;
+out_detach_iommu:
+	domain_detach_iommu(dmar_domain, iommu);
+out_free:
+	kfree(dev_pasid);
+	return ret;
+}
+
 const struct iommu_ops intel_iommu_ops = {
 	.capable		= intel_iommu_capable,
 	.domain_alloc		= intel_iommu_domain_alloc,
@@ -4777,6 +4869,7 @@ const struct iommu_ops intel_iommu_ops = {
 #endif
 	.default_domain_ops = &(const struct iommu_domain_ops) {
 		.attach_dev		= intel_iommu_attach_device,
+		.set_dev_pasid		= intel_iommu_set_dev_pasid,
 		.map_pages		= intel_iommu_map_pages,
 		.unmap_pages		= intel_iommu_unmap_pages,
 		.iotlb_sync_map		=

intel_iommu_iotlb_sync_map,

drivers/iommu/intel/iommu.h +1 -0
index 6d94a29f5d52..68bb7cdf5543 100644
@@ -595,6 +595,7 @@ struct dmar_domain {
 
 	spinlock_t lock;		/* Protect device tracking lists */
 	struct list_head devices;	/* all devices' list */
+	struct list_head dev_pasids;	/* all attached pasids */
 
 	struct dma_pte	*pgd;		/* virtual address */
 	int		gaw;		/* max guest address width */

drivers/iommu/intel/iommu.h struct device_domain_info { · line 725

Quoted 8 lines
drivers/iommu/intel/iommu.h +5 -0
@@ -717,6 +718,12 @@ struct device_domain_info {
 	struct pasid_table *pasid_table; /* pasid table */
 };
 
+struct dev_pasid_info {
+	struct list_head link_domain;	/* link to domain siblings */
+	struct device *dev;		/* the physical device */
+	ioasid_t pasid;			/* PASID of the physical device */
+};

the comment for dev/pasid is meaningless.

maintainer
#22 lore raw

Re: [PATCH v10 6/7] iommu/vt-d: Add set_dev_pasid callback for dma domain

Quoted 15 lines · Kevin
Quoted 15 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +3 -5
-	/*
-	 * Should never reach here until we add support for attaching
-	 * non-SVA domain to a pasid.
-	 */
-	WARN_ON(1);
+	dmar_domain = to_dmar_domain(domain);
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain)

{

drivers/iommu/intel/iommu.c +2 -0
+		if (curr->dev == dev && curr->pasid == pasid) {
+			list_del(&curr->link_domain);
Quoted 8 lines

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4790

Quoted 6 lines
drivers/iommu/intel/iommu.c +5 -0
+			dev_pasid = curr;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);

what about no matching dev_pasid is find?

kfree() can handle this gracefully.

Quoted 90 lines
Quoted 90 lines
drivers/iommu/intel/iommu.c +57 -0
+	domain_detach_iommu(dmar_domain, iommu);
+	kfree(dev_pasid);
  out_tear_down:
  	intel_pasid_tear_down_entry(iommu, dev, pasid, false);
  	intel_drain_pasid_prq(dev, pasid);
  }
 
+static int intel_iommu_set_dev_pasid(struct iommu_domain *domain,
+				     struct device *dev, ioasid_t pasid)
+{
+	struct device_domain_info *info = dev_iommu_priv_get(dev);
+	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct intel_iommu *iommu = info->iommu;
+	struct dev_pasid_info *dev_pasid;
+	unsigned long flags;
+	int ret;
+
+	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
+		return -EOPNOTSUPP;
+
+	if (context_copied(iommu, info->bus, info->devfn))
+		return -EBUSY;
+
+	ret = prepare_domain_attach_device(domain, dev);
+	if (ret)
+		return ret;
+
+	dev_pasid = kzalloc(sizeof(*dev_pasid), GFP_KERNEL);
+	if (!dev_pasid)
+		return -ENOMEM;
+
+	ret = domain_attach_iommu(dmar_domain, iommu);
+	if (ret)
+		goto out_free;
+
+	if (domain_type_is_si(dmar_domain))
+		ret = intel_pasid_setup_pass_through(iommu, dmar_domain,
+						     dev, pasid);
+	else if (dmar_domain->use_first_level)
+		ret = domain_setup_first_level(iommu, dmar_domain,
+					       dev, pasid);
+	else
+		ret = intel_pasid_setup_second_level(iommu, dmar_domain,
+						     dev, pasid);
+	if (ret)
+		goto out_detach_iommu;
+
+	dev_pasid->dev = dev;
+	dev_pasid->pasid = pasid;
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_add(&dev_pasid->link_domain, &dmar_domain->dev_pasids);
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);
+
+	return 0;
+out_detach_iommu:
+	domain_detach_iommu(dmar_domain, iommu);
+out_free:
+	kfree(dev_pasid);
+	return ret;
+}
+
  const struct iommu_ops intel_iommu_ops = {
  	.capable		= intel_iommu_capable,
  	.domain_alloc		= intel_iommu_domain_alloc,
@@ -4777,6 +4869,7 @@ const struct iommu_ops intel_iommu_ops = {
  #endif
  	.default_domain_ops = &(const struct iommu_domain_ops) {
  		.attach_dev		= intel_iommu_attach_device,
+		.set_dev_pasid		= intel_iommu_set_dev_pasid,
  		.map_pages		= intel_iommu_map_pages,
  		.unmap_pages		= intel_iommu_unmap_pages,
  		.iotlb_sync_map		=

intel_iommu_iotlb_sync_map,

drivers/iommu/intel/iommu.h +1 -0
index 6d94a29f5d52..68bb7cdf5543 100644
@@ -595,6 +595,7 @@ struct dmar_domain {
 
  	spinlock_t lock;		/* Protect device tracking lists */
  	struct list_head devices;	/* all devices' list */
+	struct list_head dev_pasids;	/* all attached pasids */
 
  	struct dma_pte	*pgd;		/* virtual address */
  	int		gaw;		/* max guest address width */
@@ -717,6 +718,12 @@ struct device_domain_info {
  	struct pasid_table *pasid_table; /* pasid table */
  };
 
Quoted 7 lines

drivers/iommu/intel/iommu.h struct device_domain_info { · line 725

Quoted 5 lines
drivers/iommu/intel/iommu.h +5 -0
+struct dev_pasid_info {
+	struct list_head link_domain;	/* link to domain siblings */
+	struct device *dev;		/* the physical device */
+	ioasid_t pasid;			/* PASID of the physical device */
+};

the comment for dev/pasid is meaningless.

Then we can remove them.

Best regards,
baolu

#23 lore raw

RE: [PATCH v10 6/7] iommu/vt-d: Add set_dev_pasid callback for dma domain

Quoted 28 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:34 AM

Quoted 15 lines · Kevin
Quoted 15 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +3 -5
-	/*
-	 * Should never reach here until we add support for attaching
-	 * non-SVA domain to a pasid.
-	 */
-	WARN_ON(1);
+	dmar_domain = to_dmar_domain(domain);
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain)

{

drivers/iommu/intel/iommu.c +2 -0
+		if (curr->dev == dev && curr->pasid == pasid) {
+			list_del(&curr->link_domain);
Quoted 8 lines

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4790

Quoted 6 lines
drivers/iommu/intel/iommu.c +5 -0
+			dev_pasid = curr;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);

what about no matching dev_pasid is find?

Quoted 1 line

kfree() can handle this gracefully.

but what about domain_detach_iommu()? Is it correct to adjust the refcnting when there is no matching dev_pasid?

maintainer
#24 lore raw

Re: [PATCH v10 6/7] iommu/vt-d: Add set_dev_pasid callback for dma domain

Quoted 28 lines · Kevin
Quoted 28 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:34 AM

Quoted 23 lines · Kevin
Quoted 14 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +3 -5
-	/*
-	 * Should never reach here until we add support for attaching
-	 * non-SVA domain to a pasid.
-	 */
-	WARN_ON(1);
+	dmar_domain = to_dmar_domain(domain);
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain)

{

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4790

Quoted 7 lines
drivers/iommu/intel/iommu.c +7 -0
+		if (curr->dev == dev && curr->pasid == pasid) {
+			list_del(&curr->link_domain);
+			dev_pasid = curr;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);

what about no matching dev_pasid is find?

Quoted 4 lines
Quoted 1 line

kfree() can handle this gracefully.

but what about domain_detach_iommu()? Is it correct to adjust the refcnting when there is no matching dev_pasid?

You are right.

Logically, if we get a valid domain for a pasid, we should have a dev_pasid allocated for it. Perhaps, adding a check in the code will make the code more readable?

Best regards,
baolu

#25 lore raw

RE: [PATCH v10 6/7] iommu/vt-d: Add set_dev_pasid callback for dma domain

Quoted 34 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 2:00 PM

Quoted 30 lines · Kevin
Quoted 29 lines

From: Baolu Lu <[email protected]>
Sent: Friday, July 14, 2023 11:34 AM

Quoted 15 lines · Kevin
Quoted 15 lines

From: Jacob Pan <[email protected]>
Sent: Thursday, July 13, 2023 12:34 AM

drivers/iommu/intel/iommu.c +3 -5
-	/*
-	 * Should never reach here until we add support for attaching
-	 * non-SVA domain to a pasid.
-	 */
-	WARN_ON(1);
+	dmar_domain = to_dmar_domain(domain);
+	spin_lock_irqsave(&dmar_domain->lock, flags);
+	list_for_each_entry(curr, &dmar_domain->dev_pasids, link_domain)

{

drivers/iommu/intel/iommu.c +2 -0
+		if (curr->dev == dev && curr->pasid == pasid) {
+			list_del(&curr->link_domain);
Quoted 8 lines

drivers/iommu/intel/iommu.c static void intel_iommu_remove_dev_pasid(struct device *dev, ioasid_t pasid) · line 4790

Quoted 6 lines
drivers/iommu/intel/iommu.c +5 -0
+			dev_pasid = curr;
+			break;
+		}
+	}
+	spin_unlock_irqrestore(&dmar_domain->lock, flags);

what about no matching dev_pasid is find?

kfree() can handle this gracefully.

Quoted 9 lines
Quoted 2 lines

but what about domain_detach_iommu()? Is it correct to adjust the refcnting when there is no matching dev_pasid?

You are right.

Logically, if we get a valid domain for a pasid, we should have a dev_pasid allocated for it. Perhaps, adding a check in the code will make the code more readable?

yes

author
#26 lore raw

PATCH v10 7/7 dmaengine/idxd: Re-enable kernel workqueue under DMA API

In reply to Jacob Pan

Kernel workqueues were disabled due to flawed use of kernel VA and SVA API. Now that we have the support for attaching PASID to the device's default domain and the ability to reserve global PASIDs from SVA APIs, we can re-enable the kernel work queues and use them under DMA API.

We also use non-privileged access for in-kernel DMA to be consistent with the IOMMU settings. Consequently, interrupt for user privilege is enabled for work completion IRQs.

---
v9: Set user IRQ enable when device is enabled for system PASID

---
 drivers/dma/idxd/device.c | 39 ++++++++++------------------
 drivers/dma/idxd/dma.c    |  5 ++--
 drivers/dma/idxd/idxd.h   |  9 +++++++
 drivers/dma/idxd/init.c   | 54 ++++++++++++++++++++++++++++++++++++---
 drivers/dma/idxd/sysfs.c  |  7 -----
 5 files changed, 76 insertions(+), 38 deletions(-)
drivers/dma/idxd/device.c +14 -25
index 5abbcc61c528..169b7ade8919 100644
@@ -299,21 +299,6 @@ void idxd_wqs_unmap_portal(struct idxd_device *idxd)
 	}
 }
 
-static void __idxd_wq_set_priv_locked(struct idxd_wq *wq, int priv)
-{
-	struct idxd_device *idxd = wq->idxd;
-	union wqcfg wqcfg;
-	unsigned int offset;
-
-	offset = WQCFG_OFFSET(idxd, wq->id, WQCFG_PRIVL_IDX);
-	spin_lock(&idxd->dev_lock);
-	wqcfg.bits[WQCFG_PRIVL_IDX] = ioread32(idxd->reg_base + offset);
-	wqcfg.priv = priv;
-	wq->wqcfg->bits[WQCFG_PRIVL_IDX] = wqcfg.bits[WQCFG_PRIVL_IDX];
-	iowrite32(wqcfg.bits[WQCFG_PRIVL_IDX], idxd->reg_base + offset);
-	spin_unlock(&idxd->dev_lock);
-}
-
 static void __idxd_wq_set_pasid_locked(struct idxd_wq *wq, int pasid)
 {
 	struct idxd_device *idxd = wq->idxd;
@@ -1423,15 +1408,14 @@ int drv_enable_wq(struct idxd_wq *wq)
 	}
 
 	/*
-	 * In the event that the WQ is configurable for pasid and priv bits.
-	 * For kernel wq, the driver should setup the pasid, pasid_en, and priv bit.
-	 * However, for non-kernel wq, the driver should only set the pasid_en bit for
-	 * shared wq. A dedicated wq that is not 'kernel' type will configure pasid and
+	 * In the event that the WQ is configurable for pasid, the driver
+	 * should setup the pasid, pasid_en bit. This is true for both kernel
+	 * and user shared workqueues. There is no need to setup priv bit in
+	 * that in-kernel DMA will also do user privileged requests.
+	 * A dedicated wq that is not 'kernel' type will configure pasid and
 	 * pasid_en later on so there is no need to setup.
 	 */
 	if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) {
-		int priv = 0;
-
 		if (wq_pasid_enabled(wq)) {
 			if (is_idxd_wq_kernel(wq) || wq_shared(wq)) {
 				u32 pasid = wq_dedicated(wq) ? idxd->pasid : 0;
@@ -1439,10 +1423,6 @@ int drv_enable_wq(struct idxd_wq *wq)
 				__idxd_wq_set_pasid_locked(wq, pasid);
 			}
 		}
-
-		if (is_idxd_wq_kernel(wq))
-			priv = 1;
-		__idxd_wq_set_priv_locked(wq, priv);
 	}
 
 	rc = 0;
@@ -1550,6 +1530,15 @@ int idxd_device_drv_probe(struct idxd_dev *idxd_dev)
 	if (rc < 0)
 		return -ENXIO;
 
+	/*
+	 * System PASID is preserved across device disable/enable cycle, but
+	 * genconfig register content gets cleared during device reset. We
+	 * need to re-enable user interrupts for kernel work queue completion
+	 * IRQ to function.
+	 */
+	if (idxd->pasid != IOMMU_PASID_INVALID)
+		idxd_set_user_intr(idxd, 1);
+
 	rc = idxd_device_evl_setup(idxd);
 	if (rc < 0) {
 		idxd->cmd_status = IDXD_SCMD_DEV_EVL_ERR;
drivers/dma/idxd/dma.c +3 -2
index eb35ca313684..07623fb0f52f 100644
@@ -75,9 +75,10 @@ static inline void idxd_prep_desc_common(struct idxd_wq *wq,
 	hw->xfer_size = len;
 	/*
 	 * For dedicated WQ, this field is ignored and HW will use the WQCFG.priv
-	 * field instead. This field should be set to 1 for kernel descriptors.
+	 * field instead. This field should be set to 0 for kernel descriptors
+	 * since kernel DMA on VT-d supports "user" privilege only.
 	 */
-	hw->priv = 1;
+	hw->priv = 0;
 	hw->completion_addr = compl;
 }
 
drivers/dma/idxd/idxd.h +9 -0
index 5428a2e1b1ec..502be9db63f4 100644
@@ -473,6 +473,15 @@ static inline struct idxd_device *ie_to_idxd(struct idxd_irq_entry *ie)
 	return container_of(ie, struct idxd_device, ie);
 }
 
+static inline void idxd_set_user_intr(struct idxd_device *idxd, bool enable)
+{
+	union gencfg_reg reg;
+
+	reg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET);
+	reg.user_int_en = enable;
+	iowrite32(reg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET);
+}
+
 extern struct bus_type dsa_bus_type;
 
 extern bool support_enqcmd;
drivers/dma/idxd/init.c +50 -4

72 lines not on this page · raw message

drivers/dma/idxd/sysfs.c +0 -7
index 293739ac5596..63f6966c51aa 100644
@@ -948,13 +948,6 @@ static ssize_t wq_name_store(struct device *dev,
 	if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0)
 		return -EINVAL;
 
-	/*
-	 * This is temporarily placed here until we have SVM support for
-	 * dmaengine.
-	 */
-	if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd))
-		return -EOPNOTSUPP;
-
 	input = kstrndup(buf, count, GFP_KERNEL);
 	if (!input)
 		return -ENOMEM;
Signature · 1 line
2.25.1