Skip to contentSkip to conversation

iommupt/vtd: Support dirty tracking on PASID

PATCH Applied Zhenzhong Duan 4 participants 12 messages Mar 30 – Apr 1

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

v2 cover iommupt/vtd: Support dirty tracking on PASID

Hi,

When we add pasid support in QEMU for passthrough device, we found PASID attachment to a nested parent domain with dirty tracking failed.

It's because PASID-level dirty tracking is not there yet, by adding it, we can enable PASID attachment to such domain.

Thanks
Zhenzhong

Changelog:
v2:
- a new patch1 to add a check in intel_nested_set_dev_pasid() (Kevin)
- s/device_set_dirty_tracking/domain_set_dirty_tracking (Liuyi)
- add R-b

Zhenzhong Duan (4):
iommu/vt-d: Block PASID attachment to nested domain with dirty
tracking
iommupt/vtd: Rename device_set_dirty_tracking() and pass dmar_domain
pointer
iommupt/vtd: Support dirty tracking on PASID iommufd/selftest: Test dirty tracking on PASID

 drivers/iommu/intel/iommu.c             | 33 ++++++++++++++-----------
 tools/testing/selftests/iommu/iommufd.c | 27 ++++++++++++++++++++
 2 files changed, 45 insertions(+), 15 deletions(-)
Signature · 1 line
2.47.3
#1 lore raw

PATCH v2 1/4 iommu/vt-d: Block PASID attachment to nested domain with dirty tracking

Kernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.

Fixes f35f22cc760e ("iommu/vt-d: Access/Dirty bit support for SS domains")
---
 drivers/iommu/intel/nested.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/iommu/intel/nested.c +5 -1
index 2b979bec56ce..16c82ba47d30 100644
@@ -148,6 +148,7 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,
 {
 	struct device_domain_info *info = dev_iommu_priv_get(dev);
 	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct iommu_domain *s2_domain = &dmar_domain->s2_domain->domain;
 	struct intel_iommu *iommu = info->iommu;
 	struct dev_pasid_info *dev_pasid;
 	int ret;
@@ -155,10 +156,13 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,
 	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
 		return -EOPNOTSUPP;
 
+	if (s2_domain->dirty_ops)
+		return -EINVAL;
+
 	if (context_copied(iommu, info->bus, info->devfn))
 		return -EBUSY;
 
-	ret = paging_domain_compatible(&dmar_domain->s2_domain->domain, dev);
+	ret = paging_domain_compatible(s2_domain, dev);
 	if (ret)
 		return ret;
 
Signature · 1 line
2.47.3
maintainer Reviewed-by on v2 1/4
#2 lore raw

RE: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty tracking

Quoted 11 lines

From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PM

Kernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.

Fixes f35f22cc760e ("iommu/vt-d: Access/Dirty bit support for SS domains")
Reviewed-by on v2 1/4
#3 lore raw

Re: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty tracking

In reply to Zhenzhong Duan
Quoted 11 lines · Zhenzhong Duan

Kernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.

Fixes f35f22cc760e ("iommu/vt-d: Access/Dirty bit support for SS domains")
---
  drivers/iommu/intel/nested.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)

Good catch. Just one nit. I think the below fix tag is more accurate. SS dirty was merged before PASID attachment. So this fix should be backported since the first PASID nested domain attachment.

Fixes 67f6f56b5912 ("iommu/vt-d: Add set_dev_pasid callback for nested domain")
Quoted 27 lines
drivers/iommu/intel/nested.c +5 -1
index 2b979bec56ce..16c82ba47d30 100644
@@ -148,6 +148,7 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,
  {
  	struct device_domain_info *info = dev_iommu_priv_get(dev);
  	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct iommu_domain *s2_domain = &dmar_domain->s2_domain->domain;
  	struct intel_iommu *iommu = info->iommu;
  	struct dev_pasid_info *dev_pasid;
  	int ret;
@@ -155,10 +156,13 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,
  	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
  		return -EOPNOTSUPP;
 
+	if (s2_domain->dirty_ops)
+		return -EINVAL;
+
  	if (context_copied(iommu, info->bus, info->devfn))
  		return -EBUSY;
 
-	ret = paging_domain_compatible(&dmar_domain->s2_domain->domain, dev);
+	ret = paging_domain_compatible(s2_domain, dev);
  	if (ret)
  		return ret;
 
#4 lore raw

RE: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty tracking

Quoted 20 lines

-----Original Message-----
From: Liu, Yi L <[email protected]>
Subject: Re: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty tracking

Quoted 11 lines · Zhenzhong Duan

Kernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.

Fixes f35f22cc760e ("iommu/vt-d: Access/Dirty bit support for SS domains")
---
  drivers/iommu/intel/nested.c | 6 +++++-
  1 file changed, 5 insertions(+), 1 deletion(-)
Quoted 3 lines

Good catch. Just one nit. I think the below fix tag is more accurate. SS dirty was merged before PASID attachment. So this fix should be backported since the first PASID nested domain attachment.

Oh, I see, thanks for sharing the history.

Quoted 3 lines
Fixes 67f6f56b5912 ("iommu/vt-d: Add set_dev_pasid callback for nested

domain")

I'll leave it to Baolu to decide if he want a respin or will pick this directly.

BRs,
Zhenzhong

Quoted 34 lines

drivers/iommu/intel/nested.c static int intel_nested_set_dev_pasid(struct

Quoted 5 lines
drivers/iommu/intel/nested.c +0 -0
index 2b979bec56ce..16c82ba47d30 100644
@@ -148,6 +148,7 @@ static int intel_nested_set_dev_pasid(struct

iommu_domain *domain,

drivers/iommu/intel/nested.c static int intel_nested_set_dev_pasid(struct

Quoted 9 lines
drivers/iommu/intel/nested.c +1 -0
  {
  	struct device_domain_info *info = dev_iommu_priv_get(dev);
  	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
+	struct iommu_domain *s2_domain = &dmar_domain->s2_domain-
 domain;
  	struct intel_iommu *iommu = info->iommu;
  	struct dev_pasid_info *dev_pasid;

int ret;

drivers/iommu/intel/nested.c +0 -0
@@ -155,10 +156,13 @@ static int intel_nested_set_dev_pasid(struct

iommu_domain *domain,

drivers/iommu/intel/nested.c static int intel_nested_set_dev_pasid(struct iommu_domain *domain,

Quoted 10 lines
drivers/iommu/intel/nested.c +3 -1
  	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
  		return -EOPNOTSUPP;
 
+	if (s2_domain->dirty_ops)
+		return -EINVAL;
+
  	if (context_copied(iommu, info->bus, info->devfn))
  		return -EBUSY;
 
-	ret = paging_domain_compatible(&dmar_domain->s2_domain->domain,

dev);

Quoted 4 lines
drivers/iommu/intel/nested.c +1 -0
+	ret = paging_domain_compatible(s2_domain, dev);
  	if (ret)
  		return ret;
 
#5 lore raw

PATCH v2 2/4 iommupt/vtd: Rename device_set_dirty_tracking() and pass dmar_domain pointer

In reply to Zhenzhong Duan

device_set_dirty_tracking() sets dirty tracking on all devices attached to a domain, also on all PASIDs attached to same domain in subsequent patch.

So rename it as domain_set_dirty_tracking() and pass dmar_domain pointer to better align to what it does.

No functional changes intended.

---
 drivers/iommu/intel/iommu.c | 21 +++++++++------------
 1 file changed, 9 insertions(+), 12 deletions(-)
drivers/iommu/intel/iommu.c +9 -12
index ef7613b177b9..965e0330ec4b 100644
@@ -3684,16 +3684,15 @@ static void *intel_iommu_hw_info(struct device *dev, u32 *length,
 	return vtd;
 }
 
-/*
- * Set dirty tracking for the device list of a domain. The caller must
- * hold the domain->lock when calling it.
- */
-static int device_set_dirty_tracking(struct list_head *devices, bool enable)
+/* Set dirty tracking for the devices that the domain has been attached. */
+static int domain_set_dirty_tracking(struct dmar_domain *domain, bool enable)
 {
 	struct device_domain_info *info;
 	int ret = 0;
 
-	list_for_each_entry(info, devices, link) {
+	lockdep_assert_held(&domain->lock);
+
+	list_for_each_entry(info, &domain->devices, link) {
 		ret = intel_pasid_setup_dirty_tracking(info->iommu, info->dev,
 						       IOMMU_NO_PASID, enable);
 		if (ret)
@@ -3713,7 +3712,7 @@ static int parent_domain_set_dirty_tracking(struct dmar_domain *domain,
 	spin_lock(&domain->s1_lock);
 	list_for_each_entry(s1_domain, &domain->s1_domains, s2_link) {
 		spin_lock_irqsave(&s1_domain->lock, flags);
-		ret = device_set_dirty_tracking(&s1_domain->devices, enable);
+		ret = domain_set_dirty_tracking(s1_domain, enable);
 		spin_unlock_irqrestore(&s1_domain->lock, flags);
 		if (ret)
 			goto err_unwind;
@@ -3724,8 +3723,7 @@ static int parent_domain_set_dirty_tracking(struct dmar_domain *domain,
 err_unwind:
 	list_for_each_entry(s1_domain, &domain->s1_domains, s2_link) {
 		spin_lock_irqsave(&s1_domain->lock, flags);
-		device_set_dirty_tracking(&s1_domain->devices,
-					  domain->dirty_tracking);
+		domain_set_dirty_tracking(s1_domain, domain->dirty_tracking);
 		spin_unlock_irqrestore(&s1_domain->lock, flags);
 	}
 	spin_unlock(&domain->s1_lock);
@@ -3742,7 +3740,7 @@ static int intel_iommu_set_dirty_tracking(struct iommu_domain *domain,
 	if (dmar_domain->dirty_tracking == enable)
 		goto out_unlock;
 
-	ret = device_set_dirty_tracking(&dmar_domain->devices, enable);
+	ret = domain_set_dirty_tracking(dmar_domain, enable);
 	if (ret)
 		goto err_unwind;
 
@@ -3759,8 +3757,7 @@ static int intel_iommu_set_dirty_tracking(struct iommu_domain *domain,
 	return 0;
 
 err_unwind:
-	device_set_dirty_tracking(&dmar_domain->devices,
-				  dmar_domain->dirty_tracking);
+	domain_set_dirty_tracking(dmar_domain, dmar_domain->dirty_tracking);
 	spin_unlock(&dmar_domain->lock);
 	return ret;
 }
Signature · 1 line
2.47.3
maintainer Reviewed-by on v2 2/4
#6 lore raw

RE: [PATCH v2 2/4] iommupt/vtd: Rename device_set_dirty_tracking() and pass dmar_domain pointer

Quoted 6 lines

From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PM

device_set_dirty_tracking() sets dirty tracking on all devices attached to a domain, also on all PASIDs attached to same domain in subsequent patch.

Quoted 8 lines

So rename it as domain_set_dirty_tracking() and pass dmar_domain pointer to better align to what it does.

No functional changes intended.

#7 lore raw

PATCH v2 3/4 iommupt/vtd: Support dirty tracking on PASID

In reply to Zhenzhong Duan

In order to support passthrough device with PASID capability in QEMU, e.g., DSA device, kernel needs to support attaching PASID to a domain.

But attaching is not allowed if the domain is a second stage domain or nested domain with dirty tracking.

The reason is kernel lacking support for dirty tracking on such domain attached to PASID. By adding dirty tracking on PASID, the check can be removed.

---
 drivers/iommu/intel/iommu.c  | 12 +++++++++---
 drivers/iommu/intel/nested.c |  6 +-----
 2 files changed, 10 insertions(+), 8 deletions(-)
drivers/iommu/intel/iommu.c +9 -3
index 965e0330ec4b..26135ff3a289 100644
@@ -3618,9 +3618,6 @@ static int intel_iommu_set_dev_pasid(struct iommu_domain *domain,
 	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
 		return -EOPNOTSUPP;
 
-	if (domain->dirty_ops)
-		return -EINVAL;
-
 	if (context_copied(iommu, info->bus, info->devfn))
 		return -EBUSY;
 
@@ -3688,6 +3685,7 @@ static void *intel_iommu_hw_info(struct device *dev, u32 *length,
 static int domain_set_dirty_tracking(struct dmar_domain *domain, bool enable)
 {
 	struct device_domain_info *info;
+	struct dev_pasid_info *dev_pasid;
 	int ret = 0;
 
 	lockdep_assert_held(&domain->lock);
@@ -3695,6 +3693,14 @@ static int domain_set_dirty_tracking(struct dmar_domain *domain, bool enable)
 	list_for_each_entry(info, &domain->devices, link) {
 		ret = intel_pasid_setup_dirty_tracking(info->iommu, info->dev,
 						       IOMMU_NO_PASID, enable);
+		if (ret)
+			return ret;
+	}
+
+	list_for_each_entry(dev_pasid, &domain->dev_pasids, link_domain) {
+		info = dev_iommu_priv_get(dev_pasid->dev);
+		ret = intel_pasid_setup_dirty_tracking(info->iommu, info->dev,
+						       dev_pasid->pasid, enable);
 		if (ret)
 			break;
 	}
drivers/iommu/intel/nested.c +1 -5
index 16c82ba47d30..2b979bec56ce 100644
@@ -148,7 +148,6 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,
 {
 	struct device_domain_info *info = dev_iommu_priv_get(dev);
 	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
-	struct iommu_domain *s2_domain = &dmar_domain->s2_domain->domain;
 	struct intel_iommu *iommu = info->iommu;
 	struct dev_pasid_info *dev_pasid;
 	int ret;
@@ -156,13 +155,10 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,
 	if (!pasid_supported(iommu) || dev_is_real_dma_subdevice(dev))
 		return -EOPNOTSUPP;
 
-	if (s2_domain->dirty_ops)
-		return -EINVAL;
-
 	if (context_copied(iommu, info->bus, info->devfn))
 		return -EBUSY;
 
-	ret = paging_domain_compatible(s2_domain, dev);
+	ret = paging_domain_compatible(&dmar_domain->s2_domain->domain, dev);
 	if (ret)
 		return ret;
 
Signature · 1 line
2.47.3
maintainer Reviewed-by on v2 3/4
#8 lore raw
Quoted 9 lines

From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PM

In order to support passthrough device with PASID capability in QEMU, e.g., DSA device, kernel needs to support attaching PASID to a domain.

But attaching is not allowed if the domain is a second stage domain or nested domain with dirty tracking.

Quoted 6 lines

The reason is kernel lacking support for dirty tracking on such domain attached to PASID. By adding dirty tracking on PASID, the check can be removed.

#9 lore raw

PATCH v2 4/4 iommufd/selftest: Test dirty tracking on PASID

In reply to Zhenzhong Duan

Add test case for dirty tracking on a domain attached to PASID, also confirm attachment to PASID fail if device doesn't support dirty tracking.

---
 tools/testing/selftests/iommu/iommufd.c | 27 +++++++++++++++++++++++++
 1 file changed, 27 insertions(+)
tools/testing/selftests/iommu/iommufd.c +27 -0
index dadad277f4eb..d1fe5dbc2813 100644
@@ -2275,6 +2275,33 @@ TEST_F(iommufd_dirty_tracking, set_dirty_tracking)
 	test_ioctl_destroy(hwpt_id);
 }
 
+TEST_F(iommufd_dirty_tracking, pasid_set_dirty_tracking)
+{
+	uint32_t stddev_id, ioas_id, hwpt_id, pasid = 100;
+	uint32_t dev_flags = MOCK_FLAGS_DEVICE_PASID;
+
+	/* Regular case */
+	test_cmd_hwpt_alloc(self->idev_id, self->ioas_id,
+			    IOMMU_HWPT_ALLOC_PASID | IOMMU_HWPT_ALLOC_DIRTY_TRACKING,
+			    &hwpt_id);
+	test_cmd_mock_domain_flags(hwpt_id, dev_flags, &stddev_id, NULL, NULL);
+	ASSERT_EQ(0, _test_cmd_pasid_attach(self->fd, stddev_id, pasid, hwpt_id));
+	test_cmd_set_dirty_tracking(hwpt_id, true);
+	test_cmd_set_dirty_tracking(hwpt_id, false);
+	ASSERT_EQ(0, _test_cmd_pasid_detach(self->fd, stddev_id, pasid));
+
+	test_ioctl_destroy(stddev_id);
+
+	/* IOMMU device does not support dirty tracking */
+	dev_flags |= MOCK_FLAGS_DEVICE_NO_DIRTY;
+	test_ioctl_ioas_alloc(&ioas_id);
+	test_cmd_mock_domain_flags(ioas_id, dev_flags, &stddev_id, NULL, NULL);
+	EXPECT_ERRNO(EINVAL, _test_cmd_pasid_attach(self->fd, stddev_id, pasid, hwpt_id));
+
+	test_ioctl_destroy(stddev_id);
+	test_ioctl_destroy(hwpt_id);
+}
+
 TEST_F(iommufd_dirty_tracking, device_dirty_capability)
 {
 	uint32_t caps = 0;
Signature · 1 line
2.47.3
maintainer Reviewed-by on v2 4/4
#10 lore raw

RE: [PATCH v2 4/4] iommufd/selftest: Test dirty tracking on PASID

Quoted 9 lines

From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PM

Add test case for dirty tracking on a domain attached to PASID, also confirm attachment to PASID fail if device doesn't support dirty tracking.

maintainer Applied
#11 lore raw
In reply to Zhenzhong Duan
Quoted 17 lines · Zhenzhong Duan

Hi,

When we add pasid support in QEMU for passthrough device, we found PASID attachment to a nested parent domain with dirty tracking failed.

It's because PASID-level dirty tracking is not there yet, by adding it, we can enable PASID attachment to such domain.

Thanks
Zhenzhong

Changelog:
v2:
- a new patch1 to add a check in intel_nested_set_dev_pasid() (Kevin)
- s/device_set_dirty_tracking/domain_set_dirty_tracking (Liuyi)
- add R-b

Quoted 7 lines

Zhenzhong Duan (4):
iommu/vt-d: Block PASID attachment to nested domain with dirty
tracking
iommupt/vtd: Rename device_set_dirty_tracking() and pass dmar_domain
pointer
iommupt/vtd: Support dirty tracking on PASID iommufd/selftest: Test dirty tracking on PASID

Queued for iommu next with "iommupt/vtd" changed to "iommu/vt-d" in the titles.

Thanks,
baolu