iommupt/vtd: Support dirty tracking on PASID
Subsystems (5)
v2 cover iommupt/vtd: Support dirty tracking on PASID
Hi,
When we add pasid support in QEMU for passthrough device, we found PASID attachment to a nested parent domain with dirty tracking failed.
It's because PASID-level dirty tracking is not there yet, by adding it, we can enable PASID attachment to such domain.
Thanks
Zhenzhong
Changelog:
v2:
- a new patch1 to add a check in intel_nested_set_dev_pasid() (Kevin)
- s/device_set_dirty_tracking/domain_set_dirty_tracking (Liuyi)
- add R-b
Zhenzhong Duan (4):
iommu/vt-d: Block PASID attachment to nested domain with dirty
tracking
iommupt/vtd: Rename device_set_dirty_tracking() and pass dmar_domain
pointer
iommupt/vtd: Support dirty tracking on PASID iommufd/selftest: Test dirty tracking on PASID
drivers/iommu/intel/iommu.c | 33 ++++++++++++++----------- tools/testing/selftests/iommu/iommufd.c | 27 ++++++++++++++++++++ 2 files changed, 45 insertions(+), 15 deletions(-)
Signature · 1 line
2.47.3
PATCH v2 1/4 iommu/vt-d: Block PASID attachment to nested domain with dirty tracking
Kernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.
--- drivers/iommu/intel/nested.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
drivers/iommu/intel/nested.c +5 -1
@@ -148,6 +148,7 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain, {struct device_domain_info *info =dev_iommu_priv_get (dev);struct dmar_domain *dmar_domain =to_dmar_domain (domain); +struct iommu_domain *s2_domain = &dmar_domain->s2_domain->domain;struct intel_iommu *iommu = info->iommu;struct dev_pasid_info *dev_pasid;int ret;
@@ -155,10 +156,13 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,if (!pasid_supported (iommu) ||dev_is_real_dma_subdevice (dev))return -EOPNOTSUPP ; +if (s2_domain->dirty_ops) +return -EINVAL ; +if (context_copied (iommu, info->bus, info->devfn))return -EBUSY ; - ret =paging_domain_compatible (&dmar_domain->s2_domain->domain, dev); + ret =paging_domain_compatible (s2_domain, dev);if (ret)return ret;
Signature · 1 line
2.47.3
RE: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty tracking
Quoted 11 lines
From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PMKernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.
Fixes f35f22cc760e ("iommu/vt-d: Access/Dirty bit support for SS domains")Suggested-by Kevin Tian <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>
Re: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty tracking
In reply to Zhenzhong DuanQuoted 11 lines · Zhenzhong Duan
Kernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.
Fixes f35f22cc760e ("iommu/vt-d: Access/Dirty bit support for SS domains")Suggested-by Kevin Tian <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>--- drivers/iommu/intel/nested.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Good catch. Just one nit. I think the below fix tag is more accurate. SS dirty was merged before PASID attachment. So this fix should be backported since the first PASID nested domain attachment.
Quoted 27 lines
drivers/iommu/intel/nested.c +5
-1@@ -148,6 +148,7 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain, {struct device_domain_info *info =dev_iommu_priv_get (dev);struct dmar_domain *dmar_domain =to_dmar_domain (domain); +struct iommu_domain *s2_domain = &dmar_domain->s2_domain->domain;struct intel_iommu *iommu = info->iommu;struct dev_pasid_info *dev_pasid;int ret;@@ -155,10 +156,13 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,if (!pasid_supported (iommu) ||dev_is_real_dma_subdevice (dev))return -EOPNOTSUPP ; +if (s2_domain->dirty_ops) +return -EINVAL ; +if (context_copied (iommu, info->bus, info->devfn))return -EBUSY ; - ret =paging_domain_compatible (&dmar_domain->s2_domain->domain, dev); + ret =paging_domain_compatible (s2_domain, dev);if (ret)return ret;
RE: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty tracking
Quoted 20 lines
-----Original Message-----
From: Liu, Yi L <[email protected]>
Subject: Re: [PATCH v2 1/4] iommu/vt-d: Block PASID attachment to nested domain with dirty trackingQuoted 11 lines · Zhenzhong Duan
Kernel lacks dirty tracking support on nested domain attached to PASID, fails the attachment early if nesting parent domain is dirty tracking configured, otherwise dirty pages would be lost.
Fixes f35f22cc760e ("iommu/vt-d: Access/Dirty bit support for SS domains")Suggested-by Kevin Tian <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>--- drivers/iommu/intel/nested.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Quoted 3 lines
Good catch. Just one nit. I think the below fix tag is more accurate. SS dirty was merged before PASID attachment. So this fix should be backported since the first PASID nested domain attachment.
Oh, I see, thanks for sharing the history.
Quoted 3 lines
Fixes 67f6f56b5912 ("iommu/vt-d: Add set_dev_pasid callback for nesteddomain")
I'll leave it to Baolu to decide if he want a respin or will pick this directly.
BRs,
Zhenzhong
Quoted 34 lines
Reviewed-by Yi Liu <[email protected]>drivers/iommu/intel/nested.c static int intel_nested_set_dev_pasid(struct
Quoted 5 lines
drivers/iommu/intel/nested.c +0
-0@@ -148,6 +148,7 @@ static int intel_nested_set_dev_pasid(struct
iommu_domain*domain,drivers/iommu/intel/nested.c static int intel_nested_set_dev_pasid(struct
Quoted 9 lines
drivers/iommu/intel/nested.c +1
-0{struct device_domain_info *info =dev_iommu_priv_get (dev);struct dmar_domain *dmar_domain =to_dmar_domain (domain); +struct iommu_domain *s2_domain = &dmar_domain->s2_domain- domain;struct intel_iommu *iommu = info->iommu;struct dev_pasid_info *dev_pasid;int ret;
drivers/iommu/intel/nested.c +0
-0@@ -155,10 +156,13 @@ static int intel_nested_set_dev_pasid(struct
iommu_domain*domain,drivers/iommu/intel/nested.c static int intel_nested_set_dev_pasid(struct iommu_domain *domain,
Quoted 10 lines
drivers/iommu/intel/nested.c +3
-1if (!pasid_supported (iommu) ||dev_is_real_dma_subdevice (dev))return -EOPNOTSUPP ; +if (s2_domain->dirty_ops) +return -EINVAL ; +if (context_copied (iommu, info->bus, info->devfn))return -EBUSY ; - ret = paging_domain_compatible(&dmar_domain->s2_domain->domain,dev);
Quoted 4 lines
drivers/iommu/intel/nested.c +1
-0+ ret =paging_domain_compatible (s2_domain, dev);if (ret)return ret;
PATCH v2 2/4 iommupt/vtd: Rename device_set_dirty_tracking() and pass dmar_domain pointer
In reply to Zhenzhong Duandevice_set_dirty_tracking() sets dirty tracking on all devices attached to a domain, also on all PASIDs attached to same domain in subsequent patch.
So rename it as domain_set_dirty_tracking() and pass dmar_domain pointer to better align to what it does.
No functional changes intended.
--- drivers/iommu/intel/iommu.c | 21 +++++++++------------ 1 file changed, 9 insertions(+), 12 deletions(-)
drivers/iommu/intel/iommu.c +9 -12
@@ -3684,16 +3684,15 @@ static void *intel_iommu_hw_info(struct device *dev, u32 *length,return vtd; } -/* -* Set dirty tracking for the device list of a domain. The caller must -* hold the domain->lock when calling it. -*/ -static int device_set_dirty_tracking (struct list_head *devices,bool enable) +/* Set dirty tracking for the devices that the domain has been attached. */ +static int domain_set_dirty_tracking (struct dmar_domain *domain,bool enable) {struct device_domain_info *info;int ret =0 ; -list_for_each_entry (info, devices, link) { +lockdep_assert_held (&domain->lock); + +list_for_each_entry (info, &domain->devices, link) { ret =intel_pasid_setup_dirty_tracking (info->iommu, info->dev,IOMMU_NO_PASID , enable);if (ret)
@@ -3713,7 +3712,7 @@ static int parent_domain_set_dirty_tracking(struct dmar_domain *domain,spin_lock (&domain->s1_lock);list_for_each_entry (s1_domain, &domain->s1_domains, s2_link) {spin_lock_irqsave (&s1_domain->lock, flags); - ret =device_set_dirty_tracking (&s1_domain->devices, enable); + ret =domain_set_dirty_tracking (s1_domain, enable);spin_unlock_irqrestore (&s1_domain->lock, flags);if (ret) goto err_unwind;
@@ -3724,8 +3723,7 @@ static int parent_domain_set_dirty_tracking(struct dmar_domain *domain, err_unwind:list_for_each_entry (s1_domain, &domain->s1_domains, s2_link) {spin_lock_irqsave (&s1_domain->lock, flags); -device_set_dirty_tracking (&s1_domain->devices, - domain->dirty_tracking); +domain_set_dirty_tracking (s1_domain, domain->dirty_tracking);spin_unlock_irqrestore (&s1_domain->lock, flags); }spin_unlock (&domain->s1_lock);
@@ -3742,7 +3740,7 @@ static int intel_iommu_set_dirty_tracking(struct iommu_domain *domain,if (dmar_domain->dirty_tracking == enable) goto out_unlock; - ret =device_set_dirty_tracking (&dmar_domain->devices, enable); + ret =domain_set_dirty_tracking (dmar_domain, enable);if (ret) goto err_unwind;
@@ -3759,8 +3757,7 @@ static int intel_iommu_set_dirty_tracking(struct iommu_domain *domain,return 0 ; err_unwind: -device_set_dirty_tracking (&dmar_domain->devices, - dmar_domain->dirty_tracking); +domain_set_dirty_tracking (dmar_domain, dmar_domain->dirty_tracking);spin_unlock (&dmar_domain->lock);return ret; }
Signature · 1 line
2.47.3
RE: [PATCH v2 2/4] iommupt/vtd: Rename device_set_dirty_tracking() and pass dmar_domain pointer
Quoted 6 lines
From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PM
device_set_dirty_tracking()sets dirty tracking on all devices attached to a domain, also on all PASIDs attached to same domain in subsequent patch.
Quoted 8 lines
So rename it as
domain_set_dirty_tracking()and passdmar_domainpointer to better align to what it does.No functional changes intended.
Suggested-by Lu Baolu <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>Reviewed-by Yi Liu <[email protected]>
PATCH v2 3/4 iommupt/vtd: Support dirty tracking on PASID
In reply to Zhenzhong DuanIn order to support passthrough device with PASID capability in QEMU, e.g., DSA device, kernel needs to support attaching PASID to a domain.
But attaching is not allowed if the domain is a second stage domain or nested domain with dirty tracking.
The reason is kernel lacking support for dirty tracking on such domain attached to PASID. By adding dirty tracking on PASID, the check can be removed.
--- drivers/iommu/intel/iommu.c | 12 +++++++++--- drivers/iommu/intel/nested.c | 6 +----- 2 files changed, 10 insertions(+), 8 deletions(-)
drivers/iommu/intel/iommu.c +9 -3
@@ -3618,9 +3618,6 @@ static int intel_iommu_set_dev_pasid(struct iommu_domain *domain,if (!pasid_supported (iommu) ||dev_is_real_dma_subdevice (dev))return -EOPNOTSUPP ; -if (domain->dirty_ops) -return -EINVAL ; -if (context_copied (iommu, info->bus, info->devfn))return -EBUSY ;
@@ -3688,6 +3685,7 @@ static void *intel_iommu_hw_info(struct device *dev, u32 *length,static int domain_set_dirty_tracking (struct dmar_domain *domain,bool enable) {struct device_domain_info *info; +struct dev_pasid_info *dev_pasid;int ret =0 ;lockdep_assert_held (&domain->lock);
@@ -3695,6 +3693,14 @@ static int domain_set_dirty_tracking(struct dmar_domain *domain, bool enable)list_for_each_entry (info, &domain->devices, link) { ret =intel_pasid_setup_dirty_tracking (info->iommu, info->dev,IOMMU_NO_PASID , enable); +if (ret) +return ret; + } + +list_for_each_entry (dev_pasid, &domain->dev_pasids, link_domain) { + info =dev_iommu_priv_get (dev_pasid->dev); + ret =intel_pasid_setup_dirty_tracking (info->iommu, info->dev, + dev_pasid->pasid, enable);if (ret)break ; }
drivers/iommu/intel/nested.c +1 -5
@@ -148,7 +148,6 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain, {struct device_domain_info *info =dev_iommu_priv_get (dev);struct dmar_domain *dmar_domain =to_dmar_domain (domain); -struct iommu_domain *s2_domain = &dmar_domain->s2_domain->domain;struct intel_iommu *iommu = info->iommu;struct dev_pasid_info *dev_pasid;int ret;
@@ -156,13 +155,10 @@ static int intel_nested_set_dev_pasid(struct iommu_domain *domain,if (!pasid_supported (iommu) ||dev_is_real_dma_subdevice (dev))return -EOPNOTSUPP ; -if (s2_domain->dirty_ops) -return -EINVAL ; -if (context_copied (iommu, info->bus, info->devfn))return -EBUSY ; - ret =paging_domain_compatible (s2_domain, dev); + ret =paging_domain_compatible (&dmar_domain->s2_domain->domain, dev);if (ret)return ret;
Signature · 1 line
2.47.3
Quoted 9 lines
From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PMIn order to support passthrough device with PASID capability in QEMU, e.g., DSA device, kernel needs to support attaching PASID to a domain.
But attaching is not allowed if the domain is a second stage domain or nested domain with dirty tracking.
Quoted 6 lines
The reason is kernel lacking support for dirty tracking on such domain attached to PASID. By adding dirty tracking on PASID, the check can be removed.
Signed-off-by Zhenzhong Duan <[email protected]>Reviewed-by Yi Liu <[email protected]>
PATCH v2 4/4 iommufd/selftest: Test dirty tracking on PASID
In reply to Zhenzhong DuanAdd test case for dirty tracking on a domain attached to PASID, also confirm attachment to PASID fail if device doesn't support dirty tracking.
--- tools/testing/selftests/iommu/iommufd.c | 27 +++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
tools/testing/selftests/iommu/iommufd.c +27 -0
@@ -2275,6 +2275,33 @@ TEST_F(iommufd_dirty_tracking, set_dirty_tracking)test_ioctl_destroy (hwpt_id); } +TEST_F(iommufd_dirty_tracking, pasid_set_dirty_tracking) +{ +uint32_t stddev_id, ioas_id, hwpt_id, pasid =100 ; +uint32_t dev_flags =MOCK_FLAGS_DEVICE_PASID ; + +/* Regular case */ +test_cmd_hwpt_alloc (self->idev_id, self->ioas_id, +IOMMU_HWPT_ALLOC_PASID |IOMMU_HWPT_ALLOC_DIRTY_TRACKING , + &hwpt_id); +test_cmd_mock_domain_flags (hwpt_id, dev_flags, &stddev_id,NULL ,NULL ); +ASSERT_EQ (0 ,_test_cmd_pasid_attach (self->fd, stddev_id, pasid, hwpt_id)); +test_cmd_set_dirty_tracking (hwpt_id, true); +test_cmd_set_dirty_tracking (hwpt_id, false); +ASSERT_EQ (0 ,_test_cmd_pasid_detach (self->fd, stddev_id, pasid)); + +test_ioctl_destroy (stddev_id); + +/* IOMMU device does not support dirty tracking */ + dev_flags |=MOCK_FLAGS_DEVICE_NO_DIRTY ; +test_ioctl_ioas_alloc (&ioas_id); +test_cmd_mock_domain_flags (ioas_id, dev_flags, &stddev_id,NULL ,NULL ); +EXPECT_ERRNO (EINVAL ,_test_cmd_pasid_attach (self->fd, stddev_id, pasid, hwpt_id)); + +test_ioctl_destroy (stddev_id); +test_ioctl_destroy (hwpt_id); +} + TEST_F(iommufd_dirty_tracking, device_dirty_capability) {uint32_t caps =0 ;
Signature · 1 line
2.47.3
RE: [PATCH v2 4/4] iommufd/selftest: Test dirty tracking on PASID
Quoted 9 lines
From: Duan, Zhenzhong <[email protected]>
Sent: Monday, March 30, 2026 6:11 PMAdd test case for dirty tracking on a domain attached to PASID, also confirm attachment to PASID fail if device doesn't support dirty tracking.
Suggested-by Lu Baolu <[email protected]>Signed-off-by Zhenzhong Duan <[email protected]>Reviewed-by Yi Liu <[email protected]>
Quoted 17 lines · Zhenzhong Duan
Hi,
When we add pasid support in QEMU for passthrough device, we found PASID attachment to a nested parent domain with dirty tracking failed.
It's because PASID-level dirty tracking is not there yet, by adding it, we can enable PASID attachment to such domain.
Thanks
ZhenzhongChangelog:
v2:
- a new patch1 to add a check inintel_nested_set_dev_pasid()(Kevin)
- s/device_set_dirty_tracking/domain_set_dirty_tracking (Liuyi)
- add R-b
Quoted 7 lines
Zhenzhong Duan (4):
iommu/vt-d: Block PASID attachment to nested domain with dirty
tracking
iommupt/vtd: Renamedevice_set_dirty_tracking()and passdmar_domain
pointer
iommupt/vtd: Support dirty tracking on PASID iommufd/selftest: Test dirty tracking on PASID
Queued for iommu next with "iommupt/vtd" changed to "iommu/vt-d" in the titles.
Thanks,
baolu