Skip to contentSkip to conversation

iommu/vt-d: Optimize the use of locks

PATCH Superseded Lu Baolu 2 participants 31 messages Jun 14, 2022 – Jun 16, 2022

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Lu Baolu

author maintainer
#0 lore raw

v2 cover iommu/vt-d: Optimize the use of locks

Hi folks,

This series tries to optimize the uses of two locks in the Intel IOMMU driver:

- The intel_iommu::lock is used to protect the IOMMU resources shared by devices. They include the IOMMU root and context tables, the pasid tables and the domain IDs.
- The global device_domain_lock is used to protect the global and the per-domain device tracking lists.

The optimization includes:

- Remove the unnecessary global device tracking list;
- Remove unnecessary locking;
- Reduce the scope of the lock as much as possible, that is, use the lock only where necessary;
- The global lock is transformed into a local lock to improve the efficiency.

This series is also available on github:
https://github.com/LuBaolu/intel-iommu/commits/intel-iommu-lock-optimization-v2

Your comments and suggestions are very appreciated.

Best regards,
baolu

Change log:

v2:
- Split the lock-free page walk issue into a new patch:
https://lore.kernel.org/linux-iommu/[email protected]/
- Drop the conversion from spinlock to mutex and make this series cleanup purpose only.
- Address several comments received during v1 review.

v1:
- https://lore.kernel.org/linux-iommu/[email protected]/
- Initial post.

Lu Baolu (12):
iommu/vt-d: debugfs: Remove device_domain_lock usage iommu/vt-d: Remove for_each_device_domain()
iommu/vt-d: Remove clearing translation data in disable_dmar_iommu() iommu/vt-d: Use pci_get_domain_bus_and_slot() in pgtable_walk() iommu/vt-d: Unnecessary spinlock for root table alloc and free iommu/vt-d: Acquiring lock in domain ID allocation helpers iommu/vt-d: Acquiring lock in pasid manipulation helpers iommu/vt-d: Replace spin_lock_irqsave() with spin_lock() iommu/vt-d: Check device list of domain in domain free path iommu/vt-d: Fold __dmar_remove_one_dev_info() into its caller iommu/vt-d: Use device_domain_lock accurately
iommu/vt-d: Convert global spinlock into per domain ones

 drivers/iommu/intel/iommu.h   |   5 +-
 drivers/iommu/intel/pasid.h   |   1 +
 drivers/iommu/intel/debugfs.c |  55 ++++---
 drivers/iommu/intel/iommu.c   | 279 ++++++++++------------------------
 drivers/iommu/intel/pasid.c   | 149 +++++++++---------
 drivers/iommu/intel/svm.c     |   5 +-
 6 files changed, 185 insertions(+), 309 deletions(-)
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author maintainer
#1 lore raw

PATCH v2 1/12 iommu/vt-d: debugfs: Remove device_domain_lock usage

The domain_translation_struct debugfs node is used to dump the DMAR page tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses a global spinlock device_domain_lock to avoid the races, but this is problematical as this lock is only used to protect the device tracking lists of each domain.

This replaces device_domain_lock with group->mutex to protect page tables from setting a new domain. This also makes device_domain_lock static as it is now only used inside the file.

---
 drivers/iommu/intel/iommu.h   |  1 -
 drivers/iommu/intel/debugfs.c | 49 +++++++++++++++++++++++++----------
 drivers/iommu/intel/iommu.c   |  2 +-
 3 files changed, 36 insertions(+), 16 deletions(-)
drivers/iommu/intel/iommu.h +0 -1
index a22adfbdf870..8a6d64d726c0 100644
@@ -480,7 +480,6 @@ enum {
 #define VTD_FLAG_SVM_CAPABLE		(1 << 2)
 
 extern int intel_iommu_sm;
-extern spinlock_t device_domain_lock;
 
 #define sm_supported(iommu)	(intel_iommu_sm && ecap_smts((iommu)->ecap))
 #define pasid_supported(iommu)	(sm_supported(iommu) &&			\
drivers/iommu/intel/debugfs.c +35 -14

70 lines not on this page · raw message

drivers/iommu/intel/iommu.c +1 -1
index 19024dc52735..a39d72a9d1cf 100644
@@ -314,7 +314,7 @@ static int iommu_skip_te_disable;
 #define IDENTMAP_GFX		2
 #define IDENTMAP_AZALIA		4
 
-DEFINE_SPINLOCK(device_domain_lock);
+static DEFINE_SPINLOCK(device_domain_lock);
 static LIST_HEAD(device_domain_list);
 
 /*
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#2 lore raw

RE: [PATCH v2 01/12] iommu/vt-d: debugfs: Remove device_domain_lock usage

Quoted 9 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The domain_translation_struct debugfs node is used to dump the DMAR page
tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses a global spinlock device_domain_lock to avoid the races, but this is problematical as this lock is only used to protect the device tracking lists of each domain.

is it really problematic at this point? Before following patches are applied using device_domain_lock should have similar effect as holding the group lock.

Here it might make more sense to just focus on removing the use of device_domain_lock outside of iommu.c. Just that using group lock is cleaner and more compatible to following cleanups.

and it's worth mentioning that racing with page table updates is out of the scope of this series. Probably also add a comment in the code to clarify this point.

Quoted 4 lines

This replaces device_domain_lock with group->mutex to protect page tables from setting a new domain. This also makes device_domain_lock static as it is now only used inside the file.

s/the file/iommu.c/

Quoted 27 lines
---
 drivers/iommu/intel/iommu.h   |  1 -
 drivers/iommu/intel/debugfs.c | 49 +++++++++++++++++++++++++----------
 drivers/iommu/intel/iommu.c   |  2 +-
 3 files changed, 36 insertions(+), 16 deletions(-)
drivers/iommu/intel/iommu.h +0 -1
index a22adfbdf870..8a6d64d726c0 100644
@@ -480,7 +480,6 @@ enum {
 #define VTD_FLAG_SVM_CAPABLE		(1 << 2)
 
 extern int intel_iommu_sm;
-extern spinlock_t device_domain_lock;
 
 #define sm_supported(iommu)	(intel_iommu_sm &&
 ecap_smts((iommu)->ecap))
 #define pasid_supported(iommu)	(sm_supported(iommu) &&
		\
drivers/iommu/intel/debugfs.c +0 -0
index d927ef10641b..5ebfe32265d5 100644

drivers/iommu/intel/debugfs.c static void pgtable_walk_level(struct seq_file *m, · line 349

Quoted 9 lines
drivers/iommu/intel/debugfs.c +4 -1
@@ -342,15 +342,23 @@ static void pgtable_walk_level(struct seq_file *m,
 struct dma_pte *pde,
 	}
 }
 
-static int show_device_domain_translation(struct device *dev, void *data)
+struct show_domain_opaque {
+	struct device *dev;
+	struct seq_file *m;
+};

Sounds redundant as both bus_for_each_dev() and iommu_group_for_each_dev() declares the same fn type which accepts a device pointer and void *data.

Quoted 5 lines
drivers/iommu/intel/debugfs.c +2 -2
+
+static int __show_device_domain_translation(struct device *dev, void *data)
 {
-	struct device_domain_info *info = dev_iommu_priv_get(dev);
-	struct dmar_domain *domain = info->domain;

drivers/iommu/intel/debugfs.c static void pgtable_walk_level(struct seq_file *m, struct dma_pte *pde, · line 359

Quoted 10 lines
drivers/iommu/intel/debugfs.c +5 -2
-	struct seq_file *m = data;
+	struct show_domain_opaque *opaque = data;
+	struct device_domain_info *info;
+	struct seq_file *m = opaque->m;
+	struct dmar_domain *domain;
 	u64 path[6] = { 0 };
 
-	if (!domain)
+	if (dev != opaque->dev)
 		return 0;

not required.

Quoted 63 lines
drivers/iommu/intel/debugfs.c +2 -0
+	info = dev_iommu_priv_get(dev);
+	domain = info->domain;
 
 	seq_printf(m, "Device %s @0x%llx\n", dev_name(dev),
drivers/iommu/intel/debugfs.c +22 -9
 		   (u64)virt_to_phys(domain->pgd));
@@ -359,20 +367,33 @@ static int show_device_domain_translation(struct
 device *dev, void *data)
 	pgtable_walk_level(m, domain->pgd, domain->agaw + 2, 0, path);
 	seq_putc(m, '\n');
 
-	return 0;
+	return 1;
 }
 
-static int domain_translation_struct_show(struct seq_file *m, void *unused)
+static int show_device_domain_translation(struct device *dev, void *data)
 {
-	unsigned long flags;
-	int ret;
+	struct show_domain_opaque opaque = {dev, data};
+	struct iommu_group *group;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	ret = bus_for_each_dev(&pci_bus_type, NULL, m,
-			       show_device_domain_translation);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
+	group = iommu_group_get(dev);
+	if (group) {
+		/*
+		 * The group->mutex is held across the callback, which will
+		 * block calls to iommu_attach/detach_group/device. Hence,
+		 * the domain of the device will not change during traversal.
+		 */
+		iommu_group_for_each_dev(group, &opaque,
+					 __show_device_domain_translation);
+		iommu_group_put(group);
+	}
 
-	return ret;
+	return 0;
+}
+
+static int domain_translation_struct_show(struct seq_file *m, void *unused)
+{
+	return bus_for_each_dev(&pci_bus_type, NULL, m,
+				show_device_domain_translation);
 }
 DEFINE_SHOW_ATTRIBUTE(domain_translation_struct);
drivers/iommu/intel/iommu.c +1 -2
index 19024dc52735..a39d72a9d1cf 100644
@@ -314,7 +314,7 @@ static int iommu_skip_te_disable;
 #define IDENTMAP_GFX		2
 #define IDENTMAP_AZALIA		4
 
-DEFINE_SPINLOCK(device_domain_lock);
+static DEFINE_SPINLOCK(device_domain_lock);
 static LIST_HEAD(device_domain_list);
 
 /*
--
Quoted 1 line

2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#3 lore raw

Re: [PATCH v2 01/12] iommu/vt-d: debugfs: Remove device_domain_lock usage

Hi Kevin,

Thanks for your reviewing.

Quoted 10 lines · Kevin
Quoted 9 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The domain_translation_struct debugfs node is used to dump the DMAR page
tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses a global spinlock device_domain_lock to avoid the races, but this is problematical as this lock is only used to protect the device tracking lists of each domain.

Quoted 3 lines

is it really problematic at this point? Before following patches are applied using device_domain_lock should have similar effect as holding the group lock.

The device_domain_lock only protects the device tracking list of the domain, it doesn't include the domain pointer stored in the dev_info structure. That's really protected by the group->mutex.

Quoted 4 lines

Here it might make more sense to just focus on removing the use of device_domain_lock outside of iommu.c. Just that using group lock is cleaner and more compatible to following cleanups.

Fair enough. I will update the commit message with above statement.

Quoted 3 lines

and it's worth mentioning that racing with page table updates is out of the scope of this series. Probably also add a comment in the code to clarify this point.

Sure.

Quoted 7 lines
Quoted 5 lines

This replaces device_domain_lock with group->mutex to protect page tables from setting a new domain. This also makes device_domain_lock static as it is now only used inside the file.

s/the file/iommu.c/

Sure.

Quoted 54 lines
Quoted 28 lines
---
  drivers/iommu/intel/iommu.h   |  1 -
  drivers/iommu/intel/debugfs.c | 49 +++++++++++++++++++++++++----------
  drivers/iommu/intel/iommu.c   |  2 +-
  3 files changed, 36 insertions(+), 16 deletions(-)
drivers/iommu/intel/iommu.h +0 -1
index a22adfbdf870..8a6d64d726c0 100644
@@ -480,7 +480,6 @@ enum {
  #define VTD_FLAG_SVM_CAPABLE		(1 << 2)
 
  extern int intel_iommu_sm;
-extern spinlock_t device_domain_lock;
 
  #define sm_supported(iommu)	(intel_iommu_sm &&
 ecap_smts((iommu)->ecap))
  #define pasid_supported(iommu)	(sm_supported(iommu) &&
		\
drivers/iommu/intel/debugfs.c +0 -0
index d927ef10641b..5ebfe32265d5 100644

drivers/iommu/intel/debugfs.c static void pgtable_walk_level(struct seq_file *m, · line 349

Quoted 9 lines
drivers/iommu/intel/debugfs.c +4 -1
@@ -342,15 +342,23 @@ static void pgtable_walk_level(struct seq_file *m,
 struct dma_pte *pde,
  	}
  }
 
-static int show_device_domain_translation(struct device *dev, void *data)
+struct show_domain_opaque {
+	struct device *dev;
+	struct seq_file *m;
+};

Sounds redundant as both bus_for_each_dev() and iommu_group_for_each_dev() declares the same fn type which accepts a device pointer and void *data.

Quoted 12 lines
drivers/iommu/intel/debugfs.c +6 -3
+
+static int __show_device_domain_translation(struct device *dev, void *data)
  {
-	struct device_domain_info *info = dev_iommu_priv_get(dev);
-	struct dmar_domain *domain = info->domain;
-	struct seq_file *m = data;
+	struct show_domain_opaque *opaque = data;
+	struct device_domain_info *info;
+	struct seq_file *m = opaque->m;
+	struct dmar_domain *domain;
  	u64 path[6] = { 0 };
 
Quoted 5 lines

drivers/iommu/intel/debugfs.c static void pgtable_walk_level(struct seq_file *m, struct dma_pte *pde, · line 359

Quoted 3 lines
drivers/iommu/intel/debugfs.c +1 -1
-	if (!domain)
+	if (dev != opaque->dev)
  		return 0;

not required.

Together with above comment.

The iommu group might have other devices. I only want to dump the domain of the secific @opaque->dev. It reads a bit confusing, but it's the only helper I can use outside of drivers/iommu/iommu.c.

Or, since all devices in the iommu group share the same domain, hence only dump once?

Quoted 58 lines
Quoted 58 lines
drivers/iommu/intel/debugfs.c +2 -0
+	info = dev_iommu_priv_get(dev);
+	domain = info->domain;
 
  	seq_printf(m, "Device %s @0x%llx\n", dev_name(dev),
  		   (u64)virt_to_phys(domain->pgd));
drivers/iommu/intel/debugfs.c +22 -9
@@ -359,20 +367,33 @@ static int show_device_domain_translation(struct
 device *dev, void *data)
  	pgtable_walk_level(m, domain->pgd, domain->agaw + 2, 0, path);
  	seq_putc(m, '\n');
 
-	return 0;
+	return 1;
  }
 
-static int domain_translation_struct_show(struct seq_file *m, void *unused)
+static int show_device_domain_translation(struct device *dev, void *data)
  {
-	unsigned long flags;
-	int ret;
+	struct show_domain_opaque opaque = {dev, data};
+	struct iommu_group *group;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	ret = bus_for_each_dev(&pci_bus_type, NULL, m,
-			       show_device_domain_translation);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
+	group = iommu_group_get(dev);
+	if (group) {
+		/*
+		 * The group->mutex is held across the callback, which will
+		 * block calls to iommu_attach/detach_group/device. Hence,
+		 * the domain of the device will not change during traversal.
+		 */
+		iommu_group_for_each_dev(group, &opaque,
+					 __show_device_domain_translation);
+		iommu_group_put(group);
+	}
 
-	return ret;
+	return 0;
+}
+
+static int domain_translation_struct_show(struct seq_file *m, void *unused)
+{
+	return bus_for_each_dev(&pci_bus_type, NULL, m,
+				show_device_domain_translation);
  }
  DEFINE_SHOW_ATTRIBUTE(domain_translation_struct);
drivers/iommu/intel/iommu.c +0 -0
index 19024dc52735..a39d72a9d1cf 100644
@@ -314,7 +314,7 @@ static int iommu_skip_te_disable;
  #define IDENTMAP_GFX		2
  #define IDENTMAP_AZALIA		4
 
Quoted 8 lines
Quoted 7 lines
drivers/iommu/intel/iommu.c +1 -2
-DEFINE_SPINLOCK(device_domain_lock);
+static DEFINE_SPINLOCK(device_domain_lock);
  static LIST_HEAD(device_domain_list);
 
  /*
--

2.25.1

Best regards,
baolu

author maintainer
#4 lore raw

Re: [PATCH v2 01/12] iommu/vt-d: debugfs: Remove device_domain_lock usage

In reply to Tian, Kevin
Quoted 13 lines · Kevin
Quoted 9 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The domain_translation_struct debugfs node is used to dump the DMAR page
tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses a global spinlock device_domain_lock to avoid the races, but this is problematical as this lock is only used to protect the device tracking lists of each domain.

is it really problematic at this point? Before following patches are applied using device_domain_lock should have similar effect as holding the group lock.

Quoted 8 lines

Here it might make more sense to just focus on removing the use of device_domain_lock outside of iommu.c. Just that using group lock is cleaner and more compatible to following cleanups.

and it's worth mentioning that racing with page table updates is out of the scope of this series. Probably also add a comment in the code to clarify this point.

Hi Kevin,

How do you like below updated patch?

From cecc9a0623780a11c4ea4d0a15aa6187f01541c4 Mon Sep 17 00:00:00 2001
From: Lu Baolu <[email protected]>
Date: Sun, 29 May 2022 10:18:56 +0800
Subject: [PATCH 1/1] iommu/vt-d: debugfs: Remove device_domain_lock usage

The domain_translation_struct debugfs node is used to dump the DMAR page tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses the global spinlock device_domain_lock to avoid the races.

This removes the use of device_domain_lock outside of iommu.c by replacing it with the group mutex lock. Using the group mutex lock is cleaner and more compatible to following cleanups.

---
 drivers/iommu/intel/debugfs.c | 42 +++++++++++++++++++++++++----------
 drivers/iommu/intel/iommu.c   |  2 +-
 drivers/iommu/intel/iommu.h   |  1 -
 3 files changed, 31 insertions(+), 14 deletions(-)
drivers/iommu/intel/debugfs.c +30 -12
index d927ef10641b..f4acd8993f60 100644
@@ -342,13 +342,13 @@ static void pgtable_walk_level(struct seq_file *m, struct dma_pte *pde,
 	}
 }
 
-static int show_device_domain_translation(struct device *dev, void *data)
+static int __show_device_domain_translation(struct device *dev, void *data)
 {
-	struct device_domain_info *info = dev_iommu_priv_get(dev);
-	struct dmar_domain *domain = info->domain;
+	struct dmar_domain *domain;
 	struct seq_file *m = data;
 	u64 path[6] = { 0 };
 
+	domain = to_dmar_domain(iommu_get_domain_for_dev(dev));
 	if (!domain)
 		return 0;
 
@@ -359,20 +359,38 @@ static int show_device_domain_translation(struct device *dev, void *data)
 	pgtable_walk_level(m, domain->pgd, domain->agaw + 2, 0, path);
 	seq_putc(m, '\n');
 
-	return 0;
+	return 1;
 }
 
-static int domain_translation_struct_show(struct seq_file *m, void *unused)
+static int show_device_domain_translation(struct device *dev, void *data)
 {
-	unsigned long flags;
-	int ret;
+	struct iommu_group *group;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	ret = bus_for_each_dev(&pci_bus_type, NULL, m,
-			       show_device_domain_translation);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
+	group = iommu_group_get(dev);
+	if (group) {
+		/*
+		 * The group->mutex is held across the callback, which will
+		 * block calls to iommu_attach/detach_group/device. Hence,
+		 * the domain of the device will not change during traversal.
+		 *
+		 * All devices in an iommu group share a single domain, hence
+		 * we only dump the domain of the first device. Even though,
+		 * this code still possibly races with the iommu_unmap()
+		 * interface. This could be solved by RCU-freeing the page
+		 * table pages in the iommu_unmap() path.
+		 */
+		iommu_group_for_each_dev(group, data,
+					 __show_device_domain_translation);
+		iommu_group_put(group);
+	}
 
-	return ret;
+	return 0;
+}
+
+static int domain_translation_struct_show(struct seq_file *m, void *unused)
+{
+	return bus_for_each_dev(&pci_bus_type, NULL, m,
+				show_device_domain_translation);
 }
 DEFINE_SHOW_ATTRIBUTE(domain_translation_struct);
 
drivers/iommu/intel/iommu.c +1 -1
index 19024dc52735..a39d72a9d1cf 100644
@@ -314,7 +314,7 @@ static int iommu_skip_te_disable;
 #define IDENTMAP_GFX		2
 #define IDENTMAP_AZALIA		4
 
-DEFINE_SPINLOCK(device_domain_lock);
+static DEFINE_SPINLOCK(device_domain_lock);
 static LIST_HEAD(device_domain_list);
 
 /*
drivers/iommu/intel/iommu.h +0 -1
index a22adfbdf870..8a6d64d726c0 100644
@@ -480,7 +480,6 @@ enum {
 #define VTD_FLAG_SVM_CAPABLE		(1 << 2)
 
 extern int intel_iommu_sm;
-extern spinlock_t device_domain_lock;
 
 #define sm_supported(iommu)	(intel_iommu_sm && ecap_smts((iommu)->ecap))
 #define pasid_supported(iommu)	(sm_supported(iommu) &&			\
Signature · 4 lines
2.25.1

Best regards,
baolu
#5 lore raw

RE: [PATCH v2 01/12] iommu/vt-d: debugfs: Remove device_domain_lock usage

Quoted 21 lines

From: Baolu Lu <[email protected]>
Sent: Wednesday, June 15, 2022 9:54 AM

Quoted 17 lines · Kevin
Quoted 9 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The domain_translation_struct debugfs node is used to dump the DMAR page
tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses a global spinlock device_domain_lock to avoid the races, but this is problematical as this lock is only used to protect the device tracking lists of each domain.

is it really problematic at this point? Before following patches are applied using device_domain_lock should have similar effect as holding the group lock.

Here it might make more sense to just focus on removing the use of device_domain_lock outside of iommu.c. Just that using group lock is cleaner and more compatible to following cleanups.

Quoted 8 lines
Quoted 4 lines

and it's worth mentioning that racing with page table updates is out of the scope of this series. Probably also add a comment in the code to clarify this point.

Hi Kevin,

How do you like below updated patch?

Yes, this is better.

Quoted 76 lines

From cecc9a0623780a11c4ea4d0a15aa6187f01541c4 Mon Sep 17 00:00:00
2001
From: Lu Baolu <[email protected]>
Date: Sun, 29 May 2022 10:18:56 +0800
Subject: [PATCH 1/1] iommu/vt-d: debugfs: Remove device_domain_lock usage

The domain_translation_struct debugfs node is used to dump the DMAR page
tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses the global spinlock device_domain_lock to avoid the races.

This removes the use of device_domain_lock outside of iommu.c by replacing it with the group mutex lock. Using the group mutex lock is cleaner and more compatible to following cleanups.

---
  drivers/iommu/intel/debugfs.c | 42 +++++++++++++++++++++++++----------
  drivers/iommu/intel/iommu.c   |  2 +-
  drivers/iommu/intel/iommu.h   |  1 -
  3 files changed, 31 insertions(+), 14 deletions(-)
drivers/iommu/intel/debugfs.c +14 -11
index d927ef10641b..f4acd8993f60 100644
@@ -342,13 +342,13 @@ static void pgtable_walk_level(struct seq_file *m,
 struct dma_pte *pde,
  	}
  }
 
-static int show_device_domain_translation(struct device *dev, void *data)
+static int __show_device_domain_translation(struct device *dev, void *data)
  {
-	struct device_domain_info *info = dev_iommu_priv_get(dev);
-	struct dmar_domain *domain = info->domain;
+	struct dmar_domain *domain;
  	struct seq_file *m = data;
  	u64 path[6] = { 0 };
 
+	domain = to_dmar_domain(iommu_get_domain_for_dev(dev));
  	if (!domain)
  		return 0;
@@ -359,20 +359,38 @@ static int show_device_domain_translation(struct
 device *dev, void *data)
  	pgtable_walk_level(m, domain->pgd, domain->agaw + 2, 0, path);
  	seq_putc(m, '\n');
 
-	return 0;
+	return 1;
  }
 
-static int domain_translation_struct_show(struct seq_file *m, void *unused)
+static int show_device_domain_translation(struct device *dev, void *data)
  {
-	unsigned long flags;
-	int ret;
+	struct iommu_group *group;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	ret = bus_for_each_dev(&pci_bus_type, NULL, m,
-			       show_device_domain_translation);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
+	group = iommu_group_get(dev);
+	if (group) {
+		/*
+		 * The group->mutex is held across the callback, which will
+		 * block calls to iommu_attach/detach_group/device. Hence,
+		 * the domain of the device will not change during traversal.
+		 *
+		 * All devices in an iommu group share a single domain,

hence

drivers/iommu/intel/debugfs.c static int show_device_domain_translation(struct device *dev, void *data) · line 377

Quoted 1 line
drivers/iommu/intel/debugfs.c +1 -0
+		 * we only dump the domain of the first device. Even though,

bus_for_each_dev() will still lead to duplicated dump in the same group but probably we can leave with it for a debug interface.

Quoted 47 lines
drivers/iommu/intel/debugfs.c +15 -1
+		 * this code still possibly races with the iommu_unmap()
+		 * interface. This could be solved by RCU-freeing the page
+		 * table pages in the iommu_unmap() path.
+		 */
+		iommu_group_for_each_dev(group, data,
+					 __show_device_domain_translation);
+		iommu_group_put(group);
+	}
 
-	return ret;
+	return 0;
+}
+
+static int domain_translation_struct_show(struct seq_file *m, void *unused)
+{
+	return bus_for_each_dev(&pci_bus_type, NULL, m,
+				show_device_domain_translation);
  }
  DEFINE_SHOW_ATTRIBUTE(domain_translation_struct);
drivers/iommu/intel/iommu.c +1 -1
index 19024dc52735..a39d72a9d1cf 100644
@@ -314,7 +314,7 @@ static int iommu_skip_te_disable;
  #define IDENTMAP_GFX		2
  #define IDENTMAP_AZALIA		4
 
-DEFINE_SPINLOCK(device_domain_lock);
+static DEFINE_SPINLOCK(device_domain_lock);
  static LIST_HEAD(device_domain_list);
 
  /*
drivers/iommu/intel/iommu.h +0 -1
index a22adfbdf870..8a6d64d726c0 100644
@@ -480,7 +480,6 @@ enum {
  #define VTD_FLAG_SVM_CAPABLE		(1 << 2)
 
  extern int intel_iommu_sm;
-extern spinlock_t device_domain_lock;
 
  #define sm_supported(iommu)	(intel_iommu_sm &&
 ecap_smts((iommu)->ecap))
  #define pasid_supported(iommu)	(sm_supported(iommu) &&
		\
Quoted 5 lines

--
2.25.1

Best regards,
baolu

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#6 lore raw

Re: [PATCH v2 01/12] iommu/vt-d: debugfs: Remove device_domain_lock usage

Quoted 92 lines · Kevin
Quoted 21 lines

From: Baolu Lu<[email protected]>
Sent: Wednesday, June 15, 2022 9:54 AM

Quoted 17 lines · Kevin
Quoted 9 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The domain_translation_struct debugfs node is used to dump the DMAR page
tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses a global spinlock device_domain_lock to avoid the races, but this is problematical as this lock is only used to protect the device tracking lists of each domain.

is it really problematic at this point? Before following patches are applied using device_domain_lock should have similar effect as holding the group lock.

Here it might make more sense to just focus on removing the use of device_domain_lock outside of iommu.c. Just that using group lock is cleaner and more compatible to following cleanups.

Quoted 7 lines
Quoted 4 lines

and it's worth mentioning that racing with page table updates is out of the scope of this series. Probably also add a comment in the code to clarify this point.

Hi Kevin,

How do you like below updated patch?

Yes, this is better.

Quoted 62 lines

From cecc9a0623780a11c4ea4d0a15aa6187f01541c4 Mon Sep 17 00:00:00
2001
From: Lu Baolu<[email protected]>
Date: Sun, 29 May 2022 10:18:56 +0800
Subject: [PATCH 1/1] iommu/vt-d: debugfs: Remove device_domain_lock usage

The domain_translation_struct debugfs node is used to dump the DMAR page
tables for the PCI devices. It potentially races with setting domains to devices. The existing code uses the global spinlock device_domain_lock to avoid the races.

This removes the use of device_domain_lock outside of iommu.c by replacing it with the group mutex lock. Using the group mutex lock is cleaner and more compatible to following cleanups.

---
   drivers/iommu/intel/debugfs.c | 42 +++++++++++++++++++++++++----------
   drivers/iommu/intel/iommu.c   |  2 +-
   drivers/iommu/intel/iommu.h   |  1 -
   3 files changed, 31 insertions(+), 14 deletions(-)
drivers/iommu/intel/debugfs.c +6 -7
index d927ef10641b..f4acd8993f60 100644
@@ -342,13 +342,13 @@ static void pgtable_walk_level(struct seq_file *m,
 struct dma_pte *pde,
   	}
   }
 
-static int show_device_domain_translation(struct device *dev, void *data)
+static int __show_device_domain_translation(struct device *dev, void *data)
   {
-	struct device_domain_info *info = dev_iommu_priv_get(dev);
-	struct dmar_domain *domain = info->domain;
+	struct dmar_domain *domain;
   	struct seq_file *m = data;
   	u64 path[6] = { 0 };
 
+	domain = to_dmar_domain(iommu_get_domain_for_dev(dev));
   	if (!domain)
   		return 0;
@@ -359,20 +359,38 @@ static int show_device_domain_translation(struct
 device *dev, void *data)
   	pgtable_walk_level(m, domain->pgd, domain->agaw + 2, 0, path);
   	seq_putc(m, '\n');
 
-	return 0;
+	return 1;
   }
 
-static int domain_translation_struct_show(struct seq_file *m, void *unused)
+static int show_device_domain_translation(struct device *dev, void *data)
   {
-	unsigned long flags;
-	int ret;
+	struct iommu_group *group;
 
Quoted 17 lines
Quoted 13 lines
drivers/iommu/intel/debugfs.c +8 -4
-	spin_lock_irqsave(&device_domain_lock, flags);
-	ret = bus_for_each_dev(&pci_bus_type, NULL, m,
-			       show_device_domain_translation);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
+	group = iommu_group_get(dev);
+	if (group) {
+		/*
+		 * The group->mutex is held across the callback, which will
+		 * block calls to iommu_attach/detach_group/device. Hence,
+		 * the domain of the device will not change during traversal.
+		 *
+		 * All devices in an iommu group share a single domain,

hence

drivers/iommu/intel/debugfs.c static int show_device_domain_translation(struct device *dev, void *data) · line 377

Quoted 1 line
drivers/iommu/intel/debugfs.c +1 -0
+		 * we only dump the domain of the first device. Even though,

bus_for_each_dev() will still lead to duplicated dump in the same group but probably we can leave with it for a debug interface.

Yes. This is what it was. Ideally we could walk the iommu groups and dump the device names belonging to the group and it's domain mappings, but I was not willing to add any helpers in the iommu core just for a debugfs use.

---
Best regards,
baolu

author maintainer
#7 lore raw

PATCH v2 2/12 iommu/vt-d: Remove for_each_device_domain()

In reply to Lu Baolu

The per-device device_domain_info data could be retrieved from the device itself. There's no need to search a global list.

---
 drivers/iommu/intel/iommu.h |  2 --
 drivers/iommu/intel/iommu.c | 25 ----------------------
 drivers/iommu/intel/pasid.c | 41 +++++++++++--------------------------
 3 files changed, 12 insertions(+), 56 deletions(-)
drivers/iommu/intel/iommu.h +0 -2
index 8a6d64d726c0..2f4a5b9509c0 100644
@@ -727,8 +727,6 @@ extern int dmar_ir_support(void);
 void *alloc_pgtable_page(int node);
 void free_pgtable_page(void *vaddr);
 struct intel_iommu *domain_get_iommu(struct dmar_domain *domain);
-int for_each_device_domain(int (*fn)(struct device_domain_info *info,
-				     void *data), void *data);
 void iommu_flush_write_buffer(struct intel_iommu *iommu);
 int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct device *dev);
 struct intel_iommu *device_to_iommu(struct device *dev, u8 *bus, u8 *devfn);
drivers/iommu/intel/iommu.c +0 -25
index a39d72a9d1cf..ff6018f746e0 100644
@@ -316,31 +316,6 @@ static int iommu_skip_te_disable;
 
 static DEFINE_SPINLOCK(device_domain_lock);
 static LIST_HEAD(device_domain_list);
-
-/*
- * Iterate over elements in device_domain_list and call the specified
- * callback @fn against each element.
- */
-int for_each_device_domain(int (*fn)(struct device_domain_info *info,
-				     void *data), void *data)
-{
-	int ret = 0;
-	unsigned long flags;
-	struct device_domain_info *info;
-
-	spin_lock_irqsave(&device_domain_lock, flags);
-	list_for_each_entry(info, &device_domain_list, global) {
-		ret = fn(info, data);
-		if (ret) {
-			spin_unlock_irqrestore(&device_domain_lock, flags);
-			return ret;
-		}
-	}
-	spin_unlock_irqrestore(&device_domain_lock, flags);
-
-	return 0;
-}
-
 const struct iommu_ops intel_iommu_ops;
 
 static bool translation_pre_enabled(struct intel_iommu *iommu)
drivers/iommu/intel/pasid.c +12 -29

71 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author maintainer
#8 lore raw

PATCH v2 3/12 iommu/vt-d: Remove clearing translation data in disable_dmar_iommu()

In reply to Lu Baolu

The disable_dmar_iommu() is called when IOMMU initialization fails or the IOMMU is hot-removed from the system. In both cases, there is no need to clear the IOMMU translation data structures for devices.

On the initialization path, the device probing only happens after the IOMMU is initialized successfully, hence there're no translation data structures.

On the hot-remove path, there is no real use case where the IOMMU is hot-removed, but the devices that it manages are still alive in the system. The translation data structures were torn down during device release, hence there's no need to repeat it in IOMMU hot-remove path either. This removes the unnecessary code and only leaves a check.

---
 drivers/iommu/intel/pasid.h |  1 +
 drivers/iommu/intel/iommu.c | 21 +++++++--------------
 2 files changed, 8 insertions(+), 14 deletions(-)
drivers/iommu/intel/pasid.h +1 -0
index 583ea67fc783..2afbb2afe8cc 100644
@@ -39,6 +39,7 @@
  * only and pass-through transfer modes.
  */
 #define FLPT_DEFAULT_DID		1
+#define NUM_RESERVED_DID		2
 
 /*
  * The SUPERVISOR_MODE flag indicates a first level translation which
drivers/iommu/intel/iommu.c +7 -14
index ff6018f746e0..695aed474e5d 100644
@@ -1715,23 +1715,16 @@ static int iommu_init_domains(struct intel_iommu *iommu)
 
 static void disable_dmar_iommu(struct intel_iommu *iommu)
 {
-	struct device_domain_info *info, *tmp;
-	unsigned long flags;
-
 	if (!iommu->domain_ids)
 		return;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	list_for_each_entry_safe(info, tmp, &device_domain_list, global) {
-		if (info->iommu != iommu)
-			continue;
-
-		if (!info->dev || !info->domain)
-			continue;
-
-		__dmar_remove_one_dev_info(info);
-	}
-	spin_unlock_irqrestore(&device_domain_lock, flags);
+	/*
+	 * All iommu domains must have been detached from the devices,
+	 * hence there should be no domain IDs in use.
+	 */
+	if (WARN_ON(bitmap_weight(iommu->domain_ids, cap_ndoms(iommu->cap))
+		    != NUM_RESERVED_DID))
+		return;
 
 	if (iommu->gcmd & DMA_GCMD_TE)
 		iommu_disable_translation(iommu);
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#9 lore raw

RE: [PATCH v2 03/12] iommu/vt-d: Remove clearing translation data in disable_dmar_iommu()

Quoted 10 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The disable_dmar_iommu() is called when IOMMU initialization fails or the IOMMU is hot-removed from the system. In both cases, there is no need to clear the IOMMU translation data structures for devices.

On the initialization path, the device probing only happens after the IOMMU is initialized successfully, hence there're no translation data structures.

Out of curiosity. With kexec the IOMMU may contain stale mappings from the old kernel. Then is it meaningful to disable IOMMU after the new kernel fails to initialize it properly?

Quoted 63 lines

On the hot-remove path, there is no real use case where the IOMMU is hot-removed, but the devices that it manages are still alive in the system. The translation data structures were torn down during device release, hence there's no need to repeat it in IOMMU hot-remove path either. This removes the unnecessary code and only leaves a check.

---
 drivers/iommu/intel/pasid.h |  1 +
 drivers/iommu/intel/iommu.c | 21 +++++++--------------
 2 files changed, 8 insertions(+), 14 deletions(-)
drivers/iommu/intel/pasid.h +1 -0
index 583ea67fc783..2afbb2afe8cc 100644
@@ -39,6 +39,7 @@
  * only and pass-through transfer modes.
  */
 #define FLPT_DEFAULT_DID		1
+#define NUM_RESERVED_DID		2
 
 /*
  * The SUPERVISOR_MODE flag indicates a first level translation which
drivers/iommu/intel/iommu.c +7 -14
index ff6018f746e0..695aed474e5d 100644
@@ -1715,23 +1715,16 @@ static int iommu_init_domains(struct
 intel_iommu *iommu)
 
 static void disable_dmar_iommu(struct intel_iommu *iommu)
 {
-	struct device_domain_info *info, *tmp;
-	unsigned long flags;
-
 	if (!iommu->domain_ids)
 		return;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	list_for_each_entry_safe(info, tmp, &device_domain_list, global) {
-		if (info->iommu != iommu)
-			continue;
-
-		if (!info->dev || !info->domain)
-			continue;
-
-		__dmar_remove_one_dev_info(info);
-	}
-	spin_unlock_irqrestore(&device_domain_lock, flags);
+	/*
+	 * All iommu domains must have been detached from the devices,
+	 * hence there should be no domain IDs in use.
+	 */
+	if (WARN_ON(bitmap_weight(iommu->domain_ids,
 cap_ndoms(iommu->cap))
+		    != NUM_RESERVED_DID))
+		return;
 
drivers/iommu/intel/iommu.c +0 -1
 	if (iommu->gcmd & DMA_GCMD_TE)
 		iommu_disable_translation(iommu);
--
Quoted 1 line

2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#10 lore raw

Re: [PATCH v2 03/12] iommu/vt-d: Remove clearing translation data in disable_dmar_iommu()

Quoted 7 lines · Kevin
Quoted 7 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The disable_dmar_iommu() is called when IOMMU initialization fails or the IOMMU is hot-removed from the system. In both cases, there is no need to clear the IOMMU translation data structures for devices.

Quoted 6 lines
Quoted 3 lines

On the initialization path, the device probing only happens after the IOMMU is initialized successfully, hence there're no translation data structures.

Out of curiosity. With kexec the IOMMU may contain stale mappings from the old kernel. Then is it meaningful to disable IOMMU after the new kernel fails to initialize it properly?

For kexec kernel, if the IOMMU is detected to be pre-enabled, the IOMMU driver will try to copy tables from the old kernel. If copying table fails, the IOMMU driver will disable IOMMU and do the normal initialization.

Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#11 lore raw

RE: [PATCH v2 03/12] iommu/vt-d: Remove clearing translation data in disable_dmar_iommu()

Quoted 14 lines

From: Baolu Lu <[email protected]>
Sent: Tuesday, June 14, 2022 3:21 PM

Quoted 10 lines · Kevin
Quoted 10 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The disable_dmar_iommu() is called when IOMMU initialization fails or the IOMMU is hot-removed from the system. In both cases, there is no need to clear the IOMMU translation data structures for devices.

On the initialization path, the device probing only happens after the IOMMU is initialized successfully, hence there're no translation data structures.

Quoted 9 lines
Quoted 3 lines

Out of curiosity. With kexec the IOMMU may contain stale mappings from the old kernel. Then is it meaningful to disable IOMMU after the new kernel fails to initialize it properly?

For kexec kernel, if the IOMMU is detected to be pre-enabled, the IOMMU driver will try to copy tables from the old kernel. If copying table fails, the IOMMU driver will disable IOMMU and do the normal initialization.

What about an error occurred after copying table in the initialization path? The new kernel will be in a state assuming iommu is disabled but it is still enabled using an old mapping for certain devices...

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#12 lore raw

Re: [PATCH v2 03/12] iommu/vt-d: Remove clearing translation data in disable_dmar_iommu()

Quoted 24 lines · Kevin
Quoted 14 lines

From: Baolu Lu <[email protected]>
Sent: Tuesday, June 14, 2022 3:21 PM

Quoted 10 lines · Kevin
Quoted 10 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The disable_dmar_iommu() is called when IOMMU initialization fails or the IOMMU is hot-removed from the system. In both cases, there is no need to clear the IOMMU translation data structures for devices.

On the initialization path, the device probing only happens after the IOMMU is initialized successfully, hence there're no translation data structures.

Quoted 9 lines
Quoted 3 lines

Out of curiosity. With kexec the IOMMU may contain stale mappings from the old kernel. Then is it meaningful to disable IOMMU after the new kernel fails to initialize it properly?

For kexec kernel, if the IOMMU is detected to be pre-enabled, the IOMMU driver will try to copy tables from the old kernel. If copying table fails, the IOMMU driver will disable IOMMU and do the normal initialization.

Quoted 4 lines

What about an error occurred after copying table in the initialization path? The new kernel will be in a state assuming iommu is disabled but it is still enabled using an old mapping for certain devices...

If copying table failed, the translation will be disabled and a clean root table will be used.

if (translation_pre_enabled(iommu)) {
pr_info("Translation already enabled - trying to copy
translation structures\n");

ret = copy_translation_tables(iommu);
if (ret) {
/*
* We found the IOMMU with translation
* enabled - but failed to copy over the
* old root-entry table. Try to proceed
* by disabling translation now and
* allocating a clean root-entry table.
* This might cause DMAR faults, but
* probably the dump will still succeed.
*/
pr_err("Failed to copy translation tables from previous
kernel for %s\n",

                       iommu->name);
                iommu_disable_translation(iommu);
                clear_translation_pre_enabled(iommu);
        } else {
                pr_info("Copied translation tables from previous kernel 
for %s\n",
                        iommu->name);
        }
}

Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

#13 lore raw

RE: [PATCH v2 03/12] iommu/vt-d: Remove clearing translation data in disable_dmar_iommu()

Quoted 53 lines

From: Baolu Lu <[email protected]>
Sent: Wednesday, June 15, 2022 9:10 PM

Quoted 24 lines · Kevin
Quoted 14 lines

From: Baolu Lu <[email protected]>
Sent: Tuesday, June 14, 2022 3:21 PM

Quoted 10 lines · Kevin
Quoted 10 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:51 AM

The disable_dmar_iommu() is called when IOMMU initialization fails or the IOMMU is hot-removed from the system. In both cases, there is no need to clear the IOMMU translation data structures for devices.

On the initialization path, the device probing only happens after the IOMMU is initialized successfully, hence there're no translation data structures.

Quoted 9 lines
Quoted 3 lines

Out of curiosity. With kexec the IOMMU may contain stale mappings from the old kernel. Then is it meaningful to disable IOMMU after the new kernel fails to initialize it properly?

For kexec kernel, if the IOMMU is detected to be pre-enabled, the IOMMU driver will try to copy tables from the old kernel. If copying table fails, the IOMMU driver will disable IOMMU and do the normal initialization.

Quoted 4 lines

What about an error occurred after copying table in the initialization path? The new kernel will be in a state assuming iommu is disabled but it is still enabled using an old mapping for certain devices...

If copying table failed, the translation will be disabled and a clean root table will be used.

if (translation_pre_enabled(iommu)) {
pr_info("Translation already enabled - trying to copy
translation structures\n");

ret = copy_translation_tables(iommu);
if (ret) {
/*
* We found the IOMMU with translation
* enabled - but failed to copy over the
* old root-entry table. Try to proceed
* by disabling translation now and
* allocating a clean root-entry table.
* This might cause DMAR faults, but
* probably the dump will still succeed.
*/
pr_err("Failed to copy translation tables from previous
kernel for %s\n",

Quoted 10 lines
                        iommu->name);
                 iommu_disable_translation(iommu);
                 clear_translation_pre_enabled(iommu);
         } else {
                 pr_info("Copied translation tables from previous kernel
for %s\n",
                         iommu->name);
         }
}

I meant copying table succeeds but another error occurs in the remaining path of initialization...

author maintainer
#14 lore raw

PATCH v2 4/12 iommu/vt-d: Use pci_get_domain_bus_and_slot() in pgtable_walk()

In reply to Lu Baolu

Use pci_get_domain_bus_and_slot() instead of searching the global list to retrieve the pci device pointer. This also removes the global device_domain_list as there isn't any consumer anymore.

---
 drivers/iommu/intel/iommu.h |  1 -
 drivers/iommu/intel/iommu.c | 33 ++++++---------------------------
 2 files changed, 6 insertions(+), 28 deletions(-)
drivers/iommu/intel/iommu.h +0 -1
index 2f4a5b9509c0..6724703d573b 100644
@@ -609,7 +609,6 @@ struct intel_iommu {
 /* PCI domain-device relationship */
 struct device_domain_info {
 	struct list_head link;	/* link to domain siblings */
-	struct list_head global; /* link to global list */
 	struct list_head table;	/* link to pasid table */
 	u32 segment;		/* PCI segment number */
 	u8 bus;			/* PCI bus number */
drivers/iommu/intel/iommu.c +6 -27
index 695aed474e5d..839bb5f3e000 100644
@@ -131,8 +131,6 @@ static struct intel_iommu **g_iommus;
 
 static void __init check_tylersburg_isoch(void);
 static int rwbf_quirk;
-static inline struct device_domain_info *
-dmar_search_domain_by_dev_info(int segment, int bus, int devfn);
 
 /*
  * set to 1 to panic kernel if can't successfully enable VT-d
@@ -315,7 +313,6 @@ static int iommu_skip_te_disable;
 #define IDENTMAP_AZALIA		4
 
 static DEFINE_SPINLOCK(device_domain_lock);
-static LIST_HEAD(device_domain_list);
 const struct iommu_ops intel_iommu_ops;
 
 static bool translation_pre_enabled(struct intel_iommu *iommu)
@@ -845,9 +842,14 @@ static void pgtable_walk(struct intel_iommu *iommu, unsigned long pfn, u8 bus, u
 	struct device_domain_info *info;
 	struct dma_pte *parent, *pte;
 	struct dmar_domain *domain;
+	struct pci_dev *pdev;
 	int offset, level;
 
-	info = dmar_search_domain_by_dev_info(iommu->segment, bus, devfn);
+	pdev = pci_get_domain_bus_and_slot(iommu->segment, bus, devfn);
+	if (!pdev)
+		return;
+
+	info = dev_iommu_priv_get(&pdev->dev);
 	if (!info || !info->domain) {
 		pr_info("device [%02x:%02x.%d] not probed\n",
 			bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
@@ -2356,19 +2358,6 @@ static void domain_remove_dev_info(struct dmar_domain *domain)
 	spin_unlock_irqrestore(&device_domain_lock, flags);
 }
 
-static inline struct device_domain_info *
-dmar_search_domain_by_dev_info(int segment, int bus, int devfn)
-{
-	struct device_domain_info *info;
-
-	list_for_each_entry(info, &device_domain_list, global)
-		if (info->segment == segment && info->bus == bus &&
-		    info->devfn == devfn)
-			return info;
-
-	return NULL;
-}
-
 static int domain_setup_first_level(struct intel_iommu *iommu,
 				    struct dmar_domain *domain,
 				    struct device *dev,
@@ -4572,7 +4561,6 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
 	struct pci_dev *pdev = dev_is_pci(dev) ? to_pci_dev(dev) : NULL;
 	struct device_domain_info *info;
 	struct intel_iommu *iommu;
-	unsigned long flags;
 	u8 bus, devfn;
 
 	iommu = device_to_iommu(dev, &bus, &devfn);
@@ -4615,10 +4603,7 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
 		}
 	}
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	list_add(&info->global, &device_domain_list);
 	dev_iommu_priv_set(dev, info);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
 
 	return &iommu->iommu;
 }
@@ -4626,15 +4611,9 @@ static struct iommu_device *intel_iommu_probe_device(struct device *dev)
 static void intel_iommu_release_device(struct device *dev)
 {
 	struct device_domain_info *info = dev_iommu_priv_get(dev);
-	unsigned long flags;
 
 	dmar_remove_one_dev_info(dev);
-
-	spin_lock_irqsave(&device_domain_lock, flags);
 	dev_iommu_priv_set(dev, NULL);
-	list_del(&info->global);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
-
 	kfree(info);
 	set_dma_ops(dev, NULL);
 }
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author maintainer
#15 lore raw

PATCH v2 5/12 iommu/vt-d: Unnecessary spinlock for root table alloc and free

In reply to Lu Baolu

The IOMMU root table is allocated and freed in the IOMMU initialization code in static boot or hot-remove paths. There's no need for a spinlock.

---
 drivers/iommu/intel/iommu.c | 18 +++++-------------
 1 file changed, 5 insertions(+), 13 deletions(-)
drivers/iommu/intel/iommu.c +5 -13
index 839bb5f3e000..b2ef8af0c3f3 100644
@@ -809,14 +809,12 @@ static int device_context_mapped(struct intel_iommu *iommu, u8 bus, u8 devfn)
 
 static void free_context_table(struct intel_iommu *iommu)
 {
-	int i;
-	unsigned long flags;
 	struct context_entry *context;
+	int i;
+
+	if (!iommu->root_entry)
+		return;
 
-	spin_lock_irqsave(&iommu->lock, flags);
-	if (!iommu->root_entry) {
-		goto out;
-	}
 	for (i = 0; i < ROOT_ENTRY_NR; i++) {
 		context = iommu_context_addr(iommu, i, 0, 0);
 		if (context)
@@ -828,12 +826,10 @@ static void free_context_table(struct intel_iommu *iommu)
 		context = iommu_context_addr(iommu, i, 0x80, 0);
 		if (context)
 			free_pgtable_page(context);
-
 	}
+
 	free_pgtable_page(iommu->root_entry);
 	iommu->root_entry = NULL;
-out:
-	spin_unlock_irqrestore(&iommu->lock, flags);
 }
 
 #ifdef CONFIG_DMAR_DEBUG
@@ -1232,7 +1228,6 @@ static void domain_unmap(struct dmar_domain *domain, unsigned long start_pfn,
 static int iommu_alloc_root_entry(struct intel_iommu *iommu)
 {
 	struct root_entry *root;
-	unsigned long flags;
 
 	root = (struct root_entry *)alloc_pgtable_page(iommu->node);
 	if (!root) {
@@ -1242,10 +1237,7 @@ static int iommu_alloc_root_entry(struct intel_iommu *iommu)
 	}
 
 	__iommu_flush_cache(iommu, root, ROOT_SIZE);
-
-	spin_lock_irqsave(&iommu->lock, flags);
 	iommu->root_entry = root;
-	spin_unlock_irqrestore(&iommu->lock, flags);
 
 	return 0;
 }
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author maintainer
#16 lore raw

PATCH v2 6/12 iommu/vt-d: Acquiring lock in domain ID allocation helpers

In reply to Lu Baolu

The iommu->lock is used to protect the per-IOMMU domain ID resource. Moving the lock into the ID alloc/free helpers makes the code more compact. At the same time, the device_domain_lock is irrelevant to the domain ID resource, remove its assertion as well.

On the other hand, the iommu->lock is never used in interrupt context, there's no need to use the irqsave variant of the spinlock calls.

---
 drivers/iommu/intel/iommu.c | 25 +++++++++----------------
 1 file changed, 9 insertions(+), 16 deletions(-)
drivers/iommu/intel/iommu.c +9 -16
index b2ef8af0c3f3..8fdaa01ef10d 100644
@@ -1782,16 +1782,13 @@ static struct dmar_domain *alloc_domain(unsigned int type)
 	return domain;
 }
 
-/* Must be called with iommu->lock */
 static int domain_attach_iommu(struct dmar_domain *domain,
 			       struct intel_iommu *iommu)
 {
 	unsigned long ndomains;
-	int num;
-
-	assert_spin_locked(&device_domain_lock);
-	assert_spin_locked(&iommu->lock);
+	int num, ret = 0;
 
+	spin_lock(&iommu->lock);
 	domain->iommu_refcnt[iommu->seq_id] += 1;
 	if (domain->iommu_refcnt[iommu->seq_id] == 1) {
 		ndomains = cap_ndoms(iommu->cap);
@@ -1800,7 +1797,8 @@ static int domain_attach_iommu(struct dmar_domain *domain,
 		if (num >= ndomains) {
 			pr_err("%s: No free domain ids\n", iommu->name);
 			domain->iommu_refcnt[iommu->seq_id] -= 1;
-			return -ENOSPC;
+			ret = -ENOSPC;
+			goto out_unlock;
 		}
 
 		set_bit(num, iommu->domain_ids);
@@ -1809,7 +1807,9 @@ static int domain_attach_iommu(struct dmar_domain *domain,
 		domain_update_iommu_cap(domain);
 	}
 
-	return 0;
+out_unlock:
+	spin_unlock(&iommu->lock);
+	return ret;
 }
 
 static void domain_detach_iommu(struct dmar_domain *domain,
@@ -1817,9 +1817,7 @@ static void domain_detach_iommu(struct dmar_domain *domain,
 {
 	int num;
 
-	assert_spin_locked(&device_domain_lock);
-	assert_spin_locked(&iommu->lock);
-
+	spin_lock(&iommu->lock);
 	domain->iommu_refcnt[iommu->seq_id] -= 1;
 	if (domain->iommu_refcnt[iommu->seq_id] == 0) {
 		num = domain->iommu_did[iommu->seq_id];
@@ -1827,6 +1825,7 @@ static void domain_detach_iommu(struct dmar_domain *domain,
 		domain_update_iommu_cap(domain);
 		domain->iommu_did[iommu->seq_id] = 0;
 	}
+	spin_unlock(&iommu->lock);
 }
 
 static inline int guestwidth_to_adjustwidth(int gaw)
@@ -2479,9 +2478,7 @@ static int domain_add_dev_info(struct dmar_domain *domain, struct device *dev)
 
 	spin_lock_irqsave(&device_domain_lock, flags);
 	info->domain = domain;
-	spin_lock(&iommu->lock);
 	ret = domain_attach_iommu(domain, iommu);
-	spin_unlock(&iommu->lock);
 	if (ret) {
 		spin_unlock_irqrestore(&device_domain_lock, flags);
 		return ret;
@@ -4166,7 +4163,6 @@ static void __dmar_remove_one_dev_info(struct device_domain_info *info)
 {
 	struct dmar_domain *domain;
 	struct intel_iommu *iommu;
-	unsigned long flags;
 
 	assert_spin_locked(&device_domain_lock);
 
@@ -4187,10 +4183,7 @@ static void __dmar_remove_one_dev_info(struct device_domain_info *info)
 	}
 
 	list_del(&info->link);
-
-	spin_lock_irqsave(&iommu->lock, flags);
 	domain_detach_iommu(domain, iommu);
-	spin_unlock_irqrestore(&iommu->lock, flags);
 }
 
 static void dmar_remove_one_dev_info(struct device *dev)
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#17 lore raw

RE: [PATCH v2 06/12] iommu/vt-d: Acquiring lock in domain ID allocation helpers

Quoted 10 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

The iommu->lock is used to protect the per-IOMMU domain ID resource. Moving the lock into the ID alloc/free helpers makes the code more compact. At the same time, the device_domain_lock is irrelevant to the domain ID resource, remove its assertion as well.

On the other hand, the iommu->lock is never used in interrupt context, there's no need to use the irqsave variant of the spinlock calls.

I still prefer to separating reduction of lock ranges from changing irqsave. Locking is tricky. From bisect p.o.v. it will be a lot easier if we just change one logic in one patch.

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#18 lore raw

Re: [PATCH v2 06/12] iommu/vt-d: Acquiring lock in domain ID allocation helpers

Quoted 8 lines · Kevin
Quoted 8 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

The iommu->lock is used to protect the per-IOMMU domain ID resource. Moving the lock into the ID alloc/free helpers makes the code more compact. At the same time, the device_domain_lock is irrelevant to the domain ID resource, remove its assertion as well.

Quoted 7 lines
Quoted 2 lines

On the other hand, the iommu->lock is never used in interrupt context, there's no need to use the irqsave variant of the spinlock calls.

I still prefer to separating reduction of lock ranges from changing irqsave. Locking is tricky. From bisect p.o.v. it will be a lot easier if we just change one logic in one patch.

Fair enough. I will do this in the next version.

Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#19 lore raw

PATCH v2 7/12 iommu/vt-d: Acquiring lock in pasid manipulation helpers

In reply to Lu Baolu

The iommu->lock is used to protect the per-IOMMU pasid directory table and pasid table. Move the spinlock acquisition/release into the helpers to make the code self-contained. Again, the iommu->lock is never used in interrupt contexts, hence there's no need to disable the interrupts when holding the lock.

---
 drivers/iommu/intel/iommu.c |   2 -
 drivers/iommu/intel/pasid.c | 108 +++++++++++++++++++-----------------
 drivers/iommu/intel/svm.c   |   5 +-
 3 files changed, 57 insertions(+), 58 deletions(-)
drivers/iommu/intel/iommu.c +0 -2
index 8fdaa01ef10d..12cd12fc86a4 100644
@@ -2496,7 +2496,6 @@ static int domain_add_dev_info(struct dmar_domain *domain, struct device *dev)
 		}
 
 		/* Setup the PASID entry for requests without PASID: */
-		spin_lock_irqsave(&iommu->lock, flags);
 		if (hw_pass_through && domain_type_is_si(domain))
 			ret = intel_pasid_setup_pass_through(iommu, domain,
 					dev, PASID_RID2PASID);
@@ -2506,7 +2505,6 @@ static int domain_add_dev_info(struct dmar_domain *domain, struct device *dev)
 		else
 			ret = intel_pasid_setup_second_level(iommu, domain,
 					dev, PASID_RID2PASID);
-		spin_unlock_irqrestore(&iommu->lock, flags);
 		if (ret) {
 			dev_err(dev, "Setup RID2PASID failed\n");
 			dmar_remove_one_dev_info(dev);
drivers/iommu/intel/pasid.c +56 -52

181 lines not on this page · raw message

drivers/iommu/intel/svm.c +1 -4
index 580713aa9e07..64072e628bbd 100644
@@ -328,9 +328,9 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu,
 					   unsigned int flags)
 {
 	struct device_domain_info *info = dev_iommu_priv_get(dev);
-	unsigned long iflags, sflags;
 	struct intel_svm_dev *sdev;
 	struct intel_svm *svm;
+	unsigned long sflags;
 	int ret = 0;
 
 	svm = pasid_private_find(mm->pasid);
@@ -394,11 +394,8 @@ static struct iommu_sva *intel_svm_bind_mm(struct intel_iommu *iommu,
 	sflags = (flags & SVM_FLAG_SUPERVISOR_MODE) ?
 			PASID_FLAG_SUPERVISOR_MODE : 0;
 	sflags |= cpu_feature_enabled(X86_FEATURE_LA57) ? PASID_FLAG_FL5LP : 0;
-	spin_lock_irqsave(&iommu->lock, iflags);
 	ret = intel_pasid_setup_first_level(iommu, dev, mm->pgd, mm->pasid,
 					    FLPT_DEFAULT_DID, sflags);
-	spin_unlock_irqrestore(&iommu->lock, iflags);
-
 	if (ret)
 		goto free_sdev;
 
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
author maintainer
#20 lore raw

PATCH v2 8/12 iommu/vt-d: Replace spin_lock_irqsave() with spin_lock()

In reply to Lu Baolu

The iommu->lock is used to protect changes in root/context/pasid tables and domain ID allocation. There's no use case to change these resources in any interrupt context. Hence there's no need to disable interrupts when helding the spinlock.

---
 drivers/iommu/intel/debugfs.c |  6 ++----
 drivers/iommu/intel/iommu.c   | 17 +++++++----------
 2 files changed, 9 insertions(+), 14 deletions(-)
drivers/iommu/intel/debugfs.c +2 -4
index 5ebfe32265d5..104f9d77b3f0 100644
@@ -263,10 +263,9 @@ static void ctx_tbl_walk(struct seq_file *m, struct intel_iommu *iommu, u16 bus)
 
 static void root_tbl_walk(struct seq_file *m, struct intel_iommu *iommu)
 {
-	unsigned long flags;
 	u16 bus;
 
-	spin_lock_irqsave(&iommu->lock, flags);
+	spin_lock(&iommu->lock);
 	seq_printf(m, "IOMMU %s: Root Table Address: 0x%llx\n", iommu->name,
 		   (u64)virt_to_phys(iommu->root_entry));
 	seq_puts(m, "B.D.F\tRoot_entry\t\t\t\tContext_entry\t\t\t\tPASID\tPASID_table_entry\n");
@@ -278,8 +277,7 @@ static void root_tbl_walk(struct seq_file *m, struct intel_iommu *iommu)
 	 */
 	for (bus = 0; bus < 256; bus++)
 		ctx_tbl_walk(m, iommu, bus);
-
-	spin_unlock_irqrestore(&iommu->lock, flags);
+	spin_unlock(&iommu->lock);
 }
 
 static int dmar_translation_struct_show(struct seq_file *m, void *unused)
drivers/iommu/intel/iommu.c +7 -10
index 12cd12fc86a4..aec33599c7f7 100644
@@ -797,13 +797,12 @@ static int device_context_mapped(struct intel_iommu *iommu, u8 bus, u8 devfn)
 {
 	struct context_entry *context;
 	int ret = 0;
-	unsigned long flags;
 
-	spin_lock_irqsave(&iommu->lock, flags);
+	spin_lock(&iommu->lock);
 	context = iommu_context_addr(iommu, bus, devfn, 0);
 	if (context)
 		ret = context_present(context);
-	spin_unlock_irqrestore(&iommu->lock, flags);
+	spin_unlock(&iommu->lock);
 	return ret;
 }
 
@@ -2295,16 +2294,15 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8
 {
 	struct intel_iommu *iommu = info->iommu;
 	struct context_entry *context;
-	unsigned long flags;
 	u16 did_old;
 
 	if (!iommu)
 		return;
 
-	spin_lock_irqsave(&iommu->lock, flags);
+	spin_lock(&iommu->lock);
 	context = iommu_context_addr(iommu, bus, devfn, 0);
 	if (!context) {
-		spin_unlock_irqrestore(&iommu->lock, flags);
+		spin_unlock(&iommu->lock);
 		return;
 	}
 
@@ -2319,7 +2317,7 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8
 
 	context_clear_entry(context);
 	__iommu_flush_cache(iommu, context, sizeof(*context));
-	spin_unlock_irqrestore(&iommu->lock, flags);
+	spin_unlock(&iommu->lock);
 	iommu->flush.flush_context(iommu,
 				   did_old,
 				   (((u16)bus) << 8) | devfn,
@@ -2772,7 +2770,6 @@ static int copy_translation_tables(struct intel_iommu *iommu)
 	struct root_entry *old_rt;
 	phys_addr_t old_rt_phys;
 	int ctxt_table_entries;
-	unsigned long flags;
 	u64 rtaddr_reg;
 	int bus, ret;
 	bool new_ext, ext;
@@ -2815,7 +2812,7 @@ static int copy_translation_tables(struct intel_iommu *iommu)
 		}
 	}
 
-	spin_lock_irqsave(&iommu->lock, flags);
+	spin_lock(&iommu->lock);
 
 	/* Context tables are copied, now write them to the root_entry table */
 	for (bus = 0; bus < 256; bus++) {
@@ -2834,7 +2831,7 @@ static int copy_translation_tables(struct intel_iommu *iommu)
 		iommu->root_entry[bus].hi = val;
 	}
 
-	spin_unlock_irqrestore(&iommu->lock, flags);
+	spin_unlock(&iommu->lock);
 
 	kfree(ctxt_tbls);
 
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Reviewed-by on v2 8/12
#21 lore raw

RE: [PATCH v2 08/12] iommu/vt-d: Replace spin_lock_irqsave() with spin_lock()

Quoted 9 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

The iommu->lock is used to protect changes in root/context/pasid tables and domain ID allocation. There's no use case to change these resources in any interrupt context. Hence there's no need to disable interrupts when helding the spinlock.

with this as one-place to changing irqsave for all previous patches:

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#22 lore raw

PATCH v2 9/12 iommu/vt-d: Check device list of domain in domain free path

In reply to Lu Baolu

When the IOMMU domain is about to be freed, it should not be set on any device. Instead of silently dealing with some bug cases, it's better to trigger a warning to report and fix any potential bugs at the first time.

---
 drivers/iommu/intel/iommu.c | 19 +++----------------
 1 file changed, 3 insertions(+), 16 deletions(-)
drivers/iommu/intel/iommu.c +3 -16
index aec33599c7f7..af22690f44c8 100644
@@ -294,7 +294,6 @@ static LIST_HEAD(dmar_satc_units);
 /* bitmap for indexing intel_iommus */
 static int g_num_of_iommus;
 
-static void domain_remove_dev_info(struct dmar_domain *domain);
 static void dmar_remove_one_dev_info(struct device *dev);
 static void __dmar_remove_one_dev_info(struct device_domain_info *info);
 
@@ -1843,10 +1842,6 @@ static inline int guestwidth_to_adjustwidth(int gaw)
 
 static void domain_exit(struct dmar_domain *domain)
 {
-
-	/* Remove associated devices and clear attached or cached domains */
-	domain_remove_dev_info(domain);
-
 	if (domain->pgd) {
 		LIST_HEAD(freelist);
 
@@ -1854,6 +1849,9 @@ static void domain_exit(struct dmar_domain *domain)
 		put_pages_list(&freelist);
 	}
 
+	if (WARN_ON(!list_empty(&domain->devices)))
+		return;
+
 	kfree(domain);
 }
 
@@ -2336,17 +2334,6 @@ static void domain_context_clear_one(struct device_domain_info *info, u8 bus, u8
 	__iommu_flush_dev_iotlb(info, 0, MAX_AGAW_PFN_WIDTH);
 }
 
-static void domain_remove_dev_info(struct dmar_domain *domain)
-{
-	struct device_domain_info *info, *tmp;
-	unsigned long flags;
-
-	spin_lock_irqsave(&device_domain_lock, flags);
-	list_for_each_entry_safe(info, tmp, &domain->devices, link)
-		__dmar_remove_one_dev_info(info);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
-}
-
 static int domain_setup_first_level(struct intel_iommu *iommu,
 				    struct dmar_domain *domain,
 				    struct device *dev,
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
Reviewed-by on v2 9/12
#23 lore raw

RE: [PATCH v2 09/12] iommu/vt-d: Check device list of domain in domain free path

Quoted 9 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

When the IOMMU domain is about to be freed, it should not be set on any device. Instead of silently dealing with some bug cases, it's better to trigger a warning to report and fix any potential bugs at the first time.

author maintainer
#24 lore raw

PATCH v2 10/12 iommu/vt-d: Fold __dmar_remove_one_dev_info() into its caller

In reply to Lu Baolu

Fold __dmar_remove_one_dev_info() into dmar_remove_one_dev_info() which is its only caller. Make the spin lock critical range only cover the device list change code and remove some unnecessary checks.

---
 drivers/iommu/intel/iommu.c | 34 +++++++++-------------------------
 1 file changed, 9 insertions(+), 25 deletions(-)
drivers/iommu/intel/iommu.c +9 -25
index af22690f44c8..8345e0c0824c 100644
@@ -295,7 +295,6 @@ static LIST_HEAD(dmar_satc_units);
 static int g_num_of_iommus;
 
 static void dmar_remove_one_dev_info(struct device *dev);
-static void __dmar_remove_one_dev_info(struct device_domain_info *info);
 
 int dmar_disabled = !IS_ENABLED(CONFIG_INTEL_IOMMU_DEFAULT_ON);
 int intel_iommu_sm = IS_ENABLED(CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON);
@@ -4141,20 +4140,14 @@ static void domain_context_clear(struct device_domain_info *info)
 			       &domain_context_clear_one_cb, info);
 }
 
-static void __dmar_remove_one_dev_info(struct device_domain_info *info)
+static void dmar_remove_one_dev_info(struct device *dev)
 {
-	struct dmar_domain *domain;
-	struct intel_iommu *iommu;
-
-	assert_spin_locked(&device_domain_lock);
-
-	if (WARN_ON(!info))
-		return;
-
-	iommu = info->iommu;
-	domain = info->domain;
+	struct device_domain_info *info = dev_iommu_priv_get(dev);
+	struct dmar_domain *domain = info->domain;
+	struct intel_iommu *iommu = info->iommu;
+	unsigned long flags;
 
-	if (info->dev && !dev_is_real_dma_subdevice(info->dev)) {
+	if (!dev_is_real_dma_subdevice(info->dev)) {
 		if (dev_is_pci(info->dev) && sm_supported(iommu))
 			intel_pasid_tear_down_entry(iommu, info->dev,
 					PASID_RID2PASID, false);
@@ -4164,20 +4157,11 @@ static void __dmar_remove_one_dev_info(struct device_domain_info *info)
 		intel_pasid_free_table(info->dev);
 	}
 
-	list_del(&info->link);
-	domain_detach_iommu(domain, iommu);
-}
-
-static void dmar_remove_one_dev_info(struct device *dev)
-{
-	struct device_domain_info *info;
-	unsigned long flags;
-
 	spin_lock_irqsave(&device_domain_lock, flags);
-	info = dev_iommu_priv_get(dev);
-	if (info)
-		__dmar_remove_one_dev_info(info);
+	list_del(&info->link);
 	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	domain_detach_iommu(domain, iommu);
 }
 
 static int md_domain_init(struct dmar_domain *domain, int guest_width)
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#25 lore raw

RE: [PATCH v2 10/12] iommu/vt-d: Fold __dmar_remove_one_dev_info() into its caller

Quoted 36 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

Fold __dmar_remove_one_dev_info() into dmar_remove_one_dev_info() which
is its only caller. Make the spin lock critical range only cover the device list change code and remove some unnecessary checks.

---
 drivers/iommu/intel/iommu.c | 34 +++++++++-------------------------
 1 file changed, 9 insertions(+), 25 deletions(-)
drivers/iommu/intel/iommu.c +0 -1
index af22690f44c8..8345e0c0824c 100644
@@ -295,7 +295,6 @@ static LIST_HEAD(dmar_satc_units);
 static int g_num_of_iommus;
 
 static void dmar_remove_one_dev_info(struct device *dev);
-static void __dmar_remove_one_dev_info(struct device_domain_info *info);
 
 int dmar_disabled = !IS_ENABLED(CONFIG_INTEL_IOMMU_DEFAULT_ON);
 int intel_iommu_sm =

IS_ENABLED(CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON);

drivers/iommu/intel/iommu.c +1 -3
@@ -4141,20 +4140,14 @@ static void domain_context_clear(struct
 device_domain_info *info)
 			       &domain_context_clear_one_cb, info);
 }
 
-static void __dmar_remove_one_dev_info(struct device_domain_info *info)
+static void dmar_remove_one_dev_info(struct device *dev)
 {
-	struct dmar_domain *domain;
-	struct intel_iommu *iommu;

drivers/iommu/intel/iommu.c static void domain_context_clear(struct · line 4147

Quoted 10 lines
drivers/iommu/intel/iommu.c +2 -8
-
-	assert_spin_locked(&device_domain_lock);
-
-	if (WARN_ON(!info))
-		return;
-
-	iommu = info->iommu;
-	domain = info->domain;
+	struct device_domain_info *info = dev_iommu_priv_get(dev);
+	struct dmar_domain *domain = info->domain;

this local variable is not required as there is just one reference to info->domain.

btw I didn't see info->domain is cleared in this path. Is it correct?

Quoted 32 lines
drivers/iommu/intel/iommu.c +3 -1
+	struct intel_iommu *iommu = info->iommu;
+	unsigned long flags;
 
-	if (info->dev && !dev_is_real_dma_subdevice(info->dev)) {
+	if (!dev_is_real_dma_subdevice(info->dev)) {
 		if (dev_is_pci(info->dev) && sm_supported(iommu))
 			intel_pasid_tear_down_entry(iommu, info->dev,
drivers/iommu/intel/iommu.c +3 -12
 					PASID_RID2PASID, false);
@@ -4164,20 +4157,11 @@ static void
 __dmar_remove_one_dev_info(struct device_domain_info *info)
 		intel_pasid_free_table(info->dev);
 	}
 
-	list_del(&info->link);
-	domain_detach_iommu(domain, iommu);
-}
-
-static void dmar_remove_one_dev_info(struct device *dev)
-{
-	struct device_domain_info *info;
-	unsigned long flags;
-
 	spin_lock_irqsave(&device_domain_lock, flags);
-	info = dev_iommu_priv_get(dev);
-	if (info)
-		__dmar_remove_one_dev_info(info);
+	list_del(&info->link);
 	spin_unlock_irqrestore(&device_domain_lock, flags);
+
+	domain_detach_iommu(domain, iommu);
 }
 
Quoted 3 lines

static int md_domain_init(struct dmar_domain *domain, int guest_width)
--
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#26 lore raw

Re: [PATCH v2 10/12] iommu/vt-d: Fold __dmar_remove_one_dev_info() into its caller

Quoted 40 lines · Kevin
Quoted 40 lines

From: Lu Baolu<[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

Fold __dmar_remove_one_dev_info() into dmar_remove_one_dev_info() which
is its only caller. Make the spin lock critical range only cover the device list change code and remove some unnecessary checks.

---
  drivers/iommu/intel/iommu.c | 34 +++++++++-------------------------
  1 file changed, 9 insertions(+), 25 deletions(-)
drivers/iommu/intel/iommu.c +0 -1
index af22690f44c8..8345e0c0824c 100644
@@ -295,7 +295,6 @@ static LIST_HEAD(dmar_satc_units);
  static int g_num_of_iommus;
 
  static void dmar_remove_one_dev_info(struct device *dev);
-static void __dmar_remove_one_dev_info(struct device_domain_info *info);
 
  int dmar_disabled = !IS_ENABLED(CONFIG_INTEL_IOMMU_DEFAULT_ON);
  int intel_iommu_sm =

IS_ENABLED(CONFIG_INTEL_IOMMU_SCALABLE_MODE_DEFAULT_ON);

drivers/iommu/intel/iommu.c +1 -7
@@ -4141,20 +4140,14 @@ static void domain_context_clear(struct
 device_domain_info *info)
  			       &domain_context_clear_one_cb, info);
  }
 
-static void __dmar_remove_one_dev_info(struct device_domain_info *info)
+static void dmar_remove_one_dev_info(struct device *dev)
  {
-	struct dmar_domain *domain;
-	struct intel_iommu *iommu;
-
-	assert_spin_locked(&device_domain_lock);
-
-	if (WARN_ON(!info))
Quoted 8 lines

drivers/iommu/intel/iommu.c static void domain_context_clear(struct device_domain_info *info) · line 4146

Quoted 6 lines
drivers/iommu/intel/iommu.c +2 -4
-		return;
-
-	iommu = info->iommu;
-	domain = info->domain;
+	struct device_domain_info *info = dev_iommu_priv_get(dev);
+	struct dmar_domain *domain = info->domain;

this local variable is not required as there is just one reference to info->domain.

Yes. It could be removed and use info->domain directly.

Quoted 3 lines

btw I didn't see info->domain is cleared in this path. Is it correct?

It's better to clear here. I will make this change in my in-process blocking domain series.

But it doesn't cause any real problems because the Intel IOMMU driver supports default domain, hence the logic here is info->domain is replaced, but not cleared.

Best regards,
baolu
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#27 lore raw

PATCH v2 11/12 iommu/vt-d: Use device_domain_lock accurately

In reply to Lu Baolu

The device_domain_lock is used to protect the device tracking list of a domain. Remove unnecessary spin_lock/unlock()'s and move the necessary ones around the list access.

---
 drivers/iommu/intel/iommu.c | 68 +++++++++++++++----------------------
 1 file changed, 27 insertions(+), 41 deletions(-)
drivers/iommu/intel/iommu.c +27 -41
index 8345e0c0824c..aa3dea1c9f13 100644
@@ -534,16 +534,10 @@ static int domain_update_device_node(struct dmar_domain *domain)
 {
 	struct device_domain_info *info;
 	int nid = NUMA_NO_NODE;
+	unsigned long flags;
 
-	assert_spin_locked(&device_domain_lock);
-
-	if (list_empty(&domain->devices))
-		return NUMA_NO_NODE;
-
+	spin_lock_irqsave(&device_domain_lock, flags);
 	list_for_each_entry(info, &domain->devices, link) {
-		if (!info->dev)
-			continue;
-
 		/*
 		 * There could possibly be multiple device numa nodes as devices
 		 * within the same domain may sit behind different IOMMUs. There
@@ -554,6 +548,7 @@ static int domain_update_device_node(struct dmar_domain *domain)
 		if (nid != NUMA_NO_NODE)
 			break;
 	}
+	spin_unlock_irqrestore(&device_domain_lock, flags);
 
 	return nid;
 }
@@ -1376,49 +1371,50 @@ static void __iommu_flush_iotlb(struct intel_iommu *iommu, u16 did,
 }
 
 static struct device_domain_info *
-iommu_support_dev_iotlb (struct dmar_domain *domain, struct intel_iommu *iommu,
-			 u8 bus, u8 devfn)
+iommu_support_dev_iotlb(struct dmar_domain *domain, struct intel_iommu *iommu,
+			u8 bus, u8 devfn)
 {
-	struct device_domain_info *info;
-
-	assert_spin_locked(&device_domain_lock);
+	struct device_domain_info *info = NULL, *tmp;
+	unsigned long flags;
 
 	if (!iommu->qi)
 		return NULL;
 
-	list_for_each_entry(info, &domain->devices, link)
-		if (info->iommu == iommu && info->bus == bus &&
-		    info->devfn == devfn) {
-			if (info->ats_supported && info->dev)
-				return info;
+	spin_lock_irqsave(&device_domain_lock, flags);
+	list_for_each_entry(tmp, &domain->devices, link) {
+		if (tmp->iommu == iommu && tmp->bus == bus &&
+		    tmp->devfn == devfn) {
+			if (tmp->ats_supported)
+				info = tmp;
 			break;
 		}
+	}
+	spin_unlock_irqrestore(&device_domain_lock, flags);
 
-	return NULL;
+	return info;
 }
 
 static void domain_update_iotlb(struct dmar_domain *domain)
 {
 	struct device_domain_info *info;
 	bool has_iotlb_device = false;
+	unsigned long flags;
 
-	assert_spin_locked(&device_domain_lock);
-
-	list_for_each_entry(info, &domain->devices, link)
+	spin_lock_irqsave(&device_domain_lock, flags);
+	list_for_each_entry(info, &domain->devices, link) {
 		if (info->ats_enabled) {
 			has_iotlb_device = true;
 			break;
 		}
-
+	}
 	domain->has_iotlb_device = has_iotlb_device;
+	spin_unlock_irqrestore(&device_domain_lock, flags);
 }
 
 static void iommu_enable_dev_iotlb(struct device_domain_info *info)
 {
 	struct pci_dev *pdev;
 
-	assert_spin_locked(&device_domain_lock);
-
 	if (!info || !dev_is_pci(info->dev))
 		return;
 
@@ -1464,8 +1460,6 @@ static void iommu_disable_dev_iotlb(struct device_domain_info *info)
 {
 	struct pci_dev *pdev;
 
-	assert_spin_locked(&device_domain_lock);
-
 	if (!dev_is_pci(info->dev))
 		return;
 
@@ -1908,11 +1902,11 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 				      struct pasid_table *table,
 				      u8 bus, u8 devfn)
 {
+	struct device_domain_info *info =
+			iommu_support_dev_iotlb(domain, iommu, bus, devfn);
 	u16 did = domain->iommu_did[iommu->seq_id];
 	int translation = CONTEXT_TT_MULTI_LEVEL;
-	struct device_domain_info *info = NULL;
 	struct context_entry *context;
-	unsigned long flags;
 	int ret;
 
 	WARN_ON(did == 0);
@@ -1925,7 +1919,6 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 
 	BUG_ON(!domain->pgd);
 
-	spin_lock_irqsave(&device_domain_lock, flags);
 	spin_lock(&iommu->lock);
 
 	ret = -ENOMEM;
@@ -1978,7 +1971,6 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 		 * Setup the Device-TLB enable bit and Page request
 		 * Enable bit:
 		 */
-		info = iommu_support_dev_iotlb(domain, iommu, bus, devfn);
 		if (info && info->ats_supported)
 			context_set_sm_dte(context);
 		if (info && info->pri_supported)
@@ -2001,7 +1993,6 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 					goto out_unlock;
 			}
 
-			info = iommu_support_dev_iotlb(domain, iommu, bus, devfn);
 			if (info && info->ats_supported)
 				translation = CONTEXT_TT_DEV_IOTLB;
 			else
@@ -2047,7 +2038,6 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 
 out_unlock:
 	spin_unlock(&iommu->lock);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
 
 	return ret;
 }
@@ -2460,15 +2450,14 @@ static int domain_add_dev_info(struct dmar_domain *domain, struct device *dev)
 	if (!iommu)
 		return -ENODEV;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	info->domain = domain;
 	ret = domain_attach_iommu(domain, iommu);
-	if (ret) {
-		spin_unlock_irqrestore(&device_domain_lock, flags);
+	if (ret)
 		return ret;
-	}
+
+	spin_lock_irqsave(&device_domain_lock, flags);
 	list_add(&info->link, &domain->devices);
 	spin_unlock_irqrestore(&device_domain_lock, flags);
+	info->domain = domain;
 
 	/* PASID table is mandatory for a PCI device in scalable mode. */
 	if (sm_supported(iommu) && !dev_is_real_dma_subdevice(dev)) {
@@ -4637,7 +4626,6 @@ int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct device *dev)
 	struct device_domain_info *info = dev_iommu_priv_get(dev);
 	struct context_entry *context;
 	struct dmar_domain *domain;
-	unsigned long flags;
 	u64 ctx_lo;
 	int ret;
 
@@ -4645,7 +4633,6 @@ int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct device *dev)
 	if (!domain)
 		return -EINVAL;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
 	spin_lock(&iommu->lock);
 
 	ret = -EINVAL;
@@ -4677,7 +4664,6 @@ int intel_iommu_enable_pasid(struct intel_iommu *iommu, struct device *dev)
 
  out:
 	spin_unlock(&iommu->lock);
-	spin_unlock_irqrestore(&device_domain_lock, flags);
 
 	return ret;
 }
Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu
#28 lore raw

RE: [PATCH v2 11/12] iommu/vt-d: Use device_domain_lock accurately

Quoted 12 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

The device_domain_lock is used to protect the device tracking list of a domain. Remove unnecessary spin_lock/unlock()'s and move the necessary ones around the list access.

---
 drivers/iommu/intel/iommu.c | 68 +++++++++++++++----------------------
 1 file changed, 27 insertions(+), 41 deletions(-)

[...]

Quoted 14 lines

+iommu_support_dev_iotlb(struct dmar_domain *domain, struct intel_iommu *iommu,

drivers/iommu/intel/iommu.c +3 -4
+			u8 bus, u8 devfn)
 {
-	struct device_domain_info *info;
-
-	assert_spin_locked(&device_domain_lock);
+	struct device_domain_info *info = NULL, *tmp;
+	unsigned long flags;
 
 	if (!iommu->qi)
 		return NULL;
 
-	list_for_each_entry(info, &domain->devices, link)

drivers/iommu/intel/iommu.c static void __iommu_flush_iotlb(struct intel_iommu *iommu, u16 did, · line 1388

Quoted 10 lines
drivers/iommu/intel/iommu.c +6 -4
-		if (info->iommu == iommu && info->bus == bus &&
-		    info->devfn == devfn) {
-			if (info->ats_supported && info->dev)
-				return info;
+	spin_lock_irqsave(&device_domain_lock, flags);
+	list_for_each_entry(tmp, &domain->devices, link) {
+		if (tmp->iommu == iommu && tmp->bus == bus &&
+		    tmp->devfn == devfn) {
+			if (tmp->ats_supported)
+				info = tmp;

Directly returning with unlock here is clearer than adding another tmp variable...

Quoted 9 lines
drivers/iommu/intel/iommu.c +0 -3
@@ -2460,15 +2450,14 @@ static int domain_add_dev_info(struct
 dmar_domain *domain, struct device *dev)
 	if (!iommu)
 		return -ENODEV;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	info->domain = domain;
 	ret = domain_attach_iommu(domain, iommu);
-	if (ret) {

drivers/iommu/intel/iommu.c static int domain_add_dev_info(struct · line 2462

Quoted 10 lines
drivers/iommu/intel/iommu.c +4 -2
-		spin_unlock_irqrestore(&device_domain_lock, flags);
+	if (ret)
 		return ret;
-	}
+
+	spin_lock_irqsave(&device_domain_lock, flags);
 	list_add(&info->link, &domain->devices);
 	spin_unlock_irqrestore(&device_domain_lock, flags);
+	info->domain = domain;
 

This is incorrect. You need fully initialize the object before adding it to the list. Otherwise a search right after above unlock and before assigning info->domain will get a wrong data
_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu

author maintainer
#29 lore raw

Re: [PATCH v2 11/12] iommu/vt-d: Use device_domain_lock accurately

Quoted 26 lines · Kevin
Quoted 12 lines

From: Lu Baolu <[email protected]>
Sent: Tuesday, June 14, 2022 10:52 AM

The device_domain_lock is used to protect the device tracking list of a domain. Remove unnecessary spin_lock/unlock()'s and move the necessary ones around the list access.

---
  drivers/iommu/intel/iommu.c | 68 +++++++++++++++----------------------
  1 file changed, 27 insertions(+), 41 deletions(-)

[...]

Quoted 13 lines

+iommu_support_dev_iotlb(struct dmar_domain *domain, struct intel_iommu *iommu,

drivers/iommu/intel/iommu.c +3 -3
+			u8 bus, u8 devfn)
  {
-	struct device_domain_info *info;
-
-	assert_spin_locked(&device_domain_lock);
+	struct device_domain_info *info = NULL, *tmp;
+	unsigned long flags;
 
  	if (!iommu->qi)
  		return NULL;
Quoted 14 lines

drivers/iommu/intel/iommu.c static void __iommu_flush_iotlb(struct intel_iommu *iommu, u16 did, · line 1388

Quoted 11 lines
drivers/iommu/intel/iommu.c +6 -5
-	list_for_each_entry(info, &domain->devices, link)
-		if (info->iommu == iommu && info->bus == bus &&
-		    info->devfn == devfn) {
-			if (info->ats_supported && info->dev)
-				return info;
+	spin_lock_irqsave(&device_domain_lock, flags);
+	list_for_each_entry(tmp, &domain->devices, link) {
+		if (tmp->iommu == iommu && tmp->bus == bus &&
+		    tmp->devfn == devfn) {
+			if (tmp->ats_supported)
+				info = tmp;

Directly returning with unlock here is clearer than adding another tmp variable...

Sure.

Quoted 16 lines
Quoted 16 lines
drivers/iommu/intel/iommu.c +3 -5
@@ -2460,15 +2450,14 @@ static int domain_add_dev_info(struct
 dmar_domain *domain, struct device *dev)
  	if (!iommu)
  		return -ENODEV;
 
-	spin_lock_irqsave(&device_domain_lock, flags);
-	info->domain = domain;
  	ret = domain_attach_iommu(domain, iommu);
-	if (ret) {
-		spin_unlock_irqrestore(&device_domain_lock, flags);
+	if (ret)
  		return ret;
-	}
+
+	spin_lock_irqsave(&device_domain_lock, flags);
Quoted 8 lines

drivers/iommu/intel/iommu.c static int domain_add_dev_info(struct dmar_domain *domain, struct device *dev) · line 2461

Quoted 4 lines
drivers/iommu/intel/iommu.c +1 -0
  	list_add(&info->link, &domain->devices);
  	spin_unlock_irqrestore(&device_domain_lock, flags);
+	info->domain = domain;
 

This is incorrect. You need fully initialize the object before adding it to the list. Otherwise a search right after above unlock and before assigning info->domain will get a wrong data

Fair enough. Will fix it in the next version.

Best regards,
baolu

author maintainer
#30 lore raw

PATCH v2 12/12 iommu/vt-d: Convert global spinlock into per domain ones

In reply to Lu Baolu

Using a global device_domain_lock spinlock to protect per-domain device tracking lists is an inefficient way, especially considering this lock is also needed in the hot paths. This optimizes the locking mechanism by converting the global lock to per domain lock.

On the other hand, as the device tracking lists are never accessed in any interrupt context, there is no need to disable interrupts while spinning. Replace irqsave variant with spinlock calls.

---
 drivers/iommu/intel/iommu.h |  1 +
 drivers/iommu/intel/iommu.c | 45 +++++++++++++++----------------------
 2 files changed, 19 insertions(+), 27 deletions(-)
drivers/iommu/intel/iommu.h +1 -0
index 6724703d573b..cc304ff09a7b 100644
@@ -541,6 +541,7 @@ struct dmar_domain {
 	u8 force_snooping : 1;		/* Create IOPTEs with snoop control */
 	u8 set_pte_snp:1;
 
+	spinlock_t lock;		/* Protect device tracking lists */
 	struct list_head devices;	/* all devices' list */
 
 	struct dma_pte	*pgd;		/* virtual address */
drivers/iommu/intel/iommu.c +18 -27

166 lines not on this page · raw message

Signature · 6 lines
2.25.1

_______________________________________________
iommu mailing list
[email protected]
https://lists.linuxfoundation.org/mailman/listinfo/iommu