Skip to contentSkip to conversation

fix vt-d hard lockup when hotplug ATS capable device

RFC PATCH Superseded Ethan Zhao 2 participants 18 messages Dec 28, 2023 – Dec 29, 2023

Subsystems (3)

v8 of 14 Superseded Latest is v14 →

lore
Share & tools
lore

Ethan Zhao

author
#0 lore raw

v8 cover fix vt-d hard lockup when hotplug ATS capable device

This patchset is used to fix vt-d hard lockup reported when surprise unplug ATS capable endpoint device connects to system via PCIe switch as following topology.

     +-[0000:15]-+-00.0  Intel Corporation Ice Lake Memory Map/VT-d 
     |           +-00.1  Intel Corporation Ice Lake Mesh 2 PCIe     
     |           +-00.2  Intel Corporation Ice Lake RAS             
     |           +-00.4  Intel Corporation Device 0b23              
     |           \-01.0-[16-1b]----00.0-[17-1b]--+-00.0-[18]----00.0 

NVIDIA Corporation Device 2324
| +-01.0-[19]----00.0
Mellanox Technologies MT2910 Family [ConnectX-7]

User brought endpoint device 19:00.0's link down by flapping it's hotplug capable slot 17:01.0 link control register, as sequence DLLSC response, pciehp_ist() will unload device driver and power it off, durning device driver is unloading an iommu device-TLB invalidation (Intel VT-d spec, or 'ATS Invalidation' in PCIe spec) request issued to that link down device, thus a long time completion/timeout waiting in interrupt context causes continuous hard lockup warnning and system hang.

Other detail, see every patch commit log.

patch [3&4] were tested by [email protected] on stable v6.7-rc4. patch [1-5] passed compiling on stable v6.7-rc6.

change log:
v9:
- unify all spelling of ATS Invalidation adhere to PCIe spec per Bjorn's suggestion.
v8:
- add a patch to break the loop for timeout device-TLB invalidation, as Bjorn said there is possibility device just no response but not gone.
v7:
- reorder patches and revise commit log per Bjorn's guide.
- other code and commit log revise per Lukas' suggestion.
- rebased to stable v6.7-rc6.
v6:
- add two patches to break out device-TLB invalidation if device is gone. v5:
- add a patch try to fix the rare case (surprise remove a device in safe removal process). not work because surprise removal handling can't re-enter when another safe removal is in process.
v4:
- move the PCI device state checking after ATS per Baolu's suggestion. v3:
- fix commit description typo.
v2:
- revise commit[1] description part according to Lukas' suggestion.
- revise commit[2] description to clarify the issue's impact. v1:
- https://lore.kernel.org/lkml/20231213034637.2603013-1-haifeng.zhao@
linux.intel.com/T/

Thanks,
Ethan

Ethan Zhao (5):
iommu/vt-d: add flush_target_dev member to struct intel_iommu and pass
device info to all ATS Invalidation functions
iommu/vt-d: break out ATS Invalidation if target device is gone
PCI: make pci_dev_is_disconnected() helper public for other drivers iommu/vt-d: don't issue ATS Invalidation request when device is
disconnected
iommu/vt-d: don't loop for timeout ATS Invalidation request
forever

 drivers/iommu/intel/dmar.c  | 14 +++++++++++++-
 drivers/iommu/intel/iommu.c |  1 +
 drivers/iommu/intel/iommu.h |  2 ++
 drivers/iommu/intel/pasid.c |  4 ++++
 drivers/iommu/intel/svm.c   |  1 +
 drivers/pci/pci.h           |  5 -----
 include/linux/pci.h         |  5 +++++
 7 files changed, 26 insertions(+), 6 deletions(-)
Signature · 1 line
2.31.1
author
#1 lore raw

RFC PATCH v9 1/5 iommu/vt-d: add flush_target_dev member to struct intel_iommu and pass device info to all ATS Invalidation functions

As iommu is a pointer member of device_domain_info, so can't play trick like container_of() to get the info and device instance for qi_submit_sync() low level function to check device status, add a flush_target_dev member to struct inte_iommu and pass dev info to all device-TLB invalidation (a.k.a ATS Invalidation) functions.

---
 drivers/iommu/intel/iommu.c | 1 +
 drivers/iommu/intel/iommu.h | 2 ++
 drivers/iommu/intel/pasid.c | 1 +
 drivers/iommu/intel/svm.c   | 1 +
 4 files changed, 5 insertions(+)
drivers/iommu/intel/iommu.c +1 -0
index 897159dba47d..c3724f1d86dc 100644
@@ -1461,6 +1461,7 @@ static void __iommu_flush_dev_iotlb(struct device_domain_info *info,
 
 	sid = info->bus << 8 | info->devfn;
 	qdep = info->ats_qdep;
+	info->iommu->flush_target_dev = info->dev;
 	qi_flush_dev_iotlb(info->iommu, sid, info->pfsid,
 			   qdep, addr, mask);
 	quirk_extra_dev_tlb_flush(info, addr, mask, IOMMU_NO_PASID, qdep);
drivers/iommu/intel/iommu.h +2 -0
index ce030c5b5772..e892c5c7560a 100644
@@ -731,6 +731,8 @@ struct intel_iommu {
 	void *perf_statistic;
 
 	struct iommu_pmu *pmu;
+
+	struct device *flush_target_dev; /* the target device TLB to be invalidated. */
 };
 
 /* PCI domain-device relationship */
drivers/iommu/intel/pasid.c +1 -0
index 74e8e4c17e81..1c87fb1b1039 100644
@@ -485,6 +485,7 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu,
 	qdep = info->ats_qdep;
 	pfsid = info->pfsid;
 
+	info->iommu->flush_target_dev = info->dev;
 	/*
 	 * When PASID 0 is used, it indicates RID2PASID(DMA request w/o PASID),
 	 * devTLB flush w/o PASID should be used. For non-zero PASID under
drivers/iommu/intel/svm.c +1 -0
index ac12f76c1212..d42a99801cdf 100644
@@ -181,6 +181,7 @@ static void __flush_svm_range_dev(struct intel_svm *svm,
 
 	qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages, ih);
 	if (info->ats_enabled) {
+		info->iommu->flush_target_dev = info->dev;
 		qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info->pfsid,
 					 svm->pasid, sdev->qdep, address,
 					 order_base_2(pages));
Signature · 1 line
2.31.1
#2 lore raw

RE: [RFC PATCH v9 1/5] iommu/vt-d: add flush_target_dev member to struct intel_iommu and pass device info to all ATS Invalidation functions

Quoted 12 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

(quoted diff) +1 -0
@@ -181,6 +181,7 @@ static void __flush_svm_range_dev(struct intel_svm
 *svm,
 
 	qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages,
 ih);
 	if (info->ats_enabled) {
+		info->iommu->flush_target_dev = info->dev;
 		qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info-
Quoted 1 line

pfsid,

drivers/iommu/intel/svm.c static void __flush_svm_range_dev(struct intel_svm *svm, · line 187

Quoted 2 lines
drivers/iommu/intel/svm.c +0 -0
 					 svm->pasid, sdev->qdep, address,
 					 order_base_2(pages));

this is wrong both in concept and function.

an iommu instance can be shared by many devices which may all have ongoing ATS invalidation requests to handle. Using a per-iommu field to store the flush target is limiting (and there is no lock protection at all).

if there is a real need of passing dev pointer to qi helpers, just change the helper to accept an explicit parameter.

author
#3 lore raw

Re: [RFC PATCH v9 1/5] iommu/vt-d: add flush_target_dev member to struct intel_iommu and pass device info to all ATS Invalidation functions

Quoted 11 lines · Kevin
Quoted 11 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

(quoted diff) +1 -0
@@ -181,6 +181,7 @@ static void __flush_svm_range_dev(struct intel_svm
 *svm,
 
  	qi_flush_piotlb(sdev->iommu, sdev->did, svm->pasid, address, pages,
 ih);
  	if (info->ats_enabled) {
+		info->iommu->flush_target_dev = info->dev;
  		qi_flush_dev_iotlb_pasid(sdev->iommu, sdev->sid, info-
Quoted 4 lines
Quoted 3 lines
Quoted 1 line

pfsid,

svm->pasid, sdev->qdep, address, order_base_2(pages));

this is wrong both in concept and function.

Yes, wrong.

Quoted 7 lines

an iommu instance can be shared by many devices which may all have ongoing ATS invalidation requests to handle. Using a per-iommu field to store the flush target is limiting (and there is no lock protection at all).

if there is a real need of passing dev pointer to qi helpers, just change the helper to accept an explicit parameter.

seems the only way is to add parameter and refactor all affected

functions.

Thanks,

Ethan

author
#4 lore raw

RFC PATCH v9 2/5 iommu/vt-d: break out ATS Invalidation if target device is gone

In reply to Ethan Zhao

For those endpoint devices connect to system via hotplug capable ports, users could request a warm reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for device to be sent and a long time completion/timeout waiting in interrupt context.

That would cause following continuous hard lockup warning and system hang

[ 4211.433662] pcieport 0000:17:01.0: pciehp: Slot(108): Link Down
[ 4211.433664] pcieport 0000:17:01.0: pciehp: Slot(108): Card not present
[ 4223.822591] NMI watchdog: Watchdog detected hard LOCKUP on cpu 144
[ 4223.822622] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded Tainted: G S
         OE    kernel version xxxx
[ 4223.822623] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822623] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822624] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48 8b
 57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6 1
0 74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39
[ 4223.822624] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822625] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX: 0000000000000005
[ 4223.822625] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI: ffff9f38401a8340
[ 4223.822625] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09: 0000000000000000
[ 4223.822626] R10: 0000000000000010 R11: 0000000000000018 R12: ffff9f384005e200
[ 4223.822626] R13: 0000000000000004 R14: 0000000000000046 R15: 0000000000000004
[ 4223.822626] FS:  0000000000000000(0000) GS:ffffa237ae400000(0000)
knlGS:0000000000000000
[ 4223.822627] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4223.822627] CR2: 00007ffe86515d80 CR3: 000002fd3000a001 CR4: 0000000000770ee0
[ 4223.822627] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 4223.822628] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[ 4223.822628] PKRU: 55555554
[ 4223.822628] Call Trace:
[ 4223.822628]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822628]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822629]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822629]  intel_iommu_release_device+0x1f/0x30
[ 4223.822629]  iommu_release_device+0x33/0x60
[ 4223.822629]  iommu_bus_notifier+0x7f/0x90
[ 4223.822630]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822630]  device_del+0x2e5/0x420
[ 4223.822630]  pci_remove_bus_device+0x70/0x110
[ 4223.822630]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822631]  pciehp_disable_slot+0x6b/0x100
[ 4223.822631]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822631]  pciehp_ist+0x176/0x180
[ 4223.822631]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822632]  irq_thread_fn+0x19/0x50
[ 4223.822632]  irq_thread+0x104/0x190
[ 4223.822632]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822632]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822633]  kthread+0x114/0x130
[ 4223.822633]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822633]  ret_from_fork+0x1f/0x30
[ 4223.822633] Kernel panic - not syncing: Hard LOCKUP
[ 4223.822634] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded Tainted: G S
         OE     kernel version xxxx
[ 4223.822634] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822634] Call Trace:
[ 4223.822634]  <NMI>
[ 4223.822635]  dump_stack+0x6d/0x88
[ 4223.822635]  panic+0x101/0x2d0
[ 4223.822635]  ? ret_from_fork+0x11/0x30
[ 4223.822635]  nmi_panic.cold.14+0xc/0xc
[ 4223.822636]  watchdog_overflow_callback.cold.8+0x6d/0x81
[ 4223.822636]  __perf_event_overflow+0x4f/0xf0
[ 4223.822636]  handle_pmi_common+0x1ef/0x290
[ 4223.822636]  ? __set_pte_vaddr+0x28/0x40
[ 4223.822637]  ? flush_tlb_one_kernel+0xa/0x20
[ 4223.822637]  ? __native_set_fixmap+0x24/0x30
[ 4223.822637]  ? ghes_copy_tofrom_phys+0x70/0x100
[ 4223.822637]  ? __ghes_peek_estatus.isra.16+0x49/0xa0
[ 4223.822637]  intel_pmu_handle_irq+0xba/0x2b0
[ 4223.822638]  perf_event_nmi_handler+0x24/0x40
[ 4223.822638]  nmi_handle+0x4d/0xf0
[ 4223.822638]  default_do_nmi+0x49/0x100
[ 4223.822638]  exc_nmi+0x134/0x180
[ 4223.822639]  end_repeat_nmi+0x16/0x67
[ 4223.822639] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822639] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48 8b
 57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6 10
 74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39
[ 4223.822640] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822640] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX: 0000000000000005
[ 4223.822640] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI: ffff9f38401a8340
[ 4223.822641] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09: 0000000000000000
[ 4223.822641] R10: 0000000000000010 R11: 0000000000000018 R12: ffff9f384005e200
[ 4223.822641] R13: 0000000000000004 R14: 0000000000000046 R15: 0000000000000004
[ 4223.822641]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  </NMI>
[ 4223.822642]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822642]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822643]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822643]  intel_iommu_release_device+0x1f/0x30
[ 4223.822643]  iommu_release_device+0x33/0x60
[ 4223.822643]  iommu_bus_notifier+0x7f/0x90
[ 4223.822644]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822644]  device_del+0x2e5/0x420
[ 4223.822644]  pci_remove_bus_device+0x70/0x110
[ 4223.822644]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822644]  pciehp_disable_slot+0x6b/0x100
[ 4223.822645]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822645]  pciehp_ist+0x176/0x180
[ 4223.822645]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822645]  irq_thread_fn+0x19/0x50
[ 4223.822646]  irq_thread+0x104/0x190
[ 4223.822646]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822646]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822646]  kthread+0x114/0x130
[ 4223.822647]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822647]  ret_from_fork+0x1f/0x30
[ 4223.822647] Kernel Offset: 0x6400000 from 0xffffffff81000000 (relocation

range: 0xffffffff80000000-0xffffffffbfffffff)

Furthermore even an in-process safe removal unplugged device could be surprise removed anytime, thus need to check the ATS Invalidation target device state to see if it is gone, and don't wait for the completion/ timeout blindly, thus avoid the up to 1min+50% (see Implementation Note in PCIe spec r6.1 sec 10.3.1) waiting and cause hard lockup or system hang.

---
 drivers/iommu/intel/dmar.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/iommu/intel/dmar.c +12 -0
index 23cb80d62a9a..76903a8bf963 100644
@@ -1347,6 +1347,7 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc,
 		   unsigned int count, unsigned long options)
 {
 	struct q_inval *qi = iommu->qi;
+	struct pci_dev *pdev = NULL;
 	s64 devtlb_start_ktime = 0;
 	s64 iotlb_start_ktime = 0;
 	s64 iec_start_ktime = 0;
@@ -1360,6 +1361,9 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc,
 	if (!qi)
 		return 0;
 
+	if (iommu->flush_target_dev && dev_is_pci(iommu->flush_target_dev))
+		pdev = to_pci_dev(iommu->flush_target_dev);
+
 	type = desc->qw0 & GENMASK_ULL(3, 0);
 
 	if ((type == QI_IOTLB_TYPE || type == QI_EIOTLB_TYPE) &&
@@ -1423,6 +1427,14 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc,
 	writel(qi->free_head << shift, iommu->reg + DMAR_IQT_REG);
 
 	while (qi->desc_status[wait_index] != QI_DONE) {
+		/*
+		 * if the device-TLB invalidation target device is gone, don't
+		 * wait anymore, it might take up to 1min+50%, causes system
+		 * hang. (see Implementation Note in PCIe spec r6.1 sec 10.3.1)
+		 */
+		if ((type == QI_DIOTLB_TYPE || type == QI_DEIOTLB_TYPE) && pdev)
+			if (!pci_device_is_present(pdev))
+				break;
 		/*
 		 * We will leave the interrupts disabled, to prevent interrupt
 		 * context to queue another cmd while a cmd is already submitted
Signature · 1 line
2.31.1
#5 lore raw

RE: [RFC PATCH v9 2/5] iommu/vt-d: break out ATS Invalidation if target device is gone

Quoted 10 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

For those endpoint devices connect to system via hotplug capable ports, users could request a warm reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for device to be sent and a long time completion/timeout waiting in interrupt context.

is above describing the behavior of safe removal or surprise removal?

Quoted 128 lines

That would cause following continuous hard lockup warning and system hang

[ 4211.433662] pcieport 0000:17:01.0: pciehp: Slot(108): Link Down
[ 4211.433664] pcieport 0000:17:01.0: pciehp: Slot(108): Card not present
[ 4223.822591] NMI watchdog: Watchdog detected hard LOCKUP on cpu 144
[ 4223.822622] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded
Tainted: G S
         OE    kernel version xxxx
[ 4223.822623] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822623] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822624] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48
8b
 57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6 1
0 74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39
[ 4223.822624] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822625] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX:
0000000000000005
[ 4223.822625] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI:
ffff9f38401a8340
[ 4223.822625] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09:
0000000000000000
[ 4223.822626] R10: 0000000000000010 R11: 0000000000000018 R12:
ffff9f384005e200
[ 4223.822626] R13: 0000000000000004 R14: 0000000000000046 R15:
0000000000000004
[ 4223.822626] FS:  0000000000000000(0000) GS:ffffa237ae400000(0000)
knlGS:0000000000000000
[ 4223.822627] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4223.822627] CR2: 00007ffe86515d80 CR3: 000002fd3000a001 CR4:
0000000000770ee0
[ 4223.822627] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 4223.822628] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7:
0000000000000400
[ 4223.822628] PKRU: 55555554
[ 4223.822628] Call Trace:
[ 4223.822628]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822628]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822629]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822629]  intel_iommu_release_device+0x1f/0x30
[ 4223.822629]  iommu_release_device+0x33/0x60
[ 4223.822629]  iommu_bus_notifier+0x7f/0x90
[ 4223.822630]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822630]  device_del+0x2e5/0x420
[ 4223.822630]  pci_remove_bus_device+0x70/0x110
[ 4223.822630]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822631]  pciehp_disable_slot+0x6b/0x100
[ 4223.822631]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822631]  pciehp_ist+0x176/0x180
[ 4223.822631]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822632]  irq_thread_fn+0x19/0x50
[ 4223.822632]  irq_thread+0x104/0x190
[ 4223.822632]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822632]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822633]  kthread+0x114/0x130
[ 4223.822633]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822633]  ret_from_fork+0x1f/0x30
[ 4223.822633] Kernel panic - not syncing: Hard LOCKUP
[ 4223.822634] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded
Tainted: G S
         OE     kernel version xxxx
[ 4223.822634] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822634] Call Trace:
[ 4223.822634]  <NMI>
[ 4223.822635]  dump_stack+0x6d/0x88
[ 4223.822635]  panic+0x101/0x2d0
[ 4223.822635]  ? ret_from_fork+0x11/0x30
[ 4223.822635]  nmi_panic.cold.14+0xc/0xc
[ 4223.822636]  watchdog_overflow_callback.cold.8+0x6d/0x81
[ 4223.822636]  __perf_event_overflow+0x4f/0xf0
[ 4223.822636]  handle_pmi_common+0x1ef/0x290
[ 4223.822636]  ? __set_pte_vaddr+0x28/0x40
[ 4223.822637]  ? flush_tlb_one_kernel+0xa/0x20
[ 4223.822637]  ? __native_set_fixmap+0x24/0x30
[ 4223.822637]  ? ghes_copy_tofrom_phys+0x70/0x100
[ 4223.822637]  ? __ghes_peek_estatus.isra.16+0x49/0xa0
[ 4223.822637]  intel_pmu_handle_irq+0xba/0x2b0
[ 4223.822638]  perf_event_nmi_handler+0x24/0x40
[ 4223.822638]  nmi_handle+0x4d/0xf0
[ 4223.822638]  default_do_nmi+0x49/0x100
[ 4223.822638]  exc_nmi+0x134/0x180
[ 4223.822639]  end_repeat_nmi+0x16/0x67
[ 4223.822639] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822639] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48

8b
57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6
10
74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39

[ 4223.822640] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822640] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX:
0000000000000005
[ 4223.822640] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI:
ffff9f38401a8340
[ 4223.822641] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09:
0000000000000000
[ 4223.822641] R10: 0000000000000010 R11: 0000000000000018 R12:
ffff9f384005e200
[ 4223.822641] R13: 0000000000000004 R14: 0000000000000046 R15:
0000000000000004
[ 4223.822641]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  </NMI>
[ 4223.822642]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822642]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822643]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822643]  intel_iommu_release_device+0x1f/0x30
[ 4223.822643]  iommu_release_device+0x33/0x60
[ 4223.822643]  iommu_bus_notifier+0x7f/0x90
[ 4223.822644]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822644]  device_del+0x2e5/0x420
[ 4223.822644]  pci_remove_bus_device+0x70/0x110
[ 4223.822644]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822644]  pciehp_disable_slot+0x6b/0x100
[ 4223.822645]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822645]  pciehp_ist+0x176/0x180
[ 4223.822645]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822645]  irq_thread_fn+0x19/0x50
[ 4223.822646]  irq_thread+0x104/0x190
[ 4223.822646]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822646]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822646]  kthread+0x114/0x130
[ 4223.822647]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822647]  ret_from_fork+0x1f/0x30
[ 4223.822647] Kernel Offset: 0x6400000 from 0xffffffff81000000 (relocation
Quoted 4 lines

range: 0xffffffff80000000-0xffffffffbfffffff)

Furthermore even an in-process safe removal unplugged device could be surprise removed anytime, thus need to check the ATS Invalidation target

I don't understand what this sentence is trying to say. what is "in-process safe removal unplugged device"? Are following words about safe removal or surprise removal?

Quoted 49 lines

device state to see if it is gone, and don't wait for the completion/ timeout blindly, thus avoid the up to 1min+50% (see Implementation Note in PCIe spec r6.1 sec 10.3.1) waiting and cause hard lockup or system hang.

---
 drivers/iommu/intel/dmar.c | 12 ++++++++++++
 1 file changed, 12 insertions(+)
drivers/iommu/intel/dmar.c +1 -0
index 23cb80d62a9a..76903a8bf963 100644
@@ -1347,6 +1347,7 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
 		   unsigned int count, unsigned long options)
 {
 	struct q_inval *qi = iommu->qi;
+	struct pci_dev *pdev = NULL;
 	s64 devtlb_start_ktime = 0;
 	s64 iotlb_start_ktime = 0;
drivers/iommu/intel/dmar.c +3 -0
 	s64 iec_start_ktime = 0;
@@ -1360,6 +1361,9 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
 	if (!qi)
 		return 0;
 
+	if (iommu->flush_target_dev && dev_is_pci(iommu-
 >flush_target_dev))
+		pdev = to_pci_dev(iommu->flush_target_dev);
+
 	type = desc->qw0 & GENMASK_ULL(3, 0);
drivers/iommu/intel/dmar.c +3 -0
 	if ((type == QI_IOTLB_TYPE || type == QI_EIOTLB_TYPE) &&
@@ -1423,6 +1427,14 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
 	writel(qi->free_head << shift, iommu->reg + DMAR_IQT_REG);
 
 	while (qi->desc_status[wait_index] != QI_DONE) {
+		/*
+		 * if the device-TLB invalidation target device is gone, don't
+		 * wait anymore, it might take up to 1min+50%, causes

system

drivers/iommu/intel/dmar.c +1 -0
+		 * hang. (see Implementation Note in PCIe spec r6.1 sec

10.3.1)

drivers/iommu/intel/dmar.c +2 -0
+		 */
+		if ((type == QI_DIOTLB_TYPE || type == QI_DEIOTLB_TYPE)

&& pdev)

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, · line 1437

Quoted 2 lines
drivers/iommu/intel/dmar.c +2 -0
+			if (!pci_device_is_present(pdev))
+				break;

I'm not sure it's the right thing to do. Such check should be put in the caller which has the device pointer and can already know it's absent to not call those cache invalidation helpers.

Quoted 7 lines
drivers/iommu/intel/dmar.c +0 -0
 		/*
 		 * We will leave the interrupts disabled, to prevent interrupt
 		 * context to queue another cmd while a cmd is already

submitted
--
2.31.1

author
#6 lore raw

Re: [RFC PATCH v9 2/5] iommu/vt-d: break out ATS Invalidation if target device is gone

Quoted 11 lines · Kevin
Quoted 10 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

For those endpoint devices connect to system via hotplug capable ports, users could request a warm reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for device to be sent and a long time completion/timeout waiting in interrupt context.

is above describing the behavior of safe removal or surprise removal?

bring the link down is a kind of surprise removal for hotplug capable

device.

Quoted 130 lines
Quoted 130 lines

That would cause following continuous hard lockup warning and system hang

[ 4211.433662] pcieport 0000:17:01.0: pciehp: Slot(108): Link Down
[ 4211.433664] pcieport 0000:17:01.0: pciehp: Slot(108): Card not present
[ 4223.822591] NMI watchdog: Watchdog detected hard LOCKUP on cpu 144
[ 4223.822622] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded
Tainted: G S
          OE    kernel version xxxx
[ 4223.822623] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822623] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822624] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48
8b
  57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6 1
0 74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39
[ 4223.822624] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822625] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX:
0000000000000005
[ 4223.822625] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI:
ffff9f38401a8340
[ 4223.822625] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09:
0000000000000000
[ 4223.822626] R10: 0000000000000010 R11: 0000000000000018 R12:
ffff9f384005e200
[ 4223.822626] R13: 0000000000000004 R14: 0000000000000046 R15:
0000000000000004
[ 4223.822626] FS:  0000000000000000(0000) GS:ffffa237ae400000(0000)
knlGS:0000000000000000
[ 4223.822627] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4223.822627] CR2: 00007ffe86515d80 CR3: 000002fd3000a001 CR4:
0000000000770ee0
[ 4223.822627] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 4223.822628] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7:
0000000000000400
[ 4223.822628] PKRU: 55555554
[ 4223.822628] Call Trace:
[ 4223.822628]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822628]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822629]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822629]  intel_iommu_release_device+0x1f/0x30
[ 4223.822629]  iommu_release_device+0x33/0x60
[ 4223.822629]  iommu_bus_notifier+0x7f/0x90
[ 4223.822630]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822630]  device_del+0x2e5/0x420
[ 4223.822630]  pci_remove_bus_device+0x70/0x110
[ 4223.822630]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822631]  pciehp_disable_slot+0x6b/0x100
[ 4223.822631]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822631]  pciehp_ist+0x176/0x180
[ 4223.822631]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822632]  irq_thread_fn+0x19/0x50
[ 4223.822632]  irq_thread+0x104/0x190
[ 4223.822632]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822632]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822633]  kthread+0x114/0x130
[ 4223.822633]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822633]  ret_from_fork+0x1f/0x30
[ 4223.822633] Kernel panic - not syncing: Hard LOCKUP
[ 4223.822634] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded
Tainted: G S
          OE     kernel version xxxx
[ 4223.822634] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822634] Call Trace:
[ 4223.822634]  <NMI>
[ 4223.822635]  dump_stack+0x6d/0x88
[ 4223.822635]  panic+0x101/0x2d0
[ 4223.822635]  ? ret_from_fork+0x11/0x30
[ 4223.822635]  nmi_panic.cold.14+0xc/0xc
[ 4223.822636]  watchdog_overflow_callback.cold.8+0x6d/0x81
[ 4223.822636]  __perf_event_overflow+0x4f/0xf0
[ 4223.822636]  handle_pmi_common+0x1ef/0x290
[ 4223.822636]  ? __set_pte_vaddr+0x28/0x40
[ 4223.822637]  ? flush_tlb_one_kernel+0xa/0x20
[ 4223.822637]  ? __native_set_fixmap+0x24/0x30
[ 4223.822637]  ? ghes_copy_tofrom_phys+0x70/0x100
[ 4223.822637]  ? __ghes_peek_estatus.isra.16+0x49/0xa0
[ 4223.822637]  intel_pmu_handle_irq+0xba/0x2b0
[ 4223.822638]  perf_event_nmi_handler+0x24/0x40
[ 4223.822638]  nmi_handle+0x4d/0xf0
[ 4223.822638]  default_do_nmi+0x49/0x100
[ 4223.822638]  exc_nmi+0x134/0x180
[ 4223.822639]  end_repeat_nmi+0x16/0x67
[ 4223.822639] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822639] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48

8b
57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6
10
74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39

[ 4223.822640] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822640] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX:
0000000000000005
[ 4223.822640] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI:
ffff9f38401a8340
[ 4223.822641] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09:
0000000000000000
[ 4223.822641] R10: 0000000000000010 R11: 0000000000000018 R12:
ffff9f384005e200
[ 4223.822641] R13: 0000000000000004 R14: 0000000000000046 R15:
0000000000000004
[ 4223.822641]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  </NMI>
[ 4223.822642]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822642]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822643]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822643]  intel_iommu_release_device+0x1f/0x30
[ 4223.822643]  iommu_release_device+0x33/0x60
[ 4223.822643]  iommu_bus_notifier+0x7f/0x90
[ 4223.822644]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822644]  device_del+0x2e5/0x420
[ 4223.822644]  pci_remove_bus_device+0x70/0x110
[ 4223.822644]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822644]  pciehp_disable_slot+0x6b/0x100
[ 4223.822645]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822645]  pciehp_ist+0x176/0x180
[ 4223.822645]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822645]  irq_thread_fn+0x19/0x50
[ 4223.822646]  irq_thread+0x104/0x190
[ 4223.822646]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822646]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822646]  kthread+0x114/0x130
[ 4223.822647]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822647]  ret_from_fork+0x1f/0x30
[ 4223.822647] Kernel Offset: 0x6400000 from 0xffffffff81000000 (relocation

range: 0xffffffff80000000-0xffffffffbfffffff)

Quoted 5 lines
Quoted 2 lines

Furthermore even an in-process safe removal unplugged device could be surprise removed anytime, thus need to check the ATS Invalidation target

I don't understand what this sentence is trying to say. what is "in-process safe removal unplugged device"? Are following words about safe removal or surprise removal?

That is to say surprise removal unplug action happens when the safe

removal is in the middle of handling process.

Quoted 40 lines
Quoted 40 lines

device state to see if it is gone, and don't wait for the completion/ timeout blindly, thus avoid the up to 1min+50% (see Implementation Note in PCIe spec r6.1 sec 10.3.1) waiting and cause hard lockup or system hang.

---
  drivers/iommu/intel/dmar.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
drivers/iommu/intel/dmar.c +1 -0
index 23cb80d62a9a..76903a8bf963 100644
@@ -1347,6 +1347,7 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
  		   unsigned int count, unsigned long options)
  {
  	struct q_inval *qi = iommu->qi;
+	struct pci_dev *pdev = NULL;
  	s64 devtlb_start_ktime = 0;
  	s64 iotlb_start_ktime = 0;
  	s64 iec_start_ktime = 0;
drivers/iommu/intel/dmar.c +3 -0
@@ -1360,6 +1361,9 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
  	if (!qi)
  		return 0;
 
+	if (iommu->flush_target_dev && dev_is_pci(iommu-
 > flush_target_dev))
+		pdev = to_pci_dev(iommu->flush_target_dev);
+
  	type = desc->qw0 & GENMASK_ULL(3, 0);
  	if ((type == QI_IOTLB_TYPE || type == QI_EIOTLB_TYPE) &&
drivers/iommu/intel/dmar.c +0 -0
@@ -1423,6 +1427,14 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
  	writel(qi->free_head << shift, iommu->reg + DMAR_IQT_REG);
 
Quoted 15 lines

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, · line 1437

Quoted 12 lines
drivers/iommu/intel/dmar.c +3 -0
  	while (qi->desc_status[wait_index] != QI_DONE) {
+		/*
+		 * if the device-TLB invalidation target device is gone, don't
+		 * wait anymore, it might take up to 1min+50%, causes

system

drivers/iommu/intel/dmar.c +1 -0
+		 * hang. (see Implementation Note in PCIe spec r6.1 sec

10.3.1)

drivers/iommu/intel/dmar.c +2 -0
+		 */
+		if ((type == QI_DIOTLB_TYPE || type == QI_DEIOTLB_TYPE)

&& pdev)

drivers/iommu/intel/dmar.c +2 -0
+			if (!pci_device_is_present(pdev))
+				break;

I'm not sure it's the right thing to do. Such check should be put in the caller which has the device pointer and can already know it's absent to not call those cache invalidation helpers.

Here is to handle such case, the invalidation request is sent, but the

device is just pulled out at that moment.

Thanks,

Ethan

Quoted 8 lines
Quoted 7 lines

/*
* We will leave the interrupts disabled, to prevent interrupt
* context to queue another cmd while a cmd is already
submitted
--
2.31.1

#7 lore raw

RE: [RFC PATCH v9 2/5] iommu/vt-d: break out ATS Invalidation if target device is gone

Quoted 14 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 9:03 PM

Quoted 10 lines · Kevin
Quoted 10 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

For those endpoint devices connect to system via hotplug capable ports, users could request a warm reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for device to be sent and a long time completion/timeout waiting in interrupt context.

Quoted 5 lines
Quoted 1 line

is above describing the behavior of safe removal or surprise removal?

bring the link down is a kind of surprise removal for hotplug capable

device.

then it's better to make it clear from beginning that this is about surprise removal in which device is removed and cannot respond to on-going ATS invalidation request incurred in the removal process.

safe removal should be immune from this problem as the device is still responsive in the whole removal process.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
[ 4223.822628] Call Trace:
[ 4223.822628]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822628]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822629]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822629]  intel_iommu_release_device+0x1f/0x30
[ 4223.822629]  iommu_release_device+0x33/0x60
[ 4223.822629]  iommu_bus_notifier+0x7f/0x90
[ 4223.822630]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822630]  device_del+0x2e5/0x420
[ 4223.822630]  pci_remove_bus_device+0x70/0x110
[ 4223.822630]  pciehp_unconfigure_device+0x7c/0x130

I'm curious why this doesn't occur earlier when the device is detached from the driver. At that point presumably the device should be detached from the DMA domain which involves ATS invalidation too.

Quoted 17 lines
Quoted 16 lines

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, · line 1437

Quoted 13 lines
  	while (qi->desc_status[wait_index] != QI_DONE) {
drivers/iommu/intel/dmar.c +3 -0
+		/*
+		 * if the device-TLB invalidation target device is gone, don't
+		 * wait anymore, it might take up to 1min+50%, causes

system

(quoted diff) +1 -0
+		 * hang. (see Implementation Note in PCIe spec r6.1 sec

10.3.1)

(quoted diff) +2 -0
+		 */
+		if ((type == QI_DIOTLB_TYPE || type == QI_DEIOTLB_TYPE)

&& pdev)

drivers/iommu/intel/dmar.c +2 -0
+			if (!pci_device_is_present(pdev))
+				break;

I'm not sure it's the right thing to do. Such check should be put in the caller which has the device pointer and can already know it's absent to not call those cache invalidation helpers.

Quoted 4 lines

Here is to handle such case, the invalidation request is sent, but the

device is just pulled out at that moment.

one problem - the caller could pass multiple descriptors while type only refers to the 1st descriptor.

btw is it an Intel specific problem? A quick glance at smmu driver suggests the same problem too:

arm_smmu_atc_inv_domain()
arm_smmu_cmdq_batch_submit()
arm_smmu_cmdq_issue_cmdlist()
arm_smmu_cmdq_poll_until_sync()
__arm_smmu_cmdq_poll_until_consumed()

/*
* Wait until the SMMU cons index passes llq->prod.
* Must be called with the cmdq lock held in some capacity.
*/
static int __arm_smmu_cmdq_poll_until_consumed(struct arm_smmu_device *smmu,
struct arm_smmu_ll_queue *llq)

is there a more general way to solve it?

author
#8 lore raw

Re: [RFC PATCH v9 2/5] iommu/vt-d: break out ATS Invalidation if target device is gone

Quoted 14 lines · Kevin
Quoted 14 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 9:03 PM

Quoted 10 lines · Kevin
Quoted 10 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

For those endpoint devices connect to system via hotplug capable ports, users could request a warm reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for device to be sent and a long time completion/timeout waiting in interrupt context.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

is above describing the behavior of safe removal or surprise removal?

bring the link down is a kind of surprise removal for hotplug capable

device.

then it's better to make it clear from beginning that this is about surprise removal in which device is removed and cannot respond to on-going ATS invalidation request incurred in the removal process.

This case, customer insisted he wasn't meant to do "surprise removal", but

did a warm reset, perhas by chance, they populated adapters in the hotplug

capable slots.

typical surprise removal doesn't include such case in my understanding.

1. pull out adapter directly

2. request power off via sysfs.

but the behaviour of pciehp (hotplug driver) is exactly the same as other

surprise removal operation, so just classify it as "surprise removal" , no

items in PCIe spec mentioned this is one typical surprise removal.

perhaps no one did surprise removal via setpci tool to access pci

config space to flap power/link state, why not just pull it out.

Quoted 3 lines

safe removal should be immune from this problem as the device is still responsive in the whole removal process.

Yup, agree.

Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
Quoted 12 lines
[ 4223.822628] Call Trace:
[ 4223.822628]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822628]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822629]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822629]  intel_iommu_release_device+0x1f/0x30
[ 4223.822629]  iommu_release_device+0x33/0x60
[ 4223.822629]  iommu_bus_notifier+0x7f/0x90
[ 4223.822630]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822630]  device_del+0x2e5/0x420
[ 4223.822630]  pci_remove_bus_device+0x70/0x110
[ 4223.822630]  pciehp_unconfigure_device+0x7c/0x130
Quoted 4 lines

I'm curious why this doesn't occur earlier when the device is detached from the driver. At that point presumably the device should be detached from the DMA domain which involves ATS invalidation too.

well, that is not weird as I know

I am sure the device driver was unloaded already before user

tries to do a warm reset to the device.

In fact, customer uses a firmware tool called "mlxfwreset"

the steps that tool executed

1. send reset command to firmware

2. stop driver

3. reset pci (via setpci , then hang here).

Thanks,

Ethan

Quoted 39 lines
Quoted 15 lines
Quoted 7 lines
Quoted 7 lines
   	while (qi->desc_status[wait_index] != QI_DONE) {
drivers/iommu/intel/dmar.c +3 -0
+		/*
+		 * if the device-TLB invalidation target device is gone, don't
+		 * wait anymore, it might take up to 1min+50%, causes

system

(quoted diff) +1 -0
+		 * hang. (see Implementation Note in PCIe spec r6.1 sec

10.3.1)

Quoted 8 lines

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, · line 1437

Quoted 5 lines
(quoted diff) +2 -0
+		 */
+		if ((type == QI_DIOTLB_TYPE || type == QI_DEIOTLB_TYPE)

&& pdev)

drivers/iommu/intel/dmar.c +2 -0
+			if (!pci_device_is_present(pdev))
+				break;

I'm not sure it's the right thing to do. Such check should be put in the caller which has the device pointer and can already know it's absent to not call those cache invalidation helpers.

Quoted 4 lines

Here is to handle such case, the invalidation request is sent, but the

device is just pulled out at that moment.

one problem - the caller could pass multiple descriptors while type only refers to the 1st descriptor.

btw is it an Intel specific problem? A quick glance at smmu driver suggests the same problem too:

arm_smmu_atc_inv_domain()
arm_smmu_cmdq_batch_submit()
arm_smmu_cmdq_issue_cmdlist()
arm_smmu_cmdq_poll_until_sync()
__arm_smmu_cmdq_poll_until_consumed()

/*
* Wait until the SMMU cons index passes llq->prod.
* Must be called with the cmdq lock held in some capacity.
*/
static int __arm_smmu_cmdq_poll_until_consumed(struct arm_smmu_device *smmu,
struct arm_smmu_ll_queue *llq)

is there a more general way to solve it?

author
#9 lore raw

Re: [RFC PATCH v9 2/5] iommu/vt-d: break out ATS Invalidation if target device is gone

In reply to Tian, Kevin
Quoted 56 lines · Kevin
Quoted 14 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 9:03 PM

Quoted 10 lines · Kevin
Quoted 10 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

For those endpoint devices connect to system via hotplug capable ports, users could request a warm reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for device to be sent and a long time completion/timeout waiting in interrupt context.

Quoted 4 lines
Quoted 1 line

is above describing the behavior of safe removal or surprise removal?

bring the link down is a kind of surprise removal for hotplug capable

device.

then it's better to make it clear from beginning that this is about surprise removal in which device is removed and cannot respond to on-going ATS invalidation request incurred in the removal process.

safe removal should be immune from this problem as the device is still responsive in the whole removal process.

Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
[ 4223.822628] Call Trace:
[ 4223.822628]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822628]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822629]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822629]  intel_iommu_release_device+0x1f/0x30
[ 4223.822629]  iommu_release_device+0x33/0x60
[ 4223.822629]  iommu_bus_notifier+0x7f/0x90
[ 4223.822630]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822630]  device_del+0x2e5/0x420
[ 4223.822630]  pci_remove_bus_device+0x70/0x110
[ 4223.822630]  pciehp_unconfigure_device+0x7c/0x130

I'm curious why this doesn't occur earlier when the device is detached from the driver. At that point presumably the device should be detached from the DMA domain which involves ATS invalidation too.

Quoted 15 lines
Quoted 15 lines

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, · line 1437

Quoted 12 lines
   	while (qi->desc_status[wait_index] != QI_DONE) {
drivers/iommu/intel/dmar.c +3 -0
+		/*
+		 * if the device-TLB invalidation target device is gone, don't
+		 * wait anymore, it might take up to 1min+50%, causes

system

(quoted diff) +1 -0
+		 * hang. (see Implementation Note in PCIe spec r6.1 sec

10.3.1)

(quoted diff) +2 -0
+		 */
+		if ((type == QI_DIOTLB_TYPE || type == QI_DEIOTLB_TYPE)

&& pdev)

drivers/iommu/intel/dmar.c +2 -0
+			if (!pci_device_is_present(pdev))
+				break;

I'm not sure it's the right thing to do. Such check should be put in the caller which has the device pointer and can already know it's absent to not call those cache invalidation helpers.

Quoted 6 lines
Quoted 4 lines

Here is to handle such case, the invalidation request is sent, but the

device is just pulled out at that moment.

one problem - the caller could pass multiple descriptors while type only refers to the 1st descriptor.

If the other invalidation request mixed together with ATS invalidation

in the descriptors passed to qi_submit_sync(), would be problem,

so far to Intel VT-d driver, I didn't see such kind of usage, perhaps

will see it later, no one could prevent that.

Quoted 10 lines

btw is it an Intel specific problem? A quick glance at smmu driver suggests the same problem too:

arm_smmu_atc_inv_domain()
arm_smmu_cmdq_batch_submit()
arm_smmu_cmdq_issue_cmdlist()
arm_smmu_cmdq_poll_until_sync()
__arm_smmu_cmdq_poll_until_consumed()

Quoted 8 lines

/*
* Wait until the SMMU cons index passes llq->prod.
* Must be called with the cmdq lock held in some capacity.
*/
static int __arm_smmu_cmdq_poll_until_consumed(struct arm_smmu_device *smmu,
struct arm_smmu_ll_queue *llq)

is there a more general way to solve it?

Surprise removal could happen anytime, depends on user,

no preparation could be done, so called 'surprise' :(

Thanks,

Ethan

author
#10 lore raw

Re: [RFC PATCH v9 2/5] iommu/vt-d: break out ATS Invalidation if target device is gone

In reply to Tian, Kevin
Quoted 141 lines · Kevin
Quoted 10 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

For those endpoint devices connect to system via hotplug capable ports, users could request a warm reset to the device by flapping device's link through setting the slot's link control register, as pciehp_ist() DLLSC interrupt sequence response, pciehp will unload the device driver and then power it off. thus cause an IOMMU device-TLB invalidation (Intel VT-d spec, or ATS Invalidation in PCIe spec r6.1) request for device to be sent and a long time completion/timeout waiting in interrupt context.

is above describing the behavior of safe removal or surprise removal?

Quoted 129 lines

That would cause following continuous hard lockup warning and system hang

[ 4211.433662] pcieport 0000:17:01.0: pciehp: Slot(108): Link Down
[ 4211.433664] pcieport 0000:17:01.0: pciehp: Slot(108): Card not present
[ 4223.822591] NMI watchdog: Watchdog detected hard LOCKUP on cpu 144
[ 4223.822622] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded
Tainted: G S
          OE    kernel version xxxx
[ 4223.822623] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822623] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822624] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48
8b
  57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6 1
0 74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39
[ 4223.822624] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822625] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX:
0000000000000005
[ 4223.822625] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI:
ffff9f38401a8340
[ 4223.822625] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09:
0000000000000000
[ 4223.822626] R10: 0000000000000010 R11: 0000000000000018 R12:
ffff9f384005e200
[ 4223.822626] R13: 0000000000000004 R14: 0000000000000046 R15:
0000000000000004
[ 4223.822626] FS:  0000000000000000(0000) GS:ffffa237ae400000(0000)
knlGS:0000000000000000
[ 4223.822627] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 4223.822627] CR2: 00007ffe86515d80 CR3: 000002fd3000a001 CR4:
0000000000770ee0
[ 4223.822627] DR0: 0000000000000000 DR1: 0000000000000000 DR2:
0000000000000000
[ 4223.822628] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7:
0000000000000400
[ 4223.822628] PKRU: 55555554
[ 4223.822628] Call Trace:
[ 4223.822628]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822628]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822629]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822629]  intel_iommu_release_device+0x1f/0x30
[ 4223.822629]  iommu_release_device+0x33/0x60
[ 4223.822629]  iommu_bus_notifier+0x7f/0x90
[ 4223.822630]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822630]  device_del+0x2e5/0x420
[ 4223.822630]  pci_remove_bus_device+0x70/0x110
[ 4223.822630]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822631]  pciehp_disable_slot+0x6b/0x100
[ 4223.822631]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822631]  pciehp_ist+0x176/0x180
[ 4223.822631]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822632]  irq_thread_fn+0x19/0x50
[ 4223.822632]  irq_thread+0x104/0x190
[ 4223.822632]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822632]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822633]  kthread+0x114/0x130
[ 4223.822633]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822633]  ret_from_fork+0x1f/0x30
[ 4223.822633] Kernel panic - not syncing: Hard LOCKUP
[ 4223.822634] CPU: 144 PID: 1422 Comm: irq/57-pciehp Kdump: loaded
Tainted: G S
          OE     kernel version xxxx
[ 4223.822634] Hardware name: vendorname xxxx 666-106,
BIOS 01.01.02.03.01 05/15/2023
[ 4223.822634] Call Trace:
[ 4223.822634]  <NMI>
[ 4223.822635]  dump_stack+0x6d/0x88
[ 4223.822635]  panic+0x101/0x2d0
[ 4223.822635]  ? ret_from_fork+0x11/0x30
[ 4223.822635]  nmi_panic.cold.14+0xc/0xc
[ 4223.822636]  watchdog_overflow_callback.cold.8+0x6d/0x81
[ 4223.822636]  __perf_event_overflow+0x4f/0xf0
[ 4223.822636]  handle_pmi_common+0x1ef/0x290
[ 4223.822636]  ? __set_pte_vaddr+0x28/0x40
[ 4223.822637]  ? flush_tlb_one_kernel+0xa/0x20
[ 4223.822637]  ? __native_set_fixmap+0x24/0x30
[ 4223.822637]  ? ghes_copy_tofrom_phys+0x70/0x100
[ 4223.822637]  ? __ghes_peek_estatus.isra.16+0x49/0xa0
[ 4223.822637]  intel_pmu_handle_irq+0xba/0x2b0
[ 4223.822638]  perf_event_nmi_handler+0x24/0x40
[ 4223.822638]  nmi_handle+0x4d/0xf0
[ 4223.822638]  default_do_nmi+0x49/0x100
[ 4223.822638]  exc_nmi+0x134/0x180
[ 4223.822639]  end_repeat_nmi+0x16/0x67
[ 4223.822639] RIP: 0010:qi_submit_sync+0x2c0/0x490
[ 4223.822639] Code: 48 be 00 00 00 00 00 08 00 00 49 85 74 24 20 0f 95 c1 48

8b
57 10 83 c1 04 83 3c 1a 03 0f 84 a2 01 00 00 49 8b 04 24 8b 70 34 <40> f6 c6
10
74 17 49 8b 04 24 8b 80 80 00 00 00 89 c2 d3 fa 41 39

[ 4223.822640] RSP: 0018:ffffc4f074f0bbb8 EFLAGS: 00000093
[ 4223.822640] RAX: ffffc4f040059000 RBX: 0000000000000014 RCX:
0000000000000005
[ 4223.822640] RDX: ffff9f3841315800 RSI: 0000000000000000 RDI:
ffff9f38401a8340
[ 4223.822641] RBP: ffff9f38401a8340 R08: ffffc4f074f0bc00 R09:
0000000000000000
[ 4223.822641] R10: 0000000000000010 R11: 0000000000000018 R12:
ffff9f384005e200
[ 4223.822641] R13: 0000000000000004 R14: 0000000000000046 R15:
0000000000000004
[ 4223.822641]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  ? qi_submit_sync+0x2c0/0x490
[ 4223.822642]  </NMI>
[ 4223.822642]  qi_flush_dev_iotlb+0xb1/0xd0
[ 4223.822642]  __dmar_remove_one_dev_info+0x224/0x250
[ 4223.822643]  dmar_remove_one_dev_info+0x3e/0x50
[ 4223.822643]  intel_iommu_release_device+0x1f/0x30
[ 4223.822643]  iommu_release_device+0x33/0x60
[ 4223.822643]  iommu_bus_notifier+0x7f/0x90
[ 4223.822644]  blocking_notifier_call_chain+0x60/0x90
[ 4223.822644]  device_del+0x2e5/0x420
[ 4223.822644]  pci_remove_bus_device+0x70/0x110
[ 4223.822644]  pciehp_unconfigure_device+0x7c/0x130
[ 4223.822644]  pciehp_disable_slot+0x6b/0x100
[ 4223.822645]  pciehp_handle_presence_or_link_change+0xd8/0x320
[ 4223.822645]  pciehp_ist+0x176/0x180
[ 4223.822645]  ? irq_finalize_oneshot.part.50+0x110/0x110
[ 4223.822645]  irq_thread_fn+0x19/0x50
[ 4223.822646]  irq_thread+0x104/0x190
[ 4223.822646]  ? irq_forced_thread_fn+0x90/0x90
[ 4223.822646]  ? irq_thread_check_affinity+0xe0/0xe0
[ 4223.822646]  kthread+0x114/0x130
[ 4223.822647]  ? __kthread_cancel_work+0x40/0x40
[ 4223.822647]  ret_from_fork+0x1f/0x30
[ 4223.822647] Kernel Offset: 0x6400000 from 0xffffffff81000000 (relocation

range: 0xffffffff80000000-0xffffffffbfffffff)

Quoted 5 lines
Quoted 2 lines

Furthermore even an in-process safe removal unplugged device could be surprise removed anytime, thus need to check the ATS Invalidation target

I don't understand what this sentence is trying to say. what is "in-process safe removal unplugged device"? Are following words about safe removal or surprise removal?

Surprise removal for hotplug device could happen anytime, even the

user pressed attention button to tell the OS he requested a "safe removal"

and OS blink the led to say "OS is handling the safe removal", the user

could pull out the device (surprise removal) that moment and not wait

the blinking led to turn off and tells him OS completed the handling.

Thanks,

Ethan

Quoted 63 lines
Quoted 49 lines

device state to see if it is gone, and don't wait for the completion/ timeout blindly, thus avoid the up to 1min+50% (see Implementation Note in PCIe spec r6.1 sec 10.3.1) waiting and cause hard lockup or system hang.

---
  drivers/iommu/intel/dmar.c | 12 ++++++++++++
  1 file changed, 12 insertions(+)
drivers/iommu/intel/dmar.c +1 -0
index 23cb80d62a9a..76903a8bf963 100644
@@ -1347,6 +1347,7 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
  		   unsigned int count, unsigned long options)
  {
  	struct q_inval *qi = iommu->qi;
+	struct pci_dev *pdev = NULL;
  	s64 devtlb_start_ktime = 0;
  	s64 iotlb_start_ktime = 0;
  	s64 iec_start_ktime = 0;
drivers/iommu/intel/dmar.c +3 -0
@@ -1360,6 +1361,9 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
  	if (!qi)
  		return 0;
 
+	if (iommu->flush_target_dev && dev_is_pci(iommu-
 > flush_target_dev))
+		pdev = to_pci_dev(iommu->flush_target_dev);
+
  	type = desc->qw0 & GENMASK_ULL(3, 0);
  	if ((type == QI_IOTLB_TYPE || type == QI_EIOTLB_TYPE) &&
drivers/iommu/intel/dmar.c +3 -0
@@ -1423,6 +1427,14 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
  	writel(qi->free_head << shift, iommu->reg + DMAR_IQT_REG);
 
  	while (qi->desc_status[wait_index] != QI_DONE) {
+		/*
+		 * if the device-TLB invalidation target device is gone, don't
+		 * wait anymore, it might take up to 1min+50%, causes

system

drivers/iommu/intel/dmar.c +1 -0
+		 * hang. (see Implementation Note in PCIe spec r6.1 sec

10.3.1)

drivers/iommu/intel/dmar.c +2 -0
+		 */
+		if ((type == QI_DIOTLB_TYPE || type == QI_DEIOTLB_TYPE)

&& pdev)

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc, · line 1437

Quoted 2 lines
drivers/iommu/intel/dmar.c +2 -0
+			if (!pci_device_is_present(pdev))
+				break;

I'm not sure it's the right thing to do. Such check should be put in the caller which has the device pointer and can already know it's absent to not call those cache invalidation helpers.

Quoted 7 lines

/*
* We will leave the interrupts disabled, to prevent interrupt
* context to queue another cmd while a cmd is already
submitted
--
2.31.1

author
#11 lore raw

RFC PATCH v9 3/5 PCI: make pci_dev_is_disconnected() helper public for other drivers

In reply to Ethan Zhao

Make pci_dev_is_disconnected() public so that it can be called from Intel VT-d driver to quickly fix/workaround the surprise removal unplug hang issue for those ATS capable devices on PCIe switch downstream hotplug capable ports.

Beside pci_device_is_present() function, this one has no config space space access, so is light enough to optimize the normal pure surprise removal and safe removal flow.

---
 drivers/pci/pci.h   | 5 -----
 include/linux/pci.h | 5 +++++
 2 files changed, 5 insertions(+), 5 deletions(-)
drivers/pci/pci.h +0 -5
index 5ecbcf041179..75fa2084492f 100644
@@ -366,11 +366,6 @@ static inline int pci_dev_set_disconnected(struct pci_dev *dev, void *unused)
 	return 0;
 }
 
-static inline bool pci_dev_is_disconnected(const struct pci_dev *dev)
-{
-	return dev->error_state == pci_channel_io_perm_failure;
-}
-
 /* pci_dev priv_flags */
 #define PCI_DEV_ADDED 0
 #define PCI_DPC_RECOVERED 1
include/linux/pci.h +5 -0
index dea043bc1e38..4779eec8b267 100644
@@ -2506,6 +2506,11 @@ static inline struct pci_dev *pcie_find_root_port(struct pci_dev *dev)
 	return NULL;
 }
 
+static inline bool pci_dev_is_disconnected(const struct pci_dev *dev)
+{
+	return dev->error_state == pci_channel_io_perm_failure;
+}
+
 void pci_request_acs(void);
 bool pci_acs_enabled(struct pci_dev *pdev, u16 acs_flags);
 bool pci_acs_path_enabled(struct pci_dev *start,
Signature · 1 line
2.31.1
author
#12 lore raw

RFC PATCH v9 4/5 iommu/vt-d: don't issue ATS Invalidation request when device is disconnected

In reply to Ethan Zhao

Except those aggressive hotplug cases - surprise remove a hotplug device while its safe removal is requested and handled in process by:

1. pull it out directly.
2. turn off its power.
3. bring the link down.
4. just died there that moment.

etc, in a word, 'gone' or 'disconnected'.

Mostly are regular normal safe removal and surprise removal unplug. these hot unplug handling process could be optimized for fix the ATS Invalidation hang issue by calling pci_dev_is_disconnected() in function devtlb_invalidation_with_pasid() to check target device state to avoid sending meaningless ATS Invalidation request to iommu when device is gone. (see IMPLEMENTATION NOTE in PCIe spec r6.1 section 10.3.1)

For safe removal, device wouldn't be removed untill the whole software handling process is done, it wouldn't trigger the hard lock up issue caused by too long ATS Invalidation timeout wait. In safe removal path, device state isn't set to pci_channel_io_perm_failure in pciehp_unconfigure_device() by checking 'presence' parameter, calling pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return false there, wouldn't break the function.

For surprise removal, device state is set to pci_channel_io_perm_failure in pciehp_unconfigure_device(), means device is already gone (disconnected) call pci_dev_is_disconnected() in devtlb_invalidation_with_pasid() will return true to break the function not to send ATS Invalidation request to the disconnected device blindly, thus avoid the further long time waiting triggers the hard lockup.

safe removal & surprise removal

pciehp_ist()
pciehp_handle_presence_or_link_change()
pciehp_disable_slot()
remove_board()
pciehp_unconfigure_device(presence)

---
 drivers/iommu/intel/pasid.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/iommu/intel/pasid.c +3 -0
index 1c87fb1b1039..a08bdbec90eb 100644
@@ -481,6 +481,9 @@ devtlb_invalidation_with_pasid(struct intel_iommu *iommu,
 	if (!info || !info->ats_enabled)
 		return;
 
+	if (pci_dev_is_disconnected(to_pci_dev(dev)))
+		return;
+
 	sid = info->bus << 8 | info->devfn;
 	qdep = info->ats_qdep;
 	pfsid = info->pfsid;
Signature · 1 line
2.31.1
author
#13 lore raw

RFC PATCH v9 5/5 iommu/vt-d: don't loop for timeout ATS Invalidation request forever

In reply to Ethan Zhao

When the ATS Invalidation request timeout happens, the qi_submit_sync() will restart and loop for the invalidation request forever till it is done, it will block another Invalidation thread such as the fq_timer to issue invalidation request, cause the system lockup as following

[exception RIP: native_queued_spin_lock_slowpath+92]

RIP: ffffffffa9d1025c RSP: ffffb202f268cdc8 RFLAGS: 00000002

RAX: 0000000000000101 RBX: ffffffffab36c2a0 RCX: 0000000000000000

RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffab36c2a0

RBP: ffffffffab36c2a0 R8: 0000000000000001 R9: 0000000000000000

R10: 0000000000000010 R11: 0000000000000018 R12: 0000000000000000

R13: 0000000000000004 R14: ffff9e10d71b1c88 R15: ffff9e10d71b1980

ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018

#12 [ffffb202f268cdc8] native_queued_spin_lock_slowpath at ffffffffa9d1025c

#13 [ffffb202f268cdc8] do_raw_spin_lock at ffffffffa9d121f1

#14 [ffffb202f268cdd8] _raw_spin_lock_irqsave at ffffffffaa51795b

#15 [ffffb202f268cdf8] iommu_flush_dev_iotlb at ffffffffaa20df48

#16 [ffffb202f268ce28] iommu_flush_iova at ffffffffaa20e182

#17 [ffffb202f268ce60] iova_domain_flush at ffffffffaa220e27

#18 [ffffb202f268ce70] fq_flush_timeout at ffffffffaa221c9d

#19 [ffffb202f268cea8] call_timer_fn at ffffffffa9d46661

#20 [ffffb202f268cf08] run_timer_softirq at ffffffffa9d47933

#21 [ffffb202f268cf98] __softirqentry_text_start at ffffffffaa8000e0

#22 [ffffb202f268cff0] asm_call_sysvec_on_stack at ffffffffaa60114f --- ---
(the left part of exception see the hotplug case of ATS capable device)

If one endpoint device just no response to the ATS Invalidation request, but is not gone, it will bring down the whole system, to avoid such case, don't try the timeout ATS Invalidation request forever.

---
 drivers/iommu/intel/dmar.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel/dmar.c +1 -1
index 76903a8bf963..206ab0b7294f 100644
@@ -1457,7 +1457,7 @@ int qi_submit_sync(struct intel_iommu *iommu, struct qi_desc *desc,
 	reclaim_free_desc(qi);
 	raw_spin_unlock_irqrestore(&qi->q_lock, flags);
 
-	if (rc == -EAGAIN)
+	if (rc == -EAGAIN && type !=QI_DIOTLB_TYPE && type != QI_DEIOTLB_TYPE)
 		goto restart;
 
 	if (iotlb_start_ktime)
Signature · 1 line
2.31.1
#14 lore raw

RE: [RFC PATCH v9 5/5] iommu/vt-d: don't loop for timeout ATS Invalidation request forever

Quoted 64 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

When the ATS Invalidation request timeout happens, the qi_submit_sync() will restart and loop for the invalidation request forever till it is done, it will block another Invalidation thread such as the fq_timer to issue invalidation request, cause the system lockup as following

[exception RIP: native_queued_spin_lock_slowpath+92]

RIP: ffffffffa9d1025c RSP: ffffb202f268cdc8 RFLAGS: 00000002

RAX: 0000000000000101 RBX: ffffffffab36c2a0 RCX: 0000000000000000

RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffab36c2a0

RBP: ffffffffab36c2a0 R8: 0000000000000001 R9: 0000000000000000

R10: 0000000000000010 R11: 0000000000000018 R12: 0000000000000000

R13: 0000000000000004 R14: ffff9e10d71b1c88 R15: ffff9e10d71b1980

ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018

#12 [ffffb202f268cdc8] native_queued_spin_lock_slowpath at ffffffffa9d1025c

#13 [ffffb202f268cdc8] do_raw_spin_lock at ffffffffa9d121f1

#14 [ffffb202f268cdd8] _raw_spin_lock_irqsave at ffffffffaa51795b

#15 [ffffb202f268cdf8] iommu_flush_dev_iotlb at ffffffffaa20df48

#16 [ffffb202f268ce28] iommu_flush_iova at ffffffffaa20e182

#17 [ffffb202f268ce60] iova_domain_flush at ffffffffaa220e27

#18 [ffffb202f268ce70] fq_flush_timeout at ffffffffaa221c9d

#19 [ffffb202f268cea8] call_timer_fn at ffffffffa9d46661

#20 [ffffb202f268cf08] run_timer_softirq at ffffffffa9d47933

#21 [ffffb202f268cf98] __softirqentry_text_start at ffffffffaa8000e0

#22 [ffffb202f268cff0] asm_call_sysvec_on_stack at ffffffffaa60114f --- ---
(the left part of exception see the hotplug case of ATS capable device)

If one endpoint device just no response to the ATS Invalidation request, but is not gone, it will bring down the whole system, to avoid such case, don't try the timeout ATS Invalidation request forever.

---
 drivers/iommu/intel/dmar.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel/dmar.c +0 -0
index 76903a8bf963..206ab0b7294f 100644

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, · line 1463

Quoted 8 lines
drivers/iommu/intel/dmar.c +1 -1
@@ -1457,7 +1457,7 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
 	reclaim_free_desc(qi);
 	raw_spin_unlock_irqrestore(&qi->q_lock, flags);
 
-	if (rc == -EAGAIN)
+	if (rc == -EAGAIN && type !=QI_DIOTLB_TYPE && type !=
 QI_DEIOTLB_TYPE)
 		goto restart;

this change is moot.

-EAGAIN is set only when hardware detects a ATS invalidation completion timeout in qi_check_fault(). so above just essentially kills the restart logic.

I'd wait for the maintainer of this driver to comment. this part doesn't look good but there might be some history reason so carefulness must be paid.

author
#15 lore raw

Re: [RFC PATCH v9 5/5] iommu/vt-d: don't loop for timeout ATS Invalidation request forever

Quoted 74 lines · Kevin
Quoted 64 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

When the ATS Invalidation request timeout happens, the qi_submit_sync() will restart and loop for the invalidation request forever till it is done, it will block another Invalidation thread such as the fq_timer to issue invalidation request, cause the system lockup as following

[exception RIP: native_queued_spin_lock_slowpath+92]

RIP: ffffffffa9d1025c RSP: ffffb202f268cdc8 RFLAGS: 00000002

RAX: 0000000000000101 RBX: ffffffffab36c2a0 RCX: 0000000000000000

RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffffffffab36c2a0

RBP: ffffffffab36c2a0 R8: 0000000000000001 R9: 0000000000000000

R10: 0000000000000010 R11: 0000000000000018 R12: 0000000000000000

R13: 0000000000000004 R14: ffff9e10d71b1c88 R15: ffff9e10d71b1980

ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018

#12 [ffffb202f268cdc8] native_queued_spin_lock_slowpath at ffffffffa9d1025c

#13 [ffffb202f268cdc8] do_raw_spin_lock at ffffffffa9d121f1

#14 [ffffb202f268cdd8] _raw_spin_lock_irqsave at ffffffffaa51795b

#15 [ffffb202f268cdf8] iommu_flush_dev_iotlb at ffffffffaa20df48

#16 [ffffb202f268ce28] iommu_flush_iova at ffffffffaa20e182

#17 [ffffb202f268ce60] iova_domain_flush at ffffffffaa220e27

#18 [ffffb202f268ce70] fq_flush_timeout at ffffffffaa221c9d

#19 [ffffb202f268cea8] call_timer_fn at ffffffffa9d46661

#20 [ffffb202f268cf08] run_timer_softirq at ffffffffa9d47933

#21 [ffffb202f268cf98] __softirqentry_text_start at ffffffffaa8000e0

#22 [ffffb202f268cff0] asm_call_sysvec_on_stack at ffffffffaa60114f --- ---
(the left part of exception see the hotplug case of ATS capable device)

If one endpoint device just no response to the ATS Invalidation request, but is not gone, it will bring down the whole system, to avoid such case, don't try the timeout ATS Invalidation request forever.

---
  drivers/iommu/intel/dmar.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/iommu/intel/dmar.c +0 -0
index 76903a8bf963..206ab0b7294f 100644

drivers/iommu/intel/dmar.c int qi_submit_sync(struct intel_iommu *iommu, · line 1463

Quoted 8 lines
drivers/iommu/intel/dmar.c +1 -1
@@ -1457,7 +1457,7 @@ int qi_submit_sync(struct intel_iommu *iommu,
 struct qi_desc *desc,
  	reclaim_free_desc(qi);
  	raw_spin_unlock_irqrestore(&qi->q_lock, flags);
 
-	if (rc == -EAGAIN)
+	if (rc == -EAGAIN && type !=QI_DIOTLB_TYPE && type !=
 QI_DEIOTLB_TYPE)
  		goto restart;

this change is moot.

Quoted 2 lines

-EAGAIN is set only when hardware detects a ATS invalidation completion timeout in qi_check_fault(). so above just essentially kills the restart logic.

This change is intended to break the restar login when device-TLB

invalidation timeout happens, we don't know how long the ITE took

if the device is just no reponse.

Quoted 4 lines

I'd wait for the maintainer of this driver to comment. this part doesn't look good but there might be some history reason so carefulness must be paid.

I would like to know the reason a hole is left here to hang the driver

forever.

Thanks,

Ethan

#16 lore raw

RE: [RFC PATCH v9 5/5] iommu/vt-d: don't loop for timeout ATS Invalidation request forever

Quoted 15 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 9:10 PM

Quoted 11 lines · Kevin
Quoted 9 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

(quoted diff) +1 -1
-	if (rc == -EAGAIN)
+	if (rc == -EAGAIN && type !=QI_DIOTLB_TYPE && type !=

QI_DEIOTLB_TYPE)
goto restart;

this change is moot.

Quoted 8 lines
Quoted 2 lines

-EAGAIN is set only when hardware detects a ATS invalidation completion timeout in qi_check_fault(). so above just essentially kills the restart logic.

This change is intended to break the restar login when device-TLB

invalidation timeout happens, we don't know how long the ITE took

if the device is just no reponse.

if in the end the agreement is to remove the restart logic, then do it.

it's not good to introduce a change which essentially kills the restart logic but still keeps the related code.

author
#17 lore raw

Re: [RFC PATCH v9 5/5] iommu/vt-d: don't loop for timeout ATS Invalidation request forever

Quoted 19 lines · Kevin
Quoted 19 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 9:10 PM

Quoted 11 lines · Kevin
Quoted 9 lines

From: Ethan Zhao <[email protected]>
Sent: Thursday, December 28, 2023 8:17 AM

(quoted diff) +1 -1
-	if (rc == -EAGAIN)
+	if (rc == -EAGAIN && type !=QI_DIOTLB_TYPE && type !=

QI_DEIOTLB_TYPE)
goto restart;

this change is moot.

Quoted 2 lines

-EAGAIN is set only when hardware detects a ATS invalidation completion timeout in qi_check_fault(). so above just essentially kills the restart logic.

This change is intended to break the restar login when device-TLB

Quoted 7 lines
Quoted 3 lines

invalidation timeout happens, we don't know how long the ITE took

if the device is just no reponse.

if in the end the agreement is to remove the restart logic, then do it.

it's not good to introduce a change which essentially kills the restart logic but still keeps the related code.

Here, the device-TLB invalidation, depends on devcies response,

no one could make sure the what the third party adapters will act.

but for those invalidation issued to iommu itself, should be more

likely to survive ?

Anyway, would like to see more comments.

Thanks,

Ethan