vfio/type1: avoid walking reserved-only mappings while unpinning
Subsystems (2)
PATCH v2 1/1 vfio/type1: avoid walking reserved-only mappings while unpinning
Tearing down a large device-passthrough DMA mapping can process tens of millions of reserved pfns in a single VFIO_IOMMU_UNMAP_DMA. A 128 GiB reserved mapping holds about 33.6 million 4KiB pfns, and vfio_unpin_pages_remote() walks every one of them whenever dma->has_rsvd is set, because reserved and non-reserved pfns might be mixed. For a mapping that contains only reserved pfns every put_pfn() is a no-op, but each one still pays for the pfn_valid()/PageReserved() classification.
Seen on GPU-passthrough hosts running 6.13 with the v6.18 type1 series backported, where a single VFIO_IOMMU_UNMAP_DMA held a CPU past the soft-lockup watchdog (host identification, register dump and module list trimmed):
watchdog: BUG: soft lockup - CPU#261 stuck for 22s! [qemu-system-x86:134503] RIP: 0010:vfio_unpin_pages_remote+0x139/0x2a0 [vfio_iommu_type1] Code: 90 4c 89 e1 48 c1 e9 34 75 74 4c 89 e1 48 c1 e9 22 75 6b 48 8b 0d f7 6f c2 e3 48 85 c9 74 5f 4c 89 e2 48 c1 ea 16 48 8b 0c d1 <48> 85 c9 74 4f 41 8b 55 30 4c 89 e6 48 c1 ee 0f 83 e6 7f c1 e6 05 Call Trace: <IRQ> ? watchdog_timer_fn+0x3d6/0x440 ? running_clock+0x10/0x10 ? hrtimer_interrupt+0x185/0x580 ? __sysvec_apic_timer_interrupt+0x44/0xe0 ? sysvec_apic_timer_interrupt+0x6b/0x80 </IRQ> <TASK> ? asm_sysvec_apic_timer_interrupt+0x16/0x20 ? vfio_unpin_pages_remote+0x139/0x2a0 [vfio_iommu_type1] vfio_sync_unpin+0x96/0xf0 [vfio_iommu_type1] vfio_unmap_unpin+0x324/0x3d0 [vfio_iommu_type1] vfio_remove_dma+0x25/0xa0 [vfio_iommu_type1] vfio_iommu_type1_ioctl+0xcd5/0x1760 [vfio_iommu_type1] ? amd_pmu_v2_enable_all+0xa/0x30 ? perf_pmu_sched_task+0xbf/0xf0 x64_sys_call+0x282/0x1ac0 ? syscall_trace_enter+0x1e5/0x1f0 do_syscall_64+0x68/0x130 entry_SYSCALL_64_after_hwframe+0x4b/0x53
RIP is the mem_section root test in pfn_valid(), inlined into put_pfn() through is_invalid_reserved_pfn(), so the stall is in the dma->has_rsvd per-page loop. mem_section[] has no root for that pfn, so pfn_valid() returns false and put_pfn() does nothing - for every pfn in the mapping.
Track whether the mapping also contains non-reserved pfns and skip the walk entirely for reserved-only mappings. Mixed mappings keep the existing per-pfn walk, and ordinary mappings keep the existing batched unpin path.
Tested on an affected host: reserved-only mappings up to 128 GiB take the new path and complete in microseconds. The ordinary-page path was exercised unchanged.
---
This is the reserved-only fast path you suggested on v1, tested on the affected hosts. There is indeed no evidence that the non-reserved VFIO path needs chunking, and agree any such change to unpin_user_page_range_dirty_lock() would belong in mm, so not including it in this patch.
---
Changes in v2:
- Replace v1's cond_resched() chunking with the reserved-only fast path.
- Tested on an affected host; reserved-only mappings up to 128 GiB now complete in microseconds instead of walking ~33.6M pfns.
- Link to v1: https://patch.msgid.link/[email protected]
--- drivers/vfio/vfio_iommu_type1.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)
drivers/vfio/vfio_iommu_type1.c +8 -4
@@ -94,6 +94,7 @@ struct vfio_dma {bool lock_cap;/* capable(CAP_IPC_LOCK) */ bool vaddr_invalid;bool has_rsvd;/* has 1 or more rsvd pfns */ +bool has_non_rsvd;/* has 1 or more !rsvd pfns */ struct task_struct *task;struct rb_root pfn_list;/* Ex-user pinned pfn list */ unsigned long *bitmap;
@@ -791,6 +792,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, out: dma->has_rsvd |= rsvd; + dma->has_non_rsvd |= !rsvd; ret =vfio_lock_acct (dma, lock_acct, false); unpin_out:
@@ -821,11 +823,13 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,long unlocked =0 , locked =vpfn_pages (dma, iova, npage);if (dma->has_rsvd) { -unsigned long i; +if (dma->has_non_rsvd) { +unsigned long i; -for (i =0 ; i < npage; i++) -if (put_pfn (pfn++, dma->prot)) - unlocked++; +for (i =0 ; i < npage; i++) +if (put_pfn (pfn++, dma->prot)) + unlocked++; + } }else {put_valid_unreserved_pfns (pfn, npage, dma->prot); unlocked = npage;
---
base-commit: 4e3c1fc8abcb8eff062150b4340fa4569696d645
change-id: 20260723-vfio-decd86bf3cf6
Best regards,
Signature · 1 line
Samuel Crossley <[email protected]>
Quoted 114 lines · On Sat, 19 Sep 2026 15:13:03 -0700 Samuel Crossley <[email protected]>…
Tearing down a large device-passthrough DMA mapping can process tens of millions of reserved pfns in a single
VFIO_IOMMU_UNMAP_DMA. A 128 GiB reserved mapping holds about 33.6 million 4KiB pfns, andvfio_unpin_pages_remote()walks every one of them wheneverdma->has_rsvdis set, because reserved and non-reserved pfns might be mixed. For a mapping that contains only reserved pfns everyput_pfn()is a no-op, but each one still pays for the pfn_valid()/PageReserved() classification.Seen on GPU-passthrough hosts running 6.13 with the v6.18 type1 series backported, where a single
VFIO_IOMMU_UNMAP_DMAheld a CPU past the soft-lockup watchdog (host identification, register dump and module list trimmed):watchdog: BUG: soft lockup - CPU#261 stuck for 22s! [qemu-system-x86:134503] RIP: 0010:vfio_unpin_pages_remote+0x139/0x2a0 [vfio_iommu_type1] Code: 90 4c 89 e1 48 c1 e9 34 75 74 4c 89 e1 48 c1 e9 22 75 6b 48 8b 0d f7 6f c2 e3 48 85 c9 74 5f 4c 89 e2 48 c1 ea 16 48 8b 0c d1 <48> 85 c9 74 4f 41 8b 55 30 4c 89 e6 48 c1 ee 0f 83 e6 7f c1 e6 05 Call Trace: <IRQ> ? watchdog_timer_fn+0x3d6/0x440 ? running_clock+0x10/0x10 ? hrtimer_interrupt+0x185/0x580 ? __sysvec_apic_timer_interrupt+0x44/0xe0 ? sysvec_apic_timer_interrupt+0x6b/0x80 </IRQ> <TASK> ? asm_sysvec_apic_timer_interrupt+0x16/0x20 ? vfio_unpin_pages_remote+0x139/0x2a0 [vfio_iommu_type1] vfio_sync_unpin+0x96/0xf0 [vfio_iommu_type1] vfio_unmap_unpin+0x324/0x3d0 [vfio_iommu_type1] vfio_remove_dma+0x25/0xa0 [vfio_iommu_type1] vfio_iommu_type1_ioctl+0xcd5/0x1760 [vfio_iommu_type1] ? amd_pmu_v2_enable_all+0xa/0x30 ? perf_pmu_sched_task+0xbf/0xf0 x64_sys_call+0x282/0x1ac0 ? syscall_trace_enter+0x1e5/0x1f0 do_syscall_64+0x68/0x130 entry_SYSCALL_64_after_hwframe+0x4b/0x53RIP is the
mem_sectionroot test inpfn_valid(), inlined intoput_pfn()throughis_invalid_reserved_pfn(), so the stall is in thedma->has_rsvdper-page loop. mem_section[] has no root for that pfn, sopfn_valid()returns false andput_pfn()does nothing - for every pfn in the mapping.Track whether the mapping also contains non-reserved pfns and skip the walk entirely for reserved-only mappings. Mixed mappings keep the existing per-pfn walk, and ordinary mappings keep the existing batched unpin path.
Tested on an affected host: reserved-only mappings up to 128 GiB take the new path and complete in microseconds. The ordinary-page path was exercised unchanged.
Suggested-by Alex Williamson <[email protected]>Signed-off-by Samuel Crossley <[email protected]>---
This is the reserved-only fast path you suggested on v1, tested on the affected hosts. There is indeed no evidence that the non-reserved VFIO path needs chunking, and agree any such change tounpin_user_page_range_dirty_lock()would belong in mm, so not including it in this patch.
---
Changes in v2:
- Replace v1'scond_resched()chunking with the reserved-only fast path.
- Tested on an affected host; reserved-only mappings up to 128 GiB now complete in microseconds instead of walking ~33.6M pfns.
- Link to v1: https://patch.msgid.link/[email protected]--- drivers/vfio/vfio_iommu_type1.c | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-)drivers/vfio/vfio_iommu_type1.c +8
-5@@ -94,6 +94,7 @@ struct vfio_dma {bool lock_cap;/* capable(CAP_IPC_LOCK) */ bool vaddr_invalid;bool has_rsvd;/* has 1 or more rsvd pfns */ +bool has_non_rsvd;/* has 1 or more !rsvd pfns */ struct task_struct *task;struct rb_root pfn_list;/* Ex-user pinned pfn list */ unsigned long *bitmap;@@ -791,6 +792,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr, out: dma->has_rsvd |= rsvd; + dma->has_non_rsvd |= !rsvd; ret =vfio_lock_acct (dma, lock_acct, false); unpin_out:@@ -821,11 +823,13 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,long unlocked =0 , locked =vpfn_pages (dma, iova, npage);if (dma->has_rsvd) { -unsigned long i; +if (dma->has_non_rsvd) { +unsigned long i; -for (i =0 ; i < npage; i++) -if (put_pfn (pfn++, dma->prot)) - unlocked++; +for (i =0 ; i < npage; i++) +if (put_pfn (pfn++, dma->prot)) + unlocked++; + } }else {put_valid_unreserved_pfns (pfn, npage, dma->prot); unlocked = npage; ---base-commit:
4e3c1fc8abcb8eff062150b4340fa4569696d645
change-id: 20260723-vfio-decd86bf3cf6
Quoted 4 lines
Best regards,
Signature · 1 line
Samuel Crossley <[email protected]>
Applied to vfio next for v7.4. Thanks,
Alex