Skip to contentSkip to conversation

vfio/iommufd: Prevent scheduler warnings when unmapping large regions

PATCH In review Aaron Lewis 4 participants 12 messages Jul 14 – Aug 4

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover vfio/iommufd: Prevent scheduler warnings when unmapping large regions

Both the Type1 IOMMU and IOMMUFD can be guilty of not playing nice with the scheduler during DMA unmapping on large regions (e.g., 256GB). Add rate-limited calls to cond_resched() to ensure this doesn't happen.

This series was tested using the selftest proposed here:
- https://lore.kernel.org/kvm/[email protected]/

And profiled using the bpftrace tooling proposed here:
- https://lore.kernel.org/kvm/[email protected]/

Note that the kernel must be built with preemption disabled and have LATENCY_WARN enabled in order to trigger the warnings. With those conditions met, the issue should reproduce as described in the changelogs.

Aaron Lewis (2):
vfio/type1: Periodically try rescheduling when unmapping
iommufd: Periodically reschedule when unmapping

 drivers/iommu/iommufd/pages.c   | 4 ++++
 drivers/vfio/vfio_iommu_type1.c | 3 +++
 2 files changed, 7 insertions(+)
Signature · 1 line
2.55.0.141.g00534a21ce-goog
#1 lore raw

PATCH 1/2 vfio/type1: Periodically try rescheduling when unmapping

Unmapping large DMA regions can be slow and cause a CPU core to be unresponsive for a long period of time, leading to kernel scheduler warnings.

For example:

[ 1404.810673] sched: CPU 19 need_resched set for > 100000026 ns (100 ticks) without schedule
[ 1404.810679] CPU: 19 UID: 0 PID: 34615 Comm: vfio_dma_mappin Kdump: loaded Not tainted 7.1.0-smp-DEV #2 PREEMPTLAZY
[ 1404.810682] Call trace:
[ 1404.810700]  resched_latency_warn+0x7c/0x88
[ 1404.810704]  sched_tick+0x18c/0x208
[ 1404.810708]  update_process_times+0xa8/0xd8
[ 1404.810711]  tick_nohz_handler+0x120/0x1c8
[ 1404.810713]  __hrtimer_run_queues+0x124/0x3a0
[ 1404.810716]  hrtimer_interrupt+0xdc/0x2c0
[ 1404.810718]  arch_timer_handler_phys+0x3c/0x58
[ 1404.810721]  handle_percpu_devid_irq+0x144/0x1b8
[ 1404.810723]  generic_handle_domain_irq+0x44/0x70
[ 1404.810725]  gic_handle_irq+0x1b8/0x320
[ 1404.810726]  call_on_irq_stack+0x30/0x48
[ 1404.810727]  do_interrupt_handler+0x54/0x80
[ 1404.810734]  __arm_lpae_iopte_walk+0x15c/0x1b8 (P)
[ 1404.810736]  arm_lpae_iova_to_phys+0x60/0xf0
[ 1404.810737]  arm_smmu_iova_to_phys+0x24/0x38
[ 1404.810739]  iommu_iova_to_phys+0x38/0x58
[ 1404.810752]  vfio_unmap_unpin+0x8/0x1c0
[ 1404.810753]  vfio_remove_dma+0x38/0x140
[ 1404.810757]  vfio_iommu_type1_ioctl+0x8/0xc80

Note that in order for this warning to fire, the kernel must be built without preemption, or... for testing, the code around the for loop can explicitly disable preemption.

This warning was produced on ARM using the VFIO selftest vfio_dma_mapping_perf_test:

./vfio_dma_mapping_perf_test -p -b 256G -a "-v vfio_type1_iommu_anonymous_hugetlb_1gb"

To profile the unmap flow, bpftrace was used with the tooling described at:
https://lore.kernel.org/kvm/[email protected]/

To isolate parts of the runtime, two temporary helper functions were added to vfio_unmap_unpin() during testing: vfio_try_unpin(), which encapsulated the outer while loop, and vfio_get_largest_chunk(), which encapsulated the inner for loop. Doing this helped identify where the latency issues were coming from and how the code flowed.

The results show:

vfio_iommu_type1_unmap_dma, thread_time = 1124.85ms, count = 1

    vfio_unmap_unpin, thread_time = 1124.84ms, count = 1
      vfio_try_unpin, thread_time = 1124.80ms, count = 1
        vfio_get_largest_chunk, thread_time = 1124.79ms, count = 1

This shows that the execution time is spent entirely inside the inner for loop. Furthermore, the loop is entered only once. Therefore, To avoid the scheduler warning and allow the system to remain responsive, add a cond_resched() to the inner for loop in vfio_unmap_unpin(). Also, limit the cond_resched() to once per 1GB to rate limit it.

Additional context on what consumes time within the inner loop can be seen in this sample profile from perf:

  24.13%  [vfio_dma_mapping_perf_test] ([vfio_dma_mapping_perf_test])
  24.12%  vfs_ioctl (ioctl.c) Inlined
  24.12%  vfio_fops_unl_ioctl (container.c)
  24.12%  vfio_iommu_type1_ioctl (vfio_iommu_type1.c)
  19.66%  vfio_iommu_type1_unmap_dma (vfio_iommu_type1.c) Inlined
  19.66%  vfio_dma_do_unmap (vfio_iommu_type1.c) Inlined
  19.66%  vfio_remove_dma (vfio_iommu_type1.c)
  19.66%  vfio_unmap_unpin (vfio_iommu_type1.c)
  19.65%  vfio_try_unpin (vfio_iommu_type1.c)
  19.10%  vfio_get_largest_chunk (vfio_iommu_type1.c)
  15.91%  iommu_iova_to_phys (iommu.c)
  15.18%  arm_smmu_iova_to_phys (arm-smmu-v3.c)
  14.09%  arm_lpae_iova_to_phys (io-pgtable-arm.c)
  11.59%  __arm_lpae_iopte_walk (io-pgtable-arm.c)
   8.51%  io_pgtable_visit (io-pgtable-arm.c) Inlined
   5.27%  __arm_lpae_iopte_walk (io-pgtable-arm.c)
   2.64%  io_pgtable_visit (io-pgtable-arm.c) Inlined
---
 drivers/vfio/vfio_iommu_type1.c | 3 +++
 1 file changed, 3 insertions(+)
drivers/vfio/vfio_iommu_type1.c +3 -0
index c8151ba54de3..e78bc07d9b6e 100644
@@ -1153,6 +1153,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
 	int unmapped_region_cnt = 0;
 	long unlocked = 0;
 	size_t pos = 0;
+	int i = 0;
 
 	if (!dma->size)
 		return 0;
@@ -1196,6 +1197,8 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
 			next = iommu_iova_to_phys(domain->domain, iova + len);
 			if (next != phys + len)
 				break;
+			if ((++i % BIT(PUD_ORDER)) == 0)
+				cond_resched();
 		}
 
 		/*
Signature · 1 line
2.55.0.141.g00534a21ce-goog
#2 lore raw

Re: [PATCH 1/2] vfio/type1: Periodically try rescheduling when unmapping

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] The use of `BIT(PUD_ORDER)` to rate-limit `cond_resched()` evaluates to incorrectly large values…

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] The use of BIT(PUD_ORDER) to rate-limit cond_resched() evaluates to incorrectly large values on architectures with large page sizes, failing to prevent the scheduler stall. --

--- Patch [1]: [PATCH 1/2] vfio/type1: Periodically try rescheduling when unmapping --- commit 2d6c55cffed789bb39f187cfc28ac1de6edd70b2
Author: Aaron Lewis <[email protected]>

vfio/type1: Periodically try rescheduling when unmapping

This commit adds a cond_resched() to the inner loop of vfio_unmap_unpin() to prevent CPU stalls during the unmapping of large DMA regions. It attempts to rate limit the reschedule check to once per 1GB using BIT(PUD_ORDER).

Quoted 3 lines
drivers/vfio/vfio_iommu_type1.c +0 -0

[ ... ]

drivers/vfio/vfio_iommu_type1.c static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, · line 1201

Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +2 -0
@@ -1196,6 +1197,8 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
 			next = iommu_iova_to_phys(domain->domain, iova + len);
 			if (next != phys + len)
 				break;
+			if ((++i % BIT(PUD_ORDER)) == 0)
+				cond_resched();

[Severity: Medium]
Will this rate limit fail to prevent scheduler stalls on architectures with large page sizes?

On an ARM64 system configured with 64KB pages, PUD_SHIFT is 42 and PAGE_SHIFT is 16, resulting in a PUD_ORDER of 26. BIT(26) evaluates to 67,108,864 loop iterations.

Since each iteration processes PAGE_SIZE (64KB), the loop would process 4TB of contiguous memory before calling cond_resched(). As noted in the commit message, 67 million loop iterations can take over 1 second, which vastly exceeds the 100ms scheduler tick limit.

Could this use a constant size limit instead, such as (SZ_1G >> PAGE_SHIFT), to ensure the interval remains consistent across different architectures and page sizes?

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
#3 lore raw

Re: [PATCH 1/2] vfio/type1: Periodically try rescheduling when unmapping

In reply to Aaron Lewis
Quoted 13 lines · Aaron Lewis
drivers/vfio/vfio_iommu_type1.c +1 -0
index c8151ba54de3..e78bc07d9b6e 100644
@@ -1153,6 +1153,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
        int unmapped_region_cnt = 0;
        long unlocked = 0;
        size_t pos = 0;
+       int i = 0;
 
        if (!dma->size)
                return 0;

drivers/vfio/vfio_iommu_type1.c static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, · line 1202

Quoted 6 lines
drivers/vfio/vfio_iommu_type1.c +2 -0
@@ -1196,6 +1197,8 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
                        next = iommu_iova_to_phys(domain->domain, iova + len);
                        if (next != phys + len)
                                break;
+                       if ((++i % BIT(PUD_ORDER)) == 0)
+                               cond_resched();
                }

What is the performance impact on unmap if cond_resched() is called on every iteration of the loop? It would be nice to not have a magic heuristic (which is brittle as Sashiko points out).

maintainer
#4 lore raw

Re: [PATCH 1/2] vfio/type1: Periodically try rescheduling when unmapping

Quoted 12 lines · David Matlack
Quoted 10 lines · Aaron Lewis
drivers/vfio/vfio_iommu_type1.c +1 -0
index c8151ba54de3..e78bc07d9b6e 100644
@@ -1153,6 +1153,7 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
        int unmapped_region_cnt = 0;
        long unlocked = 0;
        size_t pos = 0;
+       int i = 0;
 
Quoted 13 lines

drivers/vfio/vfio_iommu_type1.c static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma, · line 1202

Quoted 9 lines
drivers/vfio/vfio_iommu_type1.c +2 -0
        if (!dma->size)
                return 0;
@@ -1196,6 +1197,8 @@ static long vfio_unmap_unpin(struct vfio_iommu *iommu, struct vfio_dma *dma,
                        next = iommu_iova_to_phys(domain->domain, iova + len);
                        if (next != phys + len)
                                break;
+                       if ((++i % BIT(PUD_ORDER)) == 0)
+                               cond_resched();
                }

What is the performance impact on unmap if cond_resched() is called on every iteration of the loop? It would be nice to not have a magic heuristic (which is brittle as Sashiko points out).

Calling every loop is definately not a good idea.. Some magic number is a pretty typical pattern for these CPU bound loops.

Jason

#5 lore raw

PATCH 2/2 iommufd: Periodically reschedule when unmapping

In reply to Aaron Lewis

Unmapping large DMA regions can be slow and cause a CPU core to be unresponsive for a long period of time, leading to kernel scheduler warnings.

For example:

[ 1389.148910] vfio-pci 0007:01:01.3: resetting
[ 1389.252814] vfio-pci 0007:01:01.3: reset done
[ 1392.943771] sched: CPU 49 need_resched set for > 100000053 ns (100 ticks) without schedule
[ 1392.943776] CPU: 49 UID: 0 PID: 32912 Comm: vfio_dma_mappin Kdump: loaded Not tainted 7.1.0-smp-DEV #1 PREEMPTLAZY
[ 1392.943780] Call trace:
[ 1392.943796]  resched_latency_warn+0x7c/0x88
[ 1392.943800]  sched_tick+0x18c/0x208
[ 1392.943804]  update_process_times+0xa8/0xd8
[ 1392.943808]  tick_nohz_handler+0x120/0x1c8
[ 1392.943810]  __hrtimer_run_queues+0x124/0x3a0
[ 1392.943833]  arm_lpae_iova_to_phys+0x80/0xf0 (P)
[ 1392.943835]  arm_smmu_iova_to_phys+0x24/0x38
[ 1392.943837]  iommu_iova_to_phys+0x38/0x58
[ 1392.943839]  batch_from_domain+0x16c/0x218
[ 1392.943840]  __iopt_area_unfill_domain+0x2f0/0x5d8
[ 1392.943841]  iopt_area_unfill_domains+0x9c/0x1c8
[ 1392.943842]  iopt_unmap_iova_range+0xec/0x218
[ 1392.943844]  iopt_unmap_iova+0x38/0x50
[ 1392.943846]  iommufd_ioas_unmap+0xac/0x1f8
[ 1392.943847]  iommufd_fops_ioctl+0x200/0x348

Note that in order for this warning to fire, the kernel must be built without preemption, or... for testing, the code in batch_from_domain() can explicitly disable preemption.

This warning was produced on ARM using the VFIO selftest vfio_dma_mapping_perf_test:

./vfio_dma_mapping_perf_test -p -b 256G -a "-v iommufd_anonymous_hugetlb_1gb"

To profile the unmap flow, bpftrace was used with the tooling described at:
https://lore.kernel.org/kvm/[email protected]/

The results show:

iommufd_ioas_unmap, thread_time = 1086.52ms, count = 1

    __iopt_area_unfill_domain, thread_time = 1086.51ms, count = 1
      batch_from_domain, thread_time = 1086.48ms, count = 1

This shows that the execution time is spent entirely inside batch_from_domain(). Furthermore, the function is entered only once. Therefore, to avoid the scheduler warning and allow the core to remain responsive, add a cond_resched() call to the loop in batch_from_domain(). Also, limit the cond_resched() call to once per 1GB to rate limit it.

Additional context on what consumes time within the inner loop can be seen in this sample perf profile:

  22.96%  [vfio_dma_mapping_perf_test] ([vfio_dma_mapping_perf_test])
  22.96%  vfs_ioctl (ioctl.c) Inlined
  22.96%  iommufd_fops_ioctl (main.c)
  19.38%  iommufd_ioas_unmap (ioas.c)
  19.38%  iopt_unmap_iova (io_pagetable.c)
  19.38%  iopt_unmap_iova_range (io_pagetable.c)
  19.38%  iopt_area_unfill_domains (pages.c)
  19.38%  iopt_area_unfill_domain (pages.c) Inlined
  19.38%  __iopt_area_unfill_domain (pages.c)
  19.38%  iopt_area_unpin_domain (pages.c) Inlined
  18.84%  batch_from_domain (pages.c)
  15.06%  iommu_iova_to_phys (iommu.c)
  14.27%  arm_smmu_iova_to_phys (arm-smmu-v3.c)
  13.29%  arm_lpae_iova_to_phys (io-pgtable-arm.c)
  10.60%  __arm_lpae_iopte_walk (io-pgtable-arm.c)
   7.70%  io_pgtable_visit (io-pgtable-arm.c) Inlined
   4.71%  __arm_lpae_iopte_walk (io-pgtable-arm.c)
   2.31%  io_pgtable_visit (io-pgtable-arm.c) Inlined
---
 drivers/iommu/iommufd/pages.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/iommu/iommufd/pages.c +4 -0
index 9bdb2945afe1..e478771c0ccf 100644
@@ -412,6 +412,7 @@ static void batch_from_domain(struct pfn_batch *batch,
 	unsigned int page_offset = 0;
 	unsigned long iova;
 	phys_addr_t phys;
+	int i = 0;
 
 	iova = iopt_area_index_to_iova(area, start_index);
 	if (start_index == iopt_area_index(area))
@@ -428,6 +429,9 @@ static void batch_from_domain(struct pfn_batch *batch,
 		iova += PAGE_SIZE - page_offset;
 		page_offset = 0;
 		start_index++;
+
+		if ((++i % BIT(PUD_ORDER)) == 0)
+			cond_resched();
 	}
 }
 
Signature · 1 line
2.55.0.141.g00534a21ce-goog
#6 lore raw

Re: [PATCH 2/2] iommufd: Periodically reschedule when unmapping

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] The `cond_resched()` rate limiter is circumvented for fragmented memory because the loop counter…

Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] The cond_resched() rate limiter is circumvented for fragmented memory because the loop counter i is local and resets on every batch, leaving the scheduler warning unresolved.
- [High] Using BIT(PUD_ORDER) to rate-limit cond_resched() to 1GB is unportable and scales incorrectly on architectures with larger page sizes, preventing rescheduling. --

--- Patch [2]: [PATCH 2/2] iommufd: Periodically reschedule when unmapping --- commit 32d4b71fc4dce755ccc1bb27abe57a685f82b576
Author: Aaron Lewis <[email protected]>

iommufd: Periodically reschedule when unmapping

This patch adds a cond_resched() call to batch_from_domain() to avoid scheduler warnings during large DMA unmaps. It attempts to rate-limit the yields to approximately once per 1GB of processed memory.

Quoted 13 lines
drivers/iommu/iommufd/pages.c +1 -0
index 03c8379bbc347..e7abb3350b235 100644
@@ -412,6 +412,7 @@ static void batch_from_domain(struct pfn_batch *batch,
 	unsigned int page_offset = 0;
 	unsigned long iova;
 	phys_addr_t phys;
+	int i = 0;
 
 	iova = iopt_area_index_to_iova(area, start_index);
 	if (start_index == iopt_area_index(area))

drivers/iommu/iommufd/pages.c static void batch_from_domain(struct pfn_batch *batch, · line 434

Quoted 6 lines
drivers/iommu/iommufd/pages.c +3 -0
@@ -428,6 +429,9 @@ static void batch_from_domain(struct pfn_batch *batch,
 		iova += PAGE_SIZE - page_offset;
 		page_offset = 0;
 		start_index++;
+
+		if ((++i % BIT(PUD_ORDER)) == 0)
+			cond_resched();

[Severity: High]
Does this regression mean the counter fails to trigger cond_resched() when operating on fragmented memory?

Because i is a local variable, it resets to 0 on every call to batch_from_domain(). For highly fragmented memory, batch_add_pfn() will fill its array and return false early (limited to roughly 5,461 pages by TEMP_MEMORY_LIMIT).

This early return causes the outer loops, such as iopt_area_unpin_domain(), to repeatedly call batch_from_domain(). This continuously resets i to 0 before it can ever reach the threshold, meaning cond_resched() is never called and scheduler lockups can still occur.

[Severity: High]
Is this a regression on architectures where BIT(PUD_ORDER) does not equal 1GB?

The commit message mentions limiting the cond_resched() call to once per 1GB. While this holds true on architectures like x86_64, it scales incorrectly on architectures with larger page sizes.

For example, on ARM64 with 64KB pages and 3-level page tables, PUD_ORDER is 26. BIT(26) iterations of 64KB pages corresponds to 4TB of memory, not 1GB. The loop will only yield once every 4TB, which effectively defeats the mitigation on these architectures.

Quoted 2 lines
drivers/iommu/iommufd/pages.c +0 -0
 	}
 }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
maintainer
#7 lore raw
In reply to Aaron Lewis
Quoted 9 lines · Aaron Lewis

Both the Type1 IOMMU and IOMMUFD can be guilty of not playing nice with the scheduler during DMA unmapping on large regions (e.g., 256GB). Add rate-limited calls to cond_resched() to ensure this doesn't happen.

This series was tested using the selftest proposed here:
- https://lore.kernel.org/kvm/[email protected]/

And profiled using the bpftrace tooling proposed here:
- https://lore.kernel.org/kvm/[email protected]/

Given this is the same issue on both in the some place, I wonder if it is better to try to get this series done:

https://lore.kernel.org/linux-iommu/[email protected]/

And put the cond sched inside the iova_to_phys_length() ?

It would be fine to do things in steps, like add the iova_to_phys_length() that just does the loops iommufd and vfio have today with the additional cond sched inside which isn't such a big task.

Jason

#8 lore raw
Quoted 17 lines · Jason Gunthorpe
Quoted 9 lines · Aaron Lewis

Both the Type1 IOMMU and IOMMUFD can be guilty of not playing nice with the scheduler during DMA unmapping on large regions (e.g., 256GB). Add rate-limited calls to cond_resched() to ensure this doesn't happen.

This series was tested using the selftest proposed here:
- https://lore.kernel.org/kvm/[email protected]/

And profiled using the bpftrace tooling proposed here:
- https://lore.kernel.org/kvm/[email protected]/

Given this is the same issue on both in the some place, I wonder if it is better to try to get this series done:

https://lore.kernel.org/linux-iommu/[email protected]/

Quoted 8 lines

And put the cond sched inside the iova_to_phys_length() ?

It would be fine to do things in steps, like add the iova_to_phys_length() that just does the loops iommufd and vfio have today with the additional cond sched inside which isn't such a big task.

Jason

Ah, I think that is really what I wanted, but I proposed a solution to get feedback quicker. I'll wait for that series to be merged and reaccess if a cond_resched() is still needed. I'll follow up here once I've had a chance to run these tests again with it included.

Aaron

maintainer
#9 lore raw
Quoted 25 lines · Aaron Lewis
Quoted 24 lines · Jason Gunthorpe
Quoted 9 lines · Aaron Lewis

Both the Type1 IOMMU and IOMMUFD can be guilty of not playing nice with the scheduler during DMA unmapping on large regions (e.g., 256GB). Add rate-limited calls to cond_resched() to ensure this doesn't happen.

This series was tested using the selftest proposed here:
- https://lore.kernel.org/kvm/[email protected]/

And profiled using the bpftrace tooling proposed here:
- https://lore.kernel.org/kvm/[email protected]/

Given this is the same issue on both in the some place, I wonder if it is better to try to get this series done:

https://lore.kernel.org/linux-iommu/[email protected]/

And put the cond sched inside the iova_to_phys_length() ?

It would be fine to do things in steps, like add the iova_to_phys_length() that just does the loops iommufd and vfio have today with the additional cond sched inside which isn't such a big task.

Quoted 6 lines
Quoted 1 line

Jason

Ah, I think that is really what I wanted, but I proposed a solution to get feedback quicker. I'll wait for that series to be merged and reaccess if a cond_resched() is still needed. I'll follow up here once I've had a chance to run these tests again with it included.

It seems kind of stalled, so IDK if you want to propose just a bit of it that would be fine too

Jason

#10 lore raw
Quoted 34 lines · Jason Gunthorpe
Quoted 25 lines · Aaron Lewis
Quoted 24 lines · Jason Gunthorpe
Quoted 9 lines · Aaron Lewis

Both the Type1 IOMMU and IOMMUFD can be guilty of not playing nice with the scheduler during DMA unmapping on large regions (e.g., 256GB). Add rate-limited calls to cond_resched() to ensure this doesn't happen.

This series was tested using the selftest proposed here:
- https://lore.kernel.org/kvm/[email protected]/

And profiled using the bpftrace tooling proposed here:
- https://lore.kernel.org/kvm/[email protected]/

Given this is the same issue on both in the some place, I wonder if it is better to try to get this series done:

https://lore.kernel.org/linux-iommu/[email protected]/

And put the cond sched inside the iova_to_phys_length() ?

It would be fine to do things in steps, like add the iova_to_phys_length() that just does the loops iommufd and vfio have today with the additional cond sched inside which isn't such a big task.

Quoted 6 lines
Quoted 1 line

Jason

Ah, I think that is really what I wanted, but I proposed a solution to get feedback quicker. I'll wait for that series to be merged and reaccess if a cond_resched() is still needed. I'll follow up here once I've had a chance to run these tests again with it included.

Quoted 4 lines

It seems kind of stalled, so IDK if you want to propose just a bit of it that would be fine too

Jason

Any idea what a minimum proposal would look like? Looking at the series it all looks pretty relevant given the goal would be to have iova_to_phys_length() land.

Aaron

maintainer
#11 lore raw
Quoted 19 lines · Aaron Lewis
Quoted 19 lines
Quoted 18 lines
Quoted 5 lines

Given this is the same issue on both in the some place, I wonder if it is better to try to get this series done:

https://lore.kernel.org/linux-iommu/[email protected]/

Quoted 8 lines

And put the cond sched inside the iova_to_phys_length() ?

It would be fine to do things in steps, like add the iova_to_phys_length() that just does the loops iommufd and vfio have today with the additional cond sched inside which isn't such a big task.

Jason

Ah, I think that is really what I wanted, but I proposed a solution to get feedback quicker. I'll wait for that series to be merged and reaccess if a cond_resched() is still needed. I'll follow up here once I've had a chance to run these tests again with it included.

Quoted 8 lines
Quoted 4 lines

It seems kind of stalled, so IDK if you want to propose just a bit of it that would be fine too

Jason

Any idea what a minimum proposal would look like? Looking at the series it all looks pretty relevant given the goal would be to have iova_to_phys_length() land.

I think if all you did was add the iova_to_phys_length() function in the core code, implementing it as calling out to the drivers as is today that would be a fine series

Jason