Two-pass MMU interval notifiers
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v4 Latest4 patches 4 repliesIn review
- v34 patches 7 repliesSuperseded
- v24 patches 10 repliesSuperseded
Earlier versions · 1
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v1 | 6 patches | 3 replies | Superseded |
Changes in v4 from the cover letter
- Extend to wider audience and add R-Bs on the xe patches.
- Address documentation review comments from David Hildenbrand
Patches in v4 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/4 | mm/mmu_notifier: Allow two-pass struct mmu_interval_notifiersinclude/ mm/ | +98 | 4 replies | |
| 2/4 | drm/xe/userptr: Convert invalidation to two-pass MMU notifierdrivers/ drivers/ | +99 | 0 replies | |
| 3/4 | drm/xe: Split TLB invalidation into submit and wait stepsdrivers/ drivers/ drivers/ +5 more | +127 | 0 replies | |
| 4/4 | drm/xe/userptr: Defer Waiting for TLB invalidation to the second pass if possibledrivers/ drivers/ drivers/ +1 more | +104 | 0 replies |
Reviews trailers per patch
All 4 patches have a Reviewed-by or Acked-by.
| Patch | Matthew Brost | David Hildenbrand (Arm) | Maarten Lankhorst | Jason Gunthorpe |
|---|---|---|---|---|
| 1/4 | A-b | R-b | R-b | |
| 2/4 | R-b | |||
| 3/4 | R-b | |||
| 4/4 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 4 reviewers.
Cover letter [PATCH v4 0/4] Two-pass MMU interval notifiers
GPU use-cases for mmu_interval_notifiers with hmm often involve starting a gpu operation and then waiting for it to complete. These operations are typically context preemption or TLB flushing.
With single-pass notifiers per GPU this doesn't scale in multi-gpu scenarios. In those scenarios we'd want to first start preemption- or TLB flushing on all GPUs and as a second pass wait for them to complete.
This also applies in non-recoverable page-fault scenarios to starting a preemption requests on GPUs and waiting for the GPUs to preempt so that system pages they access can be reclaimed.
One can do this on per-driver basis multiplexing per-driver notifiers but that would mean sharing the notifier "user" lock across all GPUs and that doesn't scale well either, so adding support for two-pass in the core appears like the right choice.
So this series does that, with pach 1 implementing the core support and also describes the choices made.
The rest of the patches implements a POC with xeKMD userptr invalidation and potential TLB-flushing. A follow-up series will extend to drm_gpusvm.
v2 hightlights:
- Refactor the core mm patch to use the struct mmu_interval_notifier_ops for the invalidate_finish() callback.
- Rebase on xe driver tlb invalidation changes.
- Provide an initial implementation for userptr instead of drm_gpusvm. The intent is to handle drm_gpusvm in a follow-up series.
v3:
- Address review comments from Matt Brost: Code formatting, documentation, additional asserts and removal of unnecessary waits, as specified in each patch.
v4:
- Extend to wider audience and add R-Bs on the xe patches.
- Address documentation review comments from David Hildenbrand
Thomas Hellström (4):
mm/mmu_notifier: Allow two-pass struct mmu_interval_notifiers drm/xe/userptr: Convert invalidation to two-pass MMU notifier drm/xe: Split TLB invalidation into submit and wait steps drm/xe/userptr: Defer Waiting for TLB invalidation to the second pass
if possible
drivers/gpu/drm/xe/xe_svm.c | 8 +- drivers/gpu/drm/xe/xe_tlb_inval.c | 84 +++++++++++++ drivers/gpu/drm/xe/xe_tlb_inval.h | 6 + drivers/gpu/drm/xe/xe_tlb_inval_types.h | 14 +++ drivers/gpu/drm/xe/xe_userptr.c | 155 ++++++++++++++++++++---- drivers/gpu/drm/xe/xe_userptr.h | 31 ++++- drivers/gpu/drm/xe/xe_vm.c | 99 +++++---------- drivers/gpu/drm/xe/xe_vm.h | 5 +- drivers/gpu/drm/xe/xe_vm_madvise.c | 10 +- drivers/gpu/drm/xe/xe_vm_types.h | 1 + include/linux/mmu_notifier.h | 42 +++++++ mm/mmu_notifier.c | 65 ++++++++-- 12 files changed, 416 insertions(+), 104 deletions(-)
Signature · 1 line
2.53.0