Skip to content

RFC Multi-pass MMU interval notifiers

Thomas Hellström dri-devel intel-xe linux-mm lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest6 patches 15 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/6mm/mmu_notifier: Allow multiple struct mmu_interval_notifier passesinclude/linux/mmu_notifier.h mm/mmu_notifier.c+88 -913 replies
2/6drm/gpusvm: Update GPU SVM / Xe to twopass MMU notifierdrivers/gpu/drm/drm_gpusvm.c drivers/gpu/drm/xe/xe_svm.c include/drm/drm_gpusvm.h+23 -150 replies
3/6drm/gpusvm: Add drm_gpusvm_in_notifier_* helpersinclude/drm/drm_gpusvm.h+20 -20 replies
4/6drm/xe: Skip waiting on unarmed fences in xe_gt_tlb_invalidation_fence_waitdrivers/gpu/drm/xe/xe_gt_tlb_invalidation.h+2 -10 replies
5/6drm/xe: Add fences argument to xe_vm_range_tilemask_tlb_invalidationdrivers/gpu/drm/xe/xe_svm.c drivers/gpu/drm/xe/xe_vm.c drivers/gpu/drm/xe/xe_vm.h+23 -120 replies
6/6drm/xe: Implement two pass MMU notifiers for SVMdrivers/gpu/drm/drm_gpusvm.c drivers/gpu/drm/xe/xe_svm.c+63 -132 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 0/6] Multi-pass MMU interval notifiers

GPU use-cases for mmu_interval_notifiers with hmm often involve starting a gpu operation and then waiting for it to complete. These operations are typically context preemption or TLB flushing.

With single-pass notifiers per GPU this doesn't scale in multi-gpu scenarios. In those scenarios we'd want to first start preemption- or TLB flushing on all GPUs and as a second pass wait for them to complete on all GPUs.

One can do this on per-driver basis multiplexing per-driver notifiers but that would mean sharing the notifier "user" lock across all GPUs and that doesn't scale well either, so adding support for multi-pass in the core appears like the right choice.

So this series does that, with pach 1 implementing the core support and also describes the choices made.
The rest of the patches implements a POC with drm_gpusvm, but this will also come in handy for things like userptr where waiting for bind completion, starting of preemption and waiting for preemption completion can pe pipelined across GPUs.

Any feedback or suggestions for alternative approches appreciated.

Cc Matthew Brost <[email protected]>
Cc Jason Gunthorpe <[email protected]>
Cc Andrew Morton <[email protected]>
Cc Simona Vetter <[email protected]>
Cc Dave Airlie <[email protected]>

Matthew Brost (5):
drm/gpusvm: Update GPU SVM / Xe to twopass MMU notifier drm/gpusvm: Add drm_gpusvm_in_notifier_* helpers
drm/xe: Skip waiting on unarmed fences in
xe_gt_tlb_invalidation_fence_wait
drm/xe: Add fences argument to xe_vm_range_tilemask_tlb_invalidation drm/xe: Implement two pass MMU notifiers for SVM

Thomas Hellström (1):
mm/mmu_notifier: Allow multiple struct mmu_interval_notifier passes

 drivers/gpu/drm/drm_gpusvm.c                | 18 +++--
 drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h |  3 +-
 drivers/gpu/drm/xe/xe_svm.c                 | 84 +++++++++++++++++----
 drivers/gpu/drm/xe/xe_vm.c                  | 26 ++++---
 drivers/gpu/drm/xe/xe_vm.h                  |  6 +-
 include/drm/drm_gpusvm.h                    | 33 ++++++--
 include/linux/mmu_notifier.h                | 30 ++++++++
 mm/mmu_notifier.c                           | 67 +++++++++++++---
 8 files changed, 217 insertions(+), 50 deletions(-)
Signature · 1 line
2.50.1