Skip to content

Introduce GPU SVM and Xe SVM implementation

Matthew Brost dri-devel intel-xe 7 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v7 Latest32 patches 3 repliesIn review
  2. v632 patches 15 repliesSuperseded
  3. v532 patches 39 repliesSuperseded
Earlier versions · 4
VersionPostedPatchesRepliesStatus
v433 patches66 repliesSuperseded
v330 patches20 repliesSuperseded
v229 patches96 repliesSuperseded
v1 RFC28 patches68 repliesSuperseded

Patches in v7 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/32drm/xe: Retry BO allocationdrivers/gpu/drm/xe/xe_bo.c+7 -00 replies
2/32mm/migrate: Add migrate_device_pfnsinclude/linux/migrate.h mm/migrate_device.c+38 -150 replies
3/32mm/migrate: Trylock device page in do_swap_pagemm/memory.c mm/migrate_device.c+55 -220 replies
4/32drm/pagemap: Add DRM pagemapinclude/drm/drm_pagemap.h+107 -00 replies
5/32drm/xe/bo: Introduce xe_bo_put_asyncdrivers/gpu/drm/xe/xe_bo.c drivers/gpu/drm/xe/xe_bo.h drivers/gpu/drm/xe/xe_device.c +1 more+55 -00 replies
6/32drm/gpusvm: Add support for GPU Shared Virtual Memorydrivers/gpu/drm/Kconfig drivers/gpu/drm/Makefile drivers/gpu/drm/drm_gpusvm.c +1 more+2755 -00 replies
7/32drm/xe: Select DRM_GPUSVM Kconfigdrivers/gpu/drm/xe/Kconfig+1 -00 replies
8/32drm/xe/uapi: Add DRM_XE_VM_BIND_FLAG_CPU_ADDR_MIRRORdrivers/gpu/drm/xe/xe_pt.c drivers/gpu/drm/xe/xe_vm.c drivers/gpu/drm/xe/xe_vm.h +2 more+176 -750 replies
9/32drm/xe: Add SVM init / close / fini to faulting VMsdrivers/gpu/drm/xe/Makefile drivers/gpu/drm/xe/xe_svm.c drivers/gpu/drm/xe/xe_svm.h +2 more+128 -00 replies
10/32drm/xe: Add dma_addr res cursordrivers/gpu/drm/xe/xe_res_cursor.h drivers/gpu/drm/xe/xe_svm.h+125 -20 replies
11/32drm/xe: Nuke VM's mapping upon closedrivers/gpu/drm/xe/xe_gt_tlb_invalidation.c drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h drivers/gpu/drm/xe/xe_pt.c +2 more+73 -00 replies
12/32drm/xe: Add SVM range invalidation and page faultdrivers/gpu/drm/xe/xe_gt_pagefault.c drivers/gpu/drm/xe/xe_pt.c drivers/gpu/drm/xe/xe_pt.h +3 more+328 -60 replies
13/32drm/gpuvm: Add DRM_GPUVA_OP_DRIVERinclude/drm/drm_gpuvm.h+5 -00 replies
14/32drm/xe: Add (re)bind to SVM page fault handlerdrivers/gpu/drm/xe/xe_pt.c drivers/gpu/drm/xe/xe_pt_types.h drivers/gpu/drm/xe/xe_svm.c +4 more+340 -200 replies
15/32drm/xe: Add SVM garbage collectordrivers/gpu/drm/xe/xe_svm.c drivers/gpu/drm/xe/xe_svm.h drivers/gpu/drm/xe/xe_vm.c +1 more+116 -20 replies
16/32drm/xe: Add unbind to SVM garbage collectordrivers/gpu/drm/xe/xe_pt.c drivers/gpu/drm/xe/xe_svm.c drivers/gpu/drm/xe/xe_vm.c +2 more+176 -180 replies
17/32drm/xe: Do not allow CPU address mirror VMA unbind ifdrivers/gpu/drm/xe/xe_svm.c drivers/gpu/drm/xe/xe_svm.h drivers/gpu/drm/xe/xe_vm.c+39 -00 replies
18/32drm/xe: Enable CPU address mirror uAPIdrivers/gpu/drm/xe/xe_vm.c+2 -70 replies
19/32drm/xe/uapi: Add DRM_XE_QUERY_CONFIG_FLAG_HAS_CPU_ADDR_MIRRORdrivers/gpu/drm/xe/xe_query.c include/uapi/drm/xe_drm.h+7 -10 replies
20/32drm/xe: Add migrate layer functions for SVM supportdrivers/gpu/drm/xe/xe_migrate.c drivers/gpu/drm/xe/xe_migrate.h+185 -00 replies
21/32drm/xe: Add SVM device memory mirroringdrivers/gpu/drm/xe/Kconfig drivers/gpu/drm/xe/xe_device_types.h drivers/gpu/drm/xe/xe_svm.c +2 more+91 -20 replies
22/32drm/xe: Add drm_gpusvm_devmem to xe_bodrivers/gpu/drm/xe/xe_bo_types.h+4 -00 replies
23/32drm/xe: Add drm_pagemap ops to SVMdrivers/gpu/drm/xe/xe_device_types.h drivers/gpu/drm/xe/xe_svm.c+63 -00 replies
24/32drm/xe: Add GPUSVM device memory copy vfunc functionsdrivers/gpu/drm/xe/xe_svm.c+152 -00 replies
25/32drm/xe: Add Xe SVM populate_devmem_pfn GPU SVM vfuncdrivers/gpu/drm/xe/xe_svm.c+40 -00 replies
26/32drm/xe: Add Xe SVM devmem_release GPU SVM vfuncdrivers/gpu/drm/xe/xe_svm.c+9 -00 replies
27/32drm/xe: Add SVM VRAM migrationdrivers/gpu/drm/xe/xe_svm.c drivers/gpu/drm/xe/xe_svm.h+98 -40 replies
28/32drm/xe: Basic SVM BO evictiondrivers/gpu/drm/xe/xe_bo.c drivers/gpu/drm/xe/xe_bo.h drivers/gpu/drm/xe/xe_svm.c +1 more+48 -10 replies
29/32drm/xe: Add SVM debugdrivers/gpu/drm/xe/xe_pt.c drivers/gpu/drm/xe/xe_svm.c drivers/gpu/drm/xe/xe_svm.h+92 -70 replies
30/32drm/xe: Add modparam for SVM notifier sizedrivers/gpu/drm/xe/xe_module.c drivers/gpu/drm/xe/xe_module.h drivers/gpu/drm/xe/xe_svm.c+8 -10 replies
31/32drm/xe: Add always_migrate_to_vram modparamdrivers/gpu/drm/xe/xe_module.c drivers/gpu/drm/xe/xe_module.h drivers/gpu/drm/xe/xe_svm.c+7 -00 replies
32/32drm/doc: gpusvm: Add GPU SVM documentationDocumentation/gpu/rfc/gpusvm.rst Documentation/gpu/rfc/index.rst+110 -02 replies

Reviews trailers per patch

All 32 patches have a Reviewed-by or Acked-by.

PatchGwan-gyeong MunStuart SummersAlistair PoppleThomas HellströmHimal Prasad GhimirayMatthew BrostTejas Upadhyay
1/32R-b R-b
2/32R-b R-b
3/32R-b T-b
4/32R-b R-b
5/32T-b R-b R-b R-b
6/32R-b
7/32R-b R-b
8/32R-b R-b
9/32R-b R-b
10/32R-b
11/32R-b
12/32R-b R-b
13/32R-b
14/32R-b T-b
15/32R-b R-b
16/32R-b
17/32R-b R-b
18/32R-b
19/32R-b R-b R-b
20/32R-b
21/32R-b R-b
22/32R-b R-b
23/32R-b
24/32R-b
25/32R-b
26/32R-b R-b
27/32R-b R-b
28/32R-b
29/32R-b R-b
30/32R-b R-b
31/32R-b R-b
32/32A-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 7 reviewers.

Read the v7 thread

Cover letter [PATCH v7 00/32] Introduce GPU SVM and Xe SVM implementation

Version 7 of GPU SVM. Thanks to everyone (especially Sima, Thomas, Alistair, Himal) for their numerous reviews on revision 1, 2, 3 and for helping to address many design issues.

This version has been tested with IGT [1] on PVC, BMG, and LNL. Also tested with level0 (UMD) PR [2].

Major changes in v2:
- Dropped mmap write abuse
- core MM locking and retry loops instead of driver locking to avoid races
- Removed physical to virtual references
- Embedded structure/ops for drm_gpusvm_devmem
- Fixed mremap and fork issues
- Added DRM pagemap
- Included RFC documentation in the kernel doc

Major changes in v3:
- Move GPU SVM and DRM pagemap to DRM level
- Mostly addresses Thomas's feedback, lots of small changes documented in each individual patch change log

Major changes in v4:
- Pull documentation patch in
- Fix Kconfig / VRAM migration issue
- Address feedback which came out of internal multi-GPU implementation

Major changes in v5:
- Rebase on s/xe_mem_region/xe_vram_region
- Bit for uAPI has changed given PXP has landed

Major changes in v6:
- Fix selftest compile so GPU SVM compiles out on UML

Major changes in v7:
- Fix CI, addresses last nits, update GPU SVM doc

Known issues in v7:
- Check pages still exists, changed to threshold in this version which is better but still need to root cause cross process page finding on small user allocations.

Matt

[1] https://patchwork.freedesktop.org/series/137545/#rev3
[2] https://github.com/intel/compute-runtime/pull/782

Matthew Brost (28):
drm/xe: Retry BO allocation
mm/migrate: Add migrate_device_pfns
mm/migrate: Trylock device page in do_swap_page drm/gpusvm: Add support for GPU Shared Virtual Memory drm/xe: Select DRM_GPUSVM Kconfig
drm/xe/uapi: Add DRM_XE_VM_BIND_FLAG_CPU_ADDR_MIRROR drm/xe: Add SVM init / close / fini to faulting VMs drm/xe: Nuke VM's mapping upon close
drm/xe: Add SVM range invalidation and page fault drm/gpuvm: Add DRM_GPUVA_OP_DRIVER
drm/xe: Add (re)bind to SVM page fault handler drm/xe: Add SVM garbage collector
drm/xe: Add unbind to SVM garbage collector
drm/xe: Do not allow CPU address mirror VMA unbind if drm/xe: Enable CPU address mirror uAPI
drm/xe/uapi: Add DRM_XE_QUERY_CONFIG_FLAG_HAS_CPU_ADDR_MIRROR drm/xe: Add migrate layer functions for SVM support drm/xe: Add SVM device memory mirroring
drm/xe: Add drm_gpusvm_devmem to xe_bo
drm/xe: Add GPUSVM device memory copy vfunc functions drm/xe: Add Xe SVM populate_devmem_pfn GPU SVM vfunc drm/xe: Add Xe SVM devmem_release GPU SVM vfunc drm/xe: Add SVM VRAM migration
drm/xe: Basic SVM BO eviction
drm/xe: Add SVM debug
drm/xe: Add modparam for SVM notifier size
drm/xe: Add always_migrate_to_vram modparam drm/doc: gpusvm: Add GPU SVM documentation

Thomas Hellström (4):
drm/pagemap: Add DRM pagemap
drm/xe/bo: Introduce xe_bo_put_async
drm/xe: Add dma_addr res cursor
drm/xe: Add drm_pagemap ops to SVM

 Documentation/gpu/rfc/gpusvm.rst            |  106 +
 Documentation/gpu/rfc/index.rst             |    4 +
 drivers/gpu/drm/Kconfig                     |    9 +
 drivers/gpu/drm/Makefile                    |    1 +
 drivers/gpu/drm/drm_gpusvm.c                | 2236 +++++++++++++++++++
 drivers/gpu/drm/xe/Kconfig                  |   10 +
 drivers/gpu/drm/xe/Makefile                 |    1 +
 drivers/gpu/drm/xe/xe_bo.c                  |   54 +
 drivers/gpu/drm/xe/xe_bo.h                  |   20 +
 drivers/gpu/drm/xe/xe_bo_types.h            |    4 +
 drivers/gpu/drm/xe/xe_device.c              |    3 +
 drivers/gpu/drm/xe/xe_device_types.h        |   22 +
 drivers/gpu/drm/xe/xe_gt_pagefault.c        |   18 +-
 drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c |   22 +
 drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h |    2 +
 drivers/gpu/drm/xe/xe_migrate.c             |  175 ++
 drivers/gpu/drm/xe/xe_migrate.h             |   10 +
 drivers/gpu/drm/xe/xe_module.c              |    7 +
 drivers/gpu/drm/xe/xe_module.h              |    2 +
 drivers/gpu/drm/xe/xe_pt.c                  |  399 +++-
 drivers/gpu/drm/xe/xe_pt.h                  |    5 +
 drivers/gpu/drm/xe/xe_pt_types.h            |    2 +
 drivers/gpu/drm/xe/xe_query.c               |    5 +-
 drivers/gpu/drm/xe/xe_res_cursor.h          |  123 +-
 drivers/gpu/drm/xe/xe_svm.c                 |  946 ++++++++
 drivers/gpu/drm/xe/xe_svm.h                 |  150 ++
 drivers/gpu/drm/xe/xe_tile.c                |    5 +
 drivers/gpu/drm/xe/xe_vm.c                  |  381 +++-
 drivers/gpu/drm/xe/xe_vm.h                  |   15 +-
 drivers/gpu/drm/xe/xe_vm_types.h            |   57 +
 include/drm/drm_gpusvm.h                    |  509 +++++
 include/drm/drm_gpuvm.h                     |    5 +
 include/drm/drm_pagemap.h                   |  107 +
 include/linux/migrate.h                     |    1 +
 include/uapi/drm/xe_drm.h                   |   22 +-
 mm/memory.c                                 |   13 +-
 mm/migrate_device.c                         |  116 +-
 37 files changed, 5412 insertions(+), 155 deletions(-)
 create mode 100644 Documentation/gpu/rfc/gpusvm.rst
 create mode 100644 drivers/gpu/drm/drm_gpusvm.c
 create mode 100644 drivers/gpu/drm/xe/xe_svm.c
 create mode 100644 drivers/gpu/drm/xe/xe_svm.h
 create mode 100644 include/drm/drm_gpusvm.h
 create mode 100644 include/drm/drm_pagemap.h
Signature · 1 line
2.34.1