Skip to content

RFC DRM GPUVA Manager GPU-VM features

Danilo Krummrich dri-devel lkml nouveau oe-kbuild-all 4 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 RFC Latest8 patches 19 repliesIn review
  2. v3 RFC7 patches 67 repliesSuperseded
  3. v2 RFC7 patches 7 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v1 RFC3 patches23 repliesSuperseded

Changes in v4 from the cover letter

==============
- add a drm_gpuvm_resv() helper (Boris)
- add a drm_gpuvm::<list_name>::local_list field (Boris)
- remove drm_gpuvm_bo_get_unless_zero() helper (Boris)
- fix missing NULL assignment in get_next_vm_bo_from_list() (Boris)
- keep a drm_gem_object reference on potential vm_bo destroy (alternatively we could free the vm_bo and drop the vm_bo's drm_gem_object reference through async work)
- introduce DRM_GPUVM_RESV_PROTECTED flag to indicate external locking through the corresponding dma-resv locks to optimize for drivers already holding them when needed; add the corresponding lock_assert_held() calls (Thomas)
- make drm_gpuvm_bo_evict() per vm_bo and add a drm_gpuvm_bo_gem_evict() helper (Thomas)
- pass a drm_gpuvm_bo in drm_gpuvm_ops::vm_bo_validate() (Thomas)
- documentation fixes

Patches in v4 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/8drm/gpuvm: rename struct drm_gpuva_manager to struct drm_gpuvmdrivers/gpu/drm/Makefile drivers/gpu/drm/drm_debugfs.c drivers/gpu/drm/drm_gpuvm.c +5 more+304 -3052 replies
2/8drm/gpuvm: allow building as moduledrivers/gpu/drm/Kconfig drivers/gpu/drm/Makefile drivers/gpu/drm/drm_gpuvm.c +1 more+12 -11 reply
3/8drm/nouveau: uvmm: rename 'umgr' to 'base'drivers/gpu/drm/nouveau/nouveau_debugfs.c drivers/gpu/drm/nouveau/nouveau_exec.c drivers/gpu/drm/nouveau/nouveau_uvmm.c +1 more+22 -221 reply
4/8drm/gpuvm: add common dma-resv per struct drm_gpuvmdrivers/gpu/drm/drm_gpuvm.c drivers/gpu/drm/nouveau/nouveau_uvmm.c include/drm/drm_gpuvm.h+24 -48 replies
5/8drm/gpuvm: add an abstraction for a VM / BO combinationdrivers/gpu/drm/drm_gpuvm.c drivers/gpu/drm/nouveau/nouveau_uvmm.c include/drm/drm_gem.h +1 more+483 -750 replies
6/8drm/gpuvm: add drm_gpuvm_flags to drm_gpuvmdrivers/gpu/drm/drm_gpuvm.c drivers/gpu/drm/nouveau/nouveau_uvmm.c include/drm/drm_gpuvm.h+19 -34 replies
7/8drm/gpuvm: generalize dma_resv/extobj handling and GEM validationdrivers/gpu/drm/drm_gpuvm.c include/drm/drm_gpuvm.h+894 -12 replies
8/8drm/nouveau: GPUVM dma-resv/extobj handling, GEM validationdrivers/gpu/drm/nouveau/nouveau_bo.c drivers/gpu/drm/nouveau/nouveau_exec.c drivers/gpu/drm/nouveau/nouveau_exec.h +3 more+83 -850 replies

Reviews trailers per patch

3 of 8 patches have a Reviewed-by or Acked-by.

PatchChristian KönigDave AirlieBoris Brezillon
SeriesT-b
1/8A-b
2/8R-b R-b
3/8R-b
4/8
5/8
6/8
7/8
8/8

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.

Read the v4 thread

Cover letter [PATCH drm-misc-next v4 0/8] [RFC] DRM GPUVA Manager GPU-VM features

So far the DRM GPUVA manager offers common infrastructure to track GPU VA allocations and mappings, generically connect GPU VA mappings to their backing buffers and perform more complex mapping operations on the GPU VA space.

However, there are more design patterns commonly used by drivers, which can potentially be generalized in order to make the DRM GPUVA manager represent a basic GPU-VM implementation. In this context, this patch series aims at generalizing the following elements.

1) Provide a common dma-resv for GEM objects not being used outside of this GPU-VM.

2) Provide tracking of external GEM objects (GEM objects which are shared with other GPU-VMs).

3) Provide functions to efficiently lock all GEM objects dma-resv the GPU-VM contains mappings of.

4) Provide tracking of evicted GEM objects the GPU-VM contains mappings of, such that validation of evicted GEM objects is accelerated.

5) Provide some convinience functions for common patterns.

The implementation introduces struct drm_gpuvm_bo, which serves as abstraction combining a struct drm_gpuvm and struct drm_gem_object, similar to what amdgpu does with struct amdgpu_bo_vm. While this adds a bit of complexity it improves the efficiency of tracking external and evicted GEM objects.

This patch series also renames struct drm_gpuva_manager to struct drm_gpuvm including corresponding functions. This way the GPUVA manager's structures align better with the documentation of VM_BIND [1] and VM_BIND locking [2]. It also provides a better foundation for the naming of data structures and functions introduced for implementing the features of this patch series.

This patch series is also available at [3].

[1] Documentation/gpu/drm-vm-bind-async.rst
[2] Documentation/gpu/drm-vm-bind-locking.rst
[3] https://gitlab.freedesktop.org/nouvelles/kernel/-/commits/gpuvm-next

Changes in V2:
==============
- rename 'drm_gpuva_manager' -> 'drm_gpuvm' which generally leads to more consistent naming
- properly separate commits (introduce common dma-resv, drm_gpuvm_bo abstraction, etc.)
- remove maple tree for tracking external objects, use a list drm_gpuvm_bos per drm_gpuvm instead
- rework dma-resv locking helpers (Thomas)
- add a locking helper for a given range of the VA space (Christian)
- make the GPUVA manager buildable as module, rather than drm_exec builtin (Christian)

Changes in V3:
==============
- rename missing function and files (Boris)
- warn if vm_obj->obj != obj in drm_gpuva_link() (Boris)
- don't expose drm_gpuvm_bo_destroy() (Boris)
- unlink VM_BO from GEM in drm_gpuvm_bo_destroy() rather than drm_gpuva_unlink() and link within drm_gpuvm_bo_obtain() to keep drm_gpuvm_bo instances unique
- add internal locking to external and evicted object lists to support drivers updating the VA space from within the fence signalling critical path (Boris)
- unlink external objects and evicted objects from the GPUVM's list in drm_gpuvm_bo_destroy()
- add more documentation and fix some kernel doc issues

Changes in V4:
==============
- add a drm_gpuvm_resv() helper (Boris)
- add a drm_gpuvm::<list_name>::local_list field (Boris)
- remove drm_gpuvm_bo_get_unless_zero() helper (Boris)
- fix missing NULL assignment in get_next_vm_bo_from_list() (Boris)
- keep a drm_gem_object reference on potential vm_bo destroy (alternatively we could free the vm_bo and drop the vm_bo's drm_gem_object reference through async work)
- introduce DRM_GPUVM_RESV_PROTECTED flag to indicate external locking through the corresponding dma-resv locks to optimize for drivers already holding them when needed; add the corresponding lock_assert_held() calls (Thomas)
- make drm_gpuvm_bo_evict() per vm_bo and add a drm_gpuvm_bo_gem_evict() helper (Thomas)
- pass a drm_gpuvm_bo in drm_gpuvm_ops::vm_bo_validate() (Thomas)
- documentation fixes

Danilo Krummrich (8):
drm/gpuvm: rename struct drm_gpuva_manager to struct drm_gpuvm drm/gpuvm: allow building as module
drm/nouveau: uvmm: rename 'umgr' to 'base'
drm/gpuvm: add common dma-resv per struct drm_gpuvm drm/gpuvm: add an abstraction for a VM / BO combination drm/gpuvm: add drm_gpuvm_flags to drm_gpuvm
drm/gpuvm: generalize dma_resv/extobj handling and GEM validation drm/nouveau: GPUVM dma-resv/extobj handling, GEM validation

 drivers/gpu/drm/Kconfig                   |    7 +
 drivers/gpu/drm/Makefile                  |    2 +-
 drivers/gpu/drm/drm_debugfs.c             |   16 +-
 drivers/gpu/drm/drm_gpuva_mgr.c           | 1725 --------------
 drivers/gpu/drm/drm_gpuvm.c               | 2600 +++++++++++++++++++++
 drivers/gpu/drm/nouveau/Kconfig           |    1 +
 drivers/gpu/drm/nouveau/nouveau_bo.c      |    4 +-
 drivers/gpu/drm/nouveau/nouveau_debugfs.c |    2 +-
 drivers/gpu/drm/nouveau/nouveau_exec.c    |   52 +-
 drivers/gpu/drm/nouveau/nouveau_exec.h    |    4 -
 drivers/gpu/drm/nouveau/nouveau_gem.c     |    5 +-
 drivers/gpu/drm/nouveau/nouveau_sched.h   |    4 +-
 drivers/gpu/drm/nouveau/nouveau_uvmm.c    |  207 +-
 drivers/gpu/drm/nouveau/nouveau_uvmm.h    |    8 +-
 include/drm/drm_debugfs.h                 |    6 +-
 include/drm/drm_gem.h                     |   32 +-
 include/drm/drm_gpuva_mgr.h               |  706 ------
 include/drm/drm_gpuvm.h                   | 1142 +++++++++
 18 files changed, 3934 insertions(+), 2589 deletions(-)
 delete mode 100644 drivers/gpu/drm/drm_gpuva_mgr.c
 create mode 100644 drivers/gpu/drm/drm_gpuvm.c
 delete mode 100644 include/drm/drm_gpuva_mgr.h
 create mode 100644 include/drm/drm_gpuvm.h

base-commit: 1c7a387ffef894b1ab3942f0482dac7a6e0a909c

Signature · 1 line
2.41.0