drm/msm: sparse / "VM_BIND" support
Subsystems (7)
v2 cover drm/msm: sparse / "VM_BIND" support
From: Rob Clark <[email protected]>
Conversion to DRM GPU VA Manager[1], and adding support for Vulkan Sparse Memory[2] in the form of:
1. A new VM_BIND submitqueue type for executing VM MSM_SUBMIT_BO_OP_MAP/ MAP_NULL/UNMAP commands
2. Extending the SUBMIT ioctl to allow submitting batches of one or more MAP/MAP_NULL/UNMAP commands to a VM_BIND submitqueue
The UABI takes a slightly different approach from what other drivers have done, and what would make sense if starting from a clean sheet, ie separate VM_BIND and EXEC ioctls. But since we have to maintain support for the existing SUBMIT ioctl, and because the fence, syncobj, and BO pinning is largely the same between legacy "BO-table" style SUBMIT ioctls, and new- style VM updates submitted to a VM_BIND submitqueue, I chose to go the route of extending the existing SUBMIT ioctl rather than adding a new ioctl.
I also did not implement support for synchronous VM_BIND commands. Since userspace could just immediately wait for the SUBMIT to complete, I don't think we need this extra complexity in the kernel. Synchronous/immediate VM_BIND operations could be implemented with a 2nd VM_BIND submitqueue.
The corresponding mesa MR: https://gitlab.freedesktop.org/mesa/mesa/-/merge_requests/32533
This series can be found in MR form, if you prefer:
https://gitlab.freedesktop.org/drm/msm/-/merge_requests/144
Changes in v2:
- Dropped Bibek Kumar Patro's arm-smmu patches[3], which have since been
merged.
- Pre-allocate all the things, and drop HACK patch which disabled shrinker.
This includes ensuring that vm_bo objects are allocated up front, pre-
allocating VMA objects, and pre-allocating pages used for pgtable updates.
The latter utilizes io_pgtable_cfg callbacks for pgtable alloc/free, that
were initially added for panthor.
- Add back support for BO dumping for devcoredump.
- Link to v1 (RFC): https://lore.kernel.org/dri-devel/[email protected]/T/#t
[1] https://www.kernel.org/doc/html/next/gpu/drm-mm.html#drm-gpuvm
[2] https://docs.vulkan.org/spec/latest/chapters/sparsemem.html
[3] https://patchwork.kernel.org/project/linux-arm-kernel/list/?series=909700
Rob Clark (34):
drm/gpuvm: Don't require obj lock in destructor path drm/gpuvm: Remove bogus lock assert drm/gpuvm: Allow VAs to hold soft reference to BOs drm/gpuvm: Add drm_gpuvm_sm_unmap_va() drm/msm: Rename msm_file_private -> msm_context drm/msm: Improve msm_context comments drm/msm: Rename msm_gem_address_space -> msm_gem_vm drm/msm: Remove vram carveout support drm/msm: Collapse vma allocation and initialization drm/msm: Collapse vma close and delete drm/msm: drm_gpuvm conversion
drm/msm: Use drm_gpuvm types more
drm/msm: Split submit_pin_objects() drm/msm: Lazily create context VM
drm/msm: Add opt-in for VM_BIND
drm/msm: Mark VM as unusable on faults drm/msm: Extend SUBMIT ioctl for VM_BIND drm/msm: Add VM_BIND submitqueue
drm/msm: Add _NO_SHARE flag
drm/msm: Split out helper to get iommu prot flags drm/msm: Add mmu support for non-zero offset drm/msm: Add PRR support
drm/msm: Rename msm_gem_vma_purge() -> _unmap() drm/msm: Split msm_gem_vma_new()
drm/msm: Pre-allocate VMAs
drm/msm: Pre-allocate vm_bo objects drm/msm: Pre-allocate pages for pgtable entries drm/msm: Wire up gpuvm ops
drm/msm: Wire up drm_gpuvm debugfs
drm/msm: Crashdump prep for sparse mappings drm/msm: rd dumping prep for sparse mappings drm/msm: Crashdec support for sparse drm/msm: rd dumping support for sparse drm/msm: Bump UAPI version
drivers/gpu/drm/drm_gpuvm.c | 141 ++-- drivers/gpu/drm/msm/Kconfig | 1 + drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 25 +- drivers/gpu/drm/msm/adreno/a2xx_gpummu.c | 5 +- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 17 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 17 +- drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 24 +- drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +- drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 10 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 32 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 2 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 51 +- drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 6 +- drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 10 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 4 - drivers/gpu/drm/msm/adreno/adreno_gpu.c | 84 +- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 23 +- .../drm/msm/disp/dpu1/dpu_encoder_phys_wb.c | 14 +- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c | 18 +- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.h | 2 +- drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c | 18 +- drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c | 14 +- drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h | 4 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_crtc.c | 6 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c | 28 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_plane.c | 12 +- drivers/gpu/drm/msm/disp/mdp5/mdp5_crtc.c | 4 +- drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c | 19 +- drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c | 12 +- drivers/gpu/drm/msm/dsi/dsi_host.c | 14 +- drivers/gpu/drm/msm/msm_debugfs.c | 20 + drivers/gpu/drm/msm/msm_drv.c | 176 ++--- drivers/gpu/drm/msm/msm_drv.h | 35 +- drivers/gpu/drm/msm/msm_fb.c | 18 +- drivers/gpu/drm/msm/msm_fbdev.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 437 +++++----- drivers/gpu/drm/msm/msm_gem.h | 226 ++++-- drivers/gpu/drm/msm/msm_gem_prime.c | 15 + drivers/gpu/drm/msm/msm_gem_submit.c | 234 +++++- drivers/gpu/drm/msm/msm_gem_vma.c | 748 ++++++++++++++++-- drivers/gpu/drm/msm/msm_gpu.c | 146 ++-- drivers/gpu/drm/msm/msm_gpu.h | 132 +++- drivers/gpu/drm/msm/msm_iommu.c | 285 ++++++- drivers/gpu/drm/msm/msm_kms.c | 18 +- drivers/gpu/drm/msm/msm_kms.h | 2 +- drivers/gpu/drm/msm/msm_mmu.h | 38 +- drivers/gpu/drm/msm/msm_rd.c | 62 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 4 +- drivers/gpu/drm/msm/msm_submitqueue.c | 86 +- include/drm/drm_gpuvm.h | 14 +- include/uapi/drm/msm_drm.h | 98 ++- 52 files changed, 2359 insertions(+), 1060 deletions(-)
Signature · 1 line
2.48.1
PATCH v2 1/34 drm/gpuvm: Don't require obj lock in destructor path
From: Rob Clark <[email protected]>
See commit a414fe3a2129 ("drm/msm/gem: Drop obj lock in msm_gem_free_object()") for justification.
--- drivers/gpu/drm/drm_gpuvm.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
drivers/gpu/drm/drm_gpuvm.c +5 -2
@@ -1511,7 +1511,9 @@ drm_gpuvm_bo_destroy(struct kref *kref)drm_gpuvm_bo_list_del (vm_bo, extobj, lock);drm_gpuvm_bo_list_del (vm_bo, evict, lock); -drm_gem_gpuva_assert_lock_held (obj); +if (kref_read (&obj->refcount) >0 ) +drm_gem_gpuva_assert_lock_held (obj); +list_del (&vm_bo->list.entry.gem);if (ops && ops->vm_bo_free)
@@ -1871,7 +1873,8 @@ drm_gpuva_unlink(struct drm_gpuva *va)if (unlikely (!obj))return ; -drm_gem_gpuva_assert_lock_held (obj); +if (kref_read (&obj->refcount) >0 ) +drm_gem_gpuva_assert_lock_held (obj);list_del_init (&va->gem.entry); va->vm_bo =NULL ;
Signature · 1 line
2.48.1
PATCH v2 2/34 drm/gpuvm: Remove bogus lock assert
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
If the driver is using an external mutex to synchronize vm access, it doesn't need to hold vm->r_obj->resv. And if the driver is already holding obj->resv, then needing to pointlessly grab vm->r_obj->resv will be seen by lockdep as nested locking.
--- drivers/gpu/drm/drm_gpuvm.c | 3 --- 1 file changed, 3 deletions(-)
drivers/gpu/drm/drm_gpuvm.c +0 -3
@@ -1505,9 +1505,6 @@ drm_gpuvm_bo_destroy(struct kref *kref)struct drm_gem_object *obj = vm_bo->obj;bool lock = !drm_gpuvm_resv_protected (gpuvm); -if (!lock) -drm_gpuvm_resv_assert_held (gpuvm); -drm_gpuvm_bo_list_del (vm_bo, extobj, lock);drm_gpuvm_bo_list_del (vm_bo, evict, lock);
Signature · 1 line
2.48.1
PATCH v2 3/34 drm/gpuvm: Allow VAs to hold soft reference to BOs
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Eases migration for drivers where VAs don't hold hard references to their associated BO, avoiding reference loops.
In particular, msm uses soft references to optimistically keep around mappings until the BO is distroyed. Which obviously won't work if the VA (the mapping) is holding a reference to the BO.
By making this a per-VM flag, we can use normal hard-references for mappings in a "VM_BIND" managed VM, but soft references in other cases, such as kernel-internal VMs (for display scanout, etc).
--- drivers/gpu/drm/drm_gpuvm.c | 8 ++++++-- include/drm/drm_gpuvm.h | 12 ++++++++++-- 2 files changed, 16 insertions(+), 4 deletions(-)
drivers/gpu/drm/drm_gpuvm.c +6 -2
@@ -1482,7 +1482,9 @@ drm_gpuvm_bo_create(struct drm_gpuvm *gpuvm, vm_bo->vm =drm_gpuvm_get (gpuvm); vm_bo->obj = obj; -drm_gem_object_get (obj); + +if (!(gpuvm->flags &DRM_GPUVM_VA_WEAK_REF )) +drm_gem_object_get (obj);kref_init (&vm_bo->kref);INIT_LIST_HEAD (&vm_bo->list.gpuva);
@@ -1504,6 +1506,7 @@ drm_gpuvm_bo_destroy(struct kref *kref)const struct drm_gpuvm_ops *ops = gpuvm->ops;struct drm_gem_object *obj = vm_bo->obj;bool lock = !drm_gpuvm_resv_protected (gpuvm); +bool unref = !(gpuvm->flags &DRM_GPUVM_VA_WEAK_REF );drm_gpuvm_bo_list_del (vm_bo, extobj, lock);drm_gpuvm_bo_list_del (vm_bo, evict, lock);
@@ -1519,7 +1522,8 @@ drm_gpuvm_bo_destroy(struct kref *kref)kfree (vm_bo);drm_gpuvm_put (gpuvm); -drm_gem_object_put (obj); +if (unref) +drm_gem_object_put (obj); }/**
include/drm/drm_gpuvm.h +10 -2
@@ -205,10 +205,18 @@ enum drm_gpuvm_flags {*/ DRM_GPUVM_RESV_PROTECTED =BIT (0 ), +/** +* @DRM_GPUVM_VA_WEAK_REF: +* +* Flag indicating that the &drm_gpuva (or more correctly, the +* &drm_gpuvm_bo) only holds a weak reference to the &drm_gem_object. +*/ +DRM_GPUVM_VA_WEAK_REF =BIT (1 ), +/** * @DRM_GPUVM_USERBITS: user defined bits */ - DRM_GPUVM_USERBITS = BIT(1), + DRM_GPUVM_USERBITS = BIT(2), };/**
@@ -651,7 +659,7 @@ struct drm_gpuvm_bo {/** * @obj: The &drm_gem_object being mapped in @vm. This is a reference -* counted pointer. +* counted pointer, unless the &DRM_GPUVM_VA_WEAK_REF flag is set. */ struct drm_gem_object *obj;
Signature · 1 line
2.48.1
PATCH v2 4/34 drm/gpuvm: Add drm_gpuvm_sm_unmap_va()
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Add drm_gpuvm_sm_unmap_va(), which works similarly to drm_gpuvm_sm_unmap(), except that it operates on a single VA at a time.
This is needed by drm/msm, where due to locking ordering we cannot reuse drm_gpuvm_sm_unmap() as-is. This helper lets us reuse the logic about remaps vs unmaps cases.
--- drivers/gpu/drm/drm_gpuvm.c | 123 ++++++++++++++++++++++++------------ include/drm/drm_gpuvm.h | 2 + 2 files changed, 85 insertions(+), 40 deletions(-)
drivers/gpu/drm/drm_gpuvm.c +83 -40
@@ -2244,6 +2244,56 @@ __drm_gpuvm_sm_map(struct drm_gpuvm *gpuvm, req_obj, req_offset); } +static int +__drm_gpuvm_sm_unmap_va (struct drm_gpuva *va,const struct drm_gpuvm_ops *ops, +void *priv,u64 req_addr,u64 req_range) +{ +struct drm_gpuva_op_map prev = {}, next = {}; +bool prev_split = false, next_split = false; +struct drm_gem_object *obj = va->gem.obj; +u64 req_end = req_addr + req_range; +u64 offset = va->gem.offset; +u64 addr = va->va.addr; +u64 range = va->va.range; +u64 end = addr + range; +int ret; + +if (addr < req_addr) { + prev.va.addr = addr; + prev.va.range = req_addr - addr; + prev.gem.obj = obj; + prev.gem.offset = offset; + + prev_split = true; + } + +if (end > req_end) { + next.va.addr = req_end; + next.va.range = end - req_end; + next.gem.obj = obj; + next.gem.offset = offset + (req_end - addr); + + next_split = true; + } + +if (prev_split || next_split) { +struct drm_gpuva_op_unmap unmap = { .va = va }; + + ret =op_remap_cb (ops, priv, + prev_split ? &prev :NULL , + next_split ? &next :NULL , + &unmap); +if (ret) +return ret; + }else { + ret =op_unmap_cb (ops, priv, va, false); +if (ret) +return ret; + } + +return 0 ; +} + static int __drm_gpuvm_sm_unmap(struct drm_gpuvm *gpuvm, const struct drm_gpuvm_ops *ops, void *priv,
@@ -2257,46 +2307,9 @@ __drm_gpuvm_sm_unmap(struct drm_gpuvm *gpuvm,return -EINVAL ;drm_gpuvm_for_each_va_range_safe (va, next, gpuvm, req_addr, req_end) { -struct drm_gpuva_op_map prev = {}, next = {}; -bool prev_split = false, next_split = false; -struct drm_gem_object *obj = va->gem.obj; -u64 offset = va->gem.offset; -u64 addr = va->va.addr; -u64 range = va->va.range; -u64 end = addr + range; - -if (addr < req_addr) { - prev.va.addr = addr; - prev.va.range = req_addr - addr; - prev.gem.obj = obj; - prev.gem.offset = offset; - - prev_split = true; - } - -if (end > req_end) { - next.va.addr = req_end; - next.va.range = end - req_end; - next.gem.obj = obj; - next.gem.offset = offset + (req_end - addr); - - next_split = true; - } - -if (prev_split || next_split) { -struct drm_gpuva_op_unmap unmap = { .va = va }; - - ret =op_remap_cb (ops, priv, - prev_split ? &prev :NULL , - next_split ? &next :NULL , - &unmap); -if (ret) -return ret; - }else { - ret =op_unmap_cb (ops, priv, va, false); -if (ret) -return ret; - } + ret =__drm_gpuvm_sm_unmap_va (va, ops, priv, req_addr, req_range); +if (ret) +return ret; }return 0 ;
@@ -2394,6 +2407,36 @@ drm_gpuvm_sm_unmap(struct drm_gpuvm *gpuvm, void *priv, }EXPORT_SYMBOL_GPL (drm_gpuvm_sm_unmap); +/** +* drm_gpuvm_sm_unmap_va() - like drm_gpuvm_sm_unmap() but operating on a single VA +* @va: the &drm_gpuva to unmap from +* @priv: pointer to a driver private data structure +* @req_addr: the start address of the range to unmap +* @req_range: the range of the mappings to unmap +* +* This function iterates the range of a single VA. It utilizes the +* &drm_gpuvm_ops to call back into the driver providing the operations to +* unmap and, if required, split the existent mapping. +* +* There can be at most one unmap or remap operation, depending on whether the +* requested unmap range fully covers or partially covers the specified VA. +* +* Returns: 0 on success or a negative error code +*/ +int +drm_gpuvm_sm_unmap_va (struct drm_gpuva *va,void *priv, +u64 req_addr,u64 req_range) +{ +const struct drm_gpuvm_ops *ops = va->vm->ops; + +if (unlikely (!(ops && ops->sm_step_remap && + ops->sm_step_unmap))) +return -EINVAL ; + +return __drm_gpuvm_sm_unmap_va (va, ops, priv, req_addr, req_range); +} +EXPORT_SYMBOL_GPL (drm_gpuvm_sm_unmap_va); +static struct drm_gpuva_op *gpuva_op_alloc (struct drm_gpuvm *gpuvm) {
include/drm/drm_gpuvm.h +2 -0
@@ -1213,6 +1213,8 @@ int drm_gpuvm_sm_map(struct drm_gpuvm *gpuvm, void *priv,int drm_gpuvm_sm_unmap (struct drm_gpuvm *gpuvm,void *priv,u64 addr,u64 range); +int drm_gpuvm_sm_unmap_va (struct drm_gpuva *va,void *priv, +u64 req_addr,u64 req_range); void drm_gpuva_map(struct drm_gpuvm *gpuvm, struct drm_gpuva *va,
Signature · 1 line
2.48.1
PATCH v2 5/34 drm/msm: Rename msm_file_private -> msm_context
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
This is a more descriptive name.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 ++-- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 4 +-- drivers/gpu/drm/msm/msm_drv.c | 14 ++++----- drivers/gpu/drm/msm/msm_gem.c | 2 +- drivers/gpu/drm/msm/msm_gem_submit.c | 2 +- drivers/gpu/drm/msm/msm_gpu.c | 4 +-- drivers/gpu/drm/msm/msm_gpu.h | 39 ++++++++++++------------- drivers/gpu/drm/msm/msm_submitqueue.c | 27 +++++++++-------- 9 files changed, 49 insertions(+), 51 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -1
@@ -111,7 +111,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu, struct msm_ringbuffer *ring, struct msm_gem_submit *submit) { bool sysprof = refcount_read(&a6xx_gpu->base.base.sysprof_active) > 1; -struct msm_file_private *ctx = submit->queue->ctx; +struct msm_context *ctx = submit->queue->ctx;struct adreno_gpu *adreno_gpu = &a6xx_gpu->base;phys_addr_t ttbr;u32 asid;
drivers/gpu/drm/msm/adreno/adreno_gpu.c +3 -3
@@ -343,7 +343,7 @@ int adreno_fault_handler(struct msm_gpu *gpu, unsigned long iova, int flags,return 0 ; } -int adreno_get_param (struct msm_gpu *gpu,struct msm_file_private *ctx, +int adreno_get_param (struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t *value,uint32_t *len) {struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);
@@ -431,7 +431,7 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_file_private *ctx, } } -int adreno_set_param (struct msm_gpu *gpu,struct msm_file_private *ctx, +int adreno_set_param (struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t value,uint32_t len) {struct drm_device *drm = gpu->dev;
@@ -477,7 +477,7 @@ int adreno_set_param(struct msm_gpu *gpu, struct msm_file_private *ctx,case MSM_PARAM_SYSPROF :if (!capable (CAP_SYS_ADMIN ))return UERR (EPERM , drm,"invalid permissions" ); -return msm_file_private_set_sysprof (ctx, gpu, value); +return msm_context_set_sysprof (ctx, gpu, value);default :return UERR (EINVAL , drm,"%s: invalid param: %u" , gpu->name, param); }
drivers/gpu/drm/msm/adreno/adreno_gpu.h +2 -2
@@ -601,9 +601,9 @@ static inline int adreno_is_a7xx(struct adreno_gpu *gpu) }u64 adreno_private_address_space_size (struct msm_gpu *gpu); -int adreno_get_param (struct msm_gpu *gpu,struct msm_file_private *ctx, +int adreno_get_param (struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t *value,uint32_t *len); -int adreno_set_param (struct msm_gpu *gpu,struct msm_file_private *ctx, +int adreno_set_param (struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t value,uint32_t len);const struct firmware *adreno_request_fw (struct adreno_gpu *adreno_gpu,const char *fwname);
drivers/gpu/drm/msm/msm_drv.c +7 -7
@@ -333,7 +333,7 @@ static int context_init(struct drm_device *dev, struct drm_file *file) {static atomic_t ident =ATOMIC_INIT (0 );struct msm_drm_private *priv = dev->dev_private; -struct msm_file_private *ctx; +struct msm_context *ctx; ctx =kzalloc (sizeof (*ctx),GFP_KERNEL );if (!ctx)
@@ -363,23 +363,23 @@ static int msm_open(struct drm_device *dev, struct drm_file *file)return context_init (dev, file); } -static void context_close (struct msm_file_private *ctx) +static void context_close (struct msm_context *ctx) {msm_submitqueue_close (ctx); -msm_file_private_put (ctx); +msm_context_put (ctx); }static void msm_postclose (struct drm_device *dev,struct drm_file *file) {struct msm_drm_private *priv = dev->dev_private; -struct msm_file_private *ctx = file->driver_priv; +struct msm_context *ctx = file->driver_priv;/* * It is not possible to set sysprof param to non-zero if gpu * is not initialized: */ if (priv->gpu) -msm_file_private_set_sysprof (ctx, priv->gpu,0 ); +msm_context_set_sysprof (ctx, priv->gpu,0 );context_close (ctx); }
@@ -511,7 +511,7 @@ static int msm_ioctl_gem_info_iova(struct drm_device *dev, uint64_t *iova) {struct msm_drm_private *priv = dev->dev_private; -struct msm_file_private *ctx = file->driver_priv; +struct msm_context *ctx = file->driver_priv;if (!priv->gpu)return -EINVAL ;
@@ -531,7 +531,7 @@ static int msm_ioctl_gem_info_set_iova(struct drm_device *dev, uint64_t iova) {struct msm_drm_private *priv = dev->dev_private; -struct msm_file_private *ctx = file->driver_priv; +struct msm_context *ctx = file->driver_priv;if (!priv->gpu)return -EINVAL ;
drivers/gpu/drm/msm/msm_gem.c +1 -1
@@ -48,7 +48,7 @@ static void update_device_mem(struct msm_drm_private *priv, ssize_t size)static void update_ctx_mem (struct drm_file *file,ssize_t size) { -struct msm_file_private *ctx = file->driver_priv; +struct msm_context *ctx = file->driver_priv;uint64_t ctx_mem =atomic64_add_return (size, &ctx->ctx_mem);rcu_read_lock ();/* Locks file->pid! */
drivers/gpu/drm/msm/msm_gem_submit.c +1 -1
@@ -642,7 +642,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, {struct msm_drm_private *priv = dev->dev_private;struct drm_msm_gem_submit *args = data; -struct msm_file_private *ctx = file->driver_priv; +struct msm_context *ctx = file->driver_priv;struct msm_gem_submit *submit =NULL ;struct msm_gpu *gpu = priv->gpu;struct msm_gpu_submitqueue *queue;
drivers/gpu/drm/msm/msm_gpu.c +2 -2
@@ -148,7 +148,7 @@ int msm_gpu_pm_suspend(struct msm_gpu *gpu)return 0 ; } -void msm_gpu_show_fdinfo (struct msm_gpu *gpu,struct msm_file_private *ctx, +void msm_gpu_show_fdinfo (struct msm_gpu *gpu,struct msm_context *ctx,struct drm_printer *p) {drm_printf (p,"drm-engine-gpu:\t%llu ns\n" , ctx->elapsed_ns);
@@ -339,7 +339,7 @@ static void retire_submits(struct msm_gpu *gpu);static void get_comm_cmdline (struct msm_gem_submit *submit,char **comm,char **cmd) { -struct msm_file_private *ctx = submit->queue->ctx; +struct msm_context *ctx = submit->queue->ctx;struct task_struct *task;WARN_ON (!mutex_is_locked (&submit->gpu->lock));
drivers/gpu/drm/msm/msm_gpu.h +19 -20
@@ -22,7 +22,7 @@struct msm_gem_submit ;struct msm_gpu_perfcntr ;struct msm_gpu_state ; -struct msm_file_private ; +struct msm_context ;struct msm_gpu_config {const char *ioname;
@@ -44,9 +44,9 @@ struct msm_gpu_config {* + z180_gpu */ struct msm_gpu_funcs { -int (*get_param)(struct msm_gpu *gpu,struct msm_file_private *ctx, +int (*get_param)(struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t *value,uint32_t *len); -int (*set_param)(struct msm_gpu *gpu,struct msm_file_private *ctx, +int (*set_param)(struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t value,uint32_t len);int (*hw_init)(struct msm_gpu *gpu);
@@ -347,7 +347,7 @@ struct msm_gpu_perfcntr {#define NR_SCHED_PRIORITIES (1 + DRM_SCHED_PRIORITY_LOW - DRM_SCHED_PRIORITY_HIGH)/** -* struct msm_file_private - per-drm_file context +* struct msm_context - per-drm_file context * * @queuelock: synchronizes access to submitqueues list * @submitqueues: list of &msm_gpu_submitqueue created by userspace
@@ -357,7 +357,7 @@ struct msm_gpu_perfcntr {* @ref: reference count * @seqno: unique per process seqno */ -struct msm_file_private { +struct msm_context {rwlock_t queuelock;struct list_head submitqueues;int queueid;
@@ -512,7 +512,7 @@ struct msm_gpu_submitqueue {u32 ring_nr;int faults;uint32_t last_fence; -struct msm_file_private *ctx; +struct msm_context *ctx;struct list_head node;struct idr fence_idr;struct spinlock idr_lock;
@@ -608,33 +608,32 @@ static inline void gpu_write64(struct msm_gpu *gpu, u32 reg, u64 val)int msm_gpu_pm_suspend (struct msm_gpu *gpu);int msm_gpu_pm_resume (struct msm_gpu *gpu); -void msm_gpu_show_fdinfo (struct msm_gpu *gpu,struct msm_file_private *ctx, +void msm_gpu_show_fdinfo (struct msm_gpu *gpu,struct msm_context *ctx,struct drm_printer *p); -int msm_submitqueue_init (struct drm_device *drm,struct msm_file_private *ctx); -struct msm_gpu_submitqueue *msm_submitqueue_get (struct msm_file_private *ctx, +int msm_submitqueue_init (struct drm_device *drm,struct msm_context *ctx); +struct msm_gpu_submitqueue *msm_submitqueue_get (struct msm_context *ctx,u32 id);int msm_submitqueue_create (struct drm_device *drm, -struct msm_file_private *ctx, +struct msm_context *ctx,u32 prio,u32 flags,u32 *id); -int msm_submitqueue_query (struct drm_device *drm,struct msm_file_private *ctx, +int msm_submitqueue_query (struct drm_device *drm,struct msm_context *ctx,struct drm_msm_submitqueue_query *args); -int msm_submitqueue_remove (struct msm_file_private *ctx,u32 id); -void msm_submitqueue_close (struct msm_file_private *ctx); +int msm_submitqueue_remove (struct msm_context *ctx,u32 id); +void msm_submitqueue_close (struct msm_context *ctx);void msm_submitqueue_destroy (struct kref *kref); -int msm_file_private_set_sysprof (struct msm_file_private *ctx, -struct msm_gpu *gpu,int sysprof); -void __msm_file_private_destroy (struct kref *kref); +int msm_context_set_sysprof (struct msm_context *ctx,struct msm_gpu *gpu,int sysprof); +void __msm_context_destroy (struct kref *kref); -static inline void msm_file_private_put (struct msm_file_private *ctx) +static inline void msm_context_put (struct msm_context *ctx) { -kref_put (&ctx->ref, __msm_file_private_destroy); +kref_put (&ctx->ref, __msm_context_destroy); } -static inline struct msm_file_private *msm_file_private_get ( -struct msm_file_private *ctx) +static inline struct msm_context *msm_context_get ( +struct msm_context *ctx) {kref_get (&ctx->ref);return ctx;
drivers/gpu/drm/msm/msm_submitqueue.c +13 -14
@@ -7,8 +7,7 @@#include "msm_gpu.h" -int msm_file_private_set_sysprof (struct msm_file_private *ctx, -struct msm_gpu *gpu,int sysprof) +int msm_context_set_sysprof (struct msm_context *ctx,struct msm_gpu *gpu,int sysprof) {/* * Since pm_runtime and sysprof_active are both refcounts, we
@@ -46,10 +45,10 @@ int msm_file_private_set_sysprof(struct msm_file_private *ctx,return 0 ; } -void __msm_file_private_destroy (struct kref *kref) +void __msm_context_destroy (struct kref *kref) { -struct msm_file_private *ctx =container_of (kref, - struct msm_file_private, ref); +struct msm_context *ctx =container_of (kref, + struct msm_context, ref);int i;for (i =0 ; i <ARRAY_SIZE (ctx->entities); i++) {
@@ -73,12 +72,12 @@ void msm_submitqueue_destroy(struct kref *kref)idr_destroy (&queue->fence_idr); -msm_file_private_put (queue->ctx); +msm_context_put (queue->ctx);kfree (queue); } -struct msm_gpu_submitqueue *msm_submitqueue_get (struct msm_file_private *ctx, +struct msm_gpu_submitqueue *msm_submitqueue_get (struct msm_context *ctx,u32 id) {struct msm_gpu_submitqueue *entry;
@@ -101,7 +100,7 @@ struct msm_gpu_submitqueue *msm_submitqueue_get(struct msm_file_private *ctx,return NULL ; } -void msm_submitqueue_close (struct msm_file_private *ctx) +void msm_submitqueue_close (struct msm_context *ctx) {struct msm_gpu_submitqueue *entry, *tmp;
@@ -119,7 +118,7 @@ void msm_submitqueue_close(struct msm_file_private *ctx) }static struct drm_sched_entity * -get_sched_entity (struct msm_file_private *ctx,struct msm_ringbuffer *ring, +get_sched_entity (struct msm_context *ctx,struct msm_ringbuffer *ring,unsigned ring_nr,enum drm_sched_priority sched_prio) {static DEFINE_MUTEX (entity_lock);
@@ -155,7 +154,7 @@ get_sched_entity(struct msm_file_private *ctx, struct msm_ringbuffer *ring,return ctx->entities[idx]; } -int msm_submitqueue_create (struct drm_device *drm,struct msm_file_private *ctx, +int msm_submitqueue_create (struct drm_device *drm,struct msm_context *ctx,u32 prio,u32 flags,u32 *id) {struct msm_drm_private *priv = drm->dev_private;
@@ -200,7 +199,7 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx,write_lock (&ctx->queuelock); - queue->ctx =msm_file_private_get (ctx); + queue->ctx =msm_context_get (ctx); queue->id = ctx->queueid++;if (id)
@@ -221,7 +220,7 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx,* Create the default submit-queue (id==0), used for backwards compatibility * for userspace that pre-dates the introduction of submitqueues. */ -int msm_submitqueue_init (struct drm_device *drm,struct msm_file_private *ctx) +int msm_submitqueue_init (struct drm_device *drm,struct msm_context *ctx) {struct msm_drm_private *priv = drm->dev_private;int default_prio, max_priority;
@@ -261,7 +260,7 @@ static int msm_submitqueue_query_faults(struct msm_gpu_submitqueue *queue,return ret ? -EFAULT :0 ; } -int msm_submitqueue_query (struct drm_device *drm,struct msm_file_private *ctx, +int msm_submitqueue_query (struct drm_device *drm,struct msm_context *ctx,struct drm_msm_submitqueue_query *args) {struct msm_gpu_submitqueue *queue;
@@ -282,7 +281,7 @@ int msm_submitqueue_query(struct drm_device *drm, struct msm_file_private *ctx,return ret; } -int msm_submitqueue_remove (struct msm_file_private *ctx,u32 id) +int msm_submitqueue_remove (struct msm_context *ctx,u32 id) {struct msm_gpu_submitqueue *entry;
Signature · 1 line
2.48.1
Re: [PATCH v2 05/34] drm/msm: Rename msm_file_private -> msm_context
Quoted 5 lines · Rob Clark
From: Rob Clark <[email protected]>
This is a more descriptive name.
Signed-off-by Rob Clark <[email protected]>
Quoted 12 lines
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 6 ++-- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 4 +-- drivers/gpu/drm/msm/msm_drv.c | 14 ++++----- drivers/gpu/drm/msm/msm_gem.c | 2 +- drivers/gpu/drm/msm/msm_gem_submit.c | 2 +- drivers/gpu/drm/msm/msm_gpu.c | 4 +-- drivers/gpu/drm/msm/msm_gpu.h | 39 ++++++++++++------------- drivers/gpu/drm/msm/msm_submitqueue.c | 27 +++++++++-------- 9 files changed, 49 insertions(+), 51 deletions(-)
Signature · 2 lines
With best wishes Dmitry
PATCH v2 6/34 drm/msm: Improve msm_context comments
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Just some tidying up.
--- drivers/gpu/drm/msm/msm_gpu.h | 44 +++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 15 deletions(-)
drivers/gpu/drm/msm/msm_gpu.h +29 -15
@@ -348,25 +348,39 @@ struct msm_gpu_perfcntr {/** * struct msm_context - per-drm_file context -* -* @queuelock: synchronizes access to submitqueues list -* @submitqueues: list of &msm_gpu_submitqueue created by userspace -* @queueid: counter incremented each time a submitqueue is created, -* used to assign &msm_gpu_submitqueue.id -* @aspace: the per-process GPU address-space -* @ref: reference count -* @seqno: unique per process seqno */ struct msm_context { +/** @queuelock: synchronizes access to submitqueues list */ rwlock_t queuelock; + +/** @submitqueues: list of &msm_gpu_submitqueue created by userspace */ struct list_head submitqueues; + +/** +* @queueid: +* +* Counter incremented each time a submitqueue is created, used to +* assign &msm_gpu_submitqueue.id +*/ int queueid; + +/** @aspace: the per-process GPU address-space */ struct msm_gem_address_space *aspace; + +/** @kref: the reference count */ struct kref ref; + +/** +* @seqno: +* +* A unique per-process sequence number. Used to detect context +* switches, without relying on keeping a, potentially dangling, +* pointer to the previous context. +*/ int seqno;/** -* sysprof: +* @sysprof: * * The value of MSM_PARAM_SYSPROF set by userspace. This is * intended to be used by system profiling tools like Mesa's
@@ -384,21 +398,21 @@ struct msm_context {int sysprof;/** -* comm: Overridden task comm, see MSM_PARAM_COMM +* @comm: Overridden task comm, see MSM_PARAM_COMM * * Accessed under msm_gpu::lock */ char *comm;/** -* cmdline: Overridden task cmdline, see MSM_PARAM_CMDLINE +* @cmdline: Overridden task cmdline, see MSM_PARAM_CMDLINE * * Accessed under msm_gpu::lock */ char *cmdline;/** -* elapsed: +* @elapsed: * * The total (cumulative) elapsed time GPU was busy with rendering * from this context in ns.
@@ -406,7 +420,7 @@ struct msm_context {uint64_t elapsed_ns;/** -* cycles: +* @cycles: * * The total (cumulative) GPU cycles elapsed attributed to this * context.
@@ -414,7 +428,7 @@ struct msm_context {uint64_t cycles;/** -* entities: +* @entities: * * Table of per-priority-level sched entities used by submitqueues * associated with this &drm_file. Because some userspace apps
@@ -427,7 +441,7 @@ struct msm_context {struct drm_sched_entity *entities[NR_SCHED_PRIORITIES *MSM_GPU_MAX_RINGS ];/** -* ctx_mem: +* @ctx_mem: * * Total amount of memory of GEM buffers with handles attached for * this context.
Signature · 1 line
2.48.1
Re: [PATCH v2 06/34] drm/msm: Improve msm_context comments
Quoted 3 lines · Rob Clark
From: Rob Clark <[email protected]>
Just some tidying up.
Probably there is nothing more to say.
Quoted 6 lines
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_gpu.h | 44 +++++++++++++++++++++++------------ 1 file changed, 29 insertions(+), 15 deletions(-)
Signature · 2 lines
With best wishes Dmitry
PATCH v2 7/34 drm/msm: Rename msm_gem_address_space -> msm_gem_vm
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Re-aligning naming to better match drm_gpuvm terminology will make things less confusing at the end of the drm_gpuvm conversion.
This is just rename churn, no functional change.
--- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 18 ++-- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 4 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 4 +- drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 22 ++--- drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +- drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 10 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 26 +++--- drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 2 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 45 +++++---- drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 6 +- drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 10 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 47 +++++----- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 18 ++-- .../drm/msm/disp/dpu1/dpu_encoder_phys_wb.c | 14 +-- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c | 18 ++-- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.h | 2 +- drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c | 18 ++-- drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c | 14 +-- drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h | 4 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_crtc.c | 6 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c | 24 ++--- drivers/gpu/drm/msm/disp/mdp4/mdp4_plane.c | 12 +-- drivers/gpu/drm/msm/disp/mdp5/mdp5_crtc.c | 4 +- drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c | 18 ++-- drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c | 12 +-- drivers/gpu/drm/msm/dsi/dsi_host.c | 14 +-- drivers/gpu/drm/msm/msm_drv.c | 8 +- drivers/gpu/drm/msm/msm_drv.h | 10 +- drivers/gpu/drm/msm/msm_fb.c | 10 +- drivers/gpu/drm/msm/msm_fbdev.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 74 +++++++-------- drivers/gpu/drm/msm/msm_gem.h | 34 +++---- drivers/gpu/drm/msm/msm_gem_submit.c | 6 +- drivers/gpu/drm/msm/msm_gem_vma.c | 93 +++++++++---------- drivers/gpu/drm/msm/msm_gpu.c | 48 +++++----- drivers/gpu/drm/msm/msm_gpu.h | 16 ++-- drivers/gpu/drm/msm/msm_kms.c | 16 ++-- drivers/gpu/drm/msm/msm_kms.h | 2 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 4 +- drivers/gpu/drm/msm/msm_submitqueue.c | 2 +- 41 files changed, 349 insertions(+), 354 deletions(-)
drivers/gpu/drm/msm/adreno/a2xx_gpu.c +9 -9
@@ -113,7 +113,7 @@ static int a2xx_hw_init(struct msm_gpu *gpu)uint32_t *ptr, len;int i, ret; -a2xx_gpummu_params (gpu->aspace->mmu, &pt_base, &tran_error); +a2xx_gpummu_params (gpu->vm->mmu, &pt_base, &tran_error);DBG ("%s" , gpu->name);
@@ -466,19 +466,19 @@ static struct msm_gpu_state *a2xx_gpu_state_get(struct msm_gpu *gpu)return state; } -static struct msm_gem_address_space * -a2xx_create_address_space (struct msm_gpu *gpu,struct platform_device *pdev) +static struct msm_gem_vm * +a2xx_create_vm (struct msm_gpu *gpu,struct platform_device *pdev) {struct msm_mmu *mmu =a2xx_gpummu_new (&pdev->dev, gpu); -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm; - aspace =msm_gem_address_space_create (mmu,"gpu" ,SZ_16M , + vm =msm_gem_vm_create (mmu,"gpu" ,SZ_16M ,0xfff *SZ_64K ); -if (IS_ERR (aspace) && !IS_ERR (mmu)) +if (IS_ERR (vm) && !IS_ERR (mmu)) mmu->funcs->destroy (mmu); -return aspace; +return vm; }static u32 a2xx_get_rptr (struct msm_gpu *gpu,struct msm_ringbuffer *ring)
@@ -504,7 +504,7 @@ static const struct adreno_gpu_funcs funcs = {#endif .gpu_state_get = a2xx_gpu_state_get, .gpu_state_put = adreno_gpu_state_put, - .create_address_space = a2xx_create_address_space, + .create_vm = a2xx_create_vm, .get_rptr = a2xx_get_rptr, }, };
@@ -551,7 +551,7 @@ struct msm_gpu *a2xx_gpu_init(struct drm_device *dev)else adreno_gpu->registers = a220_registers; -if (!gpu->aspace) { +if (!gpu->vm) {dev_err (dev->dev,"No memory protection without MMU\n" );if (!allow_vram_carveout) { ret = -ENXIO ;
drivers/gpu/drm/msm/adreno/a3xx_gpu.c +2 -2
@@ -526,7 +526,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_busy = a3xx_gpu_busy, .gpu_state_get = a3xx_gpu_state_get, .gpu_state_put = adreno_gpu_state_put, - .create_address_space = adreno_create_address_space, + .create_vm = adreno_create_vm, .get_rptr = a3xx_get_rptr, }, };
@@ -581,7 +581,7 @@ struct msm_gpu *a3xx_gpu_init(struct drm_device *dev) goto fail; } -if (!gpu->aspace) { +if (!gpu->vm) {/* TODO we think it is possible to configure the GPU to * restrict access to VRAM carveout. But the required * registers are unknown. For now just bail out and
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +2 -2
@@ -645,7 +645,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_busy = a4xx_gpu_busy, .gpu_state_get = a4xx_gpu_state_get, .gpu_state_put = adreno_gpu_state_put, - .create_address_space = adreno_create_address_space, + .create_vm = adreno_create_vm, .get_rptr = a4xx_get_rptr, }, .get_timestamp = a4xx_get_timestamp,
@@ -695,7 +695,7 @@ struct msm_gpu *a4xx_gpu_init(struct drm_device *dev) adreno_gpu->uche_trap_base =0xffff0000ffff0000ull ; -if (!gpu->aspace) { +if (!gpu->vm) {/* TODO we think it is possible to configure the GPU to * restrict access to VRAM carveout. But the required * registers are unknown. For now just bail out and
drivers/gpu/drm/msm/adreno/a5xx_debugfs.c +2 -2
@@ -116,13 +116,13 @@ reset_set(void *data, u64 val) adreno_gpu->fw[ADRENO_FW_PFP ] =NULL ;if (a5xx_gpu->pm4_bo) { -msm_gem_unpin_iova (a5xx_gpu->pm4_bo, gpu->aspace); +msm_gem_unpin_iova (a5xx_gpu->pm4_bo, gpu->vm);drm_gem_object_put (a5xx_gpu->pm4_bo); a5xx_gpu->pm4_bo =NULL ; }if (a5xx_gpu->pfp_bo) { -msm_gem_unpin_iova (a5xx_gpu->pfp_bo, gpu->aspace); +msm_gem_unpin_iova (a5xx_gpu->pfp_bo, gpu->vm);drm_gem_object_put (a5xx_gpu->pfp_bo); a5xx_gpu->pfp_bo =NULL ; }
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +11 -11
@@ -622,7 +622,7 @@ static int a5xx_ucode_load(struct msm_gpu *gpu) a5xx_gpu->shadow =msm_gem_kernel_new (gpu->dev,sizeof (u32) * gpu->nr_rings,MSM_BO_WC |MSM_BO_MAP_PRIV , - gpu->aspace, &a5xx_gpu->shadow_bo, + gpu->vm, &a5xx_gpu->shadow_bo, &a5xx_gpu->shadow_iova);if (IS_ERR (a5xx_gpu->shadow))
@@ -1042,22 +1042,22 @@ static void a5xx_destroy(struct msm_gpu *gpu)a5xx_preempt_fini (gpu);if (a5xx_gpu->pm4_bo) { -msm_gem_unpin_iova (a5xx_gpu->pm4_bo, gpu->aspace); +msm_gem_unpin_iova (a5xx_gpu->pm4_bo, gpu->vm);drm_gem_object_put (a5xx_gpu->pm4_bo); }if (a5xx_gpu->pfp_bo) { -msm_gem_unpin_iova (a5xx_gpu->pfp_bo, gpu->aspace); +msm_gem_unpin_iova (a5xx_gpu->pfp_bo, gpu->vm);drm_gem_object_put (a5xx_gpu->pfp_bo); }if (a5xx_gpu->gpmu_bo) { -msm_gem_unpin_iova (a5xx_gpu->gpmu_bo, gpu->aspace); +msm_gem_unpin_iova (a5xx_gpu->gpmu_bo, gpu->vm);drm_gem_object_put (a5xx_gpu->gpmu_bo); }if (a5xx_gpu->shadow_bo) { -msm_gem_unpin_iova (a5xx_gpu->shadow_bo, gpu->aspace); +msm_gem_unpin_iova (a5xx_gpu->shadow_bo, gpu->vm);drm_gem_object_put (a5xx_gpu->shadow_bo); }
@@ -1457,7 +1457,7 @@ static int a5xx_crashdumper_init(struct msm_gpu *gpu, struct a5xx_crashdumper *dumper) { dumper->ptr =msm_gem_kernel_new (gpu->dev, -SZ_1M ,MSM_BO_WC , gpu->aspace, +SZ_1M ,MSM_BO_WC , gpu->vm, &dumper->bo, &dumper->iova);if (!IS_ERR (dumper->ptr))
@@ -1557,7 +1557,7 @@ static void a5xx_gpu_state_get_hlsq_regs(struct msm_gpu *gpu,if (a5xx_crashdumper_run (gpu, &dumper)) {kfree (a5xx_state->hlsqregs); -msm_gem_kernel_put (dumper.bo, gpu->aspace); +msm_gem_kernel_put (dumper.bo, gpu->vm);return ; }
@@ -1565,7 +1565,7 @@ static void a5xx_gpu_state_get_hlsq_regs(struct msm_gpu *gpu,memcpy (a5xx_state->hlsqregs, dumper.ptr + (256 *SZ_1K ), count *sizeof (u32)); -msm_gem_kernel_put (dumper.bo, gpu->aspace); +msm_gem_kernel_put (dumper.bo, gpu->vm); }static struct msm_gpu_state *a5xx_gpu_state_get (struct msm_gpu *gpu)
@@ -1713,7 +1713,7 @@ static const struct adreno_gpu_funcs funcs = { .gpu_busy = a5xx_gpu_busy, .gpu_state_get = a5xx_gpu_state_get, .gpu_state_put = a5xx_gpu_state_put, - .create_address_space = adreno_create_address_space, + .create_vm = adreno_create_vm, .get_rptr = a5xx_get_rptr, }, .get_timestamp = a5xx_get_timestamp,
@@ -1786,8 +1786,8 @@ struct msm_gpu *a5xx_gpu_init(struct drm_device *dev)return ERR_PTR (ret); } -if (gpu->aspace) -msm_mmu_set_fault_handler (gpu->aspace->mmu, gpu, a5xx_fault_handler); +if (gpu->vm) +msm_mmu_set_fault_handler (gpu->vm->mmu, gpu, a5xx_fault_handler);/* Set up the preemption specific bits and pieces for each ringbuffer */ a5xx_preempt_init (gpu);
drivers/gpu/drm/msm/adreno/a5xx_power.c +1 -1
@@ -363,7 +363,7 @@ void a5xx_gpmu_ucode_init(struct msm_gpu *gpu) bosize = (cmds_size + (cmds_size /TYPE4_MAX_PAYLOAD ) +1 ) <<2 ; ptr =msm_gem_kernel_new (drm, bosize, -MSM_BO_WC |MSM_BO_GPU_READONLY , gpu->aspace, +MSM_BO_WC |MSM_BO_GPU_READONLY , gpu->vm, &a5xx_gpu->gpmu_bo, &a5xx_gpu->gpmu_iova);if (IS_ERR (ptr))return ;
drivers/gpu/drm/msm/adreno/a5xx_preempt.c +5 -5
@@ -254,7 +254,7 @@ static int preempt_init_ring(struct a5xx_gpu *a5xx_gpu, ptr =msm_gem_kernel_new (gpu->dev,A5XX_PREEMPT_RECORD_SIZE +A5XX_PREEMPT_COUNTER_SIZE , -MSM_BO_WC |MSM_BO_MAP_PRIV , gpu->aspace, &bo, &iova); +MSM_BO_WC |MSM_BO_MAP_PRIV , gpu->vm, &bo, &iova);if (IS_ERR (ptr))return PTR_ERR (ptr);
@@ -262,9 +262,9 @@ static int preempt_init_ring(struct a5xx_gpu *a5xx_gpu,/* The buffer to store counters needs to be unprivileged */ counters =msm_gem_kernel_new (gpu->dev,A5XX_PREEMPT_COUNTER_SIZE , -MSM_BO_WC , gpu->aspace, &counters_bo, &counters_iova); +MSM_BO_WC , gpu->vm, &counters_bo, &counters_iova);if (IS_ERR (counters)) { -msm_gem_kernel_put (bo, gpu->aspace); +msm_gem_kernel_put (bo, gpu->vm);return PTR_ERR (counters); }
@@ -295,8 +295,8 @@ void a5xx_preempt_fini(struct msm_gpu *gpu)int i;for (i =0 ; i < gpu->nr_rings; i++) { -msm_gem_kernel_put (a5xx_gpu->preempt_bo[i], gpu->aspace); -msm_gem_kernel_put (a5xx_gpu->preempt_counters_bo[i], gpu->aspace); +msm_gem_kernel_put (a5xx_gpu->preempt_bo[i], gpu->vm); +msm_gem_kernel_put (a5xx_gpu->preempt_counters_bo[i], gpu->vm); } }
drivers/gpu/drm/msm/adreno/a6xx_gmu.c +13 -13
@@ -1259,15 +1259,15 @@ int a6xx_gmu_stop(struct a6xx_gpu *a6xx_gpu)static void a6xx_gmu_memory_free (struct a6xx_gmu *gmu) { -msm_gem_kernel_put (gmu->hfi.obj, gmu->aspace); -msm_gem_kernel_put (gmu->debug.obj, gmu->aspace); -msm_gem_kernel_put (gmu->icache.obj, gmu->aspace); -msm_gem_kernel_put (gmu->dcache.obj, gmu->aspace); -msm_gem_kernel_put (gmu->dummy.obj, gmu->aspace); -msm_gem_kernel_put (gmu->log.obj, gmu->aspace); - - gmu->aspace->mmu->funcs->detach (gmu->aspace->mmu); -msm_gem_address_space_put (gmu->aspace); +msm_gem_kernel_put (gmu->hfi.obj, gmu->vm); +msm_gem_kernel_put (gmu->debug.obj, gmu->vm); +msm_gem_kernel_put (gmu->icache.obj, gmu->vm); +msm_gem_kernel_put (gmu->dcache.obj, gmu->vm); +msm_gem_kernel_put (gmu->dummy.obj, gmu->vm); +msm_gem_kernel_put (gmu->log.obj, gmu->vm); + + gmu->vm->mmu->funcs->detach (gmu->vm->mmu); +msm_gem_vm_put (gmu->vm); } static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,
@@ -1296,7 +1296,7 @@ static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,if (IS_ERR (bo->obj))return PTR_ERR (bo->obj); - ret =msm_gem_get_and_pin_iova_range (bo->obj, gmu->aspace, &bo->iova, + ret =msm_gem_get_and_pin_iova_range (bo->obj, gmu->vm, &bo->iova, range_start, range_end);if (ret) {drm_gem_object_put (bo->obj);
@@ -1321,9 +1321,9 @@ static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu)if (IS_ERR (mmu))return PTR_ERR (mmu); - gmu->aspace =msm_gem_address_space_create (mmu,"gmu" ,0x0 ,0x80000000 ); -if (IS_ERR (gmu->aspace)) -return PTR_ERR (gmu->aspace); + gmu->vm =msm_gem_vm_create (mmu,"gmu" ,0x0 ,0x80000000 ); +if (IS_ERR (gmu->vm)) +return PTR_ERR (gmu->vm);return 0 ; }
drivers/gpu/drm/msm/adreno/a6xx_gmu.h +1 -1
@@ -62,7 +62,7 @@ struct a6xx_gmu {/* For serializing communication with the GMU: */ struct mutex lock; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;void __iomem *mmio;void __iomem *rscc;
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +22 -23
125 lines not on this page · raw message
drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c +3 -3
@@ -132,7 +132,7 @@ static int a6xx_crashdumper_init(struct msm_gpu *gpu, struct a6xx_crashdumper *dumper) { dumper->ptr =msm_gem_kernel_new (gpu->dev, -SZ_1M ,MSM_BO_WC , gpu->aspace, +SZ_1M ,MSM_BO_WC , gpu->vm, &dumper->bo, &dumper->iova);if (!IS_ERR (dumper->ptr))
@@ -1619,7 +1619,7 @@ struct msm_gpu_state *a6xx_gpu_state_get(struct msm_gpu *gpu)a7xx_get_clusters (gpu, a6xx_state, dumper);a7xx_get_dbgahb_clusters (gpu, a6xx_state, dumper); -msm_gem_kernel_put (dumper->bo, gpu->aspace); +msm_gem_kernel_put (dumper->bo, gpu->vm); }a7xx_get_post_crashdumper_registers (gpu, a6xx_state);
@@ -1631,7 +1631,7 @@ struct msm_gpu_state *a6xx_gpu_state_get(struct msm_gpu *gpu)a6xx_get_clusters (gpu, a6xx_state, dumper);a6xx_get_dbgahb_clusters (gpu, a6xx_state, dumper); -msm_gem_kernel_put (dumper->bo, gpu->aspace); +msm_gem_kernel_put (dumper->bo, gpu->vm); } }
drivers/gpu/drm/msm/adreno/a6xx_preempt.c +5 -5
@@ -343,7 +343,7 @@ static int preempt_init_ring(struct a6xx_gpu *a6xx_gpu, ptr =msm_gem_kernel_new (gpu->dev,PREEMPT_RECORD_SIZE (adreno_gpu), -MSM_BO_WC |MSM_BO_MAP_PRIV , gpu->aspace, &bo, &iova); +MSM_BO_WC |MSM_BO_MAP_PRIV , gpu->vm, &bo, &iova);if (IS_ERR (ptr))return PTR_ERR (ptr);
@@ -361,7 +361,7 @@ static int preempt_init_ring(struct a6xx_gpu *a6xx_gpu, ptr =msm_gem_kernel_new (gpu->dev,PREEMPT_SMMU_INFO_SIZE ,MSM_BO_WC |MSM_BO_MAP_PRIV |MSM_BO_GPU_READONLY , - gpu->aspace, &bo, &iova); + gpu->vm, &bo, &iova);if (IS_ERR (ptr))return PTR_ERR (ptr);
@@ -376,7 +376,7 @@ static int preempt_init_ring(struct a6xx_gpu *a6xx_gpu,struct a7xx_cp_smmu_info *smmu_info_ptr = ptr; -msm_iommu_pagetable_params (gpu->aspace->mmu, &ttbr, &asid); +msm_iommu_pagetable_params (gpu->vm->mmu, &ttbr, &asid); smmu_info_ptr->magic =GEN7_CP_SMMU_INFO_MAGIC ; smmu_info_ptr->ttbr0 = ttbr;
@@ -404,7 +404,7 @@ void a6xx_preempt_fini(struct msm_gpu *gpu)int i;for (i =0 ; i < gpu->nr_rings; i++) -msm_gem_kernel_put (a6xx_gpu->preempt_bo[i], gpu->aspace); +msm_gem_kernel_put (a6xx_gpu->preempt_bo[i], gpu->vm); }void a6xx_preempt_init (struct msm_gpu *gpu)
@@ -430,7 +430,7 @@ void a6xx_preempt_init(struct msm_gpu *gpu) a6xx_gpu->preempt_postamble_ptr =msm_gem_kernel_new (gpu->dev,PAGE_SIZE ,MSM_BO_WC |MSM_BO_MAP_PRIV |MSM_BO_GPU_READONLY , - gpu->aspace, &a6xx_gpu->preempt_postamble_bo, + gpu->vm, &a6xx_gpu->preempt_postamble_bo, &a6xx_gpu->preempt_postamble_iova);preempt_prepare_postamble (a6xx_gpu);
drivers/gpu/drm/msm/adreno/adreno_gpu.c +23 -24
110 lines not on this page · raw message
drivers/gpu/drm/msm/adreno/adreno_gpu.h +9 -9
@@ -600,7 +600,7 @@ static inline int adreno_is_a7xx(struct adreno_gpu *gpu)adreno_is_a740_family (gpu); } -u64 adreno_private_address_space_size (struct msm_gpu *gpu); +u64 adreno_private_vm_size (struct msm_gpu *gpu);int adreno_get_param (struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t *value,uint32_t *len); int adreno_set_param(struct msm_gpu *gpu, struct msm_context *ctx,
@@ -643,14 +643,14 @@ void adreno_show_object(struct drm_printer *p, void **ptr, int len,* Common helper function to initialize the default address space for arm-smmu * attached targets */ -struct msm_gem_address_space * -adreno_create_address_space (struct msm_gpu *gpu, -struct platform_device *pdev); - -struct msm_gem_address_space * -adreno_iommu_create_address_space (struct msm_gpu *gpu, -struct platform_device *pdev, -unsigned long quirks); +struct msm_gem_vm * +adreno_create_vm (struct msm_gpu *gpu, +struct platform_device *pdev); + +struct msm_gem_vm * +adreno_iommu_create_vm (struct msm_gpu *gpu, +struct platform_device *pdev, +unsigned long quirks); int adreno_fault_handler(struct msm_gpu *gpu, unsigned long iova, int flags, struct adreno_smmu_fault_info *info, const char *block,
drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_wb.c +7 -7
@@ -566,7 +566,7 @@ static void dpu_encoder_phys_wb_prepare_wb_job(struct dpu_encoder_phys *phys_enc struct drm_writeback_job *job) {const struct msm_format *format; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;struct dpu_hw_wb_cfg *wb_cfg;int ret;struct dpu_encoder_phys_wb *wb_enc =to_dpu_encoder_phys_wb (phys_enc);
@@ -576,13 +576,13 @@ static void dpu_encoder_phys_wb_prepare_wb_job(struct dpu_encoder_phys *phys_enc wb_enc->wb_job = job; wb_enc->wb_conn = job->connector; - aspace = phys_enc->dpu_kms->base.aspace; + vm = phys_enc->dpu_kms->base.vm; wb_cfg = &wb_enc->wb_cfg;memset (wb_cfg,0 ,sizeof (struct dpu_hw_wb_cfg )); - ret =msm_framebuffer_prepare (job->fb, aspace, false); + ret =msm_framebuffer_prepare (job->fb, vm, false);if (ret) {DPU_ERROR ("prep fb failed, %d\n" , ret);return ;
@@ -596,7 +596,7 @@ static void dpu_encoder_phys_wb_prepare_wb_job(struct dpu_encoder_phys *phys_encreturn ; } -dpu_format_populate_addrs (aspace, job->fb, &wb_cfg->dest); +dpu_format_populate_addrs (vm, job->fb, &wb_cfg->dest); wb_cfg->dest.width = job->fb->width; wb_cfg->dest.height = job->fb->height;
@@ -619,14 +619,14 @@ static void dpu_encoder_phys_wb_cleanup_wb_job(struct dpu_encoder_phys *phys_enc struct drm_writeback_job *job) {struct dpu_encoder_phys_wb *wb_enc =to_dpu_encoder_phys_wb (phys_enc); -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;if (!job->fb)return ; - aspace = phys_enc->dpu_kms->base.aspace; + vm = phys_enc->dpu_kms->base.vm; -msm_framebuffer_cleanup (job->fb, aspace, false); +msm_framebuffer_cleanup (job->fb, vm, false); wb_enc->wb_job =NULL ; wb_enc->wb_conn =NULL ; }
drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c +9 -9
@@ -274,7 +274,7 @@ int dpu_format_populate_plane_sizes(return _dpu_format_populate_plane_sizes_linear (fmt, fb, layout); } -static void _dpu_format_populate_addrs_ubwc (struct msm_gem_address_space *aspace, +static void _dpu_format_populate_addrs_ubwc (struct msm_gem_vm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout) {
@@ -282,7 +282,7 @@ static void _dpu_format_populate_addrs_ubwc(struct msm_gem_address_space *aspaceuint32_t base_addr =0 ;bool meta; - base_addr =msm_framebuffer_iova (fb, aspace,0 ); + base_addr =msm_framebuffer_iova (fb, vm,0 ); fmt =msm_framebuffer_format (fb); meta =MSM_FORMAT_IS_UBWC (fmt);
@@ -355,7 +355,7 @@ static void _dpu_format_populate_addrs_ubwc(struct msm_gem_address_space *aspace } } -static void _dpu_format_populate_addrs_linear (struct msm_gem_address_space *aspace, +static void _dpu_format_populate_addrs_linear (struct msm_gem_vm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout) {
@@ -363,17 +363,17 @@ static void _dpu_format_populate_addrs_linear(struct msm_gem_address_space *aspa/* Populate addresses for simple formats here */ for (i =0 ; i < layout->num_planes; ++i) - layout->plane_addr[i] =msm_framebuffer_iova (fb, aspace, i); -} + layout->plane_addr[i] =msm_framebuffer_iova (fb, vm, i); + }/** * dpu_format_populate_addrs - populate buffer addresses based on * mmu, fb, and format found in the fb -* @aspace: address space pointer +* @vm: address space pointer * @fb: framebuffer pointer * @layout: format layout structure to populate */ -void dpu_format_populate_addrs (struct msm_gem_address_space *aspace, +void dpu_format_populate_addrs (struct msm_gem_vm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout) {
@@ -384,7 +384,7 @@ void dpu_format_populate_addrs(struct msm_gem_address_space *aspace,/* Populate the addresses given the fb */ if (MSM_FORMAT_IS_UBWC (fmt) ||MSM_FORMAT_IS_TILE (fmt)) -_dpu_format_populate_addrs_ubwc (aspace, fb, layout); +_dpu_format_populate_addrs_ubwc (vm, fb, layout);else -_dpu_format_populate_addrs_linear (aspace, fb, layout); +_dpu_format_populate_addrs_linear (vm, fb, layout); }
drivers/gpu/drm/msm/disp/dpu1/dpu_formats.h +1 -1
@@ -31,7 +31,7 @@ static inline bool dpu_find_format(u32 format, const u32 *supported_formats,return false; } -void dpu_format_populate_addrs (struct msm_gem_address_space *aspace, +void dpu_format_populate_addrs (struct msm_gem_vm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout);
drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c +9 -9
@@ -1095,26 +1095,26 @@ static void _dpu_kms_mmu_destroy(struct dpu_kms *dpu_kms) {struct msm_mmu *mmu; -if (!dpu_kms->base.aspace) +if (!dpu_kms->base.vm)return ; - mmu = dpu_kms->base.aspace->mmu; + mmu = dpu_kms->base.vm->mmu; mmu->funcs->detach (mmu); -msm_gem_address_space_put (dpu_kms->base.aspace); +msm_gem_vm_put (dpu_kms->base.vm); - dpu_kms->base.aspace =NULL ; + dpu_kms->base.vm =NULL ; }static int _dpu_kms_mmu_init (struct dpu_kms *dpu_kms) { -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm; - aspace =msm_kms_init_aspace (dpu_kms->dev); -if (IS_ERR (aspace)) -return PTR_ERR (aspace); + vm =msm_kms_init_vm (dpu_kms->dev); +if (IS_ERR (vm)) +return PTR_ERR (vm); - dpu_kms->base.aspace = aspace; + dpu_kms->base.vm = vm;return 0 ; }
drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c +7 -7
@@ -71,7 +71,7 @@ static const uint32_t qcom_compressed_supported_formats[] = {/* * struct dpu_plane - local dpu plane structure -* @aspace: address space pointer +* @vm: address space pointer * @csc_ptr: Points to dpu_csc_cfg structure to use for current * @catalog: Points to dpu catalog structure * @revalidate: force revalidation of all the plane properties
@@ -654,8 +654,8 @@ static int dpu_plane_prepare_fb(struct drm_plane *plane,DPU_DEBUG_PLANE (pdpu,"FB[%u]\n" , fb->base.id); -/* cache aspace */ - pstate->aspace = kms->base.aspace; +/* cache vm */ + pstate->vm = kms->base.vm;/* * TODO: Need to sort out the msm_framebuffer_prepare() call below so
@@ -664,9 +664,9 @@ static int dpu_plane_prepare_fb(struct drm_plane *plane,*/ drm_gem_plane_helper_prepare_fb (plane, new_state); -if (pstate->aspace) { +if (pstate->vm) { ret =msm_framebuffer_prepare (new_state->fb, - pstate->aspace, pstate->needs_dirtyfb); + pstate->vm, pstate->needs_dirtyfb);if (ret) {DPU_ERROR ("failed to prepare framebuffer\n" );return ret;
@@ -689,7 +689,7 @@ static void dpu_plane_cleanup_fb(struct drm_plane *plane,DPU_DEBUG_PLANE (pdpu,"FB[%u]\n" , old_state->fb->base.id); -msm_framebuffer_cleanup (old_state->fb, old_pstate->aspace, +msm_framebuffer_cleanup (old_state->fb, old_pstate->vm, old_pstate->needs_dirtyfb); }
@@ -1349,7 +1349,7 @@ static void dpu_plane_sspp_atomic_update(struct drm_plane *plane, pstate->needs_qos_remap |= (is_rt_pipe != pdpu->is_rt_pipe); pdpu->is_rt_pipe = is_rt_pipe; -dpu_format_populate_addrs (pstate->aspace, new_state->fb, &pstate->layout); +dpu_format_populate_addrs (pstate->vm, new_state->fb, &pstate->layout); DPU_DEBUG_PLANE(pdpu, "FB[%u] " DRM_RECT_FP_FMT "->crtc%u " DRM_RECT_FMT ", %p4cc ubwc %d\n", fb->base.id, DRM_RECT_FP_ARG(&state->src),
drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h +2 -2
@@ -17,7 +17,7 @@/** * struct dpu_plane_state: Define dpu extension of drm plane state object * @base: base drm plane state object -* @aspace: pointer to address space for input/output buffers +* @vm: pointer to address space for input/output buffers * @pipe: software pipe description * @r_pipe: software pipe description of the second pipe * @pipe_cfg: software pipe configuration
@@ -34,7 +34,7 @@*/ struct dpu_plane_state {struct drm_plane_state base; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;struct dpu_sw_pipe pipe;struct dpu_sw_pipe r_pipe;struct dpu_sw_pipe_cfg pipe_cfg;
drivers/gpu/drm/msm/disp/mdp4/mdp4_crtc.c +3 -3
@@ -120,7 +120,7 @@ static void unref_cursor_worker(struct drm_flip_work *work, void *val)struct mdp4_kms *mdp4_kms =get_kms (&mdp4_crtc->base);struct msm_kms *kms = &mdp4_kms->base.base; -msm_gem_unpin_iova (val, kms->aspace); +msm_gem_unpin_iova (val, kms->vm);drm_gem_object_put (val); }
@@ -369,7 +369,7 @@ static void update_cursor(struct drm_crtc *crtc)if (next_bo) {/* take a obj ref + iova ref when we start scanning out: */ drm_gem_object_get (next_bo); -msm_gem_get_and_pin_iova (next_bo, kms->aspace, &iova); +msm_gem_get_and_pin_iova (next_bo, kms->vm, &iova);/* enable cursor: */ mdp4_write(mdp4_kms, REG_MDP4_DMA_CURSOR_SIZE(dma),
@@ -427,7 +427,7 @@ static int mdp4_crtc_cursor_set(struct drm_crtc *crtc, }if (cursor_bo) { - ret =msm_gem_get_and_pin_iova (cursor_bo, kms->aspace, &iova); + ret =msm_gem_get_and_pin_iova (cursor_bo, kms->vm, &iova);if (ret) goto fail; }else {
drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c +12 -12
@@ -120,15 +120,15 @@ static void mdp4_destroy(struct msm_kms *kms) {struct mdp4_kms *mdp4_kms =to_mdp4_kms (to_mdp_kms (kms));struct device *dev = mdp4_kms->dev->dev; -struct msm_gem_address_space *aspace = kms->aspace; +struct msm_gem_vm *vm = kms->vm;if (mdp4_kms->blank_cursor_iova) -msm_gem_unpin_iova (mdp4_kms->blank_cursor_bo, kms->aspace); +msm_gem_unpin_iova (mdp4_kms->blank_cursor_bo, kms->vm);drm_gem_object_put (mdp4_kms->blank_cursor_bo); -if (aspace) { - aspace->mmu->funcs->detach (aspace->mmu); -msm_gem_address_space_put (aspace); +if (vm) { + vm->mmu->funcs->detach (vm->mmu); +msm_gem_vm_put (vm); }if (mdp4_kms->rpm_enabled)
@@ -380,7 +380,7 @@ static int mdp4_kms_init(struct drm_device *dev)struct mdp4_kms *mdp4_kms =to_mdp4_kms (to_mdp_kms (priv->kms));struct msm_kms *kms =NULL ;struct msm_mmu *mmu; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;int ret;u32 major, minor;unsigned long max_clk;
@@ -449,19 +449,19 @@ static int mdp4_kms_init(struct drm_device *dev) }else if (!mmu) {DRM_DEV_INFO (dev->dev,"no iommu, fallback to phys " "contig buffers for scanout\n" ); - aspace =NULL ; + vm =NULL ; }else { - aspace =msm_gem_address_space_create (mmu, + vm =msm_gem_vm_create (mmu,"mdp4" ,0x1000 ,0x100000000 -0x1000 ); -if (IS_ERR (aspace)) { +if (IS_ERR (vm)) {if (!IS_ERR (mmu)) mmu->funcs->destroy (mmu); - ret =PTR_ERR (aspace); + ret =PTR_ERR (vm); goto fail; } - kms->aspace = aspace; + kms->vm = vm; } ret =modeset_init (mdp4_kms);
@@ -478,7 +478,7 @@ static int mdp4_kms_init(struct drm_device *dev) goto fail; } - ret =msm_gem_get_and_pin_iova (mdp4_kms->blank_cursor_bo, kms->aspace, + ret =msm_gem_get_and_pin_iova (mdp4_kms->blank_cursor_bo, kms->vm, &mdp4_kms->blank_cursor_iova);if (ret) {DRM_DEV_ERROR (dev->dev,"could not pin blank-cursor bo: %d\n" , ret);
drivers/gpu/drm/msm/disp/mdp4/mdp4_plane.c +6 -6
@@ -87,7 +87,7 @@ static int mdp4_plane_prepare_fb(struct drm_plane *plane,drm_gem_plane_helper_prepare_fb (plane, new_state); -return msm_framebuffer_prepare (new_state->fb, kms->aspace, false); +return msm_framebuffer_prepare (new_state->fb, kms->vm, false); } static void mdp4_plane_cleanup_fb(struct drm_plane *plane,
@@ -102,7 +102,7 @@ static void mdp4_plane_cleanup_fb(struct drm_plane *plane,return ;DBG ("%s: cleanup: FB[%u]" , mdp4_plane->name, fb->base.id); -msm_framebuffer_cleanup (fb, kms->aspace, false); +msm_framebuffer_cleanup (fb, kms->vm, false); }
@@ -153,13 +153,13 @@ static void mdp4_plane_set_scanout(struct drm_plane *plane, MDP4_PIPE_SRC_STRIDE_B_P3(fb->pitches[3]));mdp4_write (mdp4_kms,REG_MDP4_PIPE_SRCP0_BASE (pipe), -msm_framebuffer_iova (fb, kms->aspace,0 )); +msm_framebuffer_iova (fb, kms->vm,0 ));mdp4_write (mdp4_kms,REG_MDP4_PIPE_SRCP1_BASE (pipe), -msm_framebuffer_iova (fb, kms->aspace,1 )); +msm_framebuffer_iova (fb, kms->vm,1 ));mdp4_write (mdp4_kms,REG_MDP4_PIPE_SRCP2_BASE (pipe), -msm_framebuffer_iova (fb, kms->aspace,2 )); +msm_framebuffer_iova (fb, kms->vm,2 ));mdp4_write (mdp4_kms,REG_MDP4_PIPE_SRCP3_BASE (pipe), -msm_framebuffer_iova (fb, kms->aspace,3 )); +msm_framebuffer_iova (fb, kms->vm,3 )); } static void mdp4_write_csc_config(struct mdp4_kms *mdp4_kms,
drivers/gpu/drm/msm/disp/mdp5/mdp5_crtc.c +2 -2
@@ -169,7 +169,7 @@ static void unref_cursor_worker(struct drm_flip_work *work, void *val)struct mdp5_kms *mdp5_kms =get_kms (&mdp5_crtc->base);struct msm_kms *kms = &mdp5_kms->base.base; -msm_gem_unpin_iova (val, kms->aspace); +msm_gem_unpin_iova (val, kms->vm);drm_gem_object_put (val); }
@@ -993,7 +993,7 @@ static int mdp5_crtc_cursor_set(struct drm_crtc *crtc,if (!cursor_bo)return -ENOENT ; - ret =msm_gem_get_and_pin_iova (cursor_bo, kms->aspace, + ret =msm_gem_get_and_pin_iova (cursor_bo, kms->vm, &mdp5_crtc->cursor.iova);if (ret) {drm_gem_object_put (cursor_bo);
drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c +9 -9
@@ -198,11 +198,11 @@ static void mdp5_destroy(struct mdp5_kms *mdp5_kms);static void mdp5_kms_destroy (struct msm_kms *kms) {struct mdp5_kms *mdp5_kms =to_mdp5_kms (to_mdp_kms (kms)); -struct msm_gem_address_space *aspace = kms->aspace; +struct msm_gem_vm *vm = kms->vm; -if (aspace) { - aspace->mmu->funcs->detach (aspace->mmu); -msm_gem_address_space_put (aspace); +if (vm) { + vm->mmu->funcs->detach (vm->mmu); +msm_gem_vm_put (vm); }mdp_kms_destroy (&mdp5_kms->base);
@@ -500,7 +500,7 @@ static int mdp5_kms_init(struct drm_device *dev)struct mdp5_kms *mdp5_kms;struct mdp5_cfg *config;struct msm_kms *kms = priv->kms; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;int i, ret; ret =mdp5_init (to_platform_device (dev->dev), dev);
@@ -534,13 +534,13 @@ static int mdp5_kms_init(struct drm_device *dev) }mdelay (16 ); - aspace =msm_kms_init_aspace (mdp5_kms->dev); -if (IS_ERR (aspace)) { - ret =PTR_ERR (aspace); + vm =msm_kms_init_vm (mdp5_kms->dev); +if (IS_ERR (vm)) { + ret =PTR_ERR (vm); goto fail; } - kms->aspace = aspace; + kms->vm = vm;pm_runtime_put_sync (&pdev->dev);
drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c +6 -6
@@ -144,7 +144,7 @@ static int mdp5_plane_prepare_fb(struct drm_plane *plane,drm_gem_plane_helper_prepare_fb (plane, new_state); -return msm_framebuffer_prepare (new_state->fb, kms->aspace, needs_dirtyfb); +return msm_framebuffer_prepare (new_state->fb, kms->vm, needs_dirtyfb); } static void mdp5_plane_cleanup_fb(struct drm_plane *plane,
@@ -159,7 +159,7 @@ static void mdp5_plane_cleanup_fb(struct drm_plane *plane,return ;DBG ("%s: cleanup: FB[%u]" , plane->name, fb->base.id); -msm_framebuffer_cleanup (fb, kms->aspace, needed_dirtyfb); +msm_framebuffer_cleanup (fb, kms->vm, needed_dirtyfb); } static int mdp5_plane_atomic_check_with_state(struct drm_crtc_state *crtc_state,
@@ -478,13 +478,13 @@ static void set_scanout_locked(struct mdp5_kms *mdp5_kms, MDP5_PIPE_SRC_STRIDE_B_P3(fb->pitches[3]));mdp5_write (mdp5_kms,REG_MDP5_PIPE_SRC0_ADDR (pipe), -msm_framebuffer_iova (fb, kms->aspace,0 )); +msm_framebuffer_iova (fb, kms->vm,0 ));mdp5_write (mdp5_kms,REG_MDP5_PIPE_SRC1_ADDR (pipe), -msm_framebuffer_iova (fb, kms->aspace,1 )); +msm_framebuffer_iova (fb, kms->vm,1 ));mdp5_write (mdp5_kms,REG_MDP5_PIPE_SRC2_ADDR (pipe), -msm_framebuffer_iova (fb, kms->aspace,2 )); +msm_framebuffer_iova (fb, kms->vm,2 ));mdp5_write (mdp5_kms,REG_MDP5_PIPE_SRC3_ADDR (pipe), -msm_framebuffer_iova (fb, kms->aspace,3 )); +msm_framebuffer_iova (fb, kms->vm,3 )); }/* Note: mdp5_plane->pipe_lock must be locked */
drivers/gpu/drm/msm/dsi/dsi_host.c +7 -7
@@ -143,7 +143,7 @@ struct msm_dsi_host {/* DSI 6G TX buffer*/ struct drm_gem_object *tx_gem_obj; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;/* DSI v2 TX buffer */ void *tx_buf;
@@ -1146,10 +1146,10 @@ int dsi_tx_buf_alloc_6g(struct msm_dsi_host *msm_host, int size)uint64_t iova;u8 *data; - msm_host->aspace =msm_gem_address_space_get (priv->kms->aspace); + msm_host->vm =msm_gem_vm_get (priv->kms->vm); data =msm_gem_kernel_new (dev, size,MSM_BO_WC , - msm_host->aspace, + msm_host->vm, &msm_host->tx_gem_obj, &iova);if (IS_ERR (data)) {
@@ -1193,10 +1193,10 @@ void msm_dsi_tx_buf_free(struct mipi_dsi_host *host)return ;if (msm_host->tx_gem_obj) { -msm_gem_kernel_put (msm_host->tx_gem_obj, msm_host->aspace); -msm_gem_address_space_put (msm_host->aspace); +msm_gem_kernel_put (msm_host->tx_gem_obj, msm_host->vm); +msm_gem_vm_put (msm_host->vm); msm_host->tx_gem_obj =NULL ; - msm_host->aspace =NULL ; + msm_host->vm =NULL ; }if (msm_host->tx_buf)
@@ -1327,7 +1327,7 @@ int dsi_dma_base_get_6g(struct msm_dsi_host *msm_host, uint64_t *dma_base)return -EINVAL ;return msm_gem_get_and_pin_iova (msm_host->tx_gem_obj, - priv->kms->aspace, dma_base); + priv->kms->vm, dma_base); }int dsi_dma_base_get_v2 (struct msm_dsi_host *msm_host,uint64_t *dma_base)
drivers/gpu/drm/msm/msm_drv.c +4 -4
@@ -345,7 +345,7 @@ static int context_init(struct drm_device *dev, struct drm_file *file)kref_init (&ctx->ref);msm_submitqueue_init (dev, ctx); - ctx->aspace =msm_gpu_create_private_address_space (priv->gpu, current); + ctx->vm =msm_gpu_create_private_vm (priv->gpu, current); file->driver_priv = ctx; ctx->seqno =atomic_inc_return (&ident);
@@ -523,7 +523,7 @@ static int msm_ioctl_gem_info_iova(struct drm_device *dev,* Don't pin the memory here - just get an address so that userspace can * be productive */ -return msm_gem_get_iova (obj, ctx->aspace, iova); +return msm_gem_get_iova (obj, ctx->vm, iova); } static int msm_ioctl_gem_info_set_iova(struct drm_device *dev,
@@ -537,13 +537,13 @@ static int msm_ioctl_gem_info_set_iova(struct drm_device *dev,return -EINVAL ;/* Only supported if per-process address space is supported: */ -if (priv->gpu->aspace == ctx->aspace) +if (priv->gpu->vm == ctx->vm)return UERR (EOPNOTSUPP , dev,"requires per-process pgtables" );if (should_fail (&fail_gem_iova, obj->size))return -ENOMEM ; -return msm_gem_set_iova (obj, ctx->aspace, iova); +return msm_gem_set_iova (obj, ctx->vm, iova); } static int msm_ioctl_gem_info_set_metadata(struct drm_gem_object *obj,
drivers/gpu/drm/msm/msm_drv.h +5 -5
@@ -48,7 +48,7 @@ struct msm_rd_state;struct msm_perf_state ;struct msm_gem_submit ;struct msm_fence_context ; -struct msm_gem_address_space ; +struct msm_gem_vm ;struct msm_gem_vma ;struct msm_disp_state ;
@@ -241,7 +241,7 @@ void msm_crtc_disable_vblank(struct drm_crtc *crtc);int msm_register_mmu (struct drm_device *dev,struct msm_mmu *mmu);void msm_unregister_mmu (struct drm_device *dev,struct msm_mmu *mmu); -struct msm_gem_address_space *msm_kms_init_aspace (struct drm_device *dev); +struct msm_gem_vm *msm_kms_init_vm (struct drm_device *dev);bool msm_use_mmu (struct drm_device *dev); int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
@@ -263,11 +263,11 @@ int msm_gem_prime_pin(struct drm_gem_object *obj);void msm_gem_prime_unpin (struct drm_gem_object *obj);int msm_framebuffer_prepare (struct drm_framebuffer *fb, -struct msm_gem_address_space *aspace,bool needs_dirtyfb); +struct msm_gem_vm *vm,bool needs_dirtyfb);void msm_framebuffer_cleanup (struct drm_framebuffer *fb, -struct msm_gem_address_space *aspace,bool needed_dirtyfb); +struct msm_gem_vm *vm,bool needed_dirtyfb);uint32_t msm_framebuffer_iova (struct drm_framebuffer *fb, -struct msm_gem_address_space *aspace,int plane); +struct msm_gem_vm *vm,int plane);struct drm_gem_object *msm_framebuffer_bo (struct drm_framebuffer *fb,int plane);const struct msm_format *msm_framebuffer_format (struct drm_framebuffer *fb); struct drm_framebuffer *msm_framebuffer_create(struct drm_device *dev,
drivers/gpu/drm/msm/msm_fb.c +5 -5
@@ -76,7 +76,7 @@ void msm_framebuffer_describe(struct drm_framebuffer *fb, struct seq_file *m)/* prepare/pin all the fb's bo's for scanout. */ int msm_framebuffer_prepare (struct drm_framebuffer *fb, -struct msm_gem_address_space *aspace, +struct msm_gem_vm *vm,bool needs_dirtyfb) {struct msm_framebuffer *msm_fb =to_msm_framebuffer (fb);
@@ -88,7 +88,7 @@ int msm_framebuffer_prepare(struct drm_framebuffer *fb,atomic_inc (&msm_fb->prepare_count);for (i =0 ; i < n; i++) { - ret =msm_gem_get_and_pin_iova (fb->obj[i], aspace, &msm_fb->iova[i]); + ret =msm_gem_get_and_pin_iova (fb->obj[i], vm, &msm_fb->iova[i]);drm_dbg_state (fb->dev,"FB[%u]: iova[%d]: %08llx (%d)\n" , fb->base.id, i, msm_fb->iova[i], ret);if (ret)
@@ -99,7 +99,7 @@ int msm_framebuffer_prepare(struct drm_framebuffer *fb, }void msm_framebuffer_cleanup (struct drm_framebuffer *fb, -struct msm_gem_address_space *aspace, +struct msm_gem_vm *vm,bool needed_dirtyfb) {struct msm_framebuffer *msm_fb =to_msm_framebuffer (fb);
@@ -109,14 +109,14 @@ void msm_framebuffer_cleanup(struct drm_framebuffer *fb,refcount_dec (&msm_fb->dirtyfb);for (i =0 ; i < n; i++) -msm_gem_unpin_iova (fb->obj[i], aspace); +msm_gem_unpin_iova (fb->obj[i], vm);if (!atomic_dec_return (&msm_fb->prepare_count))memset (msm_fb->iova,0 ,sizeof (msm_fb->iova)); }uint32_t msm_framebuffer_iova (struct drm_framebuffer *fb, -struct msm_gem_address_space *aspace,int plane) +struct msm_gem_vm *vm,int plane) {struct msm_framebuffer *msm_fb =to_msm_framebuffer (fb);return msm_fb->iova[plane] + fb->offsets[plane];
drivers/gpu/drm/msm/msm_fbdev.c +1 -1
@@ -122,7 +122,7 @@ int msm_fbdev_driver_fbdev_probe(struct drm_fb_helper *helper,* in panic (ie. lock-safe, etc) we could avoid pinning the * buffer now: */ - ret =msm_gem_get_and_pin_iova (bo, priv->kms->aspace, &paddr); + ret =msm_gem_get_and_pin_iova (bo, priv->kms->vm, &paddr);if (ret) {DRM_DEV_ERROR (dev->dev,"failed to get buffer obj iova: %d\n" , ret); goto fail;
drivers/gpu/drm/msm/msm_gem.c +37 -37
244 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gem.h +17 -17
@@ -22,7 +22,7 @@#define MSM_BO_STOLEN 0x10000000/* try to use stolen/splash memory */ #define MSM_BO_MAP_PRIV 0x20000000/* use IOMMU_PRIV when mapping */ -struct msm_gem_address_space { +struct msm_gem_vm {const char *name;/* NOTE: mm managed at the page level, size is in # of pages * and position mm_node->start is in # of pages:
@@ -47,13 +47,13 @@ struct msm_gem_address_space {uint64_t va_size; }; -struct msm_gem_address_space * -msm_gem_address_space_get (struct msm_gem_address_space *aspace); +struct msm_gem_vm * +msm_gem_vm_get (struct msm_gem_vm *vm); -void msm_gem_address_space_put (struct msm_gem_address_space *aspace); +void msm_gem_vm_put (struct msm_gem_vm *vm); -struct msm_gem_address_space * -msm_gem_address_space_create (struct msm_mmu *mmu,const char *name, +struct msm_gem_vm * +msm_gem_vm_create (struct msm_mmu *mmu,const char *name,u64 va_start,u64 size);struct msm_fence_context ;
@@ -61,12 +61,12 @@ struct msm_fence_context;struct msm_gem_vma {struct drm_mm_node node;uint64_t iova; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;struct list_head list;/* node in msm_gem_object::vmas */ bool mapped; }; -struct msm_gem_vma *msm_gem_vma_new (struct msm_gem_address_space *aspace); +struct msm_gem_vma *msm_gem_vma_new (struct msm_gem_vm *vm);int msm_gem_vma_init (struct msm_gem_vma *vma,int size,u64 range_start,u64 range_end);void msm_gem_vma_purge (struct msm_gem_vma *vma);
@@ -127,18 +127,18 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma);void msm_gem_unpin_locked (struct drm_gem_object *obj);void msm_gem_unpin_active (struct drm_gem_object *obj);struct msm_gem_vma *msm_gem_get_vma_locked (struct drm_gem_object *obj, -struct msm_gem_address_space *aspace); +struct msm_gem_vm *vm);int msm_gem_get_iova (struct drm_gem_object *obj, -struct msm_gem_address_space *aspace,uint64_t *iova); +struct msm_gem_vm *vm,uint64_t *iova);int msm_gem_set_iova (struct drm_gem_object *obj, -struct msm_gem_address_space *aspace,uint64_t iova); +struct msm_gem_vm *vm,uint64_t iova);int msm_gem_get_and_pin_iova_range (struct drm_gem_object *obj, -struct msm_gem_address_space *aspace,uint64_t *iova, +struct msm_gem_vm *vm,uint64_t *iova,u64 range_start,u64 range_end);int msm_gem_get_and_pin_iova (struct drm_gem_object *obj, -struct msm_gem_address_space *aspace,uint64_t *iova); +struct msm_gem_vm *vm,uint64_t *iova);void msm_gem_unpin_iova (struct drm_gem_object *obj, -struct msm_gem_address_space *aspace); +struct msm_gem_vm *vm);void msm_gem_pin_obj_locked (struct drm_gem_object *obj);struct page **msm_gem_pin_pages_locked (struct drm_gem_object *obj);void msm_gem_unpin_pages_locked (struct drm_gem_object *obj);
@@ -160,10 +160,10 @@ int msm_gem_new_handle(struct drm_device *dev, struct drm_file *file,struct drm_gem_object *msm_gem_new (struct drm_device *dev,uint32_t size,uint32_t flags);void *msm_gem_kernel_new (struct drm_device *dev,uint32_t size, -uint32_t flags,struct msm_gem_address_space *aspace, +uint32_t flags,struct msm_gem_vm *vm,struct drm_gem_object **bo,uint64_t *iova);void msm_gem_kernel_put (struct drm_gem_object *bo, -struct msm_gem_address_space *aspace); +struct msm_gem_vm *vm);struct drm_gem_object *msm_gem_import (struct drm_device *dev,struct dma_buf *dmabuf,struct sg_table *sgt);__printf (2 ,3 )
@@ -257,7 +257,7 @@ struct msm_gem_submit {struct kref ref;struct drm_device *dev;struct msm_gpu *gpu; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;struct list_head node;/* node in ring submit list */ struct drm_exec exec;uint32_t seqno;/* Sequence number of the submit on the ring */
drivers/gpu/drm/msm/msm_gem_submit.c +3 -3
@@ -63,7 +63,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,kref_init (&submit->ref); submit->dev = dev; - submit->aspace = queue->ctx->aspace; + submit->vm = queue->ctx->vm; submit->gpu = gpu; submit->cmd = (void *)&submit->bos[nr_bos]; submit->queue = queue;
@@ -302,7 +302,7 @@ static int submit_pin_objects(struct msm_gem_submit *submit)struct msm_gem_vma *vma;/* if locking succeeded, pin bo: */ - vma =msm_gem_get_vma_locked (obj, submit->aspace); + vma =msm_gem_get_vma_locked (obj, submit->vm);if (IS_ERR (vma)) { ret =PTR_ERR (vma);break ;
@@ -659,7 +659,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,if (args->pad)return -EINVAL ; -if (unlikely (!ctx->aspace) && !capable (CAP_SYS_RAWIO )) { +if (unlikely (!ctx->vm) && !capable (CAP_SYS_RAWIO )) {DRM_ERROR_RATELIMITED ("IOMMU support or CAP_SYS_RAWIO required!\n" );return -EPERM ; }
drivers/gpu/drm/msm/msm_gem_vma.c +46 -47
193 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gpu.c +24 -24
108 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gpu.h +7 -9
@@ -78,10 +78,8 @@ struct msm_gpu_funcs {/* note: gpu_set_freq() can assume that we have been pm_resumed */ void (*gpu_set_freq)(struct msm_gpu *gpu,struct dev_pm_opp *opp,bool suspended); -struct msm_gem_address_space *(*create_address_space) - (struct msm_gpu *gpu,struct platform_device *pdev); -struct msm_gem_address_space *(*create_private_address_space) - (struct msm_gpu *gpu); +struct msm_gem_vm *(*create_vm)(struct msm_gpu *gpu,struct platform_device *pdev); +struct msm_gem_vm *(*create_private_vm)(struct msm_gpu *gpu);uint32_t (*get_rptr)(struct msm_gpu *gpu,struct msm_ringbuffer *ring);/**
@@ -236,7 +234,7 @@ struct msm_gpu {void __iomem *mmio;int irq; -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;/* Power Control: */ struct regulator *gpu_reg, *gpu_cx;
@@ -364,8 +362,8 @@ struct msm_context {*/ int queueid; -/** @aspace: the per-process GPU address-space */ -struct msm_gem_address_space *aspace; +/** @vm: the per-process GPU address-space */ +struct msm_gem_vm *vm;/** @kref: the reference count */ struct kref ref;
@@ -675,8 +673,8 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, struct msm_gpu *gpu, const struct msm_gpu_funcs *funcs, const char *name, struct msm_gpu_config *config); -struct msm_gem_address_space * -msm_gpu_create_private_address_space (struct msm_gpu *gpu,struct task_struct *task); +struct msm_gem_vm * +msm_gpu_create_private_vm (struct msm_gpu *gpu,struct task_struct *task);void msm_gpu_cleanup (struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_kms.c +8 -8
@@ -176,9 +176,9 @@ static int msm_kms_fault_handler(void *arg, unsigned long iova, int flags, voidreturn -ENOSYS ; } -struct msm_gem_address_space *msm_kms_init_aspace (struct drm_device *dev) +struct msm_gem_vm *msm_kms_init_vm (struct drm_device *dev) { -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;struct msm_mmu *mmu;struct device *mdp_dev = dev->dev;struct device *mdss_dev = mdp_dev->parent;
@@ -204,17 +204,17 @@ struct msm_gem_address_space *msm_kms_init_aspace(struct drm_device *dev)return NULL ; } - aspace =msm_gem_address_space_create (mmu,"mdp_kms" , + vm =msm_gem_vm_create (mmu,"mdp_kms" ,0x1000 ,0x100000000 -0x1000 ); -if (IS_ERR (aspace)) { -dev_err (mdp_dev,"aspace create, error %pe\n" , aspace); +if (IS_ERR (vm)) { +dev_err (mdp_dev,"vm create, error %pe\n" , vm); mmu->funcs->destroy (mmu); -return aspace; +return vm; } -msm_mmu_set_fault_handler (aspace->mmu, kms, msm_kms_fault_handler); +msm_mmu_set_fault_handler (vm->mmu, kms, msm_kms_fault_handler); -return aspace; +return vm; }void msm_drm_kms_uninit (struct device *dev)
drivers/gpu/drm/msm/msm_kms.h +1 -1
@@ -139,7 +139,7 @@ struct msm_kms {atomic_t fault_snapshot_capture;/* mapper-id used to request GEM buffer mapped for scanout: */ -struct msm_gem_address_space *aspace; +struct msm_gem_vm *vm;/* disp snapshot support */ struct kthread_worker *dump_worker;
drivers/gpu/drm/msm/msm_ringbuffer.c +2 -2
@@ -84,7 +84,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id, ring->start =msm_gem_kernel_new (gpu->dev,MSM_GPU_RINGBUFFER_SZ ,check_apriv (gpu,MSM_BO_WC |MSM_BO_GPU_READONLY ), - gpu->aspace, &ring->bo, &ring->iova); + gpu->vm, &ring->bo, &ring->iova);if (IS_ERR (ring->start)) { ret =PTR_ERR (ring->start);
@@ -131,7 +131,7 @@ void msm_ringbuffer_destroy(struct msm_ringbuffer *ring)msm_fence_context_free (ring->fctx); -msm_gem_kernel_put (ring->bo, ring->gpu->aspace); +msm_gem_kernel_put (ring->bo, ring->gpu->vm);kfree (ring); }
drivers/gpu/drm/msm/msm_submitqueue.c +1 -1
@@ -59,7 +59,7 @@ void __msm_context_destroy(struct kref *kref)kfree (ctx->entities[i]); } -msm_gem_address_space_put (ctx->aspace); +msm_gem_vm_put (ctx->vm);kfree (ctx->comm);kfree (ctx->cmdline);kfree (ctx);
Signature · 1 line
2.48.1
Re: [PATCH v2 07/34] drm/msm: Rename msm_gem_address_space -> msm_gem_vm
Quoted 43 lines · Rob Clark
From: Rob Clark <[email protected]>
Re-aligning naming to better match
drm_gpuvmterminology will make things less confusing at the end of thedrm_gpuvmconversion.This is just rename churn, no functional change.
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 18 ++-- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 4 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 4 +- drivers/gpu/drm/msm/adreno/a5xx_debugfs.c | 4 +- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 22 ++--- drivers/gpu/drm/msm/adreno/a5xx_power.c | 2 +- drivers/gpu/drm/msm/adreno/a5xx_preempt.c | 10 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 26 +++--- drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 2 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 45 +++++---- drivers/gpu/drm/msm/adreno/a6xx_gpu_state.c | 6 +- drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 10 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 47 +++++----- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 18 ++-- .../drm/msm/disp/dpu1/dpu_encoder_phys_wb.c | 14 +-- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c | 18 ++-- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.h | 2 +- drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c | 18 ++-- drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c | 14 +-- drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h | 4 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_crtc.c | 6 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c | 24 ++--- drivers/gpu/drm/msm/disp/mdp4/mdp4_plane.c | 12 +-- drivers/gpu/drm/msm/disp/mdp5/mdp5_crtc.c | 4 +- drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c | 18 ++-- drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c | 12 +-- drivers/gpu/drm/msm/dsi/dsi_host.c | 14 +-- drivers/gpu/drm/msm/msm_drv.c | 8 +- drivers/gpu/drm/msm/msm_drv.h | 10 +- drivers/gpu/drm/msm/msm_fb.c | 10 +- drivers/gpu/drm/msm/msm_fbdev.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 74 +++++++-------- drivers/gpu/drm/msm/msm_gem.h | 34 +++---- drivers/gpu/drm/msm/msm_gem_submit.c | 6 +-
Quoted 9 lines
drivers/gpu/drm/msm/msm_gem_vma.c | 93 +++++++++---------- drivers/gpu/drm/msm/msm_gpu.c | 48 +++++----- drivers/gpu/drm/msm/msm_gpu.h | 16 ++-- drivers/gpu/drm/msm/msm_kms.c | 16 ++-- drivers/gpu/drm/msm/msm_kms.h | 2 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 4 +- drivers/gpu/drm/msm/msm_submitqueue.c | 2 +- 41 files changed, 349 insertions(+), 354 deletions(-)
Signature · 2 lines
With best wishes Dmitry
PATCH v2 8/34 drm/msm: Remove vram carveout support
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
It is standing in the way of drm_gpuvm / VM_BIND support. Not to mention frequently broken and rarely tested. And I think only needed for a 10yr old not quite upstream SoC (msm8974).
Maybe we can add support back in later, but I'm doubtful.
--- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 6 +- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 4 - drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 - drivers/gpu/drm/msm/msm_drv.c | 117 +----------------- drivers/gpu/drm/msm/msm_drv.h | 11 -- drivers/gpu/drm/msm/msm_gem.c | 131 ++------------------- drivers/gpu/drm/msm/msm_gem.h | 5 - drivers/gpu/drm/msm/msm_gem_submit.c | 5 - 10 files changed, 19 insertions(+), 287 deletions(-)
drivers/gpu/drm/msm/adreno/a2xx_gpu.c +2 -4
@@ -553,10 +553,8 @@ struct msm_gpu *a2xx_gpu_init(struct drm_device *dev)if (!gpu->vm) {dev_err (dev->dev,"No memory protection without MMU\n" ); -if (!allow_vram_carveout) { - ret = -ENXIO ; - goto fail; - } + ret = -ENXIO ; + goto fail; }return gpu;
drivers/gpu/drm/msm/adreno/a3xx_gpu.c +2 -11
@@ -582,18 +582,9 @@ struct msm_gpu *a3xx_gpu_init(struct drm_device *dev) }if (!gpu->vm) { -/* TODO we think it is possible to configure the GPU to -* restrict access to VRAM carveout. But the required -* registers are unknown. For now just bail out and -* limp along with just modesetting. If it turns out -* to not be possible to restrict access, then we must -* implement a cmdstream validator. -*/ DRM_DEV_ERROR (dev->dev,"No memory protection without IOMMU\n" ); -if (!allow_vram_carveout) { - ret = -ENXIO ; - goto fail; - } + ret = -ENXIO ; + goto fail; } icc_path =devm_of_icc_get (&pdev->dev,"gfx-mem" );
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +2 -11
@@ -696,18 +696,9 @@ struct msm_gpu *a4xx_gpu_init(struct drm_device *dev) adreno_gpu->uche_trap_base =0xffff0000ffff0000ull ;if (!gpu->vm) { -/* TODO we think it is possible to configure the GPU to -* restrict access to VRAM carveout. But the required -* registers are unknown. For now just bail out and -* limp along with just modesetting. If it turns out -* to not be possible to restrict access, then we must -* implement a cmdstream validator. -*/ DRM_DEV_ERROR (dev->dev,"No memory protection without IOMMU\n" ); -if (!allow_vram_carveout) { - ret = -ENXIO ; - goto fail; - } + ret = -ENXIO ; + goto fail; } icc_path =devm_of_icc_get (&pdev->dev,"gfx-mem" );
drivers/gpu/drm/msm/adreno/adreno_device.c +0 -4
@@ -16,10 +16,6 @@ bool snapshot_debugbus = false;MODULE_PARM_DESC (snapshot_debugbus,"Include debugbus sections in GPU devcoredump (if not fused off)" );module_param_named (snapshot_debugbus, snapshot_debugbus, bool,0600 ); -bool allow_vram_carveout = false; -MODULE_PARM_DESC (allow_vram_carveout,"Allow using VRAM Carveout, in place of IOMMU" ); -module_param_named (allow_vram_carveout, allow_vram_carveout, bool,0600 ); -int enable_preemption =-1 ;MODULE_PARM_DESC (enable_preemption,"Enable preemption (A7xx only) (1=on , 0=disable, -1=auto (default))" );module_param (enable_preemption, int,0600 );
drivers/gpu/drm/msm/adreno/adreno_gpu.h +0 -1
@@ -18,7 +18,6 @@#include "adreno_pm4.xml.h" extern bool snapshot_debugbus; -extern bool allow_vram_carveout;enum {ADRENO_FW_PM4 =0 ,
drivers/gpu/drm/msm/msm_drv.c +1 -116
152 lines not on this page · raw message
drivers/gpu/drm/msm/msm_drv.h +0 -11
@@ -183,17 +183,6 @@ struct msm_drm_private {struct msm_drm_thread event_thread[MAX_CRTCS ]; -/* VRAM carveout, used when no IOMMU: */ -struct { -unsigned long size; -dma_addr_t paddr; -/* NOTE: mm managed at the page level, size is in # of pages -* and position mm_node->start is in # of pages: -*/ -struct drm_mm mm; -spinlock_t lock;/* Protects drm_mm node allocation/removal */ - } vram; -struct notifier_block vmap_notifier;struct shrinker *shrinker;
drivers/gpu/drm/msm/msm_gem.c +12 -119
185 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gem.h +0 -5
@@ -102,11 +102,6 @@ struct msm_gem_object {struct list_head vmas;/* list of msm_gem_vma */ -/* For physically contiguous buffers. Used when we don't have -* an IOMMU. Also used for stolen/splashscreen buffer. -*/ -struct drm_mm_node *vram_node; -char name[32 ];/* Identifier to print for the debugfs files */ /* userspace metadata backchannel */
drivers/gpu/drm/msm/msm_gem_submit.c +0 -5
@@ -659,11 +659,6 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,if (args->pad)return -EINVAL ; -if (unlikely (!ctx->vm) && !capable (CAP_SYS_RAWIO )) { -DRM_ERROR_RATELIMITED ("IOMMU support or CAP_SYS_RAWIO required!\n" ); -return -EPERM ; - } -/* for now, we just have 3d pipe.. eventually this would need to * be more clever to dispatch to appropriate gpu module: */
Signature · 1 line
2.48.1
Re: [PATCH v2 08/34] drm/msm: Remove vram carveout support
Quoted 7 lines · Rob Clark
From: Rob Clark <[email protected]>
It is standing in the way of
drm_gpuvm/VM_BINDsupport. Not to mention frequently broken and rarely tested. And I think only needed for a 10yr old not quite upstream SoC (msm8974).Maybe we can add support back in later, but I'm doubtful.
There is a small clean up of "VRAM carveout" required in a debug print in msm_gpu.c
Quoted 62 lines
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 6 +- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 4 - drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 - drivers/gpu/drm/msm/msm_drv.c | 117 +----------------- drivers/gpu/drm/msm/msm_drv.h | 11 -- drivers/gpu/drm/msm/msm_gem.c | 131 ++------------------- drivers/gpu/drm/msm/msm_gem.h | 5 - drivers/gpu/drm/msm/msm_gem_submit.c | 5 - 10 files changed, 19 insertions(+), 287 deletions(-)drivers/gpu/drm/msm/adreno/a2xx_gpu.c +2
-4@@ -553,10 +553,8 @@ struct msm_gpu *a2xx_gpu_init(struct drm_device *dev)if (!gpu->vm) {dev_err (dev->dev,"No memory protection without MMU\n" ); -if (!allow_vram_carveout) { - ret = -ENXIO ; - goto fail; - } + ret = -ENXIO ; + goto fail; }return gpu;drivers/gpu/drm/msm/adreno/a3xx_gpu.c +2
-11@@ -582,18 +582,9 @@ struct msm_gpu *a3xx_gpu_init(struct drm_device *dev) }if (!gpu->vm) { -/* TODO we think it is possible to configure the GPU to -* restrict access to VRAM carveout. But the required -* registers are unknown. For now just bail out and -* limp along with just modesetting. If it turns out -* to not be possible to restrict access, then we must -* implement a cmdstream validator. -*/ DRM_DEV_ERROR (dev->dev,"No memory protection without IOMMU\n" ); -if (!allow_vram_carveout) { - ret = -ENXIO ; - goto fail; - } + ret = -ENXIO ; + goto fail; } icc_path =devm_of_icc_get (&pdev->dev,"gfx-mem" );drivers/gpu/drm/msm/adreno/a4xx_gpu.c +0
-0
drivers/gpu/drm/msm/adreno/a4xx_gpu.c struct msm_gpu *a4xx_gpu_init(struct drm_device *dev) · line 698
Quoted 3 lines
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +0
-0@@ -696,18 +696,9 @@ struct msm_gpu *a4xx_gpu_init(struct drm_device *dev) adreno_gpu->uche_trap_base =0xffff0000ffff0000ull ;if (!gpu->vm) {
Is this check still required?
-Akhil
Quoted 456 lines
drivers/gpu/drm/msm/adreno/a4xx_gpu.c +2
-11-/* TODO we think it is possible to configure the GPU to -* restrict access to VRAM carveout. But the required -* registers are unknown. For now just bail out and -* limp along with just modesetting. If it turns out -* to not be possible to restrict access, then we must -* implement a cmdstream validator. -*/ DRM_DEV_ERROR (dev->dev,"No memory protection without IOMMU\n" ); -if (!allow_vram_carveout) { - ret = -ENXIO ; - goto fail; - } + ret = -ENXIO ; + goto fail; } icc_path =devm_of_icc_get (&pdev->dev,"gfx-mem" );drivers/gpu/drm/msm/adreno/adreno_device.c +0
-4@@ -16,10 +16,6 @@ bool snapshot_debugbus = false;MODULE_PARM_DESC (snapshot_debugbus,"Include debugbus sections in GPU devcoredump (if not fused off)" );module_param_named (snapshot_debugbus, snapshot_debugbus, bool,0600 ); -bool allow_vram_carveout = false; -MODULE_PARM_DESC (allow_vram_carveout,"Allow using VRAM Carveout, in place of IOMMU" ); -module_param_named (allow_vram_carveout, allow_vram_carveout, bool,0600 ); -int enable_preemption =-1 ;MODULE_PARM_DESC (enable_preemption,"Enable preemption (A7xx only) (1=on , 0=disable, -1=auto (default))" );module_param (enable_preemption, int,0600 );drivers/gpu/drm/msm/adreno/adreno_gpu.h +0
-1@@ -18,7 +18,6 @@#include "adreno_pm4.xml.h" extern bool snapshot_debugbus; -extern bool allow_vram_carveout;enum {ADRENO_FW_PM4 =0 ,drivers/gpu/drm/msm/msm_drv.c +1
-116@@ -46,12 +46,6 @@#define MSM_VERSION_MINOR 12#define MSM_VERSION_PATCHLEVEL 0 -static void msm_deinit_vram (struct drm_device *ddev); - -static char *vram ="16m" ; -MODULE_PARM_DESC (vram,"Configure VRAM size (for devices without IOMMU/GPUMMU)" ); -module_param (vram, charp,0 ); -bool dumpstate;MODULE_PARM_DESC (dumpstate,"Dump KMS state on errors" );module_param (dumpstate, bool,0600 );@@ -97,8 +91,6 @@ static int msm_drm_uninit(struct device *dev)if (priv->kms)msm_drm_kms_uninit (dev); -msm_deinit_vram (ddev); -component_unbind_all (dev, ddev); ddev->dev_private =NULL ;@@ -109,107 +101,6 @@ static int msm_drm_uninit(struct device *dev)return 0 ; } -bool msm_use_mmu (struct drm_device *dev) -{ -struct msm_drm_private *priv = dev->dev_private; - -/* -* a2xx comes with its own MMU -* On other platforms IOMMU can be declared specified either for the -* MDP/DPU device or for its parent, MDSS device. -*/ -return priv->is_a2xx || -device_iommu_mapped (dev->dev) || -device_iommu_mapped (dev->dev->parent); -} - -static int msm_init_vram (struct drm_device *dev) -{ -struct msm_drm_private *priv = dev->dev_private; -struct device_node *node; -unsigned long size =0 ; -int ret =0 ; - -/* In the device-tree world, we could have a 'memory-region' -* phandle, which gives us a link to our "vram". Allocating -* is all nicely abstracted behind the dma api, but we need -* to know the entire size to allocate it all in one go. There -* are two cases: -* 1) device with no IOMMU, in which case we need exclusive -* access to a VRAM carveout big enough for all gpu -* buffers -* 2) device with IOMMU, but where the bootloader puts up -* a splash screen. In this case, the VRAM carveout -* need only be large enough for fbdev fb. But we need -* exclusive access to the buffer to avoid the kernel -* using those pages for other purposes (which appears -* as corruption on screen before we have a chance to -* load and do initial modeset) -*/ - - node =of_parse_phandle (dev->dev->of_node,"memory-region" ,0 ); -if (node) { -struct resource r; - ret =of_address_to_resource (node,0 , &r); -of_node_put (node); -if (ret) -return ret; - size = r.end - r.start +1 ; -DRM_INFO ("using VRAM carveout: %lx@%pa\n" , size, &r.start); - -/* if we have no IOMMU, then we need to use carveout allocator. -* Grab the entire DMA chunk carved out in early startup in -* mach-msm: -*/ - }else if (!msm_use_mmu (dev)) { -DRM_INFO ("using %s VRAM carveout\n" , vram); - size =memparse (vram,NULL ); - } - -if (size) { -unsigned long attrs =0 ; -void *p; - - priv->vram.size = size; - -drm_mm_init (&priv->vram.mm,0 , (size >>PAGE_SHIFT ) -1 ); -spin_lock_init (&priv->vram.lock); - - attrs |=DMA_ATTR_NO_KERNEL_MAPPING ; - attrs |=DMA_ATTR_WRITE_COMBINE ; - -/* note that for no-kernel-mapping, the vaddr returned -* is bogus, but non-null if allocation succeeded: -*/ - p =dma_alloc_attrs (dev->dev, size, - &priv->vram.paddr,GFP_KERNEL , attrs); -if (!p) { -DRM_DEV_ERROR (dev->dev,"failed to allocate VRAM\n" ); - priv->vram.paddr =0 ; -return -ENOMEM ; - } - -DRM_DEV_INFO (dev->dev,"VRAM: %08x->%08x\n" , - (uint32_t )priv->vram.paddr, - (uint32_t )(priv->vram.paddr + size)); - } - -return ret; -} - -static void msm_deinit_vram (struct drm_device *ddev) -{ -struct msm_drm_private *priv = ddev->dev_private; -unsigned long attrs =DMA_ATTR_NO_KERNEL_MAPPING ; - -if (!priv->vram.paddr) -return ; - -drm_mm_takedown (&priv->vram.mm); -dma_free_attrs (ddev->dev, priv->vram.size,NULL , priv->vram.paddr, - attrs); -} -static int msm_drm_init (struct device *dev,const struct drm_driver *drv) {struct msm_drm_private *priv =dev_get_drvdata (dev);@@ -256,16 +147,12 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv) goto err_destroy_wq; } - ret =msm_init_vram (ddev); -if (ret) - goto err_destroy_wq; -dma_set_max_seg_size (dev,UINT_MAX );/* Bind all our sub-components: */ ret =component_bind_all (dev, ddev);if (ret) - goto err_deinit_vram; + goto err_destroy_wq; ret =msm_gem_shrinker_init (ddev);if (ret)@@ -302,8 +189,6 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv)return ret; -err_deinit_vram: -msm_deinit_vram (ddev); err_destroy_wq:destroy_workqueue (priv->wq); err_put_dev:drivers/gpu/drm/msm/msm_drv.h +0
-11@@ -183,17 +183,6 @@ struct msm_drm_private {struct msm_drm_thread event_thread[MAX_CRTCS ]; -/* VRAM carveout, used when no IOMMU: */ -struct { -unsigned long size; -dma_addr_t paddr; -/* NOTE: mm managed at the page level, size is in # of pages -* and position mm_node->start is in # of pages: -*/ -struct drm_mm mm; -spinlock_t lock;/* Protects drm_mm node allocation/removal */ - } vram; -struct notifier_block vmap_notifier;struct shrinker *shrinker;drivers/gpu/drm/msm/msm_gem.c +12
-119@@ -17,24 +17,8 @@#include <trace/events/gpu_mem.h> #include "msm_drv.h" -#include "msm_fence.h" #include "msm_gem.h" #include "msm_gpu.h" -#include "msm_mmu.h" - -static dma_addr_t physaddr (struct drm_gem_object *obj) -{ -struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_drm_private *priv = obj->dev->dev_private; -return (((dma_addr_t )msm_obj->vram_node->start) <<PAGE_SHIFT ) + - priv->vram.paddr; -} - -static bool use_pages (struct drm_gem_object *obj) -{ -struct msm_gem_object *msm_obj =to_msm_bo (obj); -return !msm_obj->vram_node; -}static int pgprot =0 ;module_param (pgprot, int,0600 );@@ -139,36 +123,6 @@ static void update_lru(struct drm_gem_object *obj)mutex_unlock (&priv->lru.lock); } -/* allocate pages from VRAM carveout, used when no IOMMU: */ -static struct page **get_pages_vram (struct drm_gem_object *obj,int npages) -{ -struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_drm_private *priv = obj->dev->dev_private; -dma_addr_t paddr; -struct page **p; -int ret, i; - - p =kvmalloc_array (npages,sizeof (struct page *),GFP_KERNEL ); -if (!p) -return ERR_PTR (-ENOMEM ); - -spin_lock (&priv->vram.lock); - ret =drm_mm_insert_node (&priv->vram.mm, msm_obj->vram_node, npages); -spin_unlock (&priv->vram.lock); -if (ret) { -kvfree (p); -return ERR_PTR (ret); - } - - paddr =physaddr (obj); -for (i =0 ; i < npages; i++) { - p[i] =pfn_to_page (__phys_to_pfn (paddr)); - paddr +=PAGE_SIZE ; - } - -return p; -} -static struct page **get_pages (struct drm_gem_object *obj) {struct msm_gem_object *msm_obj =to_msm_bo (obj);@@ -180,10 +134,7 @@ static struct page **get_pages(struct drm_gem_object *obj)struct page **p;int npages = obj->size >>PAGE_SHIFT ; -if (use_pages (obj)) - p =drm_gem_get_pages (obj); -else - p =get_pages_vram (obj, npages); + p =drm_gem_get_pages (obj);if (IS_ERR (p)) { DRM_DEV_ERROR(dev->dev, "could not get pages: %ld\n",@@ -216,18 +167,6 @@ static struct page **get_pages(struct drm_gem_object *obj)return msm_obj->pages; } -static void put_pages_vram (struct drm_gem_object *obj) -{ -struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_drm_private *priv = obj->dev->dev_private; - -spin_lock (&priv->vram.lock); -drm_mm_remove_node (msm_obj->vram_node); -spin_unlock (&priv->vram.lock); - -kvfree (msm_obj->pages); -} -static void put_pages (struct drm_gem_object *obj) {struct msm_gem_object *msm_obj =to_msm_bo (obj);@@ -248,10 +187,7 @@ static void put_pages(struct drm_gem_object *obj)update_device_mem (obj->dev->dev_private, -obj->size); -if (use_pages (obj)) -drm_gem_put_pages (obj, msm_obj->pages, true, false); -else -put_pages_vram (obj); +drm_gem_put_pages (obj, msm_obj->pages, true, false); msm_obj->pages =NULL ;update_lru (obj);@@ -1215,19 +1151,10 @@ struct drm_gem_object *msm_gem_new(struct drm_device *dev, uint32_t size, uint32struct msm_drm_private *priv = dev->dev_private;struct msm_gem_object *msm_obj;struct drm_gem_object *obj =NULL ; -bool use_vram = false;int ret; size =PAGE_ALIGN (size); -if (!msm_use_mmu (dev)) - use_vram = true; -else if ((flags & (MSM_BO_STOLEN |MSM_BO_SCANOUT )) && priv->vram.size) - use_vram = true; - -if (GEM_WARN_ON (use_vram && !priv->vram.size)) -return ERR_PTR (-EINVAL ); -/* Disallow zero sized objects as they make the underlying * infrastructure grumpy */ @@ -1240,44 +1167,16 @@ struct drm_gem_object *msm_gem_new(struct drm_device *dev, uint32_t size, uint32 msm_obj =to_msm_bo (obj); -if (use_vram) { -struct msm_gem_vma *vma; -struct page **pages; - -drm_gem_private_object_init (dev, obj, size); - -msm_gem_lock (obj); - - vma =add_vma (obj,NULL ); -msm_gem_unlock (obj); -if (IS_ERR (vma)) { - ret =PTR_ERR (vma); - goto fail; - } - -to_msm_bo (obj)->vram_node = &vma->node; - -msm_gem_lock (obj); - pages =get_pages (obj); -msm_gem_unlock (obj); -if (IS_ERR (pages)) { - ret =PTR_ERR (pages); - goto fail; - } - - vma->iova =physaddr (obj); - }else { - ret =drm_gem_object_init (dev, obj, size); -if (ret) - goto fail; -/* -* Our buffers are kept pinned, so allocating them from the -* MOVABLE zone is a really bad idea, and conflicts with CMA. -* See comments above new_inode() why this is required _and_ -* expected if you're going to pin these pages. -*/ -mapping_set_gfp_mask (obj->filp->f_mapping,GFP_HIGHUSER ); - } + ret =drm_gem_object_init (dev, obj, size); +if (ret) + goto fail; +/* +* Our buffers are kept pinned, so allocating them from the +* MOVABLE zone is a really bad idea, and conflicts with CMA. +* See comments above new_inode() why this is required _and_ +* expected if you're going to pin these pages. +*/ +mapping_set_gfp_mask (obj->filp->f_mapping,GFP_HIGHUSER );drm_gem_lru_move_tail (&priv->lru.unbacked, obj);@@ -1305,12 +1204,6 @@ struct drm_gem_object *msm_gem_import(struct drm_device *dev,uint32_t size;int ret, npages; -/* if we don't have IOMMU, don't bother pretending we can import: */ -if (!msm_use_mmu (dev)) { -DRM_DEV_ERROR (dev->dev,"cannot import without IOMMU\n" ); -return ERR_PTR (-EINVAL ); - } - size =PAGE_ALIGN (dmabuf->size); ret =msm_gem_new_impl (dev, size,MSM_BO_WC , &obj);drivers/gpu/drm/msm/msm_gem.h +0
-5@@ -102,11 +102,6 @@ struct msm_gem_object {struct list_head vmas;/* list of msm_gem_vma */ -/* For physically contiguous buffers. Used when we don't have -* an IOMMU. Also used for stolen/splashscreen buffer. -*/ -struct drm_mm_node *vram_node; -char name[32 ];/* Identifier to print for the debugfs files */ /* userspace metadata backchannel */ drivers/gpu/drm/msm/msm_gem_submit.c +0
-5@@ -659,11 +659,6 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,if (args->pad)return -EINVAL ; -if (unlikely (!ctx->vm) && !capable (CAP_SYS_RAWIO )) { -DRM_ERROR_RATELIMITED ("IOMMU support or CAP_SYS_RAWIO required!\n" ); -return -EPERM ; - } -/* for now, we just have 3d pipe.. eventually this would need to * be more clever to dispatch to appropriate gpu module: */
Re: [PATCH v2 08/34] drm/msm: Remove vram carveout support
In reply to Rob ClarkQuoted 5 lines · Rob Clark
From: Rob Clark <[email protected]>
It is standing in the way of
drm_gpuvm/VM_BINDsupport. Not to mention frequently broken and rarely tested. And I think only needed for a 10yr old not quite upstream SoC (msm8974).
Well... MSM8974 is quite upstream, but anyway, let's drop it. Maybe somebody will write an IOMMU driver.
Quoted 16 lines
Maybe we can add support back in later, but I'm doubtful.
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 6 +- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 4 - drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 - drivers/gpu/drm/msm/msm_drv.c | 117 +----------------- drivers/gpu/drm/msm/msm_drv.h | 11 -- drivers/gpu/drm/msm/msm_gem.c | 131 ++------------------- drivers/gpu/drm/msm/msm_gem.h | 5 - drivers/gpu/drm/msm/msm_gem_submit.c | 5 - 10 files changed, 19 insertions(+), 287 deletions(-)
Signature · 2 lines
With best wishes Dmitry
Re: [PATCH v2 08/34] drm/msm: Remove vram carveout support
Quoted 9 lines · Dmitry Baryshkov
Quoted 5 lines · Rob Clark
From: Rob Clark <[email protected]>
It is standing in the way of
drm_gpuvm/VM_BINDsupport. Not to mention frequently broken and rarely tested. And I think only needed for a 10yr old not quite upstream SoC (msm8974).Well... MSM8974 is quite upstream, but anyway, let's drop it. Maybe somebody will write an IOMMU driver.
msm8226 is also using this!
Sad to see this happening, but I get the reasoning. Unfortunately nobody who really knows GPU and IOMMU bits has looked at this in recent years, for msm8974 we (mostly Matti Lehtimäki and me) have a semi-working branch but hitting random issues with it.
Would've been nice if somebody made functional IOMMU support back in like 2015-2016 when more people looked at this platform.
Regards
Luca
Quoted 17 lines
Quoted 17 lines
Maybe we can add support back in later, but I'm doubtful.
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 6 +- drivers/gpu/drm/msm/adreno/a3xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/a4xx_gpu.c | 13 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 4 - drivers/gpu/drm/msm/adreno/adreno_gpu.h | 1 - drivers/gpu/drm/msm/msm_drv.c | 117 +----------------- drivers/gpu/drm/msm/msm_drv.h | 11 -- drivers/gpu/drm/msm/msm_gem.c | 131 ++------------------- drivers/gpu/drm/msm/msm_gem.h | 5 - drivers/gpu/drm/msm/msm_gem_submit.c | 5 - 10 files changed, 19 insertions(+), 287 deletions(-)
PATCH v2 9/34 drm/msm: Collapse vma allocation and initialization
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Now that we've dropped vram carveout support, we can collapse vma allocation and initialization. This better matches how things work with drm_gpuvm.
--- drivers/gpu/drm/msm/msm_gem.c | 30 +++----------------------- drivers/gpu/drm/msm/msm_gem.h | 4 ++-- drivers/gpu/drm/msm/msm_gem_vma.c | 36 +++++++++++++------------------ 3 files changed, 20 insertions(+), 50 deletions(-)
drivers/gpu/drm/msm/msm_gem.c +3 -27
@@ -337,23 +337,6 @@ uint64_t msm_gem_mmap_offset(struct drm_gem_object *obj)return offset; } -static struct msm_gem_vma *add_vma (struct drm_gem_object *obj, -struct msm_gem_vm *vm) -{ -struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_gem_vma *vma; - -msm_gem_assert_locked (obj); - - vma =msm_gem_vma_new (vm); -if (!vma) -return ERR_PTR (-ENOMEM ); - -list_add_tail (&vma->list, &msm_obj->vmas); - -return vma; -} -static struct msm_gem_vma *lookup_vma (struct drm_gem_object *obj,struct msm_gem_vm *vm) {
@@ -420,6 +403,7 @@ static struct msm_gem_vma *get_vma_locked(struct drm_gem_object *obj,struct msm_gem_vm *vm, u64 range_start, u64 range_end) { + struct msm_gem_object *msm_obj = to_msm_bo(obj);struct msm_gem_vma *vma;msm_gem_assert_locked (obj);
@@ -427,18 +411,10 @@ static struct msm_gem_vma *get_vma_locked(struct drm_gem_object *obj, vma =lookup_vma (obj, vm);if (!vma) { -int ret; - - vma =add_vma (obj, vm); + vma =msm_gem_vma_new (vm, obj, range_start, range_end);if (IS_ERR (vma))return vma; - - ret =msm_gem_vma_init (vma, obj->size, - range_start, range_end); -if (ret) { -del_vma (vma); -return ERR_PTR (ret); - } +list_add_tail (&vma->list, &msm_obj->vmas); }else {GEM_WARN_ON (vma->iova < range_start);GEM_WARN_ON ((vma->iova + obj->size) > range_end);
drivers/gpu/drm/msm/msm_gem.h +2 -2
@@ -66,8 +66,8 @@ struct msm_gem_vma {bool mapped; }; -struct msm_gem_vma *msm_gem_vma_new (struct msm_gem_vm *vm); -int msm_gem_vma_init (struct msm_gem_vma *vma,int size, +struct msm_gem_vma * +msm_gem_vma_new (struct msm_gem_vm *vm,struct drm_gem_object *obj,u64 range_start,u64 range_end);void msm_gem_vma_purge (struct msm_gem_vma *vma);int msm_gem_vma_map (struct msm_gem_vma *vma,int prot,struct sg_table *sgt,int size);
drivers/gpu/drm/msm/msm_gem_vma.c +15 -21
@@ -106,47 +106,41 @@ void msm_gem_vma_close(struct msm_gem_vma *vma)msm_gem_vm_put (vm); } -struct msm_gem_vma *msm_gem_vma_new (struct msm_gem_vm *vm) +/* Create a new vma and allocate an iova for it */ +struct msm_gem_vma * +msm_gem_vma_new (struct msm_gem_vm *vm,struct drm_gem_object *obj, +u64 range_start,u64 range_end) {struct msm_gem_vma *vma; +int ret; vma =kzalloc (sizeof (*vma),GFP_KERNEL );if (!vma) -return NULL ; +return ERR_PTR (-ENOMEM ); vma->vm = vm; -return vma; -} - -/* Initialize a new vma and allocate an iova for it */ -int msm_gem_vma_init (struct msm_gem_vma *vma,int size, -u64 range_start,u64 range_end) -{ -struct msm_gem_vm *vm = vma->vm; -int ret; - -if (GEM_WARN_ON (!vm)) -return -EINVAL ; - -if (GEM_WARN_ON (vma->iova)) -return -EBUSY ; -spin_lock (&vm->lock); ret =drm_mm_insert_node_in_range (&vm->mm, &vma->node, - size,PAGE_SIZE ,0 , + obj->size,PAGE_SIZE ,0 , range_start, range_end,0 );spin_unlock (&vm->lock);if (ret) -return ret; + goto err_free_vma; vma->iova = vma->node.start; vma->mapped = false; +INIT_LIST_HEAD (&vma->list); +kref_get (&vm->kref); -return 0 ; +return vma; + +err_free_vma: +kfree (vma); +return ERR_PTR (ret); } struct msm_gem_vm *
Signature · 1 line
2.48.1
PATCH v2 10/34 drm/msm: Collapse vma close and delete
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
This fits better drm_gpuvm/drm_gpuva.
--- drivers/gpu/drm/msm/msm_gem.c | 16 +++------------- drivers/gpu/drm/msm/msm_gem_vma.c | 2 ++ 2 files changed, 5 insertions(+), 13 deletions(-)
drivers/gpu/drm/msm/msm_gem.c +3 -13
@@ -353,15 +353,6 @@ static struct msm_gem_vma *lookup_vma(struct drm_gem_object *obj,return NULL ; } -static void del_vma (struct msm_gem_vma *vma) -{ -if (!vma) -return ; - -list_del (&vma->list); -kfree (vma); -} -/* * If close is true, this also closes the VMA (releasing the allocated * iova range) in addition to removing the iommu mapping. In the eviction
@@ -372,11 +363,11 @@ static void put_iova_spaces(struct drm_gem_object *obj, bool close) {struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_gem_vma *vma; +struct msm_gem_vma *vma, *tmp;msm_gem_assert_locked (obj); -list_for_each_entry (vma, &msm_obj->vmas, list) { +list_for_each_entry_safe (vma, tmp, &msm_obj->vmas, list) {if (vma->vm) {msm_gem_vma_purge (vma);if (close)
@@ -395,7 +386,7 @@ put_iova_vmas(struct drm_gem_object *obj)msm_gem_assert_locked (obj);list_for_each_entry_safe (vma, tmp, &msm_obj->vmas, list) { -del_vma (vma); +msm_gem_vma_close (vma); } }
@@ -564,7 +555,6 @@ static int clear_iova(struct drm_gem_object *obj,msm_gem_vma_purge (vma);msm_gem_vma_close (vma); -del_vma (vma);return 0 ; }
drivers/gpu/drm/msm/msm_gem_vma.c +2 -0
@@ -102,8 +102,10 @@ void msm_gem_vma_close(struct msm_gem_vma *vma)spin_unlock (&vm->lock); vma->iova =0 ; +list_del (&vma->list);msm_gem_vm_put (vm); +kfree (vma); }/* Create a new vma and allocate an iova for it */
Signature · 1 line
2.48.1
PATCH v2 11/34 drm/msm: drm_gpuvm conversion
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Now that we've realigned deletion and allocation, switch over to using drm_gpuvm/drm_gpuva. This allows us to support multiple VMAs per BO per VM, to allow mapping different parts of a single BO at different virtual addresses, which is a key requirement for sparse/VM_BIND.
This prepares us for using drm_gpuvm to translate a batch of MAP/ MAP_NULL/UNMAP operations from userspace into a sequence of map/remap/ unmap steps for updating the page tables.
--- drivers/gpu/drm/msm/Kconfig | 1 + drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 3 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 6 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 5 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 7 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c | 5 +- drivers/gpu/drm/msm/msm_drv.c | 1 + drivers/gpu/drm/msm/msm_gem.c | 142 ++++++++++++++--------- drivers/gpu/drm/msm/msm_gem.h | 87 +++++++++++--- drivers/gpu/drm/msm/msm_gem_submit.c | 2 +- drivers/gpu/drm/msm/msm_gem_vma.c | 139 +++++++++++++++------- drivers/gpu/drm/msm/msm_kms.c | 4 +- 12 files changed, 268 insertions(+), 134 deletions(-)
drivers/gpu/drm/msm/Kconfig +1 -0
@@ -21,6 +21,7 @@ config DRM_MSM select DRM_DISPLAY_HELPER select DRM_BRIDGE_CONNECTOR select DRM_EXEC + select DRM_GPUVM select DRM_KMS_HELPER select DRM_PANEL select DRM_BRIDGE
drivers/gpu/drm/msm/adreno/a2xx_gpu.c +1 -2
@@ -472,8 +472,7 @@ a2xx_create_vm(struct msm_gpu *gpu, struct platform_device *pdev)struct msm_mmu *mmu =a2xx_gpummu_new (&pdev->dev, gpu);struct msm_gem_vm *vm; - vm =msm_gem_vm_create (mmu,"gpu" ,SZ_16M , -0xfff *SZ_64K ); + vm =msm_gem_vm_create (gpu->dev, mmu,"gpu" ,SZ_16M ,0xfff *SZ_64K , true);if (IS_ERR (vm) && !IS_ERR (mmu)) mmu->funcs->destroy (mmu);
drivers/gpu/drm/msm/adreno/a6xx_gmu.c +3 -3
@@ -1311,7 +1311,7 @@ static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,return 0 ; } -static int a6xx_gmu_memory_probe (struct a6xx_gmu *gmu) +static int a6xx_gmu_memory_probe (struct drm_device *drm,struct a6xx_gmu *gmu) {struct msm_mmu *mmu;
@@ -1321,7 +1321,7 @@ static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu)if (IS_ERR (mmu))return PTR_ERR (mmu); - gmu->vm =msm_gem_vm_create (mmu,"gmu" ,0x0 ,0x80000000 ); + gmu->vm =msm_gem_vm_create (drm, mmu,"gmu" ,0x0 ,0x80000000 , true);if (IS_ERR (gmu->vm))return PTR_ERR (gmu->vm);
@@ -1940,7 +1940,7 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node)if (ret) goto err_put_device; - ret =a6xx_gmu_memory_probe (gmu); + ret =a6xx_gmu_memory_probe (adreno_gpu->base.dev, gmu);if (ret) goto err_put_device;
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +2 -3
@@ -2271,9 +2271,8 @@ a6xx_create_private_vm(struct msm_gpu *gpu)if (IS_ERR (mmu))return ERR_CAST (mmu); -return msm_gem_vm_create (mmu, -"gpu" ,0x100000000ULL , -adreno_private_vm_size (gpu)); +return msm_gem_vm_create (gpu->dev, mmu,"gpu" ,0x100000000ULL , +adreno_private_vm_size (gpu), true); }static uint32_t a6xx_get_rptr (struct msm_gpu *gpu,struct msm_ringbuffer *ring)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +4 -3
@@ -224,7 +224,8 @@ adreno_iommu_create_vm(struct msm_gpu *gpu, start =max_t (u64,SZ_16M , geometry->aperture_start); size = geometry->aperture_end - start +1 ; - vm =msm_gem_vm_create (mmu,"gpu" , start &GENMASK_ULL (48 ,0 ), size); + vm =msm_gem_vm_create (gpu->dev, mmu,"gpu" , start &GENMASK_ULL (48 ,0 ), + size, true);if (IS_ERR (vm) && !IS_ERR (mmu)) mmu->funcs->destroy (mmu);
@@ -403,12 +404,12 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,case MSM_PARAM_VA_START :if (ctx->vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->va_start; + *value = ctx->vm->base.mm_start;return 0 ;case MSM_PARAM_VA_SIZE :if (ctx->vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->va_size; + *value = ctx->vm->base.mm_range;return 0 ;case MSM_PARAM_HIGHEST_BANK_BIT : *value = adreno_gpu->ubwc_config.highest_bank_bit;
drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c +3 -2
@@ -451,8 +451,9 @@ static int mdp4_kms_init(struct drm_device *dev) "contig buffers for scanout\n"); vm =NULL ; }else { - vm =msm_gem_vm_create (mmu, -"mdp4" ,0x1000 ,0x100000000 -0x1000 ); + vm =msm_gem_vm_create (dev, mmu,"mdp4" , +0x1000 ,0x100000000 -0x1000 , + true);if (IS_ERR (vm)) {if (!IS_ERR (mmu))
drivers/gpu/drm/msm/msm_drv.c +1 -0
@@ -776,6 +776,7 @@ static const struct file_operations fops = {static const struct drm_driver msm_driver = { .driver_features =DRIVER_GEM | +DRIVER_GEM_GPUVA |DRIVER_RENDER |DRIVER_ATOMIC | DRIVER_MODESET |
drivers/gpu/drm/msm/msm_gem.c +84 -58
273 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gem.h +68 -19
130 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gem_submit.c +1 -1
@@ -312,7 +312,7 @@ static int submit_pin_objects(struct msm_gem_submit *submit)if (ret)break ; - submit->bos[i].iova = vma->iova; + submit->bos[i].iova = vma->base.va.addr; }/*
drivers/gpu/drm/msm/msm_gem_vma.c +98 -41
242 lines not on this page · raw message
drivers/gpu/drm/msm/msm_kms.c +2 -2
@@ -204,8 +204,8 @@ struct msm_gem_vm *msm_kms_init_vm(struct drm_device *dev)return NULL ; } - vm =msm_gem_vm_create (mmu,"mdp_kms" , -0x1000 ,0x100000000 -0x1000 ); + vm =msm_gem_vm_create (dev, mmu,"mdp_kms" , +0x1000 ,0x100000000 -0x1000 , true);if (IS_ERR (vm)) {dev_err (mdp_dev,"vm create, error %pe\n" , vm); mmu->funcs->destroy (mmu);
Signature · 1 line
2.48.1
Re: [PATCH v2 11/34] drm/msm: drm_gpuvm conversion
Quoted 793 lines · Rob Clark
From: Rob Clark <[email protected]>
Now that we've realigned deletion and allocation, switch over to using drm_gpuvm/drm_gpuva. This allows us to support multiple VMAs per BO per VM, to allow mapping different parts of a single BO at different virtual addresses, which is a key requirement for sparse/VM_BIND.
This prepares us for using
drm_gpuvmto translate a batch of MAP/ MAP_NULL/UNMAP operations from userspace into a sequence of map/remap/ unmap steps for updating the page tables.Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/Kconfig | 1 + drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 3 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 6 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 5 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 7 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c | 5 +- drivers/gpu/drm/msm/msm_drv.c | 1 + drivers/gpu/drm/msm/msm_gem.c | 142 ++++++++++++++--------- drivers/gpu/drm/msm/msm_gem.h | 87 +++++++++++--- drivers/gpu/drm/msm/msm_gem_submit.c | 2 +- drivers/gpu/drm/msm/msm_gem_vma.c | 139 +++++++++++++++------- drivers/gpu/drm/msm/msm_kms.c | 4 +- 12 files changed, 268 insertions(+), 134 deletions(-)drivers/gpu/drm/msm/Kconfig +1
-0@@ -21,6 +21,7 @@ config DRM_MSM select DRM_DISPLAY_HELPER select DRM_BRIDGE_CONNECTOR select DRM_EXEC + select DRM_GPUVM select DRM_KMS_HELPER select DRM_PANEL select DRM_BRIDGEdrivers/gpu/drm/msm/adreno/a2xx_gpu.c +1
-2@@ -472,8 +472,7 @@ a2xx_create_vm(struct msm_gpu *gpu, struct platform_device *pdev)struct msm_mmu *mmu =a2xx_gpummu_new (&pdev->dev, gpu);struct msm_gem_vm *vm; - vm =msm_gem_vm_create (mmu,"gpu" ,SZ_16M , -0xfff *SZ_64K ); + vm =msm_gem_vm_create (gpu->dev, mmu,"gpu" ,SZ_16M ,0xfff *SZ_64K , true);if (IS_ERR (vm) && !IS_ERR (mmu)) mmu->funcs->destroy (mmu);drivers/gpu/drm/msm/adreno/a6xx_gmu.c +3
-3@@ -1311,7 +1311,7 @@ static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,return 0 ; } -static int a6xx_gmu_memory_probe (struct a6xx_gmu *gmu) +static int a6xx_gmu_memory_probe (struct drm_device *drm,struct a6xx_gmu *gmu) {struct msm_mmu *mmu;@@ -1321,7 +1321,7 @@ static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu)if (IS_ERR (mmu))return PTR_ERR (mmu); - gmu->vm =msm_gem_vm_create (mmu,"gmu" ,0x0 ,0x80000000 ); + gmu->vm =msm_gem_vm_create (drm, mmu,"gmu" ,0x0 ,0x80000000 , true);if (IS_ERR (gmu->vm))return PTR_ERR (gmu->vm);@@ -1940,7 +1940,7 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node)if (ret) goto err_put_device; - ret =a6xx_gmu_memory_probe (gmu); + ret =a6xx_gmu_memory_probe (adreno_gpu->base.dev, gmu);if (ret) goto err_put_device;drivers/gpu/drm/msm/adreno/a6xx_gpu.c +2
-3@@ -2271,9 +2271,8 @@ a6xx_create_private_vm(struct msm_gpu *gpu)if (IS_ERR (mmu))return ERR_CAST (mmu); -return msm_gem_vm_create (mmu, -"gpu" ,0x100000000ULL , -adreno_private_vm_size (gpu)); +return msm_gem_vm_create (gpu->dev, mmu,"gpu" ,0x100000000ULL , +adreno_private_vm_size (gpu), true); }static uint32_t a6xx_get_rptr (struct msm_gpu *gpu,struct msm_ringbuffer *ring)drivers/gpu/drm/msm/adreno/adreno_gpu.c +4
-3@@ -224,7 +224,8 @@ adreno_iommu_create_vm(struct msm_gpu *gpu, start =max_t (u64,SZ_16M , geometry->aperture_start); size = geometry->aperture_end - start +1 ; - vm =msm_gem_vm_create (mmu,"gpu" , start &GENMASK_ULL (48 ,0 ), size); + vm =msm_gem_vm_create (gpu->dev, mmu,"gpu" , start &GENMASK_ULL (48 ,0 ), + size, true);if (IS_ERR (vm) && !IS_ERR (mmu)) mmu->funcs->destroy (mmu);@@ -403,12 +404,12 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,case MSM_PARAM_VA_START :if (ctx->vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->va_start; + *value = ctx->vm->base.mm_start;return 0 ;case MSM_PARAM_VA_SIZE :if (ctx->vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->va_size; + *value = ctx->vm->base.mm_range;return 0 ;case MSM_PARAM_HIGHEST_BANK_BIT : *value = adreno_gpu->ubwc_config.highest_bank_bit;drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c +3
-2@@ -451,8 +451,9 @@ static int mdp4_kms_init(struct drm_device *dev) "contig buffers for scanout\n"); vm =NULL ; }else { - vm =msm_gem_vm_create (mmu, -"mdp4" ,0x1000 ,0x100000000 -0x1000 ); + vm =msm_gem_vm_create (dev, mmu,"mdp4" , +0x1000 ,0x100000000 -0x1000 , + true);if (IS_ERR (vm)) {if (!IS_ERR (mmu))drivers/gpu/drm/msm/msm_drv.c +1
-0@@ -776,6 +776,7 @@ static const struct file_operations fops = {static const struct drm_driver msm_driver = { .driver_features =DRIVER_GEM | +DRIVER_GEM_GPUVA |DRIVER_RENDER |DRIVER_ATOMIC | DRIVER_MODESET |drivers/gpu/drm/msm/msm_gem.c +84
-58@@ -47,9 +47,32 @@ static int msm_gem_open(struct drm_gem_object *obj, struct drm_file *file)return 0 ; } +static void put_iova_spaces (struct drm_gem_object *obj,struct drm_gpuvm *vm,bool close); +static void msm_gem_close (struct drm_gem_object *obj,struct drm_file *file) { +struct msm_context *ctx = file->driver_priv; +update_ctx_mem (file, -obj->size); + +/* +* If VM isn't created yet, nothing to cleanup. And in fact calling +* put_iova_spaces() with vm=NULL would be bad, in that it will tear- +* down the mappings of shared buffers in other contexts. +*/ +if (!ctx->vm) +return ; + +/* +* TODO we might need to kick this to a queue to avoid blocking +* in CLOSE ioctl +*/ +dma_resv_wait_timeout (obj->resv,DMA_RESV_USAGE_READ , false, +msecs_to_jiffies (1000 )); + +msm_gem_lock (obj); +put_iova_spaces (obj, &ctx->vm->base, true); +msm_gem_unlock (obj); }/* @@ -171,6 +194,13 @@ static void put_pages(struct drm_gem_object *obj) {struct msm_gem_object *msm_obj =to_msm_bo (obj); +/* +* Skip gpuvm in the object free path to avoid a WARN_ON() splat. +* See explaination in msm_gem_assert_locked() +*/ +if (kref_read (&obj->refcount)) +drm_gpuvm_bo_gem_evict (obj, true); +if (msm_obj->pages) {if (msm_obj->sgt) {/* For non-cached buffers, ensure the new @@ -338,16 +368,25 @@ uint64_t msm_gem_mmap_offset(struct drm_gem_object *obj) }static struct msm_gem_vma *lookup_vma (struct drm_gem_object *obj, -struct msm_gem_vm *vm) +struct msm_gem_vm *vm) { -struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_gem_vma *vma; +struct drm_gpuvm_bo *vm_bo;msm_gem_assert_locked (obj); -list_for_each_entry (vma, &msm_obj->vmas, list) { -if (vma->vm == vm) -return vma; + drm_gem_for_each_gpuvm_bo (vm_bo, obj) { +struct drm_gpuva *vma; + + drm_gpuvm_bo_for_each_va (vma, vm_bo) { +if (vma->vm == &vm->base) { +/* lookup_vma() should only be used in paths +* with at most one vma per vm +*/ +GEM_WARN_ON (!list_is_singular (&vm_bo->list.gpuva)); + +return to_msm_vma (vma); + } + } }return NULL ;@@ -360,33 +399,29 @@ static struct msm_gem_vma *lookup_vma(struct drm_gem_object *obj,* mapping. */ static void -put_iova_spaces (struct drm_gem_object *obj,bool close) +put_iova_spaces (struct drm_gem_object *obj,struct drm_gpuvm *vm,bool close) { -struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_gem_vma *vma, *tmp; +struct drm_gpuvm_bo *vm_bo, *tmp;msm_gem_assert_locked (obj); -list_for_each_entry_safe (vma, tmp, &msm_obj->vmas, list) { -if (vma->vm) { -msm_gem_vma_purge (vma); -if (close) -msm_gem_vma_close (vma); - } - } -} + drm_gem_for_each_gpuvm_bo_safe (vm_bo, tmp, obj) { +struct drm_gpuva *vma, *vmatmp; -/* Called with msm_obj locked */ -static void -put_iova_vmas (struct drm_gem_object *obj) -{ -struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct msm_gem_vma *vma, *tmp; +if (vm && vm_bo->vm != vm) +continue ; -msm_gem_assert_locked (obj); +drm_gpuvm_bo_get (vm_bo); -list_for_each_entry_safe (vma, tmp, &msm_obj->vmas, list) { -msm_gem_vma_close (vma); + drm_gpuvm_bo_for_each_va_safe (vma, vmatmp, vm_bo) { +struct msm_gem_vma *msm_vma =to_msm_vma (vma); + +msm_gem_vma_purge (msm_vma); +if (close) +msm_gem_vma_close (msm_vma); + } + +drm_gpuvm_bo_put (vm_bo); } }@@ -394,7 +429,6 @@ static struct msm_gem_vma *get_vma_locked(struct drm_gem_object *obj,struct msm_gem_vm *vm, u64 range_start, u64 range_end) { - struct msm_gem_object *msm_obj = to_msm_bo(obj); struct msm_gem_vma *vma;msm_gem_assert_locked (obj);@@ -403,12 +437,9 @@ static struct msm_gem_vma *get_vma_locked(struct drm_gem_object *obj,if (!vma) { vma =msm_gem_vma_new (vm, obj, range_start, range_end); -if (IS_ERR (vma)) -return vma; -list_add_tail (&vma->list, &msm_obj->vmas); }else { -GEM_WARN_ON (vma->iova < range_start); -GEM_WARN_ON ((vma->iova + obj->size) > range_end); +GEM_WARN_ON (vma->base.va.addr < range_start); +GEM_WARN_ON ((vma->base.va.addr + obj->size) > range_end); }return vma;@@ -492,7 +523,7 @@ static int get_and_pin_iova_range_locked(struct drm_gem_object *obj, ret =msm_gem_pin_vma_locked (obj, vma);if (!ret) { - *iova = vma->iova; + *iova = vma->base.va.addr;pin_obj_locked (obj); }@@ -538,7 +569,7 @@ int msm_gem_get_iova(struct drm_gem_object *obj,if (IS_ERR (vma)) { ret =PTR_ERR (vma); }else { - *iova = vma->iova; + *iova = vma->base.va.addr; }msm_gem_unlock (obj);@@ -579,7 +610,7 @@ int msm_gem_set_iova(struct drm_gem_object *obj, vma =get_vma_locked (obj, vm, iova, iova + obj->size);if (IS_ERR (vma)) { ret =PTR_ERR (vma); - }else if (GEM_WARN_ON (vma->iova != iova)) { + }else if (GEM_WARN_ON (vma->base.va.addr != iova)) {clear_iova (obj, vm); ret = -EBUSY ; }@@ -763,7 +794,7 @@ void msm_gem_purge(struct drm_gem_object *obj)GEM_WARN_ON (!is_purgeable (msm_obj));/* Get rid of any iommu mapping(s): */ -put_iova_spaces (obj, true); +put_iova_spaces (obj,NULL , true);msm_gem_vunmap (obj);@@ -771,8 +802,6 @@ void msm_gem_purge(struct drm_gem_object *obj)put_pages (obj); -put_iova_vmas (obj); -mutex_lock (&priv->lru.lock);/* A one-way transition: */ msm_obj->madv = __MSM_MADV_PURGED;@@ -803,7 +832,7 @@ void msm_gem_evict(struct drm_gem_object *obj)GEM_WARN_ON (is_unevictable (msm_obj));/* Get rid of any iommu mapping(s): */ -put_iova_spaces (obj, false); +put_iova_spaces (obj,NULL , false);drm_vma_node_unmap (&obj->vma_node, dev->anon_inode->i_mapping);@@ -869,7 +898,6 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m, {struct msm_gem_object *msm_obj =to_msm_bo (obj);struct dma_resv *robj = obj->resv; -struct msm_gem_vma *vma;uint64_t off =drm_vma_node_start (&obj->vma_node);const char *madv;@@ -912,14 +940,17 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m,seq_printf (m," %08zu %9s %-32s\n" , obj->size, madv, msm_obj->name); -if (!list_empty (&msm_obj->vmas)) { +if (!list_empty (&obj->gpuva.list)) { +struct drm_gpuvm_bo *vm_bo;seq_puts (m," vmas:" ); -list_for_each_entry (vma, &msm_obj->vmas, list) { -const char *name, *comm; -if (vma->vm) { -struct msm_gem_vm *vm = vma->vm; + drm_gem_for_each_gpuvm_bo (vm_bo, obj) { +struct drm_gpuva *vma; + + drm_gpuvm_bo_for_each_va (vma, vm_bo) { +const char *name, *comm; +struct msm_gem_vm *vm =to_msm_vm (vma->vm);struct task_struct *task =get_pid_task (vm->pid,PIDTYPE_PID );if (task) {@@ -928,15 +959,14 @@ void msm_gem_describe(struct drm_gem_object *obj, struct seq_file *m, }else { comm =NULL ; } - name = vm->name; - }else { - name = comm =NULL ; + name = vm->base.name; + +seq_printf (m," [%s%s%s: vm=%p, %08llx,%smapped]" , + name, comm ?":" :"" , comm ? comm :"" , + vma->vm, vma->va.addr, +to_msm_vma (vma)->mapped ?"" :"un" ); +kfree (comm); } -seq_printf (m," [%s%s%s: vm=%p, %08llx,%s]" , - name, comm ?":" :"" , comm ? comm :"" , - vma->vm, vma->iova, - vma->mapped ?"mapped" :"unmapped" ); -kfree (comm); }seq_puts (m,"\n" );@@ -982,7 +1012,7 @@ static void msm_gem_free_object(struct drm_gem_object *obj)list_del (&msm_obj->node);mutex_unlock (&priv->obj_lock); -put_iova_spaces (obj, true); +put_iova_spaces (obj,NULL , true);if (obj->import_attach) {GEM_WARN_ON (msm_obj->vaddr);@@ -992,13 +1022,10 @@ static void msm_gem_free_object(struct drm_gem_object *obj)*/ kvfree (msm_obj->pages); -put_iova_vmas (obj); -drm_prime_gem_destroy (obj, msm_obj->sgt); }else {msm_gem_vunmap (obj);put_pages (obj); -put_iova_vmas (obj); }drm_gem_object_release (obj);@@ -1104,7 +1131,6 @@ static int msm_gem_new_impl(struct drm_device *dev, msm_obj->madv =MSM_MADV_WILLNEED ;INIT_LIST_HEAD (&msm_obj->node); -INIT_LIST_HEAD (&msm_obj->vmas); *obj = &msm_obj->base; (*obj)->funcs = &msm_gem_object_funcs;drivers/gpu/drm/msm/msm_gem.h +68
-19@@ -10,6 +10,7 @@#include <linux/kref.h> #include <linux/dma-resv.h> #include "drm/drm_exec.h" +#include "drm/drm_gpuvm.h" #include "drm/gpu_scheduler.h" #include "msm_drv.h" @@ -22,30 +23,67 @@#define MSM_BO_STOLEN 0x10000000/* try to use stolen/splash memory */ #define MSM_BO_MAP_PRIV 0x20000000/* use IOMMU_PRIV when mapping */ +/** +* struct msm_gem_vm - VM object +* +* A VM object representing a GPU (or display or GMU or ...) virtual address +* space. +* +* In the case of GPU, if per-process address spaces are supported, the address +* space is split into two VMs, which map to TTBR0 and TTBR1 in the SMMU. TTBR0 +* is used for userspace objects, and is unique per msm_context/drm_file, while +* TTBR1 is the same for all processes. (The kernel controlled ringbuffer and +* a few other kernel controlled buffers live in TTBR1.) +* +* The GPU TTBR0 vm can be managed by userspace or by the kernel, depending on +* whether userspace supports VM_BIND. All other vm's are managed by the kernel. +* (Managed by kernel means the kernel is responsible for VA allocation.) +* +* Note that because VM_BIND allows a given BO to be mapped multiple times in +* a VM, and therefore have multiple VMA's in a VM, there is an extra object +* provided by drm_gpuvm infrastructure.. the drm_gpuvm_bo, which is not +* embedded in any larger driver structure. The GEM object holds a list of +* drm_gpuvm_bo, which in turn holds a list of msm_gem_vma. A linked vma +* holds a reference to the vm_bo, and drops it when the vma is unlinked. +* So we just need to call drm_gpuvm_bo_obtain() to return a ref to an +* existing vm_bo, or create a new one. Once the vma is linked, the ref +* to the vm_bo can be dropped (since the vma is holding one). +*/ struct msm_gem_vm { -const char *name; -/* NOTE: mm managed at the page level, size is in # of pages -* and position mm_node->start is in # of pages: +/** @base: Inherit from drm_gpuvm. */ +struct drm_gpuvm base; + +/** +* @mm: Memory management for kernel managed VA allocations +* +* Only used for kernel managed VMs, unused for user managed VMs. +* +* Protected by @mm_lock. */ struct drm_mm mm; -spinlock_t lock;/* Protects drm_mm node allocation/removal */ + +/** @mm_lock: protects @mm node allocation/removal */ +struct spinlock mm_lock; + +/** @vm_lock: protects gpuvm insert/remove/traverse */ +struct mutex vm_lock; + +/** @mmu: The mmu object which manages the pgtables */ struct msm_mmu *mmu; -struct kref kref; -/* For address spaces associated with a specific process, this +/** +* @pid: For address spaces associated with a specific process, this * will be non-NULL: */ struct pid *pid; -/* @faults: the number of GPU hangs associated with this address space */ +/** @faults: the number of GPU hangs associated with this address space */ int faults; -/** @va_start: lowest possible address to allocate */ -uint64_t va_start; - -/** @va_size: the size of the address space (in bytes) */ -uint64_t va_size; +/** @managed: is this a kernel managed VM? */ +bool managed; }; +#define to_msm_vm (x) container_of(x, struct msm_gem_vm, base)struct msm_gem_vm *msm_gem_vm_get (struct msm_gem_vm *vm);@@ -53,18 +91,31 @@ msm_gem_vm_get(struct msm_gem_vm *vm);void msm_gem_vm_put (struct msm_gem_vm *vm);struct msm_gem_vm * -msm_gem_vm_create (struct msm_mmu *mmu,const char *name, -u64 va_start,u64 size); +msm_gem_vm_create (struct drm_device *drm,struct msm_mmu *mmu,const char *name, +u64 va_start,u64 va_size,bool managed);struct msm_fence_context ; +/** +* struct msm_gem_vma - a VMA mapping +* +* Represents a combination of a GEM object plus a VM. +*/ struct msm_gem_vma { +/** @base: inherit from drm_gpuva */ +struct drm_gpuva base; + +/** +* @node: mm node for VA allocation +* +* Only used by kernel managed VMs +*/ struct drm_mm_node node; -uint64_t iova; -struct msm_gem_vm *vm; -struct list_head list;/* node in msm_gem_object::vmas */ + +/** @mapped: Is this VMA mapped? */ bool mapped; }; +#define to_msm_vma (x) container_of(x, struct msm_gem_vma, base)struct msm_gem_vma * msm_gem_vma_new(struct msm_gem_vm *vm, struct drm_gem_object *obj,@@ -100,8 +151,6 @@ struct msm_gem_object {struct sg_table *sgt;void *vaddr; -struct list_head vmas;/* list of msm_gem_vma */ -char name[32 ];/* Identifier to print for the debugfs files */ /* userspace metadata backchannel */ drivers/gpu/drm/msm/msm_gem_submit.c +1
-1@@ -312,7 +312,7 @@ static int submit_pin_objects(struct msm_gem_submit *submit)if (ret)break ; - submit->bos[i].iova = vma->iova; + submit->bos[i].iova = vma->base.va.addr; }/* drivers/gpu/drm/msm/msm_gem_vma.c +56
-32@@ -5,14 +5,13 @@*/ #include "msm_drv.h" -#include "msm_fence.h" #include "msm_gem.h" #include "msm_mmu.h" static void -msm_gem_vm_destroy (struct kref *kref) +msm_gem_vm_free (struct drm_gpuvm *gpuvm) { - struct msm_gem_vm *vm = container_of(kref, struct msm_gem_vm, kref); + struct msm_gem_vm *vm = container_of(gpuvm, struct msm_gem_vm, base);drm_mm_takedown (&vm->mm);if (vm->mmu)@@ -25,14 +24,14 @@ msm_gem_vm_destroy(struct kref *kref)void msm_gem_vm_put (struct msm_gem_vm *vm) {if (vm) -kref_put (&vm->kref, msm_gem_vm_destroy); +drm_gpuvm_put (&vm->base); }struct msm_gem_vm *msm_gem_vm_get (struct msm_gem_vm *vm) {if (!IS_ERR_OR_NULL (vm)) -kref_get (&vm->kref); +drm_gpuvm_get (&vm->base);return vm; }@@ -40,14 +39,14 @@ msm_gem_vm_get(struct msm_gem_vm *vm)/* Actually unmap memory for the vma */ void msm_gem_vma_purge (struct msm_gem_vma *vma) { -struct msm_gem_vm *vm = vma->vm; -unsigned size = vma->node.size; +struct msm_gem_vm *vm =to_msm_vm (vma->base.vm); +unsigned size = vma->base.va.range;/* Don't do anything if the memory isn't mapped */ if (!vma->mapped)return ; - vm->mmu->funcs->unmap (vm->mmu, vma->iova, size); + vm->mmu->funcs->unmap (vm->mmu, vma->base.va.addr, size); vma->mapped = false; }@@ -57,10 +56,10 @@ int msm_gem_vma_map(struct msm_gem_vma *vma, int prot, struct sg_table *sgt, int size) { - struct msm_gem_vm *vm = vma->vm; + struct msm_gem_vm *vm = to_msm_vm(vma->base.vm);int ret; -if (GEM_WARN_ON (!vma->iova)) +if (GEM_WARN_ON (!vma->base.va.addr))return -EINVAL ;if (vma->mapped)@@ -68,9 +67,6 @@ msm_gem_vma_map(struct msm_gem_vma *vma, int prot, vma->mapped = true; -if (!vm) -return 0 ; -/* * NOTE: iommu/io-pgtable can allocate pages, so we cannot hold * a lock across map/unmap which is also used in the job_run() @@ -80,7 +76,7 @@ msm_gem_vma_map(struct msm_gem_vma *vma, int prot,* Revisit this if we can come up with a scheme to pre-alloc pages * for the pgtable in map/unmap ops. */ - ret = vm->mmu->funcs->map (vm->mmu, vma->iova, sgt, size, prot); + ret = vm->mmu->funcs->map (vm->mmu, vma->base.va.addr, sgt, size, prot);if (ret) { vma->mapped = false;@@ -92,19 +88,20 @@ msm_gem_vma_map(struct msm_gem_vma *vma, int prot,/* Close an iova. Warn if it is still in use */ void msm_gem_vma_close (struct msm_gem_vma *vma) { -struct msm_gem_vm *vm = vma->vm; +struct msm_gem_vm *vm =to_msm_vm (vma->base.vm);GEM_WARN_ON (vma->mapped); -spin_lock (&vm->lock); -if (vma->iova) +spin_lock (&vm->mm_lock); +if (vma->base.va.addr)drm_mm_remove_node (&vma->node); -spin_unlock (&vm->lock); +spin_unlock (&vm->mm_lock); - vma->iova =0 ; -list_del (&vma->list); +mutex_lock (&vm->vm_lock); +drm_gpuva_remove (&vma->base); +drm_gpuva_unlink (&vma->base); +mutex_unlock (&vm->vm_lock); -msm_gem_vm_put (vm);kfree (vma); }@@ -113,6 +110,7 @@ struct msm_gem_vma * msm_gem_vma_new(struct msm_gem_vm *vm, struct drm_gem_object *obj, u64 range_start, u64 range_end) { + struct drm_gpuvm_bo *vm_bo;struct msm_gem_vma *vma;int ret;@@ -120,36 +118,82 @@ msm_gem_vma_new(struct msm_gem_vm *vm, struct drm_gem_object *obj,if (!vma)return ERR_PTR (-ENOMEM ); - vma->vm = vm; +if (vm->managed) { +spin_lock (&vm->mm_lock); + ret =drm_mm_insert_node_in_range (&vm->mm, &vma->node, + obj->size,PAGE_SIZE ,0 , + range_start, range_end,0 ); +spin_unlock (&vm->mm_lock); -spin_lock (&vm->lock); - ret =drm_mm_insert_node_in_range (&vm->mm, &vma->node, - obj->size,PAGE_SIZE ,0 , - range_start, range_end,0 ); -spin_unlock (&vm->lock); +if (ret) + goto err_free_vma; -if (ret) - goto err_free_vma; + range_start = vma->node.start; + range_end = range_start + obj->size; + } - vma->iova = vma->node.start; +GEM_WARN_ON ((range_end - range_start) > obj->size); + +drm_gpuva_init (&vma->base, range_start, range_end - range_start, obj,0 ); vma->mapped = false; -INIT_LIST_HEAD (&vma->list); +mutex_lock (&vm->vm_lock); + ret =drm_gpuva_insert (&vm->base, &vma->base); +mutex_unlock (&vm->vm_lock); +if (ret) + goto err_free_range; -kref_get (&vm->kref); + vm_bo =drm_gpuvm_bo_obtain (&vm->base, obj); +if (IS_ERR (vm_bo)) { + ret =PTR_ERR (vm_bo); + goto err_va_remove; + } + +mutex_lock (&vm->vm_lock); +drm_gpuva_link (&vma->base, vm_bo); +mutex_unlock (&vm->vm_lock); +GEM_WARN_ON (drm_gpuvm_bo_put (vm_bo));return vma; +err_va_remove: +mutex_lock (&vm->vm_lock); +drm_gpuva_remove (&vma->base); +mutex_unlock (&vm->vm_lock); +err_free_range: +if (vm->managed) +drm_mm_remove_node (&vma->node); err_free_vma:kfree (vma);return ERR_PTR (ret); } +static const struct drm_gpuvm_ops msm_gpuvm_ops = {
drivers/gpu/drm/msm/msm_gem_vma.c msm_gem_vma_new(struct msm_gem_vm *vm, struct drm_gem_object *obj, · line 181
Quoted 10 lines
drivers/gpu/drm/msm/msm_gem_vma.c +10
-0+ .vm_free = msm_gem_vm_free, +}; + +/** +* msm_gem_vm_create() - Create and initialize a &msm_gem_vm +* @drm: the drm device +* @mmu: the backing MMU objects handling mapping/unmapping +* @name: the name of the VM +* @va_start: the start offset of the GPU VA space +* @va_size: the size of the GPU VA space
This function is used on the KMS side too. So "GPU/KMS"?
-Akhil
Quoted 71 lines
drivers/gpu/drm/msm/msm_gem_vma.c +32
-9+* @managed: is it a kernel managed VM? +* +* In a kernel managed VM, the kernel handles address allocation, and only +* synchronous operations are supported. In a user managed VM, userspace +* handles virtual address allocation, and both async and sync operations +* are supported. +*/ struct msm_gem_vm * -msm_gem_vm_create (struct msm_mmu *mmu,const char *name, -u64 va_start,u64 size) +msm_gem_vm_create (struct drm_device *drm,struct msm_mmu *mmu,const char *name, +u64 va_start,u64 va_size,bool managed) { +enum drm_gpuvm_flags flags = managed ?DRM_GPUVM_VA_WEAK_REF :0 ;struct msm_gem_vm *vm; +struct drm_gem_object *dummy_gem; +int ret =0 ;if (IS_ERR (mmu))return ERR_CAST (mmu);@@ -158,15 +202,28 @@ msm_gem_vm_create(struct msm_mmu *mmu, const char *name,if (!vm)return ERR_PTR (-ENOMEM ); -spin_lock_init (&vm->lock); - vm->name = name; - vm->mmu = mmu; - vm->va_start = va_start; - vm->va_size = size; + dummy_gem =drm_gpuvm_resv_object_alloc (drm); +if (!dummy_gem) { + ret = -ENOMEM ; + goto err_free_vm; + } + +drm_gpuvm_init (&vm->base, name, flags, drm, dummy_gem, + va_start, va_size,0 ,0 , &msm_gpuvm_ops); +drm_gem_object_put (dummy_gem); + +spin_lock_init (&vm->mm_lock); +mutex_init (&vm->vm_lock); -drm_mm_init (&vm->mm, va_start, size); + vm->mmu = mmu; + vm->managed = managed; -kref_init (&vm->kref); +drm_mm_init (&vm->mm, va_start, va_size);return vm; + +err_free_vm: +kfree (vm); +return ERR_PTR (ret); + }drivers/gpu/drm/msm/msm_kms.c +2
-2@@ -204,8 +204,8 @@ struct msm_gem_vm *msm_kms_init_vm(struct drm_device *dev)return NULL ; } - vm =msm_gem_vm_create (mmu,"mdp_kms" , -0x1000 ,0x100000000 -0x1000 ); + vm =msm_gem_vm_create (dev, mmu,"mdp_kms" , +0x1000 ,0x100000000 -0x1000 , true);if (IS_ERR (vm)) {dev_err (mdp_dev,"vm create, error %pe\n" , vm); mmu->funcs->destroy (mmu);
PATCH v2 12/34 drm/msm: Use drm_gpuvm types more
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Most of the driver code doesn't need to reach in to msm specific fields, so just use the drm_gpuvm/drm_gpuva types directly. This should hopefully improve commonality with other drivers and make the code easier to understand.
--- drivers/gpu/drm/msm/adreno/a2xx_gpu.c | 6 +- drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 6 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 6 +- drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 2 +- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 14 ++-- drivers/gpu/drm/msm/adreno/a6xx_preempt.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 21 +++-- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 4 +- .../drm/msm/disp/dpu1/dpu_encoder_phys_wb.c | 4 +- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c | 6 +- drivers/gpu/drm/msm/disp/dpu1/dpu_formats.h | 2 +- drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c | 6 +- drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h | 2 +- drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c | 11 +-- drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c | 11 +-- drivers/gpu/drm/msm/dsi/dsi_host.c | 6 +- drivers/gpu/drm/msm/msm_drv.h | 19 ++--- drivers/gpu/drm/msm/msm_fb.c | 14 ++-- drivers/gpu/drm/msm/msm_gem.c | 82 +++++++++---------- drivers/gpu/drm/msm/msm_gem.h | 53 ++++++------ drivers/gpu/drm/msm/msm_gem_submit.c | 4 +- drivers/gpu/drm/msm/msm_gem_vma.c | 72 +++++++--------- drivers/gpu/drm/msm/msm_gpu.c | 21 +++-- drivers/gpu/drm/msm/msm_gpu.h | 10 +-- drivers/gpu/drm/msm/msm_kms.c | 6 +- drivers/gpu/drm/msm/msm_kms.h | 2 +- drivers/gpu/drm/msm/msm_submitqueue.c | 2 +- 27 files changed, 192 insertions(+), 202 deletions(-)
drivers/gpu/drm/msm/adreno/a2xx_gpu.c +3 -3
@@ -113,7 +113,7 @@ static int a2xx_hw_init(struct msm_gpu *gpu)uint32_t *ptr, len;int i, ret; -a2xx_gpummu_params (gpu->vm->mmu, &pt_base, &tran_error); +a2xx_gpummu_params (to_msm_vm (gpu->vm)->mmu, &pt_base, &tran_error);DBG ("%s" , gpu->name);
@@ -466,11 +466,11 @@ static struct msm_gpu_state *a2xx_gpu_state_get(struct msm_gpu *gpu)return state; } -static struct msm_gem_vm * +static struct drm_gpuvm *a2xx_create_vm (struct msm_gpu *gpu,struct platform_device *pdev) {struct msm_mmu *mmu =a2xx_gpummu_new (&pdev->dev, gpu); -struct msm_gem_vm *vm; +struct drm_gpuvm *vm; vm =msm_gem_vm_create (gpu->dev, mmu,"gpu" ,SZ_16M ,0xfff *SZ_64K , true);
drivers/gpu/drm/msm/adreno/a5xx_gpu.c +4 -2
@@ -1786,8 +1786,10 @@ struct msm_gpu *a5xx_gpu_init(struct drm_device *dev)return ERR_PTR (ret); } -if (gpu->vm) -msm_mmu_set_fault_handler (gpu->vm->mmu, gpu, a5xx_fault_handler); +if (gpu->vm) { +msm_mmu_set_fault_handler (to_msm_vm (gpu->vm)->mmu, gpu, + a5xx_fault_handler); + }/* Set up the preemption specific bits and pieces for each ringbuffer */ a5xx_preempt_init (gpu);
drivers/gpu/drm/msm/adreno/a6xx_gmu.c +4 -2
@@ -1259,6 +1259,8 @@ int a6xx_gmu_stop(struct a6xx_gpu *a6xx_gpu)static void a6xx_gmu_memory_free (struct a6xx_gmu *gmu) { +struct msm_mmu *mmu =to_msm_vm (gmu->vm)->mmu; +msm_gem_kernel_put (gmu->hfi.obj, gmu->vm);msm_gem_kernel_put (gmu->debug.obj, gmu->vm);msm_gem_kernel_put (gmu->icache.obj, gmu->vm);
@@ -1266,8 +1268,8 @@ static void a6xx_gmu_memory_free(struct a6xx_gmu *gmu)msm_gem_kernel_put (gmu->dummy.obj, gmu->vm);msm_gem_kernel_put (gmu->log.obj, gmu->vm); - gmu->vm->mmu->funcs->detach (gmu->vm->mmu); -msm_gem_vm_put (gmu->vm); + mmu->funcs->detach (mmu); +drm_gpuvm_put (gmu->vm); } static int a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo,
drivers/gpu/drm/msm/adreno/a6xx_gmu.h +1 -1
@@ -62,7 +62,7 @@ struct a6xx_gmu {/* For serializing communication with the GMU: */ struct mutex lock; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;void __iomem *mmio;void __iomem *rscc;
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +8 -6
@@ -120,7 +120,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,if (ctx->seqno == ring->cur_ctx_seqno)return ; -if (msm_iommu_pagetable_params (ctx->vm->mmu, &ttbr, &asid)) +if (msm_iommu_pagetable_params (to_msm_vm (ctx->vm)->mmu, &ttbr, &asid))return ;if (adreno_gpu->info->family >=ADRENO_7XX_GEN1 ) {
@@ -2243,7 +2243,7 @@ static void a6xx_gpu_set_freq(struct msm_gpu *gpu, struct dev_pm_opp *opp,mutex_unlock (&a6xx_gpu->gmu.lock); } -static struct msm_gem_vm * +static struct drm_gpuvm *a6xx_create_vm (struct msm_gpu *gpu,struct platform_device *pdev) {struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);
@@ -2261,12 +2261,12 @@ a6xx_create_vm(struct msm_gpu *gpu, struct platform_device *pdev)return adreno_iommu_create_vm (gpu, pdev, quirks); } -static struct msm_gem_vm * +static struct drm_gpuvm *a6xx_create_private_vm (struct msm_gpu *gpu) {struct msm_mmu *mmu; - mmu =msm_iommu_pagetable_create (gpu->vm->mmu); + mmu =msm_iommu_pagetable_create (to_msm_vm (gpu->vm)->mmu);if (IS_ERR (mmu))return ERR_CAST (mmu);
@@ -2546,8 +2546,10 @@ struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) adreno_gpu->uche_trap_base =0x1fffffffff000ull ; -if (gpu->vm) -msm_mmu_set_fault_handler (gpu->vm->mmu, gpu, a6xx_fault_handler); +if (gpu->vm) { +msm_mmu_set_fault_handler (to_msm_vm (gpu->vm)->mmu, gpu, + a6xx_fault_handler); + }a6xx_calc_ubwc_config (adreno_gpu);/* Set up the preemption specific bits and pieces for each ringbuffer */
drivers/gpu/drm/msm/adreno/a6xx_preempt.c +1 -1
@@ -376,7 +376,7 @@ static int preempt_init_ring(struct a6xx_gpu *a6xx_gpu,struct a7xx_cp_smmu_info *smmu_info_ptr = ptr; -msm_iommu_pagetable_params (gpu->vm->mmu, &ttbr, &asid); +msm_iommu_pagetable_params (to_msm_vm (gpu->vm)->mmu, &ttbr, &asid); smmu_info_ptr->magic =GEN7_CP_SMMU_INFO_MAGIC ; smmu_info_ptr->ttbr0 = ttbr;
drivers/gpu/drm/msm/adreno/adreno_gpu.c +12 -9
@@ -191,21 +191,21 @@ int adreno_zap_shader_load(struct msm_gpu *gpu, u32 pasid)return zap_shader_load_mdt (gpu, adreno_gpu->info->zapfw, pasid); } -struct msm_gem_vm * +struct drm_gpuvm *adreno_create_vm (struct msm_gpu *gpu,struct platform_device *pdev) {return adreno_iommu_create_vm (gpu, pdev,0 ); } -struct msm_gem_vm * +struct drm_gpuvm *adreno_iommu_create_vm (struct msm_gpu *gpu,struct platform_device *pdev,unsigned long quirks) {struct iommu_domain_geometry *geometry;struct msm_mmu *mmu; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;u64 start, size; mmu =msm_iommu_gpu_new (&pdev->dev, gpu, quirks);
@@ -259,9 +259,10 @@ void adreno_check_and_reenable_stall(struct adreno_gpu *adreno_gpu)if (!adreno_gpu->stall_enabled &&ktime_after (ktime_get (), adreno_gpu->stall_reenable_time) && !READ_ONCE (gpu->crashstate)) { +struct msm_mmu *mmu =to_msm_vm (gpu->vm)->mmu; adreno_gpu->stall_enabled = true; - gpu->vm->mmu->funcs->set_stall (gpu->vm->mmu, true); + mmu->funcs->set_stall (mmu, true); }spin_unlock_irqrestore (&adreno_gpu->fault_stall_lock, flags); }
@@ -275,6 +276,7 @@ int adreno_fault_handler(struct msm_gpu *gpu, unsigned long iova, int flags, struct adreno_smmu_fault_info *info, const char *block, u32 scratch[4]) { + struct msm_mmu *mmu = to_msm_vm(gpu->vm)->mmu;struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);const char *type ="UNKNOWN" ; bool do_devcoredump = info && (info->fsr & ARM_SMMU_FSR_SS) &&
@@ -287,9 +289,10 @@ int adreno_fault_handler(struct msm_gpu *gpu, unsigned long iova, int flags,*/ spin_lock_irqsave (&adreno_gpu->fault_stall_lock, irq_flags);if (adreno_gpu->stall_enabled) { + adreno_gpu->stall_enabled = false; - gpu->vm->mmu->funcs->set_stall (gpu->vm->mmu, false); + mmu->funcs->set_stall (mmu, false); } adreno_gpu->stall_reenable_time =ktime_add_ms (ktime_get (),500 );spin_unlock_irqrestore (&adreno_gpu->fault_stall_lock, irq_flags);
@@ -299,7 +302,7 @@ int adreno_fault_handler(struct msm_gpu *gpu, unsigned long iova, int flags,* it now. */ if (!do_devcoredump) { - gpu->vm->mmu->funcs->resume_translation (gpu->vm->mmu); + mmu->funcs->resume_translation (mmu); }/*
@@ -394,7 +397,7 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,return 0 ;case MSM_PARAM_FAULTS :if (ctx->vm) - *value = gpu->global_faults + ctx->vm->faults; + *value = gpu->global_faults +to_msm_vm (ctx->vm)->faults;else *value = gpu->global_faults;return 0 ;
@@ -404,12 +407,12 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,case MSM_PARAM_VA_START :if (ctx->vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->base.mm_start; + *value = ctx->vm->mm_start;return 0 ;case MSM_PARAM_VA_SIZE :if (ctx->vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->base.mm_range; + *value = ctx->vm->mm_range;return 0 ;case MSM_PARAM_HIGHEST_BANK_BIT : *value = adreno_gpu->ubwc_config.highest_bank_bit;
drivers/gpu/drm/msm/adreno/adreno_gpu.h +2 -2
@@ -642,11 +642,11 @@ void adreno_show_object(struct drm_printer *p, void **ptr, int len,* Common helper function to initialize the default address space for arm-smmu * attached targets */ -struct msm_gem_vm * +struct drm_gpuvm *adreno_create_vm (struct msm_gpu *gpu,struct platform_device *pdev); -struct msm_gem_vm * +struct drm_gpuvm *adreno_iommu_create_vm (struct msm_gpu *gpu,struct platform_device *pdev,unsigned long quirks);
drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_wb.c +2 -2
@@ -566,7 +566,7 @@ static void dpu_encoder_phys_wb_prepare_wb_job(struct dpu_encoder_phys *phys_enc struct drm_writeback_job *job) {const struct msm_format *format; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;struct dpu_hw_wb_cfg *wb_cfg;int ret;struct dpu_encoder_phys_wb *wb_enc =to_dpu_encoder_phys_wb (phys_enc);
@@ -619,7 +619,7 @@ static void dpu_encoder_phys_wb_cleanup_wb_job(struct dpu_encoder_phys *phys_enc struct drm_writeback_job *job) {struct dpu_encoder_phys_wb *wb_enc =to_dpu_encoder_phys_wb (phys_enc); -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;if (!job->fb)return ;
drivers/gpu/drm/msm/disp/dpu1/dpu_formats.c +3 -3
@@ -274,7 +274,7 @@ int dpu_format_populate_plane_sizes(return _dpu_format_populate_plane_sizes_linear (fmt, fb, layout); } -static void _dpu_format_populate_addrs_ubwc (struct msm_gem_vm *vm, +static void _dpu_format_populate_addrs_ubwc (struct drm_gpuvm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout) {
@@ -355,7 +355,7 @@ static void _dpu_format_populate_addrs_ubwc(struct msm_gem_vm *vm, } } -static void _dpu_format_populate_addrs_linear (struct msm_gem_vm *vm, +static void _dpu_format_populate_addrs_linear (struct drm_gpuvm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout) {
@@ -373,7 +373,7 @@ static void _dpu_format_populate_addrs_linear(struct msm_gem_vm *vm,* @fb: framebuffer pointer * @layout: format layout structure to populate */ -void dpu_format_populate_addrs (struct msm_gem_vm *vm, +void dpu_format_populate_addrs (struct drm_gpuvm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout) {
drivers/gpu/drm/msm/disp/dpu1/dpu_formats.h +1 -1
@@ -31,7 +31,7 @@ static inline bool dpu_find_format(u32 format, const u32 *supported_formats,return false; } -void dpu_format_populate_addrs (struct msm_gem_vm *vm, +void dpu_format_populate_addrs (struct drm_gpuvm *vm,struct drm_framebuffer *fb,struct dpu_hw_fmt_layout *layout);
drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c +3 -3
@@ -1098,17 +1098,17 @@ static void _dpu_kms_mmu_destroy(struct dpu_kms *dpu_kms)if (!dpu_kms->base.vm)return ; - mmu = dpu_kms->base.vm->mmu; + mmu =to_msm_vm (dpu_kms->base.vm)->mmu; mmu->funcs->detach (mmu); -msm_gem_vm_put (dpu_kms->base.vm); +drm_gpuvm_put (dpu_kms->base.vm); dpu_kms->base.vm =NULL ; }static int _dpu_kms_mmu_init (struct dpu_kms *dpu_kms) { -struct msm_gem_vm *vm; +struct drm_gpuvm *vm; vm =msm_kms_init_vm (dpu_kms->dev);if (IS_ERR (vm))
drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h +1 -1
@@ -34,7 +34,7 @@*/ struct dpu_plane_state {struct drm_plane_state base; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;struct dpu_sw_pipe pipe;struct dpu_sw_pipe r_pipe;struct dpu_sw_pipe_cfg pipe_cfg;
drivers/gpu/drm/msm/disp/mdp4/mdp4_kms.c +6 -5
@@ -120,15 +120,16 @@ static void mdp4_destroy(struct msm_kms *kms) {struct mdp4_kms *mdp4_kms =to_mdp4_kms (to_mdp_kms (kms));struct device *dev = mdp4_kms->dev->dev; -struct msm_gem_vm *vm = kms->vm;if (mdp4_kms->blank_cursor_iova)msm_gem_unpin_iova (mdp4_kms->blank_cursor_bo, kms->vm);drm_gem_object_put (mdp4_kms->blank_cursor_bo); -if (vm) { - vm->mmu->funcs->detach (vm->mmu); -msm_gem_vm_put (vm); +if (kms->vm) { +struct msm_mmu *mmu =to_msm_vm (kms->vm)->mmu; + + mmu->funcs->detach (mmu); +drm_gpuvm_put (kms->vm); }if (mdp4_kms->rpm_enabled)
@@ -380,7 +381,7 @@ static int mdp4_kms_init(struct drm_device *dev)struct mdp4_kms *mdp4_kms =to_mdp4_kms (to_mdp_kms (priv->kms));struct msm_kms *kms =NULL ;struct msm_mmu *mmu; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;int ret;u32 major, minor;unsigned long max_clk;
drivers/gpu/drm/msm/disp/mdp5/mdp5_kms.c +6 -5
@@ -198,11 +198,12 @@ static void mdp5_destroy(struct mdp5_kms *mdp5_kms);static void mdp5_kms_destroy (struct msm_kms *kms) {struct mdp5_kms *mdp5_kms =to_mdp5_kms (to_mdp_kms (kms)); -struct msm_gem_vm *vm = kms->vm; -if (vm) { - vm->mmu->funcs->detach (vm->mmu); -msm_gem_vm_put (vm); +if (kms->vm) { +struct msm_mmu *mmu =to_msm_vm (kms->vm)->mmu; + + mmu->funcs->detach (mmu); +drm_gpuvm_put (kms->vm); }mdp_kms_destroy (&mdp5_kms->base);
@@ -500,7 +501,7 @@ static int mdp5_kms_init(struct drm_device *dev)struct mdp5_kms *mdp5_kms;struct mdp5_cfg *config;struct msm_kms *kms = priv->kms; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;int i, ret; ret =mdp5_init (to_platform_device (dev->dev), dev);
drivers/gpu/drm/msm/dsi/dsi_host.c +3 -3
@@ -143,7 +143,7 @@ struct msm_dsi_host {/* DSI 6G TX buffer*/ struct drm_gem_object *tx_gem_obj; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;/* DSI v2 TX buffer */ void *tx_buf;
@@ -1146,7 +1146,7 @@ int dsi_tx_buf_alloc_6g(struct msm_dsi_host *msm_host, int size)uint64_t iova;u8 *data; - msm_host->vm =msm_gem_vm_get (priv->kms->vm); + msm_host->vm =drm_gpuvm_get (priv->kms->vm); data = msm_gem_kernel_new(dev, size, MSM_BO_WC, msm_host->vm,
@@ -1194,7 +1194,7 @@ void msm_dsi_tx_buf_free(struct mipi_dsi_host *host)if (msm_host->tx_gem_obj) {msm_gem_kernel_put (msm_host->tx_gem_obj, msm_host->vm); -msm_gem_vm_put (msm_host->vm); +drm_gpuvm_put (msm_host->vm); msm_host->tx_gem_obj =NULL ; msm_host->vm =NULL ; }
drivers/gpu/drm/msm/msm_drv.h +9 -10
@@ -48,8 +48,6 @@ struct msm_rd_state;struct msm_perf_state ;struct msm_gem_submit ;struct msm_fence_context ; -struct msm_gem_vm ; -struct msm_gem_vma ;struct msm_disp_state ;#define MAX_CRTCS 8
@@ -230,7 +228,7 @@ void msm_crtc_disable_vblank(struct drm_crtc *crtc);int msm_register_mmu (struct drm_device *dev,struct msm_mmu *mmu);void msm_unregister_mmu (struct drm_device *dev,struct msm_mmu *mmu); -struct msm_gem_vm *msm_kms_init_vm (struct drm_device *dev); +struct drm_gpuvm *msm_kms_init_vm (struct drm_device *dev);bool msm_use_mmu (struct drm_device *dev); int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
@@ -251,13 +249,14 @@ struct drm_gem_object *msm_gem_prime_import_sg_table(struct drm_device *dev,int msm_gem_prime_pin (struct drm_gem_object *obj);void msm_gem_prime_unpin (struct drm_gem_object *obj); -int msm_framebuffer_prepare (struct drm_framebuffer *fb, -struct msm_gem_vm *vm,bool needs_dirtyfb); -void msm_framebuffer_cleanup (struct drm_framebuffer *fb, -struct msm_gem_vm *vm,bool needed_dirtyfb); -uint32_t msm_framebuffer_iova (struct drm_framebuffer *fb, -struct msm_gem_vm *vm,int plane); -struct drm_gem_object *msm_framebuffer_bo (struct drm_framebuffer *fb,int plane); +int msm_framebuffer_prepare (struct drm_framebuffer *fb,struct drm_gpuvm *vm, +bool needs_dirtyfb); +void msm_framebuffer_cleanup (struct drm_framebuffer *fb,struct drm_gpuvm *vm, +bool needed_dirtyfb); +uint32_t msm_framebuffer_iova (struct drm_framebuffer *fb,struct drm_gpuvm *vm, +int plane); +struct drm_gem_object *msm_framebuffer_bo (struct drm_framebuffer *fb, +int plane);const struct msm_format *msm_framebuffer_format (struct drm_framebuffer *fb);struct drm_framebuffer *msm_framebuffer_create (struct drm_device *dev,struct drm_file *file,const struct drm_mode_fb_cmd2 *mode_cmd);
drivers/gpu/drm/msm/msm_fb.c +6 -8
@@ -75,9 +75,8 @@ void msm_framebuffer_describe(struct drm_framebuffer *fb, struct seq_file *m)/* prepare/pin all the fb's bo's for scanout. */ -int msm_framebuffer_prepare (struct drm_framebuffer *fb, -struct msm_gem_vm *vm, -bool needs_dirtyfb) +int msm_framebuffer_prepare (struct drm_framebuffer *fb,struct drm_gpuvm *vm, +bool needs_dirtyfb) {struct msm_framebuffer *msm_fb =to_msm_framebuffer (fb);int ret, i, n = fb->format->num_planes;
@@ -98,9 +97,8 @@ int msm_framebuffer_prepare(struct drm_framebuffer *fb,return 0 ; } -void msm_framebuffer_cleanup (struct drm_framebuffer *fb, -struct msm_gem_vm *vm, -bool needed_dirtyfb) +void msm_framebuffer_cleanup (struct drm_framebuffer *fb,struct drm_gpuvm *vm, +bool needed_dirtyfb) {struct msm_framebuffer *msm_fb =to_msm_framebuffer (fb);int i, n = fb->format->num_planes;
@@ -115,8 +113,8 @@ void msm_framebuffer_cleanup(struct drm_framebuffer *fb,memset (msm_fb->iova,0 ,sizeof (msm_fb->iova)); } -uint32_t msm_framebuffer_iova (struct drm_framebuffer *fb, -struct msm_gem_vm *vm,int plane) +uint32_t msm_framebuffer_iova (struct drm_framebuffer *fb,struct drm_gpuvm *vm, +int plane) {struct msm_framebuffer *msm_fb =to_msm_framebuffer (fb);return msm_fb->iova[plane] + fb->offsets[plane];
drivers/gpu/drm/msm/msm_gem.c +39 -43
214 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gem.h +23 -30
87 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gem_submit.c +2 -2
@@ -299,7 +299,7 @@ static int submit_pin_objects(struct msm_gem_submit *submit)for (i =0 ; i < submit->nr_bos; i++) {struct drm_gem_object *obj = submit->bos[i].obj; -struct msm_gem_vma *vma; +struct drm_gpuva *vma;/* if locking succeeded, pin bo: */ vma =msm_gem_get_vma_locked (obj, submit->vm);
@@ -312,7 +312,7 @@ static int submit_pin_objects(struct msm_gem_submit *submit)if (ret)break ; - submit->bos[i].iova = vma->base.va.addr; + submit->bos[i].iova = vma->va.addr; }/*
drivers/gpu/drm/msm/msm_gem_vma.c +31 -41
144 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gpu.c +12 -9
@@ -283,7 +283,7 @@ static void msm_gpu_crashstate_capture(struct msm_gpu *gpu,if (state->fault_info.ttbr0) {struct msm_gpu_fault_info *info = &state->fault_info; -struct msm_mmu *mmu = submit->vm->mmu; +struct msm_mmu *mmu =to_msm_vm (submit->vm)->mmu;msm_iommu_pagetable_params (mmu, &info->pgtbl_ttbr0, &info->asid);
@@ -387,7 +387,7 @@ static void recover_worker(struct kthread_work *work)/* Increment the fault counts */ submit->queue->faults++;if (submit->vm) - submit->vm->faults++; +to_msm_vm (submit->vm)->faults++;get_comm_cmdline (submit, &comm, &cmd);
@@ -463,6 +463,7 @@ static void fault_worker(struct kthread_work *work) { struct msm_gpu *gpu = container_of(work, struct msm_gpu, fault_work);struct msm_gem_submit *submit; +struct msm_mmu *mmu =to_msm_vm (gpu->vm)->mmu;struct msm_ringbuffer *cur_ring = gpu->funcs->active_ring (gpu);char *comm =NULL , *cmd =NULL ;
@@ -492,7 +493,7 @@ static void fault_worker(struct kthread_work *work) resume_smmu:memset (&gpu->fault_info,0 ,sizeof (gpu->fault_info)); - gpu->vm->mmu->funcs->resume_translation (gpu->vm->mmu); + mmu->funcs->resume_translation (mmu);mutex_unlock (&gpu->lock); }
@@ -829,10 +830,11 @@ static int get_clocks(struct platform_device *pdev, struct msm_gpu *gpu) }/* Return a new address space for a msm_drm_private instance */ -struct msm_gem_vm * +struct drm_gpuvm *msm_gpu_create_private_vm (struct msm_gpu *gpu,struct task_struct *task) { -struct msm_gem_vm *vm =NULL ; +struct drm_gpuvm *vm =NULL ; +if (!gpu)return NULL ;
@@ -843,11 +845,11 @@ msm_gpu_create_private_vm(struct msm_gpu *gpu, struct task_struct *task)if (gpu->funcs->create_private_vm) { vm = gpu->funcs->create_private_vm (gpu);if (!IS_ERR (vm)) - vm->pid =get_pid (task_pid (task)); +to_msm_vm (vm)->pid =get_pid (task_pid (task)); }if (IS_ERR_OR_NULL (vm)) - vm =msm_gem_vm_get (gpu->vm); + vm =drm_gpuvm_get (gpu->vm);return vm; }
@@ -1020,8 +1022,9 @@ void msm_gpu_cleanup(struct msm_gpu *gpu)msm_gem_kernel_put (gpu->memptrs_bo, gpu->vm);if (!IS_ERR_OR_NULL (gpu->vm)) { - gpu->vm->mmu->funcs->detach (gpu->vm->mmu); -msm_gem_vm_put (gpu->vm); +struct msm_mmu *mmu =to_msm_vm (gpu->vm)->mmu; + mmu->funcs->detach (mmu); +drm_gpuvm_put (gpu->vm); }if (gpu->worker) {
drivers/gpu/drm/msm/msm_gpu.h +5 -5
@@ -78,8 +78,8 @@ struct msm_gpu_funcs {/* note: gpu_set_freq() can assume that we have been pm_resumed */ void (*gpu_set_freq)(struct msm_gpu *gpu,struct dev_pm_opp *opp,bool suspended); -struct msm_gem_vm *(*create_vm)(struct msm_gpu *gpu,struct platform_device *pdev); -struct msm_gem_vm *(*create_private_vm)(struct msm_gpu *gpu); +struct drm_gpuvm *(*create_vm)(struct msm_gpu *gpu,struct platform_device *pdev); +struct drm_gpuvm *(*create_private_vm)(struct msm_gpu *gpu);uint32_t (*get_rptr)(struct msm_gpu *gpu,struct msm_ringbuffer *ring);/**
@@ -234,7 +234,7 @@ struct msm_gpu {void __iomem *mmio;int irq; -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;/* Power Control: */ struct regulator *gpu_reg, *gpu_cx;
@@ -363,7 +363,7 @@ struct msm_context {int queueid;/** @vm: the per-process GPU address-space */ -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;/** @kref: the reference count */ struct kref ref;
@@ -673,7 +673,7 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, struct msm_gpu *gpu, const struct msm_gpu_funcs *funcs, const char *name, struct msm_gpu_config *config); -struct msm_gem_vm * +struct drm_gpuvm *msm_gpu_create_private_vm (struct msm_gpu *gpu,struct task_struct *task);void msm_gpu_cleanup (struct msm_gpu *gpu);
drivers/gpu/drm/msm/msm_kms.c +3 -3
@@ -176,9 +176,9 @@ static int msm_kms_fault_handler(void *arg, unsigned long iova, int flags, voidreturn -ENOSYS ; } -struct msm_gem_vm *msm_kms_init_vm (struct drm_device *dev) +struct drm_gpuvm *msm_kms_init_vm (struct drm_device *dev) { -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;struct msm_mmu *mmu;struct device *mdp_dev = dev->dev;struct device *mdss_dev = mdp_dev->parent;
@@ -212,7 +212,7 @@ struct msm_gem_vm *msm_kms_init_vm(struct drm_device *dev)return vm; } -msm_mmu_set_fault_handler (vm->mmu, kms, msm_kms_fault_handler); +msm_mmu_set_fault_handler (to_msm_vm (vm)->mmu, kms, msm_kms_fault_handler);return vm; }
drivers/gpu/drm/msm/msm_kms.h +1 -1
@@ -139,7 +139,7 @@ struct msm_kms {atomic_t fault_snapshot_capture;/* mapper-id used to request GEM buffer mapped for scanout: */ -struct msm_gem_vm *vm; +struct drm_gpuvm *vm;/* disp snapshot support */ struct kthread_worker *dump_worker;
drivers/gpu/drm/msm/msm_submitqueue.c +1 -1
@@ -59,7 +59,7 @@ void __msm_context_destroy(struct kref *kref)kfree (ctx->entities[i]); } -msm_gem_vm_put (ctx->vm); +drm_gpuvm_put (ctx->vm);kfree (ctx->comm);kfree (ctx->cmdline);kfree (ctx);
Signature · 1 line
2.48.1
PATCH v2 13/34 drm/msm: Split submit_pin_objects()
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
For VM_BIND, in the first step, we just want to get the backing pages, but defer creating the vma until the map/unmap/ops are evaluated.
--- drivers/gpu/drm/msm/msm_gem_submit.c | 27 +++++++++++++++++++-------- 1 file changed, 19 insertions(+), 8 deletions(-)
drivers/gpu/drm/msm/msm_gem_submit.c +19 -8
@@ -292,12 +292,16 @@ static int submit_fence_sync(struct msm_gem_submit *submit)return ret; } -static int submit_pin_objects (struct msm_gem_submit *submit) +static int submit_pin_vmas (struct msm_gem_submit *submit) { -struct msm_drm_private *priv = submit->dev->dev_private; -int i, ret =0 ; +int ret =0 ; -for (i =0 ; i < submit->nr_bos; i++) { +/* +* First loop, before holding the LRU lock, avoids holding the +* LRU lock while calling msm_gem_pin_vma_locked (which could +* trigger get_pages()) +*/ +for (int i =0 ; i < submit->nr_bos; i++) {struct drm_gem_object *obj = submit->bos[i].obj;struct drm_gpuva *vma;
@@ -315,6 +319,13 @@ static int submit_pin_objects(struct msm_gem_submit *submit) submit->bos[i].iova = vma->va.addr; } +return ret; +} + +static void submit_pin_objects (struct msm_gem_submit *submit) +{ +struct msm_drm_private *priv = submit->dev->dev_private; +/* * A second loop while holding the LRU lock (a) avoids acquiring/dropping * the LRU lock for each individual bo, while (b) avoiding holding the
@@ -323,14 +334,12 @@ static int submit_pin_objects(struct msm_gem_submit *submit)* could trigger deadlock with the shrinker). */ mutex_lock (&priv->lru.lock); -for (i =0 ; i < submit->nr_bos; i++) { +for (int i =0 ; i < submit->nr_bos; i++) {msm_gem_pin_obj_locked (submit->bos[i].obj); }mutex_unlock (&priv->lru.lock); submit->bos_pinned = true; - -return ret; }static void submit_unpin_objects (struct msm_gem_submit *submit)
@@ -760,10 +769,12 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, goto out; } - ret =submit_pin_objects (submit); + ret =submit_pin_vmas (submit);if (ret) goto out; +submit_pin_objects (submit); +for (i =0 ; i < args->nr_cmds; i++) {struct drm_gem_object *obj;uint64_t iova;
Signature · 1 line
2.48.1
PATCH v2 14/34 drm/msm: Lazily create context VM
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
In the next commit, a way for userspace to opt-in to userspace managed VM is added. For this to work, we need to defer creation of the VM until it is needed.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 3 ++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 14 +++++++----- drivers/gpu/drm/msm/msm_drv.c | 29 ++++++++++++++++++++----- drivers/gpu/drm/msm/msm_gem_submit.c | 2 +- drivers/gpu/drm/msm/msm_gpu.h | 9 +++++++- 5 files changed, 43 insertions(+), 14 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +2 -1
@@ -112,6 +112,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu, {bool sysprof =refcount_read (&a6xx_gpu->base.base.sysprof_active) >1 ;struct msm_context *ctx = submit->queue->ctx; +struct drm_gpuvm *vm =msm_context_vm (submit->dev, ctx);struct adreno_gpu *adreno_gpu = &a6xx_gpu->base;phys_addr_t ttbr;u32 asid;
@@ -120,7 +121,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,if (ctx->seqno == ring->cur_ctx_seqno)return ; -if (msm_iommu_pagetable_params (to_msm_vm (ctx->vm)->mmu, &ttbr, &asid)) +if (msm_iommu_pagetable_params (to_msm_vm (vm)->mmu, &ttbr, &asid))return ;if (adreno_gpu->info->family >=ADRENO_7XX_GEN1 ) {
drivers/gpu/drm/msm/adreno/adreno_gpu.c +8 -6
@@ -351,6 +351,8 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, {struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct drm_device *drm = gpu->dev; +/* Note ctx can be NULL when called from rd_open(): */ +struct drm_gpuvm *vm = ctx ?msm_context_vm (drm, ctx) :NULL ;/* No pointer params yet */ if (*len !=0 )
@@ -396,8 +398,8 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, *value =0 ;return 0 ;case MSM_PARAM_FAULTS : -if (ctx->vm) - *value = gpu->global_faults +to_msm_vm (ctx->vm)->faults; +if (vm) + *value = gpu->global_faults +to_msm_vm (vm)->faults;else *value = gpu->global_faults;return 0 ;
@@ -405,14 +407,14 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, *value = gpu->suspend_count;return 0 ;case MSM_PARAM_VA_START : -if (ctx->vm == gpu->vm) +if (vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->mm_start; + *value = vm->mm_start;return 0 ;case MSM_PARAM_VA_SIZE : -if (ctx->vm == gpu->vm) +if (vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->mm_range; + *value = vm->mm_range;return 0 ;case MSM_PARAM_HIGHEST_BANK_BIT : *value = adreno_gpu->ubwc_config.highest_bank_bit;
drivers/gpu/drm/msm/msm_drv.c +24 -5
@@ -214,10 +214,29 @@ static void load_gpu(struct drm_device *dev)mutex_unlock (&init_lock); } +/** +* msm_context_vm - lazily create the context's VM +* +* @dev: the drm device +* @ctx: the context +* +* The VM is lazily created, so that userspace has a chance to opt-in to having +* a userspace managed VM before the VM is created. +* +* Note that this does not return a reference to the VM. Once the VM is created, +* it exists for the lifetime of the context. +*/ +struct drm_gpuvm *msm_context_vm (struct drm_device *dev,struct msm_context *ctx) +{ +struct msm_drm_private *priv = dev->dev_private; +if (!ctx->vm) + ctx->vm =msm_gpu_create_private_vm (priv->gpu, current); +return ctx->vm; +} +static int context_init (struct drm_device *dev,struct drm_file *file) {static atomic_t ident =ATOMIC_INIT (0 ); -struct msm_drm_private *priv = dev->dev_private;struct msm_context *ctx; ctx =kzalloc (sizeof (*ctx),GFP_KERNEL );
@@ -230,7 +249,6 @@ static int context_init(struct drm_device *dev, struct drm_file *file)kref_init (&ctx->ref);msm_submitqueue_init (dev, ctx); - ctx->vm =msm_gpu_create_private_vm (priv->gpu, current); file->driver_priv = ctx; ctx->seqno =atomic_inc_return (&ident);
@@ -408,7 +426,7 @@ static int msm_ioctl_gem_info_iova(struct drm_device *dev,* Don't pin the memory here - just get an address so that userspace can * be productive */ -return msm_gem_get_iova (obj, ctx->vm, iova); +return msm_gem_get_iova (obj,msm_context_vm (dev, ctx), iova); } static int msm_ioctl_gem_info_set_iova(struct drm_device *dev,
@@ -417,18 +435,19 @@ static int msm_ioctl_gem_info_set_iova(struct drm_device *dev, {struct msm_drm_private *priv = dev->dev_private;struct msm_context *ctx = file->driver_priv; +struct drm_gpuvm *vm =msm_context_vm (dev, ctx);if (!priv->gpu)return -EINVAL ;/* Only supported if per-process address space is supported: */ -if (priv->gpu->vm == ctx->vm) +if (priv->gpu->vm == vm)return UERR (EOPNOTSUPP , dev,"requires per-process pgtables" );if (should_fail (&fail_gem_iova, obj->size))return -ENOMEM ; -return msm_gem_set_iova (obj, ctx->vm, iova); +return msm_gem_set_iova (obj, vm, iova); } static int msm_ioctl_gem_info_set_metadata(struct drm_gem_object *obj,
drivers/gpu/drm/msm/msm_gem_submit.c +1 -1
@@ -63,7 +63,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,kref_init (&submit->ref); submit->dev = dev; - submit->vm = queue->ctx->vm; + submit->vm =msm_context_vm (dev, queue->ctx); submit->gpu = gpu; submit->cmd = (void *)&submit->bos[nr_bos]; submit->queue = queue;
drivers/gpu/drm/msm/msm_gpu.h +8 -1
@@ -362,7 +362,12 @@ struct msm_context {*/ int queueid; -/** @vm: the per-process GPU address-space */ +/** +* @vm: +* +* The per-process GPU address-space. Do not access directly, use +* msm_context_vm(). +*/ struct drm_gpuvm *vm;/** @kref: the reference count */
@@ -447,6 +452,8 @@ struct msm_context {atomic64_t ctx_mem; }; +struct drm_gpuvm *msm_context_vm (struct drm_device *dev,struct msm_context *ctx); +/** * msm_gpu_convert_priority - Map userspace priority to ring # and sched priority *
Signature · 1 line
2.48.1
Re: [PATCH v2 14/34] drm/msm: Lazily create context VM
Quoted 93 lines · Rob Clark
From: Rob Clark <[email protected]>
In the next commit, a way for userspace to opt-in to userspace managed VM is added. For this to work, we need to defer creation of the VM until it is needed.
Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 3 ++- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 14 +++++++----- drivers/gpu/drm/msm/msm_drv.c | 29 ++++++++++++++++++++----- drivers/gpu/drm/msm/msm_gem_submit.c | 2 +- drivers/gpu/drm/msm/msm_gpu.h | 9 +++++++- 5 files changed, 43 insertions(+), 14 deletions(-)drivers/gpu/drm/msm/adreno/a6xx_gpu.c +2
-1@@ -112,6 +112,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu, {bool sysprof =refcount_read (&a6xx_gpu->base.base.sysprof_active) >1 ;struct msm_context *ctx = submit->queue->ctx; +struct drm_gpuvm *vm =msm_context_vm (submit->dev, ctx);struct adreno_gpu *adreno_gpu = &a6xx_gpu->base;phys_addr_t ttbr;u32 asid;@@ -120,7 +121,7 @@ static void a6xx_set_pagetable(struct a6xx_gpu *a6xx_gpu,if (ctx->seqno == ring->cur_ctx_seqno)return ; -if (msm_iommu_pagetable_params (to_msm_vm (ctx->vm)->mmu, &ttbr, &asid)) +if (msm_iommu_pagetable_params (to_msm_vm (vm)->mmu, &ttbr, &asid))return ;if (adreno_gpu->info->family >=ADRENO_7XX_GEN1 ) {drivers/gpu/drm/msm/adreno/adreno_gpu.c +8
-6@@ -351,6 +351,8 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, {struct adreno_gpu *adreno_gpu =to_adreno_gpu (gpu);struct drm_device *drm = gpu->dev; +/* Note ctx can be NULL when called from rd_open(): */ +struct drm_gpuvm *vm = ctx ?msm_context_vm (drm, ctx) :NULL ;/* No pointer params yet */ if (*len !=0 )@@ -396,8 +398,8 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, *value =0 ;return 0 ;case MSM_PARAM_FAULTS : -if (ctx->vm) - *value = gpu->global_faults +to_msm_vm (ctx->vm)->faults; +if (vm) + *value = gpu->global_faults +to_msm_vm (vm)->faults;else *value = gpu->global_faults;return 0 ;@@ -405,14 +407,14 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx, *value = gpu->suspend_count;return 0 ;case MSM_PARAM_VA_START : -if (ctx->vm == gpu->vm) +if (vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->mm_start; + *value = vm->mm_start;return 0 ;case MSM_PARAM_VA_SIZE : -if (ctx->vm == gpu->vm) +if (vm == gpu->vm)return UERR (EINVAL , drm,"requires per-process pgtables" ); - *value = ctx->vm->mm_range; + *value = vm->mm_range;return 0 ;case MSM_PARAM_HIGHEST_BANK_BIT : *value = adreno_gpu->ubwc_config.highest_bank_bit;drivers/gpu/drm/msm/msm_drv.c +6
-0@@ -214,10 +214,29 @@ static void load_gpu(struct drm_device *dev)mutex_unlock (&init_lock); } +/** +* msm_context_vm - lazily create the context's VM +* +* @dev: the drm device +* @ctx: the context +*
drivers/gpu/drm/msm/msm_drv.c static void load_gpu(struct drm_device *dev) · line 232
Quoted 10 lines
drivers/gpu/drm/msm/msm_drv.c +10
-0+* The VM is lazily created, so that userspace has a chance to opt-in to having +* a userspace managed VM before the VM is created. +* +* Note that this does not return a reference to the VM. Once the VM is created, +* it exists for the lifetime of the context. +*/ +struct drm_gpuvm *msm_context_vm (struct drm_device *dev,struct msm_context *ctx) +{ +struct msm_drm_private *priv = dev->dev_private; +if (!ctx->vm)
hmm. This is racy and it is in a userspace accessible path!
-Akhil
Quoted 90 lines
drivers/gpu/drm/msm/msm_drv.c +8
-5+ ctx->vm =msm_gpu_create_private_vm (priv->gpu, current); +return ctx->vm; +} +static int context_init (struct drm_device *dev,struct drm_file *file) {static atomic_t ident =ATOMIC_INIT (0 ); -struct msm_drm_private *priv = dev->dev_private;struct msm_context *ctx; ctx =kzalloc (sizeof (*ctx),GFP_KERNEL );@@ -230,7 +249,6 @@ static int context_init(struct drm_device *dev, struct drm_file *file)kref_init (&ctx->ref);msm_submitqueue_init (dev, ctx); - ctx->vm =msm_gpu_create_private_vm (priv->gpu, current); file->driver_priv = ctx; ctx->seqno =atomic_inc_return (&ident);@@ -408,7 +426,7 @@ static int msm_ioctl_gem_info_iova(struct drm_device *dev,* Don't pin the memory here - just get an address so that userspace can * be productive */ -return msm_gem_get_iova (obj, ctx->vm, iova); +return msm_gem_get_iova (obj,msm_context_vm (dev, ctx), iova); } static int msm_ioctl_gem_info_set_iova(struct drm_device *dev,@@ -417,18 +435,19 @@ static int msm_ioctl_gem_info_set_iova(struct drm_device *dev, {struct msm_drm_private *priv = dev->dev_private;struct msm_context *ctx = file->driver_priv; +struct drm_gpuvm *vm =msm_context_vm (dev, ctx);if (!priv->gpu)return -EINVAL ;/* Only supported if per-process address space is supported: */ -if (priv->gpu->vm == ctx->vm) +if (priv->gpu->vm == vm)return UERR (EOPNOTSUPP , dev,"requires per-process pgtables" );if (should_fail (&fail_gem_iova, obj->size))return -ENOMEM ; -return msm_gem_set_iova (obj, ctx->vm, iova); +return msm_gem_set_iova (obj, vm, iova); } static int msm_ioctl_gem_info_set_metadata(struct drm_gem_object *obj,drivers/gpu/drm/msm/msm_gem_submit.c +1
-1@@ -63,7 +63,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,kref_init (&submit->ref); submit->dev = dev; - submit->vm = queue->ctx->vm; + submit->vm =msm_context_vm (dev, queue->ctx); submit->gpu = gpu; submit->cmd = (void *)&submit->bos[nr_bos]; submit->queue = queue;drivers/gpu/drm/msm/msm_gpu.h +8
-1@@ -362,7 +362,12 @@ struct msm_context {*/ int queueid; -/** @vm: the per-process GPU address-space */ +/** +* @vm: +* +* The per-process GPU address-space. Do not access directly, use +* msm_context_vm(). +*/ struct drm_gpuvm *vm;/** @kref: the reference count */ @@ -447,6 +452,8 @@ struct msm_context {atomic64_t ctx_mem; }; +struct drm_gpuvm *msm_context_vm (struct drm_device *dev,struct msm_context *ctx); +/** * msm_gpu_convert_priority - Map userspace priority to ring # and sched priority *
PATCH v2 15/34 drm/msm: Add opt-in for VM_BIND
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Add a SET_PARAM for userspace to request to manage to the VM itself, instead of getting a kernel managed VM.
In order to transition to a userspace managed VM, this param must be set before any mappings are created.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 4 ++-- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 15 +++++++++++++ drivers/gpu/drm/msm/msm_drv.c | 13 +++++++++-- drivers/gpu/drm/msm/msm_gem.c | 8 +++++++ drivers/gpu/drm/msm/msm_gpu.c | 5 +++-- drivers/gpu/drm/msm/msm_gpu.h | 29 +++++++++++++++++++++++-- include/uapi/drm/msm_drm.h | 24 ++++++++++++++++++++ 7 files changed, 90 insertions(+), 8 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +2 -2
@@ -2263,7 +2263,7 @@ a6xx_create_vm(struct msm_gpu *gpu, struct platform_device *pdev) }static struct drm_gpuvm * -a6xx_create_private_vm (struct msm_gpu *gpu) +a6xx_create_private_vm (struct msm_gpu *gpu,bool kernel_managed) {struct msm_mmu *mmu;
@@ -2273,7 +2273,7 @@ a6xx_create_private_vm(struct msm_gpu *gpu)return ERR_CAST (mmu);return msm_gem_vm_create (gpu->dev, mmu,"gpu" ,0x100000000ULL , -adreno_private_vm_size (gpu), true); +adreno_private_vm_size (gpu), kernel_managed); }static uint32_t a6xx_get_rptr (struct msm_gpu *gpu,struct msm_ringbuffer *ring)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +15 -0
@@ -483,6 +483,21 @@ int adreno_set_param(struct msm_gpu *gpu, struct msm_context *ctx,if (!capable (CAP_SYS_ADMIN ))return UERR (EPERM , drm,"invalid permissions" );return msm_context_set_sysprof (ctx, gpu, value); +case MSM_PARAM_EN_VM_BIND : +/* We can only support VM_BIND with per-process pgtables: */ +if (ctx->vm == gpu->vm) +return UERR (EINVAL , drm,"requires per-process pgtables" ); + +/* +* We can only swtich to VM_BIND mode if the VM has not yet +* been created: +*/ +if (ctx->vm) +return UERR (EBUSY , drm,"VM already created" ); + + ctx->userspace_managed_vm = value; + +return 0 ;default :return UERR (EINVAL , drm,"%s: invalid param: %u" , gpu->name, param); }
drivers/gpu/drm/msm/msm_drv.c +11 -2
@@ -229,8 +229,11 @@ static void load_gpu(struct drm_device *dev)struct drm_gpuvm *msm_context_vm (struct drm_device *dev,struct msm_context *ctx) {struct msm_drm_private *priv = dev->dev_private; -if (!ctx->vm) - ctx->vm =msm_gpu_create_private_vm (priv->gpu, current); +if (!ctx->vm) { + ctx->vm =msm_gpu_create_private_vm ( + priv->gpu, current, !ctx->userspace_managed_vm); + + }return ctx->vm; }
@@ -419,6 +422,9 @@ static int msm_ioctl_gem_info_iova(struct drm_device *dev,if (!priv->gpu)return -EINVAL ; +if (msm_context_is_vmbind (ctx)) +return UERR (EINVAL , dev,"VM_BIND is enabled" ); +if (should_fail (&fail_gem_iova, obj->size))return -ENOMEM ;
@@ -440,6 +446,9 @@ static int msm_ioctl_gem_info_set_iova(struct drm_device *dev,if (!priv->gpu)return -EINVAL ; +if (msm_context_is_vmbind (ctx)) +return UERR (EINVAL , dev,"VM_BIND is enabled" ); +/* Only supported if per-process address space is supported: */ if (priv->gpu->vm == vm)return UERR (EOPNOTSUPP , dev,"requires per-process pgtables" );
drivers/gpu/drm/msm/msm_gem.c +8 -0
@@ -63,6 +63,14 @@ static void msm_gem_close(struct drm_gem_object *obj, struct drm_file *file)if (!ctx->vm)return ; +/* +* VM_BIND does not depend on implicit teardown of VMAs on handle +* close, but instead on implicit teardown of the VM when the device +* is closed (see msm_gem_vm_close()) +*/ +if (msm_context_is_vmbind (ctx)) +return ; +/* * TODO we might need to kick this to a queue to avoid blocking * in CLOSE ioctl
drivers/gpu/drm/msm/msm_gpu.c +3 -2
@@ -831,7 +831,8 @@ static int get_clocks(struct platform_device *pdev, struct msm_gpu *gpu)/* Return a new address space for a msm_drm_private instance */ struct drm_gpuvm * -msm_gpu_create_private_vm (struct msm_gpu *gpu,struct task_struct *task) +msm_gpu_create_private_vm (struct msm_gpu *gpu,struct task_struct *task, +bool kernel_managed) {struct drm_gpuvm *vm =NULL ;
@@ -843,7 +844,7 @@ msm_gpu_create_private_vm(struct msm_gpu *gpu, struct task_struct *task)* the global one */ if (gpu->funcs->create_private_vm) { - vm = gpu->funcs->create_private_vm (gpu); + vm = gpu->funcs->create_private_vm (gpu, kernel_managed);if (!IS_ERR (vm))to_msm_vm (vm)->pid =get_pid (task_pid (task)); }
drivers/gpu/drm/msm/msm_gpu.h +27 -2
@@ -79,7 +79,7 @@ struct msm_gpu_funcs {void (*gpu_set_freq)(struct msm_gpu *gpu,struct dev_pm_opp *opp,bool suspended);struct drm_gpuvm *(*create_vm)(struct msm_gpu *gpu,struct platform_device *pdev); -struct drm_gpuvm *(*create_private_vm)(struct msm_gpu *gpu); +struct drm_gpuvm *(*create_private_vm)(struct msm_gpu *gpu,bool kernel_managed);uint32_t (*get_rptr)(struct msm_gpu *gpu,struct msm_ringbuffer *ring);/**
@@ -362,6 +362,14 @@ struct msm_context {*/ int queueid; +/** +* @userspace_managed_vm: +* +* Has userspace opted-in to userspace managed VM (ie. VM_BIND) via +* MSM_PARAM_EN_VM_BIND? +*/ +bool userspace_managed_vm; +/** * @vm: *
@@ -454,6 +462,22 @@ struct msm_context {struct drm_gpuvm *msm_context_vm (struct drm_device *dev,struct msm_context *ctx); +/** +* msm_context_is_vm_bind() - has userspace opted in to VM_BIND? +* +* @ctx: the drm_file context +* +* See MSM_PARAM_EN_VM_BIND. If userspace is managing the VM, it can +* do sparse binding including having multiple, potentially partial, +* mappings in the VM. Therefore certain legacy uabi (ie. GET_IOVA, +* SET_IOVA) are rejected because they don't have a sensible meaning. +*/ +static inline bool +msm_context_is_vmbind (struct msm_context *ctx) +{ +return ctx->userspace_managed_vm; +} +/** * msm_gpu_convert_priority - Map userspace priority to ring # and sched priority *
@@ -681,7 +705,8 @@ int msm_gpu_init(struct drm_device *drm, struct platform_device *pdev, const char *name, struct msm_gpu_config *config);struct drm_gpuvm * -msm_gpu_create_private_vm (struct msm_gpu *gpu,struct task_struct *task); +msm_gpu_create_private_vm (struct msm_gpu *gpu,struct task_struct *task, +bool kernel_managed);void msm_gpu_cleanup (struct msm_gpu *gpu);
include/uapi/drm/msm_drm.h +24 -0
@@ -91,6 +91,30 @@ struct drm_msm_timespec {#define MSM_PARAM_UBWC_SWIZZLE 0x12/* RO */ #define MSM_PARAM_MACROTILE_MODE 0x13/* RO */ #define MSM_PARAM_UCHE_TRAP_BASE 0x14/* RO */ +/* MSM_PARAM_EN_VM_BIND is set to 1 to enable VM_BIND ops. +* +* With VM_BIND enabled, userspace is required to allocate iova and use the +* VM_BIND ops for map/unmap ioctls. MSM_INFO_SET_IOVA and MSM_INFO_GET_IOVA +* will be rejected. (The latter does not have a sensible meaning when a BO +* can have multiple and/or partial mappings.) +* +* With VM_BIND enabled, userspace does not include a submit_bo table in the +* SUBMIT ioctl (this will be rejected), the resident set is determined by +* the the VM_BIND ops. +* +* Enabling VM_BIND will fail on devices which do not have per-process pgtables. +* And it is not allowed to disable VM_BIND once it has been enabled. +* +* Enabling VM_BIND should be done (attempted) prior to allocating any BOs or +* submitqueues of type MSM_SUBMITQUEUE_VM_BIND. +* +* Relatedly, when VM_BIND mode is enabled, the kernel will not try to recover +* from GPU faults or failed async VM_BIND ops, in particular because it is +* difficult to communicate to userspace which op failed so that userspace +* could rewind and try again. When the VM is marked unusable, the SUBMIT +* ioctl will throw -EPIPE. +*/ +#define MSM_PARAM_EN_VM_BIND 0x15/* WO, once */ /* For backwards compat. The original support for preemption was based on * a single ring per priority level so # of priority levels equals the #
Signature · 1 line
2.48.1
PATCH v2 16/34 drm/msm: Mark VM as unusable on faults
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
If userspace has opted-in to VM_BIND, then GPU faults and VM_BIND errors will mark the VM as unusable.
--- drivers/gpu/drm/msm/msm_gem.h | 17 +++++++++++++++++ drivers/gpu/drm/msm/msm_gem_submit.c | 3 +++ drivers/gpu/drm/msm/msm_gpu.c | 16 ++++++++++++++-- 3 files changed, 34 insertions(+), 2 deletions(-)
drivers/gpu/drm/msm/msm_gem.h +17 -0
@@ -82,6 +82,23 @@ struct msm_gem_vm {/** @managed: is this a kernel managed VM? */ bool managed; + +/** +* @unusable: True if the VM has turned unusable because something +* bad happened during an asynchronous request. +* +* We don't try to recover from such failures, because this implies +* informing userspace about the specific operation that failed, and +* hoping the userspace driver can replay things from there. This all +* sounds very complicated for little gain. +* +* Instead, we should just flag the VM as unusable, and fail any +* further request targeting this VM. +* +* As an analogy, this would be mapped to a VK_ERROR_DEVICE_LOST +* situation, where the logical device needs to be re-created. +*/ +bool unusable; };#define to_msm_vm (x) container_of(x, struct msm_gem_vm, base)
drivers/gpu/drm/msm/msm_gem_submit.c +3 -0
@@ -668,6 +668,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,if (args->pad)return -EINVAL ; +if (to_msm_vm (ctx->vm)->unusable) +return UERR (EPIPE , dev,"context is unusable" ); +/* for now, we just have 3d pipe.. eventually this would need to * be more clever to dispatch to appropriate gpu module: */
drivers/gpu/drm/msm/msm_gpu.c +14 -2
@@ -386,8 +386,20 @@ static void recover_worker(struct kthread_work *work)/* Increment the fault counts */ submit->queue->faults++; -if (submit->vm) -to_msm_vm (submit->vm)->faults++; +if (submit->vm) { +struct msm_gem_vm *vm =to_msm_vm (submit->vm); + + vm->faults++; + +/* +* If userspace has opted-in to VM_BIND (and therefore userspace +* management of the VM), faults mark the VM as unusuable. This +* matches vulkan expectations (vulkan is the main target for +* VM_BIND) +*/ +if (!vm->managed) + vm->unusable = true; + }get_comm_cmdline (submit, &comm, &cmd);
Signature · 1 line
2.48.1
Re: [PATCH v2 16/34] drm/msm: Mark VM as unusable on faults
Quoted 65 lines · Rob Clark
From: Rob Clark <[email protected]>
If userspace has opted-in to
VM_BIND, then GPU faults andVM_BINDerrors will mark the VM as unusable.Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_gem.h | 17 +++++++++++++++++ drivers/gpu/drm/msm/msm_gem_submit.c | 3 +++ drivers/gpu/drm/msm/msm_gpu.c | 16 ++++++++++++++-- 3 files changed, 34 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_gem.h +17
-0@@ -82,6 +82,23 @@ struct msm_gem_vm {/** @managed: is this a kernel managed VM? */ bool managed; + +/** +* @unusable: True if the VM has turned unusable because something +* bad happened during an asynchronous request. +* +* We don't try to recover from such failures, because this implies +* informing userspace about the specific operation that failed, and +* hoping the userspace driver can replay things from there. This all +* sounds very complicated for little gain. +* +* Instead, we should just flag the VM as unusable, and fail any +* further request targeting this VM. +* +* As an analogy, this would be mapped to a VK_ERROR_DEVICE_LOST +* situation, where the logical device needs to be re-created. +*/ +bool unusable; };#define to_msm_vm (x) container_of(x, struct msm_gem_vm, base)drivers/gpu/drm/msm/msm_gem_submit.c +3
-0@@ -668,6 +668,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,if (args->pad)return -EINVAL ; +if (to_msm_vm (ctx->vm)->unusable) +return UERR (EPIPE , dev,"context is unusable" ); +/* for now, we just have 3d pipe.. eventually this would need to * be more clever to dispatch to appropriate gpu module: */ drivers/gpu/drm/msm/msm_gpu.c +0
-2@@ -386,8 +386,20 @@ static void recover_worker(struct kthread_work *work)/* Increment the fault counts */ submit->queue->faults++; -if (submit->vm) -to_msm_vm (submit->vm)->faults++;
drivers/gpu/drm/msm/msm_gpu.c static void recover_worker(struct kthread_work *work) · line 398
Quoted 10 lines
drivers/gpu/drm/msm/msm_gpu.c +10
-0+if (submit->vm) { +struct msm_gem_vm *vm =to_msm_vm (submit->vm); + + vm->faults++; + +/* +* If userspace has opted-in to VM_BIND (and therefore userspace +* management of the VM), faults mark the VM as unusuable. This +* matches vulkan expectations (vulkan is the main target for +* VM_BIND)
The bit about this matching Vulkan expectations isn't exactly true. Some Vulkan implementations do do this, but many will also just ignore the fault and try to continue going, and the spec allows either. It's a choice that we're making.
Connor
Quoted 10 lines
drivers/gpu/drm/msm/msm_gpu.c +4
-1+*/ +if (!vm->managed) + vm->unusable = true; + }get_comm_cmdline (submit, &comm, &cmd); --2.48.1
Re: [PATCH v2 16/34] drm/msm: Mark VM as unusable on faults
Quoted 74 lines · Connor Abbott
Quoted 72 lines · Rob Clark
From: Rob Clark <[email protected]>
If userspace has opted-in to
VM_BIND, then GPU faults andVM_BINDerrors will mark the VM as unusable.Signed-off-by Rob Clark <[email protected]>--- drivers/gpu/drm/msm/msm_gem.h | 17 +++++++++++++++++ drivers/gpu/drm/msm/msm_gem_submit.c | 3 +++ drivers/gpu/drm/msm/msm_gpu.c | 16 ++++++++++++++-- 3 files changed, 34 insertions(+), 2 deletions(-)drivers/gpu/drm/msm/msm_gem.h +17
-0@@ -82,6 +82,23 @@ struct msm_gem_vm {/** @managed: is this a kernel managed VM? */ bool managed; + +/** +* @unusable: True if the VM has turned unusable because something +* bad happened during an asynchronous request. +* +* We don't try to recover from such failures, because this implies +* informing userspace about the specific operation that failed, and +* hoping the userspace driver can replay things from there. This all +* sounds very complicated for little gain. +* +* Instead, we should just flag the VM as unusable, and fail any +* further request targeting this VM. +* +* As an analogy, this would be mapped to a VK_ERROR_DEVICE_LOST +* situation, where the logical device needs to be re-created. +*/ +bool unusable; };#define to_msm_vm (x) container_of(x, struct msm_gem_vm, base)drivers/gpu/drm/msm/msm_gem_submit.c +3
-0@@ -668,6 +668,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,if (args->pad)return -EINVAL ; +if (to_msm_vm (ctx->vm)->unusable) +return UERR (EPIPE , dev,"context is unusable" ); +/* for now, we just have 3d pipe.. eventually this would need to * be more clever to dispatch to appropriate gpu module: */ drivers/gpu/drm/msm/msm_gpu.c +7
-2@@ -386,8 +386,20 @@ static void recover_worker(struct kthread_work *work)/* Increment the fault counts */ submit->queue->faults++; -if (submit->vm) -to_msm_vm (submit->vm)->faults++; +if (submit->vm) { +struct msm_gem_vm *vm =to_msm_vm (submit->vm); + + vm->faults++; + +/* +* If userspace has opted-in to VM_BIND (and therefore userspace
Quoted 8 lines
drivers/gpu/drm/msm/msm_gpu.c static void recover_worker(struct kthread_work *work) · line 398
Quoted 3 lines
drivers/gpu/drm/msm/msm_gpu.c +3
-0+* management of the VM), faults mark the VM as unusuable. This +* matches vulkan expectations (vulkan is the main target for +* VM_BIND) The bit about this matching Vulkan expectations isn't exactly true. Some Vulkan implementations do do this, but many will also just ignore the fault and try to continue going, and the spec allows either. It's a choice that we're making.
As mentioned on IRC, this is actually about GPU hangs rather then smmu
faults. I guess the $subject is a bit misleading.
BR,
-R
Quoted 12 lines
Connor
Quoted 10 lines
drivers/gpu/drm/msm/msm_gpu.c +4
-1+*/ +if (!vm->managed) + vm->unusable = true; + }get_comm_cmdline (submit, &comm, &cmd); --2.48.1
PATCH v2 17/34 drm/msm: Extend SUBMIT ioctl for VM_BIND
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
This is a bit different than the path taken by other clean-slate drivers. But there is a lot in similar with BO pinning in the legacy "EXEC" path and "VM_BIND" MAP path. Also, we want the same fence and syncobj handling.
(Why bother with fence fd's? Because for virtgpu nctx for, guest syncobj's exist only as dma_fence's between the guest kernel and host.)
--- drivers/gpu/drm/msm/msm_gem.h | 10 ++--- drivers/gpu/drm/msm/msm_gem_submit.c | 65 ++++++++++++++++++++++++---- include/uapi/drm/msm_drm.h | 49 ++++++++++++++++++--- 3 files changed, 103 insertions(+), 21 deletions(-)
drivers/gpu/drm/msm/msm_gem.h +5 -5
@@ -345,13 +345,13 @@ struct msm_gem_submit {uint32_t nr_relocs;struct drm_msm_gem_submit_reloc *relocs; } *cmd;/* array of size nr_cmds */ -struct { +struct msm_gem_submit_bo {uint32_t flags; -union { -struct drm_gem_object *obj; -uint32_t handle; - }; +uint32_t handle; +struct drm_gem_object *obj;uint64_t iova; +uint64_t bo_offset; +uint64_t range; } bos[]; };
drivers/gpu/drm/msm/msm_gem_submit.c +56 -9
127 lines not on this page · raw message
include/uapi/drm/msm_drm.h +42 -7
97 lines not on this page · raw message
Signature · 1 line
2.48.1
PATCH v2 18/34 drm/msm: Add VM_BIND submitqueue
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
This submitqueue type isn't tied to a hw ringbuffer, but instead executes on the CPU for performing async VM_BIND ops.
--- drivers/gpu/drm/msm/msm_gem.c | 3 +- drivers/gpu/drm/msm/msm_gem.h | 10 ++ drivers/gpu/drm/msm/msm_gem_submit.c | 128 +++++++++++++++++++++++--- drivers/gpu/drm/msm/msm_gem_vma.c | 107 +++++++++++++++++++++ drivers/gpu/drm/msm/msm_gpu.h | 3 + drivers/gpu/drm/msm/msm_submitqueue.c | 57 +++++++++--- include/uapi/drm/msm_drm.h | 9 +- 7 files changed, 287 insertions(+), 30 deletions(-)
drivers/gpu/drm/msm/msm_gem.c +1 -2
@@ -232,8 +232,7 @@ static void put_pages(struct drm_gem_object *obj) } } -static struct page **msm_gem_get_pages_locked (struct drm_gem_object *obj, -unsigned madv) +struct page **msm_gem_get_pages_locked (struct drm_gem_object *obj,unsigned madv) {struct msm_gem_object *msm_obj =to_msm_bo (obj);
drivers/gpu/drm/msm/msm_gem.h +10 -0
@@ -53,6 +53,13 @@ struct msm_gem_vm {/** @base: Inherit from drm_gpuvm. */ struct drm_gpuvm base; +/** +* @sched: Scheduler used for asynchronous VM_BIND request. +* +* Unused for kernel managed VMs (where all operations are synchronous). +*/ +struct drm_gpu_scheduler sched; +/** * @mm: Memory management for kernel managed VA allocations *
@@ -106,6 +113,8 @@ struct drm_gpuvm * msm_gem_vm_create(struct drm_device *drm, struct msm_mmu *mmu, const char *name, u64 va_start, u64 va_size, bool managed); +void msm_gem_vm_close (struct drm_gpuvm *gpuvm); +struct msm_fence_context ;/**
@@ -195,6 +204,7 @@ int msm_gem_get_and_pin_iova(struct drm_gem_object *obj, struct drm_gpuvm *vm, uint64_t *iova);void msm_gem_unpin_iova (struct drm_gem_object *obj,struct drm_gpuvm *vm);void msm_gem_pin_obj_locked (struct drm_gem_object *obj); +struct page **msm_gem_get_pages_locked (struct drm_gem_object *obj,unsigned madv);struct page **msm_gem_pin_pages_locked (struct drm_gem_object *obj);void msm_gem_unpin_pages_locked (struct drm_gem_object *obj); int msm_gem_dumb_create(struct drm_file *file, struct drm_device *dev,
drivers/gpu/drm/msm/msm_gem_submit.c +113 -15
242 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gem_vma.c +107 -0
127 lines not on this page · raw message
drivers/gpu/drm/msm/msm_gpu.h +3 -0
@@ -562,6 +562,9 @@ struct msm_gpu_submitqueue {struct mutex lock;struct kref ref;struct drm_sched_entity *entity; + +/** @_vm_bind_entity: used for @entity pointer for VM_BIND queues */ +struct drm_sched_entity _vm_bind_entity[0 ]; };struct msm_gpu_state_bo {
drivers/gpu/drm/msm/msm_submitqueue.c +45 -12
91 lines not on this page · raw message
include/uapi/drm/msm_drm.h +8 -1
@@ -404,12 +404,19 @@ struct drm_msm_gem_madvise {/* * Draw queues allow the user to set specific submission parameter. Command * submissions specify a specific submitqueue to use. ID 0 is reserved for -* backwards compatibility as a "default" submitqueue +* backwards compatibility as a "default" submitqueue. +* +* Because VM_BIND async updates happen on the CPU, they must run on a +* virtual queue created with the flag MSM_SUBMITQUEUE_VM_BIND. If we had +* a way to do pgtable updates on the GPU, we could drop this restriction. */ #define MSM_SUBMITQUEUE_ALLOW_PREEMPT 0x00000001 +#define MSM_SUBMITQUEUE_VM_BIND 0x00000002/* virtual queue for VM_BIND ops */ +#define MSM_SUBMITQUEUE_FLAGS ( \ MSM_SUBMITQUEUE_ALLOW_PREEMPT | \ + MSM_SUBMITQUEUE_VM_BIND | \ 0)/*
Signature · 1 line
2.48.1
PATCH v2 19/34 drm/msm: Add _NO_SHARE flag
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Buffers that are not shared between contexts can share a single resv object. This way drm_gpuvm will not track them as external objects, and submit-time validating overhead will be O(1) for all N non-shared BOs, instead of O(n).
--- drivers/gpu/drm/msm/msm_drv.h | 1 + drivers/gpu/drm/msm/msm_gem.c | 23 +++++++++++++++++++++++ drivers/gpu/drm/msm/msm_gem_prime.c | 15 +++++++++++++++ include/uapi/drm/msm_drm.h | 14 ++++++++++++++ 4 files changed, 53 insertions(+)
drivers/gpu/drm/msm/msm_drv.h +1 -0
@@ -246,6 +246,7 @@ int msm_gem_prime_vmap(struct drm_gem_object *obj, struct iosys_map *map);void msm_gem_prime_vunmap (struct drm_gem_object *obj,struct iosys_map *map);struct drm_gem_object *msm_gem_prime_import_sg_table (struct drm_device *dev,struct dma_buf_attachment *attach,struct sg_table *sg); +struct dma_buf *msm_gem_prime_export (struct drm_gem_object *obj,int flags);int msm_gem_prime_pin (struct drm_gem_object *obj);void msm_gem_prime_unpin (struct drm_gem_object *obj);
drivers/gpu/drm/msm/msm_gem.c +23 -0
@@ -522,6 +522,9 @@ static int get_and_pin_iova_range_locked(struct drm_gem_object *obj,msm_gem_assert_locked (obj); +if (to_msm_bo (obj)->flags &MSM_BO_NO_SHARE ) +return -EINVAL ; + vma =get_vma_locked (obj, vm, range_start, range_end);if (IS_ERR (vma))return PTR_ERR (vma);
@@ -1032,6 +1035,16 @@ static void msm_gem_free_object(struct drm_gem_object *obj)put_pages (obj); } +if (msm_obj->flags &MSM_BO_NO_SHARE ) { +struct drm_gem_object *r_obj = + container_of(obj->resv, struct drm_gem_object, _resv); + +BUG_ON (obj->resv == &obj->_resv); + +/* Drop reference we hold to shared resv obj: */ +drm_gem_object_put (r_obj); + } +drm_gem_object_release (obj);kfree (msm_obj->metadata);
@@ -1064,6 +1077,15 @@ int msm_gem_new_handle(struct drm_device *dev, struct drm_file *file,if (name)msm_gem_object_set_name (obj,"%s" , name); +if (flags &MSM_BO_NO_SHARE ) { +struct msm_context *ctx = file->driver_priv; +struct drm_gem_object *r_obj =drm_gpuvm_resv_obj (ctx->vm); + +drm_gem_object_get (r_obj); + + obj->resv = r_obj->resv; + } + ret =drm_gem_handle_create (file, obj, handle);/* drop reference from allocate - handle holds it now */
@@ -1096,6 +1118,7 @@ static const struct drm_gem_object_funcs msm_gem_object_funcs = { .free = msm_gem_free_object, .open = msm_gem_open, .close = msm_gem_close, + .export = msm_gem_prime_export, .pin = msm_gem_prime_pin, .unpin = msm_gem_prime_unpin, .get_sg_table = msm_gem_prime_get_sg_table,
drivers/gpu/drm/msm/msm_gem_prime.c +15 -0
@@ -16,6 +16,9 @@ struct sg_table *msm_gem_prime_get_sg_table(struct drm_gem_object *obj)struct msm_gem_object *msm_obj =to_msm_bo (obj);int npages = obj->size >>PAGE_SHIFT ; +if (msm_obj->flags &MSM_BO_NO_SHARE ) +return ERR_PTR (-EINVAL ); +if (WARN_ON (!msm_obj->pages))/* should have already pinned! */ return ERR_PTR (-ENOMEM );
@@ -45,6 +48,15 @@ struct drm_gem_object *msm_gem_prime_import_sg_table(struct drm_device *dev,return msm_gem_import (dev, attach->dmabuf, sg); } + +struct dma_buf *msm_gem_prime_export (struct drm_gem_object *obj,int flags) +{ +if (to_msm_bo (obj)->flags &MSM_BO_NO_SHARE ) +return ERR_PTR (-EPERM ); + +return drm_gem_prime_export (obj, flags); +} +int msm_gem_prime_pin (struct drm_gem_object *obj) {struct page **pages;
@@ -53,6 +65,9 @@ int msm_gem_prime_pin(struct drm_gem_object *obj)if (obj->import_attach)return 0 ; +if (to_msm_bo (obj)->flags &MSM_BO_NO_SHARE ) +return -EINVAL ; + pages =msm_gem_pin_pages_locked (obj);if (IS_ERR (pages)) ret =PTR_ERR (pages);
include/uapi/drm/msm_drm.h +14 -0
@@ -138,6 +138,19 @@ struct drm_msm_param {#define MSM_BO_SCANOUT 0x00000001/* scanout capable */ #define MSM_BO_GPU_READONLY 0x00000002 +/* Private buffers do not need to be explicitly listed in the SUBMIT +* ioctl, unless referenced by a drm_msm_gem_submit_cmd. Private +* buffers may NOT be imported/exported or used for scanout (or any +* other situation where buffers can be indefinitely pinned, but +* cases other than scanout are all kernel owned BOs which are not +* visible to userspace). +* +* In exchange for those constraints, all private BOs associated with +* a single context (drm_file) share a single dma_resv, and if there +* has been no eviction since the last submit, there are no per-BO +* bookeeping to do, significantly cutting the SUBMIT overhead. +*/ +#define MSM_BO_NO_SHARE 0x00000004#define MSM_BO_CACHE_MASK 0x000f0000/* cache modes */ #define MSM_BO_CACHED 0x00010000
@@ -147,6 +160,7 @@ struct drm_msm_param {#define MSM_BO_FLAGS (MSM_BO_SCANOUT | \ MSM_BO_GPU_READONLY | \ + MSM_BO_NO_SHARE | \ MSM_BO_CACHE_MASK)struct drm_msm_gem_new {
Signature · 1 line
2.48.1
PATCH v2 20/34 drm/msm: Split out helper to get iommu prot flags
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
We'll re-use this in the vm_bind path.
--- drivers/gpu/drm/msm/msm_gem.c | 12 ++++++++++-- drivers/gpu/drm/msm/msm_gem.h | 1 + 2 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/msm/msm_gem.c +10 -2
@@ -450,10 +450,9 @@ static struct drm_gpuva *get_vma_locked(struct drm_gem_object *obj,return vma; } -int msm_gem_pin_vma_locked (struct drm_gem_object *obj,struct drm_gpuva *vma) +int msm_gem_prot (struct drm_gem_object *obj) {struct msm_gem_object *msm_obj =to_msm_bo (obj); -struct page **pages;int prot =IOMMU_READ ;if (!(msm_obj->flags &MSM_BO_GPU_READONLY ))
@@ -469,6 +468,15 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct drm_gpuva *vma)else if (prot ==2 ) prot |=IOMMU_USE_LLC_NWA ; +return prot; +} + +int msm_gem_pin_vma_locked (struct drm_gem_object *obj,struct drm_gpuva *vma) +{ +struct msm_gem_object *msm_obj =to_msm_bo (obj); +struct page **pages; +int prot =msm_gem_prot (obj); +msm_gem_assert_locked (obj); pages =msm_gem_get_pages_locked (obj,MSM_MADV_WILLNEED );
drivers/gpu/drm/msm/msm_gem.h +1 -0
@@ -188,6 +188,7 @@ struct msm_gem_object {#define to_msm_bo (x) container_of(x, struct msm_gem_object, base)uint64_t msm_gem_mmap_offset (struct drm_gem_object *obj); +int msm_gem_prot (struct drm_gem_object *obj);int msm_gem_pin_vma_locked (struct drm_gem_object *obj,struct drm_gpuva *vma);void msm_gem_unpin_locked (struct drm_gem_object *obj);void msm_gem_unpin_active (struct drm_gem_object *obj);
Signature · 1 line
2.48.1
PATCH v2 21/34 drm/msm: Add mmu support for non-zero offset
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Only needs to be supported for iopgtables mmu, the other cases are either only used for kernel managed mappings (where offset is always zero) or devices which do not support sparse bindings.
--- drivers/gpu/drm/msm/adreno/a2xx_gpummu.c | 5 ++++- drivers/gpu/drm/msm/msm_gem.c | 4 ++-- drivers/gpu/drm/msm/msm_gem.h | 4 ++-- drivers/gpu/drm/msm/msm_gem_vma.c | 13 +++++++------ drivers/gpu/drm/msm/msm_iommu.c | 22 ++++++++++++++++++++-- drivers/gpu/drm/msm/msm_mmu.h | 2 +- 6 files changed, 36 insertions(+), 14 deletions(-)
drivers/gpu/drm/msm/adreno/a2xx_gpummu.c +4 -1
@@ -29,13 +29,16 @@ static void a2xx_gpummu_detach(struct msm_mmu *mmu) }static int a2xx_gpummu_map (struct msm_mmu *mmu,uint64_t iova, -struct sg_table *sgt,size_t len,int prot) +struct sg_table *sgt,size_t off,size_t len, +int prot) {struct a2xx_gpummu *gpummu =to_a2xx_gpummu (mmu);unsigned idx = (iova -GPUMMU_VA_START ) /GPUMMU_PAGE_SIZE ;struct sg_dma_page_iter dma_iter;unsigned prot_bits =0 ; +WARN_ON (off !=0 ); +if (prot &IOMMU_WRITE ) prot_bits |=1 ;if (prot &IOMMU_READ )
drivers/gpu/drm/msm/msm_gem.c +2 -2
@@ -441,7 +441,7 @@ static struct drm_gpuva *get_vma_locked(struct drm_gem_object *obj, vma =lookup_vma (obj, vm);if (!vma) { - vma =msm_gem_vma_new (vm, obj, range_start, range_end); + vma =msm_gem_vma_new (vm, obj,0 , range_start, range_end); }else {GEM_WARN_ON (vma->va.addr < range_start);GEM_WARN_ON ((vma->va.addr + obj->size) > range_end);
@@ -483,7 +483,7 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct drm_gpuva *vma)if (IS_ERR (pages))return PTR_ERR (pages); -return msm_gem_vma_map (vma, prot, msm_obj->sgt, obj->size); +return msm_gem_vma_map (vma, prot, msm_obj->sgt); }void msm_gem_unpin_locked (struct drm_gem_object *obj)
drivers/gpu/drm/msm/msm_gem.h +2 -2
@@ -140,9 +140,9 @@ struct msm_gem_vma {struct drm_gpuva *msm_gem_vma_new (struct drm_gpuvm *vm,struct drm_gem_object *obj, -u64 range_start,u64 range_end); +u64 offset,u64 range_start,u64 range_end);void msm_gem_vma_purge (struct drm_gpuva *vma); -int msm_gem_vma_map (struct drm_gpuva *vma,int prot,struct sg_table *sgt,int size); +int msm_gem_vma_map (struct drm_gpuva *vma,int prot,struct sg_table *sgt);void msm_gem_vma_close (struct drm_gpuva *vma);struct msm_gem_object {
drivers/gpu/drm/msm/msm_gem_vma.c +7 -6
@@ -38,8 +38,7 @@ void msm_gem_vma_purge(struct drm_gpuva *vma)/* Map and pin vma: */ int -msm_gem_vma_map (struct drm_gpuva *vma,int prot, -struct sg_table *sgt,int size) +msm_gem_vma_map (struct drm_gpuva *vma,int prot,struct sg_table *sgt) {struct msm_gem_vma *msm_vma =to_msm_vma (vma);struct msm_gem_vm *vm =to_msm_vm (vma->vm);
@@ -62,8 +61,9 @@ msm_gem_vma_map(struct drm_gpuva *vma, int prot,* Revisit this if we can come up with a scheme to pre-alloc pages * for the pgtable in map/unmap ops. */ - ret = vm->mmu->funcs->map (vm->mmu, vma->va.addr, sgt, size, prot); - + ret = vm->mmu->funcs->map (vm->mmu, vma->va.addr, sgt, + vma->gem.offset, vma->va.range, + prot);if (ret) { msm_vma->mapped = false; }
@@ -97,7 +97,7 @@ void msm_gem_vma_close(struct drm_gpuva *vma)/* Create a new vma and allocate an iova for it */ struct drm_gpuva *msm_gem_vma_new (struct drm_gpuvm *_vm,struct drm_gem_object *obj, -u64 range_start,u64 range_end) +u64 offset,u64 range_start,u64 range_end) {struct msm_gem_vm *vm =to_msm_vm (_vm);struct drm_gpuvm_bo *vm_bo;
@@ -109,6 +109,7 @@ msm_gem_vma_new(struct drm_gpuvm *_vm, struct drm_gem_object *obj,return ERR_PTR (-ENOMEM );if (vm->managed) { +BUG_ON (offset !=0 );spin_lock (&vm->mm_lock); ret = drm_mm_insert_node_in_range(&vm->mm, &vma->node, obj->size, PAGE_SIZE, 0,
@@ -124,7 +125,7 @@ msm_gem_vma_new(struct drm_gpuvm *_vm, struct drm_gem_object *obj,GEM_WARN_ON ((range_end - range_start) > obj->size); -drm_gpuva_init (&vma->base, range_start, range_end - range_start, obj,0 ); +drm_gpuva_init (&vma->base, range_start, range_end - range_start, obj, offset); vma->mapped = false;mutex_lock (&vm->vm_lock);
drivers/gpu/drm/msm/msm_iommu.c +20 -2
@@ -113,7 +113,8 @@ static int msm_iommu_pagetable_unmap(struct msm_mmu *mmu, u64 iova, }static int msm_iommu_pagetable_map (struct msm_mmu *mmu,u64 iova, -struct sg_table *sgt,size_t len,int prot) +struct sg_table *sgt,size_t off,size_t len, +int prot) {struct msm_iommu_pagetable *pagetable =to_pagetable (mmu);struct io_pgtable_ops *ops = pagetable->pgtbl_ops;
@@ -125,6 +126,19 @@ static int msm_iommu_pagetable_map(struct msm_mmu *mmu, u64 iova,size_t size = sg->length;phys_addr_t phys =sg_phys (sg); +if (!len) +break ; + +if (size <= off) { + off -= size; +continue ; + } + + phys += off; + size -= off; + size =min_t (size_t, size, len); + off =0 ; +while (size) {size_t pgsize, count, mapped =0 ;int ret;
@@ -140,6 +154,7 @@ static int msm_iommu_pagetable_map(struct msm_mmu *mmu, u64 iova, phys += mapped; addr += mapped; size -= mapped; + len -= mapped;if (ret) {msm_iommu_pagetable_unmap (mmu, iova, addr - iova);
@@ -400,11 +415,14 @@ static void msm_iommu_detach(struct msm_mmu *mmu) }static int msm_iommu_map (struct msm_mmu *mmu,uint64_t iova, -struct sg_table *sgt,size_t len,int prot) +struct sg_table *sgt,size_t off,size_t len, +int prot) {struct msm_iommu *iommu =to_msm_iommu (mmu);size_t ret; +WARN_ON (off !=0 ); +/* The arm-smmu driver expects the addresses to be sign extended */ if (iova &BIT_ULL (48 )) iova |=GENMASK_ULL (63 ,49 );
drivers/gpu/drm/msm/msm_mmu.h +1 -1
@@ -12,7 +12,7 @@struct msm_mmu_funcs {void (*detach)(struct msm_mmu *mmu);int (*map)(struct msm_mmu *mmu,uint64_t iova,struct sg_table *sgt, -size_t len,int prot); +size_t off,size_t len,int prot);int (*unmap)(struct msm_mmu *mmu,uint64_t iova,size_t len);void (*destroy)(struct msm_mmu *mmu);void (*resume_translation)(struct msm_mmu *mmu);
Signature · 1 line
2.48.1
PATCH v2 22/34 drm/msm: Add PRR support
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Add PRR (Partial Resident Region) is a bypass address which make GPU writes go to /dev/null and reads return zero. This is used to implement vulkan sparse residency.
To support PRR/NULL mappings, we allocate a page to reserve a physical address which we know will not be used as part of a GEM object, and configure the SMMU to use this address for PRR/NULL mappings.
--- drivers/gpu/drm/msm/adreno/adreno_gpu.c | 10 ++++ drivers/gpu/drm/msm/msm_iommu.c | 62 ++++++++++++++++++++++++- include/uapi/drm/msm_drm.h | 2 + 3 files changed, 73 insertions(+), 1 deletion(-)
drivers/gpu/drm/msm/adreno/adreno_gpu.c +10 -0
@@ -346,6 +346,13 @@ int adreno_fault_handler(struct msm_gpu *gpu, unsigned long iova, int flags,return 0 ; } +static bool +adreno_smmu_has_prr (struct msm_gpu *gpu) +{ +struct adreno_smmu_priv *adreno_smmu =dev_get_drvdata (&gpu->pdev->dev); +return adreno_smmu && adreno_smmu->set_prr_addr; +} +int adreno_get_param (struct msm_gpu *gpu,struct msm_context *ctx,uint32_t param,uint64_t *value,uint32_t *len) {
@@ -431,6 +438,9 @@ int adreno_get_param(struct msm_gpu *gpu, struct msm_context *ctx,case MSM_PARAM_UCHE_TRAP_BASE : *value = adreno_gpu->uche_trap_base;return 0 ; +case MSM_PARAM_HAS_PRR : + *value =adreno_smmu_has_prr (gpu); +return 0 ;default :return UERR (EINVAL , drm,"%s: invalid param: %u" , gpu->name, param); }
drivers/gpu/drm/msm/msm_iommu.c +61 -1
94 lines not on this page · raw message
include/uapi/drm/msm_drm.h +2 -0
@@ -115,6 +115,8 @@ struct drm_msm_timespec {* ioctl will throw -EPIPE. */ #define MSM_PARAM_EN_VM_BIND 0x15/* WO, once */ +/* PRR (Partially Resident Region) is required for sparse residency: */ +#define MSM_PARAM_HAS_PRR 0x16/* RO */ /* For backwards compat. The original support for preemption was based on * a single ring per priority level so # of priority levels equals the #
Signature · 1 line
2.48.1
PATCH v2 23/34 drm/msm: Rename msm_gem_vma_purge() -> _unmap()
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
This is a more descriptive name.
--- drivers/gpu/drm/msm/msm_gem.c | 4 ++-- drivers/gpu/drm/msm/msm_gem.h | 2 +- drivers/gpu/drm/msm/msm_gem_vma.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
drivers/gpu/drm/msm/msm_gem.c +2 -2
@@ -421,7 +421,7 @@ put_iova_spaces(struct drm_gem_object *obj, struct drm_gpuvm *vm, bool close)drm_gpuvm_bo_get (vm_bo); drm_gpuvm_bo_for_each_va_safe (vma, vmatmp, vm_bo) { -msm_gem_vma_purge (vma); +msm_gem_vma_unmap (vma);if (close)msm_gem_vma_close (vma); }
@@ -600,7 +600,7 @@ static int clear_iova(struct drm_gem_object *obj,if (!vma)return 0 ; -msm_gem_vma_purge (vma); +msm_gem_vma_unmap (vma);msm_gem_vma_close (vma);return 0 ;
drivers/gpu/drm/msm/msm_gem.h +1 -1
@@ -141,7 +141,7 @@ struct msm_gem_vma {struct drm_gpuva *msm_gem_vma_new (struct drm_gpuvm *vm,struct drm_gem_object *obj,u64 offset,u64 range_start,u64 range_end); -void msm_gem_vma_purge (struct drm_gpuva *vma); +void msm_gem_vma_unmap (struct drm_gpuva *vma);int msm_gem_vma_map (struct drm_gpuva *vma,int prot,struct sg_table *sgt);void msm_gem_vma_close (struct drm_gpuva *vma);
drivers/gpu/drm/msm/msm_gem_vma.c +1 -1
@@ -21,7 +21,7 @@ msm_gem_vm_free(struct drm_gpuvm *gpuvm) }/* Actually unmap memory for the vma */ -void msm_gem_vma_purge (struct drm_gpuva *vma) +void msm_gem_vma_unmap (struct drm_gpuva *vma) {struct msm_gem_vma *msm_vma =to_msm_vma (vma);struct msm_gem_vm *vm =to_msm_vm (vma->vm);
Signature · 1 line
2.48.1
PATCH v2 24/34 drm/msm: Split msm_gem_vma_new()
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Split memory allocation from vma initialization. Async vm-bind happens in the fence signalling path, so it will need to use pre-allocated memory.
--- drivers/gpu/drm/msm/msm_gem_vma.c | 67 ++++++++++++++++++++++--------- 1 file changed, 49 insertions(+), 18 deletions(-)
drivers/gpu/drm/msm/msm_gem_vma.c +49 -18
@@ -71,40 +71,54 @@ msm_gem_vma_map(struct drm_gpuva *vma, int prot, struct sg_table *sgt)return ret; } -/* Close an iova. Warn if it is still in use */ -void msm_gem_vma_close (struct drm_gpuva *vma) +static void __vma_close (struct drm_gpuva *vma) {struct msm_gem_vm *vm =to_msm_vm (vma->vm);struct msm_gem_vma *msm_vma =to_msm_vma (vma);GEM_WARN_ON (msm_vma->mapped); +GEM_WARN_ON (!mutex_is_locked (&vm->vm_lock));spin_lock (&vm->mm_lock);if (vma->va.addr && vm->managed)drm_mm_remove_node (&msm_vma->node);spin_unlock (&vm->mm_lock); -dma_resv_lock (drm_gpuvm_resv (vma->vm),NULL ); -mutex_lock (&vm->vm_lock);drm_gpuva_remove (vma);drm_gpuva_unlink (vma); -mutex_unlock (&vm->vm_lock); -dma_resv_unlock (drm_gpuvm_resv (vma->vm));kfree (vma); } -/* Create a new vma and allocate an iova for it */ -struct drm_gpuva * -msm_gem_vma_new (struct drm_gpuvm *_vm,struct drm_gem_object *obj, -u64 offset,u64 range_start,u64 range_end) +/* Close an iova. Warn if it is still in use */ +void msm_gem_vma_close (struct drm_gpuva *vma) +{ +struct msm_gem_vm *vm =to_msm_vm (vma->vm); + +/* +* Only used in legacy (kernel managed) VM, if userspace is managing +* the VM, the legacy paths should be disallowed: +*/ +GEM_WARN_ON (!vm->managed); + +dma_resv_lock (drm_gpuvm_resv (vma->vm),NULL ); +mutex_lock (&vm->vm_lock); +__vma_close (vma); +mutex_unlock (&vm->vm_lock); +dma_resv_unlock (drm_gpuvm_resv (vma->vm)); +} + +static struct drm_gpuva * +__vma_init (struct msm_gem_vma *vma,struct drm_gpuvm *_vm, +struct drm_gem_object *obj,u64 offset, +u64 range_start,u64 range_end) {struct msm_gem_vm *vm =to_msm_vm (_vm);struct drm_gpuvm_bo *vm_bo; -struct msm_gem_vma *vma;int ret; - vma =kzalloc (sizeof (*vma),GFP_KERNEL ); +GEM_WARN_ON (!mutex_is_locked (&vm->vm_lock)); +if (!vma)return ERR_PTR (-ENOMEM );
@@ -128,9 +142,7 @@ msm_gem_vma_new(struct drm_gpuvm *_vm, struct drm_gem_object *obj,drm_gpuva_init (&vma->base, range_start, range_end - range_start, obj, offset); vma->mapped = false; -mutex_lock (&vm->vm_lock); ret =drm_gpuva_insert (&vm->base, &vma->base); -mutex_unlock (&vm->vm_lock);if (ret) goto err_free_range;
@@ -140,17 +152,13 @@ msm_gem_vma_new(struct drm_gpuvm *_vm, struct drm_gem_object *obj, goto err_va_remove; } -mutex_lock (&vm->vm_lock);drm_gpuva_link (&vma->base, vm_bo); -mutex_unlock (&vm->vm_lock);GEM_WARN_ON (drm_gpuvm_bo_put (vm_bo));return &vma->base; err_va_remove: -mutex_lock (&vm->vm_lock);drm_gpuva_remove (&vma->base); -mutex_unlock (&vm->vm_lock); err_free_range:if (vm->managed)drm_mm_remove_node (&vma->node);
@@ -159,6 +167,29 @@ msm_gem_vma_new(struct drm_gpuvm *_vm, struct drm_gem_object *obj,return ERR_PTR (ret); } +/* Create a new vma and allocate an iova for it */ +struct drm_gpuva * +msm_gem_vma_new (struct drm_gpuvm *_vm,struct drm_gem_object *obj, +u64 offset,u64 range_start,u64 range_end) +{ +struct msm_gem_vm *vm =to_msm_vm (_vm); +struct msm_gem_vma *vma; + +/* +* Only used in legacy (kernel managed) VM, if userspace is managing +* the VM, the legacy paths should be disallowed: +*/ +GEM_WARN_ON (!vm->managed); + + vma =kzalloc (sizeof (*vma),GFP_KERNEL ); + +mutex_lock (&vm->vm_lock); +__vma_init (vma, _vm, obj, offset, range_start, range_end); +mutex_unlock (&vm->vm_lock); + +return &vma->base; +} +static const struct drm_gpuvm_ops msm_gpuvm_ops = { .vm_free = msm_gem_vm_free, };
Signature · 1 line
2.48.1
PATCH v2 25/34 drm/msm: Pre-allocate VMAs
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Pre-allocate the VMA objects that we will need in the vm bind job.
--- drivers/gpu/drm/msm/msm_gem.h | 9 +++++ drivers/gpu/drm/msm/msm_gem_submit.c | 5 +++ drivers/gpu/drm/msm/msm_gem_vma.c | 60 ++++++++++++++++++++++++++++ 3 files changed, 74 insertions(+)
drivers/gpu/drm/msm/msm_gem.h +9 -0
@@ -115,6 +115,9 @@ msm_gem_vm_create(struct drm_device *drm, struct msm_mmu *mmu, const char *name,void msm_gem_vm_close (struct drm_gpuvm *gpuvm); +void msm_vma_job_prepare (struct msm_gem_submit *submit); +void msm_vma_job_cleanup (struct msm_gem_submit *submit); +struct msm_fence_context ;/**
@@ -339,6 +342,12 @@ struct msm_gem_submit {int fence_id;/* key into queue->fence_idr */ struct msm_gpu_submitqueue *queue; + +/* List of pre-allocated msm_gem_vma's, used to avoid memory allocation +* in fence signalling path. +*/ +struct list_head preallocated_vmas; +struct pid *pid;/* submitting process */ bool bos_pinned : 1; bool fault_dumped:1;/* Limit devcoredump dumping to one per submit */
drivers/gpu/drm/msm/msm_gem_submit.c +5 -0
@@ -80,6 +80,7 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev, submit->ident =atomic_inc_return (&ident) -1 ;INIT_LIST_HEAD (&submit->node); +INIT_LIST_HEAD (&submit->preallocated_vmas);return submit; }
@@ -584,6 +585,9 @@ void msm_submit_retire(struct msm_gem_submit *submit) {int i; +if (submit_is_vmbind (submit)) +msm_vma_job_cleanup (submit); +for (i =0 ; i < submit->nr_bos; i++) {struct drm_gem_object *obj = submit->bos[i].obj;
@@ -912,6 +916,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, }if (submit_is_vmbind (submit)) { +msm_vma_job_prepare (submit); ret =submit_get_pages (submit); }else { ret =submit_pin_vmas (submit);
drivers/gpu/drm/msm/msm_gem_vma.c +60 -0
66 lines not on this page · raw message
Signature · 1 line
2.48.1
PATCH v2 26/34 drm/msm: Pre-allocate vm_bo objects
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Use drm_gpuvm_bo_obtain() in the synchronous part of the VM_BIND submit, to hold a reference to the vm_bo for the duration of the submit. This ensures that the vm_bo already exists before the async part of the job, which is in the fence signalling path (and therefore cannot allocate memory).
--- drivers/gpu/drm/msm/msm_gem.h | 1 + drivers/gpu/drm/msm/msm_gem_vma.c | 19 +++++++++++++++++-- 2 files changed, 18 insertions(+), 2 deletions(-)
drivers/gpu/drm/msm/msm_gem.h +1 -0
@@ -369,6 +369,7 @@ struct msm_gem_submit {uint32_t flags;uint32_t handle;struct drm_gem_object *obj; +struct drm_gpuvm_bo *vm_bo;uint64_t iova;uint64_t bo_offset;uint64_t range;
drivers/gpu/drm/msm/msm_gem_vma.c +17 -2
@@ -278,8 +278,18 @@ msm_vma_job_prepare(struct msm_gem_submit *submit)for (int i =0 ; i < submit->nr_bos; i++) {unsigned op = submit->bos[i].flags &MSM_SUBMIT_BO_OP_MASK ; -if (submit->bos[i].obj) -msm_gem_assert_locked (submit->bos[i].obj); +if (submit->bos[i].obj) { +struct drm_gem_object *obj = submit->bos[i].obj; + +msm_gem_assert_locked (obj); + +/* +* Ensure the vm_bo is already allocated by +* holding a ref until the submit is retired +*/ + submit->bos[i].vm_bo = +drm_gpuvm_bo_obtain (submit->vm, obj); + }/* * OP_MAP/OP_MAP_NULL has one new VMA for the new mapping,
@@ -309,6 +319,11 @@ msm_vma_job_cleanup(struct msm_gem_submit *submit) {struct drm_gpuva *vma; +for (int i =0 ; i < submit->nr_bos; i++) { +/* If we're holding an extra ref to the vm_bo, drop it now: */ +drm_gpuvm_bo_put (submit->bos[i].vm_bo); + } +while (!list_empty (&submit->preallocated_vmas)) { vma = list_first_entry(&submit->preallocated_vmas, struct drm_gpuva,
Signature · 1 line
2.48.1
PATCH v2 27/34 drm/msm: Pre-allocate pages for pgtable entries
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Introduce a mechanism to count the worst case # of pages required in a VM_BIND op.
Note that previously we would have had to somehow account for allocations in unmap, when splitting a block. This behavior was removed in commit 33729a5fc0ca ("iommu/io-pgtable-arm: Remove split on unmap behavior)"
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +- drivers/gpu/drm/msm/msm_gem.h | 7 +- drivers/gpu/drm/msm/msm_gem_submit.c | 5 +- drivers/gpu/drm/msm/msm_gem_vma.c | 18 ++- drivers/gpu/drm/msm/msm_iommu.c | 201 +++++++++++++++++++++++++- drivers/gpu/drm/msm/msm_mmu.h | 36 ++++- 6 files changed, 260 insertions(+), 9 deletions(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -1
@@ -2267,7 +2267,7 @@ a6xx_create_private_vm(struct msm_gpu *gpu, bool kernel_managed) {struct msm_mmu *mmu; - mmu =msm_iommu_pagetable_create (to_msm_vm (gpu->vm)->mmu); + mmu =msm_iommu_pagetable_create (to_msm_vm (gpu->vm)->mmu, kernel_managed);if (IS_ERR (mmu))return ERR_CAST (mmu);
drivers/gpu/drm/msm/msm_gem.h +6 -1
@@ -7,6 +7,7 @@#ifndef __MSM_GEM_H__#define __MSM_GEM_H__ +#include "msm_mmu.h" #include <linux/kref.h> #include <linux/dma-resv.h> #include "drm/drm_exec.h"
@@ -115,7 +116,7 @@ msm_gem_vm_create(struct drm_device *drm, struct msm_mmu *mmu, const char *name,void msm_gem_vm_close (struct drm_gpuvm *gpuvm); -void msm_vma_job_prepare (struct msm_gem_submit *submit); +int msm_vma_job_prepare (struct msm_gem_submit *submit);void msm_vma_job_cleanup (struct msm_gem_submit *submit);struct msm_fence_context ;
@@ -348,6 +349,10 @@ struct msm_gem_submit {*/ struct list_head preallocated_vmas; +/* Tracking for pre-allocated pgtable pages. +*/ +struct msm_mmu_prealloc prealloc; +struct pid *pid;/* submitting process */ bool bos_pinned : 1; bool fault_dumped:1;/* Limit devcoredump dumping to one per submit */
drivers/gpu/drm/msm/msm_gem_submit.c +3 -2
@@ -916,8 +916,9 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data, }if (submit_is_vmbind (submit)) { -msm_vma_job_prepare (submit); - ret =submit_get_pages (submit); + ret =msm_vma_job_prepare (submit); +if (!ret) + ret =submit_get_pages (submit); }else { ret =submit_pin_vmas (submit); }
drivers/gpu/drm/msm/msm_gem_vma.c +16 -2
@@ -270,9 +270,10 @@ static const struct drm_sched_backend_ops msm_vm_bind_ops = {* * Called after BOs are locked. */ -void +int msm_vma_job_prepare (struct msm_gem_submit *submit) { +struct msm_mmu *mmu =to_msm_vm (submit->vm)->mmu;unsigned num_prealloc_vmas =0 ;for (int i =0 ; i < submit->nr_bos; i++) {
@@ -299,13 +300,23 @@ msm_vma_job_prepare(struct msm_gem_submit *submit)* OP_UNMAP could trigger a remap with either a prev or * next VMA, but not both. */ - num_prealloc_vmas += (op ==MSM_SUBMIT_BO_OP_UNMAP ) ?1 :3 ; +if (op !=MSM_SUBMIT_BO_OP_UNMAP ) { + num_prealloc_vmas +=3 ; + + mmu->funcs->prealloc_count (mmu, &submit->prealloc, + submit->bos[i].iova, + submit->bos[i].range); + }else { + num_prealloc_vmas +=1 ; + } }while (num_prealloc_vmas-- >0 ) {struct msm_gem_vma *vma =kzalloc (sizeof (*vma),GFP_KERNEL );list_add_tail (&vma->base.rb.entry, &submit->preallocated_vmas); } + +return mmu->funcs->prealloc_allocate (mmu, &submit->prealloc); }/**
@@ -317,6 +328,7 @@ msm_vma_job_prepare(struct msm_gem_submit *submit)void msm_vma_job_cleanup (struct msm_gem_submit *submit) { +struct msm_mmu *mmu =to_msm_vm (submit->vm)->mmu;struct drm_gpuva *vma;for (int i =0 ; i < submit->nr_bos; i++) {
@@ -331,6 +343,8 @@ msm_vma_job_cleanup(struct msm_gem_submit *submit)list_del (&vma->rb.entry);kfree (to_msm_vma (vma)); } + + mmu->funcs->prealloc_cleanup (mmu, &submit->prealloc); }/**
drivers/gpu/drm/msm/msm_iommu.c +199 -2
256 lines not on this page · raw message
drivers/gpu/drm/msm/msm_mmu.h +35 -1
@@ -9,8 +9,16 @@#include <linux/iommu.h> +struct msm_mmu_prealloc ; +struct msm_mmu ; +struct msm_gpu ; +struct msm_mmu_funcs {void (*detach)(struct msm_mmu *mmu); +void (*prealloc_count)(struct msm_mmu *mmu,struct msm_mmu_prealloc *p, +uint64_t iova,size_t len); +int (*prealloc_allocate)(struct msm_mmu *mmu,struct msm_mmu_prealloc *p); +void (*prealloc_cleanup)(struct msm_mmu *mmu,struct msm_mmu_prealloc *p);int (*map)(struct msm_mmu *mmu,uint64_t iova,struct sg_table *sgt,size_t off,size_t len,int prot);int (*unmap)(struct msm_mmu *mmu,uint64_t iova,size_t len);
@@ -25,12 +33,38 @@ enum msm_mmu_type { MSM_MMU_IOMMU_PAGETABLE, }; +/** +* struct msm_mmu_prealloc - Tracking for pre-allocated pages for MMU updates. +*/ +struct msm_mmu_prealloc { +/** @count: Number of pages reserved. */ +uint32_t count; +/** @ptr: Index of first unused page in @pages */ +uint32_t ptr; +/** +* @pages: Array of pages preallocated for MMU table updates. +* +* After a VM operation, there might be free pages remaining in this +* array (since the amount allocated is a worst-case). These are +* returned to the pt_cache at mmu->prealloc_cleanup(). +*/ +void **pages; +}; +struct msm_mmu {const struct msm_mmu_funcs *funcs;struct device *dev;int (*handler)(void *arg,unsigned long iova,int flags,void *data);void *arg;enum msm_mmu_type type; + +/** +* @prealloc: pre-allocated pages for pgtable +* +* Set while a VM_BIND job is running, serialized under +* msm_gem_vm::op_lock. +*/ +struct msm_mmu_prealloc *prealloc; }; static inline void msm_mmu_init(struct msm_mmu *mmu, struct device *dev,
@@ -52,7 +86,7 @@ static inline void msm_mmu_set_fault_handler(struct msm_mmu *mmu, void *arg, mmu->handler = handler; } -struct msm_mmu *msm_iommu_pagetable_create (struct msm_mmu *parent); +struct msm_mmu *msm_iommu_pagetable_create (struct msm_mmu *parent,bool kernel_managed);int msm_iommu_pagetable_params (struct msm_mmu *mmu,phys_addr_t *ttbr,int *asid);
Signature · 1 line
2.48.1
PATCH v2 28/34 drm/msm: Wire up gpuvm ops
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Hook up the map/remap/unmap ops to apply MAP/UNMAP operations. The MAP/UNMAP operations are split up by drm_gpuvm into a series of map/ remap/unmap ops, for example an UNMAP operation which spans multiple vmas will get split up into a sequence of unmap (and possibly remap) ops which each apply to a single vma.
--- drivers/gpu/drm/msm/msm_gem.h | 12 ++ drivers/gpu/drm/msm/msm_gem_vma.c | 329 ++++++++++++++++++++++++++++-- 2 files changed, 320 insertions(+), 21 deletions(-)
drivers/gpu/drm/msm/msm_gem.h +12 -0
@@ -76,6 +76,16 @@ struct msm_gem_vm {/** @vm_lock: protects gpuvm insert/remove/traverse */ struct mutex vm_lock; +/** +* @op_lock: +* +* Serializes VM operations. Typically operations are serialized +* by virtue of running on the VM_BIND queue, but in the cleanup +* path (or if multiple VM_BIND queues) the @op_lock provides the +* needed serialization. +*/ +struct mutex op_lock; +/** @mmu: The mmu object which manages the pgtables */ struct msm_mmu *mmu;
@@ -121,6 +131,8 @@ void msm_vma_job_cleanup(struct msm_gem_submit *submit);struct msm_fence_context ; +#define MSM_VMA_DUMP (DRM_GPUVA_USERBITS << 0) +/** * struct msm_gem_vma - a VMA mapping *
drivers/gpu/drm/msm/msm_gem_vma.c +308 -21
441 lines not on this page · raw message
Signature · 1 line
2.48.1
PATCH v2 29/34 drm/msm: Wire up drm_gpuvm debugfs
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Core drm already provides a helper to dump vm state. We just need to wire up tracking of VMs and giving userspace VMs a suitable name.
--- drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 2 +- drivers/gpu/drm/msm/msm_debugfs.c | 20 ++++++++++++++++++++ drivers/gpu/drm/msm/msm_drv.c | 3 +++ drivers/gpu/drm/msm/msm_drv.h | 4 ++++ drivers/gpu/drm/msm/msm_gem.h | 8 ++++++++ drivers/gpu/drm/msm/msm_gem_vma.c | 23 +++++++++++++++++++++++ 6 files changed, 59 insertions(+), 1 deletion(-)
drivers/gpu/drm/msm/adreno/a6xx_gpu.c +1 -1
@@ -2272,7 +2272,7 @@ a6xx_create_private_vm(struct msm_gpu *gpu, bool kernel_managed)if (IS_ERR (mmu))return ERR_CAST (mmu); -return msm_gem_vm_create (gpu->dev, mmu,"gpu" ,0x100000000ULL , +return msm_gem_vm_create (gpu->dev, mmu,NULL ,0x100000000ULL ,adreno_private_vm_size (gpu), kernel_managed); }
drivers/gpu/drm/msm/msm_debugfs.c +20 -0
@@ -10,6 +10,7 @@#include <linux/fault-inject.h> #include <drm/drm_debugfs.h> +#include <drm/drm_drv.h> #include <drm/drm_fb_helper.h> #include <drm/drm_file.h> #include <drm/drm_framebuffer.h>
@@ -238,6 +239,24 @@ static int msm_mm_show(struct seq_file *m, void *arg)return 0 ; } +static int msm_gpuvas_show (struct seq_file *m,void *arg) +{ +struct drm_info_node *node = m->private; +struct drm_device *dev = node->minor->dev; +struct msm_drm_private *priv = dev->dev_private; +struct msm_gem_vm *vm; + +mutex_lock (&priv->vm_lock); +list_for_each_entry (vm, &priv->vms, node) { +mutex_lock (&vm->op_lock); +drm_debugfs_gpuva_info (m, &vm->base); +mutex_unlock (&vm->op_lock); + } +mutex_unlock (&priv->vm_lock); + +return 0 ; +} +static int msm_fb_show (struct seq_file *m,void *arg) {struct drm_info_node *node = m->private;
@@ -266,6 +285,7 @@ static int msm_fb_show(struct seq_file *m, void *arg)static struct drm_info_list msm_debugfs_list[] = { {"gem" , msm_gem_show}, {"mm" , msm_mm_show }, +DRM_DEBUGFS_GPUVA_INFO (msm_gpuvas_show,NULL ), };static struct drm_info_list msm_kms_debugfs_list[] = {
drivers/gpu/drm/msm/msm_drv.c +3 -0
@@ -124,6 +124,9 @@ static int msm_drm_init(struct device *dev, const struct drm_driver *drv) goto err_put_dev; } +INIT_LIST_HEAD (&priv->vms); +mutex_init (&priv->vm_lock); +INIT_LIST_HEAD (&priv->objects);mutex_init (&priv->obj_lock);
drivers/gpu/drm/msm/msm_drv.h +4 -0
@@ -112,6 +112,10 @@ struct msm_drm_private {*/ atomic64_t total_mem; +/** @vms: List of all VMs, protected by @vm_lock */ +struct list_head vms; +struct mutex vm_lock; +/** * List of all GEM objects (mainly for debugfs, protected by obj_lock * (acquire before per GEM object lock)
drivers/gpu/drm/msm/msm_gem.h +8 -0
@@ -54,6 +54,9 @@ struct msm_gem_vm {/** @base: Inherit from drm_gpuvm. */ struct drm_gpuvm base; +/** @name: Storage for dynamically generated VM name for user VMs */ +char name[32 ]; +/** * @sched: Scheduler used for asynchronous VM_BIND request. *
@@ -95,6 +98,11 @@ struct msm_gem_vm {*/ struct pid *pid; +/** +* @node: List node in msm_drm_private.vms list +*/ +struct list_head node; +/** @faults: the number of GPU hangs associated with this address space */ int faults;
drivers/gpu/drm/msm/msm_gem_vma.c +23 -0
@@ -14,6 +14,11 @@ static void msm_gem_vm_free(struct drm_gpuvm *gpuvm) { struct msm_gem_vm *vm = container_of(gpuvm, struct msm_gem_vm, base); +struct msm_drm_private *priv = gpuvm->drm->dev_private; + +mutex_lock (&priv->vm_lock); +list_del (&vm->node); +mutex_unlock (&priv->vm_lock);drm_mm_takedown (&vm->mm);if (vm->mmu)
@@ -640,6 +645,7 @@ struct drm_gpuvm * msm_gem_vm_create(struct drm_device *drm, struct msm_mmu *mmu, const char *name, u64 va_start, u64 va_size, bool managed) { + struct msm_drm_private *priv = drm->dev_private;enum drm_gpuvm_flags flags = managed ?DRM_GPUVM_VA_WEAK_REF :0 ;struct msm_gem_vm *vm;struct drm_gem_object *dummy_gem;
@@ -673,6 +679,19 @@ msm_gem_vm_create(struct drm_device *drm, struct msm_mmu *mmu, const char *name, goto err_free_dummy; } +/* For userspace pgtables, generate a VM name based on comm and PID nr: */ +if (!name) { +char tmpname[TASK_COMM_LEN ]; +struct pid *pid =get_pid (task_tgid (current)); + +get_task_comm (tmpname, current); +rcu_read_lock (); +snprintf (vm->name,sizeof (name),"%s[%d]" , tmpname,pid_nr (pid)); +rcu_read_unlock (); + + name = vm->name; + } +drm_gpuvm_init (&vm->base, name, flags, drm, dummy_gem, va_start, va_size,0 ,0 , &msm_gpuvm_ops);drm_gem_object_put (dummy_gem);
@@ -686,6 +705,10 @@ msm_gem_vm_create(struct drm_device *drm, struct msm_mmu *mmu, const char *name,drm_mm_init (&vm->mm, va_start, va_size); +mutex_lock (&priv->vm_lock); +list_add_tail (&vm->node, &priv->vms); +mutex_unlock (&priv->vm_lock); +return &vm->base; err_free_dummy:
Signature · 1 line
2.48.1
PATCH v2 30/34 drm/msm: Crashdump prep for sparse mappings
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
In this case, userspace could request dumping partial GEM obj mappings. Also drop use of should_dump() helper, which really only makes sense in the old submit->bos[] table world.
--- drivers/gpu/drm/msm/msm_gpu.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
drivers/gpu/drm/msm/msm_gpu.c +10 -7
@@ -219,13 +219,14 @@ static void msm_gpu_devcoredump_free(void *data) }static void msm_gpu_crashstate_get_bo (struct msm_gpu_state *state, -struct drm_gem_object *obj,u64 iova,bool full) +struct drm_gem_object *obj,u64 iova, +bool full,size_t offset,size_t size) {struct msm_gpu_state_bo *state_bo = &state->bos[state->nr_bos];struct msm_gem_object *msm_obj =to_msm_bo (obj);/* Don't record write only objects */ - state_bo->size = obj->size; + state_bo->size = size; state_bo->flags = msm_obj->flags; state_bo->iova = iova;
@@ -236,7 +237,7 @@ static void msm_gpu_crashstate_get_bo(struct msm_gpu_state *state,if (full) {void *ptr; - state_bo->data =kvmalloc (obj->size,GFP_KERNEL ); + state_bo->data =kvmalloc (size,GFP_KERNEL );if (!state_bo->data) goto out;
@@ -249,7 +250,7 @@ static void msm_gpu_crashstate_get_bo(struct msm_gpu_state *state, goto out; } -memcpy (state_bo->data, ptr, obj->size); +memcpy (state_bo->data, ptr + offset, size);msm_gem_put_vaddr (obj); } out:
@@ -279,6 +280,7 @@ static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, state->fault_info = gpu->fault_info;if (submit) { +extern bool rd_full;int i;if (state->fault_info.ttbr0) {
@@ -294,9 +296,10 @@ static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, sizeof(struct msm_gpu_state_bo), GFP_KERNEL);for (i =0 ; state->bos && i < submit->nr_bos; i++) { -msm_gpu_crashstate_get_bo (state, submit->bos[i].obj, - submit->bos[i].iova, -should_dump (submit, i)); +struct drm_gem_object *obj = submit->bos[i].obj; +bool dump = rd_full || (submit->bos[i].flags &MSM_SUBMIT_BO_DUMP ); +msm_gpu_crashstate_get_bo (state, obj, submit->bos[i].iova, + dump,0 , obj->size); } }
Signature · 1 line
2.48.1
PATCH v2 31/34 drm/msm: rd dumping prep for sparse mappings
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Similar to the previous commit, add support for dumping partial mappings.
--- drivers/gpu/drm/msm/msm_gem.h | 10 --------- drivers/gpu/drm/msm/msm_rd.c | 38 ++++++++++++++++------------------- 2 files changed, 17 insertions(+), 31 deletions(-)
drivers/gpu/drm/msm/msm_gem.h +0 -10
@@ -420,14 +420,4 @@ static inline void msm_gem_submit_put(struct msm_gem_submit *submit)void msm_submit_retire (struct msm_gem_submit *submit); -/* helper to determine of a buffer in submit should be dumped, used for both -* devcoredump and debugfs cmdstream dumping: -*/ -static inline bool -should_dump (struct msm_gem_submit *submit,int idx) -{ -extern bool rd_full; -return rd_full || (submit->bos[idx].flags &MSM_SUBMIT_BO_DUMP ); -} -#endif /* __MSM_GEM_H__ */
drivers/gpu/drm/msm/msm_rd.c +17 -21
@@ -308,21 +308,11 @@ void msm_rd_debugfs_cleanup(struct msm_drm_private *priv) priv->hangrd =NULL ; } -static void snapshot_buf (struct msm_rd_state *rd, -struct msm_gem_submit *submit,int idx, -uint64_t iova,uint32_t size,bool full) +static void snapshot_buf (struct msm_rd_state *rd,struct drm_gem_object *obj, +uint64_t iova,bool full,size_t offset,size_t size) { -struct drm_gem_object *obj = submit->bos[idx].obj; -unsigned offset =0 ;const char *buf; -if (iova) { - offset = iova - submit->bos[idx].iova; - }else { - iova = submit->bos[idx].iova; - size = obj->size; - } -/* * Always write the GPUADDR header so can get a complete list of all the * buffers in the cmd
@@ -333,10 +323,6 @@ static void snapshot_buf(struct msm_rd_state *rd,if (!full)return ; -/* But only dump the contents of buffers marked READ */ -if (!(submit->bos[idx].flags &MSM_SUBMIT_BO_READ )) -return ; - buf =msm_gem_get_vaddr_active (obj);if (IS_ERR (buf))return ;
@@ -352,6 +338,7 @@ static void snapshot_buf(struct msm_rd_state *rd,void msm_rd_dump_submit (struct msm_rd_state *rd,struct msm_gem_submit *submit,const char *fmt, ...) { +extern bool rd_full;struct task_struct *task;char msg[256 ];int i, n;
@@ -385,16 +372,25 @@ void msm_rd_dump_submit(struct msm_rd_state *rd, struct msm_gem_submit *submit,rd_write_section (rd,RD_CMD , msg,ALIGN (n,4 )); -for (i =0 ; i < submit->nr_bos; i++) -snapshot_buf (rd, submit, i,0 ,0 ,should_dump (submit, i)); +for (i =0 ; i < submit->nr_bos; i++) { +struct drm_gem_object *obj = submit->bos[i].obj; +bool dump = rd_full || (submit->bos[i].flags &MSM_SUBMIT_BO_DUMP ); + +snapshot_buf (rd, obj, submit->bos[i].iova, dump,0 , obj->size); + }for (i =0 ; i < submit->nr_cmds; i++) {uint32_t szd = submit->cmd[i].size;/* in dwords */ +int idx = submit->cmd[i].idx; +bool dump = rd_full || (submit->bos[idx].flags &MSM_SUBMIT_BO_DUMP );/* snapshot cmdstream bo's (if we haven't already): */ -if (!should_dump (submit, i)) { -snapshot_buf (rd, submit, submit->cmd[i].idx, - submit->cmd[i].iova, szd *4 , true); +if (!dump) { +struct drm_gem_object *obj = submit->bos[idx].obj; +size_t offset = submit->cmd[i].iova - submit->bos[idx].iova; + +snapshot_buf (rd, obj, submit->cmd[i].iova, true, + offset, szd *4 ); } }
Signature · 1 line
2.48.1
PATCH v2 32/34 drm/msm: Crashdec support for sparse
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
In this case, we need to iterate the VMAs looking for ones with MSM_VMA_DUMP flag.
--- drivers/gpu/drm/msm/msm_gpu.c | 73 +++++++++++++++++++++++++---------- 1 file changed, 52 insertions(+), 21 deletions(-)
drivers/gpu/drm/msm/msm_gpu.c +52 -21
@@ -257,6 +257,50 @@ static void msm_gpu_crashstate_get_bo(struct msm_gpu_state *state, state->nr_bos++; } +static void crashstate_get_bos (struct msm_gpu_state *state,struct msm_gem_submit *submit) +{ +extern bool rd_full; + +if (!submit) +return ; + +if (msm_context_is_vmbind (submit->queue->ctx)) { +struct drm_gpuva *vma; +unsigned cnt =0 ; + +mutex_lock (&to_msm_vm (submit->vm)->vm_lock); + +drm_gpuvm_for_each_va (vma, submit->vm) + cnt++; + + state->bos =kcalloc (cnt,sizeof (struct msm_gpu_state_bo ),GFP_KERNEL ); + + drm_gpuvm_for_each_va (vma, submit->vm) { +bool dump = rd_full || (vma->flags &MSM_VMA_DUMP ); + +/* Skip MAP_NULL/PRR VMAs: */ +if (!vma->gem.obj) +continue ; + +msm_gpu_crashstate_get_bo (state, vma->gem.obj, vma->va.addr, + dump, vma->gem.offset, vma->va.range); + } + +mutex_unlock (&to_msm_vm (submit->vm)->vm_lock); + }else { + state->bos =kcalloc (submit->nr_bos, +sizeof (struct msm_gpu_state_bo ),GFP_KERNEL ); + +for (int i =0 ; state->bos && i < submit->nr_bos; i++) { +struct drm_gem_object *obj = submit->bos[i].obj; +bool dump = rd_full || (submit->bos[i].flags &MSM_SUBMIT_BO_DUMP ); + +msm_gpu_crashstate_get_bo (state, obj, submit->bos[i].iova, + dump,0 , obj->size); + } + } +} +static void msm_gpu_crashstate_capture (struct msm_gpu *gpu,struct msm_gem_submit *submit,char *comm,char *cmd) {
@@ -279,30 +323,17 @@ static void msm_gpu_crashstate_capture(struct msm_gpu *gpu, state->cmd =kstrdup (cmd,GFP_KERNEL ); state->fault_info = gpu->fault_info; -if (submit) { -extern bool rd_full; -int i; - -if (state->fault_info.ttbr0) { -struct msm_gpu_fault_info *info = &state->fault_info; -struct msm_mmu *mmu =to_msm_vm (submit->vm)->mmu; - -msm_iommu_pagetable_params (mmu, &info->pgtbl_ttbr0, - &info->asid); -msm_iommu_pagetable_walk (mmu, info->iova, info->ptes); - } +if (submit && state->fault_info.ttbr0) { +struct msm_gpu_fault_info *info = &state->fault_info; +struct msm_mmu *mmu =to_msm_vm (submit->vm)->mmu; - state->bos =kcalloc (submit->nr_bos, -sizeof (struct msm_gpu_state_bo ),GFP_KERNEL ); - -for (i =0 ; state->bos && i < submit->nr_bos; i++) { -struct drm_gem_object *obj = submit->bos[i].obj; -bool dump = rd_full || (submit->bos[i].flags &MSM_SUBMIT_BO_DUMP ); -msm_gpu_crashstate_get_bo (state, obj, submit->bos[i].iova, - dump,0 , obj->size); - } +msm_iommu_pagetable_params (mmu, &info->pgtbl_ttbr0, + &info->asid); +msm_iommu_pagetable_walk (mmu, info->iova, info->ptes); } +crashstate_get_bos (state, submit); +/* Set the active crash state to be dumped on failure */ gpu->crashstate = state;
Signature · 1 line
2.48.1
PATCH v2 33/34 drm/msm: rd dumping support for sparse
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
As with devcoredump, we need to iterate the VMAs to figure out what to dump.
--- drivers/gpu/drm/msm/msm_rd.c | 48 +++++++++++++++++++++++++----------- 1 file changed, 33 insertions(+), 15 deletions(-)
drivers/gpu/drm/msm/msm_rd.c +33 -15
@@ -372,25 +372,43 @@ void msm_rd_dump_submit(struct msm_rd_state *rd, struct msm_gem_submit *submit,rd_write_section (rd,RD_CMD , msg,ALIGN (n,4 )); -for (i =0 ; i < submit->nr_bos; i++) { -struct drm_gem_object *obj = submit->bos[i].obj; -bool dump = rd_full || (submit->bos[i].flags &MSM_SUBMIT_BO_DUMP ); +if (msm_context_is_vmbind (submit->queue->ctx)) { +struct drm_gpuva *vma; -snapshot_buf (rd, obj, submit->bos[i].iova, dump,0 , obj->size); - } +mutex_lock (&to_msm_vm (submit->vm)->vm_lock); + drm_gpuvm_for_each_va (vma, submit->vm) { +bool dump = rd_full || (vma->flags &MSM_VMA_DUMP ); -for (i =0 ; i < submit->nr_cmds; i++) { -uint32_t szd = submit->cmd[i].size;/* in dwords */ -int idx = submit->cmd[i].idx; -bool dump = rd_full || (submit->bos[idx].flags &MSM_SUBMIT_BO_DUMP ); +/* Skip MAP_NULL/PRR VMAs: */ +if (!vma->gem.obj) +continue ; + +snapshot_buf (rd, vma->gem.obj, vma->va.addr, dump, + vma->gem.offset, vma->va.range); + } +mutex_unlock (&to_msm_vm (submit->vm)->vm_lock); + + }else { +for (i =0 ; i < submit->nr_bos; i++) { +struct drm_gem_object *obj = submit->bos[i].obj; +bool dump = rd_full || (submit->bos[i].flags &MSM_SUBMIT_BO_DUMP ); + +snapshot_buf (rd, obj, submit->bos[i].iova, dump,0 , obj->size); + } + +for (i =0 ; i < submit->nr_cmds; i++) { +uint32_t szd = submit->cmd[i].size;/* in dwords */ +int idx = submit->cmd[i].idx; +bool dump = rd_full || (submit->bos[idx].flags &MSM_SUBMIT_BO_DUMP ); -/* snapshot cmdstream bo's (if we haven't already): */ -if (!dump) { -struct drm_gem_object *obj = submit->bos[idx].obj; -size_t offset = submit->cmd[i].iova - submit->bos[idx].iova; +/* snapshot cmdstream bo's (if we haven't already): */ +if (!dump) { +struct drm_gem_object *obj = submit->bos[idx].obj; +size_t offset = submit->cmd[i].iova - submit->bos[idx].iova; -snapshot_buf (rd, obj, submit->cmd[i].iova, true, - offset, szd *4 ); +snapshot_buf (rd, obj, submit->cmd[i].iova, true, + offset, szd *4 ); + } } }
Signature · 1 line
2.48.1
PATCH v2 34/34 drm/msm: Bump UAPI version
In reply to Rob ClarkFrom: Rob Clark <[email protected]>
Bump version to signal to userspace that VM_BIND is supported.
--- drivers/gpu/drm/msm/msm_drv.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/msm/msm_drv.c +2 -1
@@ -41,9 +41,10 @@* - 1.10.0 - Add MSM_SUBMIT_BO_NO_IMPLICIT * - 1.11.0 - Add wait boost (MSM_WAIT_FENCE_BOOST, MSM_PREP_BOOST) * - 1.12.0 - Add MSM_INFO_SET_METADATA and MSM_INFO_GET_METADATA +* - 1.13.0 - Add VM_BIND */ #define MSM_VERSION_MAJOR 1 -#define MSM_VERSION_MINOR 12 +#define MSM_VERSION_MINOR 13#define MSM_VERSION_PATCHLEVEL 0bool dumpstate;
Signature · 1 line
2.48.1