Refactor VM bind code
Subsystems (3)
Matthew Brost
maintainercover Refactor VM bind code
WIP on refactoring VM bind to final error handling solution.
Major changes are:
1. Allow bind / exec IOCTLs with zero binds / batches
2. Convert multiple binds from IOCTL into single job
3. CPU binds in jobs
4. Sync vs async bind per IOCTL rather than per queue
Error handling still not 100% correct but this this series structures the code to make it possible.
This series shouldn't break any uAPI compatibility until the last patch in the series.
Matt
Matthew Brost (26):
drm/xe: Allow num_binds == 0 in VM bind IOCTL
drm/xe: Allow num_batch_buffer == 0 in exec IOCTL
drm/xe: Lock all gpuva ops during VM bind IOCTL
drm/xe: Add ops_execute function which returns a fence drm/xe: Move migrate to prefetch to op_lock funtion drm/xe: Add struct xe_vma_ops abstraction
drm/xe: Update xe_vm_rebind to use dummy VMA operations drm/xe: Move drm exec loop to vm_bind_ioctl_ops_execute drm/xe: Fixup error handling / ref counting in VM bind IOCTL drm/xe: Convert pagefault rebind to use ops interface drm/xe: Add some members to xe_vma_ops
drm/xe: Add vm_bind_ioctl_ops_install_fences helper drm/xe: Drop rebind argument from xe_pt_prepare_bind drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops
drm/xe: Move setting last fence and sync wait to
vm_bind_ioctl_ops_install_fences
drm/xe: Fix vma_is_valid to use tile argument
drm/xe: Adjust tile mask in operations create
drm/xe: Add xe_gt_tlb_invalidation_range and convert PT layer to use
this
drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue drm/xe: Convert multiple bind ops into single job
drm/xe: Update clear / populate arguments
drm/xe: Add __xe_migrate_update_pgtables_cpu helper drm/xe: Add xe_hw_fence_signal helper
drm/xe: CPU binds for jobs
drm/xe: Don't use migrate exec queue for page fault binds drm/xe/uapi: Make sync vs async VM bind operations per IOCTL rather
than queue
drivers/gpu/drm/xe/xe_bo.c | 7 +- drivers/gpu/drm/xe/xe_bo.h | 6 +- drivers/gpu/drm/xe/xe_bo_types.h | 2 + drivers/gpu/drm/xe/xe_device.c | 23 + drivers/gpu/drm/xe/xe_device.h | 7 + drivers/gpu/drm/xe/xe_device_types.h | 4 + drivers/gpu/drm/xe/xe_exec.c | 48 +- drivers/gpu/drm/xe/xe_exec_queue.c | 20 +- drivers/gpu/drm/xe/xe_exec_queue_types.h | 7 +- drivers/gpu/drm/xe/xe_gt_pagefault.c | 18 +- drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c | 60 +- drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 3 + drivers/gpu/drm/xe/xe_guc_submit.c | 43 +- drivers/gpu/drm/xe/xe_hw_fence.c | 8 + drivers/gpu/drm/xe/xe_hw_fence.h | 1 + drivers/gpu/drm/xe/xe_migrate.c | 382 +++----- drivers/gpu/drm/xe/xe_migrate.h | 46 +- drivers/gpu/drm/xe/xe_pt.c | 992 ++++++++++++-------- drivers/gpu/drm/xe/xe_pt.h | 8 + drivers/gpu/drm/xe/xe_pt_types.h | 51 + drivers/gpu/drm/xe/xe_sched_job.c | 4 +- drivers/gpu/drm/xe/xe_sched_job_types.h | 31 +- drivers/gpu/drm/xe/xe_sync.c | 5 +- drivers/gpu/drm/xe/xe_sync.h | 2 +- drivers/gpu/drm/xe/xe_vm.c | 981 ++++++++----------- drivers/gpu/drm/xe/xe_vm.h | 7 + drivers/gpu/drm/xe/xe_vm_types.h | 208 ++-- include/uapi/drm/xe_drm.h | 6 +- 28 files changed, 1536 insertions(+), 1444 deletions(-)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 1/26 drm/xe: Allow num_binds == 0 in VM bind IOCTL
The idea being out-syncs can signal indicating all previous operations on the bind queue are complete. An example use case of this would be support for implementing vkQueueWaitForIdle easily.
--- drivers/gpu/drm/xe/xe_vm.c | 30 ++++++++++++++++++------------ 1 file changed, 18 insertions(+), 12 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +18 -12
@@ -2839,7 +2839,6 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe,int i;if (XE_IOCTL_DBG (xe, args->extensions) || -XE_IOCTL_DBG (xe, !args->num_binds) ||XE_IOCTL_DBG (xe, args->num_binds >MAX_BINDS ))return -EINVAL ;
@@ -2966,7 +2965,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto put_exec_queue; } -if (XE_IOCTL_DBG (xe, async != +if (XE_IOCTL_DBG (xe, args->num_binds && async != !!(q->flags &EXEC_QUEUE_FLAG_VM_ASYNC ))) { err = -EINVAL ; goto put_exec_queue;
@@ -2980,7 +2979,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) }if (!args->exec_queue_id) { -if (XE_IOCTL_DBG (xe, async != +if (XE_IOCTL_DBG (xe, args->num_binds && async != !!(vm->flags &XE_VM_FLAG_ASYNC_DEFAULT ))) { err = -EINVAL ; goto put_vm;
@@ -3017,16 +3016,18 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) } } - bos =kzalloc (sizeof (*bos) * args->num_binds,GFP_KERNEL ); -if (!bos) { - err = -ENOMEM ; - goto release_vm_lock; - } +if (args->num_binds) { + bos =kzalloc (sizeof (*bos) * args->num_binds,GFP_KERNEL ); +if (!bos) { + err = -ENOMEM ; + goto release_vm_lock; + } - ops =kzalloc (sizeof (*ops) * args->num_binds,GFP_KERNEL ); -if (!ops) { - err = -ENOMEM ; - goto release_vm_lock; + ops =kzalloc (sizeof (*ops) * args->num_binds,GFP_KERNEL ); +if (!ops) { + err = -ENOMEM ; + goto release_vm_lock; + } }for (i =0 ; i < args->num_binds; ++i) {
@@ -3081,6 +3082,11 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto free_syncs; } +if (!args->num_binds) { + err = -ENODATA ; + goto free_syncs; + } +for (i =0 ; i < args->num_binds; ++i) {u64 range = bind_ops[i].range;u64 addr = bind_ops[i].addr;
Signature · 1 line
2.34.1
Thomas Hellström
maintainerRe: [Intel-xe] [PATCH 01/26] drm/xe: Allow num_binds == 0 in VM bind IOCTL
Hi,
Quoted 5 lines · Matthew Brost
The idea being out-syncs can signal indicating all previous operations on the bind queue are complete. An example use case of this would be support for implementing vkQueueWaitForIdle easily.
Signed-off-by Matthew Brost <[email protected]>
Do we actually create a gpu job when num_binds == 0 so that in-fences and waiting for previous binds actually work as intended even when we remove the async queue?
/Thomas
Thomas Hellström
maintainerRe: [Intel-xe] [PATCH 01/26] drm/xe: Allow num_binds == 0 in VM bind IOCTL
Quoted 10 lines · Thomas Hellström
Hi,
Quoted 5 lines · Matthew Brost
The idea being out-syncs can signal indicating all previous operations on the bind queue are complete. An example use case of this would be support for implementing vkQueueWaitForIdle easily.
Signed-off-by Matthew Brost <[email protected]>Do we actually create a gpu job when
num_binds== 0 so that in-fences and waiting for previous binds actually work as intended even when we remove the async queue?
Or to rephrase since the async queue is actually already removed, Similar to review for patch 2 / 26 Do we ensure correct ordering even in the presence of in-fences here?
/Thomas
Quoted 4 lines
/Thomas
Matthew Brost
maintainerPATCH 2/26 drm/xe: Allow num_batch_buffer == 0 in exec IOCTL
In reply to Matthew BrostThe idea being out-syncs can signal indicating all previous operations on the exec queue are complete. An example use case of this would be support for implementing vkQueueWaitForIdle easily.
v2: Don't add last_fence for VM's that do not support dma fences
--- drivers/gpu/drm/xe/xe_exec.c | 22 +++++++++++++++++++--- drivers/gpu/drm/xe/xe_exec_queue.c | 5 ++++- drivers/gpu/drm/xe/xe_exec_queue_types.h | 5 +++-- drivers/gpu/drm/xe/xe_sync.c | 5 ++++- drivers/gpu/drm/xe/xe_sync.h | 2 +- drivers/gpu/drm/xe/xe_vm.c | 2 +- 6 files changed, 32 insertions(+), 9 deletions(-)
drivers/gpu/drm/xe/xe_exec.c +19 -3
@@ -161,7 +161,8 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (XE_IOCTL_DBG (xe, q->flags &EXEC_QUEUE_FLAG_VM ))return -EINVAL ; -if (XE_IOCTL_DBG (xe, q->width != args->num_batch_buffer)) +if (XE_IOCTL_DBG (xe, args->num_batch_buffer && + q->width != args->num_batch_buffer))return -EINVAL ;if (XE_IOCTL_DBG (xe, q->flags &EXEC_QUEUE_FLAG_BANNED )) {
@@ -182,12 +183,13 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)for (i =0 ; i < args->num_syncs; i++) { err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs++], &syncs_user[i], true, -xe_vm_no_dma_fences (vm)); +xe_vm_no_dma_fences (vm), + !args->num_batch_buffer);if (err) goto err_syncs; } -if (xe_exec_queue_is_parallel (q)) { +if (args->num_batch_buffer &&xe_exec_queue_is_parallel (q)) { err =__copy_from_user (addresses, addresses_user,sizeof (u64) * q->width);if (err) {
@@ -234,6 +236,18 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_exec; } +if (!args->num_batch_buffer) { +if (!xe_vm_no_dma_fences (vm)) { +struct dma_fence *fence = +xe_exec_queue_last_fence_get (q, vm); + +for (i =0 ; i < num_syncs; i++) +xe_sync_entry_signal (&syncs[i],NULL , fence); + } + + goto err_exec; + } +if (xe_exec_queue_is_lr (q) &&xe_exec_queue_ring_full (q)) { err = -EWOULDBLOCK ; goto err_exec;
@@ -327,6 +341,8 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (xe_exec_queue_is_lr (q)) q->ring_ops->emit_job (job); +if (!xe_vm_no_dma_fences (vm)) +xe_exec_queue_last_fence_set (q, vm, &job->drm.s_fence->finished);xe_sched_job_push (job);xe_vm_reactivate_rebind (vm);
drivers/gpu/drm/xe/xe_exec_queue.c +4 -1
@@ -921,7 +921,10 @@ int xe_exec_queue_set_property_ioctl(struct drm_device *dev, void *data,static void xe_exec_queue_last_fence_lockdep_assert (struct xe_exec_queue *q,struct xe_vm *vm) { -lockdep_assert_held_write (&vm->lock); +if (q->flags &EXEC_QUEUE_FLAG_VM ) +lockdep_assert_held_write (&vm->lock); +else +xe_vm_assert_held (vm); }/**
drivers/gpu/drm/xe/xe_exec_queue_types.h +3 -2
@@ -53,8 +53,9 @@ struct xe_exec_queue {struct xe_hw_fence_irq *fence_irq;/** -* @last_fence: last fence on engine, protected by vm->lock in write -* mode if bind engine +* @last_fence: last fence on exec queue, protected by vm->lock in write +* mode if bind exec queue, protected by dma resv lock if non-bind exec +* queue */ struct dma_fence *last_fence;
drivers/gpu/drm/xe/xe_sync.c +4 -1
@@ -100,7 +100,7 @@ static void user_fence_cb(struct dma_fence *fence, struct dma_fence_cb *cb)int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user, -bool exec,bool no_dma_fences) +bool exec,bool no_dma_fences,bool exec_nop) {struct drm_xe_sync sync_in;int err;
@@ -171,6 +171,9 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,break ;case DRM_XE_SYNC_USER_FENCE : +if (XE_IOCTL_DBG (xe, exec_nop)) +return -EOPNOTSUPP ; +if (XE_IOCTL_DBG (xe, !signal))return -EOPNOTSUPP ;
drivers/gpu/drm/xe/xe_sync.h +1 -1
@@ -15,7 +15,7 @@ struct xe_sched_job;int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user, -bool exec,bool compute_mode); +bool exec,bool compute_mode,bool exec_nop);int xe_sync_entry_wait (struct xe_sync_entry *sync);int xe_sync_entry_add_deps (struct xe_sync_entry *sync,struct xe_sched_job *job);
drivers/gpu/drm/xe/xe_vm.c +1 -1
@@ -3077,7 +3077,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs], &syncs_user[num_syncs], false, -xe_vm_no_dma_fences (vm)); +xe_vm_no_dma_fences (vm), false);if (err) goto free_syncs; }
Signature · 1 line
2.34.1
Thomas Hellström
maintainerRe: [Intel-xe] [PATCH 02/26] drm/xe: Allow num_batch_buffer == 0 in exec IOCTL
Quoted 7 lines · Matthew Brost
The idea being out-syncs can signal indicating all previous operations on the exec queue are complete. An example use case of this would be support for implementing vkQueueWaitForIdle easily.
v2: Don't add
last_fencefor VM's that do not support dma fencesSigned-off-by Matthew Brost <[email protected]>
Also, here, as we discussed previously, this breaks with in-fences, right?
/Thomas
Quoted 148 lines
--- drivers/gpu/drm/xe/xe_exec.c | 22 +++++++++++++++++++--- drivers/gpu/drm/xe/xe_exec_queue.c | 5 ++++- drivers/gpu/drm/xe/xe_exec_queue_types.h | 5 +++-- drivers/gpu/drm/xe/xe_sync.c | 5 ++++- drivers/gpu/drm/xe/xe_sync.h | 2 +- drivers/gpu/drm/xe/xe_vm.c | 2 +- 6 files changed, 32 insertions(+), 9 deletions(-)drivers/gpu/drm/xe/xe_exec.c +19
-3@@ -161,7 +161,8 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (XE_IOCTL_DBG (xe, q->flags &EXEC_QUEUE_FLAG_VM ))return -EINVAL ; -if (XE_IOCTL_DBG (xe, q->width != args->num_batch_buffer)) +if (XE_IOCTL_DBG (xe, args->num_batch_buffer && + q->width != args->num_batch_buffer))return -EINVAL ;if (XE_IOCTL_DBG (xe, q->flags &EXEC_QUEUE_FLAG_BANNED )) {@@ -182,12 +183,13 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)for (i =0 ; i < args->num_syncs; i++) { err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs++], &syncs_user[i], true, -xe_vm_no_dma_fences (vm)); +xe_vm_no_dma_fences (vm), + !args->num_batch_buffer);if (err) goto err_syncs; } -if (xe_exec_queue_is_parallel (q)) { +if (args->num_batch_buffer &&xe_exec_queue_is_parallel (q)) { err =__copy_from_user (addresses, addresses_user,sizeof (u64) * q->width);if (err) {@@ -234,6 +236,18 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_exec; } +if (!args->num_batch_buffer) { +if (!xe_vm_no_dma_fences (vm)) { +struct dma_fence *fence = +xe_exec_queue_last_fence_get (q, vm); + +for (i =0 ; i < num_syncs; i++) +xe_sync_entry_signal (&syncs[i],NULL , fence); + } + + goto err_exec; + } +if (xe_exec_queue_is_lr (q) &&xe_exec_queue_ring_full (q)) { err = -EWOULDBLOCK ; goto err_exec;@@ -327,6 +341,8 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (xe_exec_queue_is_lr (q)) q->ring_ops->emit_job (job); +if (!xe_vm_no_dma_fences (vm)) +xe_exec_queue_last_fence_set (q, vm, &job->drm.s_fence->finished);xe_sched_job_push (job);xe_vm_reactivate_rebind (vm);drivers/gpu/drm/xe/xe_exec_queue.c +4
-1@@ -921,7 +921,10 @@ int xe_exec_queue_set_property_ioctl(struct drm_device *dev, void *data,static void xe_exec_queue_last_fence_lockdep_assert (struct xe_exec_queue *q,struct xe_vm *vm) { -lockdep_assert_held_write (&vm->lock); +if (q->flags &EXEC_QUEUE_FLAG_VM ) +lockdep_assert_held_write (&vm->lock); +else +xe_vm_assert_held (vm); }/** drivers/gpu/drm/xe/xe_exec_queue_types.h +3
-2@@ -53,8 +53,9 @@ struct xe_exec_queue {struct xe_hw_fence_irq *fence_irq;/** -* @last_fence: last fence on engine, protected by vm->lock in write -* mode if bind engine +* @last_fence: last fence on exec queue, protected by vm->lock in write +* mode if bind exec queue, protected by dma resv lock if non-bind exec +* queue */ struct dma_fence *last_fence;drivers/gpu/drm/xe/xe_sync.c +4
-1@@ -100,7 +100,7 @@ static void user_fence_cb(struct dma_fence *fence, struct dma_fence_cb *cb)int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user, -bool exec,bool no_dma_fences) +bool exec,bool no_dma_fences,bool exec_nop) {struct drm_xe_sync sync_in;int err;@@ -171,6 +171,9 @@ int xe_sync_entry_parse(struct xe_device *xe, struct xe_file *xef,break ;case DRM_XE_SYNC_USER_FENCE : +if (XE_IOCTL_DBG (xe, exec_nop)) +return -EOPNOTSUPP ; +if (XE_IOCTL_DBG (xe, !signal))return -EOPNOTSUPP ;drivers/gpu/drm/xe/xe_sync.h +1
-1@@ -15,7 +15,7 @@ struct xe_sched_job;int xe_sync_entry_parse (struct xe_device *xe,struct xe_file *xef,struct xe_sync_entry *sync,struct drm_xe_sync __user *sync_user, -bool exec,bool compute_mode); +bool exec,bool compute_mode,bool exec_nop);int xe_sync_entry_wait (struct xe_sync_entry *sync);int xe_sync_entry_add_deps (struct xe_sync_entry *sync,struct xe_sched_job *job);drivers/gpu/drm/xe/xe_vm.c +1
-1@@ -3077,7 +3077,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)for (num_syncs =0 ; num_syncs < args->num_syncs; num_syncs++) { err =xe_sync_entry_parse (xe, xef, &syncs[num_syncs], &syncs_user[num_syncs], false, -xe_vm_no_dma_fences (vm)); +xe_vm_no_dma_fences (vm), false);if (err) goto free_syncs; }
Matthew Brost
maintainerPATCH 3/26 drm/xe: Lock all gpuva ops during VM bind IOCTL
In reply to Matthew BrostLock all gpuva ops and validate all BOs in a single step durin the VM bind IOCTL. This help with the transition to making all gpuva ops in a VM bind IOCTL a single atomic job.
--- drivers/gpu/drm/xe/xe_vm.c | 126 +++++++++++++++++++++++++++---------- 1 file changed, 94 insertions(+), 32 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +94 -32
@@ -474,19 +474,23 @@ int xe_vm_lock_dma_resv(struct xe_vm *vm, struct drm_exec *exec,#define XE_VM_REBIND_RETRY_TIMEOUT_MS 1000 -static void xe_vm_kill (struct xe_vm *vm) +static void xe_vm_kill (struct xe_vm *vm,bool unlocked) {struct xe_exec_queue *q;lockdep_assert_held (&vm->lock); -xe_vm_lock (vm, false); +if (unlocked) +xe_vm_lock (vm, false); + vm->flags |=XE_VM_FLAG_BANNED ;trace_xe_vm_kill (vm);list_for_each_entry (q, &vm->preempt.exec_queues, compute.link) q->ops->kill (q); -xe_vm_unlock (vm); + +if (unlocked) +xe_vm_unlock (vm);/* TODO: Inform user the VM is banned */ }
@@ -682,7 +686,7 @@ static void preempt_rebind_work_func(struct work_struct *w)if (err) {drm_warn (&vm->xe->drm,"VM worker error: %d\n" , err); -xe_vm_kill (vm); +xe_vm_kill (vm, true); }up_write (&vm->lock);
@@ -1879,17 +1883,9 @@ static int xe_vm_bind(struct xe_vm *vm, struct xe_vma *vma, struct xe_exec_queue u32 num_syncs, bool immediate, bool first_op, bool last_op) { - int err; -xe_vm_assert_held (vm );xe_bo_assert_held (bo); -if (bo && immediate) { - err =xe_bo_validate (bo, vm, true); -if (err) -return err; - } -return __xe_vm_bind (vm, vma, q, syncs, num_syncs, immediate, first_op, last_op); }
@@ -2527,17 +2523,12 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,return 0 ; } -static int op_execute (struct drm_exec *exec,struct xe_vm *vm, -struct xe_vma *vma,struct xe_vma_op *op) +static int op_execute (struct xe_vm *vm,struct xe_vma *vma, +struct xe_vma_op *op) {int err;lockdep_assert_held_write (&vm->lock); - - err =xe_vm_prepare_vma (exec, vma,1 ); -if (err) -return err; -xe_vm_assert_held (vm);xe_bo_assert_held (xe_vma_bo (vma));
@@ -2618,19 +2609,10 @@ static int op_execute(struct drm_exec *exec, struct xe_vm *vm,static int __xe_vma_op_execute (struct xe_vm *vm,struct xe_vma *vma,struct xe_vma_op *op) { -struct drm_exec exec;int err; retry_userptr: -drm_exec_init (&exec,DRM_EXEC_INTERRUPTIBLE_WAIT ); - drm_exec_until_all_locked(&exec) { - err =op_execute (&exec, vm, vma, op); -drm_exec_retry_on_contention (&exec); -if (err) -break ; - } -drm_exec_fini (&exec); - + err =op_execute (vm, vma, op);if (err == -EAGAIN &&xe_vma_is_userptr (vma)) {lockdep_assert_held_write (&vm->lock); err =xe_vma_userptr_pin_pages (vma);
@@ -2790,14 +2772,91 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, } } +static int vma_lock (struct drm_exec *exec,struct xe_vma *vma,bool validate) +{ +struct xe_bo *bo =xe_vma_bo (vma); +int err =0 ; + +if (bo) { +if (!bo->vm) + err =drm_exec_prepare_obj (exec, &bo->ttm.base,1 ); +if (!err && validate) + err =xe_bo_validate (bo,xe_vma_vm (vma), true); + } + +return err; +} + +static int op_lock (struct drm_exec *exec,struct xe_vm *vm, +struct xe_vma_op *op) +{ +int err =0 ; + +switch (op->base.op) { +case DRM_GPUVA_OP_MAP : + err =vma_lock (exec, op->map.vma, + op->map.immediate || !xe_vm_in_fault_mode (vm)); +break ; +case DRM_GPUVA_OP_REMAP : + err =vma_lock (exec,gpuva_to_vma (op->base.remap.unmap->va), + false); +if (!err && op->remap.prev) + err =vma_lock (exec, op->remap.prev, true); +if (!err && op->remap.next) + err =vma_lock (exec, op->remap.next, true); +break ; +case DRM_GPUVA_OP_UNMAP : + err =vma_lock (exec,gpuva_to_vma (op->base.unmap.va), false); +break ; +case DRM_GPUVA_OP_PREFETCH : + err =vma_lock (exec,gpuva_to_vma (op->base.prefetch.va), true); +break ; +default : +drm_warn (&vm->xe->drm,"NOT POSSIBLE" ); + } + +return err; +} + +static int vm_bind_ioctl_ops_lock (struct drm_exec *exec, +struct xe_vm *vm, +struct list_head *ops_list) +{ +struct xe_vma_op *op; +int err; + +drm_exec_init (exec,DRM_EXEC_INTERRUPTIBLE_WAIT ); +drm_exec_until_all_locked (exec) { + err =drm_exec_prepare_obj (exec, &xe_vm_ttm_bo (vm)->base,1 ); +drm_exec_retry_on_contention (exec); +if (err) + goto out; + +list_for_each_entry (op, ops_list, link) { + err =op_lock (exec, vm, op); +drm_exec_retry_on_contention (exec); +if (err) + goto out; + } + } + +out: +return err; +} +static int vm_bind_ioctl_ops_execute (struct xe_vm *vm,struct list_head *ops_list) { +struct drm_exec exec;struct xe_vma_op *op, *next;int err;lockdep_assert_held_write (&vm->lock); + err =vm_bind_ioctl_ops_lock (&exec, vm, ops_list); +if (err) +return err; +list_for_each_entry_safe (op, next, ops_list, link) { err =xe_vma_op_execute (vm, op);if (err) {
@@ -2806,13 +2865,16 @@ static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,/* * FIXME: Killing VM rather than proper error handling */ -xe_vm_kill (vm); -return -ENOSPC ; +xe_vm_kill (vm, false); + err = -ENOSPC ; + goto unlock; }xe_vma_op_cleanup (vm, op); } -return 0 ; +unlock: +drm_exec_fini (&exec); +return err; }#ifdef TEST_VM_ASYNC_OPS_ERROR
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 4/26 drm/xe: Add ops_execute function which returns a fence
In reply to Matthew BrostAdd ops_execute function which returns a fence. This be helpful to initiate all binds (VM bind IOCTL, rebinds in exec IOCTL, rebinds in preempt rebind worker, and rebinds in pagefaults) via a gpuva ops list. Returning a fence is needed in various paths.
--- drivers/gpu/drm/xe/xe_vm.c | 210 +++++++++++++++++++------------------ 1 file changed, 109 insertions(+), 101 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +109 -101
@@ -1842,21 +1842,22 @@ static bool xe_vm_sync_mode(struct xe_vm *vm, struct xe_exec_queue *q) !(vm->flags &XE_VM_FLAG_ASYNC_DEFAULT ); } -static int __xe_vm_bind (struct xe_vm *vm,struct xe_vma *vma, -struct xe_exec_queue *q,struct xe_sync_entry *syncs, -u32 num_syncs,bool immediate,bool first_op, -bool last_op) +static struct dma_fence * +xe_vm_bind (struct xe_vm *vm,struct xe_vma *vma,struct xe_exec_queue *q, +struct xe_bo *bo,struct xe_sync_entry *syncs,u32 num_syncs, +bool immediate,bool first_op,bool last_op) {struct dma_fence *fence;struct xe_exec_queue *wait_exec_queue =to_wait_exec_queue (vm, q);xe_vm_assert_held (vm); +xe_bo_assert_held (bo);if (immediate) { fence =xe_vm_bind_vma (vma, q, syncs, num_syncs, first_op, last_op);if (IS_ERR (fence)) -return PTR_ERR (fence); +return fence; }else {int i;
@@ -1873,26 +1874,14 @@ static int __xe_vm_bind(struct xe_vm *vm, struct xe_vma *vma,xe_exec_queue_last_fence_set (wait_exec_queue, vm, fence);if (last_op &&xe_vm_sync_mode (vm, q))dma_fence_wait (fence, true); -dma_fence_put (fence); -return 0 ; -} - -static int xe_vm_bind (struct xe_vm *vm,struct xe_vma *vma,struct xe_exec_queue *q, -struct xe_bo *bo,struct xe_sync_entry *syncs, -u32 num_syncs,bool immediate,bool first_op, -bool last_op) -{ -xe_vm_assert_held (vm); -xe_bo_assert_held (bo); - -return __xe_vm_bind (vm, vma, q, syncs, num_syncs, immediate, first_op, - last_op); +return fence; } -static int xe_vm_unbind (struct xe_vm *vm,struct xe_vma *vma, -struct xe_exec_queue *q,struct xe_sync_entry *syncs, -u32 num_syncs,bool first_op,bool last_op) +static struct dma_fence * +xe_vm_unbind (struct xe_vm *vm,struct xe_vma *vma, +struct xe_exec_queue *q,struct xe_sync_entry *syncs, +u32 num_syncs,bool first_op,bool last_op) {struct dma_fence *fence;struct xe_exec_queue *wait_exec_queue =to_wait_exec_queue (vm, q);
@@ -1902,16 +1891,15 @@ static int xe_vm_unbind(struct xe_vm *vm, struct xe_vma *vma, fence =xe_vm_unbind_vma (vma, q, syncs, num_syncs, first_op, last_op);if (IS_ERR (fence)) -return PTR_ERR (fence); +return fence;xe_vma_destroy (vma, fence);if (last_op)xe_exec_queue_last_fence_set (wait_exec_queue, vm, fence);if (last_op &&xe_vm_sync_mode (vm, q))dma_fence_wait (fence, true); -dma_fence_put (fence); -return 0 ; +return fence; }#define ALL_DRM_XE_VM_CREATE_FLAGS (DRM_XE_VM_CREATE_SCRATCH_PAGE | \
@@ -2051,10 +2039,11 @@ static const u32 region_to_mem_type[] = { XE_PL_VRAM1, }; -static int xe_vm_prefetch (struct xe_vm *vm,struct xe_vma *vma, -struct xe_exec_queue *q,u32 region, -struct xe_sync_entry *syncs,u32 num_syncs, -bool first_op,bool last_op) +static struct dma_fence * +xe_vm_prefetch (struct xe_vm *vm,struct xe_vma *vma, +struct xe_exec_queue *q,u32 region, +struct xe_sync_entry *syncs,u32 num_syncs, +bool first_op,bool last_op) {struct xe_exec_queue *wait_exec_queue =to_wait_exec_queue (vm, q);int err;
@@ -2064,26 +2053,24 @@ static int xe_vm_prefetch(struct xe_vm *vm, struct xe_vma *vma,if (!xe_vma_has_no_bo (vma)) { err =xe_bo_migrate (xe_vma_bo (vma), region_to_mem_type[region]);if (err) -return err; +return ERR_PTR (err); }if (vma->tile_mask != (vma->tile_present & ~vma->usm.tile_invalidated)) {return xe_vm_bind (vm, vma, q,xe_vma_bo (vma), syncs, num_syncs, true, first_op, last_op); }else { +struct dma_fence *fence = +xe_exec_queue_last_fence_get (wait_exec_queue, vm);int i;/* Nothing to do, signal fences now */ if (last_op) { -for (i =0 ; i < num_syncs; i++) { -struct dma_fence *fence = -xe_exec_queue_last_fence_get (wait_exec_queue, vm); - +for (i =0 ; i < num_syncs; i++)xe_sync_entry_signal (&syncs[i],NULL , fence); - } } -return 0 ; +return fence; } }
@@ -2523,10 +2510,10 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,return 0 ; } -static int op_execute (struct xe_vm *vm,struct xe_vma *vma, -struct xe_vma_op *op) +static struct dma_fence *op_execute (struct xe_vm *vm,struct xe_vma *vma, +struct xe_vma_op *op) { -int err; +struct dma_fence *fence;lockdep_assert_held_write (&vm->lock);xe_vm_assert_held (vm);
@@ -2534,11 +2521,12 @@ static int op_execute(struct xe_vm *vm, struct xe_vma *vma,switch (op->base.op) {case DRM_GPUVA_OP_MAP : - err =xe_vm_bind (vm, vma, op->q,xe_vma_bo (vma), - op->syncs, op->num_syncs, - op->map.immediate || !xe_vm_in_fault_mode (vm), - op->flags &XE_VMA_OP_FIRST , - op->flags &XE_VMA_OP_LAST ); + fence =xe_vm_bind (vm, vma, op->q,xe_vma_bo (vma), + op->syncs, op->num_syncs, + op->map.immediate || + !xe_vm_in_fault_mode (vm), + op->flags &XE_VMA_OP_FIRST , + op->flags &XE_VMA_OP_LAST );break ;case DRM_GPUVA_OP_REMAP : {
@@ -2548,37 +2536,37 @@ static int op_execute(struct xe_vm *vm, struct xe_vma *vma,if (!op->remap.unmap_done) {if (prev || next) vma->gpuva.flags |=XE_VMA_FIRST_REBIND ; - err =xe_vm_unbind (vm, vma, op->q, op->syncs, - op->num_syncs, - op->flags &XE_VMA_OP_FIRST , - op->flags &XE_VMA_OP_LAST && - !prev && !next); -if (err) + fence =xe_vm_unbind (vm, vma, op->q, op->syncs, + op->num_syncs, + op->flags &XE_VMA_OP_FIRST , + op->flags &XE_VMA_OP_LAST && + !prev && !next); +if (IS_ERR (fence))break ; op->remap.unmap_done = true; }if (prev) { op->remap.prev->gpuva.flags |=XE_VMA_LAST_REBIND ; - err =xe_vm_bind (vm, op->remap.prev, op->q, -xe_vma_bo (op->remap.prev), op->syncs, - op->num_syncs, true, false, - op->flags &XE_VMA_OP_LAST && !next); + fence =xe_vm_bind (vm, op->remap.prev, op->q, +xe_vma_bo (op->remap.prev), op->syncs, + op->num_syncs, true, false, + op->flags &XE_VMA_OP_LAST && !next); op->remap.prev->gpuva.flags &= ~XE_VMA_LAST_REBIND ; -if (err) +if (IS_ERR (fence))break ; op->remap.prev =NULL ; }if (next) { op->remap.next->gpuva.flags |=XE_VMA_LAST_REBIND ; - err =xe_vm_bind (vm, op->remap.next, op->q, -xe_vma_bo (op->remap.next), - op->syncs, op->num_syncs, - true, false, - op->flags &XE_VMA_OP_LAST ); + fence =xe_vm_bind (vm, op->remap.next, op->q, +xe_vma_bo (op->remap.next), + op->syncs, op->num_syncs, + true, false, + op->flags &XE_VMA_OP_LAST ); op->remap.next->gpuva.flags &= ~XE_VMA_LAST_REBIND ; -if (err) +if (IS_ERR (fence))break ; op->remap.next =NULL ; }
@@ -2586,61 +2574,66 @@ static int op_execute(struct xe_vm *vm, struct xe_vma *vma,break ; }case DRM_GPUVA_OP_UNMAP : - err =xe_vm_unbind (vm, vma, op->q, op->syncs, - op->num_syncs, op->flags &XE_VMA_OP_FIRST , - op->flags &XE_VMA_OP_LAST ); + fence =xe_vm_unbind (vm, vma, op->q, op->syncs, + op->num_syncs, op->flags &XE_VMA_OP_FIRST , + op->flags &XE_VMA_OP_LAST );break ;case DRM_GPUVA_OP_PREFETCH : - err =xe_vm_prefetch (vm, vma, op->q, op->prefetch.region, - op->syncs, op->num_syncs, - op->flags &XE_VMA_OP_FIRST , - op->flags &XE_VMA_OP_LAST ); + fence =xe_vm_prefetch (vm, vma, op->q, op->prefetch.region, + op->syncs, op->num_syncs, + op->flags &XE_VMA_OP_FIRST , + op->flags &XE_VMA_OP_LAST );break ;default :drm_warn (&vm->xe->drm,"NOT POSSIBLE" ); } -if (err) +if (IS_ERR (fence))trace_xe_vma_fail (vma); -return err; +return fence; } -static int __xe_vma_op_execute (struct xe_vm *vm,struct xe_vma *vma, -struct xe_vma_op *op) +static struct dma_fence * +__xe_vma_op_execute (struct xe_vm *vm,struct xe_vma *vma, +struct xe_vma_op *op) { +struct dma_fence *fence;int err; retry_userptr: - err =op_execute (vm, vma, op); -if (err == -EAGAIN &&xe_vma_is_userptr (vma)) { + fence =op_execute (vm, vma, op); +if (IS_ERR (fence) &&PTR_ERR (fence) == -EAGAIN && +xe_vma_is_userptr (vma)) {lockdep_assert_held_write (&vm->lock); err =xe_vma_userptr_pin_pages (vma);if (!err) goto retry_userptr; + fence =ERR_PTR (err);trace_xe_vma_fail (vma); } -return err; +return fence; } -static int xe_vma_op_execute (struct xe_vm *vm,struct xe_vma_op *op) +static struct dma_fence * +xe_vma_op_execute (struct xe_vm *vm,struct xe_vma_op *op) { -int ret =0 ; +struct dma_fence *fence =ERR_PTR (-ENOMEM );lockdep_assert_held_write (&vm->lock);#ifdef TEST_VM_ASYNC_OPS_ERROR if (op->inject_error) { op->inject_error = false; -return -ENOMEM ; +return fence; }#endif switch (op->base.op) {case DRM_GPUVA_OP_MAP : - ret =__xe_vma_op_execute (vm, op->map.vma, op); + fence =__xe_vma_op_execute (vm, op->map.vma, op);break ;case DRM_GPUVA_OP_REMAP : {
@@ -2653,23 +2646,23 @@ static int xe_vma_op_execute(struct xe_vm *vm, struct xe_vma_op *op)else vma = op->remap.next; - ret =__xe_vma_op_execute (vm, vma, op); + fence =__xe_vma_op_execute (vm, vma, op);break ; }case DRM_GPUVA_OP_UNMAP : - ret =__xe_vma_op_execute (vm,gpuva_to_vma (op->base.unmap.va), - op); + fence =__xe_vma_op_execute (vm,gpuva_to_vma (op->base.unmap.va), + op);break ;case DRM_GPUVA_OP_PREFETCH : - ret =__xe_vma_op_execute (vm, -gpuva_to_vma (op->base.prefetch.va), - op); + fence =__xe_vma_op_execute (vm, +gpuva_to_vma (op->base.prefetch.va), + op);break ;default :drm_warn (&vm->xe->drm,"NOT POSSIBLE" ); } -return ret; +return fence; }static void xe_vma_op_cleanup (struct xe_vm *vm,struct xe_vma_op *op)
@@ -2844,11 +2837,32 @@ static int vm_bind_ioctl_ops_lock(struct drm_exec *exec,return err; } +static struct dma_fence *ops_execute (struct xe_vm *vm, +struct list_head *ops_list, +bool cleanup) +{ +struct xe_vma_op *op, *next; +struct dma_fence *fence =NULL ; + +list_for_each_entry_safe (op, next, ops_list, link) { + fence =xe_vma_op_execute (vm, op); +if (IS_ERR (fence)) { +drm_warn (&vm->xe->drm,"VM op(%d) failed with %ld" , + op->base.op,PTR_ERR (fence)); +return ERR_PTR (-ENOSPC ); + } +if (cleanup) +xe_vma_op_cleanup (vm, op); + } + +return fence; +} +static int vm_bind_ioctl_ops_execute (struct xe_vm *vm,struct list_head *ops_list) {struct drm_exec exec; -struct xe_vma_op *op, *next; +struct dma_fence *fence;int err;lockdep_assert_held_write (&vm->lock);
@@ -2857,23 +2871,17 @@ static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,if (err)return err; -list_for_each_entry_safe (op, next, ops_list, link) { - err =xe_vma_op_execute (vm, op); -if (err) { -drm_warn (&vm->xe->drm,"VM op(%d) failed with %d" , - op->base.op, err); -/* -* FIXME: Killing VM rather than proper error handling -*/ -xe_vm_kill (vm, false); - err = -ENOSPC ; - goto unlock; - } -xe_vma_op_cleanup (vm, op); + fence =ops_execute (vm, ops_list, true); +if (IS_ERR (fence)) { + err =PTR_ERR (fence); +/* FIXME: Killing VM rather than proper error handling */ +xe_vm_kill (vm, false); + }else { +dma_fence_put (fence); } -unlock:drm_exec_fini (&exec); +return err; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 5/26 drm/xe: Move migrate to prefetch to op_lock funtion
In reply to Matthew BrostMigrates need to be done under drm exec to make lockdep happy, move the migrate done for prefetches under the op_lock function.
--- drivers/gpu/drm/xe/xe_vm.c | 30 ++++++++++++++---------------- 1 file changed, 14 insertions(+), 16 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +14 -16
@@ -2041,20 +2041,10 @@ static const u32 region_to_mem_type[] = {static struct dma_fence *xe_vm_prefetch (struct xe_vm *vm,struct xe_vma *vma, -struct xe_exec_queue *q,u32 region, -struct xe_sync_entry *syncs,u32 num_syncs, -bool first_op,bool last_op) +struct xe_exec_queue *q,struct xe_sync_entry *syncs, +u32 num_syncs,bool first_op,bool last_op) {struct xe_exec_queue *wait_exec_queue =to_wait_exec_queue (vm, q); -int err; - -xe_assert (vm->xe, region <=ARRAY_SIZE (region_to_mem_type)); - -if (!xe_vma_has_no_bo (vma)) { - err =xe_bo_migrate (xe_vma_bo (vma), region_to_mem_type[region]); -if (err) -return ERR_PTR (err); - }if (vma->tile_mask != (vma->tile_present & ~vma->usm.tile_invalidated)) { return xe_vm_bind(vm, vma, q, xe_vma_bo(vma), syncs, num_syncs,
@@ -2070,7 +2060,7 @@ xe_vm_prefetch(struct xe_vm *vm, struct xe_vma *vma,xe_sync_entry_signal (&syncs[i],NULL , fence); } -return fence; +return dma_fence_get (fence); } }
@@ -2579,8 +2569,7 @@ static struct dma_fence *op_execute(struct xe_vm *vm, struct xe_vma *vma, op->flags & XE_VMA_OP_LAST);break ;case DRM_GPUVA_OP_PREFETCH : - fence =xe_vm_prefetch (vm, vma, op->q, op->prefetch.region, - op->syncs, op->num_syncs, + fence =xe_vm_prefetch (vm, vma, op->q, op->syncs, op->num_syncs, op->flags &XE_VMA_OP_FIRST , op->flags &XE_VMA_OP_LAST );break ;
@@ -2802,8 +2791,17 @@ static int op_lock(struct drm_exec *exec, struct xe_vm *vm, err =vma_lock (exec,gpuva_to_vma (op->base.unmap.va), false);break ;case DRM_GPUVA_OP_PREFETCH : - err =vma_lock (exec,gpuva_to_vma (op->base.prefetch.va), true); + { +struct xe_vma *vma =gpuva_to_vma (op->base.prefetch.va); +u32 region = op->prefetch.region; + +xe_assert (vm->xe, region <=ARRAY_SIZE (region_to_mem_type)); + + err =vma_lock (exec, vma, false); +if (!err && !xe_vma_has_no_bo (vma)) + err =xe_bo_migrate (xe_vma_bo (vma), region);break ; + }default :drm_warn (&vm->xe->drm,"NOT POSSIBLE" ); }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 6/26 drm/xe: Add struct xe_vma_ops abstraction
In reply to Matthew BrostThis will help with upcoming change to 1 job for list VMA operations.
--- drivers/gpu/drm/xe/xe_vm.c | 37 ++++++++++++++++++-------------- drivers/gpu/drm/xe/xe_vm_types.h | 5 +++++ 2 files changed, 26 insertions(+), 16 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +21 -16
@@ -2356,7 +2356,7 @@ static int xe_vma_op_commit(struct xe_vm *vm, struct xe_vma_op *op)static int vm_bind_ioctl_ops_parse (struct xe_vm *vm,struct xe_exec_queue *q,struct drm_gpuva_ops *ops,struct xe_sync_entry *syncs,u32 num_syncs, -struct list_head *ops_list,bool last, +struct xe_vma_ops *vops,bool last,bool async) {struct xe_vma_op *last_op =NULL ;
@@ -2367,10 +2367,10 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,drm_gpuva_for_each_op (__op, ops) {struct xe_vma_op *op =gpuva_op_to_vma_op (__op); -bool first =list_empty (ops_list); +bool first =list_empty (&vops->list);INIT_LIST_HEAD (&op->link); -list_add_tail (&op->link, ops_list); +list_add_tail (&op->link, &vops->list);if (first) { op->flags |=XE_VMA_OP_FIRST ;
@@ -2485,7 +2485,7 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q, }/* FIXME: Unhandled corner case */ -XE_WARN_ON (!last_op && last && !list_empty (ops_list)); +XE_WARN_ON (!last_op && last && !list_empty (&vops->list));if (!last_op)return 0 ;
@@ -2811,7 +2811,7 @@ static int op_lock(struct drm_exec *exec, struct xe_vm *vm,static int vm_bind_ioctl_ops_lock (struct drm_exec *exec,struct xe_vm *vm, -struct list_head *ops_list) +struct xe_vma_ops *vops) {struct xe_vma_op *op;int err;
@@ -2823,7 +2823,7 @@ static int vm_bind_ioctl_ops_lock(struct drm_exec *exec,if (err) goto out; -list_for_each_entry (op, ops_list, link) { +list_for_each_entry (op, &vops->list, link) { err =op_lock (exec, vm, op);drm_exec_retry_on_contention (exec);if (err)
@@ -2836,13 +2836,13 @@ static int vm_bind_ioctl_ops_lock(struct drm_exec *exec, }static struct dma_fence *ops_execute (struct xe_vm *vm, -struct list_head *ops_list, +struct xe_vma_ops *vops,bool cleanup) {struct xe_vma_op *op, *next;struct dma_fence *fence =NULL ; -list_for_each_entry_safe (op, next, ops_list, link) { +list_for_each_entry_safe (op, next, &vops->list, link) { fence =xe_vma_op_execute (vm, op);if (IS_ERR (fence)) { drm_warn(&vm->xe->drm, "VM op(%d) failed with %ld",
@@ -2857,7 +2857,7 @@ static struct dma_fence *ops_execute(struct xe_vm *vm, }static int vm_bind_ioctl_ops_execute (struct xe_vm *vm, -struct list_head *ops_list) +struct xe_vma_ops *vops) {struct drm_exec exec;struct dma_fence *fence;
@@ -2865,11 +2865,11 @@ static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,lockdep_assert_held_write (&vm->lock); - err =vm_bind_ioctl_ops_lock (&exec, vm, ops_list); + err =vm_bind_ioctl_ops_lock (&exec, vm, vops);if (err)return err; - fence =ops_execute (vm, ops_list, true); + fence =ops_execute (vm, vops, true);if (IS_ERR (fence)) { err =PTR_ERR (fence);/* FIXME: Killing VM rather than proper error handling */
@@ -2999,6 +2999,11 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe,return err; } +static void xe_vma_ops_init (struct xe_vma_ops *vops) +{ +INIT_LIST_HEAD (&vops->list); +} +int xe_vm_bind_ioctl (struct drm_device *dev,void *data,struct drm_file *file) {struct xe_device *xe =to_xe_device (dev);
@@ -3012,7 +3017,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)u32 num_syncs;struct xe_sync_entry *syncs =NULL ;struct drm_xe_vm_bind_op *bind_ops; -LIST_HEAD (ops_list); +struct xe_vma_ops vops;bool async;int err;int i;
@@ -3155,6 +3160,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto free_syncs; } +xe_vma_ops_init (&vops);for (i =0 ; i < args->num_binds; ++i) {u64 range = bind_ops[i].range;u64 addr = bind_ops[i].addr;
@@ -3174,15 +3180,14 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) } err =vm_bind_ioctl_ops_parse (vm, q, ops[i], syncs, num_syncs, - &ops_list, - i == args->num_binds -1 , + &vops, i == args->num_binds -1 , async);if (err) goto unwind_ops; }/* Nothing to do */ -if (list_empty (&ops_list)) { +if (list_empty (&vops.list)) { err = -ENODATA ; goto unwind_ops; }
@@ -3191,7 +3196,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file)if (q)xe_exec_queue_get (q); - err =vm_bind_ioctl_ops_execute (vm, &ops_list); + err =vm_bind_ioctl_ops_execute (vm, &vops);up_write (&vm->lock);
drivers/gpu/drm/xe/xe_vm_types.h +5 -0
@@ -420,4 +420,9 @@ struct xe_vma_op { }; }; +/** struct xe_vma_ops - VMA operations */ +struct xe_vma_ops { +struct list_head list; +}; +#endif
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 7/26 drm/xe: Update xe_vm_rebind to use dummy VMA operations
In reply to Matthew BrostAll bind interfaces are transitioning to use ops.
--- drivers/gpu/drm/xe/xe_vm.c | 43 +++++-- drivers/gpu/drm/xe/xe_vm_types.h | 199 ++++++++++++++++--------------- 2 files changed, 134 insertions(+), 108 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +31 -12
@@ -829,10 +829,23 @@ int xe_vm_userptr_check_repin(struct xe_vm *vm) list_empty_careful(&vm->userptr.invalidated)) ? 0 : -EAGAIN; } -static struct dma_fence * -xe_vm_bind_vma (struct xe_vma *vma,struct xe_exec_queue *q, -struct xe_sync_entry *syncs,u32 num_syncs, -bool first_op,bool last_op); +static void xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma) +{ + vm->dummy_ops.op.base.op =DRM_GPUVA_OP_MAP ; + vm->dummy_ops.op.base.map.va.addr = vma->gpuva.va.addr; + vm->dummy_ops.op.base.map.va.range = vma->gpuva.va.range; + vm->dummy_ops.op.base.map.gem.obj = vma->gpuva.gem.obj; + vm->dummy_ops.op.base.map.gem.offset = vma->gpuva.gem.offset; + vm->dummy_ops.op.tile_mask = vma->tile_present; + vm->dummy_ops.op.map.vma = vma; + vm->dummy_ops.op.map.immediate = true; + vm->dummy_ops.op.map.read_only =xe_vma_read_only (vma); + vm->dummy_ops.op.map.is_null =xe_vma_is_null (vma); +} + +static struct dma_fence *ops_execute (struct xe_vm *vm, +struct xe_vma_ops *vops, +bool cleanup);struct dma_fence *xe_vm_rebind (struct xe_vm *vm,bool rebind_worker) {
@@ -854,7 +867,9 @@ struct dma_fence *xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)trace_xe_vma_rebind_worker (vma); elsetrace_xe_vma_rebind_exec (vma ); - fence =xe_vm_bind_vma (vma,NULL ,NULL ,0 , false, false); + +xe_vm_populate_dummy_rebind (vm, vma); + fence =ops_execute (vm, &vm->dummy_ops.vops, false);if (IS_ERR (fence))return fence; }
@@ -1333,6 +1348,11 @@ static const struct xe_pt_ops xelp_pt_ops = {static void vm_destroy_work_func (struct work_struct *w); +static void xe_vma_ops_init (struct xe_vma_ops *vops) +{ +INIT_LIST_HEAD (&vops->list); +} +struct xe_vm *xe_vm_create (struct xe_device *xe,u32 flags) {struct xe_vm *vm;
@@ -1354,6 +1374,10 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)init_rwsem (&vm->lock); +xe_vma_ops_init (&vm->dummy_ops.vops); +INIT_LIST_HEAD (&vm->dummy_ops.op.link); +list_add (&vm->dummy_ops.op.link, &vm->dummy_ops.vops.list); +INIT_LIST_HEAD (&vm->rebind_list);INIT_LIST_HEAD (&vm->userptr.repin_list);
@@ -2505,7 +2529,7 @@ static struct dma_fence *op_execute(struct xe_vm *vm, struct xe_vma *vma, {struct dma_fence *fence; -lockdep_assert_held_write (&vm->lock); +lockdep_assert_held (&vm->lock);xe_vm_assert_held (vm);xe_bo_assert_held (xe_vma_bo (vma));
@@ -2611,7 +2635,7 @@ xe_vma_op_execute(struct xe_vm *vm, struct xe_vma_op *op) {struct dma_fence *fence =ERR_PTR (-ENOMEM ); -lockdep_assert_held_write (&vm->lock); +lockdep_assert_held (&vm->lock);#ifdef TEST_VM_ASYNC_OPS_ERROR if (op->inject_error) {
@@ -2999,11 +3023,6 @@ static int vm_bind_ioctl_check_args(struct xe_device *xe,return err; } -static void xe_vma_ops_init (struct xe_vma_ops *vops) -{ -INIT_LIST_HEAD (&vops->list); -} -int xe_vm_bind_ioctl (struct drm_device *dev,void *data,struct drm_file *file) {struct xe_device *xe =to_xe_device (dev);
drivers/gpu/drm/xe/xe_vm_types.h +103 -96
221 lines not on this page · raw message
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 8/26 drm/xe: Move drm exec loop to vm_bind_ioctl_ops_execute
In reply to Matthew BrostThe execution of the ops list can allocate memory in TTM which can trigger an eviction. Lockdep is not happy if the drm exec is closed when this happens.
--- drivers/gpu/drm/xe/xe_vm.c | 46 ++++++++++++++++++-------------------- 1 file changed, 22 insertions(+), 24 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +22 -24
@@ -2840,23 +2840,17 @@ static int vm_bind_ioctl_ops_lock(struct drm_exec *exec,struct xe_vma_op *op;int err; -drm_exec_init (exec,DRM_EXEC_INTERRUPTIBLE_WAIT ); -drm_exec_until_all_locked (exec) { - err =drm_exec_prepare_obj (exec, &xe_vm_ttm_bo (vm)->base,1 ); -drm_exec_retry_on_contention (exec); -if (err) - goto out; + err =drm_exec_prepare_obj (exec, &xe_vm_ttm_bo (vm)->base,1 ); +if (err) +return err; -list_for_each_entry (op, &vops->list, link) { - err =op_lock (exec, vm, op); -drm_exec_retry_on_contention (exec); -if (err) - goto out; - } +list_for_each_entry (op, &vops->list, link) { + err =op_lock (exec, vm, op); +if (err) +return err; } -out: -return err; +return 0 ; } static struct dma_fence *ops_execute(struct xe_vm *vm,
@@ -2889,17 +2883,21 @@ static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,lockdep_assert_held_write (&vm->lock); - err =vm_bind_ioctl_ops_lock (&exec, vm, vops); -if (err) -return err; +drm_exec_init (&exec,DRM_EXEC_INTERRUPTIBLE_WAIT ); + drm_exec_until_all_locked(&exec) { + err =vm_bind_ioctl_ops_lock (&exec, vm, vops); +drm_exec_retry_on_contention (&exec); +if (err) +return err; - fence =ops_execute (vm, vops, true); -if (IS_ERR (fence)) { - err =PTR_ERR (fence); -/* FIXME: Killing VM rather than proper error handling */ -xe_vm_kill (vm, false); - }else { -dma_fence_put (fence); + fence =ops_execute (vm, vops, true); +if (IS_ERR (fence)) { + err =PTR_ERR (fence); +/* FIXME: Killing VM rather than proper error handling */ +xe_vm_kill (vm, false); + }else { +dma_fence_put (fence); + } }drm_exec_fini (&exec);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 9/26 drm/xe: Fixup error handling / ref counting in VM bind IOCTL
In reply to Matthew BrostPart of the rework for the error handling.
--- drivers/gpu/drm/xe/xe_vm.c | 62 ++++++-------------------------- drivers/gpu/drm/xe/xe_vm_types.h | 9 ++--- 2 files changed, 12 insertions(+), 59 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +10 -52
@@ -844,8 +844,7 @@ static void xe_vm_populate_dummy_rebind(struct xe_vm *vm, struct xe_vma *vma) }static struct dma_fence *ops_execute (struct xe_vm *vm, -struct xe_vma_ops *vops, -bool cleanup); +struct xe_vma_ops *vops);struct dma_fence *xe_vm_rebind (struct xe_vm *vm,bool rebind_worker) {
@@ -869,7 +868,7 @@ struct dma_fence *xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)trace_xe_vma_rebind_exec (vma);xe_vm_populate_dummy_rebind (vm, vma); - fence =ops_execute (vm, &vm->dummy_ops.vops, false); + fence =ops_execute (vm, &vm->dummy_ops.vops);if (IS_ERR (fence))return fence; }
@@ -2514,7 +2513,6 @@ static int vm_bind_ioctl_ops_parse(struct xe_vm *vm, struct xe_exec_queue *q,if (!last_op)return 0 ; - last_op->ops = ops;if (last) { last_op->flags |=XE_VMA_OP_LAST ; last_op->num_syncs = num_syncs;
@@ -2678,25 +2676,6 @@ xe_vma_op_execute(struct xe_vm *vm, struct xe_vma_op *op)return fence; } -static void xe_vma_op_cleanup (struct xe_vm *vm,struct xe_vma_op *op) -{ -bool last = op->flags &XE_VMA_OP_LAST ; - -if (last) { -while (op->num_syncs--) -xe_sync_entry_cleanup (&op->syncs[op->num_syncs]); -kfree (op->syncs); -if (op->q) -xe_exec_queue_put (op->q); - } -if (!list_empty (&op->link)) -list_del (&op->link); -if (op->ops) -drm_gpuva_ops_free (&vm->gpuvm, op->ops); -if (last) -xe_vm_put (vm); -} -static void xe_vma_op_unwind (struct xe_vm *vm,struct xe_vma_op *op,bool post_commit,bool prev_post_commit,bool next_post_commit)
@@ -2758,7 +2737,7 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, {int i; -for (i = num_ops_list -1 ; i; ++i) { +for (i = num_ops_list -1 ; i >=0 ; --i) {struct drm_gpuva_ops *__ops = ops[i];struct drm_gpuva_op *__op;
@@ -2773,8 +2752,6 @@ static void vm_bind_ioctl_ops_unwind(struct xe_vm *vm, op->flags &XE_VMA_OP_PREV_COMMITTED , op->flags & XE_VMA_OP_NEXT_COMMITTED); } - -drm_gpuva_ops_free (&vm->gpuvm, __ops); } }
@@ -2854,8 +2831,7 @@ static int vm_bind_ioctl_ops_lock(struct drm_exec *exec, }static struct dma_fence *ops_execute (struct xe_vm *vm, -struct xe_vma_ops *vops, -bool cleanup) +struct xe_vma_ops *vops) {struct xe_vma_op *op, *next;struct dma_fence *fence =NULL ;
@@ -2867,8 +2843,6 @@ static struct dma_fence *ops_execute(struct xe_vm *vm, op->base.op, PTR_ERR(fence));return ERR_PTR (-ENOSPC ); } -if (cleanup) -xe_vma_op_cleanup (vm, op); }return fence;
@@ -2890,7 +2864,7 @@ static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,if (err)return err; - fence =ops_execute (vm, vops, true); + fence =ops_execute (vm, vops);if (IS_ERR (fence)) { err =PTR_ERR (fence);/* FIXME: Killing VM rather than proper error handling */
@@ -3209,30 +3183,14 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto unwind_ops; } -xe_vm_get (vm); -if (q) -xe_exec_queue_get (q); - err =vm_bind_ioctl_ops_execute (vm, &vops); -up_write (&vm->lock); - -if (q) -xe_exec_queue_put (q); -xe_vm_put (vm); - -for (i =0 ; bos && i < args->num_binds; ++i) -xe_bo_put (bos[i]); - -kfree (bos); -kfree (ops); -if (args->num_binds >1 ) -kfree (bind_ops); - -return err; - unwind_ops: -vm_bind_ioctl_ops_unwind (vm, ops, args->num_binds); +if (err && err != -ENODATA ) +vm_bind_ioctl_ops_unwind (vm, ops, args->num_binds); +for (i = args->num_binds -1 ; i >=0 ; --i) +if (ops[i]) +drm_gpuva_ops_free (&vm->gpuvm, ops[i]); free_syncs:for (i =0 ; err == -ENODATA && i < num_syncs; i++) { struct dma_fence *fence =
drivers/gpu/drm/xe/xe_vm_types.h +2 -7
@@ -184,11 +184,6 @@ enum xe_vma_op_flags {struct xe_vma_op {/** @base: GPUVA base operation */ struct drm_gpuva_op base; -/** -* @ops: GPUVA ops, when set call drm_gpuva_ops_free after this -* operations is processed -*/ -struct drm_gpuva_ops *ops;/** @q: exec queue for this operation */ struct xe_exec_queue *q;/**
@@ -198,9 +193,9 @@ struct xe_vma_op {struct xe_sync_entry *syncs;/** @num_syncs: number of syncs */ u32 num_syncs; -/** @link: async operation link */ +/** @link: operation link */ struct list_head link; -/** @tile_mask: gt mask for this operation */ +/** @tile_mask: tile mask for this operation */ u8 tile_mask;/** @flags: operation flags */ enum xe_vma_op_flags flags;
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 10/26 drm/xe: Convert pagefault rebind to use ops interface
In reply to Matthew BrostCalling into the VM layer is correct not the PT layer. Also all bind moving towards VMA operations interfaces.
--- drivers/gpu/drm/xe/xe_gt_pagefault.c | 7 ++++--- drivers/gpu/drm/xe/xe_vm.c | 12 ++++-------- drivers/gpu/drm/xe/xe_vm.h | 3 +++ 3 files changed, 11 insertions(+), 11 deletions(-)
drivers/gpu/drm/xe/xe_gt_pagefault.c +4 -3
@@ -18,7 +18,6 @@#include "xe_guc.h" #include "xe_guc_ct.h" #include "xe_migrate.h" -#include "xe_pt.h" #include "xe_trace.h" #include "xe_vm.h"
@@ -204,8 +203,10 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf)/* Bind VMA only to the GT that has faulted */ trace_xe_vma_pf_bind (vma); - fence =__xe_pt_bind_vma (tile, vma,xe_tile_migrate_engine (tile),NULL ,0 , - vma->tile_present &BIT (tile->id)); +xe_vm_populate_dummy_rebind (vm, vma); + vm->dummy_ops.op.tile_mask =BIT (tile->id); + vm->dummy_ops.op.q =xe_tile_migrate_engine (tile); + fence =xe_vm_ops_execute (vm, &vm->dummy_ops.vops);if (IS_ERR (fence)) { ret =PTR_ERR (fence); goto unlock_dma_resv;
drivers/gpu/drm/xe/xe_vm.c +4 -8
@@ -829,7 +829,7 @@ int xe_vm_userptr_check_repin(struct xe_vm *vm) list_empty_careful(&vm->userptr.invalidated)) ? 0 : -EAGAIN; } -static void xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma) +void xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma) { vm->dummy_ops.op.base.op =DRM_GPUVA_OP_MAP ; vm->dummy_ops.op.base.map.va.addr = vma->gpuva.va.addr;
@@ -843,9 +843,6 @@ static void xe_vm_populate_dummy_rebind(struct xe_vm *vm, struct xe_vma *vma) vm->dummy_ops.op.map.is_null =xe_vma_is_null (vma); } -static struct dma_fence *ops_execute (struct xe_vm *vm, -struct xe_vma_ops *vops); -struct dma_fence *xe_vm_rebind (struct xe_vm *vm,bool rebind_worker) {struct dma_fence *fence =NULL ;
@@ -868,7 +865,7 @@ struct dma_fence *xe_vm_rebind(struct xe_vm *vm, bool rebind_worker)trace_xe_vma_rebind_exec (vma);xe_vm_populate_dummy_rebind (vm, vma); - fence =ops_execute (vm, &vm->dummy_ops.vops); + fence =xe_vm_ops_execute (vm, &vm->dummy_ops.vops);if (IS_ERR (fence))return fence; }
@@ -2830,8 +2827,7 @@ static int vm_bind_ioctl_ops_lock(struct drm_exec *exec,return 0 ; } -static struct dma_fence *ops_execute (struct xe_vm *vm, -struct xe_vma_ops *vops) +struct dma_fence *xe_vm_ops_execute (struct xe_vm *vm,struct xe_vma_ops *vops) {struct xe_vma_op *op, *next;struct dma_fence *fence =NULL ;
@@ -2864,7 +2860,7 @@ static int vm_bind_ioctl_ops_execute(struct xe_vm *vm,if (err)return err; - fence =ops_execute (vm, vops); + fence =xe_vm_ops_execute (vm, vops);if (IS_ERR (fence)) { err =PTR_ERR (fence);/* FIXME: Killing VM rather than proper error handling */
drivers/gpu/drm/xe/xe_vm.h +3 -0
@@ -224,6 +224,9 @@ int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);int xe_vm_prepare_vma (struct drm_exec *exec,struct xe_vma *vma,unsigned int num_shared); +void xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma); +struct dma_fence *xe_vm_ops_execute (struct xe_vm *vm,struct xe_vma_ops *vops); +#if IS_ENABLED (CONFIG_DRM_XE_DEBUG_VM )#define vm_dbg drm_dbg#else
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 11/26 drm/xe: Add some members to xe_vma_ops
In reply to Matthew BrostThis will help with moving to single jobs for many bind operations.
--- drivers/gpu/drm/xe/xe_vm.c | 13 ++++++++++--- drivers/gpu/drm/xe/xe_vm_types.h | 9 +++++++++ 2 files changed, 19 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +10 -3
@@ -1344,9 +1344,16 @@ static const struct xe_pt_ops xelp_pt_ops = {static void vm_destroy_work_func (struct work_struct *w); -static void xe_vma_ops_init (struct xe_vma_ops *vops) +static void xe_vma_ops_init (struct xe_vma_ops *vops,struct xe_vm *vm, +struct xe_exec_queue *q, +struct xe_sync_entry *syncs,u32 num_syncs) { +memset (vops,0 ,sizeof (*vops));INIT_LIST_HEAD (&vops->list); + vops->vm = vm; + vops->q = q; + vops->syncs = syncs; + vops->num_syncs = num_syncs; }struct xe_vm *xe_vm_create (struct xe_device *xe,u32 flags)
@@ -1370,7 +1377,7 @@ struct xe_vm *xe_vm_create(struct xe_device *xe, u32 flags)init_rwsem (&vm->lock); -xe_vma_ops_init (&vm->dummy_ops.vops); +xe_vma_ops_init (&vm->dummy_ops.vops, vm,NULL ,NULL ,0 );INIT_LIST_HEAD (&vm->dummy_ops.op.link);list_add (&vm->dummy_ops.op.link, &vm->dummy_ops.vops.list);
@@ -3147,7 +3154,7 @@ int xe_vm_bind_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto free_syncs; } -xe_vma_ops_init (&vops); +xe_vma_ops_init (&vops, vm, q, syncs, num_syncs);for (i =0 ; i < args->num_binds; ++i) {u64 range = bind_ops[i].range;u64 addr = bind_ops[i].addr;
drivers/gpu/drm/xe/xe_vm_types.h +9 -0
@@ -217,7 +217,16 @@ struct xe_vma_op {/** struct xe_vma_ops - VMA operations */ struct xe_vma_ops { +/** @list: list of VMA operations */ struct list_head list; +/** @vm: VM */ +struct xe_vm *vm; +/** @q: exec queue these operations */ +struct xe_exec_queue *q; +/** @syncs: syncs these operation */ +struct xe_sync_entry *syncs; +/** @num_syncs: number of syncs */ +u32 num_syncs; };#define xe_vm_assert_held (vm) dma_resv_assert_held(&(vm)->resv)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 12/26 drm/xe: Add vm_bind_ioctl_ops_install_fences helper
In reply to Matthew BrostWill help with transition single job for many bind ops. Also fix some reference counting errors.
--- drivers/gpu/drm/xe/xe_exec.c | 1 + drivers/gpu/drm/xe/xe_exec_queue.c | 8 +++-- drivers/gpu/drm/xe/xe_vm.c | 58 +++++++++++++----------------- 3 files changed, 31 insertions(+), 36 deletions(-)
drivers/gpu/drm/xe/xe_exec.c +1 -0
@@ -243,6 +243,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)for (i =0 ; i < num_syncs; i++)xe_sync_entry_signal (&syncs[i],NULL , fence); +dma_fence_put (fence); } goto err_exec;
drivers/gpu/drm/xe/xe_exec_queue.c +6 -2
@@ -961,20 +961,24 @@ void xe_exec_queue_last_fence_put_unlocked(struct xe_exec_queue *q)* @q: The exec queue * @vm: The VM the engine does a bind or exec for * -* Get last fence, does not take a ref +* Get last fence, takes a ref * * Returns: last fence if not signaled, dma fence stub if signaled */ struct dma_fence *xe_exec_queue_last_fence_get (struct xe_exec_queue *q,struct xe_vm *vm) { +struct dma_fence *fence; +xe_exec_queue_last_fence_lockdep_assert (q, vm);if (q->last_fence &&test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &q->last_fence->flags))xe_exec_queue_last_fence_put (q, vm); -return q->last_fence ? q->last_fence :dma_fence_get_stub (); + fence = q->last_fence ? q->last_fence :dma_fence_get_stub (); +dma_fence_get (fence); +return fence; }/**
drivers/gpu/drm/xe/xe_vm.c +24 -34
120 lines not on this page · raw message
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 13/26 drm/xe: Drop rebind argument from xe_pt_prepare_bind
In reply to Matthew BrostThis is unused, drop it.
--- drivers/gpu/drm/xe/xe_pt.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_pt.c +2 -3
@@ -914,8 +914,7 @@ static void xe_pt_commit_bind(struct xe_vma *vma,static int xe_pt_prepare_bind (struct xe_tile *tile,struct xe_vma *vma, -struct xe_vm_pgtable_update *entries,u32 *num_entries, -bool rebind) +struct xe_vm_pgtable_update *entries,u32 *num_entries) {int err;
@@ -1271,7 +1270,7 @@ __xe_pt_bind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue"Preparing bind, with range [%llx...%llx) engine %p.\n" , xe_vma_start(vma), xe_vma_end(vma), q); - err =xe_pt_prepare_bind (tile, vma, entries, &num_entries, rebind); + err =xe_pt_prepare_bind (tile, vma, entries, &num_entries);if (err) goto err;xe_tile_assert (tile, num_entries <=ARRAY_SIZE (entries));
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 14/26 drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops
In reply to Matthew BrostWill help with the converstion to 1 job per VM bind IOCTL. Allocation only implemented in this patch.
--- drivers/gpu/drm/xe/xe_pt_types.h | 12 ++++++ drivers/gpu/drm/xe/xe_vm.c | 64 ++++++++++++++++++++++++++++++-- drivers/gpu/drm/xe/xe_vm_types.h | 8 ++++ 3 files changed, 80 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_pt_types.h +12 -0
@@ -73,4 +73,16 @@ struct xe_vm_pgtable_update { u32 flags; }; +/** struct xe_vm_pgtable_update_op - Page table update operation */ +struct xe_vm_pgtable_update_op { +/** @entries: entries to update for this operation */ +struct xe_vm_pgtable_update entries[XE_VM_MAX_LEVEL *2 +1 ]; +/** @num_entries: number of entries for this update operation */ +u32 num_entries; +/** @bind: is a bind */ +bool bind; +/** @rebind: is a rebind */ +bool rebind; +}; +#endif
drivers/gpu/drm/xe/xe_vm.c +60 -4
133 lines not on this page · raw message
drivers/gpu/drm/xe/xe_vm_types.h +8 -0
@@ -21,6 +21,7 @@ struct xe_bo;struct xe_device ;struct xe_sync_entry ;struct xe_vm ; +struct xe_vm_pgtable_update_op ;#define TEST_VM_ASYNC_OPS_ERROR #define FORCE_ASYNC_OP_ERROR BIT(31)
@@ -227,6 +228,13 @@ struct xe_vma_ops {struct xe_sync_entry *syncs;/** @num_syncs: number of syncs */ u32 num_syncs; +/** @pt_update_ops: page table update operations */ +struct { +/** @ops: operations */ +struct xe_vm_pgtable_update_op *ops; +/** @num_ops: number of operations */ +u32 num_ops; + } pt_update_ops[XE_MAX_TILES_PER_DEVICE ]; };#define xe_vm_assert_held (vm) dma_resv_assert_held(&(vm)->resv)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 15/26 drm/xe: Move setting last fence and sync wait to vm_bind_ioctl_ops_install_fences
In reply to Matthew BrostThis moves setting of the last fence and sync wait to a single location.
--- drivers/gpu/drm/xe/xe_vm.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +4 -11
@@ -1922,11 +1922,6 @@ xe_vm_bind(struct xe_vm *vm, struct xe_vma *vma, struct xe_exec_queue *q, fence =xe_exec_queue_last_fence_get (wait_exec_queue, vm); } -if (last_op) -xe_exec_queue_last_fence_set (wait_exec_queue, vm, fence); -if (last_op &&xe_vm_sync_mode (vm, q)) -dma_fence_wait (fence, true); -return fence; }
@@ -1936,7 +1931,6 @@ xe_vm_unbind(struct xe_vm *vm, struct xe_vma *vma, u32 num_syncs, bool first_op, bool last_op) { struct dma_fence *fence; -struct xe_exec_queue *wait_exec_queue =to_wait_exec_queue (vm, q);xe_vm_assert_held (vm);xe_bo_assert_held (xe_vma_bo (vma));
@@ -1945,11 +1939,6 @@ xe_vm_unbind(struct xe_vm *vm, struct xe_vma *vma,if (IS_ERR (fence))return fence; -if (last_op) -xe_exec_queue_last_fence_set (wait_exec_queue, vm, fence); -if (last_op &&xe_vm_sync_mode (vm, q)) -dma_fence_wait (fence, true); -return fence; }
@@ -2876,6 +2865,7 @@ static void vm_bind_ioctl_ops_install_fences(struct xe_vm *vm,struct xe_vma_ops *vops, struct dma_fence *fence) { + struct xe_exec_queue *wait_exec_queue = to_wait_exec_queue(vm, vops->q);struct xe_vma_op *op;int i;
@@ -2888,6 +2878,9 @@ static void vm_bind_ioctl_ops_install_fences(struct xe_vm *vm, }for (i =0 ; i < vops->num_syncs; i++)xe_sync_entry_signal (vops->syncs + i,NULL , fence); +xe_exec_queue_last_fence_set (wait_exec_queue, vm, fence); +if (xe_vm_sync_mode (vm, vops->q)) +dma_fence_wait (fence, true);dma_fence_put (fence); }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 16/26 drm/xe: Fix vma_is_valid to use tile argument
In reply to Matthew BrostWe convert gt to tile in vma_is_valid. Rather than so this just pass in the tile.
--- drivers/gpu/drm/xe/xe_gt_pagefault.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_gt_pagefault.c +4 -4
@@ -69,10 +69,10 @@ static bool access_is_atomic(enum access_type access_type)return access_type ==ACCESS_TYPE_ATOMIC ; } -static bool vma_is_valid (struct xe_gt *gt,struct xe_vma *vma) +static bool vma_is_valid (struct xe_tile *tile,struct xe_vma *vma) { -return BIT (gt_to_tile (gt)->id) & vma->tile_present && - !(BIT (gt->info.id) & vma->usm.tile_invalidated); +return BIT (tile->id) & vma->tile_present && + !(BIT (tile->id) & vma->usm.tile_invalidated); }static bool vma_matches (struct xe_vma *vma,u64 page_addr)
@@ -174,7 +174,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf) atomic =access_is_atomic (pf->access_type);/* Check if VMA is valid */ -if (vma_is_valid (gt, vma) && !atomic) +if (vma_is_valid (gt_to_tile (gt), vma) && !atomic) goto unlock_vm;/* TODO: Validate fault */
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 17/26 drm/xe: Adjust tile mask in operations create
In reply to Matthew BrostThis way the operations, not just created VMA, will have the correct tile mask.
--- drivers/gpu/drm/xe/xe_vm.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +16 -9
@@ -882,8 +882,6 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm, u8 tile_mask) {struct xe_vma *vma; -struct xe_tile *tile; -u8 id;xe_assert (vm->xe, start < end);xe_assert (vm->xe, end < vm->size);
@@ -910,13 +908,7 @@ static struct xe_vma *xe_vma_create(struct xe_vm *vm, vma->gpuva.flags |=XE_VMA_READ_ONLY ;if (is_null) vma->gpuva.flags |=DRM_GPUVA_SPARSE ; - -if (tile_mask) { - vma->tile_mask = tile_mask; - }else { -for_each_tile (tile, vm->xe, id) - vma->tile_mask |=0x1 << id; - } + vma->tile_mask = tile_mask;if (GRAPHICS_VER (vm->xe) >=20 || vm->xe->info.platform ==XE_PVC ) vma->gpuva.flags |=XE_VMA_ATOMIC_PTE_BIT ;
@@ -2163,6 +2155,19 @@ static void print_op(struct xe_device *xe, struct drm_gpuva_op *op) }#endif +static u8 adj_tile_mask (struct xe_device *xe,u8 tile_mask) +{ +struct xe_tile *tile; +u8 id; + +if (!tile_mask) { +for_each_tile (tile, xe, id) + tile_mask |=0x1 << id; + } + +return tile_mask; +} +/* * Create operations list from IOCTL arguments, setup operations fields so parse * and commit steps are decoupled from IOCTL arguments. This step can fail.
@@ -2185,6 +2190,8 @@ vm_bind_ioctl_ops_create(struct xe_vm *vm, struct xe_bo *bo, operation, (ULL )addr, (ULL )range, (ULL)bo_offset_or_userptr); + tile_mask =adj_tile_mask (vm->xe, tile_mask); +switch (operation) {case XE_VM_BIND_OP_MAP :case XE_VM_BIND_OP_MAP_USERPTR :
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 18/26 drm/xe: Add xe_gt_tlb_invalidation_range and convert PT layer to use this
In reply to Matthew Brostxe_gt_tlb_invalidation_range accepts a start and end address rather than a VMA. This will enable multiple VMAs to be invalidated in a single invalidation. Update the PT layer to use this new function.
--- drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c | 60 +++++++++++++++------ drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h | 3 ++ drivers/gpu/drm/xe/xe_pt.c | 25 ++++++--- 3 files changed, 66 insertions(+), 22 deletions(-)
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c +45 -15
@@ -227,11 +227,15 @@ int xe_gt_tlb_invalidation_guc(struct xe_gt *gt) }/** -* xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA +* xe_gt_tlb_invalidation_range - Issue a TLB invalidation on this GT for an +* address range +* * @gt: graphics tile * @fence: invalidation fence which will be signal on TLB invalidation * completion, can be NULL -* @vma: VMA to invalidate +* @start: start address +* @end: end address +* @asid: address space id * * Issue a range based TLB invalidation if supported, if not fallback to a full * TLB invalidation. Completion of TLB is asynchronous and caller can either use
@@ -241,24 +245,22 @@ int xe_gt_tlb_invalidation_guc(struct xe_gt *gt)* Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, * negative error code on error. */ -int xe_gt_tlb_invalidation_vma (struct xe_gt *gt, -struct xe_gt_tlb_invalidation_fence *fence, -struct xe_vma *vma) +int xe_gt_tlb_invalidation_range (struct xe_gt *gt, +struct xe_gt_tlb_invalidation_fence *fence, +u64 start,u64 end,u32 asid) {struct xe_device *xe =gt_to_xe (gt);#define MAX_TLB_INVALIDATION_LEN 7u32 action[MAX_TLB_INVALIDATION_LEN ];int len =0 ; -xe_gt_assert (gt, vma); - action[len++] =XE_GUC_ACTION_TLB_INVALIDATION ; action[len++] =0 ;/* seqno, replaced in send_tlb_invalidation */ if (!xe->info.has_range_tlb_invalidation) { action[len++] =MAKE_INVAL_OP (XE_GUC_TLB_INVAL_FULL ); }else { -u64 start =xe_vma_start (vma); -u64 length =xe_vma_size (vma); +u64 orig_start = start; +u64 length = end - start;u64 align, end;if (length <SZ_4K )
@@ -271,12 +273,12 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,* address mask covering the required range. */ align =roundup_pow_of_two (length); - start =ALIGN_DOWN (xe_vma_start (vma), align); - end =ALIGN (xe_vma_end (vma), align); + start =ALIGN_DOWN (start, align); + end =ALIGN (end, align); length = align;while (start + length < end) { length <<=1 ; - start =ALIGN_DOWN (xe_vma_start (vma), length); + start =ALIGN_DOWN (orig_start, length); }/*
@@ -285,16 +287,17 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,*/ if (length >=SZ_2M ) { length =max_t (u64,SZ_16M , length); - start =ALIGN_DOWN (xe_vma_start (vma), length); + start =ALIGN_DOWN (orig_start, length); }xe_gt_assert (gt, length >=SZ_4K );xe_gt_assert (gt,is_power_of_2 (length)); -xe_gt_assert (gt, !(length &GENMASK (ilog2 (SZ_16M ) -1 ,ilog2 (SZ_2M ) +1 ))); +xe_gt_assert (gt, !(length &GENMASK (ilog2 (SZ_16M ) -1 , +ilog2 (SZ_2M ) +1 )));xe_gt_assert (gt,IS_ALIGNED (start, length)); action[len++] =MAKE_INVAL_OP (XE_GUC_TLB_INVAL_PAGE_SELECTIVE ); - action[len++] =xe_vma_vm (vma)->usm.asid; + action[len++] = asid; action[len++] =lower_32_bits (start); action[len++] =upper_32_bits (start); action[len++] =ilog2 (length) -ilog2 (SZ_4K );
@@ -303,6 +306,33 @@ int xe_gt_tlb_invalidation_vma(struct xe_gt *gt,xe_gt_assert (gt, len <=MAX_TLB_INVALIDATION_LEN );return send_tlb_invalidation (>->uc.guc, fence, action, len); + +} + +/** +* xe_gt_tlb_invalidation_vma - Issue a TLB invalidation on this GT for a VMA +* @gt: graphics tile +* @fence: invalidation fence which will be signal on TLB invalidation +* completion, can be NULL +* @vma: VMA to invalidate +* +* Issue a range based TLB invalidation if supported, if not fallback to a full +* TLB invalidation. Completion of TLB is asynchronous and caller can either use +* the invalidation fence or seqno + xe_gt_tlb_invalidation_wait to wait for +* completion. +* +* Return: Seqno which can be passed to xe_gt_tlb_invalidation_wait on success, +* negative error code on error. +*/ +int xe_gt_tlb_invalidation_vma (struct xe_gt *gt, +struct xe_gt_tlb_invalidation_fence *fence, +struct xe_vma *vma) +{ +xe_gt_assert (gt, vma); + +return xe_gt_tlb_invalidation_range (gt, fence,xe_vma_start (vma), +xe_vma_end (vma), +xe_vma_vm (vma)->usm.asid); }/**
drivers/gpu/drm/xe/xe_gt_tlb_invalidation.h +3 -0
@@ -20,6 +20,9 @@ int xe_gt_tlb_invalidation_guc(struct xe_gt *gt);int xe_gt_tlb_invalidation_vma (struct xe_gt *gt,struct xe_gt_tlb_invalidation_fence *fence,struct xe_vma *vma); +int xe_gt_tlb_invalidation_range (struct xe_gt *gt, +struct xe_gt_tlb_invalidation_fence *fence, +u64 start,u64 end,u32 asid);int xe_gt_tlb_invalidation_wait (struct xe_gt *gt,int seqno);int xe_guc_tlb_invalidation_done_handler (struct xe_guc *guc,u32 *msg,u32 len);
drivers/gpu/drm/xe/xe_pt.c +18 -7
@@ -1111,10 +1111,12 @@ static const struct xe_migrate_pt_update_ops userptr_bind_ops = {struct invalidation_fence {struct xe_gt_tlb_invalidation_fence base;struct xe_gt *gt; -struct xe_vma *vma;struct dma_fence *fence;struct dma_fence_cb cb;struct work_struct work; +u64 start; +u64 end; +u32 asid; }; static const char *
@@ -1157,13 +1159,14 @@ static void invalidation_fence_work_func(struct work_struct *w) container_of(w, struct invalidation_fence, work);trace_xe_gt_tlb_invalidation_fence_work_func (&ifence->base); -xe_gt_tlb_invalidation_vma (ifence->gt, &ifence->base, ifence->vma); +xe_gt_tlb_invalidation_range (ifence->gt, &ifence->base, ifence->start, + ifence->end, ifence->asid); }static int invalidation_fence_init (struct xe_gt *gt,struct invalidation_fence *ifence,struct dma_fence *fence, -struct xe_vma *vma) +u64 start,u64 end,u32 asid) {int ret;
@@ -1181,7 +1184,9 @@ static int invalidation_fence_init(struct xe_gt *gt,dma_fence_get (&ifence->base.base);/* Ref for caller */ ifence->fence = fence; ifence->gt = gt; - ifence->vma = vma; + ifence->start = start; + ifence->end = end; + ifence->asid = asid;INIT_WORK (&ifence->work, invalidation_fence_work_func); ret =dma_fence_add_callback (fence, &ifence->cb, invalidation_fence_cb);
@@ -1323,8 +1328,11 @@ __xe_pt_bind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue/* TLB invalidation must be done before signaling rebind */ if (ifence) { -int err =invalidation_fence_init (tile->primary_gt, ifence, fence, - vma); +int err =invalidation_fence_init (tile->primary_gt, + ifence, fence, +xe_vma_start (vma), +xe_vma_end (vma), +xe_vma_vm (vma)->usm.asid);if (err) {dma_fence_put (fence);kfree (ifence);
@@ -1662,7 +1670,10 @@ __xe_pt_unbind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queudma_fence_wait (fence, false);/* TLB invalidation must be done before signaling unbind */ - err =invalidation_fence_init (tile->primary_gt, ifence, fence, vma); + err =invalidation_fence_init (tile->primary_gt, ifence, fence, +xe_vma_start (vma), +xe_vma_end (vma), +xe_vma_vm (vma)->usm.asid);if (err) {dma_fence_put (fence);kfree (ifence);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 19/26 drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue
In reply to Matthew Brostxe_engine is now xe_exec_queue, adjust this function's name to reflect.
--- drivers/gpu/drm/xe/xe_gt_pagefault.c | 2 +- drivers/gpu/drm/xe/xe_migrate.c | 8 ++++---- drivers/gpu/drm/xe/xe_migrate.h | 3 ++- 3 files changed, 7 insertions(+), 6 deletions(-)
drivers/gpu/drm/xe/xe_gt_pagefault.c +1 -1
@@ -205,7 +205,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf)trace_xe_vma_pf_bind (vma);xe_vm_populate_dummy_rebind (vm, vma); vm->dummy_ops.op.tile_mask =BIT (tile->id); - vm->dummy_ops.op.q =xe_tile_migrate_engine (tile); + vm->dummy_ops.op.q =xe_tile_migrate_exec_queue (tile); fence =xe_vm_ops_execute (vm, &vm->dummy_ops.vops);if (IS_ERR (fence)) { ret =PTR_ERR (fence);
drivers/gpu/drm/xe/xe_migrate.c +4 -4
@@ -74,15 +74,15 @@ struct xe_migrate {#define NUM_PT_PER_BLIT (MAX_PREEMPTDISABLE_TRANSFER / SZ_2M)/** -* xe_tile_migrate_engine() - Get this tile's migrate engine. +* xe_tile_migrate_exec_queue() - Get this tile's migrate exec queue. * @tile: The tile. * -* Returns the default migrate engine of this tile. +* Returns the default migrate exec queue of this tile. * TODO: Perhaps this function is slightly misplaced, and even unneeded? * -* Return: The default migrate engine +* Return: The default migrate exec queue */ -struct xe_exec_queue *xe_tile_migrate_engine (struct xe_tile *tile) +struct xe_exec_queue *xe_tile_migrate_exec_queue (struct xe_tile *tile) {return tile->migrate->q; }
drivers/gpu/drm/xe/xe_migrate.h +2 -1
@@ -105,5 +105,6 @@ xe_migrate_update_pgtables(struct xe_migrate *m,void xe_migrate_wait (struct xe_migrate *m); -struct xe_exec_queue *xe_tile_migrate_engine (struct xe_tile *tile); +struct xe_exec_queue *xe_tile_migrate_exec_queue (struct xe_tile *tile); +#endif
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 20/26 drm/xe: Convert multiple bind ops into single job
In reply to Matthew BrostThis aligns with the uAPI of an array of binds or single bind that results in multiple GPUVA ops to be considered a single atomic operations.
--- drivers/gpu/drm/xe/xe_bo_types.h | 2 + drivers/gpu/drm/xe/xe_exec.c | 25 +- drivers/gpu/drm/xe/xe_gt_pagefault.c | 6 +- drivers/gpu/drm/xe/xe_migrate.c | 287 ++++---- drivers/gpu/drm/xe/xe_migrate.h | 32 +- drivers/gpu/drm/xe/xe_pt.c | 957 ++++++++++++++++----------- drivers/gpu/drm/xe/xe_pt.h | 7 + drivers/gpu/drm/xe/xe_pt_types.h | 34 + drivers/gpu/drm/xe/xe_vm.c | 516 ++++----------- drivers/gpu/drm/xe/xe_vm.h | 5 +- drivers/gpu/drm/xe/xe_vm_types.h | 29 +- 11 files changed, 899 insertions(+), 1001 deletions(-)
drivers/gpu/drm/xe/xe_bo_types.h +2 -0
@@ -74,6 +74,8 @@ struct xe_bo { } props;/** @freed: List node for delayed put. */ struct llist_node freed; +/** @update_index: Update index if PT BO */ +int update_index;/** @created: Whether the bo has passed initial creation */ bool created; };
drivers/gpu/drm/xe/xe_exec.c +2 -23
@@ -270,30 +270,9 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) err =PTR_ERR (rebind_fence); goto err_put_job; } +dma_fence_put (rebind_fence); -/* -* We store the rebind_fence in the VM so subsequent execs don't get -* scheduled before the rebinds of userptrs / evicted BOs is complete. -*/ -if (rebind_fence) { -dma_fence_put (vm->rebind_fence); - vm->rebind_fence = rebind_fence; - } -if (vm->rebind_fence) { -if (test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , - &vm->rebind_fence->flags)) { -dma_fence_put (vm->rebind_fence); - vm->rebind_fence =NULL ; - }else { -dma_fence_get (vm->rebind_fence); - err =drm_sched_job_add_dependency (&job->drm, - vm->rebind_fence); -if (err) - goto err_put_job; - } - } - -/* Wait behind munmap style rebinds */ +/* Wait for rebinds */ if (!xe_vm_no_dma_fences (vm)) { err = drm_sched_job_add_resv_dependencies(&job->drm, &vm->resv,
drivers/gpu/drm/xe/xe_gt_pagefault.c +3 -3
@@ -203,9 +203,9 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf)/* Bind VMA only to the GT that has faulted */ trace_xe_vma_pf_bind (vma); -xe_vm_populate_dummy_rebind (vm, vma); - vm->dummy_ops.op.tile_mask =BIT (tile->id); - vm->dummy_ops.op.q =xe_tile_migrate_exec_queue (tile); +xe_vm_populate_dummy_rebind (vm, vma,BIT (tile->id)); + vm->dummy_ops.vops.pt_update_ops[tile->id].q = +xe_tile_migrate_exec_queue (tile); fence =xe_vm_ops_execute (vm, &vm->dummy_ops.vops);if (IS_ERR (fence)) { ret =PTR_ERR (fence);
drivers/gpu/drm/xe/xe_migrate.c +134 -153
399 lines not on this page · raw message
drivers/gpu/drm/xe/xe_migrate.h +20 -12
@@ -47,6 +47,24 @@ struct xe_migrate_pt_update_ops { struct xe_tile *tile, struct iosys_map *map, void *pos, u32 ofs, u32 num_qwords, const struct xe_vm_pgtable_update *update); +/** +* @clear: Clear a command buffer or page-table with ptes. +* @pt_update: Embeddable callback argument. +* @tile: The tile for the current operation. +* @map: struct iosys_map into the memory to be populated. +* @pos: If @map is NULL, map into the memory to be populated. +* @ofs: qword offset into @map, unused if @map is NULL. +* @num_qwords: Number of qwords to write. +* @update: Information about the PTEs to be inserted. +* +* This interface is intended to be used as a callback into the +* page-table system to populate command buffers or shared +* page-tables with PTEs. +*/ +void (*clear)(struct xe_migrate_pt_update *pt_update, +struct xe_tile *tile,struct iosys_map *map, +void *pos,u32 ofs,u32 num_qwords, +const struct xe_vm_pgtable_update *update);/** * @pre_commit: Callback to be called just before arming the
@@ -67,14 +85,10 @@ struct xe_migrate_pt_update_ops {struct xe_migrate_pt_update {/** @ops: Pointer to the struct xe_migrate_pt_update_ops callbacks */ const struct xe_migrate_pt_update_ops *ops; -/** @vma: The vma we're updating the pagetable for. */ -struct xe_vma *vma; +/** @vops: VMA operations */ +struct xe_vma_ops *vops;/** @job: The job if a GPU page-table update. NULL otherwise */ struct xe_sched_job *job; -/** @start: Start of update for the range fence */ -u64 start; -/** @last: Last of update for the range fence */ -u64 last;/** @tile_id: Tile ID of the update */ u8 tile_id; };
@@ -95,12 +109,6 @@ struct xe_vm *xe_migrate_get_vm(struct xe_migrate *m);struct dma_fence *xe_migrate_update_pgtables (struct xe_migrate *m, -struct xe_vm *vm, -struct xe_bo *bo, -struct xe_exec_queue *q, -const struct xe_vm_pgtable_update *updates, -u32 num_updates, -struct xe_sync_entry *syncs,u32 num_syncs,struct xe_migrate_pt_update *pt_update);void xe_migrate_wait (struct xe_migrate *m);
drivers/gpu/drm/xe/xe_pt.c +569 -388
1,095 lines not on this page · raw message
drivers/gpu/drm/xe/xe_pt.h +7 -0
@@ -17,6 +17,7 @@ struct xe_sync_entry;struct xe_tile ;struct xe_vm ;struct xe_vma ; +struct xe_vma_ops ;#define xe_pt_write (xe, map, idx, data) \ xe_map_wr(xe, map, (idx) * sizeof(u64), u64, data)
@@ -34,6 +35,12 @@ void xe_pt_populate_empty(struct xe_tile *tile, struct xe_vm *vm,void xe_pt_destroy (struct xe_pt *pt,u32 flags,struct llist_head *deferred); +int xe_pt_update_ops_prepare (struct xe_tile *tile,struct xe_vma_ops *vops); +struct dma_fence *xe_pt_update_ops_run (struct xe_tile *tile, +struct xe_vma_ops *vops); +void xe_pt_update_ops_commit (struct xe_tile *tile,struct xe_vma_ops *vops); +void xe_pt_update_ops_abort (struct xe_tile *tile,struct xe_vma_ops *vops); +struct dma_fence * __xe_pt_bind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue *q, struct xe_sync_entry *syncs, u32 num_syncs,
drivers/gpu/drm/xe/xe_pt_types.h +34 -0
@@ -85,4 +85,38 @@ struct xe_vm_pgtable_update_op { bool rebind; }; +/** struct xe_vm_pgtable_update_ops: page table update operations */ +struct xe_vm_pgtable_update_ops { +/** @ops: operations */ +struct xe_vm_pgtable_update_op *ops; +/** @deferred: deferred list to destroy PT entries */ +struct llist_head deferred; +/** @q: exec queue for PT operations */ +struct xe_exec_queue *q; +/** @start: start address of ops */ +u64 start; +/** @last: last address of ops */ +u64 last; +/** @num_ops: number of operations */ +u32 num_ops; +/** @current_op: current operations */ +u32 current_op; +/** @needs_userptr_lock: Needs userptr lock */ +bool needs_userptr_lock; +/** @needs_invalidation: Needs invalidation */ +bool needs_invalidation; +/** +* @wait_vm_bookkeep: PT operations need to wait until VM is idle +* (bookkeep dma-resv slots are idle) and stage all future VM activity +* behind these operations (install PT operations into VM kernel +* dma-resv slot). +*/ +bool wait_vm_bookkeep; +/** +* @wait_vm_kernel: PT operations need to wait until VM kernel dma-resv +* slots are idle. +*/ +bool wait_vm_kernel; +}; +#endif
drivers/gpu/drm/xe/xe_vm.c +119 -397
619 lines not on this page · raw message
drivers/gpu/drm/xe/xe_vm.h +4 -1
@@ -224,9 +224,12 @@ int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);int xe_vm_prepare_vma (struct drm_exec *exec,struct xe_vma *vma,unsigned int num_shared); -void xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma); +void xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma, +u8 tile_mask);struct dma_fence *xe_vm_ops_execute (struct xe_vm *vm,struct xe_vma_ops *vops); +void xe_vm_kill (struct xe_vm *vm,bool unlocked); +#if IS_ENABLED (CONFIG_DRM_XE_DEBUG_VM )#define vm_dbg drm_dbg#else
drivers/gpu/drm/xe/xe_vm_types.h +5 -24
@@ -169,29 +169,18 @@ struct xe_vma_op_prefetch {/** enum xe_vma_op_flags - flags for VMA operation */ enum xe_vma_op_flags { -/** @XE_VMA_OP_FIRST: first VMA operation for a set of syncs */ -XE_VMA_OP_FIRST =BIT (0 ), -/** @XE_VMA_OP_LAST: last VMA operation for a set of syncs */ -XE_VMA_OP_LAST =BIT (1 ),/** @XE_VMA_OP_COMMITTED: VMA operation committed */ -XE_VMA_OP_COMMITTED =BIT (2 ), +XE_VMA_OP_COMMITTED =BIT (0 ),/** @XE_VMA_OP_PREV_COMMITTED: Previous VMA operation committed */ -XE_VMA_OP_PREV_COMMITTED =BIT (3 ), +XE_VMA_OP_PREV_COMMITTED =BIT (1 ),/** @XE_VMA_OP_NEXT_COMMITTED: Next VMA operation committed */ -XE_VMA_OP_NEXT_COMMITTED =BIT (4 ), +XE_VMA_OP_NEXT_COMMITTED =BIT (2 ), };/** struct xe_vma_op - VMA operation */ struct xe_vma_op {/** @base: GPUVA base operation */ struct drm_gpuva_op base; -/** @q: exec queue for this operation */ -struct xe_exec_queue *q; -/** -* @syncs: syncs for this operation, only used on first and last -* operation -*/ -struct xe_sync_entry *syncs;/** @num_syncs: number of syncs */ u32 num_syncs;/** @link: operation link */
@@ -222,19 +211,14 @@ struct xe_vma_ops {struct list_head list;/** @vm: VM */ struct xe_vm *vm; -/** @q: exec queue these operations */ +/** @q: exec queue for VMA operations */ struct xe_exec_queue *q;/** @syncs: syncs these operation */ struct xe_sync_entry *syncs;/** @num_syncs: number of syncs */ u32 num_syncs;/** @pt_update_ops: page table update operations */ -struct { -/** @ops: operations */ -struct xe_vm_pgtable_update_op *ops; -/** @num_ops: number of operations */ -u32 num_ops; - } pt_update_ops[XE_MAX_TILES_PER_DEVICE ]; +struct xe_vm_pgtable_update_ops pt_update_ops[XE_MAX_TILES_PER_DEVICE ]; };#define xe_vm_assert_held (vm) dma_resv_assert_held(&(vm)->resv)
@@ -295,9 +279,6 @@ struct xe_vm {*/ struct list_head rebind_list; -/** @rebind_fence: rebind fence from execbuf */ -struct dma_fence *rebind_fence; -/** * @destroy_work: worker to destroy VM, needed as a dma_fence signaling * from an irq context can be last put and the destroy needs to be able
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 21/26 drm/xe: Update clear / populate arguments
In reply to Matthew BrostThis will help implement CPU binds in run_job() as 'struct xe_migrate_pt_update' is not available at the time of run_job().
--- drivers/gpu/drm/xe/xe_migrate.c | 9 +++++---- drivers/gpu/drm/xe/xe_migrate.h | 12 +++++------- drivers/gpu/drm/xe/xe_pt.c | 12 +++++------- 3 files changed, 15 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_migrate.c +5 -4
@@ -1057,6 +1057,7 @@ static void write_pgtable(struct xe_tile *tile, struct xe_bb *bb, u64 ppgtt_ofs, struct xe_migrate_pt_update *pt_update) {const struct xe_migrate_pt_update_ops *ops = pt_update->ops; +struct xe_vm *vm = pt_update->vops->vm;u32 chunk;u32 ofs = update->ofs, size = update->qwords;
@@ -1088,10 +1089,10 @@ static void write_pgtable(struct xe_tile *tile, struct xe_bb *bb, u64 ppgtt_ofs, bb->cs[bb->len++] =lower_32_bits (addr); bb->cs[bb->len++] =upper_32_bits (addr);if (pt_op->bind) - ops->populate (pt_update, tile,NULL , bb->cs + bb->len, + ops->populate (tile,NULL , bb->cs + bb->len, ofs, chunk, update);else - ops->clear (pt_update, tile,NULL , bb->cs + bb->len, + ops->clear (vm, tile,NULL , bb->cs + bb->len, ofs, chunk, update); bb->len += chunk *2 ;
@@ -1148,12 +1149,12 @@ xe_migrate_update_pgtables_cpu(struct xe_migrate *m, &pt_op->entries[i];if (pt_op->bind) - ops->populate (pt_update, m->tile, + ops->populate (m->tile, &update->pt_bo->vmap,NULL , update->ofs, update->qwords, update);else - ops->clear (pt_update, m->tile, + ops->clear (vm, m->tile, &update->pt_bo->vmap,NULL , update->ofs, update->qwords, update); }
drivers/gpu/drm/xe/xe_migrate.h +5 -7
@@ -31,7 +31,6 @@ struct xe_vma;struct xe_migrate_pt_update_ops {/** * @populate: Populate a command buffer or page-table with ptes. -* @pt_update: Embeddable callback argument. * @tile: The tile for the current operation. * @map: struct iosys_map into the memory to be populated. * @pos: If @map is NULL, map into the memory to be populated.
@@ -43,13 +42,12 @@ struct xe_migrate_pt_update_ops {* page-table system to populate command buffers or shared * page-tables with PTEs. */ -void (*populate)(struct xe_migrate_pt_update *pt_update, -struct xe_tile *tile,struct iosys_map *map, +void (*populate)(struct xe_tile *tile,struct iosys_map *map,void *pos,u32 ofs,u32 num_qwords,const struct xe_vm_pgtable_update *update);/** * @clear: Clear a command buffer or page-table with ptes. -* @pt_update: Embeddable callback argument. +* @vm: VM being updated * @tile: The tile for the current operation. * @map: struct iosys_map into the memory to be populated. * @pos: If @map is NULL, map into the memory to be populated.
@@ -61,9 +59,9 @@ struct xe_migrate_pt_update_ops {* page-table system to populate command buffers or shared * page-tables with PTEs. */ -void (*clear)(struct xe_migrate_pt_update *pt_update, -struct xe_tile *tile,struct iosys_map *map, -void *pos,u32 ofs,u32 num_qwords, +void (*clear)(struct xe_vm *vm,struct xe_tile *tile, +struct iosys_map *map,void *pos,u32 ofs, +u32 num_qwords,const struct xe_vm_pgtable_update *update);/**
drivers/gpu/drm/xe/xe_pt.c +5 -7
@@ -831,9 +831,8 @@ bool xe_pt_zap_ptes(struct xe_tile *tile, struct xe_vma *vma) }static void -xe_vm_populate_pgtable (struct xe_migrate_pt_update *pt_update,struct xe_tile *tile, -struct iosys_map *map,void *data, -u32 qword_ofs,u32 num_qwords, +xe_vm_populate_pgtable (struct xe_tile *tile,struct iosys_map *map, +void *data,u32 qword_ofs,u32 num_qwords,const struct xe_vm_pgtable_update *update) {struct xe_pt_entry *ptes = update->pt_entries;
@@ -1455,12 +1454,11 @@ static unsigned int xe_pt_stage_unbind(struct xe_tile *tile, struct xe_vma *vma, }static void -xe_migrate_clear_pgtable_callback (struct xe_migrate_pt_update *pt_update, -struct xe_tile *tile,struct iosys_map *map, -void *ptr,u32 qword_ofs,u32 num_qwords, +xe_migrate_clear_pgtable_callback (struct xe_vm *vm,struct xe_tile *tile, +struct iosys_map *map,void *ptr, +u32 qword_ofs,u32 num_qwords,const struct xe_vm_pgtable_update *update) { -struct xe_vm *vm = pt_update->vops->vm;u64 empty =__xe_pt_empty_pte (tile, vm, update->pt->level);int i;
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 22/26 drm/xe: Add __xe_migrate_update_pgtables_cpu helper
In reply to Matthew BrostThis will help implement CPU binds as the submision backend can call this helper when a bind jobs dependencies are resolved.
--- drivers/gpu/drm/xe/xe_migrate.c | 54 +++++++++++++++++++-------------- 1 file changed, 31 insertions(+), 23 deletions(-)
drivers/gpu/drm/xe/xe_migrate.c +31 -23
@@ -1116,6 +1116,34 @@ struct migrate_test_params { container_of(_priv, struct migrate_test_params, base)#endif +static void +__xe_migrate_update_pgtables_cpu (struct xe_vm *vm,struct xe_tile *tile, +const struct xe_migrate_pt_update_ops *ops, +struct xe_vm_pgtable_update_op *pt_op, +int num_ops) +{ +u32 j, i; + +for (j =0 ; j < num_ops; ++j, ++pt_op) { +for (i =0 ; i < pt_op->num_entries; i++) { +const struct xe_vm_pgtable_update *update = + &pt_op->entries[i]; + +if (pt_op->bind) + ops->populate (tile, &update->pt_bo->vmap, +NULL , update->ofs, update->qwords, + update); +else + ops->clear (vm, tile, &update->pt_bo->vmap, +NULL , update->ofs, update->qwords, + update); + } + } + +trace_xe_vm_cpu_bind (vm); +xe_device_wmb (vm->xe); +} +static struct dma_fence *xe_migrate_update_pgtables_cpu (struct xe_migrate *m,struct xe_migrate_pt_update *pt_update)
@@ -1128,7 +1156,6 @@ xe_migrate_update_pgtables_cpu(struct xe_migrate *m,struct xe_vm_pgtable_update_ops *pt_update_ops = &pt_update->vops->pt_update_ops[pt_update->tile_id];int err; -u32 j, i;if (XE_TEST_ONLY (test && test->force_gpu))return ERR_PTR (-ETIME );
@@ -1140,28 +1167,9 @@ xe_migrate_update_pgtables_cpu(struct xe_migrate *m,return ERR_PTR (err); } -for (j =0 ; j < pt_update_ops->num_ops; ++j) { -const struct xe_vm_pgtable_update_op *pt_op = - &pt_update_ops->ops[j]; - -for (i =0 ; i < pt_op->num_entries; i++) { -const struct xe_vm_pgtable_update *update = - &pt_op->entries[i]; - -if (pt_op->bind) - ops->populate (m->tile, - &update->pt_bo->vmap,NULL , - update->ofs, update->qwords, - update); -else - ops->clear (vm, m->tile, - &update->pt_bo->vmap,NULL , - update->ofs, update->qwords, update); - } - } - -trace_xe_vm_cpu_bind (vm); -xe_device_wmb (vm->xe); +__xe_migrate_update_pgtables_cpu (vm, m->tile, ops, + pt_update_ops->ops, + pt_update_ops->num_ops);return dma_fence_get_stub (); }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 23/26 drm/xe: Add xe_hw_fence_signal helper
In reply to Matthew BrostThis help implement cpu binds in run_job by having a helper to signal the cpu bind job's fence.
--- drivers/gpu/drm/xe/xe_hw_fence.c | 8 ++++++++ drivers/gpu/drm/xe/xe_hw_fence.h | 1 + 2 files changed, 9 insertions(+)
drivers/gpu/drm/xe/xe_hw_fence.c +8 -0
@@ -208,6 +208,14 @@ static struct xe_hw_fence *to_xe_hw_fence(struct dma_fence *fence) return container_of(fence, struct xe_hw_fence, dma); } +void xe_hw_fence_signal (struct dma_fence *dma_fence) +{ +struct xe_hw_fence *fence =to_xe_hw_fence (dma_fence); +struct xe_device *xe =gt_to_xe (fence->ctx->gt); + +xe_map_wr (xe, &fence->seqno_map,0 , u32, dma_fence->seqno); +} +struct xe_hw_fence *xe_hw_fence_create (struct xe_hw_fence_ctx *ctx,struct iosys_map seqno_map) {
drivers/gpu/drm/xe/xe_hw_fence.h +1 -0
@@ -26,5 +26,6 @@ void xe_hw_fence_ctx_finish(struct xe_hw_fence_ctx *ctx);struct xe_hw_fence *xe_hw_fence_create (struct xe_hw_fence_ctx *ctx,struct iosys_map seqno_map); +void xe_hw_fence_signal (struct dma_fence *fence);#endif
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 24/26 drm/xe: CPU binds for jobs
In reply to Matthew BrostNo reason to use the GPU for binds, rather in run_job use the CPU to do binds once the bind job dependencies are resolved.
--- drivers/gpu/drm/xe/xe_bo.c | 7 +- drivers/gpu/drm/xe/xe_bo.h | 6 +- drivers/gpu/drm/xe/xe_device.c | 23 +++ drivers/gpu/drm/xe/xe_device.h | 7 + drivers/gpu/drm/xe/xe_device_types.h | 4 + drivers/gpu/drm/xe/xe_gt_pagefault.c | 5 +- drivers/gpu/drm/xe/xe_guc_submit.c | 43 ++++- drivers/gpu/drm/xe/xe_migrate.c | 206 +++--------------------- drivers/gpu/drm/xe/xe_migrate.h | 6 + drivers/gpu/drm/xe/xe_pt.c | 31 ++-- drivers/gpu/drm/xe/xe_pt.h | 1 + drivers/gpu/drm/xe/xe_pt_types.h | 5 + drivers/gpu/drm/xe/xe_sched_job.c | 4 +- drivers/gpu/drm/xe/xe_sched_job_types.h | 31 +++- drivers/gpu/drm/xe/xe_vm.c | 97 +++++------ drivers/gpu/drm/xe/xe_vm.h | 5 +- 16 files changed, 223 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_bo.c +3 -4
@@ -2066,16 +2066,16 @@ void __xe_bo_release_dummy(struct kref *kref)/** * xe_bo_put_commit() - Put bos whose put was deferred by xe_bo_put_deferred(). +( @xe: Xe device * @deferred: The lockless list used for the call to xe_bo_put_deferred(). * * Puts all bos whose put was deferred by xe_bo_put_deferred(). * The @deferred list can be either an onstack local list or a global * shared list used by a workqueue. */ -void xe_bo_put_commit (struct llist_head *deferred) +void xe_bo_put_commit (struct xe_device *xe,struct llist_head *deferred) {struct llist_node *freed; -struct xe_bo *bo, *next;if (!deferred)return ;
@@ -2084,8 +2084,7 @@ void xe_bo_put_commit(struct llist_head *deferred)if (!freed)return ; -llist_for_each_entry_safe (bo, next, freed, freed) -drm_gem_object_free (&bo->ttm.base.refcount); +xe_device_put_deferred (xe, freed); }/**
drivers/gpu/drm/xe/xe_bo.h +4 -2
@@ -10,7 +10,7 @@#include "xe_bo_types.h" #include "xe_macros.h" -#include "xe_vm_types.h" +#include "xe_vm.h" #define XE_DEFAULT_GTT_SIZE_MB 3072ULL/* 3GB by default */
@@ -287,10 +287,12 @@ xe_bo_put_deferred(struct xe_bo *bo, struct llist_head *deferred)if (!kref_put (&bo->ttm.base.refcount, __xe_bo_release_dummy))return false; +xe_vm_get (bo->vm); +return llist_add (&bo->freed, deferred); } -void xe_bo_put_commit (struct llist_head *deferred); +void xe_bo_put_commit (struct xe_device *xe,struct llist_head *deferred);struct sg_table *xe_bo_get_sg (struct xe_bo *bo);
drivers/gpu/drm/xe/xe_device.c +23 -0
@@ -186,6 +186,8 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy) {struct xe_device *xe =to_xe_device (dev); +flush_work (&xe->mem.deferred_work); +if (xe->ordered_wq)destroy_workqueue (xe->ordered_wq);
@@ -195,6 +197,24 @@ static void xe_device_destroy(struct drm_device *dev, void *dummy)ttm_device_fini (&xe->ttm); } +static void deferred_work (struct work_struct *w) +{ + struct xe_device *xe = container_of(w, struct xe_device, + mem.deferred_work); +struct llist_node *freed =llist_del_all (&xe->mem.deferred); +struct xe_bo *bo, *next; + +if (!freed) +return ; + +llist_for_each_entry_safe (bo, next, freed, freed) { +struct xe_vm *vm = bo->vm; + +drm_gem_object_free (&bo->ttm.base.refcount); +xe_vm_put (vm); + } +} +struct xe_device *xe_device_create (struct pci_dev *pdev,const struct pci_device_id *ent) {
@@ -248,6 +268,9 @@ struct xe_device *xe_device_create(struct pci_dev *pdev, goto err_put; } +init_llist_head (&xe->mem.deferred); +INIT_WORK (&xe->mem.deferred_work, deferred_work); + err =xe_display_create (xe);if (WARN_ON (err)) goto err_put;
drivers/gpu/drm/xe/xe_device.h +7 -0
@@ -160,4 +160,11 @@ static inline bool xe_device_has_flat_ccs(struct xe_device *xe)u32 xe_device_ccs_bytes (struct xe_device *xe,u64 size); +static inline void xe_device_put_deferred (struct xe_device *xe, +struct llist_node *deferred) +{ +llist_add (deferred, &xe->mem.deferred); +queue_work (system_wq, &xe->mem.deferred_work); +} +#endif
drivers/gpu/drm/xe/xe_device_types.h +4 -0
@@ -295,6 +295,10 @@ struct xe_device {struct xe_mem_region vram;/** @sys_mgr: system TTM manager */ struct ttm_resource_manager sys_mgr; +/** @deferred: deferred list to destroy PT entries */ +struct llist_head deferred; +/** @deferred_work: worker to destroy PT entries */ +struct work_struct deferred_work; } mem;/** @usm: unified memory state */
drivers/gpu/drm/xe/xe_gt_pagefault.c +4 -1
@@ -203,10 +203,13 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf)/* Bind VMA only to the GT that has faulted */ trace_xe_vma_pf_bind (vma); -xe_vm_populate_dummy_rebind (vm, vma,BIT (tile->id)); + ret =xe_vm_populate_dummy_rebind (vm, vma,BIT (tile->id)); +if (ret) + goto unlock_dma_resv; vm->dummy_ops.vops.pt_update_ops[tile->id].q =xe_tile_migrate_exec_queue (tile); fence =xe_vm_ops_execute (vm, &vm->dummy_ops.vops); +xe_vma_ops_free (&vm->dummy_ops.vops);if (IS_ERR (fence)) { ret =PTR_ERR (fence); goto unlock_dma_resv;
drivers/gpu/drm/xe/xe_guc_submit.c +38 -5
68 lines not on this page · raw message
drivers/gpu/drm/xe/xe_migrate.c +22 -184
249 lines not on this page · raw message
drivers/gpu/drm/xe/xe_migrate.h +6 -0
@@ -22,6 +22,7 @@ struct xe_pt;struct xe_tile ;struct xe_vm ;struct xe_vm_pgtable_update ; +struct xe_vm_pgtable_update_op ;struct xe_vma ;/**
@@ -105,6 +106,11 @@ struct dma_fence *xe_migrate_clear(struct xe_migrate *m,struct xe_vm *xe_migrate_get_vm (struct xe_migrate *m); +void __xe_migrate_update_pgtables_cpu (struct xe_vm *vm,struct xe_tile *tile, +const struct xe_migrate_pt_update_ops *ops, +struct xe_vm_pgtable_update_op *pt_op, +int num_ops); +struct dma_fence *xe_migrate_update_pgtables (struct xe_migrate *m,struct xe_migrate_pt_update *pt_update);
drivers/gpu/drm/xe/xe_pt.c +22 -9
@@ -366,6 +366,7 @@ xe_pt_new_shared(struct xe_walk_update *wupd, struct xe_pt *parent, entry->pt = parent; entry->flags =0 ; entry->qwords =0 ; + entry->level = parent->level; entry->pt_bo->update_index =-1 ;if (alloc_entries) {
@@ -1459,7 +1460,7 @@ xe_migrate_clear_pgtable_callback(struct xe_vm *vm, struct xe_tile *tile, u32 qword_ofs, u32 num_qwords, const struct xe_vm_pgtable_update *update) { - u64 empty = __xe_pt_empty_pte(tile, vm, update->pt->level); + u64 empty = __xe_pt_empty_pte(tile, vm, update->level);int i;if (map && map->is_iomem)
@@ -1865,31 +1866,43 @@ xe_pt_update_ops_run(struct xe_tile *tile, struct xe_vma_ops *vops)return ERR_PTR (err); } +void xe_pt_update_ops_free (struct xe_vm_pgtable_update_op *pt_op,u32 num_ops) +{ +u32 i; + +for (i =0 ; i < num_ops; ++i, ++pt_op) +if (pt_op->bind) +xe_pt_free_bind (pt_op->entries, pt_op->num_entries); +} +void xe_pt_update_ops_commit (struct xe_tile *tile,struct xe_vma_ops *vops) {struct xe_vm_pgtable_update_ops *pt_update_ops = &vops->pt_update_ops[tile->id]; -int i;lockdep_assert_held (&vops->vm->lock);xe_vm_assert_held (vops->vm);/* FIXME: Not 100% correct */ -for (i =0 ; i < pt_update_ops->num_ops; ++i) { -struct xe_vm_pgtable_update_op *pt_op = &pt_update_ops->ops[i]; -if (pt_op->bind) -xe_pt_free_bind (pt_op->entries, pt_op->num_entries); - } -xe_bo_put_commit (&vops->pt_update_ops[tile->id].deferred); +xe_bo_put_commit (tile_to_xe (tile), &pt_update_ops->deferred); +if (!pt_update_ops->skip_free) +xe_pt_update_ops_free (pt_update_ops->ops, + pt_update_ops->num_ops); }void xe_pt_update_ops_abort (struct xe_tile *tile,struct xe_vma_ops *vops) { +struct xe_vm_pgtable_update_ops *pt_update_ops = + &vops->pt_update_ops[tile->id]; +lockdep_assert_held (&vops->vm->lock);xe_vm_assert_held (vops->vm);/* FIXME: Just kill VM for now + cleanup PTs */ -xe_bo_put_commit (&vops->pt_update_ops[tile->id].deferred); +xe_bo_put_commit (tile_to_xe (tile), &pt_update_ops->deferred); +if (!pt_update_ops->skip_free) +xe_pt_update_ops_free (pt_update_ops->ops, + pt_update_ops->num_ops);xe_vm_kill (vops->vm, false); }
drivers/gpu/drm/xe/xe_pt.h +1 -0
@@ -40,6 +40,7 @@ struct dma_fence *xe_pt_update_ops_run(struct xe_tile *tile, struct xe_vma_ops *vops);void xe_pt_update_ops_commit (struct xe_tile *tile,struct xe_vma_ops *vops);void xe_pt_update_ops_abort (struct xe_tile *tile,struct xe_vma_ops *vops); +void xe_pt_update_ops_free (struct xe_vm_pgtable_update_op *pt_op,u32 num_ops);struct dma_fence * __xe_pt_bind_vma(struct xe_tile *tile, struct xe_vma *vma, struct xe_exec_queue *q,
drivers/gpu/drm/xe/xe_pt_types.h +5 -0
@@ -69,6 +69,9 @@ struct xe_vm_pgtable_update {/** @pt_entries: Newly added pagetable entries */ struct xe_pt_entry *pt_entries; +/** @level: level of update */ +unsigned int level; +/** @flags: Target flags */ u32 flags; };
@@ -117,6 +120,8 @@ struct xe_vm_pgtable_update_ops {* slots are idle. */ bool wait_vm_kernel; +/** @skip_free: Free @ops in submission backend rather than in IOCTL */ + bool skip_free; };#endif
drivers/gpu/drm/xe/xe_sched_job.c +3 -1
@@ -34,8 +34,10 @@ int __init xe_sched_job_module_init(void) xe_sched_job_parallel_slab =kmem_cache_create ("xe_sched_job_parallel" ,sizeof (struct xe_sched_job ) + +max_t (size_t,sizeof (u64) * -XE_HW_ENGINE_MAX_INSTANCE ,0 , +XE_HW_ENGINE_MAX_INSTANCE , +sizeof (struct pt_update_args )),0 ,SLAB_HWCACHE_ALIGN ,NULL );if (!xe_sched_job_parallel_slab) {kmem_cache_destroy (xe_sched_job_slab);
drivers/gpu/drm/xe/xe_sched_job_types.h +29 -2
@@ -11,6 +11,28 @@#include <drm/gpu_scheduler.h> struct xe_exec_queue ; +struct xe_migrate_pt_update_ops ; +struct xe_tile ; +struct xe_vm ; +struct xe_vm_pgtable_update_op ; + +/** +* struct pt_update_args - PT update arguments +*/ +struct pt_update_args { +/** @vm: VM */ +struct xe_vm *vm; +/** @tile: Tile */ +struct xe_tile *tile; +/** @ops: Migrate PT update ops */ +const struct xe_migrate_pt_update_ops *ops; +/** @pt_op: PT update ops */ +struct xe_vm_pgtable_update_op *pt_op; +/** @deferred: deferred list to destroy PT entries */ +struct llist_head deferred; +/** @num_ops: number of PT update ops */ +int num_ops; +};/** * struct xe_sched_job - XE schedule job (batch buffer tracking)
@@ -27,6 +49,7 @@ struct xe_sched_job {* can safely reference fence, fence cannot safely reference job. */ #define JOB_FLAG_SUBMIT DMA_FENCE_FLAG_USER_BITS +#define JOB_FLAG_PT (DMA_FENCE_FLAG_USER_BITS << 1)struct dma_fence *fence;/** @user_fence: write back value when BB is complete */ struct {
@@ -39,8 +62,12 @@ struct xe_sched_job { } user_fence;/** @migrate_flush_flags: Additional flush flags for migration jobs */ u32 migrate_flush_flags; -/** @batch_addr: batch buffer address of job */ -u64 batch_addr[]; +union { +/** @batch_addr: batch buffer address of job */ +u64 batch_addr[0 ]; +/** @pt_update: PT update arguments */ +struct pt_update_args pt_update[0 ]; + }; };#endif
drivers/gpu/drm/xe/xe_vm.c +49 -48
152 lines not on this page · raw message
drivers/gpu/drm/xe/xe_vm.h +3 -2
@@ -224,8 +224,9 @@ int xe_analyze_vm(struct drm_printer *p, struct xe_vm *vm, int gt_id);int xe_vm_prepare_vma (struct drm_exec *exec,struct xe_vma *vma,unsigned int num_shared); -void xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma, -u8 tile_mask); +int xe_vm_populate_dummy_rebind (struct xe_vm *vm,struct xe_vma *vma, +u8 tile_mask); +void xe_vma_ops_free (struct xe_vma_ops *vops);struct dma_fence *xe_vm_ops_execute (struct xe_vm *vm,struct xe_vma_ops *vops);void xe_vm_kill (struct xe_vm *vm,bool unlocked);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 25/26 drm/xe: Don't use migrate exec queue for page fault binds
In reply to Matthew BrostNow that the CPU is always used for binds even in jobs, CPU bind jobs can pass GPU jobs in the same exec queue resulting dma-fences signaling out-of-order. Use a dedicated exec queue for binds issued from page faults to avoid ordering issues.
--- drivers/gpu/drm/xe/xe_gt_pagefault.c | 2 +- drivers/gpu/drm/xe/xe_migrate.c | 22 +++++++++++++++++++++- drivers/gpu/drm/xe/xe_migrate.h | 1 + 3 files changed, 23 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_gt_pagefault.c +1 -1
@@ -207,7 +207,7 @@ static int handle_pagefault(struct xe_gt *gt, struct pagefault *pf)if (ret) goto unlock_dma_resv; vm->dummy_ops.vops.pt_update_ops[tile->id].q = -xe_tile_migrate_exec_queue (tile); +xe_tile_migrate_bind_exec_queue (tile); fence =xe_vm_ops_execute (vm, &vm->dummy_ops.vops);xe_vma_ops_free (&vm->dummy_ops.vops);if (IS_ERR (fence)) {
drivers/gpu/drm/xe/xe_migrate.c +21 -1
@@ -40,6 +40,8 @@struct xe_migrate {/** @q: Default exec queue used for migration */ struct xe_exec_queue *q; +/** @bind_q: Default exec queue used for binds */ +struct xe_exec_queue *bind_q;/** @tile: Backpointer to the tile this struct xe_migrate belongs to. */ struct xe_tile *tile;/** @job_mutex: Timeline mutex for @eng. */
@@ -87,6 +89,11 @@ struct xe_exec_queue *xe_tile_migrate_exec_queue(struct xe_tile *tile)return tile->migrate->q; } +struct xe_exec_queue *xe_tile_migrate_bind_exec_queue (struct xe_tile *tile) +{ +return tile->migrate->bind_q; +} +static void xe_migrate_fini (struct drm_device *dev,void *arg) {struct xe_migrate *m = arg;
@@ -105,6 +112,8 @@ static void xe_migrate_fini(struct drm_device *dev, void *arg)mutex_destroy (&m->job_mutex);xe_vm_close_and_put (m->q->vm);xe_exec_queue_put (m->q); +if (m->bind_q) +xe_exec_queue_put (m->bind_q); }static u64 xe_migrate_vm_addr (u64 slot,u32 level)
@@ -381,6 +390,15 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile)if (!hwe || !logical_mask)return ERR_PTR (-EINVAL ); + m->bind_q =xe_exec_queue_create (xe, vm, logical_mask,1 , hwe, +EXEC_QUEUE_FLAG_KERNEL | +EXEC_QUEUE_FLAG_PERMANENT ); +if (IS_ERR (m->bind_q)) { +xe_vm_close_and_put (vm); +return ERR_CAST (m->bind_q); + } + m->bind_q->entity->priority =DRM_SCHED_PRIORITY_KERNEL ; + m->q =xe_exec_queue_create (xe, vm, logical_mask,1 , hwe,EXEC_QUEUE_FLAG_KERNEL |EXEC_QUEUE_FLAG_PERMANENT );
@@ -391,6 +409,8 @@ struct xe_migrate *xe_migrate_init(struct xe_tile *tile) EXEC_QUEUE_FLAG_PERMANENT); }if (IS_ERR (m->q)) { +if (m->bind_q) +xe_exec_queue_put (m->bind_q);xe_vm_close_and_put (vm);return ERR_CAST (m->q); }
@@ -1132,7 +1152,7 @@ __xe_migrate_update_pgtables(struct xe_migrate *m,struct xe_tile *tile = m->tile;struct xe_sched_job *job;struct dma_fence *fence; -bool is_migrate = pt_update_ops->q == m->q; +bool is_migrate = pt_update_ops->q == m->bind_q;u64 batch_addr[2 ] = {0 ,0 };int err;
drivers/gpu/drm/xe/xe_migrate.h +1 -0
@@ -118,5 +118,6 @@ xe_migrate_update_pgtables(struct xe_migrate *m,void xe_migrate_wait (struct xe_migrate *m);struct xe_exec_queue *xe_tile_migrate_exec_queue (struct xe_tile *tile); +struct xe_exec_queue *xe_tile_migrate_bind_exec_queue (struct xe_tile *tile);#endif
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 26/26 drm/xe/uapi: Make sync vs async VM bind operations per IOCTL rather than queue
In reply to Matthew BrostMaking sync vs async per IOCTL simplifies the uAPI while also making it more flexible. Adjust the uAPI accordingly.
--- drivers/gpu/drm/xe/xe_exec_queue.c | 7 +-- drivers/gpu/drm/xe/xe_exec_queue_types.h | 2 - drivers/gpu/drm/xe/xe_vm.c | 59 +++++++++--------------- drivers/gpu/drm/xe/xe_vm_types.h | 15 +++--- include/uapi/drm/xe_drm.h | 6 +-- 5 files changed, 34 insertions(+), 55 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue.c +1 -6
@@ -622,10 +622,7 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data,if (XE_IOCTL_DBG (xe, eci[0 ].gt_id >= xe->info.gt_count))return -EINVAL ; -if (eci[0 ].engine_class >=DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC ) { -bool sync = eci[0 ].engine_class == -DRM_XE_ENGINE_CLASS_VM_BIND_SYNC ; - +if (eci[0 ].engine_class ==DRM_XE_ENGINE_CLASS_VM_BIND ) {for_each_gt (gt, xe, id) {struct xe_exec_queue *new;
@@ -651,8 +648,6 @@ int xe_exec_queue_create_ioctl(struct drm_device *dev, void *data, args->width, hwe,EXEC_QUEUE_FLAG_PERSISTENT |EXEC_QUEUE_FLAG_VM | - (sync ?0 : -EXEC_QUEUE_FLAG_VM_ASYNC ) | (id ?EXEC_QUEUE_FLAG_BIND_ENGINE_CHILD : 0));
drivers/gpu/drm/xe/xe_exec_queue_types.h +0 -2
@@ -71,8 +71,6 @@ struct xe_exec_queue {#define EXEC_QUEUE_FLAG_VM BIT(4)/* child of VM queue for multi-tile VM jobs */ #define EXEC_QUEUE_FLAG_BIND_ENGINE_CHILD BIT(5) -/* VM jobs for this queue are asynchronous */ -#define EXEC_QUEUE_FLAG_VM_ASYNC BIT(6)/** * @flags: flags for this exec queue, should statically setup aside from ban
drivers/gpu/drm/xe/xe_vm.c +23 -36
149 lines not on this page · raw message
drivers/gpu/drm/xe/xe_vm_types.h +8 -7
@@ -217,6 +217,8 @@ struct xe_vma_ops {struct xe_sync_entry *syncs;/** @num_syncs: number of syncs */ u32 num_syncs; +/** @async: VMA operations are asynchronous */ +bool async;/** @pt_update_ops: page table update operations */ struct xe_vm_pgtable_update_ops pt_update_ops[XE_MAX_TILES_PER_DEVICE ]; };
@@ -252,13 +254,12 @@ struct xe_vm {*/ #define XE_VM_FLAG_64K BIT(0)#define XE_VM_FLAG_COMPUTE_MODE BIT(1) -#define XE_VM_FLAG_ASYNC_DEFAULT BIT(2) -#define XE_VM_FLAG_MIGRATION BIT(3) -#define XE_VM_FLAG_SCRATCH_PAGE BIT(4) -#define XE_VM_FLAG_FAULT_MODE BIT(5) -#define XE_VM_FLAG_BANNED BIT(6) -#define XE_VM_FLAG_TILE_ID (flags) FIELD_GET(GENMASK(8, 7), flags) -#define XE_VM_FLAG_SET_TILE_ID (tile) FIELD_PREP(GENMASK(8, 7), (tile)->id) +#define XE_VM_FLAG_MIGRATION BIT(2) +#define XE_VM_FLAG_SCRATCH_PAGE BIT(3) +#define XE_VM_FLAG_FAULT_MODE BIT(4) +#define XE_VM_FLAG_BANNED BIT(5) +#define XE_VM_FLAG_TILE_ID (flags) FIELD_GET(GENMASK(7, 6), flags) +#define XE_VM_FLAG_SET_TILE_ID (tile) FIELD_PREP(GENMASK(7, 6), (tile)->id)unsigned long flags;/** @composite_fence_ctx: context composite fence */
include/uapi/drm/xe_drm.h +2 -4
@@ -137,8 +137,7 @@ struct drm_xe_engine_class_instance {* Kernel only classes (not actual hardware engine class). Used for * creating ordered queues of VM bind operations. */ -#define DRM_XE_ENGINE_CLASS_VM_BIND_ASYNC 5 -#define DRM_XE_ENGINE_CLASS_VM_BIND_SYNC 6 +#define DRM_XE_ENGINE_CLASS_VM_BIND 5__u16 engine_class;__u16 engine_instance;
@@ -597,8 +596,7 @@ struct drm_xe_vm_create {#define DRM_XE_VM_CREATE_SCRATCH_PAGE (0x1 << 0)#define DRM_XE_VM_CREATE_COMPUTE_MODE (0x1 << 1) -#define DRM_XE_VM_CREATE_ASYNC_DEFAULT (0x1 << 2) -#define DRM_XE_VM_CREATE_FAULT_MODE (0x1 << 3) +#define DRM_XE_VM_CREATE_FAULT_MODE (0x1 << 2)/** @flags: Flags */ __u32 flags;
Signature · 1 line
2.34.1
Patchwork
CI[Intel-xe] ✓ CI.Patch_applied: success for Refactor VM bind code
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: Refactor VM bind code
URL : https://patchwork.freedesktop.org/series/125608/
State : success
== Summary ==
=== Applying kernel patches on branch 'drm-xe-next' with base: ===
Base commit: 4c7ed0ef2 drm/xe/mocs: MOCS registers are multicast on Xe_HP and beyond === git am output follows ===
Applying: drm/xe: Allow num_binds == 0 in VM bind IOCTL
Applying: drm/xe: Allow num_batch_buffer == 0 in exec IOCTL
Applying: drm/xe: Lock all gpuva ops during VM bind IOCTL
Applying: drm/xe: Add ops_execute function which returns a fence
Applying: drm/xe: Move migrate to prefetch to op_lock funtion
Applying: drm/xe: Add struct xe_vma_ops abstraction
Applying: drm/xe: Update xe_vm_rebind to use dummy VMA operations
Applying: drm/xe: Move drm exec loop to vm_bind_ioctl_ops_execute
Applying: drm/xe: Fixup error handling / ref counting in VM bind IOCTL
Applying: drm/xe: Convert pagefault rebind to use ops interface
Applying: drm/xe: Add some members to xe_vma_ops
Applying: drm/xe: Add vm_bind_ioctl_ops_install_fences helper
Applying: drm/xe: Drop rebind argument from xe_pt_prepare_bind
Applying: drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops
Applying: drm/xe: Move setting last fence and sync wait to vm_bind_ioctl_ops_install_fences
Applying: drm/xe: Fix vma_is_valid to use tile argument
Applying: drm/xe: Adjust tile mask in operations create
Applying: drm/xe: Add xe_gt_tlb_invalidation_range and convert PT layer to use this
Applying: drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queue
Applying: drm/xe: Convert multiple bind ops into single job
Applying: drm/xe: Update clear / populate arguments
Applying: drm/xe: Add __xe_migrate_update_pgtables_cpu helper
Applying: drm/xe: Add xe_hw_fence_signal helper
Applying: drm/xe: CPU binds for jobs
Applying: drm/xe: Don't use migrate exec queue for page fault binds
Applying: drm/xe/uapi: Make sync vs async VM bind operations per IOCTL rather than queue
Patchwork
CI[Intel-xe] ✗ CI.checkpatch: warning for Refactor VM bind code
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: Refactor VM bind code
URL : https://patchwork.freedesktop.org/series/125608/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master 63c2b6b160bca2df6efc7bc4cea6f442097d7854
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 7b761f152371ab5b7e2f21f970d459e0c3778b82
Author: Matthew Brost <[email protected]> Date: Wed Oct 25 21:02:13 2023 -0700
drm/xe/uapi: Make sync vs async VM bind operations per IOCTL rather than queue
Making sync vs async per IOCTL simplifies the uAPI while also making it more flexible. Adjust the uAPI accordingly.
+ /mt/dim checkpatch 4c7ed0ef2d582ae173d9a2062ee6f7360d1aa1df drm-intel 5e06ef0ef drm/xe: Allow num_binds == 0 in VM bind IOCTL
-:52: WARNING:ALLOC_WITH_MULTIPLY: Prefer kcalloc over kzalloc with multiply #52: FILE: drivers/gpu/drm/xe/xe_vm.c:3020:
+ bos = kzalloc(sizeof(*bos) * args->num_binds, GFP_KERNEL);
total: 0 errors, 1 warnings, 0 checks, 61 lines checkedc2e508592 drm/xe: Allow num_batch_buffer == 0 in exec IOCTL 9b32c4be2 drm/xe: Lock all gpuva ops during VM bind IOCTL 7c2c36d6b drm/xe: Add ops_execute function which returns a fence 42d99b48a drm/xe: Move migrate to prefetch to op_lock funtion -:4: WARNING:TYPO_SPELLING: 'funtion' may be misspelled - perhaps 'function'? #4:
Subject: [PATCH] drm/xe: Move migrate to prefetch to op_lock funtion
^^^^^^^
total: 0 errors, 1 warnings, 0 checks, 57 lines checkedeffe5583e drm/xe: Add struct xe_vma_ops abstraction21d5f97cb drm/xe: Update xe_vm_rebind to use dummy VMA operations6508a6399 drm/xe: Move drm exec loop to vm_bind_ioctl_ops_executec36aea3fa drm/xe: Fixup error handling / ref counting in VM bind IOCTL 4dc614ca6 drm/xe: Convert pagefault rebind to use ops interface583508ab5 drm/xe: Add some members to xe_vma_ops5d2dd52fe drm/xe: Add vm_bind_ioctl_ops_install_fences helper81b301767 drm/xe: Drop rebind argument from xe_pt_prepare_binde83a5202b drm/xe: Add xe_vm_pgtable_update_op to xe_vma_ops278b05b89 drm/xe: Move setting last fence and sync wait to vm_bind_ioctl_ops_install_fences 80baec27d drm/xe: Fix vma_is_valid to use tile argument3803fe843 drm/xe: Adjust tile mask in operations create9730b8e16 drm/xe: Add xe_gt_tlb_invalidation_range and convert PT layer to use this -:107: CHECK:BRACES: Blank lines aren't necessary before a close brace '}' #107: FILE: drivers/gpu/drm/xe/xe_gt_tlb_invalidation.c:310:
+
+}
total: 0 errors, 0 warnings, 1 checks, 186 lines checked212783cca drm/xe: s/xe_tile_migrate_engine/xe_tile_migrate_exec_queueba612daa9 drm/xe: Convert multiple bind ops into single job309ba121d drm/xe: Update clear / populate arguments9348726cb drm/xe: Add __xe_migrate_update_pgtables_cpu helperd9565a0dd drm/xe: Add xe_hw_fence_signal helper
-:40: WARNING:FROM_SIGN_OFF_MISMATCH: From:/Signed-off-by: email address mismatch: 'From: Matthew Brost <[email protected]>' != 'Signed-off-by: Matthew Brost <[email protected]>'
total: 0 errors, 1 warnings, 0 checks, 20 lines checked 9e17ead37 drm/xe: CPU binds for jobs
-:467: CHECK:LINE_SPACING: Please don't use multiple blank lines #467: FILE: drivers/gpu/drm/xe/xe_migrate.c:1155:
+
+
-:645: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #645: FILE: drivers/gpu/drm/xe/xe_sched_job.c:38:
+ max_t(size_t,
sizeof(u64) *
-:704: ERROR:FLEXIBLE_ARRAY: Use C99 flexible arrays - see https://docs.kernel.org/process/deprecated.html#zero-length-and-one-element-arrays
#704: FILE: drivers/gpu/drm/xe/xe_sched_job_types.h:70:
+ struct pt_update_args pt_update[0];
+ };
-:757: CHECK:PARENTHESIS_ALIGNMENT: Alignment should match open parenthesis #757: FILE: drivers/gpu/drm/xe/xe_vm.c:868:
+int xe_vm_populate_dummy_rebind(struct xe_vm *vm, struct xe_vma *vma,
u8 tile_mask)
total: 1 errors, 0 warnings, 3 checks, 772 lines checked575cfe921 drm/xe: Don't use migrate exec queue for page fault binds 7b761f152 drm/xe/uapi: Make sync vs async VM bind operations per IOCTL rather than queue
Patchwork
CI[Intel-xe] ✗ CI.KUnit: failure for Refactor VM bind code
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: Refactor VM bind code
URL : https://patchwork.freedesktop.org/series/125608/
State : failure
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig
ERROR:root:In file included from ../drivers/gpu/drm/xe/xe_migrate.c:1253:../drivers/gpu/drm/xe/tests/xe_migrate.c:91:14: error: initialization of ‘void (*)(struct xe_tile *, struct iosys_map *, void *, u32, u32, const struct xe_vm_pgtable_update *)’ {aka ‘void (*)(struct xe_tile *, struct iosys_map *, void *, unsigned int, unsigned int, const struct xe_vm_pgtable_update *)’} from incompatible pointer type ‘void (*)(struct xe_migrate_pt_update *, struct xe_tile *, struct iosys_map *, void *, u32, u32, const struct xe_vm_pgtable_update *)’ {aka ‘void (*)(struct xe_migrate_pt_update *, struct xe_tile *, struct iosys_map *, void *, unsigned int, unsigned int, const struct xe_vm_pgtable_update *)’} [-Werror=incompatible-pointer-types]
91 | .populate = sanity_populate_cb,
| ^~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/tests/xe_migrate.c:91:14: note: (near initialization for ‘sanity_ops.populate’) ../drivers/gpu/drm/xe/tests/xe_migrate.c: In function ‘test_pt_update’:../drivers/gpu/drm/xe/tests/xe_migrate.c:239:10: error: too many arguments to function ‘xe_migrate_update_pgtables’
239 | fence = xe_migrate_update_pgtables(m, NULL, NULL, m->q, &update, 1,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_migrate.c:1223:1: note: declared here
1223 | xe_migrate_update_pgtables(struct xe_migrate *m,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~cc1: some warnings being treated as errors
make[7]: *** [../scripts/Makefile.build:243: drivers/gpu/drm/xe/xe_migrate.o] Error 1 make[7]: *** Waiting for unfinished jobs....
make[6]: *** [../scripts/Makefile.build:480: drivers/gpu/drm/xe] Error 2 make[6]: *** Waiting for unfinished jobs....
make[5]: *** [../scripts/Makefile.build:480: drivers/gpu/drm] Error 2 make[4]: *** [../scripts/Makefile.build:480: drivers/gpu] Error 2 make[3]: *** [../scripts/Makefile.build:480: drivers] Error 2
make[2]: *** [/kernel/Makefile:1913: .] Error 2
make[1]: *** [/kernel/Makefile:234: __sub-make] Error 2
make: *** [Makefile:234: __sub-make] Error 2
[04:51:47] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig [04:51:51] Building KUnit Kernel ... Populating config with: $ make ARCH=um O=.kunit olddefconfig Building with: $ make ARCH=um O=.kunit --jobs=48 + cleanup ++ stat -c %u:%g /kernel + chown -R 1003:1003 /kernel