add recovery entity and run queue
Subsystems (2)
cover add recovery entity and run queue
every vm has itself recovery entity, which is used to reovery page table from their shadow. They don't need to wait front vm completed.
And also using all pte rings can speed reovery.
every scheduler has its own recovery entity, which is used to save hw jobs, and resubmit from it, which solves the conflicts between reset thread and scheduler thread when run job.
And some fixes when doing this improment.
Chunming Zhou (11):
drm/amdgpu: hw ring should be empty when gpu reset drm/amdgpu: specify entity to amdgpu_copy_buffer drm/amd: add recover run queue for scheduler drm/amdgpu: fix vm init error path drm/amdgpu: add vm recover entity drm/amdgpu: use all pte rings to recover page table drm/amd: add recover entity for every scheduler drm/amd: use scheduler to recover hw jobs drm/amd: hw job list should be exact drm/amd: reset jobs to recover entity drm/amdgpu: no need fence wait every time
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 35 +++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 11 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_test.c | 8 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 27 ++++-- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 129 +++++++++++++------------- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 4 +- 9 files changed, 135 insertions(+), 92 deletions(-)
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1470124302-23615-1-git-send-email-David1.Zhou-5C7GfCeVMHo@public.gmane.org
PATCH 1/11 drm/amdgpu: hw ring should be empty when gpu reset
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 11 +++++++++++ 3 files changed, 13 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -1253,6 +1253,7 @@ int amdgpu_ib_ring_tests(struct amdgpu_device *adev);int amdgpu_ring_alloc (struct amdgpu_ring *ring,unsigned ndw);void amdgpu_ring_insert_nop (struct amdgpu_ring *ring,uint32_t count);void amdgpu_ring_generic_pad_ib (struct amdgpu_ring *ring,struct amdgpu_ib *ib); +void amdgpu_ring_reset (struct amdgpu_ring *ring);void amdgpu_ring_commit (struct amdgpu_ring *ring);void amdgpu_ring_undo (struct amdgpu_ring *ring); int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
@@ -2117,6 +2117,7 @@ int amdgpu_gpu_reset(struct amdgpu_device *adev)continue ;kthread_park (ring->sched.thread);amd_sched_hw_job_reset (&ring->sched); +amdgpu_ring_reset (ring); }/* after all hw jobs are reset, hw fence is meaningless, so force_completion */ amdgpu_fence_driver_force_completion (adev);
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +11 -0
@@ -110,6 +110,17 @@ void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib) ib->ptr[ib->length_dw++] = ring->nop; } +void amdgpu_ring_reset (struct amdgpu_ring *ring) +{ +u32 rptr =amdgpu_ring_get_rptr (ring); + + ring->wptr = rptr; + ring->wptr &= ring->ptr_mask; + +mb (); +amdgpu_ring_set_wptr (ring); +} +/** * amdgpu_ring_commit - tell the GPU to execute the new * commands on the ring buffer
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 2/11 drm/amdgpu: specify entity to amdgpu_copy_buffer
In reply to a message not in the archive--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_test.c | 8 ++++---- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 5 +++-- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +- 5 files changed, 11 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -432,6 +432,7 @@ struct amdgpu_mman { }; int amdgpu_copy_buffer(struct amdgpu_ring *ring, + struct amd_sched_entity *entity, uint64_t src_offset, uint64_t dst_offset, uint32_t byte_count,
drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c +2 -1
@@ -39,7 +39,8 @@ static int amdgpu_benchmark_do_move(struct amdgpu_device *adev, unsigned size, start_jiffies = jiffies;for (i =0 ; i < n; i++) {struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring; - r =amdgpu_copy_buffer (ring, saddr, daddr, size,NULL , &fence); + r =amdgpu_copy_buffer (ring, &adev->mman.entity, + saddr, daddr, size,NULL , &fence);if (r) goto exit_do_move; r =fence_wait (fence, false);
drivers/gpu/drm/amd/amdgpu/amdgpu_test.c +4 -4
@@ -110,8 +110,8 @@ static void amdgpu_do_test_moves(struct amdgpu_device *adev)amdgpu_bo_kunmap (gtt_obj[i]); - r =amdgpu_copy_buffer (ring, gtt_addr, vram_addr, - size,NULL , &fence); + r =amdgpu_copy_buffer (ring, &adev->mman.entity, gtt_addr, + vram_addr, size,NULL , &fence);if (r) {DRM_ERROR ("Failed GTT->VRAM copy %d\n" , i);
@@ -155,8 +155,8 @@ static void amdgpu_do_test_moves(struct amdgpu_device *adev)amdgpu_bo_kunmap (vram_obj); - r =amdgpu_copy_buffer (ring, vram_addr, gtt_addr, - size,NULL , &fence); + r =amdgpu_copy_buffer (ring, &adev->mman.entity, vram_addr, + gtt_addr, size,NULL , &fence);if (r) {DRM_ERROR ("Failed VRAM->GTT copy %d\n" , i);
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +3 -2
@@ -283,7 +283,7 @@ static int amdgpu_move_blit(struct ttm_buffer_object *bo,BUILD_BUG_ON ((PAGE_SIZE %AMDGPU_GPU_PAGE_SIZE ) !=0 ); - r =amdgpu_copy_buffer (ring, old_start, new_start, + r =amdgpu_copy_buffer (ring, &adev->mman.entity, old_start, new_start, new_mem->num_pages *PAGE_SIZE ,/* bytes */ bo->resv, &fence);if (r)
@@ -1147,6 +1147,7 @@ int amdgpu_mmap(struct file *filp, struct vm_area_struct *vma) } int amdgpu_copy_buffer(struct amdgpu_ring *ring, + struct amd_sched_entity *entity, uint64_t src_offset, uint64_t dst_offset, uint32_t byte_count,
@@ -1195,7 +1196,7 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring,amdgpu_ring_pad_ib (ring, &job->ibs[0 ]);WARN_ON (job->ibs[0 ].length_dw > num_dw); - r =amdgpu_job_submit (job, ring, &adev->mman.entity, + r =amdgpu_job_submit (job, ring, entity,AMDGPU_FENCE_OWNER_UNDEFINED , fence);if (r) goto error_free;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -1
@@ -739,7 +739,7 @@ static int amdgpu_vm_recover_bo_from_shadow(struct amdgpu_device *adev,if (r) goto err3; - r =amdgpu_copy_buffer (ring, gtt_addr, vram_addr, + r =amdgpu_copy_buffer (ring, &adev->mman.entity, gtt_addr, vram_addr,amdgpu_bo_size (bo), resv, fence);if (!r)amdgpu_bo_fence (bo, *fence, true);
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 3/11 drm/amd: add recover run queue for scheduler
In reply to a message not in the archive--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +2 -1
@@ -112,7 +112,8 @@ struct amd_sched_backend_ops { };enum amd_sched_priority { -AMD_SCHED_PRIORITY_KERNEL =0 , +AMD_SCHED_PRIORITY_RECOVER =0 , +AMD_SCHED_PRIORITY_KERNEL ,AMD_SCHED_PRIORITY_NORMAL ,AMD_SCHED_MAX_PRIORITY };
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 4/11 drm/amdgpu: fix vm init error path
In reply to a message not in the archive--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -1
@@ -1682,7 +1682,7 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm) r =amd_sched_entity_init (&ring->sched, &vm->entity, rq, amdgpu_sched_jobs);if (r) -return r; + goto err; vm->page_directory_fence =NULL ;
@@ -1725,6 +1725,9 @@ error_free_page_directory: error_free_sched_entity:amd_sched_entity_fini (&ring->sched, &vm->entity); +err: +drm_free_large (vm->page_tables); +return r; }
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 5/11 drm/amdgpu: add vm recover entity
In reply to a message not in the archiveevery vm uses itself recover entity to recovery page table from shadow.
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 3 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 22 +++++++++++++++------- 2 files changed, 17 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +2 -1
@@ -917,7 +917,8 @@ struct amdgpu_vm {/* Scheduler entity for page table updates */ struct amd_sched_entity entity; - +struct amd_sched_entity recover_entity; +struct amdgpu_ring *ring;/* client id */ u64 client_id;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +15 -7
@@ -714,13 +714,13 @@ error_free: }static int amdgpu_vm_recover_bo_from_shadow (struct amdgpu_device *adev, +struct amdgpu_vm *vm,struct amdgpu_bo *bo,struct amdgpu_bo *bo_shadow,struct reservation_object *resv,struct fence **fence) { -struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;int r;uint64_t vram_addr, gtt_addr;
@@ -739,8 +739,8 @@ static int amdgpu_vm_recover_bo_from_shadow(struct amdgpu_device *adev,if (r) goto err3; - r =amdgpu_copy_buffer (ring, &adev->mman.entity, gtt_addr, vram_addr, -amdgpu_bo_size (bo), resv, fence); + r =amdgpu_copy_buffer (vm->ring, &vm->recover_entity, gtt_addr, + vram_addr,amdgpu_bo_size (bo), resv, fence);if (!r)amdgpu_bo_fence (bo, *fence, true);
@@ -767,7 +767,7 @@ int amdgpu_vm_recover_page_table_from_shadow(struct amdgpu_device *adev,if (unlikely (r !=0 ))return r; - r =amdgpu_vm_recover_bo_from_shadow (adev, vm->page_directory, + r =amdgpu_vm_recover_bo_from_shadow (adev, vm, vm->page_directory, vm->page_directory->shadow,NULL , &fence);if (r) {
@@ -784,7 +784,7 @@ int amdgpu_vm_recover_page_table_from_shadow(struct amdgpu_device *adev,if (!bo || !bo_shadow)continue ; - r =amdgpu_vm_recover_bo_from_shadow (adev, bo, bo_shadow, + r =amdgpu_vm_recover_bo_from_shadow (adev, vm, bo, bo_shadow,NULL , &fence);if (r) {DRM_ERROR ("recover page table failed!\n" );
@@ -1678,12 +1678,17 @@ int amdgpu_vm_init(struct amdgpu_device *adev, struct amdgpu_vm *vm) ring_instance =atomic_inc_return (&adev->vm_manager.vm_pte_next_ring); ring_instance %= adev->vm_manager.vm_pte_num_rings; ring = adev->vm_manager.vm_pte_rings[ring_instance]; + rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_RECOVER ]; + r =amd_sched_entity_init (&ring->sched, &vm->recover_entity, + rq, amdgpu_sched_jobs); +if (r) + goto err; rq = &ring->sched.sched_rq[AMD_SCHED_PRIORITY_KERNEL ]; r =amd_sched_entity_init (&ring->sched, &vm->entity, rq, amdgpu_sched_jobs);if (r) - goto err; - + goto err1; + vm->ring = ring; vm->page_directory_fence =NULL ; r = amdgpu_bo_create(adev, pd_size, align, true,
@@ -1725,6 +1730,8 @@ error_free_page_directory: error_free_sched_entity:amd_sched_entity_fini (&ring->sched, &vm->entity); +err1: +amd_sched_entity_fini (&ring->sched, &vm->recover_entity); err:drm_free_large (vm->page_tables);
@@ -1748,6 +1755,7 @@ void amdgpu_vm_fini(struct amdgpu_device *adev, struct amdgpu_vm *vm)spin_lock (&adev->vm_list_lock);list_del (&vm->list);spin_unlock (&adev->vm_list_lock); +amd_sched_entity_fini (vm->entity.sched, &vm->recover_entity);amd_sched_entity_fini (vm->entity.sched, &vm->entity);if (!RB_EMPTY_ROOT (&vm->va)) {
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 6/11 drm/amdgpu: use all pte rings to recover page table
In reply to a message not in the archive--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 34 ++++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +25 -9
@@ -2098,6 +2098,7 @@ int amdgpu_gpu_reset(struct amdgpu_device *adev)int i, r;int resched;bool need_full_reset; +u32 unpark_bits;if (!amdgpu_check_soft_reset (adev)) {DRM_INFO ("No hardware hang detected. Did some blocks stall?\n" );
@@ -2119,6 +2120,7 @@ int amdgpu_gpu_reset(struct amdgpu_device *adev)amd_sched_hw_job_reset (&ring->sched);amdgpu_ring_reset (ring); } + unpark_bits =0 ;/* after all hw jobs are reset, hw fence is meaningless, so force_completion */ amdgpu_fence_driver_force_completion (adev);/* store modesetting */
@@ -2162,8 +2164,6 @@ retry:amdgpu_atombios_scratch_regs_restore (adev); }if (!r) { -struct amdgpu_ring *buffer_ring = adev->mman.buffer_funcs_ring; -amdgpu_irq_gpu_reset_resume_helper (adev); r =amdgpu_ib_ring_tests (adev);if (r) {
@@ -2178,11 +2178,20 @@ retry:*/ if (need_full_reset && !(adev->flags &AMD_IS_APU )) {struct amdgpu_vm *vm, *tmp; +int i;DRM_INFO ("recover page table from shadow\n" ); -amd_sched_rq_block_entity ( - &buffer_ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ], true); -kthread_unpark (buffer_ring->sched.thread); +for (i =0 ; i < adev->vm_manager.vm_pte_num_rings; i++) { +struct amdgpu_ring *ring = adev->vm_manager.vm_pte_rings[i]; + +amd_sched_rq_block_entity ( + &ring->sched.sched_rq[AMD_SCHED_PRIORITY_KERNEL ], true); +amd_sched_rq_block_entity ( + &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ], true); +kthread_unpark (ring->sched.thread); + unpark_bits |=1 << ring->idx; + } +spin_lock (&adev->vm_list_lock);list_for_each_entry_safe (vm, tmp, &adev->vm_list, list) {spin_unlock (&adev->vm_list_lock);
@@ -2190,8 +2199,15 @@ retry:spin_lock (&adev->vm_list_lock); }spin_unlock (&adev->vm_list_lock); -amd_sched_rq_block_entity ( - &buffer_ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ], false); + +for (i =0 ; i < adev->vm_manager.vm_pte_num_rings; i++) { +struct amdgpu_ring *ring = adev->vm_manager.vm_pte_rings[i]; + +amd_sched_rq_block_entity ( + &ring->sched.sched_rq[AMD_SCHED_PRIORITY_KERNEL ], false); +amd_sched_rq_block_entity ( + &ring->sched.sched_rq[AMD_SCHED_PRIORITY_NORMAL ], false); + } }for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i];
@@ -2199,9 +2215,9 @@ retry:continue ;DRM_INFO ("ring:%d recover jobs\n" , ring->idx); -kthread_park (buffer_ring->sched.thread);amd_sched_job_recovery (&ring->sched); -kthread_unpark (ring->sched.thread); +if (!((unpark_bits >> ring->idx) &0x1 )) +kthread_unpark (ring->sched.thread); } }else {dev_err (adev->dev,"asic resume failed (%d).\n" , r);
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 7/11 drm/amd: add recover entity for every scheduler
In reply to a message not in the archiveIt will be used to recover hw jobs.
--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 24 ++++++++++++++++++++---- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 1 + 2 files changed, 21 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/scheduler/gpu_scheduler.c +20 -4
@@ -635,7 +635,7 @@ int amd_sched_init(struct amd_gpu_scheduler *sched,const struct amd_sched_backend_ops *ops, unsigned hw_submission, long timeout, const char *name) { - int i; + int i, r; sched->ops = ops; sched->hw_submission_limit = hw_submission; sched->name = name;
@@ -648,22 +648,37 @@ int amd_sched_init(struct amd_gpu_scheduler *sched,INIT_LIST_HEAD (&sched->ring_mirror_list);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 ); + r =amd_sched_entity_init (sched, &sched->recover_entity, + &sched->sched_rq[AMD_SCHED_PRIORITY_RECOVER ], + hw_submission); +if (r) +return r;if (atomic_inc_return (&sched_fence_slab_ref) ==1 ) { sched_fence_slab =kmem_cache_create ("amd_sched_fence" ,sizeof (struct amd_sched_fence ),0 ,SLAB_HWCACHE_ALIGN ,NULL ); -if (!sched_fence_slab) -return -ENOMEM ; +if (!sched_fence_slab) { + r = -ENOMEM ; + goto err1; + } }/* Each scheduler will run on a seperate kernel thread */ sched->thread =kthread_run (amd_sched_main, sched, sched->name);if (IS_ERR (sched->thread)) {DRM_ERROR ("Failed to create scheduler for %s.\n" , name); -return PTR_ERR (sched->thread); + r =PTR_ERR (sched->thread); + goto err2; }return 0 ; +err2: +if (atomic_dec_and_test (&sched_fence_slab_ref)) +kmem_cache_destroy (sched_fence_slab); + +err1: +amd_sched_entity_fini (sched, &sched->recover_entity); +return r; }/**
@@ -677,4 +692,5 @@ void amd_sched_fini(struct amd_gpu_scheduler *sched)kthread_stop (sched->thread);if (atomic_dec_and_test (&sched_fence_slab_ref))kmem_cache_destroy (sched_fence_slab); +amd_sched_entity_fini (sched, &sched->recover_entity); }
drivers/gpu/drm/amd/scheduler/gpu_scheduler.h +1 -0
@@ -133,6 +133,7 @@ struct amd_gpu_scheduler {struct task_struct *thread;struct list_head ring_mirror_list;spinlock_t job_list_lock; +struct amd_sched_entity recover_entity; }; int amd_sched_init(struct amd_gpu_scheduler *sched,
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 8/11 drm/amd: use scheduler to recover hw jobs
In reply to a message not in the archiveThe old way is trying to recover hw jobs directly, which will conflict with scheduler thread.
--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 48 +++++++++------------------ 1 file changed, 16 insertions(+), 32 deletions(-)
drivers/gpu/drm/amd/scheduler/gpu_scheduler.c +16 -32
@@ -324,10 +324,12 @@ amd_sched_entity_pop_job(struct amd_sched_entity *entity)* * Returns true if we could submit the job. */ -static bool amd_sched_entity_in (struct amd_sched_job *sched_job) +static bool amd_sched_entity_in_or_recover (struct amd_sched_job *sched_job, +bool recover) {struct amd_gpu_scheduler *sched = sched_job->sched; -struct amd_sched_entity *entity = sched_job->s_entity; +struct amd_sched_entity *entity = recover ? &sched->recover_entity : + sched_job->s_entity;bool added, first = false;spin_lock (&entity->queue_lock);
@@ -348,6 +350,15 @@ static bool amd_sched_entity_in(struct amd_sched_job *sched_job)return added; } +static void amd_sched_entity_push_job_recover (struct amd_sched_job *sched_job) +{ +struct amd_sched_entity *entity = sched_job->s_entity; + +trace_amd_sched_job (sched_job); +wait_event (entity->sched->job_scheduled, +amd_sched_entity_in_or_recover (sched_job, true)); +} +/* job_finish is called after hw fence signaled, and * the job had already been deleted from ring_mirror_list */
@@ -426,39 +437,12 @@ void amd_sched_hw_job_reset(struct amd_gpu_scheduler *sched)void amd_sched_job_recovery (struct amd_gpu_scheduler *sched) {struct amd_sched_job *s_job, *tmp; -int r;spin_lock (&sched->job_list_lock); - s_job =list_first_entry_or_null (&sched->ring_mirror_list, - struct amd_sched_job, node); -if (s_job) -schedule_delayed_work (&s_job->work_tdr, sched->timeout); -list_for_each_entry_safe (s_job, tmp, &sched->ring_mirror_list, node) { -struct amd_sched_fence *s_fence = s_job->s_fence; -struct fence *fence, *dependency; - +list_del_init (&s_job->node);spin_unlock (&sched->job_list_lock); -while ((dependency = sched->ops->dependency (s_job))) { -fence_wait (dependency, false); -fence_put (dependency); - } - fence = sched->ops->run_job (s_job); -atomic_inc (&sched->hw_rq_count); -if (fence) { - s_fence->parent =fence_get (fence); - r =fence_add_callback (fence, &s_fence->cb, - amd_sched_process_job); -if (r == -ENOENT ) -amd_sched_process_job (fence, &s_fence->cb); -else if (r) -DRM_ERROR ("fence add callback failed (%d)\n" , - r); -fence_put (fence); - }else { -DRM_ERROR ("Failed to run job!\n" ); -amd_sched_process_job (NULL , &s_fence->cb); - } +amd_sched_entity_push_job_recover (s_job);spin_lock (&sched->job_list_lock); }spin_unlock (&sched->job_list_lock);
@@ -479,7 +463,7 @@ void amd_sched_entity_push_job(struct amd_sched_job *sched_job)fence_add_callback (&sched_job->s_fence->finished, &sched_job->finish_cb, amd_sched_job_finish_cb);wait_event (entity->sched->job_scheduled, -amd_sched_entity_in (sched_job)); +amd_sched_entity_in_or_recover (sched_job, false)); }/* init a sched_job with basic field */
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 9/11 drm/amd: hw job list should be exact
In reply to a message not in the archivehw job list should be exact, so deleting job node should be in irq handler instead of work thread.
And Calculating time of next job should be immediate as well.
--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 38 +++++++++++++-------------- 1 file changed, 18 insertions(+), 20 deletions(-)
drivers/gpu/drm/amd/scheduler/gpu_scheduler.c +18 -20
@@ -367,34 +367,32 @@ static void amd_sched_job_finish(struct work_struct *work) struct amd_sched_job *s_job = container_of(work, struct amd_sched_job, finish_work);struct amd_gpu_scheduler *sched = s_job->sched; -unsigned long flags; - -/* remove job from ring_mirror_list */ -spin_lock_irqsave (&sched->job_list_lock, flags); -list_del_init (&s_job->node); -if (sched->timeout !=MAX_SCHEDULE_TIMEOUT ) { -struct amd_sched_job *next; -spin_unlock_irqrestore (&sched->job_list_lock, flags); +if (sched->timeout !=MAX_SCHEDULE_TIMEOUT )cancel_delayed_work_sync (&s_job->work_tdr); -spin_lock_irqsave (&sched->job_list_lock, flags); - -/* queue TDR for next job */ - next =list_first_entry_or_null (&sched->ring_mirror_list, - struct amd_sched_job, node); -if (next) -schedule_delayed_work (&next->work_tdr, sched->timeout); - } -spin_unlock_irqrestore (&sched->job_list_lock, flags); sched->ops->free_job (s_job); }static void amd_sched_job_finish_cb (struct fence *f,struct fence_cb *cb) { - struct amd_sched_job *job = container_of(cb, struct amd_sched_job, - finish_cb); -schedule_work (&job->finish_work); + struct amd_sched_job *s_job = container_of(cb, struct amd_sched_job, + finish_cb); +struct amd_gpu_scheduler *sched = s_job->sched; +struct amd_sched_job *next; +unsigned long flags; + +/* remove job from ring_mirror_list */ +spin_lock_irqsave (&sched->job_list_lock, flags); +list_del_init (&s_job->node); +/* queue TDR for next job */ + next =list_first_entry_or_null (&sched->ring_mirror_list, + struct amd_sched_job, node); +spin_unlock_irqrestore (&sched->job_list_lock, flags); +if ((sched->timeout !=MAX_SCHEDULE_TIMEOUT ) && next) +schedule_delayed_work (&next->work_tdr, sched->timeout); + +schedule_work (&s_job->finish_work); }static void amd_sched_job_begin (struct amd_sched_job *s_job)
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 10/11 drm/amd: reset jobs to recover entity
In reply to a message not in the archiveremove recover_entity for recover_rq when reset job. add recover_entity back when recover job
--- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 19 ++++++++++--------- 1 file changed, 10 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/scheduler/gpu_scheduler.c +10 -9
@@ -417,9 +417,10 @@ static void amd_sched_job_timedout(struct work_struct *work) job->sched->ops->timedout_job (job); } +/* scheduler must be parked before job reset */ void amd_sched_hw_job_reset (struct amd_gpu_scheduler *sched) { -struct amd_sched_job *s_job; +struct amd_sched_job *s_job, *tmp;spin_lock (&sched->job_list_lock);list_for_each_entry_reverse (s_job, &sched->ring_mirror_list, node) {
@@ -429,14 +430,6 @@ void amd_sched_hw_job_reset(struct amd_gpu_scheduler *sched) } }atomic_set (&sched->hw_rq_count,0 ); -spin_unlock (&sched->job_list_lock); -} - -void amd_sched_job_recovery (struct amd_gpu_scheduler *sched) -{ -struct amd_sched_job *s_job, *tmp; - -spin_lock (&sched->job_list_lock);list_for_each_entry_safe (s_job, tmp, &sched->ring_mirror_list, node) {list_del_init (&s_job->node);spin_unlock (&sched->job_list_lock);
@@ -444,6 +437,14 @@ void amd_sched_job_recovery(struct amd_gpu_scheduler *sched)spin_lock (&sched->job_list_lock); }spin_unlock (&sched->job_list_lock); +amd_sched_rq_remove_entity (&sched->sched_rq[AMD_SCHED_PRIORITY_RECOVER ], + &sched->recover_entity); +} + +void amd_sched_job_recovery (struct amd_gpu_scheduler *sched) +{ +amd_sched_rq_add_entity (&sched->sched_rq[AMD_SCHED_PRIORITY_RECOVER ], + &sched->recover_entity); }/**
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 11/11 drm/amdgpu: no need fence wait every time
In reply to a message not in the archiverecover entities have handled very well for each dependency.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 -- 1 file changed, 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -2
@@ -797,8 +797,6 @@ int amdgpu_vm_recover_page_table_from_shadow(struct amdgpu_device *adev, err:amdgpu_bo_unreserve (vm->page_directory); -if (vm->recover_pt_fence) - r =fence_wait (vm->recover_pt_fence, false);return r; }
Signature · 6 lines
1.9.1 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Hi,
I didn't see any of my previous review remarks on your previous submission addresses in this submission of this series?
Kind Regards,
Edward.
Quoted 21 lines · Chunming Zhou
every vm has itself recovery entity, which is used to reovery page table from their shadow. They don't need to wait front vm completed.
And also using all pte rings can speed reovery.every scheduler has its own recovery entity, which is used to save hw jobs, and resubmit from it, which solves the conflicts between reset thread and scheduler thread when run job.
And some fixes when doing this improment.
Chunming Zhou (11):
drm/amdgpu: hw ring should be empty when gpu reset drm/amdgpu: specify entity toamdgpu_copy_bufferdrm/amd: add recover run queue for scheduler drm/amdgpu: fix vm init error path drm/amdgpu: add vm recover entity drm/amdgpu: use all pte rings to recover page table drm/amd: add recover entity for every scheduler drm/amd: use scheduler to recover hw jobs drm/amd: hw job list should be exact drm/amd: reset jobs to recover entity drm/amdgpu: no need fence wait every time
Quoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 35 +++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 11 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_test.c | 8 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 27 ++++-- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 129 +++++++++++++------------- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 4 +- 9 files changed, 135 insertions(+), 92 deletions(-)
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 88bda920-4da2-7115-a82c-c0b1f581603e-dczkZgxz+BNUPWh3PAxdjQ@public.gmane.org
Quoted 4 lines · Edward O'Callaghan
Hi,
I didn't see any of my previous review remarks on your previous submission addresses in this submission of this series?
Hi Edward,
Sorry for that. Since these patch sets are big feature change and we need to pin down the approach of the solution first. As you see, Christian has argued this solution and suggested another direction. When we change the direction, the patches could be different totally. So I didn't address patch itself flaw. But indeed many thanks for your review.
Thanks,
David Zhou
Quoted 39 lines
Kind Regards,
Edward.Quoted 21 lines · Chunming Zhou
every vm has itself recovery entity, which is used to reovery page table from their shadow. They don't need to wait front vm completed.
And also using all pte rings can speed reovery.every scheduler has its own recovery entity, which is used to save hw jobs, and resubmit from it, which solves the conflicts between reset thread and scheduler thread when run job.
And some fixes when doing this improment.
Chunming Zhou (11):
drm/amdgpu: hw ring should be empty when gpu reset drm/amdgpu: specify entity toamdgpu_copy_bufferdrm/amd: add recover run queue for scheduler drm/amdgpu: fix vm init error path drm/amdgpu: add vm recover entity drm/amdgpu: use all pte rings to recover page table drm/amd: add recover entity for every scheduler drm/amd: use scheduler to recover hw jobs drm/amd: hw job list should be exact drm/amd: reset jobs to recover entity drm/amdgpu: no need fence wait every timeQuoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 35 +++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 11 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_test.c | 8 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 27 ++++-- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 129 +++++++++++++------------- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 4 +- 9 files changed, 135 insertions(+), 92 deletions(-)
Quoted 4 lines
_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): [email protected]
Quoted 6 lines · On 08/04/2016 12:44 PM, zhoucm1 wrote:
Quoted 4 lines · Edward O'Callaghan
Hi,
I didn't see any of my previous review remarks on your previous submission addresses in this submission of this series?
Quoted 7 lines
Hi Edward,
Sorry for that. Since these patch sets are big feature change and we need to pin down the approach of the solution first. As you see, Christian has argued this solution and suggested another direction. When we change the direction, the patches could be different totally. So I didn't address patch itself flaw. But indeed many thanks for your review.
No problem, welcome to the community and thanks for your hard work! :)
Quoted 48 lines
Thanks,
David ZhouQuoted 43 lines
Kind Regards,
Edward.Quoted 21 lines · Chunming Zhou
every vm has itself recovery entity, which is used to reovery page table from their shadow. They don't need to wait front vm completed.
And also using all pte rings can speed reovery.every scheduler has its own recovery entity, which is used to save hw jobs, and resubmit from it, which solves the conflicts between reset thread and scheduler thread when run job.
And some fixes when doing this improment.
Chunming Zhou (11):
drm/amdgpu: hw ring should be empty when gpu reset drm/amdgpu: specify entity toamdgpu_copy_bufferdrm/amd: add recover run queue for scheduler drm/amdgpu: fix vm init error path drm/amdgpu: add vm recover entity drm/amdgpu: use all pte rings to recover page table drm/amd: add recover entity for every scheduler drm/amd: use scheduler to recover hw jobs drm/amd: hw job list should be exact drm/amd: reset jobs to recover entity drm/amdgpu: no need fence wait every timeQuoted 11 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c | 3 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 35 +++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 11 +++ drivers/gpu/drm/amd/amdgpu/amdgpu_test.c | 8 +- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 5 +- drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 27 ++++-- drivers/gpu/drm/amd/scheduler/gpu_scheduler.c | 129 +++++++++++++------------- drivers/gpu/drm/amd/scheduler/gpu_scheduler.h | 4 +- 9 files changed, 135 insertions(+), 92 deletions(-)_______________________________________________
amd-gfx mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx