Skip to contentSkip to conversation

drm/amdgpu: add sched sync for amdgpu job

PATCH NAK Chunming Zhou 3 participants 18 messages May 10, 2017 10 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/4 drm/amdgpu: add sched sync for amdgpu job

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
 3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
 	struct amdgpu_vm	*vm;
 	struct amdgpu_ring	*ring;
 	struct amdgpu_sync	sync;
+	struct amdgpu_sync	sched_sync;
 	struct amdgpu_ib	*ibs;
 	struct fence		*fence; /* the hw fence */
 	uint32_t		preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +6 -1
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
 {
 	struct amdgpu_device *adev = ring->adev;
 	struct amdgpu_ib *ib = &ibs[0];
+	struct fence *tmp;
 	bool skip_preamble, need_ctx_switch;
 	unsigned patch_offset = ~0;
 	struct amdgpu_vm *vm;
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
 		return r;
 	}
 
-	if (ring->funcs->emit_pipeline_sync && job && job->need_pipeline_sync)
+	if (ring->funcs->emit_pipeline_sync && job &&
+	    (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+		job->need_pipeline_sync = true;
 		amdgpu_ring_emit_pipeline_sync(ring);
+		fence_put(tmp);
+	}
 	if (vm) {
 		amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE go too fast than DE */
 
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +4 -1
index cfa97ab..fa0c8b1 100644
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
 	(*job)->need_pipeline_sync = false;
 
 	amdgpu_sync_create(&(*job)->sync);
+	amdgpu_sync_create(&(*job)->sched_sync);
 
 	return 0;
 }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct amd_sched_job *s_job)
 
 	fence_put(job->fence);
 	amdgpu_sync_free(&job->sync);
+	amdgpu_sync_free(&job->sched_sync);
 	kfree(job);
 }
 
@@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
 
 	fence_put(job->fence);
 	amdgpu_sync_free(&job->sync);
+	amdgpu_sync_free(&job->sched_sync);
 	kfree(job);
 }
 
@@ -154,7 +157,7 @@ static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job)
 	}
 
 	if (amd_sched_dependency_optimized(fence, sched_job->s_entity))
-		job->need_pipeline_sync = true;
+		amdgpu_sync_fence(job->adev, &job->sched_sync, fence);
 
 	return fence;
 }
Signature · 6 lines
1.9.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1494401509-7198-1-git-send-email-David1.Zhou-5C7GfCeVMHo@public.gmane.org

#2 lore raw

PATCH 2/4 drm/amdgpu: make pipeline sync be in same place

Change-Id I0ccfa0e6de0cddbcca8dd85f2862240bc5ca02b3
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h     |  1 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  |  6 +++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_job.c |  1 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c  | 34 ++++++++++++++++++++++++++++++---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h  |  2 ++
 5 files changed, 36 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0 -1
index ef018bf..a583aab 100644
@@ -1170,7 +1170,6 @@ struct amdgpu_job {
 	void			*owner;
 	uint64_t		fence_ctx; /* the fence_context this job uses */
 	bool                    vm_needs_flush;
-	bool			need_pipeline_sync;
 	unsigned		vm_id;
 	uint64_t		vm_pd_addr;
 	uint32_t		gds_base, gds_size;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +3 -3
index 86ad507..dc250d6 100644
@@ -121,7 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
 {
 	struct amdgpu_device *adev = ring->adev;
 	struct amdgpu_ib *ib = &ibs[0];
-	struct fence *tmp;
+	struct fence *tmp = NULL;
 	bool skip_preamble, need_ctx_switch;
 	unsigned patch_offset = ~0;
 	struct amdgpu_vm *vm;
@@ -169,8 +169,8 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
 	}
 
 	if (ring->funcs->emit_pipeline_sync && job &&
-	    (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
-		job->need_pipeline_sync = true;
+	    ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
+	     amdgpu_vm_need_pipeline_sync(ring, job))) {
 		amdgpu_ring_emit_pipeline_sync(ring);
 		fence_put(tmp);
 	}
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +0 -1
index fa0c8b1..adb7901 100644
@@ -57,7 +57,6 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
 	(*job)->vm = vm;
 	(*job)->ibs = (void *)&(*job)[1];
 	(*job)->num_ibs = num_ibs;
-	(*job)->need_pipeline_sync = false;
 
 	amdgpu_sync_create(&(*job)->sync);
 	amdgpu_sync_create(&(*job)->sched_sync);
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +31 -3
index b4f83fc..58cde30 100644
@@ -700,6 +700,37 @@ static u64 amdgpu_vm_adjust_mc_addr(struct amdgpu_device *adev, u64 mc_addr)
 	return addr;
 }
 
+bool amdgpu_vm_need_pipeline_sync(struct amdgpu_ring *ring,
+				  struct amdgpu_job *job)
+{
+	struct amdgpu_device *adev = ring->adev;
+	unsigned vmhub = ring->funcs->vmhub;
+	struct amdgpu_vm_id_manager *id_mgr = &adev->vm_manager.id_mgr[vmhub];
+	struct amdgpu_vm_id *id;
+	bool gds_switch_needed;
+	bool vm_flush_needed = job->vm_needs_flush ||
+		amdgpu_vm_ring_has_compute_vm_bug(ring);
+
+	if (job->vm_id == 0)
+		return false;
+	id = &id_mgr->ids[job->vm_id];
+	gds_switch_needed = ring->funcs->emit_gds_switch && (
+		id->gds_base != job->gds_base ||
+		id->gds_size != job->gds_size ||
+		id->gws_base != job->gws_base ||
+		id->gws_size != job->gws_size ||
+		id->oa_base != job->oa_base ||
+		id->oa_size != job->oa_size);
+
+	if (amdgpu_vm_had_gpu_reset(adev, id)) {
+		gds_switch_needed = true;
+		vm_flush_needed = true;
+	}
+	if (!vm_flush_needed && !gds_switch_needed)
+		return false;
+	return true;
+}
+
 /**
  * amdgpu_vm_flush - hardware flush the vm
  *
@@ -738,9 +769,6 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job)
 	if (ring->funcs->init_cond_exec)
 		patch_offset = amdgpu_ring_init_cond_exec(ring);
 
-	if (ring->funcs->emit_pipeline_sync && !job->need_pipeline_sync)
-		amdgpu_ring_emit_pipeline_sync(ring);
-
 	if (ring->funcs->emit_vm_flush && vm_flush_needed) {
 		u64 pd_addr = amdgpu_vm_adjust_mc_addr(adev, job->vm_pd_addr);
 		struct fence *fence;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -0
index 9828fcd..3d16169 100644
@@ -254,5 +254,7 @@ void amdgpu_vm_bo_rmv(struct amdgpu_device *adev,
 		      struct amdgpu_bo_va *bo_va);
 void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint64_t vm_size);
 int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp);
+bool amdgpu_vm_need_pipeline_sync(struct amdgpu_ring *ring,
+				  struct amdgpu_job *job);
 
 #endif
Signature · 6 lines
1.9.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1494401509-7198-2-git-send-email-David1.Zhou-5C7GfCeVMHo@public.gmane.org

Reviewed-by Junwei Zhang
#4 lore raw

Re: [PATCH 2/4] drm/amdgpu: make pipeline sync be in same place

Quoted 101 lines · Chunming Zhou
Change-Id I0ccfa0e6de0cddbcca8dd85f2862240bc5ca02b3
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     |  1 -
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  |  6 +++---
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c |  1 -
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c  | 34 ++++++++++++++++++++++++++++++---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h  |  2 ++
  5 files changed, 36 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0 -1
index ef018bf..a583aab 100644
@@ -1170,7 +1170,6 @@ struct amdgpu_job {
  	void			*owner;
  	uint64_t		fence_ctx; /* the fence_context this job uses */
  	bool                    vm_needs_flush;
-	bool			need_pipeline_sync;
  	unsigned		vm_id;
  	uint64_t		vm_pd_addr;
  	uint32_t		gds_base, gds_size;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +3 -3
index 86ad507..dc250d6 100644
@@ -121,7 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  {
  	struct amdgpu_device *adev = ring->adev;
  	struct amdgpu_ib *ib = &ibs[0];
-	struct fence *tmp;
+	struct fence *tmp = NULL;
  	bool skip_preamble, need_ctx_switch;
  	unsigned patch_offset = ~0;
  	struct amdgpu_vm *vm;
@@ -169,8 +169,8 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  	}
 
  	if (ring->funcs->emit_pipeline_sync && job &&
-	    (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
-		job->need_pipeline_sync = true;
+	    ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
+	     amdgpu_vm_need_pipeline_sync(ring, job))) {
  		amdgpu_ring_emit_pipeline_sync(ring);
  		fence_put(tmp);
  	}
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +0 -1
index fa0c8b1..adb7901 100644
@@ -57,7 +57,6 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
  	(*job)->vm = vm;
  	(*job)->ibs = (void *)&(*job)[1];
  	(*job)->num_ibs = num_ibs;
-	(*job)->need_pipeline_sync = false;
 
  	amdgpu_sync_create(&(*job)->sync);
  	amdgpu_sync_create(&(*job)->sched_sync);
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +31 -0
index b4f83fc..58cde30 100644
@@ -700,6 +700,37 @@ static u64 amdgpu_vm_adjust_mc_addr(struct amdgpu_device *adev, u64 mc_addr)
  	return addr;
  }
 
+bool amdgpu_vm_need_pipeline_sync(struct amdgpu_ring *ring,
+				  struct amdgpu_job *job)
+{
+	struct amdgpu_device *adev = ring->adev;
+	unsigned vmhub = ring->funcs->vmhub;
+	struct amdgpu_vm_id_manager *id_mgr = &adev->vm_manager.id_mgr[vmhub];
+	struct amdgpu_vm_id *id;
+	bool gds_switch_needed;
+	bool vm_flush_needed = job->vm_needs_flush ||
+		amdgpu_vm_ring_has_compute_vm_bug(ring);
+
+	if (job->vm_id == 0)
+		return false;
+	id = &id_mgr->ids[job->vm_id];
+	gds_switch_needed = ring->funcs->emit_gds_switch && (
+		id->gds_base != job->gds_base ||
+		id->gds_size != job->gds_size ||
+		id->gws_base != job->gws_base ||
+		id->gws_size != job->gws_size ||
+		id->oa_base != job->oa_base ||
+		id->oa_size != job->oa_size);
+
+	if (amdgpu_vm_had_gpu_reset(adev, id)) {
+		gds_switch_needed = true;
+		vm_flush_needed = true;
+	}
+	if (!vm_flush_needed && !gds_switch_needed)
+		return false;
+	return true;
+}
+
  /**
   * amdgpu_vm_flush - hardware flush the vm
   *

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job) · line 743

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -3
@@ -738,9 +769,6 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job)
  	if (ring->funcs->init_cond_exec)
  		patch_offset = amdgpu_ring_init_cond_exec(ring);
 
-	if (ring->funcs->emit_pipeline_sync && !job->need_pipeline_sync)
-		amdgpu_ring_emit_pipeline_sync(ring);
-

Just confirm:
this pipeline sync will be done when vm_flush job is scheduled?

If yes,

Jerry

Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
  	if (ring->funcs->emit_vm_flush && vm_flush_needed) {
  		u64 pd_addr = amdgpu_vm_adjust_mc_addr(adev, job->vm_pd_addr);
  		struct fence *fence;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
index 9828fcd..3d16169 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, · line 260

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -0
@@ -254,5 +254,7 @@ void amdgpu_vm_bo_rmv(struct amdgpu_device *adev,
  		      struct amdgpu_bo_va *bo_va);
  void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint64_t vm_size);
  int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp);
+bool amdgpu_vm_need_pipeline_sync(struct amdgpu_ring *ring,
+				  struct amdgpu_job *job);
 
  #endif

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#5 lore raw

Re: [PATCH 2/4] drm/amdgpu: make pipeline sync be in same place

In reply to a message not in the archive

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 81 lines
Change-Id I0ccfa0e6de0cddbcca8dd85f2862240bc5ca02b3
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     |  1 -
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  |  6 +++---
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c |  1 -
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c  | 34 ++++++++++++++++++++++++++++++---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h  |  2 ++
  5 files changed, 36 insertions(+), 8 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +0 -1
index ef018bf..a583aab 100644
@@ -1170,7 +1170,6 @@ struct amdgpu_job {
  	void			*owner;
  	uint64_t		fence_ctx; /* the fence_context this job uses */
  	bool                    vm_needs_flush;
-	bool			need_pipeline_sync;
  	unsigned		vm_id;
  	uint64_t		vm_pd_addr;
  	uint32_t		gds_base, gds_size;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +3 -3
index 86ad507..dc250d6 100644
@@ -121,7 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  {
  	struct amdgpu_device *adev = ring->adev;
  	struct amdgpu_ib *ib = &ibs[0];
-	struct fence *tmp;
+	struct fence *tmp = NULL;
  	bool skip_preamble, need_ctx_switch;
  	unsigned patch_offset = ~0;
  	struct amdgpu_vm *vm;
@@ -169,8 +169,8 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  	}
 
  	if (ring->funcs->emit_pipeline_sync && job &&
-	    (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
-		job->need_pipeline_sync = true;
+	    ((tmp = amdgpu_sync_get_fence(&job->sched_sync)) ||
+	     amdgpu_vm_need_pipeline_sync(ring, job))) {
  		amdgpu_ring_emit_pipeline_sync(ring);
  		fence_put(tmp);
  	}
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +0 -1
index fa0c8b1..adb7901 100644
@@ -57,7 +57,6 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
  	(*job)->vm = vm;
  	(*job)->ibs = (void *)&(*job)[1];
  	(*job)->num_ibs = num_ibs;
-	(*job)->need_pipeline_sync = false;
 
  	amdgpu_sync_create(&(*job)->sync);
  	amdgpu_sync_create(&(*job)->sched_sync);
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +15 -0
index b4f83fc..58cde30 100644
@@ -700,6 +700,37 @@ static u64 amdgpu_vm_adjust_mc_addr(struct amdgpu_device *adev, u64 mc_addr)
  	return addr;
  }
 
+bool amdgpu_vm_need_pipeline_sync(struct amdgpu_ring *ring,
+				  struct amdgpu_job *job)
+{
+	struct amdgpu_device *adev = ring->adev;
+	unsigned vmhub = ring->funcs->vmhub;
+	struct amdgpu_vm_id_manager *id_mgr = &adev->vm_manager.id_mgr[vmhub];
+	struct amdgpu_vm_id *id;
+	bool gds_switch_needed;
+	bool vm_flush_needed = job->vm_needs_flush ||
+		amdgpu_vm_ring_has_compute_vm_bug(ring);
+
+	if (job->vm_id == 0)
+		return false;
+	id = &id_mgr->ids[job->vm_id];
+	gds_switch_needed = ring->funcs->emit_gds_switch && (

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c static u64 amdgpu_vm_adjust_mc_addr(struct amdgpu_device *adev, u64 mc_addr) · line 727

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +10 -0
+		id->gds_base != job->gds_base ||
+		id->gds_size != job->gds_size ||
+		id->gws_base != job->gws_base ||
+		id->gws_size != job->gws_size ||
+		id->oa_base != job->oa_base ||
+		id->oa_size != job->oa_size);
+
+	if (amdgpu_vm_had_gpu_reset(adev, id)) {
+		gds_switch_needed = true;
+		vm_flush_needed = true;

You can shortcut here, e.g. just return true. Same above if you use an "if".

Apart from that looks good to me.

Christian.

Quoted 24 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +6 -3
+	}
+	if (!vm_flush_needed && !gds_switch_needed)
+		return false;
+	return true;
+}
+
  /**
   * amdgpu_vm_flush - hardware flush the vm
   *
@@ -738,9 +769,6 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job)
  	if (ring->funcs->init_cond_exec)
  		patch_offset = amdgpu_ring_init_cond_exec(ring);
 
-	if (ring->funcs->emit_pipeline_sync && !job->need_pipeline_sync)
-		amdgpu_ring_emit_pipeline_sync(ring);
-
  	if (ring->funcs->emit_vm_flush && vm_flush_needed) {
  		u64 pd_addr = amdgpu_vm_adjust_mc_addr(adev, job->vm_pd_addr);
  		struct fence *fence;
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
index 9828fcd..3d16169 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h void amdgpu_vm_bo_rmv(struct amdgpu_device *adev, · line 260

Quoted 7 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -0
@@ -254,5 +254,7 @@ void amdgpu_vm_bo_rmv(struct amdgpu_device *adev,
  		      struct amdgpu_bo_va *bo_va);
  void amdgpu_vm_adjust_size(struct amdgpu_device *adev, uint64_t vm_size);
  int amdgpu_vm_ioctl(struct drm_device *dev, void *data, struct drm_file *filp);
+bool amdgpu_vm_need_pipeline_sync(struct amdgpu_ring *ring,
+				  struct amdgpu_job *job);
 
  #endif

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#6 lore raw

PATCH 3/4 drm/amdgpu: id reset count only is updated when used end

In reply to a message not in the archive

before that, we have function to check if reset happens by using reset count.

Change-Id I2e941dd35295d4210d57a9593d39b5ee9021be9f
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -2
index 58cde30..bbb3587 100644
@@ -450,7 +450,6 @@ static int amdgpu_vm_grab_reserved_vmid_locked(struct amdgpu_vm *vm,
 		id->flushed_updates = fence_get(updates);
 	}
 	id->pd_gpu_addr = job->vm_pd_addr;
-	id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
 	atomic64_set(&id->owner, vm->client_id);
 	job->vm_needs_flush = needs_flush;
 	if (needs_flush) {
@@ -598,7 +597,6 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
 	id->pd_gpu_addr = job->vm_pd_addr;
 	fence_put(id->flushed_updates);
 	id->flushed_updates = fence_get(updates);
-	id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
 	atomic64_set(&id->owner, vm->client_id);
 
 needs_flush:
@@ -783,6 +781,8 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job)
 		mutex_lock(&id_mgr->lock);
 		fence_put(id->last_flush);
 		id->last_flush = fence;
+		if (amdgpu_vm_had_gpu_reset(adev, id))
+			id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
 		mutex_unlock(&id_mgr->lock);
 	}
 
Signature · 6 lines
1.9.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1494401509-7198-3-git-send-email-David1.Zhou-5C7GfCeVMHo@public.gmane.org

#8 lore raw

Re: [PATCH 3/4] drm/amdgpu: id reset count only is updated when used end

Could you elaborate the reason? bug fix, or sth else?

Jerry

Quoted 30 lines · Chunming Zhou

before that, we have function to check if reset happens by using reset count.

Change-Id I2e941dd35295d4210d57a9593d39b5ee9021be9f
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -2
index 58cde30..bbb3587 100644
@@ -450,7 +450,6 @@ static int amdgpu_vm_grab_reserved_vmid_locked(struct amdgpu_vm *vm,
  		id->flushed_updates = fence_get(updates);
  	}
  	id->pd_gpu_addr = job->vm_pd_addr;
-	id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
  	atomic64_set(&id->owner, vm->client_id);
  	job->vm_needs_flush = needs_flush;
  	if (needs_flush) {
@@ -598,7 +597,6 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
  	id->pd_gpu_addr = job->vm_pd_addr;
  	fence_put(id->flushed_updates);
  	id->flushed_updates = fence_get(updates);
-	id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
  	atomic64_set(&id->owner, vm->client_id);
 
  needs_flush:

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job) · line 788

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -0
@@ -783,6 +781,8 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job)
  		mutex_lock(&id_mgr->lock);
  		fence_put(id->last_flush);
  		id->last_flush = fence;
+		if (amdgpu_vm_had_gpu_reset(adev, id))
+			id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
  		mutex_unlock(&id_mgr->lock);
  	}
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#9 lore raw

Re: [PATCH 3/4] drm/amdgpu: id reset count only is updated when used end

In reply to a message not in the archive

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 29 lines

before that, we have function to check if reset happens by using reset count.

Change-Id I2e941dd35295d4210d57a9593d39b5ee9021be9f
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 4 ++--
  1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -2
index 58cde30..bbb3587 100644
@@ -450,7 +450,6 @@ static int amdgpu_vm_grab_reserved_vmid_locked(struct amdgpu_vm *vm,
  		id->flushed_updates = fence_get(updates);
  	}
  	id->pd_gpu_addr = job->vm_pd_addr;
-	id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
  	atomic64_set(&id->owner, vm->client_id);
  	job->vm_needs_flush = needs_flush;
  	if (needs_flush) {
@@ -598,7 +597,6 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
  	id->pd_gpu_addr = job->vm_pd_addr;
  	fence_put(id->flushed_updates);
  	id->flushed_updates = fence_get(updates);
-	id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);
  	atomic64_set(&id->owner, vm->client_id);
 
  needs_flush:

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job) · line 785

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -0
@@ -783,6 +781,8 @@ int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job)
  		mutex_lock(&id_mgr->lock);
  		fence_put(id->last_flush);
  		id->last_flush = fence;
+		if (amdgpu_vm_had_gpu_reset(adev, id))
+			id->current_gpu_reset_count = atomic_read(&adev->gpu_reset_counter);

Just drop the "if", checking the memory location first to avoid the write is rather pointless with an atomic.

With that fixed the patch is Reviewed-by: Christian König <[email protected]>.

Regards,
Christian.

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job) · line 788

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +0 -0
  		mutex_unlock(&id_mgr->lock);
  	}
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#10 lore raw

PATCH 4/4 drm/amdgpu: check whether the vmid can be reused first

In reply to a message not in the archive
Change-Id If24a62b9c3097c9b040225ab0e768145b7a3db1e
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 92 +++++++++++++++++-----------------
 1 file changed, 47 insertions(+), 45 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +47 -45
index bbb3587..6259608 100644
@@ -492,51 +492,6 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
 		mutex_unlock(&id_mgr->lock);
 		return r;
 	}
-	fences = kmalloc_array(sizeof(void *), id_mgr->num_ids, GFP_KERNEL);
-	if (!fences) {
-		mutex_unlock(&id_mgr->lock);
-		return -ENOMEM;
-	}
-	/* Check if we have an idle VMID */
-	i = 0;
-	list_for_each_entry(idle, &id_mgr->ids_lru, list) {
-		fences[i] = amdgpu_sync_peek_fence(&idle->active, ring);
-		if (!fences[i])
-			break;
-		++i;
-	}
-
-	/* If we can't find a idle VMID to use, wait till one becomes available */
-	if (&idle->list == &id_mgr->ids_lru) {
-		u64 fence_context = adev->vm_manager.fence_context + ring->idx;
-		unsigned seqno = ++adev->vm_manager.seqno[ring->idx];
-		struct fence_array *array;
-		unsigned j;
-
-		for (j = 0; j < i; ++j)
-			fence_get(fences[j]);
-
-		array = fence_array_create(i, fences, fence_context,
-					   seqno, true);
-		if (!array) {
-			for (j = 0; j < i; ++j)
-				fence_put(fences[j]);
-			kfree(fences);
-			r = -ENOMEM;
-			goto error;
-		}
-
-
-		r = amdgpu_sync_fence(ring->adev, sync, &array->base);
-		fence_put(&array->base);
-		if (r)
-			goto error;
-
-		mutex_unlock(&id_mgr->lock);
-		return 0;
-
-	}
-	kfree(fences);
 
 	job->vm_needs_flush = false;
 	/* Check if we can use a VMID already assigned to this VM */
@@ -586,6 +541,53 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
 
 	};
 
+	fences = kmalloc_array(sizeof(void *), id_mgr->num_ids, GFP_KERNEL);
+	if (!fences) {
+		mutex_unlock(&id_mgr->lock);
+		return -ENOMEM;
+	}
+	/* Check if we have an idle VMID */
+	i = 0;
+	list_for_each_entry(idle, &id_mgr->ids_lru, list) {
+		fences[i] = amdgpu_sync_peek_fence(&idle->active, ring);
+		if (!fences[i])
+			break;
+		++i;
+	}
+
+	/* If we can't find a idle VMID to use, wait till one becomes available */
+	if (&idle->list == &id_mgr->ids_lru) {
+		u64 fence_context = adev->vm_manager.fence_context + ring->idx;
+		unsigned seqno = ++adev->vm_manager.seqno[ring->idx];
+		struct fence_array *array;
+		unsigned j;
+
+		for (j = 0; j < i; ++j)
+			fence_get(fences[j]);
+
+		array = fence_array_create(i, fences, fence_context,
+					   seqno, true);
+		if (!array) {
+			for (j = 0; j < i; ++j)
+				fence_put(fences[j]);
+			kfree(fences);
+			r = -ENOMEM;
+			goto error;
+		}
+
+
+		r = amdgpu_sync_fence(ring->adev, sync, &array->base);
+		fence_put(&array->base);
+		if (r)
+			goto error;
+
+		mutex_unlock(&id_mgr->lock);
+		return 0;
+
+	}
+	kfree(fences);
+
+
 	/* Still no ID to use? Then use the idle one found earlier */
 	id = idle;
 
Signature · 6 lines
1.9.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1494401509-7198-4-git-send-email-David1.Zhou-5C7GfCeVMHo@public.gmane.org

Reviewed-by Junwei Zhang
#12 lore raw

Re: [PATCH 4/4] drm/amdgpu: check whether the vmid can be reused first

Quoted 2 lines · Chunming Zhou
Change-Id If24a62b9c3097c9b040225ab0e768145b7a3db1e

I had same idea when read this code before.

Quoted 105 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 92 +++++++++++++++++-----------------
  1 file changed, 47 insertions(+), 45 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +40 -45
index bbb3587..6259608 100644
@@ -492,51 +492,6 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
  		mutex_unlock(&id_mgr->lock);
  		return r;
  	}
-	fences = kmalloc_array(sizeof(void *), id_mgr->num_ids, GFP_KERNEL);
-	if (!fences) {
-		mutex_unlock(&id_mgr->lock);
-		return -ENOMEM;
-	}
-	/* Check if we have an idle VMID */
-	i = 0;
-	list_for_each_entry(idle, &id_mgr->ids_lru, list) {
-		fences[i] = amdgpu_sync_peek_fence(&idle->active, ring);
-		if (!fences[i])
-			break;
-		++i;
-	}
-
-	/* If we can't find a idle VMID to use, wait till one becomes available */
-	if (&idle->list == &id_mgr->ids_lru) {
-		u64 fence_context = adev->vm_manager.fence_context + ring->idx;
-		unsigned seqno = ++adev->vm_manager.seqno[ring->idx];
-		struct fence_array *array;
-		unsigned j;
-
-		for (j = 0; j < i; ++j)
-			fence_get(fences[j]);
-
-		array = fence_array_create(i, fences, fence_context,
-					   seqno, true);
-		if (!array) {
-			for (j = 0; j < i; ++j)
-				fence_put(fences[j]);
-			kfree(fences);
-			r = -ENOMEM;
-			goto error;
-		}
-
-
-		r = amdgpu_sync_fence(ring->adev, sync, &array->base);
-		fence_put(&array->base);
-		if (r)
-			goto error;
-
-		mutex_unlock(&id_mgr->lock);
-		return 0;
-
-	}
-	kfree(fences);
 
  	job->vm_needs_flush = false;
  	/* Check if we can use a VMID already assigned to this VM */
@@ -586,6 +541,53 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
 
  	};
 
+	fences = kmalloc_array(sizeof(void *), id_mgr->num_ids, GFP_KERNEL);
+	if (!fences) {
+		mutex_unlock(&id_mgr->lock);
+		return -ENOMEM;
+	}
+	/* Check if we have an idle VMID */
+	i = 0;
+	list_for_each_entry(idle, &id_mgr->ids_lru, list) {
+		fences[i] = amdgpu_sync_peek_fence(&idle->active, ring);
+		if (!fences[i])
+			break;
+		++i;
+	}
+
+	/* If we can't find a idle VMID to use, wait till one becomes available */
+	if (&idle->list == &id_mgr->ids_lru) {
+		u64 fence_context = adev->vm_manager.fence_context + ring->idx;
+		unsigned seqno = ++adev->vm_manager.seqno[ring->idx];
+		struct fence_array *array;
+		unsigned j;
+
+		for (j = 0; j < i; ++j)
+			fence_get(fences[j]);
+
+		array = fence_array_create(i, fences, fence_context,
+					   seqno, true);
+		if (!array) {
+			for (j = 0; j < i; ++j)
+				fence_put(fences[j]);
+			kfree(fences);
+			r = -ENOMEM;
+			goto error;
+		}
+
+
+		r = amdgpu_sync_fence(ring->adev, sync, &array->base);
+		fence_put(&array->base);
+		if (r)
+			goto error;
+

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring, · line 593

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +7 -0
+		mutex_unlock(&id_mgr->lock);
+		return 0;
+
+	}
+	kfree(fences);
+
+
  	/* Still no ID to use? Then use the idle one found earlier */
  	id = idle;
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#13 lore raw

Re: [PATCH 4/4] drm/amdgpu: check whether the vmid can be reused first

In reply to a message not in the archive

NAK, it is intentionally done like this to avoid starvation of processes.

In other words we assign a VMID only when there is at least one free.

Regards,
Christian.

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 106 lines
Change-Id If24a62b9c3097c9b040225ab0e768145b7a3db1e
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 92 +++++++++++++++++-----------------
  1 file changed, 47 insertions(+), 45 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +40 -45
index bbb3587..6259608 100644
@@ -492,51 +492,6 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
  		mutex_unlock(&id_mgr->lock);
  		return r;
  	}
-	fences = kmalloc_array(sizeof(void *), id_mgr->num_ids, GFP_KERNEL);
-	if (!fences) {
-		mutex_unlock(&id_mgr->lock);
-		return -ENOMEM;
-	}
-	/* Check if we have an idle VMID */
-	i = 0;
-	list_for_each_entry(idle, &id_mgr->ids_lru, list) {
-		fences[i] = amdgpu_sync_peek_fence(&idle->active, ring);
-		if (!fences[i])
-			break;
-		++i;
-	}
-
-	/* If we can't find a idle VMID to use, wait till one becomes available */
-	if (&idle->list == &id_mgr->ids_lru) {
-		u64 fence_context = adev->vm_manager.fence_context + ring->idx;
-		unsigned seqno = ++adev->vm_manager.seqno[ring->idx];
-		struct fence_array *array;
-		unsigned j;
-
-		for (j = 0; j < i; ++j)
-			fence_get(fences[j]);
-
-		array = fence_array_create(i, fences, fence_context,
-					   seqno, true);
-		if (!array) {
-			for (j = 0; j < i; ++j)
-				fence_put(fences[j]);
-			kfree(fences);
-			r = -ENOMEM;
-			goto error;
-		}
-
-
-		r = amdgpu_sync_fence(ring->adev, sync, &array->base);
-		fence_put(&array->base);
-		if (r)
-			goto error;
-
-		mutex_unlock(&id_mgr->lock);
-		return 0;
-
-	}
-	kfree(fences);
 
  	job->vm_needs_flush = false;
  	/* Check if we can use a VMID already assigned to this VM */
@@ -586,6 +541,53 @@ int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring,
 
  	};
 
+	fences = kmalloc_array(sizeof(void *), id_mgr->num_ids, GFP_KERNEL);
+	if (!fences) {
+		mutex_unlock(&id_mgr->lock);
+		return -ENOMEM;
+	}
+	/* Check if we have an idle VMID */
+	i = 0;
+	list_for_each_entry(idle, &id_mgr->ids_lru, list) {
+		fences[i] = amdgpu_sync_peek_fence(&idle->active, ring);
+		if (!fences[i])
+			break;
+		++i;
+	}
+
+	/* If we can't find a idle VMID to use, wait till one becomes available */
+	if (&idle->list == &id_mgr->ids_lru) {
+		u64 fence_context = adev->vm_manager.fence_context + ring->idx;
+		unsigned seqno = ++adev->vm_manager.seqno[ring->idx];
+		struct fence_array *array;
+		unsigned j;
+
+		for (j = 0; j < i; ++j)
+			fence_get(fences[j]);
+
+		array = fence_array_create(i, fences, fence_context,
+					   seqno, true);
+		if (!array) {
+			for (j = 0; j < i; ++j)
+				fence_put(fences[j]);
+			kfree(fences);
+			r = -ENOMEM;
+			goto error;
+		}
+
+
+		r = amdgpu_sync_fence(ring->adev, sync, &array->base);
+		fence_put(&array->base);
+		if (r)
+			goto error;
+

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_grab_id(struct amdgpu_vm *vm, struct amdgpu_ring *ring, · line 593

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +7 -0
+		mutex_unlock(&id_mgr->lock);
+		return 0;
+
+	}
+	kfree(fences);
+
+
  	/* Still no ID to use? Then use the idle one found earlier */
  	id = idle;
 

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by Junwei Zhang
#14 lore raw
In reply to a message not in the archive
Quoted 7 lines · Chunming Zhou

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
Quoted 74 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
  	struct amdgpu_vm	*vm;
  	struct amdgpu_ring	*ring;
  	struct amdgpu_sync	sync;
+	struct amdgpu_sync	sched_sync;
  	struct amdgpu_ib	*ibs;
  	struct fence		*fence; /* the hw fence */
  	uint32_t		preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +6 -1
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  {
  	struct amdgpu_device *adev = ring->adev;
  	struct amdgpu_ib *ib = &ibs[0];
+	struct fence *tmp;
  	bool skip_preamble, need_ctx_switch;
  	unsigned patch_offset = ~0;
  	struct amdgpu_vm *vm;
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  		return r;
  	}
 
-	if (ring->funcs->emit_pipeline_sync && job && job->need_pipeline_sync)
+	if (ring->funcs->emit_pipeline_sync && job &&
+	    (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+		job->need_pipeline_sync = true;
  		amdgpu_ring_emit_pipeline_sync(ring);
+		fence_put(tmp);
+	}
  	if (vm) {
  		amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE go too fast than DE */
 
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +3 -0
index cfa97ab..fa0c8b1 100644
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
  	(*job)->need_pipeline_sync = false;
 
  	amdgpu_sync_create(&(*job)->sync);
+	amdgpu_sync_create(&(*job)->sched_sync);
 
  	return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct amd_sched_job *s_job)
 
  	fence_put(job->fence);
  	amdgpu_sync_free(&job->sync);
+	amdgpu_sync_free(&job->sched_sync);
  	kfree(job);
  }
 
@@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
 
  	fence_put(job->fence);
  	amdgpu_sync_free(&job->sync);
+	amdgpu_sync_free(&job->sched_sync);
  	kfree(job);
  }
 

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job) · line 163

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
@@ -154,7 +157,7 @@ static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job)
  	}
 
  	if (amd_sched_dependency_optimized(fence, sched_job->s_entity))
-		job->need_pipeline_sync = true;
+		amdgpu_sync_fence(job->adev, &job->sched_sync, fence);
 
  	return fence;
  }

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#15 lore raw
In reply to a message not in the archive

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 80 lines

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
  	struct amdgpu_vm	*vm;
  	struct amdgpu_ring	*ring;
  	struct amdgpu_sync	sync;
+	struct amdgpu_sync	sched_sync;
  	struct amdgpu_ib	*ibs;
  	struct fence		*fence; /* the hw fence */
  	uint32_t		preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +6 -1
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  {
  	struct amdgpu_device *adev = ring->adev;
  	struct amdgpu_ib *ib = &ibs[0];
+	struct fence *tmp;
  	bool skip_preamble, need_ctx_switch;
  	unsigned patch_offset = ~0;
  	struct amdgpu_vm *vm;
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned num_ibs,
  		return r;
  	}
 
-	if (ring->funcs->emit_pipeline_sync && job && job->need_pipeline_sync)
+	if (ring->funcs->emit_pipeline_sync && job &&
+	    (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+		job->need_pipeline_sync = true;
  		amdgpu_ring_emit_pipeline_sync(ring);
+		fence_put(tmp);
+	}
  	if (vm) {
  		amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE go too fast than DE */
 
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +3 -0
index cfa97ab..fa0c8b1 100644
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, unsigned num_ibs,
  	(*job)->need_pipeline_sync = false;
 
  	amdgpu_sync_create(&(*job)->sync);
+	amdgpu_sync_create(&(*job)->sched_sync);
 
  	return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct amd_sched_job *s_job)
 
  	fence_put(job->fence);
  	amdgpu_sync_free(&job->sync);
+	amdgpu_sync_free(&job->sched_sync);
  	kfree(job);
  }
 
@@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
 
  	fence_put(job->fence);
  	amdgpu_sync_free(&job->sync);
+	amdgpu_sync_free(&job->sched_sync);
  	kfree(job);
  }
 

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job) · line 160

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
@@ -154,7 +157,7 @@ static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job)
  	}
 
  	if (amd_sched_dependency_optimized(fence, sched_job->s_entity))
-		job->need_pipeline_sync = true;
+		amdgpu_sync_fence(job->adev, &job->sched_sync, fence);

This can result in an -ENOMEM and additional to that we only need to remember the last fence optimized like this, not all of them.

So just keep the last one found here in job->sched_fence instead.

Regards,
Christian.

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job) · line 163

Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +0 -0
  	return fence;
  }

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 6c8f4299-d251-a0fe-c76f-b9124482fa0a-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

author
#17 lore raw
Quoted 79 lines · Christian König

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 78 lines

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
      struct amdgpu_vm    *vm;
      struct amdgpu_ring    *ring;
      struct amdgpu_sync    sync;
+    struct amdgpu_sync    sched_sync;
      struct amdgpu_ib    *ibs;
      struct fence        *fence; /* the hw fence */
      uint32_t        preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +1 -0
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, 
 unsigned num_ibs,
  {
      struct amdgpu_device *adev = ring->adev;
      struct amdgpu_ib *ib = &ibs[0];
+    struct fence *tmp;
      bool skip_preamble, need_ctx_switch;
      unsigned patch_offset = ~0;
      struct amdgpu_vm *vm;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +5 -0
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, 
 unsigned num_ibs,
          return r;
      }
  -    if (ring->funcs->emit_pipeline_sync && job && 
 job->need_pipeline_sync)
+    if (ring->funcs->emit_pipeline_sync && job &&
+        (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+        job->need_pipeline_sync = true;
          amdgpu_ring_emit_pipeline_sync(ring);
+        fence_put(tmp);
+    }
      if (vm) {
          amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE go 

too fast than DE */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index cfa97ab..fa0c8b1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +2 -0
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, 
 unsigned num_ibs,
      (*job)->need_pipeline_sync = false;
        amdgpu_sync_create(&(*job)->sync);
+    amdgpu_sync_create(&(*job)->sched_sync);
        return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct amd_sched_job 
 *s_job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);
  }
  @@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);

}

Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job) · line 160

Quoted 6 lines

@@ -154,7 +157,7 @@ static struct fence
*amdgpu_job_dependency(struct amd_sched_job *sched_job)

      }
        if (amd_sched_dependency_optimized(fence, sched_job->s_entity))
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
-        job->need_pipeline_sync = true;
+        amdgpu_sync_fence(job->adev, &job->sched_sync, fence);

This can result in an -ENOMEM

will handle it.

Quoted 3 lines

and additional to that we only need to remember the last fence optimized like this, not all of them.

So just keep the last one found here in job->sched_fence instead.

I guess this isn't enough.
The dependency is not in order when calling, so the last one is not always the last scheduled fence.
And they could be sched fence not hw fence, although they are handled by same hw ring, but the sched fence context isn't same. so we still need sched_sync here, right?

Regards,
David zhou

Quoted 8 lines

Regards,
Christian.

Quoted 2 lines
        return fence;

}

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#19 lore raw

Am 10.05.2017 um 10:38 schrieb zhoucm1:

Quoted 90 lines · Christian König
Quoted 79 lines

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 78 lines

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
      struct amdgpu_vm    *vm;
      struct amdgpu_ring    *ring;
      struct amdgpu_sync    sync;
+    struct amdgpu_sync    sched_sync;
      struct amdgpu_ib    *ibs;
      struct fence        *fence; /* the hw fence */
      uint32_t        preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +1 -0
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, 
 unsigned num_ibs,
  {
      struct amdgpu_device *adev = ring->adev;
      struct amdgpu_ib *ib = &ibs[0];
+    struct fence *tmp;
      bool skip_preamble, need_ctx_switch;
      unsigned patch_offset = ~0;
      struct amdgpu_vm *vm;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +5 -0
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
          return r;
      }
  -    if (ring->funcs->emit_pipeline_sync && job && 
 job->need_pipeline_sync)
+    if (ring->funcs->emit_pipeline_sync && job &&
+        (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+        job->need_pipeline_sync = true;
          amdgpu_ring_emit_pipeline_sync(ring);
+        fence_put(tmp);
+    }
      if (vm) {
          amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE go 

too fast than DE */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index cfa97ab..fa0c8b1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +2 -0
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, 
 unsigned num_ibs,
      (*job)->need_pipeline_sync = false;
        amdgpu_sync_create(&(*job)->sync);
+    amdgpu_sync_create(&(*job)->sched_sync);
        return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct 
 amd_sched_job *s_job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);
  }
  @@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);

}

Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c static struct fence *amdgpu_job_dependency(struct amd_sched_job *sched_job) · line 160

Quoted 6 lines

@@ -154,7 +157,7 @@ static struct fence
*amdgpu_job_dependency(struct amd_sched_job *sched_job)

      }
        if (amd_sched_dependency_optimized(fence, sched_job->s_entity))
drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
-        job->need_pipeline_sync = true;
+        amdgpu_sync_fence(job->adev, &job->sched_sync, fence);

This can result in an -ENOMEM

Quoted 8 lines

will handle it.

Quoted 3 lines

and additional to that we only need to remember the last fence optimized like this, not all of them.

So just keep the last one found here in job->sched_fence instead.

I guess this isn't enough.
The dependency is not in order when calling, so the last one is not always the last scheduled fence.
And they could be sched fence not hw fence, although they are handled by same hw ring, but the sched fence context isn't same. so we still need sched_sync here, right?

No, amdgpu_job_dependency is only called again when the returned fence is signaled (or scheduled on the same ring).

So when this is called and you find that you need to wait for another fence the order is guaranteed.

Regards,
Christian.

Quoted 12 lines

Regards,
David zhou

Quoted 8 lines

Regards,
Christian.

Quoted 2 lines
        return fence;

}

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 7769dbea-68ad-e6a0-81be-f4e07677c731-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

author
#21 lore raw
Quoted 95 lines · Christian König

Am 10.05.2017 um 10:38 schrieb zhoucm1:

Quoted 94 lines · Christian König
Quoted 80 lines

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 79 lines

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
      struct amdgpu_vm    *vm;
      struct amdgpu_ring    *ring;
      struct amdgpu_sync    sync;
+    struct amdgpu_sync    sched_sync;
      struct amdgpu_ib    *ibs;
      struct fence        *fence; /* the hw fence */
      uint32_t        preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +1 -0
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
  {
      struct amdgpu_device *adev = ring->adev;
      struct amdgpu_ib *ib = &ibs[0];
+    struct fence *tmp;
      bool skip_preamble, need_ctx_switch;
      unsigned patch_offset = ~0;
      struct amdgpu_vm *vm;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +5 -0
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
          return r;
      }
  -    if (ring->funcs->emit_pipeline_sync && job && 
 job->need_pipeline_sync)
+    if (ring->funcs->emit_pipeline_sync && job &&
+        (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+        job->need_pipeline_sync = true;
          amdgpu_ring_emit_pipeline_sync(ring);
+        fence_put(tmp);
+    }
      if (vm) {
          amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE go 

too fast than DE */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index cfa97ab..fa0c8b1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +2 -0
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, 
 unsigned num_ibs,
      (*job)->need_pipeline_sync = false;
        amdgpu_sync_create(&(*job)->sync);
+    amdgpu_sync_create(&(*job)->sched_sync);
        return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct 
 amd_sched_job *s_job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);
  }
  @@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);

}
@@ -154,7 +157,7 @@ static struct fence

Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c

Quoted 6 lines

*amdgpu_job_dependency(struct amd_sched_job *sched_job)

      }
        if (amd_sched_dependency_optimized(fence, 

sched_job->s_entity))

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
-        job->need_pipeline_sync = true;
+        amdgpu_sync_fence(job->adev, &job->sched_sync, fence);

This can result in an -ENOMEM

will handle it.

Quoted 2 lines

and additional to that we only need to remember the last fence optimized like this, not all of them.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

So just keep the last one found here in job->sched_fence instead.

I guess this isn't enough.
The dependency is not in order when calling, so the last one is not always the last scheduled fence.
And they could be sched fence not hw fence, although they are handled by same hw ring, but the sched fence context isn't same. so we still need sched_sync here, right?

No, amdgpu_job_dependency is only called again when the returned fence is signaled (or scheduled on the same ring).

Let use give an example for it:
Assume job->sync has two fences(fenceA and fenceB) which could be scheduled. fenceA is from entity1, fenceB is from entity2, but both for gfx engine, but fenceA could be submitted to hw ring behind fenceB. the order in job->sync list is: others---->fenceA---->fenceB--->others.
when calling amdgpu_job_dependency, fenceA will be checked first, and then fenceB.

If following your proposal, we only store fenceB, but fenceA is the later. Which isn't expected.

Regards,
David Zhou

Quoted 19 lines

So when this is called and you find that you need to wait for another fence the order is guaranteed.

Regards,
Christian.

Quoted 12 lines

Regards,
David zhou

Quoted 8 lines

Regards,
Christian.

Quoted 2 lines
        return fence;

}

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#23 lore raw

Am 10.05.2017 um 11:00 schrieb zhoucm1:

Quoted 104 lines · Christian König
Quoted 101 lines

Am 10.05.2017 um 10:38 schrieb zhoucm1:

Quoted 99 lines · Christian König
Quoted 80 lines

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 79 lines

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
      struct amdgpu_vm    *vm;
      struct amdgpu_ring    *ring;
      struct amdgpu_sync    sync;
+    struct amdgpu_sync    sched_sync;
      struct amdgpu_ib    *ibs;
      struct fence        *fence; /* the hw fence */
      uint32_t        preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +1 -0
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
  {
      struct amdgpu_device *adev = ring->adev;
      struct amdgpu_ib *ib = &ibs[0];
+    struct fence *tmp;
      bool skip_preamble, need_ctx_switch;
      unsigned patch_offset = ~0;
      struct amdgpu_vm *vm;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +5 -0
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
          return r;
      }
  -    if (ring->funcs->emit_pipeline_sync && job && 
 job->need_pipeline_sync)
+    if (ring->funcs->emit_pipeline_sync && job &&
+        (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+        job->need_pipeline_sync = true;
          amdgpu_ring_emit_pipeline_sync(ring);
+        fence_put(tmp);
+    }
      if (vm) {
          amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE 

go too fast than DE */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index cfa97ab..fa0c8b1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +2 -0
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device *adev, 
 unsigned num_ibs,
      (*job)->need_pipeline_sync = false;
        amdgpu_sync_create(&(*job)->sync);
+    amdgpu_sync_create(&(*job)->sched_sync);
        return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct 
 amd_sched_job *s_job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);
  }
  @@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);

}
@@ -154,7 +157,7 @@ static struct fence

Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c

Quoted 6 lines

*amdgpu_job_dependency(struct amd_sched_job *sched_job)

      }
        if (amd_sched_dependency_optimized(fence, 

sched_job->s_entity))

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
-        job->need_pipeline_sync = true;
+        amdgpu_sync_fence(job->adev, &job->sched_sync, fence);

This can result in an -ENOMEM

will handle it.

Quoted 3 lines

and additional to that we only need to remember the last fence optimized like this, not all of them.

So just keep the last one found here in job->sched_fence instead.

I guess this isn't enough.
The dependency is not in order when calling, so the last one is not always the last scheduled fence.
And they could be sched fence not hw fence, although they are handled by same hw ring, but the sched fence context isn't same. so we still need sched_sync here, right?

Quoted 7 lines
Quoted 1 line

No, amdgpu_job_dependency is only called again when the returned fence is signaled (or scheduled on the same ring).

Let use give an example for it:
Assume job->sync has two fences(fenceA and fenceB) which could be scheduled. fenceA is from entity1, fenceB is from entity2, but both for gfx engine, but fenceA could be submitted to hw ring behind fenceB. the order in job->sync list is: others---->fenceA---->fenceB--->others.
when calling amdgpu_job_dependency, fenceA will be checked first, and then fenceB.

If following your proposal, we only store fenceB, but fenceA is the later. Which isn't expected.

Ah! Indeed, I didn't realized that the dependent fence could have already been scheduled.

Mhm, how are we going to handle the out of memory situation then? Sine we are inside a kernel thread we are not supposed to fail at this point.

Regards,
Christian.

Quoted 24 lines

Regards,
David Zhou

Quoted 19 lines

So when this is called and you find that you need to wait for another fence the order is guaranteed.

Regards,
Christian.

Quoted 12 lines

Regards,
David zhou

Quoted 8 lines

Regards,
Christian.

Quoted 2 lines
        return fence;

}

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): ab63e653-6eb3-c42f-1674-ac26528fd202-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

author
#25 lore raw
Quoted 111 lines · Christian König

Am 10.05.2017 um 11:00 schrieb zhoucm1:

Quoted 110 lines · Christian König
Quoted 95 lines

Am 10.05.2017 um 10:38 schrieb zhoucm1:

Quoted 94 lines · Christian König
Quoted 80 lines

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 79 lines

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
      struct amdgpu_vm    *vm;
      struct amdgpu_ring    *ring;
      struct amdgpu_sync    sync;
+    struct amdgpu_sync    sched_sync;
      struct amdgpu_ib    *ibs;
      struct fence        *fence; /* the hw fence */
      uint32_t        preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +1 -0
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
  {
      struct amdgpu_device *adev = ring->adev;
      struct amdgpu_ib *ib = &ibs[0];
+    struct fence *tmp;
      bool skip_preamble, need_ctx_switch;
      unsigned patch_offset = ~0;
      struct amdgpu_vm *vm;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +5 -0
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
          return r;
      }
  -    if (ring->funcs->emit_pipeline_sync && job && 
 job->need_pipeline_sync)
+    if (ring->funcs->emit_pipeline_sync && job &&
+        (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+        job->need_pipeline_sync = true;
          amdgpu_ring_emit_pipeline_sync(ring);
+        fence_put(tmp);
+    }
      if (vm) {
          amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE 

go too fast than DE */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index cfa97ab..fa0c8b1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +2 -0
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device 
 *adev, unsigned num_ibs,
      (*job)->need_pipeline_sync = false;
        amdgpu_sync_create(&(*job)->sync);
+    amdgpu_sync_create(&(*job)->sched_sync);
        return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct 
 amd_sched_job *s_job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);
  }
  @@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);

}
@@ -154,7 +157,7 @@ static struct fence

Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c

Quoted 6 lines

*amdgpu_job_dependency(struct amd_sched_job *sched_job)

      }
        if (amd_sched_dependency_optimized(fence, 

sched_job->s_entity))

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
-        job->need_pipeline_sync = true;
+        amdgpu_sync_fence(job->adev, &job->sched_sync, fence);

This can result in an -ENOMEM

will handle it.

Quoted 2 lines

and additional to that we only need to remember the last fence optimized like this, not all of them.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

So just keep the last one found here in job->sched_fence instead.

I guess this isn't enough.
The dependency is not in order when calling, so the last one is not always the last scheduled fence.
And they could be sched fence not hw fence, although they are handled by same hw ring, but the sched fence context isn't same. so we still need sched_sync here, right?

No, amdgpu_job_dependency is only called again when the returned fence is signaled (or scheduled on the same ring).

Let use give an example for it:
Assume job->sync has two fences(fenceA and fenceB) which could be scheduled. fenceA is from entity1, fenceB is from entity2, but both for gfx engine, but fenceA could be submitted to hw ring behind fenceB. the order in job->sync list is: others---->fenceA---->fenceB--->others.
when calling amdgpu_job_dependency, fenceA will be checked first, and then fenceB.

Quoted 5 lines
Quoted 1 line

If following your proposal, we only store fenceB, but fenceA is the later. Which isn't expected.

Ah! Indeed, I didn't realized that the dependent fence could have already been scheduled.

Mhm, how are we going to handle the out of memory situation then? Sine we are inside a kernel thread we are not supposed to fail at this point.

like grab vmid failed case, add DRM_ERROR, is it ok?

Regards,
David Zhou

Quoted 29 lines

Regards,
Christian.

Quoted 24 lines

Regards,
David Zhou

Quoted 19 lines

So when this is called and you find that you need to wait for another fence the order is guaranteed.

Regards,
Christian.

Quoted 12 lines

Regards,
David zhou

Quoted 8 lines

Regards,
Christian.

Quoted 2 lines
        return fence;

}

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): [email protected]

#27 lore raw

Am 10.05.2017 um 11:20 schrieb zhoucm1:

Quoted 114 lines · Christian König
Quoted 111 lines

Am 10.05.2017 um 11:00 schrieb zhoucm1:

Quoted 110 lines · Christian König
Quoted 95 lines

Am 10.05.2017 um 10:38 schrieb zhoucm1:

Quoted 94 lines · Christian König
Quoted 80 lines

Am 10.05.2017 um 09:31 schrieb Chunming Zhou:

Quoted 79 lines

this is an improvement for previous patch, the sched_sync is to store fence that could be skipped as scheduled, when job is executed, we didn't need pipeline_sync if all fences in sched_sync are signalled, otherwise insert pipeline_sync still.

Change-Id I26d3a2794272ba94b25753d4bf367326d12f6939
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
  drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c  | 7 ++++++-
  drivers/gpu/drm/amd/amdgpu/amdgpu_job.c | 5 ++++-
  3 files changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 787acd7..ef018bf 100644
@@ -1162,6 +1162,7 @@ struct amdgpu_job {
      struct amdgpu_vm    *vm;
      struct amdgpu_ring    *ring;
      struct amdgpu_sync    sync;
+    struct amdgpu_sync    sched_sync;
      struct amdgpu_ib    *ibs;
      struct fence        *fence; /* the hw fence */
      uint32_t        preamble_status;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +1 -0
index 2c6624d..86ad507 100644
@@ -121,6 +121,7 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
  {
      struct amdgpu_device *adev = ring->adev;
      struct amdgpu_ib *ib = &ibs[0];
+    struct fence *tmp;
      bool skip_preamble, need_ctx_switch;
      unsigned patch_offset = ~0;
      struct amdgpu_vm *vm;
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +5 -0
@@ -167,8 +168,12 @@ int amdgpu_ib_schedule(struct amdgpu_ring 
 *ring, unsigned num_ibs,
          return r;
      }
  -    if (ring->funcs->emit_pipeline_sync && job && 
 job->need_pipeline_sync)
+    if (ring->funcs->emit_pipeline_sync && job &&
+        (tmp = amdgpu_sync_get_fence(&job->sched_sync))) {
+        job->need_pipeline_sync = true;
          amdgpu_ring_emit_pipeline_sync(ring);
+        fence_put(tmp);
+    }
      if (vm) {
          amdgpu_ring_insert_nop(ring, extra_nop); /* prevent CE 

go too fast than DE */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c index cfa97ab..fa0c8b1 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +2 -0
@@ -60,6 +60,7 @@ int amdgpu_job_alloc(struct amdgpu_device 
 *adev, unsigned num_ibs,
      (*job)->need_pipeline_sync = false;
        amdgpu_sync_create(&(*job)->sync);
+    amdgpu_sync_create(&(*job)->sched_sync);
        return 0;
  }
@@ -98,6 +99,7 @@ static void amdgpu_job_free_cb(struct 
 amd_sched_job *s_job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);
  }
  @@ -107,6 +109,7 @@ void amdgpu_job_free(struct amdgpu_job *job)
        fence_put(job->fence);
      amdgpu_sync_free(&job->sync);
+    amdgpu_sync_free(&job->sched_sync);
      kfree(job);

}
@@ -154,7 +157,7 @@ static struct fence

Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c

Quoted 6 lines

*amdgpu_job_dependency(struct amd_sched_job *sched_job)

      }
        if (amd_sched_dependency_optimized(fence, 

sched_job->s_entity))

drivers/gpu/drm/amd/amdgpu/amdgpu_job.c +1 -1
-        job->need_pipeline_sync = true;
+        amdgpu_sync_fence(job->adev, &job->sched_sync, fence);

This can result in an -ENOMEM

will handle it.

Quoted 2 lines

and additional to that we only need to remember the last fence optimized like this, not all of them.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

So just keep the last one found here in job->sched_fence instead.

I guess this isn't enough.
The dependency is not in order when calling, so the last one is not always the last scheduled fence.
And they could be sched fence not hw fence, although they are handled by same hw ring, but the sched fence context isn't same. so we still need sched_sync here, right?

No, amdgpu_job_dependency is only called again when the returned fence is signaled (or scheduled on the same ring).

Let use give an example for it:
Assume job->sync has two fences(fenceA and fenceB) which could be scheduled. fenceA is from entity1, fenceB is from entity2, but both for gfx engine, but fenceA could be submitted to hw ring behind fenceB. the order in job->sync list is: others---->fenceA---->fenceB--->others.
when calling amdgpu_job_dependency, fenceA will be checked first, and then fenceB.

Quoted 6 lines
Quoted 5 lines
Quoted 1 line

If following your proposal, we only store fenceB, but fenceA is the later. Which isn't expected.

Ah! Indeed, I didn't realized that the dependent fence could have already been scheduled.

Mhm, how are we going to handle the out of memory situation then? Sine we are inside a kernel thread we are not supposed to fail at this point.

like grab vmid failed case, add DRM_ERROR, is it ok?

Not ideal, but should at least work for the moment.

Christian.

Quoted 33 lines

Regards,
David Zhou

Quoted 29 lines

Regards,
Christian.

Quoted 24 lines

Regards,
David Zhou

Quoted 19 lines

So when this is called and you find that you need to wait for another fence the order is guaranteed.

Regards,
Christian.

Quoted 12 lines

Regards,
David zhou

Quoted 8 lines

Regards,
Christian.

Quoted 2 lines
        return fence;

}

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx