Skip to contentSkip to conversation

drm/amdgpu: always emit the job vm fence

PATCH In review Alex Deucher 3 participants 6 messages Jun 5 – Aug 10

Subsystems (2)
lore
Share & tools
lore

Alex Deucher

author maintainer
#0 lore raw

PATCH 1/3 drm/amdgpu: always emit the job vm fence

We need the fence to reemit the gds switch or spm update after a queue reset.

Fixes a17ef941212b ("drm/amdgpu: rework ring reset backup and reemit v9")
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 10 ++++------
 1 file changed, 4 insertions(+), 6 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -6
index 2f3470208829e..7e0e2281719b1 100644
@@ -853,12 +853,10 @@ void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
 					    job->oa_size);
 	}
 
-	if (vm_flush_needed || pasid_mapping_needed || cleaner_shader_needed) {
-		amdgpu_fence_emit(ring, job->hw_vm_fence, 0);
-		fence = &job->hw_vm_fence->base;
-		/* get a ref for the job */
-		dma_fence_get(fence);
-	}
+	amdgpu_fence_emit(ring, job->hw_vm_fence, 0);
+	fence = &job->hw_vm_fence->base;
+	/* get a ref for the job */
+	dma_fence_get(fence);
 
 	if (vm_flush_needed) {
 		mutex_lock(&id_mgr->lock);
Signature · 1 line
2.54.0
author maintainer
#1 lore raw

PATCH 2/3 drm/amdgpu: fix missing check in vm_flush()

We shouldn't return early if we need to emit spm update.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -1
index 7e0e2281719b1..13031e4b6f1d5 100644
@@ -809,7 +809,7 @@ void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
 		&job->base.s_fence->scheduled == isolation->spearhead;
 
 	if (!vm_flush_needed && !gds_switch_needed && !need_pipe_sync &&
-	    !cleaner_shader_needed)
+	    !cleaner_shader_needed && !spm_update_needed)
 		return;
 
 	amdgpu_ring_ib_begin(ring);
Signature · 1 line
2.54.0
#2 lore raw

Re: [PATCH 2/3] drm/amdgpu: fix missing check in vm_flush()

Ping?

Alex

Quoted 24 lines · Alex Deucher

We shouldn't return early if we need to emit spm update.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -2
index 7e0e2281719b1..13031e4b6f1d5 100644
@@ -809,7 +809,7 @@ void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
                &job->base.s_fence->scheduled == isolation->spearhead;
 
        if (!vm_flush_needed && !gds_switch_needed && !need_pipe_sync &&
-           !cleaner_shader_needed)
+           !cleaner_shader_needed && !spm_update_needed)
                return;
 
        amdgpu_ring_ib_begin(ring);
--

2.54.0

#3 lore raw

Re: [PATCH 2/3] drm/amdgpu: fix missing check in vm_flush()

+ David

Ping?

Quoted 30 lines · Alex Deucher

Ping?

Alex

Quoted 24 lines · Alex Deucher

We shouldn't return early if we need to emit spm update.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -2
index 7e0e2281719b1..13031e4b6f1d5 100644
@@ -809,7 +809,7 @@ void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
                &job->base.s_fence->scheduled == isolation->spearhead;
 
        if (!vm_flush_needed && !gds_switch_needed && !need_pipe_sync &&
-           !cleaner_shader_needed)
+           !cleaner_shader_needed && !spm_update_needed)
                return;
 
        amdgpu_ring_ib_begin(ring);
--

2.54.0

Reviewed-by
#4 lore raw

Re: [PATCH 2/3] drm/amdgpu: fix missing check in vm_flush()

Quoted 15 lines · Alex Deucher

[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]

+ David

Ping?

Quoted 8 lines · Alex Deucher

Ping?

Alex

Quoted 3 lines · Alex Deucher

We shouldn't return early if we need to emit spm update.

It may be better to move
+       spm_update_needed &= !!adev->gfx.rlc.funcs->update_spm_vmid; from the 3rd patch here.

Quoted 20 lines
Quoted 20 lines
Quoted 20 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -2
index 7e0e2281719b1..13031e4b6f1d5 100644
@@ -809,7 +809,7 @@ void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
                 &job->base.s_fence->scheduled == isolation->spearhead;
 
         if (!vm_flush_needed && !gds_switch_needed && !need_pipe_sync &&
-           !cleaner_shader_needed)
+           !cleaner_shader_needed && !spm_update_needed)
                 return;
 
         amdgpu_ring_ib_begin(ring);
--

2.54.0

author maintainer
#5 lore raw

PATCH 3/3 drm/amdgpu: handle GDS and SPM without a VM fence

In reply to Alex Deucher

If we end up emitting a VM fence keep GDS and SPM
associated with that fence. If not, emit them as
part of the IB fence.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c | 14 +++++++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 47 ++++++++++++++++++--------
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  4 ++-
 3 files changed, 48 insertions(+), 17 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c +13 -1
index f1ed4a436f5b4..d439c68eed9de 100644
@@ -131,6 +131,8 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs,
 	struct amdgpu_fence *af;
 	struct amdgpu_fence *vm_af;
 	bool need_ctx_switch;
+	bool emit_spm_needed = false;
+	bool emit_gds_needed = false;
 	struct amdgpu_vm *vm;
 	uint64_t fence_ctx;
 	uint32_t status = 0, alloc_size;
@@ -220,7 +222,8 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs,
 		vm_af = job->hw_vm_fence;
 		/* VM sequence */
 		vm_af->ib_wptr = ring->wptr;
-		amdgpu_vm_flush(ring, job, need_pipe_sync);
+		amdgpu_vm_flush(ring, job, need_pipe_sync, &emit_spm_needed,
+				&emit_gds_needed);
 		vm_af->ib_dw_size =
 			amdgpu_ring_get_dw_distance(ring, vm_af->ib_wptr, ring->wptr);
 	}
@@ -232,6 +235,15 @@ int amdgpu_ib_schedule(struct amdgpu_ring *ring, unsigned int num_ibs,
 	if (ring->funcs->insert_start)
 		ring->funcs->insert_start(ring);
 
+	if (emit_spm_needed)
+		adev->gfx.rlc.funcs->update_spm_vmid(adev, ring->xcc_id, ring, job->vmid);
+
+	if (emit_gds_needed)
+		amdgpu_ring_emit_gds_switch(ring, job->vmid, job->gds_base,
+					    job->gds_size, job->gws_base,
+					    job->gws_size, job->oa_base,
+					    job->oa_size);
+
 	if ((ib->flags & AMDGPU_IB_FLAG_EMIT_MEM_SYNC) && ring->funcs->emit_mem_sync)
 		ring->funcs->emit_mem_sync(ring);
 
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +32 -15

89 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +3 -1
index cc096c005e348..b67eeec464e09 100644
@@ -514,7 +514,9 @@ int amdgpu_vm_validate(struct amdgpu_device *adev, struct amdgpu_vm *vm,
 		       struct ww_acquire_ctx *ticket,
 		       int (*callback)(void *p, struct amdgpu_bo *bo),
 		       void *param);
-void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job, bool need_pipe_sync);
+void amdgpu_vm_flush(struct amdgpu_ring *ring, struct amdgpu_job *job,
+		     bool need_pipe_sync, bool *emit_spm_needed,
+		     bool *emit_gds_needed);
 int amdgpu_vm_update_pdes(struct amdgpu_device *adev,
 			  struct amdgpu_vm *vm, bool immediate);
 int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
Signature · 1 line
2.54.0