Skip to contentSkip to conversation

drm/amdgpu: Correct unlocked update fence handling

PATCH NAK Philip Yang 3 participants 9 messages Mar 28, 2022

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/3 drm/amdgpu: Correct unlocked update fence handling

To fix two issues with unlocked update fence:

1. vm->last_unlocked store the latest fence without taking refcount.
2. amdgpu_vm_bo_update_mapping returns old fence, not the latest fence.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +1 -1
index dbb551762805..69fba68ff88e 100644
@@ -109,7 +109,7 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
 	if (p->unlocked) {
 		struct dma_fence *tmp = dma_fence_get(f);
 
-		swap(p->vm->last_unlocked, f);
+		swap(p->vm->last_unlocked, tmp);
 		dma_fence_put(tmp);
 	} else {
 		amdgpu_bo_fence(p->vm->root.bo, f, true);
Signature · 1 line
2.35.1
#1 lore raw

PATCH v2 2/3 drm/amdgpu: Add tlb_cb for unlocked update

Flush TLB needs wait for GPU update fence done. MMU notify callback to unmap range from GPUs uses unlocked GPU page table update, so add tlb_cb to unlocked update fence to increase vm->tlb_seq.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -1
index 48f326609976..683b08f756ed 100644
@@ -927,7 +927,7 @@ int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
 
 	r = vm->update_funcs->commit(&params, fence);
 
-	if (!unlocked && (!(flags & AMDGPU_PTE_VALID) || params.table_freed)) {
+	if (!(flags & AMDGPU_PTE_VALID) || params.table_freed) {
 		tlb_cb->vm = vm;
 		if (!fence || !*fence ||
 		    dma_fence_add_callback(*fence, &tlb_cb->cb,
Signature · 1 line
2.35.1
maintainer Reviewed-by on v2 2/3
#2 lore raw

Re: [PATCH v2 2/3] drm/amdgpu: Add tlb_cb for unlocked update

Am 28.03.22 um 15:06 schrieb Philip Yang:

Quoted 5 lines

Flush TLB needs wait for GPU update fence done. MMU notify callback to unmap range from GPUs uses unlocked GPU page table update, so add tlb_cb to unlocked update fence to increase vm->tlb_seq.

Quoted 17 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +1 -1
index 48f326609976..683b08f756ed 100644
@@ -927,7 +927,7 @@ int amdgpu_vm_bo_update_mapping(struct amdgpu_device *adev,
 
  	r = vm->update_funcs->commit(&params, fence);
 
-	if (!unlocked && (!(flags & AMDGPU_PTE_VALID) || params.table_freed)) {
+	if (!(flags & AMDGPU_PTE_VALID) || params.table_freed) {
  		tlb_cb->vm = vm;
  		if (!fence || !*fence ||
  		    dma_fence_add_callback(*fence, &tlb_cb->cb,
#3 lore raw

PATCH v2 3/3 drm/amdkfd: Use atomic64_t type for pdd->tlb_seq

In reply to Philip Yang

To support multi-thread update page table.

---
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h    | 2 +-
 drivers/gpu/drm/amd/amdkfd/kfd_process.c | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +1 -1
index 945982a5d688..e1b7e6afa920 100644
@@ -705,7 +705,7 @@ struct kfd_process_device {
 	/* VM context for GPUVM allocations */
 	struct file *drm_file;
 	void *drm_priv;
-	uint64_t tlb_seq;
+	atomic64_t tlb_seq;
 
 	/* GPUVM allocations storage */
 	struct idr alloc_idr;
drivers/gpu/drm/amd/amdkfd/kfd_process.c +3 -3
index ac8123c1ee8f..43ed8ec1f975 100644
@@ -1560,7 +1560,7 @@ int kfd_process_device_init_vm(struct kfd_process_device *pdd,
 		return ret;
 	}
 	pdd->drm_priv = drm_file->private_data;
-	pdd->tlb_seq = 0;
+	atomic64_set(&pdd->tlb_seq, 0);
 
 	ret = kfd_process_device_reserve_ib_mem(pdd);
 	if (ret)
@@ -1954,10 +1954,10 @@ void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type)
 	uint64_t tlb_seq = amdgpu_vm_tlb_seq(vm);
 	struct kfd_dev *dev = pdd->dev;
 
-	if (pdd->tlb_seq == tlb_seq)
+	if (atomic64_read(&pdd->tlb_seq) == tlb_seq)
 		return;
 
-	pdd->tlb_seq = tlb_seq;
+	atomic64_set(&pdd->tlb_seq, tlb_seq);
 	if (dev->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS) {
 		/* Nothing to flush until a VMID is assigned, which
 		 * only happens when the first queue is created.
Signature · 1 line
2.35.1
maintainer
#4 lore raw

Re: [PATCH v2 3/3] drm/amdkfd: Use atomic64_t type for pdd->tlb_seq

Am 28.03.22 um 15:07 schrieb Philip Yang:

Quoted 35 lines

To support multi-thread update page table.

---
  drivers/gpu/drm/amd/amdkfd/kfd_priv.h    | 2 +-
  drivers/gpu/drm/amd/amdkfd/kfd_process.c | 6 +++---
  2 files changed, 4 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdkfd/kfd_priv.h +1 -1
index 945982a5d688..e1b7e6afa920 100644
@@ -705,7 +705,7 @@ struct kfd_process_device {
  	/* VM context for GPUVM allocations */
  	struct file *drm_file;
  	void *drm_priv;
-	uint64_t tlb_seq;
+	atomic64_t tlb_seq;
 
  	/* GPUVM allocations storage */
  	struct idr alloc_idr;
drivers/gpu/drm/amd/amdkfd/kfd_process.c +1 -1
index ac8123c1ee8f..43ed8ec1f975 100644
@@ -1560,7 +1560,7 @@ int kfd_process_device_init_vm(struct kfd_process_device *pdd,
  		return ret;
  	}
  	pdd->drm_priv = drm_file->private_data;
-	pdd->tlb_seq = 0;
+	atomic64_set(&pdd->tlb_seq, 0);
 
  	ret = kfd_process_device_reserve_ib_mem(pdd);
  	if (ret)

drivers/gpu/drm/amd/amdkfd/kfd_process.c void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type) · line 1960

Quoted 9 lines
drivers/gpu/drm/amd/amdkfd/kfd_process.c +2 -2
@@ -1954,10 +1954,10 @@ void kfd_flush_tlb(struct kfd_process_device *pdd, enum TLB_FLUSH_TYPE type)
  	uint64_t tlb_seq = amdgpu_vm_tlb_seq(vm);
  	struct kfd_dev *dev = pdd->dev;
 
-	if (pdd->tlb_seq == tlb_seq)
+	if (atomic64_read(&pdd->tlb_seq) == tlb_seq)
  		return;
 
-	pdd->tlb_seq = tlb_seq;
+	atomic64_set(&pdd->tlb_seq, tlb_seq);

That should probably use atomic64_xchg() instead of read+set or otherwise using an atomic doesn't make much sense.

Christian.

Quoted 3 lines
drivers/gpu/drm/amd/amdkfd/kfd_process.c +0 -0
  	if (dev->dqm->sched_policy == KFD_SCHED_POLICY_NO_HWS) {
  		/* Nothing to flush until a VMID is assigned, which
  		 * only happens when the first queue is created.
maintainer NAK
#5 lore raw
In reply to Philip Yang

Am 28.03.22 um 15:06 schrieb Philip Yang:

Quoted 4 lines

To fix two issues with unlocked update fence:

1. vm->last_unlocked store the latest fence without taking refcount.
2. amdgpu_vm_bo_update_mapping returns old fence, not the latest fence.

NAK, that code here is perfectly correct.

vm->last_unlocked gets a reference to the last unlocked operation.

Otherwise the last locked operation is added directly to the root resv object.

Regards,
Christian.

Quoted 19 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +1 -1
index dbb551762805..69fba68ff88e 100644
@@ -109,7 +109,7 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
  	if (p->unlocked) {
  		struct dma_fence *tmp = dma_fence_get(f);
 
-		swap(p->vm->last_unlocked, f);
+		swap(p->vm->last_unlocked, tmp);
  		dma_fence_put(tmp);
  	} else {
  		amdgpu_bo_fence(p->vm->root.bo, f, true);
#6 lore raw

On 2022-03-28 9:14 a.m., Christian König wrote:
Am 28.03.22 um 15:06 schrieb Philip Yang:
To fix two issues with unlocked update fence:

1. vm->last_unlocked store the latest fence without taking refcount.
2. amdgpu_vm_bo_update_mapping returns old fence, not the latest fence.

NAK, that code here is perfectly correct.

vm->last_unlocked gets a reference to the last unlocked operation.

Otherwise the last locked operation is added directly to the root resv object.

Regards,
Christian.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +0 -0
index dbb551762805..69fba68ff88e 100644
@@ -109,7 +109,7 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,

      if (p->unlocked) {
          struct dma_fence *tmp = dma_fence_get(f); -        swap(p->vm->last_unlocked, f);
+        swap(p->vm->last_unlocked, tmp); dma_fence_put(tmp);

tmp is the new unlock fence, so this drop the new fence refcount, fix is to move the old fence to tmp, drop the old fence refcount, take the new fence reference. f is return fence, because f swap to old fence, so old fence is returned, not new fence, fix will not change f.
Regards,
Philip

      } else { amdgpu_bo_fence(p->vm->root.bo, f, true);

maintainer
#7 lore raw

Am 28.03.22 um 15:35 schrieb philip yang:

Quoted 37 lines · Christian König
Quoted 21 lines

Am 28.03.22 um 15:06 schrieb Philip Yang:

Quoted 4 lines

To fix two issues with unlocked update fence:

1. vm->last_unlocked store the latest fence without taking refcount.
2. amdgpu_vm_bo_update_mapping returns old fence, not the latest fence.

NAK, that code here is perfectly correct.

vm->last_unlocked gets a reference to the last unlocked operation.

Otherwise the last locked operation is added directly to the root resv object.

Regards,
Christian.

Quoted 6 lines

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 +-

  1 file changed, 1 insertion(+), 1 deletion(-)
Quoted 10 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c static int amdgpu_vm_sdma_commit(struct · line 113

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +1 -0
index dbb551762805..69fba68ff88e 100644
@@ -109,7 +109,7 @@ static int amdgpu_vm_sdma_commit(struct 
 amdgpu_vm_update_params *p,
       if (p->unlocked) {
           struct dma_fence *tmp = dma_fence_get(f);
   -        swap(p->vm->last_unlocked, f);
+        swap(p->vm->last_unlocked, tmp);

          dma_fence_put(tmp);

Quoted 4 lines

tmp is the new unlock fence, so this drop the new fence refcount, fix is to move the old fence to tmp, drop the old fence refcount, take the new fence reference.

f is return fence, because f swap to old fence, so old fence is returned, not new fence, fix will not change f.

No, swap() will exchange the parameters a and b.

So tmp is pointing to the old fence when dma_fence_put() is called on it.

That reference counting is correct as far as I can see.

Regards,
Christian.

Quoted 7 lines

Regards,

Philip

Quoted 3 lines
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +0 -0
       } else {
           amdgpu_bo_fence(p->vm->root.bo, f, true);
maintainer
#8 lore raw
In reply to Philip Yang

Am 28.03.22 um 15:06 schrieb Philip Yang:

Quoted 6 lines

To fix two issues with unlocked update fence:

1. vm->last_unlocked store the latest fence without taking refcount.
2. amdgpu_vm_bo_update_mapping returns old fence, not the latest fence.

Now I see what you mean. Patch is Reviewed-by: Christian König <[email protected]>

Quoted 17 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c +1 -1
index dbb551762805..69fba68ff88e 100644
@@ -109,7 +109,7 @@ static int amdgpu_vm_sdma_commit(struct amdgpu_vm_update_params *p,
  	if (p->unlocked) {
  		struct dma_fence *tmp = dma_fence_get(f);
 
-		swap(p->vm->last_unlocked, f);
+		swap(p->vm->last_unlocked, tmp);
  		dma_fence_put(tmp);
  	} else {
  		amdgpu_bo_fence(p->vm->root.bo, f, true);