Skip to contentSkip to conversation

drm/amdgpu: Sync with VM root BO when switching VM to CPU update mode

PATCH In review Felix Kuehling 3 participants 4 messages May 20, 2020 – May 22, 2020

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/amdgpu: Sync with VM root BO when switching VM to CPU update mode

This fixes an intermittent bug where a root PD clear operation still in progress could overwrite a PDE update done by the CPU, resulting in a VM fault.

Fixes 108b4d928c03 ("drm/amd/amdgpu: Update VM function pointer")
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +9 -2
index 414a0b1c2e5a..7417754e9141 100644
@@ -3000,10 +3000,17 @@ int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm,
 		   !amdgpu_gmc_vram_full_visible(&adev->gmc)),
 		  "CPU update of VM recommended only for large BAR system\n");
 
-	if (vm->use_cpu_for_update)
+	if (vm->use_cpu_for_update) {
+		/* Sync with last SDMA update/clear before switching to CPU */
+		r = amdgpu_bo_sync_wait(vm->root.base.bo,
+					AMDGPU_FENCE_OWNER_UNDEFINED, true);
+		if (r)
+			goto free_idr;
+
 		vm->update_funcs = &amdgpu_vm_cpu_funcs;
-	else
+	} else {
 		vm->update_funcs = &amdgpu_vm_sdma_funcs;
+	}
 	dma_fence_put(vm->last_update);
 	vm->last_update = NULL;
 	vm->is_compute_context = true;
Signature · 6 lines
2.17.1

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#1 lore raw

Am 21.05.20 um 00:51 schrieb Felix Kuehling:

Quoted 3 lines

This fixes an intermittent bug where a root PD clear operation still in progress could overwrite a PDE update done by the CPU, resulting in a VM fault.

Mhm, maybe better add this to amdgpu_vm_cpu_prepare().

This way we could (in theory) switch between CPU and SDMA based updates on the fly elsewhere as well.

Christian.

Quoted 23 lines
Fixes 108b4d928c03 ("drm/amd/amdgpu: Update VM function pointer")
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 11 +++++++++--
  1 file changed, 9 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +5 -1
index 414a0b1c2e5a..7417754e9141 100644
@@ -3000,10 +3000,17 @@ int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm,
  		   !amdgpu_gmc_vram_full_visible(&adev->gmc)),
  		  "CPU update of VM recommended only for large BAR system\n");
 
-	if (vm->use_cpu_for_update)
+	if (vm->use_cpu_for_update) {
+		/* Sync with last SDMA update/clear before switching to CPU */
+		r = amdgpu_bo_sync_wait(vm->root.base.bo,
+					AMDGPU_FENCE_OWNER_UNDEFINED, true);
+		if (r)

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm, · line 3016

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -1
+			goto free_idr;
+
  		vm->update_funcs = &amdgpu_vm_cpu_funcs;
-	else
+	} else {
  		vm->update_funcs = &amdgpu_vm_sdma_funcs;
+	}
  	dma_fence_put(vm->last_update);
  	vm->last_update = NULL;
  	vm->is_compute_context = true;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#2 lore raw

Am 2020-05-21 um 9:50 a.m. schrieb Christian König:

Quoted 9 lines

Am 21.05.20 um 00:51 schrieb Felix Kuehling:

Quoted 3 lines

This fixes an intermittent bug where a root PD clear operation still in progress could overwrite a PDE update done by the CPU, resulting in a VM fault.

Mhm, maybe better add this to amdgpu_vm_cpu_prepare().

This way we could (in theory) switch between CPU and SDMA based updates on the fly elsewhere as well.

That won't work. I want to wait for FENCE_OWNER_VM fences, so I need to use FENCE_OWNER_UNDEFINED. But then I would also end up waiting for FENCE_OWNER_KFD eviction fences, which would trigger unwanted evictions.

This works OK in amdgpu_vm_make_compute because it runs before the eviction fence is attached to the VM.

Regards,
  Felix

Quoted 29 lines

Christian.

Quoted 26 lines
Fixes 108b4d928c03 ("drm/amd/amdgpu: Update VM function pointer")

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 11 +++++++++--

  1 file changed, 9 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +7 -0
index 414a0b1c2e5a..7417754e9141 100644
@@ -3000,10 +3000,17 @@ int amdgpu_vm_make_compute(struct
 amdgpu_device *adev, struct amdgpu_vm *vm,
              !amdgpu_gmc_vram_full_visible(&adev->gmc)),
             "CPU update of VM recommended only for large BAR system\n");
   -    if (vm->use_cpu_for_update)
+    if (vm->use_cpu_for_update) {
+        /* Sync with last SDMA update/clear before switching to CPU */
+        r = amdgpu_bo_sync_wait(vm->root.base.bo,
+                    AMDGPU_FENCE_OWNER_UNDEFINED, true);
+        if (r)
+            goto free_idr;
+
Quoted 9 lines
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -1
           vm->update_funcs = &amdgpu_vm_cpu_funcs;
-    else
+    } else {
           vm->update_funcs = &amdgpu_vm_sdma_funcs;
+    }
       dma_fence_put(vm->last_update);
       vm->last_update = NULL;

      vm->is_compute_context = true;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

maintainer
#3 lore raw

Am 21.05.20 um 19:06 schrieb Felix Kuehling:

Quoted 7 lines

Am 2020-05-21 um 9:50 a.m. schrieb Christian König:

Quoted 6 lines

Am 21.05.20 um 00:51 schrieb Felix Kuehling:

Quoted 3 lines

This fixes an intermittent bug where a root PD clear operation still in progress could overwrite a PDE update done by the CPU, resulting in a VM fault.

Mhm, maybe better add this to amdgpu_vm_cpu_prepare().

Quoted 8 lines
Quoted 2 lines

This way we could (in theory) switch between CPU and SDMA based updates on the fly elsewhere as well.

That won't work. I want to wait for FENCE_OWNER_VM fences, so I need to use FENCE_OWNER_UNDEFINED. But then I would also end up waiting for FENCE_OWNER_KFD eviction fences, which would trigger unwanted evictions.

This works OK in amdgpu_vm_make_compute because it runs before the eviction fence is attached to the VM.

Ok, in this case the patch is Reviewed-by: Christian König <[email protected]>.

Quoted 32 lines

Regards,
  Felix

Quoted 27 lines

Christian.

Quoted 25 lines
Fixes 108b4d928c03 ("drm/amd/amdgpu: Update VM function pointer")
---
   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 11 +++++++++--
   1 file changed, 9 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +7 -0
index 414a0b1c2e5a..7417754e9141 100644
@@ -3000,10 +3000,17 @@ int amdgpu_vm_make_compute(struct
 amdgpu_device *adev, struct amdgpu_vm *vm,
              !amdgpu_gmc_vram_full_visible(&adev->gmc)),
             "CPU update of VM recommended only for large BAR system\n");
   -    if (vm->use_cpu_for_update)
+    if (vm->use_cpu_for_update) {
+        /* Sync with last SDMA update/clear before switching to CPU */
+        r = amdgpu_bo_sync_wait(vm->root.base.bo,
+                    AMDGPU_FENCE_OWNER_UNDEFINED, true);
+        if (r)
+            goto free_idr;
+
Quoted 8 lines
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_make_compute(struct amdgpu_device *adev, struct amdgpu_vm *vm, · line 3016

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +2 -1
           vm->update_funcs = &amdgpu_vm_cpu_funcs;
-    else
+    } else {
           vm->update_funcs = &amdgpu_vm_sdma_funcs;
+    }
       dma_fence_put(vm->last_update);
       vm->last_update = NULL;
       vm->is_compute_context = true;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx