Skip to contentSkip to conversation

drm/xe: Decrement fault mode counts in xe_vm_close_and_put

PATCH In review Matthew Brost 4 participants 5 messages Mar 21, 2023 – Mar 24, 2023

Subsystems (2)
lore
Share & tools
lore

Matthew Brost

author maintainer
#0 lore raw

PATCH 1/1 drm/xe: Decrement fault mode counts in xe_vm_close_and_put

Rather waiting for the VM to be destroyed (all refs to VM go to zero), drop the fault mode counts when the VM is closed in xe_vm_close_and_put. This avoids a window where user space can create a faulting VM, close it, and a subsequent creation of a non-faulting VM fails.

v2 (Lucas): Drop VLK reference in commit message

---
 drivers/gpu/drm/xe/xe_vm.c | 15 +++++++--------
 1 file changed, 7 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +7 -8
index 68ab51f84bed..ab036a51d17e 100644
@@ -1349,6 +1349,13 @@ void xe_vm_close_and_put(struct xe_vm *vm)
 	XE_WARN_ON(!list_empty(&vm->extobj.list));
 	up_write(&vm->lock);
 
+	mutex_lock(&xe->usm.lock);
+	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
+		xe->usm.num_vm_in_fault_mode--;
+	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
+		xe->usm.num_vm_in_non_fault_mode--;
+	mutex_unlock(&xe->usm.lock);
+
 	xe_vm_put(vm);
 }
 
@@ -1393,18 +1400,10 @@ static void vm_destroy_work_func(struct work_struct *w)
 
 	drm_gpuva_manager_destroy(&vm->mgr);
 
-	mutex_lock(&xe->usm.lock);
-	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
-		xe->usm.num_vm_in_fault_mode--;
-	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
-		xe->usm.num_vm_in_non_fault_mode--;
-	mutex_unlock(&xe->usm.lock);
-
 	trace_xe_vm_free(vm);
 	dma_fence_put(vm->rebind_fence);
 	dma_resv_fini(&vm->resv);
 	kfree(vm);
-
 }
 
 void xe_vm_free(struct kref *ref)
Signature · 1 line
2.34.1
CI Reviewed-by Maarten Lankhorst Reviewed-by Gwan-gyeong Mun on v1 1/1
#1 lore raw

[Intel-xe] ✗ CI.Patch_applied: failure for drm/xe: Decrement fault mode counts in xe_vm_close_and_put (rev2)

== Series Details ==

== Series Details ==

Series: drm/xe: Decrement fault mode counts in xe_vm_close_and_put (rev2)
URL : https://patchwork.freedesktop.org/series/115303/
State : failure

== Summary ==

=== Applying kernel patches on branch 'drm-xe-next' with base: === commit 1ae4dd9e82429e075aa7121404b06ce170da5848

Author:     Matthew Auld <[email protected]>
AuthorDate: Tue Mar 21 11:44:07 2023 +0000
Commit:     Matthew Auld <[email protected]>
CommitDate: Tue Mar 21 14:03:29 2023 +0000

drm/xe/buddy: add compatible and intersects hooks

Copy this from i915. We need .compatible for vram -> vram transfers, so they don't just get nooped by ttm, if need to move something from mappable to non-mappble or vice versa. The .intersects is needed for eviction, to determine if a victim resource is worth eviction. e.g if we need mappable space there is no point in evicting a resource that has zero mappable pages.

=== git am output follows ===
error: patch failed: drivers/gpu/drm/xe/xe_vm.c:1393
error: drivers/gpu/drm/xe/xe_vm.c: patch does not apply
hint: Use 'git am --show-current-patch' to see the failed patch
Applying: drm/xe: Decrement fault mode counts in xe_vm_close_and_put Patch failed at 0001 drm/xe: Decrement fault mode counts in xe_vm_close_and_put When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

#2 lore raw
In reply to Matthew Brost
Quoted 9 lines · Matthew Brost

Rather waiting for the VM to be destroyed (all refs to VM go to zero), drop the fault mode counts when the VM is closed in xe_vm_close_and_put. This avoids a window where user space can create a faulting VM, close it, and a subsequent creation of a non-faulting VM fails.

v2 (Lucas): Drop VLK reference in commit message

This patch should probably go in the 'USM locking fixes' patch series?

Quoted 10 lines

---
drivers/gpu/drm/xe/xe_vm.c | 15 +++++++--------

1 file changed, 7 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +0 -0
index 68ab51f84bed..ab036a51d17e 100644
@@ -1349,6 +1349,13 @@ void xe_vm_close_and_put(struct xe_vm *vm)
 	XE_WARN_ON(!list_empty(&vm->extobj.list));

drivers/gpu/drm/xe/xe_vm.c void xe_vm_close_and_put(struct xe_vm *vm) · line 1359

Quoted 10 lines
drivers/gpu/drm/xe/xe_vm.c +7 -0
 	up_write(&vm->lock);
 
+	mutex_lock(&xe->usm.lock);
+	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
+		xe->usm.num_vm_in_fault_mode--;
+	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
+		xe->usm.num_vm_in_non_fault_mode--;
+	mutex_unlock(&xe->usm.lock);
+
 	xe_vm_put(vm);

My concern is that something might be holding VM reference and hence VM will continue to live after we decrement the counts here and thus might endup in a new VM created in fault mode while the old VM in non-fault mode still not freed (and vice versa). Is that fine?

Niranjana

Quoted 24 lines
drivers/gpu/drm/xe/xe_vm.c +0 -8
 }
 
@@ -1393,18 +1400,10 @@ static void vm_destroy_work_func(struct work_struct *w)
 
 	drm_gpuva_manager_destroy(&vm->mgr);
 
-	mutex_lock(&xe->usm.lock);
-	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
-		xe->usm.num_vm_in_fault_mode--;
-	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
-		xe->usm.num_vm_in_non_fault_mode--;
-	mutex_unlock(&xe->usm.lock);
-
 	trace_xe_vm_free(vm);
 	dma_fence_put(vm->rebind_fence);
 	dma_resv_fini(&vm->resv);
 	kfree(vm);
-
 }
 
 void xe_vm_free(struct kref *ref)
Signature · 1 line
2.34.1
author maintainer
#3 lore raw
Quoted 6 lines · Niranjana Vishwanathapura
Quoted 5 lines · Matthew Brost

Rather waiting for the VM to be destroyed (all refs to VM go to zero), drop the fault mode counts when the VM is closed in xe_vm_close_and_put. This avoids a window where user space can create a faulting VM, close it, and a subsequent creation of a non-faulting VM fails.

Quoted 7 lines
Quoted 4 lines

v2 (Lucas): Drop VLK reference in commit message

This patch should probably go in the 'USM locking fixes' patch series?

Sure, likely need at respin all of these anyways.

Quoted 12 lines
Quoted 12 lines

---
drivers/gpu/drm/xe/xe_vm.c | 15 +++++++--------

1 file changed, 7 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +0 -0
index 68ab51f84bed..ab036a51d17e 100644
@@ -1349,6 +1349,13 @@ void xe_vm_close_and_put(struct xe_vm *vm)
 	XE_WARN_ON(!list_empty(&vm->extobj.list));
 	up_write(&vm->lock);
 
Quoted 14 lines

drivers/gpu/drm/xe/xe_vm.c void xe_vm_close_and_put(struct xe_vm *vm) · line 1359

Quoted 8 lines
drivers/gpu/drm/xe/xe_vm.c +7 -0
+	mutex_lock(&xe->usm.lock);
+	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
+		xe->usm.num_vm_in_fault_mode--;
+	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
+		xe->usm.num_vm_in_non_fault_mode--;
+	mutex_unlock(&xe->usm.lock);
+
 	xe_vm_put(vm);

My concern is that something might be holding VM reference and hence VM will continue to live after we decrement the counts here and thus might endup in a new VM created in fault mode while the old VM in non-fault mode still not freed (and vice versa). Is that fine?

The VM is effectively dead after this as we tear down all the page tables in this function, all page faults on the VM are just dropped, and likewise no execs can be done to this VM after this function. So yes,I believe this is safe.

Matt

Quoted 26 lines

Niranjana

Quoted 24 lines
drivers/gpu/drm/xe/xe_vm.c +0 -8
 }
 
@@ -1393,18 +1400,10 @@ static void vm_destroy_work_func(struct work_struct *w)
 
 	drm_gpuva_manager_destroy(&vm->mgr);
 
-	mutex_lock(&xe->usm.lock);
-	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
-		xe->usm.num_vm_in_fault_mode--;
-	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
-		xe->usm.num_vm_in_non_fault_mode--;
-	mutex_unlock(&xe->usm.lock);
-
 	trace_xe_vm_free(vm);
 	dma_fence_put(vm->rebind_fence);
 	dma_resv_fini(&vm->resv);
 	kfree(vm);
-
 }
 
 void xe_vm_free(struct kref *ref)
Signature · 1 line
2.34.1
Reviewed-by on v1 1/1
#4 lore raw
In reply to Matthew Brost
Quoted 50 lines · Matthew Brost

Rather waiting for the VM to be destroyed (all refs to VM go to zero), drop the fault mode counts when the VM is closed in xe_vm_close_and_put. This avoids a window where user space can create a faulting VM, close it, and a subsequent creation of a non-faulting VM fails.

v2 (Lucas): Drop VLK reference in commit message

---
  drivers/gpu/drm/xe/xe_vm.c | 15 +++++++--------
  1 file changed, 7 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_vm.c +7 -8
index 68ab51f84bed..ab036a51d17e 100644
@@ -1349,6 +1349,13 @@ void xe_vm_close_and_put(struct xe_vm *vm)
  	XE_WARN_ON(!list_empty(&vm->extobj.list));
  	up_write(&vm->lock);
 
+	mutex_lock(&xe->usm.lock);
+	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
+		xe->usm.num_vm_in_fault_mode--;
+	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
+		xe->usm.num_vm_in_non_fault_mode--;
+	mutex_unlock(&xe->usm.lock);
+
  	xe_vm_put(vm);
  }
 
@@ -1393,18 +1400,10 @@ static void vm_destroy_work_func(struct work_struct *w)
 
  	drm_gpuva_manager_destroy(&vm->mgr);
 
-	mutex_lock(&xe->usm.lock);
-	if (vm->flags & XE_VM_FLAG_FAULT_MODE)
-		xe->usm.num_vm_in_fault_mode--;
-	else if (!(vm->flags & XE_VM_FLAG_MIGRATION))
-		xe->usm.num_vm_in_non_fault_mode--;
-	mutex_unlock(&xe->usm.lock);
-
  	trace_xe_vm_free(vm);
  	dma_fence_put(vm->rebind_fence);
  	dma_resv_fini(&vm->resv);
  	kfree(vm);
-
  }
 
  void xe_vm_free(struct kref *ref)