Skip to contentSkip to conversation

drm/amdgpu: restrict userptr even more

PATCH In review Christian König 3 participants 13 messages Aug 30, 2017 – Aug 31, 2017 8 messages not in the archive

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/4 drm/amdgpu: restrict userptr even more

From: Christian König <[email protected]>

Don't allow them to be GEM imported into another process.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +6 -0
index d028806..e32a2b5 100644
@@ -112,7 +112,13 @@ int amdgpu_gem_object_open(struct drm_gem_object *obj,
 	struct amdgpu_fpriv *fpriv = file_priv->driver_priv;
 	struct amdgpu_vm *vm = &fpriv->vm;
 	struct amdgpu_bo_va *bo_va;
+	struct mm_struct *mm;
 	int r;
+
+	mm = amdgpu_ttm_tt_get_usermm(abo->tbo.ttm);
+	if (mm && mm != current->mm)
+		return -EPERM;
+
 	r = amdgpu_bo_reserve(abo, false);
 	if (r)
 		return r;
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1504105209-1558-1-git-send-email-deathsimple-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

#2 lore raw

PATCH 2/4 drm/amdgpu: add support for per VM BOs v2

From: Christian König <[email protected]>

Per VM BOs are handled like VM PDs and PTs. They are always valid and don't need to be specified in the BO lists.

v2: validate PDs/PTs first

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 79 ++++++++++++++++++++++++----------
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  5 ++-
 3 files changed, 60 insertions(+), 26 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
index f68ac56..48e18cc 100644
@@ -813,7 +813,7 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p)
 
 	}
 
-	r = amdgpu_vm_clear_moved(adev, vm, &p->job->sync);
+	r = amdgpu_vm_handle_moved(adev, vm, &p->job->sync);
 
 	if (amdgpu_vm_debug && p->bo_list) {
 		/* Invalidate all BOs to test for userspace bugs */
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +56 -23

147 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +3 -2
index c3753af..90b7741 100644
@@ -249,8 +249,9 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
 int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
 			  struct amdgpu_vm *vm,
 			  struct dma_fence **fence);
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-			  struct amdgpu_sync *sync);
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+			   struct amdgpu_vm *vm,
+			   struct amdgpu_sync *sync);
 int amdgpu_vm_bo_update(struct amdgpu_device *adev,
 			struct amdgpu_bo_va *bo_va,
 			bool clear);
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1504105209-1558-2-git-send-email-deathsimple-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

Reviewed-by
#4 lore raw

Re: [PATCH 2/4] drm/amdgpu: add support for per VM BOs v2

Some more thoughts inline, but nothing that should be addressed in this change.

Regards,
Felix

Quoted 82 lines · Christian König

From: Christian König <[email protected]>

Per VM BOs are handled like VM PDs and PTs. They are always valid and don't need to be specified in the BO lists.

v2: validate PDs/PTs first

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 79 ++++++++++++++++++++++++----------
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  5 ++-
 3 files changed, 60 insertions(+), 26 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
index f68ac56..48e18cc 100644
@@ -813,7 +813,7 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p)
 
 	}
 
-	r = amdgpu_vm_clear_moved(adev, vm, &p->job->sync);
+	r = amdgpu_vm_handle_moved(adev, vm, &p->job->sync);
 
 	if (amdgpu_vm_debug && p->bo_list) {
 		/* Invalidate all BOs to test for userspace bugs */
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +14 -7
index 4cdfb70..6cd20e7 100644
@@ -189,14 +189,18 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm,
 			spin_unlock(&glob->lru_lock);
 		}
 
-		if (vm->use_cpu_for_update) {
+		if (bo->tbo.type == ttm_bo_type_kernel &&
+		    vm->use_cpu_for_update) {
 			r = amdgpu_bo_kmap(bo, NULL);
 			if (r)
 				return r;
 		}
 
 		spin_lock(&vm->status_lock);
-		list_move(&bo_base->vm_status, &vm->relocated);
+		if (bo->tbo.type != ttm_bo_type_kernel)
+			list_move(&bo_base->vm_status, &vm->moved);
+		else
+			list_move(&bo_base->vm_status, &vm->relocated);
 	}
 	spin_unlock(&vm->status_lock);
 
@@ -1994,20 +1998,23 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
 }
 
 /**
- * amdgpu_vm_clear_moved - clear moved BOs in the PT
+ * amdgpu_vm_handle_moved - handle moved BOs in the PT
  *
  * @adev: amdgpu_device pointer
  * @vm: requested vm
+ * @sync: sync object to add fences to
  *
- * Make sure all moved BOs are cleared in the PT.
+ * Make sure all BOs which are moved are updated in the PTs.
  * Returns 0 for success.
  *
- * PTs have to be reserved and mutex must be locked!
+ * PTs have to be reserved!
  */
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-			    struct amdgpu_sync *sync)
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+			   struct amdgpu_vm *vm,
+			   struct amdgpu_sync *sync)
 {
 	struct amdgpu_bo_va *bo_va = NULL;
+	bool clear;
 	int r = 0;
 
 	spin_lock(&vm->status_lock);

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, · line 2029

Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -1
@@ -2016,7 +2023,10 @@ int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
 			struct amdgpu_bo_va, base.vm_status);
 		spin_unlock(&vm->status_lock);
 
-		r = amdgpu_vm_bo_update(adev, bo_va, true);
+		/* Per VM BOs never need to bo cleared in the page tables */
+		clear = bo_va->base.bo->tbo.resv != vm->root.base.bo->tbo.resv;
+
+		r = amdgpu_vm_bo_update(adev, bo_va, clear);

Just thinking out loud: Per VM BOs don't need to be cleared because they have separate lists for evicted and moved (but valid) BOs. If we had a similar thing for non-VM BOs (but without the automatic validation), this case could be optimized. Then moved but valid BOs could be updated in the page table instead of clearing them here and updating the page table again later.

Quoted 102 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +38 -15
 		if (r)
 			return r;
 
@@ -2068,6 +2078,37 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
 	return bo_va;
 }
 
+
+/**
+ * amdgpu_vm_bo_insert_mapping - insert a new mapping
+ *
+ * @adev: amdgpu_device pointer
+ * @bo_va: bo_va to store the address
+ * @mapping: the mapping to insert
+ *
+ * Insert a new mapping into all structures.
+ */
+static void amdgpu_vm_bo_insert_map(struct amdgpu_device *adev,
+				    struct amdgpu_bo_va *bo_va,
+				    struct amdgpu_bo_va_mapping *mapping)
+{
+	struct amdgpu_vm *vm = bo_va->base.vm;
+	struct amdgpu_bo *bo = bo_va->base.bo;
+
+	list_add(&mapping->list, &bo_va->invalids);
+	amdgpu_vm_it_insert(mapping, &vm->va);
+
+	if (mapping->flags & AMDGPU_PTE_PRT)
+		amdgpu_vm_prt_get(adev);
+
+	if (bo && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
+		spin_lock(&vm->status_lock);
+		list_move(&bo_va->base.vm_status, &vm->moved);
+		spin_unlock(&vm->status_lock);
+	}
+	trace_amdgpu_vm_bo_map(bo_va, mapping);
+}
+
 /**
  * amdgpu_vm_bo_map - map bo inside a vm
  *
@@ -2119,18 +2160,12 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
 	if (!mapping)
 		return -ENOMEM;
 
-	INIT_LIST_HEAD(&mapping->list);
 	mapping->start = saddr;
 	mapping->last = eaddr;
 	mapping->offset = offset;
 	mapping->flags = flags;
 
-	list_add(&mapping->list, &bo_va->invalids);
-	amdgpu_vm_it_insert(mapping, &vm->va);
-
-	if (flags & AMDGPU_PTE_PRT)
-		amdgpu_vm_prt_get(adev);
-	trace_amdgpu_vm_bo_map(bo_va, mapping);
+	amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
 	return 0;
 }
@@ -2157,7 +2192,6 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
 {
 	struct amdgpu_bo_va_mapping *mapping;
 	struct amdgpu_bo *bo = bo_va->base.bo;
-	struct amdgpu_vm *vm = bo_va->base.vm;
 	uint64_t eaddr;
 	int r;
 
@@ -2191,12 +2225,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
 	mapping->offset = offset;
 	mapping->flags = flags;
 
-	list_add(&mapping->list, &bo_va->invalids);
-	amdgpu_vm_it_insert(mapping, &vm->va);
-
-	if (flags & AMDGPU_PTE_PRT)
-		amdgpu_vm_prt_get(adev);
-	trace_amdgpu_vm_bo_map(bo_va, mapping);
+	amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
 	return 0;
 }
@@ -2411,7 +2440,11 @@ void amdgpu_vm_bo_invalidate(struct amdgpu_device *adev,
 		bo_base->moved = true;
 		if (evicted && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
 			spin_lock(&bo_base->vm->status_lock);
-			list_move(&bo_base->vm_status, &vm->evicted);
+			if (bo->tbo.type == ttm_bo_type_kernel)
+				list_move(&bo_base->vm_status, &vm->evicted);
+			else
+				list_move_tail(&bo_base->vm_status,
+					       &vm->evicted);
 			spin_unlock(&bo_base->vm->status_lock);
 			continue;
 		}
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +0 -0
index c3753af..90b7741 100644
@@ -249,8 +249,9 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
 int amdgpu_vm_clear_freed(struct amdgpu_device *adev,

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 257

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +3 -2
 			  struct amdgpu_vm *vm,
 			  struct dma_fence **fence);
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-			  struct amdgpu_sync *sync);
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+			   struct amdgpu_vm *vm,
+			   struct amdgpu_sync *sync);
 int amdgpu_vm_bo_update(struct amdgpu_device *adev,
 			struct amdgpu_bo_va *bo_va,
 			bool clear);

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 0f0164e2-2a6e-9805-54d2-c2220614657f-5C7GfCeVMHo@public.gmane.org

#6 lore raw

Re: [PATCH 2/4] drm/amdgpu: add support for per VM BOs v2

Am 31.08.2017 um 00:50 schrieb Felix Kuehling:

Quoted 95 lines

Some more thoughts inline, but nothing that should be addressed in this change.

Regards,
Felix

Quoted 85 lines · Christian König

From: Christian König <[email protected]>

Per VM BOs are handled like VM PDs and PTs. They are always valid and don't need to be specified in the BO lists.

v2: validate PDs/PTs first

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c |  2 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 79 ++++++++++++++++++++++++----------
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  5 ++-
  3 files changed, 60 insertions(+), 26 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
index f68ac56..48e18cc 100644
@@ -813,7 +813,7 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p)
 
  	}
 
-	r = amdgpu_vm_clear_moved(adev, vm, &p->job->sync);
+	r = amdgpu_vm_handle_moved(adev, vm, &p->job->sync);
 
  	if (amdgpu_vm_debug && p->bo_list) {
  		/* Invalidate all BOs to test for userspace bugs */
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +14 -7
index 4cdfb70..6cd20e7 100644
@@ -189,14 +189,18 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm,
  			spin_unlock(&glob->lru_lock);
  		}
 
-		if (vm->use_cpu_for_update) {
+		if (bo->tbo.type == ttm_bo_type_kernel &&
+		    vm->use_cpu_for_update) {
  			r = amdgpu_bo_kmap(bo, NULL);
  			if (r)
  				return r;
  		}
 
  		spin_lock(&vm->status_lock);
-		list_move(&bo_base->vm_status, &vm->relocated);
+		if (bo->tbo.type != ttm_bo_type_kernel)
+			list_move(&bo_base->vm_status, &vm->moved);
+		else
+			list_move(&bo_base->vm_status, &vm->relocated);
  	}
  	spin_unlock(&vm->status_lock);
 
@@ -1994,20 +1998,23 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
  }
 
  /**
- * amdgpu_vm_clear_moved - clear moved BOs in the PT
+ * amdgpu_vm_handle_moved - handle moved BOs in the PT
   *
   * @adev: amdgpu_device pointer
   * @vm: requested vm
+ * @sync: sync object to add fences to
   *
- * Make sure all moved BOs are cleared in the PT.
+ * Make sure all BOs which are moved are updated in the PTs.
   * Returns 0 for success.
   *
- * PTs have to be reserved and mutex must be locked!
+ * PTs have to be reserved!
   */
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-			    struct amdgpu_sync *sync)
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+			   struct amdgpu_vm *vm,
+			   struct amdgpu_sync *sync)
  {
  	struct amdgpu_bo_va *bo_va = NULL;
+	bool clear;
  	int r = 0;
 
  	spin_lock(&vm->status_lock);
@@ -2016,7 +2023,10 @@ int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
  			struct amdgpu_bo_va, base.vm_status);
  		spin_unlock(&vm->status_lock);
 
Quoted 7 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, · line 2029

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -1
-		r = amdgpu_vm_bo_update(adev, bo_va, true);
+		/* Per VM BOs never need to bo cleared in the page tables */
+		clear = bo_va->base.bo->tbo.resv != vm->root.base.bo->tbo.resv;
+
+		r = amdgpu_vm_bo_update(adev, bo_va, clear);

Just thinking out loud: Per VM BOs don't need to be cleared because they have separate lists for evicted and moved (but valid) BOs.

Actually the reason is that we know they are reserved together with the page directory.

Quoted 5 lines

If we had a
similar thing for non-VM BOs (but without the automatic validation), this case could be optimized. Then moved but valid BOs could be updated in the page table instead of clearing them here and updating the page table again later.

Well, that won't work because other BOs can change their location while we try to use their location.

But what I'm speculating about for a while now is to use trylock on the BOs here. That might not work all the time, but clearly most of the time.

Going to give that a try now,
Christian.

Quoted 109 lines
Quoted 109 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +38 -15
  		if (r)
  			return r;
 
@@ -2068,6 +2078,37 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
  	return bo_va;
  }
 
+
+/**
+ * amdgpu_vm_bo_insert_mapping - insert a new mapping
+ *
+ * @adev: amdgpu_device pointer
+ * @bo_va: bo_va to store the address
+ * @mapping: the mapping to insert
+ *
+ * Insert a new mapping into all structures.
+ */
+static void amdgpu_vm_bo_insert_map(struct amdgpu_device *adev,
+				    struct amdgpu_bo_va *bo_va,
+				    struct amdgpu_bo_va_mapping *mapping)
+{
+	struct amdgpu_vm *vm = bo_va->base.vm;
+	struct amdgpu_bo *bo = bo_va->base.bo;
+
+	list_add(&mapping->list, &bo_va->invalids);
+	amdgpu_vm_it_insert(mapping, &vm->va);
+
+	if (mapping->flags & AMDGPU_PTE_PRT)
+		amdgpu_vm_prt_get(adev);
+
+	if (bo && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
+		spin_lock(&vm->status_lock);
+		list_move(&bo_va->base.vm_status, &vm->moved);
+		spin_unlock(&vm->status_lock);
+	}
+	trace_amdgpu_vm_bo_map(bo_va, mapping);
+}
+
  /**
   * amdgpu_vm_bo_map - map bo inside a vm
   *
@@ -2119,18 +2160,12 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
  	if (!mapping)
  		return -ENOMEM;
 
-	INIT_LIST_HEAD(&mapping->list);
  	mapping->start = saddr;
  	mapping->last = eaddr;
  	mapping->offset = offset;
  	mapping->flags = flags;
 
-	list_add(&mapping->list, &bo_va->invalids);
-	amdgpu_vm_it_insert(mapping, &vm->va);
-
-	if (flags & AMDGPU_PTE_PRT)
-		amdgpu_vm_prt_get(adev);
-	trace_amdgpu_vm_bo_map(bo_va, mapping);
+	amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
  	return 0;
  }
@@ -2157,7 +2192,6 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
  {
  	struct amdgpu_bo_va_mapping *mapping;
  	struct amdgpu_bo *bo = bo_va->base.bo;
-	struct amdgpu_vm *vm = bo_va->base.vm;
  	uint64_t eaddr;
  	int r;
 
@@ -2191,12 +2225,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
  	mapping->offset = offset;
  	mapping->flags = flags;
 
-	list_add(&mapping->list, &bo_va->invalids);
-	amdgpu_vm_it_insert(mapping, &vm->va);
-
-	if (flags & AMDGPU_PTE_PRT)
-		amdgpu_vm_prt_get(adev);
-	trace_amdgpu_vm_bo_map(bo_va, mapping);
+	amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
  	return 0;
  }
@@ -2411,7 +2440,11 @@ void amdgpu_vm_bo_invalidate(struct amdgpu_device *adev,
  		bo_base->moved = true;
  		if (evicted && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
  			spin_lock(&bo_base->vm->status_lock);
-			list_move(&bo_base->vm_status, &vm->evicted);
+			if (bo->tbo.type == ttm_bo_type_kernel)
+				list_move(&bo_base->vm_status, &vm->evicted);
+			else
+				list_move_tail(&bo_base->vm_status,
+					       &vm->evicted);
  			spin_unlock(&bo_base->vm->status_lock);
  			continue;
  		}
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -2
index c3753af..90b7741 100644
@@ -249,8 +249,9 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
  int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
  			  struct amdgpu_vm *vm,
  			  struct dma_fence **fence);
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-			  struct amdgpu_sync *sync);
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+			   struct amdgpu_vm *vm,
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 257

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -0
+			   struct amdgpu_sync *sync);
  int amdgpu_vm_bo_update(struct amdgpu_device *adev,
  			struct amdgpu_bo_va *bo_va,
  			bool clear);

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 663386b1-c6e8-5fa8-16d4-b3b9a58aee2b-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

#8 lore raw

Re: [PATCH 2/4] drm/amdgpu: add support for per VM BOs v2

Quoted 5 lines
Quoted 2 lines

Just thinking out loud: Per VM BOs don't need to be cleared because they have separate lists for evicted and moved (but valid) BOs.

Actually the reason is that we know they are reserved together with the page directory.

I mentioned separate lists because that allows us to distinguish valid and invalid BOs. Non-VM BOs are all on the same list. So it's harder to tell, which ones are still valid but have moved, and which ones have been evicted and must be revalidated.

Regards,
Felix

________________________________________
From: Christian König <[email protected]>
Sent: Thursday, August 31, 2017 8:20:58 AM
To: Kuehling, Felix; [email protected]; Koenig, Christian
Subject: Re: [PATCH 2/4] drm/amdgpu: add support for per VM BOs v2

Am 31.08.2017 um 00:50 schrieb Felix Kuehling:

Quoted 95 lines

Some more thoughts inline, but nothing that should be addressed in this change.

Regards,
Felix

Quoted 85 lines · Christian König

From: Christian König <[email protected]>

Per VM BOs are handled like VM PDs and PTs. They are always valid and don't need to be specified in the BO lists.

v2: validate PDs/PTs first

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c |  2 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 79 ++++++++++++++++++++++++----------
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  5 ++-
  3 files changed, 60 insertions(+), 26 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
index f68ac56..48e18cc 100644
@@ -813,7 +813,7 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p)
 
     }
 
-    r = amdgpu_vm_clear_moved(adev, vm, &p->job->sync);
+    r = amdgpu_vm_handle_moved(adev, vm, &p->job->sync);
 
     if (amdgpu_vm_debug && p->bo_list) {
             /* Invalidate all BOs to test for userspace bugs */
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +14 -7
index 4cdfb70..6cd20e7 100644
@@ -189,14 +189,18 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm,
                     spin_unlock(&glob->lru_lock);
             }
 
-            if (vm->use_cpu_for_update) {
+            if (bo->tbo.type == ttm_bo_type_kernel &&
+                vm->use_cpu_for_update) {
                     r = amdgpu_bo_kmap(bo, NULL);
                     if (r)
                             return r;
             }
 
             spin_lock(&vm->status_lock);
-            list_move(&bo_base->vm_status, &vm->relocated);
+            if (bo->tbo.type != ttm_bo_type_kernel)
+                    list_move(&bo_base->vm_status, &vm->moved);
+            else
+                    list_move(&bo_base->vm_status, &vm->relocated);
     }
     spin_unlock(&vm->status_lock);
 
@@ -1994,20 +1998,23 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
  }
 
  /**
- * amdgpu_vm_clear_moved - clear moved BOs in the PT
+ * amdgpu_vm_handle_moved - handle moved BOs in the PT
   *
   * @adev: amdgpu_device pointer
   * @vm: requested vm
+ * @sync: sync object to add fences to
   *
- * Make sure all moved BOs are cleared in the PT.
+ * Make sure all BOs which are moved are updated in the PTs.
   * Returns 0 for success.
   *
- * PTs have to be reserved and mutex must be locked!
+ * PTs have to be reserved!
   */
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-                        struct amdgpu_sync *sync)
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+                       struct amdgpu_vm *vm,
+                       struct amdgpu_sync *sync)
  {
     struct amdgpu_bo_va *bo_va = NULL;
+    bool clear;
     int r = 0;
 
     spin_lock(&vm->status_lock);
@@ -2016,7 +2023,10 @@ int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
                     struct amdgpu_bo_va, base.vm_status);
             spin_unlock(&vm->status_lock);
 
Quoted 7 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, · line 2029

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -1
-            r = amdgpu_vm_bo_update(adev, bo_va, true);
+            /* Per VM BOs never need to bo cleared in the page tables */
+            clear = bo_va->base.bo->tbo.resv != vm->root.base.bo->tbo.resv;
+
+            r = amdgpu_vm_bo_update(adev, bo_va, clear);

Just thinking out loud: Per VM BOs don't need to be cleared because they have separate lists for evicted and moved (but valid) BOs.

Actually the reason is that we know they are reserved together with the page directory.

Quoted 5 lines

If we had a
similar thing for non-VM BOs (but without the automatic validation), this case could be optimized. Then moved but valid BOs could be updated in the page table instead of clearing them here and updating the page table again later.

Well, that won't work because other BOs can change their location while we try to use their location.

But what I'm speculating about for a while now is to use trylock on the BOs here. That might not work all the time, but clearly most of the time.

Going to give that a try now,
Christian.

Quoted 109 lines
Quoted 109 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +38 -15
             if (r)
                     return r;
 
@@ -2068,6 +2078,37 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
     return bo_va;
  }
 
+
+/**
+ * amdgpu_vm_bo_insert_mapping - insert a new mapping
+ *
+ * @adev: amdgpu_device pointer
+ * @bo_va: bo_va to store the address
+ * @mapping: the mapping to insert
+ *
+ * Insert a new mapping into all structures.
+ */
+static void amdgpu_vm_bo_insert_map(struct amdgpu_device *adev,
+                                struct amdgpu_bo_va *bo_va,
+                                struct amdgpu_bo_va_mapping *mapping)
+{
+    struct amdgpu_vm *vm = bo_va->base.vm;
+    struct amdgpu_bo *bo = bo_va->base.bo;
+
+    list_add(&mapping->list, &bo_va->invalids);
+    amdgpu_vm_it_insert(mapping, &vm->va);
+
+    if (mapping->flags & AMDGPU_PTE_PRT)
+            amdgpu_vm_prt_get(adev);
+
+    if (bo && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
+            spin_lock(&vm->status_lock);
+            list_move(&bo_va->base.vm_status, &vm->moved);
+            spin_unlock(&vm->status_lock);
+    }
+    trace_amdgpu_vm_bo_map(bo_va, mapping);
+}
+
  /**
   * amdgpu_vm_bo_map - map bo inside a vm
   *
@@ -2119,18 +2160,12 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
     if (!mapping)
             return -ENOMEM;
 
-    INIT_LIST_HEAD(&mapping->list);
     mapping->start = saddr;
     mapping->last = eaddr;
     mapping->offset = offset;
     mapping->flags = flags;
 
-    list_add(&mapping->list, &bo_va->invalids);
-    amdgpu_vm_it_insert(mapping, &vm->va);
-
-    if (flags & AMDGPU_PTE_PRT)
-            amdgpu_vm_prt_get(adev);
-    trace_amdgpu_vm_bo_map(bo_va, mapping);
+    amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
     return 0;
  }
@@ -2157,7 +2192,6 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
  {
     struct amdgpu_bo_va_mapping *mapping;
     struct amdgpu_bo *bo = bo_va->base.bo;
-    struct amdgpu_vm *vm = bo_va->base.vm;
     uint64_t eaddr;
     int r;
 
@@ -2191,12 +2225,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
     mapping->offset = offset;
     mapping->flags = flags;
 
-    list_add(&mapping->list, &bo_va->invalids);
-    amdgpu_vm_it_insert(mapping, &vm->va);
-
-    if (flags & AMDGPU_PTE_PRT)
-            amdgpu_vm_prt_get(adev);
-    trace_amdgpu_vm_bo_map(bo_va, mapping);
+    amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
     return 0;
  }
@@ -2411,7 +2440,11 @@ void amdgpu_vm_bo_invalidate(struct amdgpu_device *adev,
             bo_base->moved = true;
             if (evicted && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
                     spin_lock(&bo_base->vm->status_lock);
-                    list_move(&bo_base->vm_status, &vm->evicted);
+                    if (bo->tbo.type == ttm_bo_type_kernel)
+                            list_move(&bo_base->vm_status, &vm->evicted);
+                    else
+                            list_move_tail(&bo_base->vm_status,
+                                           &vm->evicted);
                     spin_unlock(&bo_base->vm->status_lock);
                     continue;
             }
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -2
index c3753af..90b7741 100644
@@ -249,8 +249,9 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
  int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
                       struct amdgpu_vm *vm,
                       struct dma_fence **fence);
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-                      struct amdgpu_sync *sync);
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+                       struct amdgpu_vm *vm,
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 257

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -0
+                       struct amdgpu_sync *sync);
  int amdgpu_vm_bo_update(struct amdgpu_device *adev,
                     struct amdgpu_bo_va *bo_va,
                     bool clear);

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): DM5PR1201MB023534198780B550F5E8CBC2929D0-grEf7a3NxMBd8L2jMOIKKmrFom/aUZj6nBOFsp37pqbUKgpGm//BTAC/[email protected]

#10 lore raw

Re: [PATCH 2/4] drm/amdgpu: add support for per VM BOs v2

Am 31.08.2017 um 15:20 schrieb Kuehling, Felix:

Quoted 5 lines
Quoted 4 lines
Quoted 2 lines

Just thinking out loud: Per VM BOs don't need to be cleared because they have separate lists for evicted and moved (but valid) BOs.

Actually the reason is that we know they are reserved together with the page directory.

I mentioned separate lists because that allows us to distinguish valid and invalid BOs. Non-VM BOs are all on the same list. So it's harder to tell, which ones are still valid but have moved, and which ones have been evicted and must be revalidated.

Actually that's not an issue, even evicted BOs are handled correctly here and shouldn't be revalidated just because somebody uses the VM.

It's just that we could as well set the new location instead of clearing the VM page tables of BOs which aren't part of the current working set.

Regards,
Christian.

Quoted 247 lines

Regards,
Felix

________________________________________
From: Christian König <[email protected]>
Sent: Thursday, August 31, 2017 8:20:58 AM
To: Kuehling, Felix; [email protected]; Koenig, Christian
Subject: Re: [PATCH 2/4] drm/amdgpu: add support for per VM BOs v2

Am 31.08.2017 um 00:50 schrieb Felix Kuehling:

Quoted 95 lines

Some more thoughts inline, but nothing that should be addressed in this change.

Regards,
Felix

Quoted 85 lines · Christian König

From: Christian König <[email protected]>

Per VM BOs are handled like VM PDs and PTs. They are always valid and don't need to be specified in the BO lists.

v2: validate PDs/PTs first

---
   drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c |  2 +-
   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 79 ++++++++++++++++++++++++----------
   drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h |  5 ++-
   3 files changed, 60 insertions(+), 26 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +1 -1
index f68ac56..48e18cc 100644
@@ -813,7 +813,7 @@ static int amdgpu_bo_vm_update_pte(struct amdgpu_cs_parser *p)
 
      }
 
-    r = amdgpu_vm_clear_moved(adev, vm, &p->job->sync);
+    r = amdgpu_vm_handle_moved(adev, vm, &p->job->sync);
 
      if (amdgpu_vm_debug && p->bo_list) {
              /* Invalidate all BOs to test for userspace bugs */
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +14 -7
index 4cdfb70..6cd20e7 100644
@@ -189,14 +189,18 @@ int amdgpu_vm_validate_pt_bos(struct amdgpu_device *adev, struct amdgpu_vm *vm,
                      spin_unlock(&glob->lru_lock);
              }
 
-            if (vm->use_cpu_for_update) {
+            if (bo->tbo.type == ttm_bo_type_kernel &&
+                vm->use_cpu_for_update) {
                      r = amdgpu_bo_kmap(bo, NULL);
                      if (r)
                              return r;
              }
 
              spin_lock(&vm->status_lock);
-            list_move(&bo_base->vm_status, &vm->relocated);
+            if (bo->tbo.type != ttm_bo_type_kernel)
+                    list_move(&bo_base->vm_status, &vm->moved);
+            else
+                    list_move(&bo_base->vm_status, &vm->relocated);
      }
      spin_unlock(&vm->status_lock);
 
@@ -1994,20 +1998,23 @@ int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
   }
 
   /**
- * amdgpu_vm_clear_moved - clear moved BOs in the PT
+ * amdgpu_vm_handle_moved - handle moved BOs in the PT
    *
    * @adev: amdgpu_device pointer
    * @vm: requested vm
+ * @sync: sync object to add fences to
    *
- * Make sure all moved BOs are cleared in the PT.
+ * Make sure all BOs which are moved are updated in the PTs.
    * Returns 0 for success.
    *
- * PTs have to be reserved and mutex must be locked!
+ * PTs have to be reserved!
    */
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-                        struct amdgpu_sync *sync)
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+                       struct amdgpu_vm *vm,
+                       struct amdgpu_sync *sync)
   {
      struct amdgpu_bo_va *bo_va = NULL;
+    bool clear;
      int r = 0;
 
      spin_lock(&vm->status_lock);
@@ -2016,7 +2023,10 @@ int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
                      struct amdgpu_bo_va, base.vm_status);
              spin_unlock(&vm->status_lock);
 
Quoted 7 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm, · line 2029

Quoted 5 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +4 -1
-            r = amdgpu_vm_bo_update(adev, bo_va, true);
+            /* Per VM BOs never need to bo cleared in the page tables */
+            clear = bo_va->base.bo->tbo.resv != vm->root.base.bo->tbo.resv;
+
+            r = amdgpu_vm_bo_update(adev, bo_va, clear);

Just thinking out loud: Per VM BOs don't need to be cleared because they have separate lists for evicted and moved (but valid) BOs.

Actually the reason is that we know they are reserved together with the page directory.

Quoted 5 lines

If we had a
similar thing for non-VM BOs (but without the automatic validation), this case could be optimized. Then moved but valid BOs could be updated in the page table instead of clearing them here and updating the page table again later.

Well, that won't work because other BOs can change their location while we try to use their location.

But what I'm speculating about for a while now is to use trylock on the BOs here. That might not work all the time, but clearly most of the time.

Going to give that a try now,
Christian.

Quoted 108 lines
Quoted 108 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c +38 -15
              if (r)
                      return r;
 
@@ -2068,6 +2078,37 @@ struct amdgpu_bo_va *amdgpu_vm_bo_add(struct amdgpu_device *adev,
      return bo_va;
   }
 
+
+/**
+ * amdgpu_vm_bo_insert_mapping - insert a new mapping
+ *
+ * @adev: amdgpu_device pointer
+ * @bo_va: bo_va to store the address
+ * @mapping: the mapping to insert
+ *
+ * Insert a new mapping into all structures.
+ */
+static void amdgpu_vm_bo_insert_map(struct amdgpu_device *adev,
+                                struct amdgpu_bo_va *bo_va,
+                                struct amdgpu_bo_va_mapping *mapping)
+{
+    struct amdgpu_vm *vm = bo_va->base.vm;
+    struct amdgpu_bo *bo = bo_va->base.bo;
+
+    list_add(&mapping->list, &bo_va->invalids);
+    amdgpu_vm_it_insert(mapping, &vm->va);
+
+    if (mapping->flags & AMDGPU_PTE_PRT)
+            amdgpu_vm_prt_get(adev);
+
+    if (bo && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
+            spin_lock(&vm->status_lock);
+            list_move(&bo_va->base.vm_status, &vm->moved);
+            spin_unlock(&vm->status_lock);
+    }
+    trace_amdgpu_vm_bo_map(bo_va, mapping);
+}
+
   /**
    * amdgpu_vm_bo_map - map bo inside a vm
    *
@@ -2119,18 +2160,12 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
      if (!mapping)
              return -ENOMEM;
 
-    INIT_LIST_HEAD(&mapping->list);
      mapping->start = saddr;
      mapping->last = eaddr;
      mapping->offset = offset;
      mapping->flags = flags;
 
-    list_add(&mapping->list, &bo_va->invalids);
-    amdgpu_vm_it_insert(mapping, &vm->va);
-
-    if (flags & AMDGPU_PTE_PRT)
-            amdgpu_vm_prt_get(adev);
-    trace_amdgpu_vm_bo_map(bo_va, mapping);
+    amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
      return 0;
   }
@@ -2157,7 +2192,6 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
   {
      struct amdgpu_bo_va_mapping *mapping;
      struct amdgpu_bo *bo = bo_va->base.bo;
-    struct amdgpu_vm *vm = bo_va->base.vm;
      uint64_t eaddr;
      int r;
 
@@ -2191,12 +2225,7 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
      mapping->offset = offset;
      mapping->flags = flags;
 
-    list_add(&mapping->list, &bo_va->invalids);
-    amdgpu_vm_it_insert(mapping, &vm->va);
-
-    if (flags & AMDGPU_PTE_PRT)
-            amdgpu_vm_prt_get(adev);
-    trace_amdgpu_vm_bo_map(bo_va, mapping);
+    amdgpu_vm_bo_insert_map(adev, bo_va, mapping);
 
      return 0;
   }
@@ -2411,7 +2440,11 @@ void amdgpu_vm_bo_invalidate(struct amdgpu_device *adev,
              bo_base->moved = true;
              if (evicted && bo->tbo.resv == vm->root.base.bo->tbo.resv) {
                      spin_lock(&bo_base->vm->status_lock);
-                    list_move(&bo_base->vm_status, &vm->evicted);
+                    if (bo->tbo.type == ttm_bo_type_kernel)
+                            list_move(&bo_base->vm_status, &vm->evicted);
+                    else
+                            list_move_tail(&bo_base->vm_status,
+                                           &vm->evicted);
                      spin_unlock(&bo_base->vm->status_lock);
                      continue;
              }
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +2 -2
index c3753af..90b7741 100644
@@ -249,8 +249,9 @@ int amdgpu_vm_update_directories(struct amdgpu_device *adev,
   int amdgpu_vm_clear_freed(struct amdgpu_device *adev,
                        struct amdgpu_vm *vm,
                        struct dma_fence **fence);
-int amdgpu_vm_clear_moved(struct amdgpu_device *adev, struct amdgpu_vm *vm,
-                      struct amdgpu_sync *sync);
+int amdgpu_vm_handle_moved(struct amdgpu_device *adev,
+                       struct amdgpu_vm *vm,
Quoted 8 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h int amdgpu_vm_update_directories(struct amdgpu_device *adev, · line 257

Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h +1 -0
+                       struct amdgpu_sync *sync);
   int amdgpu_vm_bo_update(struct amdgpu_device *adev,
                      struct amdgpu_bo_va *bo_va,
                      bool clear);

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Quoted 4 lines

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#11 lore raw

PATCH 3/4 drm/amdgpu: add IOCTL interface for per VM BOs v3

In reply to a message not in the archive

From: Christian König <[email protected]>

Add the IOCTL interface so that applications can allocate per VM BOs.

Still WIP since not all corner cases are tested yet, but this reduces average CS overhead for 10K BOs from 21ms down to 48us.

v2: add some extra checks, remove the WIP tag
v3: rename new flag to AMDGPU_GEM_CREATE_VM_ALWAYS_VALID

---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h       |  7 ++--
 drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c    |  2 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c   | 63 ++++++++++++++++++++++---------
 drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c |  3 +-
 include/uapi/drm/amdgpu_drm.h             |  2 +
 5 files changed, 55 insertions(+), 22 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +4 -3
index b1e817c..21cab36 100644
@@ -457,9 +457,10 @@ struct amdgpu_sa_bo {
  */
 void amdgpu_gem_force_release(struct amdgpu_device *adev);
 int amdgpu_gem_object_create(struct amdgpu_device *adev, unsigned long size,
-				int alignment, u32 initial_domain,
-				u64 flags, bool kernel,
-				struct drm_gem_object **obj);
+			     int alignment, u32 initial_domain,
+			     u64 flags, bool kernel,
+			     struct reservation_object *resv,
+			     struct drm_gem_object **obj);
 
 int amdgpu_mode_dumb_create(struct drm_file *file_priv,
 			    struct drm_device *dev,
drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c +1 -1
index 0e907ea..7256f83 100644
@@ -144,7 +144,7 @@ static int amdgpufb_create_pinned_object(struct amdgpu_fbdev *rfbdev,
 				       AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED |
 				       AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
 				       AMDGPU_GEM_CREATE_VRAM_CLEARED,
-				       true, &gobj);
+				       true, NULL, &gobj);
 	if (ret) {
 		pr_err("failed to allocate framebuffer (%d)\n", aligned_size);
 		return -ENOMEM;
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +46 -17

171 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +2 -1
index 5b3f928..7e08264 100644
@@ -136,7 +136,8 @@ struct dma_buf *amdgpu_gem_prime_export(struct drm_device *dev,
 {
 	struct amdgpu_bo *bo = gem_to_amdgpu_bo(gobj);
 
-	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm))
+	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) ||
+	    bo->flags & AMDGPU_GEM_CREATE_VM_ALWAYS_VALID)
 		return ERR_PTR(-EPERM);
 
 	return drm_gem_prime_export(dev, gobj, flags);
include/uapi/drm/amdgpu_drm.h +2 -0
index d0ee739..fe839d9 100644
@@ -89,6 +89,8 @@ extern "C" {
 #define AMDGPU_GEM_CREATE_SHADOW		(1 << 4)
 /* Flag that allocating the BO should use linear VRAM */
 #define AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS	(1 << 5)
+/* Flag that BO is always valid in this VM */
+#define AMDGPU_GEM_CREATE_VM_ALWAYS_VALID	(1 << 6)
 
 struct drm_amdgpu_gem_create_in  {
 	/** the requested memory size */
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1504105209-1558-3-git-send-email-deathsimple-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

#13 lore raw

Re: [PATCH 3/4] drm/amdgpu: add IOCTL interface for per VM BOs v3

drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c struct dma_buf *amdgpu_gem_prime_export(struct drm_device *dev, · line 143

Quoted 12 lines · Christian König
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +2 -1
@@ -136,7 +136,8 @@ struct dma_buf *amdgpu_gem_prime_export(struct drm_device *dev,
 {
 	struct amdgpu_bo *bo = gem_to_amdgpu_bo(gobj);
 
-	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm))
+	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) ||
+	    bo->flags & AMDGPU_GEM_CREATE_VM_ALWAYS_VALID)
 		return ERR_PTR(-EPERM);
 
 	return drm_gem_prime_export(dev, gobj, flags);

Is this limitation necessary? If it weren't for this, I'd use per-VM BOs for KFD, because we always need to validate all our BOs when we restore from an eviction anyway. But we need to be able to support buffer sharing at the same time. And we don't know which buffers an application plans to shared at allocation time.

Either way, we could address this later. This patch is Reviewed-by:
Felix Kuehling <[email protected]>

Regards,
Felix

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 5ec19bf2-75a2-91f2-e3d7-aeb5719f53e7-5C7GfCeVMHo@public.gmane.org

#15 lore raw

Re: [PATCH 3/4] drm/amdgpu: add IOCTL interface for per VM BOs v3

Am 31.08.2017 um 00:55 schrieb Felix Kuehling:

Quoted 7 lines · Christian König
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +0 -0
@@ -136,7 +136,8 @@ struct dma_buf *amdgpu_gem_prime_export(struct drm_device *dev,
  {
  	struct amdgpu_bo *bo = gem_to_amdgpu_bo(gobj);
 
Quoted 7 lines

drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c struct dma_buf *amdgpu_gem_prime_export(struct drm_device *dev, · line 143

Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_prime.c +2 -1
-	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm))
+	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm) ||
+	    bo->flags & AMDGPU_GEM_CREATE_VM_ALWAYS_VALID)
  		return ERR_PTR(-EPERM);
 
  	return drm_gem_prime_export(dev, gobj, flags);

Is this limitation necessary?

Not really, there are just two issue why I would like to have it at least for the GFX side:

1. The root page directory stays allocated while we wait for all the per VM BOs to be destroyed.

This can be fixed with reservation object reference counting, but I didn't had time yet to implement this.

2. The implicit fencing causes a bunch of problems for displayable textures. So any sharing of such BOs with the X server can cause undefined delays.

We need to fix this on the userspace level and add something like a AMDGPU_GEM_CREATE_UNSYCED flag for BO allocation.

Regards,
Christian.

Quoted 9 lines

If it weren't for this, I'd use per-VM BOs
for KFD, because we always need to validate all our BOs when we restore from an eviction anyway. But we need to be able to support buffer sharing at the same time. And we don't know which buffers an application plans to shared at allocation time.

Either way, we could address this later. This patch is Reviewed-by:
Felix Kuehling <[email protected]>

Quoted 7 lines

Regards,
Felix

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#16 lore raw

PATCH 4/4 drm/amdgpu: bump version for support of local BOs

In reply to a message not in the archive

From: Christian König <[email protected]>

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +2 -1
index 21116fc..5035305c 100644
@@ -69,9 +69,10 @@
  * - 3.17.0 - Add AMDGPU_NUM_VRAM_CPU_PAGE_FAULTS.
  * - 3.18.0 - Export gpu always on cu bitmap
  * - 3.19.0 - Add support for UVD MJPEG decode
+ * - 3.20.0 - Add support for local BOs
  */
 #define KMS_DRIVER_MAJOR	3
-#define KMS_DRIVER_MINOR	19
+#define KMS_DRIVER_MINOR	20
 #define KMS_DRIVER_PATCHLEVEL	0
 
 int amdgpu_vram_limit = 0;
Signature · 6 lines
2.7.4

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 1504105209-1558-4-git-send-email-deathsimple-ANTagKRnAhcb1SvskN2V4Q@public.gmane.org

Reviewed-by
#18 lore raw

Re: [PATCH 4/4] drm/amdgpu: bump version for support of local BOs

Quoted 13 lines · Christian König

From: Christian König <[email protected]>

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +0 -0
index 21116fc..5035305c 100644
@@ -69,9 +69,10 @@
  * - 3.17.0 - Add AMDGPU_NUM_VRAM_CPU_PAGE_FAULTS.

drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c line 78

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +2 -1
  * - 3.18.0 - Export gpu always on cu bitmap
  * - 3.19.0 - Add support for UVD MJPEG decode
+ * - 3.20.0 - Add support for local BOs
  */
 #define KMS_DRIVER_MAJOR	3
-#define KMS_DRIVER_MINOR	19
+#define KMS_DRIVER_MINOR	20
 #define KMS_DRIVER_PATCHLEVEL	0
 
 int amdgpu_vram_limit = 0;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by
#19 lore raw
In reply to a message not in the archive

John and I were wondering whether this is redundant because amdgpu_gem_prime_export already prevents exporting of userptr BOs. However, I just realized this is not for prime buffer sharing but the GEM_FLINK buffer sharing API. It doesn't have a driver callback in GEM_FLINK, so the only chance to prevent sharing userptrs is in GEM_OPEN.

As I understand it, this doesn't add any new fundamental limitation, just enforces it in a different situation.

Regards,
Felix

Quoted 17 lines · Christian König

From: Christian König <[email protected]>

Don't allow them to be GEM imported into another process.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 6 ++++++
 1 file changed, 6 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +0 -0
index d028806..e32a2b5 100644
@@ -112,7 +112,13 @@ int amdgpu_gem_object_open(struct drm_gem_object *obj,
 	struct amdgpu_fpriv *fpriv = file_priv->driver_priv;
 	struct amdgpu_vm *vm = &fpriv->vm;
 	struct amdgpu_bo_va *bo_va;

drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c int amdgpu_gem_object_open(struct drm_gem_object *obj, · line 124

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +6 -0
+	struct mm_struct *mm;
 	int r;
+
+	mm = amdgpu_ttm_tt_get_usermm(abo->tbo.ttm);
+	if (mm && mm != current->mm)
+		return -EPERM;
+
 	r = amdgpu_bo_reserve(abo, false);
 	if (r)
 		return r;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Acked-by Chunming Zhou
#20 lore raw
In reply to a message not in the archive
Quoted 17 lines · Christian König

From: Christian König <[email protected]>

Don't allow them to be GEM imported into another process.

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c | 6 ++++++
  1 file changed, 6 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +0 -0
index d028806..e32a2b5 100644
@@ -112,7 +112,13 @@ int amdgpu_gem_object_open(struct drm_gem_object *obj,
  	struct amdgpu_fpriv *fpriv = file_priv->driver_priv;
  	struct amdgpu_vm *vm = &fpriv->vm;
  	struct amdgpu_bo_va *bo_va;

drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c int amdgpu_gem_object_open(struct drm_gem_object *obj, · line 124

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c +6 -0
+	struct mm_struct *mm;
  	int r;
+
+	mm = amdgpu_ttm_tt_get_usermm(abo->tbo.ttm);
+	if (mm && mm != current->mm)
+		return -EPERM;
+
  	r = amdgpu_bo_reserve(abo, false);
  	if (r)
  		return r;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx