Skip to contentSkip to conversation

drm/ttm: fix busy memory to fail other user v3

PATCH In review Chunming Zhou 4 participants 5 messages Apr 25, 2019 – Apr 26, 2019

Subsystems (5)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 drm/ttm: fix busy memory to fail other user v3

heavy gpu job could occupy memory long time, which lead other user fail to get memory.

basically pick up Christian idea:

1. Reserve the BO in DC using a ww_mutex ticket (trivial).
2. If we then run into this EBUSY condition in TTM check if the BO we need memory for (or rather the ww_mutex of its reservation object) has a ticket assigned.
3. If we have a ticket we grab a reference to the first BO on the LRU, drop the LRU lock and try to grab the reservation lock with the ticket.
4. If getting the reservation lock with the ticket succeeded we check if the BO is still the first one on the LRU in question (the BO could have moved).
5. If the BO is still the first one on the LRU in question we try to evict it as we would evict any other BO.
6. If any of the "If's" above fail we just back off and return -EBUSY.

v2: fix some minor check
v3: address Christian v2 comments.

Change-Id I21423fb922f885465f13833c41df1e134364a8e7
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |  7 ++-
 .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 21 ++++++--
 drivers/gpu/drm/ttm/ttm_bo.c                  | 48 +++++++++++++++++--
 3 files changed, 67 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +6 -1
index affde72b44db..015bf62277d0 100644
@@ -811,7 +811,12 @@ int amdgpu_bo_pin_restricted(struct amdgpu_bo *bo, u32 domain,
 			     u64 min_offset, u64 max_offset)
 {
 	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-	struct ttm_operation_ctx ctx = { false, false };
+	struct ttm_operation_ctx ctx = {
+		.interruptible = false,
+		.no_wait_gpu = false,
+		.resv = bo->tbo.resv,
+		.flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+	};
 	int r, i;
 
 	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm))
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +18 -3
index a5cacf846e1b..2957ac38dcb0 100644
@@ -4101,6 +4101,9 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane,
 	struct amdgpu_device *adev;
 	struct amdgpu_bo *rbo;
 	struct dm_plane_state *dm_plane_state_new, *dm_plane_state_old;
+	struct list_head list, duplicates;
+	struct ttm_validate_buffer tv;
+	struct ww_acquire_ctx ticket;
 	uint64_t tiling_flags;
 	uint32_t domain;
 	int r;
@@ -4120,6 +4123,18 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane,
 	r = amdgpu_bo_reserve(rbo, false);
 	if (unlikely(r != 0))
 		return r;
+	INIT_LIST_HEAD(&list);
+	INIT_LIST_HEAD(&duplicates);
+
+	tv.bo = &rbo->tbo;
+	tv.num_shared = 1;
+	list_add(&tv.head, &list);
+
+	r = ttm_eu_reserve_buffers(&ticket, &list, false, &duplicates);
+	if (r) {
+		dev_err(adev->dev, "fail to reserve bo (%d)\n", r);
+		return r;
+	}
 
 	if (plane->type != DRM_PLANE_TYPE_CURSOR)
 		domain = amdgpu_display_supported_domains(adev);
@@ -4130,21 +4145,21 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane,
 	if (unlikely(r != 0)) {
 		if (r != -ERESTARTSYS)
 			DRM_ERROR("Failed to pin framebuffer with error %d\n", r);
-		amdgpu_bo_unreserve(rbo);
+		ttm_eu_backoff_reservation(&ticket, &list);
 		return r;
 	}
 
 	r = amdgpu_ttm_alloc_gart(&rbo->tbo);
 	if (unlikely(r != 0)) {
 		amdgpu_bo_unpin(rbo);
-		amdgpu_bo_unreserve(rbo);
+		ttm_eu_backoff_reservation(&ticket, &list);
 		DRM_ERROR("%p bind failed\n", rbo);
 		return r;
 	}
 
 	amdgpu_bo_get_tiling_flags(rbo, &tiling_flags);
 
-	amdgpu_bo_unreserve(rbo);
+	ttm_eu_backoff_reservation(&ticket, &list);
 
 	afb->address = amdgpu_bo_gpu_offset(rbo);
 
drivers/gpu/drm/ttm/ttm_bo.c +43 -5

95 lines not on this page · raw message

Signature · 6 lines
2.17.1

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#1 lore raw

Am 25.04.19 um 09:39 schrieb Chunming Zhou:

Quoted 5 lines

heavy gpu job could occupy memory long time, which lead other user fail to get memory.

basically pick up Christian idea:

1. Reserve the BO in DC using a ww_mutex ticket (trivial).

Any reason you don't want to split this into a separate patch?

Quoted 22 lines

2. If we then run into this EBUSY condition in TTM check if the BO we need memory for (or rather the ww_mutex of its reservation object) has a ticket assigned.
3. If we have a ticket we grab a reference to the first BO on the LRU, drop the LRU lock and try to grab the reservation lock with the ticket.
4. If getting the reservation lock with the ticket succeeded we check if the BO is still the first one on the LRU in question (the BO could have moved).
5. If the BO is still the first one on the LRU in question we try to evict it as we would evict any other BO.
6. If any of the "If's" above fail we just back off and return -EBUSY.

v2: fix some minor check
v3: address Christian v2 comments.

Change-Id I21423fb922f885465f13833c41df1e134364a8e7
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |  7 ++-
  .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 21 ++++++--
  drivers/gpu/drm/ttm/ttm_bo.c                  | 48 +++++++++++++++++--
  3 files changed, 67 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
index affde72b44db..015bf62277d0 100644

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_pin_restricted(struct amdgpu_bo *bo, u32 domain, · line 819

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +6 -1
@@ -811,7 +811,12 @@ int amdgpu_bo_pin_restricted(struct amdgpu_bo *bo, u32 domain,
  			     u64 min_offset, u64 max_offset)
  {
  	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-	struct ttm_operation_ctx ctx = { false, false };
+	struct ttm_operation_ctx ctx = {
+		.interruptible = false,
+		.no_wait_gpu = false,
+		.resv = bo->tbo.resv,
+		.flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+	};

Please completely drop this chunk.

If we allow evicting BOs from the same process during pinning it could lead to some unforeseen side effects.

Quoted 48 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
  	int r, i;
 
  	if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm))
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +16 -1
index a5cacf846e1b..2957ac38dcb0 100644
@@ -4101,6 +4101,9 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane,
  	struct amdgpu_device *adev;
  	struct amdgpu_bo *rbo;
  	struct dm_plane_state *dm_plane_state_new, *dm_plane_state_old;
+	struct list_head list, duplicates;
+	struct ttm_validate_buffer tv;
+	struct ww_acquire_ctx ticket;
  	uint64_t tiling_flags;
  	uint32_t domain;
  	int r;
@@ -4120,6 +4123,18 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane,
  	r = amdgpu_bo_reserve(rbo, false);
  	if (unlikely(r != 0))
  		return r;
+	INIT_LIST_HEAD(&list);
+	INIT_LIST_HEAD(&duplicates);
+
+	tv.bo = &rbo->tbo;
+	tv.num_shared = 1;
+	list_add(&tv.head, &list);
+
+	r = ttm_eu_reserve_buffers(&ticket, &list, false, &duplicates);
+	if (r) {
+		dev_err(adev->dev, "fail to reserve bo (%d)\n", r);
+		return r;
+	}
 
  	if (plane->type != DRM_PLANE_TYPE_CURSOR)
  		domain = amdgpu_display_supported_domains(adev);
@@ -4130,21 +4145,21 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane,
  	if (unlikely(r != 0)) {
  		if (r != -ERESTARTSYS)
  			DRM_ERROR("Failed to pin framebuffer with error %d\n", r);
-		amdgpu_bo_unreserve(rbo);
+		ttm_eu_backoff_reservation(&ticket, &list);
  		return r;
  	}
 
  	r = amdgpu_ttm_alloc_gart(&rbo->tbo);
  	if (unlikely(r != 0)) {
  		amdgpu_bo_unpin(rbo);

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c static int dm_plane_helper_prepare_fb(struct drm_plane *plane, · line 4162

Quoted 10 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +2 -2
-		amdgpu_bo_unreserve(rbo);
+		ttm_eu_backoff_reservation(&ticket, &list);
  		DRM_ERROR("%p bind failed\n", rbo);
  		return r;
  	}
 
  	amdgpu_bo_get_tiling_flags(rbo, &tiling_flags);
 
-	amdgpu_bo_unreserve(rbo);
+	ttm_eu_backoff_reservation(&ticket, &list);

This part looks good to me, but as noted above should probably be in a separate patch.

Quoted 63 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +0 -0
  	afb->address = amdgpu_bo_gpu_offset(rbo);
 
drivers/gpu/drm/ttm/ttm_bo.c +12 -3
index 8502b3ed2d88..dbcf958d1b43 100644
@@ -766,11 +766,12 @@ EXPORT_SYMBOL(ttm_bo_eviction_valuable);
   * b. Otherwise, trylock it.
   */
  static bool ttm_bo_evict_swapout_allowable(struct ttm_buffer_object *bo,
-			struct ttm_operation_ctx *ctx, bool *locked)
+			struct ttm_operation_ctx *ctx, bool *locked, bool *busy)
  {
  	bool ret = false;
 
  	*locked = false;
+	*busy = false;
  	if (bo->resv == ctx->resv) {
  		reservation_object_assert_held(bo->resv);
  		if (ctx->flags & TTM_OPT_FLAG_ALLOW_RES_EVICT
@@ -779,6 +780,8 @@ static bool ttm_bo_evict_swapout_allowable(struct ttm_buffer_object *bo,
  	} else {
  		*locked = reservation_object_trylock(bo->resv);
  		ret = *locked;
+		if (!ret)
+			*busy = true;
  	}
 
  	return ret;
@@ -791,7 +794,7 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev,
  {
  	struct ttm_bo_global *glob = bdev->glob;
  	struct ttm_mem_type_manager *man = &bdev->man[mem_type];
-	struct ttm_buffer_object *bo = NULL;
+	struct ttm_buffer_object *bo = NULL, *first_bo = NULL;
  	bool locked = false;
  	unsigned i;
  	int ret;
@@ -799,8 +802,13 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev,
  	spin_lock(&glob->lru_lock);
  	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
  		list_for_each_entry(bo, &man->lru[i], lru) {
-			if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked))
+			bool busy = false;
+			if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked,
+							    &busy)) {
+				if (!first_bo && busy)
+					first_bo = bo;
  				continue;
+			}
 
  			if (place && !bdev->driver->eviction_valuable(bo,
  								      place)) {
@@ -808,6 +816,7 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev,
  					reservation_object_unlock(bo->resv);
  				continue;
  			}
+
  			break;
  		}
 

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_mem_evict_first(struct ttm_bo_device *bdev, · line 832

Quoted 5 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
@@ -820,7 +829,33 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev,
 
  	if (!bo) {
  		spin_unlock(&glob->lru_lock);
-		return -EBUSY;
+		if (!first_bo || !ctx || !ctx->resv || !ctx->resv->lock.ctx)

Ah, now I see why you need the reservation object here.

Quoted 5 lines
drivers/gpu/drm/ttm/ttm_bo.c +5 -0
+			return -EBUSY;
+		if (ctx->interruptible)
+			ret = ww_mutex_lock_interruptible(&bo->resv->lock,
+							  ctx->resv->lock.ctx);
+		else

drivers/gpu/drm/ttm/ttm_bo.c static int ttm_mem_evict_first(struct ttm_bo_device *bdev, · line 847

Quoted 10 lines
drivers/gpu/drm/ttm/ttm_bo.c +10 -0
+			ret = ww_mutex_lock(&bo->resv->lock, ctx->resv->lock.ctx);
+		if (ret)
+			return ret;
+		spin_lock(&glob->lru_lock);
+		for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
+			if (list_empty(&man->lru[i]))
+				continue;
+			bo = list_first_entry(&man->lru[i],
+					      struct ttm_buffer_object,
+					      lru);

You now need to check all BOs on the LRU, cause the first one where a trylock failed is not necessarily the first one on the list.

Quoted 15 lines
drivers/gpu/drm/ttm/ttm_bo.c +11 -0
+
+			break;
+		}
+		/* verify if BO have been moved */
+		if (first_bo != bo) {
+			spin_unlock(&glob->lru_lock);
+			ww_mutex_unlock(&bo->resv->lock);
+			return -EBUSY;
+		}
+		spin_unlock(&glob->lru_lock);
+		/* ok, pick up first busy BO to wait to evict */
  	}
 
  	kref_get(&bo->list_kref);

drivers/gpu/drm/ttm/ttm_bo.c int ttm_bo_swapout(struct ttm_bo_global *glob, struct ttm_operation_ctx *ctx) · line 1822

Quoted 5 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
@@ -1784,7 +1819,10 @@ int ttm_bo_swapout(struct ttm_bo_global *glob, struct ttm_operation_ctx *ctx)
  	spin_lock(&glob->lru_lock);
  	for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
  		list_for_each_entry(bo, &glob->swap_lru[i], swap) {
-			if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked)) {
+			bool busy = false;

Better make the parameter optional.

Christian.

drivers/gpu/drm/ttm/ttm_bo.c int ttm_bo_swapout(struct ttm_bo_global *glob, struct ttm_operation_ctx *ctx) · line 1828

Quoted 6 lines
drivers/gpu/drm/ttm/ttm_bo.c +3 -0
+
+			if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked,
+							   &busy)) {
  				ret = 0;
  				break;
  			}

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#2 lore raw
Quoted 8 lines · Christian König

Am 25.04.19 um 09:39 schrieb Chunming Zhou:

Quoted 5 lines

heavy gpu job could occupy memory long time, which lead other user fail to get memory.

basically pick up Christian idea:

1. Reserve the BO in DC using a ww_mutex ticket (trivial).

Any reason you don't want to split this into a separate patch?

Will do that after locking down solution, just make the solution in one patch to review conveniently.

Quoted 37 lines
Quoted 24 lines

2. If we then run into this EBUSY condition in TTM check if the BO we need memory for (or rather the ww_mutex of its reservation object) has a ticket assigned.
3. If we have a ticket we grab a reference to the first BO on the LRU, drop the LRU lock and try to grab the reservation lock with the ticket.
4. If getting the reservation lock with the ticket succeeded we check if the BO is still the first one on the LRU in question (the BO could have moved).
5. If the BO is still the first one on the LRU in question we try to evict it as we would evict any other BO.
6. If any of the "If's" above fail we just back off and return -EBUSY.

v2: fix some minor check
v3: address Christian v2 comments.

Change-Id I21423fb922f885465f13833c41df1e134364a8e7

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c    |  7 ++- .../gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c | 21 ++++++-- drivers/gpu/drm/ttm/ttm_bo.c                  | 48 +++++++++++++++++--

  3 files changed, 67 insertions(+), 9 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
index affde72b44db..015bf62277d0 100644
@@ -811,7 +811,12 @@ int amdgpu_bo_pin_restricted(struct amdgpu_bo 
 *bo, u32 domain,

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_pin_restricted(struct amdgpu_bo · line 820

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +6 -1
                    u64 min_offset, u64 max_offset)
   {
       struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-    struct ttm_operation_ctx ctx = { false, false };
+    struct ttm_operation_ctx ctx = {
+        .interruptible = false,
+        .no_wait_gpu = false,
+        .resv = bo->tbo.resv,
+        .flags = TTM_OPT_FLAG_ALLOW_RES_EVICT
+    };

Please completely drop this chunk.

Quoted 1 line

If we allow evicting BOs from the same process during pinning it could lead to some unforeseen side effects.

Will remove that flags.

Quoted 144 lines
Quoted 47 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +0 -0
       int r, i;
         if (amdgpu_ttm_tt_get_usermm(bo->tbo.ttm))
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +3 -0
index a5cacf846e1b..2957ac38dcb0 100644
@@ -4101,6 +4101,9 @@ static int dm_plane_helper_prepare_fb(struct 
 drm_plane *plane,
       struct amdgpu_device *adev;
       struct amdgpu_bo *rbo;
       struct dm_plane_state *dm_plane_state_new, *dm_plane_state_old;
+    struct list_head list, duplicates;
+    struct ttm_validate_buffer tv;
+    struct ww_acquire_ctx ticket;
       uint64_t tiling_flags;
       uint32_t domain;

      int r;

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +13 -1
@@ -4120,6 +4123,18 @@ static int dm_plane_helper_prepare_fb(struct 
 drm_plane *plane,
       r = amdgpu_bo_reserve(rbo, false);
       if (unlikely(r != 0))
           return r;
+    INIT_LIST_HEAD(&list);
+    INIT_LIST_HEAD(&duplicates);
+
+    tv.bo = &rbo->tbo;
+    tv.num_shared = 1;
+    list_add(&tv.head, &list);
+
+    r = ttm_eu_reserve_buffers(&ticket, &list, false, &duplicates);
+    if (r) {
+        dev_err(adev->dev, "fail to reserve bo (%d)\n", r);
+        return r;
+    }
         if (plane->type != DRM_PLANE_TYPE_CURSOR)
           domain = amdgpu_display_supported_domains(adev);
@@ -4130,21 +4145,21 @@ static int dm_plane_helper_prepare_fb(struct 
 drm_plane *plane,
       if (unlikely(r != 0)) {
           if (r != -ERESTARTSYS)
               DRM_ERROR("Failed to pin framebuffer with error %d\n", r);
-        amdgpu_bo_unreserve(rbo);
+        ttm_eu_backoff_reservation(&ticket, &list);
           return r;
       }
         r = amdgpu_ttm_alloc_gart(&rbo->tbo);

drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c static int dm_plane_helper_prepare_fb(struct · line 4161

Quoted 10 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +2 -1
       if (unlikely(r != 0)) {
           amdgpu_bo_unpin(rbo);
-        amdgpu_bo_unreserve(rbo);
+        ttm_eu_backoff_reservation(&ticket, &list);
           DRM_ERROR("%p bind failed\n", rbo);
           return r;
       }
         amdgpu_bo_get_tiling_flags(rbo, &tiling_flags);
   -    amdgpu_bo_unreserve(rbo);
+    ttm_eu_backoff_reservation(&ticket, &list);

This part looks good to me, but as noted above should probably be in a separate patch.

Quoted 60 lines
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +0 -0
         afb->address = amdgpu_bo_gpu_offset(rbo);

  diff --git a/drivers/gpu/drm/ttm/ttm_bo.c b/drivers/gpu/drm/ttm/ttm_bo.c index 8502b3ed2d88..dbcf958d1b43 100644

drivers/gpu/drm/ttm/ttm_bo.c +2 -1
@@ -766,11 +766,12 @@ EXPORT_SYMBOL(ttm_bo_eviction_valuable);
    * b. Otherwise, trylock it.
    */
   static bool ttm_bo_evict_swapout_allowable(struct ttm_buffer_object *bo,
-            struct ttm_operation_ctx *ctx, bool *locked)
+            struct ttm_operation_ctx *ctx, bool *locked, bool *busy)
   {
       bool ret = false;
         *locked = false;
+    *busy = false;
       if (bo->resv == ctx->resv) {
           reservation_object_assert_held(bo->resv);

          if (ctx->flags & TTM_OPT_FLAG_ALLOW_RES_EVICT

drivers/gpu/drm/ttm/ttm_bo.c +3 -1
@@ -779,6 +780,8 @@ static bool ttm_bo_evict_swapout_allowable(struct 
 ttm_buffer_object *bo,
       } else {
           *locked = reservation_object_trylock(bo->resv);
           ret = *locked;
+        if (!ret)
+            *busy = true;
       }
         return ret;
@@ -791,7 +794,7 @@ static int ttm_mem_evict_first(struct 
 ttm_bo_device *bdev,
   {
       struct ttm_bo_global *glob = bdev->glob;
       struct ttm_mem_type_manager *man = &bdev->man[mem_type];
-    struct ttm_buffer_object *bo = NULL;
+    struct ttm_buffer_object *bo = NULL, *first_bo = NULL;
       bool locked = false;
       unsigned i;

      int ret;

drivers/gpu/drm/ttm/ttm_bo.c +7 -1
@@ -799,8 +802,13 @@ static int ttm_mem_evict_first(struct 
 ttm_bo_device *bdev,
       spin_lock(&glob->lru_lock);
       for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
           list_for_each_entry(bo, &man->lru[i], lru) {
-            if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked))
+            bool busy = false;
+            if (!ttm_bo_evict_swapout_allowable(bo, ctx, &locked,
+                                &busy)) {
+                if (!first_bo && busy)
+                    first_bo = bo;
                   continue;
+            }
                 if (place && !bdev->driver->eviction_valuable(bo,
                                         place)) {
@@ -808,6 +816,7 @@ static int ttm_mem_evict_first(struct 
 ttm_bo_device *bdev,
                       reservation_object_unlock(bo->resv);
                   continue;
               }
+
               break;
           }
Quoted 5 lines

  @@ -820,7 +829,33 @@ static int ttm_mem_evict_first(struct ttm_bo_device *bdev, if (!bo) {
          spin_unlock(&glob->lru_lock);
-        return -EBUSY;
+        if (!first_bo || !ctx || !ctx->resv || !ctx->resv->lock.ctx)

Ah, now I see why you need the reservation object here.

Quoted 7 lines

+            return -EBUSY;
+        if (ctx->interruptible)
+            ret = ww_mutex_lock_interruptible(&bo->resv->lock, +                              ctx->resv->lock.ctx);
+        else
+            ret = ww_mutex_lock(&bo->resv->lock, ctx->resv->lock.ctx); +        if (ret)

Quoted 8 lines

+            return ret;
+        spin_lock(&glob->lru_lock);
+        for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { +            if (list_empty(&man->lru[i]))
+                continue;
+            bo = list_first_entry(&man->lru[i], +                          struct ttm_buffer_object, +                          lru);

Quoted 1 line

You now need to check all BOs on the LRU, cause the first one where a trylock failed is not necessarily the first one on the list.

Sorry, I don't get your opinion on this, Could you detail a bit how to do that? I though after ww_mutex_lock, cs is done, then DC can pick up any of them.

Quoted 15 lines
Quoted 15 lines

+
+            break;
+        }
+        /* verify if BO have been moved */
+        if (first_bo != bo) {
+            spin_unlock(&glob->lru_lock);
+            ww_mutex_unlock(&bo->resv->lock);
+            return -EBUSY;
+        }
+        spin_unlock(&glob->lru_lock);
+        /* ok, pick up first busy BO to wait to evict */ }
        kref_get(&bo->list_kref);

drivers/gpu/drm/ttm/ttm_bo.c +0 -0
@@ -1784,7 +1819,10 @@ int ttm_bo_swapout(struct ttm_bo_global *glob, 
Quoted 8 lines

drivers/gpu/drm/ttm/ttm_bo.c int ttm_bo_swapout(struct ttm_bo_global *glob, struct ttm_operation_ctx *ctx) · line 1822

Quoted 6 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
 struct ttm_operation_ctx *ctx)
       spin_lock(&glob->lru_lock);
       for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
           list_for_each_entry(bo, &glob->swap_lru[i], swap) {
-            if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked)) {
+            bool busy = false;

Better make the parameter optional.

Will do.

-David

Quoted 10 lines

Christian.

Quoted 6 lines
drivers/gpu/drm/ttm/ttm_bo.c +3 -0
+
+            if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked,
+                               &busy)) {
                   ret = 0;
                   break;

              }

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#3 lore raw

Am 26.04.19 um 11:07 schrieb zhoucm1:

Quoted 9 lines

[SNIP]

Quoted 8 lines
Quoted 7 lines

+ spin_lock(&glob->lru_lock);
+        for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { +            if (list_empty(&man->lru[i]))
+                continue;
+            bo = list_first_entry(&man->lru[i], +                          struct ttm_buffer_object, +                          lru);

Quoted 6 lines
Quoted 2 lines

You now need to check all BOs on the LRU, cause the first one where a trylock failed is not necessarily the first one on the list.

Sorry, I don't get your opinion on this, Could you detail a bit how to do that?
I though after ww_mutex_lock, cs is done, then DC can pick up any of them.

The first_bo is not necessarily the first BO of the LRU, but the first BO which failed in the trylock.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

+
+            break;
+        }
+        /* verify if BO have been moved */
+        if (first_bo != bo) {

And in this case we would abort here without a good reason.

Christian.

Quoted 17 lines
Quoted 9 lines
Quoted 9 lines

+ spin_unlock(&glob->lru_lock);
+            ww_mutex_unlock(&bo->resv->lock);
+            return -EBUSY;
+        }
+        spin_unlock(&glob->lru_lock);
+        /* ok, pick up first busy BO to wait to evict */ }
        kref_get(&bo->list_kref);

(quoted diff) +0 -0
@@ -1784,7 +1819,10 @@ int ttm_bo_swapout(struct ttm_bo_global 
Quoted 8 lines

drivers/gpu/drm/ttm/ttm_bo.c int ttm_bo_swapout(struct ttm_bo_global *glob, struct ttm_operation_ctx *ctx) · line 1822

Quoted 6 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
 *glob, struct ttm_operation_ctx *ctx)
       spin_lock(&glob->lru_lock);
       for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
           list_for_each_entry(bo, &glob->swap_lru[i], swap) {
-            if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked)) {
+            bool busy = false;

Better make the parameter optional.

Quoted 14 lines

Will do.

-David

Quoted 10 lines

Christian.

Quoted 6 lines
drivers/gpu/drm/ttm/ttm_bo.c +3 -0
+
+            if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked,
+                               &busy)) {
                   ret = 0;
                   break;

              }

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#4 lore raw
Quoted 18 lines · Christian

Am 26.04.19 um 11:07 schrieb zhoucm1:

Quoted 8 lines

[SNIP]

Quoted 7 lines
Quoted 7 lines

+ spin_lock(&glob->lru_lock);
+        for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) { +            if (list_empty(&man->lru[i]))
+                continue;
+            bo = list_first_entry(&man->lru[i], +                          struct ttm_buffer_object, +                          lru);

Quoted 6 lines
Quoted 2 lines

You now need to check all BOs on the LRU, cause the first one where a trylock failed is not necessarily the first one on the list.

Sorry, I don't get your opinion on this, Could you detail a bit how to do that?
I though after ww_mutex_lock, cs is done, then DC can pick up any of them.

The first_bo is not necessarily the first BO of the LRU, but the first BO which failed in the trylock.

Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

+
+            break;
+        }
+        /* verify if BO have been moved */
+        if (first_bo != bo) {

And in this case we would abort here without a good reason.

I got what you said now. I will change it soon.

-David

Quoted 30 lines

Christian.

Quoted 27 lines
Quoted 8 lines
Quoted 8 lines

+ spin_unlock(&glob->lru_lock);
+            ww_mutex_unlock(&bo->resv->lock);
+            return -EBUSY;
+        }
+        spin_unlock(&glob->lru_lock);
+        /* ok, pick up first busy BO to wait to evict */
      }
        kref_get(&bo->list_kref);

Quoted 8 lines

drivers/gpu/drm/ttm/ttm_bo.c int ttm_bo_swapout(struct ttm_bo_global · line 1823

Quoted 7 lines
drivers/gpu/drm/ttm/ttm_bo.c +1 -1
@@ -1784,7 +1819,10 @@ int ttm_bo_swapout(struct ttm_bo_global
 *glob, struct ttm_operation_ctx *ctx)
       spin_lock(&glob->lru_lock);
       for (i = 0; i < TTM_MAX_BO_PRIORITY; ++i) {
           list_for_each_entry(bo, &glob->swap_lru[i], swap) {
-            if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked)) {
+            bool busy = false;

Better make the parameter optional.

Will do.

-David

Quoted 8 lines

Christian.

Quoted 6 lines
drivers/gpu/drm/ttm/ttm_bo.c +3 -0
+
+            if (ttm_bo_evict_swapout_allowable(bo, ctx, &locked,
+                               &busy)) {
                   ret = 0;
                   break;

              }

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel