Skip to contentSkip to conversation

Nouveau fixes for TTM refcount rework

PATCH In review Danilo Krummrich 3 participants 8 messages Jul 18, 2024 – Jul 23, 2024

Subsystems (3)
lore
Share & tools
lore
author maintainer
#1 lore raw

PATCH 1/3 drm/nouveau: prime: fix refcount underflow

In reply to Danilo Krummrich

Calling nouveau_bo_ref() on a nouveau_bo without initializing it (and hence the backing ttm_bo) leads to a refcount underflow.

Instead of calling nouveau_bo_ref() in the unwind path of drm_gem_object_init(), clean things up manually.

Fixes ab9ccb96a6e6 ("drm/nouveau: use prime helpers")
---
 drivers/gpu/drm/nouveau/nouveau_prime.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/nouveau/nouveau_prime.c +2 -1
index b58ab595faf8..cd95446d6851 100644
@@ -64,7 +64,8 @@ struct drm_gem_object *nouveau_gem_prime_import_sg_table(struct drm_device *dev,
 	 * to the caller, instead of a normal nouveau_bo ttm reference. */
 	ret = drm_gem_object_init(dev, &nvbo->bo.base, size);
 	if (ret) {
-		nouveau_bo_ref(NULL, &nvbo);
+		drm_gem_object_release(&nvbo->bo.base);
+		kfree(nvbo);
 		obj = ERR_PTR(-ENOMEM);
 		goto unlock;
 	}
Signature · 1 line
2.45.2

Danilo Krummrich

author maintainer
#0 lore raw

cover Nouveau fixes for TTM refcount rework

Hi Christian,

Those three patches should unblock your series to use GEM references instead of TTM ones.

@Lyude, Dave: Can you please double check?

- Danilo

Danilo Krummrich (3):
drm/nouveau: prime: fix refcount underflow
drm/nouveau: bo: remove unused functions
drm/nouveau: use GEM references instead of TTMs

 drivers/gpu/drm/nouveau/dispnv04/crtc.c | 43 +++++++++++++++------
 drivers/gpu/drm/nouveau/dispnv50/disp.c |  4 +-
 drivers/gpu/drm/nouveau/nouveau_bo.h    | 50 ++-----------------------
 drivers/gpu/drm/nouveau/nouveau_chan.c  |  2 +-
 drivers/gpu/drm/nouveau/nouveau_dmem.c  |  4 +-
 drivers/gpu/drm/nouveau/nouveau_prime.c |  3 +-
 drivers/gpu/drm/nouveau/nv10_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv17_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv50_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv84_fence.c    |  4 +-
 10 files changed, 46 insertions(+), 70 deletions(-)

base-commit: 99e0fb8b087120b5a7019f1cff6c5c2b5b925ae5

Signature · 1 line
2.45.2
author maintainer
#3 lore raw

PATCH 2/3 drm/nouveau: bo: remove unused functions

nouveau_bo_new_pin_map() and nouveau_bo_unmap_unpin_unref() are unused, hence remove them.

---
 drivers/gpu/drm/nouveau/nouveau_bo.h | 29 ----------------------------
 1 file changed, 29 deletions(-)
drivers/gpu/drm/nouveau/nouveau_bo.h +0 -29
index 4e891752c255..3b8dfbb621da 100644
@@ -115,35 +115,6 @@ nvbo_kmap_obj_iovirtual(struct nouveau_bo *nvbo)
 	return ioptr;
 }
 
-static inline void
-nouveau_bo_unmap_unpin_unref(struct nouveau_bo **pnvbo)
-{
-	if (*pnvbo) {
-		nouveau_bo_unmap(*pnvbo);
-		nouveau_bo_unpin(*pnvbo);
-		nouveau_bo_ref(NULL, pnvbo);
-	}
-}
-
-static inline int
-nouveau_bo_new_pin_map(struct nouveau_cli *cli, u64 size, int align, u32 domain,
-		       struct nouveau_bo **pnvbo)
-{
-	int ret = nouveau_bo_new(cli, size, align, domain,
-				 0, 0, NULL, NULL, pnvbo);
-	if (ret == 0) {
-		ret = nouveau_bo_pin(*pnvbo, domain, true);
-		if (ret == 0) {
-			ret = nouveau_bo_map(*pnvbo);
-			if (ret == 0)
-				return ret;
-			nouveau_bo_unpin(*pnvbo);
-		}
-		nouveau_bo_ref(NULL, pnvbo);
-	}
-	return ret;
-}
-
 int nv04_bo_move_init(struct nouveau_channel *, u32);
 int nv04_bo_move_m2mf(struct nouveau_channel *, struct ttm_buffer_object *,
 		      struct ttm_resource *, struct ttm_resource *);
Signature · 1 line
2.45.2
author maintainer
#4 lore raw

PATCH 3/3 drm/nouveau: use GEM references instead of TTMs

In reply to Danilo Krummrich

TTM wants to get rid of the duplicate refcounting of the embedded GEM object and its own reference count.

Hence, use of GEM object references where possible.

Also get rid of nouveau_bo_ref() and replace it with nouveau_bo_fini(), which drops the initial reference we get from initializing a ttm_bo.

---
 drivers/gpu/drm/nouveau/dispnv04/crtc.c | 43 ++++++++++++++++++-------
 drivers/gpu/drm/nouveau/dispnv50/disp.c |  4 +--
 drivers/gpu/drm/nouveau/nouveau_bo.h    | 21 ++----------
 drivers/gpu/drm/nouveau/nouveau_chan.c  |  2 +-
 drivers/gpu/drm/nouveau/nouveau_dmem.c  |  4 +--
 drivers/gpu/drm/nouveau/nv10_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv17_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv50_fence.c    |  2 +-
 drivers/gpu/drm/nouveau/nv84_fence.c    |  4 +--
 9 files changed, 44 insertions(+), 40 deletions(-)
drivers/gpu/drm/nouveau/dispnv04/crtc.c +31 -12
index 4310ad71870b..438e3ec6e1ca 100644
@@ -617,9 +617,15 @@ nv_crtc_swap_fbs(struct drm_crtc *crtc, struct drm_framebuffer *old_fb)
 
 	ret = nouveau_bo_pin(nvbo, NOUVEAU_GEM_DOMAIN_VRAM, false);
 	if (ret == 0) {
-		if (disp->image[nv_crtc->index])
-			nouveau_bo_unpin(disp->image[nv_crtc->index]);
-		nouveau_bo_ref(nvbo, &disp->image[nv_crtc->index]);
+		if (disp->image[nv_crtc->index]) {
+			struct nouveau_bo *bo = disp->image[nv_crtc->index];
+
+			nouveau_bo_unpin(bo);
+			drm_gem_object_put(&bo->bo.base);
+		}
+
+		drm_gem_object_get(&nvbo->bo.base);
+		disp->image[nv_crtc->index] = nvbo;
 	}
 
 	return ret;
@@ -754,13 +760,17 @@ static void nv_crtc_destroy(struct drm_crtc *crtc)
 
 	drm_crtc_cleanup(crtc);
 
-	if (disp->image[nv_crtc->index])
-		nouveau_bo_unpin(disp->image[nv_crtc->index]);
-	nouveau_bo_ref(NULL, &disp->image[nv_crtc->index]);
+	if (disp->image[nv_crtc->index]) {
+		struct nouveau_bo *bo = disp->image[nv_crtc->index];
+
+		nouveau_bo_unpin(bo);
+		drm_gem_object_put(&bo->bo.base);
+		disp->image[nv_crtc->index] = NULL;
+	}
 
 	nouveau_bo_unmap(nv_crtc->cursor.nvbo);
 	nouveau_bo_unpin(nv_crtc->cursor.nvbo);
-	nouveau_bo_ref(NULL, &nv_crtc->cursor.nvbo);
+	nouveau_bo_fini(nv_crtc->cursor.nvbo);
 	nvif_event_dtor(&nv_crtc->vblank);
 	nvif_head_dtor(&nv_crtc->head);
 	kfree(nv_crtc);
@@ -794,9 +804,14 @@ nv_crtc_disable(struct drm_crtc *crtc)
 {
 	struct nv04_display *disp = nv04_display(crtc->dev);
 	struct nouveau_crtc *nv_crtc = nouveau_crtc(crtc);
-	if (disp->image[nv_crtc->index])
-		nouveau_bo_unpin(disp->image[nv_crtc->index]);
-	nouveau_bo_ref(NULL, &disp->image[nv_crtc->index]);
+
+	if (disp->image[nv_crtc->index]) {
+		struct nouveau_bo *bo = disp->image[nv_crtc->index];
+
+		nouveau_bo_unpin(bo);
+		drm_gem_object_put(&bo->bo.base);
+		disp->image[nv_crtc->index] = NULL;
+	}
 }
 
 static int
@@ -1210,7 +1225,11 @@ nv04_crtc_page_flip(struct drm_crtc *crtc, struct drm_framebuffer *fb,
 		PUSH_NVSQ(push, NV05F, 0x0130, 0);
 	}
 
-	nouveau_bo_ref(new_bo, &dispnv04->image[head]);
+	if (dispnv04->image[head])
+		drm_gem_object_put(&dispnv04->image[head]->bo.base);
+
+	drm_gem_object_get(&new_bo->bo.base);
+	dispnv04->image[head] = new_bo;
 
 	ret = nv04_page_flip_emit(chan, old_bo, new_bo, s, &fence);
 	if (ret)
@@ -1329,7 +1348,7 @@ nv04_crtc_create(struct drm_device *dev, int crtc_num)
 				nouveau_bo_unpin(nv_crtc->cursor.nvbo);
 		}
 		if (ret)
-			nouveau_bo_ref(NULL, &nv_crtc->cursor.nvbo);
+			nouveau_bo_fini(nv_crtc->cursor.nvbo);
 	}
 
 	nv04_cursor_init(nv_crtc);
drivers/gpu/drm/nouveau/dispnv50/disp.c +2 -2
index ac9657d7e92d..6d20d3c68fa7 100644
@@ -2819,7 +2819,7 @@ nv50_display_destroy(struct drm_device *dev)
 	nouveau_bo_unmap(disp->sync);
 	if (disp->sync)
 		nouveau_bo_unpin(disp->sync);
-	nouveau_bo_ref(NULL, &disp->sync);
+	nouveau_bo_fini(disp->sync);
 
 	nouveau_display(dev)->priv = NULL;
 	kfree(disp);
@@ -2862,7 +2862,7 @@ nv50_display_create(struct drm_device *dev)
 				nouveau_bo_unpin(disp->sync);
 		}
 		if (ret)
-			nouveau_bo_ref(NULL, &disp->sync);
+			nouveau_bo_fini(disp->sync);
 	}
 
 	if (ret)
drivers/gpu/drm/nouveau/nouveau_bo.h +3 -18
index 3b8dfbb621da..596a63a50a20 100644
@@ -53,25 +53,10 @@ nouveau_bo(struct ttm_buffer_object *bo)
 	return container_of(bo, struct nouveau_bo, bo);
 }
 
-static inline int
-nouveau_bo_ref(struct nouveau_bo *ref, struct nouveau_bo **pnvbo)
+static inline void
+nouveau_bo_fini(struct nouveau_bo *bo)
 {
-	struct nouveau_bo *prev;
-
-	if (!pnvbo)
-		return -EINVAL;
-	prev = *pnvbo;
-
-	if (ref) {
-		ttm_bo_get(&ref->bo);
-		*pnvbo = nouveau_bo(&ref->bo);
-	} else {
-		*pnvbo = NULL;
-	}
-	if (prev)
-		ttm_bo_put(&prev->bo);
-
-	return 0;
+	ttm_bo_put(&bo->bo);
 }
 
 extern struct ttm_device_funcs nouveau_bo_driver;
drivers/gpu/drm/nouveau/nouveau_chan.c +1 -1
index 66fca95c10c7..f568ea251e3b 100644
@@ -110,7 +110,7 @@ nouveau_channel_del(struct nouveau_channel **pchan)
 		nouveau_bo_unmap(chan->push.buffer);
 		if (chan->push.buffer && chan->push.buffer->bo.pin_count)
 			nouveau_bo_unpin(chan->push.buffer);
-		nouveau_bo_ref(NULL, &chan->push.buffer);
+		nouveau_bo_fini(chan->push.buffer);
 		kfree(chan);
 	}
 	*pchan = NULL;
drivers/gpu/drm/nouveau/nouveau_dmem.c +2 -2
index 6719353e2e13..7b3b8f4630a2 100644
@@ -294,7 +294,7 @@ nouveau_dmem_chunk_alloc(struct nouveau_drm *drm, struct page **ppage)
 out_bo_unpin:
 	nouveau_bo_unpin(chunk->bo);
 out_bo_free:
-	nouveau_bo_ref(NULL, &chunk->bo);
+	nouveau_bo_fini(chunk->bo);
 out_release:
 	release_mem_region(chunk->pagemap.range.start, range_len(&chunk->pagemap.range));
 out_free:
@@ -426,7 +426,7 @@ nouveau_dmem_fini(struct nouveau_drm *drm)
 	list_for_each_entry_safe(chunk, tmp, &drm->dmem->chunks, list) {
 		nouveau_dmem_evict_chunk(chunk);
 		nouveau_bo_unpin(chunk->bo);
-		nouveau_bo_ref(NULL, &chunk->bo);
+		nouveau_bo_fini(chunk->bo);
 		WARN_ON(chunk->callocated);
 		list_del(&chunk->list);
 		memunmap_pages(&chunk->pagemap);
drivers/gpu/drm/nouveau/nv10_fence.c +1 -1
index c6a0db5b9e21..1a53b8b80467 100644
@@ -88,7 +88,7 @@ nv10_fence_destroy(struct nouveau_drm *drm)
 	nouveau_bo_unmap(priv->bo);
 	if (priv->bo)
 		nouveau_bo_unpin(priv->bo);
-	nouveau_bo_ref(NULL, &priv->bo);
+	nouveau_bo_fini(priv->bo);
 	drm->fence = NULL;
 	kfree(priv);
 }
drivers/gpu/drm/nouveau/nv17_fence.c +1 -1
index 07c2e0878c24..2c99f2c1ddcd 100644
@@ -141,7 +141,7 @@ nv17_fence_create(struct nouveau_drm *drm)
 				nouveau_bo_unpin(priv->bo);
 		}
 		if (ret)
-			nouveau_bo_ref(NULL, &priv->bo);
+			nouveau_bo_fini(priv->bo);
 	}
 
 	if (ret) {
drivers/gpu/drm/nouveau/nv50_fence.c +1 -1
index ea1e1f480bfe..6fa18f9d26b6 100644
@@ -92,7 +92,7 @@ nv50_fence_create(struct nouveau_drm *drm)
 				nouveau_bo_unpin(priv->bo);
 		}
 		if (ret)
-			nouveau_bo_ref(NULL, &priv->bo);
+			nouveau_bo_fini(priv->bo);
 	}
 
 	if (ret) {
drivers/gpu/drm/nouveau/nv84_fence.c +2 -2
index 812b8c62eeba..9ce4c2d60fe3 100644
@@ -188,7 +188,7 @@ nv84_fence_destroy(struct nouveau_drm *drm)
 	nouveau_bo_unmap(priv->bo);
 	if (priv->bo)
 		nouveau_bo_unpin(priv->bo);
-	nouveau_bo_ref(NULL, &priv->bo);
+	nouveau_bo_fini(priv->bo);
 	drm->fence = NULL;
 	kfree(priv);
 }
@@ -232,7 +232,7 @@ nv84_fence_create(struct nouveau_drm *drm)
 				nouveau_bo_unpin(priv->bo);
 		}
 		if (ret)
-			nouveau_bo_ref(NULL, &priv->bo);
+			nouveau_bo_fini(priv->bo);
 	}
 
 	if (ret)
Signature · 1 line
2.45.2
Reviewed-by on the v1 series
#5 lore raw
In reply to Danilo Krummrich
Quoted 6 lines · Danilo Krummrich

Hi Christian,

Those three patches should unblock your series to use GEM references instead of TTM ones.

@Lyude, Dave: Can you please double check?

Hi Danilo,

These look fine to me, and appear to resolve the issues I see with just the refcount series applied.

Ben.

Quoted 22 lines

- Danilo

Danilo Krummrich (3):
drm/nouveau: prime: fix refcount underflow drm/nouveau: bo: remove unused functions drm/nouveau: use GEM references instead of TTMs

  drivers/gpu/drm/nouveau/dispnv04/crtc.c | 43 +++++++++++++++------
  drivers/gpu/drm/nouveau/dispnv50/disp.c |  4 +-
  drivers/gpu/drm/nouveau/nouveau_bo.h    | 50 ++-----------------------
  drivers/gpu/drm/nouveau/nouveau_chan.c  |  2 +-
  drivers/gpu/drm/nouveau/nouveau_dmem.c  |  4 +-
  drivers/gpu/drm/nouveau/nouveau_prime.c |  3 +-
  drivers/gpu/drm/nouveau/nv10_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv17_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv50_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv84_fence.c    |  4 +-
  10 files changed, 46 insertions(+), 70 deletions(-)

base-commit: 99e0fb8b087120b5a7019f1cff6c5c2b5b925ae5

#2 lore raw

Re: [PATCH 1/3] drm/nouveau: prime: fix refcount underflow

In reply to Danilo Krummrich

Am 18.07.24 um 18:58 schrieb Danilo Krummrich:

Quoted 8 lines

Calling nouveau_bo_ref() on a nouveau_bo without initializing it (and hence the backing ttm_bo) leads to a refcount underflow.

Instead of calling nouveau_bo_ref() in the unwind path of drm_gem_object_init(), clean things up manually.

Fixes ab9ccb96a6e6 ("drm/nouveau: use prime helpers")

Thanks for looking into this, feel free to add Reviewed-by: Christian König <[email protected]> to this patch.

But since especially patch #3 is not something I can fully judge correctness on I can only give an acked-by to the other two.

Regards,
Christian.

Quoted 18 lines
---
  drivers/gpu/drm/nouveau/nouveau_prime.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/drm/nouveau/nouveau_prime.c +2 -1
index b58ab595faf8..cd95446d6851 100644
@@ -64,7 +64,8 @@ struct drm_gem_object *nouveau_gem_prime_import_sg_table(struct drm_device *dev,
  	 * to the caller, instead of a normal nouveau_bo ttm reference. */
  	ret = drm_gem_object_init(dev, &nvbo->bo.base, size);
  	if (ret) {
-		nouveau_bo_ref(NULL, &nvbo);
+		drm_gem_object_release(&nvbo->bo.base);
+		kfree(nvbo);
  		obj = ERR_PTR(-ENOMEM);
  		goto unlock;
  	}
author maintainer
#6 lore raw
In reply to Ben Skeggs
Quoted 9 lines · Ben Skeggs
Quoted 6 lines · Danilo Krummrich

Hi Christian,

Those three patches should unblock your series to use GEM references instead of TTM ones.

@Lyude, Dave: Can you please double check?

Quoted 8 lines

Hi Danilo,

These look fine to me, and appear to resolve the issues I see with just the refcount series applied.

Ben.

Thanks for reviewing!

@Christian, I applied the series to drm-misc-next for you to go ahead and cherry-picked the bug fix to drm-misc-fixes.

- Danilo

Quoted 23 lines
Quoted 23 lines

- Danilo

Danilo Krummrich (3):
   drm/nouveau: prime: fix refcount underflow drm/nouveau: bo: remove unused functions drm/nouveau: use GEM references instead of TTMs

  drivers/gpu/drm/nouveau/dispnv04/crtc.c | 43 +++++++++++++++------ drivers/gpu/drm/nouveau/dispnv50/disp.c |  4 +- drivers/gpu/drm/nouveau/nouveau_bo.h    | 50 ++----------------------- drivers/gpu/drm/nouveau/nouveau_chan.c  |  2 +- drivers/gpu/drm/nouveau/nouveau_dmem.c  |  4 +- drivers/gpu/drm/nouveau/nouveau_prime.c |  3 +-

  drivers/gpu/drm/nouveau/nv10_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv17_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv50_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv84_fence.c    |  4 +-
  10 files changed, 46 insertions(+), 70 deletions(-)

base-commit: 99e0fb8b087120b5a7019f1cff6c5c2b5b925ae5

#7 lore raw

Am 22.07.24 um 23:01 schrieb Danilo Krummrich:

Quoted 17 lines · Ben Skeggs
Quoted 16 lines · Danilo Krummrich
Quoted 7 lines

Hi Christian,

Those three patches should unblock your series to use GEM references instead of TTM ones.

@Lyude, Dave: Can you please double check?

Hi Danilo,

These look fine to me, and appear to resolve the issues I see with just the refcount series applied.

Ben.

Quoted 6 lines
Quoted 1 line

Thanks for reviewing!

@Christian, I applied the series to drm-misc-next for you to go ahead and cherry-picked the bug fix to drm-misc-fixes.

Thanks, I was already about to re-base my stuff on it.

Going to send out a new series today.

Christian.

Quoted 27 lines

- Danilo

Quoted 24 lines
Quoted 24 lines

- Danilo

Danilo Krummrich (3):
   drm/nouveau: prime: fix refcount underflow drm/nouveau: bo: remove unused functions drm/nouveau: use GEM references instead of TTMs

  drivers/gpu/drm/nouveau/dispnv04/crtc.c | 43 +++++++++++++++------ drivers/gpu/drm/nouveau/dispnv50/disp.c |  4 +- drivers/gpu/drm/nouveau/nouveau_bo.h    | 50
++-----------------------
  drivers/gpu/drm/nouveau/nouveau_chan.c  |  2 +- drivers/gpu/drm/nouveau/nouveau_dmem.c  |  4 +- drivers/gpu/drm/nouveau/nouveau_prime.c |  3 +-

  drivers/gpu/drm/nouveau/nv10_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv17_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv50_fence.c    |  2 +-
  drivers/gpu/drm/nouveau/nv84_fence.c    |  4 +-
  10 files changed, 46 insertions(+), 70 deletions(-)

base-commit: 99e0fb8b087120b5a7019f1cff6c5c2b5b925ae5