Skip to contentSkip to conversation

drm/radeon: Convert to generic fbdev emulation

PATCH In review Thomas Zimmermann 4 participants 37 messages Nov 12, 2020 – Nov 26, 2020

Subsystems (3)
lore
Share & tools
lore

Thomas Zimmermann

author maintainer
#0 lore raw

cover drm/radeon: Convert to generic fbdev emulation

This patchset replaces radeon's framebuffer emulation with the generic code from fb helpers. This reduces the driver size and complexity. As a nice side effect, it also reduces memory pressure on low-memory devices.

Patches #1 and #2 add features to fbdev emulation to bring it on par with radeons code.

Patch #4 adds internal pinning to vmap operations. This guarantess that the framebuffer BO is not relocated while being updated.

Patch #5 and #6 replace radeon's framebuffer code with the generic helper with shadow buffering. All userspace mappings interact with the shadow buffer and the BO can be relocated an necessary. This allows to enable 32-bit colors on low-memory devices.

Patches #3 and #7 are merely cleanups.

Smoke tested with an AMD R5 (Oland) card by running the console and switching to/from Xorg and weston.

Thomas Zimmermann (7):
drm/fb-helper: Set framebuffer for vga-switcheroo clients drm/fb-helper: Add hint to enable VT switching during suspend/resume drm/radeon: Whitespace fixes
drm/radeon: Pin buffers while they are vmap'ed drm/radeon: Replace framebuffer console with generic implementation drm/radeon: Use fbdev shadow fb
drm/radeon: Move radeon_align_pitch() next to its only caller

 drivers/gpu/drm/drm_fb_helper.c         |  16 +-
 drivers/gpu/drm/radeon/radeon_device.c  |  11 +-
 drivers/gpu/drm/radeon/radeon_display.c |   2 -
 drivers/gpu/drm/radeon/radeon_drv.c     |   5 +-
 drivers/gpu/drm/radeon/radeon_fb.c      | 366 +-----------------------
 drivers/gpu/drm/radeon/radeon_gem.c     |  77 ++++-
 drivers/gpu/drm/radeon/radeon_kms.c     |  14 +-
 drivers/gpu/drm/radeon/radeon_mode.h    |  10 +-
 include/drm/drm_mode_config.h           |   7 +
 9 files changed, 111 insertions(+), 397 deletions(-)

--
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#1 lore raw

PATCH 1/7 drm/fb-helper: Set framebuffer for vga-switcheroo clients

Set the framebuffer info for devices that support vga switcheroo. For other devices, this does nothing.

---
 drivers/gpu/drm/drm_fb_helper.c | 8 ++++++++
 1 file changed, 8 insertions(+)
drivers/gpu/drm/drm_fb_helper.c +8 -0
index 01ba1da28511..d2ba404a23b3 100644
@@ -33,8 +33,10 @@
 #include <linux/dma-buf.h>
 #include <linux/kernel.h>
 #include <linux/module.h>
+#include <linux/pci.h>
 #include <linux/slab.h>
 #include <linux/sysrq.h>
+#include <linux/vga_switcheroo.h>
 #include <linux/vmalloc.h>
 
 #include <drm/drm_atomic.h>
@@ -1940,6 +1942,8 @@ EXPORT_SYMBOL(drm_fb_helper_hotplug_event);
 void drm_fb_helper_lastclose(struct drm_device *dev)
 {
 	drm_fb_helper_restore_fbdev_mode_unlocked(dev->fb_helper);
+
+	vga_switcheroo_process_delayed_switch();
 }
 EXPORT_SYMBOL(drm_fb_helper_lastclose);
 
@@ -2311,6 +2315,10 @@ static int drm_fb_helper_generic_probe(struct drm_fb_helper *fb_helper,
 #endif
 	}
 
+	/* Set the fb info for vgaswitcheroo clients. Does nothing otherwise. */
+	if (dev_is_pci(dev->dev))
+		vga_switcheroo_client_fb_set(to_pci_dev(dev->dev), fbi);
+
 	return 0;
 }
 
Signature · 6 lines
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#2 lore raw

PATCH 2/7 drm/fb-helper: Add hint to enable VT switching during suspend/resume

In reply to Thomas Zimmermann

Switching VTs during suspend/resume is required to reliably run radeon with generic fbdev emulation.

---
 drivers/gpu/drm/drm_fb_helper.c | 8 +++++---
 include/drm/drm_mode_config.h   | 7 +++++++
 2 files changed, 12 insertions(+), 3 deletions(-)
drivers/gpu/drm/drm_fb_helper.c +5 -3
index d2ba404a23b3..987cd781e62c 100644
@@ -508,11 +508,11 @@ EXPORT_SYMBOL(drm_fb_helper_init);
  */
 struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
 {
-	struct device *dev = fb_helper->dev->dev;
+	struct drm_device *dev = fb_helper->dev;
 	struct fb_info *info;
 	int ret;
 
-	info = framebuffer_alloc(0, dev);
+	info = framebuffer_alloc(0, dev->dev);
 	if (!info)
 		return ERR_PTR(-ENOMEM);
 
@@ -535,7 +535,9 @@ struct fb_info *drm_fb_helper_alloc_fbi(struct drm_fb_helper *fb_helper)
 	}
 
 	fb_helper->fbdev = info;
-	info->skip_vt_switch = true;
+
+	if (!dev->mode_config.require_vt_switch_fbdev)
+		info->skip_vt_switch = true;
 
 	return info;
 
include/drm/drm_mode_config.h +7 -0
index ab424ddd7665..59e0796c6928 100644
@@ -877,6 +877,13 @@ struct drm_mode_config {
 	 */
 	bool prefer_shadow_fbdev;
 
+	/**
+	 * @require_vt_switch_fbdev:
+	 *
+	 * Hint to framebuffer emulation to enable VT switching on suspend/resume.
+	 */
+	bool require_vt_switch_fbdev;
+
 	/**
 	 * @quirk_addfb_prefer_xbgr_30bpp:
 	 *
Signature · 6 lines
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#3 lore raw

PATCH 3/7 drm/radeon: Whitespace fixes

In reply to Thomas Zimmermann

Fixed to match kernel coding style.

---
 drivers/gpu/drm/radeon/radeon_kms.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/radeon/radeon_kms.c +1 -1
index abb3bdd9ca25..75b038740ea8 100644
@@ -74,7 +74,7 @@ void radeon_driver_unload_kms(struct drm_device *dev)
 	}
 
 	radeon_acpi_fini(rdev);
-	
+
 	radeon_modeset_fini(rdev);
 	radeon_device_fini(rdev);
 
Signature · 6 lines
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#4 lore raw

PATCH 4/7 drm/radeon: Pin buffers while they are vmap'ed

In reply to Thomas Zimmermann

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

---
 drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
 1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +49 -2
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct radeon_device *rdev, int r)
 	return r;
 }
 
+static int radeon_gem_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+					   RADEON_GEM_DOMAIN_GTT |
+					   RADEON_GEM_DOMAIN_CPU;
+
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return ret;
+
+	/* pin buffer at its current location */
+	ret = radeon_bo_pin(bo, any_domain, NULL);
+	if (ret)
+		goto err_radeon_bo_unreserve;
+
+	ret = drm_gem_ttm_vmap(obj, map);
+	if (ret)
+		goto err_radeon_bo_unpin;
+
+	radeon_bo_unreserve(bo);
+
+	return 0;
+
+err_radeon_bo_unpin:
+	radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+	radeon_bo_unreserve(bo);
+	return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return;
+
+	drm_gem_ttm_vunmap(obj, map);
+	radeon_bo_unpin(bo);
+	radeon_bo_unreserve(bo);
+}
+
 static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
 	.free = radeon_gem_object_free,
 	.open = radeon_gem_object_open,
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
 	.pin = radeon_gem_prime_pin,
 	.unpin = radeon_gem_prime_unpin,
 	.get_sg_table = radeon_gem_prime_get_sg_table,
-	.vmap = drm_gem_ttm_vmap,
-	.vunmap = drm_gem_ttm_vunmap,
+	.vmap = radeon_gem_object_vmap,
+	.vunmap = radeon_gem_object_vunmap,
 };
 
 /*
Signature · 6 lines
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
maintainer
#5 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 5 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

Quoted 7 lines

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

Regards,
Christian.

Quoted 65 lines
---
  drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
  1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +47 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct radeon_device *rdev, int r)
  	return r;
  }
 
+static int radeon_gem_object_vmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+					   RADEON_GEM_DOMAIN_GTT |
+					   RADEON_GEM_DOMAIN_CPU;
+
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return ret;
+
+	/* pin buffer at its current location */
+	ret = radeon_bo_pin(bo, any_domain, NULL);
+	if (ret)
+		goto err_radeon_bo_unreserve;
+
+	ret = drm_gem_ttm_vmap(obj, map);
+	if (ret)
+		goto err_radeon_bo_unpin;
+
+	radeon_bo_unreserve(bo);
+
+	return 0;
+
+err_radeon_bo_unpin:
+	radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+	radeon_bo_unreserve(bo);
+	return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj, struct dma_buf_map *map)
+{
+	struct radeon_bo *bo = gem_to_radeon_bo(obj);
+	int ret;
+
+	ret = radeon_bo_reserve(bo, false);
+	if (ret)
+		return;
+
+	drm_gem_ttm_vunmap(obj, map);
+	radeon_bo_unpin(bo);
+	radeon_bo_unreserve(bo);
+}
+
  static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
  	.free = radeon_gem_object_free,
  	.open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs radeon_gem_object_funcs = { · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
  	.pin = radeon_gem_prime_pin,
  	.unpin = radeon_gem_prime_unpin,
  	.get_sg_table = radeon_gem_prime_get_sg_table,
-	.vmap = drm_gem_ttm_vmap,
-	.vunmap = drm_gem_ttm_vunmap,
+	.vmap = radeon_gem_object_vmap,
+	.vunmap = radeon_gem_object_vunmap,
  };
 
  /*

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#6 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 10 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 9 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

Quoted 5 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Quoted 3 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/gpu/drm/drm_fb_helper.c#n432

Quoted 87 lines

Regards,
Christian.

Quoted 68 lines

---
  drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--

  1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
       return r;
   }
   +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

  static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
      .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
       .pin = radeon_gem_prime_pin,
       .unpin = radeon_gem_prime_unpin,
       .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
   };
     /*

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 10 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
maintainer
#7 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 13 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 10 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 9 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

Regards,
Christian.

Quoted 94 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 84 lines

Regards,
Christian.

Quoted 67 lines
---
   drivers/gpu/drm/radeon/radeon_gem.c | 51 +++++++++++++++++++++++++++--
   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
       return r;
   }
   +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+
   static const struct drm_gem_object_funcs radeon_gem_object_funcs = {
       .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
       .pin = radeon_gem_prime_pin,
       .unpin = radeon_gem_prime_unpin,
       .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
   };
     /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#8 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 16.11.20 um 12:28 schrieb Christian König:

Quoted 33 lines

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 15 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 12 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 11 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

Quoted 5 lines

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

If this is a problem is practice, it has never shown up with the drivers that use it already.

I think here's a modeset lock somewhere that could serialize these operations. The fbdev console is not double buffered, so there's no frequent pageflipping; hence interference should be small.

Best regards
Thomas

Quoted 102 lines

Regards,
Christian.

Quoted 97 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 86 lines

Regards,
Christian.

Quoted 68 lines

---
   drivers/gpu/drm/radeon/radeon_gem.c | 51
+++++++++++++++++++++++++++--

   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
        return r;
    }
    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

   static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
        .pin = radeon_gem_prime_pin,
        .unpin = radeon_gem_prime_unpin,
        .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
    };
      /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

Signature · 10 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#9 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

In reply to Christian König

Hi

Am 16.11.20 um 12:28 schrieb Christian König:

Quoted 33 lines

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 15 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 12 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 11 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as
shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

Quoted 5 lines

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

I sent out a patchset that addresses the issue in it's final patch. [1] I'd appreciate your feedback. It also tested the patches with the converted radeon driver.

Best regards
Thomas

[1] https://patchwork.freedesktop.org/series/83918/

Quoted 106 lines

Regards,
Christian.

Quoted 97 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 86 lines

Regards,
Christian.

Quoted 68 lines

---
   drivers/gpu/drm/radeon/radeon_gem.c | 51
+++++++++++++++++++++++++++--

   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
        return r;
    }
    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

   static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
        .pin = radeon_gem_prime_pin,
        .unpin = radeon_gem_prime_unpin,
        .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
    };
      /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 10 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#10 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

In reply to Christian König

Hi Christian

Am 16.11.20 um 12:28 schrieb Christian König:

Quoted 31 lines

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 13 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 10 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 9 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

Quoted 4 lines

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

I'd like ask for your suggestions before sending an update for this patch.

After the discussion about locking in fbdev, [1] I intended to replace the pin call with code that acquires the reservation lock.

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO. Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Best regards
Thomas

[1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1

Quoted 105 lines

Regards,
Christian.

Quoted 97 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 86 lines

Regards,
Christian.

Quoted 68 lines

---
   drivers/gpu/drm/radeon/radeon_gem.c | 51
+++++++++++++++++++++++++++--

   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
        return r;
    }
    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

   static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
        .pin = radeon_gem_prime_pin,
        .unpin = radeon_gem_prime_unpin,
        .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
    };
      /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#11 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:

Quoted 34 lines

Hi Christian

Am 16.11.20 um 12:28 schrieb Christian König:

Quoted 31 lines

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 13 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 10 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 9 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

Quoted 8 lines
Quoted 4 lines

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

I'd like ask for your suggestions before sending an update for this patch.

After the discussion about locking in fbdev, [1] I intended to replace the pin call with code that acquires the reservation lock.

Yeah, that sounds like a good idea to me as well.

Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Regards,
Christian.

Quoted 112 lines

Best regards
Thomas

[1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1

Quoted 105 lines

Regards,
Christian.

Quoted 97 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 86 lines

Regards,
Christian.

Quoted 68 lines

---
   drivers/gpu/drm/radeon/radeon_gem.c | 51
+++++++++++++++++++++++++++--

   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
        return r;
    }
    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

   static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
        .pin = radeon_gem_prime_pin,
        .unpin = radeon_gem_prime_unpin,
        .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
    };
      /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#12 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 24.11.20 um 12:30 schrieb Christian König:

Quoted 58 lines

Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:

Quoted 34 lines

Hi Christian

Am 16.11.20 um 12:28 schrieb Christian König:

Quoted 31 lines

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 13 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 10 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 9 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

Quoted 8 lines
Quoted 4 lines

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

I'd like ask for your suggestions before sending an update for this patch.

After the discussion about locking in fbdev, [1] I intended to replace the pin call with code that acquires the reservation lock.

Yeah, that sounds like a good idea to me as well.

Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

Quoted 5 lines

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Best regards
Thomas

Quoted 117 lines

Regards,
Christian.

Quoted 112 lines

Best regards
Thomas

[1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1

Quoted 105 lines

Regards,
Christian.

Quoted 97 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 86 lines

Regards,
Christian.

Quoted 68 lines

---
   drivers/gpu/drm/radeon/radeon_gem.c | 51
+++++++++++++++++++++++++++--

   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
        return r;
    }
    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

   static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
        .pin = radeon_gem_prime_pin,
        .unpin = radeon_gem_prime_unpin,
        .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
    };
      /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#13 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 24.11.20 um 12:44 schrieb Thomas Zimmermann:

Quoted 61 lines

Hi

Am 24.11.20 um 12:30 schrieb Christian König:

Quoted 58 lines

Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:

Quoted 34 lines

Hi Christian

Am 16.11.20 um 12:28 schrieb Christian König:

Quoted 31 lines

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 13 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 10 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 9 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

Quoted 8 lines
Quoted 4 lines

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

I'd like ask for your suggestions before sending an update for this patch.

After the discussion about locking in fbdev, [1] I intended to replace the pin call with code that acquires the reservation lock.

Yeah, that sounds like a good idea to me as well.

Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

Quoted 7 lines
Quoted 5 lines

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

Regards,
Christian.

Quoted 122 lines

Best regards
Thomas

Quoted 117 lines

Regards,
Christian.

Quoted 112 lines

Best regards
Thomas

[1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1

Quoted 105 lines

Regards,
Christian.

Quoted 97 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 86 lines

Regards,
Christian.

Quoted 68 lines

---
   drivers/gpu/drm/radeon/radeon_gem.c | 51
+++++++++++++++++++++++++++--

   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
        return r;
    }
    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

   static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
        .pin = radeon_gem_prime_pin,
        .unpin = radeon_gem_prime_unpin,
        .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
    };
      /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#14 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 24.11.20 um 12:54 schrieb Christian König:

Quoted 64 lines

Am 24.11.20 um 12:44 schrieb Thomas Zimmermann:

Quoted 63 lines

Hi

Am 24.11.20 um 12:30 schrieb Christian König:

Quoted 60 lines

Am 24.11.20 um 10:16 schrieb Thomas Zimmermann:

Quoted 34 lines

Hi Christian

Am 16.11.20 um 12:28 schrieb Christian König:

Quoted 31 lines

Am 13.11.20 um 08:59 schrieb Thomas Zimmermann:

Quoted 13 lines

Hi Christian

Am 12.11.20 um 18:16 schrieb Christian König:

Quoted 10 lines

Am 12.11.20 um 14:21 schrieb Thomas Zimmermann:

Quoted 9 lines

In order to avoid eviction of vmap'ed buffers, pin them in their GEM object's vmap implementation. Unpin them in the vunmap implementation. This is needed to make generic fbdev support work reliably. Without, the buffer object could be evicted while fbdev flushed its shadow buffer.

In difference to the PRIME pin/unpin functions, the vmap code does not modify the BOs prime_shared_count, so a vmap-pinned BO does not count as shared.

Quoted 5 lines
Quoted 4 lines
Quoted 3 lines

The actual pin location is not important as the vmap call returns information on how to access the buffer. Callers that require a specific location should explicitly pin the BO before vmapping it.

Well is the buffer supposed to be scanned out?

No, not by the fbdev helper.

Ok in this case that should work.

Quoted 8 lines
Quoted 2 lines

If yes then the pin location is actually rather important since the hardware can only scan out from VRAM.

For relocatable BOs, fbdev uses a shadow buffer that makes all any relocation transparent to userspace. It flushes the shadow fb into the BO's memory if there are updates. The code is in drm_fb_helper_dirty_work(). [1] During the flush operation, the vmap call now pins the BO to wherever it is. The actual location does not matter. It's vunmap'ed immediately afterwards.

Quoted 8 lines
Quoted 4 lines

The problem is what happens when it is prepared for scanout, but can't be moved to VRAM because it is vmapped?

When the shadow is never scanned out that isn't a problem, but we need to keep that in mind.

I'd like ask for your suggestions before sending an update for this patch.

After the discussion about locking in fbdev, [1] I intended to replace the pin call with code that acquires the reservation lock.

Yeah, that sounds like a good idea to me as well.

Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right? The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Quoted 4 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Best regards
Thomas

Quoted 131 lines

Regards,
Christian.

Quoted 122 lines

Best regards
Thomas

Quoted 117 lines

Regards,
Christian.

Quoted 112 lines

Best regards
Thomas

[1] https://patchwork.freedesktop.org/patch/401088/?series=83918&rev=1

Quoted 105 lines

Regards,
Christian.

Quoted 97 lines

For dma-buf sharing, the regular procedure of pin + vmap still apply. This should always move the BO into GTT-managed memory.

Best regards
Thomas

[1]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgit.kernel.org%2Fpub%2Fscm%2Flinux%2Fkernel%2Fgit%2Ftorvalds%2Flinux.git%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%23n432&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=RLauuAuXkcl0rXwWWJ%2FrKP%2BsCr2wAzU1ejGV1bnQ80w%3D&amp;reserved=0

Quoted 86 lines

Regards,
Christian.

Quoted 68 lines

---
   drivers/gpu/drm/radeon/radeon_gem.c | 51
+++++++++++++++++++++++++++--

   1 file changed, 49 insertions(+), 2 deletions(-)
drivers/gpu/drm/radeon/radeon_gem.c +46 -0
index d2876ce3bc9e..eaf7fc9a7b07 100644
@@ -226,6 +226,53 @@ static int radeon_gem_handle_lockup(struct
 radeon_device *rdev, int r)
        return r;
    }
    +static int radeon_gem_object_vmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    static const uint32_t any_domain = RADEON_GEM_DOMAIN_VRAM |
+                       RADEON_GEM_DOMAIN_GTT |
+                       RADEON_GEM_DOMAIN_CPU;
+
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return ret;
+
+    /* pin buffer at its current location */
+    ret = radeon_bo_pin(bo, any_domain, NULL);
+    if (ret)
+        goto err_radeon_bo_unreserve;
+
+    ret = drm_gem_ttm_vmap(obj, map);
+    if (ret)
+        goto err_radeon_bo_unpin;
+
+    radeon_bo_unreserve(bo);
+
+    return 0;
+
+err_radeon_bo_unpin:
+    radeon_bo_unpin(bo);
+err_radeon_bo_unreserve:
+    radeon_bo_unreserve(bo);
+    return ret;
+}
+
+static void radeon_gem_object_vunmap(struct drm_gem_object *obj,
 struct dma_buf_map *map)
+{
+    struct radeon_bo *bo = gem_to_radeon_bo(obj);
+    int ret;
+
+    ret = radeon_bo_reserve(bo, false);
+    if (ret)
+        return;
+
+    drm_gem_ttm_vunmap(obj, map);
+    radeon_bo_unpin(bo);
+    radeon_bo_unreserve(bo);
+}
+

   static const struct drm_gem_object_funcs radeon_gem_object_funcs = { .free = radeon_gem_object_free,
       .open = radeon_gem_object_open,

drivers/gpu/drm/radeon/radeon_gem.c static const struct drm_gem_object_funcs · line 288

Quoted 10 lines
drivers/gpu/drm/radeon/radeon_gem.c +2 -2
@@ -234,8 +281,8 @@ static const struct drm_gem_object_funcs
 radeon_gem_object_funcs = {
        .pin = radeon_gem_prime_pin,
        .unpin = radeon_gem_prime_unpin,
        .get_sg_table = radeon_gem_prime_get_sg_table,
-    .vmap = drm_gem_ttm_vmap,
-    .vunmap = drm_gem_ttm_vunmap,
+    .vmap = radeon_gem_object_vmap,
+    .vunmap = radeon_gem_object_vunmap,
    };
      /*

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C31b890664ca7429fc45808d887aa0842%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637408511650629569%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=h1U9Po83K7webxsiKpn3ZGFz9Fcg6SRkxtrXWZ1%2B%2FEc%3D&amp;reserved=0

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

maintainer
#15 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 19 lines

[SNIP]

Quoted 18 lines
Quoted 18 lines
Quoted 17 lines
Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Quoted 7 lines
Quoted 3 lines
Quoted 1 line

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Yes, correct.

Quoted 1 line

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Quoted 7 lines
Quoted 5 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Do you have a link to the code?

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 1 line

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#16 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 26 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 25 lines

[SNIP]

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

Quoted 8 lines
Quoted 1 line

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Yes, correct.

Quoted 1 line

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 9 lines
Quoted 7 lines
Quoted 5 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the shadow buffer into the BO, the BO has to be kept in place. I already changed it to lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could still evict the BO concurrently.

There's no recursion taking place, so I guess the reservation lock could be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of DRM client functions could do the locking.

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 22 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 1 line

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

maintainer
#17 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 24.11.20 um 14:56 schrieb Thomas Zimmermann:

Quoted 44 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 26 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 25 lines

[SNIP]

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

Quoted 8 lines
Quoted 1 line

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Yes, correct.

Quoted 1 line

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the shadow buffer into the BO, the BO has to be kept in place. I already changed it to lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could still evict the BO concurrently.

Yeah, that's correct.

But I still don't fully understand the problem. You just need to change the code like this:

    mutex_lock(&fb_helper->lock);
    dma_resv_lock(buffer->gem->resv, NULL);

    ret = drm_client_buffer_vmap(buffer, &map);
    if (ret)
        goto out;

    dst = map; drm_fb_helper_damage_blit_real(fb_helper, clip, &dst);

    drm_client_buffer_vunmap(buffer);

out:
    dma_resv_unlock(buffer->gem->resv);
    mutex_unlock(&fb_helper->lock);

You could abstract that in drm_client functions as well, but I don't really see the value in that.

Regards,
Christian.

Quoted 30 lines

There's no recursion taking place, so I guess the reservation lock could be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of DRM client functions could do the locking.

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 22 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 1 line

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#18 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 24.11.20 um 15:06 schrieb Christian König:

Quoted 64 lines

Am 24.11.20 um 14:56 schrieb Thomas Zimmermann:

Quoted 44 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 26 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 25 lines

[SNIP]

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines
Quoted 1 line

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

Quoted 8 lines
Quoted 1 line

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Yes, correct.

Quoted 1 line

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 4 lines
Quoted 4 lines
Quoted 4 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the shadow buffer into the BO, the BO has to be kept in place. I already changed it to lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could still evict the BO concurrently.

Yeah, that's correct.

But I still don't fully understand the problem. You just need to change the code like this:

    mutex_lock(&fb_helper->lock);
    dma_resv_lock(buffer->gem->resv, NULL);

    ret = drm_client_buffer_vmap(buffer, &map);
    if (ret)
        goto out;

Quoted 9 lines

    dst = map; drm_fb_helper_damage_blit_real(fb_helper, clip, &dst);

    drm_client_buffer_vunmap(buffer);

out:
    dma_resv_unlock(buffer->gem->resv);
    mutex_unlock(&fb_helper->lock);

Yes, that's the code I had in mind.

Quoted 2 lines

You could abstract that in drm_client functions as well, but I don't really see the value in that.

The fbdev code tries hard to not use GEM directly, but to wrap everything behind client interfaces. I'm not sure if I like that, but for now I'd stick to this design.

Best regards
Thomas

Quoted 40 lines

Regards,
Christian.

Quoted 31 lines

There's no recursion taking place, so I guess the reservation lock could be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of DRM client functions could do the locking.

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 22 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 1 line

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#19 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

In reply to Thomas Zimmermann
Quoted 70 lines · Thomas Zimmermann

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 44 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 35 lines

[SNIP]

Quoted 34 lines
Quoted 33 lines
Quoted 21 lines
Quoted 5 lines

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 3 lines

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Quoted 7 lines
Quoted 1 line

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Quoted 9 lines

Yes, correct.

Quoted 3 lines

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 11 lines
Quoted 10 lines
Quoted 7 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Quoted 6 lines
Quoted 1 line

Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the shadow buffer into the BO, the BO has to be kept in place. I already changed it to lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could still evict the BO concurrently.

So I'm not sure this is actually a problem: ttm could try to concurrently evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about that last few mb of vram being useable for command submission (well atm there's no driver using it).

Having the buffer pinned into system memory and trying to do a concurrent modeset that tries to pull it in is the hard failure mode. And holding fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

Quoted 3 lines

There's no recursion taking place, so I guess the reservation lock could be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 43 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 29 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
author maintainer
#20 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:

Quoted 78 lines · Thomas Zimmermann
Quoted 70 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 44 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 35 lines

[SNIP]

Quoted 34 lines
Quoted 33 lines
Quoted 21 lines
Quoted 5 lines

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 3 lines

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Quoted 7 lines
Quoted 1 line

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Quoted 9 lines

Yes, correct.

Quoted 3 lines

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 11 lines
Quoted 10 lines
Quoted 7 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Quoted 6 lines
Quoted 1 line

Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the shadow buffer into the BO, the BO has to be kept in place. I already changed it to lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could still evict the BO concurrently.

Quoted 6 lines

So I'm not sure this is actually a problem: ttm could try to concurrently evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about that last few mb of vram being useable for command submission (well atm there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and nouveau are natural next choices. Especially radeon and nouveau support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly matter.

Quoted 6 lines

Having the buffer pinned into system memory and trying to do a concurrent modeset that tries to pull it in is the hard failure mode. And holding fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM would want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. If we do a concurrent damage blit without protecting against eviction, things go boom. Same for concurrent 3d graphics with textures, model data, etc.

Best regards
Thomas

Quoted 63 lines
Quoted 4 lines

There's no recursion taking place, so I guess the reservation lock could be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 43 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 29 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#21 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:

Quoted 95 lines

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:

Quoted 78 lines · Thomas Zimmermann
Quoted 70 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 44 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 35 lines

[SNIP]

Quoted 34 lines
Quoted 33 lines
Quoted 21 lines
Quoted 5 lines

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 3 lines

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Quoted 7 lines
Quoted 1 line

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Quoted 9 lines

Yes, correct.

Quoted 3 lines

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 11 lines
Quoted 10 lines
Quoted 7 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Quoted 6 lines
Quoted 1 line

Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the shadow buffer into the BO, the BO has to be kept in place. I already changed it to lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could still evict the BO concurrently.

Quoted 6 lines

So I'm not sure this is actually a problem: ttm could try to concurrently evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about that last few mb of vram being useable for command submission (well atm there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and nouveau are natural next choices. Especially radeon and nouveau support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly matter.

Quoted 5 lines

Having the buffer pinned into system memory and trying to do a concurrent modeset that tries to pull it in is the hard failure mode. And holding fb_helper.lock fully prevents that.

Quoted 5 lines
Quoted 1 line

So not really clear on what failure mode you're seeing here?

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM would want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. If we do a concurrent damage blit without protecting against eviction, things go boom. Same for concurrent 3d graphics with textures, model data, etc.

Completely agree.

This needs proper lock protection of the memory mapped buffer. Relying on that some other code isn't run because we have some third part locks taken is not sufficient here.

Regards,
Christian.

Quoted 68 lines

Best regards
Thomas

Quoted 63 lines
Quoted 4 lines

There's no recursion taking place, so I guess the reservation lock could be acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 43 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 29 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#22 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Quoted 116 lines · Christian König

Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:

Quoted 106 lines

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:

Quoted 81 lines · Thomas Zimmermann
Quoted 72 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 44 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 35 lines

[SNIP]

Quoted 34 lines
Quoted 33 lines
Quoted 21 lines
Quoted 5 lines

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 3 lines

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Quoted 7 lines
Quoted 1 line

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Quoted 9 lines

Yes, correct.

Quoted 3 lines

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Do you have a link to the code?

Quoted 7 lines

It's the damage blitter in the fbdev code. [1] While it flushes the shadow
buffer into the BO, the BO has to be kept in place. I already changed it to
lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
still evict the BO concurrently.

Quoted 8 lines

So I'm not sure this is actually a problem: ttm could try to concurrently
evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about that last few mb of vram being useable for command submission (well atm there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and nouveau are natural next choices. Especially radeon and nouveau support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly matter.

Quoted 7 lines

Having the buffer pinned into system memory and trying to do a concurrent
modeset that tries to pull it in is the hard failure mode. And holding fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

Quoted 8 lines

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM would want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. If we do a concurrent damage blit without protecting against eviction, things go boom. Same for concurrent 3d graphics with textures, model data, etc.

Quoted 5 lines

Completely agree.

This needs proper lock protection of the memory mapped buffer. Relying on that some other code isn't run because we have some third part locks taken is not sufficient here.

We are still protected by the pin count in this scenario. Plus, with current drivers we always pin the fbdev buffer into vram, so occasionally failing to move it out isn't a regression.

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 77 lines

Regards,
Christian.

Quoted 72 lines

Best regards
Thomas

Quoted 67 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 44 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 30 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
#23 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 25.11.20 um 11:36 schrieb Daniel Vetter:

Quoted 101 lines · Christian König
Quoted 100 lines

Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:

Quoted 89 lines

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:

Quoted 67 lines · Thomas Zimmermann
Quoted 58 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 38 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 31 lines

[SNIP]

Quoted 30 lines
Quoted 30 lines
Quoted 20 lines
Quoted 5 lines

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 6 lines
Quoted 3 lines

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Quoted 6 lines
Quoted 1 line

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Quoted 8 lines

Yes, correct.

Quoted 3 lines

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 7 lines
Quoted 7 lines
Quoted 5 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Quoted 8 lines
Quoted 1 line

Do you have a link to the code?

It's the damage blitter in the fbdev code. [1] While it flushes the shadow
buffer into the BO, the BO has to be kept in place. I already changed it to
lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
still evict the BO concurrently.

Quoted 8 lines

So I'm not sure this is actually a problem: ttm could try to concurrently
evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about that last few mb of vram being useable for command submission (well atm there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and nouveau are natural next choices. Especially radeon and nouveau support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly matter.

Quoted 6 lines

Having the buffer pinned into system memory and trying to do a concurrent
modeset that tries to pull it in is the hard failure mode. And holding fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

Quoted 8 lines

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM would want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. If we do a concurrent damage blit without protecting against eviction, things go boom. Same for concurrent 3d graphics with textures, model data, etc.

Completely agree.

Quoted 8 lines
Quoted 3 lines

This needs proper lock protection of the memory mapped buffer. Relying on that some other code isn't run because we have some third part locks taken is not sufficient here.

We are still protected by the pin count in this scenario. Plus, with current drivers we always pin the fbdev buffer into vram, so occasionally failing to move it out isn't a regression.

So I'm still not seeing how this can go boom.

Well as far as I understand it the pin count is zero for this buffer in this case here :)

I might be wrong on this because I don't know the FB code at all, but Thomas seems to be pretty clear that this is the shadow buffer which is not scanned out from.

Regards,
Christian.

Quoted 71 lines

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 64 lines

Regards,
Christian.

Quoted 61 lines

Best regards
Thomas

Quoted 58 lines
Quoted 5 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 38 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 26 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 2 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#24 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

In reply to Daniel Vetter

Hi

Am 25.11.20 um 11:36 schrieb Daniel Vetter:

Quoted 119 lines · Christian König
Quoted 118 lines

Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:

Quoted 106 lines

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:

Quoted 81 lines · Thomas Zimmermann
Quoted 72 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 44 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 35 lines

[SNIP]

Quoted 34 lines
Quoted 33 lines
Quoted 21 lines
Quoted 5 lines

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 3 lines

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Quoted 7 lines
Quoted 1 line

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Quoted 9 lines

Yes, correct.

Quoted 3 lines

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Do you have a link to the code?

Quoted 7 lines

It's the damage blitter in the fbdev code. [1] While it flushes the shadow
buffer into the BO, the BO has to be kept in place. I already changed it to
lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
still evict the BO concurrently.

Quoted 8 lines

So I'm not sure this is actually a problem: ttm could try to concurrently
evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about that last few mb of vram being useable for command submission (well atm there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and nouveau are natural next choices. Especially radeon and nouveau support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly matter.

Quoted 7 lines

Having the buffer pinned into system memory and trying to do a concurrent
modeset that tries to pull it in is the hard failure mode. And holding fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

Quoted 8 lines

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM would want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. If we do a concurrent damage blit without protecting against eviction, things go boom. Same for concurrent 3d graphics with textures, model data, etc.

Completely agree.

Quoted 7 lines
Quoted 3 lines

This needs proper lock protection of the memory mapped buffer. Relying on that some other code isn't run because we have some third part locks taken is not sufficient here.

We are still protected by the pin count in this scenario. Plus, with current drivers we always pin the fbdev buffer into vram, so occasionally failing to move it out isn't a regression.

Why is this protected by the pin count? The counter should be zero in this scenario. Otherwise, we could not evict the fbdev BO on all those systems where that's a hard requirement (e.g., ast).

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 87 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 77 lines

Regards,
Christian.

Quoted 72 lines

Best regards
Thomas

Quoted 67 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 44 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 30 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

#25 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Quoted 126 lines · Thomas Zimmermann

Hi

Am 25.11.20 um 11:36 schrieb Daniel Vetter:

Quoted 123 lines · Christian König
Quoted 121 lines

Am 25.11.20 um 09:37 schrieb Thomas Zimmermann:

Quoted 106 lines

Hi

Am 24.11.20 um 15:09 schrieb Daniel Vetter:

Quoted 81 lines · Thomas Zimmermann
Quoted 72 lines

Hi

Am 24.11.20 um 14:36 schrieb Christian König:

Quoted 44 lines

Am 24.11.20 um 13:15 schrieb Thomas Zimmermann:

Quoted 35 lines

[SNIP]

Quoted 34 lines
Quoted 33 lines
Quoted 21 lines
Quoted 5 lines

First I wanted to put this into drm_gem_ttm_vmap/vunmap(), but then wondered why ttm_bo_vmap() doe not acquire the lock internally? I'd expect that vmap/vunmap are close together and do not overlap for the same BO.

We have use cases like the following during command submission:

1. lock
2. map
3. copy parts of the BO content somewhere else or patch it with additional information
4. unmap
5. submit BO to the hardware
6. add hardware fence to the BO to make sure it doesn't move
7. unlock

That use case won't be possible with vmap/vunmap if we move the lock/unlock into it and I hope to replace the kmap/kunmap functions with them in the near term.

Quoted 7 lines
Quoted 3 lines

Otherwise, acquiring the reservation lock would require another ref-counting variable or per-driver code.

Hui, why that? Just put this into drm_gem_ttm_vmap/vunmap() helper as you initially planned.

Given your example above, step one would acquire the lock, and step two would also acquire the lock as part of the vmap implementation. Wouldn't this fail (At least during unmap or unlock steps) ?

Quoted 7 lines
Quoted 1 line

Oh, so you want to nest them? No, that is a rather bad no-go.

I don't want to nest/overlap them. My question was whether that would be required. Apparently not.

While the console's BO is being set for scanout, it's protected from movement via the pin/unpin implementation, right?

Quoted 9 lines

Yes, correct.

Quoted 3 lines

The driver does not acquire the resv lock for longer periods. I'm asking because this would prevent any console-buffer updates while the console is being displayed.

Correct as well, we only hold the lock for things like command submission, pinning, unpinning etc etc....

Thanks for answering my questions.

Quoted 5 lines
Quoted 5 lines
Quoted 5 lines

You need to make sure that the lock is only taken from the FB path which wants to vmap the object.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Why don't you lock the GEM object from the caller in the generic FB implementation?

With the current blitter code, it breaks abstraction. if vmap/vunmap hold the lock implicitly, things would be easier.

Do you have a link to the code?

Quoted 7 lines

It's the damage blitter in the fbdev code. [1] While it flushes the shadow
buffer into the BO, the BO has to be kept in place. I already changed it to
lock struct drm_fb_helper.lock, but I don't think this is enough. TTM could
still evict the BO concurrently.

Quoted 8 lines

So I'm not sure this is actually a problem: ttm could try to concurrently
evict the buffer we pinned into vram, and then just skip to the next one.

Plus atm generic fbdev isn't used on any chip where we really care about that last few mb of vram being useable for command submission (well atm there's no driver using it).

Well, this is the patchset for radeon. If it works out, amdgpu and nouveau are natural next choices. Especially radeon and nouveau support cards with low- to medium-sized VRAM. The MiBs wasted on fbdev certainly matter.

Quoted 7 lines

Having the buffer pinned into system memory and trying to do a concurrent
modeset that tries to pull it in is the hard failure mode. And holding fb_helper.lock fully prevents that.

So not really clear on what failure mode you're seeing here?

Quoted 8 lines

Imagine the fbdev BO is in VRAM, but not pinned. (Maybe Xorg or Wayland is running.) The fbdev BO is a few MiBs and not in use, so TTM would want to evict it if memory gets tight.

What I have in mind is a concurrent modeset that requires the memory. If we do a concurrent damage blit without protecting against eviction, things go boom. Same for concurrent 3d graphics with textures, model data, etc.

Completely agree.

This needs proper lock protection of the memory mapped buffer. Relying on that some other code isn't run because we have some third part locks taken is not sufficient here.

Quoted 7 lines
Quoted 3 lines

We are still protected by the pin count in this scenario. Plus, with current drivers we always pin the fbdev buffer into vram, so occasionally failing to move it out isn't a regression.

Why is this protected by the pin count? The counter should be zero in this scenario. Otherwise, we could not evict the fbdev BO on all those systems where that's a hard requirement (e.g., ast).

Ah yes, I mixed that up.

I guess full locking is required :-/ I'm not exactly sure how to make this happen with the current plethora of helpers ... I think we need an _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

And then document that if the callers of the _locked version wants a permanent mapping, it also needs to pin it. Plus I guess ideally implement the unlocked/permanent versions in terms of that, so that drivers only have to implement one or the other.

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 109 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 87 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 77 lines

Regards,
Christian.

Quoted 72 lines

Best regards
Thomas

Quoted 67 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 44 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 30 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#26 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:

Quoted 4 lines

[...]
I guess full locking is required :-/ I'm not exactly sure how to make this happen with the current plethora of helpers ... I think we need an _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the implementations are without taking resv locks. For locking, we can wrap them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap code. And as I mentioned before, this is actually wrong. The pattern dates back to when the code was still in individual drivers. It's time to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it while blitting framebuffers into HW buffers and ast does some permanent mapping of the cursor BO. All this is trivial to turn into small pairs of lock+vmap and vunmap+unlock.

That leaves generic fbdev. The shadow buffer is also trivial to fix, as outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the buffer during initialization and only vunmaps it during shutdown. As this has worked so far without locking, I'd leave it as it is and put a big comment next to is.

Hardware fbdev buffers is only required by few drivers; namely those that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. We should consider to make the fbdev shadow buffer the default and have drivers opt-in for the hardware buffer, if they need it.

Quoted 5 lines

And then document that if the callers of the _locked version wants a permanent mapping, it also needs to pin it. Plus I guess ideally implement the unlocked/permanent versions in terms of that, so that drivers only have to implement one or the other.

For my understanding, pinning is only done in prepare_fb code. And ast pins its cursor BOs into vram. We should document to hols vmap/vunmap only for time and cover them with resv locks. Pinning is for cases where the hardware requires buffers in a special location, but does not protect against concurrent threat. I think those are the implicit rules already.

I updated the radeon patchset, where all this appears to be working well.

Best regards
Thomas

Quoted 120 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 109 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 87 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 77 lines

Regards,
Christian.

Quoted 72 lines

Best regards
Thomas

Quoted 67 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 44 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 30 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#27 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Quoted 45 lines · Thomas Zimmermann

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:

Quoted 4 lines

[...]
I guess full locking is required :-/ I'm not exactly sure how to make this happen with the current plethora of helpers ... I think we need an _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the implementations are without taking resv locks. For locking, we can wrap them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap code. And as I mentioned before, this is actually wrong. The pattern dates back to when the code was still in individual drivers. It's time to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it while blitting framebuffers into HW buffers and ast does some permanent mapping of the cursor BO. All this is trivial to turn into small pairs of lock+vmap and vunmap+unlock.

That leaves generic fbdev. The shadow buffer is also trivial to fix, as outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the buffer during initialization and only vunmaps it during shutdown. As this has worked so far without locking, I'd leave it as it is and put a big comment next to is.

Hardware fbdev buffers is only required by few drivers; namely those that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. We should consider to make the fbdev shadow buffer the default and have drivers opt-in for the hardware buffer, if they need it.

Quoted 5 lines

And then document that if the callers of the _locked version wants a permanent mapping, it also needs to pin it. Plus I guess ideally implement the unlocked/permanent versions in terms of that, so that drivers only have to implement one or the other.

Quoted 8 lines

For my understanding, pinning is only done in prepare_fb code. And ast pins its cursor BOs into vram. We should document to hols vmap/vunmap only for time and cover them with resv locks. Pinning is for cases where the hardware requires buffers in a special location, but does not protect against concurrent threat. I think those are the implicit rules already.

I updated the radeon patchset, where all this appears to be working well.

Hm yeah if you want to do the full change, then that works out too. It's just a pile of work.

But if we can finish off with an dma_resv_assert_locked in dma_buf_vmap/vunmap, then I think that's ok. It does mean that exporters must implement vmap caching, or performance will be terrible. So quite some update for the dma-buf docs.

But if you're willing to do all that conversion of callers, then of course I'm not stopping you. Not at all, it's great to see that kind of maze untangled.
-Daniel

Quoted 132 lines

Best regards
Thomas

Quoted 120 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 109 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 87 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 77 lines

Regards,
Christian.

Quoted 72 lines

Best regards
Thomas

Quoted 67 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 44 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 23 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel
maintainer
#28 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 26.11.20 um 12:04 schrieb Daniel Vetter:

Quoted 25 lines · Thomas Zimmermann
Quoted 24 lines

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:

Quoted 4 lines

[...]
I guess full locking is required :-/ I'm not exactly sure how to make this happen with the current plethora of helpers ... I think we need an _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the implementations are without taking resv locks. For locking, we can wrap them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap code. And as I mentioned before, this is actually wrong. The pattern dates back to when the code was still in individual drivers. It's time to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it while blitting framebuffers into HW buffers and ast does some permanent mapping of the cursor BO. All this is trivial to turn into small pairs of lock+vmap and vunmap+unlock.

Quoted 7 lines
Quoted 7 lines

That leaves generic fbdev. The shadow buffer is also trivial to fix, as outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the buffer during initialization and only vunmaps it during shutdown. As this has worked so far without locking, I'd leave it as it is and put a big comment next to is.

Please keep in mind that you only need to grab the lock if the buffer is not pinned otherwise.

In other words when we are scanning out from the BO it is guaranteed that it can't move around.

Maybe this makes the case here easier to handle.

Quoted 17 lines
Quoted 17 lines

Hardware fbdev buffers is only required by few drivers; namely those that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. We should consider to make the fbdev shadow buffer the default and have drivers opt-in for the hardware buffer, if they need it.

Quoted 4 lines

And then document that if the callers of the _locked version wants a permanent mapping, it also needs to pin it. Plus I guess ideally implement the unlocked/permanent versions in terms of that, so that drivers only have to implement one or the other.

For my understanding, pinning is only done in prepare_fb code. And ast pins its cursor BOs into vram. We should document to hols vmap/vunmap only for time and cover them with resv locks. Pinning is for cases where the hardware requires buffers in a special location, but does not protect against concurrent threat. I think those are the implicit rules already.

Quoted 8 lines
Quoted 1 line

I updated the radeon patchset, where all this appears to be working well.

Hm yeah if you want to do the full change, then that works out too. It's just a pile of work.

But if we can finish off with an dma_resv_assert_locked in dma_buf_vmap/vunmap, then I think that's ok. It does mean that exporters must implement vmap caching, or performance will be terrible. So quite some update for the dma-buf docs.

That's one possibility, but I think we should keep the ability to use pin+vmap instead of lock+vmap.

Regards,
Christian.

Quoted 120 lines

But if you're willing to do all that conversion of callers, then of course I'm not stopping you. Not at all, it's great to see that kind of maze untangled.
-Daniel

Quoted 112 lines

Best regards
Thomas

Quoted 102 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 93 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 72 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 64 lines

Regards,
Christian.

Quoted 61 lines

Best regards
Thomas

Quoted 58 lines
Quoted 5 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 38 lines

Best regards
Thomas

[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freedesktop.org%2Fdrm%2Fdrm-tip%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%3Fid%3Dac60f3f3090115d21f028bffa2dcfb67f695c4f2%23n394&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682660550%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cky%2BozENU1nsd4hlfAdsvA6wC0RXsex7gpFuvHlCROM%3D&amp;reserved=0

Quoted 20 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 6 lines
Quoted 2 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682670543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sa5ao1X5JGFgcnhNiDbCjI4SlMMWzHITBylAZsG%2BVzs%3D&amp;reserved=0

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#29 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 26.11.20 um 12:28 schrieb Christian König:

Quoted 34 lines

Am 26.11.20 um 12:04 schrieb Daniel Vetter:

Quoted 25 lines · Thomas Zimmermann
Quoted 24 lines

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:

Quoted 4 lines

[...]
I guess full locking is required :-/ I'm not exactly sure how to make this happen with the current plethora of helpers ... I think we need an _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the implementations are without taking resv locks. For locking, we can wrap them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap code. And as I mentioned before, this is actually wrong. The pattern dates back to when the code was still in individual drivers. It's time to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it while blitting framebuffers into HW buffers and ast does some permanent mapping of the cursor BO. All this is trivial to turn into small pairs of lock+vmap and vunmap+unlock.

Quoted 7 lines
Quoted 7 lines

That leaves generic fbdev. The shadow buffer is also trivial to fix, as outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the buffer during initialization and only vunmaps it during shutdown. As this has worked so far without locking, I'd leave it as it is and put a big comment next to is.

Quoted 5 lines

Please keep in mind that you only need to grab the lock if the buffer is not pinned otherwise.

In other words when we are scanning out from the BO it is guaranteed that it can't move around.

Maybe this makes the case here easier to handle.

The fbdev code is already fragile. If no shadow FB is selected, the hardware BO is vmapped, but never pinned; if only for the reason that there's no useful generic interface to do this. So we cannot lock for longer periods, but it's also not pinned either. This really only work with a few drivers that use CMA helpers, where BOs don't move.

Best regards
Thomas

Quoted 155 lines
Quoted 18 lines
Quoted 18 lines

Hardware fbdev buffers is only required by few drivers; namely those that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. We should consider to make the fbdev shadow buffer the default and have drivers opt-in for the hardware buffer, if they need it.

Quoted 4 lines

And then document that if the callers of the _locked version wants a permanent mapping, it also needs to pin it. Plus I guess ideally implement the unlocked/permanent versions in terms of that, so that drivers only have to implement one or the other.

For my understanding, pinning is only done in prepare_fb code. And ast pins its cursor BOs into vram. We should document to hols vmap/vunmap only for time and cover them with resv locks. Pinning is for cases where the hardware requires buffers in a special location, but does not protect against concurrent threat. I think those are the implicit rules already.

Quoted 8 lines
Quoted 1 line

I updated the radeon patchset, where all this appears to be working well.

Hm yeah if you want to do the full change, then that works out too. It's just a pile of work.

But if we can finish off with an dma_resv_assert_locked in dma_buf_vmap/vunmap, then I think that's ok. It does mean that exporters must implement vmap caching, or performance will be terrible. So quite some update for the dma-buf docs.

That's one possibility, but I think we should keep the ability to use pin+vmap instead of lock+vmap.

Regards,
Christian.

Quoted 122 lines

But if you're willing to do all that conversion of callers, then of course I'm not stopping you. Not at all, it's great to see that kind of maze untangled.
-Daniel

Quoted 114 lines

Best regards
Thomas

Quoted 104 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 95 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 74 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 66 lines

Regards,
Christian.

Quoted 63 lines

Best regards
Thomas

Quoted 60 lines
Quoted 5 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 40 lines

Best regards
Thomas

[1] https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcgit.freedesktop.org%2Fdrm%2Fdrm-tip%2Ftree%2Fdrivers%2Fgpu%2Fdrm%2Fdrm_fb_helper.c%3Fid%3Dac60f3f3090115d21f028bffa2dcfb67f695c4f2%23n394&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682660550%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Cky%2BozENU1nsd4hlfAdsvA6wC0RXsex7gpFuvHlCROM%3D&amp;reserved=0

Quoted 27 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 2 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Fdri-devel&amp;data=04%7C01%7Cchristian.koenig%40amd.com%7C73458d36471547ca128008d891fb0958%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637419854682670543%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Sa5ao1X5JGFgcnhNiDbCjI4SlMMWzHITBylAZsG%2BVzs%3D&amp;reserved=0

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#30 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

In reply to Daniel Vetter

Hi

Am 26.11.20 um 12:04 schrieb Daniel Vetter:

Quoted 55 lines · Thomas Zimmermann
Quoted 45 lines

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:

Quoted 4 lines

[...]
I guess full locking is required :-/ I'm not exactly sure how to make this happen with the current plethora of helpers ... I think we need an _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the implementations are without taking resv locks. For locking, we can wrap them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap code. And as I mentioned before, this is actually wrong. The pattern dates back to when the code was still in individual drivers. It's time to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it while blitting framebuffers into HW buffers and ast does some permanent mapping of the cursor BO. All this is trivial to turn into small pairs of lock+vmap and vunmap+unlock.

That leaves generic fbdev. The shadow buffer is also trivial to fix, as outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the buffer during initialization and only vunmaps it during shutdown. As this has worked so far without locking, I'd leave it as it is and put a big comment next to is.

Hardware fbdev buffers is only required by few drivers; namely those that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. We should consider to make the fbdev shadow buffer the default and have drivers opt-in for the hardware buffer, if they need it.

Quoted 5 lines

And then document that if the callers of the _locked version wants a permanent mapping, it also needs to pin it. Plus I guess ideally implement the unlocked/permanent versions in terms of that, so that drivers only have to implement one or the other.

Quoted 8 lines

For my understanding, pinning is only done in prepare_fb code. And ast pins its cursor BOs into vram. We should document to hols vmap/vunmap only for time and cover them with resv locks. Pinning is for cases where the hardware requires buffers in a special location, but does not protect against concurrent threat. I think those are the implicit rules already.

I updated the radeon patchset, where all this appears to be working well.

Quoted 7 lines

Hm yeah if you want to do the full change, then that works out too. It's just a pile of work.

But if we can finish off with an dma_resv_assert_locked in dma_buf_vmap/vunmap, then I think that's ok. It does mean that exporters must implement vmap caching, or performance will be terrible. So quite some update for the dma-buf docs.

Yeah, I remember a bug report about frequent page-table modifications wrt to vram helpers. So we implemented the lazy unmapping / vmap caching, as suggested by Christian back them. My guess is that anything TTM-based can use a similar pattern. Christian probably knows the corner cases.

CMA seems obviously working correctly already.

For SHMEM, I'd have to figure out the reference counting of the pages involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could be moved into the free callback, plus a few other modifications.

Best regards
Thomas

Quoted 141 lines

But if you're willing to do all that conversion of callers, then of course I'm not stopping you. Not at all, it's great to see that kind of maze untangled.
-Daniel

Quoted 132 lines

Best regards
Thomas

Quoted 120 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 109 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 87 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 77 lines

Regards,
Christian.

Quoted 72 lines

Best regards
Thomas

Quoted 67 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 44 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 23 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 1 line
Quoted 7 lines
Quoted 2 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

maintainer
#31 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 26.11.20 um 12:59 schrieb Thomas Zimmermann:

Quoted 61 lines

Hi

Am 26.11.20 um 12:04 schrieb Daniel Vetter:

Quoted 58 lines · Thomas Zimmermann
Quoted 45 lines

Hi

Am 25.11.20 um 17:32 schrieb Daniel Vetter:

Quoted 4 lines

[...]
I guess full locking is required :-/ I'm not exactly sure how to make this happen with the current plethora of helpers ... I think we need an _locked version of vmap/vunmap callbacks in drm_gem_object_funcs.

I think we might be able to get away without new callbacks.

I looked through the sources that implement and use vmap. All the implementations are without taking resv locks. For locking, we can wrap them in lock/unlock pairs. Having something like drm_gem_vmap_unlocked() that locks and vmaps should make this easy.

In terms of implementation, only vram helpers do a pin+map in their vmap code. And as I mentioned before, this is actually wrong. The pattern dates back to when the code was still in individual drivers. It's time to clean this up. Vram helpers can use drm_gem_ttm_vmap() instead.

Finally, there aren't that many users of vmap. A few drivers use it while blitting framebuffers into HW buffers and ast does some permanent mapping of the cursor BO. All this is trivial to turn into small pairs of lock+vmap and vunmap+unlock.

That leaves generic fbdev. The shadow buffer is also trivial to fix, as outlined during this discussion.

The code for fbdev in hardware buffers is a special case. It vmaps the buffer during initialization and only vunmaps it during shutdown. As this has worked so far without locking, I'd leave it as it is and put a big comment next to is.

Hardware fbdev buffers is only required by few drivers; namely those that require the CONFIG_DRM_FBDEV_LEAK_PHYS_SMEM config option to work. We should consider to make the fbdev shadow buffer the default and have drivers opt-in for the hardware buffer, if they need it.

Quoted 5 lines

And then document that if the callers of the _locked version wants a permanent mapping, it also needs to pin it. Plus I guess ideally implement the unlocked/permanent versions in terms of that, so that drivers only have to implement one or the other.

Quoted 8 lines

For my understanding, pinning is only done in prepare_fb code. And ast pins its cursor BOs into vram. We should document to hols vmap/vunmap only for time and cover them with resv locks. Pinning is for cases where the hardware requires buffers in a special location, but does not protect against concurrent threat. I think those are the implicit rules already.

I updated the radeon patchset, where all this appears to be working well.

Hm yeah if you want to do the full change, then that works out too. It's just a pile of work.

Quoted 6 lines
Quoted 4 lines

But if we can finish off with an dma_resv_assert_locked in dma_buf_vmap/vunmap, then I think that's ok. It does mean that exporters must implement vmap caching, or performance will be terrible. So quite some update for the dma-buf docs.

Yeah, I remember a bug report about frequent page-table modifications wrt to vram helpers. So we implemented the lazy unmapping / vmap caching, as suggested by Christian back them. My guess is that anything TTM-based can use a similar pattern. Christian probably knows the corner cases.

My memory is failing me, what corner case are you talking about?

Christian.

Quoted 150 lines

CMA seems obviously working correctly already.

For SHMEM, I'd have to figure out the reference counting of the pages involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could be moved into the free callback, plus a few other modifications.

Best regards
Thomas

Quoted 141 lines

But if you're willing to do all that conversion of callers, then of course I'm not stopping you. Not at all, it's great to see that kind of maze untangled.
-Daniel

Quoted 132 lines

Best regards
Thomas

Quoted 120 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 109 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 87 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 77 lines

Regards,
Christian.

Quoted 72 lines

Best regards
Thomas

Quoted 67 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 44 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 30 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

author maintainer
#32 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Hi

Am 26.11.20 um 13:08 schrieb Christian König:

Quoted 4 lines
Quoted 2 lines

[...]
Yeah, I remember a bug report about frequent page-table modifications wrt to vram helpers. So we implemented the lazy unmapping / vmap caching, as suggested by Christian back them. My guess is that anything TTM-based can use a similar pattern. Christian probably knows the corner cases.

My memory is failing me, what corner case are you talking about?

Haha! :D What I meant was that if there were corner cases, you'd know about them. From your comment, I guess there are none.

Best regards
Thomas

Quoted 159 lines

Christian.

Quoted 151 lines

CMA seems obviously working correctly already.

For SHMEM, I'd have to figure out the reference counting of the pages involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could be moved into the free callback, plus a few other modifications.

Best regards
Thomas

Quoted 142 lines

But if you're willing to do all that conversion of callers, then of course I'm not stopping you. Not at all, it's great to see that kind of maze untangled.
-Daniel

Quoted 133 lines

Best regards
Thomas

Quoted 121 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 110 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 88 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 78 lines

Regards,
Christian.

Quoted 73 lines

Best regards
Thomas

Quoted 68 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 45 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 30 lines

Please note that the reservation lock you need to take here is part of the GEM object.

Usually we design things in the way that the code needs to take a lock which protects an object, then do some operations with the object and then release the lock again.

Having in the lock inside the operation can be done as well, but returning with it is kind of unusual design.

Quoted 2 lines

Sorry for the noob questions. I'm still trying to understand the
implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are talking about
here. We need to take that for a couple of different operations, vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Quoted 3 lines

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

maintainer
#33 lore raw

Re: [PATCH 4/7] drm/radeon: Pin buffers while they are vmap'ed

Am 26.11.20 um 13:14 schrieb Thomas Zimmermann:

Quoted 9 lines

Hi

Am 26.11.20 um 13:08 schrieb Christian König:

Quoted 4 lines
Quoted 2 lines

[...]
Yeah, I remember a bug report about frequent page-table modifications wrt to vram helpers. So we implemented the lazy unmapping / vmap caching, as suggested by Christian back them. My guess is that anything TTM-based can use a similar pattern. Christian probably knows the corner cases.

My memory is failing me, what corner case are you talking about?

Haha! :D What I meant was that if there were corner cases, you'd know about them. From your comment, I guess there are none.

Ah, ok :) I was wondering for a moment if I have missed something.

Cheers,
Christian.

Quoted 164 lines

Best regards
Thomas

Quoted 159 lines

Christian.

Quoted 151 lines

CMA seems obviously working correctly already.

For SHMEM, I'd have to figure out the reference counting of the pages involved. My guess is that the vunmap in drm_gem_shmem_vunmap() could be moved into the free callback, plus a few other modifications.

Best regards
Thomas

Quoted 142 lines

But if you're willing to do all that conversion of callers, then of course I'm not stopping you. Not at all, it's great to see that kind of maze untangled.
-Daniel

Quoted 133 lines

Best regards
Thomas

Quoted 121 lines

That should give us at least some way forward to gradually conver all the drivers and helpers over to dma_resv locking.
-Daniel

Quoted 110 lines

The pin count is currently maintained by the vmap implementation in vram helpers. Calling vmap is an implicit pin; calling vunmap is an implicit unpin. This prevents eviction in the damage worker. But now I was told than pinning is only for BOs that are controlled by userspace and internal users should acquire the resv lock. So vram helpers have to be fixed, actually.

In vram helpers, unmapping does not mean eviction. The unmap operation only marks the BO as unmappable. The real unmap happens when the eviction takes place. This avoids many modifications to the page tables. IOW an unpinned, unmapped BO will remain in VRAM until the memory is actually needed.

Best regards
Thomas

Quoted 88 lines

So I'm still not seeing how this can go boom.

Now long term it'd be nice to cut everything over to dma_resv locking, but the issue there is that beyond ttm, none of the helpers (and few of the drivers) use dma_resv. So this is a fairly big uphill battle. Quick interim fix seems like the right solution to me.
-Daniel

Quoted 78 lines

Regards,
Christian.

Quoted 73 lines

Best regards
Thomas

Quoted 68 lines
Quoted 6 lines

There's no recursion taking place, so I guess the reservation lock could be
acquired/release in drm_client_buffer_vmap/vunmap(), or a separate pair of
DRM client functions could do the locking.

Given how this "do the right locking" is a can of worms (and I think it's
worse than what you dug out already) I think the fb_helper.lock hack is perfectly good enough.

I'm also somewhat worried that starting to use dma_resv lock in generic code, while many helpers/drivers still have their hand-rolled locking, will make conversion over to dma_resv needlessly more complicated. -Daniel

Quoted 45 lines

Best regards
Thomas

[1] https://cgit.freedesktop.org/drm/drm-tip/tree/drivers/gpu/drm/drm_fb_helper.c?id=ac60f3f3090115d21f028bffa2dcfb67f695c4f2#n394

Quoted 30 lines

Please note that the reservation lock you need to take here is part of
the GEM object.

Usually we design things in the way that the code needs to take a lock
which protects an object, then do some operations with the object and
then release the lock again.

Having in the lock inside the operation can be done as well, but
returning with it is kind of unusual design.

Sorry for the noob questions. I'm still trying to understand the
implications of acquiring these locks.

Well this is the reservation lock of the GEM object we are
talking about
here. We need to take that for a couple of different operations,
vmap/vunmap doesn't sound like a special case to me.

Regards,
Christian.

Best regards
Thomas

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer
Signature · 6 lines
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Maxfeldstr. 5, 90409 Nürnberg, Germany
(HRB 36809, AG Nürnberg)
Geschäftsführer: Felix Imendörffer

_______________________________________________
dri-devel mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/dri-devel

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

author maintainer
#34 lore raw

PATCH 5/7 drm/radeon: Replace framebuffer console with generic implementation

In reply to Thomas Zimmermann

The fbdev code in radeon can be replaced by the generic fbdev helpers. This allows for using fbdev shadow buffers and IGT testcases.

Generic fbdev acts like a regular DRM client and has to be initialized after registering the DRM device. Change the driver initialization accordingly. The release happens automatically during shutdown.

---
 drivers/gpu/drm/radeon/radeon_display.c |   2 -
 drivers/gpu/drm/radeon/radeon_drv.c     |   5 +-
 drivers/gpu/drm/radeon/radeon_fb.c      | 337 +-----------------------
 drivers/gpu/drm/radeon/radeon_kms.c     |  12 -
 drivers/gpu/drm/radeon/radeon_mode.h    |   7 +-
 5 files changed, 16 insertions(+), 347 deletions(-)
drivers/gpu/drm/radeon/radeon_display.c +0 -2
index b79686cf8bdb..a7bae33636c3 100644
@@ -1638,7 +1638,6 @@ int radeon_modeset_init(struct radeon_device *rdev)
 	/* setup afmt */
 	radeon_afmt_init(rdev);
 
-	radeon_fbdev_init(rdev);
 	drm_kms_helper_poll_init(rdev->ddev);
 
 	/* do pm late init */
@@ -1653,7 +1652,6 @@ void radeon_modeset_fini(struct radeon_device *rdev)
 		drm_kms_helper_poll_fini(rdev->ddev);
 		radeon_hpd_fini(rdev);
 		drm_helper_force_disable_all(rdev->ddev);
-		radeon_fbdev_fini(rdev);
 		radeon_afmt_fini(rdev);
 		drm_mode_config_cleanup(rdev->ddev);
 		rdev->mode_info.mode_config_initialized = false;
drivers/gpu/drm/radeon/radeon_drv.c +3 -2
index bfadb799d6a3..63c3bf1a3a3c 100644
@@ -114,7 +114,6 @@
 #define KMS_DRIVER_PATCHLEVEL	0
 int radeon_driver_load_kms(struct drm_device *dev, unsigned long flags);
 void radeon_driver_unload_kms(struct drm_device *dev);
-void radeon_driver_lastclose_kms(struct drm_device *dev);
 int radeon_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
 void radeon_driver_postclose_kms(struct drm_device *dev,
 				 struct drm_file *file_priv);
@@ -374,6 +373,8 @@ static int radeon_pci_probe(struct pci_dev *pdev,
 	if (ret)
 		goto err_agp;
 
+	radeon_fbdev_init(dev->dev_private);
+
 	return 0;
 
 err_agp:
@@ -635,7 +636,7 @@ static const struct drm_driver kms_driver = {
 	.load = radeon_driver_load_kms,
 	.open = radeon_driver_open_kms,
 	.postclose = radeon_driver_postclose_kms,
-	.lastclose = radeon_driver_lastclose_kms,
+	.lastclose = drm_fb_helper_lastclose,
 	.unload = radeon_driver_unload_kms,
 	.irq_preinstall = radeon_driver_irq_preinstall_kms,
 	.irq_postinstall = radeon_driver_irq_postinstall_kms,
drivers/gpu/drm/radeon/radeon_fb.c +12 -325

372 lines not on this page · raw message

drivers/gpu/drm/radeon/radeon_kms.c +0 -12
index 75b038740ea8..76e18e589ccd 100644
@@ -608,18 +608,6 @@ int radeon_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
 /*
  * Outdated mess for old drm with Xorg being in charge (void function now).
  */
-/**
- * radeon_driver_lastclose_kms - drm callback for last close
- *
- * @dev: drm dev pointer
- *
- * Switch vga_switcheroo state after last close (all asics).
- */
-void radeon_driver_lastclose_kms(struct drm_device *dev)
-{
-	drm_fb_helper_lastclose(dev);
-	vga_switcheroo_process_delayed_switch();
-}
 
 /**
  * radeon_driver_open_kms - drm callback for open
drivers/gpu/drm/radeon/radeon_mode.h +1 -6
index c7f223743d46..fe192ef0aafa 100644
@@ -230,8 +230,6 @@ enum radeon_dvo_chip {
 	DVO_SIL1178,
 };
 
-struct radeon_fbdev;
-
 struct radeon_afmt {
 	bool enabled;
 	int offset;
@@ -268,8 +266,6 @@ struct radeon_mode_info {
 	struct edid *bios_hardcoded_edid;
 	int bios_hardcoded_edid_size;
 
-	/* pointer to fbdev info structure */
-	struct radeon_fbdev *rfbdev;
 	/* firmware flags */
 	u16 firmware_flags;
 	/* pointer to backlight encoder */
@@ -979,8 +975,7 @@ void dce4_program_fmt(struct drm_encoder *encoder);
 void dce8_program_fmt(struct drm_encoder *encoder);
 
 /* fbdev layer */
-int radeon_fbdev_init(struct radeon_device *rdev);
-void radeon_fbdev_fini(struct radeon_device *rdev);
+void radeon_fbdev_init(struct radeon_device *rdev);
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state);
 bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj);
 
Signature · 6 lines
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#35 lore raw

PATCH 6/7 drm/radeon: Use fbdev shadow fb

In reply to Thomas Zimmermann

Fbdev framebuffer addresses are exported and mmap'ed to userspace, so the framebuffer may not change its location. This creates memory pressure on devices with little video RAM. Radeon mitigates this problem by reducing the framebuffer's color depth on devices with 32 MiB or less.

Fully resolve the problem by enabling generic fbdev's shadow FB. All userspace-visible memory is located in the shadow buffer. The underlying BO can be evicted to system memory as needed, thus freeing video RAM for more important uses.

Switching to a shadow framebuffer also allows to remove a special case in BO handling, where the fbdev's BO was not evicted. Fbdev should now be treated an any other framebuffer BO

---
 drivers/gpu/drm/radeon/radeon_device.c | 11 ++++-------
 drivers/gpu/drm/radeon/radeon_fb.c     | 17 ++---------------
 drivers/gpu/drm/radeon/radeon_mode.h   |  1 -
 3 files changed, 6 insertions(+), 23 deletions(-)
drivers/gpu/drm/radeon/radeon_device.c +4 -7
index 266e3cbbd09b..87c14f6369ee 100644
@@ -1605,13 +1605,10 @@ int radeon_suspend_kms(struct drm_device *dev, bool suspend,
 			continue;
 		}
 		robj = gem_to_radeon_bo(fb->obj[0]);
-		/* don't unpin kernel fb objects */
-		if (!radeon_fbdev_robj_is_fb(rdev, robj)) {
-			r = radeon_bo_reserve(robj, false);
-			if (r == 0) {
-				radeon_bo_unpin(robj);
-				radeon_bo_unreserve(robj);
-			}
+		r = radeon_bo_reserve(robj, false);
+		if (r == 0) {
+			radeon_bo_unpin(robj);
+			radeon_bo_unreserve(robj);
 		}
 	}
 	/* evict vram memory */
drivers/gpu/drm/radeon/radeon_fb.c +2 -15
index 50fdc2aaa463..c687d14a1b93 100644
@@ -59,14 +59,12 @@ void radeon_fbdev_init(struct radeon_device *rdev)
 	struct drm_device *ddev = rdev->ddev;
 	int bpp_sel = 32;
 
-	/* select 8 bpp console on 8MB cards, or 16 bpp on RN50 or 32MB */
-	if (rdev->mc.real_vram_size <= (8*1024*1024))
-		bpp_sel = 8;
-	else if (ASIC_IS_RN50(rdev) || rdev->mc.real_vram_size <= (32*1024*1024))
+	if (ASIC_IS_RN50(rdev))
 		bpp_sel = 16;
 
 	/* radeon resume is fragile and needs a vt switch to help it along */
 	ddev->mode_config.require_vt_switch_fbdev = true;
+	ddev->mode_config.prefer_shadow_fbdev = true;
 
 	drm_fbdev_generic_setup(ddev, bpp_sel);
 }
@@ -75,14 +73,3 @@ void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state)
 {
 	drm_fb_helper_set_suspend(rdev->ddev->fb_helper, state);
 }
-
-bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj)
-{
-	if (!rdev->ddev->fb_helper)
-		return false;
-
-	if (gem_to_radeon_bo(rdev->ddev->fb_helper->buffer->gem) != robj)
-		return false;
-
-	return true;
-}
drivers/gpu/drm/radeon/radeon_mode.h +0 -1
index fe192ef0aafa..197423ad42c1 100644
@@ -977,7 +977,6 @@ void dce8_program_fmt(struct drm_encoder *encoder);
 /* fbdev layer */
 void radeon_fbdev_init(struct radeon_device *rdev);
 void radeon_fbdev_set_suspend(struct radeon_device *rdev, int state);
-bool radeon_fbdev_robj_is_fb(struct radeon_device *rdev, struct radeon_bo *robj);
 
 void radeon_crtc_handle_vblank(struct radeon_device *rdev, int crtc_id);
 
Signature · 6 lines
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
author maintainer
#36 lore raw

PATCH 7/7 drm/radeon: Move radeon_align_pitch() next to its only caller

In reply to Thomas Zimmermann

Allows to declare the function as static. The tiled parameter is always false, so it is being removed.

---
 drivers/gpu/drm/radeon/radeon_fb.c   | 24 ------------------------
 drivers/gpu/drm/radeon/radeon_gem.c  | 26 +++++++++++++++++++++++++-
 drivers/gpu/drm/radeon/radeon_mode.h |  2 --
 3 files changed, 25 insertions(+), 27 deletions(-)
drivers/gpu/drm/radeon/radeon_fb.c +0 -24
index c687d14a1b93..0b89dd4f4af6 100644
@@ -30,30 +30,6 @@
 
 #include "radeon.h"
 
-int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp, bool tiled)
-{
-	int aligned = width;
-	int align_large = (ASIC_IS_AVIVO(rdev)) || tiled;
-	int pitch_mask = 0;
-
-	switch (cpp) {
-	case 1:
-		pitch_mask = align_large ? 255 : 127;
-		break;
-	case 2:
-		pitch_mask = align_large ? 127 : 31;
-		break;
-	case 3:
-	case 4:
-		pitch_mask = align_large ? 63 : 15;
-		break;
-	}
-
-	aligned += pitch_mask;
-	aligned &= ~pitch_mask;
-	return aligned * cpp;
-}
-
 void radeon_fbdev_init(struct radeon_device *rdev)
 {
 	struct drm_device *ddev = rdev->ddev;
drivers/gpu/drm/radeon/radeon_gem.c +25 -1
index eaf7fc9a7b07..e19956c3309f 100644
@@ -816,6 +816,30 @@ int radeon_gem_op_ioctl(struct drm_device *dev, void *data,
 	return r;
 }
 
+static int radeon_align_pitch(struct radeon_device *rdev, int width, int cpp)
+{
+	int aligned = width;
+	int align_large = ASIC_IS_AVIVO(rdev);
+	int pitch_mask = 0;
+
+	switch (cpp) {
+	case 1:
+		pitch_mask = align_large ? 255 : 127;
+		break;
+	case 2:
+		pitch_mask = align_large ? 127 : 31;
+		break;
+	case 3:
+	case 4:
+		pitch_mask = align_large ? 63 : 15;
+		break;
+	}
+
+	aligned += pitch_mask;
+	aligned &= ~pitch_mask;
+	return aligned * cpp;
+}
+
 int radeon_mode_dumb_create(struct drm_file *file_priv,
 			    struct drm_device *dev,
 			    struct drm_mode_create_dumb *args)
@@ -826,7 +850,7 @@ int radeon_mode_dumb_create(struct drm_file *file_priv,
 	int r;
 
 	args->pitch = radeon_align_pitch(rdev, args->width,
-					 DIV_ROUND_UP(args->bpp, 8), 0);
+					 DIV_ROUND_UP(args->bpp, 8));
 	args->size = args->pitch * args->height;
 	args->size = ALIGN(args->size, PAGE_SIZE);
 
drivers/gpu/drm/radeon/radeon_mode.h +0 -2
index 197423ad42c1..50ba2b5edb93 100644
@@ -982,8 +982,6 @@ void radeon_crtc_handle_vblank(struct radeon_device *rdev, int crtc_id);
 
 void radeon_crtc_handle_flip(struct radeon_device *rdev, int crtc_id);
 
-int radeon_align_pitch(struct radeon_device *rdev, int width, int bpp, bool tiled);
-
 /* mst */
 int radeon_dp_mst_init(struct radeon_connector *radeon_connector);
 int radeon_dp_mst_probe(struct radeon_connector *radeon_connector);
Signature · 6 lines
2.29.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx