Skip to contentSkip to conversation

amdgpu: Visible VRAM Management Improvements, next take

PATCH In review Michel Dänzer 3 participants 16 messages Jul 6, 2017 – Jul 11, 2017 7 messages not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover amdgpu: Visible VRAM Management Improvements, next take

From: Michel Dänzer <[email protected]>

This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.

John Brooks (4):
drm/amdgpu: Add vis_vramlimit module parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faults

Michel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  7 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  4 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 45 +++++++++++++++
 5 files changed, 157 insertions(+), 40 deletions(-)
Signature · 6 lines
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-1-michel-otUistvHUpPR7s880joybQ@public.gmane.org

#2 lore raw

PATCH v2 1/5 drm/amdgpu: Add vis_vramlimit module parameter

From: John Brooks <[email protected]>

Allow specifying a limit on visible VRAM via a module parameter. This is helpful for testing performance under visible VRAM pressure.

v2: Add cast to 64-bit (Christian König)

---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h     | 1 +
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 ++++
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 8 ++++++++
 3 files changed, 13 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
index 8b4e8ed9e9dc..b95c1074d42c 100644
@@ -75,6 +75,7 @@
  */
 extern int amdgpu_modeset;
 extern int amdgpu_vram_limit;
+extern int amdgpu_vis_vram_limit;
 extern int amdgpu_gart_size;
 extern int amdgpu_moverate;
 extern int amdgpu_benchmarking;
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +4 -0
index 271c9d528af6..72ff7223447e 100644
@@ -74,6 +74,7 @@
 #define KMS_DRIVER_PATCHLEVEL	0
 
 int amdgpu_vram_limit = 0;
+int amdgpu_vis_vram_limit = 0;
 int amdgpu_gart_size = -1; /* auto */
 int amdgpu_moverate = -1; /* auto */
 int amdgpu_benchmarking = 0;
@@ -121,6 +122,9 @@ int amdgpu_lbpw = -1;
 MODULE_PARM_DESC(vramlimit, "Restrict VRAM for testing, in megabytes");
 module_param_named(vramlimit, amdgpu_vram_limit, int, 0600);
 
+MODULE_PARM_DESC(vis_vramlimit, "Restrict visible VRAM for testing, in megabytes");
+module_param_named(vis_vramlimit, amdgpu_vis_vram_limit, int, 0444);
+
 MODULE_PARM_DESC(gartsize, "Size of PCIE/IGP gart to setup in megabytes (32, 64, etc., -1 = auto)");
 module_param_named(gartsize, amdgpu_gart_size, int, 0600);
 
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +8 -0
index a81c4e5e9f6b..9d4861aea18a 100644
@@ -1092,6 +1092,7 @@ static struct ttm_bo_driver amdgpu_bo_driver = {
 int amdgpu_ttm_init(struct amdgpu_device *adev)
 {
 	int r;
+	u64 vis_vram_limit;
 
 	r = amdgpu_ttm_global_init(adev);
 	if (r) {
@@ -1115,6 +1116,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
 		DRM_ERROR("Failed initializing VRAM heap.\n");
 		return r;
 	}
+
+	/* Reduce size of CPU-visible VRAM if requested */
+	vis_vram_limit = (u64)amdgpu_vis_vram_limit * 1024 * 1024;
+	if (amdgpu_vis_vram_limit > 0 &&
+	    vis_vram_limit <= adev->mc.visible_vram_size)
+		adev->mc.visible_vram_size = vis_vram_limit;
+
 	/* Change the size here instead of the init above so only lpfn is affected */
 	amdgpu_ttm_set_active_vram_size(adev, adev->mc.visible_vram_size);
 
Signature · 6 lines
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
#3 lore raw

PATCH v3 2/5 drm/amdgpu: Throttle visible VRAM moves separately

In reply to a message not in the archive

From: John Brooks <[email protected]>

The BO move throttling code is designed to allow VRAM to fill quickly if it is relatively empty. However, this does not take into account situations where the visible VRAM is smaller than total VRAM, and total VRAM may not be close to full but the visible VRAM segment is under pressure. In such situations, visible VRAM would experience unrestricted swapping and performance would drop.

Add a separate counter specifically for moves involving visible VRAM, and check it before moving BOs there.

v2: Only perform calculations for separate counter if visible VRAM is
smaller than total VRAM. (Michel Dänzer)
v3: [Michel Dänzer]
* Use BO's location rather than the AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag to determine whether to account a move for visible VRAM in most cases.
* Use a single

	if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {

block in amdgpu_cs_get_threshold_for_moves.

Fixes 95844d20ae02 (drm/amdgpu: throttle buffer migrations at CS using a fixed MBps limit (v2))
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 12 +++-
 3 files changed, 87 insertions(+), 23 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +5 -1
index b95c1074d42c..463d6c241157 100644
@@ -1162,7 +1162,9 @@ struct amdgpu_cs_parser {
 	struct list_head		validated;
 	struct dma_fence		*fence;
 	uint64_t			bytes_moved_threshold;
+	uint64_t			bytes_moved_vis_threshold;
 	uint64_t			bytes_moved;
+	uint64_t			bytes_moved_vis;
 	struct amdgpu_bo_list_entry	*evictable;
 
 	/* user fence */
@@ -1596,6 +1598,7 @@ struct amdgpu_device {
 		spinlock_t		lock;
 		s64			last_update_us;
 		s64			accum_us; /* accumulated microseconds */
+		s64			accum_us_vis; /* for visible VRAM */
 		u32			log2_max_MBps;
 	} mm_stats;
 
@@ -1892,7 +1895,8 @@ void amdgpu_pci_config_reset(struct amdgpu_device *adev);
 bool amdgpu_need_post(struct amdgpu_device *adev);
 void amdgpu_update_display_priority(struct amdgpu_device *adev);
 
-void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes);
+void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
+				  u64 num_vis_bytes);
 void amdgpu_ttm_placement_from_domain(struct amdgpu_bo *abo, u32 domain);
 bool amdgpu_ttm_bo_is_amdgpu_bo(struct ttm_buffer_object *bo);
 int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page **pages);
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +73 -19

171 lines not on this page · raw message

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +9 -3
index a85e75327456..e429829ae93d 100644
@@ -322,7 +322,7 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
 	struct amdgpu_bo *bo;
 	enum ttm_bo_type type;
 	unsigned long page_align;
-	u64 initial_bytes_moved;
+	u64 initial_bytes_moved, bytes_moved;
 	size_t acc_size;
 	int r;
 
@@ -398,8 +398,14 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
 	r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
 				 &bo->placement, page_align, !kernel, NULL,
 				 acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
-	amdgpu_cs_report_moved_bytes(adev,
-		atomic64_read(&adev->num_bytes_moved) - initial_bytes_moved);
+	bytes_moved = atomic64_read(&adev->num_bytes_moved) -
+		      initial_bytes_moved;
+	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
+	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)
+		amdgpu_cs_report_moved_bytes(adev, bytes_moved, bytes_moved);
+	else
+		amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0);
 
 	if (unlikely(r != 0))
 		return r;
Signature · 6 lines
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-3-michel-otUistvHUpPR7s880joybQ@public.gmane.org

Reviewed-by
#5 lore raw

Re: [PATCH v3 2/5] drm/amdgpu: Throttle visible VRAM moves separately

Am 06.07.2017 um 12:51 schrieb Michel Dänzer:

Quoted 20 lines

From: John Brooks <[email protected]>

The BO move throttling code is designed to allow VRAM to fill quickly if it is relatively empty. However, this does not take into account situations where the visible VRAM is smaller than total VRAM, and total VRAM may not be close to full but the visible VRAM segment is under pressure. In such situations, visible VRAM would experience unrestricted swapping and performance would drop.

Add a separate counter specifically for moves involving visible VRAM, and check it before moving BOs there.

v2: Only perform calculations for separate counter if visible VRAM is
smaller than total VRAM. (Michel Dänzer)
v3: [Michel Dänzer]
* Use BO's location rather than the AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag to determine whether to account a move for visible VRAM in most cases.
* Use a single

Quoted 7 lines
	if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {

block in amdgpu_cs_get_threshold_for_moves.

Fixes 95844d20ae02 (drm/amdgpu: throttle buffer migrations at CS using a fixed MBps limit (v2))
Quoted 242 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 12 +++-
  3 files changed, 87 insertions(+), 23 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +5 -1
index b95c1074d42c..463d6c241157 100644
@@ -1162,7 +1162,9 @@ struct amdgpu_cs_parser {
  	struct list_head		validated;
  	struct dma_fence		*fence;
  	uint64_t			bytes_moved_threshold;
+	uint64_t			bytes_moved_vis_threshold;
  	uint64_t			bytes_moved;
+	uint64_t			bytes_moved_vis;
  	struct amdgpu_bo_list_entry	*evictable;
 
  	/* user fence */
@@ -1596,6 +1598,7 @@ struct amdgpu_device {
  		spinlock_t		lock;
  		s64			last_update_us;
  		s64			accum_us; /* accumulated microseconds */
+		s64			accum_us_vis; /* for visible VRAM */
  		u32			log2_max_MBps;
  	} mm_stats;
 
@@ -1892,7 +1895,8 @@ void amdgpu_pci_config_reset(struct amdgpu_device *adev);
  bool amdgpu_need_post(struct amdgpu_device *adev);
  void amdgpu_update_display_priority(struct amdgpu_device *adev);
 
-void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes);
+void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
+				  u64 num_vis_bytes);
  void amdgpu_ttm_placement_from_domain(struct amdgpu_bo *abo, u32 domain);
  bool amdgpu_ttm_bo_is_amdgpu_bo(struct ttm_buffer_object *bo);
  int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page **pages);
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +73 -19
index 82131d70a06b..44ec11d4d733 100644
@@ -220,10 +220,11 @@ static s64 bytes_to_us(struct amdgpu_device *adev, u64 bytes)
   * ticks. The accumulated microseconds (us) are converted to bytes and
   * returned.
   */
-static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)
+static void amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev,
+					      u64 *max_bytes,
+					      u64 *max_vis_bytes)
  {
  	s64 time_us, increment_us;
-	u64 max_bytes;
  	u64 free_vram, total_vram, used_vram;
 
  	/* Allow a maximum of 200 accumulated ms. This is basically per-IB
@@ -235,8 +236,11 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)
  	 */
  	const s64 us_upper_bound = 200000;
 
-	if (!adev->mm_stats.log2_max_MBps)
-		return 0;
+	if (!adev->mm_stats.log2_max_MBps) {
+		*max_bytes = 0;
+		*max_vis_bytes = 0;
+		return;
+	}
 
  	total_vram = adev->mc.real_vram_size - adev->vram_pin_size;
  	used_vram = atomic64_read(&adev->vram_usage);
@@ -277,23 +281,45 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)
  		adev->mm_stats.accum_us = max(min_us, adev->mm_stats.accum_us);
  	}
 
-	/* This returns 0 if the driver is in debt to disallow (optional)
+	/* This is set to 0 if the driver is in debt to disallow (optional)
  	 * buffer moves.
  	 */
-	max_bytes = us_to_bytes(adev, adev->mm_stats.accum_us);
+	*max_bytes = us_to_bytes(adev, adev->mm_stats.accum_us);
+
+	/* Do the same for visible VRAM if half of it is free */
+	if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {
+		u64 total_vis_vram = adev->mc.visible_vram_size;
+		u64 used_vis_vram = atomic64_read(&adev->vram_vis_usage);
+
+		if (used_vis_vram < total_vis_vram) {
+			u64 free_vis_vram = total_vis_vram - used_vis_vram;
+			adev->mm_stats.accum_us_vis = min(adev->mm_stats.accum_us_vis +
+							  increment_us, us_upper_bound);
+
+			if (free_vis_vram >= total_vis_vram / 2)
+				adev->mm_stats.accum_us_vis =
+					max(bytes_to_us(adev, free_vis_vram / 2),
+					    adev->mm_stats.accum_us_vis);
+		}
+
+		*max_vis_bytes = us_to_bytes(adev, adev->mm_stats.accum_us_vis);
+	} else {
+		*max_vis_bytes = 0;
+	}
 
  	spin_unlock(&adev->mm_stats.lock);
-	return max_bytes;
  }
 
  /* Report how many bytes have really been moved for the last command
   * submission. This can result in a debt that can stop buffer migrations
   * temporarily.
   */
-void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes)
+void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
+				  u64 num_vis_bytes)
  {
  	spin_lock(&adev->mm_stats.lock);
  	adev->mm_stats.accum_us -= bytes_to_us(adev, num_bytes);
+	adev->mm_stats.accum_us_vis -= bytes_to_us(adev, num_vis_bytes);
  	spin_unlock(&adev->mm_stats.lock);
  }
 
@@ -301,7 +327,7 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p,
  				 struct amdgpu_bo *bo)
  {
  	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-	u64 initial_bytes_moved;
+	u64 initial_bytes_moved, bytes_moved;
  	uint32_t domain;
  	int r;
 
@@ -311,17 +337,35 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p,
  	/* Don't move this buffer if we have depleted our allowance
  	 * to move it. Don't move anything if the threshold is zero.
  	 */
-	if (p->bytes_moved < p->bytes_moved_threshold)
-		domain = bo->prefered_domains;
-	else
+	if (p->bytes_moved < p->bytes_moved_threshold) {
+		if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+		    (bo->flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED)) {
+			/* And don't move a CPU_ACCESS_REQUIRED BO to limited
+			 * visible VRAM if we've depleted our allowance to do
+			 * that.
+			 */
+			if (p->bytes_moved_vis < p->bytes_moved_vis_threshold)
+				domain = bo->prefered_domains;
+			else
+				domain = bo->allowed_domains;
+		} else {
+			domain = bo->prefered_domains;
+		}
+	} else {
  		domain = bo->allowed_domains;
+	}
 
  retry:
  	amdgpu_ttm_placement_from_domain(bo, domain);
  	initial_bytes_moved = atomic64_read(&adev->num_bytes_moved);
  	r = ttm_bo_validate(&bo->tbo, &bo->placement, true, false);
-	p->bytes_moved += atomic64_read(&adev->num_bytes_moved) -
-		initial_bytes_moved;
+	bytes_moved = atomic64_read(&adev->num_bytes_moved) -
+		      initial_bytes_moved;
+	p->bytes_moved += bytes_moved;
+	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
+	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)
+		p->bytes_moved_vis += bytes_moved;
 
  	if (unlikely(r == -ENOMEM) && domain != bo->allowed_domains) {
  		domain = bo->allowed_domains;
@@ -347,7 +391,8 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,
  		struct amdgpu_bo_list_entry *candidate = p->evictable;
  		struct amdgpu_bo *bo = candidate->robj;
  		struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-		u64 initial_bytes_moved;
+		u64 initial_bytes_moved, bytes_moved;
+		bool update_bytes_moved_vis;
  		uint32_t other;
 
  		/* If we reached our current BO we can forget it */
@@ -367,10 +412,17 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,
 
  		/* Good we can try to move this BO somewhere else */
  		amdgpu_ttm_placement_from_domain(bo, other);
+		update_bytes_moved_vis =
+			adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+			bo->tbo.mem.mem_type == TTM_PL_VRAM &&
+			bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT;
  		initial_bytes_moved = atomic64_read(&adev->num_bytes_moved);
  		r = ttm_bo_validate(&bo->tbo, &bo->placement, true, false);
-		p->bytes_moved += atomic64_read(&adev->num_bytes_moved) -
+		bytes_moved = atomic64_read(&adev->num_bytes_moved) -
  			initial_bytes_moved;
+		p->bytes_moved += bytes_moved;
+		if (update_bytes_moved_vis)
+			p->bytes_moved_vis += bytes_moved;
 
  		if (unlikely(r))
  			break;
@@ -550,8 +602,10 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
  		list_splice(&need_pages, &p->validated);
  	}
 
-	p->bytes_moved_threshold = amdgpu_cs_get_threshold_for_moves(p->adev);
+	amdgpu_cs_get_threshold_for_moves(p->adev, &p->bytes_moved_threshold,
+					  &p->bytes_moved_vis_threshold);
  	p->bytes_moved = 0;
+	p->bytes_moved_vis = 0;
  	p->evictable = list_last_entry(&p->validated,
  				       struct amdgpu_bo_list_entry,
  				       tv.head);
@@ -575,8 +629,8 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
  		goto error_validate;
  	}
 
-	amdgpu_cs_report_moved_bytes(p->adev, p->bytes_moved);
-
+	amdgpu_cs_report_moved_bytes(p->adev, p->bytes_moved,
+				     p->bytes_moved_vis);
  	fpriv->vm.last_eviction_counter =
  		atomic64_read(&p->adev->num_evictions);
 
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -3
index a85e75327456..e429829ae93d 100644
@@ -322,7 +322,7 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
  	struct amdgpu_bo *bo;
  	enum ttm_bo_type type;
  	unsigned long page_align;
-	u64 initial_bytes_moved;
+	u64 initial_bytes_moved, bytes_moved;
  	size_t acc_size;
  	int r;
 
@@ -398,8 +398,14 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
  	r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
  				 &bo->placement, page_align, !kernel, NULL,
  				 acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
-	amdgpu_cs_report_moved_bytes(adev,
-		atomic64_read(&adev->num_bytes_moved) - initial_bytes_moved);
+	bytes_moved = atomic64_read(&adev->num_bytes_moved) -

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_create_restricted(struct amdgpu_device *adev, · line 411

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +7 -0
+		      initial_bytes_moved;
+	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
+	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)
+		amdgpu_cs_report_moved_bytes(adev, bytes_moved, bytes_moved);
+	else
+		amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0);
 
  	if (unlikely(r != 0))
  		return r;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by
#6 lore raw

Re: [PATCH v3 2/5] drm/amdgpu: Throttle visible VRAM moves separately

In reply to a message not in the archive
Quoted 20 lines · Michel Dänzer

From: John Brooks <[email protected]>

The BO move throttling code is designed to allow VRAM to fill quickly if it is relatively empty. However, this does not take into account situations where the visible VRAM is smaller than total VRAM, and total VRAM may not be close to full but the visible VRAM segment is under pressure. In such situations, visible VRAM would experience unrestricted swapping and performance would drop.

Add a separate counter specifically for moves involving visible VRAM, and check it before moving BOs there.

v2: Only perform calculations for separate counter if visible VRAM is
smaller than total VRAM. (Michel Dänzer)
v3: [Michel Dänzer]
* Use BO's location rather than the AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag to determine whether to account a move for visible VRAM in most cases.
* Use a single

Quoted 7 lines
	if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {

block in amdgpu_cs_get_threshold_for_moves.

Fixes 95844d20ae02 (drm/amdgpu: throttle buffer migrations at CS using a fixed MBps limit (v2))

Changes look good. You can have my

if you want it.

Quoted 255 lines
---
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  6 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 12 +++-
 3 files changed, 87 insertions(+), 23 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +5 -1
index b95c1074d42c..463d6c241157 100644
@@ -1162,7 +1162,9 @@ struct amdgpu_cs_parser {
 	struct list_head		validated;
 	struct dma_fence		*fence;
 	uint64_t			bytes_moved_threshold;
+	uint64_t			bytes_moved_vis_threshold;
 	uint64_t			bytes_moved;
+	uint64_t			bytes_moved_vis;
 	struct amdgpu_bo_list_entry	*evictable;
 
 	/* user fence */
@@ -1596,6 +1598,7 @@ struct amdgpu_device {
 		spinlock_t		lock;
 		s64			last_update_us;
 		s64			accum_us; /* accumulated microseconds */
+		s64			accum_us_vis; /* for visible VRAM */
 		u32			log2_max_MBps;
 	} mm_stats;
 
@@ -1892,7 +1895,8 @@ void amdgpu_pci_config_reset(struct amdgpu_device *adev);
 bool amdgpu_need_post(struct amdgpu_device *adev);
 void amdgpu_update_display_priority(struct amdgpu_device *adev);
 
-void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes);
+void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
+				  u64 num_vis_bytes);
 void amdgpu_ttm_placement_from_domain(struct amdgpu_bo *abo, u32 domain);
 bool amdgpu_ttm_bo_is_amdgpu_bo(struct ttm_buffer_object *bo);
 int amdgpu_ttm_tt_get_user_pages(struct ttm_tt *ttm, struct page **pages);
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +73 -19
index 82131d70a06b..44ec11d4d733 100644
@@ -220,10 +220,11 @@ static s64 bytes_to_us(struct amdgpu_device *adev, u64 bytes)
  * ticks. The accumulated microseconds (us) are converted to bytes and
  * returned.
  */
-static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)
+static void amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev,
+					      u64 *max_bytes,
+					      u64 *max_vis_bytes)
 {
 	s64 time_us, increment_us;
-	u64 max_bytes;
 	u64 free_vram, total_vram, used_vram;
 
 	/* Allow a maximum of 200 accumulated ms. This is basically per-IB
@@ -235,8 +236,11 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)
 	 */
 	const s64 us_upper_bound = 200000;
 
-	if (!adev->mm_stats.log2_max_MBps)
-		return 0;
+	if (!adev->mm_stats.log2_max_MBps) {
+		*max_bytes = 0;
+		*max_vis_bytes = 0;
+		return;
+	}
 
 	total_vram = adev->mc.real_vram_size - adev->vram_pin_size;
 	used_vram = atomic64_read(&adev->vram_usage);
@@ -277,23 +281,45 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)
 		adev->mm_stats.accum_us = max(min_us, adev->mm_stats.accum_us);
 	}
 
-	/* This returns 0 if the driver is in debt to disallow (optional)
+	/* This is set to 0 if the driver is in debt to disallow (optional)
 	 * buffer moves.
 	 */
-	max_bytes = us_to_bytes(adev, adev->mm_stats.accum_us);
+	*max_bytes = us_to_bytes(adev, adev->mm_stats.accum_us);
+
+	/* Do the same for visible VRAM if half of it is free */
+	if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {
+		u64 total_vis_vram = adev->mc.visible_vram_size;
+		u64 used_vis_vram = atomic64_read(&adev->vram_vis_usage);
+
+		if (used_vis_vram < total_vis_vram) {
+			u64 free_vis_vram = total_vis_vram - used_vis_vram;
+			adev->mm_stats.accum_us_vis = min(adev->mm_stats.accum_us_vis +
+							  increment_us, us_upper_bound);
+
+			if (free_vis_vram >= total_vis_vram / 2)
+				adev->mm_stats.accum_us_vis =
+					max(bytes_to_us(adev, free_vis_vram / 2),
+					    adev->mm_stats.accum_us_vis);
+		}
+
+		*max_vis_bytes = us_to_bytes(adev, adev->mm_stats.accum_us_vis);
+	} else {
+		*max_vis_bytes = 0;
+	}
 
 	spin_unlock(&adev->mm_stats.lock);
-	return max_bytes;
 }
 
 /* Report how many bytes have really been moved for the last command
  * submission. This can result in a debt that can stop buffer migrations
  * temporarily.
  */
-void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes)
+void amdgpu_cs_report_moved_bytes(struct amdgpu_device *adev, u64 num_bytes,
+				  u64 num_vis_bytes)
 {
 	spin_lock(&adev->mm_stats.lock);
 	adev->mm_stats.accum_us -= bytes_to_us(adev, num_bytes);
+	adev->mm_stats.accum_us_vis -= bytes_to_us(adev, num_vis_bytes);
 	spin_unlock(&adev->mm_stats.lock);
 }
 
@@ -301,7 +327,7 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p,
 				 struct amdgpu_bo *bo)
 {
 	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-	u64 initial_bytes_moved;
+	u64 initial_bytes_moved, bytes_moved;
 	uint32_t domain;
 	int r;
 
@@ -311,17 +337,35 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p,
 	/* Don't move this buffer if we have depleted our allowance
 	 * to move it. Don't move anything if the threshold is zero.
 	 */
-	if (p->bytes_moved < p->bytes_moved_threshold)
-		domain = bo->prefered_domains;
-	else
+	if (p->bytes_moved < p->bytes_moved_threshold) {
+		if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+		    (bo->flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED)) {
+			/* And don't move a CPU_ACCESS_REQUIRED BO to limited
+			 * visible VRAM if we've depleted our allowance to do
+			 * that.
+			 */
+			if (p->bytes_moved_vis < p->bytes_moved_vis_threshold)
+				domain = bo->prefered_domains;
+			else
+				domain = bo->allowed_domains;
+		} else {
+			domain = bo->prefered_domains;
+		}
+	} else {
 		domain = bo->allowed_domains;
+	}
 
 retry:
 	amdgpu_ttm_placement_from_domain(bo, domain);
 	initial_bytes_moved = atomic64_read(&adev->num_bytes_moved);
 	r = ttm_bo_validate(&bo->tbo, &bo->placement, true, false);
-	p->bytes_moved += atomic64_read(&adev->num_bytes_moved) -
-		initial_bytes_moved;
+	bytes_moved = atomic64_read(&adev->num_bytes_moved) -
+		      initial_bytes_moved;
+	p->bytes_moved += bytes_moved;
+	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
+	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)
+		p->bytes_moved_vis += bytes_moved;
 
 	if (unlikely(r == -ENOMEM) && domain != bo->allowed_domains) {
 		domain = bo->allowed_domains;
@@ -347,7 +391,8 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,
 		struct amdgpu_bo_list_entry *candidate = p->evictable;
 		struct amdgpu_bo *bo = candidate->robj;
 		struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-		u64 initial_bytes_moved;
+		u64 initial_bytes_moved, bytes_moved;
+		bool update_bytes_moved_vis;
 		uint32_t other;
 
 		/* If we reached our current BO we can forget it */
@@ -367,10 +412,17 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,
 
 		/* Good we can try to move this BO somewhere else */
 		amdgpu_ttm_placement_from_domain(bo, other);
+		update_bytes_moved_vis =
+			adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+			bo->tbo.mem.mem_type == TTM_PL_VRAM &&
+			bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT;
 		initial_bytes_moved = atomic64_read(&adev->num_bytes_moved);
 		r = ttm_bo_validate(&bo->tbo, &bo->placement, true, false);
-		p->bytes_moved += atomic64_read(&adev->num_bytes_moved) -
+		bytes_moved = atomic64_read(&adev->num_bytes_moved) -
 			initial_bytes_moved;
+		p->bytes_moved += bytes_moved;
+		if (update_bytes_moved_vis)
+			p->bytes_moved_vis += bytes_moved;
 
 		if (unlikely(r))
 			break;
@@ -550,8 +602,10 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
 		list_splice(&need_pages, &p->validated);
 	}
 
-	p->bytes_moved_threshold = amdgpu_cs_get_threshold_for_moves(p->adev);
+	amdgpu_cs_get_threshold_for_moves(p->adev, &p->bytes_moved_threshold,
+					  &p->bytes_moved_vis_threshold);
 	p->bytes_moved = 0;
+	p->bytes_moved_vis = 0;
 	p->evictable = list_last_entry(&p->validated,
 				       struct amdgpu_bo_list_entry,
 				       tv.head);
@@ -575,8 +629,8 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
 		goto error_validate;
 	}
 
-	amdgpu_cs_report_moved_bytes(p->adev, p->bytes_moved);
-
+	amdgpu_cs_report_moved_bytes(p->adev, p->bytes_moved,
+				     p->bytes_moved_vis);
 	fpriv->vm.last_eviction_counter =
 		atomic64_read(&p->adev->num_evictions);
 
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +9 -3
index a85e75327456..e429829ae93d 100644
@@ -322,7 +322,7 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
 	struct amdgpu_bo *bo;
 	enum ttm_bo_type type;
 	unsigned long page_align;
-	u64 initial_bytes_moved;
+	u64 initial_bytes_moved, bytes_moved;
 	size_t acc_size;
 	int r;
 
@@ -398,8 +398,14 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
 	r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, type,
 				 &bo->placement, page_align, !kernel, NULL,
 				 acc_size, sg, resv, &amdgpu_ttm_bo_destroy);
-	amdgpu_cs_report_moved_bytes(adev,
-		atomic64_read(&adev->num_bytes_moved) - initial_bytes_moved);
+	bytes_moved = atomic64_read(&adev->num_bytes_moved) -
+		      initial_bytes_moved;
+	if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+	    bo->tbo.mem.mem_type == TTM_PL_VRAM &&
+	    bo->tbo.mem.start < adev->mc.visible_vram_size >> PAGE_SHIFT)
+		amdgpu_cs_report_moved_bytes(adev, bytes_moved, bytes_moved);
+	else
+		amdgpu_cs_report_moved_bytes(adev, bytes_moved, 0);
 
 	if (unlikely(r != 0))
 		return r;
Signature · 1 line
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#7 lore raw

PATCH v3 3/5 drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM

In reply to a message not in the archive

From: John Brooks <[email protected]>

When a BO is moved to VRAM, clear AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED. This allows it to potentially later move to invisible VRAM if the CPU does not access it again.

Setting the CPU_ACCESS flag in amdgpu_bo_fault_reserve_notify() also means that we can remove the loop to restrict lpfn to the end of visible VRAM, because amdgpu_ttm_placement_init() will do it for us.

v3 [Michel Dänzer]
* Use AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED instead of a new flag (Christian König)
* Explicitly mention amdgpu_bo_fault_reserve_notify in amdgpu_bo_move

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 20 ++++++++++----------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    |  9 +++++++++
 2 files changed, 19 insertions(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -10
index e429829ae93d..93601fbea695 100644
@@ -432,6 +432,10 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
 
 	trace_amdgpu_bo_create(bo);
 
+	/* Treat CPU_ACCESS_REQUIRED only as a hint if given by UMD */
+	if (type == ttm_bo_type_device)
+		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+
 	return 0;
 
 fail_unreserve:
@@ -945,13 +949,17 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
 {
 	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->bdev);
 	struct amdgpu_bo *abo;
-	unsigned long offset, size, lpfn;
-	int i, r;
+	unsigned long offset, size;
+	int r;
 
 	if (!amdgpu_ttm_bo_is_amdgpu_bo(bo))
 		return 0;
 
 	abo = container_of(bo, struct amdgpu_bo, tbo);
+
+	/* Remember that this BO was accessed by the CPU */
+	abo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+
 	if (bo->mem.mem_type != TTM_PL_VRAM)
 		return 0;
 
@@ -967,14 +975,6 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
 	/* hurrah the memory is not visible ! */
 	atomic64_inc(&adev->num_vram_cpu_page_faults);
 	amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM);
-	lpfn =	adev->mc.visible_vram_size >> PAGE_SHIFT;
-	for (i = 0; i < abo->placement.num_placement; i++) {
-		/* Force into visible VRAM */
-		if ((abo->placements[i].flags & TTM_PL_FLAG_VRAM) &&
-		    (!abo->placements[i].lpfn ||
-		     abo->placements[i].lpfn > lpfn))
-			abo->placements[i].lpfn = lpfn;
-	}
 	r = ttm_bo_validate(bo, &abo->placement, false, false);
 	if (unlikely(r == -ENOMEM)) {
 		amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_GTT);
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +9 -0
index 9d4861aea18a..58576375b91c 100644
@@ -493,6 +493,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo,
 		}
 	}
 
+	if (bo->type == ttm_bo_type_device &&
+	    new_mem->mem_type == TTM_PL_VRAM &&
+	    old_mem->mem_type != TTM_PL_VRAM) {
+		/* amdgpu_bo_fault_reserve_notify will re-set this if the CPU
+		 * accesses the BO after it's moved.
+		 */
+		abo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+	}
+
 	/* update statistics */
 	atomic64_add((u64)bo->num_pages << PAGE_SHIFT, &adev->num_bytes_moved);
 	return 0;
Signature · 6 lines
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-4-michel-otUistvHUpPR7s880joybQ@public.gmane.org

Reviewed-by
#9 lore raw

Re: [PATCH v3 3/5] drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM

Am 06.07.2017 um 12:51 schrieb Michel Dänzer:

Quoted 10 lines

From: John Brooks <[email protected]>

When a BO is moved to VRAM, clear AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED. This allows it to potentially later move to invisible VRAM if the CPU does not access it again.

Setting the CPU_ACCESS flag in amdgpu_bo_fault_reserve_notify() also means that we can remove the loop to restrict lpfn to the end of visible VRAM, because amdgpu_ttm_placement_init() will do it for us.

Quoted 8 lines

v3 [Michel Dänzer]
* Use AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED instead of a new flag (Christian König)
* Explicitly mention amdgpu_bo_fault_reserve_notify in amdgpu_bo_move

Quoted 65 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 20 ++++++++++----------
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    |  9 +++++++++
  2 files changed, 19 insertions(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -10
index e429829ae93d..93601fbea695 100644
@@ -432,6 +432,10 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
 
  	trace_amdgpu_bo_create(bo);
 
+	/* Treat CPU_ACCESS_REQUIRED only as a hint if given by UMD */
+	if (type == ttm_bo_type_device)
+		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+
  	return 0;
 
  fail_unreserve:
@@ -945,13 +949,17 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
  {
  	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->bdev);
  	struct amdgpu_bo *abo;
-	unsigned long offset, size, lpfn;
-	int i, r;
+	unsigned long offset, size;
+	int r;
 
  	if (!amdgpu_ttm_bo_is_amdgpu_bo(bo))
  		return 0;
 
  	abo = container_of(bo, struct amdgpu_bo, tbo);
+
+	/* Remember that this BO was accessed by the CPU */
+	abo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+
  	if (bo->mem.mem_type != TTM_PL_VRAM)
  		return 0;
 
@@ -967,14 +975,6 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
  	/* hurrah the memory is not visible ! */
  	atomic64_inc(&adev->num_vram_cpu_page_faults);
  	amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM);
-	lpfn =	adev->mc.visible_vram_size >> PAGE_SHIFT;
-	for (i = 0; i < abo->placement.num_placement; i++) {
-		/* Force into visible VRAM */
-		if ((abo->placements[i].flags & TTM_PL_FLAG_VRAM) &&
-		    (!abo->placements[i].lpfn ||
-		     abo->placements[i].lpfn > lpfn))
-			abo->placements[i].lpfn = lpfn;
-	}
  	r = ttm_bo_validate(bo, &abo->placement, false, false);
  	if (unlikely(r == -ENOMEM)) {
  		amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_GTT);
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +2 -0
index 9d4861aea18a..58576375b91c 100644
@@ -493,6 +493,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo,
  		}
  	}
 
+	if (bo->type == ttm_bo_type_device &&
+	    new_mem->mem_type == TTM_PL_VRAM &&

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c static int amdgpu_bo_move(struct ttm_buffer_object *bo, · line 507

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7 -0
+	    old_mem->mem_type != TTM_PL_VRAM) {
+		/* amdgpu_bo_fault_reserve_notify will re-set this if the CPU
+		 * accesses the BO after it's moved.
+		 */
+		abo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+	}
+
  	/* update statistics */
  	atomic64_add((u64)bo->num_pages << PAGE_SHIFT, &adev->num_bytes_moved);
  	return 0;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Reviewed-by
#10 lore raw

Re: [PATCH v3 3/5] drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM

In reply to a message not in the archive
Quoted 10 lines · Michel Dänzer

From: John Brooks <[email protected]>

When a BO is moved to VRAM, clear AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED. This allows it to potentially later move to invisible VRAM if the CPU does not access it again.

Setting the CPU_ACCESS flag in amdgpu_bo_fault_reserve_notify() also means that we can remove the loop to restrict lpfn to the end of visible VRAM, because amdgpu_ttm_placement_init() will do it for us.

Quoted 8 lines

v3 [Michel Dänzer]
* Use AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED instead of a new flag (Christian König)
* Explicitly mention amdgpu_bo_fault_reserve_notify in amdgpu_bo_move

Same.

Quoted 78 lines
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 20 ++++++++++----------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    |  9 +++++++++
 2 files changed, 19 insertions(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -10
index e429829ae93d..93601fbea695 100644
@@ -432,6 +432,10 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,
 
 	trace_amdgpu_bo_create(bo);
 
+	/* Treat CPU_ACCESS_REQUIRED only as a hint if given by UMD */
+	if (type == ttm_bo_type_device)
+		bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+
 	return 0;
 
 fail_unreserve:
@@ -945,13 +949,17 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
 {
 	struct amdgpu_device *adev = amdgpu_ttm_adev(bo->bdev);
 	struct amdgpu_bo *abo;
-	unsigned long offset, size, lpfn;
-	int i, r;
+	unsigned long offset, size;
+	int r;
 
 	if (!amdgpu_ttm_bo_is_amdgpu_bo(bo))
 		return 0;
 
 	abo = container_of(bo, struct amdgpu_bo, tbo);
+
+	/* Remember that this BO was accessed by the CPU */
+	abo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+
 	if (bo->mem.mem_type != TTM_PL_VRAM)
 		return 0;
 
@@ -967,14 +975,6 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
 	/* hurrah the memory is not visible ! */
 	atomic64_inc(&adev->num_vram_cpu_page_faults);
 	amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM);
-	lpfn =	adev->mc.visible_vram_size >> PAGE_SHIFT;
-	for (i = 0; i < abo->placement.num_placement; i++) {
-		/* Force into visible VRAM */
-		if ((abo->placements[i].flags & TTM_PL_FLAG_VRAM) &&
-		    (!abo->placements[i].lpfn ||
-		     abo->placements[i].lpfn > lpfn))
-			abo->placements[i].lpfn = lpfn;
-	}
 	r = ttm_bo_validate(bo, &abo->placement, false, false);
 	if (unlikely(r == -ENOMEM)) {
 		amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_GTT);
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +9 -0
index 9d4861aea18a..58576375b91c 100644
@@ -493,6 +493,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo,
 		}
 	}
 
+	if (bo->type == ttm_bo_type_device &&
+	    new_mem->mem_type == TTM_PL_VRAM &&
+	    old_mem->mem_type != TTM_PL_VRAM) {
+		/* amdgpu_bo_fault_reserve_notify will re-set this if the CPU
+		 * accesses the BO after it's moved.
+		 */
+		abo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+	}
+
 	/* update statistics */
 	atomic64_add((u64)bo->num_pages << PAGE_SHIFT, &adev->num_bytes_moved);
 	return 0;
Signature · 1 line
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#11 lore raw

PATCH 4/5 drm/amdgpu: Don't force BOs into visible VRAM for page faults

In reply to a message not in the archive

From: John Brooks <[email protected]>

There is no need for page faults to force BOs into visible VRAM if it's full, and the time it takes to do so is great enough to cause noticeable stuttering. Add GTT as a possible placement so that if visible VRAM is full, page faults move BOs to GTT instead of evicting other BOs from VRAM.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -7
index 93601fbea695..6e24339ecc46 100644
@@ -974,18 +974,21 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
 
 	/* hurrah the memory is not visible ! */
 	atomic64_inc(&adev->num_vram_cpu_page_faults);
-	amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM);
+	amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM |
+					 AMDGPU_GEM_DOMAIN_GTT);
+
+	/* Avoid costly evictions; only set GTT as a busy placement */
+	abo->placement.num_busy_placement = 1;
+	abo->placement.busy_placement = &abo->placements[1];
+
 	r = ttm_bo_validate(bo, &abo->placement, false, false);
-	if (unlikely(r == -ENOMEM)) {
-		amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_GTT);
-		return ttm_bo_validate(bo, &abo->placement, false, false);
-	} else if (unlikely(r != 0)) {
+	if (unlikely(r != 0))
 		return r;
-	}
 
 	offset = bo->mem.start << PAGE_SHIFT;
 	/* this should never happen */
-	if ((offset + size) > adev->mc.visible_vram_size)
+	if (bo->mem.mem_type == TTM_PL_VRAM &&
+	    (offset + size) > adev->mc.visible_vram_size)
 		return -EINVAL;
 
 	return 0;
Signature · 6 lines
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-5-michel-otUistvHUpPR7s880joybQ@public.gmane.org

Reviewed-by
#13 lore raw

Re: [PATCH 4/5] drm/amdgpu: Don't force BOs into visible VRAM for page faults

Am 06.07.2017 um 12:51 schrieb Michel Dänzer:

Quoted 11 lines

From: John Brooks <[email protected]>

There is no need for page faults to force BOs into visible VRAM if it's full, and the time it takes to do so is great enough to cause noticeable stuttering. Add GTT as a possible placement so that if visible VRAM is full, page faults move BOs to GTT instead of evicting other BOs from VRAM.

Quoted 27 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 17 ++++++++++-------
  1 file changed, 10 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +8 -5
index 93601fbea695..6e24339ecc46 100644
@@ -974,18 +974,21 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)
 
  	/* hurrah the memory is not visible ! */
  	atomic64_inc(&adev->num_vram_cpu_page_faults);
-	amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM);
+	amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM |
+					 AMDGPU_GEM_DOMAIN_GTT);
+
+	/* Avoid costly evictions; only set GTT as a busy placement */
+	abo->placement.num_busy_placement = 1;
+	abo->placement.busy_placement = &abo->placements[1];
+
  	r = ttm_bo_validate(bo, &abo->placement, false, false);
-	if (unlikely(r == -ENOMEM)) {
-		amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_GTT);
-		return ttm_bo_validate(bo, &abo->placement, false, false);
-	} else if (unlikely(r != 0)) {
+	if (unlikely(r != 0))
  		return r;

drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo) · line 994

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2 -2
-	}
 
  	offset = bo->mem.start << PAGE_SHIFT;
  	/* this should never happen */
-	if ((offset + size) > adev->mc.visible_vram_size)
+	if (bo->mem.mem_type == TTM_PL_VRAM &&
+	    (offset + size) > adev->mc.visible_vram_size)
  		return -EINVAL;
 
  	return 0;

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#14 lore raw

PATCH v2 5/5 drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

In reply to a message not in the archive

From: Michel Dänzer <[email protected]>

This gives BOs which haven't been accessed by the CPU since they were moved to visible VRAM another chance to stay in VRAM when another BO needs to go to visible VRAM.

This should allow BOs to stay in VRAM longer in some cases.

v2:
* Only do this for BOs which don't have the
AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag set.

---
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 28 ++++++++++++++++++++++++++++
 1 file changed, 28 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +28 -0
index 58576375b91c..2a9906dd7637 100644
@@ -207,7 +207,35 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo,
 		    adev->mman.buffer_funcs_ring &&
 		    adev->mman.buffer_funcs_ring->ready == false) {
 			amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_CPU);
+		} else if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+			   !(abo->flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED)) {
+			unsigned fpfn = adev->mc.visible_vram_size >> PAGE_SHIFT;
+			struct drm_mm_node *node = bo->mem.mm_node;
+			unsigned long pages_left;
+
+			for (pages_left = bo->mem.num_pages;
+			     pages_left;
+			     pages_left -= node->size, node++) {
+				if (node->start < fpfn)
+					break;
+			}
+
+			if (!pages_left)
+				goto gtt;
+
+			/* Try evicting to the CPU inaccessible part of VRAM
+			 * first, but only set GTT as busy placement, so this
+			 * BO will be evicted to GTT rather than causing other
+			 * BOs to be evicted from VRAM
+			 */
+			amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM |
+							 AMDGPU_GEM_DOMAIN_GTT);
+			abo->placements[0].fpfn = fpfn;
+			abo->placements[0].lpfn = 0;
+			abo->placement.busy_placement = &abo->placements[1];
+			abo->placement.num_busy_placement = 1;
 		} else {
+gtt:
 			amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_GTT);
 			for (i = 0; i < abo->placement.num_placement; ++i) {
 				if (!(abo->placements[i].flags &
Signature · 6 lines
2.13.2

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-6-michel-otUistvHUpPR7s880joybQ@public.gmane.org

Reviewed-by
#16 lore raw

Re: [PATCH v2 5/5] drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

Am 06.07.2017 um 12:51 schrieb Michel Dänzer:

Quoted 6 lines

From: Michel Dänzer <[email protected]>

This gives BOs which haven't been accessed by the CPU since they were moved to visible VRAM another chance to stay in VRAM when another BO needs to go to visible VRAM.

Quoted 7 lines

This should allow BOs to stay in VRAM longer in some cases.

v2:
* Only do this for BOs which don't have the AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag set.

Quoted 34 lines
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 28 ++++++++++++++++++++++++++++
  1 file changed, 28 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +22 -0
index 58576375b91c..2a9906dd7637 100644
@@ -207,7 +207,35 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo,
  		    adev->mman.buffer_funcs_ring &&
  		    adev->mman.buffer_funcs_ring->ready == false) {
  			amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_CPU);
+		} else if (adev->mc.visible_vram_size < adev->mc.real_vram_size &&
+			   !(abo->flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED)) {
+			unsigned fpfn = adev->mc.visible_vram_size >> PAGE_SHIFT;
+			struct drm_mm_node *node = bo->mem.mm_node;
+			unsigned long pages_left;
+
+			for (pages_left = bo->mem.num_pages;
+			     pages_left;
+			     pages_left -= node->size, node++) {
+				if (node->start < fpfn)
+					break;
+			}
+
+			if (!pages_left)
+				goto gtt;
+
+			/* Try evicting to the CPU inaccessible part of VRAM
+			 * first, but only set GTT as busy placement, so this
+			 * BO will be evicted to GTT rather than causing other
+			 * BOs to be evicted from VRAM
+			 */
+			amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM |

drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · line 241

Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +6 -0
+							 AMDGPU_GEM_DOMAIN_GTT);
+			abo->placements[0].fpfn = fpfn;
+			abo->placements[0].lpfn = 0;
+			abo->placement.busy_placement = &abo->placements[1];
+			abo->placement.num_busy_placement = 1;
  		} else {
+gtt:
  			amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_GTT);
  			for (i = 0; i < abo->placement.num_placement; ++i) {
  				if (!(abo->placements[i].flags &

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#17 lore raw
In reply to a message not in the archive
Quoted 15 lines · Michel Dänzer

From: Michel Dänzer <[email protected]>

This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.

John Brooks (4):
drm/amdgpu: Add vis_vramlimit module parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faults

Michel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

Quoted 10 lines
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  7 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  4 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 45 +++++++++++++++
 5 files changed, 157 insertions(+), 40 deletions(-)
Signature · 1 line
2.13.2

Thanks :)

I'll test it today or tomorrow and report back.

--
John Brooks
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

#18 lore raw
In reply to a message not in the archive
Quoted 15 lines · Michel Dänzer

From: Michel Dänzer <[email protected]>

This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.

John Brooks (4):
drm/amdgpu: Add vis_vramlimit module parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faults

Michel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

Quoted 10 lines
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  7 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  4 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 45 +++++++++++++++
 5 files changed, 157 insertions(+), 40 deletions(-)
Signature · 1 line
2.13.2

I tested these changes against drm-next-4.13. Looks good to me.

John
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 20170707225013.GA15429-6hIufAJW0g7Gr8qjsLp7YGXnswh1EIUO@public.gmane.org

#20 lore raw
Quoted 16 lines · John Brooks
Quoted 15 lines · Michel Dänzer

From: Michel Dänzer <[email protected]>

This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.

John Brooks (4):
drm/amdgpu: Add vis_vramlimit module parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faults

Michel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

Quoted 8 lines
Quoted 6 lines
 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  7 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  4 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 45 +++++++++++++++
 5 files changed, 157 insertions(+), 40 deletions(-)

I tested these changes against drm-next-4.13. Looks good to me.

Thanks for testing, pushed to the internal amd-staging-4.11 branch.

Also thanks for the R-b tag offers, but I think it's a bit weird to add those from the author of a patch. :)

Signature · 7 lines
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

A message that is not in the archive (sent off-list or not yet received): 191db6d2-c3ad-3bae-5bad-fe7a295df450-otUistvHUpPR7s880joybQ@public.gmane.org

#22 lore raw
Quoted 24 lines · Michel Dänzer
Quoted 23 lines · John Brooks
Quoted 21 lines · Michel Dänzer

From: Michel Dänzer <[email protected]>

This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.

John Brooks (4):
drm/amdgpu: Add vis_vramlimit module parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faults

Michel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first

 drivers/gpu/drm/amd/amdgpu/amdgpu.h        |  7 ++-
 drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c     | 92 ++++++++++++++++++++++++------
 drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c    |  4 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++-------
 drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c    | 45 +++++++++++++++
 5 files changed, 157 insertions(+), 40 deletions(-)
Quoted 4 lines
Quoted 1 line

I tested these changes against drm-next-4.13. Looks good to me.

Thanks for testing, pushed to the internal amd-staging-4.11 branch.

Awesome :)

Quoted 4 lines

Also thanks for the R-b tag offers, but I think it's a bit weird to add those from the author of a patch. :)

Indeed. I only wasn't sure since changes had been made on top of mine.

Thanks :)
John

Quoted 5 lines
Signature · 3 lines
-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer

_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx