amdgpu: Visible VRAM Management Improvements, next take
Subsystems (2)
cover amdgpu: Visible VRAM Management Improvements, next take
From: Michel Dänzer <[email protected]>
This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.
John Brooks (4):
drm/amdgpu: Add vis_vramlimit module parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faults
Michel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 7 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 45 +++++++++++++++ 5 files changed, 157 insertions(+), 40 deletions(-)
Signature · 6 lines
2.13.2 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-1-michel-otUistvHUpPR7s880joybQ@public.gmane.org
PATCH v2 1/5 drm/amdgpu: Add vis_vramlimit module parameter
From: John Brooks <[email protected]>
Allow specifying a limit on visible VRAM via a module parameter. This is helpful for testing performance under visible VRAM pressure.
v2: Add cast to 64-bit (Christian König)
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 1 + drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 ++++ drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 8 ++++++++ 3 files changed, 13 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +1 -0
@@ -75,6 +75,7 @@*/ extern int amdgpu_modeset;extern int amdgpu_vram_limit; +extern int amdgpu_vis_vram_limit;extern int amdgpu_gart_size;extern int amdgpu_moverate;extern int amdgpu_benchmarking;
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +4 -0
@@ -74,6 +74,7 @@#define KMS_DRIVER_PATCHLEVEL 0int amdgpu_vram_limit =0 ; +int amdgpu_vis_vram_limit =0 ;int amdgpu_gart_size =-1 ;/* auto */ int amdgpu_moverate =-1 ;/* auto */ int amdgpu_benchmarking =0 ;
@@ -121,6 +122,9 @@ int amdgpu_lbpw = -1;MODULE_PARM_DESC (vramlimit,"Restrict VRAM for testing, in megabytes" );module_param_named (vramlimit, amdgpu_vram_limit, int,0600 ); +MODULE_PARM_DESC (vis_vramlimit,"Restrict visible VRAM for testing, in megabytes" ); +module_param_named (vis_vramlimit, amdgpu_vis_vram_limit, int,0444 ); +MODULE_PARM_DESC (gartsize,"Size of PCIE/IGP gart to setup in megabytes (32, 64, etc., -1 = auto)" );module_param_named (gartsize, amdgpu_gart_size, int,0600 );
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +8 -0
@@ -1092,6 +1092,7 @@ static struct ttm_bo_driver amdgpu_bo_driver = {int amdgpu_ttm_init (struct amdgpu_device *adev) {int r; +u64 vis_vram_limit; r =amdgpu_ttm_global_init (adev);if (r) {
@@ -1115,6 +1116,13 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)DRM_ERROR ("Failed initializing VRAM heap.\n" );return r; } + +/* Reduce size of CPU-visible VRAM if requested */ + vis_vram_limit = (u64 )amdgpu_vis_vram_limit *1024 *1024 ; +if (amdgpu_vis_vram_limit >0 && + vis_vram_limit <= adev->mc.visible_vram_size) + adev->mc.visible_vram_size = vis_vram_limit; +/* Change the size here instead of the init above so only lpfn is affected */ amdgpu_ttm_set_active_vram_size (adev, adev->mc.visible_vram_size);
Signature · 6 lines
2.13.2 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH v3 2/5 drm/amdgpu: Throttle visible VRAM moves separately
In reply to a message not in the archiveFrom: John Brooks <[email protected]>
The BO move throttling code is designed to allow VRAM to fill quickly if it is relatively empty. However, this does not take into account situations where the visible VRAM is smaller than total VRAM, and total VRAM may not be close to full but the visible VRAM segment is under pressure. In such situations, visible VRAM would experience unrestricted swapping and performance would drop.
Add a separate counter specifically for moves involving visible VRAM, and check it before moving BOs there.
v2: Only perform calculations for separate counter if visible VRAM is
smaller than total VRAM. (Michel Dänzer)
v3: [Michel Dänzer]
* Use BO's location rather than the AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag to determine whether to account a move for visible VRAM in most cases.
* Use a single
if (adev->mc.visible_vram_size < adev->mc.real_vram_size) { block in amdgpu_cs_get_threshold_for_moves.
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 12 +++- 3 files changed, 87 insertions(+), 23 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu.h +5 -1
@@ -1162,7 +1162,9 @@ struct amdgpu_cs_parser {struct list_head validated;struct dma_fence *fence;uint64_t bytes_moved_threshold; +uint64_t bytes_moved_vis_threshold;uint64_t bytes_moved; +uint64_t bytes_moved_vis;struct amdgpu_bo_list_entry *evictable;/* user fence */
@@ -1596,6 +1598,7 @@ struct amdgpu_device {spinlock_t lock;s64 last_update_us;s64 accum_us;/* accumulated microseconds */ +s64 accum_us_vis;/* for visible VRAM */ u32 log2_max_MBps; } mm_stats;
@@ -1892,7 +1895,8 @@ void amdgpu_pci_config_reset(struct amdgpu_device *adev);bool amdgpu_need_post (struct amdgpu_device *adev);void amdgpu_update_display_priority (struct amdgpu_device *adev); -void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes); +void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes, +u64 num_vis_bytes);void amdgpu_ttm_placement_from_domain (struct amdgpu_bo *abo,u32 domain);bool amdgpu_ttm_bo_is_amdgpu_bo (struct ttm_buffer_object *bo);int amdgpu_ttm_tt_get_user_pages (struct ttm_tt *ttm,struct page **pages);
drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +73 -19
171 lines not on this page · raw message
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +9 -3
@@ -322,7 +322,7 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,struct amdgpu_bo *bo;enum ttm_bo_type type;unsigned long page_align; -u64 initial_bytes_moved; +u64 initial_bytes_moved, bytes_moved;size_t acc_size;int r;
@@ -398,8 +398,14 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev, r =ttm_bo_init_reserved (&adev->mman.bdev, &bo->tbo, size, type, &bo->placement, page_align, !kernel,NULL , acc_size, sg, resv, &amdgpu_ttm_bo_destroy); -amdgpu_cs_report_moved_bytes (adev, -atomic64_read (&adev->num_bytes_moved) - initial_bytes_moved); + bytes_moved =atomic64_read (&adev->num_bytes_moved) - + initial_bytes_moved; +if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + bo->tbo.mem.mem_type ==TTM_PL_VRAM && + bo->tbo.mem.start < adev->mc.visible_vram_size >>PAGE_SHIFT ) +amdgpu_cs_report_moved_bytes (adev, bytes_moved, bytes_moved); +else +amdgpu_cs_report_moved_bytes (adev, bytes_moved,0 );if (unlikely (r !=0 ))return r;
Signature · 6 lines
2.13.2 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-3-michel-otUistvHUpPR7s880joybQ@public.gmane.org
Christian König
Reviewed-byRe: [PATCH v3 2/5] drm/amdgpu: Throttle visible VRAM moves separately
Am 06.07.2017 um 12:51 schrieb Michel Dänzer:
Quoted 20 lines
From: John Brooks <[email protected]>
The BO move throttling code is designed to allow VRAM to fill quickly if it is relatively empty. However, this does not take into account situations where the visible VRAM is smaller than total VRAM, and total VRAM may not be close to full but the visible VRAM segment is under pressure. In such situations, visible VRAM would experience unrestricted swapping and performance would drop.
Add a separate counter specifically for moves involving visible VRAM, and check it before moving BOs there.
v2: Only perform calculations for separate counter if visible VRAM is
smaller than total VRAM. (Michel Dänzer)
v3: [Michel Dänzer]
* Use BO's location rather than theAMDGPU_GEM_CREATE_CPU_ACCESS_REQUIREDflag to determine whether to account a move for visible VRAM in most cases.
* Use a single
Quoted 7 lines
if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {block in
amdgpu_cs_get_threshold_for_moves.Fixes 95844d20ae02 (drm/amdgpu: throttle buffer migrations at CS using a fixed MBps limit (v2))Signed-off-by John Brooks <[email protected]>Signed-off-by Michel Dänzer <[email protected]>
Quoted 242 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 12 +++- 3 files changed, 87 insertions(+), 23 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu.h +5
-1@@ -1162,7 +1162,9 @@ struct amdgpu_cs_parser {struct list_head validated;struct dma_fence *fence;uint64_t bytes_moved_threshold; +uint64_t bytes_moved_vis_threshold;uint64_t bytes_moved; +uint64_t bytes_moved_vis;struct amdgpu_bo_list_entry *evictable;/* user fence */ @@ -1596,6 +1598,7 @@ struct amdgpu_device {spinlock_t lock;s64 last_update_us;s64 accum_us;/* accumulated microseconds */ +s64 accum_us_vis;/* for visible VRAM */ u32 log2_max_MBps; } mm_stats;@@ -1892,7 +1895,8 @@ void amdgpu_pci_config_reset(struct amdgpu_device *adev);bool amdgpu_need_post (struct amdgpu_device *adev);void amdgpu_update_display_priority (struct amdgpu_device *adev); -void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes); +void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes, +u64 num_vis_bytes);void amdgpu_ttm_placement_from_domain (struct amdgpu_bo *abo,u32 domain);bool amdgpu_ttm_bo_is_amdgpu_bo (struct ttm_buffer_object *bo);int amdgpu_ttm_tt_get_user_pages (struct ttm_tt *ttm,struct page **pages);drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +73
-19@@ -220,10 +220,11 @@ static s64 bytes_to_us(struct amdgpu_device *adev, u64 bytes)* ticks. The accumulated microseconds (us) are converted to bytes and * returned. */ -static u64 amdgpu_cs_get_threshold_for_moves (struct amdgpu_device *adev) +static void amdgpu_cs_get_threshold_for_moves (struct amdgpu_device *adev, +u64 *max_bytes, +u64 *max_vis_bytes) {s64 time_us, increment_us; -u64 max_bytes;u64 free_vram, total_vram, used_vram;/* Allow a maximum of 200 accumulated ms. This is basically per-IB @@ -235,8 +236,11 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)*/ const s64 us_upper_bound =200000 ; -if (!adev->mm_stats.log2_max_MBps) -return 0 ; +if (!adev->mm_stats.log2_max_MBps) { + *max_bytes =0 ; + *max_vis_bytes =0 ; +return ; + } total_vram = adev->mc.real_vram_size - adev->vram_pin_size; used_vram =atomic64_read (&adev->vram_usage);@@ -277,23 +281,45 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev) adev->mm_stats.accum_us =max (min_us, adev->mm_stats.accum_us); } -/* This returns 0 if the driver is in debt to disallow (optional) +/* This is set to 0 if the driver is in debt to disallow (optional) * buffer moves. */ - max_bytes =us_to_bytes (adev, adev->mm_stats.accum_us); + *max_bytes =us_to_bytes (adev, adev->mm_stats.accum_us); + +/* Do the same for visible VRAM if half of it is free */ +if (adev->mc.visible_vram_size < adev->mc.real_vram_size) { +u64 total_vis_vram = adev->mc.visible_vram_size; +u64 used_vis_vram =atomic64_read (&adev->vram_vis_usage); + +if (used_vis_vram < total_vis_vram) { +u64 free_vis_vram = total_vis_vram - used_vis_vram; + adev->mm_stats.accum_us_vis =min (adev->mm_stats.accum_us_vis + + increment_us, us_upper_bound); + +if (free_vis_vram >= total_vis_vram /2 ) + adev->mm_stats.accum_us_vis = +max (bytes_to_us (adev, free_vis_vram /2 ), + adev->mm_stats.accum_us_vis); + } + + *max_vis_bytes =us_to_bytes (adev, adev->mm_stats.accum_us_vis); + }else { + *max_vis_bytes =0 ; + }spin_unlock (&adev->mm_stats.lock); -return max_bytes; }/* Report how many bytes have really been moved for the last command * submission. This can result in a debt that can stop buffer migrations * temporarily. */ -void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes) +void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes, +u64 num_vis_bytes) {spin_lock (&adev->mm_stats.lock); adev->mm_stats.accum_us -=bytes_to_us (adev, num_bytes); + adev->mm_stats.accum_us_vis -=bytes_to_us (adev, num_vis_bytes);spin_unlock (&adev->mm_stats.lock); }@@ -301,7 +327,7 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p, struct amdgpu_bo *bo) {struct amdgpu_device *adev =amdgpu_ttm_adev (bo->tbo.bdev); -u64 initial_bytes_moved; +u64 initial_bytes_moved, bytes_moved;uint32_t domain;int r;@@ -311,17 +337,35 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p,/* Don't move this buffer if we have depleted our allowance * to move it. Don't move anything if the threshold is zero. */ -if (p->bytes_moved < p->bytes_moved_threshold) - domain = bo->prefered_domains; -else +if (p->bytes_moved < p->bytes_moved_threshold) { +if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + (bo->flags &AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED )) { +/* And don't move a CPU_ACCESS_REQUIRED BO to limited +* visible VRAM if we've depleted our allowance to do +* that. +*/ +if (p->bytes_moved_vis < p->bytes_moved_vis_threshold) + domain = bo->prefered_domains; +else + domain = bo->allowed_domains; + }else { + domain = bo->prefered_domains; + } + }else { domain = bo->allowed_domains; + } retry:amdgpu_ttm_placement_from_domain (bo, domain); initial_bytes_moved =atomic64_read (&adev->num_bytes_moved); r =ttm_bo_validate (&bo->tbo, &bo->placement, true, false); - p->bytes_moved +=atomic64_read (&adev->num_bytes_moved) - - initial_bytes_moved; + bytes_moved =atomic64_read (&adev->num_bytes_moved) - + initial_bytes_moved; + p->bytes_moved += bytes_moved; +if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + bo->tbo.mem.mem_type ==TTM_PL_VRAM && + bo->tbo.mem.start < adev->mc.visible_vram_size >>PAGE_SHIFT ) + p->bytes_moved_vis += bytes_moved;if (unlikely (r == -ENOMEM ) && domain != bo->allowed_domains) { domain = bo->allowed_domains;@@ -347,7 +391,8 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,struct amdgpu_bo_list_entry *candidate = p->evictable;struct amdgpu_bo *bo = candidate->robj;struct amdgpu_device *adev =amdgpu_ttm_adev (bo->tbo.bdev); -u64 initial_bytes_moved; +u64 initial_bytes_moved, bytes_moved; +bool update_bytes_moved_vis;uint32_t other;/* If we reached our current BO we can forget it */ @@ -367,10 +412,17 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,/* Good we can try to move this BO somewhere else */ amdgpu_ttm_placement_from_domain (bo, other); + update_bytes_moved_vis = + adev->mc.visible_vram_size < adev->mc.real_vram_size && + bo->tbo.mem.mem_type ==TTM_PL_VRAM && + bo->tbo.mem.start < adev->mc.visible_vram_size >>PAGE_SHIFT ; initial_bytes_moved =atomic64_read (&adev->num_bytes_moved); r =ttm_bo_validate (&bo->tbo, &bo->placement, true, false); - p->bytes_moved +=atomic64_read (&adev->num_bytes_moved) - + bytes_moved =atomic64_read (&adev->num_bytes_moved) - initial_bytes_moved; + p->bytes_moved += bytes_moved; +if (update_bytes_moved_vis) + p->bytes_moved_vis += bytes_moved;if (unlikely (r))break ;@@ -550,8 +602,10 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,list_splice (&need_pages, &p->validated); } - p->bytes_moved_threshold =amdgpu_cs_get_threshold_for_moves (p->adev); +amdgpu_cs_get_threshold_for_moves (p->adev, &p->bytes_moved_threshold, + &p->bytes_moved_vis_threshold); p->bytes_moved =0 ; + p->bytes_moved_vis =0 ; p->evictable =list_last_entry (&p->validated, struct amdgpu_bo_list_entry, tv.head);@@ -575,8 +629,8 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, goto error_validate; } -amdgpu_cs_report_moved_bytes (p->adev, p->bytes_moved); - +amdgpu_cs_report_moved_bytes (p->adev, p->bytes_moved, + p->bytes_moved_vis); fpriv->vm.last_eviction_counter =atomic64_read (&p->adev->num_evictions);drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2
-3@@ -322,7 +322,7 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,struct amdgpu_bo *bo;enum ttm_bo_type type;unsigned long page_align; -u64 initial_bytes_moved; +u64 initial_bytes_moved, bytes_moved;size_t acc_size;int r;@@ -398,8 +398,14 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev, r =ttm_bo_init_reserved (&adev->mman.bdev, &bo->tbo, size, type, &bo->placement, page_align, !kernel,NULL , acc_size, sg, resv, &amdgpu_ttm_bo_destroy); -amdgpu_cs_report_moved_bytes (adev, -atomic64_read (&adev->num_bytes_moved) - initial_bytes_moved); + bytes_moved = atomic64_read(&adev->num_bytes_moved) -
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_create_restricted(struct amdgpu_device *adev, · line 411
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +7
-0+ initial_bytes_moved; +if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + bo->tbo.mem.mem_type ==TTM_PL_VRAM && + bo->tbo.mem.start < adev->mc.visible_vram_size >>PAGE_SHIFT ) +amdgpu_cs_report_moved_bytes (adev, bytes_moved, bytes_moved); +else +amdgpu_cs_report_moved_bytes (adev, bytes_moved,0 );if (unlikely (r !=0 ))return r;
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
John Brooks
Reviewed-byRe: [PATCH v3 2/5] drm/amdgpu: Throttle visible VRAM moves separately
In reply to a message not in the archiveQuoted 20 lines · Michel Dänzer
From: John Brooks <[email protected]>
The BO move throttling code is designed to allow VRAM to fill quickly if it is relatively empty. However, this does not take into account situations where the visible VRAM is smaller than total VRAM, and total VRAM may not be close to full but the visible VRAM segment is under pressure. In such situations, visible VRAM would experience unrestricted swapping and performance would drop.
Add a separate counter specifically for moves involving visible VRAM, and check it before moving BOs there.
v2: Only perform calculations for separate counter if visible VRAM is
smaller than total VRAM. (Michel Dänzer)
v3: [Michel Dänzer]
* Use BO's location rather than theAMDGPU_GEM_CREATE_CPU_ACCESS_REQUIREDflag to determine whether to account a move for visible VRAM in most cases.
* Use a single
Quoted 7 lines
if (adev->mc.visible_vram_size < adev->mc.real_vram_size) {block in
amdgpu_cs_get_threshold_for_moves.Fixes 95844d20ae02 (drm/amdgpu: throttle buffer migrations at CS using a fixed MBps limit (v2))Signed-off-by John Brooks <[email protected]>Signed-off-by Michel Dänzer <[email protected]>
Changes look good. You can have my
if you want it.
Quoted 255 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu.h | 6 +- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 12 +++- 3 files changed, 87 insertions(+), 23 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu.h +5
-1@@ -1162,7 +1162,9 @@ struct amdgpu_cs_parser {struct list_head validated;struct dma_fence *fence;uint64_t bytes_moved_threshold; +uint64_t bytes_moved_vis_threshold;uint64_t bytes_moved; +uint64_t bytes_moved_vis;struct amdgpu_bo_list_entry *evictable;/* user fence */ @@ -1596,6 +1598,7 @@ struct amdgpu_device {spinlock_t lock;s64 last_update_us;s64 accum_us;/* accumulated microseconds */ +s64 accum_us_vis;/* for visible VRAM */ u32 log2_max_MBps; } mm_stats;@@ -1892,7 +1895,8 @@ void amdgpu_pci_config_reset(struct amdgpu_device *adev);bool amdgpu_need_post (struct amdgpu_device *adev);void amdgpu_update_display_priority (struct amdgpu_device *adev); -void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes); +void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes, +u64 num_vis_bytes);void amdgpu_ttm_placement_from_domain (struct amdgpu_bo *abo,u32 domain);bool amdgpu_ttm_bo_is_amdgpu_bo (struct ttm_buffer_object *bo);int amdgpu_ttm_tt_get_user_pages (struct ttm_tt *ttm,struct page **pages);drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +73
-19@@ -220,10 +220,11 @@ static s64 bytes_to_us(struct amdgpu_device *adev, u64 bytes)* ticks. The accumulated microseconds (us) are converted to bytes and * returned. */ -static u64 amdgpu_cs_get_threshold_for_moves (struct amdgpu_device *adev) +static void amdgpu_cs_get_threshold_for_moves (struct amdgpu_device *adev, +u64 *max_bytes, +u64 *max_vis_bytes) {s64 time_us, increment_us; -u64 max_bytes;u64 free_vram, total_vram, used_vram;/* Allow a maximum of 200 accumulated ms. This is basically per-IB @@ -235,8 +236,11 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev)*/ const s64 us_upper_bound =200000 ; -if (!adev->mm_stats.log2_max_MBps) -return 0 ; +if (!adev->mm_stats.log2_max_MBps) { + *max_bytes =0 ; + *max_vis_bytes =0 ; +return ; + } total_vram = adev->mc.real_vram_size - adev->vram_pin_size; used_vram =atomic64_read (&adev->vram_usage);@@ -277,23 +281,45 @@ static u64 amdgpu_cs_get_threshold_for_moves(struct amdgpu_device *adev) adev->mm_stats.accum_us =max (min_us, adev->mm_stats.accum_us); } -/* This returns 0 if the driver is in debt to disallow (optional) +/* This is set to 0 if the driver is in debt to disallow (optional) * buffer moves. */ - max_bytes =us_to_bytes (adev, adev->mm_stats.accum_us); + *max_bytes =us_to_bytes (adev, adev->mm_stats.accum_us); + +/* Do the same for visible VRAM if half of it is free */ +if (adev->mc.visible_vram_size < adev->mc.real_vram_size) { +u64 total_vis_vram = adev->mc.visible_vram_size; +u64 used_vis_vram =atomic64_read (&adev->vram_vis_usage); + +if (used_vis_vram < total_vis_vram) { +u64 free_vis_vram = total_vis_vram - used_vis_vram; + adev->mm_stats.accum_us_vis =min (adev->mm_stats.accum_us_vis + + increment_us, us_upper_bound); + +if (free_vis_vram >= total_vis_vram /2 ) + adev->mm_stats.accum_us_vis = +max (bytes_to_us (adev, free_vis_vram /2 ), + adev->mm_stats.accum_us_vis); + } + + *max_vis_bytes =us_to_bytes (adev, adev->mm_stats.accum_us_vis); + }else { + *max_vis_bytes =0 ; + }spin_unlock (&adev->mm_stats.lock); -return max_bytes; }/* Report how many bytes have really been moved for the last command * submission. This can result in a debt that can stop buffer migrations * temporarily. */ -void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes) +void amdgpu_cs_report_moved_bytes (struct amdgpu_device *adev,u64 num_bytes, +u64 num_vis_bytes) {spin_lock (&adev->mm_stats.lock); adev->mm_stats.accum_us -=bytes_to_us (adev, num_bytes); + adev->mm_stats.accum_us_vis -=bytes_to_us (adev, num_vis_bytes);spin_unlock (&adev->mm_stats.lock); }@@ -301,7 +327,7 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p, struct amdgpu_bo *bo) {struct amdgpu_device *adev =amdgpu_ttm_adev (bo->tbo.bdev); -u64 initial_bytes_moved; +u64 initial_bytes_moved, bytes_moved;uint32_t domain;int r;@@ -311,17 +337,35 @@ static int amdgpu_cs_bo_validate(struct amdgpu_cs_parser *p,/* Don't move this buffer if we have depleted our allowance * to move it. Don't move anything if the threshold is zero. */ -if (p->bytes_moved < p->bytes_moved_threshold) - domain = bo->prefered_domains; -else +if (p->bytes_moved < p->bytes_moved_threshold) { +if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + (bo->flags &AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED )) { +/* And don't move a CPU_ACCESS_REQUIRED BO to limited +* visible VRAM if we've depleted our allowance to do +* that. +*/ +if (p->bytes_moved_vis < p->bytes_moved_vis_threshold) + domain = bo->prefered_domains; +else + domain = bo->allowed_domains; + }else { + domain = bo->prefered_domains; + } + }else { domain = bo->allowed_domains; + } retry:amdgpu_ttm_placement_from_domain (bo, domain); initial_bytes_moved =atomic64_read (&adev->num_bytes_moved); r =ttm_bo_validate (&bo->tbo, &bo->placement, true, false); - p->bytes_moved +=atomic64_read (&adev->num_bytes_moved) - - initial_bytes_moved; + bytes_moved =atomic64_read (&adev->num_bytes_moved) - + initial_bytes_moved; + p->bytes_moved += bytes_moved; +if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + bo->tbo.mem.mem_type ==TTM_PL_VRAM && + bo->tbo.mem.start < adev->mc.visible_vram_size >>PAGE_SHIFT ) + p->bytes_moved_vis += bytes_moved;if (unlikely (r == -ENOMEM ) && domain != bo->allowed_domains) { domain = bo->allowed_domains;@@ -347,7 +391,8 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,struct amdgpu_bo_list_entry *candidate = p->evictable;struct amdgpu_bo *bo = candidate->robj;struct amdgpu_device *adev =amdgpu_ttm_adev (bo->tbo.bdev); -u64 initial_bytes_moved; +u64 initial_bytes_moved, bytes_moved; +bool update_bytes_moved_vis;uint32_t other;/* If we reached our current BO we can forget it */ @@ -367,10 +412,17 @@ static bool amdgpu_cs_try_evict(struct amdgpu_cs_parser *p,/* Good we can try to move this BO somewhere else */ amdgpu_ttm_placement_from_domain (bo, other); + update_bytes_moved_vis = + adev->mc.visible_vram_size < adev->mc.real_vram_size && + bo->tbo.mem.mem_type ==TTM_PL_VRAM && + bo->tbo.mem.start < adev->mc.visible_vram_size >>PAGE_SHIFT ; initial_bytes_moved =atomic64_read (&adev->num_bytes_moved); r =ttm_bo_validate (&bo->tbo, &bo->placement, true, false); - p->bytes_moved +=atomic64_read (&adev->num_bytes_moved) - + bytes_moved =atomic64_read (&adev->num_bytes_moved) - initial_bytes_moved; + p->bytes_moved += bytes_moved; +if (update_bytes_moved_vis) + p->bytes_moved_vis += bytes_moved;if (unlikely (r))break ;@@ -550,8 +602,10 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,list_splice (&need_pages, &p->validated); } - p->bytes_moved_threshold =amdgpu_cs_get_threshold_for_moves (p->adev); +amdgpu_cs_get_threshold_for_moves (p->adev, &p->bytes_moved_threshold, + &p->bytes_moved_vis_threshold); p->bytes_moved =0 ; + p->bytes_moved_vis =0 ; p->evictable =list_last_entry (&p->validated, struct amdgpu_bo_list_entry, tv.head);@@ -575,8 +629,8 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p, goto error_validate; } -amdgpu_cs_report_moved_bytes (p->adev, p->bytes_moved); - +amdgpu_cs_report_moved_bytes (p->adev, p->bytes_moved, + p->bytes_moved_vis); fpriv->vm.last_eviction_counter =atomic64_read (&p->adev->num_evictions);drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +9
-3@@ -322,7 +322,7 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,struct amdgpu_bo *bo;enum ttm_bo_type type;unsigned long page_align; -u64 initial_bytes_moved; +u64 initial_bytes_moved, bytes_moved;size_t acc_size;int r;@@ -398,8 +398,14 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev, r =ttm_bo_init_reserved (&adev->mman.bdev, &bo->tbo, size, type, &bo->placement, page_align, !kernel,NULL , acc_size, sg, resv, &amdgpu_ttm_bo_destroy); -amdgpu_cs_report_moved_bytes (adev, -atomic64_read (&adev->num_bytes_moved) - initial_bytes_moved); + bytes_moved =atomic64_read (&adev->num_bytes_moved) - + initial_bytes_moved; +if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + bo->tbo.mem.mem_type ==TTM_PL_VRAM && + bo->tbo.mem.start < adev->mc.visible_vram_size >>PAGE_SHIFT ) +amdgpu_cs_report_moved_bytes (adev, bytes_moved, bytes_moved); +else +amdgpu_cs_report_moved_bytes (adev, bytes_moved,0 );if (unlikely (r !=0 ))return r;Signature · 1 line
2.13.2
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH v3 3/5 drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM
In reply to a message not in the archiveFrom: John Brooks <[email protected]>
When a BO is moved to VRAM, clear AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED. This allows it to potentially later move to invisible VRAM if the CPU does not access it again.
Setting the CPU_ACCESS flag in amdgpu_bo_fault_reserve_notify() also means that we can remove the loop to restrict lpfn to the end of visible VRAM, because amdgpu_ttm_placement_init() will do it for us.
v3 [Michel Dänzer]
* Use AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED instead of a new flag (Christian König)
* Explicitly mention amdgpu_bo_fault_reserve_notify in amdgpu_bo_move
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 20 ++++++++++---------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 9 +++++++++ 2 files changed, 19 insertions(+), 10 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -10
@@ -432,6 +432,10 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,trace_amdgpu_bo_create (bo); +/* Treat CPU_ACCESS_REQUIRED only as a hint if given by UMD */ +if (type == ttm_bo_type_device) + bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; +return 0 ; fail_unreserve:
@@ -945,13 +949,17 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo) {struct amdgpu_device *adev =amdgpu_ttm_adev (bo->bdev);struct amdgpu_bo *abo; -unsigned long offset, size, lpfn; -int i, r; +unsigned long offset, size; +int r;if (!amdgpu_ttm_bo_is_amdgpu_bo (bo))return 0 ; abo = container_of(bo, struct amdgpu_bo, tbo); + +/* Remember that this BO was accessed by the CPU */ + abo->flags |=AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; +if (bo->mem.mem_type !=TTM_PL_VRAM )return 0 ;
@@ -967,14 +975,6 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)/* hurrah the memory is not visible ! */ atomic64_inc (&adev->num_vram_cpu_page_faults);amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM ); - lpfn = adev->mc.visible_vram_size >>PAGE_SHIFT ; -for (i =0 ; i < abo->placement.num_placement; i++) { -/* Force into visible VRAM */ -if ((abo->placements[i].flags &TTM_PL_FLAG_VRAM ) && - (!abo->placements[i].lpfn || - abo->placements[i].lpfn > lpfn)) - abo->placements[i].lpfn = lpfn; - } r =ttm_bo_validate (bo, &abo->placement, false, false);if (unlikely (r == -ENOMEM )) {amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_GTT );
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +9 -0
@@ -493,6 +493,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo, } } +if (bo->type == ttm_bo_type_device && + new_mem->mem_type ==TTM_PL_VRAM && + old_mem->mem_type !=TTM_PL_VRAM ) { +/* amdgpu_bo_fault_reserve_notify will re-set this if the CPU +* accesses the BO after it's moved. +*/ + abo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; + } +/* update statistics */ atomic64_add ((u64 )bo->num_pages <<PAGE_SHIFT , &adev->num_bytes_moved);return 0 ;
Signature · 6 lines
2.13.2 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-4-michel-otUistvHUpPR7s880joybQ@public.gmane.org
Christian König
Reviewed-byRe: [PATCH v3 3/5] drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM
Am 06.07.2017 um 12:51 schrieb Michel Dänzer:
Quoted 10 lines
From: John Brooks <[email protected]>
When a BO is moved to VRAM, clear
AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED. This allows it to potentially later move to invisible VRAM if the CPU does not access it again.Setting the
CPU_ACCESSflag inamdgpu_bo_fault_reserve_notify()also means that we can remove the loop to restrict lpfn to the end of visible VRAM, becauseamdgpu_ttm_placement_init()will do it for us.
Quoted 8 lines
v3 [Michel Dänzer]
* UseAMDGPU_GEM_CREATE_CPU_ACCESS_REQUIREDinstead of a new flag (Christian König)
* Explicitly mentionamdgpu_bo_fault_reserve_notifyinamdgpu_bo_moveSuggested-by Michel Dänzer <[email protected]>Signed-off-by John Brooks <[email protected]>Signed-off-by Michel Dänzer <[email protected]>
Quoted 65 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 20 ++++++++++---------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 9 +++++++++ 2 files changed, 19 insertions(+), 10 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10
-10@@ -432,6 +432,10 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,trace_amdgpu_bo_create (bo); +/* Treat CPU_ACCESS_REQUIRED only as a hint if given by UMD */ +if (type == ttm_bo_type_device) + bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; +return 0 ; fail_unreserve:@@ -945,13 +949,17 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo) {struct amdgpu_device *adev =amdgpu_ttm_adev (bo->bdev);struct amdgpu_bo *abo; -unsigned long offset, size, lpfn; -int i, r; +unsigned long offset, size; +int r;if (!amdgpu_ttm_bo_is_amdgpu_bo (bo))return 0 ; abo = container_of(bo, struct amdgpu_bo, tbo); + +/* Remember that this BO was accessed by the CPU */ + abo->flags |=AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; +if (bo->mem.mem_type !=TTM_PL_VRAM )return 0 ;@@ -967,14 +975,6 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)/* hurrah the memory is not visible ! */ atomic64_inc (&adev->num_vram_cpu_page_faults);amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM ); - lpfn = adev->mc.visible_vram_size >>PAGE_SHIFT ; -for (i =0 ; i < abo->placement.num_placement; i++) { -/* Force into visible VRAM */ -if ((abo->placements[i].flags &TTM_PL_FLAG_VRAM ) && - (!abo->placements[i].lpfn || - abo->placements[i].lpfn > lpfn)) - abo->placements[i].lpfn = lpfn; - } r =ttm_bo_validate (bo, &abo->placement, false, false);if (unlikely (r == -ENOMEM )) {amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_GTT );drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +2
-0@@ -493,6 +493,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo, } } + if (bo->type == ttm_bo_type_device && + new_mem->mem_type == TTM_PL_VRAM &&
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c static int amdgpu_bo_move(struct ttm_buffer_object *bo, · line 507
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +7
-0+ old_mem->mem_type != TTM_PL_VRAM) { +/* amdgpu_bo_fault_reserve_notify will re-set this if the CPU +* accesses the BO after it's moved. +*/ + abo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; + } +/* update statistics */ atomic64_add ((u64 )bo->num_pages <<PAGE_SHIFT , &adev->num_bytes_moved);return 0 ;
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
John Brooks
Reviewed-byRe: [PATCH v3 3/5] drm/amdgpu: Set/clear CPU_ACCESS flag on page fault and move to VRAM
In reply to a message not in the archiveQuoted 10 lines · Michel Dänzer
From: John Brooks <[email protected]>
When a BO is moved to VRAM, clear
AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED. This allows it to potentially later move to invisible VRAM if the CPU does not access it again.Setting the
CPU_ACCESSflag inamdgpu_bo_fault_reserve_notify()also means that we can remove the loop to restrict lpfn to the end of visible VRAM, becauseamdgpu_ttm_placement_init()will do it for us.
Quoted 8 lines
v3 [Michel Dänzer]
* UseAMDGPU_GEM_CREATE_CPU_ACCESS_REQUIREDinstead of a new flag (Christian König)
* Explicitly mentionamdgpu_bo_fault_reserve_notifyinamdgpu_bo_moveSuggested-by Michel Dänzer <[email protected]>Signed-off-by John Brooks <[email protected]>Signed-off-by Michel Dänzer <[email protected]>
Same.
Quoted 78 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 20 ++++++++++---------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 9 +++++++++ 2 files changed, 19 insertions(+), 10 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10
-10@@ -432,6 +432,10 @@ int amdgpu_bo_create_restricted(struct amdgpu_device *adev,trace_amdgpu_bo_create (bo); +/* Treat CPU_ACCESS_REQUIRED only as a hint if given by UMD */ +if (type == ttm_bo_type_device) + bo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; +return 0 ; fail_unreserve:@@ -945,13 +949,17 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo) {struct amdgpu_device *adev =amdgpu_ttm_adev (bo->bdev);struct amdgpu_bo *abo; -unsigned long offset, size, lpfn; -int i, r; +unsigned long offset, size; +int r;if (!amdgpu_ttm_bo_is_amdgpu_bo (bo))return 0 ; abo = container_of(bo, struct amdgpu_bo, tbo); + +/* Remember that this BO was accessed by the CPU */ + abo->flags |=AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; +if (bo->mem.mem_type !=TTM_PL_VRAM )return 0 ;@@ -967,14 +975,6 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)/* hurrah the memory is not visible ! */ atomic64_inc (&adev->num_vram_cpu_page_faults);amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM ); - lpfn = adev->mc.visible_vram_size >>PAGE_SHIFT ; -for (i =0 ; i < abo->placement.num_placement; i++) { -/* Force into visible VRAM */ -if ((abo->placements[i].flags &TTM_PL_FLAG_VRAM ) && - (!abo->placements[i].lpfn || - abo->placements[i].lpfn > lpfn)) - abo->placements[i].lpfn = lpfn; - } r =ttm_bo_validate (bo, &abo->placement, false, false);if (unlikely (r == -ENOMEM )) {amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_GTT );drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +9
-0@@ -493,6 +493,15 @@ static int amdgpu_bo_move(struct ttm_buffer_object *bo, } } +if (bo->type == ttm_bo_type_device && + new_mem->mem_type ==TTM_PL_VRAM && + old_mem->mem_type !=TTM_PL_VRAM ) { +/* amdgpu_bo_fault_reserve_notify will re-set this if the CPU +* accesses the BO after it's moved. +*/ + abo->flags &= ~AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED ; + } +/* update statistics */ atomic64_add ((u64 )bo->num_pages <<PAGE_SHIFT , &adev->num_bytes_moved);return 0 ;Signature · 1 line
2.13.2
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 4/5 drm/amdgpu: Don't force BOs into visible VRAM for page faults
In reply to a message not in the archiveFrom: John Brooks <[email protected]>
There is no need for page faults to force BOs into visible VRAM if it's full, and the time it takes to do so is great enough to cause noticeable stuttering. Add GTT as a possible placement so that if visible VRAM is full, page faults move BOs to GTT instead of evicting other BOs from VRAM.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +10 -7
@@ -974,18 +974,21 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)/* hurrah the memory is not visible ! */ atomic64_inc (&adev->num_vram_cpu_page_faults); -amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM ); +amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM | +AMDGPU_GEM_DOMAIN_GTT ); + +/* Avoid costly evictions; only set GTT as a busy placement */ + abo->placement.num_busy_placement =1 ; + abo->placement.busy_placement = &abo->placements[1 ]; + r =ttm_bo_validate (bo, &abo->placement, false, false); -if (unlikely (r == -ENOMEM )) { -amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_GTT ); -return ttm_bo_validate (bo, &abo->placement, false, false); - }else if (unlikely (r !=0 )) { +if (unlikely (r !=0 ))return r; - } offset = bo->mem.start <<PAGE_SHIFT ;/* this should never happen */ -if ((offset + size) > adev->mc.visible_vram_size) +if (bo->mem.mem_type ==TTM_PL_VRAM && + (offset + size) > adev->mc.visible_vram_size)return -EINVAL ;return 0 ;
Signature · 6 lines
2.13.2 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-5-michel-otUistvHUpPR7s880joybQ@public.gmane.org
Christian König
Reviewed-byRe: [PATCH 4/5] drm/amdgpu: Don't force BOs into visible VRAM for page faults
Am 06.07.2017 um 12:51 schrieb Michel Dänzer:
Quoted 11 lines
From: John Brooks <[email protected]>
There is no need for page faults to force BOs into visible VRAM if it's full, and the time it takes to do so is great enough to cause noticeable stuttering. Add GTT as a possible placement so that if visible VRAM is full, page faults move BOs to GTT instead of evicting other BOs from VRAM.
Suggested-by Michel Dänzer <[email protected]>Signed-off-by John Brooks <[email protected]>Reviewed-by Michel Dänzer <[email protected]>Signed-off-by Michel Dänzer <[email protected]>
Quoted 27 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +8
-5@@ -974,18 +974,21 @@ int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo)/* hurrah the memory is not visible ! */ atomic64_inc (&adev->num_vram_cpu_page_faults); -amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM ); +amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM | +AMDGPU_GEM_DOMAIN_GTT ); + +/* Avoid costly evictions; only set GTT as a busy placement */ + abo->placement.num_busy_placement =1 ; + abo->placement.busy_placement = &abo->placements[1 ]; + r =ttm_bo_validate (bo, &abo->placement, false, false); -if (unlikely (r == -ENOMEM )) { -amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_GTT ); -return ttm_bo_validate (bo, &abo->placement, false, false); - }else if (unlikely (r !=0 )) { +if (unlikely (r !=0 ))return r;
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo) · line 994
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_object.c +2
-2- } offset = bo->mem.start <<PAGE_SHIFT ;/* this should never happen */ -if ((offset + size) > adev->mc.visible_vram_size) +if (bo->mem.mem_type ==TTM_PL_VRAM && + (offset + size) > adev->mc.visible_vram_size)return -EINVAL ;return 0 ;
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH v2 5/5 drm/amdgpu: Try evicting from CPU visible to invisible VRAM first
In reply to a message not in the archiveFrom: Michel Dänzer <[email protected]>
This gives BOs which haven't been accessed by the CPU since they were moved to visible VRAM another chance to stay in VRAM when another BO needs to go to visible VRAM.
This should allow BOs to stay in VRAM longer in some cases.
v2:
* Only do this for BOs which don't have the
AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED flag set.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +28 -0
@@ -207,7 +207,35 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, adev->mman.buffer_funcs_ring && adev->mman.buffer_funcs_ring->ready == false) {amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_CPU ); + } else if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + !(abo->flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED)) { +unsigned fpfn = adev->mc.visible_vram_size >>PAGE_SHIFT ; +struct drm_mm_node *node = bo->mem.mm_node; +unsigned long pages_left; + +for (pages_left = bo->mem.num_pages; + pages_left; + pages_left -= node->size, node++) { +if (node->start < fpfn) +break ; + } + +if (!pages_left) + goto gtt; + +/* Try evicting to the CPU inaccessible part of VRAM +* first, but only set GTT as busy placement, so this +* BO will be evicted to GTT rather than causing other +* BOs to be evicted from VRAM +*/ +amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_VRAM | +AMDGPU_GEM_DOMAIN_GTT ); + abo->placements[0 ].fpfn = fpfn; + abo->placements[0 ].lpfn =0 ; + abo->placement.busy_placement = &abo->placements[1 ]; + abo->placement.num_busy_placement =1 ; } else { +gtt:amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_GTT );for (i =0 ; i < abo->placement.num_placement; ++i) { if (!(abo->placements[i].flags &
Signature · 6 lines
2.13.2 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170706105130.23731-6-michel-otUistvHUpPR7s880joybQ@public.gmane.org
Christian König
Reviewed-byRe: [PATCH v2 5/5] drm/amdgpu: Try evicting from CPU visible to invisible VRAM first
Am 06.07.2017 um 12:51 schrieb Michel Dänzer:
Quoted 6 lines
From: Michel Dänzer <[email protected]>
This gives BOs which haven't been accessed by the CPU since they were moved to visible VRAM another chance to stay in VRAM when another BO needs to go to visible VRAM.
Quoted 7 lines
This should allow BOs to stay in VRAM longer in some cases.
v2:
* Only do this for BOs which don't have theAMDGPU_GEM_CREATE_CPU_ACCESS_REQUIREDflag set.Signed-off-by Michel Dänzer <[email protected]>
Quoted 34 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +22
-0@@ -207,7 +207,35 @@ static void amdgpu_evict_flags(struct ttm_buffer_object *bo, adev->mman.buffer_funcs_ring && adev->mman.buffer_funcs_ring->ready == false) {amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_CPU ); + } else if (adev->mc.visible_vram_size < adev->mc.real_vram_size && + !(abo->flags & AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED)) { +unsigned fpfn = adev->mc.visible_vram_size >>PAGE_SHIFT ; +struct drm_mm_node *node = bo->mem.mm_node; +unsigned long pages_left; + +for (pages_left = bo->mem.num_pages; + pages_left; + pages_left -= node->size, node++) { +if (node->start < fpfn) +break ; + } + +if (!pages_left) + goto gtt; + +/* Try evicting to the CPU inaccessible part of VRAM +* first, but only set GTT as busy placement, so this +* BO will be evicted to GTT rather than causing other +* BOs to be evicted from VRAM +*/ + amdgpu_ttm_placement_from_domain(abo, AMDGPU_GEM_DOMAIN_VRAM |
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c static void amdgpu_evict_flags(struct ttm_buffer_object *bo, · line 241
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +6
-0+ AMDGPU_GEM_DOMAIN_GTT); + abo->placements[0 ].fpfn = fpfn; + abo->placements[0 ].lpfn =0 ; + abo->placement.busy_placement = &abo->placements[1 ]; + abo->placement.num_busy_placement =1 ; }else { +gtt:amdgpu_ttm_placement_from_domain (abo,AMDGPU_GEM_DOMAIN_GTT );for (i =0 ; i < abo->placement.num_placement; ++i) { if (!(abo->placements[i].flags &
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 15 lines · Michel Dänzer
From: Michel Dänzer <[email protected]>
This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.
John Brooks (4):
drm/amdgpu: Addvis_vramlimitmodule parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clearCPU_ACCESSflag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faultsMichel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 7 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 45 +++++++++++++++ 5 files changed, 157 insertions(+), 40 deletions(-)Signature · 1 line
2.13.2
Thanks :)
I'll test it today or tomorrow and report back.
--
John Brooks
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Quoted 15 lines · Michel Dänzer
From: Michel Dänzer <[email protected]>
This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.
John Brooks (4):
drm/amdgpu: Addvis_vramlimitmodule parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clearCPU_ACCESSflag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faultsMichel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 7 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 45 +++++++++++++++ 5 files changed, 157 insertions(+), 40 deletions(-)Signature · 1 line
2.13.2
I tested these changes against drm-next-4.13. Looks good to me.
John
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170707225013.GA15429-6hIufAJW0g7Gr8qjsLp7YGXnswh1EIUO@public.gmane.org
Quoted 16 lines · John Brooks
Quoted 15 lines · Michel Dänzer
From: Michel Dänzer <[email protected]>
This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.
John Brooks (4):
drm/amdgpu: Addvis_vramlimitmodule parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clearCPU_ACCESSflag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faultsMichel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM first
Quoted 8 lines
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu.h | 7 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 45 +++++++++++++++ 5 files changed, 157 insertions(+), 40 deletions(-)I tested these changes against drm-next-4.13. Looks good to me.
Thanks for testing, pushed to the internal amd-staging-4.11 branch.
Also thanks for the R-b tag offers, but I think it's a bit weird to add those from the author of a patch. :)
Signature · 7 lines
Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 191db6d2-c3ad-3bae-5bad-fe7a295df450-otUistvHUpPR7s880joybQ@public.gmane.org
Quoted 24 lines · Michel Dänzer
Quoted 23 lines · John Brooks
Quoted 21 lines · Michel Dänzer
From: Michel Dänzer <[email protected]>
This is a rearrangement of some patches of John's, with some improvements of mine, and a patch of mine, rebased on current amd-staging-4.11. Details about changes in the individual patches.
John Brooks (4):
drm/amdgpu: Addvis_vramlimitmodule parameter drm/amdgpu: Throttle visible VRAM moves separately drm/amdgpu: Set/clearCPU_ACCESSflag on page fault and move to VRAM drm/amdgpu: Don't force BOs into visible VRAM for page faultsMichel Dänzer (1):
drm/amdgpu: Try evicting from CPU visible to invisible VRAM firstdrivers/gpu/drm/amd/amdgpu/amdgpu.h | 7 ++- drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c | 92 ++++++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 4 ++ drivers/gpu/drm/amd/amdgpu/amdgpu_object.c | 49 +++++++++------- drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 45 +++++++++++++++ 5 files changed, 157 insertions(+), 40 deletions(-)
Quoted 4 lines
Quoted 1 line
I tested these changes against drm-next-4.13. Looks good to me.
Thanks for testing, pushed to the internal amd-staging-4.11 branch.
Awesome :)
Quoted 4 lines
Also thanks for the R-b tag offers, but I think it's a bit weird to add those from the author of a patch. :)
Indeed. I only wasn't sure since changes had been made on top of mine.
Thanks :)
John
Quoted 5 lines
Signature · 3 lines
-- Earthling Michel Dänzer | http://www.amd.com Libre software enthusiast | Mesa and X developer
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx