drm/amdgpu: set sched_hw_submission higher for KIQ (v2)
Subsystems (2)
PATCH 1/1 drm/amdgpu: set sched_hw_submission higher for KIQ (v2)
KIQ doesn't really use the GPU scheduler. The base drivers generally use the KIQ ring directly rather than submitting IBs. However, amdgpu_sched_hw_submission (which defaults to 2) limits the number of outstanding fences to 2. KFD uses the KIQ for TLB flushes and the 2 fence limit hurts performance when there are several KFD processes running.
v2: move some expressions to one line
change KIQ sched_hw_submission to at least 16
--- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +12 -4
@@ -170,6 +170,16 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, unsigned irq_type) {int r; +int sched_hw_submission = amdgpu_sched_hw_submission; + +/* Set the hw submission limit higher for KIQ because +* it's used for a number of gfx/compute tasks by both +* KFD and KGD which may have outstanding fences and +* it doesn't really use the gpu scheduler anyway; +* KIQ tasks get submitted directly to the ring. +*/ +if (ring->funcs->type ==AMDGPU_RING_TYPE_KIQ ) + sched_hw_submission =max (sched_hw_submission,16 );if (ring->adev ==NULL ) {if (adev->num_rings >=AMDGPU_MAX_RINGS )
@@ -178,8 +188,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, ring->adev = adev; ring->idx = adev->num_rings++; adev->rings[ring->idx] = ring; - r =amdgpu_fence_driver_init_ring (ring, - amdgpu_sched_hw_submission); + r =amdgpu_fence_driver_init_ring (ring, sched_hw_submission);if (r)return r; }
@@ -218,8 +227,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,return r; } - ring->ring_size =roundup_pow_of_two (max_dw *4 * - amdgpu_sched_hw_submission); + ring->ring_size =roundup_pow_of_two (max_dw *4 * sched_hw_submission); ring->buf_mask = (ring->ring_size /4 ) -1 ; ring->ptr_mask = ring->funcs->support_64bit_ptrs ?
Signature · 6 lines
2.5.5 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 1503453165-23970-1-git-send-email-alexander.deucher-5C7GfCeVMHo@public.gmane.org
This change is Reviewed-by: Felix Kuehling <[email protected]>
Quoted 48 lines · Alex Deucher
KIQ doesn't really use the GPU scheduler. The base drivers generally use the KIQ ring directly rather than submitting IBs. However,
amdgpu_sched_hw_submission(which defaults to 2) limits the number of outstanding fences to 2. KFD uses the KIQ for TLB flushes and the 2 fence limit hurts performance when there are several KFD processes running.v2: move some expressions to one line
change KIQsched_hw_submissionto at least 16Signed-off-by Alex Deucher <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +11
-2@@ -170,6 +170,16 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, unsigned irq_type) {int r; +int sched_hw_submission = amdgpu_sched_hw_submission; + +/* Set the hw submission limit higher for KIQ because +* it's used for a number of gfx/compute tasks by both +* KFD and KGD which may have outstanding fences and +* it doesn't really use the gpu scheduler anyway; +* KIQ tasks get submitted directly to the ring. +*/ +if (ring->funcs->type ==AMDGPU_RING_TYPE_KIQ ) + sched_hw_submission =max (sched_hw_submission,16 );if (ring->adev ==NULL ) {if (adev->num_rings >=AMDGPU_MAX_RINGS )@@ -178,8 +188,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, ring->adev = adev; ring->idx = adev->num_rings++; adev->rings[ring->idx] = ring; - r =amdgpu_fence_driver_init_ring (ring, - amdgpu_sched_hw_submission); + r =amdgpu_fence_driver_init_ring (ring, sched_hw_submission);if (r)return r; }
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, · line 233
Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +1
-2@@ -218,8 +227,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,return r; } - ring->ring_size =roundup_pow_of_two (max_dw *4 * - amdgpu_sched_hw_submission); + ring->ring_size =roundup_pow_of_two (max_dw *4 * sched_hw_submission); ring->buf_mask = (ring->ring_size /4 ) -1 ; ring->ptr_mask = ring->funcs->support_64bit_ptrs ?
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 14fa938a-9110-ae3f-374c-03a0c98a8af6-5C7GfCeVMHo@public.gmane.org
As noted in the otehr thread we could set it to 256 as well and still only use one page, but 16 is fine with me as well.
Patch is Reviewed-by: Christian König <[email protected]>
Regards,
Christian.
Am 23.08.2017 um 04:25 schrieb Felix Kuehling:
Quoted 59 lines
This change is Reviewed-by: Felix Kuehling <[email protected]>
Quoted 55 lines · Alex Deucher
KIQ doesn't really use the GPU scheduler. The base drivers generally use the KIQ ring directly rather than submitting IBs. However,
amdgpu_sched_hw_submission(which defaults to 2) limits the number of outstanding fences to 2. KFD uses the KIQ for TLB flushes and the 2 fence limit hurts performance when there are several KFD processes running.v2: move some expressions to one line
change KIQsched_hw_submissionto at least 16Signed-off-by Alex Deucher <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +12
-4@@ -170,6 +170,16 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, unsigned irq_type) {int r; +int sched_hw_submission = amdgpu_sched_hw_submission; + +/* Set the hw submission limit higher for KIQ because +* it's used for a number of gfx/compute tasks by both +* KFD and KGD which may have outstanding fences and +* it doesn't really use the gpu scheduler anyway; +* KIQ tasks get submitted directly to the ring. +*/ +if (ring->funcs->type ==AMDGPU_RING_TYPE_KIQ ) + sched_hw_submission =max (sched_hw_submission,16 );if (ring->adev ==NULL ) {if (adev->num_rings >=AMDGPU_MAX_RINGS )@@ -178,8 +188,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, ring->adev = adev; ring->idx = adev->num_rings++; adev->rings[ring->idx] = ring; - r =amdgpu_fence_driver_init_ring (ring, - amdgpu_sched_hw_submission); + r =amdgpu_fence_driver_init_ring (ring, sched_hw_submission);if (r)return r; }@@ -218,8 +227,7 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,return r; } - ring->ring_size =roundup_pow_of_two (max_dw *4 * - amdgpu_sched_hw_submission); + ring->ring_size =roundup_pow_of_two (max_dw *4 * sched_hw_submission);
Quoted 6 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring, · line 233
Quoted 2 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +0
-0ring->buf_mask = (ring->ring_size /4 ) -1 ; ring->ptr_mask = ring->funcs->support_64bit_ptrs ?_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx