Use LRU mapping policy for SDMA engines v2
Subsystems (2)
PATCH 1/1 Use LRU mapping policy for SDMA engines v2
Previously we discussed some issues with applying the LRU mapping policy to the SDMA engines. Mainly, if both ring ids end up aliased to the same HW block, the total transfer bandwith an application can achieve will be severely impacted. E.g. an app uses one SDMA ring for upload and one for download.
This small series prevents ring id aliasing and enables LRU mapping for the DMA ip blocks.
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170327223607.2878-1-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
PATCH 1/2 drm/amdgpu: guarantee bijective mapping of ring ids for LRU
Depending on usage patterns, the current LRU policy may create a non-injective mapping between userspace ring ids and kernel rings.
This behaviour is undesired as apps that attempt to fill all HW blocks would be unable to reach some of them.
This change forces the LRU policy to create bijective mappings only.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c | 16 +++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 33 +++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 4 ++-- 3 files changed, 42 insertions(+), 11 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +14 -2
@@ -108,24 +108,36 @@ static enum amdgpu_ring_type amdgpu_hw_ip_to_ring_type(int hw_ip)DRM_ERROR ("Invalid HW IP specified %d\n" , hw_ip);return -1 ; } }static int amdgpu_lru_map (struct amdgpu_device *adev,struct amdgpu_queue_mapper *mapper,int user_ring,struct amdgpu_ring **out_ring) { -int r; +int r, i, j;int ring_type =amdgpu_hw_ip_to_ring_type (mapper->hw_ip); +int ring_blacklist[AMDGPU_MAX_RINGS ]; +struct amdgpu_ring *ring; - r =amdgpu_ring_lru_get (adev, ring_type, out_ring); +/* 0 is a valid ring index, so initialize to -1 */ +memset (ring_blacklist,0xff ,sizeof (ring_blacklist)); + +for (i =0 , j =0 ; i <AMDGPU_MAX_RINGS ; i++) { + ring = mapper->queue_map[i]; +if (ring) + ring_blacklist[j++] = ring->idx; + } + + r =amdgpu_ring_lru_get (adev, ring_type, ring_blacklist, +AMDGPU_MAX_RINGS , out_ring);if (r)return r;return amdgpu_update_cached_map (mapper, user_ring, *out_ring); }/** * amdgpu_queue_mgr_init - init an amdgpu_queue_mgr struct * * @adev: amdgpu_device pointer
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +26 -7
@@ -393,46 +393,65 @@ void amdgpu_ring_fini(struct amdgpu_ring *ring) ring->adev->rings[ring->idx] =NULL ; }static void amdgpu_ring_lru_touch_locked (struct amdgpu_device *adev,struct amdgpu_ring *ring) {/* list_move_tail handles the case where ring isn't part of the list */ list_move_tail (&ring->lru_list, &adev->ring_lru_list); } +static bool amdgpu_ring_is_blacklisted (struct amdgpu_ring *ring, +int *blacklist,int num_blacklist) +{ +int i; + +for (i =0 ; i < num_blacklist && blacklist[i] !=-1 ; i++) { +if (ring->idx == blacklist[i]) +return true; + } + +return false; +} +/** * amdgpu_ring_lru_get - get the least recently used ring for a HW IP block * * @adev: amdgpu_device pointer * @type: amdgpu_ring_type enum +* @blacklist: blacklisted ring ids array +* @num_blacklist: number of entries in @blacklist * @ring: output ring * * Retrieve the amdgpu_ring structure for the least recently used ring of * a specific IP block (all asics). * Returns 0 on success, error on failure. */ -int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type, -struct amdgpu_ring **ring) +int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type,int *blacklist, +int num_blacklist,struct amdgpu_ring **ring) {struct amdgpu_ring *entry;/* List is sorted in LRU order, find first entry corresponding * to the desired HW IP */ *ring =NULL ;spin_lock (&adev->ring_lru_list_lock);list_for_each_entry (entry, &adev->ring_lru_list, lru_list) { -if (entry->funcs->type == type) { - *ring = entry; -amdgpu_ring_lru_touch_locked (adev, *ring); -break ; - } +if (entry->funcs->type != type) +continue ; + +if (amdgpu_ring_is_blacklisted (entry, blacklist, num_blacklist)) +continue ; + + *ring = entry; +amdgpu_ring_lru_touch_locked (adev, *ring); +break ; }spin_unlock (&adev->ring_lru_list_lock);if (!*ring) {DRM_ERROR ("Ring LRU contains no entries for ring type:%d\n" , type);return -EINVAL ; }return 0 ; }
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +2 -2
@@ -195,15 +195,15 @@ void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib);void amdgpu_ring_commit (struct amdgpu_ring *ring);void amdgpu_ring_undo (struct amdgpu_ring *ring);int amdgpu_ring_priority_get (struct amdgpu_ring *ring,enum amd_sched_priority priority);void amdgpu_ring_priority_put (struct amdgpu_ring *ring,enum amd_sched_priority priority);int amdgpu_ring_init (struct amdgpu_device *adev,struct amdgpu_ring *ring,unsigned ring_size,struct amdgpu_irq_src *irq_src,unsigned irq_type);void amdgpu_ring_fini (struct amdgpu_ring *ring); -int amdgpu_ring_lru_get (struct amdgpu_device *adev,int hw_ip, -struct amdgpu_ring **ring); +int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type,int *blacklist, +int num_blacklist,struct amdgpu_ring **ring);void amdgpu_ring_lru_touch (struct amdgpu_device *adev,struct amdgpu_ring *ring);#endif
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170327223607.2878-2-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Re: [PATCH 1/2] drm/amdgpu: guarantee bijective mapping of ring ids for LRU
Forgot to add
v2: compressed ring_blacklist (nha)
Quoted 148 lines · Andres Rodriguez
Depending on usage patterns, the current LRU policy may create a non-injective mapping between userspace ring ids and kernel rings.
This behaviour is undesired as apps that attempt to fill all HW blocks would be unable to reach some of them.
This change forces the LRU policy to create bijective mappings only.
Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c | 16 +++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 33 +++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 4 ++-- 3 files changed, 42 insertions(+), 11 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +14
-2@@ -108,24 +108,36 @@ static enum amdgpu_ring_type amdgpu_hw_ip_to_ring_type(int hw_ip)DRM_ERROR ("Invalid HW IP specified %d\n" , hw_ip);return -1 ; } }static int amdgpu_lru_map (struct amdgpu_device *adev,struct amdgpu_queue_mapper *mapper,int user_ring,struct amdgpu_ring **out_ring) { -int r; +int r, i, j;int ring_type =amdgpu_hw_ip_to_ring_type (mapper->hw_ip); +int ring_blacklist[AMDGPU_MAX_RINGS ]; +struct amdgpu_ring *ring; - r =amdgpu_ring_lru_get (adev, ring_type, out_ring); +/* 0 is a valid ring index, so initialize to -1 */ +memset (ring_blacklist,0xff ,sizeof (ring_blacklist)); + +for (i =0 , j =0 ; i <AMDGPU_MAX_RINGS ; i++) { + ring = mapper->queue_map[i]; +if (ring) + ring_blacklist[j++] = ring->idx; + } + + r =amdgpu_ring_lru_get (adev, ring_type, ring_blacklist, +AMDGPU_MAX_RINGS , out_ring);if (r)return r;return amdgpu_update_cached_map (mapper, user_ring, *out_ring); }/** * amdgpu_queue_mgr_init - init an amdgpu_queue_mgr struct * * @adev: amdgpu_device pointer drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +26
-7@@ -393,46 +393,65 @@ void amdgpu_ring_fini(struct amdgpu_ring *ring) ring->adev->rings[ring->idx] =NULL ; }static void amdgpu_ring_lru_touch_locked (struct amdgpu_device *adev,struct amdgpu_ring *ring) {/* list_move_tail handles the case where ring isn't part of the list */ list_move_tail (&ring->lru_list, &adev->ring_lru_list); } +static bool amdgpu_ring_is_blacklisted (struct amdgpu_ring *ring, +int *blacklist,int num_blacklist) +{ +int i; + +for (i =0 ; i < num_blacklist && blacklist[i] !=-1 ; i++) { +if (ring->idx == blacklist[i]) +return true; + } + +return false; +} +/** * amdgpu_ring_lru_get - get the least recently used ring for a HW IP block * * @adev: amdgpu_device pointer * @type: amdgpu_ring_type enum +* @blacklist: blacklisted ring ids array +* @num_blacklist: number of entries in @blacklist * @ring: output ring * * Retrieve the amdgpu_ring structure for the least recently used ring of * a specific IP block (all asics). * Returns 0 on success, error on failure. */ -int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type, -struct amdgpu_ring **ring) +int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type,int *blacklist, +int num_blacklist,struct amdgpu_ring **ring) {struct amdgpu_ring *entry;/* List is sorted in LRU order, find first entry corresponding * to the desired HW IP */ *ring =NULL ;spin_lock (&adev->ring_lru_list_lock);list_for_each_entry (entry, &adev->ring_lru_list, lru_list) { -if (entry->funcs->type == type) { - *ring = entry; -amdgpu_ring_lru_touch_locked (adev, *ring); -break ; - } +if (entry->funcs->type != type) +continue ; + +if (amdgpu_ring_is_blacklisted (entry, blacklist, num_blacklist)) +continue ; + + *ring = entry; +amdgpu_ring_lru_touch_locked (adev, *ring); +break ; }spin_unlock (&adev->ring_lru_list_lock);if (!*ring) {DRM_ERROR ("Ring LRU contains no entries for ring type:%d\n" , type);return -EINVAL ; }return 0 ; }drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +0
-0@@ -195,15 +195,15 @@ void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib);void amdgpu_ring_commit (struct amdgpu_ring *ring);void amdgpu_ring_undo (struct amdgpu_ring *ring);int amdgpu_ring_priority_get (struct amdgpu_ring *ring,enum amd_sched_priority priority);void amdgpu_ring_priority_put (struct amdgpu_ring *ring,enum amd_sched_priority priority); int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib); · line 209
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +2
-2unsigned ring_size, struct amdgpu_irq_src *irq_src, unsigned irq_type);void amdgpu_ring_fini (struct amdgpu_ring *ring); -int amdgpu_ring_lru_get (struct amdgpu_device *adev,int hw_ip, -struct amdgpu_ring **ring); +int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type,int *blacklist, +int num_blacklist,struct amdgpu_ring **ring);void amdgpu_ring_lru_touch (struct amdgpu_device *adev,struct amdgpu_ring *ring);#endif
_______________________________________________
amd-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
Nicolai Hähnle
Reviewed-byRe: [PATCH 1/2] drm/amdgpu: guarantee bijective mapping of ring ids for LRU
In reply to a message not in the archiveQuoted 29 lines · Andres Rodriguez
Depending on usage patterns, the current LRU policy may create a non-injective mapping between userspace ring ids and kernel rings.
This behaviour is undesired as apps that attempt to fill all HW blocks would be unable to reach some of them.
This change forces the LRU policy to create bijective mappings only.
Signed-off-by Andres Rodriguez <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c | 16 +++++++++++-- drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c | 33 +++++++++++++++++++++------ drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 4 ++-- 3 files changed, 42 insertions(+), 11 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +0
-0@@ -108,24 +108,36 @@ static enum amdgpu_ring_type amdgpu_hw_ip_to_ring_type(int hw_ip)DRM_ERROR ("Invalid HW IP specified %d\n" , hw_ip);return -1 ; } }static int amdgpu_lru_map (struct amdgpu_device *adev,struct amdgpu_queue_mapper *mapper,int user_ring,struct amdgpu_ring **out_ring)
drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c static enum amdgpu_ring_type amdgpu_hw_ip_to_ring_type(int hw_ip) · line 124
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +5
-2{ -int r; +int r, i, j;int ring_type =amdgpu_hw_ip_to_ring_type (mapper->hw_ip); +int ring_blacklist[AMDGPU_MAX_RINGS ]; +struct amdgpu_ring *ring; - r =amdgpu_ring_lru_get (adev, ring_type, out_ring); +/* 0 is a valid ring index, so initialize to -1 */ +memset (ring_blacklist,0xff ,sizeof (ring_blacklist));
Unless I'm mistaken, you don't need this if ...
drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c static enum amdgpu_ring_type amdgpu_hw_ip_to_ring_type(int hw_ip) · line 133
Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +9
-0+ +for (i =0 , j =0 ; i <AMDGPU_MAX_RINGS ; i++) { + ring = mapper->queue_map[i]; +if (ring) + ring_blacklist[j++] = ring->idx; + } + + r =amdgpu_ring_lru_get (adev, ring_type, ring_blacklist, +AMDGPU_MAX_RINGS , out_ring);
... you s/AMDGPU_MAX_RINGS/j/ here.
Quoted 21 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +0
-0if (r)return r;return amdgpu_update_cached_map (mapper, user_ring, *out_ring); }/** * amdgpu_queue_mgr_init - init an amdgpu_queue_mgr struct * * @adev: amdgpu_device pointer drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +0
-0@@ -393,46 +393,65 @@ void amdgpu_ring_fini(struct amdgpu_ring *ring) ring->adev->rings[ring->idx] =NULL ; }static void amdgpu_ring_lru_touch_locked (struct amdgpu_device *adev,struct amdgpu_ring *ring) {
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c void amdgpu_ring_fini(struct amdgpu_ring *ring) · line 408
Quoted 10 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +6
-0/* list_move_tail handles the case where ring isn't part of the list */ list_move_tail (&ring->lru_list, &adev->ring_lru_list); } +static bool amdgpu_ring_is_blacklisted (struct amdgpu_ring *ring, +int *blacklist,int num_blacklist) +{ +int i; + +for (i =0 ; i < num_blacklist && blacklist[i] !=-1 ; i++) {
... and then you can also drop the blacklist[i] != -1 here.
With those changes:
Quoted 79 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c +20
-7+if (ring->idx == blacklist[i]) +return true; + } + +return false; +} +/** * amdgpu_ring_lru_get - get the least recently used ring for a HW IP block * * @adev: amdgpu_device pointer * @type: amdgpu_ring_type enum +* @blacklist: blacklisted ring ids array +* @num_blacklist: number of entries in @blacklist * @ring: output ring * * Retrieve the amdgpu_ring structure for the least recently used ring of * a specific IP block (all asics). * Returns 0 on success, error on failure. */ -int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type, -struct amdgpu_ring **ring) +int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type,int *blacklist, +int num_blacklist,struct amdgpu_ring **ring) {struct amdgpu_ring *entry;/* List is sorted in LRU order, find first entry corresponding * to the desired HW IP */ *ring =NULL ;spin_lock (&adev->ring_lru_list_lock);list_for_each_entry (entry, &adev->ring_lru_list, lru_list) { -if (entry->funcs->type == type) { - *ring = entry; -amdgpu_ring_lru_touch_locked (adev, *ring); -break ; - } +if (entry->funcs->type != type) +continue ; + +if (amdgpu_ring_is_blacklisted (entry, blacklist, num_blacklist)) +continue ; + + *ring = entry; +amdgpu_ring_lru_touch_locked (adev, *ring); +break ; }spin_unlock (&adev->ring_lru_list_lock);if (!*ring) {DRM_ERROR ("Ring LRU contains no entries for ring type:%d\n" , type);return -EINVAL ; }return 0 ; }drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h +2
-2@@ -195,15 +195,15 @@ void amdgpu_ring_generic_pad_ib(struct amdgpu_ring *ring, struct amdgpu_ib *ib);void amdgpu_ring_commit (struct amdgpu_ring *ring);void amdgpu_ring_undo (struct amdgpu_ring *ring);int amdgpu_ring_priority_get (struct amdgpu_ring *ring,enum amd_sched_priority priority);void amdgpu_ring_priority_put (struct amdgpu_ring *ring,enum amd_sched_priority priority);int amdgpu_ring_init (struct amdgpu_device *adev,struct amdgpu_ring *ring,unsigned ring_size,struct amdgpu_irq_src *irq_src,unsigned irq_type);void amdgpu_ring_fini (struct amdgpu_ring *ring); -int amdgpu_ring_lru_get (struct amdgpu_device *adev,int hw_ip, -struct amdgpu_ring **ring); +int amdgpu_ring_lru_get (struct amdgpu_device *adev,int type,int *blacklist, +int num_blacklist,struct amdgpu_ring **ring);void amdgpu_ring_lru_touch (struct amdgpu_device *adev,struct amdgpu_ring *ring);#endif
Signature · 6 lines
Lerne, wie die Welt wirklich ist, Aber vergiss niemals, wie sie sein sollte. _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
PATCH 2/2 drm/amdgpu: use LRU mapping policy for SDMA engines
In reply to a message not in the archiveSpreading the load across multiple SDMA engines can increase memory transfer performance.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +1 -1
@@ -251,25 +251,25 @@ int amdgpu_queue_mgr_map(struct amdgpu_device *adev, *out_ring =amdgpu_get_cached_map (mapper, ring);if (*out_ring) {/* cache hit */ r =0 ; goto out_unlock; }switch (mapper->hw_ip) {case AMDGPU_HW_IP_GFX : -case AMDGPU_HW_IP_DMA :case AMDGPU_HW_IP_UVD :case AMDGPU_HW_IP_VCE : r =amdgpu_identity_map (adev, mapper, ring, out_ring);break ; +case AMDGPU_HW_IP_DMA :case AMDGPU_HW_IP_COMPUTE : r =amdgpu_lru_map (adev, mapper, ring, out_ring);break ;default : *out_ring =NULL ; r = -EINVAL ;DRM_ERROR ("unknown HW IP type: %d\n" , mapper->hw_ip); } out_unlock:
Signature · 6 lines
2.9.3 _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx
A message that is not in the archive (sent off-list or not yet received): 20170327223607.2878-3-andresx7-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org
Nicolai Hähnle
Reviewed-byRe: [PATCH 2/2] drm/amdgpu: use LRU mapping policy for SDMA engines
Quoted 4 lines · Andres Rodriguez
Spreading the load across multiple SDMA engines can increase memory transfer performance.
Signed-off-by Andres Rodriguez <[email protected]>
Quoted 36 lines
--- drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)drivers/gpu/drm/amd/amdgpu/amdgpu_queue_mgr.c +1
-1@@ -251,25 +251,25 @@ int amdgpu_queue_mgr_map(struct amdgpu_device *adev, *out_ring =amdgpu_get_cached_map (mapper, ring);if (*out_ring) {/* cache hit */ r =0 ; goto out_unlock; }switch (mapper->hw_ip) {case AMDGPU_HW_IP_GFX : -case AMDGPU_HW_IP_DMA :case AMDGPU_HW_IP_UVD :case AMDGPU_HW_IP_VCE : r =amdgpu_identity_map (adev, mapper, ring, out_ring);break ; +case AMDGPU_HW_IP_DMA :case AMDGPU_HW_IP_COMPUTE : r =amdgpu_lru_map (adev, mapper, ring, out_ring);break ;default : *out_ring =NULL ; r = -EINVAL ;DRM_ERROR ("unknown HW IP type: %d\n" , mapper->hw_ip); } out_unlock:
Signature · 6 lines
Lerne, wie die Welt wirklich ist, Aber vergiss niemals, wie sie sein sollte. _______________________________________________ amd-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/amd-gfx