drm/scheduler: set current_entity to next when remove from rq
Subsystems (3)
PATCH 1/1 drm/scheduler: set current_entity to next when remove from rq
When entity move from one rq to another, current_entity will be set to NULL if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multiple drm_gpu_scheduler.
Make current_entity to next when removing from rq.
--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/gpu/drm/scheduler/sched_main.c +3 -2
@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity);
Signature · 1 line
2.34.1
+ Luben
Quoted 34 lines · brolerliew
When entity move from one rq to another,
current_entitywill be set to NULL if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multipledrm_gpu_scheduler.Make
current_entityto next when removing from rq.Signed-off-by brolerliew <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +3
-3@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity); --2.34.1
Looking...
Regards,
Luben
Quoted 37 lines · Alex Deucher
+ Luben
Quoted 34 lines · brolerliew
When entity move from one rq to another,
current_entitywill be set to NULL if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multipledrm_gpu_scheduler.Make
current_entityto next when removing from rq.Signed-off-by brolerliew <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +3
-3@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity); --2.34.1
Quoted 38 lines · Luben Tuikov
Looking...
Regards,
LubenQuoted 32 lines · Alex Deucher
+ Luben
Quoted 29 lines · brolerliew
When entity move from one rq to another,
current_entitywill be set to NULL if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multipledrm_gpu_scheduler.Make
current_entityto next when removing from rq.Signed-off-by brolerliew <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +3
-2@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +0
-1if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity); --2.34.1
Looks good. I'll pick it up into some other changes I've in tow, and repost along with my changes, as they're somewhat related.
Regards,
Luben
Quoted 46 lines · Luben Tuikov
Quoted 38 lines · Luben Tuikov
Looking...
Regards,
LubenQuoted 32 lines · Alex Deucher
+ Luben
Quoted 29 lines · brolerliew
When entity move from one rq to another,
current_entitywill be set to NULL if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multipledrm_gpu_scheduler.Make
current_entityto next when removing from rq.Signed-off-by brolerliew <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +3
-2@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +0
-1if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity); --2.34.1
Quoted 2 lines
Looks good. I'll pick it up into some other changes I've in tow, and repost along with my changes, as they're somewhat related.
Actually, the more I look at it, the more I think that we do want to set rq->current_entity to NULL in that function, in order to pick the next best entity (or scheduler for that matter), the next time around. See sched_entity.c, and drm_sched_rq_select_entity() where we start evaluating from the _next_ entity.
So, it is best to leave it to set it to NULL, for now.
Regards,
Luben
Am 27.10.22 um 10:07 schrieb Luben Tuikov:
Quoted 46 lines · Luben Tuikov
Quoted 38 lines · Luben Tuikov
Quoted 37 lines
Looking...
Regards,
LubenQuoted 31 lines · Alex Deucher
+ Luben
Quoted 28 lines · brolerliew
When entity move from one rq to another,
current_entitywill be set to NULL if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multipledrm_gpu_scheduler.Make
current_entityto next when removing from rq.Signed-off-by brolerliew <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +3
-2@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +0
-1if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity); --2.34.1
Looks good. I'll pick it up into some other changes I've in tow, and repost along with my changes, as they're somewhat related.
Quoted 7 lines
Actually, the more I look at it, the more I think that we do want to set
rq->current_entityto NULL in that function, in order to pick the next best entity (or scheduler for that matter), the next time around. Seesched_entity.c, anddrm_sched_rq_select_entity()where we start evaluating from the _next_ entity.So, it is best to leave it to set it to NULL, for now.
Apart from that this patch here could cause a crash when the entity is the last one in the list.
In this case current current_entity would be set to an incorrect upcast of the head of the list.
Regards,
Christian.
Quoted 4 lines
Regards,
Luben
Quoted 53 lines · Christian König
Am 27.10.22 um 10:07 schrieb Luben Tuikov:
Quoted 52 lines · Luben Tuikov
Quoted 38 lines · Luben Tuikov
Quoted 37 lines
Looking...
Regards,
LubenQuoted 31 lines · Alex Deucher
+ Luben
Quoted 28 lines · brolerliew
When entity move from one rq to another,
current_entitywill be set to NULL if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multipledrm_gpu_scheduler.Make
current_entityto next when removing from rq.Signed-off-by brolerliew <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +3
-2@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);Quoted 7 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +0
-1if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity); --2.34.1
Looks good. I'll pick it up into some other changes I've in tow, and repost along with my changes, as they're somewhat related.
Actually, the more I look at it, the more I think that we do want to set
rq->current_entityto NULL in that function, in order to pick the next best entity (or scheduler for that matter), the next time around. Seesched_entity.c, anddrm_sched_rq_select_entity()where we start evaluating from the _next_ entity.
Quoted 7 lines
Quoted 1 line
So, it is best to leave it to set it to NULL, for now.
Apart from that this patch here could cause a crash when the entity is the last one in the list.
In this case current
current_entitywould be set to an incorrect upcast of the head of the list.
Absolutely. I saw that, but in rejecting the patch, I didn't feel the need to mention it.
Thanks for looking into this.
Regards,
Luben
It's very nice of you-all to finger it out that it may crash when it is the
last entity in the list. Absolutely I made a mistake about that.
But I still cannot understand why we need to restart the selection from the list head when the current entity is removed from rq.
In drm_sched_rq_select_entity, starting from head may cause the first entity to be selected more often than others, which breaks the equal rule the scheduler wants to achieve.
Maybe the previous one is the better choice when current_entity == entity?
Quoted 79 lines · Luben Tuikov <[email protected]> 于2022年10月27日周四 16:24写道:
Quoted 12 lines · Christian König
Am 27.10.22 um 10:07 schrieb Luben Tuikov:
Quoted 11 lines · Luben Tuikov
Quoted 10 lines · Luben Tuikov
Quoted 9 lines
Looking...
Regards,
LubenQuoted 3 lines · Alex Deucher
+ Luben
On Tue, Oct 25, 2022 at 2:55 AM brolerliew <[email protected]>
Quoted 1 line · wrote:
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
Quoted 1 line
When entity move from one rq to another,
current_entitywill be setto NULL
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
if it is the moving entity. This make entities close to rq head got selected more frequently, especially when doing load balance between multiple
drm_gpu_scheduler.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Review
Quoted 8 lines
Make
current_entityto next when removing from rq.Signed-off-by brolerliew <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/gpu/drm/scheduler/sched_main.cQuoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/scheduler/sched_main.c void drm_sched_rq_remove_entity(struct
Quoted 4 lines
index
2fab218d7082..00b22cc50f08100644drivers/gpu/drm/scheduler/sched_main.c +0
-0@@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct
drm_sched_rq*rq,Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
Quoted 11 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-2spin_lock (&rq->lock);atomic_dec (rq->sched->score); -list_del_init (&entity->list);if (rq->current_entity == entity) - rq->current_entity =NULL ; + rq->current_entity =list_next_entry (entity, list); + +list_del_init (&entity->list);Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +0
-1if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity); --2.34.1
Looks good. I'll pick it up into some other changes I've in tow, and
repost
Quoted 3 lines
Quoted 3 lines
Quoted 1 line
along with my changes, as they're somewhat related.
Actually, the more I look at it, the more I think that we do want to set
rq->current_entityto NULL in that function, in order to pick the nextbest entity
Quoted 1 line
Quoted 1 line
(or scheduler for that matter), the next time around. See
sched_entity.c,Quoted 1 line
Quoted 1 line
and
drm_sched_rq_select_entity()where we start evaluating from the_next_
Quoted 9 lines
Quoted 3 lines
entity.
So, it is best to leave it to set it to NULL, for now.
Apart from that this patch here could cause a crash when the entity is the last one in the list.
In this case current
current_entitywould be set to an incorrect upcast of the head of the list.Absolutely. I saw that, but in rejecting the patch, I didn't feel the need to mention it.
Thanks for looking into this.
Regards,
Luben
Am 27.10.22 um 11:00 schrieb broler Liew:
Quoted 4 lines
It's very nice of you-all to finger it out that it may crash when it is the last entity in the list. Absolutely I made a mistake about that.
But I still cannot understand why we need to restart the selection from the list head when the current entity is removed from rq.
Indrm_sched_rq_select_entity, starting from head may cause the first entity to be selected more often than others, which breaks the equal rule the scheduler wants to achieve. Maybe the previous one is the better choice whencurrent_entity== entity?
That's a good argument, but we want to get rid of the round robin algorithm anyway and switch over to the fifo.
So this is some code which is already not used by default any more and improving it doesn't make much sense.
Regards,
Christian.
Quoted 87 lines
Luben Tuikov <[email protected]> 于2022年10月27日周四 16:24写道:
On 2022-10-27 04:19, Christian König wrote: > Am 27.10.22 um 10:07 schrieb Luben Tuikov: >> On 2022-10-27 03:01, Luben Tuikov wrote: >>> On 2022-10-25 13:50, Luben Tuikov wrote: >>>> Looking... >>>> >>>> Regards, >>>> Luben >>>> >>>> On 2022-10-25 09:35, Alex Deucher wrote: >>>>> + Luben >>>>> >>>>> On Tue, Oct 25, 2022 at 2:55 AM brolerliew <[email protected]> wrote: >>>>>> When entity move from one rq to another, current_entity will be set to NULL >>>>>> if it is the moving entity. This make entities close to rq head got >>>>>> selected more frequently, especially when doing load balance between >>>>>> multiple drm_gpu_scheduler. >>>>>> >>>>>> Make current_entity to next when removing from rq. >>>>>> >>>>>> Signed-off-by: brolerliew <[email protected]> >>>>>> --- >>>>>> drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- >>>>>> 1 file changed, 3 insertions(+), 2 deletions(-) >>>>>> >>>>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c >>>>>> index 2fab218d7082..00b22cc50f08 100644 >>>>>> --- a/drivers/gpu/drm/scheduler/sched_main.c >>>>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c >>>>>> @@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq, >>>>>> spin_lock(&rq->lock); >>>>>> >>>>>> atomic_dec(rq->sched->score); >>>>>> - list_del_init(&entity->list); >>>>>> >>>>>> if (rq->current_entity == entity) >>>>>> - rq->current_entity = NULL; >>>>>> + rq->current_entity = list_next_entry(entity, list); >>>>>> + >>>>>> + list_del_init(&entity->list); >>>>>> >>>>>> if (drm_sched_policy == DRM_SCHED_POLICY_FIFO) >>>>>> drm_sched_rq_remove_fifo_locked(entity); >>>>>> -- >>>>>> 2.34.1 >>>>>> >>> Looks good. I'll pick it up into some other changes I've in tow, and repost >>> along with my changes, as they're somewhat related. >> Actually, the more I look at it, the more I think that we do want to set >> rq->current_entity to NULL in that function, in order to pick the next best entity >> (or scheduler for that matter), the next time around. See sched_entity.c, >> and drm_sched_rq_select_entity() where we start evaluating from the _next_ >> entity. >> >> So, it is best to leave it to set it to NULL, for now. > > Apart from that this patch here could cause a crash when the entity is > the last one in the list. > > In this case current current_entity would be set to an incorrect upcast > of the head of the list. Absolutely. I saw that, but in rejecting the patch, I didn't feel the need to mention it. Thanks for looking into this. Regards, Luben
So, I started fixing this, including the bug taking the next element as an entity, but it could be actually the list_head... a la your patch being fixed, and then went down the rabbit whole of also fixing drm_sched_rq_select_entity(), but the problem is that at that point we don't know if we should start from the _next_ entity (as it is currently the case) or from the current entity (a la list_for_each_entry_from()) as it would be the case with this patch (if it were fixed for the list_head bug).
But the problem is that elsewhere in the GPU scheduler (sched_entity.c), we do want to start from rq->current_entity->next, and picking "next" in drm_sched_rq_remove_entity(), would then skip an entity, or be biased for an entity twice. This is why this function is called drm_sched_rq_remove_entity() and not drm_sched_rq_next_entity_or_null().
So all this work seemed moot, given that we've already switched to FIFO-based scheduling in drm-misc-next, and so I didn't see a point in developing this further at this point (it's been working alright)--we're going with FIFO-based scheduling.
Regards,
Luben
Quoted 86 lines · Christian König
Am 27.10.22 um 11:00 schrieb broler Liew:
Quoted 4 lines
It's very nice of you-all to finger it out that it may crash when it is the last entity in the list. Absolutely I made a mistake about that.
But I still cannot understand why we need to restart the selection from the list head when the current entity is removed from rq.
Indrm_sched_rq_select_entity, starting from head may cause the first entity to be selected more often than others, which breaks the equal rule the scheduler wants to achieve. Maybe the previous one is the better choice whencurrent_entity== entity?That's a good argument, but we want to get rid of the round robin algorithm anyway and switch over to the fifo.
So this is some code which is already not used by default any more and improving it doesn't make much sense.
Regards,
Christian.Quoted 72 lines
Luben Tuikov <[email protected]> 于2022年10月27日周四 16:24写道:
On 2022-10-27 04:19, Christian König wrote: > Am 27.10.22 um 10:07 schrieb Luben Tuikov: >> On 2022-10-27 03:01, Luben Tuikov wrote: >>> On 2022-10-25 13:50, Luben Tuikov wrote: >>>> Looking... >>>> >>>> Regards, >>>> Luben >>>> >>>> On 2022-10-25 09:35, Alex Deucher wrote: >>>>> + Luben >>>>> >>>>> On Tue, Oct 25, 2022 at 2:55 AM brolerliew <[email protected]> wrote: >>>>>> When entity move from one rq to another, current_entity will be set to NULL >>>>>> if it is the moving entity. This make entities close to rq head got >>>>>> selected more frequently, especially when doing load balance between >>>>>> multiple drm_gpu_scheduler. >>>>>> >>>>>> Make current_entity to next when removing from rq. >>>>>> >>>>>> Signed-off-by: brolerliew <[email protected]> >>>>>> --- >>>>>> drivers/gpu/drm/scheduler/sched_main.c | 5 +++-- >>>>>> 1 file changed, 3 insertions(+), 2 deletions(-) >>>>>> >>>>>> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c >>>>>> index 2fab218d7082..00b22cc50f08 100644 >>>>>> --- a/drivers/gpu/drm/scheduler/sched_main.c >>>>>> +++ b/drivers/gpu/drm/scheduler/sched_main.c >>>>>> @@ -168,10 +168,11 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq, >>>>>> spin_lock(&rq->lock); >>>>>> >>>>>> atomic_dec(rq->sched->score); >>>>>> - list_del_init(&entity->list); >>>>>> >>>>>> if (rq->current_entity == entity) >>>>>> - rq->current_entity = NULL; >>>>>> + rq->current_entity = list_next_entry(entity, list); >>>>>> + >>>>>> + list_del_init(&entity->list); >>>>>> >>>>>> if (drm_sched_policy == DRM_SCHED_POLICY_FIFO) >>>>>> drm_sched_rq_remove_fifo_locked(entity); >>>>>> -- >>>>>> 2.34.1 >>>>>> >>> Looks good. I'll pick it up into some other changes I've in tow, and repost >>> along with my changes, as they're somewhat related. >> Actually, the more I look at it, the more I think that we do want to set >> rq->current_entity to NULL in that function, in order to pick the next best entity >> (or scheduler for that matter), the next time around. See sched_entity.c, >> and drm_sched_rq_select_entity() where we start evaluating from the _next_ >> entity. >> >> So, it is best to leave it to set it to NULL, for now. > > Apart from that this patch here could cause a crash when the entity is > the last one in the list. > > In this case current current_entity would be set to an incorrect upcast > of the head of the list. Absolutely. I saw that, but in rejecting the patch, I didn't feel the need to mention it. Thanks for looking into this. Regards, Luben