Skip to contentSkip to conversation

sched/fair: tweak pick_next_entity

PATCH Applied Peter Oskolkov 4 participants 4 messages Sep 30, 2020 – Oct 5, 2020 Scheduler

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched/fair: tweak pick_next_entity

Currently, pick_next_entity(...) has the following structure (simplified):

[...]
if (last_buddy_ok())
result = last_buddy;
if (next_buddy_ok())
result = next_buddy;
[...]

The intended behavior is to prefer next buddy over last buddy; the current code somewhat obfuscates this, and also wastes cycles checking the last buddy when eventually the next buddy is picked up.

So this patch refactors two 'ifs' above into

[...]
if (next_buddy_ok())

    result = next_buddy;
else if (last_buddy_ok())
    result = last_buddy;

[...]

---
 kernel/sched/fair.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
kernel/sched/fair.c +10 -10
index fc3410b8b990..cec6cf9b2bb3 100644
@@ -4465,17 +4465,17 @@ pick_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *curr)
 			se = second;
 	}
 
-	/*
-	 * Prefer last buddy, try to return the CPU to a preempted task.
-	 */
-	if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, left) < 1)
-		se = cfs_rq->last;
-
-	/*
-	 * Someone really wants this to run. If it's not unfair, run it.
-	 */
-	if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < 1)
+	if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < 1) {
+		/*
+		 * Someone really wants this to run. If it's not unfair, run it.
+		 */
 		se = cfs_rq->next;
+	} else if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, left) < 1) {
+		/*
+		 * Prefer last buddy, try to return the CPU to a preempted task.
+		 */
+		se = cfs_rq->last;
+	}
 
 	clear_buddies(cfs_rq, se);
 
Signature · 1 line
2.28.0.709.gb0816b6eb0-goog
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 18 lines · Peter Oskolkov

Currently, pick_next_entity(...) has the following structure (simplified):

[...]
if (last_buddy_ok())
result = last_buddy;
if (next_buddy_ok())
result = next_buddy;
[...]

The intended behavior is to prefer next buddy over last buddy; the current code somewhat obfuscates this, and also wastes cycles checking the last buddy when eventually the next buddy is picked up.

So this patch refactors two 'ifs' above into

Quoted 8 lines

[...]
if (next_buddy_ok())

    result = next_buddy;
else if (last_buddy_ok())
    result = last_buddy;

[...]

Quoted 39 lines
---
 kernel/sched/fair.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
kernel/sched/fair.c +10 -11
index fc3410b8b990..cec6cf9b2bb3 100644
@@ -4465,17 +4465,17 @@ pick_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *curr)
                        se = second;
        }
 
-       /*
-        * Prefer last buddy, try to return the CPU to a preempted task.
-        */
-       if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, left) < 1)
-               se = cfs_rq->last;
-
-       /*
-        * Someone really wants this to run. If it's not unfair, run it.
-        */
-       if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < 1)
+       if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < 1) {
+               /*
+                * Someone really wants this to run. If it's not unfair, run it.
+                */
                se = cfs_rq->next;
+       } else if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, left) < 1) {
+               /*
+                * Prefer last buddy, try to return the CPU to a preempted task.
+                */
+               se = cfs_rq->last;
+       }
 
        clear_buddies(cfs_rq, se);
 
--

2.28.0.709.gb0816b6eb0-goog

maintainer
#2 lore raw
Quoted 26 lines · Vincent Guittot
Quoted 25 lines · Peter Oskolkov

Currently, pick_next_entity(...) has the following structure (simplified):

[...]
if (last_buddy_ok())
result = last_buddy;
if (next_buddy_ok())
result = next_buddy;
[...]

The intended behavior is to prefer next buddy over last buddy; the current code somewhat obfuscates this, and also wastes cycles checking the last buddy when eventually the next buddy is picked up.

So this patch refactors two 'ifs' above into

[...]
if (next_buddy_ok())

    result = next_buddy;
else if (last_buddy_ok())
    result = last_buddy;

[...]

Quoted 3 lines
Quoted 1 line

Thanks!

merge bot Applied
#3 lore raw

[tip: sched/core] sched/fair: Tweak pick_next_entity()

In reply to Peter Oskolkov
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     9abb897345ce1d41257567f571a78137c961c405
Gitweb:        https://git.kernel.org/tip/9abb897345ce1d41257567f571a78137c961c405
Author:        Peter Oskolkov <[email protected]>
AuthorDate:    Wed, 30 Sep 2020 10:35:32 -07:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Sat, 03 Oct 2020 16:30:52 +02:00

sched/fair: Tweak pick_next_entity()

Currently, pick_next_entity(...) has the following structure (simplified):

  [...]
  if (last_buddy_ok())
    result = last_buddy;
  if (next_buddy_ok())
    result = next_buddy;
  [...]

The intended behavior is to prefer next buddy over last buddy; the current code somewhat obfuscates this, and also wastes cycles checking the last buddy when eventually the next buddy is picked up.

So this patch refactors two 'ifs' above into

  [...]
  if (next_buddy_ok())
      result = next_buddy;
  else if (last_buddy_ok())
      result = last_buddy;
  [...]
---
 kernel/sched/fair.c | 20 ++++++++++----------
 1 file changed, 10 insertions(+), 10 deletions(-)
kernel/sched/fair.c +10 -10
index fc3410b..cec6cf9 100644
@@ -4465,17 +4465,17 @@ pick_next_entity(struct cfs_rq *cfs_rq, struct sched_entity *curr)
 			se = second;
 	}
 
-	/*
-	 * Prefer last buddy, try to return the CPU to a preempted task.
-	 */
-	if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, left) < 1)
-		se = cfs_rq->last;
-
-	/*
-	 * Someone really wants this to run. If it's not unfair, run it.
-	 */
-	if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < 1)
+	if (cfs_rq->next && wakeup_preempt_entity(cfs_rq->next, left) < 1) {
+		/*
+		 * Someone really wants this to run. If it's not unfair, run it.
+		 */
 		se = cfs_rq->next;
+	} else if (cfs_rq->last && wakeup_preempt_entity(cfs_rq->last, left) < 1) {
+		/*
+		 * Prefer last buddy, try to return the CPU to a preempted task.
+		 */
+		se = cfs_rq->last;
+	}
 
 	clear_buddies(cfs_rq, se);