Skip to contentSkip to conversation

sched: Spare IPI on single task renice

PATCH Applied Frederic Weisbecker 3 participants 6 messages Dec 3, 2019 – Dec 17, 2019 Scheduler

lore
Share & tools
lore
#0 lore raw

cover sched: Spare IPI on single task renice

A couple of patches to avoid disturbing nohz_full CPUs when a single fair task get its priority changed. I should probably check other sched policies as well...

Frederic Weisbecker (2):
sched: Spare resched IPI when prio changes on a single fair task
sched: Use fair:prio_changed() instead of ad-hoc implementation

 kernel/sched/core.c | 16 ++++++++--------
 kernel/sched/fair.c |  2 ++
 2 files changed, 10 insertions(+), 8 deletions(-)
Signature · 1 line
2.23.0
#1 lore raw

PATCH 1/2 sched: Spare resched IPI when prio changes on a single fair task

The runqueue of a fair task being remotely reniced is going to get a resched IPI in order to reassess which task should be the current running on the CPU. However that evaluation is useless if the fair task is running alone, in which case we can spare that IPI, preventing nohz_full CPUs from being disturbed.

---
 kernel/sched/fair.c | 2 ++
 1 file changed, 2 insertions(+)
kernel/sched/fair.c +2 -0
index 08a233e97a01..6d2560cb24f0 100644
@@ -10322,6 +10322,8 @@ prio_changed_fair(struct rq *rq, struct task_struct *p, int oldprio)
 	if (!task_on_rq_queued(p))
 		return;
 
+	if (rq->cfs.nr_running == 1)
+		return;
 	/*
 	 * Reschedule if we are currently running on this runqueue and
 	 * our priority decreased, or if we are not currently running on
Signature · 1 line
2.23.0
#2 lore raw

[tip: sched/core] sched: Spare resched IPI when prio changes on a single fair task

The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     7c2e8bbd87db661122e92d71a394dd7bb3ada4d3
Gitweb:        https://git.kernel.org/tip/7c2e8bbd87db661122e92d71a394dd7bb3ada4d3
Author:        Frederic Weisbecker <[email protected]>
AuthorDate:    Tue, 03 Dec 2019 17:01:05 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 17 Dec 2019 13:32:50 +01:00

sched: Spare resched IPI when prio changes on a single fair task

The runqueue of a fair task being remotely reniced is going to get a resched IPI in order to reassess which task should be the current running on the CPU. However that evaluation is useless if the fair task is running alone, in which case we can spare that IPI, preventing nohz_full CPUs from being disturbed.

---
 kernel/sched/fair.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/sched/fair.c +3 -0
index 08a233e..846f50b 100644
@@ -10322,6 +10322,9 @@ prio_changed_fair(struct rq *rq, struct task_struct *p, int oldprio)
 	if (!task_on_rq_queued(p))
 		return;
 
+	if (rq->cfs.nr_running == 1)
+		return;
+
 	/*
 	 * Reschedule if we are currently running on this runqueue and
 	 * our priority decreased, or if we are not currently running on
#3 lore raw

PATCH 2/2 sched: Use fair:prio_changed() instead of ad-hoc implementation

In reply to Frederic Weisbecker

set_user_nice() implements its own version of fair::prio_changed() and therefore misses a specific optimization towards nohz_full CPUs that avoid sending an resched IPI to a reniced task running alone. Use the proper callback instead.

---
 kernel/sched/core.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
kernel/sched/core.c +8 -8
index 90e4b00ace89..66d1af94455e 100644
@@ -4540,17 +4540,17 @@ void set_user_nice(struct task_struct *p, long nice)
 	p->prio = effective_prio(p);
 	delta = p->prio - old_prio;
 
-	if (queued) {
+	if (queued)
 		enqueue_task(rq, p, ENQUEUE_RESTORE | ENQUEUE_NOCLOCK);
-		/*
-		 * If the task increased its priority or is running and
-		 * lowered its priority, then reschedule its CPU:
-		 */
-		if (delta < 0 || (delta > 0 && task_running(rq, p)))
-			resched_curr(rq);
-	}
+
 	if (running)
 		set_next_task(rq, p);
+	/*
+	 * If the task increased its priority or is running and
+	 * lowered its priority, then reschedule its CPU:
+	 */
+	p->sched_class->prio_changed(rq, p, old_prio);
+
 out_unlock:
 	task_rq_unlock(rq, p, &rf);
 }
Signature · 1 line
2.23.0
#4 lore raw

[tip: sched/core] sched: Use fair:prio_changed() instead of ad-hoc implementation

The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     5443a0be6121d557e12951537e10159e4c61035d
Gitweb:        https://git.kernel.org/tip/5443a0be6121d557e12951537e10159e4c61035d
Author:        Frederic Weisbecker <[email protected]>
AuthorDate:    Tue, 03 Dec 2019 17:01:06 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 17 Dec 2019 13:32:50 +01:00

sched: Use fair:prio_changed() instead of ad-hoc implementation

set_user_nice() implements its own version of fair::prio_changed() and therefore misses a specific optimization towards nohz_full CPUs that avoid sending an resched IPI to a reniced task running alone. Use the proper callback instead.

---
 kernel/sched/core.c | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
kernel/sched/core.c +8 -8
index 90e4b00..15508c2 100644
@@ -4540,17 +4540,17 @@ void set_user_nice(struct task_struct *p, long nice)
 	p->prio = effective_prio(p);
 	delta = p->prio - old_prio;
 
-	if (queued) {
+	if (queued)
 		enqueue_task(rq, p, ENQUEUE_RESTORE | ENQUEUE_NOCLOCK);
-		/*
-		 * If the task increased its priority or is running and
-		 * lowered its priority, then reschedule its CPU:
-		 */
-		if (delta < 0 || (delta > 0 && task_running(rq, p)))
-			resched_curr(rq);
-	}
 	if (running)
 		set_next_task(rq, p);
+
+	/*
+	 * If the task increased its priority or is running and
+	 * lowered its priority, then reschedule its CPU:
+	 */
+	p->sched_class->prio_changed(rq, p, old_prio);
+
 out_unlock:
 	task_rq_unlock(rq, p, &rf);
 }
maintainer
#5 lore raw
In reply to Frederic Weisbecker
Quoted 3 lines · Frederic Weisbecker

A couple of patches to avoid disturbing nohz_full CPUs when a single fair task get its priority changed. I should probably check other sched policies as well...

Yeah, auditing all that sounds like a good idea.

Quoted 3 lines

Frederic Weisbecker (2):
sched: Spare resched IPI when prio changes on a single fair task
sched: Use fair:prio_changed() instead of ad-hoc implementation

These look good though, thanks!