Skip to contentSkip to conversation

sched: Remove unlikely likelys

GIT PULL New Steven Rostedt 1 participant 5 messages Dec 14, 2010

lore
Share & tools
lore
#0 lore raw

Ingo,

Please pull the latest unlikely/sched tree, which can be found at:

git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-2.6-trace.git
unlikely/sched

Steven Rostedt (3):
sched: Change pick_next_task_rt from unlikely to likely
sched: Remove unlikely() from rt_policy() in sched.c
sched: Remove unlikely() from ttwu_post_activation

Yong Zhang (1):
sched: Cleanup pre_schedule_rt

----

 kernel/sched.c    |    4 ++--
 kernel/sched_rt.c |    4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)
#1 lore raw

PATCH 1/4 sched: Cleanup pre_schedule_rt

From: Yong Zhang <[email protected]>

Since [commit 9a897c5a:
sched: RT-balance, replace hooks with pre/post schedule and wakeup methods] we must call pre_schedule_rt if prev is rt task.
So condition rt_task(prev) is always true and the 'unlikely' declaration is simply incorrect.

Cc Peter Zijlstra <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Rusty Russell <[email protected]>
---
 kernel/sched_rt.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched_rt.c +1 -1
index bea7d79..1ab66a2 100644
@@ -1474,7 +1474,7 @@ skip:
 static void pre_schedule_rt(struct rq *rq, struct task_struct *prev)
 {
 	/* Try to pull RT tasks here if we lower this rq's prio */
-	if (unlikely(rt_task(prev)) && rq->rt.highest_prio.curr > prev->prio)
+	if (rq->rt.highest_prio.curr > prev->prio)
 		pull_rt_task(rq);
 }
 
Signature · 1 line
1.7.2.3
#2 lore raw

PATCH 2/4 sched: Change pick_next_task_rt from unlikely to likely

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The if (unlikely(!rt_rq->rt_nr_running)) test in pick_next_task_rt() tests if there is another rt task ready to run. If so, then pick it.

In most systems, only one RT task runs at a time most of the time. Running the branch unlikely annotator profiler on a system doing average work "running firefox, evolution, xchat, distcc builds, etc", it showed the following:

 correct incorrect  %        Function                  File              Line
 ------- ---------  -        --------                  ----              ----
  324344 135104992  99 _pick_next_task_rt             sched_rt.c           1064

99% of the time the condition is true. When an RT task schedules out, it is unlikely that another RT task is waiting to run on that same run queue.

Simply remove the unlikely() condition.

---
 kernel/sched_rt.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched_rt.c +1 -1
index 1ab66a2..c2266c4 100644
@@ -1062,7 +1062,7 @@ static struct task_struct *_pick_next_task_rt(struct rq *rq)
 
 	rt_rq = &rq->rt;
 
-	if (unlikely(!rt_rq->rt_nr_running))
+	if (!rt_rq->rt_nr_running)
 		return NULL;
 
 	if (rt_rq_throttled(rt_rq))
Signature · 1 line
1.7.2.3
#3 lore raw

PATCH 3/4 sched: Remove unlikely() from rt_policy() in sched.c

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The rt_policy() has an unlikely() that the policy it is checking is of RT priority (SCHED_FIFO or SCHED_RR).

According to the annotate branch profiler it is incorrect most of the time:

 correct incorrect  %        Function                  File              Line
 ------- ---------  -        --------                  ----              ----

36667 654674 94 rt_policy sched.c 126

This makes sense because the rt_policy() is used by the sched_set_scheduler() and nice(). Although users may use sys_nice a bit, all RT users use the sched_set_scheduler() to set their RT priority, including kernel threads.

The above numbers were from a normal desktop computer running firefox, evolution, xchat and was part of a distcc compile farm.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +1 -1
index dc91a4d..269a045 100644
@@ -123,7 +123,7 @@
 
 static inline int rt_policy(int policy)
 {
-	if (unlikely(policy == SCHED_FIFO || policy == SCHED_RR))
+	if (policy == SCHED_FIFO || policy == SCHED_RR)
 		return 1;
 	return 0;
 }
Signature · 1 line
1.7.2.3
#4 lore raw

PATCH 4/4 sched: Remove unlikely() from ttwu_post_activation

In reply to Steven Rostedt

From: Steven Rostedt <[email protected]>

The unlikely() used in ttwu_post_activation() tests if the rq->idle_stamp is set. But since this is for a wakeup, and wakeups happen when tasks block on IO, and blocking tasks on IO may put the system into idle, this can actually be a common occurence.

Running the annotated branch profiler on an average desktop running firefox, evolution, xchat and distcc, the report shows:

 correct incorrect  %        Function             File              Line
 ------- ---------  -        --------             ----              ----
34884862 146110926  80 ttwu_post_activation      sched.c            2309

80% of the time, this unlikely is incorrect. Best not to assume what the result is, and just remove the branch annotation.

---
 kernel/sched.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/sched.c +1 -1
index 269a045..6d24b2e 100644
@@ -2458,7 +2458,7 @@ static inline void ttwu_post_activation(struct task_struct *p, struct rq *rq,
 	if (p->sched_class->task_woken)
 		p->sched_class->task_woken(rq, p);
 
-	if (unlikely(rq->idle_stamp)) {
+	if (rq->idle_stamp) {
 		u64 delta = rq->clock - rq->idle_stamp;
 		u64 max = 2*sysctl_sched_migration_cost;
 
Signature · 1 line
1.7.2.3