sched: Optimize branch hint in context_switch()
PATCH 1/1 sched: Optimize branch hint in context_switch()
Branch hint profiling on my nehalem machine showed 88% incorrect branch hints:
42017484 326957902 88 context_switch sched.c 3043 42038493 326953687 88 context_switch sched.c 3050
--- kernel/sched.c | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-)
kernel/sched.c +2 -2
@@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,*/ arch_start_context_switch (prev); -if (likely (!mm)) { +if (unlikely (!mm)) { next->active_mm = oldmm;atomic_inc (&oldmm->mm_count);enter_lazy_tlb (oldmm, next); }else switch_mm (oldmm, mm, next); -if (likely (!prev->mm)) { +if (unlikely (!prev->mm)) { prev->active_mm =NULL ; rq->prev_mm = oldmm; }
Quoted 14 lines · Tim Blechmann
Branch hint profiling on my nehalem machine showed 88% incorrect branch hints:
42017484 326957902 88 context_switch sched.c 3043 42038493 326953687 88 context_switch sched.c 3050kernel/sched.c +1
-1@@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,*/ arch_start_context_switch (prev); -if (likely (!mm)) { +if (unlikely (!mm)) { next->active_mm = oldmm;
kernel/sched.c context_switch(struct rq *rq, struct task_struct *prev, · line 3053
Quoted 10 lines
kernel/sched.c +1
-1atomic_inc (&oldmm->mm_count);enter_lazy_tlb (oldmm, next); }else switch_mm (oldmm, mm, next); -if (likely (!prev->mm)) { +if (unlikely (!prev->mm)) { prev->active_mm =NULL ; rq->prev_mm = oldmm; }
I don't think either the original or the patch is correct. Whether or not a task has an mm is entirely workload dependent, we shouldn't be giving hints here.
Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 19 lines · Avi Kivity
Quoted 18 lines · Tim Blechmann
Branch hint profiling on my nehalem machine showed 88% incorrect branch hints:
42017484 326957902 88 context_switch sched.c 3043 42038493 326953687 88 context_switch sched.c 3050kernel/sched.c +1
-1@@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,*/ arch_start_context_switch (prev); -if (likely (!mm)) { +if (unlikely (!mm)) { next->active_mm = oldmm;atomic_inc (&oldmm->mm_count);enter_lazy_tlb (oldmm, next); }else switch_mm (oldmm, mm, next);
Quoted 10 lines
kernel/sched.c context_switch(struct rq *rq, struct task_struct *prev, · line 3053
Quoted 6 lines
kernel/sched.c +1
-1-if (likely (!prev->mm)) { +if (unlikely (!prev->mm)) { prev->active_mm =NULL ; rq->prev_mm = oldmm; }I don't think either the original or the patch is correct. Whether or not a task has an mm is entirely workload dependent, we shouldn't be giving hints here.
There are reasons to still use branch hints, for example if the unlikely branch is very expensive anyway and it pays to have the likely branch be ever so slightly less expensive.
Now I don't think that applies here, but there are cases where such code generation issues are the main motivator not the actual usage patterns.
Quoted 33 lines · Peter Zijlstra
Quoted 20 lines · Avi Kivity
Quoted 18 lines · Tim Blechmann
Branch hint profiling on my nehalem machine showed 88% incorrect branch hints:
42017484 326957902 88 context_switch sched.c 3043 42038493 326953687 88 context_switch sched.c 3050kernel/sched.c +1
-1@@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,*/ arch_start_context_switch (prev); -if (likely (!mm)) { +if (unlikely (!mm)) { next->active_mm = oldmm;atomic_inc (&oldmm->mm_count);enter_lazy_tlb (oldmm, next); }else switch_mm (oldmm, mm, next);Quoted 11 lines
kernel/sched.c context_switch(struct rq *rq, struct task_struct *prev, · line 3053
Quoted 7 lines
kernel/sched.c +1
-1-if (likely (!prev->mm)) { +if (unlikely (!prev->mm)) { prev->active_mm =NULL ; rq->prev_mm = oldmm; }I don't think either the original or the patch is correct. Whether or not a task has an mm is entirely workload dependent, we shouldn't be giving hints here.
Quoted 7 lines
There are reasons to still use branch hints, for example if the unlikely branch is very expensive anyway and it pays to have the likely branch be ever so slightly less expensive.
Now I don't think that applies here, but there are cases where such code generation issues are the main motivator not the actual usage patterns.
These should be documented then to avoid patches removing them:
#define slowpath(x) unlikely(x)
if (slowpath(condition))
expensive_operation();Signature · 1 line
error compiling committee.c: too many arguments to function
Quoted 34 lines · Avi Kivity
Quoted 33 lines · Peter Zijlstra
Quoted 31 lines · Avi Kivity
Quoted 15 lines · Tim Blechmann
Branch hint profiling on my nehalem machine showed 88% incorrect branch hints:
42017484 326957902 88 context_switch sched.c 3043 42038493 326953687 88 context_switch sched.c 3050kernel/sched.c +1
-1@@ -3040,14 +3040,14 @@ context_switch(struct rq *rq, struct task_struct *prev,*/ arch_start_context_switch (prev); -if (likely (!mm)) { +if (unlikely (!mm)) { next->active_mm = oldmm;kernel/sched.c context_switch(struct rq *rq, struct task_struct *prev, · line 3053
Quoted 10 lines
kernel/sched.c +1
-1atomic_inc (&oldmm->mm_count);enter_lazy_tlb (oldmm, next); }else switch_mm (oldmm, mm, next); -if (likely (!prev->mm)) { +if (unlikely (!prev->mm)) { prev->active_mm =NULL ; rq->prev_mm = oldmm; }I don't think either the original or the patch is correct. Whether or not a task has an mm is entirely workload dependent, we shouldn't be giving hints here.
Quoted 6 lines
Quoted 6 lines
There are reasons to still use branch hints, for example if the unlikely branch is very expensive anyway and it pays to have the likely branch be ever so slightly less expensive.
Now I don't think that applies here, but there are cases where such code generation issues are the main motivator not the actual usage patterns.
would be nice, if you commit a patch, removing this hint
Quoted 6 lines
These should be documented then to avoid patches removing them:
#define slowpath(x) unlikely(x) if (slowpath(condition)) expensive_operation();
this would definitely improve the expressive power ...
thnx, tim
Signature · 4 lines
[email protected] http://tim.klingt.org Only very good and very bad programmers use goto in C
Am Sonntag 29 November 2009 16:25:43 schrieb Avi Kivity:
Quoted 6 lines
These should be documented then to avoid patches removing them:
#define slowpath(x) unlikely(x) if (slowpath(condition)) expensive_operation();
Neat. If we also modify the likelyhood tracer to __not__ check the slowpath annotation by default this really looks like a good idea.
Christian