sched/core: Create new task with twice disabled preemption
Subsystems (3)
PATCH 1/1 sched/core: Create new task with twice disabled preemption
Preemption state on enter in finish_task_switch() is different in cases of context_switch() and schedule_tail().
In the first case we have it twice disabled: at the start of schedule() and during spin locking. In the second it is only
once: the value which was set in init_task_preempt_count().
For archs without __ARCH_WANT_UNLOCKED_CTXSW set this means that all newly created tasks execute finish_arch_post_lock_switch() and post_schedule() with preemption enabled.
It seems there is possible a problem in rare situations on arm64, when one freshly created thread preempts another before finish_arch_post_lock_switch() has finished. If mm is the same, then TIF_SWITCH_MM on the second won't be set.
The second rare but possible issue is zeroing of post_schedule() on a wrong cpu.
So, lets fix this and unify preempt_count state.
--- arch/x86/include/asm/preempt.h | 6 ++++-- include/asm-generic/preempt.h | 6 ++++-- include/linux/sched.h | 2 ++ kernel/sched/core.c | 6 ++---- 4 files changed, 12 insertions(+), 8 deletions(-)
arch/x86/include/asm/preempt.h +4 -2
@@ -32,9 +32,11 @@ static __always_inline void preempt_count_set(int pc)*/ #define task_preempt_count (p) \ (task_thread_info(p)->saved_preempt_count & ~PREEMPT_NEED_RESCHED) - +/* +* Disable it twice to enter schedule_tail() with preemption disabled. +*/ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->saved_preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->saved_preempt_count = PREEMPT_DISABLED_TWICE; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
include/asm-generic/preempt.h +4 -2
@@ -25,9 +25,11 @@ static __always_inline void preempt_count_set(int pc)*/ #define task_preempt_count (p) \ (task_thread_info(p)->preempt_count & ~PREEMPT_NEED_RESCHED) - +/* +* Disable it twice to enter schedule_tail() with preemption disabled. +*/ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->preempt_count = PREEMPT_DISABLED_TWICE; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
include/linux/sched.h +2 -0
@@ -520,8 +520,10 @@ struct task_cputime {#ifdef CONFIG_PREEMPT_COUNT #define PREEMPT_DISABLED (1 + PREEMPT_ENABLED) +#define PREEMPT_DISABLED_TWICE (2 + PREEMPT_ENABLED)#else #define PREEMPT_DISABLED PREEMPT_ENABLED +#define PREEMPT_DISABLED_TWICE PREEMPT_ENABLED#endif /*
kernel/sched/core.c +2 -4
@@ -2203,12 +2203,10 @@ asmlinkage void schedule_tail(struct task_struct *prev)finish_task_switch (rq, prev); -/* -* FIXME: do we need to worry about rq being invalidated by the -* task_switch? -*/ post_schedule (rq); +preempt_enable (); +#ifdef __ARCH_WANT_UNLOCKED_CTXSW/* In this case, finish_task_switch does not reenable preemption */ preempt_enable ();
Peter Zijlstra
maintainerQuoted 3 lines · Kirill Tkhai
For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.
That's IA64 and MIPS; do they have a 'good' reason to use this?
That is; the alternative is to fix those two archs and remove the __ARCH_WANT_UNLOCKED_CTXSW clutter alltogether; which seems like a big win to me.
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Kirill Tkhai
For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.That's IA64 and MIPS; do they have a 'good' reason to use this?
It seems my description misleads reader, I'm sorry if so.
I mean all architectures *except* IA64 and MIPS. All, which has no __ARCH_WANT_UNLOCKED_CTXSW defined.
IA64 and MIPS already have preempt_enable() in schedule_tail():
#ifdef __ARCH_WANT_UNLOCKED_CTXSW
/* In this case, finish_task_switch does not reenable preemption */
preempt_enable();#endif
Their initial preemption is not decremented in finish_lock_switch().
So, we speak about x86, ARM64 etc.
Look at ARM64's finish_arch_post_lock_switch(). It looks a task must to not be preempted between switch_mm() and this function. But in case of new task this is possible.
Example:
RT thread p0 and RT thread p1 are on shared mm. System has 2 cpu.
p0 is bound to CPU0.
p1 is bound to CPU1.
p1 has set timer and it is sleeping.
p0 create fair thread f. Task f wakes on CPU1.
When f is between raw_spin_unlock_irq() and finish_arch_post_lock_switch(), preemption is enabled. In this moment the process p1 is waking on CPU1.
For p1 the check
if (!cpumask_test_and_set_cpu(cpu, mm_cpumask(next)) || prev != next)
in switch_mm() is not passed, because mm is the same. So, later we do not do cpu_switch_mm() in finish_arch_post_lock_switch() and we just go to userspace.
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
But it looks the place is buggy.
Kirill
Quoted 11 lines
That is; the alternative is to fix those two archs and remove the
__ARCH_WANT_UNLOCKED_CTXSWclutter alltogether; which seems like a big win to me.--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Quoted 20 lines · Kirill Tkhai
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Kirill Tkhai
For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.That's IA64 and MIPS; do they have a 'good' reason to use this?
It seems my description misleads reader, I'm sorry if so.
I mean all architectures *except* IA64 and MIPS. All, which has no
__ARCH_WANT_UNLOCKED_CTXSWdefined.IA64 and MIPS already have
preempt_enable()inschedule_tail():#ifdef __ARCH_WANT_UNLOCKED_CTXSW /* In this case, finish_task_switch does not reenable preemption */ preempt_enable();#endif
Quoted 7 lines
Their initial preemption is not decremented in
finish_lock_switch().So, we speak about x86, ARM64 etc.
Look at ARM64's
finish_arch_post_lock_switch(). It looks a task must to not be preempted betweenswitch_mm()and this function. But in case of new task this is possible.
We had a thread about this at the end of last year:
https://lkml.org/lkml/2013/11/15/82
There is indeed a problem on arm64, something like this (and I think s390 also needs a fix):
1. switch_mm() via check_and_switch_context() defers the actual mm switch by setting TIF_SWITCH_MM
2. the context switch is considered 'done' by the kernel before finish_arch_post_lock_switch() and therefore we can be preempted to a new thread before finish_arch_post_lock_switch()
3. The new thread has the same mm as the preempted thread but we actually missed the mm switching in finish_arch_post_lock_switch() because TIF_SWITCH_MM is per thread rather than mm
Quoted 2 lines
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
Have you managed to reproduce this? I don't say it doesn't exist, but I want to make sure that any patch actually fixes it.
So we have more solutions, one of the first two suitable for stable:
1. Propagate the TIF_SWITCH_MM to the next thread (suggested by Martin)
2. Get rid of TIF_SWITCH_MM and use mm_cpumask for tracking (I already have the patch, it just needs a lot more testing)
3. Re-write the ASID allocation algorithm to no longer require IPIs and therefore drop finish_arch_post_lock_switch() (this can be done, so pretty intrusive for stable)
4. Replace finish_arch_post_lock_switch() with finish_mm_switch() as per Martin's patch and I think this would guarantee a call always, we can move the mm switching from switch_mm() to finish_mm_switch() and no need for flags to mark deferred mm switching
For arm64, we'll most likely go with 2 for stable and move to 3 shortly after, no need for other deferred mm switching.
Signature · 1 line
Catalin
В Птн, 14/02/2014 в 10:52 +0000, Catalin Marinas пишет:
Quoted 45 lines · Kirill Tkhai
Quoted 20 lines · Peter Zijlstra
Quoted 6 lines · Kirill Tkhai
Quoted 3 lines
For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.That's IA64 and MIPS; do they have a 'good' reason to use this?
It seems my description misleads reader, I'm sorry if so.
I mean all architectures *except* IA64 and MIPS. All, which has no
__ARCH_WANT_UNLOCKED_CTXSWdefined.IA64 and MIPS already have
preempt_enable()inschedule_tail():#ifdef __ARCH_WANT_UNLOCKED_CTXSW /* In this case, finish_task_switch does not reenable preemption */ preempt_enable();#endif
Quoted 7 lines
Their initial preemption is not decremented in
finish_lock_switch().So, we speak about x86, ARM64 etc.
Look at ARM64's
finish_arch_post_lock_switch(). It looks a task must to not be preempted betweenswitch_mm()and this function. But in case of new task this is possible.We had a thread about this at the end of last year:
https://lkml.org/lkml/2013/11/15/82
There is indeed a problem on arm64, something like this (and I think s390 also needs a fix):
1.
switch_mm()viacheck_and_switch_context()defers the actual mm switch by settingTIF_SWITCH_MM
2. the context switch is considered 'done' by the kernel beforefinish_arch_post_lock_switch()and therefore we can be preempted to a new thread beforefinish_arch_post_lock_switch()
3. The new thread has the same mm as the preempted thread but we actually missed the mm switching infinish_arch_post_lock_switch()becauseTIF_SWITCH_MMis per thread rather than mm
Quoted 5 lines
Quoted 2 lines
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
Have you managed to reproduce this? I don't say it doesn't exist, but I want to make sure that any patch actually fixes it.
No, I have not tried. I found this place while analysing scheduler code. But it seems with the RT technics suggested previous message it's quite possible.
Quoted 13 lines
So we have more solutions, one of the first two suitable for stable:
1. Propagate the
TIF_SWITCH_MMto the next thread (suggested by Martin)
2. Get rid ofTIF_SWITCH_MMand usemm_cpumaskfor tracking (I already have the patch, it just needs a lot more testing)
3. Re-write the ASID allocation algorithm to no longer require IPIs and therefore dropfinish_arch_post_lock_switch()(this can be done, so pretty intrusive for stable)
4. Replacefinish_arch_post_lock_switch()withfinish_mm_switch()as per Martin's patch and I think this would guarantee a call always, we can move the mm switching fromswitch_mm()tofinish_mm_switch()and no need for flags to mark deferred mm switching
Quoted 3 lines
For arm64, we'll most likely go with 2 for stable and move to 3 shortly after, no need for other deferred mm switching.
It's good, but one of architectures is a corner case. It seems to me it's better to have the same enviroment/preemption for the first schedule and for others.
This minimises the number of rare errors and this may help in the future. The first schedule happens rare and it's tested worse.
Kirill
Quoted 25 lines · Kirill Tkhai
В Птн, 14/02/2014 в 10:52 +0000, Catalin Marinas пишет:
Quoted 24 lines · Kirill Tkhai
Quoted 3 lines
Look at ARM64's
finish_arch_post_lock_switch(). It looks a task must to not be preempted betweenswitch_mm()and this function. But in case of new task this is possible.We had a thread about this at the end of last year:
https://lkml.org/lkml/2013/11/15/82
There is indeed a problem on arm64, something like this (and I think s390 also needs a fix):
1.
switch_mm()viacheck_and_switch_context()defers the actual mm switch by settingTIF_SWITCH_MM
2. the context switch is considered 'done' by the kernel beforefinish_arch_post_lock_switch()and therefore we can be preempted to a new thread beforefinish_arch_post_lock_switch()
3. The new thread has the same mm as the preempted thread but we actually missed the mm switching infinish_arch_post_lock_switch()becauseTIF_SWITCH_MMis per thread rather than mmQuoted 2 lines
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
Quoted 6 lines
Quoted 2 lines
Have you managed to reproduce this? I don't say it doesn't exist, but I want to make sure that any patch actually fixes it.
No, I have not tried. I found this place while analysing scheduler code. But it seems with the RT technics suggested previous message it's quite possible.
Now I think I confused myself. Looking through the __schedule() code, context_switch() and therefore finish_arch_post_lock_switch() are called with preemption disabled. So the scenario above cannot exist since the current thread cannot be preempted between switch_mm() and finish_arch_post_lock_switch(). Do I miss anything?
Now I get your point about schedule_tail() which calls finish_task_switch() with a preempt count of 0. I'll get back to your original patch.
Thanks.
Signature · 1 line
Catalin
В Птн, 14/02/2014 в 12:21 +0000, Catalin Marinas пишет:
Quoted 33 lines · Kirill Tkhai
Quoted 25 lines
В Птн, 14/02/2014 в 10:52 +0000, Catalin Marinas пишет:
Quoted 24 lines · Kirill Tkhai
Quoted 3 lines
Look at ARM64's
finish_arch_post_lock_switch(). It looks a task must to not be preempted betweenswitch_mm()and this function. But in case of new task this is possible.We had a thread about this at the end of last year:
https://lkml.org/lkml/2013/11/15/82
There is indeed a problem on arm64, something like this (and I think s390 also needs a fix):
1.
switch_mm()viacheck_and_switch_context()defers the actual mm switch by settingTIF_SWITCH_MM
2. the context switch is considered 'done' by the kernel beforefinish_arch_post_lock_switch()and therefore we can be preempted to a new thread beforefinish_arch_post_lock_switch()
3. The new thread has the same mm as the preempted thread but we actually missed the mm switching infinish_arch_post_lock_switch()becauseTIF_SWITCH_MMis per thread rather than mmQuoted 2 lines
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
Quoted 6 lines
Quoted 2 lines
Have you managed to reproduce this? I don't say it doesn't exist, but I want to make sure that any patch actually fixes it.
No, I have not tried. I found this place while analysing scheduler code. But it seems with the RT technics suggested previous message it's quite possible.
Quoted 5 lines
Now I think I confused myself. Looking through the
__schedule()code,context_switch()and thereforefinish_arch_post_lock_switch()are called with preemption disabled. So the scenario above cannot exist since the current thread cannot be preempted betweenswitch_mm()andfinish_arch_post_lock_switch(). Do I miss anything?
Everything is right, the only case, which we have to worry, is schedule_tail()
Quoted 6 lines
Now I get your point about
schedule_tail()which callsfinish_task_switch()with a preempt count of 0. I'll get back to your original patch.Thanks.
Kirill
Quoted 48 lines · On Fri, 14 Feb 2014 10:52:55 +0000 Catalin Marinas <[email protected]>…
Quoted 20 lines · Kirill Tkhai
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Kirill Tkhai
For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.That's IA64 and MIPS; do they have a 'good' reason to use this?
It seems my description misleads reader, I'm sorry if so.
I mean all architectures *except* IA64 and MIPS. All, which has no
__ARCH_WANT_UNLOCKED_CTXSWdefined.IA64 and MIPS already have
preempt_enable()inschedule_tail():#ifdef __ARCH_WANT_UNLOCKED_CTXSW /* In this case, finish_task_switch does not reenable preemption */ preempt_enable();#endif
Quoted 7 lines
Their initial preemption is not decremented in
finish_lock_switch().So, we speak about x86, ARM64 etc.
Look at ARM64's
finish_arch_post_lock_switch(). It looks a task must to not be preempted betweenswitch_mm()and this function. But in case of new task this is possible.We had a thread about this at the end of last year:
https://lkml.org/lkml/2013/11/15/82
There is indeed a problem on arm64, something like this (and I think s390 also needs a fix):
1.
switch_mm()viacheck_and_switch_context()defers the actual mm switch by settingTIF_SWITCH_MM
2. the context switch is considered 'done' by the kernel beforefinish_arch_post_lock_switch()and therefore we can be preempted to a new thread beforefinish_arch_post_lock_switch()
3. The new thread has the same mm as the preempted thread but we actually missed the mm switching infinish_arch_post_lock_switch()becauseTIF_SWITCH_MMis per thread rather than mmQuoted 2 lines
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
Quoted 6 lines
Have you managed to reproduce this? I don't say it doesn't exist, but I want to make sure that any patch actually fixes it.
So we have more solutions, one of the first two suitable for stable:
1. Propagate the
TIF_SWITCH_MMto the next thread (suggested by Martin)
This is what I put in place for s390 but with the name TIF_TLB_WAIT instead of TIF_SWITCH_MM. I took the liberty to add the code to the features branch of the linux-s390 tree including the common code change that is necessary:
https://git.kernel.org/cgit/linux/kernel/git/s390/linux.git/commit/?h=features&id=09ddfb4d5602095aad04eada8bc8df59e873a6ef
https://git.kernel.org/cgit/linux/kernel/git/s390/linux.git/commit/?h=features&id=525d65f8f66ac29136ba6d2336f5a73b038701e2
These patches will be included in a please-pull request with the next merge window.
Quoted 13 lines
2. Get rid of
TIF_SWITCH_MMand usemm_cpumaskfor tracking (I already have the patch, it just needs a lot more testing)
3. Re-write the ASID allocation algorithm to no longer require IPIs and therefore dropfinish_arch_post_lock_switch()(this can be done, so pretty intrusive for stable)
4. Replacefinish_arch_post_lock_switch()withfinish_mm_switch()as per Martin's patch and I think this would guarantee a call always, we can move the mm switching fromswitch_mm()tofinish_mm_switch()and no need for flags to mark deferred mm switchingFor arm64, we'll most likely go with 2 for stable and move to 3 shortly after, no need for other deferred mm switching.
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.
Quoted 56 lines · Martin Schwidefsky
Quoted 53 lines · On Fri, 14 Feb 2014 10:52:55 +0000 Catalin Marinas <[email protected]>…
Quoted 20 lines · Kirill Tkhai
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Kirill Tkhai
For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.That's IA64 and MIPS; do they have a 'good' reason to use this?
It seems my description misleads reader, I'm sorry if so.
I mean all architectures *except* IA64 and MIPS. All, which has no
__ARCH_WANT_UNLOCKED_CTXSWdefined.IA64 and MIPS already have
preempt_enable()inschedule_tail():#ifdef __ARCH_WANT_UNLOCKED_CTXSW /* In this case, finish_task_switch does not reenable preemption */ preempt_enable();#endif
Quoted 7 lines
Their initial preemption is not decremented in
finish_lock_switch().So, we speak about x86, ARM64 etc.
Look at ARM64's
finish_arch_post_lock_switch(). It looks a task must to not be preempted betweenswitch_mm()and this function. But in case of new task this is possible.We had a thread about this at the end of last year:
https://lkml.org/lkml/2013/11/15/82
There is indeed a problem on arm64, something like this (and I think s390 also needs a fix):
1.
switch_mm()viacheck_and_switch_context()defers the actual mm switch by settingTIF_SWITCH_MM
2. the context switch is considered 'done' by the kernel beforefinish_arch_post_lock_switch()and therefore we can be preempted to a new thread beforefinish_arch_post_lock_switch()
3. The new thread has the same mm as the preempted thread but we actually missed the mm switching infinish_arch_post_lock_switch()becauseTIF_SWITCH_MMis per thread rather than mmQuoted 2 lines
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
Have you managed to reproduce this? I don't say it doesn't exist, but I want to make sure that any patch actually fixes it.
So we have more solutions, one of the first two suitable for stable:
Quoted 7 lines
Quoted 1 line
1. Propagate the
TIF_SWITCH_MMto the next thread (suggested by Martin)This is what I put in place for s390 but with the name
TIF_TLB_WAITinstead ofTIF_SWITCH_MM. I took the liberty to add the code to the features branch of the linux-s390 tree including the common code change that is necessary:
I don't see a problem with additional calls to finish_arch_post_lock_switch() on arm and arm64 but I would have done this in more than one step:
1. Introduce finish_switch_mm()
2. Convert arm and arm64 to finish_switch_mm() (which means we no longer check whether the interrupts are disabled in switch_mm() to defer the switch
3. Remove generic finish_arch_post_lock_switch() because its functionality has been entirely replaced by finish_switch_mm()
Anyway, we probably end up in the same place anyway.
But does this solve the problem of being preempted between switch_mm() and finish_arch_post_lock_switch()? I guess we still need the same guarantees that both switch_mm() and the hook happen on the same CPU.
That's a way to solve it for s390. I don't particularly like transferring the mm switch pending TIF flag to the next task but I think it does the job (just personal preference).
Signature · 1 line
Catalin
Quoted 69 lines · On Mon, 17 Feb 2014 10:40:06 +0000 Catalin Marinas <[email protected]>…
Quoted 56 lines · Martin Schwidefsky
Quoted 53 lines · On Fri, 14 Feb 2014 10:52:55 +0000 Catalin Marinas <[email protected]>…
Quoted 20 lines · Kirill Tkhai
Quoted 6 lines · Peter Zijlstra
Quoted 3 lines · Kirill Tkhai
For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.That's IA64 and MIPS; do they have a 'good' reason to use this?
It seems my description misleads reader, I'm sorry if so.
I mean all architectures *except* IA64 and MIPS. All, which has no
__ARCH_WANT_UNLOCKED_CTXSWdefined.IA64 and MIPS already have
preempt_enable()inschedule_tail():#ifdef __ARCH_WANT_UNLOCKED_CTXSW /* In this case, finish_task_switch does not reenable preemption */ preempt_enable();#endif
Quoted 7 lines
Their initial preemption is not decremented in
finish_lock_switch().So, we speak about x86, ARM64 etc.
Look at ARM64's
finish_arch_post_lock_switch(). It looks a task must to not be preempted betweenswitch_mm()and this function. But in case of new task this is possible.We had a thread about this at the end of last year:
https://lkml.org/lkml/2013/11/15/82
There is indeed a problem on arm64, something like this (and I think s390 also needs a fix):
1.
switch_mm()viacheck_and_switch_context()defers the actual mm switch by settingTIF_SWITCH_MM
2. the context switch is considered 'done' by the kernel beforefinish_arch_post_lock_switch()and therefore we can be preempted to a new thread beforefinish_arch_post_lock_switch()
3. The new thread has the same mm as the preempted thread but we actually missed the mm switching infinish_arch_post_lock_switch()becauseTIF_SWITCH_MMis per thread rather than mmQuoted 2 lines
This is the problem I tried to solve. I don't know arm64, and I can't say how it is serious.
Have you managed to reproduce this? I don't say it doesn't exist, but I want to make sure that any patch actually fixes it.
So we have more solutions, one of the first two suitable for stable:
Quoted 7 lines
Quoted 1 line
1. Propagate the
TIF_SWITCH_MMto the next thread (suggested by Martin)This is what I put in place for s390 but with the name
TIF_TLB_WAITinstead ofTIF_SWITCH_MM. I took the liberty to add the code to the features branch of the linux-s390 tree including the common code change that is necessary:I don't see a problem with additional calls to
finish_arch_post_lock_switch()on arm and arm64 but I would have done this in more than one step:
Quoted 8 lines
1. Introduce
finish_switch_mm()
2. Convert arm and arm64 tofinish_switch_mm()(which means we no longer check whether the interrupts are disabled inswitch_mm()to defer the switch
3. Remove genericfinish_arch_post_lock_switch()because its functionality has been entirely replaced byfinish_switch_mm()Anyway, we probably end up in the same place anyway.
Peter pointed me to finish_arch_post_lock switch as a replacement for finish_switch_mm. They are basically doing the same thing and I do not care too much how the function is called. finish_arch_post_lock_switch is ok from my point of view. If you want to change it be aware of the header file hell you are getting into.
Quoted 3 lines
But does this solve the problem of being preempted between
switch_mm()andfinish_arch_post_lock_switch()? I guess we still need the same guarantees that bothswitch_mm()and the hook happen on the same CPU.
By itself no, I do not think so. finish_arch_post_lock_switch is supposed to be called with no locks held, so preemption should be enabled as well.
Quoted 5 lines
That's a way to solve it for s390. I don't particularly like transferring the mm switch pending TIF flag to the next task but I think it does the job (just personal preference).
It gets the job done. The alternative is another per-cpu bitmap for each mm. I prefer the transfer of the TIF flag to the next task, we do that for the machine check flag TIF_MCCK_PENDING anyway. One more bit to transfer does not hurt.
Signature · 4 lines
blue skies, Martin. "Reality continues to ruin my life." - Calvin.
Quoted 16 lines · Kirill Tkhai
Preemption state on enter in
finish_task_switch()is different in cases ofcontext_switch()andschedule_tail().In the first case we have it twice disabled: at the start of
schedule()and during spin locking. In the second it is only
once: the value which was set ininit_task_preempt_count().For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.It seems there is possible a problem in rare situations on arm64, when one freshly created thread preempts another before
finish_arch_post_lock_switch()has finished. If mm is the same, thenTIF_SWITCH_MMon the second won't be set.
Quoted 4 lines
The second rare but possible issue is zeroing of
post_schedule()on a wrong cpu.So, lets fix this and unify
preempt_countstate.
An alternative to your patch:
kernel/sched/core.c +4 -3
@@ -2210,6 +2210,10 @@ asmlinkage void schedule_tail(struct task_struct *prev) {struct rq *rq =this_rq (); +#ifndef __ARCH_WANT_UNLOCKED_CTXSW +/* In this case, finish_task_switch reenables preemption */ +preempt_disable (); +#endif finish_task_switch (rq, prev);/*
@@ -2218,10 +2222,7 @@ asmlinkage void schedule_tail(struct task_struct *prev)*/ post_schedule (rq); -#ifdef __ARCH_WANT_UNLOCKED_CTXSW -/* In this case, finish_task_switch does not reenable preemption */ preempt_enable (); -#endif if (current->set_child_tid)put_user (task_pid_vnr (current), current->set_child_tid); }
Signature · 1 line
Catalin
В Птн, 14/02/2014 в 12:35 +0000, Catalin Marinas пишет:
Quoted 17 lines · Kirill Tkhai
Quoted 16 lines
Preemption state on enter in
finish_task_switch()is different in cases ofcontext_switch()andschedule_tail().In the first case we have it twice disabled: at the start of
schedule()and during spin locking. In the second it is only
once: the value which was set ininit_task_preempt_count().For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.It seems there is possible a problem in rare situations on arm64, when one freshly created thread preempts another before
finish_arch_post_lock_switch()has finished. If mm is the same, thenTIF_SWITCH_MMon the second won't be set.
Quoted 6 lines
Quoted 4 lines
The second rare but possible issue is zeroing of
post_schedule()on a wrong cpu.So, lets fix this and unify
preempt_countstate.An alternative to your patch:
It looks better, than the initial.
You may add my Acked-by if you want.
Quoted 27 lines
kernel/sched/core.c +4
-3@@ -2210,6 +2210,10 @@ asmlinkage void schedule_tail(struct task_struct *prev) {struct rq *rq =this_rq (); +#ifndef __ARCH_WANT_UNLOCKED_CTXSW +/* In this case, finish_task_switch reenables preemption */ +preempt_disable (); +#endif finish_task_switch (rq, prev);/* @@ -2218,10 +2222,7 @@ asmlinkage void schedule_tail(struct task_struct *prev)*/ post_schedule (rq); -#ifdef __ARCH_WANT_UNLOCKED_CTXSW -/* In this case, finish_task_switch does not reenable preemption */ preempt_enable (); -#endif if (current->set_child_tid)put_user (task_pid_vnr (current), current->set_child_tid); }
Quoted 21 lines · Kirill Tkhai
В Птн, 14/02/2014 в 12:35 +0000, Catalin Marinas пишет:
Quoted 20 lines · Kirill Tkhai
Quoted 19 lines
Preemption state on enter in
finish_task_switch()is different in cases ofcontext_switch()andschedule_tail().In the first case we have it twice disabled: at the start of
schedule()and during spin locking. In the second it is only
once: the value which was set ininit_task_preempt_count().For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.It seems there is possible a problem in rare situations on arm64, when one freshly created thread preempts another before
finish_arch_post_lock_switch()has finished. If mm is the same, thenTIF_SWITCH_MMon the second won't be set.The second rare but possible issue is zeroing of
post_schedule()on a wrong cpu.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
So, lets fix this and unify
preempt_countstate.An alternative to your patch:
It looks better, than the initial.
You may add my Acked-by if you want.
Thanks for the ack. But apart from arm64, are there any other problems with running part of finish_task_switch() and post_schedule() with preemption enabled?
The finish_arch_post_lock_switch() is currently only used by arm and arm64 (the former UP only) and arm no longer has the preemption problem (see commit bdae73cd374e2). So I can either disable the preemption around schedule_tail() call in arm64 or do it globally as per yours or my patch.
Peter, Ingo, any thoughts? Do we care about preempt count consistency across finish_task_switch() and post_schedule()?
Thanks.
Signature · 1 line
Catalin
В Птн, 14/02/2014 в 15:49 +0000, Catalin Marinas пишет:
Quoted 26 lines · Kirill Tkhai
Quoted 25 lines
В Птн, 14/02/2014 в 12:35 +0000, Catalin Marinas пишет:
Quoted 23 lines · Kirill Tkhai
Quoted 16 lines
Preemption state on enter in
finish_task_switch()is different in cases ofcontext_switch()andschedule_tail().In the first case we have it twice disabled: at the start of
schedule()and during spin locking. In the second it is only
once: the value which was set ininit_task_preempt_count().For archs without
__ARCH_WANT_UNLOCKED_CTXSWset this means that all newly created tasks executefinish_arch_post_lock_switch()andpost_schedule()with preemption enabled.It seems there is possible a problem in rare situations on arm64, when one freshly created thread preempts another before
finish_arch_post_lock_switch()has finished. If mm is the same, thenTIF_SWITCH_MMon the second won't be set.Quoted 4 lines
The second rare but possible issue is zeroing of
post_schedule()on a wrong cpu.So, lets fix this and unify
preempt_countstate.An alternative to your patch:
Quoted 7 lines
Quoted 3 lines
It looks better, than the initial.
You may add my Acked-by if you want.
Thanks for the ack. But apart from arm64, are there any other problems with running part of
finish_task_switch()andpost_schedule()with preemption enabled?
1)We have architecture-dependent finish_arch_post_lock_switch()
which is possible(?) to be fixed for every arch at the moment, but someone may run into it in the future.
2)The second is fire_sched_in_preempt_notifiers(). It's generic interface
which is currently unused. It notifies about preemption, so it's bad if additional preemption happens when it has not finished.
3)tick_nohz_task_switch() seems to be without problems. Just very-very
slightly performance.
4)If post_schedule() happens on wrong CPU, the system may occur imbalanced
for a short period. This happens, when post_schedule() of wrong class is executed.
If we fix that once in scheduler we'll decide everything above now and in the future. Also we'll decrease number of rare situations.
Quoted 11 lines
The
finish_arch_post_lock_switch()is currently only used by arm and arm64 (the former UP only) and arm no longer has the preemption problem (see commitbdae73cd374e2). So I can either disable the preemption aroundschedule_tail()call in arm64 or do it globally as per yours or my patch.Peter, Ingo, any thoughts? Do we care about preempt count consistency across
finish_task_switch()andpost_schedule()?Thanks.