sched: Killing PREEMPT_ACTIVE
Subsystems (6)
Peter Zijlstra
maintainercover sched: Killing PREEMPT_ACTIVE
This series kills PREEMPT_ACTIVE dead, its a tad risky, but survives light testing on x86_64 and seems to compile on everything else.
--- arch/x86/include/asm/preempt.h | 5 +-- arch/x86/include/asm/thread_info.h | 2 -- arch/x86/kernel/process_32.c | 8 ----- arch/x86/kernel/process_64.c | 8 ----- include/asm-generic/preempt.h | 2 +- include/linux/preempt.h | 20 +---------- include/linux/sched.h | 7 ++-- include/trace/events/sched.h | 22 +++++------- kernel/exit.c | 4 ++- kernel/sched/core.c | 70 +++++++++++++++++++++----------------- kernel/trace/ftrace.c | 2 +- kernel/trace/trace_sched_switch.c | 3 +- kernel/trace/trace_sched_wakeup.c | 2 +- 13 files changed, 60 insertions(+), 95 deletions(-)
Peter Zijlstra
maintainerRFC PATCH 1/11 sched: Simplify INIT_PREEMPT_COUNT
As per commit d86ee4809d03 ("sched: optimize cond_resched()") we need PREEMPT_ACTIVE to avoid cond_resched() from working before the scheduler is setup.
However, keeping preemption disabled should do the same thing already, making the PREEMPT_ACTIVE part entirely redundant.
--- include/linux/sched.h | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
include/linux/sched.h +2 -5
@@ -608,17 +608,14 @@ struct task_cputime_atomic {/* * Disable preemption until the scheduler is running. * Reset by start_kernel()->sched_init()->init_idle(). -* -* We include PREEMPT_ACTIVE to avoid cond_resched() from working -* before the scheduler is active -- see should_resched(). */ -#define INIT_PREEMPT_COUNT (PREEMPT_DISABLED + PREEMPT_ACTIVE) +#define INIT_PREEMPT_COUNT PREEMPT_DISABLED/** * struct thread_group_cputimer - thread group interval timer counts * @cputime_atomic: atomic thread group interval timers. * @running: non-zero when there are timers running and -* @cputime receives updates. +* @cputime receives updates. * * This structure contains the version of task_cputime, above, that is * used for thread group CPU timer calculations.
tip-bot for Peter Zijlstra
merge bot[tip:sched/core] sched/core: Simplify INIT_PREEMPT_COUNT
sched/core: Simplify INIT_PREEMPT_COUNT
Commit-ID: aed586c2f11e5cddd52c180ccde651d0d35b736b Gitweb: http://git.kernel.org/tip/aed586c2f11e5cddd52c180ccde651d0d35b736b Author: Peter Zijlstra <[email protected]> AuthorDate: Tue, 29 Sep 2015 11:28:26 +0200 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 29 Sep 2015 12:27:39 +0200
sched/core: Simplify INIT_PREEMPT_COUNT
As per commit d86ee4809d03 ("sched: optimize cond_resched()") we need PREEMPT_ACTIVE to avoid cond_resched() from working before the scheduler is setup.
However, keeping preemption disabled should do the same thing already, making the PREEMPT_ACTIVE part entirely redundant.
--- include/linux/sched.h | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
include/linux/sched.h +2 -5
@@ -608,17 +608,14 @@ struct task_cputime_atomic {/* * Disable preemption until the scheduler is running. * Reset by start_kernel()->sched_init()->init_idle(). -* -* We include PREEMPT_ACTIVE to avoid cond_resched() from working -* before the scheduler is active -- see should_resched(). */ -#define INIT_PREEMPT_COUNT (PREEMPT_DISABLED + PREEMPT_ACTIVE) +#define INIT_PREEMPT_COUNT PREEMPT_DISABLED/** * struct thread_group_cputimer - thread group interval timer counts * @cputime_atomic: atomic thread group interval timers. * @running: non-zero when there are timers running and -* @cputime receives updates. +* @cputime receives updates. * * This structure contains the version of task_cputime, above, that is * used for thread group CPU timer calculations.
Re: [RFC][PATCH 01/11] sched: Simplify INIT_PREEMPT_COUNT
In reply to Peter ZijlstraQuoted 8 lines · Peter Zijlstra
As per commit
d86ee4809d03("sched: optimizecond_resched()") we needPREEMPT_ACTIVEto avoidcond_resched()from working before the scheduler is setup.However, keeping preemption disabled should do the same thing already, making the
PREEMPT_ACTIVEpart entirely redundant.Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Re: [RFC][PATCH 01/11] sched: Simplify INIT_PREEMPT_COUNT
In reply to Peter Zijlstra[ Resend with a "Reply-all" this time! ]
Quoted 6 lines · On Tue, 29 Sep 2015 11:28:26 +0200 Peter Zijlstra <[email protected]> wrote:
As per commit
d86ee4809d03("sched: optimizecond_resched()") we needPREEMPT_ACTIVEto avoidcond_resched()from working before the scheduler is setup.However, keeping preemption disabled should do the same thing already, making the
PREEMPT_ACTIVEpart entirely redundant.
Thus PREEMPT_ACTIVE wasn't needed at that commit either, or was it?
Quoted 9 lines
Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- include/linux/sched.h | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-)
include/linux/sched.h struct task_cputime_atomic { · line 612
Quoted 9 lines
include/linux/sched.h +1
-4@@ -608,17 +608,14 @@ struct task_cputime_atomic {/* * Disable preemption until the scheduler is running. * Reset by start_kernel()->sched_init()->init_idle(). -* -* We include PREEMPT_ACTIVE to avoid cond_resched() from working -* before the scheduler is active -- see should_resched(). */ -#define INIT_PREEMPT_COUNT (PREEMPT_DISABLED + PREEMPT_ACTIVE) +#define INIT_PREEMPT_COUNT PREEMPT_DISABLED
I can't find anything wrong with this.
-- Steve
Quoted 11 lines
include/linux/sched.h +1
-1/** * struct thread_group_cputimer - thread group interval timer counts * @cputime_atomic: atomic thread group interval timers. * @running: non-zero when there are timers running and -* @cputime receives updates. +* @cputime receives updates. * * This structure contains the version of task_cputime, above, that is * used for thread group CPU timer calculations.
Peter Zijlstra
maintainerRe: [RFC][PATCH 01/11] sched: Simplify INIT_PREEMPT_COUNT
Quoted 6 lines · Steven Rostedt
[ Resend with a "Reply-all" this time! ]
On Tue, 29 Sep 2015 11:28:26 +0200
Peter Zijlstra <[email protected]> wrote:
Quoted 8 lines
Quoted 6 lines
As per commit
d86ee4809d03("sched: optimizecond_resched()") we needPREEMPT_ACTIVEto avoidcond_resched()from working before the scheduler is setup.However, keeping preemption disabled should do the same thing already, making the
PREEMPT_ACTIVEpart entirely redundant.Thus
PREEMPT_ACTIVEwasn't needed at that commit either, or was it?
Correct. We might have actually discussed that at that time but decided to be conservative -- but these be vague memories..
A message that is not in the archive (sent off-list or not yet received): [email protected]
A message that is not in the archive (sent off-list or not yet received): [email protected]
Steven Rostedt
maintainerRe: [RFC][PATCH 01/11] sched: Simplify INIT_PREEMPT_COUNT
[ Resend, with the Cc list back ]
Quoted 8 lines · On Tue, 29 Sep 2015 16:35:59 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 7 lines · Steven Rostedt
Quoted 4 lines · On Tue, 29 Sep 2015 11:28:26 +0200 Peter Zijlstra <[email protected]> wrote:
As per commit
d86ee4809d03("sched: optimizecond_resched()") we needPREEMPT_ACTIVEto avoidcond_resched()from working before the scheduler is setup.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
However, keeping preemption disabled should do the same thing already, making the
PREEMPT_ACTIVEpart entirely redundant.Thus
PREEMPT_ACTIVEwasn't needed at that commit either, or was it?Correct. We might have actually discussed that at that time but decided to be conservative -- but these be vague memories..
Hmm, I just looked closer and I'm wondering if this still works?
We have:
#ifdef CONFIG_PREEMPT_COUNT
#define PREEMPT_DISABLED (1 + PREEMPT_ENABLED)
#else
#define PREEMPT_DISABLED PREEMPT_ENABLED
#endif
Now if we just remove the PREEMPT_ACTIVE from the INIT_PREEMPT_COUNT, when CONFIG_PREEMPT_COUNT is not set, wouldn't a _cond_resched() in boot up schedule?
-- Steve
Peter Zijlstra
maintainerRe: [RFC][PATCH 01/11] sched: Simplify INIT_PREEMPT_COUNT
Quoted 2 lines · Steven Rostedt
[ Resend, with the Cc list back ]
You seem to have a hard time with these buttons today ;-)
Quoted 10 lines
Hmm, I just looked closer and I'm wondering if this still works?
We have:
#ifdef
CONFIG_PREEMPT_COUNT
#definePREEMPT_DISABLED(1 +PREEMPT_ENABLED)
#else
#definePREEMPT_DISABLEDPREEMPT_ENABLED
#endif
Quoted 3 lines
Now if we just remove the
PREEMPT_ACTIVEfrom theINIT_PREEMPT_COUNT, whenCONFIG_PREEMPT_COUNTis not set, wouldn't a_cond_resched()in boot up schedule?
Good point, PREEMPT_COUNT isn't required for voluntary preemption; when I tested the voluntary build nothing bad happened, but that could have been luck of course.
So what we can do is use an unconditional 1 for INIT_PREEMPT_COUNT, such that the init task will have a raised preempt_count() during boot, even on !PREEMPT_COUNT kernels.
But that means we cannot use INIT_PREEMPT_COUNT for init_task_preempt_count() and such like, as I wanted to. I'll make FORK_PREEMPT_COUNT for that instead.
Peter Zijlstra
maintainerRFC PATCH 2/11 sched: Create preempt_count invariant
In reply to Peter ZijlstraEnsure that upon scheduling preempt_count == 2; although currently an additional PREEMPT_ACTIVE is still possible.
--- arch/x86/include/asm/preempt.h | 3 ++- include/asm-generic/preempt.h | 2 +- kernel/sched/core.c | 14 ++++++++++---- 3 files changed, 13 insertions(+), 6 deletions(-)
arch/x86/include/asm/preempt.h +2 -1
@@ -31,7 +31,8 @@ static __always_inline void preempt_coun* must be macros to avoid header recursion hell */ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->saved_preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->saved_preempt_count = \ + 2*PREEMPT_DISABLE_OFFSET + PREEMPT_NEED_RESCHED; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
include/asm-generic/preempt.h +1 -1
@@ -24,7 +24,7 @@ static __always_inline void preempt_coun* must be macros to avoid header recursion hell */ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->preempt_count = 2*PREEMPT_DISABLED; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
kernel/sched/core.c +10 -4
@@ -2588,11 +2588,17 @@ asmlinkage __visible void schedule_tail( {struct rq *rq; -/* finish_task_switch() drops rq->lock and enables preemtion */ -preempt_disable (); - rq =finish_task_switch (prev); +/* +* Still have preempt_count() == 2, from: +* +* schedule() +* preempt_disable(); // 1 +* __schedule() +* raw_spin_lock_irq(&rq->lock) // 2 +*/ + rq =finish_task_switch (prev);/* drops rq->lock, preempt_count() == 1 */ balance_callback (rq); -preempt_enable (); +preempt_enable ();/* preempt_count() == 0 */ if (current->set_child_tid)put_user (task_pid_vnr (current), current->set_child_tid);
tip-bot for Peter Zijlstra
merge bot[tip:sched/core] sched/core: Create preempt_count invariant
sched/core: Create preempt_count invariant
Commit-ID: e26d555f695ac8a3aa38055dd04bd23c1334723b Gitweb: http://git.kernel.org/tip/e26d555f695ac8a3aa38055dd04bd23c1334723b Author: Peter Zijlstra <[email protected]> AuthorDate: Tue, 29 Sep 2015 11:28:27 +0200 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 29 Sep 2015 12:27:40 +0200
sched/core: Create preempt_count invariant
Ensure that upon scheduling preempt_count == 2; although currently an additional PREEMPT_ACTIVE is still possible.
--- arch/x86/include/asm/preempt.h | 3 ++- include/asm-generic/preempt.h | 2 +- kernel/sched/core.c | 14 ++++++++++---- 3 files changed, 13 insertions(+), 6 deletions(-)
arch/x86/include/asm/preempt.h +2 -1
@@ -31,7 +31,8 @@ static __always_inline void preempt_count_set(int pc)* must be macros to avoid header recursion hell */ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->saved_preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->saved_preempt_count = \ + 2*PREEMPT_DISABLE_OFFSET + PREEMPT_NEED_RESCHED; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
include/asm-generic/preempt.h +1 -1
@@ -24,7 +24,7 @@ static __always_inline void preempt_count_set(int pc)* must be macros to avoid header recursion hell */ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->preempt_count = 2*PREEMPT_DISABLED; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
kernel/sched/core.c +10 -4
@@ -2588,11 +2588,17 @@ asmlinkage __visible void schedule_tail(struct task_struct *prev) {struct rq *rq; -/* finish_task_switch() drops rq->lock and enables preemtion */ -preempt_disable (); - rq =finish_task_switch (prev); +/* +* Still have preempt_count() == 2, from: +* +* schedule() +* preempt_disable(); // 1 +* __schedule() +* raw_spin_lock_irq(&rq->lock) // 2 +*/ + rq =finish_task_switch (prev);/* drops rq->lock, preempt_count() == 1 */ balance_callback (rq); -preempt_enable (); +preempt_enable ();/* preempt_count() == 0 */ if (current->set_child_tid)put_user (task_pid_vnr (current), current->set_child_tid);
Re: [RFC][PATCH 02/11] sched: Create preempt_count invariant
In reply to Peter ZijlstraQuoted 25 lines · Peter Zijlstra
Ensure that upon scheduling
preempt_count== 2; although currently an additionalPREEMPT_ACTIVEis still possible.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- arch/x86/include/asm/preempt.h | 3 ++- include/asm-generic/preempt.h | 2 +- kernel/sched/core.c | 14 ++++++++++---- 3 files changed, 13 insertions(+), 6 deletions(-)arch/x86/include/asm/preempt.h +2
-1@@ -31,7 +31,8 @@ static __always_inline void preempt_coun* must be macros to avoid header recursion hell */ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->saved_preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->saved_preempt_count = \ + 2*PREEMPT_DISABLE_OFFSET + PREEMPT_NEED_RESCHED; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
include/asm-generic/preempt.h static __always_inline void preempt_coun · line 27
Quoted 5 lines
include/asm-generic/preempt.h +1
-1@@ -24,7 +24,7 @@ static __always_inline void preempt_coun* must be macros to avoid header recursion hell */ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->preempt_count = PREEMPT_DISABLED; \ + task_thread_info(p)->preempt_count = 2*PREEMPT_DISABLED; \
Since it's not quite obvious why we use this magic value without looking
at schedule_tail() details, maybe add a little comment? (Just "/* see schedule_tail() */").
Quoted 10 lines
include/asm-generic/preempt.h +0
-0} while (0)#define init_idle_preempt_count (p, cpu) do { \kernel/sched/core.c +0
-1@@ -2588,11 +2588,17 @@ asmlinkage __visible void schedule_tail( {struct rq *rq; -/* finish_task_switch() drops rq->lock and enables preemtion */
kernel/sched/core.c asmlinkage __visible void schedule_tail( · line 2598
Quoted 10 lines
kernel/sched/core.c +8
-2-preempt_disable (); - rq =finish_task_switch (prev); +/* +* Still have preempt_count() == 2, from: +* +* schedule() +* preempt_disable(); // 1 +* __schedule() +* raw_spin_lock_irq(&rq->lock) // 2 +*/
I found that a bit confusing first, because that's a preempt_count() we actually emulate for a new task. Maybe something like:
+ /*
+ * New task is init with preempt_count() == 2 because prev task left
+ * us after:
+ *
+ * schedule()
+ * preempt_disable(); // 1
+ * __schedule()
+ * raw_spin_lock_irq(&rq->lock) // 2
+ */
kernel/sched/core.c asmlinkage __visible void schedule_tail( · line 2601
Quoted 4 lines
kernel/sched/core.c +2
-1+ rq =finish_task_switch (prev);/* drops rq->lock, preempt_count() == 1 */ balance_callback (rq); -preempt_enable (); +preempt_enable ();/* preempt_count() == 0 */
Thanks!
Peter Zijlstra
maintainerRe: [RFC][PATCH 02/11] sched: Create preempt_count invariant
Quoted 7 lines · Frederic Weisbecker
include/asm-generic/preempt.h static __always_inline void preempt_coun · line 27
Quoted 3 lines · Peter Zijlstra
#define
init_task_preempt_count(p)do { \include/asm-generic/preempt.h +1
-1-task_thread_info (p)->preempt_count =PREEMPT_DISABLED ; \ +task_thread_info (p)->preempt_count =2 *PREEMPT_DISABLED ; \Since it's not quite obvious why we use this magic value without looking
atschedule_tail()details, maybe add a little comment? (Just "/* seeschedule_tail()*/").
Right, I fixed that in 12/11 v2. I'll change that around a bit.
Quoted 12 lines
kernel/sched/core.c asmlinkage __visible void schedule_tail( · line 2598
Quoted 8 lines
kernel/sched/core.c +8
-0+/* +* Still have preempt_count() == 2, from: +* +* schedule() +* preempt_disable(); // 1 +* __schedule() +* raw_spin_lock_irq(&rq->lock) // 2 +*/ I found that a bit confusing first, because that's a
preempt_count()we actually emulate for a new task. Maybe something like:
kernel/sched/core.c asmlinkage __visible void schedule_tail( · line 2598
Quoted 9 lines
kernel/sched/core.c +9
-0+/* +* New task is init with preempt_count() == 2 because prev task left +* us after: +* +* schedule() +* preempt_disable(); // 1 +* __schedule() +* raw_spin_lock_irq(&rq->lock) // 2 +*/
I think I'll move the comment to finish_task_switch(), but yes.
Steven Rostedt
maintainerRe: [RFC][PATCH 02/11] sched: Create preempt_count invariant
Quoted 10 lines · On Tue, 29 Sep 2015 15:02:01 +0200 Peter Zijlstra <[email protected]> wrote:
Quoted 7 lines · Frederic Weisbecker
include/asm-generic/preempt.h static __always_inline void preempt_coun · line 27
Quoted 3 lines · Peter Zijlstra
#define
init_task_preempt_count(p)do { \include/asm-generic/preempt.h +1
-1-task_thread_info (p)->preempt_count =PREEMPT_DISABLED ; \ +task_thread_info (p)->preempt_count =2 *PREEMPT_DISABLED ; \Since it's not quite obvious why we use this magic value without looking
atschedule_tail()details, maybe add a little comment? (Just "/* seeschedule_tail()*/").Right, I fixed that in 12/11 v2. I'll change that around a bit.
I'll wait for v2 because this confused me too.
-- Steve
Re: [RFC][PATCH 02/11] sched: Create preempt_count invariant
In reply to Peter Zijlstrakernel/sched/core.c asmlinkage __visible void schedule_tail( · line 2601
Quoted 12 lines · Peter Zijlstra
kernel/sched/core.c +10
-1+/* +* Still have preempt_count() == 2, from: +* +* schedule() +* preempt_disable(); // 1 +* __schedule() +* raw_spin_lock_irq(&rq->lock) // 2 +*/ + rq =finish_task_switch (prev);/* drops rq->lock, preempt_count() == 1 */ balance_callback (rq); -preempt_enable (); +preempt_enable ();/* preempt_count() == 0 */
Bah. I so hate tail comments. What's wrong with
+ /* preempt_count() ==> 0 */
preempt_enable();
Hmm?
tglx
Peter Zijlstra
maintainerRe: [RFC][PATCH 02/11] sched: Create preempt_count invariant
Quoted 19 lines · Thomas Gleixner
kernel/sched/core.c asmlinkage __visible void schedule_tail( · line 2601
Quoted 12 lines · Peter Zijlstra
kernel/sched/core.c +10
-1+/* +* Still have preempt_count() == 2, from: +* +* schedule() +* preempt_disable(); // 1 +* __schedule() +* raw_spin_lock_irq(&rq->lock) // 2 +*/ + rq =finish_task_switch (prev);/* drops rq->lock, preempt_count() == 1 */ balance_callback (rq); -preempt_enable (); +preempt_enable ();/* preempt_count() == 0 */ Bah. I so hate tail comments. What's wrong with
(quoted diff) +1
-0+ /* preempt_count() ==> 0 */preempt_enable();
Quoted 1 line
Hmm?
I find the tail comments more readable in this case; clearly I don't share your hatred :-). But I can change them if you insist.
Peter Zijlstra
maintainerRFC PATCH 3/11 sched: Robustify preemption leak checks
In reply to Peter ZijlstraWhen we warn about a preempt_count leak; reset the preempt_count to the known good value such that the problem does not ripple forward.
--- kernel/exit.c | 4 +++- kernel/sched/core.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
kernel/exit.c +3 -1
@@ -706,10 +706,12 @@ void do_exit(long code)smp_mb ();raw_spin_unlock_wait (&tsk->pi_lock); -if (unlikely (in_atomic ())) +if (unlikely (in_atomic ())) {pr_info ("note: %s[%d] exited with preempt_count %d\n" , current->comm,task_pid_nr (current),preempt_count ()); +preempt_count_set (PREEMPT_ENABLED ); + }/* sync mm's RSS info before statistics gathering */ if (tsk->mm)
kernel/sched/core.c +3 -1
@@ -2960,8 +2960,10 @@ static inline void schedule_debug(struct* schedule() atomically, we ignore that path. Otherwise whine * if we are scheduling when we should not. */ -if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) +if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) {__schedule_bug (prev); +preempt_count_set (PREEMPT_DISABLED ); + }rcu_sleep_check ();profile_hit (SCHED_PROFILING ,__builtin_return_address (0 ));
tip-bot for Peter Zijlstra
merge bot[tip:sched/core] sched/core: Robustify preemption leak checks
sched/core: Robustify preemption leak checks
Commit-ID: 185b42420b0948dfc00af6cc5d93aa00b7fef581 Gitweb: http://git.kernel.org/tip/185b42420b0948dfc00af6cc5d93aa00b7fef581 Author: Peter Zijlstra <[email protected]> AuthorDate: Tue, 29 Sep 2015 11:28:28 +0200 Committer: Ingo Molnar <[email protected]> CommitDate: Tue, 29 Sep 2015 12:27:40 +0200
sched/core: Robustify preemption leak checks
When we warn about a preempt_count leak; reset the preempt_count to the known good value such that the problem does not ripple forward.
--- kernel/exit.c | 4 +++- kernel/sched/core.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
kernel/exit.c +3 -1
@@ -706,10 +706,12 @@ void do_exit(long code)smp_mb ();raw_spin_unlock_wait (&tsk->pi_lock); -if (unlikely (in_atomic ())) +if (unlikely (in_atomic ())) {pr_info ("note: %s[%d] exited with preempt_count %d\n" , current->comm,task_pid_nr (current),preempt_count ()); +preempt_count_set (PREEMPT_ENABLED ); + }/* sync mm's RSS info before statistics gathering */ if (tsk->mm)
kernel/sched/core.c +3 -1
@@ -2953,8 +2953,10 @@ static inline void schedule_debug(struct task_struct *prev)* schedule() atomically, we ignore that path. Otherwise whine * if we are scheduling when we should not. */ -if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) +if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) {__schedule_bug (prev); +preempt_count_set (PREEMPT_DISABLED ); + }rcu_sleep_check ();profile_hit (SCHED_PROFILING ,__builtin_return_address (0 ));
Re: [RFC][PATCH 03/11] sched: Robustify preemption leak checks
In reply to Peter ZijlstraQuoted 2 lines · Peter Zijlstra
When we warn about a
preempt_countleak; reset thepreempt_countto the known good value such that the problem does not ripple forward.
Re: [RFC][PATCH 03/11] sched: Robustify preemption leak checks
In reply to Peter ZijlstraQuoted 28 lines · Peter Zijlstra
When we warn about a
preempt_countleak; reset thepreempt_countto the known good value such that the problem does not ripple forward.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- kernel/exit.c | 4 +++- kernel/sched/core.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)kernel/exit.c +3
-1@@ -706,10 +706,12 @@ void do_exit(long code)smp_mb ();raw_spin_unlock_wait (&tsk->pi_lock); -if (unlikely (in_atomic ())) +if (unlikely (in_atomic ())) {pr_info ("note: %s[%d] exited with preempt_count %d\n" , current->comm,task_pid_nr (current),preempt_count ()); +preempt_count_set (PREEMPT_ENABLED ); + }/* sync mm's RSS info before statistics gathering */ if (tsk->mm)
kernel/sched/core.c static inline void schedule_debug(struct · line 2965
Quoted 7 lines
kernel/sched/core.c +2
-1@@ -2960,8 +2960,10 @@ static inline void schedule_debug(struct* schedule() atomically, we ignore that path. Otherwise whine * if we are scheduling when we should not. */ -if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) +if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) {__schedule_bug (prev); +preempt_count_set (PREEMPT_DISABLED );
That one would be a bit fragile if we kept PREEMPT_ACTIVE, but since we are removing it...
Peter Zijlstra
maintainerRe: [RFC][PATCH 03/11] sched: Robustify preemption leak checks
Quoted 8 lines · Frederic Weisbecker
kernel/sched/core.c static inline void schedule_debug(struct · line 2965
Quoted 3 lines · Peter Zijlstra
kernel/sched/core.c +2
-0+if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) {__schedule_bug (prev); +preempt_count_set (PREEMPT_DISABLED );That one would be a bit fragile if we kept
PREEMPT_ACTIVE, but since we are removing it...
Right, let me move this patch after 7 (the one that stops setting PREEMPT_ACTIVE).
Re: [RFC][PATCH 03/11] sched: Robustify preemption leak checks
Quoted 10 lines · Peter Zijlstra
Quoted 8 lines · Frederic Weisbecker
kernel/sched/core.c static inline void schedule_debug(struct · line 2965
Quoted 3 lines · Peter Zijlstra
kernel/sched/core.c +2
-0+if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) {__schedule_bug (prev); +preempt_count_set (PREEMPT_DISABLED );That one would be a bit fragile if we kept
PREEMPT_ACTIVE, but since we are removing it...
Quoted 2 lines
Right, let me move this patch after 7 (the one that stops setting
PREEMPT_ACTIVE).
Perfect!
Steven Rostedt
maintainerRe: [RFC][PATCH 03/11] sched: Robustify preemption leak checks
In reply to Peter ZijlstraQuoted 12 lines · On Tue, 29 Sep 2015 11:28:28 +0200 Peter Zijlstra <[email protected]> wrote:
When we warn about a
preempt_countleak; reset thepreempt_countto the known good value such that the problem does not ripple forward.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- kernel/exit.c | 4 +++- kernel/sched/core.c | 4 +++- 2 files changed, 6 insertions(+), 2 deletions(-)
kernel/exit.c void do_exit(long code) · line 714
Quoted 10 lines
kernel/exit.c +3
-1@@ -706,10 +706,12 @@ void do_exit(long code)smp_mb ();raw_spin_unlock_wait (&tsk->pi_lock); -if (unlikely (in_atomic ())) +if (unlikely (in_atomic ())) {pr_info ("note: %s[%d] exited with preempt_count %d\n" , current->comm,task_pid_nr (current),preempt_count ()); +preempt_count_set (PREEMPT_ENABLED ); + }
Looks good.
Quoted 6 lines
kernel/exit.c +0
-0/* sync mm's RSS info before statistics gathering */ if (tsk->mm)
kernel/sched/core.c static inline void schedule_debug(struct · line 2966
Quoted 8 lines
kernel/sched/core.c +3
-1@@ -2960,8 +2960,10 @@ static inline void schedule_debug(struct* schedule() atomically, we ignore that path. Otherwise whine * if we are scheduling when we should not. */ -if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) +if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) {__schedule_bug (prev); +preempt_count_set (PREEMPT_DISABLED ); + }
Of course, if this was not a preemption leak, but something that called schedule within a preempt_disable()/preempt_enable() section, when it returns, preemption will be enabled, right?
-- Steve
Quoted 4 lines
kernel/sched/core.c +0
-0rcu_sleep_check ();profile_hit (SCHED_PROFILING ,__builtin_return_address (0 ));
Peter Zijlstra
maintainerRe: [RFC][PATCH 03/11] sched: Robustify preemption leak checks
Quoted 18 lines · Steven Rostedt
kernel/sched/core.c static inline void schedule_debug(struct · line 2966
Quoted 11 lines · On Tue, 29 Sep 2015 11:28:28 +0200 Peter Zijlstra <[email protected]> wrote:
kernel/sched/core.c +3
-1@@ -2960,8 +2960,10 @@ static inline void schedule_debug(struct* schedule() atomically, we ignore that path. Otherwise whine * if we are scheduling when we should not. */ -if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) +if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) {__schedule_bug (prev); +preempt_count_set (PREEMPT_DISABLED ); + }Of course, if this was not a preemption leak, but something that called schedule within a preempt_disable()/preempt_enable() section, when it returns, preemption will be enabled, right?
Indeed.. But it ensures only the task that incorrectly called schedule() gets screwed and not everybody else.
This is most important on x86 which has a per cpu preempt_count that is not saved/restored (after this series). So if you schedule with an invalid (!2*PREEMPT_DISABLE_OFFSET) preempt_count the next task is messed up too.
Enforcing this invariant limits the borkage to just the one task.
Peter Zijlstra
maintainerRFC PATCH 4/11 sched: Rework TASK_DEAD preemption exception
In reply to Peter ZijlstraTASK_DEAD is special in that the final schedule call from do_exit() must be done with preemption disabled.
This leads to a violation of our new scheduling invariant which states that the preempt count should be 2. Move the check for TASK_DEAD out of the debug check and use it to decrement the preempt count (from 2 to 1).
--- kernel/sched/core.c | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-)
kernel/sched/core.c +13 -6
@@ -2955,12 +2955,8 @@ static inline void schedule_debug(struct#ifdef CONFIG_SCHED_STACK_END_CHECK BUG_ON (unlikely (task_stack_end_corrupted (prev)));#endif -/* -* Test if we are atomic. Since do_exit() needs to call into -* schedule() atomically, we ignore that path. Otherwise whine -* if we are scheduling when we should not. -*/ -if (unlikely (in_atomic_preempt_off () && prev->state !=TASK_DEAD )) { + +if (unlikely (in_atomic_preempt_off ())) {__schedule_bug (prev);preempt_count_set (PREEMPT_DISABLED ); }
@@ -3061,6 +3057,17 @@ static void __sched __schedule(void)rcu_note_context_switch (); prev = rq->curr; +/* +* do_exit() calls schedule() with preemption disabled as an exception; +* however we must fix that up, otherwise the next task will see an +* inconsistent preempt count. +* +* It also avoids the below schedule_debug() test from complaining +* about this. +*/ +if (unlikely (prev->state ==TASK_DEAD )) +preempt_enable_no_resched_notrace (); +schedule_debug (prev);if (sched_feat (HRTICK ))
Re: [RFC][PATCH 04/11] sched: Rework TASK_DEAD preemption exception
Quoted 7 lines · Peter Zijlstra
TASK_DEADis special in that the final schedule call fromdo_exit()must be done with preemption disabled.This leads to a violation of our new scheduling invariant which states that the preempt count should be 2. Move the check for
TASK_DEADout of the debug check and use it to decrement the preempt count (from 2 to 1).
Re: [RFC][PATCH 04/11] sched: Rework TASK_DEAD preemption exception
In reply to Peter ZijlstraQuoted 9 lines · Peter Zijlstra
TASK_DEADis special in that the final schedule call fromdo_exit()must be done with preemption disabled.This leads to a violation of our new scheduling invariant which states that the preempt count should be 2. Move the check for
TASK_DEADout of the debug check and use it to decrement the preempt count (from 2 to 1).Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Re: [RFC][PATCH 04/11] sched: Rework TASK_DEAD preemption exception
In reply to Peter ZijlstraQuoted 9 lines · On Tue, 29 Sep 2015 11:28:29 +0200 Peter Zijlstra <[email protected]> wrote:
TASK_DEADis special in that the final schedule call fromdo_exit()must be done with preemption disabled.This leads to a violation of our new scheduling invariant which states that the preempt count should be 2. Move the check for
TASK_DEADout of the debug check and use it to decrement the preempt count (from 2 to 1).Signed-off-by Peter Zijlstra (Intel) <[email protected]>
-- Steve
Peter Zijlstra
maintainerRFC PATCH 5/11 sched: Add preempt argument to __schedule()
In reply to Peter ZijlstraThere is only a single PREEMPT_ACTIVE use in the regular __schedule() path and that is to circumvent the task->state check. Since the code setting PREEMPT_ACTIVE is the immediate caller of __schedule() we can replace this with a function argument.
--- kernel/sched/core.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
kernel/sched/core.c +6 -6
@@ -3045,7 +3045,7 @@ pick_next_task(struct rq *rq, struct tas* * WARNING: must be called with preemption disabled! */ -static void __sched __schedule(void) +static void __sched __schedule(bool preempt) {struct task_struct *prev, *next;unsigned long *switch_count;
@@ -3085,7 +3085,7 @@ static void __sched __schedule(void) rq->clock_skip_update <<=1 ;/* promote REQ to ACT */ switch_count = &prev->nivcsw; -if (prev->state && !(preempt_count () &PREEMPT_ACTIVE )) { +if (!preempt && prev->state) {if (unlikely (signal_pending_state (prev->state, prev))) { prev->state =TASK_RUNNING ; }else {
@@ -3150,7 +3150,7 @@ asmlinkage __visible void __sched schedusched_submit_work (tsk);do {preempt_disable (); -__schedule (); +__schedule (false);sched_preempt_enable_no_resched (); }while (need_resched ()); }
@@ -3191,7 +3191,7 @@ static void __sched notrace preempt_sche { do { preempt_active_enter(); - __schedule(); + __schedule(true); preempt_active_exit(); /*
@@ -3256,7 +3256,7 @@ asmlinkage __visible void __sched notrac* an infinite recursion. */ prev_ctx =exception_enter (); -__schedule (); +__schedule (true);exception_exit (prev_ctx);barrier ();
@@ -3285,7 +3285,7 @@ asmlinkage __visible void __sched preempdo {preempt_active_enter ();local_irq_enable (); -__schedule (); +__schedule (true);local_irq_disable ();preempt_active_exit (); }while (need_resched ());
Re: [RFC][PATCH 05/11] sched: Add preempt argument to __schedule()
Quoted 4 lines · Peter Zijlstra
There is only a single
PREEMPT_ACTIVEuse in the regular__schedule()path and that is to circumvent thetask->statecheck. Since the code settingPREEMPT_ACTIVEis the immediate caller of__schedule()we can replace this with a function argument.
Re: [RFC][PATCH 05/11] sched: Add preempt argument to __schedule()
In reply to Peter ZijlstraQuoted 6 lines · Peter Zijlstra
There is only a single
PREEMPT_ACTIVEuse in the regular__schedule()path and that is to circumvent thetask->statecheck. Since the code settingPREEMPT_ACTIVEis the immediate caller of__schedule()we can replace this with a function argument.Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Re: [RFC][PATCH 05/11] sched: Add preempt argument to __schedule()
In reply to Peter ZijlstraQuoted 7 lines · On Tue, 29 Sep 2015 11:28:30 +0200 Peter Zijlstra <[email protected]> wrote:
There is only a single
PREEMPT_ACTIVEuse in the regular__schedule()path and that is to circumvent thetask->statecheck. Since the code settingPREEMPT_ACTIVEis the immediate caller of__schedule()we can replace this with a function argument.Signed-off-by Peter Zijlstra (Intel) <[email protected]>---
-- Steve
Steven Rostedt
maintainerRe: [RFC][PATCH 05/11] sched: Add preempt argument to __schedule()
Quoted 8 lines · On Tue, 29 Sep 2015 11:28:39 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 5 lines · On Tue, 29 Sep 2015 11:28:30 +0200 Peter Zijlstra <[email protected]> wrote:
There is only a single
PREEMPT_ACTIVEuse in the regular__schedule()path and that is to circumvent thetask->statecheck. Since the code settingPREEMPT_ACTIVEis the immediate caller of__schedule()we can replace this with a function argument.
Quoted 4 lines
Quoted 2 lines
Signed-off-by Peter Zijlstra (Intel) <[email protected]>---
Reviewed-by Steven Rostedt <[email protected]>
I just want to note that this scared me at first, because __schedule() can be traced by the function tracer that can also do a preempt_enable() which would schedule if preemption was enabled. But it looks as __schedule() is always called with preemption disabled, thus, no worries.
-- Steve
Peter Zijlstra
maintainerRe: [RFC][PATCH 05/11] sched: Add preempt argument to __schedule()
Quoted 11 lines · Steven Rostedt
Quoted 8 lines · On Tue, 29 Sep 2015 11:28:39 -0400 Steven Rostedt <[email protected]> wrote:
Quoted 5 lines · On Tue, 29 Sep 2015 11:28:30 +0200 Peter Zijlstra <[email protected]> wrote:
There is only a single
PREEMPT_ACTIVEuse in the regular__schedule()path and that is to circumvent thetask->statecheck. Since the code settingPREEMPT_ACTIVEis the immediate caller of__schedule()we can replace this with a function argument.
Quoted 7 lines
Quoted 4 lines
Quoted 2 lines
Signed-off-by Peter Zijlstra (Intel) <[email protected]>---
Reviewed-by Steven Rostedt <[email protected]>I just want to note that this scared me at first, because
__schedule()can be traced by the function tracer that can also do a
See 11/11 :-)
Peter Zijlstra
maintainerRFC PATCH 6/11 sched: Fix trace_sched_switch()
In reply to Peter Zijlstra__trace_sched_switch_state() is the last remaining PREEMPT_ACTIVE user, move trace_sched_switch() from prepare_task_switch() to __schedule() and propagate the @preempt argument.
--- include/trace/events/sched.h | 22 +++++++++------------- kernel/sched/core.c | 2 +- kernel/trace/ftrace.c | 2 +- kernel/trace/trace_sched_switch.c | 3 ++- kernel/trace/trace_sched_wakeup.c | 2 +- 5 files changed, 14 insertions(+), 17 deletions(-)
include/trace/events/sched.h +9 -13
@@ -104,22 +104,17 @@ DEFINE_EVENT(sched_wakeup_template, sche TP_ARGS(p));#ifdef CREATE_TRACE_POINTS -static inline long __trace_sched_switch_state (struct task_struct *p) +static inline long __trace_sched_switch_state (bool preempt,struct task_struct *p) { -long state = p->state; - -#ifdef CONFIG_PREEMPT #ifdef CONFIG_SCHED_DEBUG BUG_ON (p != current);#endif /* CONFIG_SCHED_DEBUG */ +/* -* For all intents and purposes a preempted task is a running task. +* Preemption ignores task state, therefore preempted tasks are always +* RUNNING (we will not have dequeued if state != RUNNING). */ -if (preempt_count () &PREEMPT_ACTIVE ) - state =TASK_RUNNING |TASK_STATE_MAX ; -#endif /* CONFIG_PREEMPT */ - -return state; +return preempt ?TASK_RUNNING |TASK_STATE_MAX : p->state; }#endif /* CREATE_TRACE_POINTS */
@@ -128,10 +123,11 @@ static inline long __trace_sched_switch_*/ TRACE_EVENT(sched_switch, - TP_PROTO(struct task_struct *prev, + TP_PROTO(bool preempt, + struct task_struct *prev, struct task_struct *next), - TP_ARGS(prev, next), + TP_ARGS(preempt, prev, next), TP_STRUCT__entry( __array( char, prev_comm, TASK_COMM_LEN )
@@ -147,7 +143,7 @@ TRACE_EVENT(sched_switch,memcpy (__entry->next_comm, next->comm,TASK_COMM_LEN ); __entry->prev_pid = prev->pid; __entry->prev_prio = prev->prio; - __entry->prev_state =__trace_sched_switch_state (prev); + __entry->prev_state =__trace_sched_switch_state (preempt, prev);memcpy (__entry->prev_comm, prev->comm,TASK_COMM_LEN ); __entry->next_pid = next->pid; __entry->next_prio = next->prio;
kernel/sched/core.c +1 -1
@@ -2470,7 +2470,6 @@ static inline void prepare_task_switch(struct rq *rq, struct task_struct *prev, struct task_struct *next) { -trace_sched_switch (prev, next);sched_info_switch (rq, prev, next);perf_event_task_sched_out (prev, next);fire_sched_out_preempt_notifiers (prev, next);
@@ -3121,6 +3120,7 @@ static void __sched __schedule(bool pree rq->curr = next; ++*switch_count; +trace_sched_switch (preempt, prev, next); rq =context_switch (rq, prev, next);/* unlocks the rq */ cpu =cpu_of (rq); }else {
kernel/trace/ftrace.c +1 -1
@@ -5697,7 +5697,7 @@ static int alloc_retstack_tasklist(struc }static void -ftrace_graph_probe_sched_switch (void *ignore, +ftrace_graph_probe_sched_switch (void *ignore,bool preempt,struct task_struct *prev,struct task_struct *next) {unsigned long long timestamp;
kernel/trace/trace_sched_switch.c +2 -1
@@ -16,7 +16,8 @@ static int sched_ref;static DEFINE_MUTEX (sched_register_mutex);static void -probe_sched_switch (void *ignore,struct task_struct *prev,struct task_struct *next) +probe_sched_switch (void *ignore,bool preempt, +struct task_struct *prev,struct task_struct *next) {if (unlikely (!sched_ref))return ;
kernel/trace/trace_sched_wakeup.c +1 -1
@@ -420,7 +420,7 @@ tracing_sched_wakeup_trace(struct trace_ } static void notrace -probe_wakeup_sched_switch (void *ignore, +probe_wakeup_sched_switch (void *ignore,bool preempt,struct task_struct *prev,struct task_struct *next) {struct trace_array_cpu *data;
Re: [RFC][PATCH 06/11] sched: Fix trace_sched_switch()
Quoted 3 lines · Peter Zijlstra
__trace_sched_switch_state()is the last remainingPREEMPT_ACTIVEuser, movetrace_sched_switch()fromprepare_task_switch()to__schedule()and propagate the @preempt argument.
Re: [RFC][PATCH 06/11] sched: Fix trace_sched_switch()
In reply to Peter ZijlstraQuoted 31 lines · On Tue, 29 Sep 2015 11:28:31 +0200 Peter Zijlstra <[email protected]> wrote:
__trace_sched_switch_state()is the last remainingPREEMPT_ACTIVEuser, movetrace_sched_switch()fromprepare_task_switch()to__schedule()and propagate the @preempt argument.Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- include/trace/events/sched.h | 22 +++++++++------------- kernel/sched/core.c | 2 +- kernel/trace/ftrace.c | 2 +- kernel/trace/trace_sched_switch.c | 3 ++- kernel/trace/trace_sched_wakeup.c | 2 +- 5 files changed, 14 insertions(+), 17 deletions(-)include/trace/events/sched.h +2
-5@@ -104,22 +104,17 @@ DEFINE_EVENT(sched_wakeup_template, sche TP_ARGS(p));#ifdef CREATE_TRACE_POINTS -static inline long __trace_sched_switch_state (struct task_struct *p) +static inline long __trace_sched_switch_state (bool preempt,struct task_struct *p) { -long state = p->state; - -#ifdef CONFIG_PREEMPT #ifdef CONFIG_SCHED_DEBUG BUG_ON (p != current);#endif /* CONFIG_SCHED_DEBUG */ +/* -* For all intents and purposes a preempted task is a running task.
include/trace/events/sched.h DEFINE_EVENT(sched_wakeup_template, sche · line 118
Quoted 10 lines
include/trace/events/sched.h +3
-5+* Preemption ignores task state, therefore preempted tasks are always +* RUNNING (we will not have dequeued if state != RUNNING). */ -if (preempt_count () &PREEMPT_ACTIVE ) - state =TASK_RUNNING |TASK_STATE_MAX ; -#endif /* CONFIG_PREEMPT */ - -return state; +return preempt ?TASK_RUNNING |TASK_STATE_MAX : p->state; }
Hmm, this original change screwed up kernelshark, as it used the state to determine if something was preempted or not. Because now you always show a task as running, it can't do that anymore. I think I bitched about this before.
What about nuking the above and just export to the sched_switch tracepoint the fact that it was preempted. We now have that information passed to it.
As everything should be using the parsing files, it should not break any tools to export it.
As for this patch and with the current state, I see nothing wrong with it.
-- Steve
Peter Zijlstra
maintainerRe: [RFC][PATCH 06/11] sched: Fix trace_sched_switch()
Quoted 13 lines · Steven Rostedt
include/trace/events/sched.h DEFINE_EVENT(sched_wakeup_template, sche · line 118
Quoted 4 lines · On Tue, 29 Sep 2015 11:28:31 +0200 Peter Zijlstra <[email protected]> wrote:
include/trace/events/sched.h +2
-0+static inline long __trace_sched_switch_state (bool preempt,struct task_struct *p) { +return preempt ?TASK_RUNNING |TASK_STATE_MAX : p->state; }Hmm, this original change screwed up kernelshark, as it used the state to determine if something was preempted or not. Because now you always show a task as running, it can't do that anymore. I think I bitched about this before.
Quoted 6 lines
What about nuking the above and just export to the
sched_switchtracepoint the fact that it was preempted. We now have that information passed to it.As everything should be using the parsing files, it should not break any tools to export it.
/SHOULD/ being the operative word. Experience has taught me that changing the sched tracepoint leads to borkage.
But we can sure try, see if someone notices etc.. Same with trace_sched_wakeup(), that still prints a dummy value.
Steven Rostedt
maintainerRe: [RFC][PATCH 06/11] sched: Fix trace_sched_switch()
Quoted 2 lines · On Tue, 29 Sep 2015 17:48:28 +0200 Peter Zijlstra <[email protected]> wrote:
/SHOULD/ being the operative word. Experience has taught me that changing the sched tracepoint leads to borkage.
Right. But experience also tells us that those relying on offsets will get brokage if they run 32 bit userspace on a 64 bit kernel too.
Quoted 3 lines
But we can sure try, see if someone notices etc.. Same with
trace_sched_wakeup(), that still prints a dummy value.
Removing a field can break things that even use the parsing library. That's why we still have that.
-- Steve
Peter Zijlstra
maintainerRFC PATCH 7/11 sched: Stop setting PREEMPT_ACTIVE
In reply to Peter ZijlstraNow that nothing tests for PREEMPT_ACTIVE anymore, stop setting it.
--- include/linux/preempt.h | 12 ------------ kernel/sched/core.c | 19 ++++++------------- 2 files changed, 6 insertions(+), 25 deletions(-)
include/linux/preempt.h +0 -12
@@ -146,18 +146,6 @@ extern void preempt_count_sub(int val);#define preempt_count_inc () preempt_count_add(1)#define preempt_count_dec () preempt_count_sub(1) -#define preempt_active_enter () \ -do { \ - preempt_count_add(PREEMPT_ACTIVE + PREEMPT_DISABLE_OFFSET); \ - barrier(); \ -} while (0) - -#define preempt_active_exit () \ -do { \ - barrier(); \ - preempt_count_sub(PREEMPT_ACTIVE + PREEMPT_DISABLE_OFFSET); \ -} while (0) -#ifdef CONFIG_PREEMPT_COUNT #define preempt_disable () \
kernel/sched/core.c +6 -13
@@ -3190,9 +3190,9 @@ void __sched schedule_preempt_disabled(v static void __sched notrace preempt_schedule_common(void) { do { - preempt_active_enter(); + preempt_disable(); __schedule(true); - preempt_active_exit(); + sched_preempt_enable_no_resched(); /* * Check again in case we missed a preemption opportunity
@@ -3243,13 +3243,7 @@ asmlinkage __visible void __sched notracreturn ; do { - /* - * Use raw __prempt_count() ops that don't call function. - * We can't call functions before disabling preemption which - * disarm preemption tracing recursions. - */ - __preempt_count_add(PREEMPT_ACTIVE + PREEMPT_DISABLE_OFFSET); - barrier(); + preempt_disable_notrace(); /* * Needs preempt disabled in case user_exit() is traced * and the tracer calls preempt_enable_notrace() causing
@@ -3259,8 +3253,7 @@ asmlinkage __visible void __sched notrac__schedule (true);exception_exit (prev_ctx); -barrier (); -__preempt_count_sub (PREEMPT_ACTIVE +PREEMPT_DISABLE_OFFSET ); +preempt_enable_no_resched_notrace (); }while (need_resched ()); }EXPORT_SYMBOL_GPL (preempt_schedule_notrace);
@@ -3283,11 +3276,11 @@ asmlinkage __visible void __sched preemp prev_state =exception_enter ();do { -preempt_active_enter (); +preempt_disable ();local_irq_enable ();__schedule (true);local_irq_disable (); -preempt_active_exit (); +sched_preempt_enable_no_resched (); }while (need_resched ());exception_exit (prev_state);
Re: [RFC][PATCH 07/11] sched: Stop setting PREEMPT_ACTIVE
Quoted 1 line · Peter Zijlstra
Now that nothing tests for
PREEMPT_ACTIVEanymore, stop setting it.
Good riddance.
Re: [RFC][PATCH 07/11] sched: Stop setting PREEMPT_ACTIVE
In reply to Peter ZijlstraQuoted 21 lines · On Tue, 29 Sep 2015 11:28:32 +0200 Peter Zijlstra <[email protected]> wrote:
Now that nothing tests for
PREEMPT_ACTIVEanymore, stop setting it.Signed-off-by Peter Zijlstra (Intel) <[email protected]>kernel/sched/core.c +1
-7@@ -3243,13 +3243,7 @@ asmlinkage __visible void __sched notracreturn ; do { - /* - * Use raw __prempt_count() ops that don't call function. - * We can't call functions before disabling preemption which - * disarm preemption tracing recursions. - */ - __preempt_count_add(PREEMPT_ACTIVE + PREEMPT_DISABLE_OFFSET); - barrier(); + preempt_disable_notrace(); /* * Needs preempt disabled in case user_exit() is traced * and the tracer calls preempt_enable_notrace() causing
kernel/sched/core.c asmlinkage __visible void __sched notrac · line 3256
Quoted 6 lines
kernel/sched/core.c +1
-2@@ -3259,8 +3253,7 @@ asmlinkage __visible void __sched notrac__schedule (true);exception_exit (prev_ctx); -barrier (); -__preempt_count_sub (PREEMPT_ACTIVE +PREEMPT_DISABLE_OFFSET ); +preempt_enable_no_resched_notrace ();
Nice, you used the notrace variants for this function.
-- Steve
Quoted 2 lines
kernel/sched/core.c +0
-0}while (need_resched ()); }
Peter Zijlstra
maintainerRe: [RFC][PATCH 07/11] sched: Stop setting PREEMPT_ACTIVE
Quoted 27 lines · Steven Rostedt
Quoted 24 lines · On Tue, 29 Sep 2015 11:28:32 +0200 Peter Zijlstra <[email protected]> wrote:
Now that nothing tests for
PREEMPT_ACTIVEanymore, stop setting it.Signed-off-by Peter Zijlstra (Intel) <[email protected]>kernel/sched/core.c +1
-7@@ -3243,13 +3243,7 @@ asmlinkage __visible void __sched notracreturn ; do { - /* - * Use raw __prempt_count() ops that don't call function. - * We can't call functions before disabling preemption which - * disarm preemption tracing recursions. - */ - __preempt_count_add(PREEMPT_ACTIVE + PREEMPT_DISABLE_OFFSET); - barrier(); + preempt_disable_notrace(); /* * Needs preempt disabled in case user_exit() is traced * and the tracer calls preempt_enable_notrace() causing@@ -3259,8 +3253,7 @@ asmlinkage __visible void __sched notrac__schedule (true);exception_exit (prev_ctx);
Quoted 5 lines
kernel/sched/core.c asmlinkage __visible void __sched notrac · line 3256
Quoted 3 lines
kernel/sched/core.c +1
-2-barrier (); -__preempt_count_sub (PREEMPT_ACTIVE +PREEMPT_DISABLE_OFFSET ); +preempt_enable_no_resched_notrace ();Nice, you used the notrace variants for this function.
There was a comment stating this was important, and its a notrace annotated function :-)
Peter Zijlstra
maintainerRFC PATCH 8/11 sched: Simplify preempt_count tests
In reply to Peter ZijlstraSince we stopped setting PREEMPT_ACTIVE, there is no need to mask it out of preempt_count() tests.
--- include/linux/preempt.h | 3 +-- kernel/sched/core.c | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-)
include/linux/preempt.h +1 -2
@@ -126,8 +126,7 @@* Check whether we were atomic before we did preempt_disable(): * (used by the scheduler) */ -#define in_atomic_preempt_off () \ - ((preempt_count() & ~PREEMPT_ACTIVE) != PREEMPT_DISABLE_OFFSET) +#define in_atomic_preempt_off () (preempt_count() != PREEMPT_DISABLE_OFFSET)#if defined(CONFIG_DEBUG_PREEMPT ) || defined(CONFIG_PREEMPT_TRACER )extern void preempt_count_add (int val);
kernel/sched/core.c +1 -1
@@ -7472,7 +7472,7 @@ void __init sched_init(void)#ifdef CONFIG_DEBUG_ATOMIC_SLEEP static inline int preempt_count_equals (int preempt_offset) { -int nested = (preempt_count () & ~PREEMPT_ACTIVE ) +rcu_preempt_depth (); +int nested =preempt_count () +rcu_preempt_depth ();return (nested == preempt_offset); }
Re: [RFC][PATCH 08/11] sched: Simplify preempt_count tests
Quoted 2 lines · Peter Zijlstra
Since we stopped setting
PREEMPT_ACTIVE, there is no need to mask it out ofpreempt_count()tests.
Re: [RFC][PATCH 08/11] sched: Simplify preempt_count tests
In reply to Peter ZijlstraQuoted 5 lines · On Tue, 29 Sep 2015 11:28:33 +0200 Peter Zijlstra <[email protected]> wrote:
Since we stopped setting
PREEMPT_ACTIVE, there is no need to mask it out ofpreempt_count()tests.Signed-off-by Peter Zijlstra (Intel) <[email protected]>---
-- Steve
Quoted 28 lines
include/linux/preempt.h | 3 +-- kernel/sched/core.c | 2 +- 2 files changed, 2 insertions(+), 3 deletions(-)include/linux/preempt.h +1
-2@@ -126,8 +126,7 @@* Check whether we were atomic before we did preempt_disable(): * (used by the scheduler) */ -#define in_atomic_preempt_off () \ - ((preempt_count() & ~PREEMPT_ACTIVE) != PREEMPT_DISABLE_OFFSET) +#define in_atomic_preempt_off () (preempt_count() != PREEMPT_DISABLE_OFFSET)#if defined(CONFIG_DEBUG_PREEMPT ) || defined(CONFIG_PREEMPT_TRACER )extern void preempt_count_add (int val);kernel/sched/core.c +1
-1@@ -7472,7 +7472,7 @@ void __init sched_init(void)#ifdef CONFIG_DEBUG_ATOMIC_SLEEP static inline int preempt_count_equals (int preempt_offset) { -int nested = (preempt_count () & ~PREEMPT_ACTIVE ) +rcu_preempt_depth (); +int nested =preempt_count () +rcu_preempt_depth ();return (nested == preempt_offset); }
Peter Zijlstra
maintainerRFC PATCH 9/11 sched, x86: Kill saved_preempt_count
In reply to Peter ZijlstraWith the introduction of the context switch preempt_count invariant, and the demise of PREEMPT_ACTIVE, its pointless to save/restore the per-cpu preemption count, it must always be 2.
--- arch/x86/include/asm/preempt.h | 6 +----- arch/x86/include/asm/thread_info.h | 2 -- arch/x86/kernel/process_32.c | 8 -------- arch/x86/kernel/process_64.c | 8 -------- 4 files changed, 1 insertion(+), 23 deletions(-)
arch/x86/include/asm/preempt.h +1 -5
@@ -30,13 +30,9 @@ static __always_inline void preempt_coun/* * must be macros to avoid header recursion hell */ -#define init_task_preempt_count (p) do { \ - task_thread_info(p)->saved_preempt_count = \ - 2*PREEMPT_DISABLE_OFFSET + PREEMPT_NEED_RESCHED; \ -} while (0) +#define init_task_preempt_count (p) do { } while (0)#define init_idle_preempt_count (p, cpu) do { \ - task_thread_info(p)->saved_preempt_count = PREEMPT_ENABLED; \ per_cpu(__preempt_count, (cpu)) = PREEMPT_ENABLED; \ } while (0)
arch/x86/include/asm/thread_info.h +0 -2
@@ -57,7 +57,6 @@ struct thread_info {__u32 flags;/* low level flags */ __u32 status;/* thread synchronous flags */ __u32 cpu;/* current CPU */ -int saved_preempt_count;mm_segment_t addr_limit;void __user *sysenter_return; unsigned int sig_on_uaccess_error:1;
@@ -69,7 +68,6 @@ struct thread_info { .task = &tsk, \ .flags = 0, \ .cpu = 0, \ - .saved_preempt_count = INIT_PREEMPT_COUNT, \ .addr_limit = KERNEL_DS, \ }
arch/x86/kernel/process_32.c +0 -8
@@ -280,14 +280,6 @@ __switch_to(struct task_struct *prev_p,set_iopl_mask (next->iopl);/* -* If it were not for PREEMPT_ACTIVE we could guarantee that the -* preempt_count of all tasks was equal here and this would not be -* needed. -*/ -task_thread_info (prev_p)->saved_preempt_count =this_cpu_read (__preempt_count); -this_cpu_write (__preempt_count,task_thread_info (next_p)->saved_preempt_count); - -/* * Now maybe handle debug registers and/or IO bitmaps */ if (unlikely(task_thread_info(prev_p)->flags & _TIF_WORK_CTXSW_PREV ||
arch/x86/kernel/process_64.c +0 -8
@@ -401,14 +401,6 @@ __switch_to(struct task_struct *prev_p,*/ this_cpu_write (current_task, next_p); -/* -* If it were not for PREEMPT_ACTIVE we could guarantee that the -* preempt_count of all tasks was equal here and this would not be -* needed. -*/ -task_thread_info (prev_p)->saved_preempt_count =this_cpu_read (__preempt_count); -this_cpu_write (__preempt_count,task_thread_info (next_p)->saved_preempt_count); -/* Reload esp0 and ss1. This changes current_thread_info(). */ load_sp0 (tss, next);
Re: [RFC][PATCH 09/11] sched, x86: Kill saved_preempt_count
Quoted 3 lines · Peter Zijlstra
With the introduction of the context switch
preempt_countinvariant, and the demise ofPREEMPT_ACTIVE, its pointless to save/restore the per-cpu preemption count, it must always be 2.
Steven Rostedt
maintainerRe: [RFC][PATCH 09/11] sched, x86: Kill saved_preempt_count
In reply to Peter ZijlstraQuoted 5 lines · On Tue, 29 Sep 2015 11:28:34 +0200 Peter Zijlstra <[email protected]> wrote:
With the introduction of the context switch
preempt_countinvariant, and the demise ofPREEMPT_ACTIVE, its pointless to save/restore the per-cpu preemption count, it must always be 2.Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Should we add a warn on someplace:
WARN_ON_ONCE(preempt_count() != 2)
?
-- Steve
Peter Zijlstra
maintainerRe: [RFC][PATCH 09/11] sched, x86: Kill saved_preempt_count
Quoted 7 lines · Steven Rostedt
Quoted 4 lines · On Tue, 29 Sep 2015 11:28:34 +0200 Peter Zijlstra <[email protected]> wrote:
With the introduction of the context switch
preempt_countinvariant, and the demise ofPREEMPT_ACTIVE, its pointless to save/restore the per-cpu preemption count, it must always be 2.
Quoted 5 lines
Quoted 1 line
Signed-off-by Peter Zijlstra (Intel) <[email protected]>Should we add a warn on someplace:
WARN_ON_ONCE(preempt_count() != 2)
See v2 12/11 :-)
Peter Zijlstra
maintainerRFC PATCH 10/11 sched: Kill PREEMPT_ACTIVE
In reply to Peter ZijlstraIts unused, kill the definition.
--- include/linux/preempt.h | 5 ----- 1 file changed, 5 deletions(-)
include/linux/preempt.h +0 -5
@@ -26,7 +26,6 @@* SOFTIRQ_MASK: 0x0000ff00 * HARDIRQ_MASK: 0x000f0000 * NMI_MASK: 0x00100000 -* PREEMPT_ACTIVE: 0x00200000 * PREEMPT_NEED_RESCHED: 0x80000000 */ #define PREEMPT_BITS 8
@@ -53,10 +52,6 @@#define SOFTIRQ_DISABLE_OFFSET (2 * SOFTIRQ_OFFSET) -#define PREEMPT_ACTIVE_BITS 1 -#define PREEMPT_ACTIVE_SHIFT (NMI_SHIFT + NMI_BITS) -#define PREEMPT_ACTIVE (__IRQ_MASK(PREEMPT_ACTIVE_BITS) << PREEMPT_ACTIVE_SHIFT) -/* We use the MSB mostly because its available */ #define PREEMPT_NEED_RESCHED 0x80000000
Re: [RFC][PATCH 10/11] sched: Kill PREEMPT_ACTIVE
Quoted 1 line · Peter Zijlstra
Its unused, kill the definition.
Re: [RFC][PATCH 10/11] sched: Kill PREEMPT_ACTIVE
In reply to Peter ZijlstraQuoted 3 lines · On Tue, 29 Sep 2015 11:28:35 +0200 Peter Zijlstra <[email protected]> wrote:
Its unused, kill the definition.
Signed-off-by Peter Zijlstra (Intel) <[email protected]>
-- Steve
Peter Zijlstra
maintainerRFC PATCH 11/11 sched: More notrace
In reply to Peter Zijlstrapreempt_schedule_common() is marked notrace, but it does not use _notrace() preempt_count functions and __schedule() is also not marked notrace, which means that its perfectly possible to end up in the tracer from preempt_schedule_common().
So either we need more notrace, or we need to remove notrace from preempt_schedule_common().
--- kernel/sched/core.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/sched/core.c +3 -3
@@ -3044,7 +3044,7 @@ pick_next_task(struct rq *rq, struct tas* * WARNING: must be called with preemption disabled! */ -static void __sched __schedule(bool preempt) +static void __sched notrace __schedule(bool preempt) {struct task_struct *prev, *next;unsigned long *switch_count;
@@ -3190,9 +3190,9 @@ void __sched schedule_preempt_disabled(v static void __sched notrace preempt_schedule_common(void) { do { - preempt_disable(); + preempt_disable_notrace(); __schedule(true); - sched_preempt_enable_no_resched(); + preempt_enable_no_resched_notrace(); /* * Check again in case we missed a preemption opportunity
Re: [RFC][PATCH 11/11] sched: More notrace
Quoted 7 lines · Peter Zijlstra
preempt_schedule_common()is marked notrace, but it does not use_notrace()preempt_countfunctions and__schedule()is also not marked notrace, which means that its perfectly possible to end up in the tracer frompreempt_schedule_common().So either we need more notrace, or we need to remove notrace from
preempt_schedule_common().
More notrace is the right thing to do. We have enough pointless information in tracing already.
Re: [RFC][PATCH 11/11] sched: More notrace
In reply to Peter ZijlstraQuoted 7 lines · On Tue, 29 Sep 2015 11:28:36 +0200 Peter Zijlstra <[email protected]> wrote:
preempt_schedule_common()is marked notrace, but it does not use_notrace()preempt_countfunctions and__schedule()is also not marked notrace, which means that its perfectly possible to end up in the tracer frompreempt_schedule_common().So either we need more notrace, or we need to remove notrace from
preempt_schedule_common().
Yep, there's some history to this. This was originally the issue that caused function tracing to go into infinite recursion. But now we have preempt_schedule_notrace(), which is used by the function tracer, and that function must not be traced till preemption is disabled.
Now if function tracing is running and we take an interrupt when NEED_RESCHED is set, it calls
preempt_schedule_common() (not traced)
But then that calls preempt_disable() (traced)
function tracer calls preempt_disable_notrace() followed by preempt_enable_notrace() which will see NEED_RESCHED set, and it will call preempt_schedule_notrace(), which stops the recursion, but still calls __schedule() here, and that means when we return, we call the __schedule() from preempt_schedule_common().
That said, I prefer this patch. Preemption is disabled before calling __schedule(), and we get rid of a one round recursion with the scheduler.
Feel free to add any of the above to your change log.
-- Steve
Quoted 30 lines
Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- kernel/sched/core.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)kernel/sched/core.c +3
-3@@ -3044,7 +3044,7 @@ pick_next_task(struct rq *rq, struct tas* * WARNING: must be called with preemption disabled! */ -static void __sched __schedule(bool preempt) +static void __sched notrace __schedule(bool preempt) {struct task_struct *prev, *next;unsigned long *switch_count;@@ -3190,9 +3190,9 @@ void __sched schedule_preempt_disabled(v static void __sched notrace preempt_schedule_common(void) { do { - preempt_disable(); + preempt_disable_notrace(); __schedule(true); - sched_preempt_enable_no_resched(); + preempt_enable_no_resched_notrace(); /* * Check again in case we missed a preemption opportunity
Peter Zijlstra
maintainer[RFC][PATCH 12/11] sched: Add preempt_count invariant check
In reply to Peter ZijlstraIngo requested I keep my debug check for the preempt_count invariant.
--- kernel/sched/core.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/sched/core.c +5 -0
@@ -2503,6 +2503,11 @@ static struct rq *finish_task_switch(strstruct mm_struct *mm = rq->prev_mm;long prev_state; +if (unlikely (WARN_ONCE (preempt_count () !=2 , +"corrupted preempt_count: %s/%d/0x%x\n" , + current->comm, current->pid,preempt_count ()))) +preempt_count_set (INIT_PREEMPT_COUNT ); + rq->prev_mm =NULL ;/*
Peter Zijlstra
maintainerRe: [RFC][PATCH 12/11] sched: Add preempt_count invariant check
Quoted 12 lines · Peter Zijlstra
Ingo requested I keep my debug check for the
preempt_countinvariant.Requested-by Ingo Molnar <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>--- kernel/sched/core.c | 5 +++++ 1 file changed, 5 insertions(+)
kernel/sched/core.c static struct rq *finish_task_switch(str · line 2506
Quoted 4 lines
kernel/sched/core.c +1
-0@@ -2503,6 +2503,11 @@ static struct rq *finish_task_switch(strstruct mm_struct *mm = rq->prev_mm;long prev_state; + if (unlikely(WARN_ONCE(preempt_count() != 2,
That should have: s/2/2*PREEMPT_DISABLE_OFFSET/ or something, otherwise !PREEMPT_COUNT kernels will go funny.
Quoted 7 lines
kernel/sched/core.c +4
-0+ "corrupted preempt_count: %s/%d/0x%x\n", + current->comm, current->pid, preempt_count()))) +preempt_count_set (INIT_PREEMPT_COUNT ); + rq->prev_mm =NULL ;/*
Peter Zijlstra
maintainer[RFC][PATCH v2 12/11] sched: Add preempt_count invariant check
In reply to Peter ZijlstraIngo requested I keep my debug check for the preempt_count invariant.
--- include/asm-generic/preempt.h | 2 +- include/linux/sched.h | 17 +++++++++-------- kernel/sched/core.c | 5 +++++ 3 files changed, 15 insertions(+), 9 deletions(-)
include/asm-generic/preempt.h +1 -1
@@ -24,7 +24,7 @@ static __always_inline void preempt_coun* must be macros to avoid header recursion hell */ #define init_task_preempt_count (p) do { \ - task_thread_info(p)->preempt_count = 2*PREEMPT_DISABLED; \ + task_thread_info(p)->preempt_count = INIT_PREEMPT_COUNT; \ } while (0)#define init_idle_preempt_count (p, cpu) do { \
include/linux/sched.h +9 -8
@@ -599,17 +599,18 @@ struct task_cputime_atomic { .sum_exec_runtime = ATOMIC64_INIT(0), \ } -#ifdef CONFIG_PREEMPT_COUNT -#define PREEMPT_DISABLED (1 + PREEMPT_ENABLED) -#else -#define PREEMPT_DISABLED PREEMPT_ENABLED -#endif +#define PREEMPT_DISABLED (PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)/* -* Disable preemption until the scheduler is running. -* Reset by start_kernel()->sched_init()->init_idle(). +* Initial preempt_count value; reflects the preempt_count schedule invariant +* which states that during schedule preempt_count() == 2. +* +* This also results in the kernel starting with preemption disabled until +* the scheduler is initialized, see: +* +* start_kernel()->sched_init()->init_idle(). */ -#define INIT_PREEMPT_COUNT PREEMPT_DISABLED +#define INIT_PREEMPT_COUNT (2*PREEMPT_DISABLE_OFFSET + PREEMPT_ENABLED)/** * struct thread_group_cputimer - thread group interval timer counts
kernel/sched/core.c +5 -0
@@ -2503,6 +2503,11 @@ static struct rq *finish_task_switch(strstruct mm_struct *mm = rq->prev_mm;long prev_state; +if (unlikely (WARN_ONCE (preempt_count () !=2 *PREEMPT_DISABLE_OFFSET , +"corrupted preempt_count: %s/%d/0x%x\n" , + current->comm, current->pid,preempt_count ()))) +preempt_count_set (INIT_PREEMPT_COUNT ); + rq->prev_mm =NULL ;/*
Re: [RFC][PATCH v2 12/11] sched: Add preempt_count invariant check
Quoted 3 lines · Peter Zijlstra
+ * Initial
preempt_countvalue; reflects thepreempt_countschedule invariant
+ * which states that during schedulepreempt_count()== 2.
Is this actually *true*?
preempt_count() is two only if preemption is enabled. But spinlocks do not actually update the preemption count if there is no preemption, so these kinds of checks and comments that aren't even inside #ifdef CONFIG_PREEMPT seem to be actively misleading.
I do believe that the preempt count is stable - but the actual value would seem to depend on config options.
Linus
Peter Zijlstra
maintainerRe: [RFC][PATCH v2 12/11] sched: Add preempt_count invariant check
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Peter Zijlstra
+ * Initial
preempt_countvalue; reflects thepreempt_countschedule invariant
+ * which states that during schedulepreempt_count()== 2.Is this actually *true*?
Quoted 7 lines
preempt_count()is two only if preemption is enabled. But spinlocks do not actually update the preemption count if there is no preemption, so these kinds of checks and comments that aren't even inside #ifdefCONFIG_PREEMPTseem to be actively misleading.I do believe that the preempt count is stable - but the actual value would seem to depend on config options.
Right, 2*PREEMPT_DISABLE_OFFSET is the 'right' number. That ends up being 0 for !PREEMPT_COUNT configs.
I'll update the Changelog to be more accurate on this.
Quoted 2 lines · Peter Zijlstra
This series kills
PREEMPT_ACTIVEdead, its a tad risky, but survives light testing onx86_64and seems to compile on everything else.
I've applied the first three patches because they look low risk.
Will wait a bit with the rest.
Thanks,
Ingo