sched: Better document ttwu()
Peter Zijlstra
maintainerPATCH 1/1 sched: Better document ttwu()
Dave hit the problem fixed by commit:
b6e13e85829f ("sched/core: Fix ttwu() race")
and failed to understand much of the code involved. Per his request a few comments to (hopefully) clarify things.
--- include/linux/sched.h | 12 +-- kernel/sched/core.c | 196 +++++++++++++++++++++++++++++++++++++++++++------- kernel/sched/sched.h | 10 ++ 3 files changed, 187 insertions(+), 31 deletions(-)
include/linux/sched.h +6 -6
@@ -158,24 +158,24 @@ struct task_group;* * for (;;) { * set_current_state(TASK_UNINTERRUPTIBLE); -* if (!need_sleep) -* break; +* if (CONDITION) +* break; * * schedule(); * } * __set_current_state(TASK_RUNNING); * * If the caller does not need such serialisation (because, for instance, the -* condition test and condition change and wakeup are under the same lock) then +* CONDITION test and condition change and wakeup are under the same lock) then * use __set_current_state(). * * The above is typically ordered against the wakeup, which does: * -* need_sleep = false; +* CONDITION = 1; * wake_up_state(p, TASK_UNINTERRUPTIBLE); * -* where wake_up_state() executes a full memory barrier before accessing the -* task state. +* where wake_up_state()/try_to_wake_up() executes a full memory barrier before +* accessing p->state. * * Wakeup will do: if (@state & p->state) p->state = TASK_RUNNING, that is, * once it observes the TASK_UNINTERRUPTIBLE store the waking CPU can issue a
kernel/sched/core.c +171 -25
282 lines not on this page · raw message
kernel/sched/sched.h +10 -0
@@ -1203,6 +1203,16 @@ struct rq_flags {#endif }; +/* +* Lockdep annotation that avoids accidental unlocks; it's like a +* sticky/continuous lockdep_assert_held(). +* +* This avoids code that has access to 'struct rq *rq' (basically everything in +* the scheduler) from accidentally unlocking the rq if they do not also have a +* copy of the (on-stack) 'struct rq_flags rf'. +* +* Also see Documentation/locking/lockdep-design.rst. +*/ static inline void rq_pin_lock (struct rq *rq,struct rq_flags *rf) { rf->cookie =lockdep_pin_lock (&rq->lock);
Quoted 9 lines · Peter Zijlstra
Dave hit the problem fixed by commit:
b6e13e85829f("sched/core: Fixttwu()race")and failed to understand much of the code involved. Per his request a few comments to (hopefully) clarify things.
Requested-by Dave Chinner <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>
Small extra comment below, otherwise FWIW:
Quoted 35 lines
+++
b/kernel/sched/core.ckernel/sched/core.c +30
-0@@ -79,6 +79,100 @@ __read_mostly int scheduler_running;*/ int sysctl_sched_rt_runtime =950000 ; + +/* +* Serialization rules: +* +* Lock order: +* +* p->pi_lock +* rq->lock +* hrtimer_cpu_base->lock (hrtimer_start() for bandwidth controls) +* +* rq1->lock +* rq2->lock where: rq1 < rq2 +* +* Regular state: +* +* Normal scheduling state is serialized by rq->lock. __schedule() takes the +* local CPU's rq->lock, it optionally removes the task from the runqueue and +* always looks at the local rq data structures to find the most elegible task +* to run next. +* +* Task enqueue is also under rq->lock, possibly taken from another CPU. +* Wakeups from another LLC domain might use an IPI to transfer the enqueue to +* the local CPU to avoid bouncing the runqueue state around [ see +* ttwu_queue_wakelist() ] +* +* Task wakeup, specifically wakeups that involve migration, are horribly +* complicated to avoid having to take two rq->locks. +* +* Special state: +*
kernel/sched/core.c __read_mostly int scheduler_running; · line 121
Quoted 10 lines
kernel/sched/core.c +10
-0+* System-calls and anything external will use task_rq_lock() which acquires +* both p->pi_lock and rq->lock. As a consequence the state they change is +* stable while holding either lock: +* +* - sched_setaffinity()/ +* set_cpus_allowed_ptr(): p->cpus_ptr, p->nr_cpus_allowed +* - set_user_nice(): p->se.load, p->*prio +* - __sched_setscheduler(): p->sched_class, p->policy, p->*prio, +* p->se.load, p->rt_priority, +* p->dl.dl_{runtime, deadline, period, flags, bw, density}
Uclamp stuff can also get updated in __sched_setscheduler(); see __setscheduler_uclamp(). It's only p->uclamp_req AFAICT, but I don't think there's harm in just saying p->uclamp*.
Quoted 5 lines
kernel/sched/core.c +5
-0+* - sched_setnuma(): p->numa_preferred_nid +* - sched_move_task()/ +* cpu_cgroup_fork(): p->sched_task_group +* - uclamp_update_active() p->uclamp* +*