Skip to contentSkip to conversation

sched/fair: Disable runtime_enabled on dying rq

PATCH Superseded Kirill Tkhai 4 participants 13 messages Jun 17, 2014 – Jun 23, 2014 1 message not in the archive

Subsystems (2)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): 20140617130442.29933.54945.stgit@tkhai

#1 lore raw

PATCH 1/3 sched/fair: Disable runtime_enabled on dying rq

We kill rq->rd on the CPU_DOWN_PREPARE stage:

	cpuset_cpu_inactive -> cpuset_update_active_cpus -> partition_sched_domains ->
	-> cpu_attach_domain -> rq_attach_root -> set_rq_offline

This unthrottles all throttled cfs_rqs.

But the cpu is still able to call schedule() till

	take_cpu_down->__cpu_disable()

is called from stop_machine.

This case the tasks from just unthrottled cfs_rqs are pickable in a standard scheduler way, and they are picked by dying cpu. The cfs_rqs becomes throttled again, and migrate_tasks() in migration_call skips their tasks (one more unthrottle in migrate_tasks()->CPU_DYING does not happen, because rq->rd is already NULL).

Patch sets runtime_enabled to zero. This guarantees, the runtime is not accounted, and the cfs_rqs won't exceed given cfs_rq->runtime_remaining = 1, and tasks will be pickable in migrate_tasks(). runtime_enabled is recalculated again when rq becomes online again.

CC Srikar Dronamraju <[email protected]>
CC Mike Galbraith <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Ingo Molnar <[email protected]>
---
 kernel/sched/fair.c |   22 ++++++++++++++++++++++
 1 file changed, 22 insertions(+)
kernel/sched/fair.c +22 -0
index fea7d33..f120525 100644
@@ -3775,6 +3775,19 @@ static void destroy_cfs_bandwidth(struct cfs_bandwidth *cfs_b)
 	hrtimer_cancel(&cfs_b->slack_timer);
 }
 
+static void __maybe_unused update_runtime_enabled(struct rq *rq)
+{
+	struct cfs_rq *cfs_rq;
+
+	for_each_leaf_cfs_rq(rq, cfs_rq) {
+		struct cfs_bandwidth *cfs_b = &cfs_rq->tg->cfs_bandwidth;
+
+		raw_spin_lock(&cfs_b->lock);
+		cfs_rq->runtime_enabled = cfs_b->quota != RUNTIME_INF;
+		raw_spin_unlock(&cfs_b->lock);
+	}
+}
+
 static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 {
 	struct cfs_rq *cfs_rq;
@@ -3790,6 +3803,12 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 		cfs_rq->runtime_remaining = 1;
 		if (cfs_rq_throttled(cfs_rq))
 			unthrottle_cfs_rq(cfs_rq);
+
+		/*
+		 * Offline rq is schedulable till cpu is completely disabled
+		 * in take_cpu_down(), so we prevent new cfs throttling here.
+		 */
+		cfs_rq->runtime_enabled = 0;
 	}
 }
 
@@ -3831,6 +3850,7 @@ static inline struct cfs_bandwidth *tg_cfs_bandwidth(struct task_group *tg)
 	return NULL;
 }
 static inline void destroy_cfs_bandwidth(struct cfs_bandwidth *cfs_b) {}
+static inline void update_runtime_enabled(struct rq *rq) {}
 static inline void unthrottle_offline_cfs_rqs(struct rq *rq) {}
 
 #endif /* CONFIG_CFS_BANDWIDTH */
@@ -7325,6 +7345,8 @@ void trigger_load_balance(struct rq *rq)
 static void rq_online_fair(struct rq *rq)
 {
 	update_sysctl();
+
+	update_runtime_enabled(rq);
 }
 
 static void rq_offline_fair(struct rq *rq)
maintainer
#2 lore raw

kernel/sched/fair.c static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq) · line 3811

Quoted 10 lines · Kirill Tkhai
kernel/sched/fair.c +6 -0
@@ -3790,6 +3803,12 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 		cfs_rq->runtime_remaining = 1;
 		if (cfs_rq_throttled(cfs_rq))
 			unthrottle_cfs_rq(cfs_rq);
+
+		/*
+		 * Offline rq is schedulable till cpu is completely disabled
+		 * in take_cpu_down(), so we prevent new cfs throttling here.
+		 */
+		cfs_rq->runtime_enabled = 0;

Does it make sense to clear this before calling unthrottle_cfs_rq()? Just to make sure they're in the right order..

Quoted 2 lines
(quoted diff) +0 -0
 	}
 }
#3 lore raw

В Пн, 23/06/2014 в 12:07 +0200, Peter Zijlstra пишет:

Quoted 6 lines · Kirill Tkhai
Quoted 5 lines
kernel/sched/fair.c +1 -0
@@ -3790,6 +3803,12 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 		cfs_rq->runtime_remaining = 1;
 		if (cfs_rq_throttled(cfs_rq))
 			unthrottle_cfs_rq(cfs_rq);
+
Quoted 8 lines

kernel/sched/fair.c static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq) · line 3811

Quoted 5 lines
kernel/sched/fair.c +5 -0
+		/*
+		 * Offline rq is schedulable till cpu is completely disabled
+		 * in take_cpu_down(), so we prevent new cfs throttling here.
+		 */
+		cfs_rq->runtime_enabled = 0;

Does it make sense to clear this before calling unthrottle_cfs_rq()? Just to make sure they're in the right order..

This looks good for me. I'll test and resend.

Quoted 3 lines
Quoted 3 lines
(quoted diff) +0 -0
 	}
 }
#4 lore raw
In reply to Peter Zijlstra
Quoted 14 lines · Peter Zijlstra

kernel/sched/fair.c static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq) · line 3811

Quoted 10 lines · Kirill Tkhai
kernel/sched/fair.c +6 -0
@@ -3790,6 +3803,12 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 		cfs_rq->runtime_remaining = 1;
 		if (cfs_rq_throttled(cfs_rq))
 			unthrottle_cfs_rq(cfs_rq);
+
+		/*
+		 * Offline rq is schedulable till cpu is completely disabled
+		 * in take_cpu_down(), so we prevent new cfs throttling here.
+		 */
+		cfs_rq->runtime_enabled = 0;

Does it make sense to clear this before calling unthrottle_cfs_rq()? Just to make sure they're in the right order..

I believe that order is irrelevant here - I do not believe that unthrottle_cfs_rq(a) can cause a throttle_cfs_rq(a). In fact, I don't see any code that will check it at all from unthrottle, although I might be missing something. It _can_ cause a throttle_cfs_rq(parent_cfs_rq(a)), but that should be fine as long as for_each_leaf_cfs_rq is sorted correctly.

That said, migrate_tasks drops rq->lock, and I /think/ another cpu could wake another task onto this cpu, which could then enqueue_throttle its cfs_rq (which previously had no tasks and thus wasn't on leaf_cfs_rq_list). You certainly could have tg_set_bandwidth come in and turn runtime_enabled on.

I think the general idea of turning runtime_enabled off during offline is fine, ccing pjt to double check.

#5 lore raw
Quoted 17 lines · On 23.06.2014 21:29, [email protected] wrote:
Quoted 14 lines · Peter Zijlstra

kernel/sched/fair.c static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq) · line 3811

Quoted 10 lines · Kirill Tkhai
kernel/sched/fair.c +6 -0
@@ -3790,6 +3803,12 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 		cfs_rq->runtime_remaining = 1;
 		if (cfs_rq_throttled(cfs_rq))
 			unthrottle_cfs_rq(cfs_rq);
+
+		/*
+		 * Offline rq is schedulable till cpu is completely disabled
+		 * in take_cpu_down(), so we prevent new cfs throttling here.
+		 */
+		cfs_rq->runtime_enabled = 0;

Does it make sense to clear this before calling unthrottle_cfs_rq()? Just to make sure they're in the right order..

Quoted 6 lines

I believe that order is irrelevant here - I do not believe that unthrottle_cfs_rq(a) can cause a throttle_cfs_rq(a). In fact, I don't see any code that will check it at all from unthrottle, although I might be missing something. It _can_ cause a throttle_cfs_rq(parent_cfs_rq(a)), but that should be fine as long as for_each_leaf_cfs_rq is sorted correctly.

I think this is correct. We may change the order just for the hope, that anybody will work on it in some way in the future, and this person could skip this opaque place. Ok, I don't know how is better :)

Quoted 5 lines

That said, migrate_tasks drops rq->lock, and I /think/ another cpu could wake another task onto this cpu, which could then enqueue_throttle its cfs_rq (which previously had no tasks and thus wasn't on leaf_cfs_rq_list). You certainly could have tg_set_bandwidth come in and turn runtime_enabled on.

We mask cpu inactive on CPU_DOWN_PREPARE stage (in sched_cpu_inactive). Other cpu is not able to wake a task there after that.

rq is masked offline in cpuset_cpu_inactive() (during the same stage). But priority of sched_cpu_inactive() is higher than priority of cpuset_cpu_inactive().

CPU_PRI_SCHED_INACTIVE = INT_MIN + 1,
CPU_PRI_CPUSET_INACTIVE = INT_MIN,

This guarantees that nobody could use dying_cpu even before we start unthrottling. Another cpu will see dying_cpu is inactive.

So, it looks like we are free of this problem.

Quoted 2 lines

I think the general idea of turning runtime_enabled off during offline is fine, ccing pjt to double check.

Thanks,
Kirill

#6 lore raw
Quoted 42 lines · Kirill Tkhai
Quoted 17 lines · On 23.06.2014 21:29, [email protected] wrote:
Quoted 14 lines · Peter Zijlstra

kernel/sched/fair.c static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq) · line 3811

Quoted 10 lines · Kirill Tkhai
kernel/sched/fair.c +6 -0
@@ -3790,6 +3803,12 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 		cfs_rq->runtime_remaining = 1;
 		if (cfs_rq_throttled(cfs_rq))
 			unthrottle_cfs_rq(cfs_rq);
+
+		/*
+		 * Offline rq is schedulable till cpu is completely disabled
+		 * in take_cpu_down(), so we prevent new cfs throttling here.
+		 */
+		cfs_rq->runtime_enabled = 0;

Does it make sense to clear this before calling unthrottle_cfs_rq()? Just to make sure they're in the right order..

Quoted 6 lines

I believe that order is irrelevant here - I do not believe that unthrottle_cfs_rq(a) can cause a throttle_cfs_rq(a). In fact, I don't see any code that will check it at all from unthrottle, although I might be missing something. It _can_ cause a throttle_cfs_rq(parent_cfs_rq(a)), but that should be fine as long as for_each_leaf_cfs_rq is sorted correctly.

I think this is correct. We may change the order just for the hope, that anybody will work on it in some way in the future, and this person could skip this opaque place. Ok, I don't know how is better :)

Quoted 5 lines

That said, migrate_tasks drops rq->lock, and I /think/ another cpu could wake another task onto this cpu, which could then enqueue_throttle its cfs_rq (which previously had no tasks and thus wasn't on leaf_cfs_rq_list). You certainly could have tg_set_bandwidth come in and turn runtime_enabled on.

We mask cpu inactive on CPU_DOWN_PREPARE stage (in sched_cpu_inactive). Other cpu is not able to wake a task there after that.

rq is masked offline in cpuset_cpu_inactive() (during the same stage). But priority of sched_cpu_inactive() is higher than priority of cpuset_cpu_inactive().

Quoted 7 lines

CPU_PRI_SCHED_INACTIVE = INT_MIN + 1,
CPU_PRI_CPUSET_INACTIVE = INT_MIN,

This guarantees that nobody could use dying_cpu even before we start unthrottling. Another cpu will see dying_cpu is inactive.

So, it looks like we are free of this problem.

Ah, ok, I haven't looked that hard at hotplug, and wasn't sure of the ordering there. We still have the tg_set_cfs_bandwidth issue, because that uses for_each_possible_cpu. However, with the addition of rq_online_fair, that can probably be changed to for_each_active_cpu, and then I think we would be fine.

#7 lore raw
Quoted 50 lines · On 24.06.2014 01:05, [email protected] wrote:
Quoted 48 lines · Kirill Tkhai
Quoted 17 lines · On 23.06.2014 21:29, [email protected] wrote:
Quoted 14 lines · Peter Zijlstra

kernel/sched/fair.c static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq) · line 3811

Quoted 10 lines · Kirill Tkhai
kernel/sched/fair.c +6 -0
@@ -3790,6 +3803,12 @@ static void __maybe_unused unthrottle_offline_cfs_rqs(struct rq *rq)
 		cfs_rq->runtime_remaining = 1;
 		if (cfs_rq_throttled(cfs_rq))
 			unthrottle_cfs_rq(cfs_rq);
+
+		/*
+		 * Offline rq is schedulable till cpu is completely disabled
+		 * in take_cpu_down(), so we prevent new cfs throttling here.
+		 */
+		cfs_rq->runtime_enabled = 0;

Does it make sense to clear this before calling unthrottle_cfs_rq()? Just to make sure they're in the right order..

Quoted 6 lines

I believe that order is irrelevant here - I do not believe that unthrottle_cfs_rq(a) can cause a throttle_cfs_rq(a). In fact, I don't see any code that will check it at all from unthrottle, although I might be missing something. It _can_ cause a throttle_cfs_rq(parent_cfs_rq(a)), but that should be fine as long as for_each_leaf_cfs_rq is sorted correctly.

I think this is correct. We may change the order just for the hope, that anybody will work on it in some way in the future, and this person could skip this opaque place. Ok, I don't know how is better :)

Quoted 5 lines

That said, migrate_tasks drops rq->lock, and I /think/ another cpu could wake another task onto this cpu, which could then enqueue_throttle its cfs_rq (which previously had no tasks and thus wasn't on leaf_cfs_rq_list). You certainly could have tg_set_bandwidth come in and turn runtime_enabled on.

We mask cpu inactive on CPU_DOWN_PREPARE stage (in sched_cpu_inactive). Other cpu is not able to wake a task there after that.

rq is masked offline in cpuset_cpu_inactive() (during the same stage). But priority of sched_cpu_inactive() is higher than priority of cpuset_cpu_inactive().

CPU_PRI_SCHED_INACTIVE = INT_MIN + 1,
CPU_PRI_CPUSET_INACTIVE = INT_MIN,

This guarantees that nobody could use dying_cpu even before we start unthrottling. Another cpu will see dying_cpu is inactive.

Quoted 7 lines
Quoted 1 line

So, it looks like we are free of this problem.

Ah, ok, I haven't looked that hard at hotplug, and wasn't sure of the ordering there. We still have the tg_set_cfs_bandwidth issue, because that uses for_each_possible_cpu. However, with the addition of rq_online_fair, that can probably be changed to for_each_active_cpu, and then I think we would be fine.

Ok, now I see. Thanks, Ben.

#8 lore raw

PATCH 2/3 sched/rt: __disable_runtime: Enqueue just unthrottled rt_rq back on the stack

In reply to a message not in the archive

Make rt_rq available for pick_next_task(). Otherwise, their tasks stay prisoned long time till dead cpu becomes alive again.

CC Srikar Dronamraju <[email protected]>
CC Mike Galbraith <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Ingo Molnar <[email protected]>
---
 kernel/sched/rt.c |    3 +++
 1 file changed, 3 insertions(+)
kernel/sched/rt.c +3 -0
index a490831..671a8b5 100644
@@ -740,6 +740,9 @@ balanced:
 		rt_rq->rt_throttled = 0;
 		raw_spin_unlock(&rt_rq->rt_runtime_lock);
 		raw_spin_unlock(&rt_b->rt_runtime_lock);
+
+		/* Make rt_rq available for pick_next_task() */
+		sched_rt_rq_enqueue(rt_rq);
 	}
 }
 
maintainer
#9 lore raw

Re: [PATCH 2/3] sched/rt: __disable_runtime: Enqueue just unthrottled rt_rq back on the stack

kernel/sched/rt.c balanced:

Quoted 18 lines · Kirill Tkhai

Make rt_rq available for pick_next_task(). Otherwise, their tasks stay prisoned long time till dead cpu becomes alive again.

CC Srikar Dronamraju <[email protected]>
CC Mike Galbraith <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Ingo Molnar <[email protected]>
---
 kernel/sched/rt.c |    3 +++
 1 file changed, 3 insertions(+)
kernel/sched/rt.c +0 -0
index a490831..671a8b5 100644
@@ -740,6 +740,9 @@ balanced:

Depending on what you use to generate patches, can you:

QUILT_DIFF_OPTS="-F ^[[:alpha:]\$_].*[^:]\$"

or .gitconfig:

[diff "default"]

        xfuncname = "^[[:alpha:]$_].*[^:]$"

That avoids mistaking labels (like the above balanced:) for function names.

The patch itself makes sense though.

#10 lore raw

PATCH 3/3 sched: Rework check_for_tasks()

In reply to a message not in the archive

1)Iterate throw all of threads in the system.
Check for all threads, not only for group leaders.

2)Check for p->on_rq instead of p->state and cputime.
Preempted task in !TASK_RUNNING state OR just created task may be queued, that we want to be reported too.

3)Use read_lock() instead of write_lock().
This function does not change any structures, and read_lock() is enough.

CC Srikar Dronamraju <[email protected]>
CC Mike Galbraith <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Ingo Molnar <[email protected]>
---
 kernel/cpu.c |   33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)
kernel/cpu.c +20 -13
index a343bde..81e2a38 100644
@@ -274,21 +274,28 @@ void clear_tasks_mm_cpumask(int cpu)
 	rcu_read_unlock();
 }
 
-static inline void check_for_tasks(int cpu)
+static inline void check_for_tasks(int dead_cpu)
 {
-	struct task_struct *p;
-	cputime_t utime, stime;
+	struct task_struct *g, *p;
 
-	write_lock_irq(&tasklist_lock);
-	for_each_process(p) {
-		task_cputime(p, &utime, &stime);
-		if (task_cpu(p) == cpu && p->state == TASK_RUNNING &&
-		    (utime || stime))
-			pr_warn("Task %s (pid = %d) is on cpu %d (state = %ld, flags = %x)\n",
-				p->comm, task_pid_nr(p), cpu,
-				p->state, p->flags);
-	}
-	write_unlock_irq(&tasklist_lock);
+	read_lock_irq(&tasklist_lock);
+	do_each_thread(g, p) {
+		if (!p->on_rq)
+			continue;
+		/*
+		 * We do the check with unlocked task_rq(p)->lock.
+		 * Order the reading to do not warn about a task,
+		 * which was running on this cpu in the past, and
+		 * it's just been woken on another cpu.
+		 */
+		rmb();
+		if (task_cpu(p) != dead_cpu)
+			continue;
+
+		pr_warn("Task %s (pid=%d) is on cpu %d (state=%ld, flags=%x)\n",
+			p->comm, task_pid_nr(p), dead_cpu, p->state, p->flags);
+	} while_each_thread(g, p);
+	read_unlock_irq(&tasklist_lock);
 }
 
 struct take_cpu_down_param {
maintainer
#11 lore raw

Re: [PATCH 3/3] sched: Rework check_for_tasks()

Quoted 2 lines · Kirill Tkhai

1)Iterate throw all of threads in the system.

thru

Quoted 46 lines

Check for all threads, not only for group leaders.

2)Check for p->on_rq instead of p->state and cputime.
Preempted task in !TASK_RUNNING state OR just created task may be queued, that we want to be reported too.

3)Use read_lock() instead of write_lock().
This function does not change any structures, and read_lock() is enough.

CC Srikar Dronamraju <[email protected]>
CC Mike Galbraith <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Ingo Molnar <[email protected]>
---
 kernel/cpu.c |   33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)
kernel/cpu.c +3 -13
index a343bde..81e2a38 100644
@@ -274,21 +274,28 @@ void clear_tasks_mm_cpumask(int cpu)
 	rcu_read_unlock();
 }
 
-static inline void check_for_tasks(int cpu)
+static inline void check_for_tasks(int dead_cpu)
 {
-	struct task_struct *p;
-	cputime_t utime, stime;
+	struct task_struct *g, *p;
 
-	write_lock_irq(&tasklist_lock);
-	for_each_process(p) {
-		task_cputime(p, &utime, &stime);
-		if (task_cpu(p) == cpu && p->state == TASK_RUNNING &&
-		    (utime || stime))
-			pr_warn("Task %s (pid = %d) is on cpu %d (state = %ld, flags = %x)\n",
-				p->comm, task_pid_nr(p), cpu,
-				p->state, p->flags);
-	}
-	write_unlock_irq(&tasklist_lock);
+	read_lock_irq(&tasklist_lock);

kernel/cpu.c void clear_tasks_mm_cpumask(int cpu) · line 291

Quoted 10 lines
kernel/cpu.c +10 -0
+	do_each_thread(g, p) {
+		if (!p->on_rq)
+			continue;
+		/*
+		 * We do the check with unlocked task_rq(p)->lock.
+		 * Order the reading to do not warn about a task,
+		 * which was running on this cpu in the past, and
+		 * it's just been woken on another cpu.
+		 */
+		rmb();
		smp_rmb();

kernel/cpu.c void clear_tasks_mm_cpumask(int cpu) · line 293

Quoted 2 lines
kernel/cpu.c +2 -0
+		if (task_cpu(p) != dead_cpu)
+			continue;

But because we don't have rq->lock held, we can be in the middle of a wakeup and miss a task.

Then again, I suppose anything without rq->lock can and will miss tasks.

Quoted 10 lines
kernel/cpu.c +4 -0
+		pr_warn("Task %s (pid=%d) is on cpu %d (state=%ld, flags=%x)\n",
+			p->comm, task_pid_nr(p), dead_cpu, p->state, p->flags);
+	} while_each_thread(g, p);
+	read_unlock_irq(&tasklist_lock);
 }
 
 struct take_cpu_down_param {
#12 lore raw

Re: [PATCH 3/3] sched: Rework check_for_tasks()

В Пн, 23/06/2014 в 12:24 +0200, Peter Zijlstra пишет:

Quoted 5 lines · Kirill Tkhai
Quoted 2 lines

1)Iterate throw all of threads in the system.

thru

Thanks :)

Quoted 60 lines
Quoted 47 lines

Check for all threads, not only for group leaders.

2)Check for p->on_rq instead of p->state and cputime.
Preempted task in !TASK_RUNNING state OR just created task may be queued, that we want to be reported too.

3)Use read_lock() instead of write_lock().
This function does not change any structures, and read_lock() is enough.

CC Srikar Dronamraju <[email protected]>
CC Mike Galbraith <[email protected]>
CC Peter Zijlstra <[email protected]>
CC Ingo Molnar <[email protected]>
---
 kernel/cpu.c |   33 ++++++++++++++++++++-------------
 1 file changed, 20 insertions(+), 13 deletions(-)
kernel/cpu.c +3 -13
index a343bde..81e2a38 100644
@@ -274,21 +274,28 @@ void clear_tasks_mm_cpumask(int cpu)
 	rcu_read_unlock();
 }
 
-static inline void check_for_tasks(int cpu)
+static inline void check_for_tasks(int dead_cpu)
 {
-	struct task_struct *p;
-	cputime_t utime, stime;
+	struct task_struct *g, *p;
 
-	write_lock_irq(&tasklist_lock);
-	for_each_process(p) {
-		task_cputime(p, &utime, &stime);
-		if (task_cpu(p) == cpu && p->state == TASK_RUNNING &&
-		    (utime || stime))
-			pr_warn("Task %s (pid = %d) is on cpu %d (state = %ld, flags = %x)\n",
-				p->comm, task_pid_nr(p), cpu,
-				p->state, p->flags);
-	}
-	write_unlock_irq(&tasklist_lock);
+	read_lock_irq(&tasklist_lock);

kernel/cpu.c void clear_tasks_mm_cpumask(int cpu) · line 291

Quoted 10 lines
kernel/cpu.c +10 -0
+	do_each_thread(g, p) {
+		if (!p->on_rq)
+			continue;
+		/*
+		 * We do the check with unlocked task_rq(p)->lock.
+		 * Order the reading to do not warn about a task,
+		 * which was running on this cpu in the past, and
+		 * it's just been woken on another cpu.
+		 */
+		rmb();
		smp_rmb();
Quoted 7 lines

kernel/cpu.c void clear_tasks_mm_cpumask(int cpu) · line 293

Quoted 2 lines
kernel/cpu.c +2 -0
+		if (task_cpu(p) != dead_cpu)
+			continue;

But because we don't have rq->lock held, we can be in the middle of a wakeup and miss a task.

Then again, I suppose anything without rq->lock can and will miss tasks.

If we use rq->lock it's possible to move check_for_tasks() to kernel/sched/core.c.

And we can leave TASK_RUNNING check for waking tasks. Maybe something like this?

static inline void check_for_tasks(int dead_cpu)

{
	struct task_struct *g, *p;
	struct rq *rq = cpu_rq(dead_cpu);

	read_lock_irq(&tasklist_lock);
	raw_spin_lock(&rq->lock)

	do_each_thread(g, p) {
		if (!p->on_rq && p->state != TASK_RUNNING)
			continue;
		if (task_cpu(p) != dead_cpu)
			continue;

		pr_warn("Task %s (pid=%d) is on cpu %d (state=%ld, flags=%x)\n",
			p->comm, task_pid_nr(p), dead_cpu, p->state, p->flags);
	} while_each_thread(g, p);

	raw_spin_unlock(&rq->lock)
	read_unlock_irq(&tasklist_lock);
}

It still does not give a 100% guarantee... Should we take p->pi_lock for every task?

Quoted 10 lines

kernel/cpu.c void clear_tasks_mm_cpumask(int cpu) · line 301

Quoted 10 lines
kernel/cpu.c +4 -0
+		pr_warn("Task %s (pid=%d) is on cpu %d (state=%ld, flags=%x)\n",
+			p->comm, task_pid_nr(p), dead_cpu, p->state, p->flags);
+	} while_each_thread(g, p);
+	read_unlock_irq(&tasklist_lock);
 }
 
 struct take_cpu_down_param {

Regards,
Kirill

maintainer
#13 lore raw

Re: [PATCH 3/3] sched: Rework check_for_tasks()

Quoted 28 lines · Kirill Tkhai
Quoted 1 line

Then again, I suppose anything without rq->lock can and will miss tasks.

If we use rq->lock it's possible to move check_for_tasks() to kernel/sched/core.c.

And we can leave TASK_RUNNING check for waking tasks. Maybe something like this?

static inline void check_for_tasks(int dead_cpu)

{
	struct task_struct *g, *p;
	struct rq *rq = cpu_rq(dead_cpu);

	read_lock_irq(&tasklist_lock);
	raw_spin_lock(&rq->lock)

	do_each_thread(g, p) {
		if (!p->on_rq && p->state != TASK_RUNNING)
			continue;
		if (task_cpu(p) != dead_cpu)
			continue;

		pr_warn("Task %s (pid=%d) is on cpu %d (state=%ld, flags=%x)\n",
			p->comm, task_pid_nr(p), dead_cpu, p->state, p->flags);
	} while_each_thread(g, p);

	raw_spin_unlock(&rq->lock)
	read_unlock_irq(&tasklist_lock);
}
Quoted 1 line

It still does not give a 100% guarantee... Should we take p->pi_lock for every task?

seeing how rq->lock nests inside of ->pi_lock that's going to be somewhat icky.

I think we can live with a false negative, given how much people run this nonsense it'll trigger eventually.

False positives would be bad though :-)

So I think we can keep your original (lock-free) proposal.