Skip to contentSkip to conversation

sched: Address idle task vs pcpu kthread checks

PATCH Applied Valentin Schneider 4 participants 13 messages May 10, 2021 – Jun 1, 2021

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover sched: Address idle task vs pcpu kthread checks

Commit

5ba2ffba13a1 ("sched: Fix CPU hotplug / tighten is_per_cpu_kthread()")

had to special-case the idle task when checking for per-CPU kthreads. This is due to the idle task not having its own struct kthread, which is where we'd store KTHREAD_IS_PER_CPU.

From staring at Yejune's recent patch [1], it turns out the idle task is also missing PF_NO_SETAFFINITY.

Patch 1 cleans this up, patch 2 is Yejune's v1 which depends on it.

Note: I remember seeing some patch(es) from Peter tackling this exact problem, but I couldn't find them again.

[1]: http://lore.kernel.org/r/[email protected]

Cheers,
Valentin

Valentin Schneider (1):
sched: Make the idle task quack like a per-CPU kthread

Yejune Deng (1):
lib/smp_processor_id: Use is_percpu_thread() instead of
nr_cpus_allowed

 include/linux/kthread.h |  2 ++
 kernel/kthread.c        | 30 ++++++++++++++++++------------
 kernel/sched/core.c     | 21 +++++++++++++++------
 lib/smp_processor_id.c  |  6 +-----
 4 files changed, 36 insertions(+), 23 deletions(-)

--
2.25.1

#1 lore raw

PATCH 1/2 sched: Make the idle task quack like a per-CPU kthread

For all intents and purposes, the idle task is a per-CPU kthread. It isn't created via the same route as other pcpu kthreads however, and as a result it is missing a few bells and whistles: it fails kthread_is_per_cpu() and it doesn't have PF_NO_SETAFFINITY set.

Fix the former by giving the idle task a kthread struct along with the KTHREAD_IS_PER_CPU flag. This requires some extra iffery as init_idle() call be called more than once on the same idle task.

---
 include/linux/kthread.h |  2 ++
 kernel/kthread.c        | 30 ++++++++++++++++++------------
 kernel/sched/core.c     | 21 +++++++++++++++------
 3 files changed, 35 insertions(+), 18 deletions(-)
include/linux/kthread.h +2 -0
index 2484ed97e72f..d9133d6db308 100644
@@ -33,6 +33,8 @@ struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data),
 					  unsigned int cpu,
 					  const char *namefmt);
 
+void set_kthread_struct(struct task_struct *p);
+
 void kthread_set_per_cpu(struct task_struct *k, int cpu);
 bool kthread_is_per_cpu(struct task_struct *k);
 
kernel/kthread.c +18 -12
index 6d3c488a0f82..77dd0ea1d35d 100644
@@ -68,16 +68,6 @@ enum KTHREAD_BITS {
 	KTHREAD_SHOULD_PARK,
 };
 
-static inline void set_kthread_struct(void *kthread)
-{
-	/*
-	 * We abuse ->set_child_tid to avoid the new member and because it
-	 * can't be wrongly copied by copy_process(). We also rely on fact
-	 * that the caller can't exec, so PF_KTHREAD can't be cleared.
-	 */
-	current->set_child_tid = (__force void __user *)kthread;
-}
-
 static inline struct kthread *to_kthread(struct task_struct *k)
 {
 	WARN_ON(!(k->flags & PF_KTHREAD));
@@ -103,6 +93,22 @@ static inline struct kthread *__to_kthread(struct task_struct *p)
 	return kthread;
 }
 
+void set_kthread_struct(struct task_struct *p)
+{
+	struct kthread *kthread;
+
+	if (__to_kthread(p))
+		return;
+
+	kthread = kzalloc(sizeof(*kthread), GFP_KERNEL);
+	/*
+	 * We abuse ->set_child_tid to avoid the new member and because it
+	 * can't be wrongly copied by copy_process(). We also rely on fact
+	 * that the caller can't exec, so PF_KTHREAD can't be cleared.
+	 */
+	p->set_child_tid = (__force void __user *)kthread;
+}
+
 void free_kthread_struct(struct task_struct *k)
 {
 	struct kthread *kthread;
@@ -272,8 +278,8 @@ static int kthread(void *_create)
 	struct kthread *self;
 	int ret;
 
-	self = kzalloc(sizeof(*self), GFP_KERNEL);
-	set_kthread_struct(self);
+	set_kthread_struct(current);
+	self = to_kthread(current);
 
 	/* If user was SIGKILLed, I release the structure. */
 	done = xchg(&create->done, NULL);
kernel/sched/core.c +15 -6
index 4a0668acd876..5bb720829d93 100644
@@ -7440,12 +7440,25 @@ void init_idle(struct task_struct *idle, int cpu)
 
 	__sched_fork(0, idle);
 
+	/*
+	 * The idle task doesn't need the kthread struct to function, but it
+	 * is dressed up as a per-CPU kthread and thus needs to play the part
+	 * if we want to avoid special-casing it in code that deals with per-CPU
+	 * kthreads.
+	 */
+	set_kthread_struct(idle);
+
 	raw_spin_lock_irqsave(&idle->pi_lock, flags);
 	raw_spin_lock(&rq->lock);
 
 	idle->state = TASK_RUNNING;
 	idle->se.exec_start = sched_clock();
-	idle->flags |= PF_IDLE;
+	/*
+	 * PF_KTHREAD should already be set at this point; regardless, make it
+	 * look like a proper per-CPU kthread.
+	 */
+	idle->flags |= PF_IDLE | PF_KTHREAD | PF_NO_SETAFFINITY;
+	kthread_set_per_cpu(idle, cpu);
 
 	scs_task_reset(idle);
 	kasan_unpoison_task_stack(idle);
@@ -7662,12 +7675,8 @@ static void balance_push(struct rq *rq)
 	/*
 	 * Both the cpu-hotplug and stop task are in this case and are
 	 * required to complete the hotplug process.
-	 *
-	 * XXX: the idle task does not match kthread_is_per_cpu() due to
-	 * histerical raisins.
 	 */
-	if (rq->idle == push_task ||
-	    kthread_is_per_cpu(push_task) ||
+	if (kthread_is_per_cpu(push_task) ||
 	    is_migration_disabled(push_task)) {
 
 		/*
Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH 1/2] sched: Make the idle task quack like a per-CPU kthread

Quoted 3 lines · Valentin Schneider

This requires some extra iffery as init_idle() call be called more than once on the same idle task.

While I'm at it, do we actually still need to suffer through this?

AFAICT the extra calls are due to idle_thread_get() (used in cpuhp) calling init_idle(). However it looks to me that since

3bb5d2ee396a ("smp, idle: Allocate idle thread for each possible cpu during boot")

we don't need to do that: we already have a

for_each_possible_cpu(cpu)
init_idle(cpu)

issued at init. So can't we "simply" rely on that init-time creation, given it's done against the possible mask? I think the only thing that might need doing at later hotplug is making sure the preempt count is right (secondary startups seem to all prepare the idle task by issuing a preempt_disable()).

#3 lore raw

Re: [PATCH 1/2] sched: Make the idle task quack like a per-CPU kthread

Quoted 6 lines · Valentin Schneider
Quoted 3 lines · Valentin Schneider

This requires some extra iffery as init_idle() call be called more than once on the same idle task.

While I'm at it, do we actually still need to suffer through this?

No.

Quoted 7 lines

AFAICT the extra calls are due to idle_thread_get() (used in cpuhp) calling init_idle(). However it looks to me that since

3bb5d2ee396a ("smp, idle: Allocate idle thread for each possible cpu during boot")

we don't need to do that: we already have a

Quoted 8 lines

for_each_possible_cpu(cpu)
init_idle(cpu)

issued at init. So can't we "simply" rely on that init-time creation, given it's done against the possible mask? I think the only thing that might need doing at later hotplug is making sure the preempt count is right (secondary startups seem to all prepare the idle task by issuing a preempt_disable()).

Best-case it works, worst-case we discover an unclean assumption in the init sequence and it works after we fix that.

Win-win. :-)

Thanks,

Ingo

#4 lore raw

Re: [PATCH 1/2] sched: Make the idle task quack like a per-CPU kthread

Quoted 17 lines · Ingo Molnar
Quoted 7 lines · Valentin Schneider

AFAICT the extra calls are due to idle_thread_get() (used in cpuhp) calling init_idle(). However it looks to me that since

3bb5d2ee396a ("smp, idle: Allocate idle thread for each possible cpu during boot")

we don't need to do that: we already have a

Quoted 8 lines

for_each_possible_cpu(cpu)
init_idle(cpu)

issued at init. So can't we "simply" rely on that init-time creation, given it's done against the possible mask? I think the only thing that might need doing at later hotplug is making sure the preempt count is right (secondary startups seem to all prepare the idle task by issuing a preempt_disable()).

Quoted 5 lines

Best-case it works, worst-case we discover an unclean assumption in the init sequence and it works after we fix that.

Win-win. :-)

Well I got something that seems to work, let me it test it some more and convince myself it isn't completely bonkers and I'll toss it out.

Quoted 3 lines

Thanks,

Ingo

#5 lore raw

[tip: sched/core] sched: Make the idle task quack like a per-CPU kthread

In reply to Valentin Schneider
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     00b89fe0197f0c55a045775c11553c0cdb7082fe
Gitweb:        https://git.kernel.org/tip/00b89fe0197f0c55a045775c11553c0cdb7082fe
Author:        Valentin Schneider <[email protected]>
AuthorDate:    Mon, 10 May 2021 16:10:23 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 18 May 2021 12:53:53 +02:00

sched: Make the idle task quack like a per-CPU kthread

For all intents and purposes, the idle task is a per-CPU kthread. It isn't created via the same route as other pcpu kthreads however, and as a result it is missing a few bells and whistles: it fails kthread_is_per_cpu() and it doesn't have PF_NO_SETAFFINITY set.

Fix the former by giving the idle task a kthread struct along with the KTHREAD_IS_PER_CPU flag. This requires some extra iffery as init_idle() call be called more than once on the same idle task.

---
 include/linux/kthread.h |  2 ++
 kernel/kthread.c        | 30 ++++++++++++++++++------------
 kernel/sched/core.c     | 21 +++++++++++++++------
 3 files changed, 35 insertions(+), 18 deletions(-)
include/linux/kthread.h +2 -0
index 2484ed9..d9133d6 100644
@@ -33,6 +33,8 @@ struct task_struct *kthread_create_on_cpu(int (*threadfn)(void *data),
 					  unsigned int cpu,
 					  const char *namefmt);
 
+void set_kthread_struct(struct task_struct *p);
+
 void kthread_set_per_cpu(struct task_struct *k, int cpu);
 bool kthread_is_per_cpu(struct task_struct *k);
 
kernel/kthread.c +18 -12
index fe3f2a4..3d32683 100644
@@ -68,16 +68,6 @@ enum KTHREAD_BITS {
 	KTHREAD_SHOULD_PARK,
 };
 
-static inline void set_kthread_struct(void *kthread)
-{
-	/*
-	 * We abuse ->set_child_tid to avoid the new member and because it
-	 * can't be wrongly copied by copy_process(). We also rely on fact
-	 * that the caller can't exec, so PF_KTHREAD can't be cleared.
-	 */
-	current->set_child_tid = (__force void __user *)kthread;
-}
-
 static inline struct kthread *to_kthread(struct task_struct *k)
 {
 	WARN_ON(!(k->flags & PF_KTHREAD));
@@ -103,6 +93,22 @@ static inline struct kthread *__to_kthread(struct task_struct *p)
 	return kthread;
 }
 
+void set_kthread_struct(struct task_struct *p)
+{
+	struct kthread *kthread;
+
+	if (__to_kthread(p))
+		return;
+
+	kthread = kzalloc(sizeof(*kthread), GFP_KERNEL);
+	/*
+	 * We abuse ->set_child_tid to avoid the new member and because it
+	 * can't be wrongly copied by copy_process(). We also rely on fact
+	 * that the caller can't exec, so PF_KTHREAD can't be cleared.
+	 */
+	p->set_child_tid = (__force void __user *)kthread;
+}
+
 void free_kthread_struct(struct task_struct *k)
 {
 	struct kthread *kthread;
@@ -272,8 +278,8 @@ static int kthread(void *_create)
 	struct kthread *self;
 	int ret;
 
-	self = kzalloc(sizeof(*self), GFP_KERNEL);
-	set_kthread_struct(self);
+	set_kthread_struct(current);
+	self = to_kthread(current);
 
 	/* If user was SIGKILLed, I release the structure. */
 	done = xchg(&create->done, NULL);
kernel/sched/core.c +15 -6
index 24fd795..6a5124c 100644
@@ -8234,12 +8234,25 @@ void __init init_idle(struct task_struct *idle, int cpu)
 
 	__sched_fork(0, idle);
 
+	/*
+	 * The idle task doesn't need the kthread struct to function, but it
+	 * is dressed up as a per-CPU kthread and thus needs to play the part
+	 * if we want to avoid special-casing it in code that deals with per-CPU
+	 * kthreads.
+	 */
+	set_kthread_struct(idle);
+
 	raw_spin_lock_irqsave(&idle->pi_lock, flags);
 	raw_spin_rq_lock(rq);
 
 	idle->state = TASK_RUNNING;
 	idle->se.exec_start = sched_clock();
-	idle->flags |= PF_IDLE;
+	/*
+	 * PF_KTHREAD should already be set at this point; regardless, make it
+	 * look like a proper per-CPU kthread.
+	 */
+	idle->flags |= PF_IDLE | PF_KTHREAD | PF_NO_SETAFFINITY;
+	kthread_set_per_cpu(idle, cpu);
 
 	scs_task_reset(idle);
 	kasan_unpoison_task_stack(idle);
@@ -8456,12 +8469,8 @@ static void balance_push(struct rq *rq)
 	/*
 	 * Both the cpu-hotplug and stop task are in this case and are
 	 * required to complete the hotplug process.
-	 *
-	 * XXX: the idle task does not match kthread_is_per_cpu() due to
-	 * histerical raisins.
 	 */
-	if (rq->idle == push_task ||
-	    kthread_is_per_cpu(push_task) ||
+	if (kthread_is_per_cpu(push_task) ||
 	    is_migration_disabled(push_task)) {
 
 		/*
#6 lore raw

PATCH 2/2 lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed

In reply to Valentin Schneider

From: Yejune Deng <[email protected]>

is_percpu_thread() more elegantly handles SMP vs UP, and further checks the presence of PF_NO_SETAFFINITY. This lets us catch cases where check_preemption_disabled() can race with a concurrent sched_setaffinity().

---
 lib/smp_processor_id.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
lib/smp_processor_id.c +1 -5
index 1c1dbd300325..046ac6297c78 100644
@@ -19,11 +19,7 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)
 	if (irqs_disabled())
 		goto out;
 
-	/*
-	 * Kernel threads bound to a single CPU can safely use
-	 * smp_processor_id():
-	 */
-	if (current->nr_cpus_allowed == 1)
+	if (is_percpu_thread())
 		goto out;
 
 #ifdef CONFIG_SMP
Signature · 1 line
2.25.1
merge bot Applied
#7 lore raw

[tip: sched/core] lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed

The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     0019699518cc026b5bd912425be8e424843d5b33
Gitweb:        https://git.kernel.org/tip/0019699518cc026b5bd912425be8e424843d5b33
Author:        Yejune Deng <[email protected]>
AuthorDate:    Mon, 10 May 2021 16:10:24 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 18 May 2021 12:53:54 +02:00

lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed

is_percpu_thread() more elegantly handles SMP vs UP, and further checks the presence of PF_NO_SETAFFINITY. This lets us catch cases where check_preemption_disabled() can race with a concurrent sched_setaffinity().

---
 lib/smp_processor_id.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
lib/smp_processor_id.c +1 -5
index 1c1dbd3..046ac62 100644
@@ -19,11 +19,7 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)
 	if (irqs_disabled())
 		goto out;
 
-	/*
-	 * Kernel threads bound to a single CPU can safely use
-	 * smp_processor_id():
-	 */
-	if (current->nr_cpus_allowed == 1)
+	if (is_percpu_thread())
 		goto out;
 
 #ifdef CONFIG_SMP
merge bot Applied
#8 lore raw

[tip: sched/core] lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed

In reply to Valentin Schneider
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     570a752b7a9bd03b50ad6420cd7f10092cc11bd3
Gitweb:        https://git.kernel.org/tip/570a752b7a9bd03b50ad6420cd7f10092cc11bd3
Author:        Yejune Deng <[email protected]>
AuthorDate:    Mon, 10 May 2021 16:10:24 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 19 May 2021 10:51:40 +02:00

lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed

is_percpu_thread() more elegantly handles SMP vs UP, and further checks the presence of PF_NO_SETAFFINITY. This lets us catch cases where check_preemption_disabled() can race with a concurrent sched_setaffinity().

---
 lib/smp_processor_id.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
lib/smp_processor_id.c +1 -5
index 1c1dbd3..046ac62 100644
@@ -19,11 +19,7 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)
 	if (irqs_disabled())
 		goto out;
 
-	/*
-	 * Kernel threads bound to a single CPU can safely use
-	 * smp_processor_id():
-	 */
-	if (current->nr_cpus_allowed == 1)
+	if (is_percpu_thread())
 		goto out;
 
 #ifdef CONFIG_SMP
maintainer
#9 lore raw

PATCH 1/1 sched,init: Fix DEBUG_PREEMPT vs early boot

Quoted 29 lines · On Wed, May 19, 2021 at 09:02:34AM -0000, tip-bot2 for Yejune Deng wrote:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     570a752b7a9bd03b50ad6420cd7f10092cc11bd3
Gitweb:        https://git.kernel.org/tip/570a752b7a9bd03b50ad6420cd7f10092cc11bd3
Author:        Yejune Deng <[email protected]>
AuthorDate:    Mon, 10 May 2021 16:10:24 +01:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Wed, 19 May 2021 10:51:40 +02:00

lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed

is_percpu_thread() more elegantly handles SMP vs UP, and further checks the presence of PF_NO_SETAFFINITY. This lets us catch cases where check_preemption_disabled() can race with a concurrent sched_setaffinity().

---
 lib/smp_processor_id.c | 6 +-----
 1 file changed, 1 insertion(+), 5 deletions(-)
lib/smp_processor_id.c +0 -0
index 1c1dbd3..046ac62 100644

lib/smp_processor_id.c unsigned int check_preemption_disabled(const char *what1, const char *what2) · line 23

Quoted 10 lines
lib/smp_processor_id.c +1 -5
@@ -19,11 +19,7 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)
 	if (irqs_disabled())
 		goto out;
 
-	/*
-	 * Kernel threads bound to a single CPU can safely use
-	 * smp_processor_id():
-	 */
-	if (current->nr_cpus_allowed == 1)
+	if (is_percpu_thread())
 		goto out;

So my test box was unhappy with all this and started spewing lots of DEBUG_PREEMPT warns on boot.

This extends 8fb12156b8db6 to cover the new requirement.

---
Subject: sched,init: Fix DEBUG_PREEMPT vs early boot

Extend 8fb12156b8db ("init: Pin init task to the boot CPU, initially") to cover the new PF_NO_SETAFFINITY requirement.

While there, move wait_for_completion(&kthreadd_done) into kernel_init() to make it absolutely clear it is the very first thing done by the init thread.

Fixes 570a752b7a9b ("lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed")
---
 init/main.c         | 11 ++++++-----
 kernel/sched/core.c |  1 +
 2 files changed, 7 insertions(+), 5 deletions(-)
init/main.c +6 -5
index 7b027d9c5c89..e945ec82b8a5 100644
@@ -692,6 +692,7 @@ noinline void __ref rest_init(void)
 	 */
 	rcu_read_lock();
 	tsk = find_task_by_pid_ns(pid, &init_pid_ns);
+	tsk->flags |= PF_NO_SETAFFINITY;
 	set_cpus_allowed_ptr(tsk, cpumask_of(smp_processor_id()));
 	rcu_read_unlock();
 
@@ -1440,6 +1441,11 @@ static int __ref kernel_init(void *unused)
 {
 	int ret;
 
+	/*
+	 * Wait until kthreadd is all set-up.
+	 */
+	wait_for_completion(&kthreadd_done);
+
 	kernel_init_freeable();
 	/* need to finish all async __init code before freeing the memory */
 	async_synchronize_full();
@@ -1520,11 +1526,6 @@ void __init console_on_rootfs(void)
 
 static noinline void __init kernel_init_freeable(void)
 {
-	/*
-	 * Wait until kthreadd is all set-up.
-	 */
-	wait_for_completion(&kthreadd_done);
-
 	/* Now the scheduler is fully set up and can do blocking allocations */
 	gfp_allowed_mask = __GFP_BITS_MASK;
 
kernel/sched/core.c +1 -0
index adea0b1e8036..ae7737e6c2b2 100644
@@ -8867,6 +8867,7 @@ void __init sched_init_smp(void)
 	/* Move init over to a non-isolated CPU */
 	if (set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_FLAG_DOMAIN)) < 0)
 		BUG();
+	current->flags &= ~PF_NO_SETAFFINITY;
 	sched_init_granularity();
 
 	init_sched_rt_class();
author Tested-by Reviewed-by on v1 1/1
#10 lore raw

Re: [PATCH] sched,init: Fix DEBUG_PREEMPT vs early boot

Quoted 5 lines · Peter Zijlstra
Quoted 4 lines · On Wed, May 19, 2021 at 09:02:34AM -0000, tip-bot2 for Yejune Deng wrote:
(quoted diff) +0 -0
@@ -19,11 +19,7 @@ unsigned int check_preemption_disabled(const char *what1, const char *what2)
     if (irqs_disabled())
             goto out;
 
Quoted 11 lines

Review unsigned int check_preemption_disabled(const char *what1, const char *what2) · line 23

Quoted 7 lines
(quoted diff) +1 -5
-	/*
-	 * Kernel threads bound to a single CPU can safely use
-	 * smp_processor_id():
-	 */
-	if (current->nr_cpus_allowed == 1)
+	if (is_percpu_thread())
             goto out;

So my test box was unhappy with all this and started spewing lots of DEBUG_PREEMPT warns on boot.

I get these too, though can't recall getting them when testing the above. I think it's tied with what Frederic found out with copy_process() copying PF_NO_SETAFFINITY, which it now no longer does.

Quoted 8 lines

This extends 8fb12156b8db6 to cover the new requirement.

---
Subject: sched,init: Fix DEBUG_PREEMPT vs early boot

Extend 8fb12156b8db ("init: Pin init task to the boot CPU, initially") to cover the new PF_NO_SETAFFINITY requirement.

Quoted 6 lines

While there, move wait_for_completion(&kthreadd_done) into kernel_init() to make it absolutely clear it is the very first thing done by the init thread.

Fixes 570a752b7a9b ("lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed")
merge bot Applied
#11 lore raw

[tip: sched/core] sched,init: Fix DEBUG_PREEMPT vs early boot

In reply to Peter Zijlstra
The following commit has been merged into the sched/core branch of tip:

The following commit has been merged into the sched/core branch of tip:

Commit-ID:     15faafc6b449777a85c0cf82dd8286c293fed4eb
Gitweb:        https://git.kernel.org/tip/15faafc6b449777a85c0cf82dd8286c293fed4eb
Author:        Peter Zijlstra <[email protected]>
AuthorDate:    Mon, 31 May 2021 12:21:13 +02:00
Committer:     Peter Zijlstra <[email protected]>
CommitterDate: Tue, 01 Jun 2021 16:00:11 +02:00

sched,init: Fix DEBUG_PREEMPT vs early boot

Extend 8fb12156b8db ("init: Pin init task to the boot CPU, initially") to cover the new PF_NO_SETAFFINITY requirement.

While there, move wait_for_completion(&kthreadd_done) into kernel_init() to make it absolutely clear it is the very first thing done by the init thread.

Fixes 570a752b7a9b ("lib/smp_processor_id: Use is_percpu_thread() instead of nr_cpus_allowed")
---
 init/main.c         | 11 ++++++-----
 kernel/sched/core.c |  1 +
 2 files changed, 7 insertions(+), 5 deletions(-)
init/main.c +6 -5
index 7b027d9..e945ec8 100644
@@ -692,6 +692,7 @@ noinline void __ref rest_init(void)
 	 */
 	rcu_read_lock();
 	tsk = find_task_by_pid_ns(pid, &init_pid_ns);
+	tsk->flags |= PF_NO_SETAFFINITY;
 	set_cpus_allowed_ptr(tsk, cpumask_of(smp_processor_id()));
 	rcu_read_unlock();
 
@@ -1440,6 +1441,11 @@ static int __ref kernel_init(void *unused)
 {
 	int ret;
 
+	/*
+	 * Wait until kthreadd is all set-up.
+	 */
+	wait_for_completion(&kthreadd_done);
+
 	kernel_init_freeable();
 	/* need to finish all async __init code before freeing the memory */
 	async_synchronize_full();
@@ -1520,11 +1526,6 @@ void __init console_on_rootfs(void)
 
 static noinline void __init kernel_init_freeable(void)
 {
-	/*
-	 * Wait until kthreadd is all set-up.
-	 */
-	wait_for_completion(&kthreadd_done);
-
 	/* Now the scheduler is fully set up and can do blocking allocations */
 	gfp_allowed_mask = __GFP_BITS_MASK;
 
kernel/sched/core.c +1 -0
index 3d25272..e205c19 100644
@@ -8862,6 +8862,7 @@ void __init sched_init_smp(void)
 	/* Move init over to a non-isolated CPU */
 	if (set_cpus_allowed_ptr(current, housekeeping_cpumask(HK_FLAG_DOMAIN)) < 0)
 		BUG();
+	current->flags &= ~PF_NO_SETAFFINITY;
 	sched_init_granularity();
 
 	init_sched_rt_class();
maintainer
#12 lore raw
In reply to Valentin Schneider
Quoted 2 lines · Valentin Schneider

Note: I remember seeing some patch(es) from Peter tackling this exact problem, but I couldn't find them again.

Found it (by accident), yours is nicer though :-)

Quoted 7 lines

Valentin Schneider (1):
sched: Make the idle task quack like a per-CPU kthread

Yejune Deng (1):
lib/smp_processor_id: Use is_percpu_thread() instead of
nr_cpus_allowed

Quoted 5 lines
 include/linux/kthread.h |  2 ++
 kernel/kthread.c        | 30 ++++++++++++++++++------------
 kernel/sched/core.c     | 21 +++++++++++++++------
 lib/smp_processor_id.c  |  6 +-----
 4 files changed, 36 insertions(+), 23 deletions(-)

Thanks!