init_idle reaped before final call
I'm working with a (approximately) 2.4.17 kernel from the mips-linux tree (oss.sgi.com).
I'd like to propose removing the "__init" designation from init_idle in kernel/sched.c, since this is called from rest_init via cpu_idle. Notice that rest_init isn't in an init section, and explicitly mentions that it's avoiding a race with free_initmem. In my kernel (an SMP kernel running on a system with only 1 available CPU), cpu_idle isn't getting called until after free_initmem().
My CPU is MIPS, but it looks like x86 could experience the same problem.
Kip
Index: kernel/sched.c
===================================================================
RCS file:
/projects/bbp/cvsroot/systemsw/linux/src/kernel/kernel/sched.c,v retrieving revision 1.10
sched.c 2002/03/05 19:40:14 renamed from sched.c 2002/01/15 04:13:43 1.10 +1 -1
@@ -1304,7 +1304,7 @@ extern unsigned long wait_init_idle; -void __init init_idle(void) +void init_idle(void) { struct schedule_data * sched_data; sched_data = &aligned_data[smp_processor_id()].schedule_data;
Quoted 11 lines
I'm working with a (approximately) 2.4.17 kernel from the mips-linux tree (oss.sgi.com).
I'd like to propose removing the "
__init" designation frominit_idleinkernel/sched.c, since this is called fromrest_initviacpu_idle. Notice thatrest_initisn't in an init section, and explicitly mentions that it's avoiding a race withfree_initmem. In my kernel (an SMP kernel running on a system with only 1 available CPU),cpu_idleisn't getting called until afterfree_initmem().My CPU is MIPS, but it looks like x86 could experience the same problem.
I fixed something in this area for x86, looks like the same code path for MIPS unless I'm misreading.
smp_init spins waiting on wait_init_idle until every cpu has done init_idle. rest_init() isn't called until smp_init returns, so I'm not sure how you could hit this (possibly there's a minute window after init_idle clears the bit, but before it returns?).
M.
Quoted 13 lines · Martin J. Bligh
Quoted 4 lines
I'm working with a (approximately) 2.4.17 kernel from the mips-linux tree (oss.sgi.com).
Quoted 8 lines
I'd like to propose removing the "
__init" designation frominit_idleinkernel/sched.c, since this is called fromrest_initviacpu_idle. Notice thatrest_initisn't in an init section, and explicitly mentions that it's avoiding a race withfree_initmem. In my kernel (an SMP kernel running on a system with only 1 available CPU),cpu_idleisn't getting called until afterfree_initmem().My CPU is MIPS, but it looks like x86 could experience the same problem.
Quoted 7 lines
I fixed something in this area for x86, looks like the same code path for MIPS unless I'm misreading.
smp_initspins waiting onwait_init_idleuntil every cpu has doneinit_idle.rest_init()isn't called untilsmp_initreturns, so I'm not sure how you could hit this (possibly there's a minute window afterinit_idleclears the bit, but before it returns?).
This synchronization doesn't help: cpu0 (even in the multi-cpu case) calls init_idle twice -- once from smp_init (through smp_boot_cpus), and then again from cpu_idle. In my failing case (CONFIG_SMP=y, but only 1 cpu in the system) the second call, the one from cpu_idle, doesn't happen until long after the init kernel thread has been running and has freed the initmem.
Maybe a better fix is to avoid this double calling of init_idle for the "master" CPU? From my reading the code, x86 seems to behave the same.
Kip
Quoted 4 lines · Kip Walker
Maybe a better fix is to avoid this double calling of
init_idlefor the "master" CPU? From my reading the code, x86 seems to behave the same.
Looks to me like the clean fix would be to call init_idle() from rest_init() before the init() thread is spawned, and remove it from cpu_idle(). It looks like a pretty straightforward race condition that no one else has happened to trigger in a bad way. I'm no scheduler pro, but I don't see any problems with calling init_idle() earlier.
That fix assumes that bringup of non-primary cpus on other architectures call init_idle() explicitly before allowing smp_init() to return; this is true of mips, but I can't vouch for any other arch's.
I'd submit a patch, but I'm sadly lacking in SMP machines for testing. Anyone who wants to rectify that, I'm open to charity. :)
-Justin