bad patch in sched.c
It appears that
commit 6c99e9ad47d9c082bd096f42fb49e397b05d58a8
Author: Rusty Russell <[email protected]> Date: Tue Nov 25 02:35:04 2008 +1030
sched: convert struct sched_group/sched_domain cpumask_ts to variable bitmaps
Impact: (future) size reduction for large NR_CPUS.
We move the 'cpumask' member of sched_group to the end, so when we kmalloc it we can do a minimal allocation: saves space for small nr_cpu_ids but big CONFIG_NR_CPUS. Similar trick for 'span' in sched_domain.
This isn't quite as good as converting to a cpumask_var_t, as some sched_groups are actually static, but it's safer: we don't have to figure out where to call alloc_cpumask_var/free_cpumask_var.
causes a panic in ia64 with NR_CPUS=1024. Config attached though I think it's the standard config file in:
//git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
Version: v2.6.28-5274-g597b0d2
Revert it and the problem goes away.
Thanks!
Mike
Calibrating delay loop... 3325.95 BogoMIPS (lpj=6651904) Brought up 128 CPUs
Total of 128 processors activated (412352.51 BogoMIPS).
Unable to handle kernel paging request at virtual address 0001000000000007 swapper[1]: Oops 8804682956800 [1] Modules linked in:
Pid: 1, CPU 0, comm: swapper
psr : 00001010085a6018 ifs : 8000000000000691 ip : [<a0000001000959a0>] Not tainted (2.6.28)
ip is at cpu_attach_domain+0x1a0/0x800
unat: 0000000000000000 pfs : 0000000000000691 rsc : 0000000000000003
rnat: 0000000000000000 bsps: 0000000000000000 pr : 0000000000005941
ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c8a70433f csd : 0000000000000000 ssd : 0000000000000000
b0 : a000000100095890 b6 : a00000010008f180 b7 : a00000010008f180 f6 : 0fff2fffffffff0000000 f7 : 0fffb8000080000000000 f8 : 1001f800007ff80000000 f9 : 1000b8000000000000000 f10 : 10013800007ff77ffff80 f11 : 1003e0000000000100000 r1 : a000000101192740 r2 : 000000000000028f r3 : 0000000000000400 r8 : 0000000000000001 r9 : 0000000000000000 r10 : 0000000000000000 r11 : 0000000000000000 r12 : e0001160f0d4f820 r13 : e0001160f0d40000 r14 : 0000000000000001 r15 : 0000000000000000 r16 : 0000000000000001 r17 : 0001000000000007 r18 : 0000000000000010 r19 : a000000100eb5028 r20 : a000000101004010 r21 : a000000101004010 r22 : 0000000000000080 r23 : 0000000000000080 r24 : a000000100df4658 r25 : 0000ffffffffffff r26 : a000000101004010 r27 : a000000101004010 r28 : a000000101004010 r29 : 0000000000000080 r30 : a000000100db0080 r31 : a000000100f92c80
Call Trace:
[<a000000100015e20>] show_stack+0x40/0xa0
sp=e0001160f0d4f3f0 bsp=e0001160f0d41008
[<a000000100016730>] show_regs+0x850/0x8a0
sp=e0001160f0d4f5c0 bsp=e0001160f0d40fa8
[<a000000100039cd0>] die+0x1b0/0x2c0
sp=e0001160f0d4f5c0 bsp=e0001160f0d40f60
[<a00000010005dc30>] ia64_do_page_fault+0x8d0/0xa40
sp=e0001160f0d4f5c0 bsp=e0001160f0d40f10
[<a00000010000c340>] ia64_native_leave_kernel+0x0/0x270
sp=e0001160f0d4f650 bsp=e0001160f0d40f10
[<a0000001000959a0>] cpu_attach_domain+0x1a0/0x800
sp=e0001160f0d4f820 bsp=e0001160f0d40e88
[<a00000010009e400>] __build_sched_domains+0xe20/0xfe0
sp=e0001160f0d4f9a0 bsp=e0001160f0d40df0
[<a000000100b7c8c0>] sched_init_smp+0x1a0/0x640
sp=e0001160f0d4fdb0 bsp=e0001160f0d40dc8
[<a000000100b60680>] kernel_init+0x460/0x5a0
sp=e0001160f0d4fe30 bsp=e0001160f0d40da8
[<a000000100014370>] kernel_thread_helper+0xd0/0x100
sp=e0001160f0d4fe30 bsp=e0001160f0d40d80
[<a00000010000a4c0>] start_kernel_thread+0x20/0x40
sp=e0001160f0d4fe30 bsp=e0001160f0d40d80
Kernel panic - not syncing: Attempted to kill init! Quoted 20 lines · On Saturday 10 January 2009 05:00:28 Mike Travis wrote:
It appears that
commit
6c99e9ad47d9c082bd096f42fb49e397b05d58a8Author: Rusty Russell <[email protected]> Date: Tue Nov 25 02:35:04 2008 +1030sched: convert struct sched_group/sched_domain
cpumask_tsto variable bitmapsImpact: (future) size reduction for large
NR_CPUS.We move the 'cpumask' member of
sched_groupto the end, so when we kmalloc it we can do a minimal allocation: saves space for smallnr_cpu_idsbut bigCONFIG_NR_CPUS. Similar trick for 'span' insched_domain.This isn't quite as good as converting to a
cpumask_var_t, as somesched_groupsare actually static, but it's safer: we don't have to figure out where to call alloc_cpumask_var/free_cpumask_var.
Quoted 5 lines
Signed-off-by Rusty Russell <[email protected]>Signed-off-by Ingo Molnar <[email protected]>causes a panic in ia64 with NR_CPUS=1024.
Thanks, that focussed me on the right place. Does this fix it?
cpumask: fix CONFIG_NUMA=y sched.c
struct sched_domain is now a dangling structure; where we really want static ones, we need to use static_sched_domain.
(As the FIXME in this file says, cpumask_var_t would be better, but this code is hairy enough without trying to add initialization code to the right places).
kernel/sched.c +5 -5
@@ -7282,10 +7282,10 @@ cpu_to_phys_group(int cpu, const struct* groups, so roll our own. Now each node has its own list of groups which * gets dynamically allocated. */ -static DEFINE_PER_CPU(struct sched_domain, node_domains); +static DEFINE_PER_CPU(struct static_sched_domain, node_domains);static struct sched_group ***sched_group_nodes_bycpu; -static DEFINE_PER_CPU(struct sched_domain, allnodes_domains); +static DEFINE_PER_CPU(struct static_sched_domain, allnodes_domains); static DEFINE_PER_CPU(struct static_sched_group, sched_group_allnodes); static int cpu_to_allnodes_group(int cpu, const struct cpumask *cpu_map,
@@ -7560,7 +7560,7 @@ static int __build_sched_domains(const s#ifdef CONFIG_NUMA if (cpumask_weight (cpu_map) >SD_NODES_PER_DOMAIN *cpumask_weight (nodemask)) { - sd = &per_cpu (allnodes_domains, i); + sd = &per_cpu (allnodes_domains, i).sg;SD_INIT (sd,ALLNODES );set_domain_attribute (sd, attr);cpumask_copy (sched_domain_span (sd), cpu_map);
@@ -7570,7 +7570,7 @@ static int __build_sched_domains(const s }else p =NULL ; - sd = &per_cpu (node_domains, i); + sd = &per_cpu (node_domains, i).sd;SD_INIT (sd,NODE );set_domain_attribute (sd, attr);sched_domain_node_span (cpu_to_node (i),sched_domain_span (sd));
@@ -7688,7 +7688,7 @@ static int __build_sched_domains(const sfor_each_cpu (j, nodemask) {struct sched_domain *sd; - sd = &per_cpu (node_domains, j); + sd = &per_cpu (node_domains, j).sd; sd->groups = sg; } sg->__cpu_power =0 ;
Quoted 1 line · Rusty Russell
cpumask: fix
CONFIG_NUMA=ysched.c
doesnt build:
kernel/sched.c: In function ‘__build_sched_domains’:kernel/sched.c:7701: error: ‘struct static_sched_domain’ has no member named ‘sg’
Ingo
Quoted 10 lines · Ingo Molnar
Quoted 1 line · Rusty Russell
cpumask: fix
CONFIG_NUMA=ysched.cdoesnt build:
kernel/sched.c: In function ‘__build_sched_domains’:kernel/sched.c:7701: error: ‘struct static_sched_domain’ has no member named ‘sg’Ingo
Hi,
This one does compile and fix it.
Thanks,
Mike
---
cpumask: fix CONFIG_NUMA=y sched.c
struct sched_domain is now a dangling structure; where we really want static ones, we need to use static_sched_domain.
(As the FIXME in this file says, cpumask_var_t would be better, but this code is hairy enough without trying to add initialization code to the right places).
--- kernel/sched.c | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
kernel/sched.c +5 -5
@@ -7278,10 +7278,10 @@ cpu_to_phys_group(int cpu, const struct* groups, so roll our own. Now each node has its own list of groups which * gets dynamically allocated. */ -static DEFINE_PER_CPU(struct sched_domain, node_domains); +static DEFINE_PER_CPU(struct static_sched_domain, node_domains);static struct sched_group ***sched_group_nodes_bycpu; -static DEFINE_PER_CPU(struct sched_domain, allnodes_domains); +static DEFINE_PER_CPU(struct static_sched_domain, allnodes_domains); static DEFINE_PER_CPU(struct static_sched_group, sched_group_allnodes); static int cpu_to_allnodes_group(int cpu, const struct cpumask *cpu_map,
@@ -7556,7 +7556,7 @@ static int __build_sched_domains(const s#ifdef CONFIG_NUMA if (cpumask_weight (cpu_map) >SD_NODES_PER_DOMAIN *cpumask_weight (nodemask)) { - sd = &per_cpu (allnodes_domains, i); + sd = &per_cpu (allnodes_domains, i).sd;SD_INIT (sd,ALLNODES );set_domain_attribute (sd, attr);cpumask_copy (sched_domain_span (sd), cpu_map);
@@ -7566,7 +7566,7 @@ static int __build_sched_domains(const s }else p =NULL ; - sd = &per_cpu (node_domains, i); + sd = &per_cpu (node_domains, i).sd;SD_INIT (sd,NODE );set_domain_attribute (sd, attr);sched_domain_node_span (cpu_to_node (i),sched_domain_span (sd));
@@ -7684,7 +7684,7 @@ static int __build_sched_domains(const sfor_each_cpu (j, nodemask) {struct sched_domain *sd; - sd = &per_cpu (node_domains, j); + sd = &per_cpu (node_domains, j).sd; sd->groups = sg; } sg->__cpu_power =0 ;
Ingo Molnar
AppliedQuoted 7 lines · Mike Travis
Quoted 6 lines · Ingo Molnar
Quoted 1 line · Rusty Russell
cpumask: fix
CONFIG_NUMA=ysched.cdoesnt build:
Quoted 7 lines
Quoted 4 lines
kernel/sched.c: In function ‘__build_sched_domains’:kernel/sched.c:7701: error: ‘struct static_sched_domain’ has no member named ‘sg’Ingo
Hi,
This one does compile and fix it.
applied to tip/sched/urgent, thanks Mike!
Ingo