Skip to contentSkip to conversation

bad patch in sched.c

DISCUSSION Applied Mike Travis 3 participants 5 messages Jan 9, 2009 – Jan 11, 2009

lore
Share & tools
lore
#0 lore raw

It appears that

commit 6c99e9ad47d9c082bd096f42fb49e397b05d58a8

Author: Rusty Russell <[email protected]>
Date:   Tue Nov 25 02:35:04 2008 +1030

sched: convert struct sched_group/sched_domain cpumask_ts to variable bitmaps

Impact: (future) size reduction for large NR_CPUS.

We move the 'cpumask' member of sched_group to the end, so when we kmalloc it we can do a minimal allocation: saves space for small nr_cpu_ids but big CONFIG_NR_CPUS. Similar trick for 'span' in sched_domain.

This isn't quite as good as converting to a cpumask_var_t, as some sched_groups are actually static, but it's safer: we don't have to figure out where to call alloc_cpumask_var/free_cpumask_var.

causes a panic in ia64 with NR_CPUS=1024. Config attached though I think it's the standard config file in:

	//git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6

Version: v2.6.28-5274-g597b0d2

Revert it and the problem goes away.

Thanks!
Mike

Calibrating delay loop... 3325.95 BogoMIPS (lpj=6651904) Brought up 128 CPUs
Total of 128 processors activated (412352.51 BogoMIPS).

Unable to handle kernel paging request at virtual address 0001000000000007
swapper[1]: Oops 8804682956800 [1]
Modules linked in:

Pid: 1, CPU 0, comm: swapper
psr : 00001010085a6018 ifs : 8000000000000691 ip : [<a0000001000959a0>] Not tainted (2.6.28)
ip is at cpu_attach_domain+0x1a0/0x800
unat: 0000000000000000 pfs : 0000000000000691 rsc : 0000000000000003
rnat: 0000000000000000 bsps: 0000000000000000 pr : 0000000000005941
ldrs: 0000000000000000 ccv : 0000000000000000 fpsr: 0009804c8a70433f csd : 0000000000000000 ssd : 0000000000000000
b0 : a000000100095890 b6 : a00000010008f180 b7 : a00000010008f180 f6 : 0fff2fffffffff0000000 f7 : 0fffb8000080000000000 f8 : 1001f800007ff80000000 f9 : 1000b8000000000000000 f10 : 10013800007ff77ffff80 f11 : 1003e0000000000100000 r1 : a000000101192740 r2 : 000000000000028f r3 : 0000000000000400 r8 : 0000000000000001 r9 : 0000000000000000 r10 : 0000000000000000 r11 : 0000000000000000 r12 : e0001160f0d4f820 r13 : e0001160f0d40000 r14 : 0000000000000001 r15 : 0000000000000000 r16 : 0000000000000001 r17 : 0001000000000007 r18 : 0000000000000010 r19 : a000000100eb5028 r20 : a000000101004010 r21 : a000000101004010 r22 : 0000000000000080 r23 : 0000000000000080 r24 : a000000100df4658 r25 : 0000ffffffffffff r26 : a000000101004010 r27 : a000000101004010 r28 : a000000101004010 r29 : 0000000000000080 r30 : a000000100db0080 r31 : a000000100f92c80

Call Trace:
 [<a000000100015e20>] show_stack+0x40/0xa0
                                sp=e0001160f0d4f3f0 bsp=e0001160f0d41008
 [<a000000100016730>] show_regs+0x850/0x8a0
                                sp=e0001160f0d4f5c0 bsp=e0001160f0d40fa8
 [<a000000100039cd0>] die+0x1b0/0x2c0
                                sp=e0001160f0d4f5c0 bsp=e0001160f0d40f60
 [<a00000010005dc30>] ia64_do_page_fault+0x8d0/0xa40
                                sp=e0001160f0d4f5c0 bsp=e0001160f0d40f10
 [<a00000010000c340>] ia64_native_leave_kernel+0x0/0x270
                                sp=e0001160f0d4f650 bsp=e0001160f0d40f10
 [<a0000001000959a0>] cpu_attach_domain+0x1a0/0x800
                                sp=e0001160f0d4f820 bsp=e0001160f0d40e88
 [<a00000010009e400>] __build_sched_domains+0xe20/0xfe0
                                sp=e0001160f0d4f9a0 bsp=e0001160f0d40df0
 [<a000000100b7c8c0>] sched_init_smp+0x1a0/0x640
                                sp=e0001160f0d4fdb0 bsp=e0001160f0d40dc8
 [<a000000100b60680>] kernel_init+0x460/0x5a0
                                sp=e0001160f0d4fe30 bsp=e0001160f0d40da8
 [<a000000100014370>] kernel_thread_helper+0xd0/0x100
                                sp=e0001160f0d4fe30 bsp=e0001160f0d40d80
 [<a00000010000a4c0>] start_kernel_thread+0x20/0x40
                                sp=e0001160f0d4fe30 bsp=e0001160f0d40d80
Kernel panic - not syncing: Attempted to kill init! 
#1 lore raw
Quoted 20 lines · On Saturday 10 January 2009 05:00:28 Mike Travis wrote:

It appears that

commit 6c99e9ad47d9c082bd096f42fb49e397b05d58a8

Author: Rusty Russell <[email protected]>
Date:   Tue Nov 25 02:35:04 2008 +1030

sched: convert struct sched_group/sched_domain cpumask_ts to variable bitmaps

Impact: (future) size reduction for large NR_CPUS.

We move the 'cpumask' member of sched_group to the end, so when we kmalloc it we can do a minimal allocation: saves space for small nr_cpu_ids but big CONFIG_NR_CPUS. Similar trick for 'span' in sched_domain.

This isn't quite as good as converting to a cpumask_var_t, as some sched_groups are actually static, but it's safer: we don't have to figure out where to call alloc_cpumask_var/free_cpumask_var.

Quoted 5 lines

causes a panic in ia64 with NR_CPUS=1024.

Thanks, that focussed me on the right place. Does this fix it?

cpumask: fix CONFIG_NUMA=y sched.c

struct sched_domain is now a dangling structure; where we really want static ones, we need to use static_sched_domain.

(As the FIXME in this file says, cpumask_var_t would be better, but this code is hairy enough without trying to add initialization code to the right places).

kernel/sched.c +5 -5
@@ -7282,10 +7282,10 @@ cpu_to_phys_group(int cpu, const struct 
  * groups, so roll our own. Now each node has its own list of groups which
  * gets dynamically allocated.
  */
-static DEFINE_PER_CPU(struct sched_domain, node_domains);
+static DEFINE_PER_CPU(struct static_sched_domain, node_domains);
 static struct sched_group ***sched_group_nodes_bycpu;
 
-static DEFINE_PER_CPU(struct sched_domain, allnodes_domains);
+static DEFINE_PER_CPU(struct static_sched_domain, allnodes_domains);
 static DEFINE_PER_CPU(struct static_sched_group, sched_group_allnodes);
 
 static int cpu_to_allnodes_group(int cpu, const struct cpumask *cpu_map,
@@ -7560,7 +7560,7 @@ static int __build_sched_domains(const s
 #ifdef CONFIG_NUMA
 		if (cpumask_weight(cpu_map) >
 				SD_NODES_PER_DOMAIN*cpumask_weight(nodemask)) {
-			sd = &per_cpu(allnodes_domains, i);
+			sd = &per_cpu(allnodes_domains, i).sg;
 			SD_INIT(sd, ALLNODES);
 			set_domain_attribute(sd, attr);
 			cpumask_copy(sched_domain_span(sd), cpu_map);
@@ -7570,7 +7570,7 @@ static int __build_sched_domains(const s
 		} else
 			p = NULL;
 
-		sd = &per_cpu(node_domains, i);
+		sd = &per_cpu(node_domains, i).sd;
 		SD_INIT(sd, NODE);
 		set_domain_attribute(sd, attr);
 		sched_domain_node_span(cpu_to_node(i), sched_domain_span(sd));
@@ -7688,7 +7688,7 @@ static int __build_sched_domains(const s
 		for_each_cpu(j, nodemask) {
 			struct sched_domain *sd;
 
-			sd = &per_cpu(node_domains, j);
+			sd = &per_cpu(node_domains, j).sd;
 			sd->groups = sg;
 		}
 		sg->__cpu_power = 0;
#2 lore raw
Quoted 1 line · Rusty Russell

cpumask: fix CONFIG_NUMA=y sched.c

doesnt build:

kernel/sched.c: In function ‘__build_sched_domains’:
kernel/sched.c:7701: error: ‘struct static_sched_domain’ has no member named ‘sg’

Ingo

#3 lore raw
Quoted 10 lines · Ingo Molnar
Quoted 1 line · Rusty Russell

cpumask: fix CONFIG_NUMA=y sched.c

doesnt build:

kernel/sched.c: In function ‘__build_sched_domains’:
kernel/sched.c:7701: error: ‘struct static_sched_domain’ has no member named ‘sg’

Ingo

Hi,

This one does compile and fix it.

Thanks,
Mike
---
cpumask: fix CONFIG_NUMA=y sched.c

struct sched_domain is now a dangling structure; where we really want static ones, we need to use static_sched_domain.

(As the FIXME in this file says, cpumask_var_t would be better, but this code is hairy enough without trying to add initialization code to the right places).

---
 kernel/sched.c |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
kernel/sched.c +5 -5
@@ -7278,10 +7278,10 @@ cpu_to_phys_group(int cpu, const struct 
  * groups, so roll our own. Now each node has its own list of groups which
  * gets dynamically allocated.
  */
-static DEFINE_PER_CPU(struct sched_domain, node_domains);
+static DEFINE_PER_CPU(struct static_sched_domain, node_domains);
 static struct sched_group ***sched_group_nodes_bycpu;
 
-static DEFINE_PER_CPU(struct sched_domain, allnodes_domains);
+static DEFINE_PER_CPU(struct static_sched_domain, allnodes_domains);
 static DEFINE_PER_CPU(struct static_sched_group, sched_group_allnodes);
 
 static int cpu_to_allnodes_group(int cpu, const struct cpumask *cpu_map,
@@ -7556,7 +7556,7 @@ static int __build_sched_domains(const s
 #ifdef CONFIG_NUMA
 		if (cpumask_weight(cpu_map) >
 				SD_NODES_PER_DOMAIN*cpumask_weight(nodemask)) {
-			sd = &per_cpu(allnodes_domains, i);
+			sd = &per_cpu(allnodes_domains, i).sd;
 			SD_INIT(sd, ALLNODES);
 			set_domain_attribute(sd, attr);
 			cpumask_copy(sched_domain_span(sd), cpu_map);
@@ -7566,7 +7566,7 @@ static int __build_sched_domains(const s
 		} else
 			p = NULL;
 
-		sd = &per_cpu(node_domains, i);
+		sd = &per_cpu(node_domains, i).sd;
 		SD_INIT(sd, NODE);
 		set_domain_attribute(sd, attr);
 		sched_domain_node_span(cpu_to_node(i), sched_domain_span(sd));
@@ -7684,7 +7684,7 @@ static int __build_sched_domains(const s
 		for_each_cpu(j, nodemask) {
 			struct sched_domain *sd;
 
-			sd = &per_cpu(node_domains, j);
+			sd = &per_cpu(node_domains, j).sd;
 			sd->groups = sg;
 		}
 		sg->__cpu_power = 0;
Applied
#4 lore raw
Quoted 7 lines · Mike Travis
Quoted 6 lines · Ingo Molnar
Quoted 1 line · Rusty Russell

cpumask: fix CONFIG_NUMA=y sched.c

doesnt build:

Quoted 7 lines
Quoted 4 lines

kernel/sched.c: In function ‘__build_sched_domains’:
kernel/sched.c:7701: error: ‘struct static_sched_domain’ has no member named ‘sg’

Ingo

Hi,

This one does compile and fix it.

applied to tip/sched/urgent, thanks Mike!

Ingo