failed to boot on IA64 with CONFIG_DISCONTIGMEM=y

memcg BUG ? Li Zefan 3 participants 7 messages Dec 5, 2008 – Dec 7, 2008 Memory management

lore

Share & tools

lore

Li Zefan

author

2008-12-05 03:09 UTC #0 lore raw

Kernel version: 2.6.28-rc7
Arch: IA64
Memory model: DISCONTIGMEM

ELILO boot: Uncompressing Linux... done
Loading file initrd-2.6.28-rc7-lizf.img...done
(frozen)

Booted successfully with cgroup_disable=memory, here is the dmesg:

...
Number of logical nodes in system = 16
Number of memory chunks in system = 18
SMP: Allowing 128 CPUs, 120 hotplug CPUs
Initial ramdisk at: 0xe00000006afbc000 (4644000 bytes)
SAL 3.20: FUJITSU LIMITED PRIMEQUEST version 3.4
SAL Platform features: None
SAL: AP wakeup using external interrupt vector 0xf0
ACPI: Local APIC address c0000000fee00000
PLATFORM int CPEI (0x3): GSI 22 (level, low) -> CPU 0 (0x0000) vector 30 8 CPUs available, 128 CPUs total
MCA related initialization done
Virtual mem_map starts at 0xa07fffffc0c80000
Zone PFN ranges:
DMA 0x00000100 -> 0x00010000
Normal 0x00010000 -> 0x01210000
Movable zone start PFN for each node
early_node_map[3] active PFN ranges

    0: 0x00000100 -> 0x00006d00
    0: 0x00408000 -> 0x00410000
    1: 0x01200400 -> 0x01210000

On node 0 totalpages: 60416
free_area_init_node: node 0, pgdat e0000000011c0000, node_mem_map a07fffffc0c83800
DMA zone: 56 pages used for memmap
DMA zone: 0 pages reserved
DMA zone: 27592 pages, LIFO batch:1
Normal zone: 3584 pages used for memmap
Normal zone: 29184 pages, LIFO batch:1
Movable zone: 0 pages used for memmap
On node 1 totalpages: 64512
free_area_init_node: node 1, pgdat e0000120040d0080, node_mem_map a07fffffffc8e000
DMA zone: 0 pages used for memmap
Normal zone: 56 pages used for memmap
Normal zone: 64456 pages, LIFO batch:1
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 2
On node 2 totalpages: 0
free_area_init_node: node 2, pgdat e000004080180100, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 3
On node 3 totalpages: 0
free_area_init_node: node 3, pgdat e0000040802d0180, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 4
On node 4 totalpages: 0
free_area_init_node: node 4, pgdat e0000040804a0200, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 5
On node 5 totalpages: 0
free_area_init_node: node 5, pgdat e0000040805f0280, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 6
On node 6 totalpages: 0
free_area_init_node: node 6, pgdat e0000040807c0300, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 7
On node 7 totalpages: 0
free_area_init_node: node 7, pgdat e000004080910380, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 8
On node 8 totalpages: 0
free_area_init_node: node 8, pgdat e000004080ad0400, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 9
On node 9 totalpages: 0
free_area_init_node: node 9, pgdat e000004080c10480, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 10
On node 10 totalpages: 0
free_area_init_node: node 10, pgdat e000004080dd0500, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 11
On node 11 totalpages: 0
free_area_init_node: node 11, pgdat e000004080f10580, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 12
On node 12 totalpages: 0
free_area_init_node: node 12, pgdat e0000040810d0600, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 13
On node 13 totalpages: 0
free_area_init_node: node 13, pgdat e000004081210680, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 14
On node 14 totalpages: 0
free_area_init_node: node 14, pgdat e0000040813d0700, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Could not find start_pfn for node 15
On node 15 totalpages: 0
free_area_init_node: node 15, pgdat e000004081510780, node_mem_map a07fffffc0c80000
DMA zone: 0 pages used for memmap
Normal zone: 0 pages used for memmap
Movable zone: 0 pages used for memmap
Built 16 zonelists in Zone order, mobility grouping on. Total pages: 121232
Policy zone: Normal
Kernel command line: BOOT_IMAGE=scsi0:EFI\redhat\vmlinuz-2.6.28-rc7-lizf console=ttyS0 rhgb cgroup_disable=memory root=/dev/sda2 ro Disabling memory control group subsystem
PID hash table entries: 4096 (order: 12, 32768 bytes)
CPU 0: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz
Console: colour VGA+ 80x25
Placing software IO TLB between 0x53d0000 - 0x93d0000
Memory: 7870144k/7970368k available (6617k code, 125248k reserved, 5996k data, 1728k init)
SLUB: Genslabs=17, HWalign=128, Order=0-3, MinObjects=0, CPUs=128, Nodes=1024
Calibrating delay loop... 3186.68 BogoMIPS (lpj=1593344)
Security Framework initialized

SELinux:  Initializing.
SELinux:  Starting in permissive mode
Dentry cache hash table entries: 1048576 (order: 7, 8388608 bytes)
Inode-cache hash table entries: 524288 (order: 6, 4194304 bytes)
Mount-cache hash table entries: 4096

Initializing cgroup subsys debug
Initializing cgroup subsys ns
Initializing cgroup subsys cpuacct
Initializing cgroup subsys memory
Initializing cgroup subsys devices
Initializing cgroup subsys freezer
ACPI: Core revision 20080926
Boot processor id 0x0/0x0
Fixed BSP b0 value from CPU 1
CPU 1: synchronized ITC with CPU 0 (last diff 1 cycles, maxerr 139 cycles) CPU 1: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz CPU 2: synchronized ITC with CPU 0 (last diff -1 cycles, maxerr 139 cycles) CPU 2: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz CPU 3: synchronized ITC with CPU 0 (last diff 1 cycles, maxerr 139 cycles) CPU 3: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz CPU 4: synchronized ITC with CPU 0 (last diff 0 cycles, maxerr 512 cycles) CPU 4: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz CPU 5: synchronized ITC with CPU 0 (last diff 0 cycles, maxerr 513 cycles) CPU 5: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz CPU 6: synchronized ITC with CPU 0 (last diff 0 cycles, maxerr 512 cycles) CPU 6: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz CPU 7: synchronized ITC with CPU 0 (last diff 0 cycles, maxerr 512 cycles) CPU 7: base freq=266.666MHz, ITC ratio=6/4, ITC freq=399.999MHz Brought up 8 CPUs
Total of 8 processors activated (25493.50 BogoMIPS).
...

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

KAMEZAWA Hiroyuki

2008-12-05 03:21 UTC #1 lore raw

Quoted 11 lines · On Fri, 05 Dec 2008 11:09:29 +0800 Li Zefan <[email protected]> wrote:

Kernel version: 2.6.28-rc7
Arch: IA64
Memory model: DISCONTIGMEM
ELILO boot: Uncompressing Linux... done
Loading file initrd-2.6.28-rc7-lizf.img...done
(frozen)
Booted successfully with cgroup_disable=memory, here is the dmesg:

thx, will dig into...Maybe you're the first person using DISCONTIGMEM with empty_node after page_cgroup-alloc-at-boot.

How about this ?
-Kame
==

From: [email protected]

page_cgroup should ignore empty-nodes.

Signed-off-by KAMEZAWA Hiroyuki <[email protected]>

---
 mm/page_cgroup.c |    3 +++
 1 file changed, 3 insertions(+)

mm/page_cgroup.c +3 -0

Index: mmotm-2.6.28-Dec03/mm/page_cgroup.c
===================================================================
@@ -51,6 +51,9 @@ static int __init alloc_node_page_cgroup
 	start_pfn = NODE_DATA(nid)->node_start_pfn;
 	nr_pages = NODE_DATA(nid)->node_spanned_pages;
 
+	if (!nr_pages)
+		return;
+
 	table_size = sizeof(struct page_cgroup) * nr_pages;
 
 	base = __alloc_bootmem_node_nopanic(NODE_DATA(nid),

KAMEZAWA Hiroyuki

2008-12-05 03:25 UTC #2 lore raw

Quoted 12 lines · On Fri, 5 Dec 2008 12:20:24 +0900 KAMEZAWA Hiroyuki…

Quoted 9 lines · On Fri, 05 Dec 2008 11:09:29 +0800 Li Zefan <[email protected]> wrote:
Kernel version: 2.6.28-rc7
Arch: IA64
Memory model: DISCONTIGMEM
ELILO boot: Uncompressing Linux... done
Loading file initrd-2.6.28-rc7-lizf.img...done
(frozen)

Quoted 7 lines

Quoted 2 lines
Booted successfully with cgroup_disable=memory, here is the dmesg:
thx, will dig into...Maybe you're the first person using DISCONTIGMEM with empty_node after page_cgroup-alloc-at-boot.
How about this ?

Ahhh..sorry.

this one please.
==

From: [email protected]

page_cgroup should ignore empty-nodes.

Signed-off-by KAMEZAWA Hiroyuki <[email protected]>

---
 mm/page_cgroup.c |    3 +++
 1 file changed, 3 insertions(+)

mm/page_cgroup.c +3 -0

Index: mmotm-2.6.28-Dec03/mm/page_cgroup.c
===================================================================
@@ -51,6 +51,9 @@ static int __init alloc_node_page_cgroup
 	start_pfn = NODE_DATA(nid)->node_start_pfn;
 	nr_pages = NODE_DATA(nid)->node_spanned_pages;
 
+	if (!nr_pages)
+		return 0;
+
 	table_size = sizeof(struct page_cgroup) * nr_pages;
 
 	base = __alloc_bootmem_node_nopanic(NODE_DATA(nid),

Li Zefan

author Tested-by

2008-12-05 03:31 UTC #3 lore raw

Quoted 15 lines · KAMEZAWA Hiroyuki

Quoted 12 lines · On Fri, 5 Dec 2008 12:20:24 +0900 KAMEZAWA Hiroyuki…
Quoted 9 lines · On Fri, 05 Dec 2008 11:09:29 +0800 Li Zefan <[email protected]> wrote:
Kernel version: 2.6.28-rc7
Arch: IA64
Memory model: DISCONTIGMEM
ELILO boot: Uncompressing Linux... done
Loading file initrd-2.6.28-rc7-lizf.img...done
(frozen)

Quoted 5 lines

Quoted 5 lines
Quoted 2 lines
Booted successfully with cgroup_disable=memory, here is the dmesg:
thx, will dig into...Maybe you're the first person using DISCONTIGMEM with empty_node after page_cgroup-alloc-at-boot.

I was reading the code in page_cgroup.c, and It came to my mind that maybe no one ever tried DISCONTIGMEM+memcg, so I had it a try..

Quoted 4 lines

Quoted 1 line
How about this ?
Ahhh..sorry.

Quoted 9 lines

this one please.
==
From: [email protected]
page_cgroup should ignore empty-nodes.
Signed-off-by KAMEZAWA Hiroyuki <[email protected]>

Now it booted successfully. :)

Tested-by Li Zefan <[email protected]>

Quoted 11 lines

---
 mm/page_cgroup.c |    3 +++
 1 file changed, 3 insertions(+)
Index: mmotm-2.6.28-Dec03/mm/page_cgroup.c
===================================================================

mm/page_cgroup.c static int __init alloc_node_page_cgroup · line 59

Quoted 9 lines

mm/page_cgroup.c +3 -0

@@ -51,6 +51,9 @@ static int __init alloc_node_page_cgroup
 	start_pfn = NODE_DATA(nid)->node_start_pfn;
 	nr_pages = NODE_DATA(nid)->node_spanned_pages;
 
+	if (!nr_pages)
+		return 0;
+
 	table_size = sizeof(struct page_cgroup) * nr_pages;
 
 	base = __alloc_bootmem_node_nopanic(NODE_DATA(nid),

KAMEZAWA Hiroyuki

2008-12-05 03:50 UTC #4 lore raw

Quoted 9 lines · On Fri, 05 Dec 2008 11:31:20 +0800 Li Zefan <[email protected]> wrote:

Quoted 9 lines
Quoted 1 line
How about this ?
Ahhh..sorry.
this one please.
==
From: [email protected]

Quoted 9 lines

Quoted 4 lines
page_cgroup should ignore empty-nodes.
Signed-off-by KAMEZAWA Hiroyuki <[email protected]>
Now it booted successfully. :)
Tested-by Li Zefan <[email protected]>

Thank you!

-Kame

Andrew Morton

maintainer

2008-12-07 08:31 UTC #5 lore raw

In reply to KAMEZAWA Hiroyuki

Quoted 43 lines · On Fri, 5 Dec 2008 12:24:58 +0900 KAMEZAWA Hiroyuki…

Quoted 12 lines · On Fri, 5 Dec 2008 12:20:24 +0900 KAMEZAWA Hiroyuki…
Quoted 9 lines · On Fri, 05 Dec 2008 11:09:29 +0800 Li Zefan <[email protected]> wrote:
Kernel version: 2.6.28-rc7
Arch: IA64
Memory model: DISCONTIGMEM
ELILO boot: Uncompressing Linux... done
Loading file initrd-2.6.28-rc7-lizf.img...done
(frozen)
Quoted 7 lines
Quoted 2 lines
Booted successfully with cgroup_disable=memory, here is the dmesg:
thx, will dig into...Maybe you're the first person using DISCONTIGMEM with empty_node after page_cgroup-alloc-at-boot.
How about this ?
Ahhh..sorry.
this one please.
==
From: [email protected]
page_cgroup should ignore empty-nodes.
Signed-off-by KAMEZAWA Hiroyuki <[email protected]>
---
 mm/page_cgroup.c |    3 +++
 1 file changed, 3 insertions(+)
Index: mmotm-2.6.28-Dec03/mm/page_cgroup.c
===================================================================

mm/page_cgroup.c static int __init alloc_node_page_cgroup · line 59

Quoted 9 lines

mm/page_cgroup.c +3 -0

@@ -51,6 +51,9 @@ static int __init alloc_node_page_cgroup
 	start_pfn = NODE_DATA(nid)->node_start_pfn;
 	nr_pages = NODE_DATA(nid)->node_spanned_pages;
 
+	if (!nr_pages)
+		return 0;
+
 	table_size = sizeof(struct page_cgroup) * nr_pages;
 
 	base = __alloc_bootmem_node_nopanic(NODE_DATA(nid),

Why did the kernel fail?

Either __alloc_bootmem_node_nopanic() succeeds, in which case the code looks like it handles that OK.

Or __alloc_bootmem_node_nopanic() fails this zero-sized allocation, and the code attempts to handle that, but fails to do so, which might be a bug, and the above patch just papers over it.

Of course, a full description of the problem will clear all this up. Better changelogs, please.

KAMEZAWA Hiroyuki

2008-12-07 12:34 UTC #6 lore raw

Quoted 59 lines · Andrew Morton

Quoted 44 lines · On Fri, 5 Dec 2008 12:24:58 +0900 KAMEZAWA Hiroyuki…
Quoted 12 lines · On Fri, 5 Dec 2008 12:20:24 +0900 KAMEZAWA Hiroyuki…
Quoted 9 lines · On Fri, 05 Dec 2008 11:09:29 +0800 Li Zefan <[email protected]> wrote:
Kernel version: 2.6.28-rc7
Arch: IA64
Memory model: DISCONTIGMEM
ELILO boot: Uncompressing Linux... done
Loading file initrd-2.6.28-rc7-lizf.img...done
(frozen)
Quoted 4 lines
Quoted 2 lines
Booted successfully with cgroup_disable=memory, here is the dmesg:
thx, will dig into...Maybe you're the first person using DISCONTIGMEM
with
Quoted 3 lines
empty_node after page_cgroup-alloc-at-boot.
How about this ?
Ahhh..sorry.
this one please.
==
From: [email protected]
page_cgroup should ignore empty-nodes.
Signed-off-by KAMEZAWA Hiroyuki <[email protected]>
---
 mm/page_cgroup.c |    3 +++
 1 file changed, 3 insertions(+)
Index: mmotm-2.6.28-Dec03/mm/page_cgroup.c
===================================================================
mm/page_cgroup.c static int __init alloc_node_page_cgroup · line 59
Quoted 9 lines
mm/page_cgroup.c +3 -0
@@ -51,6 +51,9 @@ static int __init alloc_node_page_cgroup
 	start_pfn = NODE_DATA(nid)->node_start_pfn;
 	nr_pages = NODE_DATA(nid)->node_spanned_pages;
 
+	if (!nr_pages)
+		return 0;
+
 	table_size = sizeof(struct page_cgroup) * nr_pages;
 
 	base = __alloc_bootmem_node_nopanic(NODE_DATA(nid),
Why did the kernel fail?

Quoted 7 lines

Either __alloc_bootmem_node_nopanic() succeeds, in which case the code looks like it handles that OK.
Or __alloc_bootmem_node_nopanic() fails this zero-sized allocation, and the code attempts to handle that, but fails to do so, which might be a bug, and the above patch just papers over it.

After this, print a message like "memcg cannot allocate memory, please try cgroup_disable=memory as boot option", and panic.
This is a boot time failure which can be avoided by boot option.

Quoted 3 lines

Of course, a full description of the problem will clear all this up. Better changelogs, please.

will do.

-Kame