Fix crash with FLAT_MEMORY and ARCH_PFN_OFFSET != 0
Subsystems (3)
Thomas Bogendoerfer
maintainerPATCH 1/1 Fix crash with FLAT_MEMORY and ARCH_PFN_OFFSET != 0
When using FLAT_MEMORY and ARCH_PFN_OFFSET is not 0, the kernel crashes in memmap_init_zone(). This bug got introduced by commit c713216deebd95d2b0ab38fef8bb2361c0180c2d
---
mm/page_alloc.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
mm/page_alloc.c +1 -1
@@ -3427,7 +3427,7 @@ static void __init_refok alloc_node_mem_map(struct pglist_data *pgdat) mem_map =NODE_DATA (0 )->node_mem_map;#ifdef CONFIG_ARCH_POPULATES_NODE_MAP if (page_to_pfn (mem_map) != pgdat->node_start_pfn) - mem_map -= pgdat->node_start_pfn; + mem_map -= (pgdat->node_start_pfn -ARCH_PFN_OFFSET ); #endif /* CONFIG_ARCH_POPULATES_NODE_MAP */ }#endif
Andrew Morton
maintainerQuoted 15 lines · On Tue, 18 Dec 2007 13:03:00 +0100 (CET) Thomas Bogendoerfer…
When using
FLAT_MEMORYandARCH_PFN_OFFSETis not 0, the kernel crashes inmemmap_init_zone(). This bug got introduced by commitc713216deebd95d2b0ab38fef8bb2361c0180c2dSigned-off-by Thomas Bogendoerfer <[email protected]>---
mm/page_alloc.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)mm/page_alloc.c +0
-0
mm/page_alloc.c static void __init_refok alloc_node_mem_map(struct pglist_data *pgdat) · line 3433
Quoted 8 lines
mm/page_alloc.c +1
-1@@ -3427,7 +3427,7 @@ static void __init_refok alloc_node_mem_map(struct pglist_data *pgdat) mem_map =NODE_DATA (0 )->node_mem_map;#ifdef CONFIG_ARCH_POPULATES_NODE_MAP if (page_to_pfn (mem_map) != pgdat->node_start_pfn) - mem_map -= pgdat->node_start_pfn; + mem_map -= (pgdat->node_start_pfn -ARCH_PFN_OFFSET ); #endif /* CONFIG_ARCH_POPULATES_NODE_MAP */ }#endif
hm. It's rather non-trivial to verify that this will compile OK on all architectures.
Thomas Bogendoerfer
maintainerQuoted 15 lines · Andrew Morton
mm/page_alloc.c static void __init_refok alloc_node_mem_map(struct pglist_data *pgdat) · line 3433
Quoted 11 lines · On Tue, 18 Dec 2007 13:03:00 +0100 (CET) Thomas Bogendoerfer…
mm/page_alloc.c +1
-1@@ -3427,7 +3427,7 @@ static void __init_refok alloc_node_mem_map(struct pglist_data *pgdat) mem_map =NODE_DATA (0 )->node_mem_map;#ifdef CONFIG_ARCH_POPULATES_NODE_MAP if (page_to_pfn (mem_map) != pgdat->node_start_pfn) - mem_map -= pgdat->node_start_pfn; + mem_map -= (pgdat->node_start_pfn -ARCH_PFN_OFFSET ); #endif /* CONFIG_ARCH_POPULATES_NODE_MAP */ }#endif hm. It's rather non-trivial to verify that this will compile OK on all architectures.
I grepped through include/asm-* and found:
#if defined(CONFIG_FLATMEM)
#ifndef ARCH_PFN_OFFSET
#define ARCH_PFN_OFFSET (0UL)
#endif
in include/asm-generic/memory_model.h. So I'm still confident, that this works on all architectures. It's tested on MIPS.
Thomas.
Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessary a good idea. [ RFC1925, 2.3 ]
On (18/12/07 13:03), Thomas Bogendoerfer didst pronounce:
Quoted 4 lines
When using
FLAT_MEMORYandARCH_PFN_OFFSETis not 0, the kernel crashes inmemmap_init_zone(). This bug got introduced by commitc713216deebd95d2b0ab38fef8bb2361c0180c2d
That commit is over a year old and it initially distressed me that it would take this long to show up on a boot test. However, you said this was to fix MIPS in a follow-on mail and I never made it use arch-independent zone-sizing which is where CONFIG_ARCH_POPULATES_NODE_MAP comes from. Support for arch-independent zone-sizing was added for MIPS on Nov 3rd 2007 by commit cce335ae47e231398269fb05fa48e0e9cbf289e0 (relevant cc added) which is a much more reasonable timeframe for catching this sort of problem. I believe the real bug might be in there.
Can you post a full dmesg log with loglevel=8 so I can see what the layout looks like? My aim is to find out why the start of your mem_map is not at the start of node 0's mem_map which is what I generally expected with the FLATMEM model.
Thanks.
Quoted 20 lines
Signed-off-by Thomas Bogendoerfer <[email protected]>---
mm/page_alloc.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)mm/page_alloc.c +1
-1@@ -3427,7 +3427,7 @@ static void __init_refok alloc_node_mem_map(struct pglist_data *pgdat) mem_map =NODE_DATA (0 )->node_mem_map;#ifdef CONFIG_ARCH_POPULATES_NODE_MAP if (page_to_pfn (mem_map) != pgdat->node_start_pfn) - mem_map -= pgdat->node_start_pfn; + mem_map -= (pgdat->node_start_pfn -ARCH_PFN_OFFSET ); #endif /* CONFIG_ARCH_POPULATES_NODE_MAP */ }#endif
Signature · 3 lines
Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab
Thomas Bogendoerfer
maintainerQuoted 8 lines · Mel Gorman
That commit is over a year old and it initially distressed me that it would take this long to show up on a boot test. However, you said this was to fix MIPS in a follow-on mail and I never made it use arch-independent zone-sizing which is where
CONFIG_ARCH_POPULATES_NODE_MAPcomes from. Support for arch-independent zone-sizing was added for MIPS on Nov 3rd 2007 by commitcce335ae47e231398269fb05fa48e0e9cbf289e0(relevant cc added) which is a much more reasonable timeframe for catching this sort of problem. I believe the real bug might be in there.
either that or it got uncovered. But I'm not too deep into that part of the kernel at the moment.
Quoted 5 lines
Can you post a full dmesg log with loglevel=8 so I can see what the layout looks like? My aim is to find out why the start of your
mem_mapis not at the start of node 0'smem_mapwhich is what I generally expected with the FLATMEM model.
Linux version 2.6.24-rc5-g4b1e2593-dirty ([email protected]) (gcc version 4.2.1) #170 Tue Dec 18 17:03:53 CET 2007
ARCH: SGI-IP28
PROMLIB: ARC firmware Version 64 Revision 0
console [early0] enabled
CPU revision is: 00000925 (R10000)
FPU revision is: 00000900
MC: SGI memory controller Revision 5
MC: Probing memory configuration:
bank0: 256M @ 20000000
bank1: 128M @ 30000000
bank2: 128M @ 38000000
Determined physical RAM map:
memory: 0000000020000000 @ 0000000020000000 (usable)
Entering add_active_range(0, 131946, 262144) 0 entries of 256 used
Zone PFN ranges:
Normal 131946 -> 262144
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
0: 131946 -> 262144
On node 0 totalpages: 130198
Normal zone: 1780 pages used for memmap
Normal zone: 0 pages reserved
Normal zone: 128418 pages, LIFO batch:31
Movable zone: 0 pages used for memmap
Built 1 zonelists in Zone order, mobility grouping on. Total pages: 128418
Kernel command line: root=scsi(0)disk(1)rdisk(0)partition(0) loglevel=8 ip=dhcp Primary instruction cache 32kB, VIPT, 2-way, linesize 64 bytes.
Primary data cache 32kB, 2-way, VIPT, no aliases, linesize 32 bytes
Unified secondary cache 1024kB 2-way, linesize 128 bytes.
Synthesized clear page handler (44 instructions).
Synthesized copy page handler (82 instructions).
Synthesized TLB refill handler (37 instructions).
Synthesized TLB load handler fastpath (49 instructions).
Synthesized TLB store handler fastpath (49 instructions).
Synthesized TLB modify handler fastpath (48 instructions).
EISA: Probing bus...
EISA: Detected 0 card.
ISA support compiled in.
PID hash table entries: 2048 (order: 11, 16384 bytes)
Calibrating system timer... warning: timer counts differ, retrying... 87500 [175.0000 MHz CPU]
Console: colour dummy device 80x25
Dentry cache hash table entries: 65536 (order: 7, 524288 bytes)
Inode-cache hash table entries: 32768 (order: 6, 262144 bytes)
Memory: 512132k/524288k available (2659k kernel code, 8412k reserved, 817k data, 280k init, 0k highmem)
That's with my fix/workaround applied.
Thomas.
Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessary a good idea. [ RFC1925, 2.3 ]
Mel Gorman
Acked-byOn (18/12/07 17:09), Thomas Bogendoerfer didst pronounce:
Quoted 11 lines · Mel Gorman
Quoted 8 lines
That commit is over a year old and it initially distressed me that it would take this long to show up on a boot test. However, you said this was to fix MIPS in a follow-on mail and I never made it use arch-independent zone-sizing which is where
CONFIG_ARCH_POPULATES_NODE_MAPcomes from. Support for arch-independent zone-sizing was added for MIPS on Nov 3rd 2007 by commitcce335ae47e231398269fb05fa48e0e9cbf289e0(relevant cc added) which is a much more reasonable timeframe for catching this sort of problem. I believe the real bug might be in there.either that or it got uncovered.
You're right, the arches using POPULATES_NODE_MAP would have had ARCH_PFN_OFFSET at 0. Your patch corrects that and should not affect other arches so;
Thanks for fixing.
While looking at this, it occured to me that maybe you could avoid a subtraction at runtime for MIPS using page_to_pfn() on FLATMEM by getting rid of ARCH_PFN_OFFSET. It's probably only of miniscule benefit but something like this (totally untested) patch may work if you want to try it out. It should have fixed your problem too but would it would have been achieved by hiding the problem. I believe your patch is still correct.
====
Get rid of ARCH_PFN_OFFSET calculation in MIPS with FLATMEM memory model.
---
arch/mips/kernel/setup.c | 13 +++++++------ include/asm-mips/page.h | 9 +-------- 2 files changed, 8 insertions(+), 14 deletions(-)
arch/mips/kernel/setup.c +7 -6
@@ -273,6 +273,7 @@ static void __init bootmem_init(void)unsigned long mapstart = ~0UL ;unsigned long bootmap_size;int i; +unsigned long phys_pfn_offset =PFN_UP (PHYS_OFFSET );/* * Init any data related to initrd. It's a nop if INITRD is
@@ -314,17 +315,17 @@ static void __init bootmem_init(void)if (min_low_pfn >= max_low_pfn)panic ("Incorrect memory mapping !!!" ); -if (min_low_pfn >ARCH_PFN_OFFSET ) { +if (min_low_pfn > phys_pfn_offset) {printk (KERN_INFO "Wasting %lu bytes for tracking %lu unused pages\n" , - (min_low_pfn -ARCH_PFN_OFFSET ) *sizeof (struct page ), - min_low_pfn -ARCH_PFN_OFFSET ); - }else if (min_low_pfn <ARCH_PFN_OFFSET ) { + (min_low_pfn - phys_pfn_offset) *sizeof (struct page ), + min_low_pfn - phys_pfn_offset); + }else if (min_low_pfn < phys_pfn_offset) {printk (KERN_INFO "%lu free pages won't be used\n" , -ARCH_PFN_OFFSET - min_low_pfn); + phys_pfn_offset - min_low_pfn); } - min_low_pfn =ARCH_PFN_OFFSET ; + min_low_pfn = phys_pfn_offset;/* * Determine low and high memory ranges
include/asm-mips/page.h +1 -8
@@ -37,13 +37,6 @@#include <linux/pfn.h> #include <asm/io.h> -/* -* It's normally defined only for FLATMEM config but it's -* used in our early mem init code for all memory models. -* So always define it. -*/ -#define ARCH_PFN_OFFSET PFN_UP(PHYS_OFFSET) -extern void clear_page (void * page);extern void copy_page (void * to,void * from);
@@ -159,7 +152,7 @@ typedef struct { unsigned long pgprot; } pgprot_t;#ifdef CONFIG_FLATMEM -#define pfn_valid (pfn) ((pfn) >= ARCH_PFN_OFFSET && (pfn) < max_mapnr) +#define pfn_valid (pfn) ((pfn) >= PFN_UP(PHYS_OFFSET) && (pfn) < max_mapnr)#elif defined(CONFIG_SPARSEMEM )
Signature · 3 lines
Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab
Thomas Bogendoerfer
maintainerinclude/asm-mips/page.h line 46
Quoted 13 lines · Mel Gorman
include/asm-mips/page.h +0
-7@@ -37,13 +37,6 @@#include <linux/pfn.h> #include <asm/io.h> -/* -* It's normally defined only for FLATMEM config but it's -* used in our early mem init code for all memory models. -* So always define it. -*/ -#define ARCH_PFN_OFFSET PFN_UP(PHYS_OFFSET) -
hmm, doesn't this break what I've fixed ? Without this #define ARCH_PFN_OFFSET gets defined to 0 and the bug is back. Or did I miss anything ?
Thomas.
Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessary a good idea. [ RFC1925, 2.3 ]
On (20/12/07 13:43), Thomas Bogendoerfer didst pronounce:
Quoted 7 lines · Mel Gorman
Quoted 6 lines
include/asm-mips/page.h +0
-0@@ -37,13 +37,6 @@#include <linux/pfn.h> #include <asm/io.h>
Quoted 12 lines
include/asm-mips/page.h line 46
Quoted 7 lines
include/asm-mips/page.h +0
-7-/* -* It's normally defined only for FLATMEM config but it's -* used in our early mem init code for all memory models. -* So always define it. -*/ -#define ARCH_PFN_OFFSET PFN_UP(PHYS_OFFSET) -hmm, doesn't this break what I've fixed ? Without this #define
ARCH_PFN_OFFSETgets defined to 0 and the bug is back. Or did I miss anything ?
ARCH_PFN_OFFSET goes to 0, so page_to_pfn() is no longer adjusting by PFN_UP(PHYS_OFFSET) like it was when your problem occured. I am guessing that the nature of the crash was that page_to_pfn() was returning bogus values early in boot and trying to initialise memmap that didn't exist.
Signature · 3 lines
Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab
Thomas Bogendoerfer
maintainerQuoted 21 lines · Mel Gorman
On (20/12/07 13:43), Thomas Bogendoerfer didst pronounce:
Quoted 7 lines · Mel Gorman
Quoted 6 lines
include/asm-mips/page.h +0
-0@@ -37,13 +37,6 @@#include <linux/pfn.h> #include <asm/io.h> Quoted 12 lines
include/asm-mips/page.h line 46
Quoted 7 lines
include/asm-mips/page.h +0
-7-/* -* It's normally defined only for FLATMEM config but it's -* used in our early mem init code for all memory models. -* So always define it. -*/ -#define ARCH_PFN_OFFSET PFN_UP(PHYS_OFFSET) -hmm, doesn't this break what I've fixed ? Without this #define
ARCH_PFN_OFFSETgets defined to 0 and the bug is back. Or did I miss anything ?
Quoted 4 lines
ARCH_PFN_OFFSETgoes to 0, sopage_to_pfn()is no longer adjusting byPFN_UP(PHYS_OFFSET)like it was when your problem occured. I am guessing that the nature of the crash was thatpage_to_pfn()was returning bogus values early in boot and trying to initialise memmap that didn't exist.
ic, your patch works for me.
Thomas.
Signature · 2 lines
Crap can work. Given enough thrust pigs will fly, but it's not necessary a good idea. [ RFC1925, 2.3 ]
On (30/12/07 12:37), Thomas Bogendoerfer didst pronounce:
Quoted 22 lines · Mel Gorman
Quoted 21 lines
On (20/12/07 13:43), Thomas Bogendoerfer didst pronounce:
Quoted 19 lines · Mel Gorman
include/asm-mips/page.h line 46
Quoted 13 lines
include/asm-mips/page.h +0
-7@@ -37,13 +37,6 @@#include <linux/pfn.h> #include <asm/io.h> -/* -* It's normally defined only for FLATMEM config but it's -* used in our early mem init code for all memory models. -* So always define it. -*/ -#define ARCH_PFN_OFFSET PFN_UP(PHYS_OFFSET) -hmm, doesn't this break what I've fixed ? Without this #define
ARCH_PFN_OFFSETgets defined to 0 and the bug is back. Or did I miss anything ?
Quoted 7 lines
Quoted 4 lines
ARCH_PFN_OFFSETgoes to 0, sopage_to_pfn()is no longer adjusting byPFN_UP(PHYS_OFFSET)like it was when your problem occured. I am guessing that the nature of the crash was thatpage_to_pfn()was returning bogus values early in boot and trying to initialise memmap that didn't exist.ic, your patch works for me.
Grand. For the moment, your patch is the correct fix. This is a micro-optimisation that can be picked up again at a later date. Thanks for the testing and the fix.
Signature · 3 lines
Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab