Slab panic on 2.6.19-rc3-git5 (-git4 was OK)
Subsystems (2)
-git4 was fine. -git5 is broken (on PPC64 blade)
As -rc2-mm2 seemed fine on this box, I'm guessing it's something that didn't go via Andrew ;-( Looks like it might be something JFS or slab specific. Bigger PPC64 box with different config was OK though.
Full log is here: http://test.kernel.org/abat/59046/debug/console.log
Good -git4 run: http://test.kernel.org/abat/58997/debug/console.log
kernel BUG in cache_grow at mm/slab.c:2705!
cpu 0x1: Vector: 700 (Program Check) at [c0000000fffb7710]
pc: c0000000000c8ad4: .cache_grow+0x64/0x4f0
lr: c0000000000c91a8: .cache_alloc_refill+0x248/0x2cc
sp: c0000000fffb7990 msr: 8000000000021032
current = 0xc0000000fffab800
paca = 0xc00000000047e780
pid = 1, comm = swapper
kernel BUG in cache_grow at mm/slab.c:2705!
enter ? for help
[c0000000fffb7a60] c0000000000c91a8 .cache_alloc_refill+0x248/0x2cc [c0000000fffb7b20] c0000000000c9708 .kmem_cache_alloc_node+0xd0/0x10c [c0000000fffb7bc0] c0000000000b69cc .__get_vm_area_node+0xcc/0x230 [c0000000fffb7c70] c0000000000b7640 .__vmalloc_node+0x60/0xc0 [c0000000fffb7d10] c0000000001ad4c8 .txInit+0x2a0/0x3a8 [c0000000fffb7e20] c00000000044c1ec .init_jfs_fs+0x78/0x27c [c0000000fffb7ec0] c0000000000094c0 .init+0x1f4/0x3e4 [c0000000fffb7f90] c000000000027270 .kernel_thread+0x4c/0x68
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Hi,
Quoted 16 lines · Martin J. Bligh
-git4 was fine. -git5 is broken (on PPC64 blade)
As -rc2-mm2 seemed fine on this box, I'm guessing it's something that didn't go via Andrew ;-( Looks like it might be something JFS or slab specific. Bigger PPC64 box with different config was OK though.
Full log is here: http://test.kernel.org/abat/59046/debug/console.log
Good -git4 run: http://test.kernel.org/abat/58997/debug/console.logkernel BUG in
cache_growatmm/slab.c:2705!
cpu0x1: Vector: 700 (Program Check) at [c0000000fffb7710]pc: c0000000000c8ad4: .cache_grow+0x64/0x4f0 lr: c0000000000c91a8: .cache_alloc_refill+0x248/0x2cc sp: c0000000fffb7990 msr: 8000000000021032
Quoted 13 lines
current =
0xc0000000fffab800
paca =0xc00000000047e780
pid = 1, comm = swapper
kernel BUG incache_growatmm/slab.c:2705!
enter ? for help
[c0000000fffb7a60]c0000000000c91a8.cache_alloc_refill+0x248/0x2cc [c0000000fffb7b20]c0000000000c9708.kmem_cache_alloc_node+0xd0/0x10c [c0000000fffb7bc0]c0000000000b69cc.__get_vm_area_node+0xcc/0x230 [c0000000fffb7c70]c0000000000b7640.__vmalloc_node+0x60/0xc0 [c0000000fffb7d10]c0000000001ad4c8.txInit+0x2a0/0x3a8 [c0000000fffb7e20]c00000000044c1ec.init_jfs_fs+0x78/0x27c [c0000000fffb7ec0]c0000000000094c0.init+0x1f4/0x3e4 [c0000000fffb7f90]c000000000027270.kernel_thread+0x4c/0x68
I only skimmed through this briefly but it looks like due to 52fd24ca1db3a741f144bbc229beefe044202cac __get_vm_area_node is passing GFP_HIGHMEM to kmem_cache_alloc_node which is a no-no.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 33 lines · --- Pekka Enberg
Hi,
Quoted 16 lines · Martin J. Bligh
-git4 was fine. -git5 is broken (on PPC64 blade)
As -rc2-mm2 seemed fine on this box, I'm guessing it's something that didn't go via Andrew ;-( Looks like it might be something JFS or slab specific. Bigger PPC64 box with different config was OK though
Full log is here: http://test.kernel.org/abat/59046/debug/console.log
Good -git4 run: http://test.kernel.org/abat/58997/debug/console.logkernel BUG in
cache_growatmm/slab.c:2705!
cpu0x1: Vector: 700 (Program Check) at [c0000000fffb7710]pc: c0000000000c8ad4: .cache_grow+0x64/0x4f0 lr: c0000000000c91a8: .cache_alloc_refill+0x248/0x2cc sp: c0000000fffb7990 msr: 8000000000021032Quoted 13 lines
current =
0xc0000000fffab800
paca =0xc00000000047e780
pid = 1, comm = swapper
kernel BUG incache_growatmm/slab.c:2705!
enter ? for help
[c0000000fffb7a60]c0000000000c91a8.cache_alloc_refill+0x248/0x2cc [c0000000fffb7b20]c0000000000c9708.kmem_cache_alloc_node+0xd0/0x10c [c0000000fffb7bc0]c0000000000b69cc.__get_vm_area_node+0xcc/0x230 [c0000000fffb7c70]c0000000000b7640.__vmalloc_node+0x60/0xc0 [c0000000fffb7d10]c0000000001ad4c8.txInit+0x2a0/0x3a8 [c0000000fffb7e20]c00000000044c1ec.init_jfs_fs+0x78/0x27c [c0000000fffb7ec0]c0000000000094c0.init+0x1f4/0x3e4 [c0000000fffb7f90]c000000000027270.kernel_thread+0x4c/0x68
Quoted 4 lines
I only skimmed through this briefly but it looks like due to
52fd24ca1db3a741f144bbc229beefe044202cac__get_vm_area_nodeis passingGFP_HIGHMEMtokmem_cache_alloc_nodewhich is a no-no.
I haven't been able to reproduce this, although I understand why it happens:
vmalloc allocates memory with
GFP_KERNEL | __GFP_HIGHMEM
and with git5, the same flags are passed down to cache_alloc_refill, causing the BUG. The following patch against 2.6.19-rc3-git5 (also attached as attachment, as this mailer may mess up inline copying) should fix it.
Note that when calling kmalloc_node, I am masking off __GFP_HIGHMEM with GFP_LEVEL_MASK, whereas __vmalloc_area_node does the same with
~(__GFP_HIGHMEM | __GFP_ZERO).
IMHO, using GFP_LEVEL_MASK is preferable, but either should fix this problem.
diff -Naur linux-2.6.19-rc3-git5.orig/mm/vmalloc.c linux-2.6.19-rc3-git5/mm/vmalloc.c
--- linux-2.6.19-rc3-git5.orig/mm/vmalloc.c 2006-10-29 07:26:34.000000000
-0500
+++ linux-2.6.19-rc3-git5/mm/vmalloc.c 2006-10-29 07:28:12.000000000 -0500 @@ -182,7 +182,7 @@
addr = ALIGN(start, align);
size = PAGE_ALIGN(size);- area = kmalloc_node(sizeof(*area), gfp_mask, node);
+ area = kmalloc_node(sizeof(*area), gfp_mask & GFP_LEVEL_MASK, node);
if (unlikely(!area))
return NULL;____________________________________________________________________________________
Access over 1 million songs - Yahoo! Music Unlimited
(http://music.yahoo.com/unlimited)
--0-295431475-1162126015=:4410
Content-Type: text/x-diff; name="__get_vm_area_node-should-mask-off-gfp-highmem.patch"
Content-Description: 16165293-__get_vm_area_node-should-mask-off-gfp-highmem.patch
Content-Disposition: inline; filename="__get_vm_area_node-should-mask-off-gfp-highmem.patch"
mm/vmalloc.c +1 -1
@@ -182,7 +182,7 @@ addr =ALIGN (start, align); size =PAGE_ALIGN (size); - area =kmalloc_node (sizeof (*area), gfp_mask, node); + area =kmalloc_node (sizeof (*area), gfp_mask &GFP_LEVEL_MASK , node);if (unlikely (!area))return NULL ;
--0-295431475-1162126015=:4410--
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines
Quoted 3 lines
I only skimmed through this briefly but it looks like due to
52fd24ca1db3a741f144bbc229beefe044202cac__get_vm_area_nodeis passingGFP_HIGHMEMtokmem_cache_alloc_nodewhich is a no-no.
Quoted 8 lines
I haven't been able to reproduce this, although I understand why it happens:
vmalloc allocates memory with
GFP_KERNEL|__GFP_HIGHMEMand with git5, the same flags are passed down to
cache_alloc_refill, causing the BUG. The following patch against 2.6.19-rc3-git5 (also attached as attachment, as this mailer may mess up inline copying) should fix it.
Thanks for the patch ... but more worrying is how this got broken. Wasn't the point of having the -mm tree that patches like this went through it for testing, and we avoid breaking mainline? especially this late in the -rc cycle.
M.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 4 lines · --- "Martin J. Bligh
Thanks for the patch ... but more worrying is how this got broken. Wasn't the point of having the -mm tree that patches like this went through it for testing, and we avoid breaking mainline? especially this late in the -rc cycle.
I don't know how it got into Linus's tree, but the breakage was due to my earlier patch - sorry.
Giri
____________________________________________________________________________________
Get your email and see which of your friends are online - Right on the New Yahoo.com
(http://www.yahoo.com/preview)
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 11 lines · On Sat, 28 Oct 2006 22:57:48 -0700 "Martin J. Bligh" <[email protected]> wrote:
-git4 was fine. -git5 is broken (on PPC64 blade)
As -rc2-mm2 seemed fine on this box, I'm guessing it's something that didn't go via Andrew ;-( Looks like it might be something JFS or slab specific. Bigger PPC64 box with different config was OK though.
Full log is here: http://test.kernel.org/abat/59046/debug/console.log
Good -git4 run: http://test.kernel.org/abat/58997/debug/console.logkernel BUG in
cache_growatmm/slab.c:2705!
This?
mm/vmalloc.c renamed from mm/vmalloc.c~__vmalloc_area_node-fix +2 -1
@@ -428,7 +428,8 @@ void *__vmalloc_area_node(struct vm_stru area->nr_pages = nr_pages;/* Please note that the recursion is strictly bounded. */ if (array_size >PAGE_SIZE ) { - pages =__vmalloc_node (array_size, gfp_mask,PAGE_KERNEL , node); + pages =__vmalloc_node (array_size, gfp_mask & ~__GFP_HIGHMEM, +PAGE_KERNEL , node); area->flags |=VM_VPAGES ; }else { pages = kmalloc_node(array_size,
_
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 22 lines · Andrew Morton
Quoted 11 lines · On Sat, 28 Oct 2006 22:57:48 -0700 "Martin J. Bligh" <[email protected]> wrote:
-git4 was fine. -git5 is broken (on PPC64 blade)
As -rc2-mm2 seemed fine on this box, I'm guessing it's something that didn't go via Andrew ;-( Looks like it might be something JFS or slab specific. Bigger PPC64 box with different config was OK though.
Full log is here: http://test.kernel.org/abat/59046/debug/console.log
Good -git4 run: http://test.kernel.org/abat/58997/debug/console.logkernel BUG in
cache_growatmm/slab.c:2705!This?
mm/vmalloc.c void *__vmalloc_area_node(struct vm_stru · line 435
Quoted 9 lines
mm/vmalloc.c renamed from mm/vmalloc.c~__vmalloc_area_node-fix +2
-1@@ -428,7 +428,8 @@ void *__vmalloc_area_node(struct vm_stru area->nr_pages = nr_pages;/* Please note that the recursion is strictly bounded. */ if (array_size >PAGE_SIZE ) { - pages =__vmalloc_node (array_size, gfp_mask,PAGE_KERNEL , node); + pages =__vmalloc_node (array_size, gfp_mask & ~__GFP_HIGHMEM, +PAGE_KERNEL , node); area->flags |=VM_VPAGES ; }else { pages = kmalloc_node(array_size,
Don't you actually *want* the page array to be allocated from highmem? So the gfp mask here should be just for whether we're allowed to sleep / reclaim (ie gfp_mask & ~(__GFP_DMA|__GFP_DMA32) | (__GFP_HIGHMEM))?
Slab allocations should be (gfp_mask & ~(__GFP_DMA|__GFP_DMA32|__GFP_HIGHMEM)), which you could mask in __get_vm_area_node
Signature · 8 lines
SUSE Labs, Novell Inc. Send instant messages to your online friends http://au.messenger.yahoo.com -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 18 lines · Nick Piggin
mm/vmalloc.c void *__vmalloc_area_node(struct vm_stru · line 435
Quoted 12 lines · Andrew Morton
mm/vmalloc.c renamed from mm/vmalloc.c~__vmalloc_area_node-fix +2
-1@@ -428,7 +428,8 @@ void *__vmalloc_area_node(struct vm_stru area->nr_pages = nr_pages;/* Please note that the recursion is strictly bounded. */ if (array_size >PAGE_SIZE ) { - pages =__vmalloc_node (array_size, gfp_mask,PAGE_KERNEL , node); + pages =__vmalloc_node (array_size, gfp_mask & ~__GFP_HIGHMEM, +PAGE_KERNEL , node); area->flags |=VM_VPAGES ; }else { pages = kmalloc_node(array_size,Don't you actually *want* the page array to be allocated from highmem? So the gfp mask here should be just for whether we're allowed to sleep / reclaim (ie
gfp_mask& ~(__GFP_DMA|__GFP_DMA32) | (__GFP_HIGHMEM))?
Quoted 4 lines
Slab allocations should be (
gfp_mask& ~(__GFP_DMA|__GFP_DMA32|__GFP_HIGHMEM)), which you could mask in__get_vm_area_node
Since gfp_mask there would also have GFP_ZERO, we need to mask off that too. How about my earlier suggestion of masking off flags in __get_vm_area_node with GFP_LEVEL_MASK?
Giri
PS: I am not sure if this mail gets to all recipients in the original thread - I am not subscribed to lkml and I haven't found a way to reply to all people and the group.
Quoted 29 lines · Andrew Morton
Quoted 11 lines · On Sat, 28 Oct 2006 22:57:48 -0700 "Martin J. Bligh" <[email protected]> wrote:
-git4 was fine. -git5 is broken (on PPC64 blade)
As -rc2-mm2 seemed fine on this box, I'm guessing it's something that didn't go via Andrew ;-( Looks like it might be something JFS or slab specific. Bigger PPC64 box with different config was OK though.
Full log is here: http://test.kernel.org/abat/59046/debug/console.log
Good -git4 run: http://test.kernel.org/abat/58997/debug/console.logkernel BUG in
cache_growatmm/slab.c:2705!This?
mm/vmalloc.c renamed from mm/vmalloc.c~__vmalloc_area_node-fix +2
-1@@ -428,7 +428,8 @@ void *__vmalloc_area_node(struct vm_stru area->nr_pages = nr_pages;/* Please note that the recursion is strictly bounded. */ if (array_size >PAGE_SIZE ) { - pages =__vmalloc_node (array_size, gfp_mask,PAGE_KERNEL , node); + pages =__vmalloc_node (array_size, gfp_mask & ~__GFP_HIGHMEM, +PAGE_KERNEL , node); area->flags |=VM_VPAGES ; }else { pages = kmalloc_node(array_size,
Quoted 1 line
_
/me shoves it into the tests... results in a couple of hours.
-apw
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 18 lines
Quoted 15 lines
Quoted 1 line
kernel BUG in
cache_growatmm/slab.c:2705!This?
mm/vmalloc.c renamed from mm/vmalloc.c~__vmalloc_area_node-fix +2
-1@@ -428,7 +428,8 @@ void *__vmalloc_area_node(struct vm_stru area->nr_pages = nr_pages;/* Please note that the recursion is strictly bounded. */ if (array_size >PAGE_SIZE ) { - pages =__vmalloc_node (array_size, gfp_mask,PAGE_KERNEL , node); + pages =__vmalloc_node (array_size, gfp_mask & ~__GFP_HIGHMEM, +PAGE_KERNEL , node); area->flags |=VM_VPAGES ; }else { pages = kmalloc_node(array_size,Quoted 1 line
_
/me shoves it into the tests... results in a couple of hours.
Seems like that doesn't fix it, I'm afraid.
M.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 2 lines · Martin J. Bligh
Seems like that doesn't fix it, I'm afraid.
Does the one in the current -git tree? It's commit 5211e6e6c671f0d4b1e1a1023384d20227d8ee65, as below..
Linus
---
commit 5211e6e6c671f0d4b1e1a1023384d20227d8ee65
Author: Giridhar Pemmasani <[email protected]> Date: Sun Oct 29 04:46:55 2006 -0800
[PATCH] Fix GFP_HIGHMEM slab panic
As reported by Martin J. Bligh <[email protected]>, we let through some non-slab bits to slab allocation through __get_vm_area_node when doing a vmalloc.
I haven't been able to reproduce this, although I understand why it
happens: vmalloc allocates memory with
GFP_KERNEL | __GFP_HIGHMEM
and commit 52fd24ca1db3a741f144bbc229beefe044202cac resulted in the same flags are passed down to cache_alloc_refill, causing the BUG. The following patch fixes it.
Note that when calling kmalloc_node, I am masking off __GFP_HIGHMEM with GFP_LEVEL_MASK, whereas __vmalloc_area_node does the same with
~(__GFP_HIGHMEM | __GFP_ZERO).
IMHO, using GFP_LEVEL_MASK is preferable, but either should fix this problem.
mm/vmalloc.c +1 -1
@@ -182,7 +182,7 @@ static struct vm_struct *__get_vm_area_n addr =ALIGN (start, align); size =PAGE_ALIGN (size); - area =kmalloc_node (sizeof (*area), gfp_mask, node); + area =kmalloc_node (sizeof (*area), gfp_mask &GFP_LEVEL_MASK , node);if (unlikely (!area))return NULL ;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Linus Torvalds
Quoted 1 line · Martin J. Bligh
Seems like that doesn't fix it, I'm afraid.
Does the one in the current -git tree? It's commit
5211e6e6c671f0d4b1e1a1023384d20227d8ee65, as below..Linus
Submitted that commit, results in a couple of hours.
-apw
Quoted 39 lines
---
commit5211e6e6c671f0d4b1e1a1023384d20227d8ee65Author: Giridhar Pemmasani <[email protected]> Date: Sun Oct 29 04:46:55 2006 -0800[PATCH] Fix
GFP_HIGHMEMslab panicAs reported by Martin J. Bligh <[email protected]>, we let through some non-slab bits to slab allocation through
__get_vm_area_nodewhen doing a vmalloc.I haven't been able to reproduce this, although I understand why it
happens: vmalloc allocates memory with
GFP_KERNEL|__GFP_HIGHMEMand commit
52fd24ca1db3a741f144bbc229beefe044202cacresulted in the same flags are passed down tocache_alloc_refill, causing the BUG. The following patch fixes it.Note that when calling
kmalloc_node, I am masking off__GFP_HIGHMEMwithGFP_LEVEL_MASK, whereas__vmalloc_area_nodedoes the same with~(__GFP_HIGHMEM |
__GFP_ZERO).IMHO, using
GFP_LEVEL_MASKis preferable, but either should fix this problem.Signed-off-by Giridhar Pemmasani ([email protected])Cc Martin J. Bligh <[email protected]>Cc Andrew Morton <[email protected]>Signed-off-by Linus Torvalds <[email protected]>mm/vmalloc.c +0
-0
mm/vmalloc.c static struct vm_struct *__get_vm_area_n · line 188
Quoted 8 lines
mm/vmalloc.c +1
-1@@ -182,7 +182,7 @@ static struct vm_struct *__get_vm_area_n addr =ALIGN (start, align); size =PAGE_ALIGN (size); - area =kmalloc_node (sizeof (*area), gfp_mask, node); + area =kmalloc_node (sizeof (*area), gfp_mask &GFP_LEVEL_MASK , node);if (unlikely (!area))return NULL ;
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 8 lines · Linus Torvalds
Quoted 1 line · Martin J. Bligh
Seems like that doesn't fix it, I'm afraid.
Does the one in the current -git tree? It's commit
5211e6e6c671f0d4b1e1a1023384d20227d8ee65, as below..Linus
Test results are back on the version of the slab panic fix which Linus' has committed in his tree. This change on top of 2.6.19-rc3-git5 is good. 2.6.19-rc3-git6 is also showing good on this machine.
-apw
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 3 lines · Andy Whitcroft
Test results are back on the version of the slab panic fix which Linus' has committed in his tree. This change on top of 2.6.19-rc3-git5 is good. 2.6.19-rc3-git6 is also showing good on this machine.
FWIW, the patch looks correct to me also.
Pekka
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>