Skip to contentSkip to conversation

mm: fix oom roll-back of __vmalloc_area_node

PATCH In review Jan Kiszka 2 participants 4 messages Jul 11, 2006 – Jul 13, 2006

Subsystems (2)
Memory management vmalloc
lore
Share & tools
lore

Jan Kiszka

author
#0 lore raw

PATCH 1/1 mm: fix oom roll-back of __vmalloc_area_node

__vunmap must not rely on area->nr_pages when picking the release methode for area->pages. It may be too small when __vmalloc_area_node failed early due to lacking memory. Instead, use a flag in vmstruct to differentiate.

Applies against 2.6.18-rc1, but sighted, developed, and tested on a 2.6.16 kernel.

Signed-off-by Jan Kiszka <[email protected]>
---
 include/linux/vmalloc.h |    1 +
 mm/vmalloc.c            |    7 ++++---
 2 files changed, 5 insertions(+), 3 deletions(-)
include/linux/vmalloc.h +1 -0
Index: linux-2.6/include/linux/vmalloc.h
===================================================================
@@ -11,6 +11,7 @@ struct vm_area_struct;
 #define VM_ALLOC	0x00000002	/* vmalloc() */
 #define VM_MAP		0x00000004	/* vmap()ed pages */
 #define VM_USERMAP	0x00000008	/* suitable for remap_vmalloc_range */
+#define VM_VPAGES	0x00000010	/* buffer for pages was vmalloc'ed */
 /* bits [20..32] reserved for arch specific ioremap internals */
 
 /*
mm/vmalloc.c +4 -3
Index: linux-2.6/mm/vmalloc.c
===================================================================
@@ -340,7 +340,7 @@ void __vunmap(void *addr, int deallocate
 			__free_page(area->pages[i]);
 		}
 
-		if (area->nr_pages > PAGE_SIZE/sizeof(struct page *))
+		if (area->flags & VM_VPAGES)
 			vfree(area->pages);
 		else
 			kfree(area->pages);

@@ -427,9 +427,10 @@ void *__vmalloc_area_node(struct vm_stru
 
 	area->nr_pages = nr_pages;
 	/* Please note that the recursion is strictly bounded. */
-	if (array_size > PAGE_SIZE)
+	if (array_size > PAGE_SIZE) {
 		pages = __vmalloc_node(array_size, gfp_mask, PAGE_KERNEL, node);
-	else
+		area->flags |= VM_VPAGES;
+	} else
 		pages = kmalloc_node(array_size, (gfp_mask & ~__GFP_HIGHMEM), node);
 	area->pages = pages;
 	if (!area->pages) {
#1 lore raw
Quoted 4 lines · On Tue, 11 Jul 2006 21:37:08 +0200 Jan Kiszka <[email protected]> wrote:

__vunmap must not rely on area->nr_pages when picking the release methode for area->pages. It may be too small when __vmalloc_area_node failed early due to lacking memory. Instead, use a flag in vmstruct to differentiate.

So you mean that when this:

		if (unlikely(!area->pages[i])) {
			/* Successfully allocated i pages, free them in __vunmap() */
			area->nr_pages = i;
			goto fail;

happens, it could be that i <= PAGE_SIZE/sizeof(struct page *) and __vunmap kfree()s something which it should have vfree()d, yes?

That sounds like a dead box, or worse.

I think the change would be a good one even if it didn't fix a bug, thanks.

author
#2 lore raw
Quoted 12 lines · Andrew Morton
Quoted 5 lines · On Tue, 11 Jul 2006 21:37:08 +0200 Jan Kiszka <[email protected]> wrote:

__vunmap must not rely on area->nr_pages when picking the release methode for area->pages. It may be too small when __vmalloc_area_node failed early due to lacking memory. Instead, use a flag in vmstruct to differentiate.

So you mean that when this:

Quoted 9 lines
		if (unlikely(!area->pages[i])) {
			/* Successfully allocated i pages, free them in __vunmap() */
			area->nr_pages = i;
			goto fail;

happens, it could be that i <= PAGE_SIZE/sizeof(struct page *) and __vunmap kfree()s something which it should have vfree()d, yes?

Yes, exactly. It then causes some BUG in kfree during unroll.

Quoted 3 lines

That sounds like a dead box, or worse.

Someone triggered a too large vmalloc request, that was the scenario here.

Quoted 3 lines

I think the change would be a good one even if it didn't fix a bug, thanks.

Meanwhile I thought about an even simpler solution:

__vunmap must not rely on area->nr_pages when picking the release methode for area->pages. It may be too small when __vmalloc_area_node failed due to lacking memory. Check for the vmalloc address range instead.

Signed-off by: Jan Kiszka <[email protected]>

mm/vmalloc.c +1 -1
Index: linux-2.6/mm/vmalloc.c
===================================================================
@@ -340,7 +340,7 @@ void __vunmap(void *addr, int deallocate
 			__free_page(area->pages[i]);
 		}
 
-		if (area->nr_pages > PAGE_SIZE/sizeof(struct page *))
+		if (area->pages >= VMALLOC_START && area->pages < VMALLOC_END)
 			vfree(area->pages);
 		else
 			kfree(area->pages);
#3 lore raw
Quoted 18 lines · On Thu, 13 Jul 2006 09:38:38 +0200 Jan Kiszka <[email protected]> wrote:
Quoted 3 lines

I think the change would be a good one even if it didn't fix a bug, thanks.

Meanwhile I thought about an even simpler solution:

__vunmap must not rely on area->nr_pages when picking the release methode for area->pages. It may be too small when __vmalloc_area_node failed due to lacking memory. Check for the vmalloc address range instead.

Signed-off by: Jan Kiszka <[email protected]>

Index: linux-2.6/mm/vmalloc.c
===================================================================

mm/vmalloc.c void __vunmap(void *addr, int deallocate · line 346

Quoted 8 lines
mm/vmalloc.c +1 -1
@@ -340,7 +340,7 @@ void __vunmap(void *addr, int deallocate
 			__free_page(area->pages[i]);
 		}
 
-		if (area->nr_pages > PAGE_SIZE/sizeof(struct page *))
+		if (area->pages >= VMALLOC_START && area->pages < VMALLOC_END)
 			vfree(area->pages);
 		else
 			kfree(area->pages);

Nah, your first patch was better. Very clear, direct, explicit.

It even had a comment.