Skip to contentSkip to conversation

i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

PATCH In review Anatoli Antonovitch 3 participants 3 messages Sep 15, 2006 – Sep 24, 2006

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 i386: fix overflow in vmap on an x86 system which has more than 4GB memory.

Description
(max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more than 4GB memory, and hence cause vmap to fail every time.

Patch
diff -Nur linux-2.4.21-40.EL/mm/vmalloc.c
linux-2.4.21-40.EL.diff/mm/vmalloc.c
--- linux-2.4.21-40.EL/mm/vmalloc.c 2006-02-02 21:13:20.000000000
-0600
+++ linux-2.4.21-40.EL.diff/mm/vmalloc.c 2006-09-04
11:29:33.000000000 -0500
@@ -298,8 +298,8 @@

        struct vm_struct *area;
        unsigned long size = count << PAGE_SHIFT;

- if (!size || size > (max_mapnr << PAGE_SHIFT))
- return NULL;
+ if (!count || count > max_mapnr)

+        return NULL;
        area = get_vm_area(size, flags);
        if (!area) {
                return NULL;
#1 lore raw

This is a 2.4 fix (not needed in 2.6): let's CC maintainer Willy Tarreau.

Quoted 3 lines · Anatoli Antonovitch

Description
(max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more than 4GB memory, and hence cause vmap to fail every time.

Good point, thanks for the patch. Sorry I'm so slow to get to it.

Quoted 9 lines

Patch
diff -Nur linux-2.4.21-40.EL/mm/vmalloc.c
linux-2.4.21-40.EL.diff/mm/vmalloc.c
--- linux-2.4.21-40.EL/mm/vmalloc.c 2006-02-02 21:13:20.000000000
-0600
+++ linux-2.4.21-40.EL.diff/mm/vmalloc.c 2006-09-04

And still needs fixing in latest mainline 2.4.

Review line 302

Quoted 9 lines

11:29:33.000000000 -0500

(quoted diff) +2 -2
@@ -298,8 +298,8 @@
        struct vm_struct *area;
        unsigned long size = count << PAGE_SHIFT;
 
-       if (!size || size > (max_mapnr << PAGE_SHIFT))
-               return NULL;
+    if (!count || count > max_mapnr)
+        return NULL;

I'm afraid the tabs got messed up in both the old and new lines. Also, count is a signed int (whereas size and max_mapnr are both unsigned longs), so best reject "count <= 0" rather than just "!count".

Review line 305

Quoted 3 lines
(quoted diff) +0 -0
        area = get_vm_area(size, flags);
        if (!area) {
                return NULL;

Here's a replacement patch for Willy. Anatoli, you didn't sign off the patch yourself: so I'm assuming Michael is the originator.

From: Michael Chen <[email protected]>

(max_mapnr << PAGE_SHIFT) would overflow on a system which has 4GB memory or more, and so could cause vmap to fail every time.

---

 mm/vmalloc.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmalloc.c +1 -1
@@ -293,7 +293,7 @@ void * vmap(struct page **pages, int cou
 	struct vm_struct *area;
 	unsigned long size = count << PAGE_SHIFT;
 
-	if (!size || size > (max_mapnr << PAGE_SHIFT))
+	if (count <= 0 || count > max_mapnr)
 		return NULL;
 	area = get_vm_area(size, flags);
 	if (!area) {
#2 lore raw

Hi Hugh !

Quoted 9 lines · Hugh Dickins

This is a 2.4 fix (not needed in 2.6): let's CC maintainer Willy Tarreau.

Quoted 3 lines · Anatoli Antonovitch

Description
(max_mapnr << PAGE_SHIFT) would overflow on an x86 system which has more than 4GB memory, and hence cause vmap to fail every time.

Good point, thanks for the patch. Sorry I'm so slow to get to it.

Don't worry, Andrew already forwarded it to me. BTW, thanks for your review and comment, I'll finally apply yours since it's better.

Cheers,
Willy