Skip to contentSkip to conversation

New: Crash when allocating > 2 TB memory

Bug 210023 Andrew Morton 3 participants 5 messages Nov 4, 2020 – Nov 5, 2020 3 messages not in the archive

Subsystems (2)
Memory management vmalloc
lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]/

Andrew Morton

author maintainer
#1 lore raw

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 18 lines · On Tue, 03 Nov 2020 18:50:07 +0000 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=210023

Bug ID: 210023
Summary: Crash when allocating > 2 TB memory
Product: Memory Management
Version: 2.5
Kernel Version: 3.10.0-957.27.2.el7.x86_64
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: blocking
Priority: P1
Component: Slab Allocator
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Quoted 2 lines

With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to allocate > 2 TB memory, the array_size below will be overflowed.

How was this observed?

Is there any know userspace operation which causes the kernel to try to vmalloc such a large hunk of memory?

Quoted 7 lines

The array_size is an unsigned int and can only be used to allocate less than 2 TB memory. If you pass 2*1028*1028*1024*1024 = 2 * 2^40 in the argument of vmalloc. The array_size will become 2*2^31 = 2^32. The 2^32 cannot be store with a 32 bit integer.

The fix is to change the type of array_size to unsigned long.

Quoted 7 lines

vmalloc.c

1762 void *vmalloc(unsigned long size)
1763 {
1764 return __vmalloc_node_flags(size, NUMA_NO_NODE,
1765 GFP_KERNEL | __GFP_HIGHMEM);
1766 }

OK, thanks. Against current mainline your proposed change would look like this, yes?

mm/vmalloc.c renamed from mm/vmalloc.c~a +3 -1
@@ -2461,9 +2461,11 @@ static void *__vmalloc_area_node(struct
 {
 	const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) | __GFP_ZERO;
 	unsigned int nr_pages = get_vm_area_size(area) >> PAGE_SHIFT;
-	unsigned int array_size = nr_pages * sizeof(struct page *), i;
+	unsigned long array_size
+	unsigned int i;
 	struct page **pages;
 
+	array_size = (unsigned long)nr_pages * sizeof(struct page *);
 	gfp_mask |= __GFP_NOWARN;
 	if (!(gfp_mask & (GFP_DMA | GFP_DMA32)))
 		gfp_mask |= __GFP_HIGHMEM;

_

#2 lore raw

mm/vmalloc.c static void *__vmalloc_area_node(struct · line 2468

Quoted 8 lines · Andrew Morton
mm/vmalloc.c renamed from mm/vmalloc.c~a +3 -1
 	const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) | __GFP_ZERO;
 	unsigned int nr_pages = get_vm_area_size(area) >> PAGE_SHIFT;
-	unsigned int array_size = nr_pages * sizeof(struct page *), i;
+	unsigned long array_size
+	unsigned int i;
 	struct page **pages;
 
+	array_size = (unsigned long)nr_pages * sizeof(struct page *);

This is only pushing the problem out ~5 years. If somebody tries to allocate a 16TB area, 16TB / 4kB is 4GB and nr_pages overflows. That's only 3 doublings of RAM away.

I think we need to change vm_struct's nr_pages from an unsigned int to an unsigned long. It won't take up any more room because it's sandwiched between a pointer and a phys_addr_t.

#3 lore raw
In reply to Andrew Morton
Quoted 2 lines

With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to allocate > 2 TB memory, the array_size below will be overflowed.

How was this observed?

Is there any know userspace operation which causes the kernel to try to vmalloc such a large hunk of memory?

[Frank] The Dell PowerEdge R740/R940 can have up to 3TB/6TB memory. installed. Our application requires reserve consecutive memory in the kernel space and protected from userspace programs.

----------------------------------------------------------------------------------------------------
OK, thanks. Against current mainline your proposed change would look like this, yes?

[Frank] Yes. This will support up to less than 16 TB. If you want to support
more than 16 TB, we need to expand nr_pages to unsigned long as
Matthew pointed out.

Will it be possible to add this to kernel 3.10.0-957.27.2.el7.x86_64?

Thanks,
Frank

Quoted 72 lines · Andrew Morton

(switched to email. Please respond via emailed reply-to-all, not via the bugzilla web interface).

Quoted 18 lines · On Tue, 03 Nov 2020 18:50:07 +0000 [email protected] wrote:

https://bugzilla.kernel.org/show_bug.cgi?id=210023

Bug ID: 210023
Summary: Crash when allocating > 2 TB memory
Product: Memory Management
Version: 2.5
Kernel Version: 3.10.0-957.27.2.el7.x86_64
Hardware: All
OS: Linux
Tree: Mainline
Status: NEW
Severity: blocking
Priority: P1
Component: Slab Allocator
Assignee: [email protected]
Reporter: [email protected]
Regression: No

Quoted 2 lines

With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to allocate > 2 TB memory, the array_size below will be overflowed.

How was this observed?

Is there any know userspace operation which causes the kernel to try to vmalloc such a large hunk of memory?

Quoted 1 line

The array_size is an unsigned int and can only be used to allocate less

than 2

Quoted 1 line

TB memory. If you pass 2*1028*1028*1024*1024 = 2 * 2^40 in the argument

of

Quoted 1 line

vmalloc. The array_size will become 2*2^31 = 2^32. The 2^32 cannot be

store

Quoted 11 lines

with a 32 bit integer.

The fix is to change the type of array_size to unsigned long.

vmalloc.c

1762 void *vmalloc(unsigned long size)
1763 {
1764 return __vmalloc_node_flags(size, NUMA_NO_NODE,
1765 GFP_KERNEL | __GFP_HIGHMEM);
1766 }

OK, thanks. Against current mainline your proposed change would look like this, yes?

mm/vmalloc.c renamed from mm/vmalloc.c~a +3 -1
@@ -2461,9 +2461,11 @@ static void *__vmalloc_area_node(struct
 {
        const gfp_t nested_gfp = (gfp_mask & GFP_RECLAIM_MASK) |
 __GFP_ZERO;
        unsigned int nr_pages = get_vm_area_size(area) >> PAGE_SHIFT;
-       unsigned int array_size = nr_pages * sizeof(struct page *), i;
+       unsigned long array_size
+       unsigned int i;
        struct page **pages;
 
+       array_size = (unsigned long)nr_pages * sizeof(struct page *);
        gfp_mask |= __GFP_NOWARN;
        if (!(gfp_mask & (GFP_DMA | GFP_DMA32)))

                gfp_mask |= __GFP_HIGHMEM;

_

author maintainer
#4 lore raw
Quoted 11 lines · On Wed, 4 Nov 2020 08:55:11 -0500 Hsin-Hui Wu <[email protected]> wrote:
Quoted 2 lines

With a machine with 3 TB (more than 2 TB memory). If you use vmalloc to allocate > 2 TB memory, the array_size below will be overflowed.

How was this observed?

Is there any know userspace operation which causes the kernel to try to vmalloc such a large hunk of memory?

[Frank] The Dell PowerEdge R740/R940 can have up to 3TB/6TB memory. installed. Our application requires reserve consecutive memory in the kernel space and protected from userspace programs.

Did this require custom kernel changes? If not, precisely which system calls were used to cause this allocation attempt?

Quoted 9 lines

----------------------------------------------------------------------------------------------------
OK, thanks. Against current mainline your proposed change would look like this, yes?

[Frank] Yes. This will support up to less than 16 TB. If you want to support
more than 16 TB, we need to expand nr_pages to unsigned long as
Matthew pointed out.

Will it be possible to add this to kernel 3.10.0-957.27.2.el7.x86_64?

That is up to Red Hat to decide.

A message that is not in the archive (sent off-list or not yet received): CANpob6LY6oXqTG0GTJk-Z9qJtp9kpCCdoDAsbOJgTA8QfPCe2Q@mail.gmail.com

A message that is not in the archive (sent off-list or not yet received): [email protected]

#7 lore raw

We don't use userspace program to allocate memory. We use kernel module to do so.
Here is the sample kernel program you can use to test on a machine that has more than 2 TB memory.

Use "insmod memtest.ko mb=2100000" to allocate more than 2 TB memory. Use "rmmod memtest" to free the memory.

memtest.c
=============================================================
#include <linux/version.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/fs.h>
#include <linux/cdev.h>
#include <linux/mm.h>
#include <linux/vmalloc.h>
#ifdef MODVERSIONS
#include <linux/modversions.h>
#endif
#include <asm/io.h>

static char *mem_ptr;
MODULE_LICENSE("GPL");
int mb = 0;
MODULE_PARM_DESC(mb, "alloc MB memory");
module_param(mb, int, S_IWUSR | S_IRUGO);

static int __init memtest_init(void) {
    mem_ptr = NULL;
    if (mb > 0) {
        unsigned long alloc_len;
        alloc_len = mb * 1024 * 1024;

        mem_ptr = vmalloc(alloc_len);

        if (mem_ptr) {
            printk("memtest: vmalloc for size %u MB.\n", mb);
        } else {
            printk("memtest: failed to vmalloc for bytes %u MB.\n", mb);
        }
    } else {
        printk("memtest len_MB\n");
    }
    return(0);
}
static void __exit memtest_exit(void) {
    if (mem_ptr) {
        vfree(mem_ptr);
        printk("memtest: vfree for size %u MB.\n", mb);
        mem_ptr = NULL;
    }
}

module_init(memtest_init);
module_exit(memtest_exit);

Quoted 14 lines · Andrew Morton
Quoted 5 lines · On Wed, 4 Nov 2020 15:53:36 -0500 Hsin-Hui Wu <[email protected]> wrote:

Did this require custom kernel changes? If not, precisely which system calls were used to cause this allocation attempt?

[Frank] No. vmalloc().

vmalloc() is not a system call.

Please fully describe how vmalloc() came to be called with such a large request size. Fully. Are you able to provide example userspace code?