Skip to contentSkip to conversation

tune vmalloc size

PATCH Andi Kleen 6 participants 7 messages Sep 15, 2004 – Sep 17, 2004 1 message not in the archive Documentation

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Andi Kleen

author
#1 lore raw
Quoted 6 lines · Ingo Molnar

there are a few devices that use lots of ioremap space. vmalloc space is a showstopper problem for them.

this patch adds the vmalloc=<size> boot parameter to override __VMALLOC_RESERVE. The default is 128mb right now - e.g. vmalloc=256m doubles the size.

Ah, Karsten Keil did a similar patch some months ago. There is clearly a need.

But I think this should be self tuning instead. For a machine with less than 900MB of memory the vmalloc area can be automagically increased, growing into otherwise unused address space.

This way many users wouldn't need to specify weird options. So far most machines still don't have more than 512MB.

-Andi

#2 lore raw

How i know the best value to set vmalloc, it's full size of ram???? ----- Original Message -----
From: "Andi Kleen" <[email protected]>
To: "Ingo Molnar" <[email protected]>
Cc: <[email protected]>; <[email protected]>
Sent: Wednesday, September 15, 2004 10:29 AM
Subject: Re: [patch] tune vmalloc size

Quoted 26 lines
Quoted 6 lines · Ingo Molnar

there are a few devices that use lots of ioremap space. vmalloc space is a showstopper problem for them.

this patch adds the vmalloc=<size> boot parameter to override __VMALLOC_RESERVE. The default is 128mb right now - e.g. vmalloc=256m doubles the size.

Ah, Karsten Keil did a similar patch some months ago. There is clearly a need.

But I think this should be self tuning instead. For a machine with less than 900MB of memory the vmalloc area can be automagically increased, growing into otherwise unused address space.

This way many users wouldn't need to specify weird options. So far most machines still don't have more than 512MB.

(quoted diff) +0 -2
-Andi
 
-

To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#3 lore raw
In reply to Andi Kleen
Quoted 3 lines · Andi Kleen

But I think this should be self tuning instead. For a machine with less than 900MB of memory the vmalloc area can be automagically increased, growing into otherwise unused address space.

that is the case already

this patch is for the other case

#4 lore raw
Quoted 1 line · Arjan van de Ven

that is the case already

why do we still use 128MB as a default then? this is way over-kill from what i can tell looking on what my machines use. i'd rather have this be a bit smaller and enable the slab/whatever to grow a little more

#5 lore raw
Quoted 8 lines · Chris Wedgwood
Quoted 1 line · Arjan van de Ven

that is the case already

why do we still use 128MB as a default then? this is way over-kill from what i can tell looking on what my machines use. i'd rather have this be a bit smaller and enable the slab/whatever to grow a little more

if you have an old glibc it will use ldt's which in turn use vmalloc for threading... 128Mb is no luxury there.

#6 lore raw
In reply to Andi Kleen
Quoted 12 lines · Andi Kleen
Quoted 6 lines · Ingo Molnar

there are a few devices that use lots of ioremap space. vmalloc space is a showstopper problem for them.

this patch adds the vmalloc=<size> boot parameter to override __VMALLOC_RESERVE. The default is 128mb right now - e.g. vmalloc=256m doubles the size.

Ah, Karsten Keil did a similar patch some months ago. There is clearly a need.

Quoted 7 lines

But I think this should be self tuning instead. For a machine with less than 900MB of memory the vmalloc area can be automagically increased, growing into otherwise unused address space.

This way many users wouldn't need to specify weird options. So far most machines still don't have more than 512MB.

Yes my patch has this autotune already in.

Hmm, I though I did sent it on LKML begin on this year, but after looking through the archives, it didn't happend.

Here is my version.

arch/i386/boot/setup.S +1 -1
diff -urN linux-2.6.9-rc2-bk2.org/arch/i386/boot/setup.S linux-2.6.9-rc2-bk2/arch/i386/boot/setup.S
@@ -156,7 +156,7 @@
 					# can be located anywhere in
 					# low memory 0x10000 or higher.
 
-ramdisk_max:	.long (MAXMEM-1) & 0x7fffffff
+ramdisk_max:	.long (__MAXMEM-1) & 0x7fffffff
 					# (Header version 0x0203 or later)
 					# The highest safe address for
 					# the contents of an initrd
arch/i386/kernel/setup.c +36 -1
diff -urN linux-2.6.9-rc2-bk2.org/arch/i386/kernel/setup.c linux-2.6.9-rc2-bk2/arch/i386/kernel/setup.c
@@ -97,6 +97,11 @@
 /* For PCI or other memory-mapped resources */
 unsigned long pci_mem_start = 0x10000000;
 
+/* reserved mapping space for vmalloc and ioremap */
+unsigned long vmalloc_reserve = __VMALLOC_RESERVE_DEFAULT;
+EXPORT_SYMBOL(vmalloc_reserve);
+static unsigned long vm_reserve __initdata = -1;
+
 /* user-defined highmem size */
 static unsigned int highmem_pages = -1;
 
@@ -814,7 +819,16 @@
 		 */
 		if (c == ' ' && !memcmp(from, "highmem=", 8))
 			highmem_pages = memparse(from+8, &from) >> PAGE_SHIFT;
-	
+
+		/*
+		 * vm_reserve=size forces to reserve 'size' bytes for vmalloc and
+		 * ioremap areas minimum is 32 MB maximum is 800 MB
+		 * the default without vm_reserve depends on the total amount of
+		 * memory the minimum default is 128 MB
+		 */
+		if (c == ' ' && !memcmp(from, "vm_reserve=", 11))
+			vm_reserve = memparse(from+11, &from);
+
 		c = *(from++);
 		if (!c)
 			break;
@@ -1019,7 +1033,28 @@
 	start_pfn = PFN_UP(init_pg_tables_end);
 
 	find_max_pfn();
+	
+	/* 
+	 * calculate the default size of vmalloc/ioremap area
+	 * overwrite with the value of the vm_reserve= option
+	 * if set
+	 */
 
+	if (max_pfn >= PFN_UP(KERNEL_MAXMEM - __VMALLOC_RESERVE_DEFAULT))
+		vmalloc_reserve = __VMALLOC_RESERVE_DEFAULT;
+	else
+		vmalloc_reserve = KERNEL_MAXMEM - PFN_PHYS(max_pfn);
+	if (vm_reserve != -1) {
+		if (vm_reserve < __VMALLOC_RESERVE_MIN)
+			vm_reserve = __VMALLOC_RESERVE_MIN;
+		if (vm_reserve > __VMALLOC_RESERVE_MAX)
+			vm_reserve = __VMALLOC_RESERVE_MAX;
+		vmalloc_reserve = vm_reserve;
+	}
+	
+        printk(KERN_NOTICE "%ldMB vmalloc/ioremap area available.\n",
+                        vmalloc_reserve>>20);
+                        	
 	max_low_pfn = find_max_low_pfn();
 
 #ifdef CONFIG_HIGHMEM
arch/i386/mm/discontig.c +13 -0
diff -urN linux-2.6.9-rc2-bk2.org/arch/i386/mm/discontig.c linux-2.6.9-rc2-bk2/arch/i386/mm/discontig.c
@@ -266,6 +266,19 @@
 	system_start_pfn = min_low_pfn = PFN_UP(init_pg_tables_end);
 
 	find_max_pfn();
+
+	/* Added 2004-03-02, <[email protected]>, copied from i386/setup.c
+	 * but leave out automatic vmalloc size increase ... */
+	if (vm_reserve != -1) {
+		if (vm_reserve < __VMALLOC_RESERVE_MIN)
+			vm_reserve = __VMALLOC_RESERVE_MIN;
+		if (vm_reserve > __VMALLOC_RESERVE_MAX)
+			vm_reserve = __VMALLOC_RESERVE_MAX;
+		vmalloc_reserve = vm_reserve;
+	}
+	printk(KERN_NOTICE "%ldMB vmalloc/ioremap area available.\n",
+		vmalloc_reserve>>20);
+
 	system_max_low_pfn = max_low_pfn = find_max_low_pfn() - reserve_pages;
 	printk("reserve_pages = %ld find_max_low_pfn() ~ %ld\n",
 			reserve_pages, max_low_pfn + reserve_pages);
Documentation/kernel-parameters.txt +5 -0
diff -urN linux-2.6.9-rc2-bk2.org/Documentation/kernel-parameters.txt linux-2.6.9-rc2-bk2/Documentation/kernel-parameters.txt
@@ -1280,6 +1280,11 @@
 			This is actually a boot loader parameter; the value is
 			passed to the kernel using a special protocol.
 
+	vm_reserve=nn[KM]
+			[KNL,BOOT,IA-32] force use of a specific amount of
+			virtual memory for vmalloc and ioremap allocations
+			minimum 32 MB maximum 800 MB
+
 	vmhalt=		[KNL,S390]
 
 	vmpoff=		[KNL,S390] 
include/asm-i386/page.h +12 -4
diff -urN linux-2.6.9-rc2-bk2.org/include/asm-i386/page.h linux-2.6.9-rc2-bk2/include/asm-i386/page.h
@@ -98,10 +98,15 @@
  * This much address space is reserved for vmalloc() and iomap()
  * as well as fixmap mappings.
  */
-#define __VMALLOC_RESERVE	(128 << 20)
+#define __VMALLOC_RESERVE_MIN		(32 << 20)
+#define __VMALLOC_RESERVE_DEFAULT	(128 << 20)
+#define __VMALLOC_RESERVE_MAX		(800 << 20)
+#define __RESERVED_AREA			(10 << 20)
 
 #ifndef __ASSEMBLY__
 
+extern unsigned long vmalloc_reserve;
+
 /* Pure 2^n version of get_order */
 static __inline__ int get_order(unsigned long size)
 {
@@ -128,11 +133,14 @@
 
 
 #define PAGE_OFFSET		((unsigned long)__PAGE_OFFSET)
-#define VMALLOC_RESERVE		((unsigned long)__VMALLOC_RESERVE)
-#define MAXMEM			(-__PAGE_OFFSET-__VMALLOC_RESERVE)
+#define KERNEL_MEMORY		((unsigned long)(FIXADDR_START - __PAGE_OFFSET))
+#define RESERVED_AREA		((unsigned long)__RESERVED_AREA) 
+#define KERNEL_MAXMEM		((unsigned long)(KERNEL_MEMORY - RESERVED_AREA))
+#define __MAXMEM		(-__PAGE_OFFSET-__VMALLOC_RESERVE_MAX)
+#define MAXMEM			((unsigned long)(-PAGE_OFFSET-vmalloc_reserve))
 #define __pa(x)			((unsigned long)(x)-PAGE_OFFSET)
 #define __va(x)			((void *)((unsigned long)(x)+PAGE_OFFSET))
-#define pfn_to_kaddr(pfn)      __va((pfn) << PAGE_SHIFT)
+#define pfn_to_kaddr(pfn)	__va((pfn) << PAGE_SHIFT)
 #ifndef CONFIG_DISCONTIGMEM
 #define pfn_to_page(pfn)	(mem_map + (pfn))
 #define page_to_pfn(page)	((unsigned long)((page) - mem_map))
Signature · 3 lines
Karsten Keil
SuSE Labs
ISDN development
#7 lore raw
In reply to Andi Kleen

--Andi Kleen <[email protected]> wrote (on Wednesday, September 15, 2004 15:29:53 +0200):

Quoted 12 lines
Quoted 6 lines · Ingo Molnar

there are a few devices that use lots of ioremap space. vmalloc space is a showstopper problem for them.

this patch adds the vmalloc=<size> boot parameter to override __VMALLOC_RESERVE. The default is 128mb right now - e.g. vmalloc=256m doubles the size.

Ah, Karsten Keil did a similar patch some months ago. There is clearly a need.

Quoted 6 lines

But I think this should be self tuning instead. For a machine with less than 900MB of memory the vmalloc area can be automagically increased, growing into otherwise unused address space.

This way many users wouldn't need to specify weird options. So far most machines still don't have more than 512MB.

It already does that, IIRC.

M.