accurately calculate pages managed by buddy system

RFC PATCH Vineet Gupta 1 participant 1 message Mar 7, 2013 1 message not in the archive

Subsystems (13)

Memory management Arm (32-bit) x86 PowerPC +9 more subsystems

lore

Share & tools

lore

A message that is not in the archive (sent off-list or not yet received): [email protected]

Vineet Gupta

author Acked-by

2013-03-07 06:25 UTC #1 lore raw

Somehow the CC to linux-arch and linux-mm got lost in the trail - Also adding James for metag and clipped the extended CC list.

Acked-by Vineet Gupta <[email protected]> for patch 01 (ARC bits)

-------- Original Message --------
Subject: Re: [RFC PATCH v1 00/33] accurately calculate pages managed by buddy system
Date: Wed, 6 Mar 2013 23:44:04 +0800
From: Jiang Liu <[email protected]>
To: Vineet Gupta <[email protected]>
CC: Andrew Morton <[email protected]>, David Rientjes
<[email protected]>, Jiang Liu <[email protected]>, Wen Congyang <[email protected]>, Maciej Rutecki <[email protected]>, "Chris Clayton" <[email protected]>, "Rafael J . Wysocki" <[email protected]>, "Mel Gorman" <[email protected]>, Minchan Kim <[email protected]>, "KAMEZAWA Hiroyuki" <[email protected]>, Michal Hocko <[email protected]>, Jianguo Wu <[email protected]>, Anatolij Gustschin <[email protected]>, Aurelien Jacquiot <[email protected]>, Benjamin Herrenschmidt <[email protected]>, Catalin Marinas <[email protected]>, "Chen Liqin" <[email protected]>, Chris Metcalf <[email protected]>, Chris Zankel <[email protected]>, David Howells <[email protected]>, "David S. Miller" <[email protected]>, Eric Biederman <[email protected]>, Fenghua Yu <[email protected]>, "Geert Uytterhoeven" <[email protected]>

Quoted 69 lines · Vineet Gupta

Hi Jiang,
Quoted 61 lines · Jiang Liu
The original goal of this patchset is to fix the bug reported by
https://bugzilla.kernel.org/show_bug.cgi?id=53501
Now it has also been expanded to reduce common code used by memory initializion. In total it has reduced about 550 lines of code.
Patch 1:
Extract common help functions from free_init_mem() and free_initrd_mem() on different architectures.
Patch 2-27:
Use help functions to simplify free_init_mem() and free_initrd_mem() on different architectures. This has reduced about 500 lines of code.
Patch 28:
Introduce common help function to free highmem pages when initializing memory subsystem.
Patch 29-32:
	Adjust totalhigh_pages, totalram_pages and zone->managed_pages
	altogether when reserving/unreserving pages.
Patch 33:
Change /sys/.../node/nodex/meminfo to report available pages within the node as "MemTotal".
We have only tested these patchset on x86 platforms, and have done basic compliation tests using cross-compilers from ftp.kernel.org. That means some code may not pass compilation on some architectures. So any help to test this patchset are welcomed!
Jiang Liu (33):
mm: introduce common help functions to deal with reserved/managed
pages
mm/alpha: use common help functions to free reserved pages mm/ARM: use common help functions to free reserved pages mm/avr32: use common help functions to free reserved pages mm/blackfin: use common help functions to free reserved pages mm/c6x: use common help functions to free reserved pages mm/cris: use common help functions to free reserved pages mm/FRV: use common help functions to free reserved pages mm/h8300: use common help functions to free reserved pages mm/IA64: use common help functions to free reserved pages mm/m32r: use common help functions to free reserved pages mm/m68k: use common help functions to free reserved pages mm/microblaze: use common help functions to free reserved pages mm/MIPS: use common help functions to free reserved pages mm/mn10300: use common help functions to free reserved pages mm/openrisc: use common help functions to free reserved pages mm/parisc: use common help functions to free reserved pages mm/ppc: use common help functions to free reserved pages mm/s390: use common help functions to free reserved pages mm/score: use common help functions to free reserved pages mm/SH: use common help functions to free reserved pages mm/SPARC: use common help functions to free reserved pages mm/um: use common help functions to free reserved pages mm/unicore32: use common help functions to free reserved pages mm/x86: use common help functions to free reserved pages mm/xtensa: use common help functions to free reserved pages mm,kexec: use common help functions to free reserved pages
mm: introduce free_highmem_page() helper to free highmem pages inti
buddy system
mm: accurately calculate zone->managed_pages for highmem zones
mm: use a dedicated lock to protect totalram_pages and
Quoted 4 lines
    zone->managed_pages
mm: avoid using __free_pages_bootmem() at runtime
mm: correctly update zone->mamaged_pages
mm: report available pages as "MemTotal" for each NUMA node

Quoted 6 lines

I'm not sure what baseline your patches are based off of - however as part of 3.9-rc1, two new architectures were merged (arc and metag). It would be ideal if they got updated them as part of this series itself - if possible. Please let me know.
Thx,
-Vineet

Hi Vineet,
I have rebased the patchset to v3.9-rc1, but haven't noticed these two
new architectures. So how about the attached three patches?
Regards!
Gerry

arch/arc/mm/init.c +2 ~~-21~~

index caf797d..727d479 100644
@@ -144,37 +144,18 @@ void __init mem_init(void)
 		PAGES_TO_KB(reserved_pages));
 }
 
-static void __init free_init_pages(const char *what, unsigned long begin,
-				   unsigned long end)
-{
-	unsigned long addr;
-
-	pr_info("Freeing %s: %ldk [%lx] to [%lx]\n",
-		what, TO_KB(end - begin), begin, end);
-
-	/* need to check that the page we free is not a partial page */
-	for (addr = begin; addr + PAGE_SIZE <= end; addr += PAGE_SIZE) {
-		ClearPageReserved(virt_to_page(addr));
-		init_page_count(virt_to_page(addr));
-		free_page(addr);
-		totalram_pages++;
-	}
-}
-
 /*
  * free_initmem: Free all the __init memory.
  */
 void __init_refok free_initmem(void)
 {
-	free_init_pages("unused kernel memory",
-			(unsigned long)__init_begin,
-			(unsigned long)__init_end);
+	free_initmem_default(0);
 }
 
 #ifdef CONFIG_BLK_DEV_INITRD
 void __init free_initrd_mem(unsigned long start, unsigned long end)
 {
-	free_init_pages("initrd memory", start, end);
+	free_reserved_area(start, end, 0, "initrd");
 }
 #endif

Signature · 1 line

1.7.9.5

arch/metag/mm/init.c +2 ~~-19~~

index 504a398..c6784fb 100644
@@ -412,32 +412,15 @@ void __init mem_init(void)
 	return;
 }
 
-static void free_init_pages(char *what, unsigned long begin, unsigned long end)
-{
-	unsigned long addr;
-
-	for (addr = begin; addr < end; addr += PAGE_SIZE) {
-		ClearPageReserved(virt_to_page(addr));
-		init_page_count(virt_to_page(addr));
-		memset((void *)addr, POISON_FREE_INITMEM, PAGE_SIZE);
-		free_page(addr);
-		totalram_pages++;
-	}
-	pr_info("Freeing %s: %luk freed\n", what, (end - begin) >> 10);
-}
-
 void free_initmem(void)
 {
-	free_init_pages("unused kernel memory",
-			(unsigned long)(&__init_begin),
-			(unsigned long)(&__init_end));
+	free_initmem_default(POISON_FREE_INITMEM);
 }
 
 #ifdef CONFIG_BLK_DEV_INITRD
 void free_initrd_mem(unsigned long start, unsigned long end)
 {
-	end = end & PAGE_MASK;
-	free_init_pages("initrd memory", start, end);
+	free_reserved_area(start, end, POISON_FREE_INITMEM, "initrd");
 }
 #endif

Signature · 1 line

1.7.9.5

arch/arm/mm/init.c +2 -4

index 40a5bc2..400a383 100644
@@ -519,10 +519,8 @@ static void __init free_unused_memmap(struct meminfo *mi)
 #ifdef CONFIG_HIGHMEM
 static inline void free_area_high(unsigned long pfn, unsigned long end)
 {
-	for (; pfn < end; pfn++) {
-		__free_reserved_page(pfn_to_page(pfn));
-		totalhigh_pages++;
-	}
+	for (; pfn < end; pfn++)
+		free_highmem_page(pfn_to_page(pfn));
 }
 #endif

arch/metag/mm/init.c +2 -7

index c6784fb..7449af0 100644
@@ -380,13 +380,8 @@ void __init mem_init(void)
 
 #ifdef CONFIG_HIGHMEM
 	unsigned long tmp;
-	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++) {
-		struct page *page = pfn_to_page(tmp);
-		ClearPageReserved(page);
-		init_page_count(page);
-		__free_page(page);
-		totalhigh_pages++;
-	}
+	for (tmp = highstart_pfn; tmp < highend_pfn; tmp++)
+		free_highmem_page(pfn_to_page(tmp));
 	totalram_pages += totalhigh_pages;
 	num_physpages += totalhigh_pages;
 #endif /* CONFIG_HIGHMEM */

arch/microblaze/mm/init.c +1 -4

index 9be5302..d0fe2a8 100644
@@ -82,10 +82,7 @@ static unsigned long highmem_setup(void)
 		/* FIXME not sure about */
 		if (memblock_is_reserved(pfn << PAGE_SHIFT))
 			continue;
-		ClearPageReserved(page);
-		init_page_count(page);
-		__free_page(page);
-		totalhigh_pages++;
+		free_highmem_page(page);
 		reservedpages++;
 	}
 	totalram_pages += totalhigh_pages;

arch/mips/mm/init.c +1 -4

index 60f7c61..3105494 100644
@@ -393,10 +393,7 @@ void __init mem_init(void)
 			SetPageReserved(page);
 			continue;
 		}
-		ClearPageReserved(page);
-		init_page_count(page);
-		__free_page(page);
-		totalhigh_pages++;
+		free_highmem_page(page);
 	}
 	totalram_pages += totalhigh_pages;
 	num_physpages += totalhigh_pages;

arch/powerpc/mm/mem.c +1 -4

index c756713..79eb16b 100644
@@ -352,10 +352,7 @@ void __init mem_init(void)
 			struct page *page = pfn_to_page(pfn);
 			if (memblock_is_reserved(paddr))
 				continue;
-			ClearPageReserved(page);
-			init_page_count(page);
-			__free_page(page);
-			totalhigh_pages++;
+			free_higmem_page(page);
 			reservedpages--;
 		}
 		totalram_pages += totalhigh_pages;

arch/sparc/mm/init_32.c +2 -8

index 2a7b6eb..cd4c78c 100644
@@ -282,14 +282,8 @@ static void map_high_region(unsigned long start_pfn, unsigned long end_pfn)
 	printk("mapping high region %08lx - %08lx\n", start_pfn, end_pfn);
 #endif
 
-	for (tmp = start_pfn; tmp < end_pfn; tmp++) {
-		struct page *page = pfn_to_page(tmp);
-
-		ClearPageReserved(page);
-		init_page_count(page);
-		__free_page(page);
-		totalhigh_pages++;
-	}
+	for (tmp = start_pfn; tmp < end_pfn; tmp++)
+		free_higmem_page(pfn_to_page(tmp));
 }
 
 void __init mem_init(void)

arch/um/kernel/mem.c +3 ~~-12~~

index d5ac802..fea5c9d 100644
@@ -42,17 +42,12 @@ static unsigned long brk_end;
 static void setup_highmem(unsigned long highmem_start,
 			  unsigned long highmem_len)
 {
-	struct page *page;
 	unsigned long highmem_pfn;
 	int i;
 
 	highmem_pfn = __pa(highmem_start) >> PAGE_SHIFT;
-	for (i = 0; i < highmem_len >> PAGE_SHIFT; i++) {
-		page = &mem_map[highmem_pfn + i];
-		ClearPageReserved(page);
-		init_page_count(page);
-		__free_page(page);
-	}
+	for (i = 0; i < highmem_len >> PAGE_SHIFT; i++)
+		free_highmem_page(&mem_map[highmem_pfn + i]);
 }
 #endif

@@ -73,7 +68,7 @@ void __init mem_init(void)
 	totalram_pages = free_all_bootmem();
 	max_low_pfn = totalram_pages;
 #ifdef CONFIG_HIGHMEM
-	totalhigh_pages = highmem >> PAGE_SHIFT;
+	setup_highmem(end_iomem, highmem);
 	totalram_pages += totalhigh_pages;
 #endif
 	num_physpages = totalram_pages;

@@ -81,10 +76,6 @@ void __init mem_init(void)
 	printk(KERN_INFO "Memory: %luk available\n",
 	       nr_free_pages() << (PAGE_SHIFT-10));
 	kmalloc_ok = 1;
-
-#ifdef CONFIG_HIGHMEM
-	setup_highmem(end_iomem, highmem);
-#endif
 }
 
 /*

arch/x86/mm/init_32.c +1 -9

index 2d19001..3ac7e31 100644
@@ -427,14 +427,6 @@ static void __init permanent_kmaps_init(pgd_t *pgd_base)
 	pkmap_page_table = pte;
 }
 
-static void __init add_one_highpage_init(struct page *page)
-{
-	ClearPageReserved(page);
-	init_page_count(page);
-	__free_page(page);
-	totalhigh_pages++;
-}
-
 void __init add_highpages_with_active_regions(int nid,
 			 unsigned long start_pfn, unsigned long end_pfn)
 {

@@ -448,7 +440,7 @@ void __init add_highpages_with_active_regions(int nid,
 					      start_pfn, end_pfn);
 		for ( ; pfn < e_pfn; pfn++)
 			if (pfn_valid(pfn))
-				add_one_highpage_init(pfn_to_page(pfn));
+				free_highmem_page(pfn_to_page(pfn));
 	}
 }
 #else

include/linux/mm.h +3 -0

index 881461c..4d1509b 100644
@@ -1296,6 +1296,9 @@ extern void free_area_init_node(int nid, unsigned long * zones_size,
 extern void free_initmem(void);
 
 /* Help functions to deal with reserved/managed pages. */
+#ifdef	CONFIG_HIGHMEM
+extern void free_highmem_page(struct page *page);
+#endif
 extern unsigned long free_reserved_area(unsigned long start, unsigned long end,
 					int poison, char *s);

mm/page_alloc.c +8 -0

index 0fadb09..ad2f619 100644
@@ -5133,6 +5133,14 @@ unsigned long free_reserved_area(unsigned long start, unsigned long end,
 	return pages;
 }
 
+#ifdef	CONFIG_HIGHMEM
+void free_highmem_page(struct page *page)
+{
+	__free_reserved_page(page);
+	totalhigh_pages++;
+}
+#endif
+
 /**
  * set_dma_reserve - set the specified number of pages reserved in the first zone
  * @new_dma_reserve: The number of pages to mark reserved