Skip to contentSkip to conversation

mm: move core MM initialization to mm/mm_init.c

PATCH Superseded Mike Rapoport 3 participants 36 messages Mar 19, 2023 – Mar 20, 2023

Subsystems (8)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Mike Rapoport

author maintainer
#0 lore raw

cover mm: move core MM initialization to mm/mm_init.c

From: "Mike Rapoport (IBM)" <[email protected]>

Also in git:
https://git.kernel.org/rppt/h/mm-init/v1

This set moves most of the core MM initialization to mm/mm_init.c.

This largely includes free_area_init() and its helpers, functions used at boot time, mm_init() from init/main.c and some of the functions it calls.

Aside from gaining some more space before mm/page_alloc.c hits 10k lines, this makes mm/page_alloc.c to be mostly about buddy allocator and moves the init code out of the way, which IMO improves maintainability.

Besides, this allows to move a couple of declarations out of include/linux and make them private to mm/.

And as an added bonus there a slight decrease in vmlinux size. For tinyconfig and defconfig on x86 I've got

tinyconfig:
text data bss dec hex filename
853206 289376 1200128 2342710 23bf36 a/vmlinux
853198 289344 1200128 2342670 23bf0e b/vmlinux

defconfig:
text data bss dec hex filename
26152959 9730634 2170884 38054477 244aa4d a/vmlinux
26152945 9730602 2170884 38054431 244aa1f b/vmlinux

Mike Rapoport (IBM) (15):
mips: fix comment about pgtable_init()
mm/cma: move init_cma_reserved_pages() to cma.c and make it static mm/page_alloc: add helper for checking if check_pages_enabled
mm: move most of core MM initialization to mm/mm_init.c
mm: handle hashdist initialization in mm/mm_init.c mm/page_alloc: rename page_alloc_init() to page_alloc_init_cpuhp()
init: fold build_all_zonelists() and page_alloc_init_cpuhp() to mm_init() init,mm: move mm_init() to mm/mm_init.c and rename it to mm_core_init()
mm: move pgtable_init() to mm/mm_init.c and make it static
mm: move init_mem_debugging_and_hardening() to mm/mm_init.c init,mm: fold late call to page_ext_init() to page_alloc_init_late()
mm: move mem_init_print_info() to mm_init.c
mm: move kmem_cache_init() declaration to mm/slab.h
mm: move vmalloc_init() declaration to mm/internal.h
MAINTAINERS: extend memblock entry to include MM initialization

 MAINTAINERS                    |    3 +-
 arch/mips/include/asm/fixmap.h |    2 +-
 include/linux/gfp.h            |    7 +-
 include/linux/mm.h             |    9 +-
 include/linux/page_ext.h       |    2 -
 include/linux/slab.h           |    1 -
 include/linux/vmalloc.h        |    4 -
 init/main.c                    |   74 +-
 mm/cma.c                       |   21 +
 mm/internal.h                  |   45 +
 mm/mm_init.c                   | 2532 +++++++++++++++++++++++++++
 mm/page_alloc.c                | 2981 +++-----------------------------
 mm/slab.h                      |    1 +
 13 files changed, 2855 insertions(+), 2827 deletions(-)

base-commit: 4018ab1f7cec061b8425737328edefebdc0ab832

Signature · 1 line
2.35.1
author maintainer
#1 lore raw

PATCH 1/15 mips: fix comment about pgtable_init()

From: "Mike Rapoport (IBM)" <[email protected]>

Comment about fixrange_init() says that its called from pgtable_init() while the actual caller is pagetabe_init().

Update comment to match the code.

---
 arch/mips/include/asm/fixmap.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/mips/include/asm/fixmap.h +1 -1
index beea14761cef..b037718d7e8b 100644
@@ -70,7 +70,7 @@ enum fixed_addresses {
 #include <asm-generic/fixmap.h>
 
 /*
- * Called from pgtable_init()
+ * Called from pagetable_init()
  */
 extern void fixrange_init(unsigned long start, unsigned long end,
 	pgd_t *pgd_base);
Signature · 1 line
2.35.1
Reviewed-by on v1 1/15
#2 lore raw

Re: [PATCH 01/15] mips: fix comment about pgtable_init()

Quoted 17 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

Comment about fixrange_init() says that its called from pgtable_init() while the actual caller is pagetabe_init().

Update comment to match the code.

---
  arch/mips/include/asm/fixmap.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
arch/mips/include/asm/fixmap.h +0 -0
index beea14761cef..b037718d7e8b 100644

arch/mips/include/asm/fixmap.h enum fixed_addresses { · line 76

Quoted 8 lines
arch/mips/include/asm/fixmap.h +1 -1
@@ -70,7 +70,7 @@ enum fixed_addresses {
  #include <asm-generic/fixmap.h>
 
  /*
- * Called from pgtable_init()
+ * Called from pagetable_init()
   */
  extern void fixrange_init(unsigned long start, unsigned long end,
  	pgd_t *pgd_base);
Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#3 lore raw

PATCH 2/15 mm/cma: move init_cma_reserved_pages() to cma.c and make it static

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_cma_reserved_pages() only used in cma.c, no point of having it in page_alloc.c.

Move init_cma_reserved_pages() to cma.c and make it static.

---
 include/linux/gfp.h |  5 -----
 mm/cma.c            | 21 +++++++++++++++++++++
 mm/page_alloc.c     | 21 ---------------------
 3 files changed, 21 insertions(+), 26 deletions(-)
include/linux/gfp.h +0 -5
index 65a78773dcca..7c554e4bd49f 100644
@@ -361,9 +361,4 @@ extern struct page *alloc_contig_pages(unsigned long nr_pages, gfp_t gfp_mask,
 #endif
 void free_contig_range(unsigned long pfn, unsigned long nr_pages);
 
-#ifdef CONFIG_CMA
-/* CMA stuff */
-extern void init_cma_reserved_pageblock(struct page *page);
-#endif
-
 #endif /* __LINUX_GFP_H */
mm/cma.c +21 -0
index a7263aa02c92..ce08fb9825b4 100644
@@ -31,8 +31,10 @@
 #include <linux/highmem.h>
 #include <linux/io.h>
 #include <linux/kmemleak.h>
+#include <linux/page-isolation.h>
 #include <trace/events/cma.h>
 
+#include "internal.h"
 #include "cma.h"
 
 struct cma cma_areas[MAX_CMA_AREAS];
@@ -93,6 +95,25 @@ static void cma_clear_bitmap(struct cma *cma, unsigned long pfn,
 	spin_unlock_irqrestore(&cma->lock, flags);
 }
 
+/* Free whole pageblock and set its migration type to MIGRATE_CMA. */
+static void init_cma_reserved_pageblock(struct page *page)
+{
+	unsigned i = pageblock_nr_pages;
+	struct page *p = page;
+
+	do {
+		__ClearPageReserved(p);
+		set_page_count(p, 0);
+	} while (++p, --i);
+
+	set_pageblock_migratetype(page, MIGRATE_CMA);
+	set_page_refcounted(page);
+	__free_pages(page, pageblock_order);
+
+	adjust_managed_page_count(page, pageblock_nr_pages);
+	page_zone(page)->cma_pages += pageblock_nr_pages;
+}
+
 static void __init cma_activate_area(struct cma *cma)
 {
 	unsigned long base_pfn = cma->base_pfn, pfn;
mm/page_alloc.c +0 -21
index 87d760236dba..22e3da842e3f 100644
@@ -2280,27 +2280,6 @@ void __init page_alloc_init_late(void)
 		set_zone_contiguous(zone);
 }
 
-#ifdef CONFIG_CMA
-/* Free whole pageblock and set its migration type to MIGRATE_CMA. */
-void __init init_cma_reserved_pageblock(struct page *page)
-{
-	unsigned i = pageblock_nr_pages;
-	struct page *p = page;
-
-	do {
-		__ClearPageReserved(p);
-		set_page_count(p, 0);
-	} while (++p, --i);
-
-	set_pageblock_migratetype(page, MIGRATE_CMA);
-	set_page_refcounted(page);
-	__free_pages(page, pageblock_order);
-
-	adjust_managed_page_count(page, pageblock_nr_pages);
-	page_zone(page)->cma_pages += pageblock_nr_pages;
-}
-#endif
-
 /*
  * The order of subdivision here is critical for the IO subsystem.
  * Please do not alter this order without good reasons and regression
Signature · 1 line
2.35.1
#4 lore raw

Re: [PATCH 02/15] mm/cma: move init_cma_reserved_pages() to cma.c and make it static

Quoted 6 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_cma_reserved_pages() only used in cma.c, no point of having it in page_alloc.c.

Move init_cma_reserved_pages() to cma.c and make it static.

I guess the motivation is to avoid letting too many subsystems mess with pageblock migratetypes, managed pages, PG_reserved ...

So it kind of makes sense to have these low-level details out of common CMA code, no?

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#5 lore raw

Re: [PATCH 02/15] mm/cma: move init_cma_reserved_pages() to cma.c and make it static

Quoted 10 lines · David Hildenbrand
Quoted 6 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_cma_reserved_pages() only used in cma.c, no point of having it in page_alloc.c.

Move init_cma_reserved_pages() to cma.c and make it static.

I guess the motivation is to avoid letting too many subsystems mess with pageblock migratetypes, managed pages, PG_reserved ...

Judging by the git log it just ended up in page_alloc.c because set_pageblock_migratetype() was static back then ;)

Quoted 2 lines

So it kind of makes sense to have these low-level details out of common CMA code, no?

I don't mind keeping it out of cma and folding this into "grand move" patch.

Quoted 5 lines
Signature · 3 lines
Thanks,

David / dhildenb
Signature · 2 lines
Sincerely yours,
Mike.
#6 lore raw

Re: [PATCH 02/15] mm/cma: move init_cma_reserved_pages() to cma.c and make it static

Quoted 7 lines · Mike Rapoport
Quoted 6 lines · David Hildenbrand
Quoted 5 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_cma_reserved_pages() only used in cma.c, no point of having it in page_alloc.c.

Quoted 7 lines
Quoted 4 lines
Quoted 1 line

Move init_cma_reserved_pages() to cma.c and make it static.

I guess the motivation is to avoid letting too many subsystems mess with pageblock migratetypes, managed pages, PG_reserved ...

Judging by the git log it just ended up in page_alloc.c because set_pageblock_migratetype() was static back then ;)

Yeah, it leaked into page_isolation, which heavily relies on migratetype handling :)

Signature · 3 lines
Thanks,

David / dhildenb
#7 lore raw

Re: [PATCH 02/15] mm/cma: move init_cma_reserved_pages() to cma.c and make it static

In reply to Mike Rapoport
Quoted 12 lines · Mike Rapoport
Quoted 10 lines · David Hildenbrand
Quoted 6 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_cma_reserved_pages() only used in cma.c, no point of having it in page_alloc.c.

Move init_cma_reserved_pages() to cma.c and make it static.

I guess the motivation is to avoid letting too many subsystems mess with pageblock migratetypes, managed pages, PG_reserved ...

Quoted 8 lines

Judging by the git log it just ended up in page_alloc.c because set_pageblock_migratetype() was static back then ;)

Quoted 2 lines

So it kind of makes sense to have these low-level details out of common CMA code, no?

I don't mind keeping it out of cma and folding this into "grand move" patch.

Just an FYI, this conflicts with my patch:
https://lore.kernel.org/lkml/[email protected]/

So it would work better for me if it was folded into your "grand move" (assuming that refers to your patch 4) and init_cma_reserved_pageblock() could be retained as a wrapper there in my patch if we want to still keep set_pageblock_migratetype() out of the common CMA code.

--
Thanks,
Doug

author maintainer
#8 lore raw

PATCH 3/15 mm/page_alloc: add helper for checking if check_pages_enabled

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

Instead of duplicating long static_branch_enabled(&check_pages_enabled) wrap it in a helper function is_check_pages_enabled()

---
 mm/page_alloc.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
mm/page_alloc.c +8 -3
index 22e3da842e3f..e52f90d5d6a3 100644
@@ -245,6 +245,11 @@ EXPORT_SYMBOL(init_on_free);
 /* perform sanity checks on struct pages being allocated or freed */
 static DEFINE_STATIC_KEY_MAYBE(CONFIG_DEBUG_VM, check_pages_enabled);
 
+static inline bool is_check_pages_enabled(void)
+{
+	return static_branch_unlikely(&check_pages_enabled);
+}
+
 static bool _init_on_alloc_enabled_early __read_mostly
 				= IS_ENABLED(CONFIG_INIT_ON_ALLOC_DEFAULT_ON);
 static int __init early_init_on_alloc(char *buf)
@@ -1443,7 +1448,7 @@ static __always_inline bool free_pages_prepare(struct page *page,
 		for (i = 1; i < (1 << order); i++) {
 			if (compound)
 				bad += free_tail_pages_check(page, page + i);
-			if (static_branch_unlikely(&check_pages_enabled)) {
+			if (is_check_pages_enabled()) {
 				if (unlikely(free_page_is_bad(page + i))) {
 					bad++;
 					continue;
@@ -1456,7 +1461,7 @@ static __always_inline bool free_pages_prepare(struct page *page,
 		page->mapping = NULL;
 	if (memcg_kmem_online() && PageMemcgKmem(page))
 		__memcg_kmem_uncharge_page(page, order);
-	if (static_branch_unlikely(&check_pages_enabled)) {
+	if (is_check_pages_enabled()) {
 		if (free_page_is_bad(page))
 			bad++;
 		if (bad)
@@ -2345,7 +2350,7 @@ static int check_new_page(struct page *page)
 
 static inline bool check_new_pages(struct page *page, unsigned int order)
 {
-	if (static_branch_unlikely(&check_pages_enabled)) {
+	if (is_check_pages_enabled()) {
 		for (int i = 0; i < (1 << order); i++) {
 			struct page *p = page + i;
 
Signature · 1 line
2.35.1
Reviewed-by on v1 3/15
#9 lore raw

Re: [PATCH 03/15] mm/page_alloc: add helper for checking if check_pages_enabled

Quoted 6 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

Instead of duplicating long static_branch_enabled(&check_pages_enabled) wrap it in a helper function is_check_pages_enabled()

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#10 lore raw

PATCH 4/15 mm: move most of core MM initialization to mm/mm_init.c

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

The bulk of memory management initialization code is spread all over mm/page_alloc.c and makes navigating through page allocator functionality difficult.

Move most of the functions marked __init and __meminit to mm/mm_init.c to make it better localized and allow some more spare room before mm/page_alloc.c reaches 10k lines.

No functional changes.

---
 mm/internal.h   |   31 +
 mm/mm_init.c    | 2284 ++++++++++++++++++++++++++++++++++++++
 mm/page_alloc.c | 2837 +++++------------------------------------------
 3 files changed, 2583 insertions(+), 2569 deletions(-)
mm/internal.h +31 -0
index fce94775819c..6b154b4a538f 100644
@@ -202,6 +202,8 @@ pmd_t *mm_find_pmd(struct mm_struct *mm, unsigned long address);
  * in mm/page_alloc.c
  */
 
+extern char * const zone_names[MAX_NR_ZONES];
+
 /*
  * Structure for holding the mostly immutable allocation parameters passed
  * between functions involved in allocations, including the alloc_pages*
@@ -366,7 +368,29 @@ extern void __putback_isolated_page(struct page *page, unsigned int order,
 extern void memblock_free_pages(struct page *page, unsigned long pfn,
 					unsigned int order);
 extern void __free_pages_core(struct page *page, unsigned int order);
+
+static inline void prep_compound_head(struct page *page, unsigned int order)
+{
+	struct folio *folio = (struct folio *)page;
+
+	set_compound_page_dtor(page, COMPOUND_PAGE_DTOR);
+	set_compound_order(page, order);
+	atomic_set(&folio->_entire_mapcount, -1);
+	atomic_set(&folio->_nr_pages_mapped, 0);
+	atomic_set(&folio->_pincount, 0);
+}
+
+static inline void prep_compound_tail(struct page *head, int tail_idx)
+{
+	struct page *p = head + tail_idx;
+
+	p->mapping = TAIL_MAPPING;
+	set_compound_head(p, head);
+	set_page_private(p, 0);
+}
+
 extern void prep_compound_page(struct page *page, unsigned int order);
+
 extern void post_alloc_hook(struct page *page, unsigned int order,
 					gfp_t gfp_flags);
 extern int user_min_free_kbytes;
@@ -377,6 +401,7 @@ extern void free_unref_page_list(struct list_head *list);
 extern void zone_pcp_reset(struct zone *zone);
 extern void zone_pcp_disable(struct zone *zone);
 extern void zone_pcp_enable(struct zone *zone);
+extern void zone_pcp_init(struct zone *zone);
 
 extern void *memmap_alloc(phys_addr_t size, phys_addr_t align,
 			  phys_addr_t min_addr,
@@ -679,6 +704,12 @@ static inline loff_t fadvise_calc_endbyte(loff_t offset, loff_t len)
 }
 
 /* Memory initialisation debug and verification */
+#ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
+DECLARE_STATIC_KEY_TRUE(deferred_pages);
+
+bool __init deferred_grow_zone(struct zone *zone, unsigned int order);
+#endif /* CONFIG_DEFERRED_STRUCT_PAGE_INIT */
+
 enum mminit_level {
 	MMINIT_WARNING,
 	MMINIT_VERIFY,
mm/mm_init.c +2284 -0

2,294 lines not on this page · raw message

mm/page_alloc.c +268 -2569

650 lines not on this page · raw message

The rest of this diff is not shown. View the raw message.

Signature · 1 line
2.35.1

Part of the diff is not shown here. View the raw message.

Acked-by on v1 4/15
#11 lore raw

Re: [PATCH 04/15] mm: move most of core MM initialization to mm/mm_init.c

Quoted 6 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

The bulk of memory management initialization code is spread all over mm/page_alloc.c and makes navigating through page allocator functionality difficult.

Quoted 8 lines

Move most of the functions marked __init and __meminit to mm/mm_init.c to make it better localized and allow some more spare room before mm/page_alloc.c reaches 10k lines.

No functional changes.

---

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#12 lore raw

PATCH 5/15 mm: handle hashdist initialization in mm/mm_init.c

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

The hashdist variable must be initialized before the first call to alloc_large_system_hash() and free_area_init() looks like a better place for it than page_alloc_init().

Move hashdist handling to mm/mm_init.c

---
 mm/mm_init.c    | 22 ++++++++++++++++++++++
 mm/page_alloc.c | 18 ------------------
 2 files changed, 22 insertions(+), 18 deletions(-)
mm/mm_init.c +22 -0
index 63aa7b6b2880..8aaaddd13a20 100644
@@ -607,6 +607,25 @@ int __meminit early_pfn_to_nid(unsigned long pfn)
 
 	return nid;
 }
+
+int hashdist = HASHDIST_DEFAULT;
+
+static int __init set_hashdist(char *str)
+{
+	if (!str)
+		return 0;
+	hashdist = simple_strtoul(str, &str, 0);
+	return 1;
+}
+__setup("hashdist=", set_hashdist);
+
+static inline void fixup_hashdist(void)
+{
+	if (num_node_state(N_MEMORY) == 1)
+		hashdist = 0;
+}
+#else
+static inline void fixup_hashdist(void) {}
 #endif /* CONFIG_NUMA */
 
 #ifdef CONFIG_DEFERRED_STRUCT_PAGE_INIT
@@ -1855,6 +1874,9 @@ void __init free_area_init(unsigned long *max_zone_pfn)
 	}
 
 	memmap_init();
+
+	/* disable hash distribution for systems with a single node */
+	fixup_hashdist();
 }
 
 /**
mm/page_alloc.c +0 -18
index c56c147bdf27..ff6a2fff2880 100644
@@ -6383,28 +6383,10 @@ static int page_alloc_cpu_online(unsigned int cpu)
 	return 0;
 }
 
-#ifdef CONFIG_NUMA
-int hashdist = HASHDIST_DEFAULT;
-
-static int __init set_hashdist(char *str)
-{
-	if (!str)
-		return 0;
-	hashdist = simple_strtoul(str, &str, 0);
-	return 1;
-}
-__setup("hashdist=", set_hashdist);
-#endif
-
 void __init page_alloc_init(void)
 {
 	int ret;
 
-#ifdef CONFIG_NUMA
-	if (num_node_state(N_MEMORY) == 1)
-		hashdist = 0;
-#endif
-
 	ret = cpuhp_setup_state_nocalls(CPUHP_PAGE_ALLOC,
 					"mm/page_alloc:pcp",
 					page_alloc_cpu_online,
Signature · 1 line
2.35.1
Acked-by on v1 5/15
#13 lore raw

Re: [PATCH 05/15] mm: handle hashdist initialization in mm/mm_init.c

Quoted 10 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

The hashdist variable must be initialized before the first call to alloc_large_system_hash() and free_area_init() looks like a better place for it than page_alloc_init().

Move hashdist handling to mm/mm_init.c

---

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#14 lore raw

PATCH 6/15 mm/page_alloc: rename page_alloc_init() to page_alloc_init_cpuhp()

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

The page_alloc_init() name is really misleading because all this function does is sets up CPU hotplug callbacks for the page allocator.

Rename it to page_alloc_init_cpuhp() so that name will reflect what the function does.

---
 include/linux/gfp.h | 2 +-
 init/main.c         | 2 +-
 mm/page_alloc.c     | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)
include/linux/gfp.h +1 -1
index 7c554e4bd49f..ed8cb537c6a7 100644
@@ -319,7 +319,7 @@ extern void page_frag_free(void *addr);
 #define __free_page(page) __free_pages((page), 0)
 #define free_page(addr) free_pages((addr), 0)
 
-void page_alloc_init(void);
+void page_alloc_init_cpuhp(void);
 void drain_zone_pages(struct zone *zone, struct per_cpu_pages *pcp);
 void drain_all_pages(struct zone *zone);
 void drain_local_pages(struct zone *zone);
init/main.c +1 -1
index 4425d1783d5c..b2499bee7a3c 100644
@@ -969,7 +969,7 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
 	boot_cpu_hotplug_init();
 
 	build_all_zonelists(NULL);
-	page_alloc_init();
+	page_alloc_init_cpuhp();
 
 	pr_notice("Kernel command line: %s\n", saved_command_line);
 	/* parameters may set static keys */
mm/page_alloc.c +1 -1
index ff6a2fff2880..d1276bfe7a30 100644
@@ -6383,7 +6383,7 @@ static int page_alloc_cpu_online(unsigned int cpu)
 	return 0;
 }
 
-void __init page_alloc_init(void)
+void __init page_alloc_init_cpuhp(void)
 {
 	int ret;
 
Signature · 1 line
2.35.1
Reviewed-by on v1 6/15
#15 lore raw

Re: [PATCH 06/15] mm/page_alloc: rename page_alloc_init() to page_alloc_init_cpuhp()

Quoted 10 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

The page_alloc_init() name is really misleading because all this function does is sets up CPU hotplug callbacks for the page allocator.

Rename it to page_alloc_init_cpuhp() so that name will reflect what the function does.

---

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#16 lore raw

PATCH 7/15 init: fold build_all_zonelists() and page_alloc_init_cpuhp() to mm_init()

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

Both build_all_zonelists() and page_alloc_init_cpuhp() must be called after SMP setup is complete but before the page allocator is set up.

Still, they both are a part of memory management initialization, so move them to mm_init().

---
 init/main.c | 7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
init/main.c +4 -3
index b2499bee7a3c..4423906177c1 100644
@@ -833,6 +833,10 @@ static void __init report_meminit(void)
  */
 static void __init mm_init(void)
 {
+	/* Initializations relying on SMP setup */
+	build_all_zonelists(NULL);
+	page_alloc_init_cpuhp();
+
 	/*
 	 * page_ext requires contiguous pages,
 	 * bigger than MAX_ORDER unless SPARSEMEM.
@@ -968,9 +972,6 @@ asmlinkage __visible void __init __no_sanitize_address start_kernel(void)
 	smp_prepare_boot_cpu();	/* arch-specific boot-cpu hooks */
 	boot_cpu_hotplug_init();
 
-	build_all_zonelists(NULL);
-	page_alloc_init_cpuhp();
-
 	pr_notice("Kernel command line: %s\n", saved_command_line);
 	/* parameters may set static keys */
 	jump_label_init();
Signature · 1 line
2.35.1
Acked-by on v1 7/15
#17 lore raw

Re: [PATCH 07/15] init: fold build_all_zonelists() and page_alloc_init_cpuhp() to mm_init()

Quoted 9 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

Both build_all_zonelists() and page_alloc_init_cpuhp() must be called after SMP setup is complete but before the page allocator is set up.

Still, they both are a part of memory management initialization, so move them to mm_init().

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#18 lore raw

PATCH 8/15 init,mm: move mm_init() to mm/mm_init.c and rename it to mm_core_init()

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

Make mm_init() a part of mm/ codebase. mm_core_init() better describes what the function does and does not clash with mm_init() in kernel/fork.c

---
 include/linux/mm.h |  1 +
 init/main.c        | 71 ++------------------------------------------
 mm/mm_init.c       | 73 ++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 76 insertions(+), 69 deletions(-)
include/linux/mm.h +1 -0
index ee755bb4e1c1..2d7f095136fc 100644
@@ -39,6 +39,7 @@ struct pt_regs;
 
 extern int sysctl_page_lock_unfairness;
 
+void mm_core_init(void);
 void init_mm_internals(void);
 
 #ifndef CONFIG_NUMA		/* Don't use mapnrs, do it properly */
init/main.c +2 -69

88 lines not on this page · raw message

mm/mm_init.c +73 -0

85 lines not on this page · raw message

Signature · 1 line
2.35.1
Acked-by on v1 8/15
#19 lore raw

Re: [PATCH 08/15] init,mm: move mm_init() to mm/mm_init.c and rename it to mm_core_init()

Quoted 7 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

Make mm_init() a part of mm/ codebase. mm_core_init() better describes what the function does and does not clash with mm_init() in kernel/fork.c

---

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#20 lore raw

PATCH 9/15 mm: move pgtable_init() to mm/mm_init.c and make it static

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

pgtable_init() is only called from mm_core_init().

Move it close to the caller and make it static.

---
 include/linux/mm.h | 6 ------
 mm/mm_init.c       | 6 ++++++
 2 files changed, 6 insertions(+), 6 deletions(-)
include/linux/mm.h +0 -6
index 2d7f095136fc..c3c67d8bc833 100644
@@ -2782,12 +2782,6 @@ static inline bool ptlock_init(struct page *page) { return true; }
 static inline void ptlock_free(struct page *page) {}
 #endif /* USE_SPLIT_PTE_PTLOCKS */
 
-static inline void pgtable_init(void)
-{
-	ptlock_cache_init();
-	pgtable_cache_init();
-}
-
 static inline bool pgtable_pte_page_ctor(struct page *page)
 {
 	if (!ptlock_init(page))
mm/mm_init.c +6 -0
index 1da48762e4a2..a91fbb57c4cc 100644
@@ -2511,6 +2511,12 @@ void __init memblock_free_pages(struct page *page, unsigned long pfn,
 	__free_pages_core(page, order);
 }
 
+static void __init pgtable_init(void)
+{
+	ptlock_cache_init();
+	pgtable_cache_init();
+}
+
 /* Report memory auto-initialization states for this boot. */
 static void __init report_meminit(void)
 {
Signature · 1 line
2.35.1
#21 lore raw

Re: [PATCH 09/15] mm: move pgtable_init() to mm/mm_init.c and make it static

Quoted 5 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

pgtable_init() is only called from mm_core_init().

Move it close to the caller and make it static.

Why not inline it completely into the single caller? I don't see too much value of this indirection.

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#22 lore raw

PATCH 10/15 mm: move init_mem_debugging_and_hardening() to mm/mm_init.c

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_mem_debugging_and_hardening() is only called from mm_core_init().

Move it close to the caller and make it static.

---
 include/linux/mm.h |  1 -
 mm/internal.h      |  8 ++++
 mm/mm_init.c       | 89 +++++++++++++++++++++++++++++++++++++++++++
 mm/page_alloc.c    | 95 ----------------------------------------------
 4 files changed, 97 insertions(+), 96 deletions(-)
include/linux/mm.h +0 -1
index c3c67d8bc833..2fecabb1a328 100644
@@ -3394,7 +3394,6 @@ extern int apply_to_existing_page_range(struct mm_struct *mm,
 				   unsigned long address, unsigned long size,
 				   pte_fn_t fn, void *data);
 
-extern void __init init_mem_debugging_and_hardening(void);
 #ifdef CONFIG_PAGE_POISONING
 extern void __kernel_poison_pages(struct page *page, int numpages);
 extern void __kernel_unpoison_pages(struct page *page, int numpages);
mm/internal.h +8 -0
index 6b154b4a538f..827499e39d78 100644
@@ -204,6 +204,14 @@ pmd_t *mm_find_pmd(struct mm_struct *mm, unsigned long address);
 
 extern char * const zone_names[MAX_NR_ZONES];
 
+/* perform sanity checks on struct pages being allocated or freed */
+DECLARE_STATIC_KEY_MAYBE(CONFIG_DEBUG_VM, check_pages_enabled);
+
+static inline bool is_check_pages_enabled(void)
+{
+	return static_branch_unlikely(&check_pages_enabled);
+}
+
 /*
  * Structure for holding the mostly immutable allocation parameters passed
  * between functions involved in allocations, including the alloc_pages*
mm/mm_init.c +89 -0

95 lines not on this page · raw message

mm/page_alloc.c +0 -95

107 lines not on this page · raw message

Signature · 1 line
2.35.1
Acked-by on v1 10/15
#23 lore raw

Re: [PATCH 10/15] mm: move init_mem_debugging_and_hardening() to mm/mm_init.c

Quoted 4 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_mem_debugging_and_hardening() is only called from mm_core_init().

While at it, maybe rename it to ..._init() like most other such functions in there?

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#24 lore raw

Re: [PATCH 10/15] mm: move init_mem_debugging_and_hardening() to mm/mm_init.c

Quoted 8 lines · David Hildenbrand
Quoted 4 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_mem_debugging_and_hardening() is only called from mm_core_init().

While at it, maybe rename it to ..._init() like most other such functions in there?

Makes sense, but I wouldn't want to post v2 only for this.

Quoted 7 lines
Signature · 3 lines
Thanks,

David / dhildenb
Signature · 2 lines
Sincerely yours,
Mike.
#25 lore raw

Re: [PATCH 10/15] mm: move init_mem_debugging_and_hardening() to mm/mm_init.c

Quoted 11 lines · Mike Rapoport
Quoted 8 lines · David Hildenbrand
Quoted 4 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

init_mem_debugging_and_hardening() is only called from mm_core_init().

While at it, maybe rename it to ..._init() like most other such functions in there?

Makes sense, but I wouldn't want to post v2 only for this.

We can fixup on top, sure.

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#26 lore raw

PATCH 11/15 init,mm: fold late call to page_ext_init() to page_alloc_init_late()

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

When deferred initialization of struct pages is enabled, page_ext_init() must be called after all the deferred initialization is done, but there is no point to keep it a separate call from kernel_init_freeable() right after page_alloc_init_late().

Fold the call to page_ext_init() into page_alloc_init_late() and localize deferred_struct_pages variable.

---
 include/linux/page_ext.h | 2 --
 init/main.c              | 4 ----
 mm/mm_init.c             | 6 +++++-
 3 files changed, 5 insertions(+), 7 deletions(-)
include/linux/page_ext.h +0 -2
index bc2e39090a1f..67314f648aeb 100644
@@ -29,8 +29,6 @@ struct page_ext_operations {
 	bool need_shared_flags;
 };
 
-extern bool deferred_struct_pages;
-
 #ifdef CONFIG_PAGE_EXTENSION
 
 /*
init/main.c +0 -4
index 8a20b4c25f24..04113514e56a 100644
@@ -62,7 +62,6 @@
 #include <linux/rmap.h>
 #include <linux/mempolicy.h>
 #include <linux/key.h>
-#include <linux/page_ext.h>
 #include <linux/debug_locks.h>
 #include <linux/debugobjects.h>
 #include <linux/lockdep.h>
@@ -1561,9 +1560,6 @@ static noinline void __init kernel_init_freeable(void)
 
 	padata_init();
 	page_alloc_init_late();
-	/* Initialize page ext after all struct pages are initialized. */
-	if (deferred_struct_pages)
-		page_ext_init();
 
 	do_basic_setup();
 
mm/mm_init.c +5 -1
index ae6bd26cf5a2..2d73d8b05a69 100644
@@ -225,7 +225,7 @@ static unsigned long nr_kernel_pages __initdata;
 static unsigned long nr_all_pages __initdata;
 static unsigned long dma_reserve __initdata;
 
-bool deferred_struct_pages __meminitdata;
+static bool deferred_struct_pages __meminitdata;
 
 static DEFINE_PER_CPU(struct per_cpu_nodestat, boot_nodestats);
 
@@ -2338,6 +2338,10 @@ void __init page_alloc_init_late(void)
 
 	for_each_populated_zone(zone)
 		set_zone_contiguous(zone);
+
+	/* Initialize page ext after all struct pages are initialized. */
+	if (deferred_struct_pages)
+		page_ext_init();
 }
 
 #ifndef __HAVE_ARCH_RESERVED_KERNEL_PAGES
Signature · 1 line
2.35.1
Reviewed-by on v1 11/15
#27 lore raw

Re: [PATCH 11/15] init,mm: fold late call to page_ext_init() to page_alloc_init_late()

Quoted 11 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

When deferred initialization of struct pages is enabled, page_ext_init() must be called after all the deferred initialization is done, but there is no point to keep it a separate call from kernel_init_freeable() right after page_alloc_init_late().

Fold the call to page_ext_init() into page_alloc_init_late() and localize deferred_struct_pages variable.

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#28 lore raw

PATCH 12/15 mm: move mem_init_print_info() to mm_init.c

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

mem_init_print_info() is only called from mm_core_init().

Move it close to the caller and make it static.

---
 include/linux/mm.h |  1 -
 mm/internal.h      |  1 +
 mm/mm_init.c       | 53 ++++++++++++++++++++++++++++++++++++++++++++++
 mm/page_alloc.c    | 53 ----------------------------------------------
 4 files changed, 54 insertions(+), 54 deletions(-)
include/linux/mm.h +0 -1
index 2fecabb1a328..e249208f8fbe 100644
@@ -2925,7 +2925,6 @@ extern unsigned long free_reserved_area(void *start, void *end,
 					int poison, const char *s);
 
 extern void adjust_managed_page_count(struct page *page, long count);
-extern void mem_init_print_info(void);
 
 extern void reserve_bootmem_region(phys_addr_t start, phys_addr_t end);
 
mm/internal.h +1 -0
index 827499e39d78..1be4278d7913 100644
@@ -201,6 +201,7 @@ pmd_t *mm_find_pmd(struct mm_struct *mm, unsigned long address);
 /*
  * in mm/page_alloc.c
  */
+#define K(x) ((x) << (PAGE_SHIFT-10))
 
 extern char * const zone_names[MAX_NR_ZONES];
 
mm/mm_init.c +53 -0

65 lines not on this page · raw message

mm/page_alloc.c +0 -53

65 lines not on this page · raw message

Signature · 1 line
2.35.1
author maintainer
#30 lore raw

PATCH 13/15 mm: move kmem_cache_init() declaration to mm/slab.h

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

kmem_cache_init() is called only from mm_core_init(), there is no need to declare it in include/linux/slab.h

Move kmem_cache_init() declaration to mm/slab.h

---
 include/linux/slab.h | 1 -
 mm/mm_init.c         | 1 +
 mm/slab.h            | 1 +
 3 files changed, 2 insertions(+), 1 deletion(-)
include/linux/slab.h +0 -1
index aa4575ef2965..f8b1d63c63a3 100644
@@ -167,7 +167,6 @@ struct mem_cgroup;
 /*
  * struct kmem_cache related prototypes
  */
-void __init kmem_cache_init(void);
 bool slab_is_available(void);
 
 struct kmem_cache *kmem_cache_create(const char *name, unsigned int size,
mm/mm_init.c +1 -0
index 73964449669e..78b2366e4407 100644
@@ -27,6 +27,7 @@
 #include <linux/swap.h>
 #include <linux/cma.h>
 #include "internal.h"
+#include "slab.h"
 #include "shuffle.h"
 
 #include <asm/setup.h>
mm/slab.h +1 -0
index 43966aa5fadf..3f8df2244f5a 100644
@@ -4,6 +4,7 @@
 /*
  * Internal slab definitions
  */
+void __init kmem_cache_init(void);
 
 /* Reuses the bits in struct page */
 struct slab {
Signature · 1 line
2.35.1
Reviewed-by on v1 13/15
#31 lore raw

Re: [PATCH 13/15] mm: move kmem_cache_init() declaration to mm/slab.h

Quoted 9 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

kmem_cache_init() is called only from mm_core_init(), there is no need to declare it in include/linux/slab.h

Move kmem_cache_init() declaration to mm/slab.h

---

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#32 lore raw

PATCH 14/15 mm: move vmalloc_init() declaration to mm/internal.h

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

vmalloc_init() is called only from mm_core_init(), there is no need to declare it in include/linux/vmalloc.h

Move vmalloc_init() declaration to mm/internal.h

---
 include/linux/vmalloc.h | 4 ----
 mm/internal.h           | 5 +++++
 2 files changed, 5 insertions(+), 4 deletions(-)
include/linux/vmalloc.h +0 -4
index 69250efa03d1..351fc7697214 100644
@@ -131,12 +131,8 @@ extern void *vm_map_ram(struct page **pages, unsigned int count, int node);
 extern void vm_unmap_aliases(void);
 
 #ifdef CONFIG_MMU
-extern void __init vmalloc_init(void);
 extern unsigned long vmalloc_nr_pages(void);
 #else
-static inline void vmalloc_init(void)
-{
-}
 static inline unsigned long vmalloc_nr_pages(void) { return 0; }
 #endif
 
mm/internal.h +5 -0
index 1be4278d7913..7e22137b4e86 100644
@@ -895,9 +895,14 @@ size_t splice_folio_into_pipe(struct pipe_inode_info *pipe,
  * mm/vmalloc.c
  */
 #ifdef CONFIG_MMU
+void __init vmalloc_init(void);
 int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
                 pgprot_t prot, struct page **pages, unsigned int page_shift);
 #else
+static inline void vmalloc_init(void)
+{
+}
+
 static inline
 int vmap_pages_range_noflush(unsigned long addr, unsigned long end,
                 pgprot_t prot, struct page **pages, unsigned int page_shift)
Signature · 1 line
2.35.1
Reviewed-by on v1 14/15
#33 lore raw

Re: [PATCH 14/15] mm: move vmalloc_init() declaration to mm/internal.h

Quoted 9 lines · Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

vmalloc_init() is called only from mm_core_init(), there is no need to declare it in include/linux/vmalloc.h

Move vmalloc_init() declaration to mm/internal.h

---

Signature · 3 lines
Thanks,

David / dhildenb
author maintainer
#34 lore raw

PATCH 15/15 MAINTAINERS: extend memblock entry to include MM initialization

In reply to Mike Rapoport

From: "Mike Rapoport (IBM)" <[email protected]>

and add mm/mm_init.c to memblock entry in MAINTAINERS

---
 MAINTAINERS | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
MAINTAINERS +2 -1
index 7002a5d3eb62..b79463ea1049 100644
@@ -13368,13 +13368,14 @@ F:	arch/powerpc/include/asm/membarrier.h
 F:	include/uapi/linux/membarrier.h
 F:	kernel/sched/membarrier.c
 
-MEMBLOCK
+MEMBLOCK AND MEMORY MANAGEMENT INITIALIZATION
 M:	Mike Rapoport <[email protected]>
 L:	[email protected]
 S:	Maintained
 F:	Documentation/core-api/boot-time-mm.rst
 F:	include/linux/memblock.h
 F:	mm/memblock.c
+F:	mm/mm_init.c
 F:	tools/testing/memblock/
 
 MEMORY CONTROLLER DRIVERS
Signature · 1 line
2.35.1