Skip to contentSkip to conversation

Clean up hugetlb boot command line processing

PATCH Superseded Mike Kravetz 10 participants 23 messages Apr 17, 2020 – Apr 28, 2020

Subsystems (12)

v3 of 4 Superseded 1 of 4 applied Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v3 cover Clean up hugetlb boot command line processing

v3 -
Used weak attribute method of defining arch_hugetlb_valid_size.
This eliminates changes to arch specific hugetlb.h files (Peter)
Updated documentation (Peter, Randy)
Fixed handling of implicitly specified gigantic page preallocation
in existing code and removed documentation of such. There is now no difference between handling of gigantic and non-gigantic pages. (Peter, Nitesh).
This requires the most review as there is a small change to undocumented behavior. See patch 4 commit message for details.
Added Acks and Reviews (Mina, Peter)

v2 -
Fix build errors with patch 1 (Will)
Change arch_hugetlb_valid_size arg to unsigned long and remove
irrelevant 'extern' keyword (Christophe)
Documentation and other misc changes (Randy, Christophe, Mina) Do not process command line options if !hugepages_supported()
(Dave, but it sounds like we may want to additional changes to
hugepages_supported() for x86? If that is needed I would prefer a separate patch.)

Longpeng(Mike) reported a weird message from hugetlb command line processing and proposed a solution [1]. While the proposed patch does address the specific issue, there are other related issues in command line processing. As hugetlbfs evolved, updates to command line processing have been made to meet immediate needs and not necessarily in a coordinated manner. The result is that some processing is done in arch specific code, some is done in arch independent code and coordination is problematic. Semantics can vary between architectures.

The patch series does the following:
- Define arch specific arch_hugetlb_valid_size routine used to validate passed huge page sizes.
- Move hugepagesz= command line parsing out of arch specific code and into an arch independent routine.
- Clean up command line processing to follow desired semantics and document those semantics.

[1] https://lore.kernel.org/linux-mm/[email protected]

Mike Kravetz (4):
hugetlbfs: add arch_hugetlb_valid_size
hugetlbfs: move hugepagesz= parsing to arch independent code
hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate
hugetlbfs: clean up command line processing

 .../admin-guide/kernel-parameters.txt         |  40 ++--
 Documentation/admin-guide/mm/hugetlbpage.rst  |  35 ++++
 arch/arm64/mm/hugetlbpage.c                   |  30 +--
 arch/powerpc/mm/hugetlbpage.c                 |  30 +--
 arch/riscv/mm/hugetlbpage.c                   |  24 +--
 arch/s390/mm/hugetlbpage.c                    |  24 +--
 arch/sparc/mm/init_64.c                       |  43 +---
 arch/x86/mm/hugetlbpage.c                     |  23 +--
 include/linux/hugetlb.h                       |   2 +-
 mm/hugetlb.c                                  | 190 +++++++++++++++---
 10 files changed, 271 insertions(+), 170 deletions(-)
Signature · 1 line
2.25.2
#1 lore raw

PATCH v3 1/4 hugetlbfs: add arch_hugetlb_valid_size

The architecture independent routine hugetlb_default_setup sets up the default huge pages size. It has no way to verify if the passed value is valid, so it accepts it and attempts to validate at a later time. This requires undocumented cooperation between the arch specific and arch independent code.

For architectures that support more than one huge page size, provide a routine arch_hugetlb_valid_size to validate a huge page size. hugetlb_default_setup can use this to validate passed values.

arch_hugetlb_valid_size will also be used in a subsequent patch to move processing of the "hugepagesz=" in arch specific code to a common routine in arch independent code.

---
 arch/arm64/mm/hugetlbpage.c   | 17 +++++++++++++----
 arch/powerpc/mm/hugetlbpage.c | 20 +++++++++++++-------
 arch/riscv/mm/hugetlbpage.c   | 26 +++++++++++++++++---------
 arch/s390/mm/hugetlbpage.c    | 16 ++++++++++++----
 arch/sparc/mm/init_64.c       | 24 ++++++++++++++++--------
 arch/x86/mm/hugetlbpage.c     | 17 +++++++++++++----
 include/linux/hugetlb.h       |  1 +
 mm/hugetlb.c                  | 21 ++++++++++++++++++---
 8 files changed, 103 insertions(+), 39 deletions(-)
arch/arm64/mm/hugetlbpage.c +13 -4
index bbeb6a5a6ba6..069b96ee2aec 100644
@@ -462,17 +462,26 @@ static int __init hugetlbpage_init(void)
 }
 arch_initcall(hugetlbpage_init);
 
-static __init int setup_hugepagesz(char *opt)
+bool __init arch_hugetlb_valid_size(unsigned long size)
 {
-	unsigned long ps = memparse(opt, &opt);
-
-	switch (ps) {
+	switch (size) {
 #ifdef CONFIG_ARM64_4K_PAGES
 	case PUD_SIZE:
 #endif
 	case CONT_PMD_SIZE:
 	case PMD_SIZE:
 	case CONT_PTE_SIZE:
+		return true;
+	}
+
+	return false;
+}
+
+static __init int setup_hugepagesz(char *opt)
+{
+	unsigned long ps = memparse(opt, &opt);
+
+	if (arch_hugetlb_valid_size(ps)) {
 		add_huge_page_size(ps);
 		return 1;
 	}
arch/powerpc/mm/hugetlbpage.c +13 -7
index 33b3461d91e8..de54d2a37830 100644
@@ -558,7 +558,7 @@ unsigned long vma_mmu_pagesize(struct vm_area_struct *vma)
 	return vma_kernel_pagesize(vma);
 }
 
-static int __init add_huge_page_size(unsigned long long size)
+bool __init arch_hugetlb_valid_size(unsigned long size)
 {
 	int shift = __ffs(size);
 	int mmu_psize;
@@ -566,20 +566,26 @@ static int __init add_huge_page_size(unsigned long long size)
 	/* Check that it is a page size supported by the hardware and
 	 * that it fits within pagetable and slice limits. */
 	if (size <= PAGE_SIZE || !is_power_of_2(size))
-		return -EINVAL;
+		return false;
 
 	mmu_psize = check_and_get_huge_psize(shift);
 	if (mmu_psize < 0)
-		return -EINVAL;
+		return false;
 
 	BUG_ON(mmu_psize_defs[mmu_psize].shift != shift);
 
-	/* Return if huge page size has already been setup */
-	if (size_to_hstate(size))
-		return 0;
+	return true;
+}
 
-	hugetlb_add_hstate(shift - PAGE_SHIFT);
+static int __init add_huge_page_size(unsigned long long size)
+{
+	int shift = __ffs(size);
+
+	if (!arch_hugetlb_valid_size((unsigned long)size))
+		return -EINVAL;
 
+	if (!size_to_hstate(size))
+		hugetlb_add_hstate(shift - PAGE_SHIFT);
 	return 0;
 }
 
arch/riscv/mm/hugetlbpage.c +17 -9
index a6189ed36c5f..da1f516bc451 100644
@@ -12,21 +12,29 @@ int pmd_huge(pmd_t pmd)
 	return pmd_leaf(pmd);
 }
 
+bool __init arch_hugetlb_valid_size(unsigned long size)
+{
+	if (size = HPAGE_SIZE)
+		return true;
+	else if (IS_ENABLED(CONFIG_64BIT) && size = PUD_SIZE)
+		return true;
+	else
+		return false;
+}
+
 static __init int setup_hugepagesz(char *opt)
 {
 	unsigned long ps = memparse(opt, &opt);
 
-	if (ps = HPAGE_SIZE) {
-		hugetlb_add_hstate(HPAGE_SHIFT - PAGE_SHIFT);
-	} else if (IS_ENABLED(CONFIG_64BIT) && ps = PUD_SIZE) {
-		hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
-	} else {
-		hugetlb_bad_size();
-		pr_err("hugepagesz: Unsupported page size %lu M\n", ps >> 20);
-		return 0;
+	if (arch_hugetlb_valid_size(ps)) {
+		hugetlb_add_hstate(ilog2(ps) - PAGE_SHIFT);
+		return 1;
 	}
 
-	return 1;
+	hugetlb_bad_size();
+	pr_err("hugepagesz: Unsupported page size %lu M\n", ps >> 20);
+	return 0;
+
 }
 __setup("hugepagesz=", setup_hugepagesz);
 
arch/s390/mm/hugetlbpage.c +12 -4
index f01daddcbc5e..ac25b207624c 100644
@@ -251,16 +251,24 @@ follow_huge_pud(struct mm_struct *mm, unsigned long address,
 	return pud_page(*pud) + ((address & ~PUD_MASK) >> PAGE_SHIFT);
 }
 
+bool __init arch_hugetlb_valid_size(unsigned long size)
+{
+	if (MACHINE_HAS_EDAT1 && size = PMD_SIZE)
+		return true;
+	else if (MACHINE_HAS_EDAT2 && size = PUD_SIZE)
+		return true;
+	else
+		return false;
+}
+
 static __init int setup_hugepagesz(char *opt)
 {
 	unsigned long size;
 	char *string = opt;
 
 	size = memparse(opt, &opt);
-	if (MACHINE_HAS_EDAT1 && size = PMD_SIZE) {
-		hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-	} else if (MACHINE_HAS_EDAT2 && size = PUD_SIZE) {
-		hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+	if (arch_hugetlb_valid_size(size)) {
+		hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
 	} else {
 		hugetlb_bad_size();
 		pr_err("hugepagesz= specifies an unsupported page size %s\n",
arch/sparc/mm/init_64.c +16 -8
index 1cf0d666dea3..2bfe8e22b706 100644
@@ -360,16 +360,11 @@ static void __init pud_huge_patch(void)
 	__asm__ __volatile__("flush %0" : : "r" (addr));
 }
 
-static int __init setup_hugepagesz(char *string)
+bool __init arch_hugetlb_valid_size(unsigned long size)
 {
-	unsigned long long hugepage_size;
-	unsigned int hugepage_shift;
+	unsigned int hugepage_shift = ilog2(size);
 	unsigned short hv_pgsz_idx;
 	unsigned int hv_pgsz_mask;
-	int rc = 0;
-
-	hugepage_size = memparse(string, &string);
-	hugepage_shift = ilog2(hugepage_size);
 
 	switch (hugepage_shift) {
 	case HPAGE_16GB_SHIFT:
@@ -397,7 +392,20 @@ static int __init setup_hugepagesz(char *string)
 		hv_pgsz_mask = 0;
 	}
 
-	if ((hv_pgsz_mask & cpu_pgsz_mask) = 0U) {
+	if ((hv_pgsz_mask & cpu_pgsz_mask) = 0U)
+		return false;
+
+	return true;
+}
+
+static int __init setup_hugepagesz(char *string)
+{
+	unsigned long long hugepage_size;
+	int rc = 0;
+
+	hugepage_size = memparse(string, &string);
+
+	if (!arch_hugetlb_valid_size((unsigned long)hugepage_size)) {
 		hugetlb_bad_size();
 		pr_err("hugepagesz=%llu not supported by MMU.\n",
 			hugepage_size);
arch/x86/mm/hugetlbpage.c +13 -4
index 5bfd5aef5378..1c4372bfe782 100644
@@ -181,13 +181,22 @@ hugetlb_get_unmapped_area(struct file *file, unsigned long addr,
 #endif /* CONFIG_HUGETLB_PAGE */
 
 #ifdef CONFIG_X86_64
+bool __init arch_hugetlb_valid_size(unsigned long size)
+{
+	if (size = PMD_SIZE)
+		return true;
+	else if (size = PUD_SIZE && boot_cpu_has(X86_FEATURE_GBPAGES))
+		return true;
+	else
+		return false;
+}
+
 static __init int setup_hugepagesz(char *opt)
 {
 	unsigned long ps = memparse(opt, &opt);
-	if (ps = PMD_SIZE) {
-		hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-	} else if (ps = PUD_SIZE && boot_cpu_has(X86_FEATURE_GBPAGES)) {
-		hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+
+	if (arch_hugetlb_valid_size(ps)) {
+		hugetlb_add_hstate(ilog2(ps) - PAGE_SHIFT);
 	} else {
 		hugetlb_bad_size();
 		printk(KERN_ERR "hugepagesz: Unsupported page size %lu M\n",
include/linux/hugetlb.h +1 -0
index 43a1cef8f0f1..2eb15f5ab01e 100644
@@ -521,6 +521,7 @@ int __init alloc_bootmem_huge_page(struct hstate *h);
 
 void __init hugetlb_bad_size(void);
 void __init hugetlb_add_hstate(unsigned order);
+bool __init arch_hugetlb_valid_size(unsigned long size);
 struct hstate *size_to_hstate(unsigned long size);
 
 #ifndef HUGE_MAX_HSTATE
mm/hugetlb.c +18 -3
index cd459155d28a..871ac1681f37 100644
@@ -3256,6 +3256,12 @@ static int __init hugetlb_init(void)
 }
 subsys_initcall(hugetlb_init);
 
+/* Overwritten by architectures with more huge page sizes */
+bool __init __attribute((weak)) arch_hugetlb_valid_size(unsigned long size)
+{
+	return size = HPAGE_SIZE;
+}
+
 /* Should be called on processing a hugepagesz=... option */
 void __init hugetlb_bad_size(void)
 {
@@ -3331,12 +3337,21 @@ static int __init hugetlb_nrpages_setup(char *s)
 }
 __setup("hugepages=", hugetlb_nrpages_setup);
 
-static int __init hugetlb_default_setup(char *s)
+static int __init default_hugepagesz_setup(char *s)
 {
-	default_hstate_size = memparse(s, &s);
+	unsigned long size;
+
+	size = (unsigned long)memparse(s, NULL);
+
+	if (!arch_hugetlb_valid_size(size)) {
+		pr_err("HugeTLB: unsupported default_hugepagesz %s\n", s);
+		return 0;
+	}
+
+	default_hstate_size = size;
 	return 1;
 }
-__setup("default_hugepagesz=", hugetlb_default_setup);
+__setup("default_hugepagesz=", default_hugepagesz_setup);
 
 static unsigned int cpuset_mems_nr(unsigned int *array)
 {
Signature · 1 line
2.25.2
#2 lore raw

PATCH v3 2/4 hugetlbfs: move hugepagesz= parsing to arch independent code

In reply to Mike Kravetz

Now that architectures provide arch_hugetlb_valid_size(), parsing of "hugepagesz=" can be done in architecture independent code. Create a single routine to handle hugepagesz= parsing and remove all arch specific routines. We can also remove the interface hugetlb_bad_size() as this is no longer used outside arch independent code.

This also provides consistent behavior of hugetlbfs command line options. The hugepagesz= option should only be specified once for a specific size, but some architectures allow multiple instances. This appears to be more of an oversight when code was added by some architectures to set up ALL huge pages sizes.

---
 arch/arm64/mm/hugetlbpage.c   | 15 ---------------
 arch/powerpc/mm/hugetlbpage.c | 15 ---------------
 arch/riscv/mm/hugetlbpage.c   | 16 ----------------
 arch/s390/mm/hugetlbpage.c    | 18 ------------------
 arch/sparc/mm/init_64.c       | 22 ----------------------
 arch/x86/mm/hugetlbpage.c     | 16 ----------------
 include/linux/hugetlb.h       |  1 -
 mm/hugetlb.c                  | 23 +++++++++++++++++------
 8 files changed, 17 insertions(+), 109 deletions(-)
arch/arm64/mm/hugetlbpage.c +0 -15
index 069b96ee2aec..f706b821aba6 100644
@@ -476,18 +476,3 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 
 	return false;
 }
-
-static __init int setup_hugepagesz(char *opt)
-{
-	unsigned long ps = memparse(opt, &opt);
-
-	if (arch_hugetlb_valid_size(ps)) {
-		add_huge_page_size(ps);
-		return 1;
-	}
-
-	hugetlb_bad_size();
-	pr_err("hugepagesz: Unsupported page size %lu K\n", ps >> 10);
-	return 0;
-}
-__setup("hugepagesz=", setup_hugepagesz);
arch/powerpc/mm/hugetlbpage.c +0 -15
index de54d2a37830..2c3fa0a7787b 100644
@@ -589,21 +589,6 @@ static int __init add_huge_page_size(unsigned long long size)
 	return 0;
 }
 
-static int __init hugepage_setup_sz(char *str)
-{
-	unsigned long long size;
-
-	size = memparse(str, &str);
-
-	if (add_huge_page_size(size) != 0) {
-		hugetlb_bad_size();
-		pr_err("Invalid huge page size specified(%llu)\n", size);
-	}
-
-	return 1;
-}
-__setup("hugepagesz=", hugepage_setup_sz);
-
 static int __init hugetlbpage_init(void)
 {
 	bool configured = false;
arch/riscv/mm/hugetlbpage.c +0 -16
index da1f516bc451..4e5d7e9f0eef 100644
@@ -22,22 +22,6 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 		return false;
 }
 
-static __init int setup_hugepagesz(char *opt)
-{
-	unsigned long ps = memparse(opt, &opt);
-
-	if (arch_hugetlb_valid_size(ps)) {
-		hugetlb_add_hstate(ilog2(ps) - PAGE_SHIFT);
-		return 1;
-	}
-
-	hugetlb_bad_size();
-	pr_err("hugepagesz: Unsupported page size %lu M\n", ps >> 20);
-	return 0;
-
-}
-__setup("hugepagesz=", setup_hugepagesz);
-
 #ifdef CONFIG_CONTIG_ALLOC
 static __init int gigantic_pages_init(void)
 {
arch/s390/mm/hugetlbpage.c +0 -18
index ac25b207624c..242dfc0d462d 100644
@@ -261,24 +261,6 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 		return false;
 }
 
-static __init int setup_hugepagesz(char *opt)
-{
-	unsigned long size;
-	char *string = opt;
-
-	size = memparse(opt, &opt);
-	if (arch_hugetlb_valid_size(size)) {
-		hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
-	} else {
-		hugetlb_bad_size();
-		pr_err("hugepagesz= specifies an unsupported page size %s\n",
-			string);
-		return 0;
-	}
-	return 1;
-}
-__setup("hugepagesz=", setup_hugepagesz);
-
 static unsigned long hugetlb_get_unmapped_area_bottomup(struct file *file,
 		unsigned long addr, unsigned long len,
 		unsigned long pgoff, unsigned long flags)
arch/sparc/mm/init_64.c +0 -22
index 2bfe8e22b706..4618f96fd30f 100644
@@ -397,28 +397,6 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 
 	return true;
 }
-
-static int __init setup_hugepagesz(char *string)
-{
-	unsigned long long hugepage_size;
-	int rc = 0;
-
-	hugepage_size = memparse(string, &string);
-
-	if (!arch_hugetlb_valid_size((unsigned long)hugepage_size)) {
-		hugetlb_bad_size();
-		pr_err("hugepagesz=%llu not supported by MMU.\n",
-			hugepage_size);
-		goto out;
-	}
-
-	add_huge_page_size(hugepage_size);
-	rc = 1;
-
-out:
-	return rc;
-}
-__setup("hugepagesz=", setup_hugepagesz);
 #endif	/* CONFIG_HUGETLB_PAGE */
 
 void update_mmu_cache(struct vm_area_struct *vma, unsigned long address, pte_t *ptep)
arch/x86/mm/hugetlbpage.c +0 -16
index 1c4372bfe782..937d640a89e3 100644
@@ -191,22 +191,6 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 		return false;
 }
 
-static __init int setup_hugepagesz(char *opt)
-{
-	unsigned long ps = memparse(opt, &opt);
-
-	if (arch_hugetlb_valid_size(ps)) {
-		hugetlb_add_hstate(ilog2(ps) - PAGE_SHIFT);
-	} else {
-		hugetlb_bad_size();
-		printk(KERN_ERR "hugepagesz: Unsupported page size %lu M\n",
-			ps >> 20);
-		return 0;
-	}
-	return 1;
-}
-__setup("hugepagesz=", setup_hugepagesz);
-
 #ifdef CONFIG_CONTIG_ALLOC
 static __init int gigantic_pages_init(void)
 {
include/linux/hugetlb.h +0 -1
index 2eb15f5ab01e..0c13706054ef 100644
@@ -519,7 +519,6 @@ int huge_add_to_page_cache(struct page *page, struct address_space *mapping,
 int __init __alloc_bootmem_huge_page(struct hstate *h);
 int __init alloc_bootmem_huge_page(struct hstate *h);
 
-void __init hugetlb_bad_size(void);
 void __init hugetlb_add_hstate(unsigned order);
 bool __init arch_hugetlb_valid_size(unsigned long size);
 struct hstate *size_to_hstate(unsigned long size);
mm/hugetlb.c +17 -6
index 871ac1681f37..b2d276408cec 100644
@@ -3262,12 +3262,6 @@ bool __init __attribute((weak)) arch_hugetlb_valid_size(unsigned long size)
 	return size == HPAGE_SIZE;
 }
 
-/* Should be called on processing a hugepagesz=... option */
-void __init hugetlb_bad_size(void)
-{
-	parsed_valid_hugepagesz = false;
-}
-
 void __init hugetlb_add_hstate(unsigned int order)
 {
 	struct hstate *h;
@@ -3337,6 +3331,23 @@ static int __init hugetlb_nrpages_setup(char *s)
 }
 __setup("hugepages=", hugetlb_nrpages_setup);
 
+static int __init hugepagesz_setup(char *s)
+{
+	unsigned long size;
+
+	size = (unsigned long)memparse(s, NULL);
+
+	if (!arch_hugetlb_valid_size(size)) {
+		parsed_valid_hugepagesz = false;
+		pr_err("HugeTLB: unsupported hugepagesz %s\n", s);
+		return 0;
+	}
+
+	hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
+	return 1;
+}
+__setup("hugepagesz=", hugepagesz_setup);
+
 static int __init default_hugepagesz_setup(char *s)
 {
 	unsigned long size;
Signature · 1 line
2.25.2
#3 lore raw

Re: [PATCH v3 2/4] hugetlbfs: move hugepagesz= parsing to arch independent code

Hi Mike,

Quoted 29 lines · Mike Kravetz

Now that architectures provide arch_hugetlb_valid_size(), parsing of "hugepagesz=" can be done in architecture independent code. Create a single routine to handle hugepagesz= parsing and remove all arch specific routines. We can also remove the interface hugetlb_bad_size() as this is no longer used outside arch independent code.

This also provides consistent behavior of hugetlbfs command line options. The hugepagesz= option should only be specified once for a specific size, but some architectures allow multiple instances. This appears to be more of an oversight when code was added by some architectures to set up ALL huge pages sizes.

[...]

arch/powerpc/mm/hugetlbpage.c +0 -6
index de54d2a37830..2c3fa0a7787b 100644
@@ -589,21 +589,6 @@ static int __init add_huge_page_size(unsigned long long size)
 	return 0;
 }
 
-static int __init hugepage_setup_sz(char *str)
-{
-	unsigned long long size;
-
-	size = memparse(str, &str);
-

arch/powerpc/mm/hugetlbpage.c static int __init add_huge_page_size(unsigned long long size) · line 606

Quoted 10 lines
arch/powerpc/mm/hugetlbpage.c +0 -9
-	if (add_huge_page_size(size) != 0) {
-		hugetlb_bad_size();
-		pr_err("Invalid huge page size specified(%llu)\n", size);
-	}
-
-	return 1;
-}
-__setup("hugepagesz=", hugepage_setup_sz);
-
[...]

This isn't working as expected on powerpc64.

  [    0.000000] Kernel command line: root=UUID=dc7b49cf-95a2-4996-8e7d-7c64ddc7a6ff hugepagesz=16G hugepages=2 
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepagesz = 16G
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepages = 2
  [    0.284177] HugeTLB registered 16.0 MiB page size, pre-allocated 0 pages
  [    0.284182] HugeTLB registered 16.0 GiB page size, pre-allocated 0 pages
  [    2.585062]     hugepagesz=16G
  [    2.585063]     hugepages=2

The "huge pages not supported" messages are under a !hugepages_supported() condition which checks if HPAGE_SHIFT is non-zero. On powerpc64, HPAGE_SHIFT comes from the hpage_shift variable. At this point, it is still zero and yet to be set. Hence the check fails. The reason being hugetlbpage_init_default(), which sets hpage_shift, it now called after hugepage_setup_sz().

- Sandipan

#4 lore raw

Re: [PATCH v3 2/4] hugetlbfs: move hugepagesz= parsing to arch independent code

Quoted 53 lines · Sandipan Das

Hi Mike,

Quoted 29 lines · Mike Kravetz

Now that architectures provide arch_hugetlb_valid_size(), parsing of "hugepagesz=" can be done in architecture independent code. Create a single routine to handle hugepagesz= parsing and remove all arch specific routines. We can also remove the interface hugetlb_bad_size() as this is no longer used outside arch independent code.

This also provides consistent behavior of hugetlbfs command line options. The hugepagesz= option should only be specified once for a specific size, but some architectures allow multiple instances. This appears to be more of an oversight when code was added by some architectures to set up ALL huge pages sizes.

[...]

arch/powerpc/mm/hugetlbpage.c +0 -6
index de54d2a37830..2c3fa0a7787b 100644
@@ -589,21 +589,6 @@ static int __init add_huge_page_size(unsigned long long size)
 	return 0;
 }
 
-static int __init hugepage_setup_sz(char *str)
-{
-	unsigned long long size;
-
-	size = memparse(str, &str);
-

arch/powerpc/mm/hugetlbpage.c static int __init add_huge_page_size(unsigned long long size) · line 606

Quoted 10 lines
arch/powerpc/mm/hugetlbpage.c +0 -9
-	if (add_huge_page_size(size) != 0) {
-		hugetlb_bad_size();
-		pr_err("Invalid huge page size specified(%llu)\n", size);
-	}
-
-	return 1;
-}
-__setup("hugepagesz=", hugepage_setup_sz);
-
[...]

This isn't working as expected on powerpc64.

  [    0.000000] Kernel command line: root=UUID=dc7b49cf-95a2-4996-8e7d-7c64ddc7a6ff hugepagesz=16G hugepages=2 
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepagesz = 16G
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepages = 2
  [    0.284177] HugeTLB registered 16.0 MiB page size, pre-allocated 0 pages
  [    0.284182] HugeTLB registered 16.0 GiB page size, pre-allocated 0 pages
  [    2.585062]     hugepagesz=16G
  [    2.585063]     hugepages=2
Quoted 5 lines

The "huge pages not supported" messages are under a !hugepages_supported() condition which checks if HPAGE_SHIFT is non-zero. On powerpc64, HPAGE_SHIFT comes from the hpage_shift variable. At this point, it is still zero and yet to be set. Hence the check fails. The reason being hugetlbpage_init_default(), which sets hpage_shift, it now called after hugepage_setup_sz().

Thanks for catching this Sandipan.

In the new arch independent version of hugepages_setup, I added the following code in patch 4 off this series:

Review

Quoted 11 lines
(quoted diff) +6 -0
+static int __init hugepages_setup(char *s)
 {
 	unsigned long *mhp;
 	static unsigned long *last_mhp;
 
+	if (!hugepages_supported()) {
+		pr_warn("HugeTLB: huge pages not supported, ignoring hugepages = %s\n", s);
+		return 0;
+	}
+
 	if (!parsed_valid_hugepagesz) {

In fact, I added it to the beginning of all the hugetlb command line parsing routines. My 'thought' was to warn early if hugetlb pages were not supported. Previously, the first check for hugepages_supported() was in hugetlb_init() which ran after hugetlbpage_init_default().

The easy solution is to remove all the hugepages_supported() checks from command line parsing routines and rely on the later check in hugetlb_init().

Another reason for adding those early checks was to possibly prevent the
preallocation of gigantic pages at command line parsing time. Gigantic
pages are allocated at command line parsing time as they need to be allocated with the bootmem allocator. My concern is that there could be some strange configuration where !hugepages_supported(), yet we allocate gigantic pages from bootmem that can not be used or freeed later.

powerpc is the only architecture which has it's own alloc_bootmem_huge_page routine. So, it handles this potential issue.

I'll send out a fix shortly.

Signature · 1 line
Mike Kravetz
#5 lore raw

Re: [PATCH v3 2/4] hugetlbfs: move hugepagesz= parsing to arch independent code

Quoted 19 lines · Mike Kravetz
Quoted 6 lines · Sandipan Das
Quoted 2 lines · Mike Kravetz

Now that architectures provide arch_hugetlb_valid_size(), parsing of "hugepagesz=" can be done in architecture independent code.

This isn't working as expected on powerpc64.

Quoted 8 lines
  [    0.000000] Kernel command line: root=UUIDÜ7b49cf-95a2-4996-8e7d-7c64ddc7a6ff hugepageszG hugepages=2 
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepagesz = 16G
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepages = 2
  [    0.284177] HugeTLB registered 16.0 MiB page size, pre-allocated 0 pages
  [    0.284182] HugeTLB registered 16.0 GiB page size, pre-allocated 0 pages
  [    2.585062]     hugepageszG
  [    2.585063]     hugepages=2

In the new arch independent version of hugepages_setup, I added the following code in patch 4 off this series:

Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+	if (!hugepages_supported()) {
+		pr_warn("HugeTLB: huge pages not supported, ignoring hugepages = %s\n", s);
+		return 0;
+	}
+

The easy solution is to remove all the hugepages_supported() checks from command line parsing routines and rely on the later check in hugetlb_init().

Here is a patch to address the issue. Sorry, as my series breaks all hugetlb command line processing on powerpc.

Sandipan, can you test the following patch?

From 480fe2847361e2a85aeec1fb39fe643bb7100a07 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 27 Apr 2020 11:37:30 -0700
Subject: [PATCH] hugetlbfs: fix changes to command line processing

Previously, a check for hugepages_supported was added before processing hugetlb command line parameters. On some architectures such as powerpc, hugepages_supported() is not set to true until after command line processing. Therefore, no hugetlb command line parameters would be accepted.

Remove the additional checks for hugepages_supported. In hugetlb_init, print a warning if !hugepages_supported and command line parameters were specified.

---
 mm/hugetlb.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
mm/hugetlb.c +4 -16
index 1075abdb5717..5548e8851b93 100644
@@ -3212,8 +3212,11 @@ static int __init hugetlb_init(void)
 {
 	int i;
 
-	if (!hugepages_supported())
+	if (!hugepages_supported()) {
+		if (hugetlb_max_hstate || default_hstate_max_huge_pages)
+			pr_warn("HugeTLB: huge pages not supported, ignoring associated command-line parameters\n");
 		return 0;
+	}
 
 	/*
 	 * Make sure HPAGE_SIZE (HUGETLB_PAGE_ORDER) hstate exists.  Some
@@ -3315,11 +3318,6 @@ static int __init hugepages_setup(char *s)
 	unsigned long *mhp;
 	static unsigned long *last_mhp;
 
-	if (!hugepages_supported()) {
-		pr_warn("HugeTLB: huge pages not supported, ignoring hugepages = %s\n", s);
-		return 0;
-	}
-
 	if (!parsed_valid_hugepagesz) {
 		pr_warn("HugeTLB: hugepages=%s does not follow a valid hugepagesz, ignoring\n", s);
 		parsed_valid_hugepagesz = true;
@@ -3372,11 +3370,6 @@ static int __init hugepagesz_setup(char *s)
 	struct hstate *h;
 
 	parsed_valid_hugepagesz = false;
-	if (!hugepages_supported()) {
-		pr_warn("HugeTLB: huge pages not supported, ignoring hugepagesz = %s\n", s);
-		return 0;
-	}
-
 	size = (unsigned long)memparse(s, NULL);
 
 	if (!arch_hugetlb_valid_size(size)) {
@@ -3424,11 +3417,6 @@ static int __init default_hugepagesz_setup(char *s)
 	unsigned long size;
 
 	parsed_valid_hugepagesz = false;
-	if (!hugepages_supported()) {
-		pr_warn("HugeTLB: huge pages not supported, ignoring default_hugepagesz = %s\n", s);
-		return 0;
-	}
-
 	if (parsed_default_hugepagesz) {
 		pr_err("HugeTLB: default_hugepagesz previously specified, ignoring %s\n", s);
 		return 0;
Signature · 1 line
2.25.4
maintainer
#6 lore raw

Re: [PATCH v3 2/4] hugetlbfs: move hugepagesz= parsing to arch independent code

Quoted 9 lines · On Mon, 27 Apr 2020 12:09:47 -0700 Mike Kravetz <[email protected]> wrote:

Previously, a check for hugepages_supported was added before processing hugetlb command line parameters. On some architectures such as powerpc, hugepages_supported() is not set to true until after command line processing. Therefore, no hugetlb command line parameters would be accepted.

Remove the additional checks for hugepages_supported. In hugetlb_init, print a warning if !hugepages_supported and command line parameters were specified.

This applies to [4/4] instead of fixing [2/4]. I guess you'll straighten that out in v4?

btw, was
http://lkml.kernel.org/r/CADYN=9Koefrq9H1Y82Q8nMNbeyN4tzhEfvDu5u=sVFjFZCYorA@mail.gmail.com
addressed?

#7 lore raw

Re: [PATCH v3 2/4] hugetlbfs: move hugepagesz= parsing to arch independent code

Quoted 8 lines · Andrew Morton
Quoted 6 lines · On Mon, 27 Apr 2020 12:09:47 -0700 Mike Kravetz <[email protected]> wrote:

Previously, a check for hugepages_supported was added before processing hugetlb command line parameters. On some architectures such as powerpc, hugepages_supported() is not set to true until after command line processing. Therefore, no hugetlb command line parameters would be accepted.

Quoted 6 lines
Quoted 3 lines

Remove the additional checks for hugepages_supported. In hugetlb_init, print a warning if !hugepages_supported and command line parameters were specified.

This applies to [4/4] instead of fixing [2/4]. I guess you'll straighten that out in v4?

Yes.

Quoted 3 lines

btw, was
http://lkml.kernel.org/r/CADYN=9Koefrq9H1Y82Q8nMNbeyN4tzhEfvDu5u=sVFjFZCYorA@mail.gmail.com
addressed?

Yes, you pulled a patch into your tree to address this. hugetlbfs-remove-hugetlb_add_hstate-warning-for-existing-hstate-fix.patch

I'll send out a v4 with both these issues addressed. Would like to wait until receiving confirmation from someone who can test on powerpc.

Signature · 1 line
Mike Kravetz
Tested-by
#8 lore raw

Re: [PATCH v3 2/4] hugetlbfs: move hugepagesz= parsing to arch independent code

In reply to Mike Kravetz

Hi Mike,

Quoted 29 lines · Mike Kravetz
Quoted 19 lines · Mike Kravetz
Quoted 6 lines · Sandipan Das
Quoted 2 lines · Mike Kravetz

Now that architectures provide arch_hugetlb_valid_size(), parsing of "hugepagesz=" can be done in architecture independent code.

This isn't working as expected on powerpc64.

Quoted 8 lines
  [    0.000000] Kernel command line: root=UUID=dc7b49cf-95a2-4996-8e7d-7c64ddc7a6ff hugepagesz=16G hugepages=2 
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepagesz = 16G
  [    0.000000] HugeTLB: huge pages not supported, ignoring hugepages = 2
  [    0.284177] HugeTLB registered 16.0 MiB page size, pre-allocated 0 pages
  [    0.284182] HugeTLB registered 16.0 GiB page size, pre-allocated 0 pages
  [    2.585062]     hugepagesz=16G
  [    2.585063]     hugepages=2

In the new arch independent version of hugepages_setup, I added the following code in patch 4 off this series:

Quoted 8 lines

Review

Quoted 5 lines
(quoted diff) +5 -0
+	if (!hugepages_supported()) {
+		pr_warn("HugeTLB: huge pages not supported, ignoring hugepages = %s\n", s);
+		return 0;
+	}
+

The easy solution is to remove all the hugepages_supported() checks from command line parsing routines and rely on the later check in hugetlb_init().

Quoted 5 lines

Here is a patch to address the issue. Sorry, as my series breaks all hugetlb command line processing on powerpc.

Sandipan, can you test the following patch?

The following patch does fix the issue. Thanks.

Quoted 74 lines

From 480fe2847361e2a85aeec1fb39fe643bb7100a07 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 27 Apr 2020 11:37:30 -0700
Subject: [PATCH] hugetlbfs: fix changes to command line processing

Previously, a check for hugepages_supported was added before processing hugetlb command line parameters. On some architectures such as powerpc, hugepages_supported() is not set to true until after command line processing. Therefore, no hugetlb command line parameters would be accepted.

Remove the additional checks for hugepages_supported. In hugetlb_init, print a warning if !hugepages_supported and command line parameters were specified.

---
 mm/hugetlb.c | 20 ++++----------------
 1 file changed, 4 insertions(+), 16 deletions(-)
mm/hugetlb.c +4 -16
index 1075abdb5717..5548e8851b93 100644
@@ -3212,8 +3212,11 @@ static int __init hugetlb_init(void)
 {
 	int i;
 
-	if (!hugepages_supported())
+	if (!hugepages_supported()) {
+		if (hugetlb_max_hstate || default_hstate_max_huge_pages)
+			pr_warn("HugeTLB: huge pages not supported, ignoring associated command-line parameters\n");
 		return 0;
+	}
 
 	/*
 	 * Make sure HPAGE_SIZE (HUGETLB_PAGE_ORDER) hstate exists.  Some
@@ -3315,11 +3318,6 @@ static int __init hugepages_setup(char *s)
 	unsigned long *mhp;
 	static unsigned long *last_mhp;
 
-	if (!hugepages_supported()) {
-		pr_warn("HugeTLB: huge pages not supported, ignoring hugepages = %s\n", s);
-		return 0;
-	}
-
 	if (!parsed_valid_hugepagesz) {
 		pr_warn("HugeTLB: hugepages=%s does not follow a valid hugepagesz, ignoring\n", s);
 		parsed_valid_hugepagesz = true;
@@ -3372,11 +3370,6 @@ static int __init hugepagesz_setup(char *s)
 	struct hstate *h;
 
 	parsed_valid_hugepagesz = false;
-	if (!hugepages_supported()) {
-		pr_warn("HugeTLB: huge pages not supported, ignoring hugepagesz = %s\n", s);
-		return 0;
-	}
-
 	size = (unsigned long)memparse(s, NULL);
 
 	if (!arch_hugetlb_valid_size(size)) {
@@ -3424,11 +3417,6 @@ static int __init default_hugepagesz_setup(char *s)
 	unsigned long size;
 
 	parsed_valid_hugepagesz = false;
-	if (!hugepages_supported()) {
-		pr_warn("HugeTLB: huge pages not supported, ignoring default_hugepagesz = %s\n", s);
-		return 0;
-	}
-
 	if (parsed_default_hugepagesz) {
 		pr_err("HugeTLB: default_hugepagesz previously specified, ignoring %s\n", s);
 		return 0;
#9 lore raw

PATCH v3 3/4 hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate

In reply to Mike Kravetz

The routine hugetlb_add_hstate prints a warning if the hstate already exists. This was originally done as part of kernel command line parsing. If 'hugepagesz=' was specified more than once, the warning

	pr_warn("hugepagesz= specified twice, ignoring\n");

would be printed.

Some architectures want to enable all huge page sizes. They would call hugetlb_add_hstate for all supported sizes. However, this was done after command line processing and as a result hstates could have already been created for some sizes. To make sure no warning were printed, there would often be code like:
if (!size_to_hstate(size)
hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT)

The only time we want to print the warning is as the result of command line processing. So, remove the warning from hugetlb_add_hstate and add it to the single arch independent routine processing "hugepagesz=". After this, calls to size_to_hstate() in arch specific code can be removed and hugetlb_add_hstate can be called without worrying about warning messages.

---
 arch/arm64/mm/hugetlbpage.c   | 16 ++++------------
 arch/powerpc/mm/hugetlbpage.c |  3 +--
 arch/riscv/mm/hugetlbpage.c   |  2 +-
 arch/sparc/mm/init_64.c       | 19 ++++---------------
 arch/x86/mm/hugetlbpage.c     |  2 +-
 mm/hugetlb.c                  |  9 ++++++---
 6 files changed, 17 insertions(+), 34 deletions(-)
arch/arm64/mm/hugetlbpage.c +4 -12
index f706b821aba6..21fa98b51e00 100644
@@ -441,22 +441,14 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
 	clear_flush(vma->vm_mm, addr, ptep, pgsize, ncontig);
 }
 
-static void __init add_huge_page_size(unsigned long size)
-{
-	if (size_to_hstate(size))
-		return;
-
-	hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
-}
-
 static int __init hugetlbpage_init(void)
 {
 #ifdef CONFIG_ARM64_4K_PAGES
-	add_huge_page_size(PUD_SIZE);
+	hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
 #endif
-	add_huge_page_size(CONT_PMD_SIZE);
-	add_huge_page_size(PMD_SIZE);
-	add_huge_page_size(CONT_PTE_SIZE);
+	hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
 
 	return 0;
 }
arch/powerpc/mm/hugetlbpage.c +1 -2
index 2c3fa0a7787b..4d5ed1093615 100644
@@ -584,8 +584,7 @@ static int __init add_huge_page_size(unsigned long long size)
 	if (!arch_hugetlb_valid_size((unsigned long)size))
 		return -EINVAL;
 
-	if (!size_to_hstate(size))
-		hugetlb_add_hstate(shift - PAGE_SHIFT);
+	hugetlb_add_hstate(shift - PAGE_SHIFT);
 	return 0;
 }
 
arch/riscv/mm/hugetlbpage.c +1 -1
index 4e5d7e9f0eef..932dadfdca54 100644
@@ -26,7 +26,7 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 static __init int gigantic_pages_init(void)
 {
 	/* With CONTIG_ALLOC, we can allocate gigantic pages at runtime */
-	if (IS_ENABLED(CONFIG_64BIT) && !size_to_hstate(1UL << PUD_SHIFT))
+	if (IS_ENABLED(CONFIG_64BIT))
 		hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
 	return 0;
 }
arch/sparc/mm/init_64.c +4 -15
index 4618f96fd30f..ae819a16d07a 100644
@@ -325,23 +325,12 @@ static void __update_mmu_tsb_insert(struct mm_struct *mm, unsigned long tsb_inde
 }
 
 #ifdef CONFIG_HUGETLB_PAGE
-static void __init add_huge_page_size(unsigned long size)
-{
-	unsigned int order;
-
-	if (size_to_hstate(size))
-		return;
-
-	order = ilog2(size) - PAGE_SHIFT;
-	hugetlb_add_hstate(order);
-}
-
 static int __init hugetlbpage_init(void)
 {
-	add_huge_page_size(1UL << HPAGE_64K_SHIFT);
-	add_huge_page_size(1UL << HPAGE_SHIFT);
-	add_huge_page_size(1UL << HPAGE_256MB_SHIFT);
-	add_huge_page_size(1UL << HPAGE_2GB_SHIFT);
+	hugetlb_add_hstate(HPAGE_64K_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(HPAGE_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(HPAGE_256MB_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(HPAGE_2GB_SHIFT - PAGE_SHIFT);
 
 	return 0;
 }
arch/x86/mm/hugetlbpage.c +1 -1
index 937d640a89e3..cf5781142716 100644
@@ -195,7 +195,7 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 static __init int gigantic_pages_init(void)
 {
 	/* With compaction or CMA we can allocate gigantic pages at runtime */
-	if (boot_cpu_has(X86_FEATURE_GBPAGES) && !size_to_hstate(1UL << PUD_SHIFT))
+	if (boot_cpu_has(X86_FEATURE_GBPAGES))
 		hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
 	return 0;
 }
mm/hugetlb.c +6 -3
index b2d276408cec..0e6eb755ae94 100644
@@ -3222,8 +3222,7 @@ static int __init hugetlb_init(void)
 		}
 
 		default_hstate_size = HPAGE_SIZE;
-		if (!size_to_hstate(default_hstate_size))
-			hugetlb_add_hstate(HUGETLB_PAGE_ORDER);
+		hugetlb_add_hstate(HUGETLB_PAGE_ORDER);
 	}
 	default_hstate_idx = hstate_index(size_to_hstate(default_hstate_size));
 	if (default_hstate_max_huge_pages) {
@@ -3268,7 +3267,6 @@ void __init hugetlb_add_hstate(unsigned int order)
 	unsigned long i;
 
 	if (size_to_hstate(PAGE_SIZE << order)) {
-		pr_warn("hugepagesz= specified twice, ignoring\n");
 		return;
 	}
 	BUG_ON(hugetlb_max_hstate >= HUGE_MAX_HSTATE);
@@ -3343,6 +3341,11 @@ static int __init hugepagesz_setup(char *s)
 		return 0;
 	}
 
+	if (size_to_hstate(size)) {
+		pr_warn("HugeTLB: hugepagesz %s specified twice, ignoring\n", s);
+		return 0;
+	}
+
 	hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
 	return 1;
 }
Signature · 1 line
2.25.2
#10 lore raw

Re: [PATCH v3 3/4] hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate

Quoted 22 lines · Mike Kravetz

The routine hugetlb_add_hstate prints a warning if the hstate already exists. This was originally done as part of kernel command line parsing. If 'hugepagesz=' was specified more than once, the warning

        pr_warn("hugepagesz= specified twice, ignoring\n");

would be printed.

Some architectures want to enable all huge page sizes. They would call hugetlb_add_hstate for all supported sizes. However, this was done after command line processing and as a result hstates could have already been created for some sizes. To make sure no warning were printed, there would often be code like:
if (!size_to_hstate(size)
hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT)

The only time we want to print the warning is as the result of command line processing. So, remove the warning from hugetlb_add_hstate and add it to the single arch independent routine processing "hugepagesz=". After this, calls to size_to_hstate() in arch specific code can be removed and hugetlb_add_hstate can be called without worrying about warning messages.

Quoted 2 lines

When I build an arm64 kernel on today's next-20200420 and ran that in qemu I got the following output [1]:

...

[  311.326817][    T1] kobject: 'drivers' ((____ptrval____)):
kobject_add_internal: parent: 'coresight', set: '<NULL>'
[  311.331513][    T1] kobject: 'drivers' ((____ptrval____)): kobject_uevent_env
[  311.334514][    T1] kobject: 'drivers' ((____ptrval____)):
kobject_uevent_env: filter function caused the event to drop!
[  311.340127][    T1] bus: 'coresight': registered
[  311.342228][    T1] initcall coresight_init+0x0/0x64 returned 0
after 27343 usecs
[  311.349740][    T1] calling  debug_traps_init+0x0/0xa4 @ 1
[  311.352138][    T1] initcall debug_traps_init+0x0/0xa4 returned 0
after 0 usecs
[  311.355550][    T1] calling  reserve_memblock_reserved_regions+0x0/0x374 @ 1
[  311.364913][    T1] initcall
reserve_memblock_reserved_regions+0x0/0x374 returned 0 after 7812
usecs
[  311.368937][    T1] calling  aarch32_alloc_vdso_pages+0x0/0x1d0 @ 1
[  311.371819][    T1] initcall aarch32_alloc_vdso_pages+0x0/0x1d0
returned 0 after 0 usecs
[  311.375608][    T1] calling  vdso_init+0x0/0x52c @ 1
[  311.378092][    T1] initcall vdso_init+0x0/0x52c returned 0 after 0 usecs
[  311.381386][    T1] calling  arch_hw_breakpoint_init+0x0/0x178 @ 1
[  311.384007][    T1] hw-breakpoint: found 6 breakpoint and 4
watchpoint registers.
[  311.388120][    T1] initcall arch_hw_breakpoint_init+0x0/0x178
returned 0 after 3906 usecs
[  311.391924][    T1] calling  asids_update_limit+0x0/0x110 @ 1
[  311.394390][    T1] ASID allocator initialised with 65536 entries
[  311.397427][    T1] initcall asids_update_limit+0x0/0x110 returned
0 after 3906 usecs
[  311.400749][    T1] calling  hugetlbpage_init+0x0/0x7c @ 1
[  311.403581][    T1] Unexpected kernel BRK exception at EL1
[  311.405771][    T1] Internal error: ptrace BRK handler: f20003e8
[#1] PREEMPT SMP
[  311.408759][    T1] Modules linked in:
[  311.410514][    T1] CPU: 0 PID: 1 Comm: swapper/0 Tainted: G
        T 5.7.0-rc2-next-20200420-03722-ge4ba9b47e4ed #1
[  311.415175][    T1] Hardware name: linux,dummy-virt (DT)
[  311.417466][    T1] pstate: 80400005 (Nzcv daif +PAN -UAO)
[  311.419887][    T1] pc : hugetlb_add_hstate+0x68/0x4f0
[  311.422171][    T1] lr : hugetlb_add_hstate+0x68/0x4f0
[  311.424354][    T1] sp : ffff000069c07c60
[  311.426124][    T1] x29: ffff000069c07c60 x28: ffff00006a7f8058
[  311.428754][    T1] x27: 0000000000000000 x26: ffffa00013f56950
[  311.431376][    T1] x25: ffffa000141b8000 x24: ffff00006a7f8040
[  311.433987][    T1] x23: 1fffe0000d380fae x22: 00000000fffffff8
[  311.436574][    T1] x21: 0000000100000000 x20: ffffa000141b8000
[  311.439167][    T1] x19: ec632d51be3d2507 x18: 0000000000001a68
[  311.441763][    T1] x17: 00000000000013e0 x16: 0000000000001a94
[  311.444386][    T1] x15: 0000000000001a68 x14: 6573752036303933
[  311.447034][    T1] x13: 2072657466612030 x12: 00000000000025b0
[  311.449639][    T1] x11: 00000000f1f1f1f1 x10: 0000000041b58ab3
[  311.452238][    T1] x9 : ffffa000139a833c x8 : 1ffff40002bf2c23
[  311.454849][    T1] x7 : ffff940002bf2c23 x6 : ffffa00015f9611b
[  311.457480][    T1] x5 : ffff00006a7f8040 x4 : 0000000000000000
[  311.460124][    T1] x3 : ffffa000139fd98c x2 : 00000000fffffff8
[  311.462737][    T1] x1 : ffff00006a7f8040 x0 : 0000000000000000
[  311.465322][    T1] Call trace:
[  311.466818][    T1]  hugetlb_add_hstate+0x68/0x4f0
[  311.468934][    T1]  hugetlbpage_init+0x34/0x7c
[  311.470934][    T1]  do_one_initcall+0x480/0xa40
[  311.472996][    T1]  kernel_init_freeable+0x7a0/0x968
[  311.475224][    T1]  kernel_init+0x20/0x1f8
[  311.477078][    T1]  ret_from_fork+0x10/0x18
[  311.479053][    T1] Code: 972762be 7100fedf 54000069 97276197 (d4207d00)
[  311.482106][    T1] _warn_unseeded_randomness: 18 callbacks suppressed
[  311.482255][    T1] random: get_random_bytes called from
print_oops_end_marker+0x48/0x80 with crng_init=0
[  311.482321][    T1] ---[ end trace 60df362baad50718 ]---
[  311.491423][    T1] Kernel panic - not syncing: Fatal exception
[  311.494038][    T1] ---[ end Kernel panic - not syncing: Fatal exception ]---

If I revert this patch I can't see the problem anymore...

Any idea what happens?

This is the kernel.config [1] I'm using, its from an allmodconfig kernel build

Cheers,
Anders
[1] https://people.linaro.org/~anders.roxell/output-next-20200420.log
[2] https://builds.tuxbuild.com/U7ufblLydTsSvle27GSPAA/kernel.config

Quoted 149 lines
---
 arch/arm64/mm/hugetlbpage.c   | 16 ++++------------
 arch/powerpc/mm/hugetlbpage.c |  3 +--
 arch/riscv/mm/hugetlbpage.c   |  2 +-
 arch/sparc/mm/init_64.c       | 19 ++++---------------
 arch/x86/mm/hugetlbpage.c     |  2 +-
 mm/hugetlb.c                  |  9 ++++++---
 6 files changed, 17 insertions(+), 34 deletions(-)
arch/arm64/mm/hugetlbpage.c +4 -12
index f706b821aba6..21fa98b51e00 100644
@@ -441,22 +441,14 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
        clear_flush(vma->vm_mm, addr, ptep, pgsize, ncontig);
 }
 
-static void __init add_huge_page_size(unsigned long size)
-{
-       if (size_to_hstate(size))
-               return;
-
-       hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
-}
-
 static int __init hugetlbpage_init(void)
 {
 #ifdef CONFIG_ARM64_4K_PAGES
-       add_huge_page_size(PUD_SIZE);
+       hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
 #endif
-       add_huge_page_size(CONT_PMD_SIZE);
-       add_huge_page_size(PMD_SIZE);
-       add_huge_page_size(CONT_PTE_SIZE);
+       hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
 
        return 0;
 }
arch/powerpc/mm/hugetlbpage.c +1 -2
index 2c3fa0a7787b..4d5ed1093615 100644
@@ -584,8 +584,7 @@ static int __init add_huge_page_size(unsigned long long size)
        if (!arch_hugetlb_valid_size((unsigned long)size))
                return -EINVAL;
 
-       if (!size_to_hstate(size))
-               hugetlb_add_hstate(shift - PAGE_SHIFT);
+       hugetlb_add_hstate(shift - PAGE_SHIFT);
        return 0;
 }
 
arch/riscv/mm/hugetlbpage.c +1 -1
index 4e5d7e9f0eef..932dadfdca54 100644
@@ -26,7 +26,7 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 static __init int gigantic_pages_init(void)
 {
        /* With CONTIG_ALLOC, we can allocate gigantic pages at runtime */
-       if (IS_ENABLED(CONFIG_64BIT) && !size_to_hstate(1UL << PUD_SHIFT))
+       if (IS_ENABLED(CONFIG_64BIT))
                hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
        return 0;
 }
arch/sparc/mm/init_64.c +4 -15
index 4618f96fd30f..ae819a16d07a 100644
@@ -325,23 +325,12 @@ static void __update_mmu_tsb_insert(struct mm_struct *mm, unsigned long tsb_inde
 }
 
 #ifdef CONFIG_HUGETLB_PAGE
-static void __init add_huge_page_size(unsigned long size)
-{
-       unsigned int order;
-
-       if (size_to_hstate(size))
-               return;
-
-       order = ilog2(size) - PAGE_SHIFT;
-       hugetlb_add_hstate(order);
-}
-
 static int __init hugetlbpage_init(void)
 {
-       add_huge_page_size(1UL << HPAGE_64K_SHIFT);
-       add_huge_page_size(1UL << HPAGE_SHIFT);
-       add_huge_page_size(1UL << HPAGE_256MB_SHIFT);
-       add_huge_page_size(1UL << HPAGE_2GB_SHIFT);
+       hugetlb_add_hstate(HPAGE_64K_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(HPAGE_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(HPAGE_256MB_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(HPAGE_2GB_SHIFT - PAGE_SHIFT);
 
        return 0;
 }
arch/x86/mm/hugetlbpage.c +1 -1
index 937d640a89e3..cf5781142716 100644
@@ -195,7 +195,7 @@ bool __init arch_hugetlb_valid_size(unsigned long size)
 static __init int gigantic_pages_init(void)
 {
        /* With compaction or CMA we can allocate gigantic pages at runtime */
-       if (boot_cpu_has(X86_FEATURE_GBPAGES) && !size_to_hstate(1UL << PUD_SHIFT))
+       if (boot_cpu_has(X86_FEATURE_GBPAGES))
                hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
        return 0;
 }
mm/hugetlb.c +6 -4
index b2d276408cec..0e6eb755ae94 100644
@@ -3222,8 +3222,7 @@ static int __init hugetlb_init(void)
                }
 
                default_hstate_size = HPAGE_SIZE;
-               if (!size_to_hstate(default_hstate_size))
-                       hugetlb_add_hstate(HUGETLB_PAGE_ORDER);
+               hugetlb_add_hstate(HUGETLB_PAGE_ORDER);
        }
        default_hstate_idx = hstate_index(size_to_hstate(default_hstate_size));
        if (default_hstate_max_huge_pages) {
@@ -3268,7 +3267,6 @@ void __init hugetlb_add_hstate(unsigned int order)
        unsigned long i;
 
        if (size_to_hstate(PAGE_SIZE << order)) {
-               pr_warn("hugepagesz= specified twice, ignoring\n");
                return;
        }
        BUG_ON(hugetlb_max_hstate >= HUGE_MAX_HSTATE);
@@ -3343,6 +3341,11 @@ static int __init hugepagesz_setup(char *s)
                return 0;
        }
 
+       if (size_to_hstate(size)) {
+               pr_warn("HugeTLB: hugepagesz %s specified twice, ignoring\n", s);
+               return 0;
+       }
+
        hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT);
        return 1;
 }
--

2.25.2

#11 lore raw

Re: [PATCH v3 3/4] hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate

In reply to Mike Kravetz
Quoted 14 lines · Mike Kravetz

The routine hugetlb_add_hstate prints a warning if the hstate already exists. This was originally done as part of kernel command line parsing. If 'hugepagesz=' was specified more than once, the warning

	pr_warn("hugepagesz= specified twice, ignoring\n");

would be printed.

Some architectures want to enable all huge page sizes. They would call hugetlb_add_hstate for all supported sizes. However, this was done after command line processing and as a result hstates could have already been created for some sizes. To make sure no warning were printed, there would often be code like:
if (!size_to_hstate(size)
hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT)

Quoted 2 lines

The only time we want to print the warning is as the result of command line processing.

Does this patch break hugepages=x command line? I haven't tested this yet. But one of the details w.r.t. skipping that hugetlb_add_hstate is to make sure we can configure the max_huge_pages.

Quoted 6 lines

So, remove the warning from hugetlb_add_hstate and
add it to the single arch independent routine processing "hugepagesz=". After this, calls to size_to_hstate() in arch specific code can be removed and hugetlb_add_hstate can be called without worrying about warning messages.

-aneesh

#12 lore raw

Re: [PATCH v3 3/4] hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate

Quoted 16 lines · Aneesh Kumar K.V
Quoted 14 lines · Mike Kravetz

The routine hugetlb_add_hstate prints a warning if the hstate already exists. This was originally done as part of kernel command line parsing. If 'hugepagesz=' was specified more than once, the warning

	pr_warn("hugepagesz= specified twice, ignoring\n");

would be printed.

Some architectures want to enable all huge page sizes. They would call hugetlb_add_hstate for all supported sizes. However, this was done after command line processing and as a result hstates could have already been created for some sizes. To make sure no warning were printed, there would often be code like:
if (!size_to_hstate(size)
hugetlb_add_hstate(ilog2(size) - PAGE_SHIFT)

Quoted 7 lines
Quoted 2 lines

The only time we want to print the warning is as the result of command line processing.

Does this patch break hugepages=x command line? I haven't tested this yet. But one of the details w.r.t. skipping that hugetlb_add_hstate is to make sure we can configure the max_huge_pages.

Are you asking about hugepages=x being the only option on the command line? If so, then the behavior is not changed. This will result in x pages of default huge page size being allocated. Where default huge page size is of course architecture dependent. On an x86 VM,

[    0.040474] Kernel command line: BOOT_IMAGE=/vmlinuz-5.6.0-mm1+ root=/dev/mapper/fedora_new--host-root ro rd.lvm.lv=fedora_new-host/root rd.lvm.lv=fedora_new-host/swap console=tty0 console=ttyS0,115200 audit=0 transparent_hugepage=always hugepages=128
[    0.332618] HugeTLB registered 1.00 GiB page size, pre-allocated 0 pages
[    0.333245] HugeTLB registered 2.00 MiB page size, pre-allocated 128 pages

BTW - Here are the command line options I tested on x86 with this series.

No errors or warnings
---------------------
hugepages=128
hugepagesz=2M hugepages=128
default_hugepagesz=2M hugepages=128
hugepages=128 default_hugepagesz=2M
hugepagesz=1G hugepages=2
hugepages=2 default_hugepagesz=1G
default_hugepagesz=1G hugepages=2
hugepages=128 hugepagesz=1G hugepages=2
hugepagesz=1G hugepages=2 hugepagesz=2M hugepages=128 default_hugepagesz=2M hugepages=128 hugepagesz=1G hugepages=2 hugepages=128 default_hugepagesz=2M hugepagesz=1G hugepages=2 hugepages=2 default_hugepagesz=1G hugepagesz=2M hugepages=128 default_hugepagesz=1G hugepages=2 hugepagesz=2M hugepages=128 default_hugepagesz=2M hugepagesz=2M hugepages=128 default_hugepagesz=2M hugepagesz=1G hugepages=2 hugepagesz=2M hugepages=128

Error or warning
----------------
hugepages=128 hugepagesz=2M hugepages=256 hugepagesz=2M hugepages=128 hugepagesz=2M hugepages=256 default_hugepagesz=2M hugepages=128 hugepagesz=2M hugepages=256 hugepages=128 hugepages=256
hugepagesz=2M hugepages=128 hugepages=2 default_hugepagesz=1G

Signature · 1 line
Mike Kravetz
#13 lore raw

PATCH v3 4/4 hugetlbfs: clean up command line processing

In reply to Mike Kravetz

With all hugetlb page processing done in a single file clean up code.
- Make code match desired semantics
- Update documentation with semantics
- Make all warnings and errors messages start with 'HugeTLB:'.
- Consistently name command line parsing routines.
- Check for hugepages_supported() before processing parameters.
- Add comments to code
- Describe some of the subtle interactions
- Describe semantics of command line arguments

This patch also fixes issues with implicitly setting the number of gigantic huge pages to preallocate. Previously on X86 command line,
hugepages=2 default_hugepagesz=1G
would result in zero 1G pages being preallocated and,

        # grep HugePages_Total /proc/meminfo
        HugePages_Total:       0
        # sysctl -a | grep nr_hugepages
        vm.nr_hugepages = 2
        vm.nr_hugepages_mempolicy = 2
        # cat /proc/sys/vm/nr_hugepages
        2
After this patch 2 gigantic pages will be preallocated and all the
proc, sysfs, sysctl and meminfo files will accurately reflect this.

To address the issue with gigantic pages, a small change in behavior was made to command line processing. Previously the command line,
hugepages8 default_hugepagesz=2M hugepagesz=2M hugepages%6
would result in the allocation of 256 2M huge pages. The value 128 would be ignored without any warning. After this patch, 128 2M pages will be allocated and a warning message will be displayed indicating the value of 256 is ignored. This change in behavior is required because allocation of implicitly specified gigantic pages must be done when the default_hugepagesz= is encountered for gigantic pages. Previously the code waited until later in the boot process (hugetlb_init), to allocate pages of default size. However the bootmem allocator required for gigantic allocations is not available at this time.

---
 .../admin-guide/kernel-parameters.txt         |  40 +++--
 Documentation/admin-guide/mm/hugetlbpage.rst  |  35 ++++
 mm/hugetlb.c                                  | 159 ++++++++++++++----
 3 files changed, 190 insertions(+), 44 deletions(-)
Documentation/admin-guide/kernel-parameters.txt +26 -12
index f2a93c8679e8..8cd78cc87a1c 100644
@@ -834,12 +834,15 @@
 			See also Documentation/networking/decnet.txt.
 
 	default_hugepagesz-			[same as hugepagesz=] The size of the default
-			HugeTLB page size. This is the size represented by
-			the legacy /proc/ hugepages APIs, used for SHM, and
-			default size when mounting hugetlbfs filesystems.
-			Defaults to the default architecture's huge page size
-			if not specified.
+			[HW] The size of the default HugeTLB page. This is
+			the size represented by the legacy /proc/ hugepages
+			APIs.  In addition, this is the default hugetlb size
+			used for shmget(), mmap() and mounting hugetlbfs
+			filesystems.  If not specified, defaults to the
+			architecture's default huge page size.  Huge page
+			sizes are architecture dependent.  See also
+			Documentation/admin-guide/mm/hugetlbpage.rst.
+			Format: size[KMG]
 
 	deferred_probe_timeout 			[KNL] Debugging option to set a timeout in seconds for
@@ -1479,13 +1482,24 @@
 			hugepages using the cma allocator. If enabled, the
 			boot-time allocation of gigantic hugepages is skipped.
 
-	hugepages=	[HW,X86-32,IA-64] HugeTLB pages to allocate at boot.
-	hugepagesz=	[HW,IA-64,PPC,X86-64] The size of the HugeTLB pages.
-			On x86-64 and powerpc, this option can be specified
-			multiple times interleaved with hugepages= to reserve
-			huge pages of different sizes. Valid pages sizes on
-			x86-64 are 2M (when the CPU supports "pse") and 1G
-			(when the CPU supports the "pdpe1gb" cpuinfo flag).
+	hugepages=	[HW] Number of HugeTLB pages to allocate at boot.
+			If this follows hugepagesz (below), it specifies
+			the number of pages of hugepagesz to be allocated.
+			If this is the first HugeTLB parameter on the command
+			line, it specifies the number of pages to allocate for
+			the default huge page size.  See also
+			Documentation/admin-guide/mm/hugetlbpage.rst.
+			Format: <integer>
+
+	hugepagesz+			[HW] The size of the HugeTLB pages.  This is used in
+			conjunction with hugepages (above) to allocate huge
+			pages of a specific size at boot.  The pair
+			hugepagesz=X hugepages=Y can be specified once for
+			each supported huge page size. Huge page sizes are
+			architecture dependent.  See also
+			Documentation/admin-guide/mm/hugetlbpage.rst.
+			Format: size[KMG]
 
 	hung_task_panic 			[KNL] Should the hung task detector generate panics.
Documentation/admin-guide/mm/hugetlbpage.rst +35 -0
index 1cc0bc78d10e..5026e58826e2 100644
@@ -100,6 +100,41 @@ with a huge page size selection parameter "hugepagesz=<size>".  <size> must
 be specified in bytes with optional scale suffix [kKmMgG].  The default huge
 page size may be selected with the "default_hugepagesz=<size>" boot parameter.
 
+Hugetlb boot command line parameter semantics
+hugepagesz - Specify a huge page size.  Used in conjunction with hugepages
+	parameter to preallocate a number of huge pages of the specified
+	size.  Hence, hugepagesz and hugepages are typically specified in
+	pairs such as:
+		hugepagesz=2M hugepagesQ2
+	hugepagesz can only be specified once on the command line for a
+	specific huge page size.  Valid huge page sizes are architecture
+	dependent.
+hugepages - Specify the number of huge pages to preallocate.  This typically
+	follows a valid hugepagesz or default_hugepagesz parameter.  However,
+	if hugepages is the first or only hugetlb command line parameter it
+	implicitly specifies the number of huge pages of default size to
+	allocate.  If the number of huge pages of default size is implicitly
+	specified, it can not be overwritten by a hugepagesz,hugepages
+	parameter pair for the default size.
+	For example, on an architecture with 2M default huge page size:
+		hugepages%6 hugepagesz=2M hugepagesQ2
+	will result in 256 2M huge pages being allocated and a warning message
+	indicating that the hugepagesQ2 parameter is ignored.  If a hugepages
+	parameter is preceded by an invalid hugepagesz parameter, it will
+	be ignored.
+default_hugepagesz - Specify the default huge page size.  This parameter can
+	only be specified once on the command line.  default_hugepagesz can
+	optionally be followed by the hugepages parameter to preallocate a
+	specific number of huge pages of default size.  The number of default
+	sized huge pages to preallocate can also be implicitly specified as
+	mentioned in the hugepages section above.  Therefore, on an
+	architecture with 2M default huge page size:
+		hugepages%6
+		default_hugepagesz=2M hugepages%6
+		hugepages%6 default_hugepagesz=2M
+	will all result in 256 2M huge pages being allocated.  Valid default
+	huge page size is architecture dependent.
+
 When multiple huge page sizes are supported, ``/proc/sys/vm/nr_hugepages``
 indicates the current number of pre-allocated huge pages of the default size.
 Thus, one can use the following command to dynamically allocate/deallocate
mm/hugetlb.c +127 -31

238 lines not on this page · raw message

Signature · 1 line
2.25.2
#14 lore raw
In reply to Mike Kravetz
Quoted 21 lines · Mike Kravetz

Longpeng(Mike) reported a weird message from hugetlb command line processing and proposed a solution [1]. While the proposed patch does address the specific issue, there are other related issues in command line processing. As hugetlbfs evolved, updates to command line processing have been made to meet immediate needs and not necessarily in a coordinated manner. The result is that some processing is done in arch specific code, some is done in arch independent code and coordination is problematic. Semantics can vary between architectures.

The patch series does the following:
- Define arch specific arch_hugetlb_valid_size routine used to validate passed huge page sizes.
- Move hugepagesz= command line parsing out of arch specific code and into an arch independent routine.
- Clean up command line processing to follow desired semantics and document those semantics.

Quoted 7 lines

[1] https://lore.kernel.org/linux-mm/[email protected]

Mike Kravetz (4):
hugetlbfs: add arch_hugetlb_valid_size
hugetlbfs: move hugepagesz= parsing to arch independent code
hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate
hugetlbfs: clean up command line processing

Reverted this series fixed many undefined behaviors on arm64 with the config,

https://raw.githubusercontent.com/cailca/linux-mm/master/arm64.config

[   54.172683][    T1] UBSAN: shift-out-of-bounds in ./include/linux/hugetlb.h:555:34
[   54.180411][    T1] shift exponent 4294967285 is too large for 64-bit type 'unsigned long'
[   54.188885][    T1] CPU: 130 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc2-next-20200420 #1
[   54.197284][    T1] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[   54.207888][    T1] Call trace:
[   54.211100][    T1]  dump_backtrace+0x0/0x224
[   54.215565][    T1]  show_stack+0x20/0x2c
[   54.219651][    T1]  dump_stack+0xfc/0x184
[   54.223829][    T1]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[   54.230204][    T1]  hugetlb_add_hstate+0x3ec/0x414
huge_page_size at include/linux/hugetlb.h:555
(inlined by) hugetlb_add_hstate at mm/hugetlb.c:3301
[   54.235191][    T1]  hugetlbpage_init+0x14/0x30
[   54.239824][    T1]  do_one_initcall+0x6c/0x144
[   54.244446][    T1]  do_initcall_level+0x158/0x1c4
[   54.249336][    T1]  do_initcalls+0x68/0xb0
[   54.253597][    T1]  do_basic_setup+0x28/0x30
[   54.258049][    T1]  kernel_init_freeable+0x19c/0x228
[   54.263188][    T1]  kernel_init+0x14/0x208
[   54.267473][    T1]  ret_from_fork+0x10/0x18
[   55.534338][    T1] UBSAN: shift-out-of-bounds in ./include/linux/hugetlb.h:555:34
[   55.542064][    T1] shift exponent 4294967285 is too large for 64-bit type 'unsigned long'
[   55.550555][    T1] CPU: 129 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc2-next-20200420 #1
[   55.558992][    T1] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[   55.569659][    T1] Call trace:
[   55.572898][    T1]  dump_backtrace+0x0/0x224
[   55.577335][    T1]  show_stack+0x20/0x2c
[   55.581442][    T1]  dump_stack+0xfc/0x184
[   55.585621][    T1]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[   55.592031][    T1]  __hugetlb_cgroup_file_dfl_init+0x37c/0x384
[   55.598062][    T1]  hugetlb_cgroup_file_init+0x9c/0xd8
[   55.603399][    T1]  hugetlb_init+0x248/0x448
[   55.607840][    T1]  do_one_initcall+0x6c/0x144
[   55.612493][    T1]  do_initcall_level+0x158/0x1c4
[   55.617404][    T1]  do_initcalls+0x68/0xb0
[   55.621664][    T1]  do_basic_setup+0x28/0x30
[   55.626107][    T1]  kernel_init_freeable+0x19c/0x228
[   55.631253][    T1]  kernel_init+0x14/0x208
[   55.635519][    T1]  ret_from_fork+0x10/0x18
[  153.283648][    T1] ================================================================================
[  153.293078][    T1] UBSAN: shift-out-of-bounds in ./include/linux/hugetlb.h:555:34
[  153.300841][    T1] shift exponent 4294967285 is too large for 64-bit type 'unsigned long'
[  153.309185][    T1] CPU: 161 PID: 1 Comm: swapper/0 Tainted: G             L    5.7.0-rc2-next-20200420 #1
[  153.318879][    T1] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[  153.329352][    T1] Call trace:
[  153.332545][    T1]  dump_backtrace+0x0/0x224
[  153.336945][    T1]  show_stack+0x20/0x2c
[  153.341000][    T1]  dump_stack+0xfc/0x184
[  153.345149][    T1]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[  153.351465][    T1]  hugetlbfs_fill_super+0x424/0x43c
[  153.356560][    T1]  vfs_get_super+0xcc/0x170
[  153.360959][    T1]  get_tree_nodev+0x28/0x34
[  153.365358][    T1]  hugetlbfs_get_tree+0xfc/0x128
[  153.370193][    T1]  vfs_get_tree+0x54/0x158
[  153.374513][    T1]  fc_mount+0x1c/0x5c
[  153.378399][    T1]  mount_one_hugetlbfs+0x54/0xc8
[  153.383233][    T1]  init_hugetlbfs_fs+0x18c/0x268
[  153.388068][    T1]  do_one_initcall+0x6c/0x144
[  153.392647][    T1]  do_initcall_level+0x158/0x1c4
[  153.397480][    T1]  do_initcalls+0x68/0xb0
[  153.401706][    T1]  do_basic_setup+0x28/0x30
[  153.406105][    T1]  kernel_init_freeable+0x19c/0x228
[  153.411208][    T1]  kernel_init+0x14/0x208
[  153.415436][    T1]  ret_from_fork+0x10/0x18
[  194.312926][ T1828] UBSAN: shift-out-of-bounds in ./include/linux/hugetlb.h:584:11
[  194.320664][ T1828] shift exponent 4294967285 is too large for 32-bit type 'int'
[  194.328103][ T1828] CPU: 194 PID: 1828 Comm: systemd-journal Tainted: G             L    5.7.0-rc2-next-20200420 #1
[  194.338558][ T1828] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[  194.349010][ T1828] Call trace:
[  194.352183][ T1828]  dump_backtrace+0x0/0x224
[  194.356560][ T1828]  show_stack+0x20/0x2c
[  194.360595][ T1828]  dump_stack+0xfc/0x184
[  194.364724][ T1828]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[  194.371020][ T1828]  hugetlb_total_pages+0x100/0x128
[  194.376017][ T1828]  vm_commit_limit+0x54/0xb0
[  194.380484][ T1828]  meminfo_proc_show+0x8f4/0xc4c
[  194.385297][ T1828]  seq_read+0x380/0x930
[  194.389353][ T1828]  pde_read+0x5c/0x78
[  194.393232][ T1828]  proc_reg_read+0x74/0xc0
[  194.397528][ T1828]  __vfs_read+0x84/0x1d0
[  194.401646][ T1828]  vfs_read+0xec/0x124
[  194.405588][ T1828]  ksys_read+0xb0/0x120
[  194.409643][ T1828]  __arm64_sys_read+0x54/0x88
[  194.414195][ T1828]  do_el0_svc+0x128/0x1dc
[  194.418405][ T1828]  el0_sync_handler+0x150/0x250
[  194.423157][ T1828]  el0_sync+0x164/0x180
[  194.427425][ T1828] ================================================================================
[  194.436930][ T1828] ================================================================================
[  194.446355][ T1828] UBSAN: shift-out-of-bounds in mm/hugetlb.c:3564:23
[  194.453190][ T1828] shift exponent 4294967285 is too large for 64-bit type 'unsigned long'
[  194.461752][ T1828] CPU: 194 PID: 1828 Comm: systemd-journal Tainted: G             L    5.7.0-rc2-next-20200420 #1
[  194.472245][ T1828] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[  194.482720][ T1828] Call trace:
[  194.485909][ T1828]  dump_backtrace+0x0/0x224
[  194.490312][ T1828]  show_stack+0x20/0x2c
[  194.494368][ T1828]  dump_stack+0xfc/0x184
[  194.498513][ T1828]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[  194.504828][ T1828]  hugetlb_report_meminfo+0x25c/0x2ac
[  194.510103][ T1828]  meminfo_proc_show+0xc08/0xc4c
[  194.514938][ T1828]  seq_read+0x380/0x930
[  194.518993][ T1828]  pde_read+0x5c/0x78
[  194.522874][ T1828]  proc_reg_read+0x74/0xc0
[  194.527190][ T1828]  __vfs_read+0x84/0x1d0
[  194.531335][ T1828]  vfs_read+0xec/0x124
[  194.535304][ T1828]  ksys_read+0xb0/0x120
[  194.539371][ T1828]  __arm64_sys_read+0x54/0x88
[  194.543958][ T1828]  do_el0_svc+0x128/0x1dc
[  194.548187][ T1828]  el0_sync_handler+0x150/0x250
[  194.552936][ T1828]  el0_sync+0x164/0x180
Quoted 17 lines
.../admin-guide/kernel-parameters.txt         |  40 ++--
Documentation/admin-guide/mm/hugetlbpage.rst  |  35 ++++
arch/arm64/mm/hugetlbpage.c                   |  30 +--
arch/powerpc/mm/hugetlbpage.c                 |  30 +--
arch/riscv/mm/hugetlbpage.c                   |  24 +--
arch/s390/mm/hugetlbpage.c                    |  24 +--
arch/sparc/mm/init_64.c                       |  43 +---
arch/x86/mm/hugetlbpage.c                     |  23 +--
include/linux/hugetlb.h                       |   2 +-
mm/hugetlb.c                                  | 190 +++++++++++++++---
10 files changed, 271 insertions(+), 170 deletions(-)
Signature · 1 line
2.25.2
#15 lore raw
Quoted 45 lines · Qian Cai
Quoted 21 lines · Mike Kravetz

Longpeng(Mike) reported a weird message from hugetlb command line processing and proposed a solution [1]. While the proposed patch does address the specific issue, there are other related issues in command line processing. As hugetlbfs evolved, updates to command line processing have been made to meet immediate needs and not necessarily in a coordinated manner. The result is that some processing is done in arch specific code, some is done in arch independent code and coordination is problematic. Semantics can vary between architectures.

The patch series does the following:
- Define arch specific arch_hugetlb_valid_size routine used to validate passed huge page sizes.
- Move hugepagesz= command line parsing out of arch specific code and into an arch independent routine.
- Clean up command line processing to follow desired semantics and document those semantics.

Quoted 7 lines

[1] https://lore.kernel.org/linux-mm/[email protected]

Mike Kravetz (4):
hugetlbfs: add arch_hugetlb_valid_size
hugetlbfs: move hugepagesz= parsing to arch independent code
hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate
hugetlbfs: clean up command line processing

Reverted this series fixed many undefined behaviors on arm64 with the config,

https://raw.githubusercontent.com/cailca/linux-mm/master/arm64.config

[   54.172683][    T1] UBSAN: shift-out-of-bounds in ./include/linux/hugetlb.h:555:34
[   54.180411][    T1] shift exponent 4294967285 is too large for 64-bit type 'unsigned long'
[   54.188885][    T1] CPU: 130 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc2-next-20200420 #1
[   54.197284][    T1] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[   54.207888][    T1] Call trace:
[   54.211100][    T1]  dump_backtrace+0x0/0x224
[   54.215565][    T1]  show_stack+0x20/0x2c
[   54.219651][    T1]  dump_stack+0xfc/0x184
[   54.223829][    T1]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[   54.230204][    T1]  hugetlb_add_hstate+0x3ec/0x414

huge_page_size at include/linux/hugetlb.h:555 (inlined by) hugetlb_add_hstate at mm/hugetlb.c:3301

Quoted 8 lines
[   54.235191][    T1]  hugetlbpage_init+0x14/0x30
[   54.239824][    T1]  do_one_initcall+0x6c/0x144
[   54.244446][    T1]  do_initcall_level+0x158/0x1c4
[   54.249336][    T1]  do_initcalls+0x68/0xb0
[   54.253597][    T1]  do_basic_setup+0x28/0x30
[   54.258049][    T1]  kernel_init_freeable+0x19c/0x228
[   54.263188][    T1]  kernel_init+0x14/0x208
[   54.267473][    T1]  ret_from_fork+0x10/0x18

While rearranging the code (patch 3 in series), I made the incorrect assumption that CONT_XXX_SIZE == (1UL << CONT_XXX_SHIFT). However, this is not the case. Does the following patch fix these issues?

From b75cb4a0852e208bee8c4eb347dc076fcaa88859 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 10:41:18 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
 arch/arm64/mm/hugetlbpage.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/mm/hugetlbpage.c +4 -4
index 9ca840527296..a02411a1f19a 100644
@@ -453,11 +453,11 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
 static int __init hugetlbpage_init(void)
 {
 #ifdef CONFIG_ARM64_4K_PAGES
-	hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(PUD_SIZE) - PAGE_SHIFT);
 #endif
-	hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
-	hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-	hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(CONT_PMD_SIZE) - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(PMD_SIZE) - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(CONT_PTE_SIZE) - PAGE_SHIFT);
 
 	return 0;
 }
Signature · 1 line
2.25.2
maintainer
#16 lore raw
Quoted 79 lines · Mike Kravetz
Quoted 45 lines · Qian Cai
Quoted 21 lines · Mike Kravetz

Longpeng(Mike) reported a weird message from hugetlb command line processing and proposed a solution [1]. While the proposed patch does address the specific issue, there are other related issues in command line processing. As hugetlbfs evolved, updates to command line processing have been made to meet immediate needs and not necessarily in a coordinated manner. The result is that some processing is done in arch specific code, some is done in arch independent code and coordination is problematic. Semantics can vary between architectures.

The patch series does the following:
- Define arch specific arch_hugetlb_valid_size routine used to validate passed huge page sizes.
- Move hugepagesz= command line parsing out of arch specific code and into an arch independent routine.
- Clean up command line processing to follow desired semantics and document those semantics.

Quoted 7 lines

[1] https://lore.kernel.org/linux-mm/[email protected]

Mike Kravetz (4):
hugetlbfs: add arch_hugetlb_valid_size
hugetlbfs: move hugepagesz= parsing to arch independent code
hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate
hugetlbfs: clean up command line processing

Reverted this series fixed many undefined behaviors on arm64 with the config,

https://raw.githubusercontent.com/cailca/linux-mm/master/arm64.config

[   54.172683][    T1] UBSAN: shift-out-of-bounds in ./include/linux/hugetlb.h:555:34
[   54.180411][    T1] shift exponent 4294967285 is too large for 64-bit type 'unsigned long'
[   54.188885][    T1] CPU: 130 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc2-next-20200420 #1
[   54.197284][    T1] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[   54.207888][    T1] Call trace:
[   54.211100][    T1]  dump_backtrace+0x0/0x224
[   54.215565][    T1]  show_stack+0x20/0x2c
[   54.219651][    T1]  dump_stack+0xfc/0x184
[   54.223829][    T1]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[   54.230204][    T1]  hugetlb_add_hstate+0x3ec/0x414

huge_page_size at include/linux/hugetlb.h:555 (inlined by) hugetlb_add_hstate at mm/hugetlb.c:3301

Quoted 8 lines
[   54.235191][    T1]  hugetlbpage_init+0x14/0x30
[   54.239824][    T1]  do_one_initcall+0x6c/0x144
[   54.244446][    T1]  do_initcall_level+0x158/0x1c4
[   54.249336][    T1]  do_initcalls+0x68/0xb0
[   54.253597][    T1]  do_basic_setup+0x28/0x30
[   54.258049][    T1]  kernel_init_freeable+0x19c/0x228
[   54.263188][    T1]  kernel_init+0x14/0x208
[   54.267473][    T1]  ret_from_fork+0x10/0x18

While rearranging the code (patch 3 in series), I made the incorrect assumption that CONT_XXX_SIZE = (1UL << CONT_XXX_SHIFT). However, this is not the case. Does the following patch fix these issues?

From b75cb4a0852e208bee8c4eb347dc076fcaa88859 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 10:41:18 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
 arch/arm64/mm/hugetlbpage.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/mm/hugetlbpage.c +0 -0
index 9ca840527296..a02411a1f19a 100644
@@ -453,11 +453,11 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
 static int __init hugetlbpage_init(void)
 {

arch/arm64/mm/hugetlbpage.c void huge_ptep_clear_flush(struct vm_area_struct *vma, · line 460

Quoted 10 lines
arch/arm64/mm/hugetlbpage.c +4 -4
 #ifdef CONFIG_ARM64_4K_PAGES
-	hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(PUD_SIZE) - PAGE_SHIFT);
 #endif
-	hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
-	hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-	hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(CONT_PMD_SIZE) - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(PMD_SIZE) - PAGE_SHIFT);
+	hugetlb_add_hstate(ilog2(CONT_PTE_SIZE) - PAGE_SHIFT);

Might be clearer to leave the non CONT_* definitions alone and instead convert the CONT versions along the lines of:

	hugetlb_add_hstate(CONT_PMD_SHIFT + PMD_SHIFT - PAGE_SHIFT);

(untested, but I think that's right...)

But that doesn't matter until Qian has confirmed that your patch fixes the issue.

Will

#17 lore raw
In reply to Mike Kravetz
Quoted 83 lines · Mike Kravetz
Quoted 45 lines · Qian Cai
Quoted 21 lines · Mike Kravetz

Longpeng(Mike) reported a weird message from hugetlb command line processing and proposed a solution [1]. While the proposed patch does address the specific issue, there are other related issues in command line processing. As hugetlbfs evolved, updates to command line processing have been made to meet immediate needs and not necessarily in a coordinated manner. The result is that some processing is done in arch specific code, some is done in arch independent code and coordination is problematic. Semantics can vary between architectures.

The patch series does the following:
- Define arch specific arch_hugetlb_valid_size routine used to validate passed huge page sizes.
- Move hugepagesz= command line parsing out of arch specific code and into an arch independent routine.
- Clean up command line processing to follow desired semantics and document those semantics.

Quoted 7 lines

[1] https://lore.kernel.org/linux-mm/[email protected]

Mike Kravetz (4):
hugetlbfs: add arch_hugetlb_valid_size
hugetlbfs: move hugepagesz= parsing to arch independent code
hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate
hugetlbfs: clean up command line processing

Reverted this series fixed many undefined behaviors on arm64 with the config,

https://raw.githubusercontent.com/cailca/linux-mm/master/arm64.config

[   54.172683][    T1] UBSAN: shift-out-of-bounds in ./include/linux/hugetlb.h:555:34
[   54.180411][    T1] shift exponent 4294967285 is too large for 64-bit type 'unsigned long'
[   54.188885][    T1] CPU: 130 PID: 1 Comm: swapper/0 Not tainted 5.7.0-rc2-next-20200420 #1
[   54.197284][    T1] Hardware name: HPE Apollo 70             /C01_APACHE_MB         , BIOS L50_5.13_1.11 06/18/2019
[   54.207888][    T1] Call trace:
[   54.211100][    T1]  dump_backtrace+0x0/0x224
[   54.215565][    T1]  show_stack+0x20/0x2c
[   54.219651][    T1]  dump_stack+0xfc/0x184
[   54.223829][    T1]  __ubsan_handle_shift_out_of_bounds+0x304/0x344
[   54.230204][    T1]  hugetlb_add_hstate+0x3ec/0x414

huge_page_size at include/linux/hugetlb.h:555 (inlined by) hugetlb_add_hstate at mm/hugetlb.c:3301

Quoted 8 lines
[   54.235191][    T1]  hugetlbpage_init+0x14/0x30
[   54.239824][    T1]  do_one_initcall+0x6c/0x144
[   54.244446][    T1]  do_initcall_level+0x158/0x1c4
[   54.249336][    T1]  do_initcalls+0x68/0xb0
[   54.253597][    T1]  do_basic_setup+0x28/0x30
[   54.258049][    T1]  kernel_init_freeable+0x19c/0x228
[   54.263188][    T1]  kernel_init+0x14/0x208
[   54.267473][    T1]  ret_from_fork+0x10/0x18

While rearranging the code (patch 3 in series), I made the incorrect assumption that CONT_XXX_SIZE = (1UL << CONT_XXX_SHIFT). However, this is not the case. Does the following patch fix these issues?

From b75cb4a0852e208bee8c4eb347dc076fcaa88859 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 10:41:18 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
 arch/arm64/mm/hugetlbpage.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/mm/hugetlbpage.c +1 -1
index 9ca840527296..a02411a1f19a 100644
@@ -453,11 +453,11 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
 static int __init hugetlbpage_init(void)
 {
 #ifdef CONFIG_ARM64_4K_PAGES
-       hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PUD_SIZE) - PAGE_SHIFT);

arch/arm64/mm/hugetlbpage.c void huge_ptep_clear_flush(struct vm_area_struct *vma, · line 463

Quoted 10 lines
arch/arm64/mm/hugetlbpage.c +3 -3
 #endif
-       hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PTE_SIZE) - PAGE_SHIFT);
 
        return 0;
 }

I build this for an arm64 kernel and ran it in qemu and it worked.

Cheers,
Anders

#18 lore raw
Quoted 39 lines · Anders Roxell
Quoted 38 lines · Mike Kravetz
Quoted 2 lines · Qian Cai

Reverted this series fixed many undefined behaviors on arm64 with the config,

While rearranging the code (patch 3 in series), I made the incorrect assumption that CONT_XXX_SIZE = (1UL << CONT_XXX_SHIFT). However, this is not the case. Does the following patch fix these issues?

From b75cb4a0852e208bee8c4eb347dc076fcaa88859 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 10:41:18 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
 arch/arm64/mm/hugetlbpage.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/mm/hugetlbpage.c +4 -4
index 9ca840527296..a02411a1f19a 100644
@@ -453,11 +453,11 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
 static int __init hugetlbpage_init(void)
 {
 #ifdef CONFIG_ARM64_4K_PAGES
-       hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PUD_SIZE) - PAGE_SHIFT);
 #endif
-       hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PTE_SIZE) - PAGE_SHIFT);
 
Quoted 4 lines

arch/arm64/mm/hugetlbpage.c void huge_ptep_clear_flush(struct vm_area_struct *vma, · line 463

Quoted 2 lines
arch/arm64/mm/hugetlbpage.c +0 -0
        return 0;
 }

I build this for an arm64 kernel and ran it in qemu and it worked.

Thanks for testing Anders!

Will, here is an updated version of the patch based on your suggestion. I added the () for emphasis but that may just be noise for some. Also, the naming differences and values for CONT_PTE may make some people look twice. Not sure if being consistent here helps?

I have only built this. No testing.

From daf833ab6b806ecc0816d84d45dcbacc052a7eec Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 13:56:15 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
 arch/arm64/mm/hugetlbpage.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/mm/hugetlbpage.c +2 -2
index 9ca840527296..bed6dc7c4276 100644
@@ -455,9 +455,9 @@ static int __init hugetlbpage_init(void)
 #ifdef CONFIG_ARM64_4K_PAGES
 	hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
 #endif
-	hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate((CONT_PMD_SHIFT + PMD_SHIFT) - PAGE_SHIFT);
 	hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-	hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+	hugetlb_add_hstate((CONT_PTE_SHIFT + PAGE_SHIFT) - PAGE_SHIFT);
 
 	return 0;
 }
Signature · 1 line
2.25.2
Tested-by
#19 lore raw
Quoted 60 lines · Mike Kravetz
Quoted 39 lines · Anders Roxell
Quoted 38 lines · Mike Kravetz
Quoted 2 lines · Qian Cai

Reverted this series fixed many undefined behaviors on arm64 with the config,

While rearranging the code (patch 3 in series), I made the incorrect assumption that CONT_XXX_SIZE = (1UL << CONT_XXX_SHIFT). However, this is not the case. Does the following patch fix these issues?

From b75cb4a0852e208bee8c4eb347dc076fcaa88859 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 10:41:18 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
 arch/arm64/mm/hugetlbpage.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/mm/hugetlbpage.c +4 -4
index 9ca840527296..a02411a1f19a 100644
@@ -453,11 +453,11 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
 static int __init hugetlbpage_init(void)
 {
 #ifdef CONFIG_ARM64_4K_PAGES
-       hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PUD_SIZE) - PAGE_SHIFT);
 #endif
-       hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PTE_SIZE) - PAGE_SHIFT);
 
Quoted 4 lines

arch/arm64/mm/hugetlbpage.c static int __init hugetlbpage_init(void) · line 463

Quoted 2 lines
arch/arm64/mm/hugetlbpage.c +0 -0
        return 0;
 }

I build this for an arm64 kernel and ran it in qemu and it worked.

Thanks for testing Anders!

Will, here is an updated version of the patch based on your suggestion. I added the () for emphasis but that may just be noise for some. Also, the naming differences and values for CONT_PTE may make some people look twice. Not sure if being consistent here helps?

I have only built this. No testing.

From daf833ab6b806ecc0816d84d45dcbacc052a7eec Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 13:56:15 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

Quoted 4 lines

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

I tested this patch on qemu-aarch64.

Cheers,
Anders

Quoted 23 lines
---
 arch/arm64/mm/hugetlbpage.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/mm/hugetlbpage.c +2 -3
index 9ca840527296..bed6dc7c4276 100644
@@ -455,9 +455,9 @@ static int __init hugetlbpage_init(void)
 #ifdef CONFIG_ARM64_4K_PAGES
        hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
 #endif
-       hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate((CONT_PMD_SHIFT + PMD_SHIFT) - PAGE_SHIFT);
        hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate((CONT_PTE_SHIFT + PAGE_SHIFT) - PAGE_SHIFT);
 
        return 0;
 }
--

2.25.2

Tested-by Anders Roxell
#20 lore raw
 

    On Wednesday, April 22, 2020, 05:52:05 PM EDT, Anders Roxell <[email protected]> wrote:  
Quoted 60 lines · Mike Kravetz
Quoted 39 lines · Anders Roxell
Quoted 38 lines · Mike Kravetz
Quoted 2 lines · Qian Cai

Reverted this series fixed many undefined behaviors on arm64 with the config,

While rearranging the code (patch 3 in series), I made the incorrect assumption that CONT_XXX_SIZE == (1UL << CONT_XXX_SHIFT).  However, this is not the case.  Does the following patch fix these issues?

From b75cb4a0852e208bee8c4eb347dc076fcaa88859 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 10:41:18 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
  arch/arm64/mm/hugetlbpage.c | 8 ++++----

  1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/mm/hugetlbpage.c +4 -4
index 9ca840527296..a02411a1f19a 100644
@@ -453,11 +453,11 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
   static int __init hugetlbpage_init(void)
   {
   #ifdef CONFIG_ARM64_4K_PAGES
-      hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+      hugetlb_add_hstate(ilog2(PUD_SIZE) - PAGE_SHIFT);
   #endif
-      hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
-      hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-      hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+      hugetlb_add_hstate(ilog2(CONT_PMD_SIZE) - PAGE_SHIFT);
+      hugetlb_add_hstate(ilog2(PMD_SIZE) - PAGE_SHIFT);
+      hugetlb_add_hstate(ilog2(CONT_PTE_SIZE) - PAGE_SHIFT);
 
Quoted 4 lines

arch/arm64/mm/hugetlbpage.c static int __init hugetlbpage_init(void) · line 463

Quoted 2 lines
arch/arm64/mm/hugetlbpage.c +0 -0
         return 0;
   }

I build this for an arm64 kernel and ran it in qemu and it worked.

Thanks for testing Anders!

Will, here is an updated version of the patch based on your suggestion. I added the () for emphasis but that may just be noise for some.  Also, the naming differences and values for CONT_PTE may make some people look twice.  Not sure if being consistent here helps?

I have only built this.  No testing.

From daf833ab6b806ecc0816d84d45dcbacc052a7eec Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 13:56:15 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

Quoted 4 lines

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

I tested this patch on qemu-aarch64.

Cheers,
Anders

Quoted 21 lines

---
  arch/arm64/mm/hugetlbpage.c | 4 ++--

  1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/mm/hugetlbpage.c +2 -3
index 9ca840527296..bed6dc7c4276 100644
@@ -455,9 +455,9 @@ static int __init hugetlbpage_init(void)
   #ifdef CONFIG_ARM64_4K_PAGES
         hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
   #endif
-      hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
+      hugetlb_add_hstate((CONT_PMD_SHIFT + PMD_SHIFT) - PAGE_SHIFT);
         hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-      hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+      hugetlb_add_hstate((CONT_PTE_SHIFT + PAGE_SHIFT) - PAGE_SHIFT);
 
         return 0;
   }
--
Quoted 2 lines

2.25.2

_______________________________________________
linux-arm-kernel mailing list
[email protected]
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Signature · 202 lines
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp
derp 
derp
derp
derp
derp
derp
derp
derp
--- 
You received this message because you are subscribed to the Google Groups "derpyderpderp" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [email protected].
maintainer Acked-by
#21 lore raw
In reply to Mike Kravetz
Quoted 45 lines · Mike Kravetz
Quoted 39 lines · Anders Roxell
Quoted 38 lines · Mike Kravetz
Quoted 2 lines · Qian Cai

Reverted this series fixed many undefined behaviors on arm64 with the config,

While rearranging the code (patch 3 in series), I made the incorrect assumption that CONT_XXX_SIZE = (1UL << CONT_XXX_SHIFT). However, this is not the case. Does the following patch fix these issues?

From b75cb4a0852e208bee8c4eb347dc076fcaa88859 Mon Sep 17 00:00:00 2001
From: Mike Kravetz <[email protected]>
Date: Mon, 20 Apr 2020 10:41:18 -0700
Subject: [PATCH] arm64/hugetlb: fix hugetlb initialization

When calling hugetlb_add_hstate() to initialize a new hugetlb size, be sure to use correct huge pages size order.

---
 arch/arm64/mm/hugetlbpage.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/arm64/mm/hugetlbpage.c +4 -4
index 9ca840527296..a02411a1f19a 100644
@@ -453,11 +453,11 @@ void huge_ptep_clear_flush(struct vm_area_struct *vma,
 static int __init hugetlbpage_init(void)
 {
 #ifdef CONFIG_ARM64_4K_PAGES
-       hugetlb_add_hstate(PUD_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PUD_SIZE) - PAGE_SHIFT);
 #endif
-       hugetlb_add_hstate(CONT_PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(PMD_SHIFT - PAGE_SHIFT);
-       hugetlb_add_hstate(CONT_PTE_SHIFT - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(PMD_SIZE) - PAGE_SHIFT);
+       hugetlb_add_hstate(ilog2(CONT_PTE_SIZE) - PAGE_SHIFT);
 
Quoted 4 lines

arch/arm64/mm/hugetlbpage.c static int __init hugetlbpage_init(void) · line 463

Quoted 2 lines
arch/arm64/mm/hugetlbpage.c +0 -0
        return 0;
 }

I build this for an arm64 kernel and ran it in qemu and it worked.

Quoted 6 lines

Thanks for testing Anders!

Will, here is an updated version of the patch based on your suggestion. I added the () for emphasis but that may just be noise for some. Also, the naming differences and values for CONT_PTE may make some people look twice. Not sure if being consistent here helps?

Cheers, thanks for this. I think being consistent is worthwhile, as it's the definitions themselves that are weird and we can conceivably clean that up as a separate patch.

So,

Looks like Andrew already picked it up (thanks!)

Thanks,

Will

Acked-by on the v3 series
#22 lore raw
In reply to Mike Kravetz
Quoted 47 lines · On Fri, 17 Apr 2020 11:50:45 -0700 Mike Kravetz <[email protected]> wrote:

v3 -
Used weak attribute method of defining arch_hugetlb_valid_size.
This eliminates changes to arch specific hugetlb.h files (Peter)
Updated documentation (Peter, Randy)
Fixed handling of implicitly specified gigantic page preallocation
in existing code and removed documentation of such. There is now no difference between handling of gigantic and non-gigantic pages. (Peter, Nitesh).
This requires the most review as there is a small change to undocumented behavior. See patch 4 commit message for details.
Added Acks and Reviews (Mina, Peter)

v2 -
Fix build errors with patch 1 (Will)
Change arch_hugetlb_valid_size arg to unsigned long and remove
irrelevant 'extern' keyword (Christophe)
Documentation and other misc changes (Randy, Christophe, Mina) Do not process command line options if !hugepages_supported()
(Dave, but it sounds like we may want to additional changes to
hugepages_supported() for x86? If that is needed I would prefer a separate patch.)

Longpeng(Mike) reported a weird message from hugetlb command line processing and proposed a solution [1]. While the proposed patch does address the specific issue, there are other related issues in command line processing. As hugetlbfs evolved, updates to command line processing have been made to meet immediate needs and not necessarily in a coordinated manner. The result is that some processing is done in arch specific code, some is done in arch independent code and coordination is problematic. Semantics can vary between architectures.

The patch series does the following:
- Define arch specific arch_hugetlb_valid_size routine used to validate passed huge page sizes.
- Move hugepagesz= command line parsing out of arch specific code and into an arch independent routine.
- Clean up command line processing to follow desired semantics and document those semantics.

[1] https://lore.kernel.org/linux-mm/[email protected]

Mike Kravetz (4):
hugetlbfs: add arch_hugetlb_valid_size
hugetlbfs: move hugepagesz= parsing to arch independent code
hugetlbfs: remove hugetlb_add_hstate() warning for existing hstate
hugetlbfs: clean up command line processing

Quoted 12 lines
 .../admin-guide/kernel-parameters.txt         |  40 ++--
 Documentation/admin-guide/mm/hugetlbpage.rst  |  35 ++++
 arch/arm64/mm/hugetlbpage.c                   |  30 +--
 arch/powerpc/mm/hugetlbpage.c                 |  30 +--
 arch/riscv/mm/hugetlbpage.c                   |  24 +--
 arch/s390/mm/hugetlbpage.c                    |  24 +--
 arch/sparc/mm/init_64.c                       |  43 +---
 arch/x86/mm/hugetlbpage.c                     |  23 +--
 include/linux/hugetlb.h                       |   2 +-
 mm/hugetlb.c                                  | 190 +++++++++++++++---
 10 files changed, 271 insertions(+), 170 deletions(-)

Looks good and works fine for s390, thanks for cleaning up!