mm: thp: always enable mTHP support

Today, shmem_getattr() sets stat->blksize to PMD size whenever shmem_huge_global_enabled() returns non-zero. While this works fine for the normal THP-enabled case as explained by Baolin in [1], this has two problems:

1. Theoretically, when shmem is configured for within_size, this could set blksize to PMD size even though the allocation may be a smaller mTHP order

2. A future commit will allow shmem THP support to be enabled even when the CPU doesn't support PMD-sized pages. We should not allow blksize to be set to PMD size in this case

In order to fix #1 and prepare for #2, this commit sets blksize to the size of the highest supported order returned by shmem_huge_global_enabled().

[1] https://lore.kernel.org/linux-mm/[email protected]/

Suggested-by Baolin Wang <[email protected]>

Acked-by Zi Yan <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
 mm/shmem.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

mm/shmem.c +1 -0

index b572c60f2af8..776dff8a848e 100644
@@ -1506,6 +1506,7 @@ static int shmem_getattr(struct mnt_idmap *idmap,
 {
 	struct inode *inode = path->dentry->d_inode;
 	struct shmem_inode_info *info = SHMEM_I(inode);
+	unsigned int orders;
 
 	/* Fast-path hint; recalc under info->lock corrects any stale read. */
 	if (data_race(info->alloced - info->swapped != inode->i_mapping->nrpages))

@@ -1522,8 +1523,9 @@ static int shmem_getattr(struct mnt_idmap *idmap,
 			STATX_ATTR_NODUMP);

Currently, we have two helpers that check for PMD-sized pages but have different names and slightly different semantics:

- has_transparent_hugepage(): the name suggests it checks if THP is enabled, but when CONFIG_TRANSPARENT_HUGEPAGE=y and the architecture implements this helper, it actually checks if the CPU supports PMD-sized pages

- thp_disabled_by_hw(): the name suggests it checks if THP is disabled by the hardware, but it just returns a cached value acquired with has_transparent_hugepage() during boot. This helper is used in fast paths

A better design would be to separate CONFIG_TRANSPARENT_HUGEPAGE checking, which can be done with the IS_ENABLED() macro, from checking if a CPU supports PMD-sized pages.

To this end, this commit introduces a new helper called pgtable_has_pmd_leaves() which offers the following advantages:

1. Well defined and clear semantics: it returns true if the CPU supports PMD-sized pages and false otherwise
2. It always returns a cached value, so it can be used in fast paths
3. It's implemented with a static key: it's a no-op for archs not implemeting it and for archs implementing it the static key is changed only during boot

The new helper requires an initialization step which is performed by pgtable_leaf_support_init(). We call pgtable_leaf_support_init() early during boot from mm_core_init().

The next commits will convert users of both has_transparent_hugepage() and thp_disabled_by_hw() to pgtable_has_pmd_leaves() and/or IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE).

Signed-off-by Luiz Capitulino <[email protected]>

---
 include/linux/pgtable.h | 17 +++++++++++++++++
 mm/memory.c             |  9 +++++++++
 mm/mm_init.c            |  1 +
 3 files changed, 27 insertions(+)

Currently, we have two helpers that check for PMD-sized pages but have different names and slightly different semantics:

- has_transparent_hugepage(): the name suggests it checks if THP is enabled, but when CONFIG_TRANSPARENT_HUGEPAGE=y and the architecture implements this helper, it actually checks if the CPU supports PMD-sized pages

- thp_disabled_by_hw(): the name suggests it checks if THP is disabled by the hardware, but it just returns a cached value acquired with has_transparent_hugepage() during boot. This helper is used in fast paths

A better design would be to separate CONFIG_TRANSPARENT_HUGEPAGE checking, which can be done with the IS_ENABLED() macro, from checking if a CPU supports PMD-sized pages.

To this end, this commit introduces a new helper called pgtable_has_pmd_leaves() which offers the following advantages:

1. Well defined and clear semantics: it returns true if the CPU supports PMD-sized pages and false otherwise
2. It always returns a cached value, so it can be used in fast paths
3. It's implemented with a static key: it's a no-op for archs not implemeting it and for archs implementing it the static key is changed only during boot

The new helper requires an initialization step which is performed by pgtable_leaf_support_init(). We call pgtable_leaf_support_init() early during boot from mm_core_init().

The next commits will convert users of both has_transparent_hugepage() and thp_disabled_by_hw() to pgtable_has_pmd_leaves() and/or IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE).

Signed-off-by Luiz Capitulino <[email protected]>

---

dax_align_valid() uses has_transparent_hugepage() to check if PMD-sized pages are supported, use pgtable_has_pmd_leaves() instead.

Acked-by David Hildenbrand (Red Hat) <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
 drivers/dax/dax-private.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

dax_align_valid() uses has_transparent_hugepage() to check if PMD-sized pages are supported, use pgtable_has_pmd_leaves() instead.

Acked-by David Hildenbrand (Red Hat) <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
 drivers/dax/dax-private.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

drivers/dax/dax-private.h +0 -0

index 607a53a91f58..18a6295aa766 100644

Quoted 14 lines · Luiz Capitulino

dax_align_valid() uses has_transparent_hugepage() to check if PMD-sized pages are supported, use pgtable_has_pmd_leaves() instead.

Acked-by David Hildenbrand (Red Hat) <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
  drivers/dax/dax-private.h | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

drivers/dax/dax-private.h +0 -0

index 607a53a91f58..18a6295aa766 100644

drivers/dax/dax-private.h static inline bool dax_align_valid(unsigned long align) · line 127

Quoted 8 lines

drivers/dax/dax-private.h +1 -1

@@ -121,7 +121,7 @@ static inline bool dax_align_valid(unsigned long align)
  {
  	if (align == PUD_SIZE && IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE_PUD))
  		return true;
-	if (align == PMD_SIZE && has_transparent_hugepage())
+	if (align == PMD_SIZE && pgtable_has_pmd_leaves())
  		return true;
  	if (align == PAGE_SIZE)
  		return true;

The main part why we don't see any change is that dax_align_valid() is wrapped in the #ifdef CONFIG_TRANSPARENT_HUGEPAGE today.

Might we worth spelling that out in the patch description.

nd_pfn_supported_alignments() and nd_pfn_default_alignment() use has_transparent_hugepage() to check if THP is supported with PMD-sized pages. Use pgtable_has_pmd_leaves() instead. Also, check for IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) to preserve the current implementation semantics.

Acked-by Zi Yan <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
 drivers/nvdimm/pfn_devs.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

drivers/nvdimm/pfn_devs.c +0 -0

index 8fa9c16aba7e..457eb54e7ab6 100644

debug_vm_pgtable calls has_transparent_hugepage() in multiple places to check if PMD-sized pages are supported, use pgtable_has_pmd_leaves() instead.

Reviewed-by Lance Yang <[email protected]>

Reviewed-by Zi Yan <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---

Quoted 4 lines · Luiz Capitulino

debug_vm_pgtable calls has_transparent_hugepage() in multiple places to check if PMD-sized pages are supported, use pgtable_has_pmd_leaves() instead.

Quoted 4 lines

Reviewed-by Lance Yang <[email protected]>

Reviewed-by Zi Yan <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---

Best to mention in the description that there is no effective change as all calls to has_transparent_hugepage() are predicated by CONFIG_TRANSPARENT_HUGEPAGE.

Acked-by David Hildenbrand (Arm) <[email protected]>

In order to enable THP support in shmem today, besides the user configuration required, the CPU must support PMD-sized pages. This is the case because of the following has_transparent_hugepage() usage:

- shmem_parse_one() and shmem_parse_huge(): Check if THP is built-in and if the CPU supports PMD-sized pages

- shmem_init(): Since the CONFIG_TRANSPARENT_HUGEPAGE guard is outside the code block calling has_transparent_hugepage(), the has_transparent_hugepage() call is exclusively checking if the CPU supports PMD-sized pages

While it's necessary to check if CONFIG_TRANSPARENT_HUGEPAGE is enabled in all cases, shmem can determine THP size support at folio allocation time. Therefore, drop the has_transparent_hugepage() usage listed above while keeping the CONFIG_TRANSPARENT_HUGEPAGE checks.

Additionally, we need to check if PMD size order is supported in shmem_getattr(). Use pgtable_has_pmd_leaves() for that.

Reviewed-by Baolin Wang <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
 mm/shmem.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

mm/shmem.c +0 -0

index 776dff8a848e..930657d05375 100644

Quoted 32 lines · Luiz Capitulino

In order to enable THP support in shmem today, besides the user configuration required, the CPU must support PMD-sized pages. This is the case because of the following has_transparent_hugepage() usage:

- shmem_parse_one() and shmem_parse_huge(): Check if THP is built-in and if the CPU supports PMD-sized pages

- shmem_init(): Since the CONFIG_TRANSPARENT_HUGEPAGE guard is outside the code block calling has_transparent_hugepage(), the has_transparent_hugepage() call is exclusively checking if the CPU supports PMD-sized pages

While it's necessary to check if CONFIG_TRANSPARENT_HUGEPAGE is enabled in all cases, shmem can determine THP size support at folio allocation time. Therefore, drop the has_transparent_hugepage() usage listed above while keeping the CONFIG_TRANSPARENT_HUGEPAGE checks.

Additionally, we need to check if PMD size order is supported in shmem_getattr(). Use pgtable_has_pmd_leaves() for that.

Reviewed-by Baolin Wang <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
  mm/shmem.c | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)

mm/shmem.c +0 -0

index 776dff8a848e..930657d05375 100644

mm/shmem.c static int shmem_parse_huge(const char *str) · line 695

Quoted 7 lines

mm/shmem.c +1 -1

@@ -690,7 +690,7 @@ static int shmem_parse_huge(const char *str)
  	else
  		return -EINVAL;
 
-	if (!has_transparent_hugepage() &&
+	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
  	    huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY)
  		return -EINVAL;

This can be deleted, since shmem_parse_huge() is guarded by CONFIG_TRANSPARENT_HUGEPAGE and after the change the if is always false under that guard.

Otherwise, LGTM. With the above code deleted, feel free to add

Reviewed-by Zi Yan <[email protected]>

In order to enable THP support in shmem today, besides the user configuration required, the CPU must support PMD-sized pages. This is the case because of the following has_transparent_hugepage() usage:

- shmem_parse_one() and shmem_parse_huge(): Check if THP is built-in and if the CPU supports PMD-sized pages

- shmem_init(): Since the CONFIG_TRANSPARENT_HUGEPAGE guard is outside the code block calling has_transparent_hugepage(), the has_transparent_hugepage() call is exclusively checking if the CPU supports PMD-sized pages

While it's necessary to check if CONFIG_TRANSPARENT_HUGEPAGE is enabled in all cases, shmem can determine THP size support at folio allocation time. Therefore, drop the has_transparent_hugepage() usage listed above while keeping the CONFIG_TRANSPARENT_HUGEPAGE checks.

Additionally, we need to check if PMD size order is supported in shmem_getattr(). Use pgtable_has_pmd_leaves() for that.

Reviewed-by Baolin Wang <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
 mm/shmem.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

mm/shmem.c +1 -1

index 776dff8a848e..930657d05375 100644
@@ -690,7 +690,7 @@ static int shmem_parse_huge(const char *str)
 	else
 		return -EINVAL;
 
-	if (!has_transparent_hugepage() &&
+	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
 	    huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY)
 		return -EINVAL;
 

Quoted 40 lines · Luiz Capitulino

In order to enable THP support in shmem today, besides the user configuration required, the CPU must support PMD-sized pages. This is the case because of the following has_transparent_hugepage() usage:

- shmem_parse_one() and shmem_parse_huge(): Check if THP is built-in and if the CPU supports PMD-sized pages

- shmem_init(): Since the CONFIG_TRANSPARENT_HUGEPAGE guard is outside the code block calling has_transparent_hugepage(), the has_transparent_hugepage() call is exclusively checking if the CPU supports PMD-sized pages

While it's necessary to check if CONFIG_TRANSPARENT_HUGEPAGE is enabled in all cases, shmem can determine THP size support at folio allocation time. Therefore, drop the has_transparent_hugepage() usage listed above while keeping the CONFIG_TRANSPARENT_HUGEPAGE checks.

Additionally, we need to check if PMD size order is supported in shmem_getattr(). Use pgtable_has_pmd_leaves() for that.

Reviewed-by Baolin Wang <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
  mm/shmem.c | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)

mm/shmem.c +1 -1

index 776dff8a848e..930657d05375 100644
@@ -690,7 +690,7 @@ static int shmem_parse_huge(const char *str)
  	else
  		return -EINVAL;
 
-	if (!has_transparent_hugepage() &&
+	if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
  	    huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY)
  		return -EINVAL;
 

mm/shmem.c static int shmem_getattr(struct mnt_idmap *idmap, · line 1526

Quoted 4 lines

mm/shmem.c +0 -0

@@ -1524,6 +1524,8 @@ static int shmem_getattr(struct mnt_idmap *idmap,
  	generic_fillattr(idmap, request_mask, inode, stat);
 
  	orders = shmem_huge_global_enabled(inode, 0, 0, false, NULL, 0);

I'm curious: why is that not handled inside shmem_huge_global_enabled() ? If PMD order is impossible (well, okay, it is possible, but we simply cannot map these things through PMDs), I would expect that we never list them as "enabled".

(while shmem could allocate PMD folios, it would have to map them always through PTEs. I think this would be possible through some changes, but we can defer that to some future work).

Quoted 38 lines · David Hildenbrand (Arm)

Quoted 37 lines · Luiz Capitulino

In order to enable THP support in shmem today, besides the user configuration required, the CPU must support PMD-sized pages. This is the case because of the following has_transparent_hugepage() usage:

- shmem_parse_one() and shmem_parse_huge(): Check if THP is built-in and if the CPU supports PMD-sized pages

- shmem_init(): Since the CONFIG_TRANSPARENT_HUGEPAGE guard is outside the code block calling has_transparent_hugepage(), the has_transparent_hugepage() call is exclusively checking if the CPU supports PMD-sized pages

While it's necessary to check if CONFIG_TRANSPARENT_HUGEPAGE is enabled in all cases, shmem can determine THP size support at folio allocation time. Therefore, drop the has_transparent_hugepage() usage listed above while keeping the CONFIG_TRANSPARENT_HUGEPAGE checks.

Additionally, we need to check if PMD size order is supported in shmem_getattr(). Use pgtable_has_pmd_leaves() for that.

Reviewed-by Baolin Wang <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
  mm/shmem.c | 9 +++++----

  1 file changed, 5 insertions(+), 4 deletions(-)

mm/shmem.c +0 -0

index 776dff8a848e..930657d05375 100644
@@ -690,7 +690,7 @@ static int shmem_parse_huge(const char *str)

      else
          return -EINVAL;
-    if (!has_transparent_hugepage() &&
+    if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY)

Quoted 8 lines

mm/shmem.c static int shmem_getattr(struct mnt_idmap *idmap, · line 1525

Quoted 4 lines

          return -EINVAL;

mm/shmem.c +0 -0

@@ -1524,6 +1524,8 @@ static int shmem_getattr(struct mnt_idmap *idmap,
       generic_fillattr(idmap, request_mask, inode, stat);
       orders = shmem_huge_global_enabled(inode, 0, 0, false, NULL, 0);

I'm curious: why is that not handled inside shmem_huge_global_enabled() ? If PMD order is impossible (well, okay, it is possible, but we simply cannot map these things through PMDs), I would expect that we never list them as "enabled".

Yes, you're right. What about renaming current shmem_huge_global_enabled() to __shmem_huge_global_enabled() and then having:

static unsigned int shmem_huge_global_enabled(struct inode *inode, pgoff_t index,
                          loff_t write_end, bool shmem_huge_force, struct vm_area_struct *vma,
                          vm_flags_t vm_flags)
{
    unsigned int orders;

    orders = __shmem_huge_global_enabled(inode, index, write_end, shmem_huge_force, vma, vm_flags); if (!pgtable_has_pmd_leaves())
        orders &= ~BIT(PMD_ORDER);

    return orders;
}

Would this be acceptable?

Quoted 65 lines · Luiz Capitulino

Quoted 38 lines · David Hildenbrand (Arm)

Quoted 37 lines · Luiz Capitulino

In order to enable THP support in shmem today, besides the user configuration required, the CPU must support PMD-sized pages. This is the case because of the following has_transparent_hugepage() usage:

- shmem_parse_one() and shmem_parse_huge(): Check if THP is built-in and if the CPU supports PMD-sized pages

- shmem_init(): Since the CONFIG_TRANSPARENT_HUGEPAGE guard is outside the code block calling has_transparent_hugepage(), the has_transparent_hugepage() call is exclusively checking if the CPU supports PMD-sized pages

While it's necessary to check if CONFIG_TRANSPARENT_HUGEPAGE is enabled in all cases, shmem can determine THP size support at folio allocation time. Therefore, drop the has_transparent_hugepage() usage listed above while keeping the CONFIG_TRANSPARENT_HUGEPAGE checks.

Additionally, we need to check if PMD size order is supported in shmem_getattr(). Use pgtable_has_pmd_leaves() for that.

Reviewed-by Baolin Wang <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---
  mm/shmem.c | 9 +++++----
  1 file changed, 5 insertions(+), 4 deletions(-)

mm/shmem.c +1 -1

index 776dff8a848e..930657d05375 100644
@@ -690,7 +690,7 @@ static int shmem_parse_huge(const char *str)
      else
          return -EINVAL;
-    if (!has_transparent_hugepage() &&
+    if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) &&
          huge != SHMEM_HUGE_NEVER && huge != SHMEM_HUGE_DENY)

Quoted 8 lines

mm/shmem.c static int shmem_getattr(struct mnt_idmap *idmap, · line 1525

Quoted 4 lines

mm/shmem.c +0 -0

          return -EINVAL;

@@ -1524,6 +1524,8 @@ static int shmem_getattr(struct mnt_idmap *idmap,
      generic_fillattr(idmap, request_mask, inode, stat);
      orders = shmem_huge_global_enabled(inode, 0, 0, false, NULL, 0);

I'm curious: why is that not handled inside shmem_huge_global_enabled() ? If PMD order is impossible (well, okay, it is possible, but we simply cannot map these things through PMDs), I would expect that we never list them as "enabled".

Yes, you're right. What about renaming current shmem_huge_global_enabled() to __shmem_huge_global_enabled() and then having:

static unsigned int shmem_huge_global_enabled(struct inode *inode, pgoff_t index,
loff_t write_end, bool shmem_huge_force, struct vm_area_struct *vma,
vm_flags_t vm_flags)

{
     unsigned int orders;

     orders = __shmem_huge_global_enabled(inode, index, write_end,
                          shmem_huge_force, vma, vm_flags);
     if (!pgtable_has_pmd_leaves())
         orders &= ~BIT(PMD_ORDER);

Quoted 4 lines

     return orders;
}

Would this be acceptable?

Not a fan of re-adding the wrapper. I think you could refer to the changes to shmem_allowable_huge_orders() in patch 14 and filter out the 'disabled_orders' in shmem_huge_global_enabled() directly.

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Move its definition out of the THP guard.

Signed-off-by Luiz Capitulino <[email protected]>

---
 arch/s390/include/asm/pgtable.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Move its definition out of the THP guard.

Signed-off-by Luiz Capitulino <[email protected]>

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Move its definition out of the THP guard.

Signed-off-by Luiz Capitulino <[email protected]>

---
 arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
 arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
 arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 +++++++++---------
 arch/powerpc/include/asm/book3s/64/radix.h    | 14 +++++++-------
 arch/powerpc/mm/book3s64/hash_pgtable.c       |  4 ++--
 5 files changed, 20 insertions(+), 20 deletions(-)

arch/powerpc/include/asm/book3s/64/hash-4k.h +1 -1

index 8e5bd9902bed..79511e6abfca 100644
@@ -165,9 +165,9 @@ extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
 extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
 extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
 				       unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);
 #endif
 
+extern int hash__has_transparent_hugepage(void);
 #endif /* !__ASSEMBLER__ */
 
 #endif /* _ASM_POWERPC_BOOK3S_64_HASH_4K_H */

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -0

index 7deb3a66890b..a4a44a112ff9 100644

arch/powerpc/include/asm/book3s/64/hash-64k.h +4 -3

-static inline int radix__has_transparent_hugepage(void)
+static inline int radix__has_transparent_pud_hugepage(void)
 {
-	/* For radix 2M at PMD level means thp */
-	if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT)
+	/* For radix 1G at PUD level means pud hugepage support */
+	if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT)
 		return 1;
 	return 0;
 }
+#endif
 

Quoted 30 lines · Luiz Capitulino

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Move its definition out of the THP guard.

Signed-off-by Luiz Capitulino <[email protected]>

---
  arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
  arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
  arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 +++++++++---------
  arch/powerpc/include/asm/book3s/64/radix.h    | 14 +++++++-------
  arch/powerpc/mm/book3s64/hash_pgtable.c       |  4 ++--
  5 files changed, 20 insertions(+), 20 deletions(-)

arch/powerpc/include/asm/book3s/64/hash-4k.h +1 -1

index 8e5bd9902bed..79511e6abfca 100644
@@ -165,9 +165,9 @@ extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
  extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
  extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
  				       unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);
  #endif
 
+extern int hash__has_transparent_hugepage(void);
  #endif /* !__ASSEMBLER__ */
 

arch/powerpc/include/asm/book3s/64/hash-64k.h extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, · line 281

Quoted 10 lines

arch/powerpc/include/asm/book3s/64/hash-4k.h +0 -0

  #endif /* _ASM_POWERPC_BOOK3S_64_HASH_4K_H */

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -1

index 7deb3a66890b..a4a44a112ff9 100644
@@ -278,9 +278,9 @@ extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
  extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
  extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
  				       unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);

Without some of these helpers in place, I assume actually using PMD leafs without THP would require some more work. (which is not the goal of this series, just asking).

I do wonder whether the architecture should instead simply say "not supported" if !CONFIG_TRANSPARENT_HUGEPAGE?

That should still enable your series: using mTHP without PMD support.

[...]

Quoted 13 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +4 -3

-static inline int radix__has_transparent_hugepage(void)
+static inline int radix__has_transparent_pud_hugepage(void)
  {
-	/* For radix 2M at PMD level means thp */
-	if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT)
+	/* For radix 1G at PUD level means pud hugepage support */
+	if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT)
  		return 1;
  	return 0;
  }
+#endif

arch/powerpc/include/asm/book3s/64/hash-64k.h

Quoted 10 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +3 -3

-static inline int radix__has_transparent_pud_hugepage(void)
+static inline int radix__has_transparent_hugepage(void)
  {
-	/* For radix 1G at PUD level means pud hugepage support */
-	if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT)
+	/* For radix 2M at PMD level means thp */
+	if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT)
  		return 1;
  	return 0;
  }

You are swapping both implementations, which might create some unnecessary churn I think.

Quoted 37 lines · Luiz Capitulino

Quoted 36 lines

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Move its definition out of the THP guard.

Signed-off-by Luiz Capitulino <[email protected]>

---
  arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +- arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +- arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 +++++++++--------- arch/powerpc/include/asm/book3s/64/radix.h    | 14 +++++++------- arch/powerpc/mm/book3s64/hash_pgtable.c       |  4 ++--

  5 files changed, 20 insertions(+), 20 deletions(-)

arch/powerpc/include/asm/book3s/64/hash-4k.h +1 -1

index 8e5bd9902bed..79511e6abfca 100644
@@ -165,9 +165,9 @@ extern void 
 hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
   extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
   extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
                          unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);
   #endif
+extern int hash__has_transparent_hugepage(void);
   #endif /* !__ASSEMBLER__ */
   #endif /* _ASM_POWERPC_BOOK3S_64_HASH_4K_H */

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -0

index 7deb3a66890b..a4a44a112ff9 100644
@@ -278,9 +278,9 @@ extern void 
 hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,

Quoted 8 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h extern void · line 282

Quoted 4 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -1

   extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
   extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
                          unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);

Without some of these helpers in place, I assume actually using PMD leafs without THP would require some more work. (which is not the goal of this series, just asking).

Yes, you're right.

Quoted 4 lines

I do wonder whether the architecture should instead simply say "not supported" if !CONFIG_TRANSPARENT_HUGEPAGE?

That should still enable your series: using mTHP without PMD support.

Yes, it would. However, I think this introduces an inconsistency.

In in its current form, arch_has_pmd_leaves() should always report a hardware capability. This is true even for the default case where arch_has_pmd_leaves() defaults to IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE) in the assumption that archs supporting PMD-sized pages by default will have this config enabled.

Your suggestion will change this and for some archs arch_has_pmd_leaves() may have different behavior depending on the user configuration. Additionally, I intended to decouple the base API implementation from THP.

If you feel strongly about this I can implement your suggestion, but I'd still vote for keeping the API about consistently reporting the hardware capability. Even if the only user is THP code today, new use cases can be added incrementally.

Quoted 19 lines

[...]

Quoted 16 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +1 -1

-static inline int radix__has_transparent_hugepage(void)
+static inline int radix__has_transparent_pud_hugepage(void)

  {
-    /* For radix 2M at PMD level means thp */
-    if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT) +    /* For radix 1G at PUD level means pud hugepage support */ +    if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT) return 1;
      return 0;
  }

arch/powerpc/include/asm/book3s/64/hash-64k.h +2 -1

+#endif
-static inline int radix__has_transparent_pud_hugepage(void)
+static inline int radix__has_transparent_hugepage(void)

  {
-    /* For radix 1G at PUD level means pud hugepage support */ -    if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT)

Quoted 8 lines

Quoted 5 lines

+    /* For radix 2M at PMD level means thp */ +    if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT) return 1;
      return 0;
  }

You are swapping both implementations, which might create some unnecessary churn I think.

I suspect this was done by git diff as I just moved the functions, but I'll take a better look.

Hi,

Don't forget <[email protected]> when you address powerpc architecture.

Quoted 47 lines · David Hildenbrand (Arm)

Quoted 35 lines · Luiz Capitulino

Quoted 34 lines

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Move its definition out of the THP guard.

Signed-off-by Luiz Capitulino <[email protected]>

---
  arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
  arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
  arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 +++++++++---------
  arch/powerpc/include/asm/book3s/64/radix.h    | 14 +++++++-------
  arch/powerpc/mm/book3s64/hash_pgtable.c       |  4 ++--
  5 files changed, 20 insertions(+), 20 deletions(-)

arch/powerpc/include/asm/book3s/64/hash-4k.h +1 -1

index 8e5bd9902bed..79511e6abfca 100644
@@ -165,9 +165,9 @@ extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
  extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
  extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
                         unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);
  #endif
+extern int hash__has_transparent_hugepage(void);
  #endif /* !__ASSEMBLER__ */
  #endif /* _ASM_POWERPC_BOOK3S_64_HASH_4K_H */

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -0

index 7deb3a66890b..a4a44a112ff9 100644
@@ -278,9 +278,9 @@ extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,

Quoted 8 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h extern void hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp, · line 281

Quoted 4 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -1

  extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
  extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
                         unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);

Without some of these helpers in place, I assume actually using PMD leafs without THP would require some more work. (which is not the goal of this series, just asking).

Quoted 8 lines

Yes, you're right.

Quoted 4 lines

I do wonder whether the architecture should instead simply say "not supported" if !CONFIG_TRANSPARENT_HUGEPAGE?

That should still enable your series: using mTHP without PMD support.

Yes, it would. However, I think this introduces an inconsistency.

powerpc has two types of MMU (HASH and RADIX) with different page layout. MMU type is selected at boottime based on the capabilities of the CPU. For transparent page you have:

static inline int has_transparent_hugepage(void)

{
     if (radix_enabled())
         return radix__has_transparent_hugepage();
     return hash__has_transparent_hugepage();
}

Quoted 50 lines

In in its current form, arch_has_pmd_leaves() should always report a hardware capability. This is true even for the default case where arch_has_pmd_leaves() defaults to IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE) in the assumption that archs supporting PMD-sized pages by default will have this config enabled.

Your suggestion will change this and for some archs arch_has_pmd_leaves() may have different behavior depending on the user configuration. Additionally, I intended to decouple the base API implementation from THP.

If you feel strongly about this I can implement your suggestion, but I'd still vote for keeping the API about consistently reporting the hardware capability. Even if the only user is THP code today, new use cases can be added incrementally.

Quoted 19 lines

[...]

Quoted 16 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +5 -6

-static inline int radix__has_transparent_hugepage(void)
+static inline int radix__has_transparent_pud_hugepage(void)
  {
-    /* For radix 2M at PMD level means thp */
-    if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT)
+    /* For radix 1G at PUD level means pud hugepage support */
+    if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT)
          return 1;
      return 0;
  }
+#endif
-static inline int radix__has_transparent_pud_hugepage(void)
+static inline int radix__has_transparent_hugepage(void)
  {
-    /* For radix 1G at PUD level means pud hugepage support */
-    if (mmu_psize_defs[MMU_PAGE_1G].shift == PUD_SHIFT)

Quoted 8 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h

Quoted 5 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +2 -0

+    /* For radix 2M at PMD level means thp */
+    if (mmu_psize_defs[MMU_PAGE_2M].shift == PMD_SHIFT)
          return 1;
      return 0;
  }

You are swapping both implementations, which might create some unnecessary churn I think.

I suspect this was done by git diff as I just moved the functions, but I'll take a better look.

Quoted 3 lines

Hi,

Don't forget <[email protected]> when you address powerpc architecture.

Will do.

Quoted 61 lines · Luiz Capitulino

Quoted 50 lines · David Hildenbrand (Arm)

Quoted 38 lines · Luiz Capitulino

Quoted 37 lines

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Move its definition out of the THP guard.

Signed-off-by Luiz Capitulino <[email protected]>

---
  arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +- arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +- arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 ++++++++

+---------
  arch/powerpc/include/asm/book3s/64/radix.h    | 14 +++++++-------
  arch/powerpc/mm/book3s64/hash_pgtable.c       |  4 ++--
  5 files changed, 20 insertions(+), 20 deletions(-)

arch/powerpc/include/asm/book3s/64/hash-4k.h +1 -1

index 8e5bd9902bed..79511e6abfca 100644
@@ -165,9 +165,9 @@ extern void 
 hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,
   extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
   extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
                          unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);
   #endif
+extern int hash__has_transparent_hugepage(void);
   #endif /* !__ASSEMBLER__ */
   #endif /* _ASM_POWERPC_BOOK3S_64_HASH_4K_H */

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -0

index 7deb3a66890b..a4a44a112ff9 100644
@@ -278,9 +278,9 @@ extern void 
 hash__pgtable_trans_huge_deposit(struct mm_struct *mm, pmd_t *pmdp,

Quoted 8 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h extern void · line 282

Quoted 4 lines

arch/powerpc/include/asm/book3s/64/hash-64k.h +0 -1

   extern pgtable_t hash__pgtable_trans_huge_withdraw(struct mm_struct *mm, pmd_t *pmdp);
   extern pmd_t hash__pmdp_huge_get_and_clear(struct mm_struct *mm,
                          unsigned long addr, pmd_t *pmdp);
-extern int hash__has_transparent_hugepage(void);

Without some of these helpers in place, I assume actually using PMD leafs without THP would require some more work. (which is not the goal of this series, just asking).

Quoted 8 lines

Yes, you're right.

Quoted 4 lines

I do wonder whether the architecture should instead simply say "not supported" if !CONFIG_TRANSPARENT_HUGEPAGE?

That should still enable your series: using mTHP without PMD support.

Yes, it would. However, I think this introduces an inconsistency.

Quoted 8 lines

powerpc has two types of MMU (HASH and RADIX) with different page layout. MMU type is selected at boottime based on the capabilities of the CPU. For transparent page you have:

static inline int has_transparent_hugepage(void) {
     if (radix_enabled())
         return radix__has_transparent_hugepage(); return hash__has_transparent_hugepage(); }

Right and, unless I'm misunderstanding your comment, this supports my position that the new arch_has_pmd_leaves() API is all about reporting a hardware capability and not tied to THP support.

Quoted 5 lines

Quoted 5 lines

Quoted 5 lines

I do wonder whether the architecture should instead simply say "not supported"
if !CONFIG_TRANSPARENT_HUGEPAGE?

That should still enable your series: using mTHP without PMD support.

A future commit will introduce a kernel API to allow for checking if the CPU supports PMD-sized pages. This API will be based on the has_transparent_hugepage() implementation but will be orthogonal to THP and therefore must work when CONFIG_TRANSPARENT_HUGEPAGE=n.

Now that all the has_transparent_hugepage() callers have been converted to pgtable_has_pmd_leaves(), this commit does two things:

1. Rename has_transparent_hugepage() arch implementations to arch_has_pmd_leaves(), since that's what the helper checks for

2. Introduce the default implementation of arch_has_pmd_leaves() as IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE). This means that if the arch doesn't implement arch_has_pmd_leaves() we default to checking CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE as a way to determine if PMD-sized pages are supported

Note that arch_has_pmd_leaves() is supposed to be called only by pgtable_leaf_support_init(). The remaining exception is hugepage_init() which will be converted in a future commit.

Signed-off-by Luiz Capitulino <[email protected]>

---
 arch/mips/include/asm/pgtable.h               |  4 ++--
 arch/mips/mm/pgtable.c                        |  6 +++---
 arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
 arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
 arch/powerpc/include/asm/book3s/64/pgtable.h  | 10 +++++-----
 arch/powerpc/include/asm/book3s/64/radix.h    |  2 +-
 arch/powerpc/mm/book3s64/hash_pgtable.c       |  4 ++--
 arch/s390/include/asm/pgtable.h               |  4 ++--
 arch/x86/include/asm/pgtable.h                |  4 ++--
 include/linux/pgtable.h                       |  4 ++--
 mm/huge_memory.c                              |  2 +-
 mm/memory.c                                   |  2 +-
 12 files changed, 23 insertions(+), 23 deletions(-)

Despite its name, thp_disabled_by_hw() just checks whether the architecture supports PMD-sized pages. It returns true when TRANSPARENT_HUGEPAGE_UNSUPPORTED is set in transparent_hugepage_flags, this only occurs if the architecture implements arch_has_pmd_leaves() and that function returns false.

Since pgtable_has_pmd_leaves() provides the same semantics, use it instead.

Reviewed-by Lance Yang <[email protected]>

Reviewed-by Baolin Wang <[email protected]>

Acked-by Zi Yan <[email protected]>

Acked-by David Hildenbrand (Arm) <[email protected]>

Signed-off-by Luiz Capitulino <[email protected]>

---

If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP.

This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time.

Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders():

- Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered
- Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU

Signed-off-by Luiz Capitulino <[email protected]>

---

If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP.

This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time.

Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders():

- Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered
- Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU

Signed-off-by Luiz Capitulino <[email protected]>

---
 mm/huge_memory.c | 25 ++++++++++++++++++++-----
 mm/shmem.c       | 14 +++++++++-----
 2 files changed, 29 insertions(+), 10 deletions(-)

mm/huge_memory.c +20 -2

index a06025b87e7c..a2d6de3ea988 100644
@@ -189,6 +189,15 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
 	else
 		supported_orders = THP_ORDERS_ALL_FILE_DEFAULT;
 
+	if (!pgtable_has_pmd_leaves()) {
+		/*
+		 * If the CPU does not support PMD leaves, assume for
+		 * now that it does not support PUD leaves and disable
+		 * both folio orders.
+		 */
+		supported_orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+	}
+
 	orders &= supported_orders;
 	if (!orders)
 		return 0;

@@ -196,7 +205,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
 	if (!vma->vm_mm)		/* vdso */
 		return 0;
 
-	if (!pgtable_has_pmd_leaves() || vma_thp_disabled(vma, vm_flags, forced_collapse))
+	if (vma_thp_disabled(vma, vm_flags, forced_collapse))
 		return 0;
 
 	/* khugepaged doesn't collapse DAX vma, but page fault is fine. */

@@ -979,7 +988,7 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
 	 * disable all other sizes. powerpc's PMD_ORDER isn't a compile-time
 	 * constant so we have to do this here.
 	 */
-	if (!anon_orders_configured)
+	if (!anon_orders_configured && pgtable_has_pmd_leaves())
 		huge_anon_orders_inherit = BIT(PMD_ORDER);
 
 	*hugepage_kobj = kobject_create_and_add("transparent_hugepage", mm_kobj);

@@ -1001,6 +1010,15 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
 	}
 
 	orders = THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_FILE_DEFAULT;
+	if (!pgtable_has_pmd_leaves()) {
+		/*
+		 * If the CPU does not support PMD leaves, assume for
+		 * now that it does not support PUD leaves and disable
+		 * both folio orders.
+		 */
+		orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+	}
+
 	order = highest_order(orders);
 	while (orders) {
 		thpsize = thpsize_create(order, *hugepage_kobj);

mm/huge_memory.c +0 -0

 	/*
 	 * hugepages can't be allocated by the buddy allocator
 	 */

mm/shmem.c +9 -5

index bc2de3a7c1ea..8c0f7e3efeeb 100644
@@ -2046,11 +2046,14 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 	unsigned long mask = READ_ONCE(huge_shmem_orders_always);
 	unsigned long within_size_orders = READ_ONCE(huge_shmem_orders_within_size);
 	vm_flags_t vm_flags = vma ? vma->vm_flags : 0;
-	unsigned int global_orders;
+	unsigned int global_orders, disabled_orders = 0;
 
-	if (!pgtable_has_pmd_leaves() || (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force)))
+	if (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force))
 		return 0;
 
+	if (!pgtable_has_pmd_leaves())
+		disabled_orders = BIT(PMD_ORDER);
+
 	global_orders = shmem_huge_global_enabled(inode, index, write_end,
 						  shmem_huge_force, vma, vm_flags);
 	/*

@@ -2058,7 +2061,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 	 * sysfs configs.
 	 */
 	if (!vma || !vma_is_anon_shmem(vma) || shmem_huge_force)
-		return global_orders;
+		return global_orders & ~disabled_orders;
 
 	/*
 	 * Following the 'deny' semantics of the top level, force the huge

@@ -2072,7 +2075,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 	 * means non-PMD sized THP can not override 'huge' mount option now.
 	 */
 	if (shmem_huge == SHMEM_HUGE_FORCE)
-		return READ_ONCE(huge_shmem_orders_inherit);
+		return READ_ONCE(huge_shmem_orders_inherit) & ~disabled_orders;
 
 	/* Allow mTHP that will be fully within i_size. */
 	mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0);

@@ -2083,6 +2086,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
 	if (global_orders > 0)
 		mask |= READ_ONCE(huge_shmem_orders_inherit);
 
+	mask &= ~disabled_orders;
 	return THP_ORDERS_ALL_FILE_DEFAULT & mask;
 }
 

@@ -5630,7 +5634,7 @@ void __init shmem_init(void)
 	 * Default to setting PMD-sized THP to inherit the global setting and
 	 * disable all other multi-size THPs.
 	 */
-	if (!shmem_orders_configured)
+	if (!shmem_orders_configured && pgtable_has_pmd_leaves())
 		huge_shmem_orders_inherit = BIT(HPAGE_PMD_ORDER);
 #endif
 	return;

Signature · 1 line

2.55.0

Quoted 77 lines · On Thu, 17 Sep 2026 21:45:35 -0400 Luiz Capitulino <[email protected]> wrote:

If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP.

This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time.

Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders():

- Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered
- Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU

Signed-off-by Luiz Capitulino <[email protected]>

---
  mm/huge_memory.c | 25 ++++++++++++++++++++-----
  mm/shmem.c       | 14 +++++++++-----
  2 files changed, 29 insertions(+), 10 deletions(-)

mm/huge_memory.c +20 -2

index a06025b87e7c..a2d6de3ea988 100644
@@ -189,6 +189,15 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
  	else
  		supported_orders = THP_ORDERS_ALL_FILE_DEFAULT;
 
+	if (!pgtable_has_pmd_leaves()) {
+		/*
+		 * If the CPU does not support PMD leaves, assume for
+		 * now that it does not support PUD leaves and disable
+		 * both folio orders.
+		 */
+		supported_orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+	}
+
  	orders &= supported_orders;
  	if (!orders)
  		return 0;

@@ -196,7 +205,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
  	if (!vma->vm_mm)		/* vdso */
  		return 0;
 
-	if (!pgtable_has_pmd_leaves() || vma_thp_disabled(vma, vm_flags, forced_collapse))
+	if (vma_thp_disabled(vma, vm_flags, forced_collapse))
  		return 0;
 
  	/* khugepaged doesn't collapse DAX vma, but page fault is fine. */

@@ -979,7 +988,7 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
  	 * disable all other sizes. powerpc's PMD_ORDER isn't a compile-time
  	 * constant so we have to do this here.
  	 */
-	if (!anon_orders_configured)
+	if (!anon_orders_configured && pgtable_has_pmd_leaves())
  		huge_anon_orders_inherit = BIT(PMD_ORDER);
 
  	*hugepage_kobj = kobject_create_and_add("transparent_hugepage", mm_kobj);

@@ -1001,6 +1010,15 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
  	}
 
  	orders = THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_FILE_DEFAULT;
+	if (!pgtable_has_pmd_leaves()) {
+		/*
+		 * If the CPU does not support PMD leaves, assume for
+		 * now that it does not support PUD leaves and disable
+		 * both folio orders.
+		 */
+		orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+	}
+
  	order = highest_order(orders);
  	while (orders) {
  		thpsize = thpsize_create(order, *hugepage_kobj);

mm/huge_memory.c static int __init hugepage_init(void) · line 1096

Quoted 6 lines

mm/huge_memory.c +0 -3

@@ -1091,9 +1109,6 @@ static int __init hugepage_init(void)
  	int err;
  	struct kobject *hugepage_kobj;
 
-	if (!pgtable_has_pmd_leaves())
-		return -EINVAL;
-

Removing this guard lets start_stop_khugepaged() run on a system without PMD leaves. All mTHP orders default to never, but the global always or madvise flag still makes hugepage_enabled() return true.

That starts an idle khugepaged thread and can unnecessarily raise min_free_kbytes. Could hugepage_enabled() instead test whether an enabled order remains after masking PMD_ORDER when pgtable_has_pmd_leaves() is false?

Quoted 64 lines

mm/huge_memory.c +0 -0

  	/*
  	 * hugepages can't be allocated by the buddy allocator
  	 */

mm/shmem.c +9 -5

index bc2de3a7c1ea..8c0f7e3efeeb 100644
@@ -2046,11 +2046,14 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
  	unsigned long mask = READ_ONCE(huge_shmem_orders_always);
  	unsigned long within_size_orders = READ_ONCE(huge_shmem_orders_within_size);
  	vm_flags_t vm_flags = vma ? vma->vm_flags : 0;
-	unsigned int global_orders;
+	unsigned int global_orders, disabled_orders = 0;
 
-	if (!pgtable_has_pmd_leaves() || (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force)))
+	if (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force))
  		return 0;
 
+	if (!pgtable_has_pmd_leaves())
+		disabled_orders = BIT(PMD_ORDER);
+
  	global_orders = shmem_huge_global_enabled(inode, index, write_end,
  						  shmem_huge_force, vma, vm_flags);
  	/*

@@ -2058,7 +2061,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
  	 * sysfs configs.
  	 */
  	if (!vma || !vma_is_anon_shmem(vma) || shmem_huge_force)
-		return global_orders;
+		return global_orders & ~disabled_orders;
 
  	/*
  	 * Following the 'deny' semantics of the top level, force the huge

@@ -2072,7 +2075,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
  	 * means non-PMD sized THP can not override 'huge' mount option now.
  	 */
  	if (shmem_huge == SHMEM_HUGE_FORCE)
-		return READ_ONCE(huge_shmem_orders_inherit);
+		return READ_ONCE(huge_shmem_orders_inherit) & ~disabled_orders;
 
  	/* Allow mTHP that will be fully within i_size. */
  	mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0);

@@ -2083,6 +2086,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
  	if (global_orders > 0)
  		mask |= READ_ONCE(huge_shmem_orders_inherit);
 
+	mask &= ~disabled_orders;
  	return THP_ORDERS_ALL_FILE_DEFAULT & mask;
  }
 

@@ -5630,7 +5634,7 @@ void __init shmem_init(void)
  	 * Default to setting PMD-sized THP to inherit the global setting and
  	 * disable all other multi-size THPs.
  	 */
-	if (!shmem_orders_configured)
+	if (!shmem_orders_configured && pgtable_has_pmd_leaves())
  		huge_shmem_orders_inherit = BIT(HPAGE_PMD_ORDER);
  #endif
  	return;

Signature · 1 line

2.55.0

If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP.

This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time.

Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders():

- Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered
- Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU

Quoted 8 lines · Luiz Capitulino

If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP.

This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time.

Quoted 7 lines

Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders():

- Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered
- Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU

Nit: "and PUD" can go.

David mentioned

"
There is no need to disable PUD-orders, as shmem does not support PUD-orders (unlike DAX). So you can keep it simpler here.
"

in v4

https://lore.kernel.org/all/[email protected]/

Cheers, Lance

If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP.

This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time.

Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders():

- Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered
- Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU

Signed-off-by Luiz Capitulino <[email protected]>

---
  mm/huge_memory.c | 25 ++++++++++++++++++++-----
  mm/shmem.c       | 14 +++++++++-----
  2 files changed, 29 insertions(+), 10 deletions(-)

mm/huge_memory.c +20 -5

index a06025b87e7c..a2d6de3ea988 100644
@@ -189,6 +189,15 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
  	else
  		supported_orders = THP_ORDERS_ALL_FILE_DEFAULT;
 
+	if (!pgtable_has_pmd_leaves()) {
+		/*
+		 * If the CPU does not support PMD leaves, assume for
+		 * now that it does not support PUD leaves and disable
+		 * both folio orders.
+		 */
+		supported_orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+	}
+
  	orders &= supported_orders;
  	if (!orders)
  		return 0;

@@ -196,7 +205,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
  	if (!vma->vm_mm)		/* vdso */
  		return 0;
 
-	if (!pgtable_has_pmd_leaves() || vma_thp_disabled(vma, vm_flags, forced_collapse))
+	if (vma_thp_disabled(vma, vm_flags, forced_collapse))
  		return 0;
 
  	/* khugepaged doesn't collapse DAX vma, but page fault is fine. */

@@ -979,7 +988,7 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
  	 * disable all other sizes. powerpc's PMD_ORDER isn't a compile-time
  	 * constant so we have to do this here.
  	 */
-	if (!anon_orders_configured)
+	if (!anon_orders_configured && pgtable_has_pmd_leaves())
  		huge_anon_orders_inherit = BIT(PMD_ORDER);
 
  	*hugepage_kobj = kobject_create_and_add("transparent_hugepage", mm_kobj);

@@ -1001,6 +1010,15 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
  	}
 
  	orders = THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_FILE_DEFAULT;
+	if (!pgtable_has_pmd_leaves()) {
+		/*
+		 * If the CPU does not support PMD leaves, assume for
+		 * now that it does not support PUD leaves and disable
+		 * both folio orders.
+		 */
+		orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+	}
+
  	order = highest_order(orders);
  	while (orders) {
  		thpsize = thpsize_create(order, *hugepage_kobj);

@@ -1091,9 +1109,6 @@ static int __init hugepage_init(void)
  	int err;
  	struct kobject *hugepage_kobj;
 
-	if (!pgtable_has_pmd_leaves())
-		return -EINVAL;
-
  	/*
  	 * hugepages can't be allocated by the buddy allocator
  	 */

mm/shmem.c +5 -2

index bc2de3a7c1ea..8c0f7e3efeeb 100644
@@ -2046,11 +2046,14 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
  	unsigned long mask = READ_ONCE(huge_shmem_orders_always);
  	unsigned long within_size_orders = READ_ONCE(huge_shmem_orders_within_size);
  	vm_flags_t vm_flags = vma ? vma->vm_flags : 0;
-	unsigned int global_orders;
+	unsigned int global_orders, disabled_orders = 0;
 
-	if (!pgtable_has_pmd_leaves() || (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force)))
+	if (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force))
  		return 0;
 
+	if (!pgtable_has_pmd_leaves())
+		disabled_orders = BIT(PMD_ORDER);
+
  	global_orders = shmem_huge_global_enabled(inode, index, write_end,
  						  shmem_huge_force, vma, vm_flags);
  	/*

mm/shmem.c +0 -0

  	return THP_ORDERS_ALL_FILE_DEFAULT & mask;
  }

Quoted 100 lines · Luiz Capitulino

If PMD-sized pages are not supported on an architecture (ie. the arch implements arch_has_pmd_leaves() and it returns false) then the current code disables all THP, including mTHP.

This commit fixes this by allowing mTHP to be always enabled for all archs. When PMD-sized pages are not supported, its sysfs entry won't be created and their mapping will be disallowed at page-fault time.

Similarly, this commit implements the following changes for shmem in shmem_allowable_huge_orders():

- Drop the pgtable_has_pmd_leaves() check so that mTHP sizes are considered
- Filter out PMD and PUD orders from allowable orders when PMD-sized pages are not supported by the CPU

Signed-off-by Luiz Capitulino <[email protected]>

---
  mm/huge_memory.c | 25 ++++++++++++++++++++-----
  mm/shmem.c       | 14 +++++++++-----
  2 files changed, 29 insertions(+), 10 deletions(-)

mm/huge_memory.c +20 -5

index a06025b87e7c..a2d6de3ea988 100644
@@ -189,6 +189,15 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
      else
          supported_orders = THP_ORDERS_ALL_FILE_DEFAULT;
+    if (!pgtable_has_pmd_leaves()) {
+        /*
+         * If the CPU does not support PMD leaves, assume for
+         * now that it does not support PUD leaves and disable
+         * both folio orders.
+         */
+        supported_orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+    }
+
      orders &= supported_orders;
      if (!orders)
          return 0;

@@ -196,7 +205,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,
      if (!vma->vm_mm)        /* vdso */
          return 0;
-    if (!pgtable_has_pmd_leaves() || vma_thp_disabled(vma, vm_flags, forced_collapse))
+    if (vma_thp_disabled(vma, vm_flags, forced_collapse))
          return 0;
      /* khugepaged doesn't collapse DAX vma, but page fault is fine. */

@@ -979,7 +988,7 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
       * disable all other sizes. powerpc's PMD_ORDER isn't a compile-time
       * constant so we have to do this here.
       */
-    if (!anon_orders_configured)
+    if (!anon_orders_configured && pgtable_has_pmd_leaves())
          huge_anon_orders_inherit = BIT(PMD_ORDER);
      *hugepage_kobj = kobject_create_and_add("transparent_hugepage", mm_kobj);

@@ -1001,6 +1010,15 @@ static int __init hugepage_init_sysfs(struct kobject **hugepage_kobj)
      }
      orders = THP_ORDERS_ALL_ANON | THP_ORDERS_ALL_FILE_DEFAULT;
+    if (!pgtable_has_pmd_leaves()) {
+        /*
+         * If the CPU does not support PMD leaves, assume for
+         * now that it does not support PUD leaves and disable
+         * both folio orders.
+         */
+        orders &= ~(BIT(PMD_ORDER) | BIT(PUD_ORDER));
+    }
+
      order = highest_order(orders);
      while (orders) {
          thpsize = thpsize_create(order, *hugepage_kobj);

@@ -1091,9 +1109,6 @@ static int __init hugepage_init(void)
      int err;
      struct kobject *hugepage_kobj;
-    if (!pgtable_has_pmd_leaves())
-        return -EINVAL;
-
      /*
       * hugepages can't be allocated by the buddy allocator
       */

mm/shmem.c +5 -2

index bc2de3a7c1ea..8c0f7e3efeeb 100644
@@ -2046,11 +2046,14 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
      unsigned long mask = READ_ONCE(huge_shmem_orders_always);
      unsigned long within_size_orders = READ_ONCE(huge_shmem_orders_within_size);
      vm_flags_t vm_flags = vma ? vma->vm_flags : 0;
-    unsigned int global_orders;
+    unsigned int global_orders, disabled_orders = 0;
-    if (!pgtable_has_pmd_leaves() || (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force)))
+    if (vma && vma_thp_disabled(vma, vm_flags, shmem_huge_force))
          return 0;
+    if (!pgtable_has_pmd_leaves())
+        disabled_orders = BIT(PMD_ORDER);
+
      global_orders = shmem_huge_global_enabled(inode, index, write_end,
                            shmem_huge_force, vma, vm_flags);
      /*

mm/shmem.c unsigned long shmem_allowable_huge_orders(struct inode *inode, · line 2064

Quoted 5 lines

mm/shmem.c +1 -1

@@ -2058,7 +2061,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
       * sysfs configs.
       */
      if (!vma || !vma_is_anon_shmem(vma) || shmem_huge_force)
-        return global_orders;
+        return global_orders & ~disabled_orders;

If you move the 'disabled_orders' filtering logic into shmem_huge_global_enabled(), as I mentioned in patch 8, then this change can be removed.

mm/shmem.c unsigned long shmem_allowable_huge_orders(struct inode *inode, · line 2078

Quoted 8 lines

mm/shmem.c +1 -1

      /*
       * Following the 'deny' semantics of the top level, force the huge

@@ -2072,7 +2075,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
       * means non-PMD sized THP can not override 'huge' mount option now.
       */
      if (shmem_huge == SHMEM_HUGE_FORCE)
-        return READ_ONCE(huge_shmem_orders_inherit);
+        return READ_ONCE(huge_shmem_orders_inherit) & ~disabled_orders;

Since you've already disabled PMD-sized and PUD-sized orders for the 'shmem_enabled' interface in hugepage_init_sysfs(), this change can also be dropped.

mm/shmem.c unsigned long shmem_allowable_huge_orders(struct inode *inode, · line 2088

Quoted 6 lines

mm/shmem.c +1 -0

      /* Allow mTHP that will be fully within i_size. */
      mask |= shmem_get_orders_within_size(inode, within_size_orders, index, 0);

@@ -2083,6 +2086,7 @@ unsigned long shmem_allowable_huge_orders(struct inode *inode,
      if (global_orders > 0)
          mask |= READ_ONCE(huge_shmem_orders_inherit);
+    mask &= ~disabled_orders;

Ditto.

Quoted 3 lines

mm/shmem.c +0 -0

      return THP_ORDERS_ALL_FILE_DEFAULT & mask;
  }

Introduction
============

Today, if an architecture implements has_transparent_hugepage() and the CPU lacks support for PMD-sized pages, the THP code disables all THP, including mTHP.

This happens because the has_transparent_hugepage() helper is overloaded:
its name implies it checks whether THP is enabled, but on some architectures it actually checks whether the CPU supports PMD-sized pages. In addition, the THP and shmem code have a big switch on has_transparent_hugepage().

This series solves this by decoupling THP availability checking from querying CPU support for PMD-sized pages. THP availability can be checked with IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE). For querying PMD-sized page support, this series introduces a new helper called pgtable_has_pmd_leaves(), which is independent of THP, has well defined semantics and can be used in fast paths.

Core THP code and shmem can use pgtable_has_pmd_leaves() to determine PMD-sized THP support at page fault time (or folio allocation time for shmem), leaving THP and shmem always enabled for other THP sizes. For architectures and CPUs that do support PMD-sized pages there's no intended change in behavior.

We also convert each user of has_transparent_hugepage(), and the related helper thp_disabled_by_hw(), to IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) and/or pgtable_has_pmd_leaves() according to the call site's check semantics. On s390, powerpc, mips and x86 the arch has_transparent_hugepage() implementation is moved out of the CONFIG_TRANSPARENT_HUGEPAGE guard and renamed to arch_has_pmd_leaves().

Thanks to David Hildenbrand for suggesting this improvement and for providing initial guidance (all bugs and misconceptions are mine).

This applies to mm-new 93f615a22169 ("mm/swap, PM: hibernate: atomically replace hibernation pin").

Reporting availability of PMD-sized pages to user-space
=======================================================

Before this series, not having PMD pages available would shut down THP support meaning that /sys/kernel/mm/transparent_hugepage would not be available. After this series, /sys/kernel/mm/transparent_hugepage and hpage_pmd_size are always available but hugepages-<PMD-size>kB is only available if PMD-sized pages are supported.

To me this seems logical, as hugepages-<size>kB is only available if <size> is supported. If this is correct, then MM kselftests that depend on PMD-sized pages will need to be updated to check for it as they fail with this series applied when PMD-sized pages are not available.

The alternative is changing hpage_pmd_size: either don't create the file when PMD-sized pages are not available or set hpage_pmd_size=0. My concern is that this could be considered an ABI breakage as this file was never reported to be optional or report a zero value.

Testing
=======

- Ran MM kselftests on x86_64 and s390
- Tested all mTHP sizes allocation on x86_64 (with and without PMD-sized pages available)
- Tested shmem with within_size w/ mTHP on x86_64 (with and without PMD-sized pages available)
- Performed defconfig build on x86_64, s390, arm64, powerpc, and mips

NOTES:
* I'm forcing arch_has_pmd_leaves() off on x86 to simulate not having PMD-sized pages available

* Running the MM kselftests when PMD pages are not available causes some tests that depend on PMD-sized THP pages to fail as noted earlier

Changelog
=========

v8
--
- Fixed build breakage on MIPS (Lance)
- Rebased on top of mm-new (fixed conflicts in include/linux/pgtable.h and mm/shmem.c - dropped a Reviewed-by as a result)

v7
--
- Applied on top of latest mm-new
- Improved various changelogs including cover-letter
- Changed arch_has_pmd_leaves() default implementation to use IS_ENABLED() instead of IS_BUILTIN()

v6
--
- Rebased on top of mm-unstable (required a minor conflict resolution)
- Added more Reviewed-by and Acked-by tags

v5
--
- Moved init_arch_has_pmd_leaves() to mm_core_init() (David)
- Renamed init_arch_has_pmd_leaves() to pgtable_leaf_support_init() (Lance)
- Added new patch changing shmem_getattr() to set blksize according to highest supported THP order (Baolin)
- Added new patches to move has_transparent_hugepage() out of CONFIG_TRANSPARENT_HUGEPAGE guards
- shmem_allowable_huge_orders(): rename variable and only disable PMD_ORDER (David)
- Added <linux/jmp_label.h> to include/linux/pgtable.h
- Added new tags and removed Reviewed-by from changed patches

v4
--
- Used static key for pgtable_has_pmd_leaves() API (Lance)
- Moved shmem pgtable_has_pmd_leaves() check to shmem_allowable_huge_orders() (Baolin)
- Default pgtable_has_pmd_leaves() implementation to IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE) (Zi)
- Dropped patch “mm: thp: x86: cleanup PSE feature bit usage” (Dave)

v3
--
- Rebased on top of latest Linus tree
- Removed i915 patch as driver dropped has_transparent_hugepage() usage
- Moved init_arch_has_pmd_leaves() call in start_kernel() to avoid conflict with early_param handlers clearing CPU feature flags
- Fixed build error with CONFIG_MMU=n (kernel test robot)
- Fixed huge_anon_orders_inherit default setting when !pgtable_has_pmd_leaves() (Baolin)
- Small commit changelog improvements

v2
--
- Added support for always enabling mTHPs for shmem (Baolin)
- Improved commits changelog & added reviewed-by

v1
--
- Call init_arch_has_pmd_leaves() from start_kernel()
- Keep pgtable_has_pmd_leaves() calls tied to CONFIG_TRANSPARENT_HUGEPAGE (David)
- Clear PUD_ORDER when clearing PMD_ORDER (David)
- Small changelog improvements (David)
- Rebased on top of latest mm-new

Luiz Capitulino (14):
docs: tmpfs: remove implementation detail reference
mm: shmem: shmem_getattr(): set blksize to highest supported THP order
mm: introduce pgtable_has_pmd_leaves()
drivers: dax: use pgtable_has_pmd_leaves()
drivers: nvdimm: use pgtable_has_pmd_leaves()
mm: debug_vm_pgtable: use pgtable_has_pmd_leaves()
mm: shmem: allow THP support determination at folio allocation time
s390: move has_transparent_hugepage() out of THP guard
powerpc: move has_transparent_hugepage() out of THP guard
mips: move has_transparent_hugepage() out of THP guard
x86: move has_transparent_hugepage() out of THP guard
treewide: introduce arch_has_pmd_leaves()
mm: replace thp_disabled_by_hw() with pgtable_has_pmd_leaves()
mm: thp: always enable mTHP support

 Documentation/filesystems/tmpfs.rst           |  5 ++--
 arch/mips/include/asm/pgtable.h               |  6 ++--
 arch/mips/mm/pgtable.c                        | 25 ++++++++++++++++
 arch/mips/mm/tlb-r4k.c                        | 22 --------------
 arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
 arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
 arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 ++++++------
 arch/powerpc/include/asm/book3s/64/radix.h    | 14 ++++-----
 arch/powerpc/mm/book3s64/hash_pgtable.c       |  8 ++---
 arch/s390/include/asm/pgtable.h               |  6 ++--
 arch/x86/include/asm/pgtable.h                | 12 ++++----
 drivers/dax/dax-private.h                     |  2 +-
 drivers/nvdimm/pfn_devs.c                     |  6 ++--
 include/linux/huge_mm.h                       |  7 -----
 include/linux/pgtable.h                       | 21 ++++++++++++--
 mm/debug_vm_pgtable.c                         | 20 ++++++-------
 mm/huge_memory.c                              | 27 ++++++++++++-----
 mm/memory.c                                   | 11 ++++++-
 mm/mm_init.c                                  |  1 +
 mm/shmem.c                                    | 29 ++++++++++++-------
 20 files changed, 144 insertions(+), 100 deletions(-)

Quoted 6 lines · Luiz Capitulino

Introduction
============

Today, if an architecture implements has_transparent_hugepage() and the CPU lacks support for PMD-sized pages, the THP code disables all THP, including mTHP.

Hi Luiz,

Sorry for asking this so late in the series, but which CPUs lack support for PMD sized pages?

Is this series mainly for PowerPC?

Thanks,
Usama

Quoted 172 lines

This happens because the has_transparent_hugepage() helper is overloaded:
its name implies it checks whether THP is enabled, but on some architectures it actually checks whether the CPU supports PMD-sized pages. In addition, the THP and shmem code have a big switch on has_transparent_hugepage().

This series solves this by decoupling THP availability checking from querying CPU support for PMD-sized pages. THP availability can be checked with IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE). For querying PMD-sized page support, this series introduces a new helper called pgtable_has_pmd_leaves(), which is independent of THP, has well defined semantics and can be used in fast paths.

Core THP code and shmem can use pgtable_has_pmd_leaves() to determine PMD-sized THP support at page fault time (or folio allocation time for shmem), leaving THP and shmem always enabled for other THP sizes. For architectures and CPUs that do support PMD-sized pages there's no intended change in behavior.

We also convert each user of has_transparent_hugepage(), and the related helper thp_disabled_by_hw(), to IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) and/or pgtable_has_pmd_leaves() according to the call site's check semantics. On s390, powerpc, mips and x86 the arch has_transparent_hugepage() implementation is moved out of the CONFIG_TRANSPARENT_HUGEPAGE guard and renamed to arch_has_pmd_leaves().

Thanks to David Hildenbrand for suggesting this improvement and for providing initial guidance (all bugs and misconceptions are mine).

This applies to mm-new 93f615a22169 ("mm/swap, PM: hibernate: atomically replace hibernation pin").

Reporting availability of PMD-sized pages to user-space
=======================================================

Before this series, not having PMD pages available would shut down THP support meaning that /sys/kernel/mm/transparent_hugepage would not be available. After this series, /sys/kernel/mm/transparent_hugepage and hpage_pmd_size are always available but hugepages-<PMD-size>kB is only available if PMD-sized pages are supported.

To me this seems logical, as hugepages-<size>kB is only available if <size> is supported. If this is correct, then MM kselftests that depend on PMD-sized pages will need to be updated to check for it as they fail with this series applied when PMD-sized pages are not available.

The alternative is changing hpage_pmd_size: either don't create the file when PMD-sized pages are not available or set hpage_pmd_size=0. My concern is that this could be considered an ABI breakage as this file was never reported to be optional or report a zero value.

Testing
=======

- Ran MM kselftests on x86_64 and s390
- Tested all mTHP sizes allocation on x86_64 (with and without PMD-sized pages available)
- Tested shmem with within_size w/ mTHP on x86_64 (with and without PMD-sized pages available)
- Performed defconfig build on x86_64, s390, arm64, powerpc, and mips

NOTES:
* I'm forcing arch_has_pmd_leaves() off on x86 to simulate not having PMD-sized pages available

* Running the MM kselftests when PMD pages are not available causes some tests that depend on PMD-sized THP pages to fail as noted earlier

Changelog
=========

v8
--
- Fixed build breakage on MIPS (Lance)
- Rebased on top of mm-new (fixed conflicts in include/linux/pgtable.h and mm/shmem.c - dropped a Reviewed-by as a result)

v7
--
- Applied on top of latest mm-new
- Improved various changelogs including cover-letter
- Changed arch_has_pmd_leaves() default implementation to use IS_ENABLED() instead of IS_BUILTIN()

v6
--
- Rebased on top of mm-unstable (required a minor conflict resolution)
- Added more Reviewed-by and Acked-by tags

v5
--
- Moved init_arch_has_pmd_leaves() to mm_core_init() (David)
- Renamed init_arch_has_pmd_leaves() to pgtable_leaf_support_init() (Lance)
- Added new patch changing shmem_getattr() to set blksize according to highest supported THP order (Baolin)
- Added new patches to move has_transparent_hugepage() out of CONFIG_TRANSPARENT_HUGEPAGE guards
- shmem_allowable_huge_orders(): rename variable and only disable PMD_ORDER (David)
- Added <linux/jmp_label.h> to include/linux/pgtable.h
- Added new tags and removed Reviewed-by from changed patches

v4
--
- Used static key for pgtable_has_pmd_leaves() API (Lance)
- Moved shmem pgtable_has_pmd_leaves() check to shmem_allowable_huge_orders() (Baolin)
- Default pgtable_has_pmd_leaves() implementation to IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE) (Zi)
- Dropped patch “mm: thp: x86: cleanup PSE feature bit usage” (Dave)

v3
--
- Rebased on top of latest Linus tree
- Removed i915 patch as driver dropped has_transparent_hugepage() usage
- Moved init_arch_has_pmd_leaves() call in start_kernel() to avoid conflict with early_param handlers clearing CPU feature flags
- Fixed build error with CONFIG_MMU=n (kernel test robot)
- Fixed huge_anon_orders_inherit default setting when !pgtable_has_pmd_leaves() (Baolin)
- Small commit changelog improvements

v2
--
- Added support for always enabling mTHPs for shmem (Baolin)
- Improved commits changelog & added reviewed-by

v1
--
- Call init_arch_has_pmd_leaves() from start_kernel()
- Keep pgtable_has_pmd_leaves() calls tied to CONFIG_TRANSPARENT_HUGEPAGE (David)
- Clear PUD_ORDER when clearing PMD_ORDER (David)
- Small changelog improvements (David)
- Rebased on top of latest mm-new

Luiz Capitulino (14):
docs: tmpfs: remove implementation detail reference
mm: shmem: shmem_getattr(): set blksize to highest supported THP order
mm: introduce pgtable_has_pmd_leaves()
drivers: dax: use pgtable_has_pmd_leaves()
drivers: nvdimm: use pgtable_has_pmd_leaves()
mm: debug_vm_pgtable: use pgtable_has_pmd_leaves()
mm: shmem: allow THP support determination at folio allocation time
s390: move has_transparent_hugepage() out of THP guard
powerpc: move has_transparent_hugepage() out of THP guard
mips: move has_transparent_hugepage() out of THP guard
x86: move has_transparent_hugepage() out of THP guard
treewide: introduce arch_has_pmd_leaves()
mm: replace thp_disabled_by_hw() with pgtable_has_pmd_leaves()
mm: thp: always enable mTHP support

  Documentation/filesystems/tmpfs.rst           |  5 ++--
  arch/mips/include/asm/pgtable.h               |  6 ++--
  arch/mips/mm/pgtable.c                        | 25 ++++++++++++++++
  arch/mips/mm/tlb-r4k.c                        | 22 --------------
  arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
  arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
  arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 ++++++------
  arch/powerpc/include/asm/book3s/64/radix.h    | 14 ++++-----
  arch/powerpc/mm/book3s64/hash_pgtable.c       |  8 ++---
  arch/s390/include/asm/pgtable.h               |  6 ++--
  arch/x86/include/asm/pgtable.h                | 12 ++++----
  drivers/dax/dax-private.h                     |  2 +-
  drivers/nvdimm/pfn_devs.c                     |  6 ++--
  include/linux/huge_mm.h                       |  7 -----
  include/linux/pgtable.h                       | 21 ++++++++++++--
  mm/debug_vm_pgtable.c                         | 20 ++++++-------
  mm/huge_memory.c                              | 27 ++++++++++++-----
  mm/memory.c                                   | 11 ++++++-
  mm/mm_init.c                                  |  1 +
  mm/shmem.c                                    | 29 ++++++++++++-------
  20 files changed, 144 insertions(+), 100 deletions(-)

Quoted 11 lines · Usama Arif

Quoted 6 lines · Luiz Capitulino

Introduction
============

Today, if an architecture implements has_transparent_hugepage() and the CPU lacks support for PMD-sized pages, the THP code disables all THP, including mTHP.

Quoted 6 lines

Hi Luiz,

Sorry for asking this so late in the series, but which CPUs lack support for PMD sized pages?

Is this series mainly for PowerPC?

No :)

The architectures that have conditional PMD page support (ie. discovered at run-time via has_transparent_hugepage()) are x86, s390, powerpc and mips.

For x86 for example, even on 64-bit there are cases where the PSE bit may not be present (eg. some errata and by hypervisor CPUID masking).

That being said, the main goal of the series is that has_transparent_hugepage() is overloaded: it has different semantics on those architectures and is used to gate all THP support (which is not the right thing to do for mTHP).

Quoted 11 lines · Usama Arif

Quoted 6 lines · Luiz Capitulino

Introduction
============

Today, if an architecture implements has_transparent_hugepage() and the CPU lacks support for PMD-sized pages, the THP code disables all THP, including mTHP.

Quoted 6 lines

Hi Luiz,

Sorry for asking this so late in the series, but which CPUs lack support for PMD sized pages?

Is this series mainly for PowerPC?

No :)

The architectures that have conditional PMD page support (ie. discovered at run-time via has_transparent_hugepage()) are x86, s390, powerpc and mips.

For x86 for example, even on 64-bit there are cases where the PSE bit may not be present (eg. some errata and by hypervisor CPUID masking).

That being said, the main goal of the series is that has_transparent_hugepage() is overloaded: it has different semantics on those architectures and is used to gate all THP support (which is not the right thing to do for mTHP).

Quoted 178 lines

Thanks,
Usama

Quoted 172 lines

This happens because the has_transparent_hugepage() helper is overloaded:
its name implies it checks whether THP is enabled, but on some architectures it actually checks whether the CPU supports PMD-sized pages. In addition, the THP and shmem code have a big switch on has_transparent_hugepage().

This series solves this by decoupling THP availability checking from querying CPU support for PMD-sized pages. THP availability can be checked with IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE). For querying PMD-sized page support, this series introduces a new helper called pgtable_has_pmd_leaves(), which is independent of THP, has well defined semantics and can be used in fast paths.

Core THP code and shmem can use pgtable_has_pmd_leaves() to determine PMD-sized THP support at page fault time (or folio allocation time for shmem), leaving THP and shmem always enabled for other THP sizes. For architectures and CPUs that do support PMD-sized pages there's no intended change in behavior.

We also convert each user of has_transparent_hugepage(), and the related helper thp_disabled_by_hw(), to IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) and/or pgtable_has_pmd_leaves() according to the call site's check semantics. On s390, powerpc, mips and x86 the arch has_transparent_hugepage() implementation is moved out of the CONFIG_TRANSPARENT_HUGEPAGE guard and renamed to arch_has_pmd_leaves().

Thanks to David Hildenbrand for suggesting this improvement and for providing initial guidance (all bugs and misconceptions are mine).

This applies to mm-new 93f615a22169 ("mm/swap, PM: hibernate: atomically replace hibernation pin").

Reporting availability of PMD-sized pages to user-space
=======================================================

Before this series, not having PMD pages available would shut down THP support meaning that /sys/kernel/mm/transparent_hugepage would not be available. After this series, /sys/kernel/mm/transparent_hugepage and hpage_pmd_size are always available but hugepages-<PMD-size>kB is only available if PMD-sized pages are supported.

To me this seems logical, as hugepages-<size>kB is only available if <size> is supported. If this is correct, then MM kselftests that depend on PMD-sized pages will need to be updated to check for it as they fail with this series applied when PMD-sized pages are not available.

The alternative is changing hpage_pmd_size: either don't create the file when PMD-sized pages are not available or set hpage_pmd_size=0. My concern is that this could be considered an ABI breakage as this file was never reported to be optional or report a zero value.

Testing
=======

- Ran MM kselftests on x86_64 and s390
- Tested all mTHP sizes allocation on x86_64 (with and without PMD-sized pages available)
- Tested shmem with within_size w/ mTHP on x86_64 (with and without PMD-sized pages available)
- Performed defconfig build on x86_64, s390, arm64, powerpc, and mips

NOTES:
* I'm forcing arch_has_pmd_leaves() off on x86 to simulate not having PMD-sized pages available

* Running the MM kselftests when PMD pages are not available causes some tests that depend on PMD-sized THP pages to fail as noted earlier

Changelog
=========

v8
--
- Fixed build breakage on MIPS (Lance)
- Rebased on top of mm-new (fixed conflicts in include/linux/pgtable.h and mm/shmem.c - dropped a Reviewed-by as a result)

v7
--
- Applied on top of latest mm-new
- Improved various changelogs including cover-letter
- Changed arch_has_pmd_leaves() default implementation to use IS_ENABLED() instead of IS_BUILTIN()

v6
--
- Rebased on top of mm-unstable (required a minor conflict resolution)
- Added more Reviewed-by and Acked-by tags

v5
--
- Moved init_arch_has_pmd_leaves() to mm_core_init() (David)
- Renamed init_arch_has_pmd_leaves() to pgtable_leaf_support_init() (Lance)
- Added new patch changing shmem_getattr() to set blksize according to highest supported THP order (Baolin)
- Added new patches to move has_transparent_hugepage() out of CONFIG_TRANSPARENT_HUGEPAGE guards
- shmem_allowable_huge_orders(): rename variable and only disable PMD_ORDER (David)
- Added <linux/jmp_label.h> to include/linux/pgtable.h
- Added new tags and removed Reviewed-by from changed patches

v4
--
- Used static key for pgtable_has_pmd_leaves() API (Lance)
- Moved shmem pgtable_has_pmd_leaves() check to shmem_allowable_huge_orders() (Baolin)
- Default pgtable_has_pmd_leaves() implementation to IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE) (Zi)
- Dropped patch “mm: thp: x86: cleanup PSE feature bit usage” (Dave)

v3
--
- Rebased on top of latest Linus tree
- Removed i915 patch as driver dropped has_transparent_hugepage() usage
- Moved init_arch_has_pmd_leaves() call in start_kernel() to avoid conflict with early_param handlers clearing CPU feature flags
- Fixed build error with CONFIG_MMU=n (kernel test robot)
- Fixed huge_anon_orders_inherit default setting when !pgtable_has_pmd_leaves() (Baolin)
- Small commit changelog improvements

v2
--
- Added support for always enabling mTHPs for shmem (Baolin)
- Improved commits changelog & added reviewed-by

v1
--
- Call init_arch_has_pmd_leaves() from start_kernel()
- Keep pgtable_has_pmd_leaves() calls tied to CONFIG_TRANSPARENT_HUGEPAGE (David)
- Clear PUD_ORDER when clearing PMD_ORDER (David)
- Small changelog improvements (David)
- Rebased on top of latest mm-new

Luiz Capitulino (14):
docs: tmpfs: remove implementation detail reference
mm: shmem: shmem_getattr(): set blksize to highest supported THP order
mm: introduce pgtable_has_pmd_leaves()
drivers: dax: use pgtable_has_pmd_leaves()
drivers: nvdimm: use pgtable_has_pmd_leaves()
mm: debug_vm_pgtable: use pgtable_has_pmd_leaves()
mm: shmem: allow THP support determination at folio allocation time
s390: move has_transparent_hugepage() out of THP guard
powerpc: move has_transparent_hugepage() out of THP guard
mips: move has_transparent_hugepage() out of THP guard
x86: move has_transparent_hugepage() out of THP guard
treewide: introduce arch_has_pmd_leaves()
mm: replace thp_disabled_by_hw() with pgtable_has_pmd_leaves()
mm: thp: always enable mTHP support

  Documentation/filesystems/tmpfs.rst           |  5 ++--
  arch/mips/include/asm/pgtable.h               |  6 ++--
  arch/mips/mm/pgtable.c                        | 25 ++++++++++++++++
  arch/mips/mm/tlb-r4k.c                        | 22 --------------
  arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
  arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
  arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 ++++++------
  arch/powerpc/include/asm/book3s/64/radix.h    | 14 ++++-----
  arch/powerpc/mm/book3s64/hash_pgtable.c       |  8 ++---
  arch/s390/include/asm/pgtable.h               |  6 ++--
  arch/x86/include/asm/pgtable.h                | 12 ++++----
  drivers/dax/dax-private.h                     |  2 +-
  drivers/nvdimm/pfn_devs.c                     |  6 ++--
  include/linux/huge_mm.h                       |  7 -----
  include/linux/pgtable.h                       | 21 ++++++++++++--
  mm/debug_vm_pgtable.c                         | 20 ++++++-------
  mm/huge_memory.c                              | 27 ++++++++++++-----
  mm/memory.c                                   | 11 ++++++-
  mm/mm_init.c                                  |  1 +
  mm/shmem.c                                    | 29 ++++++++++++-------
  20 files changed, 144 insertions(+), 100 deletions(-)

Quoted 27 lines · On Fri, 18 Sep 2026 10:01:35 -0400 Luiz Capitulino <[email protected]> wrote:

Quoted 11 lines · Usama Arif

Quoted 6 lines · Luiz Capitulino

Introduction
============

Today, if an architecture implements has_transparent_hugepage() and the CPU lacks support for PMD-sized pages, the THP code disables all THP, including mTHP.

Quoted 6 lines

Hi Luiz,

Sorry for asking this so late in the series, but which CPUs lack support for PMD sized pages?

Is this series mainly for PowerPC?

No :)

The architectures that have conditional PMD page support (ie. discovered at run-time via has_transparent_hugepage()) are x86, s390, powerpc and mips.

Quoted 7 lines

For x86 for example, even on 64-bit there are cases where the PSE bit may not be present (eg. some errata and by hypervisor CPUID masking).

That being said, the main goal of the series is that has_transparent_hugepage() is overloaded: it has different semantics on those architectures and is used to gate all THP support (which is not the right thing to do for mTHP).

Thanks for clearing that and the patches.

They all look good to me, apart from the the comment in the last patch.

Feel free to add

Acked-by Usama Arif <[email protected]>

to all of them apart from last.

One nit: I would put the above goal of clean up at the start of the coverletter, instead of starting with adding mTHP support for CPUs that lack PMD.

Thanks,
Usama

Quoted 180 lines

Quoted 178 lines

Thanks,
Usama

Quoted 172 lines

This happens because the has_transparent_hugepage() helper is overloaded:
its name implies it checks whether THP is enabled, but on some architectures it actually checks whether the CPU supports PMD-sized pages. In addition, the THP and shmem code have a big switch on has_transparent_hugepage().

This series solves this by decoupling THP availability checking from querying CPU support for PMD-sized pages. THP availability can be checked with IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE). For querying PMD-sized page support, this series introduces a new helper called pgtable_has_pmd_leaves(), which is independent of THP, has well defined semantics and can be used in fast paths.

Core THP code and shmem can use pgtable_has_pmd_leaves() to determine PMD-sized THP support at page fault time (or folio allocation time for shmem), leaving THP and shmem always enabled for other THP sizes. For architectures and CPUs that do support PMD-sized pages there's no intended change in behavior.

We also convert each user of has_transparent_hugepage(), and the related helper thp_disabled_by_hw(), to IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) and/or pgtable_has_pmd_leaves() according to the call site's check semantics. On s390, powerpc, mips and x86 the arch has_transparent_hugepage() implementation is moved out of the CONFIG_TRANSPARENT_HUGEPAGE guard and renamed to arch_has_pmd_leaves().

Thanks to David Hildenbrand for suggesting this improvement and for providing initial guidance (all bugs and misconceptions are mine).

This applies to mm-new 93f615a22169 ("mm/swap, PM: hibernate: atomically replace hibernation pin").

Reporting availability of PMD-sized pages to user-space
=======================================================

Before this series, not having PMD pages available would shut down THP support meaning that /sys/kernel/mm/transparent_hugepage would not be available. After this series, /sys/kernel/mm/transparent_hugepage and hpage_pmd_size are always available but hugepages-<PMD-size>kB is only available if PMD-sized pages are supported.

To me this seems logical, as hugepages-<size>kB is only available if <size> is supported. If this is correct, then MM kselftests that depend on PMD-sized pages will need to be updated to check for it as they fail with this series applied when PMD-sized pages are not available.

The alternative is changing hpage_pmd_size: either don't create the file when PMD-sized pages are not available or set hpage_pmd_size=0. My concern is that this could be considered an ABI breakage as this file was never reported to be optional or report a zero value.

Testing
=======

- Ran MM kselftests on x86_64 and s390
- Tested all mTHP sizes allocation on x86_64 (with and without PMD-sized pages available)
- Tested shmem with within_size w/ mTHP on x86_64 (with and without PMD-sized pages available)
- Performed defconfig build on x86_64, s390, arm64, powerpc, and mips

NOTES:
* I'm forcing arch_has_pmd_leaves() off on x86 to simulate not having PMD-sized pages available

* Running the MM kselftests when PMD pages are not available causes some tests that depend on PMD-sized THP pages to fail as noted earlier

Changelog
=========

v8
--
- Fixed build breakage on MIPS (Lance)
- Rebased on top of mm-new (fixed conflicts in include/linux/pgtable.h and mm/shmem.c - dropped a Reviewed-by as a result)

v7
--
- Applied on top of latest mm-new
- Improved various changelogs including cover-letter
- Changed arch_has_pmd_leaves() default implementation to use IS_ENABLED() instead of IS_BUILTIN()

v6
--
- Rebased on top of mm-unstable (required a minor conflict resolution)
- Added more Reviewed-by and Acked-by tags

v5
--
- Moved init_arch_has_pmd_leaves() to mm_core_init() (David)
- Renamed init_arch_has_pmd_leaves() to pgtable_leaf_support_init() (Lance)
- Added new patch changing shmem_getattr() to set blksize according to highest supported THP order (Baolin)
- Added new patches to move has_transparent_hugepage() out of CONFIG_TRANSPARENT_HUGEPAGE guards
- shmem_allowable_huge_orders(): rename variable and only disable PMD_ORDER (David)
- Added <linux/jmp_label.h> to include/linux/pgtable.h
- Added new tags and removed Reviewed-by from changed patches

v4
--
- Used static key for pgtable_has_pmd_leaves() API (Lance)
- Moved shmem pgtable_has_pmd_leaves() check to shmem_allowable_huge_orders() (Baolin)
- Default pgtable_has_pmd_leaves() implementation to IS_ENABLED(CONFIG_HAVE_ARCH_TRANSPARENT_HUGEPAGE) (Zi)
- Dropped patch “mm: thp: x86: cleanup PSE feature bit usage” (Dave)

v3
--
- Rebased on top of latest Linus tree
- Removed i915 patch as driver dropped has_transparent_hugepage() usage
- Moved init_arch_has_pmd_leaves() call in start_kernel() to avoid conflict with early_param handlers clearing CPU feature flags
- Fixed build error with CONFIG_MMU=n (kernel test robot)
- Fixed huge_anon_orders_inherit default setting when !pgtable_has_pmd_leaves() (Baolin)
- Small commit changelog improvements

v2
--
- Added support for always enabling mTHPs for shmem (Baolin)
- Improved commits changelog & added reviewed-by

v1
--
- Call init_arch_has_pmd_leaves() from start_kernel()
- Keep pgtable_has_pmd_leaves() calls tied to CONFIG_TRANSPARENT_HUGEPAGE (David)
- Clear PUD_ORDER when clearing PMD_ORDER (David)
- Small changelog improvements (David)
- Rebased on top of latest mm-new

Luiz Capitulino (14):
docs: tmpfs: remove implementation detail reference
mm: shmem: shmem_getattr(): set blksize to highest supported THP order
mm: introduce pgtable_has_pmd_leaves()
drivers: dax: use pgtable_has_pmd_leaves()
drivers: nvdimm: use pgtable_has_pmd_leaves()
mm: debug_vm_pgtable: use pgtable_has_pmd_leaves()
mm: shmem: allow THP support determination at folio allocation time
s390: move has_transparent_hugepage() out of THP guard
powerpc: move has_transparent_hugepage() out of THP guard
mips: move has_transparent_hugepage() out of THP guard
x86: move has_transparent_hugepage() out of THP guard
treewide: introduce arch_has_pmd_leaves()
mm: replace thp_disabled_by_hw() with pgtable_has_pmd_leaves()
mm: thp: always enable mTHP support

   Documentation/filesystems/tmpfs.rst           |  5 ++--
   arch/mips/include/asm/pgtable.h               |  6 ++--
   arch/mips/mm/pgtable.c                        | 25 ++++++++++++++++
   arch/mips/mm/tlb-r4k.c                        | 22 --------------
   arch/powerpc/include/asm/book3s/64/hash-4k.h  |  2 +-
   arch/powerpc/include/asm/book3s/64/hash-64k.h |  2 +-
   arch/powerpc/include/asm/book3s/64/pgtable.h  | 18 ++++++------
   arch/powerpc/include/asm/book3s/64/radix.h    | 14 ++++-----
   arch/powerpc/mm/book3s64/hash_pgtable.c       |  8 ++---
   arch/s390/include/asm/pgtable.h               |  6 ++--
   arch/x86/include/asm/pgtable.h                | 12 ++++----
   drivers/dax/dax-private.h                     |  2 +-
   drivers/nvdimm/pfn_devs.c                     |  6 ++--
   include/linux/huge_mm.h                       |  7 -----
   include/linux/pgtable.h                       | 21 ++++++++++++--
   mm/debug_vm_pgtable.c                         | 20 ++++++-------
   mm/huge_memory.c                              | 27 ++++++++++++-----
   mm/memory.c                                   | 11 ++++++-
   mm/mm_init.c                                  |  1 +
   mm/shmem.c                                    | 29 ++++++++++++-------
   20 files changed, 144 insertions(+), 100 deletions(-)