Skip to contentSkip to conversation

mm/huge_memory: fix shrinking of all-zero THPs with max_ptes_none default

PATCH In review David Hildenbrand 7 participants 31 messages Sep 5, 2025 – Sep 15, 2025

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 mm/huge_memory: fix shrinking of all-zero THPs with max_ptes_none default

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

const size_t size = 1024*1024*1024;

int main(void)

 {
         size_t offs;
         char *area;

         area = mmap(0, size, PROT_READ | PROT_WRITE,
                     MAP_ANON | MAP_PRIVATE, -1, 0);
         if (area == MAP_FAILED) {
                 printf("mmap failed\n");
                 exit(-1);
         }
         madvise(area, size, MADV_HUGEPAGE);

         for (offs = 0; offs < size; offs += getpagesize())
                 area[offs] = 0;
         pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
---
 mm/huge_memory.c | 3 ---
 1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -3
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
 	void *kaddr;
 	int i;
 
-	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-		return false;
-
 	for (i = 0; i < folio_nr_pages(folio); i++) {
 		kaddr = kmap_local_folio(folio, i * PAGE_SIZE);
 		if (!memchr_inv(kaddr, 0, PAGE_SIZE)) {
Signature · 1 line
2.50.1
reviewer
#1 lore raw
Quoted 78 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

const size_t size = 1024*1024*1024;

int main(void)

 {
         size_t offs;
         char *area;

         area = mmap(0, size, PROT_READ | PROT_WRITE,
                     MAP_ANON | MAP_PRIVATE, -1, 0);
         if (area == MAP_FAILED) {
                 printf("mmap failed\n");
                 exit(-1);
         }
         madvise(area, size, MADV_HUGEPAGE);

         for (offs = 0; offs < size; offs += getpagesize())
                 area[offs] = 0;
         pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
Quoted 4 lines
---
 mm/huge_memory.c | 3 ---
 1 file changed, 3 deletions(-)

LGTM. Acked-by: Zi Yan <[email protected]>

I also notice that thp_underused() checks num_zero_pages directly against khugepaged_max_ptes_none. This means mTHPs will never be regarded as underused. A similar issue you are discussing in Nico’s khugepaged mTHP support. Maybe checks against these khugepaged_max* variables should be calculated based on nr_pages of a large folio, like making these variables a ratio in other discussion.

Best Regards,
Yan, Zi

#2 lore raw
Quoted 84 lines · Zi Yan
Quoted 78 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

  const size_t size = 1024*1024*1024;

  int main(void)
  {
          size_t offs;
          char *area;
          area = mmap(0, size, PROT_READ | PROT_WRITE,
                      MAP_ANON | MAP_PRIVATE, -1, 0);
          if (area == MAP_FAILED) {
                  printf("mmap failed\n");
                  exit(-1);
          }
          madvise(area, size, MADV_HUGEPAGE);

          for (offs = 0; offs < size; offs += getpagesize())
                  area[offs] = 0;
          pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
Quoted 4 lines
---
  mm/huge_memory.c | 3 ---
  1 file changed, 3 deletions(-)
Quoted 8 lines

LGTM. Acked-by: Zi Yan <[email protected]>

I also notice that thp_underused() checks num_zero_pages directly against khugepaged_max_ptes_none. This means mTHPs will never be regarded as underused. A similar issue you are discussing in Nico’s khugepaged mTHP support. Maybe checks against these khugepaged_max* variables should be calculated based on nr_pages of a large folio, like making these variables a ratio in other discussion.

Yeah, factoring that out and cleaning it up is my next step.

But note that mTHPs are not a candidate for the shrinker right now. (see my explanation in reply to Nicos patch)

Signature · 3 lines
Cheers

David / dhildenb
reviewer
#3 lore raw
Quoted 95 lines · David Hildenbrand
Quoted 84 lines · Zi Yan
Quoted 78 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

  const size_t size = 1024*1024*1024;

  int main(void)
  {
          size_t offs;
          char *area;
          area = mmap(0, size, PROT_READ | PROT_WRITE,
                      MAP_ANON | MAP_PRIVATE, -1, 0);
          if (area == MAP_FAILED) {
                  printf("mmap failed\n");
                  exit(-1);
          }
          madvise(area, size, MADV_HUGEPAGE);

          for (offs = 0; offs < size; offs += getpagesize())
                  area[offs] = 0;
          pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
Quoted 4 lines
---
  mm/huge_memory.c | 3 ---
  1 file changed, 3 deletions(-)
Quoted 8 lines

LGTM. Acked-by: Zi Yan <[email protected]>

I also notice that thp_underused() checks num_zero_pages directly against khugepaged_max_ptes_none. This means mTHPs will never be regarded as underused. A similar issue you are discussing in Nico’s khugepaged mTHP support. Maybe checks against these khugepaged_max* variables should be calculated based on nr_pages of a large folio, like making these variables a ratio in other discussion.

Quoted 4 lines

Yeah, factoring that out and cleaning it up is my next step.

But note that mTHPs are not a candidate for the shrinker right now. (see my explanation in reply to Nicos patch)

Right. IIUC, the logic in deferred_split_scan() is gated by !folio_test_partially_mapped(folio). This creates two paths:

1) Splits for mTHPs that are partially mapped. These set the flag and corectly bypass thp_underused().

2) Optional shrinker splits for new or collapsed THPs. These don't set the flag and must go through thp_underused().

So, thp_underused() is only for whole, PMD-sized THPs right now. Please correct me if my understanding is wrong here ;)

Cheers,
Lance

#4 lore raw
In reply to Zi Yan
Quoted 84 lines · Zi Yan
Quoted 78 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

const size_t size = 1024*1024*1024;

int main(void)

 {
         size_t offs;
         char *area;

         area = mmap(0, size, PROT_READ | PROT_WRITE,
                     MAP_ANON | MAP_PRIVATE, -1, 0);
         if (area == MAP_FAILED) {
                 printf("mmap failed\n");
                 exit(-1);
         }
         madvise(area, size, MADV_HUGEPAGE);

         for (offs = 0; offs < size; offs += getpagesize())
                 area[offs] = 0;
         pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
Quoted 4 lines
---
 mm/huge_memory.c | 3 ---
 1 file changed, 3 deletions(-)
Quoted 8 lines

LGTM. Acked-by: Zi Yan <[email protected]>

I also notice that thp_underused() checks num_zero_pages directly against khugepaged_max_ptes_none. This means mTHPs will never be regarded as underused. A similar issue you are discussing in Nico’s khugepaged mTHP support. Maybe checks against these khugepaged_max* variables should be calculated based on nr_pages of a large folio, like making these variables a ratio in other discussion.

I unfortunately didnt follow the series in the latest revisions.

In the earlier revisions, I think it was decided to not add mTHPs to shrinker as a start, as there are diminshing returns for smaller THPs and having a lot of smaller mTHPs in the deferred list might mean that we get to PMD mapped THPs a lot slower?

Quoted 3 lines

Best Regards,
Yan, Zi

#5 lore raw
Quoted 96 lines · Usama Arif
Quoted 84 lines · Zi Yan
Quoted 78 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

  const size_t size = 1024*1024*1024;

  int main(void)
  {
          size_t offs;
          char *area;
          area = mmap(0, size, PROT_READ | PROT_WRITE,
                      MAP_ANON | MAP_PRIVATE, -1, 0);
          if (area == MAP_FAILED) {
                  printf("mmap failed\n");
                  exit(-1);
          }
          madvise(area, size, MADV_HUGEPAGE);

          for (offs = 0; offs < size; offs += getpagesize())
                  area[offs] = 0;
          pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
Quoted 4 lines
---
  mm/huge_memory.c | 3 ---
  1 file changed, 3 deletions(-)
Quoted 8 lines

LGTM. Acked-by: Zi Yan <[email protected]>

I also notice that thp_underused() checks num_zero_pages directly against khugepaged_max_ptes_none. This means mTHPs will never be regarded as underused. A similar issue you are discussing in Nico’s khugepaged mTHP support. Maybe checks against these khugepaged_max* variables should be calculated based on nr_pages of a large folio, like making these variables a ratio in other discussion.

Quoted 6 lines

I unfortunately didnt follow the series in the latest revisions.

In the earlier revisions, I think it was decided to not add mTHPs to shrinker as a start, as there are diminshing returns for smaller THPs and having a lot of smaller mTHPs in the deferred list might mean that we get to PMD mapped THPs a lot slower?

Probably we would want lists per order etc.

Signature · 3 lines
Cheers

David / dhildenb
#6 lore raw
Quoted 97 lines · David Hildenbrand
Quoted 96 lines · Usama Arif
Quoted 92 lines · Zi Yan
Quoted 78 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
  #include <string.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <unistd.h>
  #include <sys/mman.h>

  const size_t size = 1024*1024*1024;

  int main(void)
  {
          size_t offs;
          char *area;

          area = mmap(0, size, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0); if (area == MAP_FAILED) { printf("mmap failed\n"); exit(-1);
          }
          madvise(area, size, MADV_HUGEPAGE);

          for (offs = 0; offs < size; offs += getpagesize()) area[offs] = 0;
          pause();
  }
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
Quoted 4 lines

---
  mm/huge_memory.c | 3 ---

  1 file changed, 3 deletions(-)

LGTM. Acked-by: Zi Yan <[email protected]>

I also notice that thp_underused() checks num_zero_pages directly against khugepaged_max_ptes_none. This means mTHPs will never be regarded as underused. A similar issue you are discussing in Nico’s khugepaged mTHP support. Maybe checks against these khugepaged_max* variables should be calculated based on nr_pages of a large folio, like making these variables a ratio in other discussion.

Quoted 9 lines
Quoted 6 lines

I unfortunately didnt follow the series in the latest revisions.

In the earlier revisions, I think it was decided to not add mTHPs to shrinker as a start, as there are diminshing returns for smaller THPs and having a lot of smaller mTHPs in the deferred list might mean that we get to PMD mapped THPs a lot slower?

Probably we would want lists per order etc.

Yes that makes sense! and we start with the highest order list.

#7 lore raw
In reply to David Hildenbrand
Quoted 3 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

Yes, that was the main reason.

Quoted 5 lines

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

Agree with this point for this patch.

Quoted 6 lines

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Quoted 9 lines

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 64 lines

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

const size_t size = 1024*1024*1024;

int main(void)

 {
         size_t offs;
         char *area;

         area = mmap(0, size, PROT_READ | PROT_WRITE,
                     MAP_ANON | MAP_PRIVATE, -1, 0);
         if (area == MAP_FAILED) {
                 printf("mmap failed\n");
                 exit(-1);
         }
         madvise(area, size, MADV_HUGEPAGE);

         for (offs = 0; offs < size; offs += getpagesize())
                 area[offs] = 0;
         pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
---
 mm/huge_memory.c | 3 ---
 1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -0
index 26cedfcd74189..aa3ed7a86435b 100644

mm/huge_memory.c static bool thp_underused(struct folio *folio) · line 4115

Quoted 6 lines
mm/huge_memory.c +0 -3
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
 	void *kaddr;
 	int i;
 
-	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-		return false;
-

I do agree with your usecase, but I am really worried about the amount of work and cpu time the THP shrinker will consume when max_ptes_none is 511 (I dont have any numbers to back up my worry :)), and its less likely that we will have these completely zeroed out THPs (again no numbers to back up this statement). We have the huge_zero_folio as well which is installed on read.

Quoted 3 lines
mm/huge_memory.c +0 -0
 	for (i = 0; i < folio_nr_pages(folio); i++) {
 		kaddr = kmap_local_folio(folio, i * PAGE_SIZE);
 		if (!memchr_inv(kaddr, 0, PAGE_SIZE)) {
#8 lore raw

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Quoted 68 lines
Quoted 68 lines

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

  const size_t size = 1024*1024*1024;

  int main(void)
  {
          size_t offs;
          char *area;
          area = mmap(0, size, PROT_READ | PROT_WRITE,
                      MAP_ANON | MAP_PRIVATE, -1, 0);
          if (area == MAP_FAILED) {
                  printf("mmap failed\n");
                  exit(-1);
          }
          madvise(area, size, MADV_HUGEPAGE);

          for (offs = 0; offs < size; offs += getpagesize())
                  area[offs] = 0;
          pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
---
  mm/huge_memory.c | 3 ---
  1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -0
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
  	void *kaddr;
  	int i;
 
Quoted 9 lines

mm/huge_memory.c static bool thp_underused(struct folio *folio) · line 4115

Quoted 3 lines
mm/huge_memory.c +0 -3
-	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-		return false;
-

I do agree with your usecase, but I am really worried about the amount of work and cpu time the THP shrinker will consume when max_ptes_none is 511 (I dont have any numbers to back up my worry :)), and its less likely that we will have these completely zeroed out THPs (again no numbers to back up this statement).

Then then shrinker shall be deactivated as default if that becomes a problem.

Fortunately you documented the desired semantics:

"All THPs at fault and collapse time will be added to _deferred_list, and will therefore be split under memory pressure if they are considered "underused". A THP is underused if the number of zero-filled pages in the THP is above max_ptes_none (see below)."

Quoted 1 line

We have the huge_zero_folio as well which is installed on read.

Yes, only if the huge zero folio is not available. Which will then also get properly reclaimed.

Signature · 3 lines
Cheers

David / dhildenb
#9 lore raw
Quoted 10 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

Quoted 90 lines
Quoted 60 lines
Quoted 60 lines

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
  #include <string.h>
  #include <stdio.h>
  #include <stdlib.h>
  #include <unistd.h>
  #include <sys/mman.h>

  const size_t size = 1024*1024*1024;

  int main(void)
  {
          size_t offs;
          char *area;

          area = mmap(0, size, PROT_READ | PROT_WRITE, MAP_ANON | MAP_PRIVATE, -1, 0); if (area == MAP_FAILED) { printf("mmap failed\n"); exit(-1);
          }
          madvise(area, size, MADV_HUGEPAGE);

          for (offs = 0; offs < size; offs += getpagesize()) area[offs] = 0;
          pause();
  }
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>

---
  mm/huge_memory.c | 3 ---

  1 file changed, 3 deletions(-)
Quoted 16 lines

mm/huge_memory.c static bool thp_underused(struct folio *folio) · line 4114

Quoted 10 lines
mm/huge_memory.c +0 -2
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
       void *kaddr;
       int i;
   -    if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-        return false;
-

I do agree with your usecase, but I am really worried about the amount of work and cpu time the THP shrinker will consume when max_ptes_none is 511 (I dont have any numbers to back up my worry :)), and its less likely that we will have these completely zeroed out THPs (again no numbers to back up this statement).

Then then shrinker shall be deactivated as default if that becomes a problem.

Fortunately you documented the desired semantics:

"All THPs at fault and collapse time will be added to _deferred_list, and will therefore be split under memory pressure if they are considered "underused". A THP is underused if the number of zero-filled pages in the THP is above max_ptes_none (see below)."

Quoted 1 line

We have the huge_zero_folio as well which is installed on read.

Yes, only if the huge zero folio is not available. Which will then also get properly reclaimed.

#10 lore raw
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

Signature · 3 lines
Cheers

David / dhildenb
#11 lore raw
Quoted 20 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

Quoted 8 lines

I don't really see why we should do that.

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

#12 lore raw
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

Signature · 3 lines
Cheers

David / dhildenb
#13 lore raw
Quoted 33 lines · David Hildenbrand
Quoted 32 lines · Usama Arif
Quoted 28 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

Quoted 8 lines
Quoted 1 line

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

In deferred_split_folio, if split_underused_thp is false, we dont add them to the list (unless partially_mapped).

Unless you are referring to non pmd mapped THPs?

#14 lore raw
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)

Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

mm/huge_memory: always add THPs to the deferred split list

When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.

That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it.

So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.

This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +2 -3
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
        if (folio_order(folio) <= 1)
                return;
 
-       if (!partially_mapped && !split_underused_thp)
-               return;
-
        /*
         * Exclude swapcache: originally to avoid a corrupt deferred split
         * queue. Nowadays that is fully prevented by memcg1_swapout();
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                bool underused = false;
 
                if (!folio_test_partially_mapped(folio)) {
+                       if (!split_underused_thp)
+                               goto next;
                        underused = thp_underused(folio);
                        if (!underused)
                                goto next;
Signature · 3 lines
Cheers

David / dhildenb
#15 lore raw
Quoted 83 lines · David Hildenbrand
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)
Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

    mm/huge_memory: always add THPs to the deferred split list When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.
        That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it. So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.
        This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +0 -0
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
         if (folio_order(folio) <= 1)
                 return;


-       if (!partially_mapped && !split_underused_thp)
-               return;
-
        /*
         * Exclude swapcache: originally to avoid a corrupt deferred split
         * queue. Nowadays that is fully prevented by memcg1_swapout();

mm/huge_memory.c +0 -0
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                 bool underused = false;
Quoted 9 lines


                if (!folio_test_partially_mapped(folio)) { +                       if (!split_underused_thp) +                               goto next; underused = thp_underused(folio); if (!underused)
                                goto next;

Thanks for sending the diff! Now I know what you meant lol.

In the case of when shrinker is disabled, this could make the deferred split scan for partially mapped folios very ineffective?

I am making up numbers, but lets there are 128 THPs in the system, only 2 of them are partially mapped and sc->nr_to_scan is 32.

In the current code, with shrinker disabled, only the 2 partially mapped THPs will be on the deferred list, so we will reclaim them in the first go.

With your patch, the worst case scenario is that the partially mapped THPs are at the end of the deferred_list and we would need 4 calls for the shrinker to split them.

#16 lore raw
Quoted 102 lines · Usama Arif
Quoted 83 lines · David Hildenbrand
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)
Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

    mm/huge_memory: always add THPs to the deferred split list When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.
        That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it. So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.
        This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +0 -0
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
         if (folio_order(folio) <= 1)
                 return;


-       if (!partially_mapped && !split_underused_thp)
-               return;
-
        /*
         * Exclude swapcache: originally to avoid a corrupt deferred split
         * queue. Nowadays that is fully prevented by memcg1_swapout();

mm/huge_memory.c +0 -0
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                 bool underused = false;
Quoted 9 lines


                if (!folio_test_partially_mapped(folio)) { +                       if (!split_underused_thp) +                               goto next; underused = thp_underused(folio); if (!underused)
                                goto next;

Thanks for sending the diff! Now I know what you meant lol.

In the case of when shrinker is disabled, this could make the deferred split scan for partially mapped folios very ineffective?

Quoted 8 lines

I am making up numbers, but lets there are 128 THPs in the system, only 2 of them are partially mapped and sc->nr_to_scan is 32.

In the current code, with shrinker disabled, only the 2 partially mapped THPs will be on the deferred list, so we will reclaim them in the first go.

With your patch, the worst case scenario is that the partially mapped THPs are at the end of the deferred_list and we would need 4 calls for the shrinker to split them.

And I am hoping people are not dynamically enabling/disabling THP shrinker :)

I have ideas about dynamically changing max_ptes_none, maybe based on system metrics like memory pressure, but not enabling/disabling shrinker.

#17 lore raw
In reply to Usama Arif
Quoted 99 lines · Usama Arif
Quoted 84 lines · David Hildenbrand
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)
Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

    mm/huge_memory: always add THPs to the deferred split list When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.
        That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it. So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.
        This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +0 -3
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
         if (folio_order(folio) <= 1)
                 return;
 
-       if (!partially_mapped && !split_underused_thp)
-               return;
-
         /*
          * Exclude swapcache: originally to avoid a corrupt deferred split
          * queue. Nowadays that is fully prevented by memcg1_swapout();

mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4179

Quoted 8 lines
mm/huge_memory.c +2 -0
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                 bool underused = false;
 
                 if (!folio_test_partially_mapped(folio)) {
+                       if (!split_underused_thp)
+                               goto next;
                         underused = thp_underused(folio);
                         if (!underused)
                                 goto next;

Thanks for sending the diff! Now I know what you meant lol.

Quoted 2 lines

In the case of when shrinker is disabled, this could make the deferred split scan for partially mapped folios very ineffective?

I hope you realize that that's the default on each and every system out there that ships this feature :)

And don't ask me how many people even know about disabling the shrinker or would do it, when the default setting is mostly not splitting many THPs ever.

Quoted 9 lines

I am making up numbers, but lets there are 128 THPs in the system, only 2 of them are partially mapped and sc->nr_to_scan is 32.

In the current code, with shrinker disabled, only the 2 partially mapped THPs will be on the deferred list, so we will reclaim them in the first go.

With your patch, the worst case scenario is that the partially mapped THPs are at the end of the deferred_list and we would need 4 calls for the shrinker to split them.

Probably at some point we would want split lists as well, not sure how feasible that is.

Signature · 3 lines
Cheers

David / dhildenb
#18 lore raw
Quoted 98 lines · David Hildenbrand
Quoted 97 lines · Usama Arif
Quoted 82 lines · David Hildenbrand
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)
Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

     mm/huge_memory: always add THPs to the deferred split list When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.
         That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it. So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.
         This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +0 -2
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
          if (folio_order(folio) <= 1)
                  return;
   -       if (!partially_mapped && !split_underused_thp)
-               return;
-
          /*
           * Exclude swapcache: originally to avoid a corrupt deferred split
           * queue. Nowadays that is fully prevented by memcg1_swapout();

mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4179

Quoted 8 lines
mm/huge_memory.c +2 -0
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                  bool underused = false;
                    if (!folio_test_partially_mapped(folio)) {
+                       if (!split_underused_thp)
+                               goto next;
                          underused = thp_underused(folio);
                          if (!underused)
                                  goto next;
 

Thanks for sending the diff! Now I know what you meant lol.

Quoted 5 lines
Quoted 2 lines

In the case of when shrinker is disabled, this could make the deferred split scan for partially mapped folios very ineffective?

I hope you realize that that's the default on each and every system out there that ships this feature :)

Yes, I made it default :)

I am assuming people either keep shrinker enabled (which is an extremely large majority as its default), or disable shrinker and they dont flip flop between the 2 settings.
There are 2 scenarios for the above patch:

- shrinker is enabled (default): the above patch wont make a difference.
- shrinker is disabled: the above patch makes splitting partially mapped folios inefficient.

I didnt talk about the shrinker enabled case as it was a no-op and just talked about the shrinker disabled case.

Quoted 14 lines

And don't ask me how many people even know about disabling the shrinker or would do it, when the default setting is mostly not splitting many THPs ever.

Quoted 9 lines

I am making up numbers, but lets there are 128 THPs in the system, only 2 of them are partially mapped and sc->nr_to_scan is 32.

In the current code, with shrinker disabled, only the 2 partially mapped THPs will be on the deferred list, so we will reclaim them in the first go.

With your patch, the worst case scenario is that the partially mapped THPs are at the end of the deferred_list and we would need 4 calls for the shrinker to split them.

Probably at some point we would want split lists as well, not sure how feasible that is.

#19 lore raw
Quoted 113 lines · Usama Arif
Quoted 98 lines · David Hildenbrand
Quoted 97 lines · Usama Arif
Quoted 82 lines · David Hildenbrand
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)
Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

     mm/huge_memory: always add THPs to the deferred split list When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.
         That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it. So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.
         This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +0 -2
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
          if (folio_order(folio) <= 1)
                  return;
   -       if (!partially_mapped && !split_underused_thp)
-               return;
-
          /*
           * Exclude swapcache: originally to avoid a corrupt deferred split
           * queue. Nowadays that is fully prevented by memcg1_swapout();

mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4179

Quoted 8 lines
mm/huge_memory.c +2 -0
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                  bool underused = false;
                    if (!folio_test_partially_mapped(folio)) {
+                       if (!split_underused_thp)
+                               goto next;
                          underused = thp_underused(folio);
                          if (!underused)
                                  goto next;
 

Thanks for sending the diff! Now I know what you meant lol.

Quoted 5 lines
Quoted 2 lines

In the case of when shrinker is disabled, this could make the deferred split scan for partially mapped folios very ineffective?

I hope you realize that that's the default on each and every system out there that ships this feature :)

Yes, I made it default :)

I am assuming people either keep shrinker enabled (which is an extremely large majority as its default), or disable shrinker and they dont flip flop between the 2 settings.
There are 2 scenarios for the above patch:

Quoted 5 lines

- shrinker is enabled (default): the above patch wont make a difference.
- shrinker is disabled: the above patch makes splitting partially mapped folios inefficient.

I didnt talk about the shrinker enabled case as it was a no-op and just talked about the shrinker disabled case.

Yeah, and I am saying that all you raised as a concern would be a problem already today in all default setups (-> 99.999999%). :)

Probably we should not just disable the shrinker during boot, and once enabled, it would only split THPs created afterwards.

With this patch it would also split ones created previously.

Signature · 3 lines
Cheers

David / dhildenb
#20 lore raw
Quoted 117 lines · David Hildenbrand
Quoted 116 lines · Usama Arif
Quoted 98 lines · David Hildenbrand
Quoted 97 lines · Usama Arif
Quoted 82 lines · David Hildenbrand
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added to deferred split list but *before* memory pressure, i.e. before the shrinker runs, so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)
Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

      mm/huge_memory: always add THPs to the deferred split list When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.
          That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it. So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.
          This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +0 -2
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
           if (folio_order(folio) <= 1)
                   return;
    -       if (!partially_mapped && !split_underused_thp)
-               return;
-
           /*
            * Exclude swapcache: originally to avoid a corrupt deferred split
            * queue. Nowadays that is fully prevented by memcg1_swapout();

mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4179

Quoted 8 lines
mm/huge_memory.c +2 -0
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                   bool underused = false;
                     if (!folio_test_partially_mapped(folio)) {
+                       if (!split_underused_thp)
+                               goto next;
                           underused = thp_underused(folio);
                           if (!underused)
                                   goto next;
 

Thanks for sending the diff! Now I know what you meant lol.

Quoted 5 lines
Quoted 2 lines

In the case of when shrinker is disabled, this could make the deferred split scan for partially mapped folios very ineffective?

I hope you realize that that's the default on each and every system out there that ships this feature :)

Yes, I made it default :)

I am assuming people either keep shrinker enabled (which is an extremely large majority as its default), or disable shrinker and they dont flip flop between the 2 settings.
There are 2 scenarios for the above patch:

- shrinker is enabled (default): the above patch wont make a difference.
- shrinker is disabled: the above patch makes splitting partially mapped folios inefficient.

Quoted 8 lines
Quoted 2 lines

I didnt talk about the shrinker enabled case as it was a no-op and just talked about the shrinker disabled case.

Yeah, and I am saying that all you raised as a concern would be a problem already today in all default setups (-> 99.999999%). :)

Probably we should not just disable the shrinker during boot, and once enabled, it would only split THPs created afterwards.

I probably didnt understand this again lol. Sorry its friday evening :)

split_underused_thp is true at boot time [1]. You are saying we should not disable shrinker during boot, but it is already not disabled during boot, right?

If someone goes with system default, which is THP shrinker enabled (from boot and runtime), the above patch is a no-op, right?

Quoted 2 lines

With this patch it would also split ones created previously.

yes, if someone changes from shrinker being disabled to shrinker being enabled before memory pressure.

[1] https://elixir.bootlin.com/linux/v6.16.4/source/mm/huge_memory.c#L76

#21 lore raw
Quoted 126 lines · Usama Arif
Quoted 123 lines · David Hildenbrand
Quoted 113 lines · Usama Arif
Quoted 98 lines · David Hildenbrand
Quoted 97 lines · Usama Arif
Quoted 82 lines · David Hildenbrand
Quoted 40 lines · Usama Arif
Quoted 37 lines · David Hildenbrand
Quoted 25 lines · Usama Arif
Quoted 22 lines · David Hildenbrand
Quoted 10 lines · Usama Arif
Quoted 7 lines · David Hildenbrand

[...]

Quoted 4 lines

The reason I did this is for the case if you change max_ptes_none after the THP is added
to deferred split list but *before* memory pressure, i.e. before the shrinker runs,
so that its considered for splitting.

Quoted 8 lines
Quoted 3 lines

Yeah, I was assuming that was the reason why the shrinker is enabled as default.

But in any sane system, the admin would enable the shrinker early. If not, we can look into handling it differently.

Yes, I do this as well, i.e. have a low value from the start.

Does it make sense to disable shrinker if max_ptes_none is 511? It wont shrink the usecase you are describing below, but we wont encounter the increased CPU usage.>

I don't really see why we should do that.

Quoted 8 lines
Quoted 6 lines

If the shrinker is a problem than the shrinker should be disabled. But if it is enabled, we should be shrinking as documented.

Without more magic around our THP toggles (we want less) :)

Shrinking happens when we are under memory pressure, so I am not really sure how relevant the scanning bit is, and if it is relevant enought to change the shrinker default.

yes agreed, I also dont have numbers to back up my worry, its all theoretical :)

BTW, I was also wondering if we should just always add all THP to the deferred split list, and make the split toggle just affect whether we process them or not (scan or not).

Quoted 8 lines
Quoted 4 lines

I mean, as a default we add all of them to the list already right now, even though nothing would ever get reclaimed as default.

What's your take?

hmm I probably didnt understand what you meant to say here:
we already add all of them to the list in __do_huge_pmd_anonymous_page and collapse_huge_page and shrink_underused sets/clears split_underused_thp in deferred_split_folio decides whether we process or not.

This is what I mean:

commit 3952b6f6b671ca7d69fd1783b1abf4806f90d436 (HEAD -> max_ptes_none)
Author: David Hildenbrand <[email protected]>
Date:   Fri Sep 5 17:22:01 2025 +0200

      mm/huge_memory: always add THPs to the deferred split list When disabling the shrinker and then re-enabling it, any anon THPs allocated in the meantime.
          That also means that we cannot disable the shrinker as default during boot, because we would miss some THPs later when enabling it. So always add them to the deferred split list, and only skip the scanning if the shrinker is disabled.
          This is effectively what we do on all systems out there already, unless they disable the shrinker.

mm/huge_memory.c +0 -2
index aa3ed7a86435b..3ee857c1d3754 100644
@@ -4052,9 +4052,6 @@ void deferred_split_folio(struct folio *folio, bool partially_mapped)
           if (folio_order(folio) <= 1)
                   return;
    -       if (!partially_mapped && !split_underused_thp)
-               return;
-
           /*
            * Exclude swapcache: originally to avoid a corrupt deferred split
            * queue. Nowadays that is fully prevented by memcg1_swapout();

mm/huge_memory.c static unsigned long deferred_split_scan(struct shrinker *shrink, · line 4179

Quoted 8 lines
mm/huge_memory.c +2 -0
@@ -4175,6 +4172,8 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
                   bool underused = false;
                     if (!folio_test_partially_mapped(folio)) {
+                       if (!split_underused_thp)
+                               goto next;
                           underused = thp_underused(folio);
                           if (!underused)
                                   goto next;
 

Thanks for sending the diff! Now I know what you meant lol.

Quoted 5 lines
Quoted 2 lines

In the case of when shrinker is disabled, this could make the deferred split scan for partially mapped folios very ineffective?

I hope you realize that that's the default on each and every system out there that ships this feature :)

Yes, I made it default :)

I am assuming people either keep shrinker enabled (which is an extremely large majority as its default), or disable shrinker and they dont flip flop between the 2 settings.
There are 2 scenarios for the above patch:

Quoted 5 lines

- shrinker is enabled (default): the above patch wont make a difference.
- shrinker is disabled: the above patch makes splitting partially mapped folios inefficient.

I didnt talk about the shrinker enabled case as it was a no-op and just talked about the shrinker disabled case.

Yeah, and I am saying that all you raised as a concern would be a problem already today in all default setups (-> 99.999999%). :)

Quoted 7 lines
Quoted 2 lines

Probably we should not just disable the shrinker during boot, and once enabled, it would only split THPs created afterwards.

I probably didnt understand this again lol. Sorry its friday evening :)

split_underused_thp is true at boot time [1]. You are saying we should not disable shrinker during boot, but it is already not disabled during boot, right?

It's all a mess, sorry, currently it's "enabled" as default but actually "disabled". Let me see if I can clean all that up.

Signature · 3 lines
Cheers

David / dhildenb
reviewer
#22 lore raw
In reply to Usama Arif
Quoted 12 lines
Quoted 12 lines
---
  mm/huge_memory.c | 3 ---
  1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -0
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
  	void *kaddr;
  	int i;
 
Quoted 8 lines

mm/huge_memory.c static bool thp_underused(struct folio *folio) · line 4115

Quoted 3 lines
mm/huge_memory.c +0 -3
-	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-		return false;
-

I do agree with your usecase, but I am really worried about the amount of work and cpu time the THP shrinker will consume when max_ptes_none is 511 (I dont have any numbers to back up my worry :)), and its less likely that we will have these completely zeroed out THPs (again no numbers to back up this statement). We have the huge_zero_folio as well which is installed on read.

How about just doing a memcmp() between huge_zero_folio and folio? We know exactly how this folio looks like, in case of max_ptes_none == 511, if it is to be eligible for shrinking.

Quoted 4 lines
Quoted 4 lines
mm/huge_memory.c +0 -0
  	for (i = 0; i < folio_nr_pages(folio); i++) {
  		kaddr = kmap_local_folio(folio, i * PAGE_SIZE);
  		if (!memchr_inv(kaddr, 0, PAGE_SIZE)) {
#23 lore raw
Quoted 12 lines · Dev Jain
Quoted 12 lines
Quoted 12 lines
---
   mm/huge_memory.c | 3 ---
   1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -0
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
   	void *kaddr;
   	int i;
 
Quoted 9 lines
Quoted 8 lines

mm/huge_memory.c static bool thp_underused(struct folio *folio) · line 4115

Quoted 3 lines
mm/huge_memory.c +0 -3
-	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-		return false;
-

I do agree with your usecase, but I am really worried about the amount of work and cpu time the THP shrinker will consume when max_ptes_none is 511 (I dont have any numbers to back up my worry :)), and its less likely that we will have these completely zeroed out THPs (again no numbers to back up this statement). We have the huge_zero_folio as well which is installed on read.

FWIW, I am still hesitant on this patch because I think the whole deferred split lists + interaction with the shrinker should be cleaned up along this patch.

Quoted 3 lines

How about just doing a memcmp() between huge_zero_folio and folio? We know exactly how this folio looks like, in case of max_ptes_none == 511, if it is to be eligible for shrinking.

I wouldn't really want to optimize for this case, especially given that

(a) I want to remove all direct dependencies on max_ptes_none and any special values.

(b) The huge zero folio is not always around

(c) The kmap local prevents us from doing that (in particular on 32bit).

Signature · 3 lines
Cheers

David / dhildenb
Acked-by on v1 1/1
#24 lore raw
In reply to David Hildenbrand
Quoted 66 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

const size_t size = 1024*1024*1024;

int main(void)

 {
         size_t offs;
         char *area;

         area = mmap(0, size, PROT_READ | PROT_WRITE,
                     MAP_ANON | MAP_PRIVATE, -1, 0);
         if (area == MAP_FAILED) {
                 printf("mmap failed\n");
                 exit(-1);
         }
         madvise(area, size, MADV_HUGEPAGE);

         for (offs = 0; offs < size; offs += getpagesize())
                 area[offs] = 0;
         pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Quoted 12 lines
Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
#25 lore raw
In reply to David Hildenbrand
Quoted 16 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Quoted 9 lines

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

Just a note that I will send a v2 with an updated patch description after the discussions here settled.

Signature · 3 lines
Cheers

David / dhildenb
Reviewed-by on v1 1/1
#26 lore raw
In reply to David Hildenbrand
Quoted 6 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that

Freeing 'all pages which are part of a THP' rather?

Quoted 1 line

is completely zero (512) and khugepaged will for sure not try to

that is -> that are?

Quoted 1 line

instantiate a THP in that case (512 shared zeropages).

But if you write faulted they're not the zero page? And how are they shared? I mean be being dumb here.

Quoted 7 lines

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Surely this is worth having?

Quoted 5 lines

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If

Nit but 'we would not do that' is kinda duplicative here :)

Quoted 37 lines

that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

const size_t size = 1024*1024*1024;

int main(void)

 {
         size_t offs;
         char *area;

         area = mmap(0, size, PROT_READ | PROT_WRITE,
                     MAP_ANON | MAP_PRIVATE, -1, 0);
         if (area == MAP_FAILED) {
                 printf("mmap failed\n");
                 exit(-1);
         }
         madvise(area, size, MADV_HUGEPAGE);

         for (offs = 0; offs < size; offs += getpagesize())
                 area[offs] = 0;
         pause();

}
<\prog.c>

2) Trigger the shrinker

Quoted 8 lines

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Yikes!

Quoted 13 lines
Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>

LGTM, so:

Quoted 21 lines
---
 mm/huge_memory.c | 3 ---
 1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -4
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
 	void *kaddr;
 	int i;
 
-	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-		return false;
-
 	for (i = 0; i < folio_nr_pages(folio); i++) {
 		kaddr = kmap_local_folio(folio, i * PAGE_SIZE);
 		if (!memchr_inv(kaddr, 0, PAGE_SIZE)) {
--

2.50.1

#27 lore raw
Quoted 9 lines · Lorenzo Stoakes
Quoted 6 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that

Freeing 'all pages which are part of a THP' rather?

I'm German, I don't know what I'm doing. :D

Quoted 4 lines
Quoted 2 lines

is completely zero (512) and khugepaged will for sure not try to

that is -> that are?

the THP is zero?

Quoted 5 lines
Quoted 2 lines

instantiate a THP in that case (512 shared zeropages).

But if you write faulted they're not the zero page? And how are they shared? I mean be being dumb here.

The shrinker will replace zeroed pages by the shared zeropages.

Quoted 10 lines
Quoted 8 lines

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Surely this is worth having?

Alrighty, let me cc stable.

Quoted 8 lines
Quoted 6 lines

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If

Nit but 'we would not do that' is kinda duplicative here :)

Yeah, fixed it already while rewriting: this was meant to be "would now".

Thanks!

Signature · 3 lines
Cheers

David / dhildenb
#28 lore raw
Quoted 5 lines · David Hildenbrand
Quoted 4 lines · Lorenzo Stoakes
Quoted 3 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that

Freeing 'all pages which are part of a THP' rather?

I'm German, I don't know what I'm doing. :D

Whereas I have no excuse :P

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

is completely zero (512) and khugepaged will for sure not try to

that is -> that are?

the THP is zero?

I mean "all pages part of a THP that is completely zero' -> "all pages part of a THP that are completely zero', I'm referring to the 'all pages' bit, but I guess you mean the THP is entirely zero.

So maybe rephrase to 'all pages which are part of a zero THP' or similar? :)

Quoted 8 lines
Quoted 6 lines
Quoted 3 lines

instantiate a THP in that case (512 shared zeropages).

But if you write faulted they're not the zero page? And how are they shared? I mean be being dumb here.

The shrinker will replace zeroed pages by the shared zeropages.

Ah thanks, I was being dumb :) too stuck in vanilla mm land...

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

Quoted 7 lines
Quoted 5 lines
Quoted 3 lines

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Surely this is worth having?

Alrighty, let me cc stable.

Thanks!

Quoted 11 lines
Quoted 9 lines
Quoted 7 lines

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If

Nit but 'we would not do that' is kinda duplicative here :)

Yeah, fixed it already while rewriting: this was meant to be "would now".

Cheers!

Quoted 10 lines

Thanks!

--
Cheers

David / dhildenb

Cheers, Lorenzo

reviewer Reviewed-by on v1 1/1
#29 lore raw
In reply to David Hildenbrand
Quoted 67 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

const size_t size = 1024*1024*1024;

int main(void)

 {
         size_t offs;
         char *area;

         area = mmap(0, size, PROT_READ | PROT_WRITE,
                     MAP_ANON | MAP_PRIVATE, -1, 0);
         if (area == MAP_FAILED) {
                 printf("mmap failed\n");
                 exit(-1);
         }
         madvise(area, size, MADV_HUGEPAGE);

         for (offs = 0; offs < size; offs += getpagesize())
                 area[offs] = 0;
         pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Quoted 12 lines
Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>

Good catch! Feel free to add:

Cheers,
Lance

Quoted 22 lines
---
 mm/huge_memory.c | 3 ---
 1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -4
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
        void *kaddr;
        int i;
 
-       if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-               return false;
-
        for (i = 0; i < folio_nr_pages(folio); i++) {
                kaddr = kmap_local_folio(folio, i * PAGE_SIZE);
                if (!memchr_inv(kaddr, 0, PAGE_SIZE)) {
--

2.50.1

reviewer Reviewed-by on v1 1/1
#30 lore raw
In reply to David Hildenbrand
Quoted 78 lines · David Hildenbrand

We added an early exit in thp_underused(), probably to avoid scanning pages when there is no chance for success.

However, assume we have max_ptes_none = 511 (default).

Nothing should stop us from freeing all pages part of a THP that is completely zero (512) and khugepaged will for sure not try to instantiate a THP in that case (512 shared zeropages).

This can just trivially happen if someone writes a single 0 byte into a PMD area, or of course, when data ends up being zero later.

So let's remove that early exit.

Do we want to CC stable? Hm, not sure. Probably not urgent.

Note that, as default, the THP shrinker is active (/sys/kernel/mm/transparent_hugepage/shrink_underused = 1), and all THPs are added to the deferred split lists. However, with the max_ptes_none default we would never scan them. We would not do that. If that's not desirable, we should just disable the shrinker as default, also not adding all THPs to the deferred split lists.

Easy to reproduce:

1) Allocate some THPs filled with 0s

<prog.c>
#include <string.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <sys/mman.h>

  const size_t size = 1024*1024*1024;

  int main(void)
  {
          size_t offs;
          char *area;
          area = mmap(0, size, PROT_READ | PROT_WRITE,
                      MAP_ANON | MAP_PRIVATE, -1, 0);
          if (area == MAP_FAILED) {
                  printf("mmap failed\n");
                  exit(-1);
          }
          madvise(area, size, MADV_HUGEPAGE);

          for (offs = 0; offs < size; offs += getpagesize())
                  area[offs] = 0;
          pause();

}
<\prog.c>

2) Trigger the shrinker

E.g., memory pressure through memhog

3) Observe that THPs are not getting reclaimed

$ cat /proc/`pgrep prog`/smaps_rollup

Would list ~1GiB of AnonHugePages. With this fix, they would get reclaimed as expected.

Fixes dafff3f4c850 ("mm: split underused THPs")
Cc Andrew Morton <[email protected]>
Cc Lorenzo Stoakes <[email protected]>
Cc Zi Yan <[email protected]>
Cc Baolin Wang <[email protected]>
Cc "Liam R. Howlett" <[email protected]>
Cc Nico Pache <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Dev Jain <[email protected]>
Cc Barry Song <[email protected]>
Cc Usama Arif <[email protected]>
Quoted 1 line

---

Thanks for sorting this out. Make sense to me.

Quoted 17 lines
  mm/huge_memory.c | 3 ---
  1 file changed, 3 deletions(-)
mm/huge_memory.c +0 -3
index 26cedfcd74189..aa3ed7a86435b 100644
@@ -4110,9 +4110,6 @@ static bool thp_underused(struct folio *folio)
  	void *kaddr;
  	int i;
 
-	if (khugepaged_max_ptes_none == HPAGE_PMD_NR - 1)
-		return false;
-
  	for (i = 0; i < folio_nr_pages(folio); i++) {
  		kaddr = kmap_local_folio(folio, i * PAGE_SIZE);
  		if (!memchr_inv(kaddr, 0, PAGE_SIZE)) {