mm: Avoiding split large folios if swap has no space

When swap is disabled or exhausted, swap slot allocation may fail during swapout, causing large folios to be split into small folios. The splitting is reasonable when we truly fail to obtain contiguous swap slots, but it is pointless in the no-space case.

A simple way to reproduce this is to invoke MADV_PAGEOUT on a system with mTHP enabled but without swap configured.

#define SIZE (16 * 1024 * 1024)
int main(void)

 {
         char *buf = mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
         memset(buf, 1, SIZE);
         madvise(buf, SIZE, MADV_PAGEOUT);
         munmap(buf, SIZE);
         return 0;

}

With 16KB mTHP enabled, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 1024

This patch checks swap space before splitting. If there is no available space, it skips splitting. After the patch, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 0

Reported-by Nanzhe Zhao <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Lorenzo Stoakes <[email protected]>

Cc Zi Yan <[email protected]>

Cc Baolin Wang <[email protected]>

Cc Liam R. Howlett <[email protected]>

Cc Nico Pache <[email protected]>

Cc Ryan Roberts <[email protected]>

Cc Dev Jain <[email protected]>

Cc Lance Yang <[email protected]>

Cc Kairui Song <[email protected]>

Cc Qi Zheng <[email protected]>

Cc Shakeel Butt <[email protected]>

Cc Axel Rasmussen <[email protected]>

Cc Yuanchu Xie <[email protected]>

Cc Wei Xu <[email protected]>

Signed-off-by Barry Song (Xiaomi) <[email protected]>

---
 mm/vmscan.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

mm/vmscan.c +13 -3

index 299b5d9e8836..33f84a5fe7ee 100644
@@ -339,8 +339,7 @@ static bool can_demote(int nid, struct scan_control *sc,
        return !nodes_empty(allowed_mask);
 }
 
-static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
-                                         int nid,
+static inline bool __can_reclaim_anon_pages(struct mem_cgroup *memcg,
                                          struct scan_control *sc)
 {
        if (memcg == NULL) {

@@ -356,6 +355,16 @@ static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
                        return true;
        }
 
+       return false;
+}
+
+static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
+                                         int nid,
+                                         struct scan_control *sc)
+{
+       if (__can_reclaim_anon_pages(memcg, sc))
+               return true;
+
        /*
         * The page can not be swapped.
         *

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                                if (!folio_test_large(folio))
                                        goto activate_locked_split;
+                               if (!__can_reclaim_anon_pages(memcg, sc))
+                                       goto activate_locked_split;
                                /* Fallback to swap normal pages */
                                if (split_folio_to_list(folio, folio_list))
                                        goto activate_locked;
--

2.39.3 (Apple Git-146)

Quoted 6 lines · Barry Song (Xiaomi)

When swap is disabled or exhausted, swap slot allocation may fail during swapout, causing large folios to be split into small folios. The splitting is reasonable when we truly fail to obtain contiguous swap slots, but it is pointless in the no-space case.

Hi Barry,

Are we sure splitting is pointless? Since b1f202060afe ("mm: remap unused subpages to shared zeropage when splitting isolated thp") a split can lead to memory reclaim.

I've been far removed from the reclaim code for quite some time but im assuming in this case we can actually reclaim some unused memory which might be beneficial if there is memory pressure. However there are some discussions on whether we should further guard this zeropage remapping behavior [1].

[1] - https://lore.kernel.org/lkml/[email protected]/

When swap is disabled or exhausted, swap slot allocation may fail during swapout, causing large folios to be split into small folios. The splitting is reasonable when we truly fail to obtain contiguous swap slots, but it is pointless in the no-space case.

A simple way to reproduce this is to invoke MADV_PAGEOUT on a system with mTHP enabled but without swap configured.

Quoted 6 lines · Barry Song (Xiaomi)

When swap is disabled or exhausted, swap slot allocation may fail during swapout, causing large folios to be split into small folios. The splitting is reasonable when we truly fail to obtain contiguous swap slots, but it is pointless in the no-space case.

Quoted 2 lines

A simple way to reproduce this is to invoke MADV_PAGEOUT on a system with mTHP enabled but without swap configured.

Would we ever run into that code without MADV_PAGEOUT, or is that a MADV_PAGEOUT-special thing?

I recall that can_reclaim_anon_pages() would make vmscan skip such code, but it's tricky.

When swap is disabled or exhausted, swap slot allocation may fail during swapout, causing large folios to be split into small folios. The splitting is reasonable when we truly fail to obtain contiguous swap slots, but it is pointless in the no-space case.

A simple way to reproduce this is to invoke MADV_PAGEOUT on a system with mTHP enabled but without swap configured.

#define SIZE (16 * 1024 * 1024)
int main(void)

 {
         char *buf = mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
         memset(buf, 1, SIZE);
         madvise(buf, SIZE, MADV_PAGEOUT);
         munmap(buf, SIZE);
         return 0;

}

With 16KB mTHP enabled, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 1024

This patch checks swap space before splitting. If there is no available space, it skips splitting. After the patch, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 0

Reported-by Nanzhe Zhao <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Lorenzo Stoakes <[email protected]>

Cc Zi Yan <[email protected]>

Cc Baolin Wang <[email protected]>

Cc Liam R. Howlett <[email protected]>

Cc Nico Pache <[email protected]>

Cc Ryan Roberts <[email protected]>

Cc Dev Jain <[email protected]>

Cc Lance Yang <[email protected]>

Cc Kairui Song <[email protected]>

Cc Qi Zheng <[email protected]>

Cc Shakeel Butt <[email protected]>

Cc Axel Rasmussen <[email protected]>

Cc Yuanchu Xie <[email protected]>

Cc Wei Xu <[email protected]>

Signed-off-by Barry Song (Xiaomi) <[email protected]>

---
 mm/vmscan.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

mm/vmscan.c +11 -2

index 299b5d9e8836..33f84a5fe7ee 100644
@@ -339,8 +339,7 @@ static bool can_demote(int nid, struct scan_control *sc,
 	return !nodes_empty(allowed_mask);
 }
 
-static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
-					  int nid,
+static inline bool __can_reclaim_anon_pages(struct mem_cgroup *memcg,
 					  struct scan_control *sc)
 {
 	if (memcg == NULL) {

@@ -356,6 +355,16 @@ static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
 			return true;
 	}
 
+	return false;
+}
+
+static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
+					  int nid,
+					  struct scan_control *sc)
+{
+	if (__can_reclaim_anon_pages(memcg, sc))
+		return true;
+
 	/*
 	 * The page can not be swapped.
 	 *

Review static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1293

Quoted 9 lines

      /*
       * The page can not be swapped.
       *

(quoted diff) +2 -0

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                              if (!folio_test_large(folio))
                                      goto activate_locked_split;
+                             if (!__can_reclaim_anon_pages(memcg, sc))
+                                     goto activate_locked_split;

Why are we even trying to allocate swap space if we cannot reclaim such pages? Makes we wonder whether we would want to have that check earlier, before the folio_alloc_swap().

Any downsides?

On Fri, Jun 19, 2026 at 10:04 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]

Quoted 14 lines

Review static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1293

Quoted 9 lines

       /*
        * The page can not be swapped.
        *

(quoted diff) +2 -0

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                               if (!folio_test_large(folio))
                                       goto activate_locked_split;
+                             if (!__can_reclaim_anon_pages(memcg, sc))
+                                     goto activate_locked_split;

Why are we even trying to allocate swap space if we cannot reclaim such pages? Makes we wonder whether we would want to have that check earlier, before the folio_alloc_swap().

Quoted 1 line

Any downsides?

I don't think there are any obvious downsides there. One issue is that the memcg may not be passed from reclaim_pages(), so memcg would always be NULL. However, the folio could still belong to a memcg whose swap quota has been exhausted. In that case, my __can_reclaim_anon_pages() will fail when checking whether we can swap out. But switching to folio_memcg() also seems awkward.

So I feel Kairui’s suggestion [1] might be the best approach. In folio_alloc_swap(), we return -EAGAIN to tell vmscan.c that we can split the folio and retry the swap-out.
only when there are sufficient swap slots and sufficient memcg swap quota do we return -EAGAIN, allowing vmscan to perform a split.

mm/swapfile.c +0 -0

index 78b49b0658ad..62e2c506ccae 100644

mm/swapfile.c +4 -2

  	}
 
  again:

@@ -1769,11 +1772,13 @@ int folio_alloc_swap(struct folio *folio)
  	}
 
  	/* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-	if (unlikely(mem_cgroup_try_charge_swap(folio)))
+	if (unlikely(mem_cgroup_try_charge_swap(folio))) {
  		swap_cache_del_folio(folio);
+		return -ENOMEM;
+	}
 
  	if (unlikely(!folio_test_swapcache(folio)))
-		return -ENOMEM;
+		return -EAGAIN;
 
  	return 0;
  }

mm/vmscan.c +2 -0

index 299b5d9e8836..63e8578454ea 100644
@@ -1257,6 +1257,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
  		 */
  		if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
  				!folio_test_swapcache(folio)) {
+			int ret;
+
  			if (!(sc->gfp_mask & __GFP_IO))
  				goto keep_locked;
  			if (folio_maybe_dma_pinned(folio))

Quoted 31 lines · Barry Song (Xiaomi)

On Fri, Jun 19, 2026 at 10:04 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]

Quoted 14 lines

Review static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1293

Quoted 9 lines

       /*
        * The page can not be swapped.
        *

(quoted diff) +2 -0

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                               if (!folio_test_large(folio))
                                       goto activate_locked_split;
+                             if (!__can_reclaim_anon_pages(memcg, sc))
+                                     goto activate_locked_split;

Why are we even trying to allocate swap space if we cannot reclaim such pages? Makes we wonder whether we would want to have that check earlier, before the folio_alloc_swap().

Quoted 1 line

Any downsides?

I don't think there are any obvious downsides there. One issue is that the memcg may not be passed from reclaim_pages(), so memcg would always be NULL. However, the folio could still belong to a memcg whose swap quota has been exhausted. In that case, my __can_reclaim_anon_pages() will fail when checking whether we can swap out. But switching to folio_memcg() also seems awkward.

So I feel Kairui’s suggestion [1] might be the best approach. In folio_alloc_swap(), we return -EAGAIN to tell vmscan.c that we can split the folio and retry the swap-out.
only when there are sufficient swap slots and sufficient memcg swap quota do we return -EAGAIN, allowing vmscan to perform a split.

mm/swapfile.c int folio_alloc_swap(struct folio *folio) · line 1760

Quoted 11 lines

mm/swapfile.c +3 -0

index 78b49b0658ad..62e2c506ccae 100644
@@ -1755,6 +1755,9 @@ int folio_alloc_swap(struct folio *folio)
                      VM_WARN_ON_ONCE(1);
                      return -EINVAL;
              }
+
+             if (get_nr_swap_pages() < (1 << order))
+                     return -ENOMEM;

Shouldn't this return -EAGAIN? Suppose we try to swap out an order-9 large folio but get_nr_swap_pages() returns 256, then we'd still need to split the order-9 large folio to reclaim some memory.

Quoted 34 lines

mm/swapfile.c +4 -2

      }
 
  again:

@@ -1769,11 +1772,13 @@ int folio_alloc_swap(struct folio *folio)
      }
 
      /* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-     if (unlikely(mem_cgroup_try_charge_swap(folio)))
+     if (unlikely(mem_cgroup_try_charge_swap(folio))) {
              swap_cache_del_folio(folio);
+             return -ENOMEM;
+     }
 
      if (unlikely(!folio_test_swapcache(folio)))
-             return -ENOMEM;
+             return -EAGAIN;
 
      return 0;
  }

mm/vmscan.c +2 -0

index 299b5d9e8836..63e8578454ea 100644
@@ -1257,6 +1257,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
               */
              if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
                              !folio_test_swapcache(folio)) {
+                     int ret;
+
                      if (!(sc->gfp_mask & __GFP_IO))
                              goto keep_locked;
                      if (folio_maybe_dma_pinned(folio))

@@ -1275,10 +1277,10 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,

mm/vmscan.c static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1280

Quoted 5 lines

mm/vmscan.c +1 -1

                                  split_folio_to_list(folio, folio_list))
                                      goto activate_locked;
                      }
-                     if (folio_alloc_swap(folio)) {
+                     if ((ret = folio_alloc_swap(folio))) {

Also, please give shmem some love (shmem also calls folio_alloc_swap() when swapping out) :)

Quoted 35 lines · Baolin Wang

Quoted 31 lines · Barry Song (Xiaomi)

On Fri, Jun 19, 2026 at 10:04 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]

Quoted 14 lines

Review static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1293

Quoted 9 lines

        /*
         * The page can not be swapped.
         *

(quoted diff) +2 -0

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                                if (!folio_test_large(folio))
                                        goto activate_locked_split;
+                             if (!__can_reclaim_anon_pages(memcg, sc))
+                                     goto activate_locked_split;

Why are we even trying to allocate swap space if we cannot reclaim such pages? Makes we wonder whether we would want to have that check earlier, before the folio_alloc_swap().

Quoted 1 line

Any downsides?

I don't think there are any obvious downsides there. One issue is that the memcg may not be passed from reclaim_pages(), so memcg would always be NULL. However, the folio could still belong to a memcg whose swap quota has been exhausted. In that case, my __can_reclaim_anon_pages() will fail when checking whether we can swap out. But switching to folio_memcg() also seems awkward.

So I feel Kairui’s suggestion [1] might be the best approach. In folio_alloc_swap(), we return -EAGAIN to tell vmscan.c that we can split the folio and retry the swap-out.
only when there are sufficient swap slots and sufficient memcg swap quota do we return -EAGAIN, allowing vmscan to perform a split.

Quoted 15 lines

mm/swapfile.c int folio_alloc_swap(struct folio *folio) · line 1760

Quoted 11 lines

mm/swapfile.c +3 -0

index 78b49b0658ad..62e2c506ccae 100644
@@ -1755,6 +1755,9 @@ int folio_alloc_swap(struct folio *folio)
                       VM_WARN_ON_ONCE(1);
                       return -EINVAL;
               }
+
+             if (get_nr_swap_pages() < (1 << order))
+                     return -ENOMEM;

Shouldn't this return -EAGAIN? Suppose we try to swap out an order-9 large folio but get_nr_swap_pages() returns 256, then we'd still need to split the order-9 large folio to reclaim some memory.

I guess Kairui has the opposite view. Quoting Kairui:

"- 1. the mem_cgroup_try_charge_swap in it failed
- 2. allocation failed but nr_swap_pages > folio size
- 3. allocation failed because all devices are full or unusable (roughly nr_swap_pages < folio size)

Only case 2 requires splitting. __can_reclaim_anon_pages also checks demote which is not related to swap."

For example, if get_nr_swap_pages() returns 1, we can only swap out a single page after splitting. Is that really worth it? On the other hand, if it returns 256, we could at least swap out half of an order-9 large folio. Isn't that worthwhile?

Quoted 31 lines

Quoted 31 lines

mm/swapfile.c +4 -2

   again:

@@ -1769,11 +1772,13 @@ int folio_alloc_swap(struct folio *folio)
       }
 
       /* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-     if (unlikely(mem_cgroup_try_charge_swap(folio)))
+     if (unlikely(mem_cgroup_try_charge_swap(folio))) {
               swap_cache_del_folio(folio);
+             return -ENOMEM;
+     }
 
       if (unlikely(!folio_test_swapcache(folio)))
-             return -ENOMEM;
+             return -EAGAIN;
 
       return 0;
   }

mm/vmscan.c +2 -0

index 299b5d9e8836..63e8578454ea 100644
@@ -1257,6 +1257,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
                */
               if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
                               !folio_test_swapcache(folio)) {
+                     int ret;
+
                       if (!(sc->gfp_mask & __GFP_IO))
                               goto keep_locked;
                       if (folio_maybe_dma_pinned(folio))

@@ -1275,10 +1277,10 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,

Quoted 8 lines

mm/vmscan.c static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1280

Quoted 5 lines

mm/vmscan.c +1 -1

                                   split_folio_to_list(folio, folio_list))
                                       goto activate_locked;
                       }
-                     if (folio_alloc_swap(folio)) {
+                     if ((ret = folio_alloc_swap(folio))) {

Also, please give shmem some love (shmem also calls folio_alloc_swap() when swapping out) :)

Sure. I'm pretty sure I love shmem. :-)

On Fri, Jun 19, 2026 at 10:04 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]

Quoted 14 lines

Review static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1293

Quoted 9 lines

      /*
       * The page can not be swapped.
       *

(quoted diff) +2 -0

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                              if (!folio_test_large(folio))
                                      goto activate_locked_split;
+                             if (!__can_reclaim_anon_pages(memcg, sc))
+                                     goto activate_locked_split;

Why are we even trying to allocate swap space if we cannot reclaim such pages? Makes we wonder whether we would want to have that check earlier, before the folio_alloc_swap().

Quoted 1 line

Any downsides?

I don't think there are any obvious downsides there. One issue is that the memcg may not be passed from reclaim_pages(), so memcg would always be NULL. However, the folio could still belong to a memcg whose swap quota has been exhausted. In that case, my __can_reclaim_anon_pages() will fail when checking whether we can swap out. But switching to folio_memcg() also seems awkward.

So I feel Kairui’s suggestion [1] might be the best approach. In folio_alloc_swap(), we return -EAGAIN to tell vmscan.c that we can split the folio and retry the swap-out.
only when there are sufficient swap slots and sufficient memcg swap quota do we return -EAGAIN, allowing vmscan to perform a split.

mm/swapfile.c +0 -0

index 78b49b0658ad..62e2c506ccae 100644

mm/swapfile.c +2 -1

+	}
 
 	if (unlikely(!folio_test_swapcache(folio)))
-		return -ENOMEM;
+		return -EAGAIN;
 
 	return 0;
 }

mm/vmscan.c +2 -0

index 299b5d9e8836..63e8578454ea 100644
@@ -1257,6 +1257,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 		 */
 		if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
 				!folio_test_swapcache(folio)) {
+			int ret;
+
 			if (!(sc->gfp_mask & __GFP_IO))
 				goto keep_locked;
 			if (folio_maybe_dma_pinned(folio))

mm/vmscan.c +1 -1

 				int __maybe_unused order = folio_order(folio);
 
-				if (!folio_test_large(folio))
+				if (!folio_test_large(folio) || ret != -EAGAIN)
 					goto activate_locked_split;
 				/* Fallback to swap normal pages */
 				if (split_folio_to_list(folio, folio_list))

What’s your view on this, David?

Quoted 36 lines · Barry Song (Xiaomi)

On Fri, Jun 19, 2026 at 10:04 PM David Hildenbrand (Arm) <[email protected]> wrote:
[...]

Quoted 14 lines

Review static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1293

Quoted 9 lines

      /*
       * The page can not be swapped.
       *

(quoted diff) +2 -0

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                              if (!folio_test_large(folio))
                                      goto activate_locked_split;
+                             if (!__can_reclaim_anon_pages(memcg, sc))
+                                     goto activate_locked_split;

Why are we even trying to allocate swap space if we cannot reclaim such pages? Makes we wonder whether we would want to have that check earlier, before the folio_alloc_swap().

Quoted 1 line

Any downsides?

I don't think there are any obvious downsides there. One issue is that the memcg may not be passed from reclaim_pages(), so memcg would always be NULL. However, the folio could still belong to a memcg whose swap quota has been exhausted. In that case, my __can_reclaim_anon_pages() will fail when checking whether we can swap out. But switching to folio_memcg() also seems awkward.

So I feel Kairui’s suggestion [1] might be the best approach. In folio_alloc_swap(), we return -EAGAIN to tell vmscan.c that we can split the folio and retry the swap-out.
only when there are sufficient swap slots and sufficient memcg swap quota do we return -EAGAIN, allowing vmscan to perform a split.

mm/swapfile.c +0 -0

index 78b49b0658ad..62e2c506ccae 100644

mm/swapfile.c int folio_alloc_swap(struct folio *folio) · line 1760

Quoted 6 lines

mm/swapfile.c +3 -0

@@ -1755,6 +1755,9 @@ int folio_alloc_swap(struct folio *folio)
                      VM_WARN_ON_ONCE(1);
                      return -EINVAL;
              }
+
+             if (get_nr_swap_pages() < (1 << order))
+                     return -ENOMEM;

I guess we could be clearer with the return value:

-> !get_nr_swap_pages() -> -ENOSPC / -ENOMEM

(no space at all)

-> get_nr_swap_pages() < (1 << order) -> -E2BIG

(there is some space, but not for the full thing)

But now I wonder whether we would also want to check "is there any free swap space", not just "is there any swap".

Essentially, try returning -E2BIG if there is the chance to swap out after split, and -ENOSPC / -ENOMEM if a split wouldn't help.

mm/swapfile.c int folio_alloc_swap(struct folio *folio) · line 1777

Quoted 11 lines

mm/swapfile.c +2 -1

      }
 
 again:

@@ -1769,11 +1772,13 @@ int folio_alloc_swap(struct folio *folio)
      }
 
      /* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-     if (unlikely(mem_cgroup_try_charge_swap(folio)))
+     if (unlikely(mem_cgroup_try_charge_swap(folio))) {
              swap_cache_del_folio(folio);
+             return -ENOMEM;

Here we wouldn't have the information whether we could charge after a split.

So that would require a rework to signal this more cleanly to the caller.

Quoted 22 lines

mm/swapfile.c +2 -1

+     }
 
      if (unlikely(!folio_test_swapcache(folio)))
-             return -ENOMEM;
+             return -EAGAIN;
 
      return 0;
 }

mm/vmscan.c +2 -0

index 299b5d9e8836..63e8578454ea 100644
@@ -1257,6 +1257,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
               */
              if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
                              !folio_test_swapcache(folio)) {
+                     int ret;
+
                      if (!(sc->gfp_mask & __GFP_IO))
                              goto keep_locked;
                      if (folio_maybe_dma_pinned(folio))

mm/vmscan.c static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1280

Quoted 6 lines

mm/vmscan.c +1 -1

@@ -1275,10 +1277,10 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
                                  split_folio_to_list(folio, folio_list))
                                      goto activate_locked;
                      }
-                     if (folio_alloc_swap(folio)) {
+                     if ((ret = folio_alloc_swap(folio))) {

I prefer doing the assignment outside the conditional.

Quoted 10 lines

mm/vmscan.c +1 -1

                              int __maybe_unused order = folio_order(folio);
 
-                             if (!folio_test_large(folio))
+                             if (!folio_test_large(folio) || ret != -EAGAIN)
                                      goto activate_locked_split;
                              /* Fallback to swap normal pages */
                              if (split_folio_to_list(folio, folio_list))

What’s your view on this, David?

I guess returning from folio_alloc_swap() whether a split could allow for swapout (e.g., -E2BIG) would be reasonable.

To catch all the cases where it makes a difference:
* No free swap space (split won't work)
* Some free swap space (split would work)
* Sufficient free swap space, but fragmented (split would work)
* No memcg space (split won't work)
* Some memcg space (split would work)

Quoted 3 lines

But now I wonder whether we would also want to check "is there any free swap space", not just "is there any swap".

I don't quite understand you. get_nr_swap_pages() returns nr_swap_pages, which increases or decreases as swap is allocated or freed. I guess it just reflects how many swaps we currently have available?

Quoted 20 lines

Essentially, try returning -E2BIG if there is the chance to swap out after split, and -ENOSPC / -ENOMEM if a split wouldn't help.

mm/swapfile.c int folio_alloc_swap(struct folio *folio) · line 1777

Quoted 11 lines

      }

again:

mm/swapfile.c +2 -1

@@ -1769,11 +1772,13 @@ int folio_alloc_swap(struct folio *folio)
      }
 
      /* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-     if (unlikely(mem_cgroup_try_charge_swap(folio)))
+     if (unlikely(mem_cgroup_try_charge_swap(folio))) {
              swap_cache_del_folio(folio);
+             return -ENOMEM;

Here we wouldn't have the information whether we could charge after a split.

Quoted 1 line

So that would require a rework to signal this more cleanly to the caller.

Yep. The tricky part is that mem_cgroup_try_charge_swap() cannot return how much swap quota is available in the memcg. Do you prefer to add an output argument to mem_cgroup_try_charge_swap() to expose that

Quoted 8 lines

Quoted 3 lines

But now I wonder whether we would also want to check "is there any free swap space", not just "is there any swap".

I don't quite understand you. get_nr_swap_pages() returns nr_swap_pages, which increases or decreases as swap is allocated or freed. I guess it just reflects how many swaps we currently have available?

Indeed, I was confused by the function name it's "free swap pages". So all goof :)

Quoted 20 lines

Quoted 20 lines

Essentially, try returning -E2BIG if there is the chance to swap out after split, and -ENOSPC / -ENOMEM if a split wouldn't help.

mm/swapfile.c int folio_alloc_swap(struct folio *folio) · line 1777

Quoted 11 lines

      }

again:

mm/swapfile.c +2 -1

@@ -1769,11 +1772,13 @@ int folio_alloc_swap(struct folio *folio)
      }
 
      /* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-     if (unlikely(mem_cgroup_try_charge_swap(folio)))
+     if (unlikely(mem_cgroup_try_charge_swap(folio))) {
              swap_cache_del_folio(folio);
+             return -ENOMEM;

Here we wouldn't have the information whether we could charge after a split.

Quoted 6 lines

Quoted 1 line

So that would require a rework to signal this more cleanly to the caller.

Yep. The tricky part is that mem_cgroup_try_charge_swap() cannot return how much swap quota is available in the memcg. Do you prefer to add an output argument to mem_cgroup_try_charge_swap() to expose that

That would probably be cleanest, if that is easily possible. We would want to get memcg maintainer feedback on that.

@memcg folks: we'd like to know whether splitting a large folio would make mem_cgroup_try_charge_swap() succeed on a split (smaller) part, to distinguish "there is no way we can swap out anything, don't split" vs. "we could swap out, split".

Quoted 17 lines · David Hildenbrand (Arm)

Quoted 5 lines

I don't quite understand you. get_nr_swap_pages() returns nr_swap_pages, which increases or decreases as swap is allocated or freed. I guess it just reflects how many swaps we currently have available?

Indeed, I was confused by the function name it's "free swap pages". So all goof :)

Quoted 6 lines

Yep. The tricky part is that mem_cgroup_try_charge_swap() cannot return how much swap quota is available in the memcg. Do you prefer to add an output argument to mem_cgroup_try_charge_swap() to expose that

That would probably be cleanest, if that is easily possible. We would want to get memcg maintainer feedback on that.

Quoted 4 lines

@memcg folks: we'd like to know whether splitting a large folio would make mem_cgroup_try_charge_swap() succeed on a split (smaller) part, to distinguish "there is no way we can swap out anything, don't split" vs. "we could swap out, split".

It's technically doable, but is this worth the bother? The remaining headroom is less than a large folio. You can split this one, but you cannot even swap out all of its subpages anymore?

From the cgroup
side, we don't need the limit to be obeyed this rigidly. We overcharge temporarily in other places if it's convenient to do so. A fuzz factor around the limit is acceptable.

But if you still want to do it, here is how:

The page_counter_try_charge() in __mem_cgroup_try_charge_swap() walks the hierarchy upwards. If it fails, it will store the first level that failed against its limit. You can do the mem_cgroup_margin() math against this counter to determine headroom. An ancestor *could* be more restrictive, so you need to finish the hierarchy walk to the root and use the min() of all the swap.max - page_counter_read(swap). Then return that in a return argument from __mem_cgroup_try_charge_swap().

Quoted 23 lines · Johannes Weiner

Quoted 21 lines · David Hildenbrand (Arm)

Quoted 5 lines

I don't quite understand you. get_nr_swap_pages() returns nr_swap_pages, which increases or decreases as swap is allocated or freed. I guess it just reflects how many swaps we currently have available?

Indeed, I was confused by the function name it's "free swap pages". So all goof :)

Quoted 6 lines

Yep. The tricky part is that mem_cgroup_try_charge_swap() cannot return how much swap quota is available in the memcg. Do you prefer to add an output argument to mem_cgroup_try_charge_swap() to expose that

That would probably be cleanest, if that is easily possible. We would want to get memcg maintainer feedback on that.

@memcg folks: we'd like to know whether splitting a large folio would make mem_cgroup_try_charge_swap() succeed on a split (smaller) part, to distinguish "there is no way we can swap out anything, don't split" vs. "we could swap out, split".

Quoted 3 lines

It's technically doable, but is this worth the bother? The remaining headroom is less than a large folio. You can split this one, but you cannot even swap out all of its subpages anymore?

I was asking myself the same, but when we think in terms of THPs on arm64 64k we're in the range of double-digit MiBs.

Quoted 5 lines

From the cgroup
side, we don't need the limit to be obeyed this rigidly. We overcharge temporarily in other places if it's convenient to do so. A fuzz factor around the limit is acceptable.

Thanks for that information.

Quoted 10 lines

But if you still want to do it, here is how:

The page_counter_try_charge() in __mem_cgroup_try_charge_swap() walks the hierarchy upwards. If it fails, it will store the first level that failed against its limit. You can do the mem_cgroup_margin() math against this counter to determine headroom. An ancestor *could* be more restrictive, so you need to finish the hierarchy walk to the root and use the min() of all the swap.max - page_counter_read(swap). Then return that in a return argument from __mem_cgroup_try_charge_swap().

Thanks! @Barry, up to you if we want to implement that right away or if we're simply going to assume that if charging fails, not worth splitting (changing the existing handling IIUC).

mm/memcontrol.c +1 -1

@@ -5578,7 +5578,7 @@ int __init mem_cgroup_init(void)
  *
  * Returns 0 on success, -ENOMEM on failure.
  */
-int __mem_cgroup_try_charge_swap(struct folio *folio)
+int __mem_cgroup_try_charge_swap(struct folio *folio, long *left_space)
 {
        unsigned int nr_pages = folio_nr_pages(folio);
        struct swap_cluster_info *ci;

Quoted 13 lines · Barry Song

mm/memcontrol.c +1 -1

@@ -5578,7 +5578,7 @@ int __init mem_cgroup_init(void)
  *
  * Returns 0 on success, -ENOMEM on failure.
  */
-int __mem_cgroup_try_charge_swap(struct folio *folio)
+int __mem_cgroup_try_charge_swap(struct folio *folio, long *left_space)
 {
        unsigned int nr_pages = folio_nr_pages(folio);
        struct swap_cluster_info *ci;

@@ -5611,6 +5611,10 @@ int __mem_cgroup_try_charge_swap(struct folio *folio)
                memcg_memory_event(memcg, MEMCG_SWAP_MAX);

mm/memcontrol.c int __mem_cgroup_try_charge_swap(struct folio *folio) · line 5615

Quoted 4 lines

mm/memcontrol.c +2 -0

                memcg_memory_event(memcg, MEMCG_SWAP_FAIL);
                mem_cgroup_private_id_put(memcg, nr_pages);
+               if (folio_test_large(folio))
+                       *left_space = mem_cgroup_get_nr_swap_pages(memcg);

It's a bit awkward to walk up the whole hierarchy again when we already have the counter that failed. Please do something like this (not tested!), then use page_counter_margin() against @counter:

When swap is disabled or exhausted, swap slot allocation may fail during swapout, causing large folios to be split into small folios. The splitting is reasonable when we truly fail to obtain contiguous swap slots, but it is pointless in the no-space case.

A simple way to reproduce this is to invoke MADV_PAGEOUT on a system with mTHP enabled but without swap configured.

#define SIZE (16 * 1024 * 1024)
int main(void)

 {
         char *buf = mmap(NULL, SIZE, PROT_READ | PROT_WRITE,
                    MAP_PRIVATE | MAP_ANONYMOUS, -1, 0);
         memset(buf, 1, SIZE);
         madvise(buf, SIZE, MADV_PAGEOUT);
         munmap(buf, SIZE);
         return 0;

}

With 16KB mTHP enabled, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 1024

This patch checks swap space before splitting. If there is no available space, it skips splitting. After the patch, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 0

Reported-by Nanzhe Zhao <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Lorenzo Stoakes <[email protected]>

Cc Zi Yan <[email protected]>

Cc Baolin Wang <[email protected]>

Cc Liam R. Howlett <[email protected]>

Cc Nico Pache <[email protected]>

Cc Ryan Roberts <[email protected]>

Cc Dev Jain <[email protected]>

Cc Lance Yang <[email protected]>

Cc Kairui Song <[email protected]>

Cc Qi Zheng <[email protected]>

Cc Shakeel Butt <[email protected]>

Cc Axel Rasmussen <[email protected]>

Cc Yuanchu Xie <[email protected]>

Cc Wei Xu <[email protected]>

Signed-off-by Barry Song (Xiaomi) <[email protected]>

---
 mm/vmscan.c | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

mm/vmscan.c +11 -2

index 299b5d9e8836..33f84a5fe7ee 100644
@@ -339,8 +339,7 @@ static bool can_demote(int nid, struct scan_control *sc,
        return !nodes_empty(allowed_mask);
 }
 
-static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
-                                         int nid,
+static inline bool __can_reclaim_anon_pages(struct mem_cgroup *memcg,
                                          struct scan_control *sc)
 {
        if (memcg == NULL) {

@@ -356,6 +355,16 @@ static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
                        return true;
        }
 
+       return false;
+}
+
+static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
+                                         int nid,
+                                         struct scan_control *sc)
+{
+       if (__can_reclaim_anon_pages(memcg, sc))
+               return true;
+
        /*
         * The page can not be swapped.
         *

Quoted 84 lines · Barry Song (Xiaomi)

When swap is disabled or exhausted, swap slot allocation may fail during swapout, causing large folios to be split into small folios. The splitting is reasonable when we truly fail to obtain contiguous swap slots, but it is pointless in the no-space case.

A simple way to reproduce this is to invoke MADV_PAGEOUT on a system with mTHP enabled but without swap configured.

 #define SIZE (16 * 1024 * 1024)
 int main(void)
 {
         char *buf = mmap(NULL, SIZE, PROT_READ | PROT_WRITE, MAP_PRIVATE | MAP_ANONYMOUS, -1, 0); memset(buf, 1, SIZE);
         madvise(buf, SIZE, MADV_PAGEOUT); munmap(buf, SIZE);
         return 0;
 }

With 16KB mTHP enabled, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 1024

This patch checks swap space before splitting. If there is no available space, it skips splitting. After the patch, we observe:
~ # cat /sys/kernel/mm/transparent_hugepage/hugepages-16kB/stats/split 0

Reported-by Nanzhe Zhao <[email protected]>

Cc David Hildenbrand <[email protected]>

Cc Lorenzo Stoakes <[email protected]>

Cc Zi Yan <[email protected]>

Cc Baolin Wang <[email protected]>

Cc Liam R. Howlett <[email protected]>

Cc Nico Pache <[email protected]>

Cc Ryan Roberts <[email protected]>

Cc Dev Jain <[email protected]>

Cc Lance Yang <[email protected]>

Cc Kairui Song <[email protected]>

Cc Qi Zheng <[email protected]>

Cc Shakeel Butt <[email protected]>

Cc Axel Rasmussen <[email protected]>

Cc Yuanchu Xie <[email protected]>

Cc Wei Xu <[email protected]>

Signed-off-by Barry Song (Xiaomi) <[email protected]>

---
 mm/vmscan.c | 15 +++++++++++++--

 1 file changed, 13 insertions(+), 2 deletions(-)

mm/vmscan.c +11 -2

index 299b5d9e8836..33f84a5fe7ee 100644
@@ -339,8 +339,7 @@ static bool can_demote(int nid, struct scan_control *sc,
         return !nodes_empty(allowed_mask);
  }
 
-static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
-                                         int nid,
+static inline bool __can_reclaim_anon_pages(struct mem_cgroup *memcg,
                                           struct scan_control *sc)
  {
         if (memcg == NULL) {

@@ -356,6 +355,16 @@ static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
                         return true;
         }
 
+       return false;
+}
+
+static inline bool can_reclaim_anon_pages(struct mem_cgroup *memcg,
+                                         int nid,
+                                         struct scan_control *sc)
+{
+       if (__can_reclaim_anon_pages(memcg, sc))
+               return true;
+
         /*
          * The page can not be swapped.
          *

mm/vmscan.c static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1296

Quoted 8 lines

mm/vmscan.c +2 -0

@@ -1280,6 +1289,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 
                                 if (!folio_test_large(folio))
                                         goto activate_locked_split;
+                               if (!__can_reclaim_anon_pages(memcg, sc))
+                                       goto activate_locked_split;
                                 /* Fallback to swap normal pages */
                                 if (split_folio_to_list(folio, folio_list))
                                         goto activate_locked;

Hello Barry,

Thanks for raising this issue. I saw a similar issue report in the mail list before and was thinking that, perhaps another approach is to

let folio_alloc_swap return a more detailed error code, for example:

- 1. the mem_cgroup_try_charge_swap in it failed
- 2. allocation failed but nr_swap_pages > folio size
- 3. allocation failed because all devices are full or unusable (roughly nr_swap_pages < folio size)

Only case 2 requires splitting. __can_reclaim_anon_pages also checks demote which is not related to swap.

Quoted 4 lines · Kairui Song

Hello Barry,

Quoted 2 lines

Thanks for raising this issue. I saw a similar issue report in the mail list before and was thinking that, perhaps another approach is to

Hi Kairui,

Could you please post the link to your report? I'd like to add your Reported-by and Closes tags as well.

Quoted 8 lines

let folio_alloc_swap return a more detailed error code, for example:

- 1. the mem_cgroup_try_charge_swap in it failed
- 2. allocation failed but nr_swap_pages > folio size
- 3. allocation failed because all devices are full or unusable (roughly nr_swap_pages < folio size)

folio_alloc_swap() returns error codes such as -EAGAIN, -EINVAL, and -ENOMEM. For cases 1, 2, and 3, I assume it would return -ENOMEM?

I assume you mean that we might want folio_alloc_swap() to return an enum instead?

another approach is that I can return -EAGAIN for those cases we want to retry swapping-out after splitting folios:

mm/swapfile.c +7 -2

index 78b49b0658ad..62e2c506ccae 100644
@@ -1755,6 +1755,9 @@ int folio_alloc_swap(struct folio *folio)
                        VM_WARN_ON_ONCE(1);
                        return -EINVAL;
                }
+
+               if (get_nr_swap_pages() < (1 << order))
+                       return -ENOMEM;
        }
 
 again:

@@ -1769,11 +1772,13 @@ int folio_alloc_swap(struct folio *folio)
        }
 
        /* Need to call this even if allocation failed, for MEMCG_SWAP_FAIL. */
-       if (unlikely(mem_cgroup_try_charge_swap(folio)))
+       if (unlikely(mem_cgroup_try_charge_swap(folio))) {
                swap_cache_del_folio(folio);
+               return -ENOMEM;
+       }
 
        if (unlikely(!folio_test_swapcache(folio)))
-               return -ENOMEM;
+               return -EAGAIN;
 
        return 0;
 }

mm/vmscan.c +16 -15

index 299b5d9e8836..4c4cbd72c013 100644
@@ -1257,6 +1257,8 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
                 */
                if (folio_test_anon(folio) && folio_test_swapbacked(folio) &&
                                !folio_test_swapcache(folio)) {
+                       int ret;
+
                        if (!(sc->gfp_mask & __GFP_IO))
                                goto keep_locked;
                        if (folio_maybe_dma_pinned(folio))

@@ -1275,25 +1277,24 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
                                    split_folio_to_list(folio, folio_list))
                                        goto activate_locked;
                        }
-                       if (folio_alloc_swap(folio)) {
-                               int __maybe_unused order = folio_order(folio);
+                       ret = folio_alloc_swap(folio);
+                       if (!folio_test_large(folio) || ret != -EAGAIN)
+                               goto activate_locked_split;
 
-                               if (!folio_test_large(folio))
-                                       goto activate_locked_split;
-                               /* Fallback to swap normal pages */
-                               if (split_folio_to_list(folio, folio_list))
-                                       goto activate_locked;
+                       /* Fallback to swap normal pages */
+                       if (split_folio_to_list(folio, folio_list))
+                               goto activate_locked;
 #ifdef CONFIG_TRANSPARENT_HUGEPAGE
-                               if (nr_pages >= HPAGE_PMD_NR) {
-                                       count_memcg_folio_events(folio,
+                       if (nr_pages >= HPAGE_PMD_NR) {
+                               count_memcg_folio_events(folio,
                                                THP_SWPOUT_FALLBACK, 1);
-                                       count_vm_event(THP_SWPOUT_FALLBACK);
-                               }
-#endif
-                               count_mthp_stat(order, MTHP_STAT_SWPOUT_FALLBACK);
-                               if (folio_alloc_swap(folio))
-                                       goto activate_locked_split;
+                               count_vm_event(THP_SWPOUT_FALLBACK);
                        }
+#endif
+                       count_mthp_stat(folio_order(folio), MTHP_STAT_SWPOUT_FALLBACK);
+                       if (folio_alloc_swap(folio))
+                               goto activate_locked_split;
+
                        /*
                         * Normally the folio will be dirtied in unmap because
                         * its pte should be dirty. A special case is MADV_FREE

Quoted 2 lines

Only case 2 requires splitting. __can_reclaim_anon_pages also checks demote which is not related to swap.

I actually extracted __can_reclaim_anon_pages(), which only checks swap, whereas can_reclaim_anon_pages() checks both swap and demotion. :-)