Skip to contentSkip to conversation

mm/khugepaged: fix PMD collapse swap PTE accounting

PATCH Applied Lance Yang 7 participants 15 messages Jun 9 – Jun 13

Subsystems (2)
lore
Share & tools
lore

Lance Yang

author reviewer
#0 lore raw

PATCH 1/1 mm/khugepaged: fix PMD collapse swap PTE accounting

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +8 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
 		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
 						      offset + nr_ptes);
 
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+
 		if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
 			enum scan_result ret;
 
Signature · 1 line
2.49.0
maintainer Acked-by on v1 1/1
#1 lore raw
Quoted 37 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +0 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
 		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,

mm/khugepaged.c static enum scan_result mthp_collapse(struct mm_struct *mm, · line 1518

Quoted 10 lines
mm/khugepaged.c +8 -0
 						      offset + nr_ptes);
 
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+

LGTM, there is a bit of opportunity for cleanup in the future :)

For example, as we no longer have the VMA here, collapse_max_ptes_none is imprecise in uffd VMAs. We might try collapsing where there sure is nothing to collapse.

We could likely handle the userfaultfd_armed() part easier: some indication that we must not have any pte_none() would be sufficient.

Also, I don't see a good reason why uffd would not be allowed to collapse with zeropages ... it's really just about missing faults due to pte_none().

Signature · 3 lines
Cheers,

David
#7 lore raw
In reply to Lance Yang
Quoted 20 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

Quoted 4 lines

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

Right, probably we just want to add the Fixes: tag once Andrew moves the series to mm-stable?

Signature · 3 lines
Cheers,

David
author reviewer
#8 lore raw
Quoted 21 lines · David Hildenbrand (Arm)
Quoted 20 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

Quoted 7 lines
Quoted 4 lines

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

Right, probably we just want to add the Fixes: tag once Andrew moves the series to mm-stable?

Yep, hopefully Andrew can add the Fixes: tag when applying this, once the series lands in mm-stable. Should be soon, I guess :P

Reviewed-by on v1 1/1
#11 lore raw
In reply to Lance Yang
Quoted 15 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Quoted 6 lines

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

LGTM, so:

Quoted 30 lines

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +8 -1
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
 		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
 						      offset + nr_ptes);
 
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+
 		if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
 			enum scan_result ret;
 
--
Quoted 2 lines

2.49.0

Thanks, Lorenzo

#2 lore raw
In reply to David Hildenbrand (Arm)
Quoted 40 lines · David Hildenbrand (Arm)
Quoted 39 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +0 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
 		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
 						      offset + nr_ptes);
 
Quoted 10 lines

mm/khugepaged.c static enum scan_result mthp_collapse(struct mm_struct *mm, · line 1518

Quoted 8 lines
mm/khugepaged.c +8 -0
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+

LGTM, there is a bit of opportunity for cleanup in the future :)

From my point of view, accepting the mTHP khugepaged changes was essentially a big compromise on how much it adds to the mess of the existing code base, and AFAIC we shouldn't accept any further major changes until we actually sort this mess out :)

Quoted 8 lines

For example, as we no longer have the VMA here, collapse_max_ptes_none is imprecise in uffd VMAs. We might try collapsing where there sure is nothing to collapse.

Quoted 5 lines

We could likely handle the userfaultfd_armed() part easier: some indication that we must not have any pte_none() would be sufficient.

Also, I don't see a good reason why uffd would not be allowed to collapse with zeropages ... it's really just about missing faults due to pte_none().

Ugh uffd.

Quoted 5 lines

--
Cheers,

David

Cheers, Lorenzo

author reviewer
#3 lore raw
In reply to David Hildenbrand (Arm)
Quoted 40 lines · David Hildenbrand (Arm)
Quoted 39 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

  mm/khugepaged.c | 8 ++++++++
  1 file changed, 8 insertions(+)
mm/khugepaged.c +0 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
  		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
  						      offset + nr_ptes);
 
Quoted 10 lines

mm/khugepaged.c static enum scan_result mthp_collapse(struct mm_struct *mm, · line 1518

Quoted 8 lines
mm/khugepaged.c +8 -0
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+

LGTM, there is a bit of opportunity for cleanup in the future :)

Yes, follow-up cleanup material :)

Quoted 1 line

Thanks!

Quoted 3 lines

For example, as we no longer have the VMA here, collapse_max_ptes_none is imprecise in uffd VMAs. We might try collapsing where there sure is nothing to collapse.

Oh, good catch. We may end up trying a collapse that cannot really go anywhere ... One for a follow-up.

Quoted 2 lines

We could likely handle the userfaultfd_armed() part easier: some indication that we must not have any pte_none() would be sufficient.

Right. By the time we get to mthp_collapse(), we probably only need to carry that as a small "no pte_none" constraint for the candidate range.

Quoted 2 lines

Also, I don't see a good reason why uffd would not be allowed to collapse with zeropages ... it's really just about missing faults due to pte_none().

Makes sense to me. I'll take a look when I get a chance. And yeah, as Lorenzo said, better to clean up the khugepaged mess first before piling more on top :)

Cheers, Lance

#4 lore raw
In reply to David Hildenbrand (Arm)
Quoted 59 lines · David Hildenbrand (Arm)
Quoted 37 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +0 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
              nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,

mm/khugepaged.c static enum scan_result mthp_collapse(struct mm_struct *mm, · line 1518

Quoted 10 lines
mm/khugepaged.c +8 -0
                                                    offset + nr_ptes);
 
+             /*
+              * Swap PTEs accepted during the scan are counted in @unmapped,
+              * not in the present-PTE bitmap. Account them for the PMD-order
+              * candidate.
+              */
+             if (is_pmd_order(order))
+                     nr_occupied_ptes += unmapped;
+

LGTM, there is a bit of opportunity for cleanup in the future :)

For example, as we no longer have the VMA here, collapse_max_ptes_none is imprecise in uffd VMAs. We might try collapsing where there sure is nothing to collapse.

Quoted 5 lines

We could likely handle the userfaultfd_armed() part easier: some indication that we must not have any pte_none() would be sufficient.

Also, I don't see a good reason why uffd would not be allowed to collapse with zeropages ... it's really just about missing faults due to pte_none().

I have some patches exactly for this :) so far 2-3 patches for better uffd handling

Quoted 6 lines

--
Cheers,

David

Acked-by on v1 1/1
#12 lore raw
In reply to Lance Yang
Quoted 21 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

Quoted 4 lines

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

Thanks for sending the fixup :)

Quoted 29 lines

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +8 -1
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
                nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
                                                      offset + nr_ptes);
 
+               /*
+                * Swap PTEs accepted during the scan are counted in @unmapped,
+                * not in the present-PTE bitmap. Account them for the PMD-order
+                * candidate.
+                */
+               if (is_pmd_order(order))
+                       nr_occupied_ptes += unmapped;
+
                if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
                        enum scan_result ret;
 
--

2.49.0

maintainer
#9 lore raw
In reply to Lance Yang
Quoted 10 lines · On Tue, 9 Jun 2026 21:56:13 +0800 Lance Yang <[email protected]> wrote:
Quoted 7 lines
Quoted 4 lines

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

Right, probably we just want to add the Fixes: tag once Andrew moves the series to mm-stable?

Yep, hopefully Andrew can add the Fixes: tag when applying this, once the series lands in mm-stable.

No probs.

Quoted 1 line

Should be soon, I guess :P

I move the second batch into mm-stable after Linus has merged the first batch, usually.

author reviewer
#10 lore raw
Quoted 7 lines · Andrew Morton
Quoted 5 lines · On Tue, 9 Jun 2026 21:56:13 +0800 Lance Yang <[email protected]> wrote:
Quoted 5 lines
Quoted 4 lines

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

Quoted 7 lines
Quoted 5 lines
Quoted 2 lines

Right, probably we just want to add the Fixes: tag once Andrew moves the series to mm-stable?

Yep, hopefully Andrew can add the Fixes: tag when applying this, once the series lands in mm-stable.

No probs.

Thanks!

Quoted 4 lines
Quoted 1 line

Should be soon, I guess :P

I move the second batch into mm-stable after Linus has merged the first batch, usually.

Okay, got it :)

author reviewer
#5 lore raw
In reply to Nico Pache
Quoted 60 lines · Nico Pache
Quoted 59 lines · David Hildenbrand (Arm)
Quoted 37 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

  mm/khugepaged.c | 8 ++++++++
  1 file changed, 8 insertions(+)
mm/khugepaged.c +0 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
               nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,

mm/khugepaged.c static enum scan_result mthp_collapse(struct mm_struct *mm, · line 1518

Quoted 10 lines
mm/khugepaged.c +8 -0
                                                     offset + nr_ptes);
 
+             /*
+              * Swap PTEs accepted during the scan are counted in @unmapped,
+              * not in the present-PTE bitmap. Account them for the PMD-order
+              * candidate.
+              */
+             if (is_pmd_order(order))
+                     nr_occupied_ptes += unmapped;
+

LGTM, there is a bit of opportunity for cleanup in the future :)

For example, as we no longer have the VMA here, collapse_max_ptes_none is imprecise in uffd VMAs. We might try collapsing where there sure is nothing to collapse.

Quoted 8 lines
Quoted 5 lines

We could likely handle the userfaultfd_armed() part easier: some indication that we must not have any pte_none() would be sufficient.

Also, I don't see a good reason why uffd would not be allowed to collapse with zeropages ... it's really just about missing faults due to pte_none().

I have some patches exactly for this :) so far 2-3 patches for better uffd handling

Neat, I'll leave that to you then :) Looking forward to seeing those patches!

Cheers, Lance

reviewer Reviewed-by on v1 1/1
#13 lore raw
In reply to Lance Yang
Quoted 15 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Quoted 7 lines

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---

LGTM. Feel free to add:

Quoted 28 lines

Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

  mm/khugepaged.c | 8 ++++++++
  1 file changed, 8 insertions(+)
mm/khugepaged.c +8 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
  		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,
  						      offset + nr_ptes);
 
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+
  		if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
  			enum scan_result ret;
 
Reviewed-by on v1 1/1
#14 lore raw
In reply to Lance Yang
Quoted 37 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

mm/khugepaged.c | 8 ++++++++

1 file changed, 8 insertions(+)
mm/khugepaged.c +0 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
 		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,

mm/khugepaged.c static enum scan_result mthp_collapse(struct mm_struct *mm, · line 1518

Quoted 10 lines
mm/khugepaged.c +8 -0
 						      offset + nr_ptes);
 
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+

Nice catch, I didn't spot it.

Quoted 6 lines
mm/khugepaged.c +0 -0
 		if (nr_occupied_ptes >= nr_ptes - max_ptes_none) {
 			enum scan_result ret;
 
Signature · 1 line
2.49.0
Signature · 2 lines
Wei Yang
Help you, Help me
#6 lore raw
In reply to David Hildenbrand (Arm)
Quoted 51 lines · David Hildenbrand (Arm)
Quoted 37 lines · Lance Yang

From: Lance Yang <[email protected]>

mthp_collapse() uses mthp_present_ptes to decide whether a range has enough occupied PTEs to try collapse. Swap PTEs accepted by collapse_scan_pmd() are counted in unmapped, but are not represented in mthp_present_ptes.

When lower orders are enabled, collapse_scan_pmd() relaxes max_ptes_none so the scan can cover the whole PMD and build the bitmap. mthp_collapse() then checks the PMD-order candidate using the bitmap.

With max_ptes_none set to 0, a range with 511 present PTEs and one swap PTE no longer reaches collapse_huge_page(), even though PMD collapse can handle swap PTEs up to max_ptes_swap.

Account unmapped PTEs only for PMD order. PMD collapse supports swap PTEs through max_ptes_swap, while lower-order mTHP collapse does not currently support non-present PTEs. Keep non-present PTEs out of the lower-order eligibility check.

---
Sent separately, as discussed in [1], to spell out the PMD-order swap PTE case. Patch [2] is still only in mm-unstable, so no Fixes: tag.

[1] https://lore.kernel.org/linux-mm/CAA1CXcD7WAiA1b9GTLAuNZ+kHaFx0SzZwpBkqAZ=s+RHsTUaow@mail.gmail.com/
[2] https://lore.kernel.org/linux-mm/[email protected]/

 mm/khugepaged.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/khugepaged.c +0 -0
index b12187709f6d..617bca76db49 100644
@@ -1508,6 +1508,14 @@ static enum scan_result mthp_collapse(struct mm_struct *mm,
 		nr_occupied_ptes = bitmap_weight_from(cc->mthp_present_ptes, offset,

mm/khugepaged.c static enum scan_result mthp_collapse(struct mm_struct *mm, · line 1518

Quoted 10 lines
mm/khugepaged.c +8 -0
 						      offset + nr_ptes);
 
+		/*
+		 * Swap PTEs accepted during the scan are counted in @unmapped,
+		 * not in the present-PTE bitmap. Account them for the PMD-order
+		 * candidate.
+		 */
+		if (is_pmd_order(order))
+			nr_occupied_ptes += unmapped;
+

LGTM, there is a bit of opportunity for cleanup in the future :)

Quoted 6 lines

For example, as we no longer have the VMA here, collapse_max_ptes_none is imprecise in uffd VMAs. We might try collapsing where there sure is nothing to collapse.

I mentioned it in [1] too.

[1]: https://lore.kernel.org/linux-mm/20260512154431.jxcs632mqqatqtsw@master/

Quoted 11 lines

We could likely handle the userfaultfd_armed() part easier: some indication that we must not have any pte_none() would be sufficient.

Also, I don't see a good reason why uffd would not be allowed to collapse with zeropages ... it's really just about missing faults due to pte_none().

Signature · 3 lines
Cheers,

David
Signature · 2 lines
Wei Yang
Help you, Help me