Skip to contentSkip to conversation

mm-khugepaged-generalize-hugepage_vma_revalidate-for-mthp-support.patch removed from -mm tree

merged mm-stable Andrew Morton 1 participant 1 message Jun 21

lore
Share & tools
lore

Andrew Morton

author tree notice Applied
#0 lore raw

The quilt patch titled
Subject: mm/khugepaged: generalize hugepage_vma_revalidate for mTHP support
has been removed from the -mm tree. Its filename was
mm-khugepaged-generalize-hugepage_vma_revalidate-for-mthp-support.patch

This patch was dropped because it was merged into the mm-stable branch of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm

------------------------------------------------------
From: Nico Pache <[email protected]>
Subject: mm/khugepaged: generalize hugepage_vma_revalidate for mTHP support
Date: Fri, 5 Jun 2026 10:14:08 -0600

Patch series "khugepaged: add mTHP collapse support", v19.

The following series provides khugepaged with the capability to collapse anonymous memory regions to mTHPs.

To achieve this we generalize the khugepaged functions to no longer depend on PMD_ORDER. Then during the PMD scan, we use a bitmap to track individual pages that are occupied (!none/zero). After the PMD scan is done, we use the bitmap to find the optimal mTHP sizes for the PMD range. The restriction on max_ptes_none is removed during the scan, to make sure we account for the whole PMD range in the bitmap. When no mTHP size is enabled, the legacy behavior of khugepaged is maintained.

We currently only support max_ptes_none values of 0 or HPAGE_PMD_NR - 1 (ie 511). If any other value is specified, the kernel will emit a warning and mTHP collapse will default to max_ptes_none=0. If a mTHP collapse is attempted, but contains swapped out, or shared pages, we don't perform the collapse.

It is now also possible to collapse to mTHPs without requiring the PMD THP size to be enabled. These limitations are to prevent collapse "creep" behavior. This prevents constantly promoting mTHPs to the next available size, which would occur because a collapse introduces more non-zero pages that would satisfy the promotion condition on subsequent scans.

Patch 1-2: Generalize hugepage_vma_revalidate and alloc_charge_folio
for arbitrary orders.

Patch 3:     Rework max_ptes_* handling into helper functions
Patch 4:     Generalize __collapse_huge_page_* for mTHP support
Patch 5:     Require collapse_huge_page to enter/exit with the lock dropped
Patch 6:     Generalize collapse_huge_page for mTHP collapse
Patch 7:     Skip collapsing mTHP to smaller orders
Patch 8-9:   Add per-order mTHP statistics and tracepoints
Patch 10:    Introduce collapse_possible_orders helper functions
Patch 11-13: Introduce bitmap and mTHP collapse support, fully enabled
Patch 14:    Documentation

Testing:
- Built for x86_64, aarch64, ppc64le, and s390x
- ran all arches on test suites provided by the kernel-tests project
- internal testing suites: functional testing and performance testing
- selftests mm
- I created a test script that I used to push khugepaged to its limits while monitoring a number of stats and tracepoints. The code is available here[1] (Run in legacy mode for these changes and set mthp sizes to inherit)
The summary from my testings was that there was no significant regression noticed through this test. In some cases my changes had better collapse latencies, and was able to scan more pages in the same amount of time/work, but for the most part the results were consistent.
- redis testing. I did some testing with these changes along with my defer changes (see followup [2] post for more details). We've decided to get the mTHP changes merged first before attempting the defer series.
- some basic testing on 64k page size.
- lots of general use.

This patch (of 14):

For khugepaged to support different mTHP orders, we must generalize this to check if the PMD is not shared by another VMA and that the order is enabled.

We cannot collapse VMA regions that do not span the full PMD. This is due to the potential of the PMD being shared by another VMA which leaves us vulnerable to race conditions if neighboring VMAs are resized. Always check the PMD order here to ensure its not shared by another VMA. We'd need to lock all VMAs in the PMD range to support this which may lead to increased lock contention and code complexity.

No functional change in this patch. Also correct a comment about the functionality of the revalidation and fix a double space issues.

Cc Alistair Popple <[email protected]>
Cc Andrea Arcangeli <[email protected]>
Cc Anshuman Khandual <[email protected]>
Cc Bagas Sanjaya <[email protected]>
Cc Barry Song <[email protected]>
Cc Brendan Jackman <[email protected]>
Cc Byungchul Park <[email protected]>
Cc Catalin Marinas <[email protected]>
Cc David Rientjes <[email protected]>
Cc Gregory Price <[email protected]>
Cc "Huang, Ying" <[email protected]>
Cc Hugh Dickins <[email protected]>
Cc Jan Kara <[email protected]>
Cc Jann Horn <[email protected]>
Cc Johannes Weiner <[email protected]>
Cc Jonathan Corbet <[email protected]>
Cc Joshua Hahn <[email protected]>
Cc Kefeng Wang <[email protected]>
Cc Liam R. Howlett <[email protected]>
Cc "Masami Hiramatsu (Google)" <[email protected]>
Cc Mathieu Desnoyers <[email protected]>
Cc Matthew Brost <[email protected]>
Cc Matthew Wilcox (Oracle) <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Mike Rapoport <[email protected]>
Cc Nanyong Sun <[email protected]>
Cc Pedro Falcato <[email protected]>
Cc Peter Xu <[email protected]>
Cc Rafael Aquini <[email protected]>
Cc Rakie Kim <[email protected]>
Cc Randy Dunlap <[email protected]>
Cc Ryan Roberts <[email protected]>
Cc Shivank Garg <[email protected]>
Cc Steven Rostedt <[email protected]>
Cc Suren Baghdasaryan <[email protected]>
Cc Takashi Iwai (SUSE) <[email protected]>
Cc Thomas Hellström <[email protected]>
Cc Usama Arif <[email protected]>
Cc Vishal Moola (Oracle) <[email protected]>
Cc Vlastimil Babka <[email protected]>
Cc Will Deacon <[email protected]>
Cc Yang Shi <[email protected]>
Cc Zach O'Keefe <[email protected]>

---

 mm/khugepaged.c |   26 ++++++++++++++++++--------
 1 file changed, 18 insertions(+), 8 deletions(-)
mm/khugepaged.c renamed from mm/khugepaged.c~mm-khugepaged-generalize-hugepage_vma_revalidate-for-mthp-support +18 -8
@@ -905,12 +905,13 @@ static int collapse_find_target_node(str
 
 /*
  * If mmap_lock temporarily dropped, revalidate vma
- * before taking mmap_lock.
+ * after taking the mmap_lock again.
  * Returns enum scan_result value.
  */
 
 static enum scan_result hugepage_vma_revalidate(struct mm_struct *mm, unsigned long address,
-		bool expect_anon, struct vm_area_struct **vmap, struct collapse_control *cc)
+		bool expect_anon, struct vm_area_struct **vmap,
+		struct collapse_control *cc, unsigned int order)
 {
 	struct vm_area_struct *vma;
 	enum tva_type type = cc->is_khugepaged ? TVA_KHUGEPAGED :
@@ -923,15 +924,22 @@ static enum scan_result hugepage_vma_rev
 	if (!vma)
 		return SCAN_VMA_NULL;
 
+	/*
+	 * We cannot collapse VMA regions that do not span the full PMD. This is
+	 * due to the potential of the PMD being shared by another VMA leaving
+	 * us vulnerable to a race condition. Always check the PMD order here to
+	 * ensure its not shared by another VMA. We'd need to lock all VMAs in
+	 * the PMD range to support this.
+	 */
 	if (!thp_vma_suitable_order(vma, address, PMD_ORDER))
 		return SCAN_ADDRESS_RANGE;
-	if (!thp_vma_allowable_order(vma, vma->vm_flags, type, PMD_ORDER))
+	if (!thp_vma_allowable_orders(vma, vma->vm_flags, type, BIT(order)))
 		return SCAN_VMA_CHECK;
 	/*
 	 * Anon VMA expected, the address may be unmapped then
 	 * remapped to file after khugepaged reaquired the mmap_lock.
 	 *
-	 * thp_vma_allowable_order may return true for qualified file
+	 * thp_vma_allowable_orders may return true for qualified file
 	 * vmas.
 	 */
 	if (expect_anon && (!(*vmap)->anon_vma || !vma_is_anonymous(*vmap)))
@@ -1129,7 +1137,8 @@ static enum scan_result collapse_huge_pa
 	}
 
 	mmap_read_lock(mm);
-	result = hugepage_vma_revalidate(mm, address, true, &vma, cc);
+	result = hugepage_vma_revalidate(mm, address, true, &vma, cc,
+					 HPAGE_PMD_ORDER);
 	if (result != SCAN_SUCCEED) {
 		mmap_read_unlock(mm);
 		goto out_nolock;
@@ -1163,7 +1172,8 @@ static enum scan_result collapse_huge_pa
 	 * mmap_lock.
 	 */
 	mmap_write_lock(mm);
-	result = hugepage_vma_revalidate(mm, address, true, &vma, cc);
+	result = hugepage_vma_revalidate(mm, address, true, &vma, cc,
+					 HPAGE_PMD_ORDER);
 	if (result != SCAN_SUCCEED)
 		goto out_up_write;
 	/* check if the pmd is still valid */
@@ -2866,8 +2876,8 @@ int madvise_collapse(struct vm_area_stru
 			mmap_unlocked = false;
 			*lock_dropped = true;
 			result = hugepage_vma_revalidate(mm, addr, false, &vma,
-							 cc);
-			if (result  != SCAN_SUCCEED) {
+							 cc, HPAGE_PMD_ORDER);
+			if (result != SCAN_SUCCEED) {
 				last_fail = result;
 				goto out_nolock;
 			}

_

Patches currently in -mm which might be from [email protected] are