Skip to content

Optimizations for khugepaged

Dev Jain linux-mm lkml 4 versions – Applied

Status from this message: [merged mm-stable] khugepaged-optimize-collapse_pte_mapped_thp-by-pte-batching.patch removed from -mm tree

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v4 Latest3 patches 10 repliesApplied 3/3
  2. v33 patches 8 repliesSuperseded
  3. v23 patches 24 repliesSuperseded
Earlier versions · 1
VersionPostedPatchesRepliesStatus
v11 patch10 repliesSuperseded

Changes in v4 from the cover letter

- Use unsigned int for nr_ptes and max_nr_ptes (David)
- Define the functions in patch 1 as inline functions with kernel docs instead of macros (akpm)

Patches in v4 All 3 applied

#Subject and filesSizeRepliesApplied
1/3mm: add get_and_clear_ptes() and clear_ptes()arch/arm64/mm/mmu.c include/linux/pgtable.h mm/mremap.c +1 more+48 -33 repliesApplied
2/3khugepaged: Optimize __collapse_huge_page_copy_succeeded() by PTE batchingmm/khugepaged.c+18 -75 repliesApplied
3/3khugepaged: Optimize collapse_pte_mapped_thp() by PTE batchingmm/khugepaged.c+21 -122 repliesApplied

Reviews trailers per patch

All 3 patches have a Reviewed-by or Acked-by.

PatchBaolin WangDavid HildenbrandBarry SongLorenzo StoakesZi Yan
1/3R-b R-b R-b R-b
2/3R-b A-b R-b R-b
3/3R-b A-b R-b R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 5 reviewers.

Read the v4 thread

Cover letter [PATCH v4 0/3] Optimizations for khugepaged

If the underlying folio mapped by the ptes is large, we can process those ptes in a batch using folio_pte_batch().

For arm64 specifically, this results in a 16x reduction in the number of ptep_get() calls, since on a contig block, ptep_get() on arm64 will iterate through all 16 entries to collect a/d bits. Next, ptep_clear() will cause a TLBI for every contig block in the range via contpte_try_unfold(). Instead, use clear_ptes() to only do the TLBI at the first and last contig block of the range.

For split folios, there will be no pte batching; the batch size returned by folio_pte_batch() will be 1. For pagetable split folios, the ptes will still point to the same large folio; for arm64, this results in the optimization described above, and for other arches, a minor improvement is expected due to a reduction in the number of function calls and batching atomic operations.

---
Rebased on today's mm-new - the v3 of this patchset was already in, so I reverted those commits and then rebased on top of that. mm-selftests pass.

v3->v4:
- Use unsigned int for nr_ptes and max_nr_ptes (David)
- Define the functions in patch 1 as inline functions with kernel docs instead of macros (akpm)

v2->v3:
- Drop patch 3 (was merged separately)
- Add patch 1 (David)
- Coding style change, drop mapped_folio (Lorenzo)

v1->v2:
- Use for loop instead of do-while loop (Lorenzo)
- Remove folio_test_large check since the subpage-check condition will imply that (Baolin)
- Combine patch 1 and 2 into this series, add new patch 3

David Hildenbrand (1):
mm: add get_and_clear_ptes() and clear_ptes()

Dev Jain (2):
khugepaged: Optimize __collapse_huge_page_copy_succeeded() by PTE
batching
khugepaged: Optimize collapse_pte_mapped_thp() by PTE batching

 arch/arm64/mm/mmu.c     |  2 +-
 include/linux/pgtable.h | 45 ++++++++++++++++++++++++++++++++
 mm/khugepaged.c         | 58 +++++++++++++++++++++++++++--------------
 mm/mremap.c             |  2 +-
 mm/rmap.c               |  2 +-
 5 files changed, 87 insertions(+), 22 deletions(-)
Signature · 1 line
2.30.2