mm/mglru: lazily activate folios while folios are really mapped
Subsystems (3)
RFC PATCH 1/1 mm/mglru: lazily activate folios while folios are really mapped
From: Barry Song <[email protected]>
MGLRU activates folios when a new folio is added and lru_gen_in_fault() returns true. The problem is that when a page fault occurs at address N, readahead may bring in many folios around N, and those folios are also activated even though many of them may never be accessed.
A previous attempt by Lei Liu proposed introducing a separate LRU for readahead[1], but that approach is likely over-designed.
This patch instead activates folios lazily, only when they are actually mapped, so that unused folios do not occupy higher- priority positions in the LRU and become harder to reclaim.
A similar optimization could also be applied to swapin readahead, but this RFC limits the change to file-backed folios for now.
Based on Tangquan's observations, this can significantly reduce file refaults on Android devices when using MGLRU.
BTW, it seems somewhat odd that all LRU APIs are defined in swap.c and swap.h.
[1] https://lore.kernel.org/linux-mm/[email protected]/
--- include/linux/swap.h | 1 + mm/filemap.c | 2 ++ mm/swap.c | 17 ++++++++++++++++- 3 files changed, 19 insertions(+), 1 deletion(-)
include/linux/swap.h +1 -0
@@ -335,6 +335,7 @@ void folio_add_lru(struct folio *);void folio_add_lru_vma (struct folio *,struct vm_area_struct *);void mark_page_accessed (struct page *);void folio_mark_accessed (struct folio *); +void folio_activate_on_mapped (struct folio *folio);static inline bool folio_may_be_lru_cached (struct folio *folio) {
mm/filemap.c +2 -0
@@ -3567,6 +3567,7 @@ vm_fault_t filemap_fault(struct vm_fault *vmf) } } +folio_activate_on_mapped (folio);if (!lock_folio_maybe_drop_mmap (vmf, folio, &fpin)) goto out_retry;
@@ -3926,6 +3927,7 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, nr_pages, &rss, &mmap_miss, file_end);folio_unlock (folio); +folio_activate_on_mapped (folio); }while ((folio =next_uptodate_folio (&xas, mapping, end_pgoff)) !=NULL );add_mm_counter (vma->vm_mm, folio_type, rss);pte_unmap_unlock (vmf->pte, vmf->ptl);
mm/swap.c +16 -1
@@ -488,6 +488,20 @@ void folio_mark_accessed(struct folio *folio) }EXPORT_SYMBOL (folio_mark_accessed); +void folio_activate_on_mapped (struct folio *folio) +{ +if (lru_gen_enabled () &&lru_gen_in_fault () && + !(current->flags &PF_MEMALLOC ) && + !folio_test_active (folio) && + !folio_test_unevictable (folio)) { +if (folio_test_lru (folio)) +folio_activate (folio); +else /* still in lru cache */ +__lru_cache_activate_folio (folio); + } +} +EXPORT_SYMBOL (folio_activate_on_mapped); +/** * folio_add_lru - Add a folio to an LRU list. * @folio: The folio to be added to the LRU.
@@ -506,7 +520,8 @@ void folio_add_lru(struct folio *folio)/* see the comment in lru_gen_folio_seq() */ if (lru_gen_enabled () && !folio_test_unevictable (folio) &&lru_gen_in_fault () && !(current->flags &PF_MEMALLOC )) -folio_set_active (folio); +if (!folio_is_file_lru (folio)) +folio_set_active (folio);folio_batch_add_and_move (folio, lru_add); }
Signature · 1 line
2.39.3 (Apple Git-146)