promote mapped executable folios after first usage for MGLRU
Subsystems (4)
Baolin Wang
reviewerv4 cover promote mapped executable folios after first usage for MGLRU
Now MGLRU's protection of mapped executable file folios is less reliable. Follow the classical LRU's logic, promoting mapped executable file folios after their first usage to give executable code a better chance to stay in memory and improve workload performance (See patch 2 for more details).
Changes from v3:
https://lore.kernel.org/all/[email protected]/
- Collect tags from Johannes, Axel, Barry and Kairui. Thanks.
- Split the helper addition out into a separate patch (per Johannes).
- Set PG_workingset before promoting in lru_gen_set_refs() (per Kairui).
- Some code cleanups (per Kairui).
Changes from v2:
https://lore.kernel.org/all/[email protected]/
- Add acked tag from David. Thanks.
- Rename the exec-file-folio helper and move it into vmscan.c (per David).
Changes from v1:
https://lore.kernel.org/all/4b921ed528c483e13c9e22d1ae44ba58b4a15b0b.1784096432.git.baolin.wang@linux.alibaba.com/
- Add a new patch to clean up vma flags as preparation.
- Add a new helper to check exec file folios (per Kaisui).
- Promote exec file folios in lru_gen_set_refs (Per Sashiko).
Baolin Wang (3):
mm: vmscan: convert folio_referenced() to use vma_flags_t
mm: vmscan: add a helper to identify file-backed executable folios
mm: mglru: promote mapped executable folios after first usage
include/linux/rmap.h | 7 +++-- mm/rmap.c | 19 +++++++------ mm/vmscan.c | 63 ++++++++++++++++++++++++++++---------------- 3 files changed, 54 insertions(+), 35 deletions(-)
Signature · 1 line
2.47.3
Baolin Wang
reviewerPATCH v4 1/3 mm: vmscan: convert folio_referenced() to use vma_flags_t
Replace use of the legacy vm_flags_t flags with vma_flags_t values for folio_referenced() and related logic. This is also a preparation for the following changes.
No functional changes.
--- include/linux/rmap.h | 7 +++---- mm/rmap.c | 19 +++++++++++-------- mm/vmscan.c | 14 +++++++------- 3 files changed, 21 insertions(+), 19 deletions(-)
include/linux/rmap.h +3 -4
@@ -843,7 +843,7 @@ static inline int folio_try_share_anon_rmap_pmd(struct folio *folio,* Called from mm/vmscan.c to handle paging out */ int folio_referenced (struct folio *,int is_locked, -struct mem_cgroup *memcg,vm_flags_t *vm_flags); +struct mem_cgroup *memcg,vma_flags_t *vma_flags);void try_to_migrate (struct folio *folio,enum ttu_flags flags);void try_to_unmap (struct folio *,enum ttu_flags flags);
@@ -975,10 +975,9 @@ struct anon_vma *folio_lock_anon_vma_read(const struct folio *folio,#define anon_vma_prepare (vma) (0)static inline int folio_referenced (struct folio *folio,int is_locked, -struct mem_cgroup *memcg, -vm_flags_t *vm_flags) +struct mem_cgroup *memcg,vma_flags_t *vma_flags) { - *vm_flags =0 ; +vma_flags_clear_all (vma_flags);return 0 ; }
mm/rmap.c +11 -8
@@ -907,7 +907,7 @@ pmd_t *mm_find_pmd(struct mm_struct *mm, unsigned long address)struct folio_referenced_arg {int mapcount;int referenced; -vm_flags_t vm_flags; +vma_flags_t vma_flags;struct mem_cgroup *memcg; };
@@ -926,7 +926,7 @@ static bool folio_referenced_one(struct folio *folio, address = pvmw.address; nr =1 ; -if (vma->vm_flags &VM_LOCKED ) { +if (vma_test (vma,VMA_LOCKED_BIT )) { ptes++; pra->mapcount--;
@@ -947,7 +947,7 @@ static bool folio_referenced_one(struct folio *folio,/* Restore the mlock which got missed */ mlock_vma_folio (folio, vma);page_vma_mapped_walk_done (&pvmw); - pra->vm_flags |=VM_LOCKED ; +vma_flags_set (&pra->vma_flags,VMA_LOCKED_BIT );return false;/* To break the loop */ }
@@ -1015,8 +1015,11 @@ static bool folio_referenced_one(struct folio *folio, referenced++;if (referenced) { +vma_flags_t vma_flags = vma->flags; + pra->referenced++; - pra->vm_flags |= vma->vm_flags & ~VM_LOCKED ; +vma_flags_clear (&vma_flags,VMA_LOCKED_BIT ); +vma_flags_set_mask (&pra->vma_flags, vma_flags); }if (!pra->mapcount)
@@ -1054,7 +1057,7 @@ static bool invalid_folio_referenced_vma(struct vm_area_struct *vma, void *arg)* @folio: The folio to test. * @is_locked: Caller holds lock on the folio. * @memcg: target memory cgroup -* @vm_flags: A combination of all the vma->vm_flags which referenced the folio. +* @vma_flags: A combination of all the vma->flags which referenced the folio. * * Quick test_and_clear_referenced for all mappings of a folio, *
@@ -1062,7 +1065,7 @@ static bool invalid_folio_referenced_vma(struct vm_area_struct *vma, void *arg)* the function bailed out due to rmap lock contention. */ int folio_referenced (struct folio *folio,int is_locked, -struct mem_cgroup *memcg,vm_flags_t *vm_flags) +struct mem_cgroup *memcg,vma_flags_t *vma_flags) {bool we_locked = false;struct folio_referenced_arg pra = {
@@ -1078,7 +1081,7 @@ int folio_referenced(struct folio *folio, int is_locked, };VM_WARN_ON_ONCE_FOLIO (folio_is_zone_device (folio), folio); - *vm_flags =0 ; +vma_flags_clear_all (vma_flags);if (!pra.mapcount)return 0 ;
@@ -1092,7 +1095,7 @@ int folio_referenced(struct folio *folio, int is_locked, }rmap_walk (folio, &rwc); - *vm_flags = pra.vm_flags; +vma_flags_set_mask (vma_flags, pra.vma_flags);if (we_locked)folio_unlock (folio);
mm/vmscan.c +7 -7
@@ -861,16 +861,16 @@ static enum folio_references folio_check_references(struct folio *folio, struct scan_control *sc) {int referenced_ptes, referenced_folio; -vm_flags_t vm_flags; +vma_flags_t vma_flags; referenced_ptes =folio_referenced (folio,1 , sc->target_mem_cgroup, - &vm_flags); + &vma_flags);/* * The supposedly reclaimable folio was found to be in a VM_LOCKED vma. * Let the folio, now marked Mlocked, be moved to the unevictable list. */ -if (vm_flags &VM_LOCKED ) +if (vma_flags_test (&vma_flags,VMA_LOCKED_BIT ))return FOLIOREF_ACTIVATE ;/*
@@ -914,7 +914,7 @@ static enum folio_references folio_check_references(struct folio *folio,/* * Activate file-backed executable folios after first usage. */ -if ((vm_flags &VM_EXEC ) &&folio_is_file_lru (folio)) +if (vma_flags_test (&vma_flags,VMA_EXEC_BIT ) &&folio_is_file_lru (folio))return FOLIOREF_ACTIVATE ;return FOLIOREF_KEEP ;
@@ -2063,7 +2063,7 @@ static void shrink_active_list(unsigned long nr_to_scan, {unsigned long nr_taken;unsigned long nr_scanned; -vm_flags_t vm_flags; +vma_flags_t vma_flags;LIST_HEAD (l_hold);/* The folios which were snipped off */ LIST_HEAD (l_active);LIST_HEAD (l_inactive);
@@ -2107,7 +2107,7 @@ static void shrink_active_list(unsigned long nr_to_scan,/* Referenced or rmap lock contention: rotate */ if (folio_referenced (folio,0 , sc->target_mem_cgroup, - &vm_flags) !=0 ) { + &vma_flags) !=0 ) {/* * Identify referenced, file-backed active folios and * give them one more trip around the active list. So
@@ -2117,7 +2117,7 @@ static void shrink_active_list(unsigned long nr_to_scan,* IO, plus JVM can create lots of anon VM_EXEC folios, * so we ignore them here. */ -if ((vm_flags &VM_EXEC ) &&folio_is_file_lru (folio)) { +if (vma_flags_test (&vma_flags,VMA_EXEC_BIT ) &&folio_is_file_lru (folio)) { nr_rotated +=folio_nr_pages (folio);list_add (&folio->lru, &l_active);continue ;
Signature · 1 line
2.47.3
Baolin Wang
reviewerPATCH v4 2/3 mm: vmscan: add a helper to identify file-backed executable folios
In reply to Baolin WangAdd a helper to identify file-backed executable folios to avoid duplicate code.
No functional changes.
--- mm/vmscan.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
mm/vmscan.c +8 -2
@@ -268,6 +268,12 @@ static int sc_swappiness(struct scan_control *sc, struct mem_cgroup *memcg) }#endif +static inline bool is_exec_file_folio (const struct folio *folio, +const vma_flags_t *vma_flags) +{ +return vma_flags_test (vma_flags,VMA_EXEC_BIT ) &&folio_is_file_lru (folio); +} +static void set_task_reclaim_state (struct task_struct *task,struct reclaim_state *rs) {
@@ -914,7 +920,7 @@ static enum folio_references folio_check_references(struct folio *folio,/* * Activate file-backed executable folios after first usage. */ -if (vma_flags_test (&vma_flags,VMA_EXEC_BIT ) &&folio_is_file_lru (folio)) +if (is_exec_file_folio (folio, &vma_flags))return FOLIOREF_ACTIVATE ;return FOLIOREF_KEEP ;
@@ -2117,7 +2123,7 @@ static void shrink_active_list(unsigned long nr_to_scan,* IO, plus JVM can create lots of anon VM_EXEC folios, * so we ignore them here. */ -if (vma_flags_test (&vma_flags,VMA_EXEC_BIT ) &&folio_is_file_lru (folio)) { +if (is_exec_file_folio (folio, &vma_flags)) { nr_rotated +=folio_nr_pages (folio);list_add (&folio->lru, &l_active);continue ;
Signature · 1 line
2.47.3
Re: [PATCH v4 2/3] mm: vmscan: add a helper to identify file-backed executable folios
Quoted 9 lines · Baolin Wang
Add a helper to identify file-backed executable folios to avoid duplicate code.
No functional changes.
Signed-off-by Baolin Wang <[email protected]>Acked-by Johannes Weiner <[email protected]>Reviewed-by Axel Rasmussen <[email protected]>---
Signature · 3 lines
Cheers, David
Re: [PATCH v4 2/3] mm: vmscan: add a helper to identify file-backed executable folios
In reply to Baolin WangQuoted 4 lines · Baolin Wang
Add a helper to identify file-backed executable folios to avoid duplicate code.
Quoted 9 lines
No functional changes.
Signed-off-by Baolin Wang <[email protected]>Acked-by Johannes Weiner <[email protected]>Reviewed-by Axel Rasmussen <[email protected]>--- mm/vmscan.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
Re: [PATCH v4 2/3] mm: vmscan: add a helper to identify file-backed executable folios
In reply to Baolin WangQuoted 10 lines · Baolin Wang
Add a helper to identify file-backed executable folios to avoid duplicate code.
No functional changes.
Signed-off-by Baolin Wang <[email protected]>Acked-by Johannes Weiner <[email protected]>Reviewed-by Axel Rasmussen <[email protected]>---
Baolin Wang
reviewerPATCH v4 3/3 mm: mglru: promote mapped executable folios after first usage
In reply to Baolin WangClassical LRU protects mapped executable file folios through commit 8cab4754d24a0 ("vmscan: make mapped executable pages the first class citizen") and commit c909e99364c8 ("vmscan: activate executable pages after first usage"), giving executable code a better chance to stay in memory, avoiding IO thrashing and improving workload performance.
However, MGLRU's protection of mapped executable file folios is less reliable. Although shrink_folio_list() checks references, the access flag of mapped executable file folios may have already been checked and cleared by lru_gen_look_around() or walk_mm(). Additionally, folio_update_gen() or lru_gen_set_refs() only sets the 'PG_referenced' flag for mapped executable file folios, which causes shrink_folio_list() to ignore the first usage of these mapped executable file folios and reclaim them easily.
Follow the classical LRU's logic, promoting mapped executable file folios after their first usage in folio_update_gen() and lru_gen_set_refs(), giving executable code a better chance to stay in memory.
On my 32-core Arm machine, with the memcg limit set to 2G, running 'make -j32' to build kernel showed some improvement in sys time.
base patched
9248.543s 7861.579s
--- mm/vmscan.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-)
mm/vmscan.c +27 -16
@@ -841,10 +841,16 @@ enum folio_references {* with PG_active set. In contrast, the aging (page table walk) path uses * folio_update_gen(). */ -static bool lru_gen_set_refs (struct folio *folio) +static bool lru_gen_set_refs (struct folio *folio,const vma_flags_t *vma_flags) {/* see the comment on LRU_REFS_FLAGS */ if (!folio_test_referenced (folio) && !folio_test_workingset (folio)) { +/* Activate file-backed executable folios after first usage. */ +if (is_exec_file_folio (folio, vma_flags)) { +set_mask_bits (&folio->flags.f,LRU_REFS_FLAGS ,BIT (PG_workingset)); +return true; + } +set_mask_bits (&folio->flags.f,LRU_REFS_MASK ,BIT (PG_referenced));return false; }
@@ -857,7 +863,7 @@ static bool lru_gen_set_refs(struct folio *folio) return true; }#else -static bool lru_gen_set_refs (struct folio *folio) +static bool lru_gen_set_refs (struct folio *folio,const vma_flags_t *vma_flags) {return false; }
@@ -892,7 +898,7 @@ static enum folio_references folio_check_references(struct folio *folio,if (!referenced_ptes)return FOLIOREF_RECLAIM ; -return lru_gen_set_refs (folio) ?FOLIOREF_ACTIVATE :FOLIOREF_KEEP ; +return lru_gen_set_refs (folio, &vma_flags) ?FOLIOREF_ACTIVATE :FOLIOREF_KEEP ; } referenced_folio =folio_test_clear_referenced (folio);
@@ -3192,14 +3198,19 @@ static bool positive_ctrl_err(struct ctrl_pos *sp, struct ctrl_pos *pv)******************************************************************************/ /* promote pages accessed through page tables */ -static int folio_update_gen (struct folio *folio,int gen) +static int folio_update_gen (struct folio *folio,int gen,const vma_flags_t *vma_flags) {unsigned long new_flags, old_flags =READ_ONCE (folio->flags.f);VM_WARN_ON_ONCE (gen >=MAX_NR_GENS ); -/* see the comment on LRU_REFS_FLAGS */ -if (!folio_test_referenced (folio) && !folio_test_workingset (folio)) { +/* +* See the comment on LRU_REFS_FLAGS, and activate file-backed +* executable folios after first usage to avoid typical IO +* thrashing from reclaiming. +*/ +if (!folio_test_referenced (folio) && !folio_test_workingset (folio) && + !is_exec_file_folio (folio, vma_flags)) {set_mask_bits (&folio->flags.f,LRU_REFS_MASK ,BIT (PG_referenced));return -1 ; }
@@ -3432,8 +3443,8 @@ static bool suitable_to_scan(int total, int young)return young * n >= total; } -static void walk_update_folio (struct lru_gen_mm_walk *walk,struct folio *folio, -int new_gen,bool dirty) +static void walk_update_folio (struct lru_gen_mm_walk *walk,struct vm_area_struct *vma, +struct folio *folio,int new_gen,bool dirty) {int old_gen;
@@ -3446,10 +3457,10 @@ static void walk_update_folio(struct lru_gen_mm_walk *walk, struct folio *folio,folio_mark_dirty (folio);if (walk) { - old_gen =folio_update_gen (folio, new_gen); + old_gen =folio_update_gen (folio, new_gen, &vma->flags);if (old_gen >=0 && old_gen != new_gen)update_batch_size (walk, folio, old_gen, new_gen); - }else if (lru_gen_set_refs (folio)) { + }else if (lru_gen_set_refs (folio, &vma->flags)) { old_gen =folio_lru_gen (folio);if (old_gen >=0 && old_gen != new_gen)folio_activate (folio);
@@ -3522,7 +3533,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end,continue ;if (last != folio) { -walk_update_folio (walk, last, gen, dirty); +walk_update_folio (walk, args->vma, last, gen, dirty); last = folio; dirty = false;
@@ -3535,7 +3546,7 @@ static bool walk_pte_range(pmd_t *pmd, unsigned long start, unsigned long end, walk->mm_stats[MM_LEAF_YOUNG ] += nr; } -walk_update_folio (walk, last, gen, dirty); +walk_update_folio (walk, args->vma, last, gen, dirty); last =NULL ;if (i <PTRS_PER_PTE &&get_next_vma (PMD_MASK ,PAGE_SIZE , args, &start, &end))
@@ -3613,7 +3624,7 @@ static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area goto next;if (last != folio) { -walk_update_folio (walk, last, gen, dirty); +walk_update_folio (walk, vma, last, gen, dirty); last = folio; dirty = false;
@@ -3627,7 +3638,7 @@ static void walk_pmd_range_locked(pud_t *pud, unsigned long addr, struct vm_area i = i >MIN_LRU_BATCH ?0 :find_next_bit (bitmap,MIN_LRU_BATCH , i) +1 ; }while (i <=MIN_LRU_BATCH ); -walk_update_folio (walk, last, gen, dirty); +walk_update_folio (walk, vma, last, gen, dirty);lazy_mmu_mode_disable ();spin_unlock (ptl);
@@ -4262,7 +4273,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr)continue ;if (last != folio) { -walk_update_folio (walk, last, gen, dirty); +walk_update_folio (walk, vma, last, gen, dirty); last = folio; dirty = false;
@@ -4274,7 +4285,7 @@ bool lru_gen_look_around(struct page_vma_mapped_walk *pvmw, unsigned int nr) young += nr; } -walk_update_folio (walk, last, gen, dirty); +walk_update_folio (walk, vma, last, gen, dirty);lazy_mmu_mode_disable ();
Signature · 1 line
2.47.3
Barry Song
reviewerRe: [PATCH v4 3/3] mm: mglru: promote mapped executable folios after first usage
Quoted 42 lines · Baolin Wang
Classical LRU protects mapped executable file folios through commit
8cab4754d24a0("vmscan: make mapped executable pages the first class citizen") and commitc909e99364c8("vmscan: activate executable pages after first usage"), giving executable code a better chance to stay in memory, avoiding IO thrashing and improving workload performance.However, MGLRU's protection of mapped executable file folios is less reliable. Although
shrink_folio_list()checks references, the access flag of mapped executable file folios may have already been checked and cleared bylru_gen_look_around()orwalk_mm(). Additionally,folio_update_gen()orlru_gen_set_refs()only sets the 'PG_referenced' flag for mapped executable file folios, which causesshrink_folio_list()to ignore the first usage of these mapped executable file folios and reclaim them easily.Follow the classical LRU's logic, promoting mapped executable file folios after their first usage in
folio_update_gen()andlru_gen_set_refs(), giving executable code a better chance to stay in memory.On my 32-core Arm machine, with the memcg limit set to 2G, running 'make -j32' to build kernel showed some improvement in sys time.
base patched
9248.543s 7861.579sSigned-off-by Baolin Wang <[email protected]>Acked-by Johannes Weiner <[email protected]>Reviewed-by Axel Rasmussen <[email protected]>--- mm/vmscan.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-)mm/vmscan.c +0
-1@@ -841,10 +841,16 @@ enum folio_references {* with PG_active set. In contrast, the aging (page table walk) path uses * folio_update_gen(). */ -static bool lru_gen_set_refs (struct folio *folio)
mm/vmscan.c enum folio_references { · line 853
Quoted 10 lines
mm/vmscan.c +7
-0+static bool lru_gen_set_refs (struct folio *folio,const vma_flags_t *vma_flags) {/* see the comment on LRU_REFS_FLAGS */ if (!folio_test_referenced (folio) && !folio_test_workingset (folio)) { +/* Activate file-backed executable folios after first usage. */ +if (is_exec_file_folio (folio, vma_flags)) { +set_mask_bits (&folio->flags.f,LRU_REFS_FLAGS ,BIT (PG_workingset)); +return true; + } +
Hi Baolin,
Do you see any performance difference if you change the code to something like the following?
-static bool lru_gen_set_refs(struct folio *folio)
+static bool lru_gen_set_refs(struct folio *folio, const vma_flags_t *vma_flags)
{
+ /* Activate file-backed executable folios after first usage. */
+ if (is_exec_file_folio(folio, vma_flags)) {
+ set_mask_bits(&folio->flags.f, LRU_REFS_FLAGS,BIT(PG_workingset));
+ return true;
+ }
+
Best Regards
Barry
Baolin Wang
reviewerRe: [PATCH v4 3/3] mm: mglru: promote mapped executable folios after first usage
Quoted 67 lines · Barry Song
Quoted 42 lines · Baolin Wang
Classical LRU protects mapped executable file folios through commit
8cab4754d24a0("vmscan: make mapped executable pages the first class citizen") and commitc909e99364c8("vmscan: activate executable pages after first usage"), giving executable code a better chance to stay in memory, avoiding IO thrashing and improving workload performance.However, MGLRU's protection of mapped executable file folios is less reliable. Although
shrink_folio_list()checks references, the access flag of mapped executable file folios may have already been checked and cleared bylru_gen_look_around()orwalk_mm(). Additionally,folio_update_gen()orlru_gen_set_refs()only sets the 'PG_referenced' flag for mapped executable file folios, which causesshrink_folio_list()to ignore the first usage of these mapped executable file folios and reclaim them easily.Follow the classical LRU's logic, promoting mapped executable file folios after their first usage in
folio_update_gen()andlru_gen_set_refs(), giving executable code a better chance to stay in memory.On my 32-core Arm machine, with the memcg limit set to 2G, running 'make -j32' to build kernel showed some improvement in sys time.
base patched
9248.543s 7861.579sSigned-off-by Baolin Wang <[email protected]>Acked-by Johannes Weiner <[email protected]>Reviewed-by Axel Rasmussen <[email protected]>--- mm/vmscan.c | 43 +++++++++++++++++++++++++++---------------- 1 file changed, 27 insertions(+), 16 deletions(-)mm/vmscan.c +0
-1@@ -841,10 +841,16 @@ enum folio_references {* with PG_active set. In contrast, the aging (page table walk) path uses * folio_update_gen(). */ -static bool lru_gen_set_refs (struct folio *folio)mm/vmscan.c enum folio_references { · line 853
Quoted 10 lines
mm/vmscan.c +7
-0+static bool lru_gen_set_refs (struct folio *folio,const vma_flags_t *vma_flags) {/* see the comment on LRU_REFS_FLAGS */ if (!folio_test_referenced (folio) && !folio_test_workingset (folio)) { +/* Activate file-backed executable folios after first usage. */ +if (is_exec_file_folio (folio, vma_flags)) { +set_mask_bits (&folio->flags.f,LRU_REFS_FLAGS ,BIT (PG_workingset)); +return true; + } +Hi Baolin,
Do you see any performance difference if you change the code to something like the following?
mm/vmscan.c +4
-1-static bool lru_gen_set_refs (struct folio *folio) +static bool lru_gen_set_refs (struct folio *folio,const vma_flags_t *vma_flags) { +/* Activate file-backed executable folios after first usage. */ +if (is_exec_file_folio (folio, vma_flags)) { + set_mask_bits(&folio->flags.f, LRU_REFS_FLAGS,
BIT(PG_workingset));
mm/vmscan.c enum folio_references { · line 852
Quoted 2 lines
mm/vmscan.c +2
-0+return true; + }
I tried that, and there was no obvious difference. I think the original logic also promotes the exec folio when it is accessed again, so I prefer to keep the same logic as folio_update_gen().
As I mentioned earlier[1], lru_gen_set_refs() needs some rework, and I'll think about how to optimize this function later.
Hi Baolin,
We backport and test an Android handset(qcom8850, 6.12, 12 GB). Patch 1's vma_flags_t isn't on 6.12, so we kept only the core check in its legacy form:
(vma->vm_flags & VM_EXEC)
Since this is a backport, the numbers below just show the direction of change. Overall the patch does well, and ran for several hours with no crashes or hangs:
Camera (launch + capture, 22 background apps resident):
cold-start p50 -6.3% capture on par memavail at startup +4 .. +7%
Reclaim over the same window, as the patch intends:
pgsteal_kswapd -46.5% pgscan_kswapd -52.9% pgsteal_direct +0.8% pgscan_direct -6.7% pgsteal_anon -13.8% pgscan_anon -13.8% pgsteal_file -14.5% pgscan_file -30.1%
Dynamic jank (cold start + scroll, x10): missed-frame <1% on both base and patched; not memory-bound enough to discriminate.
One trade-off, same run. PSI (memory) creeps up on the sustained window:
avg10 avg60 avg300
some base 0.00 0.03 0.33
some patched 0.00 0.02 0.37 (+12%)
full base 0.00 0.00 0.10
full patched 0.00 0.00 0.13 (+30%)and kernel stack footprint -5.3%. Reads as LMKD keeping fewer background apps alive. This seems an Android policy interaction, not a kernel regression. Downstreams with aggressive LMKD may want to retune.
Just curious — did PSI move at all in your make -j32 run?
Best,
Zicheng
Baolin Wang
reviewerHi Zicheng,
Quoted 16 lines · Zicheng Wang
Hi Baolin,
We backport and test an Android
handset(qcom8850, 6.12, 12 GB). Patch 1'svma_flags_tisn't on 6.12, so we kept only the core check in its legacy form:(vma->vm_flags & VM_EXEC)Since this is a backport, the numbers below just show the direction of change. Overall the patch does well, and ran for several hours with no crashes or hangs:
Camera (launch + capture, 22 background apps resident):
cold-start p50 -6.3% capture on par memavail at startup +4 .. +7%
Quoted 8 lines
Reclaim over the same window, as the patch intends:
pgsteal_kswapd -46.5% pgscan_kswapd -52.9% pgsteal_direct +0.8% pgscan_direct -6.7% pgsteal_anon -13.8% pgscan_anon -13.8% pgsteal_file -14.5% pgscan_file -30.1%Dynamic jank (cold start + scroll, x10): missed-frame <1% on both base and patched; not memory-bound enough to discriminate.
Thanks for testing and sharing the performance number.
Quoted 9 lines
One trade-off, same run. PSI (memory) creeps up on the sustained window:
avg10 avg60 avg300 some base 0.00 0.03 0.33 some patched 0.00 0.02 0.37 (+12%) full base 0.00 0.00 0.10 full patched 0.00 0.00 0.13 (+30%)
Quoted 4 lines
and kernel stack footprint -5.3%. Reads as LMKD keeping fewer background apps alive. This seems an Android policy interaction, not a kernel regression. Downstreams with aggressive LMKD may want to retune.
I didn't pay attention to PSI before, but I think this is explainable.
Since we try to keep executable file folios in memory to avoid IO thrashing, which could cause more severe problems (such as the system hanging), this also means that the stall time of some other folios may increase. Moreover, the stall time of the original exec folios might not have been counted at all, for example, in filemap_read_folio(), PSI is only accounted for file folios marked with workingset, but before my patch, exec file folios might not have had a chance to set the workingset flag.
However, the overhead of IO thrashing is more severe than the stall time of other memory allocations, which is why the end-to-end performance improves, as seen from your data.
Quoted 8 lines · Baolin Wang
Since we try to keep executable file folios in memory to avoid IO thrashing, which could cause more severe problems (such as the system hanging), this also means that the stall time of some other folios may increase. Moreover, the stall time of the original exec folios might not have been counted at all, for example, in
filemap_read_folio(), PSI is only accounted for file folios marked with workingset, but before my patch, exec file folios might not have had a chance to set the workingset flag.
Thanks Baolin, that makes sense.
Best,
Zicheng