mm/ksm: clean up scan_get_next_rmap_item() initialization
Xu Xin (vivo)
reviewerv3 cover mm/ksm: clean up scan_get_next_rmap_item() initialization
Currently, scan_get_next_rmap_item() has grown quite large and complex. When introducing new features, such as the only-merge-zero-page mode we are currently working on, we don't want to add more complexity to it and don't want to touch rmap_item structures to avoid unnecessary overhead, so we need to extract some common logic.
With these changes, the entry of scan_get_next_rmap_item() becomes much cleaner and easier to follow, instead of being cluttered with unrelated boilerplate such as stale stable_node pruning and cycle initialization.
Let's start by cleaning up these two parts at the beginning (where I think abstraction is necessary), and we can continue cleaning up further if needed later.
Changes in v3:
- [PATCH 2/2] Initialize mm_slot = ksm_scan.mm_slot in the else branch when continuing the current scan cycle, fixing an uninitialized variable warning/bug (kernel test robot).
- [PATCH 2/2] Drop Lorenzo's Reviewed-by tag due to the v2 bug fix.
Changes in v2:
- [PATCH 2/2] Rename scan_begin_new_cycle() to scan_should_start_new_cycle and keep it simple (David).
- [PATCH 2/2] Keep lru_add_drain_all() in scan_get_next_rmap_item (David).
- [PATCH 2/2] Use scoped_guard(spinlock, &ksm_mmlist_lock) in
scan_get_first_mm() (Lorenzo).
Xu Xin (vivo) (2):
mm/ksm: extract helper functions for scan cycle cleanup mm/ksm: add initialization helpers for scan_get_next_rmap_item()
mm/ksm.c | 101 +++++++++++++++++++++++++++++++++++-------------------- 1 file changed, 65 insertions(+), 36 deletions(-)
Signature · 1 line
2.34.1
Xu Xin (vivo)
reviewerPATCH v3 1/2 mm/ksm: extract helper functions for scan cycle cleanup
Since scan_get_next_rmap_item() is getting too large, extract two new helper functions, ksm_prune_migrate_nodes() and ksm_reset_unstable_trees(), to reduce indentation levels, isolate local variables, and improve the readability of the main scan loop.
No functional change intended.
--- mm/ksm.c | 54 +++++++++++++++++++++++++++++++----------------------- 1 file changed, 31 insertions(+), 23 deletions(-)
mm/ksm.c +31 -23
@@ -2609,6 +2609,35 @@ static struct mm_walk_ops ksm_next_page_ops = { .walk_lock = PGWALK_RDLOCK, }; +static void ksm_prune_migrate_nodes (void ) +{ +struct ksm_stable_node *stable_node, *next; +struct folio *folio; + +if (ksm_merge_across_nodes) +return ; + +/* +* Whereas stale stable_nodes on the stable_tree itself +* get pruned in the regular course of stable_tree_search(), +* those moved out to the migrate_nodes list can accumulate: +* so prune them once before each full scan. +*/ +list_for_each_entry_safe (stable_node, next, &migrate_nodes, list) { + folio =ksm_get_folio (stable_node,KSM_GET_FOLIO_NOLOCK ); + +if (folio) +folio_put (folio); +cond_resched (); + } +} + +static void ksm_reset_unstable_trees (void ) +{ +for (int nid =0 ; nid < ksm_nr_node_ids; nid++) + root_unstable_tree[nid] =RB_ROOT ; +} +static struct ksm_rmap_item *scan_get_next_rmap_item (struct page **page) {struct mm_struct *mm;
@@ -2617,7 +2646,6 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)struct vm_area_struct *vma;struct ksm_rmap_item *rmap_item;struct vma_iterator vmi; -int nid;if (list_empty (&ksm_mm_head.slot.mm_node))return NULL ;
@@ -2639,28 +2667,8 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)*/ lru_add_drain_all (); -/* -* Whereas stale stable_nodes on the stable_tree itself -* get pruned in the regular course of stable_tree_search(), -* those moved out to the migrate_nodes list can accumulate: -* so prune them once before each full scan. -*/ -if (!ksm_merge_across_nodes) { -struct ksm_stable_node *stable_node, *next; -struct folio *folio; - -list_for_each_entry_safe (stable_node, next, - &migrate_nodes, list) { - folio =ksm_get_folio (stable_node, -KSM_GET_FOLIO_NOLOCK ); -if (folio) -folio_put (folio); -cond_resched (); - } - } - -for (nid =0 ; nid < ksm_nr_node_ids; nid++) - root_unstable_tree[nid] =RB_ROOT ; +ksm_prune_migrate_nodes (); +ksm_reset_unstable_trees ();spin_lock (&ksm_mmlist_lock); slot = list_entry(mm_slot->slot.mm_node.next,
Signature · 1 line
2.34.1
Xu Xin (vivo)
reviewerPATCH v3 2/2 mm/ksm: add initialization helpers for scan_get_next_rmap_item()
In reply to Xu Xin (vivo)Extract the new scan cycle check into scan_should_start_new_cycle() and add the helper scan_get_first_mm() that advances the scan cursor to the first mm_slot.
This keeps the exact execution order intact while making scan_get_next_rmap_item() cleaner and easier to follow.
No functional change intended.
---
Changes in v3:
- Initialize mm_slot = ksm_scan.mm_slot in the else branch when continuing the current scan cycle, fixing an uninitialized variable warning/bug (kernel test robot).
- Drop Lorenzo's Reviewed-by tag due to the v2 bug fix.
Changes in v2:
- Rename scan_begin_new_cycle() to scan_should_start_new_cycle() and keep it simple (David).
- Keep lru_add_drain_all() in scan_get_next_rmap_item() (David).
- Use scoped_guard(spinlock, &ksm_mmlist_lock) in scan_get_first_mm() (Lorenzo).
mm/ksm.c | 47 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 13 deletions(-)
mm/ksm.c +34 -13
@@ -2638,6 +2638,34 @@ static void ksm_reset_unstable_trees(void) root_unstable_tree[nid] =RB_ROOT ; } +static bool scan_should_start_new_cycle (void ) +{ +/* Check if the cursor points to the head. */ +return ksm_scan.mm_slot == &ksm_mm_head; +} + +static struct ksm_mm_slot *scan_get_first_mm (void ) +{ +struct ksm_mm_slot *mm_slot = &ksm_mm_head; +struct mm_slot *slot; + + scoped_guard(spinlock, &ksm_mmlist_lock) { + slot =list_entry (mm_slot->slot.mm_node.next, + struct mm_slot, mm_node); + mm_slot = mm_slot_entry(slot, struct ksm_mm_slot, slot); + ksm_scan.mm_slot = mm_slot; + } + +/* +* Although we tested list_empty() above, a racing __ksm_exit +* of the last mm on the list may have removed it since then. +*/ +if (mm_slot == &ksm_mm_head) +return NULL ; + +return mm_slot; +} +static struct ksm_rmap_item *scan_get_next_rmap_item (struct page **page) {struct mm_struct *mm;
@@ -2650,8 +2678,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)if (list_empty (&ksm_mm_head.slot.mm_node))return NULL ; - mm_slot = ksm_scan.mm_slot; -if (mm_slot == &ksm_mm_head) { +if (scan_should_start_new_cycle ()) {advisor_start_scan ();trace_ksm_start_scan (ksm_scan.seqnr, ksm_rmap_items);
@@ -2670,21 +2697,15 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)ksm_prune_migrate_nodes ();ksm_reset_unstable_trees (); -spin_lock (&ksm_mmlist_lock); - slot =list_entry (mm_slot->slot.mm_node.next, - struct mm_slot, mm_node); - mm_slot = mm_slot_entry(slot, struct ksm_mm_slot, slot); - ksm_scan.mm_slot = mm_slot; -spin_unlock (&ksm_mmlist_lock); -/* -* Although we tested list_empty() above, a racing __ksm_exit -* of the last mm on the list may have removed it since then. -*/ -if (mm_slot == &ksm_mm_head) + mm_slot =scan_get_first_mm (); +if (!mm_slot)return NULL ; next_mm: ksm_scan.address =0 ; ksm_scan.rmap_list = &mm_slot->rmap_list; + }else { +/* Continue scanning the current mm. */ + mm_slot = ksm_scan.mm_slot; } slot = &mm_slot->slot;
Signature · 1 line
2.34.1
Re: [PATCH v3 2/2] mm/ksm: add initialization helpers for scan_get_next_rmap_item()
Quoted 10 lines · Xu Xin (vivo)
Extract the new scan cycle check into
scan_should_start_new_cycle()and add the helperscan_get_first_mm()that advances the scan cursor to the firstmm_slot.This keeps the exact execution order intact while making
scan_get_next_rmap_item()cleaner and easier to follow.No functional change intended.
Signed-off-by Xu Xin (vivo) <[email protected]>
Nits below but otherwise LGTM, with those addressed:
Thanks!
Quoted 22 lines
---
Changes in v3:
- Initializemm_slot= ksm_scan.mm_slot in the else branch when continuing the current scan cycle, fixing an uninitialized variable warning/bug (kernel test robot).
- Drop Lorenzo's Reviewed-by tag due to the v2 bug fix.Changes in v2:
- Renamescan_begin_new_cycle()toscan_should_start_new_cycle()and keep it simple (David).
- Keeplru_add_drain_all()inscan_get_next_rmap_item()(David).
- Usescoped_guard(spinlock, &ksm_mmlist_lock)inscan_get_first_mm()(Lorenzo).mm/ksm.c | 47 ++++++++++++++++++++++++++++++++++------------- 1 file changed, 34 insertions(+), 13 deletions(-)mm/ksm.c +0
-0
mm/ksm.c static void ksm_reset_unstable_trees(void) · line 2643
Quoted 6 lines
mm/ksm.c +3
-0@@ -2638,6 +2638,34 @@ static void ksm_reset_unstable_trees(void) root_unstable_tree[nid] =RB_ROOT ; } +static bool scan_should_start_new_cycle (void ) +{ +/* Check if the cursor points to the head. */
Probably this comment is redundant, I think the code already makes this clear.
Quoted 7 lines
mm/ksm.c +7
-0+return ksm_scan.mm_slot == &ksm_mm_head; +} + +static struct ksm_mm_slot *scan_get_first_mm (void ) +{ +struct ksm_mm_slot *mm_slot = &ksm_mm_head; +struct mm_slot *slot;
mm/ksm.c static void ksm_reset_unstable_trees(void) · line 2660
Quoted 10 lines
mm/ksm.c +10
-0+ + scoped_guard(spinlock, &ksm_mmlist_lock) { + slot =list_entry (mm_slot->slot.mm_node.next, + struct mm_slot, mm_node); + mm_slot = mm_slot_entry(slot, struct ksm_mm_slot, slot); + ksm_scan.mm_slot = mm_slot; + } + +/* +* Although we tested list_empty() above, a racing __ksm_exit
I think you need to update this comment to reflect the fact that you moved the list_empty() check :)
Quoted 49 lines
mm/ksm.c +14
-14+* of the last mm on the list may have removed it since then. +*/ +if (mm_slot == &ksm_mm_head) +return NULL ; + +return mm_slot; +} +static struct ksm_rmap_item *scan_get_next_rmap_item (struct page **page) {struct mm_struct *mm;@@ -2650,8 +2678,7 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)if (list_empty (&ksm_mm_head.slot.mm_node))return NULL ; - mm_slot = ksm_scan.mm_slot; -if (mm_slot == &ksm_mm_head) { +if (scan_should_start_new_cycle ()) {advisor_start_scan ();trace_ksm_start_scan (ksm_scan.seqnr, ksm_rmap_items);@@ -2670,21 +2697,15 @@ static struct ksm_rmap_item *scan_get_next_rmap_item(struct page **page)ksm_prune_migrate_nodes ();ksm_reset_unstable_trees (); -spin_lock (&ksm_mmlist_lock); - slot =list_entry (mm_slot->slot.mm_node.next, - struct mm_slot, mm_node); - mm_slot = mm_slot_entry(slot, struct ksm_mm_slot, slot); - ksm_scan.mm_slot = mm_slot; -spin_unlock (&ksm_mmlist_lock); -/* -* Although we tested list_empty() above, a racing __ksm_exit -* of the last mm on the list may have removed it since then. -*/ -if (mm_slot == &ksm_mm_head) + mm_slot =scan_get_first_mm (); +if (!mm_slot)return NULL ; next_mm: ksm_scan.address =0 ; ksm_scan.rmap_list = &mm_slot->rmap_list; + }else { +/* Continue scanning the current mm. */ + mm_slot = ksm_scan.mm_slot; } slot = &mm_slot->slot; --
Quoted 2 lines
2.34.1
--
Cheers, Lorenzo
Re: [PATCH v3 2/2] mm/ksm: add initialization helpers for scan_get_next_rmap_item()
Quoted 10 lines
Quoted 10 lines
mm/ksm.c +9
-0+static struct ksm_mm_slot *scan_get_first_mm (void ) +{ +struct ksm_mm_slot *mm_slot = &ksm_mm_head; +struct mm_slot *slot; + + scoped_guard(spinlock, &ksm_mmlist_lock) { + slot =list_entry (mm_slot->slot.mm_node.next, + struct mm_slot, mm_node); + mm_slot = mm_slot_entry(slot, struct ksm_mm_slot, slot);
Quoted 8 lines
mm/ksm.c static void ksm_reset_unstable_trees(void) · line 2660
Quoted 5 lines
mm/ksm.c +5
-0+ ksm_scan.mm_slot = mm_slot; + } + +/* +* Although we tested list_empty() above, a racing __ksm_exit I think you need to update this comment to reflect the fact that you moved the
list_empty()check :)
Yeah, with both things
Xu, can you send a v4 that I can pick up?
Signature · 3 lines
Cheers, David