mm/mglru: use a smaller batch in fill/drain_evictable()
Subsystems (3)
PATCH v3 1/1 mm/mglru: use a smaller batch in fill/drain_evictable()
From: Hui Zhu <[email protected]>
lru_gen_change_state() migrates folios in batches of up to MAX_LRU_BATCH (4096) folios under lruvec_lock_irq(). A full batch keeps local interrupts off for up to about a millisecond, and CPUs contending for the same lru_lock spin with interrupts disabled for that long.
The caller already drops the lock and reschedules between batches, and the restart protocol does not depend on the batch size: both functions make monotone progress, so a smaller batch only shortens the irq-off section. Use MIN_LRU_BATCH (64) instead, bounding it to about 15us. This is a cold path that runs once per kill-switch toggle; the extra unlock/resched/relock round trips cost a few percent of wall time on an otherwise idle machine.
Same change as commit 6e9be217a3ce ("mm/mglru: use a smaller batch for reclaim") made for try_to_shrink_lruvec().
---
Changlog:
v3:
Drop the need_resched()/spin_is_contended() checks and shrink the batch to MIN_LRU_BATCH instead (Baolin Wang).
v2:
Also yield on lock contention after a minimum batch (Kairui Song).
mm/vmscan.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
mm/vmscan.c +2 -2
@@ -5499,7 +5499,7 @@ static bool __maybe_unused state_is_valid(struct lruvec *lruvec)static bool fill_evictable (struct lruvec *lruvec) {enum lru_list lru; -int remaining =MAX_LRU_BATCH ; +int remaining =MIN_LRU_BATCH ;for_each_evictable_lru (lru) {int type =is_file_lru (lru);
@@ -5544,7 +5544,7 @@ static bool fill_evictable(struct lruvec *lruvec)static bool drain_evictable (struct lruvec *lruvec) {int gen, type, zone; -int remaining =MAX_LRU_BATCH ; +int remaining =MIN_LRU_BATCH ;for_each_gen_type_zone (gen, type, zone) {struct list_head *head = &lruvec->lrugen.folios[gen][type][zone];
Signature · 1 line
2.53.0
Quoted 16 lines · Hui Zhu
From: Hui Zhu <[email protected]>
lru_gen_change_state()migrates folios in batches of up toMAX_LRU_BATCH(4096) folios underlruvec_lock_irq(). A full batch keeps local interrupts off for up to about a millisecond, and CPUs contending for the samelru_lockspin with interrupts disabled for that long.The caller already drops the lock and reschedules between batches, and the restart protocol does not depend on the batch size: both functions make monotone progress, so a smaller batch only shortens the irq-off section. Use
MIN_LRU_BATCH(64) instead, bounding it to about 15us. This is a cold path that runs once per kill-switch toggle; the extra unlock/resched/relock round trips cost a few percent of wall time on an otherwise idle machine.
Quoted 5 lines
Same change as commit
6e9be217a3ce("mm/mglru: use a smaller batch for reclaim") made fortry_to_shrink_lruvec().Signed-off-by Hui Zhu <[email protected]>---
I'm fine if the small change can fix your issue in this cold path. Let's also see others' input. Feel free to add:
Quoted 23 lines · Baolin Wang
Quoted 19 lines · Hui Zhu
From: Hui Zhu <[email protected]>
lru_gen_change_state()migrates folios in batches of up toMAX_LRU_BATCH(4096) folios underlruvec_lock_irq(). A full batch keeps local interrupts off for up to about a millisecond, and CPUs contending for the samelru_lockspin with interrupts disabled for that long.The caller already drops the lock and reschedules between batches, and the restart protocol does not depend on the batch size: both functions make monotone progress, so a smaller batch only shortens the irq-off section. Use
MIN_LRU_BATCH(64) instead, bounding it to about 15us. This is a cold path that runs once per kill-switch toggle; the extra unlock/resched/relock round trips cost a few percent of wall time on an otherwise idle machine.Same change as commit
6e9be217a3ce("mm/mglru: use a smaller batch for reclaim") made fortry_to_shrink_lruvec().
Quoted 6 lines
Quoted 2 lines
Signed-off-by Hui Zhu <[email protected]>---
I'm fine if the small change can fix your issue in this cold path. Let's also see others' input. Feel free to add:
Reviewed-by Baolin Wang <[email protected]>
+1
This is a rarely used code path, and we may eventually remove it entirely. So let's avoid over-engineering it.