Skip to contentSkip to conversation

mm/mglru: use a smaller batch in fill/drain_evictable()

PATCH In review Hui Zhu 3 participants 3 messages Nov 16

Subsystems (3)
lore
Share & tools
lore

Hui Zhu

author
#0 lore raw

PATCH v3 1/1 mm/mglru: use a smaller batch in fill/drain_evictable()

From: Hui Zhu <[email protected]>

lru_gen_change_state() migrates folios in batches of up to MAX_LRU_BATCH (4096) folios under lruvec_lock_irq(). A full batch keeps local interrupts off for up to about a millisecond, and CPUs contending for the same lru_lock spin with interrupts disabled for that long.

The caller already drops the lock and reschedules between batches, and the restart protocol does not depend on the batch size: both functions make monotone progress, so a smaller batch only shortens the irq-off section. Use MIN_LRU_BATCH (64) instead, bounding it to about 15us. This is a cold path that runs once per kill-switch toggle; the extra unlock/resched/relock round trips cost a few percent of wall time on an otherwise idle machine.

Same change as commit 6e9be217a3ce ("mm/mglru: use a smaller batch for reclaim") made for try_to_shrink_lruvec().

---
Changlog:
v3:
Drop the need_resched()/spin_is_contended() checks and shrink the batch to MIN_LRU_BATCH instead (Baolin Wang).
v2:
Also yield on lock contention after a minimum batch (Kairui Song).

 mm/vmscan.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/vmscan.c +2 -2
index 6a1af9b92349..054af6cf6a2e 100644
@@ -5499,7 +5499,7 @@ static bool __maybe_unused state_is_valid(struct lruvec *lruvec)
 static bool fill_evictable(struct lruvec *lruvec)
 {
 	enum lru_list lru;
-	int remaining = MAX_LRU_BATCH;
+	int remaining = MIN_LRU_BATCH;
 
 	for_each_evictable_lru(lru) {
 		int type = is_file_lru(lru);
@@ -5544,7 +5544,7 @@ static bool fill_evictable(struct lruvec *lruvec)
 static bool drain_evictable(struct lruvec *lruvec)
 {
 	int gen, type, zone;
-	int remaining = MAX_LRU_BATCH;
+	int remaining = MIN_LRU_BATCH;
 
 	for_each_gen_type_zone(gen, type, zone) {
 		struct list_head *head = &lruvec->lrugen.folios[gen][type][zone];
Signature · 1 line
2.53.0
reviewer Reviewed-by on v3 1/1
#1 lore raw
Quoted 16 lines · Hui Zhu

From: Hui Zhu <[email protected]>

lru_gen_change_state() migrates folios in batches of up to MAX_LRU_BATCH (4096) folios under lruvec_lock_irq(). A full batch keeps local interrupts off for up to about a millisecond, and CPUs contending for the same lru_lock spin with interrupts disabled for that long.

The caller already drops the lock and reschedules between batches, and the restart protocol does not depend on the batch size: both functions make monotone progress, so a smaller batch only shortens the irq-off section. Use MIN_LRU_BATCH (64) instead, bounding it to about 15us. This is a cold path that runs once per kill-switch toggle; the extra unlock/resched/relock round trips cost a few percent of wall time on an otherwise idle machine.

Quoted 5 lines

Same change as commit 6e9be217a3ce ("mm/mglru: use a smaller batch for reclaim") made for try_to_shrink_lruvec().

---

I'm fine if the small change can fix your issue in this cold path. Let's also see others' input. Feel free to add:

reviewer Reviewed-by
#2 lore raw
Quoted 23 lines · Baolin Wang
Quoted 19 lines · Hui Zhu

From: Hui Zhu <[email protected]>

lru_gen_change_state() migrates folios in batches of up to MAX_LRU_BATCH (4096) folios under lruvec_lock_irq(). A full batch keeps local interrupts off for up to about a millisecond, and CPUs contending for the same lru_lock spin with interrupts disabled for that long.

The caller already drops the lock and reschedules between batches, and the restart protocol does not depend on the batch size: both functions make monotone progress, so a smaller batch only shortens the irq-off section. Use MIN_LRU_BATCH (64) instead, bounding it to about 15us. This is a cold path that runs once per kill-switch toggle; the extra unlock/resched/relock round trips cost a few percent of wall time on an otherwise idle machine.

Same change as commit 6e9be217a3ce ("mm/mglru: use a smaller batch for reclaim") made for try_to_shrink_lruvec().

Quoted 6 lines
Quoted 2 lines

---

I'm fine if the small change can fix your issue in this cold path. Let's also see others' input. Feel free to add:

+1

This is a rarely used code path, and we may eventually remove it entirely. So let's avoid over-engineering it.