mm: fix condition for throttle_direct_reclaim
Subsystems (3)
PATCH v3 1/1 mm: fix condition for throttle_direct_reclaim
Since "mm: fix 100% CPU kswapd busyloop on unreclaimable nodes" kswapd has been modified to give up after MAX_RECLAIM_RETRIES number of unsucessful iterations. Before going to sleep, kswapd thread will unconditionally wakeup all threads sleeping on pfmemalloc_wait. However the awoken threads will recheck the watermarks and wake the kswapd thread and sleep again on pfmemalloc_wait. There is a chance that the system might end up in livelock between unsuccessful kswapd and direct reclaimers because all direct reclaimer might end up in throttle_direct_reclaim and there is nobody to make a forward progress. So, add kswapd_failures check on the throttle_direct_reclaim condition.
---
v3:
Commit message updated.
v2:
Instead of separate helper function for checking kswapd_failures, added the check into pfmemalloc_watermark_ok() and renamed that function.
mm/vmscan.c | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-)
mm/vmscan.c +9 -6
@@ -2783,7 +2783,7 @@ static unsigned long do_try_to_free_pages(struct zonelist *zonelist,return 0 ; } -static bool pfmemalloc_watermark_ok (pg_data_t *pgdat) +static bool allow_direct_reclaim (pg_data_t *pgdat) {struct zone *zone;unsigned long pfmemalloc_reserve =0 ;
@@ -2791,6 +2791,9 @@ static bool pfmemalloc_watermark_ok(pg_data_t *pgdat)int i;bool wmark_ok; +if (pgdat->kswapd_failures >=MAX_RECLAIM_RETRIES ) +return true; +for (i =0 ; i <=ZONE_NORMAL ; i++) { zone = &pgdat->node_zones[i];if (!managed_zone (zone))
@@ -2873,7 +2876,7 @@ static bool throttle_direct_reclaim(gfp_t gfp_mask, struct zonelist *zonelist,/* Throttle based on the first usable node */ pgdat = zone->zone_pgdat; -if (pfmemalloc_watermark_ok (pgdat)) +if (allow_direct_reclaim (pgdat)) goto out;break ; }
@@ -2895,14 +2898,14 @@ static bool throttle_direct_reclaim(gfp_t gfp_mask, struct zonelist *zonelist,*/ if (!(gfp_mask & __GFP_FS)) {wait_event_interruptible_timeout (pgdat->pfmemalloc_wait, -pfmemalloc_watermark_ok (pgdat),HZ ); +allow_direct_reclaim (pgdat),HZ ); goto check_pending; }/* Throttle until kswapd wakes the process */ wait_event_killable (zone->zone_pgdat->pfmemalloc_wait, -pfmemalloc_watermark_ok (pgdat)); +allow_direct_reclaim (pgdat)); check_pending:if (fatal_signal_pending (current))
@@ -3102,7 +3105,7 @@ static bool prepare_kswapd_sleep(pg_data_t *pgdat, int order, int classzone_idx) {/* * The throttled processes are normally woken up in balance_pgdat() as -* soon as pfmemalloc_watermark_ok() is true. But there is a potential +* soon as allow_direct_reclaim() is true. But there is a potential * race between when kswapd checks the watermarks and a process gets * throttled. There is also a potential race if processes get * throttled, kswapd wakes, a large process exits thereby balancing the
@@ -3271,7 +3274,7 @@ static int balance_pgdat(pg_data_t *pgdat, int order, int classzone_idx)* able to safely make forward progress. Wake them */ if (waitqueue_active (&pgdat->pfmemalloc_wait) && -pfmemalloc_watermark_ok (pgdat)) +allow_direct_reclaim (pgdat))wake_up_all (&pgdat->pfmemalloc_wait);/* Check if kswapd should be suspending */
Signature · 7 lines
2.12.0.367.g23dc2f6d3c-goog -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>