follow up nodereclaim for 32b fix

From: Michal Hocko <[email protected]>

get_scan_count considers the whole node LRU size when
- doing SCAN_FILE due to many page cache inactive pages
- calculating the number of pages to scan

in both cases this might lead to unexpected behavior especially on 32b systems where we can expect lowmem memory pressure very often.

A large highmem zone can easily distort SCAN_FILE heuristic because there might be only few file pages from the eligible zones on the node lru and we would still enforce file lru scanning which can lead to trashing while we could still scan anonymous pages.

The later use of lruvec_lru_size can be problematic as well. Especially when there are not many pages from the eligible zones. We would have to skip over many pages to find anything to reclaim but shrink_node_memcg would only reduce the remaining number to scan by SWAP_CLUSTER_MAX at maximum. Therefore we can end up going over a large LRU many times without actually having chance to reclaim much if anything at all. The closer we are out of memory on lowmem zone the worse the problem will be.

Changes since v1
- s@lruvec_lru_size_zone_idx@lruvec_lru_size_eligibe_zones@

Acked-by Minchan Kim <[email protected]>

Signed-off-by Michal Hocko <[email protected]>

---

On Tue 10-01-17 13:55:51, Michal Hocko wrote:
[...]

Quoted 4 lines

mm/vmscan.c +0 -0

@@ -2280,7 +2306,7 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
 			unsigned long size;
 			unsigned long scan;
 

mm/vmscan.c static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg, · line 2312

Quoted 5 lines

mm/vmscan.c +1 -1

-			size = lruvec_lru_size(lruvec, lru);
+			size = lruvec_lru_size_eligibe_zones(lruvec, lru, sc->reclaim_idx);
 			scan = size >> sc->priority;
 
 			if (!scan && pass && force_scan)

I have just come across inactive_reclaimable_pages and it seems it is unnecessary after this, right Minchan?

From: Michal Hocko <[email protected]>

get_scan_count considers the whole node LRU size when
- doing SCAN_FILE due to many page cache inactive pages
- calculating the number of pages to scan

in both cases this might lead to unexpected behavior especially on 32b systems where we can expect lowmem memory pressure very often.

/*
+ * Return the number of pages on the given lru which are eligible for the
+ * given zone_idx
+ */

mm/vmscan.c +10 -0

+static unsigned long lruvec_lru_size_eligibe_zones(struct lruvec *lruvec,
+		enum lru_list lru, int zone_idx)
+{
+	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
+	unsigned long lru_size;
+	int zid;
+
+	lru_size = lruvec_lru_size(lruvec, lru);
+	for (zid = zone_idx + 1; zid < MAX_NR_ZONES; zid++) {
+		struct zone *zone = &pgdat->node_zones[zid];

Quoted 8 lines · Michal Hocko

From: Michal Hocko <[email protected]>

get_scan_count considers the whole node LRU size when
- doing SCAN_FILE due to many page cache inactive pages
- calculating the number of pages to scan

in both cases this might lead to unexpected behavior especially on 32b systems where we can expect lowmem memory pressure very often.

The amount of retrofitting zones back into reclaim is disappointing :/

Quoted 14 lines

/*
+ * Return the number of pages on the given lru which are eligible for the
+ * given zone_idx
+ */

mm/vmscan.c +10 -0

+static unsigned long lruvec_lru_size_eligibe_zones(struct lruvec *lruvec,
+		enum lru_list lru, int zone_idx)
+{
+	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
+	unsigned long lru_size;
+	int zid;
+
+	lru_size = lruvec_lru_size(lruvec, lru);
+	for (zid = zone_idx + 1; zid < MAX_NR_ZONES; zid++) {
+		struct zone *zone = &pgdat->node_zones[zid];

mm/vmscan.c unsigned long lruvec_zone_lru_size(struct lruvec *lruvec, enum lru_list lru, · line 278

Quoted 10 lines

mm/vmscan.c +10 -0

+		unsigned long size;
+
+		if (!managed_zone(zone))
+			continue;
+
+		size = lruvec_zone_lru_size(lruvec, lru, zid);
+		lru_size -= min(size, lru_size);
+	}
+
+	return lru_size;

The only other use of lruvec_lru_size() is also in get_scan_count(), where it decays the LRU pressure balancing ratios. That caller wants to operate on the entire lruvec.

Can you instead add the filtering logic to lruvec_lru_size() directly, and pass MAX_NR_ZONES when operating on the entire lruvec? That would make the code quite a bit clearer than having 3 different lruvec size querying functions.

From 39824aac7504b38f943a80b7d98ec4e87a5607a7 Mon Sep 17 00:00:00 2001
From: Michal Hocko <[email protected]>
Date: Tue, 27 Dec 2016 16:28:44 +0100
Subject: [PATCH] mm, vmscan: consider eligible zones in get_scan_count

get_scan_count considers the whole node LRU size when
- doing SCAN_FILE due to many page cache inactive pages
- calculating the number of pages to scan

in both cases this might lead to unexpected behavior especially on 32b systems where we can expect lowmem memory pressure very often.

A large highmem zone can easily distort SCAN_FILE heuristic because there might be only few file pages from the eligible zones on the node lru and we would still enforce file lru scanning which can lead to trashing while we could still scan anonymous pages.

The later use of lruvec_lru_size can be problematic as well. Especially when there are not many pages from the eligible zones. We would have to skip over many pages to find anything to reclaim but shrink_node_memcg would only reduce the remaining number to scan by SWAP_CLUSTER_MAX at maximum. Therefore we can end up going over a large LRU many times without actually having chance to reclaim much if anything at all. The closer we are out of memory on lowmem zone the worse the problem will be.

Fix this by making lruvec_lru_size zone aware. zone_idx will tell the the maximum eligible zone.

Changes since v2
- move the zone filtering logic to lruvec_lru_size so that we do not have too many lruvec_lru_size* functions - Johannes

Changes since v1
- s@lruvec_lru_size_zone_idx@lruvec_lru_size_eligibe_zones@

Acked-by Minchan Kim <[email protected]>

Signed-off-by Michal Hocko <[email protected]>

(quoted diff) +17 -6

@@ -234,22 +234,44 @@ bool pgdat_reclaimable(struct pglist_data *pgdat)
 		pgdat_reclaimable_pages(pgdat) * 6;
 }
 
-unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru)
+static unsigned long lruvec_zone_lru_size(struct lruvec *lruvec,
+		enum lru_list lru, int zone_idx)
 {
 	if (!mem_cgroup_disabled())
-		return mem_cgroup_get_lru_size(lruvec, lru);
+		return mem_cgroup_get_zone_lru_size(lruvec, lru, zone_idx);
 
-	return node_page_state(lruvec_pgdat(lruvec), NR_LRU_BASE + lru);
+	return zone_page_state(&lruvec_pgdat(lruvec)->node_zones[zone_idx],
+			       NR_ZONE_LRU_BASE + lru);
 }
 
-unsigned long lruvec_zone_lru_size(struct lruvec *lruvec, enum lru_list lru,
-				   int zone_idx)
+/** lruvec_lru_size -  Returns the number of pages on the given LRU list.
+ * @lruvec: lru vector
+ * @lru: lru to use
+ * @zone_idx: zones to consider (use MAX_NR_ZONES for the whole LRU list)
+ */
+unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone_idx)
 {
+	unsigned long lru_size;
+	int zid;
+
 	if (!mem_cgroup_disabled())
-		return mem_cgroup_get_zone_lru_size(lruvec, lru, zone_idx);
+		lru_size = mem_cgroup_get_lru_size(lruvec, lru);
+	else
+		lru_size = node_page_state(lruvec_pgdat(lruvec), NR_LRU_BASE + lru);

From: Michal Hocko <[email protected]>

get_scan_count considers the whole node LRU size when
- doing SCAN_FILE due to many page cache inactive pages
- calculating the number of pages to scan

in both cases this might lead to unexpected behavior especially on 32b systems where we can expect lowmem memory pressure very often.

A large highmem zone can easily distort SCAN_FILE heuristic because there might be only few file pages from the eligible zones on the node lru and we would still enforce file lru scanning which can lead to trashing while we could still scan anonymous pages.

The later use of lruvec_lru_size can be problematic as well. Especially when there are not many pages from the eligible zones. We would have to skip over many pages to find anything to reclaim but shrink_node_memcg would only reduce the remaining number to scan by SWAP_CLUSTER_MAX at maximum. Therefore we can end up going over a large LRU many times without actually having chance to reclaim much if anything at all. The closer we are out of memory on lowmem zone the worse the problem will be.

Fix this by filtering out all the ineligible zones when calculating the lru size for both paths and consider only sc->reclaim_idx zones.

Acked-by Minchan Kim <[email protected]>

Signed-off-by Michal Hocko <[email protected]>

---

From: Michal Hocko <[email protected]>

This reverts 91dcade47a3d0e7c31464ef05f56c08e92a0e9c2. inactive_reclaimable_pages shouldn't be needed anymore since that get_scan_count is aware of the eligble zones ("mm, vmscan: consider eligible zones in get_scan_count").

Signed-off-by Michal Hocko <[email protected]>

---

 mm/vmscan.c | 27 ---------------------------
 1 file changed, 27 deletions(-)

mm/vmscan.c +0 -28

index a88e222784ea..486ba6d7dc4c 100644
@@ -1700,30 +1700,6 @@ static int current_may_throttle(void)
 		bdi_write_congested(current->backing_dev_info);
 }
 
-static bool inactive_reclaimable_pages(struct lruvec *lruvec,
-				struct scan_control *sc, enum lru_list lru)
-{
-	int zid;
-	struct zone *zone;
-	int file = is_file_lru(lru);
-	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
-
-	if (!global_reclaim(sc))
-		return true;
-
-	for (zid = sc->reclaim_idx; zid >= 0; zid--) {
-		zone = &pgdat->node_zones[zid];
-		if (!managed_zone(zone))
-			continue;
-
-		if (zone_page_state_snapshot(zone, NR_ZONE_LRU_BASE +
-				LRU_FILE * file) >= SWAP_CLUSTER_MAX)
-			return true;
-	}
-
-	return false;
-}
-
 /*
  * shrink_inactive_list() is a helper for shrink_node().  It returns the number
  * of reclaimed pages

@@ -1742,9 +1718,6 @@ shrink_inactive_list(unsigned long nr_to_scan, struct lruvec *lruvec,
 	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
 	struct zone_reclaim_stat *reclaim_stat = &lruvec->reclaim_stat;
 
-	if (!inactive_reclaimable_pages(lruvec, sc, lru))
-		return 0;
-
 	while (unlikely(too_many_isolated(pgdat, file, sc))) {
 		congestion_wait(BLK_RW_ASYNC, HZ/10);
 
--

2.11.0

From: Michal Hocko <[email protected]>

This reverts 91dcade47a3d0e7c31464ef05f56c08e92a0e9c2. inactive_reclaimable_pages shouldn't be needed anymore since that get_scan_count is aware of the eligble zones ("mm, vmscan: consider eligible zones in get_scan_count").

Signed-off-by Michal Hocko <[email protected]>

From: Michal Hocko <[email protected]>

lruvec_lru_size returns the full size of the LRU list while we sometimes need a value reduced only to eligible zones (e.g. for lowmem requests). inactive_list_is_low is one such user. Later patches will add more of them. Add a new parameter to lruvec_lru_size and allow it filter out zones which are not eligible for the given context.

Acked-by Johannes Weiner <[email protected]>

Signed-off-by Michal Hocko <[email protected]>

---

From: Michal Hocko <[email protected]>

lruvec_lru_size returns the full size of the LRU list while we sometimes need a value reduced only to eligible zones (e.g. for lowmem requests). inactive_list_is_low is one such user. Later patches will add more of them. Add a new parameter to lruvec_lru_size and allow it filter out zones which are not eligible for the given context.

Acked-by Johannes Weiner <[email protected]>

Signed-off-by Michal Hocko <[email protected]>

---
 include/linux/mmzone.h |  2 +-
 mm/vmscan.c            | 88 ++++++++++++++++++++++++--------------------------
 mm/workingset.c        |  2 +-
 3 files changed, 45 insertions(+), 47 deletions(-)

include/linux/mmzone.h +1 -1

index d1d440cff60e..91f69aa0d581 100644
@@ -780,7 +780,7 @@ static inline struct pglist_data *lruvec_pgdat(struct lruvec *lruvec)
 #endif
 }
 
-extern unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru);
+extern unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone_idx);
 
 #ifdef CONFIG_HAVE_MEMORY_PRESENT
 void memory_present(int nid, unsigned long start, unsigned long end);

mm/vmscan.c +0 -0

index cf940af609fd..1cb0ebdef305 100644

From: Michal Hocko <[email protected]>

inactive_list_is_low is duplicating logic implemented by lruvec_lru_size_eligibe_zones. Let's use the dedicated function to get the number of eligible pages on the lru list and ask use lruvec_lru_size to get the total LRU lize only when the tracing is really requested. We are still iterating over all LRUs two times in that case but a) inactive_list_is_low is not a hot path and b) this can be addressed at the tracing layer and only evaluate arguments only when the tracing is enabled in future if that ever matters.

Signed-off-by Michal Hocko <[email protected]>

---
 mm/vmscan.c | 38 ++++++++++----------------------------
 1 file changed, 10 insertions(+), 28 deletions(-)

mm/vmscan.c +10 -28

index 137bc85067d3..a9c881f06c0e 100644
@@ -2054,11 +2054,10 @@ static bool inactive_list_is_low(struct lruvec *lruvec, bool file,
 						struct scan_control *sc, bool trace)
 {
 	unsigned long inactive_ratio;
-	unsigned long total_inactive, inactive;
-	unsigned long total_active, active;
+	unsigned long inactive, active;
+	enum lru_list inactive_lru = file * LRU_FILE;
+	enum lru_list active_lru = file * LRU_FILE + LRU_ACTIVE;
 	unsigned long gb;
-	struct pglist_data *pgdat = lruvec_pgdat(lruvec);
-	int zid;
 
 	/*
 	 * If we don't have swap space, anonymous page deactivation

@@ -2067,27 +2066,8 @@ static bool inactive_list_is_low(struct lruvec *lruvec, bool file,
 	if (!file && !total_swap_pages)
 		return false;
 
-	total_inactive = inactive = lruvec_lru_size(lruvec, file * LRU_FILE);
-	total_active = active = lruvec_lru_size(lruvec, file * LRU_FILE + LRU_ACTIVE);
-
-	/*
-	 * For zone-constrained allocations, it is necessary to check if
-	 * deactivations are required for lowmem to be reclaimed. This
-	 * calculates the inactive/active pages available in eligible zones.
-	 */
-	for (zid = sc->reclaim_idx + 1; zid < MAX_NR_ZONES; zid++) {
-		struct zone *zone = &pgdat->node_zones[zid];
-		unsigned long inactive_zone, active_zone;
-
-		if (!managed_zone(zone))
-			continue;
-
-		inactive_zone = lruvec_zone_lru_size(lruvec, file * LRU_FILE, zid);
-		active_zone = lruvec_zone_lru_size(lruvec, (file * LRU_FILE) + LRU_ACTIVE, zid);
-
-		inactive -= min(inactive, inactive_zone);
-		active -= min(active, active_zone);
-	}
+	inactive = lruvec_lru_size_eligibe_zones(lruvec, inactive_lru, sc->reclaim_idx);
+	active = lruvec_lru_size_eligibe_zones(lruvec, active_lru, sc->reclaim_idx);
 
 	gb = (inactive + active) >> (30 - PAGE_SHIFT);
 	if (gb)

@@ -2096,10 +2076,12 @@ static bool inactive_list_is_low(struct lruvec *lruvec, bool file,
 		inactive_ratio = 1;
 
 	if (trace)
-		trace_mm_vmscan_inactive_list_is_low(pgdat->node_id,
+		trace_mm_vmscan_inactive_list_is_low(lruvec_pgdat(lruvec)->node_id,
 				sc->reclaim_idx,
-				total_inactive, inactive,
-				total_active, active, inactive_ratio, file);
+				lruvec_lru_size(lruvec, inactive_lru), inactive,
+				lruvec_lru_size(lruvec, active_lru), active,
+				inactive_ratio, file);
+
 	return inactive * inactive_ratio < active;
 }
 

Signature · 7 lines

2.11.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

From: Michal Hocko <[email protected]>

inactive_list_is_low is duplicating logic implemented by lruvec_lru_size_eligibe_zones. Let's use the dedicated function to get the number of eligible pages on the lru list and ask use lruvec_lru_size to get the total LRU lize only when the tracing is really requested. We are still iterating over all LRUs two times in that case but a) inactive_list_is_low is not a hot path and b) this can be addressed at the tracing layer and only evaluate arguments only when the tracing is enabled in future if that ever matters.

Signed-off-by Michal Hocko <[email protected]>

---

From: Michal Hocko <[email protected]>

inactive_list_is_low is duplicating logic implemented by lruvec_lru_size_eligibe_zones. Let's use the dedicated function to get the number of eligible pages on the lru list and ask use lruvec_lru_size to get the total LRU lize only when the tracing is really requested. We are still iterating over all LRUs two times in that case but a) inactive_list_is_low is not a hot path and b) this can be addressed at the tracing layer and only evaluate arguments only when the tracing is enabled in future if that ever matters.

Signed-off-by Michal Hocko <[email protected]>