Skip to contentSkip to conversation

mm: follow up oom fixes for 32b

PATCH In review Michal Hocko 3 participants 6 messages Feb 28, 2017 – Mar 8, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover mm: follow up oom fixes for 32b

Hi,
later in the 4.10 release cycle it turned out that b4536f0c829c ("mm,
memcg: fix the active list aging for lowmem requests when memcg is enabled") was not sufficient to fully close the regression introduced by f8d1a31163fc ("mm: consider whether to decivate based on eligible zones inactive ratio") [1]. mmotm tree behaved properly and it turned out the Linus tree was missing 71ab6cfe88dc ("mm, vmscan: consider eligible zones in get_scan_count") merged in 4.11 merge window. The patch heavily depends on 4a9494a3d827 ("mm, vmscan: cleanup lru size claculations") which has been backported as well (patch 1).

Please add these two to 4.9+ trees (they should apply to 4.10 as they are). 4.8 tree will need them as well but I do not see this stable tree being maintained.

[1] http://lkml.kernel.org/r/[email protected]

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#1 lore raw

PATCH 1/2 mm, vmscan: cleanup lru size claculations

From: Michal Hocko <[email protected]>

commit fd538803731e50367b7c59ce4ad3454426a3d671 upstream.

lruvec_lru_size returns the full size of the LRU list while we sometimes need a value reduced only to eligible zones (e.g. for lowmem requests). inactive_list_is_low is one such user. Later patches will add more of them. Add a new parameter to lruvec_lru_size and allow it filter out zones which are not eligible for the given context.

---
 include/linux/mmzone.h |  2 +-
 mm/vmscan.c            | 81 ++++++++++++++++++++++++--------------------------
 mm/workingset.c        |  2 +-
 3 files changed, 41 insertions(+), 44 deletions(-)
include/linux/mmzone.h +1 -1
index f99c993dd500..7e273e243a13 100644
@@ -779,7 +779,7 @@ static inline struct pglist_data *lruvec_pgdat(struct lruvec *lruvec)
 #endif
 }
 
-extern unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru);
+extern unsigned long lruvec_lru_size(struct lruvec *lruvec, enum lru_list lru, int zone_idx);
 
 #ifdef CONFIG_HAVE_MEMORY_PRESENT
 void memory_present(int nid, unsigned long start, unsigned long end);
mm/vmscan.c +39 -42

123 lines not on this page · raw message

mm/workingset.c +1 -1
index fb1f9183d89a..33f6f4db32fd 100644
@@ -266,7 +266,7 @@ bool workingset_refault(void *shadow)
 	}
 	lruvec = mem_cgroup_lruvec(pgdat, memcg);
 	refault = atomic_long_read(&lruvec->inactive_age);
-	active_file = lruvec_lru_size(lruvec, LRU_ACTIVE_FILE);
+	active_file = lruvec_lru_size(lruvec, LRU_ACTIVE_FILE, MAX_NR_ZONES);
 	rcu_read_unlock();
 
 	/*
Signature · 7 lines
2.11.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#2 lore raw

PATCH 2/2 mm, vmscan: consider eligible zones in get_scan_count

In reply to Michal Hocko

From: Michal Hocko <[email protected]>

commit 71ab6cfe88dcf9f6e6a65eb85cf2bda20a257682 upstream.

get_scan_count() considers the whole node LRU size when

- doing SCAN_FILE due to many page cache inactive pages
- calculating the number of pages to scan

In both cases this might lead to unexpected behavior especially on 32b systems where we can expect lowmem memory pressure very often.

A large highmem zone can easily distort SCAN_FILE heuristic because there might be only few file pages from the eligible zones on the node lru and we would still enforce file lru scanning which can lead to trashing while we could still scan anonymous pages.

The later use of lruvec_lru_size can be problematic as well. Especially when there are not many pages from the eligible zones. We would have to skip over many pages to find anything to reclaim but shrink_node_memcg would only reduce the remaining number to scan by SWAP_CLUSTER_MAX at maximum. Therefore we can end up going over a large LRU many times without actually having chance to reclaim much if anything at all. The closer we are out of memory on lowmem zone the worse the problem will be.

Fix this by filtering out all the ineligible zones when calculating the lru size for both paths and consider only sc->reclaim_idx zones.

The patch would need to be tweaked a bit to apply to 4.10 and older but I will do that as soon as it hits the Linus tree in the next merge window.

---
 mm/vmscan.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/vmscan.c +2 -2
index cd516c632e8f..30a88b945a44 100644
@@ -2205,7 +2205,7 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
 	 * system is under heavy pressure.
 	 */
 	if (!inactive_list_is_low(lruvec, true, sc) &&
-	    lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, MAX_NR_ZONES) >> sc->priority) {
+	    lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, sc->reclaim_idx) >> sc->priority) {
 		scan_balance = SCAN_FILE;
 		goto out;
 	}
@@ -2272,7 +2272,7 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
 			unsigned long size;
 			unsigned long scan;
 
-			size = lruvec_lru_size(lruvec, lru, MAX_NR_ZONES);
+			size = lruvec_lru_size(lruvec, lru, sc->reclaim_idx);
 			scan = size >> sc->priority;
 
 			if (!scan && pass && force_scan)
Signature · 7 lines
2.11.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#3 lore raw

Re: [PATCH stable-4.9 2/2] mm, vmscan: consider eligible zones in get_scan_count

Hi Michal,

I'm over a very long kernel bisection, and if I made no mistake in testing commits, this patch fixes a kernel bug which affects my HP Compaq dc5800 machine with 32 bit Ubuntu OS.

The bug manifests itself with "NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kswapd0:38]" messages in 4.8 kernels, and "page allocation stalls for 47608ms, order:1, mode:0x17000c0(GFP_KERNEL_ACCOUNT|__GFP_NOTRACK)" in 4.10 kernels up to this commit.

Michal, can you confirm that this patch may fix issues like the ones I encountered? If so, I'll try to get the Ubuntu kernel staff to backport this commit to Yakkety's 4.8 kernel. On the other hand, I can't seem to be able to backport this commit to 4.8 with "git cherry-pick", so maybe I need to wait for your tweaks you mentioned.

Anyway, thank you very much for the fix!

Regards,
MegaBrutal

2017-02-28 16:11 GMT+01:00 Michal Hocko <[email protected]>:

Quoted 72 lines

From: Michal Hocko <[email protected]>

commit 71ab6cfe88dcf9f6e6a65eb85cf2bda20a257682 upstream.

get_scan_count() considers the whole node LRU size when

- doing SCAN_FILE due to many page cache inactive pages
- calculating the number of pages to scan

In both cases this might lead to unexpected behavior especially on 32b systems where we can expect lowmem memory pressure very often.

A large highmem zone can easily distort SCAN_FILE heuristic because there might be only few file pages from the eligible zones on the node lru and we would still enforce file lru scanning which can lead to trashing while we could still scan anonymous pages.

The later use of lruvec_lru_size can be problematic as well. Especially when there are not many pages from the eligible zones. We would have to skip over many pages to find anything to reclaim but shrink_node_memcg would only reduce the remaining number to scan by SWAP_CLUSTER_MAX at maximum. Therefore we can end up going over a large LRU many times without actually having chance to reclaim much if anything at all. The closer we are out of memory on lowmem zone the worse the problem will be.

Fix this by filtering out all the ineligible zones when calculating the lru size for both paths and consider only sc->reclaim_idx zones.

The patch would need to be tweaked a bit to apply to 4.10 and older but I will do that as soon as it hits the Linus tree in the next merge window.

---
 mm/vmscan.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
mm/vmscan.c +2 -3
index cd516c632e8f..30a88b945a44 100644
@@ -2205,7 +2205,7 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
         * system is under heavy pressure.
         */
        if (!inactive_list_is_low(lruvec, true, sc) &&
-           lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, MAX_NR_ZONES) >> sc->priority) {
+           lruvec_lru_size(lruvec, LRU_INACTIVE_FILE, sc->reclaim_idx) >> sc->priority) {
                scan_balance = SCAN_FILE;
                goto out;
        }
@@ -2272,7 +2272,7 @@ static void get_scan_count(struct lruvec *lruvec, struct mem_cgroup *memcg,
                        unsigned long size;
                        unsigned long scan;
 
-                       size = lruvec_lru_size(lruvec, lru, MAX_NR_ZONES);
+                       size = lruvec_lru_size(lruvec, lru, sc->reclaim_idx);
                        scan = size >> sc->priority;
 
                        if (!scan && pass && force_scan)
--
Quoted 2 lines

2.11.0

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

#4 lore raw

Re: [PATCH stable-4.9 2/2] mm, vmscan: consider eligible zones in get_scan_count

Quoted 11 lines · MegaBrutal

Hi Michal,

I'm over a very long kernel bisection, and if I made no mistake in testing commits, this patch fixes a kernel bug which affects my HP Compaq dc5800 machine with 32 bit Ubuntu OS.

The bug manifests itself with "NMI watchdog: BUG: soft lockup - CPU#0 stuck for 23s! [kswapd0:38]" messages in 4.8 kernels, and "page allocation stalls for 47608ms, order:1, mode:0x17000c0(GFP_KERNEL_ACCOUNT|__GFP_NOTRACK)" in 4.10 kernels up to this commit.

This is really hard to say without seeing the traces.

Quoted 5 lines

Michal, can you confirm that this patch may fix issues like the ones I encountered? If so, I'll try to get the Ubuntu kernel staff to backport this commit to Yakkety's 4.8 kernel. On the other hand, I can't seem to be able to backport this commit to 4.8 with "git cherry-pick", so maybe I need to wait for your tweaks you mentioned.

The backport from 4.9 to 4.8 shouldn't be very complicated I believe.

Quoted 1 line

Anyway, thank you very much for the fix!

Good to hear the patch helped you though.

Signature · 8 lines
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
#5 lore raw
In reply to Michal Hocko
Quoted 11 lines · Michal Hocko

Hi,
later in the 4.10 release cycle it turned out that b4536f0c829c ("mm,
memcg: fix the active list aging for lowmem requests when memcg is enabled") was not sufficient to fully close the regression introduced by f8d1a31163fc ("mm: consider whether to decivate based on eligible zones inactive ratio") [1]. mmotm tree behaved properly and it turned out the Linus tree was missing 71ab6cfe88dc ("mm, vmscan: consider eligible zones in get_scan_count") merged in 4.11 merge window. The patch heavily depends on 4a9494a3d827 ("mm, vmscan: cleanup lru size claculations") which has been backported as well (patch 1).

Quoted 3 lines

Please add these two to 4.9+ trees (they should apply to 4.10 as they are). 4.8 tree will need them as well but I do not see this stable tree being maintained.

For 4.10-stable I needed both of your backports, many thanks for them.

greg k-h

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>