Skip to contentSkip to conversation

memcg, vmscan: Do not wait for writeback if killed

PATCH In review Michal Hocko 2 participants 4 messages Dec 2, 2015 – Dec 5, 2015

Subsystems (4)
lore
Share & tools
lore

Michal Hocko

author reviewer
#0 lore raw

PATCH 1/1 memcg, vmscan: Do not wait for writeback if killed

From: Michal Hocko <[email protected]>

Legacy memcg reclaim waits for pages under writeback to prevent from a premature oom killer invocation because there was no memcg dirty limit throttling implemented back then.

This heuristic might complicate situation when the writeback cannot make forward progress because of the global OOM situation. E.g. filesystem backed by the loop device relies on the underlying filesystem hosting the image to make forward progress which cannot be guaranteed and so we might end up triggering OOM killer to resolve the situation. If the oom victim happens to be the task stuck in wait_on_page_writeback in the memcg reclaim then we are basically deadlocked.

Introduce wait_on_page_writeback_killable and use it in this path to prevent from the issue. shrink_page_list will back off if the wait was interrupted.

---
Hi,
this came out from the discussion with Vladimir [1]. I haven't seen a deadlock presented in the above description but I fail to see why it would be impossible either. wait_on_page_writeback_killable is an utterly long name but wait_on_page_bit_killable is quite long already and I felt it is better to use the helper.

Thoughts?

[1] http://lkml.kernel.org/r/[email protected]

 include/linux/pagemap.h |  8 ++++++++
 mm/vmscan.c             | 11 ++++++++++-
 2 files changed, 18 insertions(+), 1 deletion(-)
include/linux/pagemap.h +8 -0
index 4d08b6c33557..d3bb8963f8f0 100644
@@ -532,6 +532,14 @@ static inline void wait_on_page_writeback(struct page *page)
 		wait_on_page_bit(page, PG_writeback);
 }
 
+static inline int wait_on_page_writeback_killable(struct page *page)
+{
+	if (PageWriteback(page))
+		return wait_on_page_bit_killable(page, PG_writeback);
+
+	return 0;
+}
+
 extern void end_page_writeback(struct page *page);
 void wait_for_stable_page(struct page *page);
 
mm/vmscan.c +10 -1
index 4589cfdbe405..98a1934493af 100644
@@ -1021,10 +1021,19 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 
 			/* Case 3 above */
 			} else {
+				int ret;
+
 				unlock_page(page);
-				wait_on_page_writeback(page);
+				ret = wait_on_page_writeback_killable(page);
 				/* then go back and try same page again */
 				list_add_tail(&page->lru, page_list);
+
+				/*
+				 * We've got killed while waiting here so
+				 * expedite our way out from the reclaim
+				 */
+				if (ret)
+					break;
 				continue;
 			}
 		}
Signature · 7 lines
2.6.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
maintainer
#1 lore raw
Quoted 33 lines · On Wed, 2 Dec 2015 15:26:18 +0100 Michal Hocko <[email protected]> wrote:

From: Michal Hocko <[email protected]>

Legacy memcg reclaim waits for pages under writeback to prevent from a premature oom killer invocation because there was no memcg dirty limit throttling implemented back then.

This heuristic might complicate situation when the writeback cannot make forward progress because of the global OOM situation. E.g. filesystem backed by the loop device relies on the underlying filesystem hosting the image to make forward progress which cannot be guaranteed and so we might end up triggering OOM killer to resolve the situation. If the oom victim happens to be the task stuck in wait_on_page_writeback in the memcg reclaim then we are basically deadlocked.

Introduce wait_on_page_writeback_killable and use it in this path to prevent from the issue. shrink_page_list will back off if the wait was interrupted.

...

mm/vmscan.c +3 -1
@@ -1021,10 +1021,19 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 
 			/* Case 3 above */
 			} else {
+				int ret;
+
 				unlock_page(page);
-				wait_on_page_writeback(page);
+				ret = wait_on_page_writeback_killable(page);
 				/* then go back and try same page again */
 				list_add_tail(&page->lru, page_list);

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 1039

Quoted 10 lines
mm/vmscan.c +7 -0
+
+				/*
+				 * We've got killed while waiting here so
+				 * expedite our way out from the reclaim
+				 */
+				if (ret)
+					break;
 				continue;
 			}
 		}

This function is 350 lines long and it takes a bit of effort to work out what that `break' is breaking from and where it goes next. I think you want a "goto keep_killed" here for consistency and sanity.

Also, there's high risk here of a pending signal causing the code to fall into some busy loop where it repeatedly tries to do something but then bales out without doing it. It's unobvious how this change avoids such things. (Maybe it *does* avoid such things, but it should be obvious!).

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>

author reviewer
#2 lore raw
Quoted 41 lines · Andrew Morton
Quoted 39 lines · On Wed, 2 Dec 2015 15:26:18 +0100 Michal Hocko <[email protected]> wrote:

From: Michal Hocko <[email protected]>

Legacy memcg reclaim waits for pages under writeback to prevent from a premature oom killer invocation because there was no memcg dirty limit throttling implemented back then.

This heuristic might complicate situation when the writeback cannot make forward progress because of the global OOM situation. E.g. filesystem backed by the loop device relies on the underlying filesystem hosting the image to make forward progress which cannot be guaranteed and so we might end up triggering OOM killer to resolve the situation. If the oom victim happens to be the task stuck in wait_on_page_writeback in the memcg reclaim then we are basically deadlocked.

Introduce wait_on_page_writeback_killable and use it in this path to prevent from the issue. shrink_page_list will back off if the wait was interrupted.

...

mm/vmscan.c +9 -1
@@ -1021,10 +1021,19 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 
 			/* Case 3 above */
 			} else {
+				int ret;
+
 				unlock_page(page);
-				wait_on_page_writeback(page);
+				ret = wait_on_page_writeback_killable(page);
 				/* then go back and try same page again */
 				list_add_tail(&page->lru, page_list);
+
+				/*
+				 * We've got killed while waiting here so
+				 * expedite our way out from the reclaim
+				 */
+				if (ret)
Quoted 8 lines

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 1039

Quoted 4 lines
mm/vmscan.c +1 -0
+					break;
 				continue;
 			}
 		}

This function is 350 lines long and it takes a bit of effort to work out what that `break' is breaking from and where it goes next. I think you want a "goto keep_killed" here for consistency and sanity.

Yeah, sounds better. See an update below:

Quoted 5 lines

Also, there's high risk here of a pending signal causing the code to fall into some busy loop where it repeatedly tries to do something but then bales out without doing it. It's unobvious how this change avoids such things. (Maybe it *does* avoid such things, but it should be obvious!).

shrink_page_list is called from __alloc_contig_migrate_range and shrink_inactive_list. Both of them handle fatal_signal_pending and bail out. I was relying on this behavior. I realize this is far from optimal wrt. readability but I do not have a great idea how to improve it without sticking more fatal_signal_pending checks into the reclaim path.

So you think a comment would be sufficient?
---

mm/vmscan.c +5 -1
index 98a1934493af..2e8ee9e5fcb5 100644
@@ -1031,9 +1031,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 				/*
 				 * We've got killed while waiting here so
 				 * expedite our way out from the reclaim
+				 *
+				 * Our callers should make sure we do not
+				 * get here with fatal signals again.
 				 */
 				if (ret)
-					break;
+					goto keep_killed;
 				continue;
 			}
 		}
@@ -1227,6 +1230,7 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 		VM_BUG_ON_PAGE(PageLRU(page) || PageUnevictable(page), page);
 	}
 
+keep_killed:
 	mem_cgroup_uncharge_list(&free_pages);
 	try_to_unmap_flush();
 	free_hot_cold_page_list(&free_pages, true);
Signature · 8 lines
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to [email protected].  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
maintainer
#3 lore raw
Quoted 7 lines · On Thu, 3 Dec 2015 10:08:26 +0100 Michal Hocko <[email protected]> wrote:

So you think a comment would be sufficient?
---

mm/vmscan.c +0 -0
index 98a1934493af..2e8ee9e5fcb5 100644

mm/vmscan.c static unsigned long shrink_page_list(struct list_head *page_list, · line 1037

Quoted 7 lines
mm/vmscan.c +3 -0
@@ -1031,9 +1031,12 @@ static unsigned long shrink_page_list(struct list_head *page_list,
 				/*
 				 * We've got killed while waiting here so
 				 * expedite our way out from the reclaim
+				 *
+				 * Our callers should make sure we do not
+				 * get here with fatal signals again.
 				 */

Seems OK. s/should/must/

Please resend it all after the usual exhaustive testing ;)