FAILED: patch "[PATCH] mm, vmscan: prevent infinite loop for costly GFP_NOIO |" failed to apply to 4.19-stable tree
Subsystems (6)
[email protected]
maintainerThe patch below does not apply to the 4.19-stable tree.
If someone wants it applied there, or to any other stable or longterm tree, then please email the backport, including the original git commit id to <[email protected]>.
To reproduce the conflict and resubmit, you may use the following commands:
git fetch https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/ linux-4.19.y
git checkout FETCH_HEAD
git cherry-pick -x 803de9000f334b771afacb6ff3e78622916668b0
# <resolve conflicts, build, test, etc.>
git commit -s
git send-email --to '<[email protected]>' --in-reply-to '2024032732-prowess-craving-9106@gregkh' --subject-prefix 'PATCH 4.19.y' HEAD^..
Possible dependencies:
803de9000f33 ("mm, vmscan: prevent infinite loop for costly GFP_NOIO | __GFP_RETRY_MAYFAIL allocations") f98a497e1f16 ("mm: compaction: remove unnecessary is_via_compact_memory() checks") e8606320e9af ("mm: compaction: refactor __compaction_suitable()")fe573327ffb1 ("tracing: incorrect gfp_t conversion")cff387d6a294 ("mm: compaction: make compaction_zonelist_suitable return false when COMPACT_SUCCESS") 9353ffa6e9e9 ("kasan, page_alloc: allow skipping memory init for HW_TAGS")53ae233c30a6 ("kasan, page_alloc: allow skipping unpoisoning for HW_TAGS")f49d9c5bb15c ("kasan, mm: only define ___GFP_SKIP_KASAN_POISON with HW_TAGS") e9d0ca922816 ("kasan, page_alloc: rework kasan_unpoison_pages call site")7e3cbba65de2 ("kasan, page_alloc: move kernel_init_free_pages in post_alloc_hook") 89b271163328 ("kasan, page_alloc: move SetPageSkipKASanPoison in post_alloc_hook") 9294b1281d0a ("kasan, page_alloc: combine tag_clear_highpage calls in post_alloc_hook") b42090ae6f3a ("kasan, page_alloc: merge kasan_alloc_pages into post_alloc_hook") b8491b9052fe ("kasan, page_alloc: refactor init checks in post_alloc_hook") 1c0e5b24f117 ("kasan: only apply __GFP_ZEROTAGS when memory is zeroed")c82ce3195fd1 ("mm: clarify __GFP_ZEROTAGS comment")7c13c163e036 ("kasan, page_alloc: merge kasan_free_pages into free_pages_prepare") 5b2c07138cbd ("kasan, page_alloc: move tag_clear_highpage out of kernel_init_free_pages") 94ae8b83fefc ("kasan, page_alloc: deduplicate should_skip_kasan_poison")3bf03b9a0839 ("Merge branch 'akpm' (patches from Andrew)")
thanks,
greg k-h
------------------ original commit in Linus's tree ------------------
From 803de9000f334b771afacb6ff3e78622916668b0 Mon Sep 17 00:00:00 2001
From: Vlastimil Babka <[email protected]>
Date: Wed, 21 Feb 2024 12:43:58 +0100
Subject: [PATCH] mm, vmscan: prevent infinite loop for costly GFP_NOIO |
__GFP_RETRY_MAYFAIL allocations
Sven reports an infinite loop in __alloc_pages_slowpath() for costly order __GFP_RETRY_MAYFAIL allocations that are also GFP_NOIO. Such combination can happen in a suspend/resume context where a GFP_KERNEL allocation can have __GFP_IO masked out via gfp_allowed_mask.
Quoting Sven:
1. try to do a "costly" allocation (order > PAGE_ALLOC_COSTLY_ORDER) with __GFP_RETRY_MAYFAIL set.
2. page alloc's __alloc_pages_slowpath tries to get a page from the freelist. This fails because there is nothing free of that costly order.
3. page alloc tries to reclaim by calling __alloc_pages_direct_reclaim, which bails out because a zone is ready to be compacted; it pretends to have made a single page of progress.
4. page alloc tries to compact, but this always bails out early because __GFP_IO is not set (it's not passed by the snd allocator, and even if it were, we are suspending so the __GFP_IO flag would be cleared anyway).
5. page alloc believes reclaim progress was made (because of the pretense in item 3) and so it checks whether it should retry compaction. The compaction retry logic thinks it should try again, because:
a) reclaim is needed because of the early bail-out in item 4
b) a zonelist is suitable for compaction
6. goto 2. indefinite stall.
(end quote)
The immediate root cause is confusing the COMPACT_SKIPPED returned from __alloc_pages_direct_compact() (step 4) due to lack of __GFP_IO to be indicating a lack of order-0 pages, and in step 5 evaluating that in should_compact_retry() as a reason to retry, before incrementing and limiting the number of retries. There are however other places that wrongly assume that compaction can happen while we lack __GFP_IO.
To fix this, introduce gfp_compaction_allowed() to abstract the __GFP_IO evaluation and switch the open-coded test in try_to_compact_pages() to use it.
Also use the new helper in:
- compaction_ready(), which will make reclaim not bail out in step 3, so there's at least one attempt to actually reclaim, even if chances are small for a costly order
- in_reclaim_compaction() which will make should_continue_reclaim() return false and we don't over-reclaim unnecessarily
- in __alloc_pages_slowpath() to set a local variable can_compact, which is then used to avoid retrying reclaim/compaction for costly allocations (step 5) if we can't compact and also to skip the early compaction attempt that we do in some cases
include/linux/gfp.h +9 -0
@@ -353,6 +353,15 @@ static inline bool gfp_has_io_fs(gfp_t gfp)return (gfp & (__GFP_IO | __GFP_FS)) == (__GFP_IO | __GFP_FS); } +/* +* Check if the gfp flags allow compaction - GFP_NOIO is a really +* tricky context because the migration might require IO. +*/ +static inline bool gfp_compaction_allowed (gfp_t gfp_mask) +{ +return IS_ENABLED (CONFIG_COMPACTION ) && (gfp_mask & __GFP_IO); +} +extern gfp_t vma_thp_gfp_mask (struct vm_area_struct *vma);#ifdef CONFIG_CONTIG_ALLOC
mm/compaction.c +1 -6
@@ -2723,16 +2723,11 @@ enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order, unsigned int alloc_flags, const struct alloc_context *ac, enum compact_priority prio, struct page **capture) { - int may_perform_io = (__force int)(gfp_mask & __GFP_IO); struct zoneref *z;struct zone *zone;enum compact_result rc =COMPACT_SKIPPED ; -/* -* Check if the GFP flags allow compaction - GFP_NOIO is really -* tricky context because the migration might require IO -*/ -if (!may_perform_io) +if (!gfp_compaction_allowed (gfp_mask))return COMPACT_SKIPPED ;trace_mm_compaction_try_to_compact_pages (order, gfp_mask, prio);
mm/page_alloc.c +6 -4
@@ -4041,6 +4041,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, struct alloc_context *ac) {bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM; +bool can_compact =gfp_compaction_allowed (gfp_mask);const bool costly_order = order >PAGE_ALLOC_COSTLY_ORDER ;struct page *page =NULL ;unsigned int alloc_flags;
@@ -4111,7 +4112,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,* Don't try this for allocations that are allowed to ignore * watermarks, as the ALLOC_NO_WATERMARKS attempt didn't yet happen. */ -if (can_direct_reclaim && +if (can_direct_reclaim && can_compact && (costly_order || (order >0 && ac->migratetype !=MIGRATE_MOVABLE )) && !gfp_pfmemalloc_allowed (gfp_mask)) {
@@ -4209,9 +4210,10 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,/* * Do not retry costly high order allocations unless they are -* __GFP_RETRY_MAYFAIL +* __GFP_RETRY_MAYFAIL and we can compact */ -if (costly_order && !(gfp_mask & __GFP_RETRY_MAYFAIL)) +if (costly_order && (!can_compact || + !(gfp_mask & __GFP_RETRY_MAYFAIL))) goto nopage; if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags,
@@ -4224,7 +4226,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,* implementation of the compaction depends on the sufficient amount * of free memory (see __compaction_suitable) */ -if (did_some_progress >0 && +if (did_some_progress >0 && can_compact &&should_compact_retry (ac, order, alloc_flags, compact_result, &compact_priority, &compaction_retries))
mm/vmscan.c +4 -1
@@ -5753,7 +5753,7 @@ static void shrink_lruvec(struct lruvec *lruvec, struct scan_control *sc)/* Use reclaim/compaction for costly allocs or under memory pressure */ static bool in_reclaim_compaction (struct scan_control *sc) { -if (IS_ENABLED (CONFIG_COMPACTION ) && sc->order && +if (gfp_compaction_allowed (sc->gfp_mask) && sc->order && (sc->order >PAGE_ALLOC_COSTLY_ORDER || sc->priority <DEF_PRIORITY -2 ))return true;
@@ -5998,6 +5998,9 @@ static inline bool compaction_ready(struct zone *zone, struct scan_control *sc) {unsigned long watermark; +if (!gfp_compaction_allowed (sc->gfp_mask)) +return false; +/* Allocation can already succeed, nothing to do */ if (zone_watermark_ok (zone, sc->order,min_wmark_pages (zone), sc->reclaim_idx,0 ))
PATCH 1/1 mm, vmscan: prevent infinite loop for costly GFP_NOIO | __GFP_RETRY_MAYFAIL allocations
Sven reports an infinite loop in __alloc_pages_slowpath() for costly order __GFP_RETRY_MAYFAIL allocations that are also GFP_NOIO. Such combination can happen in a suspend/resume context where a GFP_KERNEL allocation can have __GFP_IO masked out via gfp_allowed_mask.
Quoting Sven:
1. try to do a "costly" allocation (order > PAGE_ALLOC_COSTLY_ORDER) with __GFP_RETRY_MAYFAIL set.
2. page alloc's __alloc_pages_slowpath tries to get a page from the freelist. This fails because there is nothing free of that costly order.
3. page alloc tries to reclaim by calling __alloc_pages_direct_reclaim, which bails out because a zone is ready to be compacted; it pretends to have made a single page of progress.
4. page alloc tries to compact, but this always bails out early because __GFP_IO is not set (it's not passed by the snd allocator, and even if it were, we are suspending so the __GFP_IO flag would be cleared anyway).
5. page alloc believes reclaim progress was made (because of the pretense in item 3) and so it checks whether it should retry compaction. The compaction retry logic thinks it should try again, because:
a) reclaim is needed because of the early bail-out in item 4
b) a zonelist is suitable for compaction
6. goto 2. indefinite stall.
(end quote)
The immediate root cause is confusing the COMPACT_SKIPPED returned from __alloc_pages_direct_compact() (step 4) due to lack of __GFP_IO to be indicating a lack of order-0 pages, and in step 5 evaluating that in should_compact_retry() as a reason to retry, before incrementing and limiting the number of retries. There are however other places that wrongly assume that compaction can happen while we lack __GFP_IO.
To fix this, introduce gfp_compaction_allowed() to abstract the __GFP_IO evaluation and switch the open-coded test in try_to_compact_pages() to use it.
Also use the new helper in:
- compaction_ready(), which will make reclaim not bail out in step 3, so there's at least one attempt to actually reclaim, even if chances are small for a costly order
- in_reclaim_compaction() which will make should_continue_reclaim() return false and we don't over-reclaim unnecessarily
- in __alloc_pages_slowpath() to set a local variable can_compact, which is then used to avoid retrying reclaim/compaction for costly allocations (step 5) if we can't compact and also to skip the early compaction attempt that we do in some cases
--- include/linux/gfp.h | 9 +++++++++ mm/compaction.c | 7 +------ mm/page_alloc.c | 10 ++++++---- mm/vmscan.c | 5 ++++- 4 files changed, 20 insertions(+), 11 deletions(-)
include/linux/gfp.h +9 -0
@@ -608,6 +608,15 @@ static inline bool pm_suspended_storage(void) }#endif /* CONFIG_PM_SLEEP */ +/* +* Check if the gfp flags allow compaction - GFP_NOIO is a really +* tricky context because the migration might require IO. +*/ +static inline bool gfp_compaction_allowed (gfp_t gfp_mask) +{ +return IS_ENABLED (CONFIG_COMPACTION ) && (gfp_mask & __GFP_IO); +} +#if (defined(CONFIG_MEMORY_ISOLATION ) && defined(CONFIG_COMPACTION )) || defined(CONFIG_CMA )/* The below functions must be run on a range from a single zone. */ extern int alloc_contig_range(unsigned long start, unsigned long end,
mm/compaction.c +1 -6
@@ -1751,16 +1751,11 @@ enum compact_result try_to_compact_pages(gfp_t gfp_mask, unsigned int order, unsigned int alloc_flags, const struct alloc_context *ac, enum compact_priority prio) { - int may_perform_io = gfp_mask & __GFP_IO; struct zoneref *z;struct zone *zone;enum compact_result rc =COMPACT_SKIPPED ; -/* -* Check if the GFP flags allow compaction - GFP_NOIO is really -* tricky context because the migration might require IO -*/ -if (!may_perform_io) +if (!gfp_compaction_allowed (gfp_mask))return COMPACT_SKIPPED ;trace_mm_compaction_try_to_compact_pages (order, gfp_mask, prio);
mm/page_alloc.c +6 -4
@@ -4099,6 +4099,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order, struct alloc_context *ac) {bool can_direct_reclaim = gfp_mask & __GFP_DIRECT_RECLAIM; +bool can_compact =gfp_compaction_allowed (gfp_mask);const bool costly_order = order >PAGE_ALLOC_COSTLY_ORDER ;struct page *page =NULL ;unsigned int alloc_flags;
@@ -4164,7 +4165,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,* Don't try this for allocations that are allowed to ignore * watermarks, as the ALLOC_NO_WATERMARKS attempt didn't yet happen. */ -if (can_direct_reclaim && +if (can_direct_reclaim && can_compact && (costly_order || (order >0 && ac->migratetype !=MIGRATE_MOVABLE )) && !gfp_pfmemalloc_allowed (gfp_mask)) {
@@ -4251,9 +4252,10 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,/* * Do not retry costly high order allocations unless they are -* __GFP_RETRY_MAYFAIL +* __GFP_RETRY_MAYFAIL and we can compact */ -if (costly_order && !(gfp_mask & __GFP_RETRY_MAYFAIL)) +if (costly_order && (!can_compact || + !(gfp_mask & __GFP_RETRY_MAYFAIL))) goto nopage; if (should_reclaim_retry(gfp_mask, order, ac, alloc_flags,
@@ -4266,7 +4268,7 @@ __alloc_pages_slowpath(gfp_t gfp_mask, unsigned int order,* implementation of the compaction depends on the sufficient amount * of free memory (see __compaction_suitable) */ -if (did_some_progress >0 && +if (did_some_progress >0 && can_compact &&should_compact_retry (ac, order, alloc_flags, compact_result, &compact_priority, &compaction_retries))
mm/vmscan.c +4 -1
@@ -2595,7 +2595,7 @@ static void shrink_node_memcg(struct pglist_data *pgdat, struct mem_cgroup *memc/* Use reclaim/compaction for costly allocs or under memory pressure */ static bool in_reclaim_compaction (struct scan_control *sc) { -if (IS_ENABLED (CONFIG_COMPACTION ) && sc->order && +if (gfp_compaction_allowed (sc->gfp_mask) && sc->order && (sc->order >PAGE_ALLOC_COSTLY_ORDER || sc->priority <DEF_PRIORITY -2 ))return true;
@@ -2869,6 +2869,9 @@ static inline bool compaction_ready(struct zone *zone, struct scan_control *sc)unsigned long watermark;enum compact_result suitable; +if (!gfp_compaction_allowed (sc->gfp_mask)) +return false; + suitable =compaction_suitable (zone, sc->order,0 , sc->reclaim_idx);if (suitable ==COMPACT_SUCCESS )/* Allocation should succeed already. Don't reclaim. */
Signature · 1 line
2.44.0
Re: [PATCH 4.19.y] mm, vmscan: prevent infinite loop for costly GFP_NOIO | __GFP_RETRY_MAYFAIL allocations
Quoted 56 lines · Vlastimil Babka
Sven reports an infinite loop in
__alloc_pages_slowpath()for costly order__GFP_RETRY_MAYFAILallocations that are alsoGFP_NOIO. Such combination can happen in a suspend/resume context where aGFP_KERNELallocation can have__GFP_IOmasked out viagfp_allowed_mask.Quoting Sven:
1. try to do a "costly" allocation (order >
PAGE_ALLOC_COSTLY_ORDER) with__GFP_RETRY_MAYFAILset.2. page alloc's
__alloc_pages_slowpathtries to get a page from the freelist. This fails because there is nothing free of that costly order.3. page alloc tries to reclaim by calling
__alloc_pages_direct_reclaim, which bails out because a zone is ready to be compacted; it pretends to have made a single page of progress.4. page alloc tries to compact, but this always bails out early because
__GFP_IOis not set (it's not passed by the snd allocator, and even if it were, we are suspending so the__GFP_IOflag would be cleared anyway).5. page alloc believes reclaim progress was made (because of the pretense in item 3) and so it checks whether it should retry compaction. The compaction retry logic thinks it should try again, because:
a) reclaim is needed because of the early bail-out in item 4
b) a zonelist is suitable for compaction6. goto 2. indefinite stall.
(end quote)
The immediate root cause is confusing the
COMPACT_SKIPPEDreturned from__alloc_pages_direct_compact()(step 4) due to lack of__GFP_IOto be indicating a lack of order-0 pages, and in step 5 evaluating that inshould_compact_retry()as a reason to retry, before incrementing and limiting the number of retries. There are however other places that wrongly assume that compaction can happen while we lack__GFP_IO.To fix this, introduce
gfp_compaction_allowed()to abstract the__GFP_IOevaluation and switch the open-coded test intry_to_compact_pages()to use it.Also use the new helper in:
-compaction_ready(), which will make reclaim not bail out in step 3, so there's at least one attempt to actually reclaim, even if chances are small for a costly order
-in_reclaim_compaction()which will makeshould_continue_reclaim()return false and we don't over-reclaim unnecessarily
- in__alloc_pages_slowpath()to set a local variablecan_compact, which is then used to avoid retrying reclaim/compaction for costly allocations (step 5) if we can't compact and also to skip the early compaction attempt that we do in some cases
Quoted 16 lines
Fixes 3250845d0526 ("Revert "mm, oom: prevent premature OOM killer invocation for high order request"")Signed-off-by Vlastimil Babka <[email protected]>Reported-by Sven van Ashbrook <[email protected]>Closes https://lore.kernel.org/all/CAG-rBihs_xMKb3wrMO1%2B-%2Bp4fowP9oy1pa_OTkfxBzPUVOZF%[email protected]/ open hereCc Brian Geffon <[email protected]>Cc Curtis Malainey <[email protected]>Cc Jaroslav Kysela <[email protected]>Cc Mel Gorman <[email protected]>Cc Michal Hocko <[email protected]>Cc Takashi Iwai <[email protected]>Cc <[email protected]>Signed-off-by Andrew Morton <[email protected]>(cherry picked from commit 803de9000f334b771afacb6ff3e78622916668b0)Signed-off-by Vlastimil Babka <[email protected]>
All backports now queued up, thanks!
greg k-h