mm/vmscan: restore allowed mask in alloc_demote_folio()
Subsystems (3)
PATCH 1/1 mm/vmscan: restore allowed mask in alloc_demote_folio()
In alloc_demote_folio(), mtc->nmask is set to NULL for the first allocation. If that succeeds, it returns without restoring mtc->nmask to allowed_mask. For subsequent allocations from the migrate_pages() batch, mtc->nmask will be NULL. If the target node then becomes full, the fallback allocation will use nmask = NULL, allocating from any node allowed by the task cpuset, which for kswapd is all nodes.
To address this issue, restore the mtc->nmask to its original allowed nodemask after the first allocation.
--- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/vmscan.c +1 -1
@@ -985,11 +985,11 @@ static struct folio *alloc_demote_folio(struct folio *src, mtc->nmask =NULL ; mtc->gfp_mask |= __GFP_THISNODE; dst =alloc_migration_target (src, (unsigned long )mtc); + mtc->nmask = allowed_mask;if (dst)return dst; mtc->gfp_mask &= ~__GFP_THISNODE; - mtc->nmask = allowed_mask;return alloc_migration_target (src, (unsigned long )mtc); }
--
2.53.0.473.g4a7958ca14-goog
David Hildenbrand (Arm)
reviewerQuoted 10 lines · Bing Jiao
In
alloc_demote_folio(),mtc->nmaskis set to NULL for the first allocation. If that succeeds, it returns without restoringmtc->nmasktoallowed_mask. For subsequent allocations from themigrate_pages()batch,mtc->nmaskwill be NULL. If the target node then becomes full, the fallback allocation will use nmask = NULL, allocating from any node allowed by the task cpuset, which for kswapd is all nodes.To address this issue, restore the
mtc->nmaskto its original allowed nodemask after the first allocation.
That would be
?
Quoted 23 lines
Signed-off-by Bing Jiao <[email protected]>--- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/vmscan.c +1
-2@@ -985,11 +985,11 @@ static struct folio *alloc_demote_folio(struct folio *src, mtc->nmask =NULL ; mtc->gfp_mask |= __GFP_THISNODE; dst =alloc_migration_target (src, (unsigned long )mtc); + mtc->nmask = allowed_mask;if (dst)return dst; mtc->gfp_mask &= ~__GFP_THISNODE; - mtc->nmask = allowed_mask;return alloc_migration_target (src, (unsigned long )mtc); } --
Quoted 2 lines
2.53.0.473.g4a7958ca14-goog
Maybe we should just not touch the original mtc?
mm/vmscan.c +5 -8
@@ -985,9 +985,9 @@ static void folio_check_dirty_writeback(struct folio *folio,static struct folio *alloc_demote_folio (struct folio *src,unsigned long private) { +struct migration_target_control *mtc, target_nid_mtc;struct folio *dst;nodemask_t *allowed_mask; -struct migration_target_control *mtc; mtc = (struct migration_target_control *)private;
@@ -1001,15 +1001,12 @@ static struct folio *alloc_demote_folio(struct folio *src,* a demotion of cold pages from the target memtier. This can result * in the kernel placing hot pages in slower(lower) memory tiers. */ - mtc->nmask =NULL ; - mtc->gfp_mask |= __GFP_THISNODE; - dst =alloc_migration_target (src, (unsigned long )mtc); + target_nid_mtc = *mtc; + target_nid_mtc.nmask =NULL ; + target_nid_mtc.gfp_mask |= __GFP_THISNODE; + dst =alloc_migration_target (src, (unsigned long )&target_nid_mtc);if (dst)return dst; - - mtc->gfp_mask &= ~__GFP_THISNODE; - mtc->nmask = allowed_mask; -return alloc_migration_target (src, (unsigned long )mtc); }
Signature · 3 lines
Cheers, David
Quoted 12 lines · David Hildenbrand (Arm)
Quoted 10 lines · Bing Jiao
In
alloc_demote_folio(),mtc->nmaskis set to NULL for the first allocation. If that succeeds, it returns without restoringmtc->nmasktoallowed_mask. For subsequent allocations from themigrate_pages()batch,mtc->nmaskwill be NULL. If the target node then becomes full, the fallback allocation will use nmask = NULL, allocating from any node allowed by the task cpuset, which for kswapd is all nodes.To address this issue, restore the
mtc->nmaskto its original allowed nodemask after the first allocation.
Quoted 5 lines
That would be
Fixes 320080272892 ("mm/demotion: demote pages according to allocation fallback order")?
Thanks for pointing it out. Will add it in the new patch.
Quoted 68 lines
Quoted 23 lines
Signed-off-by Bing Jiao <[email protected]>--- mm/vmscan.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/vmscan.c +1
-2@@ -985,11 +985,11 @@ static struct folio *alloc_demote_folio(struct folio *src, mtc->nmask =NULL ; mtc->gfp_mask |= __GFP_THISNODE; dst =alloc_migration_target (src, (unsigned long )mtc); + mtc->nmask = allowed_mask;if (dst)return dst; mtc->gfp_mask &= ~__GFP_THISNODE; - mtc->nmask = allowed_mask;return alloc_migration_target (src, (unsigned long )mtc); } --Quoted 2 lines
2.53.0.473.g4a7958ca14-goog
Maybe we should just not touch the original mtc?
mm/vmscan.c +5
-9@@ -985,9 +985,9 @@ static void folio_check_dirty_writeback(struct folio *folio,static struct folio *alloc_demote_folio (struct folio *src,unsigned long private) { +struct migration_target_control *mtc, target_nid_mtc;struct folio *dst;nodemask_t *allowed_mask; -struct migration_target_control *mtc; mtc = (struct migration_target_control *)private;@@ -1001,15 +1001,12 @@ static struct folio *alloc_demote_folio(struct folio *src,* a demotion of cold pages from the target memtier. This can result * in the kernel placing hot pages in slower(lower) memory tiers. */ - mtc->nmask =NULL ; - mtc->gfp_mask |= __GFP_THISNODE; - dst =alloc_migration_target (src, (unsigned long )mtc); + target_nid_mtc = *mtc; + target_nid_mtc.nmask =NULL ; + target_nid_mtc.gfp_mask |= __GFP_THISNODE; + dst =alloc_migration_target (src, (unsigned long )&target_nid_mtc);if (dst)return dst; - - mtc->gfp_mask &= ~__GFP_THISNODE; - mtc->nmask = allowed_mask; -return alloc_migration_target (src, (unsigned long )mtc); } --Cheers,
Quoted 1 line
David
Thank you for the suggestion, David.
I agree that not touching the original mtc is a better. It makes the distinction between the two allocation attempts much clearer and avoids the side-effect bug. Will update it then.
Best,
Bing
PATCH v2 1/1 mm/vmscan: fix unintended mtc->nmask mutation in alloc_demote_folio()
In reply to Bing JiaoIn alloc_demote_folio(), mtc->nmask is set to NULL for the first allocation. If that succeeds, it returns without restoring mtc->nmask to allowed_mask. For subsequent allocations from the migrate_pages() batch, mtc->nmask will be NULL. If the target node then becomes full, the fallback allocation will use nmask = NULL, allocating from any node allowed by the task cpuset, which for kswapd is all nodes.
To address this issue, use a local copy of the mtc structure with nmask = NULL for the first allocation attempt specifically, ensuring the original mtc remains unmodified.
--- mm/vmscan.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)
mm/vmscan.c +5 -9
@@ -966,13 +966,11 @@ static void folio_check_dirty_writeback(struct folio *folio,static struct folio *alloc_demote_folio (struct folio *src,unsigned long private) { +struct migration_target_control *mtc, target_nid_mtc;struct folio *dst; -nodemask_t *allowed_mask; -struct migration_target_control *mtc; mtc = (struct migration_target_control *)private; - allowed_mask = mtc->nmask;/* * make sure we allocate from the target node first also trying to * demote or reclaim pages from the target node via kswapd if we are
@@ -982,15 +980,13 @@ static struct folio *alloc_demote_folio(struct folio *src,* a demotion of cold pages from the target memtier. This can result * in the kernel placing hot pages in slower(lower) memory tiers. */ - mtc->nmask =NULL ; - mtc->gfp_mask |= __GFP_THISNODE; - dst =alloc_migration_target (src, (unsigned long )mtc); + target_nid_mtc = *mtc; + target_nid_mtc.nmask =NULL ; + target_nid_mtc.gfp_mask |= __GFP_THISNODE; + dst =alloc_migration_target (src, (unsigned long )&target_nid_mtc);if (dst)return dst; - mtc->gfp_mask &= ~__GFP_THISNODE; - mtc->nmask = allowed_mask; -return alloc_migration_target (src, (unsigned long )mtc); }
--
2.53.0.473.g4a7958ca14-goog
Re: [PATCH v2] mm/vmscan: fix unintended mtc->nmask mutation in alloc_demote_folio()
Quoted 11 lines · Bing Jiao
In
alloc_demote_folio(),mtc->nmaskis set to NULL for the first allocation. If that succeeds, it returns without restoringmtc->nmasktoallowed_mask. For subsequent allocations from themigrate_pages()batch,mtc->nmaskwill be NULL. If the target node then becomes full, the fallback allocation will use nmask = NULL, allocating from any node allowed by the task cpuset, which for kswapd is all nodes.To address this issue, use a local copy of the mtc structure with nmask = NULL for the first allocation attempt specifically, ensuring the original mtc remains unmodified.
Quoted 5 lines
Fixes 320080272892 ("mm/demotion: demote pages according to allocation fallback order")Signed-off-by Bing Jiao <[email protected]>--- mm/vmscan.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)
Signature · 3 lines
Cheers, David
Re: [PATCH v2] mm/vmscan: fix unintended mtc->nmask mutation in alloc_demote_folio()
In reply to Bing JiaoQuoted 7 lines · Bing Jiao
In
alloc_demote_folio(),mtc->nmaskis set to NULL for the first allocation. If that succeeds, it returns without restoringmtc->nmasktoallowed_mask. For subsequent allocations from themigrate_pages()batch,mtc->nmaskwill be NULL. If the target node then becomes full, the fallback allocation will use nmask = NULL, allocating from any node allowed by the task cpuset, which for kswapd is all nodes.
Quoted 6 lines
To address this issue, use a local copy of the mtc structure with nmask = NULL for the first allocation attempt specifically, ensuring the original mtc remains unmodified.
Fixes 320080272892 ("mm/demotion: demote pages according to allocation fallback order")Signed-off-by Bing Jiao <[email protected]>
LGTM, so:
Quoted 46 lines
--- mm/vmscan.c | 14 +++++--------- 1 file changed, 5 insertions(+), 9 deletions(-)mm/vmscan.c +5
-10@@ -966,13 +966,11 @@ static void folio_check_dirty_writeback(struct folio *folio,static struct folio *alloc_demote_folio (struct folio *src,unsigned long private) { +struct migration_target_control *mtc, target_nid_mtc;struct folio *dst; -nodemask_t *allowed_mask; -struct migration_target_control *mtc; mtc = (struct migration_target_control *)private; - allowed_mask = mtc->nmask;/* * make sure we allocate from the target node first also trying to * demote or reclaim pages from the target node via kswapd if we are @@ -982,15 +980,13 @@ static struct folio *alloc_demote_folio(struct folio *src,* a demotion of cold pages from the target memtier. This can result * in the kernel placing hot pages in slower(lower) memory tiers. */ - mtc->nmask =NULL ; - mtc->gfp_mask |= __GFP_THISNODE; - dst =alloc_migration_target (src, (unsigned long )mtc); + target_nid_mtc = *mtc; + target_nid_mtc.nmask =NULL ; + target_nid_mtc.gfp_mask |= __GFP_THISNODE; + dst =alloc_migration_target (src, (unsigned long )&target_nid_mtc);if (dst)return dst; - mtc->gfp_mask &= ~__GFP_THISNODE; - mtc->nmask = allowed_mask; -return alloc_migration_target (src, (unsigned long )mtc); } --2.53.0.473.g4a7958ca14-goog