mm: madvise: use walk_page_range_vma() instead of walk_page_range()
Subsystems (2)
PATCH v2 1/1 mm: madvise: use walk_page_range_vma() instead of walk_page_range()
From: Barry Song <[email protected]>
We've already found the VMA within madvise_walk_vmas() before calling specific madvise behavior functions like madvise_free_single_vma(). So calling walk_page_range() and doing find_vma() again seems unnecessary. It also prevents potential optimizations in those madvise callbacks, particularly the use of dedicated per-VMA locking.
---
-v2:
* Also extend the modification to callbacks beyond madvise_free_single_vma() since the code flow is the same - Dev, Lorenzo
-rfc:
https://lore.kernel.org/linux-mm/[email protected]/
mm/madvise.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
mm/madvise.c +6 -6
@@ -282,7 +282,7 @@ static long madvise_willneed(struct vm_area_struct *vma, *prev = vma;#ifdef CONFIG_SWAP if (!file) { -walk_page_range (vma->vm_mm, start, end, &swapin_walk_ops, vma); +walk_page_range_vma (vma, start, end, &swapin_walk_ops, vma);lru_add_drain ();/* Push any new pages onto the LRU now */ return 0 ; }
@@ -581,7 +581,7 @@ static void madvise_cold_page_range(struct mmu_gather *tlb, };tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, addr, end, &cold_walk_ops, &walk_private); +walk_page_range_vma (vma, addr, end, &cold_walk_ops, &walk_private);tlb_end_vma (tlb, vma); }
@@ -619,7 +619,7 @@ static void madvise_pageout_page_range(struct mmu_gather *tlb, };tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, addr, end, &cold_walk_ops, &walk_private); +walk_page_range_vma (vma, addr, end, &cold_walk_ops, &walk_private);tlb_end_vma (tlb, vma); }
@@ -825,7 +825,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);
@@ -1160,7 +1160,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_mm (vma->vm_mm, start, end, + err =walk_page_range_vma (vma, start, end, &guard_install_walk_ops, &nr_pages);if (err <0 )return err;
@@ -1244,7 +1244,7 @@ static long madvise_guard_remove(struct vm_area_struct *vma,if (!is_valid_guard_vma (vma,/* allow_locked = */ true))return -EINVAL ; -return walk_page_range (vma->vm_mm, start, end, +return walk_page_range_vma (vma, start, end, &guard_remove_walk_ops,NULL ); }
Signature · 1 line
2.39.3 (Apple Git-146)
Quoted 8 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA within
madvise_walk_vmas()before calling specific madvise behavior functions likemadvise_free_single_vma(). So callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations in those madvise callbacks, particularly the use of dedicated per-VMA locking.
Quoted 12 lines
Reviewed-by Anshuman Khandual <[email protected]>Reviewed-by Lorenzo Stoakes <[email protected]>Reviewed-by Oscar Salvador <[email protected]>Cc "Liam R. Howlett" <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Dev Jain <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>
Nice!
Looks good to me,
Signature · 2 lines
Cheers, Harry / Hyeonggon
Quoted 73 lines
---
-v2:
* Also extend the modification to callbacks beyondmadvise_free_single_vma()since the code flow is the same - Dev, Lorenzo
-rfc:
https://lore.kernel.org/linux-mm/[email protected]/mm/madvise.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)mm/madvise.c +6
-6@@ -282,7 +282,7 @@ static long madvise_willneed(struct vm_area_struct *vma, *prev = vma;#ifdef CONFIG_SWAP if (!file) { -walk_page_range (vma->vm_mm, start, end, &swapin_walk_ops, vma); +walk_page_range_vma (vma, start, end, &swapin_walk_ops, vma);lru_add_drain ();/* Push any new pages onto the LRU now */ return 0 ; }@@ -581,7 +581,7 @@ static void madvise_cold_page_range(struct mmu_gather *tlb, };tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, addr, end, &cold_walk_ops, &walk_private); +walk_page_range_vma (vma, addr, end, &cold_walk_ops, &walk_private);tlb_end_vma (tlb, vma); }@@ -619,7 +619,7 @@ static void madvise_pageout_page_range(struct mmu_gather *tlb, };tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, addr, end, &cold_walk_ops, &walk_private); +walk_page_range_vma (vma, addr, end, &cold_walk_ops, &walk_private);tlb_end_vma (tlb, vma); }@@ -825,7 +825,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);@@ -1160,7 +1160,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_mm (vma->vm_mm, start, end, + err =walk_page_range_vma (vma, start, end, &guard_install_walk_ops, &nr_pages);if (err <0 )return err;@@ -1244,7 +1244,7 @@ static long madvise_guard_remove(struct vm_area_struct *vma,if (!is_valid_guard_vma (vma,/* allow_locked = */ true))return -EINVAL ; -return walk_page_range (vma->vm_mm, start, end, +return walk_page_range_vma (vma, start, end, &guard_remove_walk_ops,NULL ); }Signature · 1 line
2.39.3 (Apple Git-146)
Quoted 20 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA within
madvise_walk_vmas()before calling specific madvise behavior functions likemadvise_free_single_vma(). So callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations in those madvise callbacks, particularly the use of dedicated per-VMA locking.Reviewed-by Anshuman Khandual <[email protected]>Reviewed-by Lorenzo Stoakes <[email protected]>Reviewed-by Oscar Salvador <[email protected]>Cc "Liam R. Howlett" <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Dev Jain <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>
Quoted 1 line
---
Quoted 8 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA within
madvise_walk_vmas()before calling specific madvise behavior functions likemadvise_free_single_vma(). So callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations in those madvise callbacks, particularly the use of dedicated per-VMA locking.
Quoted 12 lines
Reviewed-by Anshuman Khandual <[email protected]>Reviewed-by Lorenzo Stoakes <[email protected]>Reviewed-by Oscar Salvador <[email protected]>Cc "Liam R. Howlett" <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Dev Jain <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>
Thanks!
mm/madvise.c static long madvise_guard_install(struct vm_area_struct *vma, · line 1164
Quoted 7 lines
mm/madvise.c +1
-1@@ -1160,7 +1160,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_mm (vma->vm_mm, start, end, + err =walk_page_range_vma (vma, start, end, &guard_install_walk_ops, &nr_pages);
Nit: breaks the parameter alignment. Do we care? It's Lorenzo's code so maybe not ;P
mm/madvise.c static long madvise_guard_remove(struct vm_area_struct *vma, · line 1248
Quoted 9 lines
mm/madvise.c +1
-1if (err <0 )return err;@@ -1244,7 +1244,7 @@ static long madvise_guard_remove(struct vm_area_struct *vma,if (!is_valid_guard_vma (vma,/* allow_locked = */ true))return -EINVAL ; -return walk_page_range (vma->vm_mm, start, end, +return walk_page_range_vma (vma, start, end, &guard_remove_walk_ops,NULL );
Same.
Quoted 2 lines
(quoted diff) +0
-0}
Quoted 29 lines · Vlastimil Babka
Quoted 8 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA within
madvise_walk_vmas()before calling specific madvise behavior functions likemadvise_free_single_vma(). So callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations in those madvise callbacks, particularly the use of dedicated per-VMA locking.Quoted 12 lines
Reviewed-by Anshuman Khandual <[email protected]>Reviewed-by Lorenzo Stoakes <[email protected]>Reviewed-by Oscar Salvador <[email protected]>Cc "Liam R. Howlett" <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Dev Jain <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>Reviewed-by Vlastimil Babka <[email protected]>Thanks!
Quoted 3 lines
mm/madvise.c +0
-0@@ -1160,7 +1160,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;
Quoted 7 lines
mm/madvise.c static long madvise_guard_install(struct vm_area_struct *vma, · line 1164
Quoted 4 lines
mm/madvise.c +1
-1/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_mm (vma->vm_mm, start, end, + err =walk_page_range_vma (vma, start, end, &guard_install_walk_ops, &nr_pages);Nit: breaks the parameter alignment. Do we care? It's Lorenzo's code so maybe not ;P
OMG!!
Nah it's fine leave it as-is :P
Can always go fix it up at some later date... or make it more sane with just tab alignment...
Quoted 16 lines
mm/madvise.c static long madvise_guard_remove(struct vm_area_struct *vma, · line 1248
Quoted 10 lines
mm/madvise.c +1
-1if (err <0 )return err;@@ -1244,7 +1244,7 @@ static long madvise_guard_remove(struct vm_area_struct *vma,if (!is_valid_guard_vma (vma,/* allow_locked = */ true))return -EINVAL ; -return walk_page_range (vma->vm_mm, start, end, +return walk_page_range_vma (vma, start, end, &guard_remove_walk_ops,NULL );Same.
Quoted 2 lines
(quoted diff) +0
-0}
Quoted 7 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA within
madvise_walk_vmas()before calling specific madvise behavior functions likemadvise_free_single_vma(). So callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations in those madvise callbacks, particularly the use of dedicated per-VMA locking.
FYI it looks like this patch breaks all the guard-region mm selftests with:
# guard-regions.c:719:split_merge:Expected madvise(ptr, 10 * page_size, MADV_GUARD_INSTALL) (-1) == 0 (0)
Am I the only one that runs these things? :)
[...]
Quoted 3 lines
mm/madvise.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)
[...]
mm/madvise.c static long madvise_guard_install(struct vm_area_struct *vma, · line 1164
Quoted 7 lines
mm/madvise.c +1
-1@@ -1160,7 +1160,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_mm (vma->vm_mm, start, end, + err =walk_page_range_vma (vma, start, end, &guard_install_walk_ops, &nr_pages);
IIRC walk_page_range_mm() is an internal API that allows the install_pte() callback, and the other (public) APIs explicitly disallow it, so presumably walk_page_range_vma() is now returning an error due to install_pte != NULL?
Thanks,
Ryan
Quoted 11 lines
mm/madvise.c +1
-1if (err <0 )return err;@@ -1244,7 +1244,7 @@ static long madvise_guard_remove(struct vm_area_struct *vma,if (!is_valid_guard_vma (vma,/* allow_locked = */ true))return -EINVAL ; -return walk_page_range (vma->vm_mm, start, end, +return walk_page_range_vma (vma, start, end, &guard_remove_walk_ops,NULL ); }
Andrew - could you drop this patch until this is fixed? Whoops!
Quoted 9 lines · Ryan Roberts
Quoted 7 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA within
madvise_walk_vmas()before calling specific madvise behavior functions likemadvise_free_single_vma(). So callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations in those madvise callbacks, particularly the use of dedicated per-VMA locking.
Quoted 6 lines
FYI it looks like this patch breaks all the guard-region mm selftests with:
# guard-regions.c:719:split_merge:Expected
madvise(ptr, 10 * page_size, MADV_GUARD_INSTALL)(-1) == 0 (0)Am I the only one that runs these things? :)
I normally do :) the one time I don't... :P
Quoted 12 lines
[...]
Quoted 3 lines
mm/madvise.c | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-)[...]
Quoted 3 lines
mm/madvise.c +0
-0@@ -1160,7 +1160,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;
Quoted 9 lines
mm/madvise.c static long madvise_guard_install(struct vm_area_struct *vma, · line 1164
Quoted 4 lines
mm/madvise.c +1
-1/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_mm (vma->vm_mm, start, end, + err =walk_page_range_vma (vma, start, end, &guard_install_walk_ops, &nr_pages);IIRC
walk_page_range_mm()is an internal API that allows theinstall_pte()callback, and the other (public) APIs explicitly disallow it, so presumablywalk_page_range_vma()is now returning an error due toinstall_pte!= NULL?
Yeah dear god I missed this oops!
Yeah Barry - could you revert this change for the guard region bits please? So this is intentional as we do not want anything non-mm to have access to install_pte.
I'll maybe have a think about this in terms of whether we want to do this a different way but for now this will fix the issue and get the patch in.
Otherwise patch is ok AFAICT...
With that fixed feel free to propagate tag to a v2.
Quoted 15 lines
Thanks,
RyanQuoted 11 lines
mm/madvise.c +1
-1if (err <0 )return err;@@ -1244,7 +1244,7 @@ static long madvise_guard_remove(struct vm_area_struct *vma,if (!is_valid_guard_vma (vma,/* allow_locked = */ true))return -EINVAL ; -return walk_page_range (vma->vm_mm, start, end, +return walk_page_range_vma (vma, start, end, &guard_remove_walk_ops,NULL ); }
Quoted 5 lines
Yeah dear god I missed this oops!
Yeah Barry - could you revert this change for the guard region bits please? So this is intentional as we do not want anything non-mm to have access to
install_pte.
All my fault! I wrote a multi-process/thread test to issue lots of madvise calls, but it looks like I missed INSTALL_GUARD.
Thanks, Ryan & Lorenzo! Does Andrew prefer to pick up the fix below, or would it be better to send a new version? He’s handled fixes like this in the past—happy to resend if needed.
From: Barry Song <[email protected]>
Date: Mon, 9 Jun 2025 22:42:13 +1200
Subject: [PATCH] mm: madvise: revert the walk_page_range_vma change for
MADV_GUARD_INSTALL
Fix the broken MADV_GUARD_INSTALL reported by Ryan.
# guard-regions.c:719:split_merge:Expected madvise(ptr, 10 * page_size, MADV_GUARD_INSTALL) (-1) == 0 (0)
--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/madvise.c +1 -1
@@ -1141,7 +1141,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_vma (vma, start, end, + err =walk_page_range_mm (vma->vm_mm, start, end, &guard_install_walk_ops, &nr_pages);if (err <0 )return err;
Signature · 1 line
2.39.3 (Apple Git-146)
Ryan Roberts
Reviewed-byQuoted 18 lines · Barry Song
Quoted 5 lines
Yeah dear god I missed this oops!
Yeah Barry - could you revert this change for the guard region bits please? So this is intentional as we do not want anything non-mm to have access to
install_pte.All my fault! I wrote a multi-process/thread test to issue lots of madvise calls, but it looks like I missed
INSTALL_GUARD.Thanks, Ryan & Lorenzo! Does Andrew prefer to pick up the fix below, or would it be better to send a new version? He’s handled fixes like this in the past—happy to resend if needed.
From: Barry Song <[email protected]>
Date: Mon, 9 Jun 2025 22:42:13 +1200
Subject: [PATCH] mm: madvise: revert thewalk_page_range_vmachange for
MADV_GUARD_INSTALL
Quoted 7 lines
Fix the broken
MADV_GUARD_INSTALLreported by Ryan.
# guard-regions.c:719:split_merge:Expectedmadvise(ptr, 10 * page_size, MADV_GUARD_INSTALL)(-1) == 0 (0)Reported-by Ryan Roberts <[email protected]>Signed-off-by Barry Song <[email protected]>
FWIW, the rest of the original patch looks good to me, ands this fix looks correct, so:
Quoted 17 lines
--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +1
-1@@ -1141,7 +1141,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_vma (vma, start, end, + err =walk_page_range_mm (vma->vm_mm, start, end, &guard_install_walk_ops, &nr_pages);if (err <0 )return err;
Quoted 6 lines · Barry Song
Quoted 5 lines
Yeah dear god I missed this oops!
Yeah Barry - could you revert this change for the guard region bits please? So this is intentional as we do not want anything non-mm to have access to
install_pte.
Quoted 6 lines
All my fault! I wrote a multi-process/thread test to issue lots of madvise calls, but it looks like I missed
INSTALL_GUARD.Thanks, Ryan & Lorenzo! Does Andrew prefer to pick up the fix below, or would it be better to send a new version? He’s handled fixes like this in the past—happy to resend if needed.
I'm totally fine with this thanks!
I can't speak for Andrew of course but he highlighted at LSF/MM his preference for incremental fixes like this so I would say this is probably his preference also :)
Quoted 22 lines
From: Barry Song <[email protected]>
Date: Mon, 9 Jun 2025 22:42:13 +1200
Subject: [PATCH] mm: madvise: revert thewalk_page_range_vmachange for
MADV_GUARD_INSTALLFix the broken
MADV_GUARD_INSTALLreported by Ryan.
# guard-regions.c:719:split_merge:Expectedmadvise(ptr, 10 * page_size, MADV_GUARD_INSTALL)(-1) == 0 (0)Reported-by Ryan Roberts <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +0
-0
mm/madvise.c static long madvise_guard_install(struct vm_area_struct *vma, · line 1145
Quoted 6 lines
mm/madvise.c +1
-1@@ -1141,7 +1141,7 @@ static long madvise_guard_install(struct vm_area_struct *vma,unsigned long nr_pages =0 ;/* Returns < 0 on error, == 0 if success, > 0 if zap needed. */ - err =walk_page_range_vma (vma, start, end, + err =walk_page_range_mm (vma->vm_mm, start, end, &guard_install_walk_ops, &nr_pages);
Actually ack that only the install needs it, we can keep this fix for the remove, as that doesn't of course use .install_pte().
Also it's kind of neat to only use the internal one in the instance we need that and the 'public' interface when we don't.
Quoted 5 lines
mm/madvise.c +0
-1if (err <0 )return err; --2.39.3 (Apple Git-146)
I also did actually test this this time :P and can confirm it fixes the issue.