mm: madvise: use walk_page_range_vma() for madvise_free_single_vma()
Subsystems (2)
RFC PATCH 1/1 mm: madvise: use walk_page_range_vma() for madvise_free_single_vma()
From: Barry Song <[email protected]>
We've already found the VMA before calling madvise_free_single_vma(), so calling walk_page_range() and doing find_vma() again seems unnecessary. It also prevents potential optimizations for MADV_FREE to use a per-VMA lock.
--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
mm/madvise.c +1 -1
@@ -826,7 +826,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);
Signature · 1 line
2.39.3 (Apple Git-146)
Quoted 25 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +0
-0
mm/madvise.c static int madvise_free_single_vma(struct madvise_behavior *madv_behavior, · line 832
Quoted 8 lines
mm/madvise.c +1
-1@@ -826,7 +826,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);
This indeed looks like an improvement dropping a redundant VMA look up and also opening up potential optimization later using per-VMA lock.
Quoted 25 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +0
-0
mm/madvise.c static int madvise_free_single_vma(struct madvise_behavior *madv_behavior, · line 832
Quoted 8 lines
mm/madvise.c +1
-1@@ -826,7 +826,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);
Can similar optimizations be made in madvise_willneed(), madvise_cold_page_range(), etc?
Quoted 29 lines · Dev Jain
Quoted 26 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +0
-0@@ -826,7 +826,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,
Quoted 9 lines
mm/madvise.c static int madvise_free_single_vma(struct madvise_behavior *madv_behavior, · line 832
Quoted 7 lines
mm/madvise.c +1
-1mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);Can similar optimizations be made in
madvise_willneed(),madvise_cold_page_range(), etc?
Yes, I think the same code flow applies to madvise_willneed, madvise_cold_page_range, and similar functions, though my current interest is more on madvise_free.
Let me prepare a v2 that includes those as well.
Quoted 1 line
Thanks
Barry
Quoted 40 lines · Barry Song
Quoted 29 lines · Dev Jain
Quoted 26 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +0
-0@@ -826,7 +826,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,Quoted 9 lines
mm/madvise.c static int madvise_free_single_vma(struct madvise_behavior *madv_behavior, · line 832
Quoted 7 lines
mm/madvise.c +1
-1mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);Can similar optimizations be made in
madvise_willneed(),madvise_cold_page_range(), etc?
Quoted 5 lines
Yes, I think the same code flow applies to
madvise_willneed,madvise_cold_page_range, and similar functions, though my current interest is more onmadvise_free.Let me prepare a v2 that includes those as well.
FWIW Dev makes a great point here and I agree wholeheartedly, let's fix all such cases...
As an aside, I wonder if we previously didn't do this because we hadn't previously exposed the walk_page_range_vma() API or something?
Quoted 5 lines
Quoted 2 lines
Thanks
Barry
Cheers!
Quoted 45 lines · Lorenzo Stoakes
Quoted 44 lines · Barry Song
Quoted 29 lines · Dev Jain
Quoted 26 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +0
-0@@ -826,7 +826,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,Quoted 9 lines
mm/madvise.c static int madvise_free_single_vma(struct madvise_behavior *madv_behavior, · line 832
Quoted 7 lines
mm/madvise.c +1
-1mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range);Can similar optimizations be made in
madvise_willneed(),madvise_cold_page_range(), etc?Yes, I think the same code flow applies to
madvise_willneed,madvise_cold_page_range, and similar functions, though my current interest is more onmadvise_free.
Quoted 7 lines
Quoted 1 line
Let me prepare a v2 that includes those as well.
FWIW Dev makes a great point here and I agree wholeheartedly, let's fix all such cases...
As an aside, I wonder if we previously didn't do this because we hadn't previously exposed the
walk_page_range_vma()API or something?
IIRC, yes:
commit e07cda5f232fac4de0925d8a4c92e51e41fa2f6e
Author: David Hildenbrand <[email protected]> Date: Fri Oct 21 12:11:39 2022 +0200
mm/pagewalk: add walk_page_range_vma()
Let's add walk_page_range_vma(), which is similar to walk_page_vma(), however, is only interested in a subset of the VMA range.
To be used in KSM code to stop using follow_page() next.
Signature · 3 lines
Cheers, David / dhildenb
Quoted 16 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>
Quoted 1 line
---
Signature · 3 lines
Cheers, David / dhildenb
Quoted 7 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.
Quoted 9 lines
Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>
Signature · 2 lines
Oscar Salvador SUSE Labs
Quoted 6 lines · Barry Song
From: Barry Song <[email protected]>
We've already found the VMA before calling
madvise_free_single_vma(), so callingwalk_page_range()and doingfind_vma()again seems unnecessary. It also prevents potential optimizations forMADV_FREEto use a per-VMA lock.
Really nice find, great work Barry!
Lord above, why on earth weren't we doing this before...
Quoted 10 lines
Cc "Liam R. Howlett" <[email protected]>Cc Lorenzo Stoakes <[email protected]>Cc David Hildenbrand <[email protected]>Cc Vlastimil Babka <[email protected]>Cc Jann Horn <[email protected]>Cc Suren Baghdasaryan <[email protected]>Cc Lokesh Gidra <[email protected]>Cc Tangquan Zheng <[email protected]>Signed-off-by Barry Song <[email protected]>
I can't see anything wrong with this, so:
Quoted 20 lines
--- mm/madvise.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)mm/madvise.c +1
-2@@ -826,7 +826,7 @@ static int madvise_free_single_vma(struct madvise_behavior *madv_behavior,mmu_notifier_invalidate_range_start (&range);tlb_start_vma (tlb, vma); -walk_page_range (vma->vm_mm, range.start, range.end, +walk_page_range_vma (vma, range.start, range.end, &madvise_free_walk_ops, tlb);tlb_end_vma (tlb, vma);mmu_notifier_invalidate_range_end (&range); --2.39.3 (Apple Git-146)