mm: make VMA page offset handling more consistent
Subsystems (47)
Lorenzo Stoakes
maintainercover mm: make VMA page offset handling more consistent
This series performs a series of cleanups and improvements around how the vma->vm_pgoff field is used.
Folios belonging to file-backed mappings are simply indexed by the page offset within the file they map.
However, anonymous folios belonging to pure anonymous mappings are indexed by their "virtual" page offset, which is equal to addr >> PAGE_SHIFT at the time at which the VMA was first faulted in.
The page offset of a VMA is stored in vma->vm_pgoff and indicates the page offset of the start of the VMA range, whether it be file-backed or anonymous.
The work here both cleans up how we reference this field, as well as laying the foundations for a future series which addresses the inconsistency of CoW'd folios in MAP_PRIVATE-file backed mappings, which are indexed as if they were file-backed but behave as if they were anonymous.
This future series will make it such that all anonymous folios are indexed by virtual page offset whether belonging to VMAs who satisfy vma_is_anonymous() or MAP_PRIVATE-mapped file-backed mappings.
This series:
* Exposes vma_start_pgoff() and updates the kernel to use it consistently.
* Adds and uses the useful vma_end_pgoff() helper.
* Parameterises the file-backed mapping helpers vma_interval_tree_*() by adress_space rather than rb_root_cached.
* Renames the misleadingly-named vma_interval_tree_*() helpers to mapping_interval_tree_*() to be consistent with anon_vma_interval_tree_*().
* Parameterises anon_vma_interval_tree_*() by anon_vma.
* Moves mm/interval_tree.c to the rmap section.
* Adds vmg_*() helpers for page offset.
* Clarifies the confusing vmg_adjust_set_range() function.
* Introduces linear_page_delta() to provide relative pgoff within a VMA.
* Replaces open-coded versions of linear_page_delta() and linear_page_index() with invocations of these functions.
* Introduces and uses vma_assert_can_modify() to account for whether a VMA can be modified (detached or write locked).
* Adds and uses vma_[add,sub]_pgoff() to adjust VMA page offset.
* Moves __install_special_mapping() to vma.c.
* Makes vma_set_range() static and internal to vma.c.
* Introduces and makes use of vma_set_pgoff().
* Fixes incorrect vma.h header inclusion.
* Defaults VMA userland tests to 64-bit vma flags size.
* Updates VMA userland tests to give better output on failure.
* Various smaller cleanups.
Lorenzo Stoakes (30):
mm: move vma_start_pgoff() into mm.h and clean up
mm: add kdoc comments for vma_start/last_pgoff() tools/testing/vma: use vma_start_pgoff() in merge tests
mm: introduce and use vma_end_pgoff()
mm/rmap: update mm/interval_tree.c comments
mm/rmap: parameterise vma_interval_tree_*() by address_space mm/rmap: elide unnecessary static inline's in interval_tree.c mm/rmap: rename vma_interval_tree_*() to mapping_interval_tree_*() mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
MAINTAINERS: Move mm/interval_tree.c to rmap section mm/vma: introduce and use vmg_pages(), vmg_[start, end]_pgoff() mm/vma: clean up anon_vma_compatible()
mm/vma: refactor vmg_adjust_set_range() for clarity mm/vma: minor cleanup of expand_[upwards, downwards]()
mm: introduce and use linear_page_delta()
mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/ mm/vma: remove duplicative vma_pgoff_offset() helper
mm: use linear_page_[index, delta]() consistently mm/vma: introduce vma_assert_can_modify()
mm/vma: add and use vma_[add/sub]_pgoff()
mm/vma: move __install_special_mapping() to vma.c mm/vma: make vma_set_range() static, drop insert_vm_struct() decl mm/vma: update vma_shrink() to not pass unnecessary pgoff parameter mm/vma: update vmg_adjust_set_range() to offset pgoff instead mm/vma: introduce and use vma_set_pgoff()
mm/vma: correct incorrect vma.h inclusion
mm/vma: use guard clauses in can_vma_merge_[before, after]() tools/testing/vma: default VMA flag bits to 64-bit tools/testing/vma: output compared expression on ASSERT_[EQ, NE]()
MAINTAINERS | 2 +- arch/arm/mm/fault-armv.c | 4 +- arch/arm/mm/flush.c | 2 +- arch/nios2/mm/cacheflush.c | 2 +- arch/parisc/kernel/cache.c | 2 +- arch/x86/kernel/cpu/sgx/virt.c | 3 +- drivers/comedi/comedi_fops.c | 3 +- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 +- drivers/gpu/drm/gma500/gem.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 3 +- drivers/gpu/drm/omapdrm/omap_gem.c | 5 +- drivers/gpu/drm/tegra/gem.c | 3 +- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 +- drivers/vfio/pci/nvgrace-gpu/main.c | 3 +- drivers/vfio/pci/vfio_pci_core.c | 3 +- fs/dax.c | 2 +- fs/hugetlbfs/inode.c | 15 +- include/linux/huge_mm.h | 1 + include/linux/hugetlb.h | 3 +- include/linux/mm.h | 118 +++++++++---- include/linux/mmap_lock.h | 8 + include/linux/pagemap.h | 39 ++++- kernel/dma/coherent.c | 7 +- kernel/dma/direct.c | 6 +- kernel/dma/mapping.c | 8 +- kernel/dma/ops_helpers.c | 4 +- kernel/events/core.c | 20 ++- kernel/events/uprobes.c | 13 +- kernel/kcov.c | 2 +- kernel/trace/ring_buffer.c | 3 +- mm/damon/vaddr.c | 5 +- mm/debug.c | 2 +- mm/filemap.c | 7 +- mm/huge_memory.c | 2 +- mm/hugetlb.c | 15 +- mm/internal.h | 33 ++-- mm/interval_tree.c | 113 +++++++----- mm/khugepaged.c | 7 +- mm/ksm.c | 7 +- mm/madvise.c | 6 +- mm/mapping_dirty_helpers.c | 2 +- mm/memory-failure.c | 10 +- mm/memory.c | 33 ++-- mm/mempolicy.c | 13 +- mm/mmap.c | 41 +---- mm/mmu_notifier.c | 2 +- mm/mremap.c | 12 +- mm/msync.c | 4 +- mm/nommu.c | 22 +-- mm/pagewalk.c | 6 +- mm/rmap.c | 14 +- mm/shmem.c | 9 +- mm/userfaultfd.c | 4 +- mm/util.c | 4 +- mm/vma.c | 239 ++++++++++++++++++-------- mm/vma.h | 59 ++++++- mm/vma_exec.c | 12 +- mm/vma_init.c | 6 +- mm/vma_internal.h | 4 +- tools/testing/vma/Makefile | 2 +- tools/testing/vma/include/dup.h | 41 ++++- tools/testing/vma/include/stubs.h | 12 +- tools/testing/vma/shared.c | 9 - tools/testing/vma/shared.h | 36 ++-- tools/testing/vma/tests/merge.c | 40 ++--- virt/kvm/guest_memfd.c | 2 +- 66 files changed, 699 insertions(+), 432 deletions(-)
--
2.54.0
Lorenzo Stoakes
maintainerPATCH 1/30 mm: move vma_start_pgoff() into mm.h and clean up
vma_last_pgoff() already lives there, so it's a bit odd to keep vma_start_pgoff() in mm/interval_tree.c. Move them together.
These each return unsigned long, which pgoff_t is typedef'd to. Make this consistent and have these functions return pgoff_t instead.
Additionally, express vma_last_pgoff() in terms of vma_start_pgoff(), since we wrap the vma->vm_pgoff access, we may as well use it here.
Also while we're here, const-ify the VMA and cleanup a bit.
No functional change intended.
--- include/linux/mm.h | 9 +++++++-- mm/interval_tree.c | 5 ----- 2 files changed, 7 insertions(+), 7 deletions(-)
include/linux/mm.h +7 -2
@@ -4278,9 +4278,14 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; } -static inline unsigned long vma_last_pgoff (struct vm_area_struct *vma) +static inline pgoff_t vma_start_pgoff (const struct vm_area_struct *vma) { -return vma->vm_pgoff +vma_pages (vma) -1 ; +return vma->vm_pgoff; +} + +static inline pgoff_t vma_last_pgoff (const struct vm_area_struct *vma) +{ +return vma_start_pgoff (vma) +vma_pages (vma) -1 ; }static inline unsigned long vma_desc_size (const struct vm_area_desc *desc)
mm/interval_tree.c +0 -5
@@ -10,11 +10,6 @@#include <linux/rmap.h> #include <linux/interval_tree_generic.h> -static inline unsigned long vma_start_pgoff (struct vm_area_struct *v) -{ -return v->vm_pgoff; -} - INTERVAL_TREE_DEFINE(struct vm_area_struct, shared.rb, unsigned long, shared.rb_subtree_last, vma_start_pgoff, vma_last_pgoff, /* empty */, vma_interval_tree)
Signature · 1 line
2.54.0
Re: [PATCH 01/30] mm: move vma_start_pgoff() into mm.h and clean up
Quoted 6 lines · Lorenzo Stoakes
vma_last_pgoff()already lives there, so it's a bit odd to keepvma_start_pgoff()inmm/interval_tree.c. Move them together.These each return unsigned long, which
pgoff_tis typedef'd to. Make this consistent and have these functions returnpgoff_tinstead.
Quoted 8 lines
Additionally, express
vma_last_pgoff()in terms ofvma_start_pgoff(), since we wrap thevma->vm_pgoffaccess, we may as well use it here.Also while we're here, const-ify the VMA and cleanup a bit.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Re: [PATCH 01/30] mm: move vma_start_pgoff() into mm.h and clean up
In reply to Lorenzo StoakesQuoted 2 lines · Lorenzo Stoakes
vma_last_pgoff()already lives there, so it's a bit odd to keepvma_start_pgoff()inmm/interval_tree.c. Move them together.
Hmm, a part of me wonders if this is the part where we should start cleaning up mm.h into vma.h or something. Probably not, it would be extra churn right now.
Quoted 4 lines
These each return unsigned long, which
pgoff_tis typedef'd to. Make this consistent and have these functions returnpgoff_tinstead.
Quoted 8 lines
Additionally, express
vma_last_pgoff()in terms ofvma_start_pgoff(), since we wrap thevma->vm_pgoffaccess, we may as well use it here.Also while we're here, const-ify the VMA and cleanup a bit.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 45 lines
--- include/linux/mm.h | 9 +++++++-- mm/interval_tree.c | 5 ----- 2 files changed, 7 insertions(+), 7 deletions(-)include/linux/mm.h +7
-2@@ -4278,9 +4278,14 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; } -static inline unsigned long vma_last_pgoff (struct vm_area_struct *vma) +static inline pgoff_t vma_start_pgoff (const struct vm_area_struct *vma) { -return vma->vm_pgoff +vma_pages (vma) -1 ; +return vma->vm_pgoff; +} + +static inline pgoff_t vma_last_pgoff (const struct vm_area_struct *vma) +{ +return vma_start_pgoff (vma) +vma_pages (vma) -1 ; }static inline unsigned long vma_desc_size (const struct vm_area_desc *desc)mm/interval_tree.c +0
-5@@ -10,11 +10,6 @@#include <linux/rmap.h> #include <linux/interval_tree_generic.h> -static inline unsigned long vma_start_pgoff (struct vm_area_struct *v) -{ -return v->vm_pgoff; -} - INTERVAL_TREE_DEFINE(struct vm_area_struct, shared.rb, unsigned long, shared.rb_subtree_last, vma_start_pgoff, vma_last_pgoff, /* empty */, vma_interval_tree)Signature · 1 line
2.54.0
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 01/30] mm: move vma_start_pgoff() into mm.h and clean up
Quoted 7 lines · Pedro Falcato
Quoted 2 lines · Lorenzo Stoakes
vma_last_pgoff()already lives there, so it's a bit odd to keepvma_start_pgoff()inmm/interval_tree.c. Move them together.Hmm, a part of me wonders if this is the part where we should start cleaning up
mm.hintovma.hor something. Probably not, it would be extra churn right now.
Yeah the issue is there's some confusion about vma.h - mm.h should be for stuff that is used outside of mm, and these helpers are definitely like that.
vma.h is purely for internal mm vma stuff, and most people should be accessing that via internal.h (I address that in patch 27).
I do wonder if that could be done more nicely but punt that to another time.
But also probably worth doing a pass over some of the defines, I have a bunch of series chur^W changing stuff lately so can do a follow up on that maybe.
Quoted 8 lines
Quoted 8 lines
These each return unsigned long, which
pgoff_tis typedef'd to. Make this consistent and have these functions returnpgoff_tinstead.Additionally, express
vma_last_pgoff()in terms ofvma_start_pgoff(), since we wrap thevma->vm_pgoffaccess, we may as well use it here.
Quoted 7 lines
Quoted 5 lines
Also while we're here, const-ify the VMA and cleanup a bit.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>Reviewed-by Pedro Falcato <[email protected]>
Thanks!
Quoted 35 lines
Quoted 35 lines
--- include/linux/mm.h | 9 +++++++-- mm/interval_tree.c | 5 ----- 2 files changed, 7 insertions(+), 7 deletions(-)include/linux/mm.h +7
-2@@ -4278,9 +4278,14 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; } -static inline unsigned long vma_last_pgoff (struct vm_area_struct *vma) +static inline pgoff_t vma_start_pgoff (const struct vm_area_struct *vma) { -return vma->vm_pgoff +vma_pages (vma) -1 ; +return vma->vm_pgoff; +} + +static inline pgoff_t vma_last_pgoff (const struct vm_area_struct *vma) +{ +return vma_start_pgoff (vma) +vma_pages (vma) -1 ; }static inline unsigned long vma_desc_size (const struct vm_area_desc *desc)mm/interval_tree.c +0
-0@@ -10,11 +10,6 @@#include <linux/rmap.h> #include <linux/interval_tree_generic.h>
Quoted 14 lines
Quoted 11 lines
mm/interval_tree.c +0
-6-static inline unsigned long vma_start_pgoff (struct vm_area_struct *v) -{ -return v->vm_pgoff; -} - INTERVAL_TREE_DEFINE(struct vm_area_struct, shared.rb, unsigned long, shared.rb_subtree_last, vma_start_pgoff, vma_last_pgoff, /* empty */, vma_interval_tree) --2.54.0
--
Pedro
Cheers, Lorenzo
Re: [PATCH 01/30] mm: move vma_start_pgoff() into mm.h and clean up
In reply to Lorenzo StoakesQuoted 6 lines · Lorenzo Stoakes
vma_last_pgoff()already lives there, so it's a bit odd to keepvma_start_pgoff()inmm/interval_tree.c. Move them together.These each return unsigned long, which
pgoff_tis typedef'd to. Make this consistent and have these functions returnpgoff_tinstead.
Quoted 8 lines
Additionally, express
vma_last_pgoff()in terms ofvma_start_pgoff(), since we wrap thevma->vm_pgoffaccess, we may as well use it here.Also while we're here, const-ify the VMA and cleanup a bit.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Re: [PATCH 01/30] mm: move vma_start_pgoff() into mm.h and clean up
In reply to Lorenzo StoakesQuoted 9 lines · Lorenzo Stoakes
vma_last_pgoff()already lives there, so it's a bit odd to keepvma_start_pgoff()inmm/interval_tree.c. Move them together.These each return unsigned long, which
pgoff_tis typedef'd to. Make this consistent and have these functions returnpgoff_tinstead.Additionally, express
vma_last_pgoff()in terms ofvma_start_pgoff(), since we wrap thevma->vm_pgoffaccess, we may as well use it here.
Quoted 6 lines
Also while we're here, const-ify the VMA and cleanup a bit.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>---
Signature · 3 lines
Cheers, David
Lorenzo Stoakes
maintainerPATCH 2/30 mm: add kdoc comments for vma_start/last_pgoff()
In reply to Lorenzo StoakesDescribe what vma_start_pgoff() and vma_last_pgoff() actually provide in detail.
This is in order that we can differentiate this between functions that will be added in a subsequent patch which provide a different page offset.
No functional change intended.
--- include/linux/mm.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
include/linux/mm.h +26 -0
@@ -4278,11 +4278,37 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; } +/** +* vma_start_pgoff() - Get the page offset of the start of @vma +* @vma: The VMA whose page offset is required. +* +* If the VMA is file-backed, this is the page offset into the file. +* +* If the VMA is anonymous, this is the virtual page offset of the start of the +* VMA - if unfaulted, then vma->vm_start >> PAGE_SHIFT, if faulted then the +* virtual page offset at the time of first fault. +* +* Note that if @vma is a MAP_PRIVATE file-backed mapping, then this returns the +* file offset. +* +* Returns: The page offset of the start of @vma. +*/ static inline pgoff_t vma_start_pgoff (const struct vm_area_struct *vma) {return vma->vm_pgoff; } +/** +* vma_last_pgoff() - Get the page offset of the last page in @vma +* @vma: The VMA whose last page offset is required. +* +* This returns the last page offset contained within @vma. +* +* See the description of vma_start_pgoff() for a description of VMA page +* offsets. +* +* Returns: The last page offset of @vma. +*/ static inline pgoff_t vma_last_pgoff (const struct vm_area_struct *vma) {return vma_start_pgoff (vma) +vma_pages (vma) -1 ;
Signature · 1 line
2.54.0
Re: [PATCH 02/30] mm: add kdoc comments for vma_start/last_pgoff()
Quoted 9 lines · Lorenzo Stoakes
Describe what
vma_start_pgoff()andvma_last_pgoff()actually provide in detail.This is in order that we can differentiate this between functions that will be added in a subsequent patch which provide a different page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Re: [PATCH 02/30] mm: add kdoc comments for vma_start/last_pgoff()
In reply to Lorenzo StoakesQuoted 9 lines · Lorenzo Stoakes
Describe what
vma_start_pgoff()andvma_last_pgoff()actually provide in detail.This is in order that we can differentiate this between functions that will be added in a subsequent patch which provide a different page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 02/30] mm: add kdoc comments for vma_start/last_pgoff()
In reply to Lorenzo StoakesQuoted 9 lines · Lorenzo Stoakes
Describe what
vma_start_pgoff()andvma_last_pgoff()actually provide in detail.This is in order that we can differentiate this between functions that will be added in a subsequent patch which provide a different page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 46 lines
--- include/linux/mm.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)include/linux/mm.h +26
-0@@ -4278,11 +4278,37 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; } +/** +* vma_start_pgoff() - Get the page offset of the start of @vma +* @vma: The VMA whose page offset is required. +* +* If the VMA is file-backed, this is the page offset into the file. +* +* If the VMA is anonymous, this is the virtual page offset of the start of the +* VMA - if unfaulted, then vma->vm_start >> PAGE_SHIFT, if faulted then the +* virtual page offset at the time of first fault. +* +* Note that if @vma is a MAP_PRIVATE file-backed mapping, then this returns the +* file offset. +* +* Returns: The page offset of the start of @vma. +*/ static inline pgoff_t vma_start_pgoff (const struct vm_area_struct *vma) {return vma->vm_pgoff; } +/** +* vma_last_pgoff() - Get the page offset of the last page in @vma +* @vma: The VMA whose last page offset is required. +* +* This returns the last page offset contained within @vma. +* +* See the description of vma_start_pgoff() for a description of VMA page +* offsets. +* +* Returns: The last page offset of @vma. +*/ static inline pgoff_t vma_last_pgoff (const struct vm_area_struct *vma) {return vma_start_pgoff (vma) +vma_pages (vma) -1 ;
David Hildenbrand (Arm)
maintainerRe: [PATCH 02/30] mm: add kdoc comments for vma_start/last_pgoff()
In reply to Lorenzo StoakesQuoted 23 lines · Lorenzo Stoakes
Describe what
vma_start_pgoff()andvma_last_pgoff()actually provide in detail.This is in order that we can differentiate this between functions that will be added in a subsequent patch which provide a different page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- include/linux/mm.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)include/linux/mm.h +2
-0@@ -4278,11 +4278,37 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; } +/** +* vma_start_pgoff() - Get the page offset of the start of @vma
include/linux/mm.h static inline unsigned long vma_pages(const struct vm_area_struct *vma) · line 4292
Quoted 10 lines
include/linux/mm.h +10
-0+* @vma: The VMA whose page offset is required. +* +* If the VMA is file-backed, this is the page offset into the file. +* +* If the VMA is anonymous, this is the virtual page offset of the start of the +* VMA - if unfaulted, then vma->vm_start >> PAGE_SHIFT, if faulted then the +* virtual page offset at the time of first fault. +* +* Note that if @vma is a MAP_PRIVATE file-backed mapping, then this returns the +* file offset.
There is the ugly case of @vma being a MAP_PRIVATE PFNMAP mapping, where it returns something different.
(remap_pfn_range_prepare_vma() -> vma->vm_pgoff set to PFN)
Signature · 3 lines
Cheers, David
Lorenzo Stoakes
maintainerRe: [PATCH 02/30] mm: add kdoc comments for vma_start/last_pgoff()
Quoted 22 lines · David Hildenbrand (Arm)
Quoted 21 lines · Lorenzo Stoakes
Describe what
vma_start_pgoff()andvma_last_pgoff()actually provide in detail.This is in order that we can differentiate this between functions that will be added in a subsequent patch which provide a different page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- include/linux/mm.h | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)include/linux/mm.h +0
-0@@ -4278,11 +4278,37 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; }
Quoted 15 lines
include/linux/mm.h static inline unsigned long vma_pages(const struct vm_area_struct *vma) · line 4292
Quoted 12 lines
include/linux/mm.h +12
-0+/** +* vma_start_pgoff() - Get the page offset of the start of @vma +* @vma: The VMA whose page offset is required. +* +* If the VMA is file-backed, this is the page offset into the file. +* +* If the VMA is anonymous, this is the virtual page offset of the start of the +* VMA - if unfaulted, then vma->vm_start >> PAGE_SHIFT, if faulted then the +* virtual page offset at the time of first fault. +* +* Note that if @vma is a MAP_PRIVATE file-backed mapping, then this returns the +* file offset. There is the ugly case of @vma being a
MAP_PRIVATEPFNMAP mapping, where it returns something different.
Yup, the pfn I will update doc to mention this abomination too.
The virt pgoff stuff in the RFC fixes this also.
Quoted 7 lines
(
remap_pfn_range_prepare_vma()->vma->vm_pgoffset to PFN)--
Cheers,David
Cheers, Lorenzo
Lorenzo Stoakes
maintainerPATCH 3/30 tools/testing/vma: use vma_start_pgoff() in merge tests
In reply to Lorenzo StoakesNow we have the vma_start_pgoff() helper, update the merge tests to make use of it for consistency.
No functional change intended.
--- tools/testing/vma/tests/merge.c | 38 ++++++++++++++++----------------- 1 file changed, 19 insertions(+), 19 deletions(-)
tools/testing/vma/tests/merge.c +19 -19
@@ -118,7 +118,7 @@ static bool test_simple_merge(void)ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x3000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_FLAGS_SAME_MASK (&vma->flags, vma_flags);detach_free_vma (vma);
@@ -150,7 +150,7 @@ static bool test_simple_modify(void)ASSERT_EQ (vma->vm_start,0x1000 );ASSERT_EQ (vma->vm_end,0x2000 ); -ASSERT_EQ (vma->vm_pgoff,1 ); +ASSERT_EQ (vma_start_pgoff (vma),1 );/* * Now walk through the three split VMAs and make sure they are as
@@ -162,7 +162,7 @@ static bool test_simple_modify(void)ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x1000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );detach_free_vma (vma);vma_iter_clear (&vmi);
@@ -171,7 +171,7 @@ static bool test_simple_modify(void)ASSERT_EQ (vma->vm_start,0x1000 );ASSERT_EQ (vma->vm_end,0x2000 ); -ASSERT_EQ (vma->vm_pgoff,1 ); +ASSERT_EQ (vma_start_pgoff (vma),1 );detach_free_vma (vma);vma_iter_clear (&vmi);
@@ -180,7 +180,7 @@ static bool test_simple_modify(void)ASSERT_EQ (vma->vm_start,0x2000 );ASSERT_EQ (vma->vm_end,0x3000 ); -ASSERT_EQ (vma->vm_pgoff,2 ); +ASSERT_EQ (vma_start_pgoff (vma),2 );detach_free_vma (vma);mtree_destroy (&mm.mm_mt);
@@ -209,7 +209,7 @@ static bool test_simple_expand(void)ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x3000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );detach_free_vma (vma);mtree_destroy (&mm.mm_mt);
@@ -231,7 +231,7 @@ static bool test_simple_shrink(void)ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x1000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );detach_free_vma (vma);mtree_destroy (&mm.mm_mt);
@@ -324,7 +324,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,ASSERT_TRUE (merged);ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x4000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_EQ (vma->anon_vma, &dummy_anon_vma);ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,3 );
@@ -343,7 +343,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,ASSERT_TRUE (merged);ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x5000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_EQ (vma->anon_vma, &dummy_anon_vma);ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,3 );
@@ -364,7 +364,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,ASSERT_TRUE (merged);ASSERT_EQ (vma->vm_start,0x6000 );ASSERT_EQ (vma->vm_end,0x9000 ); -ASSERT_EQ (vma->vm_pgoff,6 ); +ASSERT_EQ (vma_start_pgoff (vma),6 );ASSERT_EQ (vma->anon_vma, &dummy_anon_vma);ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,3 );
@@ -384,7 +384,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,ASSERT_TRUE (merged);ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x9000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_EQ (vma->anon_vma, &dummy_anon_vma);ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,2 );
@@ -404,7 +404,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,ASSERT_TRUE (merged);ASSERT_EQ (vma->vm_start,0xa000 );ASSERT_EQ (vma->vm_end,0xc000 ); -ASSERT_EQ (vma->vm_pgoff,0xa ); +ASSERT_EQ (vma_start_pgoff (vma),0xa );ASSERT_EQ (vma->anon_vma, &dummy_anon_vma);ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,2 );
@@ -423,7 +423,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,ASSERT_TRUE (merged);ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0xc000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_EQ (vma->anon_vma, &dummy_anon_vma);ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,1 );
@@ -443,7 +443,7 @@ static bool __test_merge_new(bool is_sticky, bool a_is_sticky, bool b_is_sticky,ASSERT_NE (vma,NULL );ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0xc000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_EQ (vma->anon_vma, &dummy_anon_vma);detach_free_vma (vma);
@@ -805,7 +805,7 @@ static bool test_vma_merge_new_with_close(void)ASSERT_EQ (vmg.state,VMA_MERGE_SUCCESS );ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x5000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_EQ (vma->vm_ops, &vm_ops);ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,2 );
@@ -865,7 +865,7 @@ static bool __test_merge_existing(bool prev_is_sticky, bool middle_is_sticky, boASSERT_EQ (vma_next->anon_vma, &dummy_anon_vma);ASSERT_EQ (vma->vm_start,0x2000 );ASSERT_EQ (vma->vm_end,0x3000 ); -ASSERT_EQ (vma->vm_pgoff,2 ); +ASSERT_EQ (vma_start_pgoff (vma),2 );ASSERT_TRUE (vma_write_started (vma));ASSERT_TRUE (vma_write_started (vma_next));ASSERT_EQ (mm.map_count,2 );
@@ -931,7 +931,7 @@ static bool __test_merge_existing(bool prev_is_sticky, bool middle_is_sticky, boASSERT_EQ (vma_prev->anon_vma, &dummy_anon_vma);ASSERT_EQ (vma->vm_start,0x6000 );ASSERT_EQ (vma->vm_end,0x7000 ); -ASSERT_EQ (vma->vm_pgoff,6 ); +ASSERT_EQ (vma_start_pgoff (vma),6 );ASSERT_TRUE (vma_write_started (vma_prev));ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,2 );
@@ -1416,7 +1416,7 @@ static bool test_merge_extend(void)ASSERT_EQ (vma_merge_extend (&vmi, vma,0x2000 ), vma);ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x4000 ); -ASSERT_EQ (vma->vm_pgoff,0 ); +ASSERT_EQ (vma_start_pgoff (vma),0 );ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (mm.map_count,1 );
@@ -1456,7 +1456,7 @@ static bool test_expand_only_mode(void)ASSERT_EQ (vmg.state,VMA_MERGE_SUCCESS );ASSERT_EQ (vma->vm_start,0x3000 );ASSERT_EQ (vma->vm_end,0x9000 ); -ASSERT_EQ (vma->vm_pgoff,3 ); +ASSERT_EQ (vma_start_pgoff (vma),3 );ASSERT_TRUE (vma_write_started (vma));ASSERT_EQ (vma_iter_addr (&vmi),0x3000 );vma_assert_attached (vma);
Signature · 1 line
2.54.0
Re: [PATCH 03/30] tools/testing/vma: use vma_start_pgoff() in merge tests
Quoted 6 lines · Lorenzo Stoakes
Now we have the
vma_start_pgoff()helper, update the merge tests to make use of it for consistency.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Question: Should we have primitive tests for vma_*_pgoff() since the behavior changes depending on file/anon?
Nice to have the cleanup and clarity. Maybe worth asserting no one ever breaks this.
for this patch though
Lorenzo Stoakes
maintainerRe: [PATCH 03/30] tools/testing/vma: use vma_start_pgoff() in merge tests
Quoted 6 lines · Gregory Price
Quoted 5 lines · Lorenzo Stoakes
Now we have the
vma_start_pgoff()helper, update the merge tests to make use of it for consistency.No functional change intended.
Quoted 7 lines
Quoted 1 line
Signed-off-by Lorenzo Stoakes <[email protected]>Question: Should we have primitive tests for vma_*_pgoff() since the behavior changes depending on file/anon?
Nice to have the cleanup and clarity. Maybe worth asserting no one ever breaks this.
Well funny you should mention that :) I do add some asserts as I go.
In my RFC series which this series is the predicate for, I add more as then we track virtal page off separately (see [0]).
Amusingly (or not) /dev/zero breaks assumptions a bit (anonymous VMA with vma->vm_file that tracks by file index, just glroious). But I plan to fix that later!
Quoted 4 lines
for this patch though
Reviewed-by Gregory Price <[email protected]>
Thanks!
Cheers, Lorenzo
Re: [PATCH 03/30] tools/testing/vma: use vma_start_pgoff() in merge tests
In reply to Lorenzo StoakesQuoted 6 lines · Lorenzo Stoakes
Now we have the
vma_start_pgoff()helper, update the merge tests to make use of it for consistency.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 03/30] tools/testing/vma: use vma_start_pgoff() in merge tests
In reply to Lorenzo StoakesQuoted 6 lines · Lorenzo Stoakes
Now we have the
vma_start_pgoff()helper, update the merge tests to make use of it for consistency.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 4/30 mm: introduce and use vma_end_pgoff()
In reply to Lorenzo StoakesWe already have vma_last_pgoff() which retrieves the last page offset within a VMA.
However, code often wishes to span a page offset range, which requires the exclusive end of this range.
So provide this in vma_end_pgoff() and update vma_last_pgoff() to use this function.
No functional change intended.
--- include/linux/mm.h | 19 ++++++++++++++++++- 1 file changed, 18 insertions(+), 1 deletion(-)
include/linux/mm.h +18 -1
@@ -4298,6 +4298,23 @@ static inline pgoff_t vma_start_pgoff(const struct vm_area_struct *vma)return vma->vm_pgoff; } +/** +* vma_end_pgoff() - Get the page offset of the exclusive end of @vma +* @vma: The VMA whose end page offset is required. +* +* This returns the exclusive end page offset of @vma, which is useful for +* expressing page offset ranges. +* +* See the description of vma_start_pgoff() for a description of VMA page +* offsets. +* +* Returns: The exclusive end page offset of @vma. +*/ +static inline pgoff_t vma_end_pgoff (const struct vm_area_struct *vma) +{ +return vma_start_pgoff (vma) +vma_pages (vma); +} +/** * vma_last_pgoff() - Get the page offset of the last page in @vma * @vma: The VMA whose last page offset is required.
@@ -4311,7 +4328,7 @@ static inline pgoff_t vma_start_pgoff(const struct vm_area_struct *vma)*/ static inline pgoff_t vma_last_pgoff (const struct vm_area_struct *vma) { -return vma_start_pgoff (vma) +vma_pages (vma) -1 ; +return vma_end_pgoff (vma) -1 ; }static inline unsigned long vma_desc_size (const struct vm_area_desc *desc)
Signature · 1 line
2.54.0
Re: [PATCH 04/30] mm: introduce and use vma_end_pgoff()
Quoted 6 lines · Lorenzo Stoakes
We already have
vma_last_pgoff()which retrieves the last page offset within a VMA.However, code often wishes to span a page offset range, which requires the exclusive end of this range.
Quoted 6 lines
So provide this in
vma_end_pgoff()and updatevma_last_pgoff()to use this function.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Re: [PATCH 04/30] mm: introduce and use vma_end_pgoff()
In reply to Lorenzo StoakesQuoted 6 lines · Lorenzo Stoakes
We already have
vma_last_pgoff()which retrieves the last page offset within a VMA.However, code often wishes to span a page offset range, which requires the exclusive end of this range.
Quoted 6 lines
So provide this in
vma_end_pgoff()and updatevma_last_pgoff()to use this function.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 04/30] mm: introduce and use vma_end_pgoff()
In reply to Lorenzo StoakesQuoted 6 lines · Lorenzo Stoakes
We already have
vma_last_pgoff()which retrieves the last page offset within a VMA.However, code often wishes to span a page offset range, which requires the exclusive end of this range.
Quoted 6 lines
So provide this in
vma_end_pgoff()and updatevma_last_pgoff()to use this function.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 5/30 mm/rmap: update mm/interval_tree.c comments
In reply to Lorenzo StoakesUpdate the file comment to clarify that both file-backed and anonymous interval trees are provided, referencing the relevant data types for clarity.
Also add comments to indicate which parts of the file apply to each.
While we're here, convert the VM_BUG_ON_VMA() to VM_WARN_ON_ONCE_VMA().
--- mm/interval_tree.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
mm/interval_tree.c +7 -2
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0-only /* -* mm/interval_tree.c - interval tree for mapping->i_mmap +* mm/interval_tree.c - interval tree for address_space->i_mmap and +* anon_vma->rb_root * * Copyright (C) 2012, Michel Lespinasse <[email protected]> */
@@ -10,6 +11,8 @@#include <linux/rmap.h> #include <linux/interval_tree_generic.h> +/* File-backed interval tree (address_space->i_mmap) */ + INTERVAL_TREE_DEFINE(struct vm_area_struct, shared.rb, unsigned long, shared.rb_subtree_last, vma_start_pgoff, vma_last_pgoff, /* empty */, vma_interval_tree)
@@ -23,7 +26,7 @@ void vma_interval_tree_insert_after(struct vm_area_struct *node,struct vm_area_struct *parent;unsigned long last =vma_last_pgoff (node); -VM_BUG_ON_VMA (vma_start_pgoff (node) !=vma_start_pgoff (prev), node); +VM_WARN_ON_ONCE_VMA (vma_start_pgoff (node) !=vma_start_pgoff (prev), node);if (!prev->shared.rb.rb_right) { parent = prev;
@@ -48,6 +51,8 @@ void vma_interval_tree_insert_after(struct vm_area_struct *node, &vma_interval_tree_augment); } +/* Anonymous interval tree (anon_vma->rb_root) */ +static inline unsigned long avc_start_pgoff (struct anon_vma_chain *avc) {return vma_start_pgoff (avc->vma);
Signature · 1 line
2.54.0
Gregory Price
maintainerRe: [PATCH 05/30] mm/rmap: update mm/interval_tree.c comments
Quoted 4 lines · Lorenzo Stoakes
Update the file comment to clarify that both file-backed and anonymous interval trees are provided, referencing the relevant data types for clarity.
Isn't this self-evident by nature of the function definitions? (one takes a vm_area_struct, the other takes an anon_vma_chain)
mm/interval_tree.c void vma_interval_tree_insert_after(struct vm_area_struct *node, · line 29
Quoted 3 lines
mm/interval_tree.c +1
-1-VM_BUG_ON_VMA (vma_start_pgoff (node) !=vma_start_pgoff (prev), node); +VM_WARN_ON_ONCE_VMA (vma_start_pgoff (node) !=vma_start_pgoff (prev), node);
For my own edification - I know not to add new BUG(), should I be converting BUG->WARN/something when i find them in areas i happen to be working in?
~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 05/30] mm/rmap: update mm/interval_tree.c comments
Quoted 8 lines · Gregory Price
Quoted 4 lines · Lorenzo Stoakes
Update the file comment to clarify that both file-backed and anonymous interval trees are provided, referencing the relevant data types for clarity.
Isn't this self-evident by nature of the function definitions? (one takes a
vm_area_struct, the other takes ananon_vma_chain)
Well you see you're already hitting up on issues there, they both take an rb_root_cached and the vma_*() ones do not instantly scream 'file-backed' do they? As VMAs are obviously used for buth anon and file-backed...
But later patches fix this stuff :)
And I feel it's hard visually to see where one set of definitions end and another begins, which was really the motive for this, as trivial as it is!
Quoted 8 lines
mm/interval_tree.c void vma_interval_tree_insert_after(struct vm_area_struct *node, · line 29
Quoted 4 lines
mm/interval_tree.c +1
-1-VM_BUG_ON_VMA (vma_start_pgoff (node) !=vma_start_pgoff (prev), node); +VM_WARN_ON_ONCE_VMA (vma_start_pgoff (node) !=vma_start_pgoff (prev), node);For my own edification - I know not to add new
BUG(), should I be converting BUG->WARN/something when i find them in areas i happen to be working in?
Yeah pretty much in all cases.
It's very rare that you'd want the kernel to definitely oops, and I can't think of any circumstance where you'd only what that if CONFIG_DEBUG_VM was set :))
Quoted 2 lines
~Gregory
Cheers, Lorenzo
Re: [PATCH 05/30] mm/rmap: update mm/interval_tree.c comments
Quoted 10 lines · Lorenzo Stoakes
Quoted 8 lines · Gregory Price
Quoted 4 lines · Lorenzo Stoakes
Update the file comment to clarify that both file-backed and anonymous interval trees are provided, referencing the relevant data types for clarity.
Isn't this self-evident by nature of the function definitions? (one takes a
vm_area_struct, the other takes ananon_vma_chain)
Quoted 9 lines
Well you see you're already hitting up on issues there, they both take an
rb_root_cachedand the vma_*() ones do not instantly scream 'file-backed' do they? As VMAs are obviously used for buth anon and file-backed...But later patches fix this stuff :)
And I feel it's hard visually to see where one set of definitions end and another begins, which was really the motive for this, as trivial as it is!
Fair enough, I scanned the rest initially but trying to wrap my head around everything as i go through one by one. Generally this really screams "fix the apis" not "comment the bad ones" - but i suppose that's the whole point here.
It's definitely an improvement either way.
~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 05/30] mm/rmap: update mm/interval_tree.c comments
Quoted 17 lines · Gregory Price
Quoted 16 lines · Lorenzo Stoakes
Quoted 8 lines · Gregory Price
Quoted 4 lines · Lorenzo Stoakes
Update the file comment to clarify that both file-backed and anonymous interval trees are provided, referencing the relevant data types for clarity.
Isn't this self-evident by nature of the function definitions? (one takes a
vm_area_struct, the other takes ananon_vma_chain)Well you see you're already hitting up on issues there, they both take an
rb_root_cachedand the vma_*() ones do not instantly scream 'file-backed' do they? As VMAs are obviously used for buth anon and file-backed...But later patches fix this stuff :)
Quoted 8 lines
Quoted 3 lines
And I feel it's hard visually to see where one set of definitions end and another begins, which was really the motive for this, as trivial as it is!
Fair enough, I scanned the rest initially but trying to wrap my head around everything as i go through one by one. Generally this really screams "fix the apis" not "comment the bad ones" - but i suppose that's the whole point here.
Yeah intent is to eventually completely remove the anon stuff from here at least :)
And Pedro I think is looking at the file rmap so we'll get there :)
Quoted 6 lines
It's definitely an improvement either way.
Reviewed-by Gregory Price <[email protected]>~Gregory
Cheers, Lorenzo
Re: [PATCH 05/30] mm/rmap: update mm/interval_tree.c comments
In reply to Lorenzo StoakesQuoted 9 lines · Lorenzo Stoakes
Update the file comment to clarify that both file-backed and anonymous interval trees are provided, referencing the relevant data types for clarity.
Also add comments to indicate which parts of the file apply to each.
While we're here, convert the
VM_BUG_ON_VMA()toVM_WARN_ON_ONCE_VMA().Signed-off-by Lorenzo Stoakes <[email protected]>
This is fine for now, but I'm wondering if it doesn't make sense to, in the long term, have:
mm/rmap.c - common rmap mechanismsmm/anon_rmap.c - anon rmap gunkmm/file_rmap.c - file rmap gunk
or even something like mm/rmap/{core,anon,file,ksm??}.c
While working on my file rmap patches I noticed there's so much stuff just splurged all over rmap.c - interval_tree.c - fs.h - fs/inode.c. It's a little silly.
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 05/30] mm/rmap: update mm/interval_tree.c comments
Quoted 20 lines · Pedro Falcato
Quoted 9 lines · Lorenzo Stoakes
Update the file comment to clarify that both file-backed and anonymous interval trees are provided, referencing the relevant data types for clarity.
Also add comments to indicate which parts of the file apply to each.
While we're here, convert the
VM_BUG_ON_VMA()toVM_WARN_ON_ONCE_VMA().Signed-off-by Lorenzo Stoakes <[email protected]>Reviewed-by Pedro Falcato <[email protected]>This is fine for now, but I'm wondering if it doesn't make sense to, in the long term, have:
mm/rmap.c- common rmap mechanismsmm/anon_rmap.c- anon rmap gunkmm/file_rmap.c- file rmap gunk
Quoted 5 lines
or even something like mm/rmap/{core,anon,file,ksm??}.c
While working on my file rmap patches I noticed there's so much stuff just splurged all over
rmap.c-interval_tree.c-fs.h-fs/inode.c. It's a little silly.
Well, Wei had something like this idea a way back, but I'd rather avoid it.
Firstly, with scalable cow coming, I'd rather us not make anon_vma a special citizen in any way, and I'm going to be heavily modifying all this anyway.
On the interval tree side, I'm simply going to get rid of the anon side of it altogether with scalable CoW also so that can live as it is now for the time being.
In general we try to have generalised rmap walk logic for file vs. anon with rmap_walk -> rmap_walk_[anon, file]() for one obviously.
But obviously there's separate rmap walker logic for file vs. anon.
I used to harbour a belief that we could make anon like file-backed but now I'm kind of thinking that's not possible :)
So I guess the real answer is - yeah, but let's revisit it after scalable CoW (and anyway I'm likely to do sensible architectural breakouts as part of that work anyway).
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Lorenzo Stoakes
maintainerPATCH 6/30 mm/rmap: parameterise vma_interval_tree_*() by address_space
In reply to Lorenzo StoakesThe file-backed mapping interval tree functions vma_interval_tree_*() accept a raw rb_root_cached pointer to determine the tree in which they are operating.
However, in each case, this is always associated with an address_space data type.
So simply pass a pointer to that instead to simplify the code, and more clearly differentiate between these operations and those concerning anonymous mappings.
While we're here, make the generated interval tree functions static as they do not need to be used externally (any previously existing external users have now been removed).
We also rename VMA parameters from 'node' to 'vma' as calling this a node is simply confusing, update the input index types to pgoff_t since they reference page offsets and rename the parameters to pgoff_start and pgoff_last.
No functional change intended.
--- arch/arm/mm/fault-armv.c | 2 +- arch/arm/mm/flush.c | 2 +- arch/nios2/mm/cacheflush.c | 2 +- arch/parisc/kernel/cache.c | 2 +- fs/dax.c | 2 +- fs/hugetlbfs/inode.c | 15 ++++---- include/linux/mm.h | 34 +++++++++--------- kernel/events/uprobes.c | 2 +- mm/hugetlb.c | 4 +-- mm/interval_tree.c | 58 +++++++++++++++++++++++-------- mm/khugepaged.c | 4 +-- mm/memory-failure.c | 7 ++-- mm/memory.c | 8 ++--- mm/mmap.c | 3 +- mm/nommu.c | 8 ++--- mm/pagewalk.c | 2 +- mm/rmap.c | 3 +- mm/vma.c | 14 ++++---- tools/testing/vma/include/stubs.h | 4 +-- 19 files changed, 100 insertions(+), 76 deletions(-)
arch/arm/mm/fault-armv.c +1 -1
@@ -140,7 +140,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma,* cache coherency. */ flush_dcache_mmap_lock (mapping); - vma_interval_tree_foreach(mpnt, &mapping->i_mmap, pgoff, pgoff) { + vma_interval_tree_foreach(mpnt, mapping, pgoff, pgoff) {/* * If we are using split PTE locks, then we need to take the pte * lock. Otherwise we are using shared mm->page_table_lock which
arch/arm/mm/flush.c +1 -1
@@ -251,7 +251,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct folio * pgoff_end = pgoff +folio_nr_pages (folio) -1 ;flush_dcache_mmap_lock (mapping); - vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff_end) { + vma_interval_tree_foreach(vma, mapping, pgoff, pgoff_end) {unsigned long start, offset, pfn;unsigned int nr;
arch/nios2/mm/cacheflush.c +1 -1
@@ -82,7 +82,7 @@ static void flush_aliases(struct address_space *mapping, struct folio *folio) pgoff = folio->index;flush_dcache_mmap_lock_irqsave (mapping, flags); - vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff + nr - 1) { + vma_interval_tree_foreach(vma, mapping, pgoff, pgoff + nr - 1) {unsigned long start;if (vma->vm_mm != mm)
arch/parisc/kernel/cache.c +1 -1
@@ -503,7 +503,7 @@ void flush_dcache_folio(struct folio *folio)* on machines that support equivalent aliasing */ flush_dcache_mmap_lock_irqsave (mapping, flags); - vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff + nr - 1) { + vma_interval_tree_foreach(vma, mapping, pgoff, pgoff + nr - 1) {unsigned long offset = pgoff - vma->vm_pgoff;unsigned long pfn =folio_pfn (folio);
fs/dax.c +1 -1
@@ -1201,7 +1201,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev,/* Walk all mappings of a given index of a file and writeprotect them */ i_mmap_lock_read (mapping); - vma_interval_tree_foreach(vma, &mapping->i_mmap, index, end) { + vma_interval_tree_foreach(vma, mapping, index, end) {pfn_mkclean_range (pfn, count, index, vma);cond_resched (); }
fs/hugetlbfs/inode.c +7 -8
@@ -380,7 +380,6 @@ static void hugetlb_unmap_file_folio(struct hstate *h,struct address_space *mapping, struct folio *folio, pgoff_t index) { - struct rb_root_cached *root = &mapping->i_mmap; struct hugetlb_vma_lock *vma_lock;unsigned long pfn =folio_pfn (folio);struct vm_area_struct *vma;
@@ -394,7 +393,7 @@ static void hugetlb_unmap_file_folio(struct hstate *h,i_mmap_lock_write (mapping); retry: vma_lock =NULL ; - vma_interval_tree_foreach(vma, root, start, end - 1) { + vma_interval_tree_foreach(vma, mapping, start, end - 1) { v_start =vma_offset_start (vma, start); v_end =vma_offset_end (vma, end);
@@ -460,8 +459,8 @@ static void hugetlb_unmap_file_folio(struct hstate *h, }static void -hugetlb_vmdelete_list (struct rb_root_cached *root,pgoff_t start,pgoff_t end, -zap_flags_t zap_flags) +hugetlb_vmdelete_list (struct address_space *mapping,pgoff_t start, +pgoff_t end,zap_flags_t zap_flags) {struct vm_area_struct *vma;
@@ -470,7 +469,8 @@ hugetlb_vmdelete_list(struct rb_root_cached *root, pgoff_t start, pgoff_t end,* unmapped. Note, end is exclusive, whereas the interval tree takes * an inclusive "last". */ - vma_interval_tree_foreach(vma, root, start, end ? end - 1 : ULONG_MAX) { +vma_interval_tree_foreach (vma, mapping, start, + end ? end - 1 : ULONG_MAX) {unsigned long v_start;unsigned long v_end;
@@ -615,8 +615,7 @@ static void hugetlb_vmtruncate(struct inode *inode, loff_t offset)i_size_write (inode, offset);i_mmap_lock_write (mapping);if (mapping_mapped (mapping)) -hugetlb_vmdelete_list (&mapping->i_mmap, pgoff,0 , -ZAP_FLAG_DROP_MARKER ); +hugetlb_vmdelete_list (mapping, pgoff,0 ,ZAP_FLAG_DROP_MARKER );i_mmap_unlock_write (mapping);remove_inode_hugepages (inode, offset,LLONG_MAX ); }
@@ -676,7 +675,7 @@ static long hugetlbfs_punch_hole(struct inode *inode, loff_t offset, loff_t len)/* Unmap users of full pages in the hole. */ if (hole_end > hole_start) {if (mapping_mapped (mapping)) -hugetlb_vmdelete_list (&mapping->i_mmap, +hugetlb_vmdelete_list (mapping, hole_start >>PAGE_SHIFT , hole_end >>PAGE_SHIFT ,0 ); }
include/linux/mm.h +18 -16
@@ -4061,23 +4061,25 @@ extern atomic_long_t mmap_pages_allocated;extern int nommu_shrink_inode_mappings (struct inode *,size_t ,size_t );/* interval_tree.c */ -void vma_interval_tree_insert (struct vm_area_struct *node, -struct rb_root_cached *root); -void vma_interval_tree_insert_after (struct vm_area_struct *node, +void vma_interval_tree_insert (struct vm_area_struct *vma, +struct address_space *mapping); +void vma_interval_tree_insert_after (struct vm_area_struct *vma,struct vm_area_struct *prev, -struct rb_root_cached *root); -void vma_interval_tree_remove (struct vm_area_struct *node, -struct rb_root_cached *root); -struct vm_area_struct *vma_interval_tree_subtree_search (struct vm_area_struct *node, -unsigned long start,unsigned long last); -struct vm_area_struct *vma_interval_tree_iter_first (struct rb_root_cached *root, -unsigned long start,unsigned long last); -struct vm_area_struct *vma_interval_tree_iter_next (struct vm_area_struct *node, -unsigned long start,unsigned long last); - -#define vma_interval_tree_foreach (vma, root, start, last) \ - for (vma = vma_interval_tree_iter_first(root, start, last); \ - vma; vma = vma_interval_tree_iter_next(vma, start, last)) +struct address_space *mapping); +void vma_interval_tree_remove (struct vm_area_struct *vma, +struct address_space *mapping); +struct vm_area_struct * +vma_interval_tree_iter_first (struct address_space *mapping, +pgoff_t pgoff_start,pgoff_t pgoff_last); +struct vm_area_struct * +vma_interval_tree_iter_next (struct vm_area_struct *vma, +pgoff_t pgoff_start,pgoff_t pgoff_last); + +#define vma_interval_tree_foreach (vma, mapping, pgoff_start, pgoff_last) \ + for (vma = vma_interval_tree_iter_first(mapping, pgoff_start, \ + pgoff_last); \ + vma; vma = vma_interval_tree_iter_next(vma, pgoff_start, \ + pgoff_last))void anon_vma_interval_tree_insert (struct anon_vma_chain *node,struct rb_root_cached *root);
kernel/events/uprobes.c +1 -1
@@ -1210,7 +1210,7 @@ build_map_info(struct address_space *mapping, loff_t offset, bool is_register) again:i_mmap_lock_read (mapping); - vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) { + vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) {if (!valid_vma (vma, is_register))continue ;
mm/hugetlb.c +2 -2
@@ -5382,7 +5382,7 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,* __unmap_hugepage_range() is called as the lock is already held */ i_mmap_lock_write (mapping); - vma_interval_tree_foreach(iter_vma, &mapping->i_mmap, pgoff, pgoff) { + vma_interval_tree_foreach(iter_vma, mapping, pgoff, pgoff) {/* Do not unmap the current VMA */ if (iter_vma == vma)continue ;
@@ -6864,7 +6864,7 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,pte_t *pte;i_mmap_lock_read (mapping); - vma_interval_tree_foreach(svma, &mapping->i_mmap, idx, idx) { + vma_interval_tree_foreach(svma, mapping, idx, idx) {if (svma == vma)continue ;
mm/interval_tree.c +43 -15
83 lines not on this page · raw message
mm/khugepaged.c +2 -2
@@ -2136,7 +2136,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)struct vm_area_struct *vma;i_mmap_lock_read (mapping); - vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) { + vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) {struct mmu_notifier_range range;struct mm_struct *mm;unsigned long addr;
@@ -2568,7 +2568,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,* not be able to observe any missing pages due to the * previously inserted retry entries. */ - vma_interval_tree_foreach(vma, &mapping->i_mmap, start, end) { + vma_interval_tree_foreach(vma, mapping, start, end) {if (userfaultfd_missing (vma)) { result =SCAN_EXCEED_NONE_PTE ; goto immap_locked;
mm/memory-failure.c +3 -4
@@ -586,8 +586,7 @@ static void collect_procs_file(const struct folio *folio,if (!t)continue ; -vma_interval_tree_foreach (vma, &mapping->i_mmap, pgoff, - pgoff) { + vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) {/* * Send early kill signal to tasks where a vma covers * the page but the corrupted page is not necessarily
@@ -638,7 +637,7 @@ static void collect_procs_fsdax(const struct page *page, t =task_early_kill (tsk, true);if (!t)continue ; - vma_interval_tree_foreach(vma, &mapping->i_mmap, pgoff, pgoff) { + vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) {if (vma->vm_mm == t->mm)add_to_kill_fsdax (t, page, vma, to_kill, pgoff); }
@@ -2239,7 +2238,7 @@ static void collect_procs_pfn(struct pfn_address_space *pfn_space, t =task_early_kill (tsk, true);if (!t)continue ; - vma_interval_tree_foreach(vma, &mapping->i_mmap, 0, ULONG_MAX) { + vma_interval_tree_foreach(vma, mapping, 0, ULONG_MAX) {pgoff_t pgoff; if (vma->vm_mm == t->mm &&
mm/memory.c +4 -4
@@ -4336,7 +4336,7 @@ static vm_fault_t do_wp_page(struct vm_fault *vmf)return wp_page_copy (vmf); } -static inline void unmap_mapping_range_tree (struct rb_root_cached *root, +static inline void unmap_mapping_range_tree (struct address_space *mapping,pgoff_t first_index,pgoff_t last_index,struct zap_details *details)
@@ -4345,7 +4345,7 @@ static inline void unmap_mapping_range_tree(struct rb_root_cached *root,unsigned long start, size;struct mmu_gather tlb; - vma_interval_tree_foreach(vma, root, first_index, last_index) { + vma_interval_tree_foreach(vma, mapping, first_index, last_index) {const pgoff_t start_idx =max (first_index, vma->vm_pgoff);const pgoff_t end_idx =min (last_index,vma_last_pgoff (vma)) +1 ;
@@ -4387,7 +4387,7 @@ void unmap_mapping_folio(struct folio *folio)i_mmap_lock_read (mapping);if (unlikely (mapping_mapped (mapping))) -unmap_mapping_range_tree (&mapping->i_mmap, first_index, +unmap_mapping_range_tree (mapping, first_index, last_index, &details);i_mmap_unlock_read (mapping); }
@@ -4417,7 +4417,7 @@ void unmap_mapping_pages(struct address_space *mapping, pgoff_t start,i_mmap_lock_read (mapping);if (unlikely (mapping_mapped (mapping))) -unmap_mapping_range_tree (&mapping->i_mmap, first_index, +unmap_mapping_range_tree (mapping, first_index, last_index, &details);i_mmap_unlock_read (mapping); }
mm/mmap.c +1 -2
@@ -1830,8 +1830,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)mapping_allow_writable (mapping);flush_dcache_mmap_lock (mapping);/* insert tmp into the share list, just after mpnt */ -vma_interval_tree_insert_after (tmp, mpnt, - &mapping->i_mmap); +vma_interval_tree_insert_after (tmp, mpnt, mapping);flush_dcache_mmap_unlock (mapping);i_mmap_unlock_write (mapping); }
mm/nommu.c +4 -4
@@ -569,7 +569,7 @@ static void setup_vma_to_mm(struct vm_area_struct *vma, struct mm_struct *mm)i_mmap_lock_write (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_insert (vma, &mapping->i_mmap); +vma_interval_tree_insert (vma, mapping);flush_dcache_mmap_unlock (mapping);i_mmap_unlock_write (mapping); }
@@ -585,7 +585,7 @@ static void cleanup_vma_from_mm(struct vm_area_struct *vma)i_mmap_lock_write (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_remove (vma, &mapping->i_mmap); +vma_interval_tree_remove (vma, mapping);flush_dcache_mmap_unlock (mapping);i_mmap_unlock_write (mapping); }
@@ -1816,7 +1816,7 @@ int nommu_shrink_inode_mappings(struct inode *inode, size_t size,i_mmap_lock_read (inode->i_mapping);/* search for VMAs that fall within the dead zone */ - vma_interval_tree_foreach(vma, &inode->i_mapping->i_mmap, low, high) { + vma_interval_tree_foreach(vma, inode->i_mapping, low, high) {/* found one - only interested if it's shared out of the page * cache */ if (vma->vm_flags &VM_SHARED ) {
@@ -1832,7 +1832,7 @@ int nommu_shrink_inode_mappings(struct inode *inode, size_t size,* we don't check for any regions that start beyond the EOF as there * shouldn't be any */ - vma_interval_tree_foreach(vma, &inode->i_mapping->i_mmap, 0, ULONG_MAX) { + vma_interval_tree_foreach(vma, inode->i_mapping, 0, ULONG_MAX) {if (!(vma->vm_flags &VM_SHARED ))continue ;
mm/pagewalk.c +1 -1
@@ -810,7 +810,7 @@ int walk_page_mapping(struct address_space *mapping, pgoff_t first_index,return -EINVAL ;lockdep_assert_held (&mapping->i_mmap_rwsem); -vma_interval_tree_foreach (vma, &mapping->i_mmap, first_index, +vma_interval_tree_foreach (vma, mapping, first_index, first_index + nr - 1) {/* Clip to the vma */ vba = vma->vm_pgoff;
mm/rmap.c +1 -2
@@ -3051,8 +3051,7 @@ static void __rmap_walk_file(struct folio *folio, struct address_space *mapping,i_mmap_lock_read (mapping); } lookup: -vma_interval_tree_foreach (vma, &mapping->i_mmap, - pgoff_start, pgoff_end) { + vma_interval_tree_foreach(vma, mapping, pgoff_start, pgoff_end) {unsigned long address =vma_address (vma, pgoff_start, nr_pages);VM_BUG_ON_VMA (address == -EFAULT , vma);
mm/vma.c +6 -8
@@ -234,7 +234,7 @@ static void __vma_link_file(struct vm_area_struct *vma,mapping_allow_writable (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_insert (vma, &mapping->i_mmap); +vma_interval_tree_insert (vma, mapping);flush_dcache_mmap_unlock (mapping); }
@@ -248,7 +248,7 @@ static void __remove_shared_vm_struct(struct vm_area_struct *vma,mapping_unmap_writable (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_remove (vma, &mapping->i_mmap); +vma_interval_tree_remove (vma, mapping);flush_dcache_mmap_unlock (mapping); }
@@ -319,10 +319,9 @@ static void vma_prepare(struct vma_prepare *vp)if (vp->file) {flush_dcache_mmap_lock (vp->mapping); -vma_interval_tree_remove (vp->vma, &vp->mapping->i_mmap); +vma_interval_tree_remove (vp->vma, vp->mapping);if (vp->adj_next) -vma_interval_tree_remove (vp->adj_next, - &vp->mapping->i_mmap); +vma_interval_tree_remove (vp->adj_next, vp->mapping); } }
@@ -340,9 +339,8 @@ static void vma_complete(struct vma_prepare *vp, struct vma_iterator *vmi, {if (vp->file) {if (vp->adj_next) -vma_interval_tree_insert (vp->adj_next, - &vp->mapping->i_mmap); -vma_interval_tree_insert (vp->vma, &vp->mapping->i_mmap); +vma_interval_tree_insert (vp->adj_next, vp->mapping); +vma_interval_tree_insert (vp->vma, vp->mapping);flush_dcache_mmap_unlock (vp->mapping); }
tools/testing/vma/include/stubs.h +2 -2
@@ -258,12 +258,12 @@ static inline void vm_acct_memory(long pages) }static inline void vma_interval_tree_insert (struct vm_area_struct *vma, -struct rb_root_cached *rb) +struct address_space *mapping) { }static inline void vma_interval_tree_remove (struct vm_area_struct *vma, -struct rb_root_cached *rb) +struct address_space *mapping) { }
Signature · 1 line
2.54.0
Re: [PATCH 06/30] mm/rmap: parameterise vma_interval_tree_*() by address_space
Quoted 15 lines · Lorenzo Stoakes
The file-backed mapping interval tree functions vma_interval_tree_*() accept a raw
rb_root_cachedpointer to determine the tree in which they are operating.However, in each case, this is always associated with an
address_spacedata type.So simply pass a pointer to that instead to simplify the code, and more clearly differentiate between these operations and those concerning anonymous mappings.
While we're here, make the generated interval tree functions static as they do not need to be used externally (any previously existing external users have now been removed).
Quoted 8 lines
We also rename VMA parameters from 'node' to 'vma' as calling this a node is simply confusing, update the input index types to
pgoff_tsince they reference page offsets and rename the parameters topgoff_startandpgoff_last.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
1) This is fantastic
2) I need to rebase my local work :)
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 06/30] mm/rmap: parameterise vma_interval_tree_*() by address_space
Quoted 25 lines · Pedro Falcato
Quoted 15 lines · Lorenzo Stoakes
The file-backed mapping interval tree functions vma_interval_tree_*() accept a raw
rb_root_cachedpointer to determine the tree in which they are operating.However, in each case, this is always associated with an
address_spacedata type.So simply pass a pointer to that instead to simplify the code, and more clearly differentiate between these operations and those concerning anonymous mappings.
While we're here, make the generated interval tree functions static as they do not need to be used externally (any previously existing external users have now been removed).
Quoted 8 lines
We also rename VMA parameters from 'node' to 'vma' as calling this a node is simply confusing, update the input index types to
pgoff_tsince they reference page offsets and rename the parameters topgoff_startandpgoff_last.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 7 lines
1) This is fantastic
2) I need to rebase my local work :)Reviewed-by Pedro Falcato <[email protected]>--
Pedro
1) thanks :)
2) sorry :P
Cheers, Lorenzo
Re: [PATCH 06/30] mm/rmap: parameterise vma_interval_tree_*() by address_space
In reply to Lorenzo StoakesQuoted 15 lines · Lorenzo Stoakes
The file-backed mapping interval tree functions vma_interval_tree_*() accept a raw
rb_root_cachedpointer to determine the tree in which they are operating.However, in each case, this is always associated with an
address_spacedata type.So simply pass a pointer to that instead to simplify the code, and more clearly differentiate between these operations and those concerning anonymous mappings.
While we're here, make the generated interval tree functions static as they do not need to be used externally (any previously existing external users have now been removed).
Quoted 8 lines
We also rename VMA parameters from 'node' to 'vma' as calling this a node is simply confusing, update the input index types to
pgoff_tsince they reference page offsets and rename the parameters topgoff_startandpgoff_last.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
nice
Lorenzo Stoakes
maintainerPATCH 7/30 mm/rmap: elide unnecessary static inline's in interval_tree.c
In reply to Lorenzo StoakesIt's not necessary to declare these functions static inline as they are contained within a single compilation unit.
This makes the anonymous interval tree code consistent with the newly updated file-backed interval tree code.
No functional change intended.
--- mm/interval_tree.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
mm/interval_tree.c +3 -3
@@ -81,19 +81,19 @@ vma_interval_tree_iter_next(struct vm_area_struct *vma,/* Anonymous interval tree (anon_vma->rb_root) */ -static inline unsigned long avc_start_pgoff (struct anon_vma_chain *avc) +static unsigned long avc_start_pgoff (struct anon_vma_chain *avc) {return vma_start_pgoff (avc->vma); } -static inline unsigned long avc_last_pgoff (struct anon_vma_chain *avc) +static unsigned long avc_last_pgoff (struct anon_vma_chain *avc) {return vma_last_pgoff (avc->vma); } INTERVAL_TREE_DEFINE(struct anon_vma_chain, rb, unsigned long, rb_subtree_last, avc_start_pgoff, avc_last_pgoff, - static inline, __anon_vma_interval_tree) + static, __anon_vma_interval_tree)void anon_vma_interval_tree_insert (struct anon_vma_chain *node,struct rb_root_cached *root)
Signature · 1 line
2.54.0
Re: [PATCH 07/30] mm/rmap: elide unnecessary static inline's in interval_tree.c
Quoted 9 lines · Lorenzo Stoakes
It's not necessary to declare these functions static inline as they are contained within a single compilation unit.
This makes the anonymous interval tree code consistent with the newly updated file-backed interval tree code.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Re: [PATCH 07/30] mm/rmap: elide unnecessary static inline's in interval_tree.c
In reply to Lorenzo StoakesQuoted 2 lines · Lorenzo Stoakes
It's not necessary to declare these functions static inline as they are contained within a single compilation unit.
FWIW I don't think it's technically strictly equivalent (I have a vague memory of inline lightly suggesting inlining a function to GCC, because of compat reasons). Anyway, doesn't really matter, practically any static function gets inlined with -O2 anyway.
Quoted 7 lines
This makes the anonymous interval tree code consistent with the newly updated file-backed interval tree code.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerPATCH 8/30 mm/rmap: rename vma_interval_tree_*() to mapping_interval_tree_*()
In reply to Lorenzo StoakesThe family of vma_interval_tree_() functions manipulate the address_space (which, of course, is generally referred to as 'mapping') reverse mapping, but are named the 'VMA' interval tree.
VMAs may be mapped by an anon_vma, an address_space, or both. Therefore calling the mapping interval tree a 'VMA' interval tree is rather confusing.
This is also inconsistent with the anon_vma_interval_tree_*() functions which explicitly reference the rmap object to which they pertain.
Rename the vma_interval_tree_*() functions to mapping_interval_tree_*() to correct this.
No functional change intended.
--- arch/arm/mm/fault-armv.c | 2 +- arch/arm/mm/flush.c | 2 +- arch/nios2/mm/cacheflush.c | 2 +- arch/parisc/kernel/cache.c | 2 +- fs/dax.c | 2 +- fs/hugetlbfs/inode.c | 6 +++--- include/linux/mm.h | 34 +++++++++++++++---------------- kernel/events/uprobes.c | 2 +- mm/hugetlb.c | 4 ++-- mm/interval_tree.c | 22 ++++++++++---------- mm/khugepaged.c | 4 ++-- mm/memory-failure.c | 6 +++--- mm/memory.c | 2 +- mm/mmap.c | 2 +- mm/nommu.c | 8 ++++---- mm/pagewalk.c | 4 ++-- mm/rmap.c | 2 +- mm/vma.c | 12 +++++------ tools/testing/vma/include/stubs.h | 8 ++++---- 19 files changed, 63 insertions(+), 63 deletions(-)
arch/arm/mm/fault-armv.c +1 -1
@@ -140,7 +140,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma,* cache coherency. */ flush_dcache_mmap_lock (mapping); - vma_interval_tree_foreach(mpnt, mapping, pgoff, pgoff) { + mapping_interval_tree_foreach(mpnt, mapping, pgoff, pgoff) {/* * If we are using split PTE locks, then we need to take the pte * lock. Otherwise we are using shared mm->page_table_lock which
arch/arm/mm/flush.c +1 -1
@@ -251,7 +251,7 @@ static void __flush_dcache_aliases(struct address_space *mapping, struct folio * pgoff_end = pgoff +folio_nr_pages (folio) -1 ;flush_dcache_mmap_lock (mapping); - vma_interval_tree_foreach(vma, mapping, pgoff, pgoff_end) { + mapping_interval_tree_foreach(vma, mapping, pgoff, pgoff_end) {unsigned long start, offset, pfn;unsigned int nr;
arch/nios2/mm/cacheflush.c +1 -1
@@ -82,7 +82,7 @@ static void flush_aliases(struct address_space *mapping, struct folio *folio) pgoff = folio->index;flush_dcache_mmap_lock_irqsave (mapping, flags); - vma_interval_tree_foreach(vma, mapping, pgoff, pgoff + nr - 1) { + mapping_interval_tree_foreach(vma, mapping, pgoff, pgoff + nr - 1) {unsigned long start;if (vma->vm_mm != mm)
arch/parisc/kernel/cache.c +1 -1
@@ -503,7 +503,7 @@ void flush_dcache_folio(struct folio *folio)* on machines that support equivalent aliasing */ flush_dcache_mmap_lock_irqsave (mapping, flags); - vma_interval_tree_foreach(vma, mapping, pgoff, pgoff + nr - 1) { + mapping_interval_tree_foreach(vma, mapping, pgoff, pgoff + nr - 1) {unsigned long offset = pgoff - vma->vm_pgoff;unsigned long pfn =folio_pfn (folio);
fs/dax.c +1 -1
@@ -1201,7 +1201,7 @@ static int dax_writeback_one(struct xa_state *xas, struct dax_device *dax_dev,/* Walk all mappings of a given index of a file and writeprotect them */ i_mmap_lock_read (mapping); - vma_interval_tree_foreach(vma, mapping, index, end) { + mapping_interval_tree_foreach(vma, mapping, index, end) {pfn_mkclean_range (pfn, count, index, vma);cond_resched (); }
fs/hugetlbfs/inode.c +3 -3
@@ -393,7 +393,7 @@ static void hugetlb_unmap_file_folio(struct hstate *h,i_mmap_lock_write (mapping); retry: vma_lock =NULL ; - vma_interval_tree_foreach(vma, mapping, start, end - 1) { + mapping_interval_tree_foreach(vma, mapping, start, end - 1) { v_start =vma_offset_start (vma, start); v_end =vma_offset_end (vma, end);
@@ -469,8 +469,8 @@ hugetlb_vmdelete_list(struct address_space *mapping, pgoff_t start,* unmapped. Note, end is exclusive, whereas the interval tree takes * an inclusive "last". */ -vma_interval_tree_foreach (vma, mapping, start, - end ? end - 1 : ULONG_MAX) { +mapping_interval_tree_foreach (vma, mapping, start, + end ? end - 1 : ULONG_MAX) {unsigned long v_start;unsigned long v_end;
include/linux/mm.h +17 -17
@@ -4061,25 +4061,25 @@ extern atomic_long_t mmap_pages_allocated;extern int nommu_shrink_inode_mappings (struct inode *,size_t ,size_t );/* interval_tree.c */ -void vma_interval_tree_insert (struct vm_area_struct *vma, -struct address_space *mapping); -void vma_interval_tree_insert_after (struct vm_area_struct *vma, -struct vm_area_struct *prev, -struct address_space *mapping); -void vma_interval_tree_remove (struct vm_area_struct *vma, -struct address_space *mapping); +void mapping_interval_tree_insert (struct vm_area_struct *vma, +struct address_space *mapping); +void mapping_interval_tree_insert_after (struct vm_area_struct *vma, +struct vm_area_struct *prev, +struct address_space *mapping); +void mapping_interval_tree_remove (struct vm_area_struct *vma, +struct address_space *mapping);struct vm_area_struct * -vma_interval_tree_iter_first (struct address_space *mapping, -pgoff_t pgoff_start,pgoff_t pgoff_last); +mapping_interval_tree_iter_first (struct address_space *mapping, +pgoff_t pgoff_start,pgoff_t pgoff_last);struct vm_area_struct * -vma_interval_tree_iter_next (struct vm_area_struct *vma, -pgoff_t pgoff_start,pgoff_t pgoff_last); - -#define vma_interval_tree_foreach (vma, mapping, pgoff_start, pgoff_last) \ - for (vma = vma_interval_tree_iter_first(mapping, pgoff_start, \ - pgoff_last); \ - vma; vma = vma_interval_tree_iter_next(vma, pgoff_start, \ - pgoff_last)) +mapping_interval_tree_iter_next (struct vm_area_struct *vma, +pgoff_t pgoff_start,pgoff_t pgoff_last); + +#define mapping_interval_tree_foreach (vma, mapping, pgoff_start, pgoff_last) \ + for (vma = mapping_interval_tree_iter_first(mapping, pgoff_start, \ + pgoff_last); \ + vma; vma = mapping_interval_tree_iter_next(vma, pgoff_start, \ + pgoff_last))void anon_vma_interval_tree_insert (struct anon_vma_chain *node,struct rb_root_cached *root);
kernel/events/uprobes.c +1 -1
@@ -1210,7 +1210,7 @@ build_map_info(struct address_space *mapping, loff_t offset, bool is_register) again:i_mmap_lock_read (mapping); - vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) { + mapping_interval_tree_foreach(vma, mapping, pgoff, pgoff) {if (!valid_vma (vma, is_register))continue ;
mm/hugetlb.c +2 -2
@@ -5382,7 +5382,7 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,* __unmap_hugepage_range() is called as the lock is already held */ i_mmap_lock_write (mapping); - vma_interval_tree_foreach(iter_vma, mapping, pgoff, pgoff) { + mapping_interval_tree_foreach(iter_vma, mapping, pgoff, pgoff) {/* Do not unmap the current VMA */ if (iter_vma == vma)continue ;
@@ -6864,7 +6864,7 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma,pte_t *pte;i_mmap_lock_read (mapping); - vma_interval_tree_foreach(svma, mapping, idx, idx) { + mapping_interval_tree_foreach(svma, mapping, idx, idx) {if (svma == vma)continue ;
mm/interval_tree.c +11 -11
@@ -18,16 +18,16 @@ INTERVAL_TREE_DEFINE(struct vm_area_struct, shared.rb, vma_start_pgoff, vma_last_pgoff, static, __vma_interval_tree) -void vma_interval_tree_insert (struct vm_area_struct *vma, -struct address_space *mapping) +void mapping_interval_tree_insert (struct vm_area_struct *vma, +struct address_space *mapping) {__vma_interval_tree_insert (vma, &mapping->i_mmap); }/* Insert vma immediately after prev in the interval tree */ -void vma_interval_tree_insert_after (struct vm_area_struct *vma, -struct vm_area_struct *prev, -struct address_space *mapping) +void mapping_interval_tree_insert_after (struct vm_area_struct *vma, +struct vm_area_struct *prev, +struct address_space *mapping) {struct rb_node **link;struct vm_area_struct *parent;
@@ -58,23 +58,23 @@ void vma_interval_tree_insert_after(struct vm_area_struct *vma, &__vma_interval_tree_augment); } -void vma_interval_tree_remove (struct vm_area_struct *vma, -struct address_space *mapping) +void mapping_interval_tree_remove (struct vm_area_struct *vma, +struct address_space *mapping) {__vma_interval_tree_remove (vma, &mapping->i_mmap); }struct vm_area_struct * -vma_interval_tree_iter_first (struct address_space *mapping, -pgoff_t pgoff_start,pgoff_t pgoff_last) +mapping_interval_tree_iter_first (struct address_space *mapping, +pgoff_t pgoff_start,pgoff_t pgoff_last) {return __vma_interval_tree_iter_first (&mapping->i_mmap, pgoff_start, pgoff_last); }struct vm_area_struct * -vma_interval_tree_iter_next (struct vm_area_struct *vma, -pgoff_t pgoff_start,pgoff_t pgoff_last) +mapping_interval_tree_iter_next (struct vm_area_struct *vma, +pgoff_t pgoff_start,pgoff_t pgoff_last) {return __vma_interval_tree_iter_next (vma, pgoff_start, pgoff_last); }
mm/khugepaged.c +2 -2
@@ -2136,7 +2136,7 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)struct vm_area_struct *vma;i_mmap_lock_read (mapping); - vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) { + mapping_interval_tree_foreach(vma, mapping, pgoff, pgoff) {struct mmu_notifier_range range;struct mm_struct *mm;unsigned long addr;
@@ -2568,7 +2568,7 @@ static enum scan_result collapse_file(struct mm_struct *mm, unsigned long addr,* not be able to observe any missing pages due to the * previously inserted retry entries. */ - vma_interval_tree_foreach(vma, mapping, start, end) { + mapping_interval_tree_foreach(vma, mapping, start, end) {if (userfaultfd_missing (vma)) { result =SCAN_EXCEED_NONE_PTE ; goto immap_locked;
mm/memory-failure.c +3 -3
@@ -586,7 +586,7 @@ static void collect_procs_file(const struct folio *folio,if (!t)continue ; - vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) { + mapping_interval_tree_foreach(vma, mapping, pgoff, pgoff) {/* * Send early kill signal to tasks where a vma covers * the page but the corrupted page is not necessarily
@@ -637,7 +637,7 @@ static void collect_procs_fsdax(const struct page *page, t =task_early_kill (tsk, true);if (!t)continue ; - vma_interval_tree_foreach(vma, mapping, pgoff, pgoff) { + mapping_interval_tree_foreach(vma, mapping, pgoff, pgoff) {if (vma->vm_mm == t->mm)add_to_kill_fsdax (t, page, vma, to_kill, pgoff); }
@@ -2238,7 +2238,7 @@ static void collect_procs_pfn(struct pfn_address_space *pfn_space, t =task_early_kill (tsk, true);if (!t)continue ; - vma_interval_tree_foreach(vma, mapping, 0, ULONG_MAX) { + mapping_interval_tree_foreach(vma, mapping, 0, ULONG_MAX) {pgoff_t pgoff; if (vma->vm_mm == t->mm &&
mm/memory.c +1 -1
@@ -4345,7 +4345,7 @@ static inline void unmap_mapping_range_tree(struct address_space *mapping,unsigned long start, size;struct mmu_gather tlb; - vma_interval_tree_foreach(vma, mapping, first_index, last_index) { + mapping_interval_tree_foreach(vma, mapping, first_index, last_index) {const pgoff_t start_idx =max (first_index, vma->vm_pgoff);const pgoff_t end_idx =min (last_index,vma_last_pgoff (vma)) +1 ;
mm/mmap.c +1 -1
@@ -1830,7 +1830,7 @@ __latent_entropy int dup_mmap(struct mm_struct *mm, struct mm_struct *oldmm)mapping_allow_writable (mapping);flush_dcache_mmap_lock (mapping);/* insert tmp into the share list, just after mpnt */ -vma_interval_tree_insert_after (tmp, mpnt, mapping); +mapping_interval_tree_insert_after (tmp, mpnt, mapping);flush_dcache_mmap_unlock (mapping);i_mmap_unlock_write (mapping); }
mm/nommu.c +4 -4
@@ -569,7 +569,7 @@ static void setup_vma_to_mm(struct vm_area_struct *vma, struct mm_struct *mm)i_mmap_lock_write (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_insert (vma, mapping); +mapping_interval_tree_insert (vma, mapping);flush_dcache_mmap_unlock (mapping);i_mmap_unlock_write (mapping); }
@@ -585,7 +585,7 @@ static void cleanup_vma_from_mm(struct vm_area_struct *vma)i_mmap_lock_write (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_remove (vma, mapping); +mapping_interval_tree_remove (vma, mapping);flush_dcache_mmap_unlock (mapping);i_mmap_unlock_write (mapping); }
@@ -1816,7 +1816,7 @@ int nommu_shrink_inode_mappings(struct inode *inode, size_t size,i_mmap_lock_read (inode->i_mapping);/* search for VMAs that fall within the dead zone */ - vma_interval_tree_foreach(vma, inode->i_mapping, low, high) { + mapping_interval_tree_foreach(vma, inode->i_mapping, low, high) {/* found one - only interested if it's shared out of the page * cache */ if (vma->vm_flags &VM_SHARED ) {
@@ -1832,7 +1832,7 @@ int nommu_shrink_inode_mappings(struct inode *inode, size_t size,* we don't check for any regions that start beyond the EOF as there * shouldn't be any */ - vma_interval_tree_foreach(vma, inode->i_mapping, 0, ULONG_MAX) { + mapping_interval_tree_foreach(vma, inode->i_mapping, 0, ULONG_MAX) {if (!(vma->vm_flags &VM_SHARED ))continue ;
mm/pagewalk.c +2 -2
@@ -810,8 +810,8 @@ int walk_page_mapping(struct address_space *mapping, pgoff_t first_index,return -EINVAL ;lockdep_assert_held (&mapping->i_mmap_rwsem); -vma_interval_tree_foreach (vma, mapping, first_index, - first_index + nr - 1) { +mapping_interval_tree_foreach (vma, mapping, first_index, + first_index + nr - 1) {/* Clip to the vma */ vba = vma->vm_pgoff; vea = vba +vma_pages (vma);
mm/rmap.c +1 -1
@@ -3051,7 +3051,7 @@ static void __rmap_walk_file(struct folio *folio, struct address_space *mapping,i_mmap_lock_read (mapping); } lookup: - vma_interval_tree_foreach(vma, mapping, pgoff_start, pgoff_end) { + mapping_interval_tree_foreach(vma, mapping, pgoff_start, pgoff_end) {unsigned long address =vma_address (vma, pgoff_start, nr_pages);VM_BUG_ON_VMA (address == -EFAULT , vma);
mm/vma.c +6 -6
@@ -234,7 +234,7 @@ static void __vma_link_file(struct vm_area_struct *vma,mapping_allow_writable (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_insert (vma, mapping); +mapping_interval_tree_insert (vma, mapping);flush_dcache_mmap_unlock (mapping); }
@@ -248,7 +248,7 @@ static void __remove_shared_vm_struct(struct vm_area_struct *vma,mapping_unmap_writable (mapping);flush_dcache_mmap_lock (mapping); -vma_interval_tree_remove (vma, mapping); +mapping_interval_tree_remove (vma, mapping);flush_dcache_mmap_unlock (mapping); }
@@ -319,9 +319,9 @@ static void vma_prepare(struct vma_prepare *vp)if (vp->file) {flush_dcache_mmap_lock (vp->mapping); -vma_interval_tree_remove (vp->vma, vp->mapping); +mapping_interval_tree_remove (vp->vma, vp->mapping);if (vp->adj_next) -vma_interval_tree_remove (vp->adj_next, vp->mapping); +mapping_interval_tree_remove (vp->adj_next, vp->mapping); } }
@@ -339,8 +339,8 @@ static void vma_complete(struct vma_prepare *vp, struct vma_iterator *vmi, {if (vp->file) {if (vp->adj_next) -vma_interval_tree_insert (vp->adj_next, vp->mapping); -vma_interval_tree_insert (vp->vma, vp->mapping); +mapping_interval_tree_insert (vp->adj_next, vp->mapping); +mapping_interval_tree_insert (vp->vma, vp->mapping);flush_dcache_mmap_unlock (vp->mapping); }
tools/testing/vma/include/stubs.h +4 -4
@@ -257,13 +257,13 @@ static inline void vm_acct_memory(long pages) { } -static inline void vma_interval_tree_insert (struct vm_area_struct *vma, -struct address_space *mapping) +static inline void mapping_interval_tree_insert (struct vm_area_struct *vma, +struct address_space *mapping) { } -static inline void vma_interval_tree_remove (struct vm_area_struct *vma, -struct address_space *mapping) +static inline void mapping_interval_tree_remove (struct vm_area_struct *vma, +struct address_space *mapping) { }
Signature · 1 line
2.54.0
Re: [PATCH 08/30] mm/rmap: rename vma_interval_tree_*() to mapping_interval_tree_*()
Quoted 11 lines · Lorenzo Stoakes
The family of
vma_interval_tree_()functions manipulate theaddress_space(which, of course, is generally referred to as 'mapping') reverse mapping, but are named the 'VMA' interval tree.VMAs may be mapped by an
anon_vma, anaddress_space, or both. Therefore calling the mapping interval tree a 'VMA' interval tree is rather confusing.This is also inconsistent with the anon_vma_interval_tree_*() functions which explicitly reference the rmap object to which they pertain.
Quoted 6 lines
Rename the vma_interval_tree_*() functions to mapping_interval_tree_*() to correct this.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
obligatory "naming is hard", this patch helps, thank you.
Pedro Falcato
reviewerRe: [PATCH 08/30] mm/rmap: rename vma_interval_tree_*() to mapping_interval_tree_*()
In reply to Lorenzo StoakesQuoted 11 lines · Lorenzo Stoakes
The family of
vma_interval_tree_()functions manipulate theaddress_space(which, of course, is generally referred to as 'mapping') reverse mapping, but are named the 'VMA' interval tree.VMAs may be mapped by an
anon_vma, anaddress_space, or both. Therefore calling the mapping interval tree a 'VMA' interval tree is rather confusing.This is also inconsistent with the anon_vma_interval_tree_*() functions which explicitly reference the rmap object to which they pertain.
Quoted 6 lines
Rename the vma_interval_tree_*() functions to mapping_interval_tree_*() to correct this.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
I'll have to nitpick this and say that I prefer [1] file_rmap_tree_, or mapping_rmap_tree. Or possibly even better - mapping_ (so mapping_for_each_vma, mapping_insert_vma, etc).
A bit of bikeshedding never hurts ;)
[1] locally I was naming things file_rmap, but I never actually got to churn these names away
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 08/30] mm/rmap: rename vma_interval_tree_*() to mapping_interval_tree_*()
Quoted 15 lines · Pedro Falcato
Quoted 14 lines · Lorenzo Stoakes
The family of
vma_interval_tree_()functions manipulate theaddress_space(which, of course, is generally referred to as 'mapping') reverse mapping, but are named the 'VMA' interval tree.VMAs may be mapped by an
anon_vma, anaddress_space, or both. Therefore calling the mapping interval tree a 'VMA' interval tree is rather confusing.This is also inconsistent with the anon_vma_interval_tree_*() functions which explicitly reference the rmap object to which they pertain.
Rename the vma_interval_tree_*() functions to mapping_interval_tree_*() to correct this.
Quoted 7 lines
Quoted 3 lines
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>I'll have to nitpick this and say that I prefer [1]
file_rmap_tree_, ormapping_rmap_tree. Or possibly even better -mapping_(somapping_for_each_vma,mapping_insert_vma, etc).
Haha of course.
I don't like mapping_ because that is such an overloaded term and it's confusing really.
I'm iffy on mapping_rmap_tree as it feels like that implies 'this is the whole of how the rmap lookup works', and it also makes it less obvious what _kind_ of tree it is, though I suppose you could look up the types, but the crap macro generation makes that more of a pain.
Also, for file 'rmap' you are potentially not actually doing an rmap at all in the classical sense of folio -> rmap object -> related VMAs, but rather are going from file/inode -> <page cache abstraction> -> related VMAs...
But then again a quick spot check suggests it's usually from a folio... I guess only the rmap really needs to find the VMAs.
Also if I renamed it to mapping_rmap_tree I'd have to respin and churn all the anon_vma code but I guess that's not impossible... :)
I guess I'm a bit stubborn about doing much with the anon_vma stuff until I get rid of it.
But I couldn't live with the stupidity of calling it vma_interval_tree_*() anymore here...
OK I'm a bit torn, mapping_rmap_tree*() and anon_rmap_tree*() are the workable contenders from your suggestions.
Let me think about that on respin... :)
Quoted 7 lines
A bit of bikeshedding never hurts ;)
[1] locally I was naming things
file_rmap, but I never actually got to churn these names away
--
Pedro
Cheers, Lorenzo
Lorenzo Stoakes
maintainerPATCH 9/30 mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
In reply to Lorenzo StoakesSimilar to what we did with mapping_interval_tree*(), let's declare anon_vma_interval_tree*() in terms of anon_vma rather than rb_root_cached.
In each case the rb tree referenced is &anon_vma->rb_root, so just pass anon_vma and the functions can figure this out themselves.
Additionally, rename 'node' to 'avc', 'index' to 'pgoff_start', and 'last' to 'pgoff_last' to make clear what is being passed.
Finally express page offsets in terms of pgoff_t to be consistent.
No functional change intended.
--- include/linux/mm.h | 27 +++++++++++--------- mm/interval_tree.c | 41 ++++++++++++++++--------------- mm/ksm.c | 7 ++---- mm/memory-failure.c | 3 +-- mm/rmap.c | 11 ++++----- mm/vma.c | 4 +-- tools/testing/vma/include/stubs.h | 4 +-- 7 files changed, 48 insertions(+), 49 deletions(-)
include/linux/mm.h +15 -12
@@ -4081,22 +4081,25 @@ mapping_interval_tree_iter_next(struct vm_area_struct *vma, vma; vma = mapping_interval_tree_iter_next(vma, pgoff_start, \ pgoff_last)) -void anon_vma_interval_tree_insert (struct anon_vma_chain *node, -struct rb_root_cached *root); -void anon_vma_interval_tree_remove (struct anon_vma_chain *node, -struct rb_root_cached *root); +void anon_vma_interval_tree_insert (struct anon_vma_chain *avc, +struct anon_vma *anon_vma); +void anon_vma_interval_tree_remove (struct anon_vma_chain *avc, +struct anon_vma *anon_vma);struct anon_vma_chain * -anon_vma_interval_tree_iter_first (struct rb_root_cached *root, -unsigned long start,unsigned long last); -struct anon_vma_chain *anon_vma_interval_tree_iter_next ( -struct anon_vma_chain *node,unsigned long start,unsigned long last); +anon_vma_interval_tree_iter_first (struct anon_vma *anon_vma, +pgoff_t pgoff_start,pgoff_t pgoff_last); +struct anon_vma_chain * +anon_vma_interval_tree_iter_next (struct anon_vma_chain *avc, +pgoff_t pgoff_start,pgoff_t pgoff_last);#ifdef CONFIG_DEBUG_VM_RB -void anon_vma_interval_tree_verify (struct anon_vma_chain *node); +void anon_vma_interval_tree_verify (struct anon_vma_chain *avc);#endif -#define anon_vma_interval_tree_foreach (avc, root, start, last) \ - for (avc = anon_vma_interval_tree_iter_first(root, start, last); \ - avc; avc = anon_vma_interval_tree_iter_next(avc, start, last)) +#define anon_vma_interval_tree_foreach (avc, anon_vma, pgoff_start, pgoff_last) \ + for (avc = anon_vma_interval_tree_iter_first(anon_vma, pgoff_start, \ + pgoff_last); \ + avc; avc = anon_vma_interval_tree_iter_next(avc, pgoff_start, \ + pgoff_last))/* mmap.c */ extern int __vm_enough_memory (const struct mm_struct *mm,long pages,int cap_sys_admin);
mm/interval_tree.c +21 -20
75 lines not on this page · raw message
mm/ksm.c +2 -5
@@ -3186,8 +3186,7 @@ void rmap_walk_ksm(struct folio *folio, struct rmap_walk_control *rwc)anon_vma_lock_read (anon_vma); } -anon_vma_interval_tree_foreach (vmac, &anon_vma->rb_root, - 0, ULONG_MAX) { + anon_vma_interval_tree_foreach(vmac, anon_vma, 0, ULONG_MAX) {cond_resched (); vma = vmac->vma;
@@ -3248,9 +3247,7 @@ void collect_procs_ksm(const struct folio *folio, const struct page *page,task_early_kill (tsk, force_early);if (!t)continue ; -anon_vma_interval_tree_foreach (vmac, &av->rb_root,0 , - ULONG_MAX) - { + anon_vma_interval_tree_foreach(vmac, av, 0, ULONG_MAX) { vma = vmac->vma;if (vma->vm_mm == t->mm) { addr = rmap_item->address &PAGE_MASK ;
mm/memory-failure.c +1 -2
@@ -552,8 +552,7 @@ static void collect_procs_anon(const struct folio *folio,if (!t)continue ; -anon_vma_interval_tree_foreach (vmac, &av->rb_root, - pgoff, pgoff) { + anon_vma_interval_tree_foreach(vmac, av, pgoff, pgoff) { vma = vmac->vma;if (vma->vm_mm != t->mm)continue ;
mm/rmap.c +5 -6
@@ -211,7 +211,7 @@ int __anon_vma_prepare(struct vm_area_struct *vma)if (likely (!vma->anon_vma)) { vma->anon_vma = anon_vma;anon_vma_chain_assign (vma, avc, anon_vma); -anon_vma_interval_tree_insert (avc, &anon_vma->rb_root); +anon_vma_interval_tree_insert (avc, anon_vma); anon_vma->num_active_vmas++; allocated =NULL ; avc =NULL ;
@@ -354,7 +354,7 @@ int anon_vma_clone(struct vm_area_struct *dst, struct vm_area_struct *src,list_for_each_entry_reverse (avc, &dst->anon_vma_chain, same_vma) {struct anon_vma *anon_vma = avc->anon_vma; -anon_vma_interval_tree_insert (avc, &anon_vma->rb_root); +anon_vma_interval_tree_insert (avc, anon_vma);if (operation ==VMA_OP_FORK )maybe_reuse_anon_vma (dst, anon_vma); }
@@ -434,7 +434,7 @@ int anon_vma_fork(struct vm_area_struct *vma, struct vm_area_struct *pvma)anon_vma_chain_assign (vma, avc, anon_vma);/* Now let rmap see it. */ anon_vma_lock_write (anon_vma); -anon_vma_interval_tree_insert (avc, &anon_vma->rb_root); +anon_vma_interval_tree_insert (avc, anon_vma); anon_vma->parent->num_children++;anon_vma_unlock_write (anon_vma);
@@ -499,7 +499,7 @@ void unlink_anon_vmas(struct vm_area_struct *vma)list_for_each_entry_safe (avc, next, &vma->anon_vma_chain, same_vma) {struct anon_vma *anon_vma = avc->anon_vma; -anon_vma_interval_tree_remove (avc, &anon_vma->rb_root); +anon_vma_interval_tree_remove (avc, anon_vma);/* * Leave empty anon_vmas on the list - we'll need
@@ -2986,8 +2986,7 @@ static void rmap_walk_anon(struct folio *folio, pgoff_start =folio_pgoff (folio); pgoff_end = pgoff_start +folio_nr_pages (folio) -1 ; -anon_vma_interval_tree_foreach (avc, &anon_vma->rb_root, - pgoff_start, pgoff_end) { + anon_vma_interval_tree_foreach(avc, anon_vma, pgoff_start, pgoff_end) {struct vm_area_struct *vma = avc->vma;unsigned long address =vma_address (vma, pgoff_start,folio_nr_pages (folio));
mm/vma.c +2 -2
@@ -272,7 +272,7 @@ anon_vma_interval_tree_pre_update_vma(struct vm_area_struct *vma)struct anon_vma_chain *avc;list_for_each_entry (avc, &vma->anon_vma_chain, same_vma) -anon_vma_interval_tree_remove (avc, &avc->anon_vma->rb_root); +anon_vma_interval_tree_remove (avc, avc->anon_vma); } static void
@@ -281,7 +281,7 @@ anon_vma_interval_tree_post_update_vma(struct vm_area_struct *vma)struct anon_vma_chain *avc;list_for_each_entry (avc, &vma->anon_vma_chain, same_vma) -anon_vma_interval_tree_insert (avc, &avc->anon_vma->rb_root); +anon_vma_interval_tree_insert (avc, avc->anon_vma); }/*
tools/testing/vma/include/stubs.h +2 -2
@@ -272,12 +272,12 @@ static inline void flush_dcache_mmap_unlock(struct address_space *mapping) }static inline void anon_vma_interval_tree_insert (struct anon_vma_chain *avc, -struct rb_root_cached *rb) +struct anon_vma *anon_vma) { }static inline void anon_vma_interval_tree_remove (struct anon_vma_chain *avc, -struct rb_root_cached *rb) +struct anon_vma *anon_vma) { }
Signature · 1 line
2.54.0
Gregory Price
maintainerRe: [PATCH 09/30] mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
Quoted 9 lines · Lorenzo Stoakes
Similar to what we did with mapping_interval_tree*(), let's declare anon_vma_interval_tree*() in terms of
anon_vmarather thanrb_root_cached.In each case the rb tree referenced is
&anon_vma->rb_root, so just passanon_vmaand the functions can figure this out themselves.Additionally, rename 'node' to 'avc', 'index' to '
pgoff_start', and 'last' to 'pgoff_last' to make clear what is being passed.
would it be possible to split the pure rename changes out from the changed function declarations? It's hard to pick out this as something that needs to be looked at as more than just a %s/x/y/
Quoted 2 lines
(quoted diff) +2
-0+void anon_vma_interval_tree_insert(struct anon_vma_chain *avc, + struct anon_vma *anon_vma)
...
mm/interval_tree.c mapping_interval_tree_iter_next(struct vm_area_struct *vma, · line 105
Quoted 2 lines
mm/interval_tree.c +1
-1-__anon_vma_interval_tree_insert (node, root); +__anon_vma_interval_tree_insert (avc, &anon_vma->rb_root);
an annoying request, sorry
~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 09/30] mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
Quoted 7 lines · Gregory Price
Quoted 6 lines · Lorenzo Stoakes
Similar to what we did with mapping_interval_tree*(), let's declare anon_vma_interval_tree*() in terms of
anon_vmarather thanrb_root_cached.In each case the rb tree referenced is
&anon_vma->rb_root, so just passanon_vmaand the functions can figure this out themselves.
Quoted 7 lines
Quoted 3 lines
Additionally, rename 'node' to 'avc', 'index' to '
pgoff_start', and 'last' to 'pgoff_last' to make clear what is being passed.would it be possible to split the pure rename changes out from the changed function declarations? It's hard to pick out this as something that needs to be looked at as more than just a %s/x/y/
Hmmm do I have to? :P I mean sure I can on a respin potentially, but it is a pretty trivial change? Just mechnically as above.
Quoted 8 lines
Quoted 3 lines
(quoted diff) +2
-0+void anon_vma_interval_tree_insert(struct anon_vma_chain *avc, + struct anon_vma *anon_vma)...
mm/interval_tree.c mapping_interval_tree_iter_next(struct vm_area_struct *vma, · line 105
Quoted 2 lines
mm/interval_tree.c +1
-1-__anon_vma_interval_tree_insert (node, root); +__anon_vma_interval_tree_insert (avc, &anon_vma->rb_root);an annoying request, sorry
:)) well it's ok I've made enough annoying requests of my own on review :)
Quoted 2 lines
~Gregory
Cheers, Lorenzo
Gregory Price
maintainerRe: [PATCH 09/30] mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
Quoted 12 lines · Lorenzo Stoakes
Quoted 11 lines · Gregory Price
Quoted 9 lines · Lorenzo Stoakes
Similar to what we did with mapping_interval_tree*(), let's declare anon_vma_interval_tree*() in terms of
anon_vmarather thanrb_root_cached.In each case the rb tree referenced is
&anon_vma->rb_root, so just passanon_vmaand the functions can figure this out themselves.Additionally, rename 'node' to 'avc', 'index' to '
pgoff_start', and 'last' to 'pgoff_last' to make clear what is being passed.
Quoted 5 lines
Quoted 3 lines
would it be possible to split the pure rename changes out from the changed function declarations? It's hard to pick out this as something that needs to be looked at as more than just a %s/x/y/
Hmmm do I have to? :P
I mean, no :]
Quoted 3 lines
I mean sure I can on a respin potentially, but it is a pretty trivial change? Just mechnically as above.
And yeah certainly not worth a respin. Just learning some of the friction points of reviewing as I spend a little more time doing it every day.
~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 09/30] mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
Quoted 21 lines · Gregory Price
Quoted 12 lines · Lorenzo Stoakes
Quoted 11 lines · Gregory Price
Quoted 9 lines · Lorenzo Stoakes
Similar to what we did with mapping_interval_tree*(), let's declare anon_vma_interval_tree*() in terms of
anon_vmarather thanrb_root_cached.In each case the rb tree referenced is
&anon_vma->rb_root, so just passanon_vmaand the functions can figure this out themselves.Additionally, rename 'node' to 'avc', 'index' to '
pgoff_start', and 'last' to 'pgoff_last' to make clear what is being passed.Quoted 5 lines
Quoted 3 lines
would it be possible to split the pure rename changes out from the changed function declarations? It's hard to pick out this as something that needs to be looked at as more than just a %s/x/y/
Hmmm do I have to? :P
I mean, no :]
Quoted 7 lines
Quoted 3 lines
I mean sure I can on a respin potentially, but it is a pretty trivial change? Just mechnically as above.
And yeah certainly not worth a respin. Just learning some of the friction points of reviewing as I spend a little more time doing it every day.
Oh in the end you will be driven absolutely insane, David and I are already there and soon you will join us! :) <- nervous smiley
But yeah, sorry if that made it harder to track what I'm doing here! Usually I really try to split up to make each patch as easy as possibel to review, but this one I thought 'why not at the same time' :P
Quoted 2 lines
~Gregory
Cheers, Lorenzo
Re: [PATCH 09/30] mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
In reply to Lorenzo StoakesQuoted 6 lines · Lorenzo Stoakes
Similar to what we did with mapping_interval_tree*(), let's declare anon_vma_interval_tree*() in terms of
anon_vmarather thanrb_root_cached.In each case the rb tree referenced is
&anon_vma->rb_root, so just passanon_vmaand the functions can figure this out themselves.
Quoted 8 lines
Additionally, rename 'node' to 'avc', 'index' to '
pgoff_start', and 'last' to 'pgoff_last' to make clear what is being passed.Finally express page offsets in terms of
pgoff_tto be consistent.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Yay!
I have a vaguely similar comment as for the file part (names could be simpler, doesn't really matter whether it's an interval tree or possibly even a tree), but I care less strongly about anon rmap :)
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 09/30] mm/rmap: parameterise anon_vma_interval_tree_*() by anon_vma
Quoted 12 lines · Pedro Falcato
Quoted 11 lines · Lorenzo Stoakes
Similar to what we did with mapping_interval_tree*(), let's declare anon_vma_interval_tree*() in terms of
anon_vmarather thanrb_root_cached.In each case the rb tree referenced is
&anon_vma->rb_root, so just passanon_vmaand the functions can figure this out themselves.Additionally, rename 'node' to 'avc', 'index' to '
pgoff_start', and 'last' to 'pgoff_last' to make clear what is being passed.Finally express page offsets in terms of
pgoff_tto be consistent.
Quoted 7 lines
Quoted 3 lines
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>Yay!
Reviewed-by Pedro Falcato <[email protected]>
Thanks :)
Quoted 4 lines
I have a vaguely similar comment as for the file part (names could be simpler, doesn't really matter whether it's an interval tree or possibly even a tree), but I care less strongly about anon rmap :)
And yeah ack, let me agonise over that a bit :P
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Lorenzo Stoakes
maintainerPATCH 10/30 MAINTAINERS: Move mm/interval_tree.c to rmap section
In reply to Lorenzo StoakesThis file implements code for the interval trees used by the file and anon rmap implementation, so belongs in the rmap section.
--- MAINTAINERS | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
MAINTAINERS +1 -1
@@ -17208,6 +17208,7 @@ R: Jann Horn <[email protected]> L: [email protected] S: Maintained F: include/linux/rmap.h +F: mm/interval_tree.c F: mm/page_vma_mapped.c F: mm/rmap.c F: tools/testing/selftests/mm/rmap.c
@@ -17313,7 +17314,6 @@ T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm F: include/trace/events/mmap.h F: fs/proc/task_mmu.c F: fs/proc/task_nommu.c -F: mm/interval_tree.c F: mm/mincore.c F: mm/mlock.c F: mm/mmap.c
Signature · 1 line
2.54.0
Re: [PATCH 10/30] MAINTAINERS: Move mm/interval_tree.c to rmap section
Quoted 4 lines · Lorenzo Stoakes
This file implements code for the interval trees used by the file and anon rmap implementation, so belongs in the rmap section.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerPATCH 11/30 mm/vma: introduce and use vmg_pages(), vmg_[start, end]_pgoff()
In reply to Lorenzo StoakesIn the VMA logic we often need to determine the number of pages in the specified merge range, as well as the start and end page offsets of that range.
Introduce and use helpers for these purposes.
No functional change intended.
--- mm/vma.c | 11 ++++------- mm/vma.h | 17 +++++++++++++++++ tools/testing/vma/include/dup.h | 10 ++++++++++ 3 files changed, 31 insertions(+), 7 deletions(-)
mm/vma.c +4 -7
@@ -197,11 +197,9 @@ static void init_multi_vma_prep(struct vma_prepare *vp,*/ static bool can_vma_merge_before (struct vma_merge_struct *vmg) { -pgoff_t pglen =PHYS_PFN (vmg->end - vmg->start); -if (is_mergeable_vma (vmg,/* merge_next = */ true) &&is_mergeable_anon_vma (vmg,/* merge_next = */ true)) { -if (vmg->next->vm_pgoff == vmg->pgoff + pglen) +if (vmg_end_pgoff (vmg) ==vma_start_pgoff (vmg->next))return true; }
@@ -221,7 +219,7 @@ static bool can_vma_merge_after(struct vma_merge_struct *vmg) {if (is_mergeable_vma (vmg,/* merge_next = */ false) &&is_mergeable_anon_vma (vmg,/* merge_next = */ false)) { -if (vmg->prev->vm_pgoff +vma_pages (vmg->prev) == vmg->pgoff) +if (vma_end_pgoff (vmg->prev) ==vmg_start_pgoff (vmg))return true; }return false;
@@ -759,7 +757,7 @@ static int commit_merge(struct vma_merge_struct *vmg)*/ vma_adjust_trans_huge (vma, vmg->start, vmg->end, vmg->__adjust_middle_start ? vmg->middle :NULL ); -vma_set_range (vma, vmg->start, vmg->end, vmg->pgoff); +vma_set_range (vma, vmg->start, vmg->end,vmg_start_pgoff (vmg));vmg_adjust_set_range (vmg);vma_iter_store_overwrite (vmg->vmi, vmg->target);
@@ -962,8 +960,7 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(* middle next * shrink/delete extend */ - -pgoff_t pglen =PHYS_PFN (vmg->end - vmg->start); +const pgoff_t pglen =vmg_pages (vmg);VM_WARN_ON_VMG (!merge_right, vmg);/* If we are offset into a VMA, then prev must be middle. */
mm/vma.h +17 -0
@@ -230,6 +230,23 @@ static inline bool vmg_nomem(struct vma_merge_struct *vmg)return vmg->state ==VMA_MERGE_ERROR_NOMEM ; } +static inline pgoff_t vmg_start_pgoff (const struct vma_merge_struct *vmg) +{ +return vmg->pgoff; +} + +static inline pgoff_t vmg_pages (const struct vma_merge_struct *vmg) +{ +const unsigned long size = vmg->end - vmg->start; + +return size >>PAGE_SHIFT ; +} + +static inline pgoff_t vmg_end_pgoff (const struct vma_merge_struct *vmg) +{ +return vmg_start_pgoff (vmg) +vmg_pages (vmg); +} +/* Assumes addr >= vma->vm_start. */ static inline pgoff_t vma_pgoff_offset (struct vm_area_struct *vma,unsigned long addr)
tools/testing/vma/include/dup.h +10 -0
@@ -1301,6 +1301,16 @@ static inline unsigned long vma_pages(const struct vm_area_struct *vma)return (vma->vm_end - vma->vm_start) >>PAGE_SHIFT ; } +static inline pgoff_t vma_start_pgoff (const struct vm_area_struct *vma) +{ +return vma->vm_pgoff; +} + +static inline pgoff_t vma_end_pgoff (const struct vm_area_struct *vma) +{ +return vma_start_pgoff (vma) +vma_pages (vma); +} +static inline int vfs_mmap_prepare (struct file *file,struct vm_area_desc *desc) {return file->f_op->mmap_prepare (desc);
Signature · 1 line
2.54.0
Re: [PATCH 11/30] mm/vma: introduce and use vmg_pages(), vmg_[start, end]_pgoff()
Quoted 9 lines · Lorenzo Stoakes
In the VMA logic we often need to determine the number of pages in the specified merge range, as well as the start and end page offsets of that range.
Introduce and use helpers for these purposes.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 11/30] mm/vma: introduce and use vmg_pages(), vmg_[start, end]_pgoff()
In reply to Lorenzo StoakesQuoted 9 lines · Lorenzo Stoakes
In the VMA logic we often need to determine the number of pages in the specified merge range, as well as the start and end page offsets of that range.
Introduce and use helpers for these purposes.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 12/30 mm/vma: clean up anon_vma_compatible()
In reply to Lorenzo StoakesBreak up the existing very large conditional, add comments and use vma_[start/end]_pgoff() to make clearer what we're doing here.
No functional change intended.
--- mm/vma.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)
mm/vma.c +16 -5
@@ -1967,14 +1967,25 @@ static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct * {vma_flags_t diff =vma_flags_diff_pair (&a->flags, &b->flags); +/* Ignore flags that mprotect() can change. */ vma_flags_clear_mask (&diff,VMA_ACCESS_FLAGS ); +/* Ignore flags that do not impact merging. */ vma_flags_clear_mask (&diff,VMA_IGNORE_MERGE_FLAGS ); -return a->vm_end == b->vm_start && -mpol_equal (vma_policy (a),vma_policy (b)) && - a->vm_file == b->vm_file && -vma_flags_empty (&diff) && - b->vm_pgoff == a->vm_pgoff + ((b->vm_start - a->vm_start) >>PAGE_SHIFT ); +/* Must be adjacent. */ +if (a->vm_end != b->vm_start) +return false; +/* Must have matching policy. */ +if (!mpol_equal (vma_policy (a),vma_policy (b))) +return false; +/* Must both be anon or map the same file (MAP_PRIVATE case). */ +if (a->vm_file != b->vm_file) +return false; +/* Flags must be equivalent modulo mprotect(). */ +if (!vma_flags_empty (&diff)) +return false; +/* Page offset must align. */ +return vma_end_pgoff (a) ==vma_start_pgoff (b); }/*
Signature · 1 line
2.54.0
Re: [PATCH 12/30] mm/vma: clean up anon_vma_compatible()
Quoted 32 lines · Lorenzo Stoakes
Break up the existing very large conditional, add comments and use vma_[start/end]_pgoff() to make clearer what we're doing here.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/vma.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)mm/vma.c +6
-5@@ -1967,14 +1967,25 @@ static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct * {vma_flags_t diff =vma_flags_diff_pair (&a->flags, &b->flags); +/* Ignore flags that mprotect() can change. */ vma_flags_clear_mask (&diff,VMA_ACCESS_FLAGS ); +/* Ignore flags that do not impact merging. */ vma_flags_clear_mask (&diff,VMA_IGNORE_MERGE_FLAGS ); -return a->vm_end == b->vm_start && -mpol_equal (vma_policy (a),vma_policy (b)) && - a->vm_file == b->vm_file && -vma_flags_empty (&diff) && - b->vm_pgoff == a->vm_pgoff + ((b->vm_start - a->vm_start) >>PAGE_SHIFT ); +/* Must be adjacent. */ +if (a->vm_end != b->vm_start) +return false; +/* Must have matching policy. */
mm/vma.c static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct * · line 1988
Quoted 10 lines
mm/vma.c +10
-0+if (!mpol_equal (vma_policy (a),vma_policy (b))) +return false; +/* Must both be anon or map the same file (MAP_PRIVATE case). */ +if (a->vm_file != b->vm_file) +return false; +/* Flags must be equivalent modulo mprotect(). */ +if (!vma_flags_empty (&diff)) +return false; +/* Page offset must align. */ +return vma_end_pgoff (a) ==vma_start_pgoff (b);
Very nice.
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 12/30] mm/vma: clean up anon_vma_compatible()
Quoted 46 lines · Pedro Falcato
Quoted 32 lines · Lorenzo Stoakes
Break up the existing very large conditional, add comments and use vma_[start/end]_pgoff() to make clearer what we're doing here.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/vma.c | 21 ++++++++++++++++----- 1 file changed, 16 insertions(+), 5 deletions(-)mm/vma.c +6
-5@@ -1967,14 +1967,25 @@ static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct * {vma_flags_t diff =vma_flags_diff_pair (&a->flags, &b->flags); +/* Ignore flags that mprotect() can change. */ vma_flags_clear_mask (&diff,VMA_ACCESS_FLAGS ); +/* Ignore flags that do not impact merging. */ vma_flags_clear_mask (&diff,VMA_IGNORE_MERGE_FLAGS ); -return a->vm_end == b->vm_start && -mpol_equal (vma_policy (a),vma_policy (b)) && - a->vm_file == b->vm_file && -vma_flags_empty (&diff) && - b->vm_pgoff == a->vm_pgoff + ((b->vm_start - a->vm_start) >>PAGE_SHIFT ); +/* Must be adjacent. */ +if (a->vm_end != b->vm_start) +return false; +/* Must have matching policy. */ mm/vma.c static int anon_vma_compatible(struct vm_area_struct *a, struct vm_area_struct * · line 1988
Quoted 10 lines
mm/vma.c +10
-0+if (!mpol_equal (vma_policy (a),vma_policy (b))) +return false; +/* Must both be anon or map the same file (MAP_PRIVATE case). */ +if (a->vm_file != b->vm_file) +return false; +/* Flags must be equivalent modulo mprotect(). */ +if (!vma_flags_empty (&diff)) +return false; +/* Page offset must align. */ +return vma_end_pgoff (a) ==vma_start_pgoff (b);Very nice.
Quoted 1 line
Reviewed-by Pedro Falcato <[email protected]>
Thanks :)
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Re: [PATCH 12/30] mm/vma: clean up anon_vma_compatible()
In reply to Lorenzo StoakesQuoted 6 lines · Lorenzo Stoakes
Break up the existing very large conditional, add comments and use vma_[start/end]_pgoff() to make clearer what we're doing here.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 13/30 mm/vma: refactor vmg_adjust_set_range() for clarity
In reply to Lorenzo StoakesAdd comments with ASCII diagrams to describe what we're doing, avoid dubious use of PHYS_PFN(), and use vma_start_pgoff().
The most complicated scenario represented here is vmg->__adjust_next_start
- when this is set, vmg->[start, end] actually indicate the range to be retained, so take special care to describe this accurately.
No functional change intended.
--- mm/vma.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 4 deletions(-)
mm/vma.c +47 -4
@@ -704,11 +704,54 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)pgoff_t pgoff;if (vmg->__adjust_middle_start) { - adjust = vmg->middle; - pgoff = adjust->vm_pgoff +PHYS_PFN (vmg->end - adjust->vm_start); +/* +* vmg->start vmg->end +* | | +* v merge v +* <-------------> +* delta +* <------> +* |------|----------------| +* | prev | middle | +* |------|----------------| +* ^ +* | +* middle->vm_start +*/ +struct vm_area_struct *middle = vmg->middle; +const unsigned long delta = vmg->end - middle->vm_start; + + pgoff =vma_start_pgoff (middle) + (delta >>PAGE_SHIFT ); + adjust = middle; }else if (vmg->__adjust_next_start) { - adjust = vmg->next; - pgoff = adjust->vm_pgoff -PHYS_PFN (adjust->vm_start - vmg->end); +/* +* Originally: +* +* vmg->start vmg->end +* | | +* v merge v +* <------------> +* . . +* merge_existing_range() updates to: +* . . +* vmg->start vmg->end . +* | | . +* v retain v . +* <----------> . +* delta . +* <-----> . +* |----------------|------| +* | middle | next | +* |----------------|------| +* ^ +* | +* next->vm_start +*/ +struct vm_area_struct *next = vmg->next; +const unsigned long delta = next->vm_start - vmg->end; + + pgoff =vma_start_pgoff (next) - (delta >>PAGE_SHIFT ); + adjust = next; }else {return ; }
Signature · 1 line
2.54.0
Re: [PATCH 13/30] mm/vma: refactor vmg_adjust_set_range() for clarity
Quoted 29 lines · Lorenzo Stoakes
Add comments with ASCII diagrams to describe what we're doing, avoid dubious use of
PHYS_PFN(), and usevma_start_pgoff().The most complicated scenario represented here is
vmg->__adjust_next_start
- when this is set, vmg->[start, end] actually indicate the range to be retained, so take special care to describe this accurately.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/vma.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 4 deletions(-)mm/vma.c +5
-2@@ -704,11 +704,54 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)pgoff_t pgoff;if (vmg->__adjust_middle_start) { - adjust = vmg->middle; - pgoff = adjust->vm_pgoff +PHYS_PFN (vmg->end - adjust->vm_start); +/* +* vmg->start vmg->end +* | | +* v merge v +* <------------->
mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 721
Quoted 10 lines
mm/vma.c +10
-0+* delta +* <------> +* |------|----------------| +* | prev | middle | +* |------|----------------| +* ^ +* | +* middle->vm_start +*/ +struct vm_area_struct *middle = vmg->middle;
FWIW this can be simplified to
adjust = middle; const unsigned long delta = vmg->end - adjust->vm_start;
But I guess you're looking for explicitness here?
Quoted 28 lines
mm/vma.c +25
-2+const unsigned long delta = vmg->end - middle->vm_start; + + pgoff =vma_start_pgoff (middle) + (delta >>PAGE_SHIFT ); + adjust = middle; }else if (vmg->__adjust_next_start) { - adjust = vmg->next; - pgoff = adjust->vm_pgoff -PHYS_PFN (adjust->vm_start - vmg->end); +/* +* Originally: +* +* vmg->start vmg->end +* | | +* v merge v +* <------------> +* . . +* merge_existing_range() updates to: +* . . +* vmg->start vmg->end . +* | | . +* v retain v . +* <----------> . +* delta . +* <-----> . +* |----------------|------| +* | middle | next | +* |----------------|------| +* ^ +* |
mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 757
Quoted 10 lines
mm/vma.c +7
-0+* next->vm_start +*/ +struct vm_area_struct *next = vmg->next; +const unsigned long delta = next->vm_start - vmg->end; + + pgoff =vma_start_pgoff (next) - (delta >>PAGE_SHIFT ); + adjust = next; }else {return ; }
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 13/30] mm/vma: refactor vmg_adjust_set_range() for clarity
Quoted 42 lines · Pedro Falcato
Quoted 29 lines · Lorenzo Stoakes
Add comments with ASCII diagrams to describe what we're doing, avoid dubious use of
PHYS_PFN(), and usevma_start_pgoff().The most complicated scenario represented here is
vmg->__adjust_next_start
- when this is set, vmg->[start, end] actually indicate the range to be retained, so take special care to describe this accurately.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/vma.c | 51 +++++++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 47 insertions(+), 4 deletions(-)mm/vma.c +5
-2@@ -704,11 +704,54 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)pgoff_t pgoff;if (vmg->__adjust_middle_start) { - adjust = vmg->middle; - pgoff = adjust->vm_pgoff +PHYS_PFN (vmg->end - adjust->vm_start); +/* +* vmg->start vmg->end +* | | +* v merge v +* <-------------> mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 721
Quoted 10 lines
mm/vma.c +10
-0+* delta +* <------> +* |------|----------------| +* | prev | middle | +* |------|----------------| +* ^ +* | +* middle->vm_start +*/ +struct vm_area_struct *middle = vmg->middle;FWIW this can be simplified to
Quoted 4 lines
adjust = middle; const unsigned long delta = vmg->end - adjust->vm_start;But I guess you're looking for explicitness here?
Yeah I'm intentionally trying to make that explicit as this code is very confusing, so people don't have to think 'oh what was adjust again?'.
Quoted 33 lines
Quoted 33 lines
mm/vma.c +29
-2+const unsigned long delta = vmg->end - middle->vm_start; + + pgoff =vma_start_pgoff (middle) + (delta >>PAGE_SHIFT ); + adjust = middle; }else if (vmg->__adjust_next_start) { - adjust = vmg->next; - pgoff = adjust->vm_pgoff -PHYS_PFN (adjust->vm_start - vmg->end); +/* +* Originally: +* +* vmg->start vmg->end +* | | +* v merge v +* <------------> +* . . +* merge_existing_range() updates to: +* . . +* vmg->start vmg->end . +* | | . +* v retain v . +* <----------> . +* delta . +* <-----> . +* |----------------|------| +* | middle | next | +* |----------------|------| +* ^ +* | +* next->vm_start +*/ +struct vm_area_struct *next = vmg->next; +const unsigned long delta = next->vm_start - vmg->end;
Quoted 8 lines
mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 757
Quoted 6 lines
mm/vma.c +3
-0+ + pgoff =vma_start_pgoff (next) - (delta >>PAGE_SHIFT ); + adjust = next; }else {return ; }Reviewed-by Pedro Falcato <[email protected]>
Thanks!
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Re: [PATCH 13/30] mm/vma: refactor vmg_adjust_set_range() for clarity
In reply to Lorenzo StoakesQuoted 10 lines · Lorenzo Stoakes
Add comments with ASCII diagrams to describe what we're doing, avoid dubious use of
PHYS_PFN(), and usevma_start_pgoff().The most complicated scenario represented here is
vmg->__adjust_next_start
- when this is set, vmg->[start, end] actually indicate the range to be retained, so take special care to describe this accurately.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 4 lines
mm/vma.c +4
-0+/* +* vmg->start vmg->end +* | | +* v merge v
mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 720
Quoted 10 lines
mm/vma.c +10
-0+* <-------------> +* delta +* <------> +* |------|----------------| +* | prev | middle | +* |------|----------------| +* ^ +* | +* middle->vm_start +*/
Even with these diagrams, it's a bit difficult to understand what the actual intent/result of this chunk is (but that may be a limitation of me not spending enough time reading the surrounding code, not a comment of your work here).
~Gregory
Quoted 23 lines
mm/vma.c +23
-0+/* +* Originally: +* +* vmg->start vmg->end +* | | +* v merge v +* <------------> +* . . +* merge_existing_range() updates to: +* . . +* vmg->start vmg->end . +* | | . +* v retain v . +* <----------> . +* delta . +* <-----> . +* |----------------|------| +* | middle | next | +* |----------------|------| +* ^ +* | +* next->vm_start +*/
Lorenzo Stoakes
maintainerRe: [PATCH 13/30] mm/vma: refactor vmg_adjust_set_range() for clarity
Quoted 25 lines · Gregory Price
Quoted 10 lines · Lorenzo Stoakes
Add comments with ASCII diagrams to describe what we're doing, avoid dubious use of
PHYS_PFN(), and usevma_start_pgoff().The most complicated scenario represented here is
vmg->__adjust_next_start
- when this is set, vmg->[start, end] actually indicate the range to be retained, so take special care to describe this accurately.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>Reviewed-by Gregory Price <[email protected]>Quoted 11 lines
mm/vma.c +11
-0+/* +* vmg->start vmg->end +* | | +* v merge v +* <-------------> +* delta +* <------> +* |------|----------------| +* | prev | middle | +* |------|----------------| +* ^
Quoted 8 lines
mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 720
Quoted 3 lines
mm/vma.c +3
-0+* | +* middle->vm_start +*/ Even with these diagrams, it's a bit difficult to understand what the actual intent/result of this chunk is (but that may be a limitation of me not spending enough time reading the surrounding code, not a comment of your work here).
Yeah I know, I do hope to refactor all this at some point to make it simpler, but for the time being tried to make the confusing thing at least _slightly_ less confusing :)
Quoted 2 lines
~Gregory
Cheers, Lorenzo
Quoted 24 lines
Quoted 24 lines
mm/vma.c +23
-0+/* +* Originally: +* +* vmg->start vmg->end +* | | +* v merge v +* <------------> +* . . +* merge_existing_range() updates to: +* . . +* vmg->start vmg->end . +* | | . +* v retain v . +* <----------> . +* delta . +* <-----> . +* |----------------|------| +* | middle | next | +* |----------------|------| +* ^ +* | +* next->vm_start +*/
Lorenzo Stoakes
maintainerPATCH 14/30 mm/vma: minor cleanup of expand_[upwards, downwards]()
In reply to Lorenzo StoakesAdjust the stack expansion functions expand_upwards() and expand_downwards() such that they are expressed in terms of named constant values, and make use of vma_start_pgoff().
This clearly documents that we are referencing the page offset of the start of the VMA.
Additionally this cleans up the overflow check in expand_upwards().
No functional change intended.
--- mm/vma.c | 17 +++++++---------- 1 file changed, 7 insertions(+), 10 deletions(-)
mm/vma.c +7 -10
@@ -3216,13 +3216,12 @@ int expand_upwards(struct vm_area_struct *vma, unsigned long address)/* Somebody else might have raced and expanded it already */ if (address > vma->vm_end) { -unsigned long size, grow; - - size = address - vma->vm_start; - grow = (address - vma->vm_end) >>PAGE_SHIFT ; +const unsigned long size = address - vma->vm_start; +const unsigned long grow = (address - vma->vm_end) >>PAGE_SHIFT ; +const pgoff_t pgoff =vma_start_pgoff (vma); error = -ENOMEM ; -if (vma->vm_pgoff + (size >>PAGE_SHIFT ) >= vma->vm_pgoff) { +if (pgoff + (size >>PAGE_SHIFT ) >= pgoff) { error =acct_stack_growth (vma, size, grow);if (!error) {if (vma_test (vma,VMA_LOCKED_BIT ))
@@ -3295,13 +3294,11 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address)/* Somebody else might have raced and expanded it already */ if (address < vma->vm_start) { -unsigned long size, grow; - - size = vma->vm_end - address; - grow = (vma->vm_start - address) >>PAGE_SHIFT ; +const unsigned long size = vma->vm_end - address; +const unsigned long grow = (vma->vm_start - address) >>PAGE_SHIFT ; error = -ENOMEM ; -if (grow <= vma->vm_pgoff) { +if (grow <=vma_start_pgoff (vma)) { error =acct_stack_growth (vma, size, grow);if (!error) {if (vma_test (vma,VMA_LOCKED_BIT ))
Signature · 1 line
2.54.0
Re: [PATCH 14/30] mm/vma: minor cleanup of expand_[upwards, downwards]()
Quoted 4 lines · Lorenzo Stoakes
Adjust the stack expansion functions
expand_upwards()andexpand_downwards()such that they are expressed in terms of named constant values, and make use ofvma_start_pgoff().
Quoted 8 lines
This clearly documents that we are referencing the page offset of the start of the VMA.
Additionally this cleans up the overflow check in
expand_upwards().No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 14/30] mm/vma: minor cleanup of expand_[upwards, downwards]()
In reply to Lorenzo StoakesQuoted 4 lines · Lorenzo Stoakes
Adjust the stack expansion functions
expand_upwards()andexpand_downwards()such that they are expressed in terms of named constant values, and make use ofvma_start_pgoff().
Quoted 8 lines
This clearly documents that we are referencing the page offset of the start of the VMA.
Additionally this cleans up the overflow check in
expand_upwards().No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 15/30 mm: introduce and use linear_page_delta()
In reply to Lorenzo StoakesIt's often useful to obtain the number of pages a given address lies at within a VMA.
Add linear_page_delta() to determine this and update linear_page_index() to make use of it.
Add comments to describe both linear_page_delta() and linear_page_index().
No functional change intended.
--- include/linux/pagemap.h | 37 +++++++++++++++++++++++++++++++++++-- 1 file changed, 35 insertions(+), 2 deletions(-)
include/linux/pagemap.h +35 -2
@@ -1063,11 +1063,44 @@ static inline pgoff_t folio_pgoff(const struct folio *folio)return folio->index; } +/** +* linear_page_delta() - Determine the relative page offset of @address within +* @vma. +* @vma: The VMA in which @address resides. +* @address: The address whose relative page offset is required. +* +* The result is identical for both file-backed and anonymous mappings and +* simply determines how many pages @address lies from @vma->vm_start. +* +* Returns: The number of pages @address is offset by within @vma. +*/ +static inline pgoff_t linear_page_delta (const struct vm_area_struct *vma, +const unsigned long address) +{ +return (address - vma->vm_start) >>PAGE_SHIFT ; +} + +/** +* linear_page_index() - Determine the absolute page offset of @address within +* @vma. +* @vma: The VMA in which @address resides. +* @address: The address whose absolute page offset is required. +* +* For file-backed mappings, this returns the page offset of @address within the +* file. +* +* For anonymous mappings, this returns the virtual page offset of @address, +* which is the page offset the address possessed at the time the VMA was first +* faulted. +* +* Returns: The absolute page offset of @address within @vma. +*/ static inline pgoff_t linear_page_index (const struct vm_area_struct *vma,const unsigned long address) {pgoff_t pgoff; - pgoff = (address - vma->vm_start) >>PAGE_SHIFT ; + + pgoff =linear_page_delta (vma, address); pgoff += vma->vm_pgoff;return pgoff; }
@@ -1219,7 +1252,7 @@ static inline vm_fault_t folio_lock_or_retry(struct folio *folio,void folio_wait_bit (struct folio *folio,int bit_nr);int folio_wait_bit_killable (struct folio *folio,int bit_nr); -/* +/* * Wait for a folio to be unlocked. * * This must be called with the caller "holding" the folio,
Signature · 1 line
2.54.0
Re: [PATCH 15/30] mm: introduce and use linear_page_delta()
Quoted 11 lines · Lorenzo Stoakes
It's often useful to obtain the number of pages a given address lies at within a VMA.
Add
linear_page_delta()to determine this and updatelinear_page_index()to make use of it.Add comments to describe both
linear_page_delta()andlinear_page_index().No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 15/30] mm: introduce and use linear_page_delta()
In reply to Lorenzo StoakesQuoted 11 lines · Lorenzo Stoakes
It's often useful to obtain the number of pages a given address lies at within a VMA.
Add
linear_page_delta()to determine this and updatelinear_page_index()to make use of it.Add comments to describe both
linear_page_delta()andlinear_page_index().No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 3 lines
[...snip...]
Re: [PATCH 15/30] mm: introduce and use linear_page_delta()
In reply to Lorenzo StoakesQuoted 11 lines · Lorenzo Stoakes
It's often useful to obtain the number of pages a given address lies at within a VMA.
Add
linear_page_delta()to determine this and updatelinear_page_index()to make use of it.Add comments to describe both
linear_page_delta()andlinear_page_index().No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 16/30 mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
In reply to Lorenzo StoakesThere are many instances in which linear_page_index() (as well as linear_page_delta()) is open-coded, which is confusing and inconsistent.
Additionally, vma->vm_pgoff doesn't necessarily make it clear that this is the page offset of the start of the VMA range.
Doing so also aids greppability.
So use vma_start_pgoff() in favour of directly accessing vma->vm_pgoff, and linear_page_index() where we can.
This also lays the ground for future changes which will add an anonymous page offset in order to be able to index MAP_PRIVATE-file backed anon folios in terms of their virtual page offset.
No functional change intended.
--- include/linux/huge_mm.h | 1 + include/linux/hugetlb.h | 3 +-- include/linux/pagemap.h | 2 +- mm/damon/vaddr.c | 5 +++-- mm/debug.c | 2 +- mm/filemap.c | 7 ++++--- mm/huge_memory.c | 2 +- mm/hugetlb.c | 11 ++++------- mm/internal.h | 24 ++++++++++++++---------- mm/khugepaged.c | 3 ++- mm/madvise.c | 6 +++--- mm/mapping_dirty_helpers.c | 2 +- mm/memory.c | 25 +++++++++++++------------ mm/mempolicy.c | 13 +++++++------ mm/mremap.c | 12 ++++-------- mm/msync.c | 4 ++-- mm/nommu.c | 7 ++++--- mm/pagewalk.c | 2 +- mm/shmem.c | 9 +++++---- mm/userfaultfd.c | 4 ++-- mm/util.c | 4 ++-- mm/vma.c | 15 +++++++-------- mm/vma_exec.c | 4 ++-- mm/vma_init.c | 2 +- 24 files changed, 86 insertions(+), 83 deletions(-)
include/linux/huge_mm.h +1 -0
@@ -230,6 +230,7 @@ static inline bool thp_vma_suitable_order(struct vm_area_struct *vma,/* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous (vma)) { +/* vma_start_pgoff() in mm.h so not available. */ if (!IS_ALIGNED ((vma->vm_start >>PAGE_SHIFT ) - vma->vm_pgoff, hpage_size >>PAGE_SHIFT ))return false;
include/linux/hugetlb.h +1 -2
@@ -792,8 +792,7 @@ static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma, {struct hstate *h =hstate_vma (vma); -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }static inline bool order_is_gigantic (unsigned int order)
include/linux/pagemap.h +1 -1
@@ -1101,7 +1101,7 @@ static inline pgoff_t linear_page_index(const struct vm_area_struct *vma,pgoff_t pgoff; pgoff =linear_page_delta (vma, address); - pgoff += vma->vm_pgoff; + pgoff +=vma_start_pgoff (vma);return pgoff; }
mm/damon/vaddr.c +3 -2
@@ -12,6 +12,7 @@#include <linux/mman.h> #include <linux/mmu_notifier.h> #include <linux/page_idle.h> +#include <linux/pagemap.h> #include <linux/pagewalk.h> #include <linux/sched/mm.h>
@@ -627,8 +628,8 @@ static void damos_va_migrate_dests_add(struct folio *folio, } order =folio_order (folio); - ilx = vma->vm_pgoff >> order; - ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order); + ilx =vma_start_pgoff (vma) >> order; + ilx +=linear_page_delta (vma, addr) >> order;for (i =0 ; i < dests->nr_dests; i++) weight_total += dests->weight_arr[i];
mm/debug.c +1 -1
@@ -163,7 +163,7 @@ void dump_vma(const struct vm_area_struct *vma)"flags: %#lx(%pGv)\n" , vma, (void *)vma->vm_start, (void *)vma->vm_end, vma->vm_mm, (unsigned long )pgprot_val (vma->vm_page_prot), - vma->anon_vma, vma->vm_ops, vma->vm_pgoff, + vma->anon_vma, vma->vm_ops,vma_start_pgoff (vma), vma->vm_file, vma->vm_private_data, #ifdef CONFIG_PER_VMA_LOCK refcount_read(&vma->vm_refcnt),
mm/filemap.c +4 -3
@@ -3402,8 +3402,8 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)* of memory. */ struct vm_area_struct *vma = vmf->vma; -unsigned long start = vma->vm_pgoff; -unsigned long end = start +vma_pages (vma); +const unsigned long start =vma_start_pgoff (vma); +const unsigned long end =vma_end_pgoff (vma);unsigned long ra_end; ra->order =exec_folio_order ();
@@ -3921,7 +3921,8 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, goto out; } - addr = vma->vm_start + ((start_pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + + ((start_pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT ); vmf->pte =pte_offset_map_lock (vma->vm_mm, vmf->pmd, addr, &vmf->ptl);if (!vmf->pte) {folio_unlock (folio);
mm/huge_memory.c +1 -1
@@ -180,7 +180,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,*/ if (!in_pf &&shmem_file (vma->vm_file))return orders &shmem_allowable_huge_orders (file_inode (vma->vm_file), - vma, vma->vm_pgoff,0 , + vma,vma_start_pgoff (vma),0 , forced_collapse);if (!vma_is_anonymous (vma)) {
mm/hugetlb.c +4 -7
@@ -1011,8 +1011,7 @@ static long region_count(struct resv_map *resv, long f, long t)static pgoff_t vma_hugecache_offset (struct hstate *h,struct vm_area_struct *vma,unsigned long address) { -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }/*
@@ -5372,8 +5371,7 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,* from page cache lookup which is in HPAGE_SIZE units. */ address = address &huge_page_mask (h); - pgoff = ((address - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff; + pgoff =linear_page_index (vma, address); mapping = vma->vm_file->f_mapping;/*
@@ -6771,7 +6769,7 @@ static unsigned long page_table_shareable(struct vm_area_struct *svma,struct vm_area_struct *vma, unsigned long addr, pgoff_t idx) { - unsigned long saddr = ((idx - svma->vm_pgoff) << PAGE_SHIFT) + + unsigned long saddr = ((idx - vma_start_pgoff(svma)) << PAGE_SHIFT) + svma->vm_start;unsigned long sbase = saddr &PUD_MASK ;unsigned long s_end = sbase +PUD_SIZE ;
@@ -6856,8 +6854,7 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, pud_t *pud) { struct address_space *mapping = vma->vm_file->f_mapping; -pgoff_t idx = ((addr - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff; +const pgoff_t idx =linear_page_index (vma, addr);struct vm_area_struct *svma;unsigned long saddr;pte_t *spte =NULL ;
mm/internal.h +14 -10
@@ -1143,26 +1143,28 @@ static inline bool folio_within_range(struct folio *folio, struct vm_area_struct *vma, unsigned long start, unsigned long end) { - pgoff_t pgoff, addr; -unsigned long vma_pglen =vma_pages (vma); + const unsigned long vma_pglen = vma_pages(vma); +pgoff_t pgoff_folio, pgoff_vma_start; +unsigned long addr;VM_WARN_ON_FOLIO (folio_test_ksm (folio), folio);if (start > end)return false; + pgoff_folio =folio_pgoff (folio); + pgoff_vma_start =vma_start_pgoff (vma); +if (start < vma->vm_start) start = vma->vm_start;if (end > vma->vm_end) end = vma->vm_end; - pgoff =folio_pgoff (folio); -/* if folio start address is not in vma range */ -if (!in_range (pgoff, vma->vm_pgoff, vma_pglen)) +if (!in_range (pgoff_folio, pgoff_vma_start, vma_pglen))return false; - addr = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + ((pgoff_folio - pgoff_vma_start) <<PAGE_SHIFT );return !(addr < start || end - addr <folio_size (folio)); }
@@ -1234,15 +1236,16 @@ extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);static inline unsigned long vma_address (const struct vm_area_struct *vma,pgoff_t pgoff,unsigned long nr_pages) { +const pgoff_t pgoff_start =vma_start_pgoff (vma);unsigned long address; -if (pgoff >= vma->vm_pgoff) { +if (pgoff >= pgoff_start) { address = vma->vm_start + - ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + ((pgoff - pgoff_start) <<PAGE_SHIFT );/* Check for address beyond vma (or wrapped through 0?) */ if (address < vma->vm_start || address >= vma->vm_end) address = -EFAULT ; - }else if (pgoff + nr_pages -1 >= vma->vm_pgoff) { + }else if (pgoff + nr_pages -1 >= pgoff_start) {/* Test above avoids possibility of wrap to 0 on 32-bit */ address = vma->vm_start; }else {
@@ -1266,7 +1269,8 @@ static inline unsigned long vma_address_end(struct page_vma_mapped_walk *pvmw)return pvmw->address +PAGE_SIZE ; pgoff = pvmw->pgoff + pvmw->nr_pages; - address = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + address = vma->vm_start + + ((pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT );/* Check for address beyond vma (or wrapped through 0?) */ if (address < vma->vm_start || address > vma->vm_end) address = vma->vm_end;
mm/khugepaged.c +2 -1
@@ -2145,7 +2145,8 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)spinlock_t *ptl;bool success = false; - addr = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + + ((pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT );if (addr & ~HPAGE_PMD_MASK || vma->vm_end < addr +HPAGE_PMD_SIZE )continue ;
mm/madvise.c +3 -3
@@ -253,7 +253,7 @@ static void shmem_swapin_range(struct vm_area_struct *vma,continue ; addr = vma->vm_start + - ((xas.xa_index - vma->vm_pgoff) <<PAGE_SHIFT ); + ((xas.xa_index -vma_start_pgoff (vma)) <<PAGE_SHIFT );xas_pause (&xas);rcu_read_unlock ();
@@ -318,7 +318,7 @@ static long madvise_willneed(struct madvise_behavior *madv_behavior)mark_mmap_lock_dropped (madv_behavior);get_file (file); offset = (loff_t)(start - vma->vm_start) - + ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT );mmap_read_unlock (mm);vfs_fadvise (file, offset, end - start,POSIX_FADV_WILLNEED );fput (file);
@@ -1023,7 +1023,7 @@ static long madvise_remove(struct madvise_behavior *madv_behavior)return -EACCES ; offset = (loff_t)(start - vma->vm_start) - + ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT );/* * Filesystem's fallocate may need to take i_rwsem. We need to
mm/mapping_dirty_helpers.c +1 -1
@@ -95,7 +95,7 @@ static int clean_record_pte(pte_t *pte, unsigned long addr,if (pte_dirty (ptent)) {pgoff_t pgoff = ((addr - walk->vma->vm_start) >>PAGE_SHIFT ) + - walk->vma->vm_pgoff - cwalk->bitmap_pgoff; +vma_start_pgoff (walk->vma) - cwalk->bitmap_pgoff;pte_t old_pte =ptep_modify_prot_start (walk->vma, addr, pte); ptent =pte_mkclean (old_pte);
mm/memory.c +13 -12
@@ -725,10 +725,10 @@ static inline struct page *__vm_normal_page(struct vm_area_struct *vma,if (!pfn_valid (pfn))return NULL ; }else { -unsigned long off = (addr - vma->vm_start) >>PAGE_SHIFT ; +const pgoff_t index =linear_page_index (vma, addr);/* Only CoW'ed anon folios are "normal". */ -if (pfn == vma->vm_pgoff + off) +if (pfn == index)return NULL ;if (!is_cow_mapping (vma->vm_flags))return NULL ;
@@ -2643,7 +2643,7 @@ static int __vm_map_pages(struct vm_area_struct *vma, struct page **pages,int vm_map_pages (struct vm_area_struct *vma,struct page **pages,unsigned long num) { -return __vm_map_pages (vma, pages, num, vma->vm_pgoff); +return __vm_map_pages (vma, pages, num,vma_start_pgoff (vma)); }EXPORT_SYMBOL (vm_map_pages);
@@ -3298,7 +3298,8 @@ int vm_iomap_memory(struct vm_area_struct *vma, phys_addr_t start, unsigned longunsigned long pfn;int err; - err =__simple_ioremap_prep (vm_len, vma->vm_pgoff, start, len, &pfn); + err =__simple_ioremap_prep (vm_len,vma_start_pgoff (vma), start, len, + &pfn);if (err)return err;
@@ -4342,15 +4343,15 @@ static inline void unmap_mapping_range_tree(struct address_space *mapping, struct zap_details *details) {struct vm_area_struct *vma; -unsigned long start, size;struct mmu_gather tlb; mapping_interval_tree_foreach(vma, mapping, first_index, last_index) { -const pgoff_t start_idx =max (first_index, vma->vm_pgoff); +const pgoff_t start_idx =max (first_index,vma_start_pgoff (vma));const pgoff_t end_idx =min (last_index,vma_last_pgoff (vma)) +1 ; - - start = vma->vm_start + ((start_idx - vma->vm_pgoff) <<PAGE_SHIFT ); - size = (end_idx - start_idx) <<PAGE_SHIFT ; +const pgoff_t offset = start_idx -vma_start_pgoff (vma); +const unsigned long offset_bytes = offset <<PAGE_SHIFT ; +const unsigned long start = vma->vm_start + offset_bytes; +const unsigned long size = (end_idx - start_idx) <<PAGE_SHIFT ;tlb_gather_mmu (&tlb, vma->vm_mm);zap_vma_range_batched (&tlb, vma, start, size, details);
@@ -5684,7 +5685,7 @@ vm_fault_t finish_fault(struct vm_fault *vmf) }else if (nr_pages >1 ) {pgoff_t idx =folio_page_idx (folio, page);/* The page offset of vmf->address within the VMA. */ -pgoff_t vma_off = vmf->pgoff - vmf->vma->vm_pgoff; +pgoff_t vma_off = vmf->pgoff -vma_start_pgoff (vmf->vma);/* The index of the entry in the pagetable for fault page. */ pgoff_t pte_off =pte_index (vmf->address);
@@ -5796,7 +5797,7 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)pgoff_t nr_pages =READ_ONCE (fault_around_pages);pgoff_t pte_off =pte_index (vmf->address);/* The page offset of vmf->address within the VMA. */ -pgoff_t vma_off = vmf->pgoff - vmf->vma->vm_pgoff; +pgoff_t vma_off = vmf->pgoff -vma_start_pgoff (vmf->vma);pgoff_t from_pte, to_pte;vm_fault_t ret;
@@ -7274,7 +7275,7 @@ void print_vma_addr(char *prefix, unsigned long ip)if (vma && vma->vm_file) {struct file *f = vma->vm_file; ip -= vma->vm_start; - ip += vma->vm_pgoff <<PAGE_SHIFT ; + ip +=vma_start_pgoff (vma) <<PAGE_SHIFT ;printk ("%s%pD[%lx,%lx+%lx]" , prefix, f, ip, vma->vm_start, vma->vm_end - vma->vm_start);
mm/mempolicy.c +7 -6
@@ -2048,8 +2048,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struct *vma, pol =get_task_policy (current);if (pol->mode ==MPOL_INTERLEAVE || pol->mode ==MPOL_WEIGHTED_INTERLEAVE ) { - *ilx += vma->vm_pgoff >> order; - *ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order); + *ilx +=vma_start_pgoff (vma) >> order; + *ilx +=linear_page_delta (vma, addr) >> order; }return pol; }
@@ -3250,16 +3250,17 @@ EXPORT_SYMBOL_FOR_MODULES(mpol_shared_policy_init, "kvm");int mpol_set_shared_policy (struct shared_policy *sp,struct vm_area_struct *vma,struct mempolicy *pol) { -int err; +const pgoff_t pgoff =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);struct sp_node *new =NULL ; -unsigned long sz =vma_pages (vma); +int err;if (pol) { - new =sp_alloc (vma->vm_pgoff, vma->vm_pgoff + sz, pol); + new =sp_alloc (pgoff, pgoff_end, pol);if (!new)return -ENOMEM ; } - err =shared_policy_replace (sp, vma->vm_pgoff, vma->vm_pgoff + sz, new); + err =shared_policy_replace (sp, pgoff, pgoff_end, new);if (err && new)sp_free (new);return err;
mm/mremap.c +4 -8
@@ -948,8 +948,7 @@ static unsigned long vrm_set_new_addr(struct vma_remap_struct *vrm)struct vm_area_struct *vma = vrm->vma;unsigned long map_flags =0 ;/* Page Offset _into_ the VMA. */ -pgoff_t internal_pgoff = (vrm->addr - vma->vm_start) >>PAGE_SHIFT ; -pgoff_t pgoff = vma->vm_pgoff + internal_pgoff; +const pgoff_t pgoff =linear_page_index (vma, vrm->addr);unsigned long new_addr =vrm_implies_new_addr (vrm) ? vrm->new_addr :0 ;unsigned long res;
@@ -1255,12 +1254,10 @@ static void unmap_source_vma(struct vma_remap_struct *vrm)static int copy_vma_and_data (struct vma_remap_struct *vrm,struct vm_area_struct **new_vma_ptr) { -unsigned long internal_offset = vrm->addr - vrm->vma->vm_start; -unsigned long internal_pgoff = internal_offset >>PAGE_SHIFT ; -unsigned long new_pgoff = vrm->vma->vm_pgoff + internal_pgoff; -unsigned long moved_len; +const unsigned long new_pgoff =linear_page_index (vrm->vma, vrm->addr);struct vm_area_struct *vma = vrm->vma;struct vm_area_struct *new_vma; +unsigned long moved_len;int err =0 ;PAGETABLE_MOVE (pmc,NULL ,NULL , vrm->addr, vrm->new_addr, vrm->old_len);
@@ -1802,8 +1799,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm) vrm->populate_expand = true;/* Need to be careful about a growing mapping */ - pgoff = (addr - vma->vm_start) >>PAGE_SHIFT ; - pgoff += vma->vm_pgoff; + pgoff =linear_page_index (vma, addr);if (pgoff + (new_len >>PAGE_SHIFT ) < pgoff)return -EINVAL ;
mm/msync.c +2 -2
@@ -12,6 +12,7 @@#include <linux/mm.h> #include <linux/mman.h> #include <linux/file.h> +#include <linux/pagemap.h> #include <linux/syscalls.h> #include <linux/sched.h>
@@ -85,8 +86,7 @@ SYSCALL_DEFINE3(msync, unsigned long, start, size_t, len, int, flags) goto out_unlock; } file = vma->vm_file; - fstart = (start - vma->vm_start) + - ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + fstart = (loff_t )linear_page_index (vma, start) <<PAGE_SHIFT ; fend = fstart + (min (end, vma->vm_end) - start) -1 ; start = vma->vm_end; if ((flags & MS_SYNC) && file &&
mm/nommu.c +4 -3
@@ -975,7 +975,7 @@ static int do_mmap_private(struct vm_area_struct *vma,/* read the contents of a file into the copy */ loff_t fpos; - fpos = vma->vm_pgoff; + fpos =vma_start_pgoff (vma); fpos <<=PAGE_SHIFT ; ret =kernel_read (vma->vm_file, base, len, &fpos);
@@ -1355,7 +1355,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,delete_nommu_region (vma->vm_region);if (new_below) { vma->vm_region->vm_start = vma->vm_start = addr; - vma->vm_region->vm_pgoff = vma->vm_pgoff += npages; + vma->vm_pgoff += npages; + vma->vm_region->vm_pgoff =vma_start_pgoff (vma); }else { vma->vm_region->vm_end = vma->vm_end = addr; vma->vm_region->vm_top = addr;
@@ -1603,7 +1604,7 @@ int vm_iomap_memory(struct vm_area_struct *vma, phys_addr_t start, unsigned longunsigned long pfn = start >>PAGE_SHIFT ;unsigned long vm_len = vma->vm_end - vma->vm_start; - pfn += vma->vm_pgoff; + pfn +=vma_start_pgoff (vma);return io_remap_pfn_range (vma, vma->vm_start, pfn, vm_len, vma->vm_page_prot); }EXPORT_SYMBOL (vm_iomap_memory);
mm/pagewalk.c +1 -1
@@ -813,7 +813,7 @@ int walk_page_mapping(struct address_space *mapping, pgoff_t first_index,mapping_interval_tree_foreach (vma, mapping, first_index, first_index + nr - 1) {/* Clip to the vma */ - vba = vma->vm_pgoff; + vba =vma_start_pgoff (vma); vea = vba +vma_pages (vma); cba = first_index; cba =max (cba, vba);
mm/shmem.c +5 -4
@@ -1032,6 +1032,8 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma)struct inode *inode =file_inode (vma->vm_file);struct shmem_inode_info *info =SHMEM_I (inode);struct address_space *mapping = inode->i_mapping; +const pgoff_t pgoff =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);unsigned long swapped;/* Be careful as we don't hold info->lock */
@@ -1045,12 +1047,11 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma)if (!swapped)return 0 ; -if (!vma->vm_pgoff && vma->vm_end - vma->vm_start >= inode->i_size) +if (!pgoff && vma->vm_end - vma->vm_start >= inode->i_size)return swapped <<PAGE_SHIFT ;/* Here comes the more involved part */ -return shmem_partial_swap_usage (mapping, vma->vm_pgoff, - vma->vm_pgoff +vma_pages (vma)); +return shmem_partial_swap_usage (mapping, pgoff, pgoff_end); }/*
@@ -2839,7 +2840,7 @@ static struct mempolicy *shmem_get_policy(struct vm_area_struct *vma,* by page order, as in shmem_get_pgoff_policy() and get_vma_policy()). */ *ilx = inode->i_ino; - index = ((addr - vma->vm_start) >>PAGE_SHIFT ) + vma->vm_pgoff; + index =linear_page_index (vma, addr);return mpol_shared_policy_lookup (&SHMEM_I (inode)->policy, index); }
mm/userfaultfd.c +2 -2
@@ -481,7 +481,7 @@ static void mfill_retry_state_save(struct mfill_retry_state *s, { s->flags =vma_flags_and_mask (&vma->flags,MFILL_RETRY_STATE_VMA_FLAGS ); s->ops =vma_uffd_ops (vma); - s->pgoff = vma->vm_pgoff; + s->pgoff =vma_start_pgoff (vma);if (vma->vm_file) s->file =get_file (vma->vm_file);
@@ -507,7 +507,7 @@ static bool mfill_retry_state_changed(struct mfill_retry_state *state,/* VMA was file backed, but file, inode or offset has changed */ if (!vma->vm_file || vma->vm_file->f_inode != state->file->f_inode || - state->file != vma->vm_file || vma->vm_pgoff != state->pgoff) + state->file != vma->vm_file ||vma_start_pgoff (vma) != state->pgoff)return true;return false;
mm/util.c +2 -2
@@ -1188,7 +1188,7 @@ void compat_set_desc_from_vma(struct vm_area_desc *desc, desc->start = vma->vm_start; desc->end = vma->vm_end; - desc->pgoff = vma->vm_pgoff; + desc->pgoff =vma_start_pgoff (vma); desc->vm_file = vma->vm_file; desc->vma_flags = vma->flags; desc->page_prot = vma->vm_page_prot;
@@ -1379,7 +1379,7 @@ static int call_vma_mapped(struct vm_area_struct *vma)if (!vm_ops || !vm_ops->mapped)return 0 ; - err = vm_ops->mapped (vma->vm_start, vma->vm_end, vma->vm_pgoff, + err = vm_ops->mapped (vma->vm_start, vma->vm_end,vma_start_pgoff (vma), vma->vm_file, &vm_private_data);if (err)return err;
mm/vma.c +7 -8
@@ -967,10 +967,9 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(* prev middle next * extend delete delete */ - vmg->start = prev->vm_start; vmg->end = next->vm_end; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);/* * We already ensured anon_vma compatibility above, so now it's
@@ -987,9 +986,8 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(* prev middle * extend shrink/delete */ - vmg->start = prev->vm_start; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);if (!vmg->__remove_middle) vmg->__adjust_middle_start = true;
@@ -1011,13 +1009,13 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(if (vmg->__remove_middle) { vmg->end = next->vm_end; - vmg->pgoff = next->vm_pgoff - pglen; + vmg->pgoff =vma_start_pgoff (next) - pglen; }else {/* We shrink middle and expand next. */ vmg->__adjust_next_start = true; vmg->start = middle->vm_start; vmg->end = start; - vmg->pgoff = middle->vm_pgoff; + vmg->pgoff =vma_start_pgoff (middle); } err =dup_anon_vma (next, middle, &anon_dup);
@@ -1126,7 +1124,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)if (can_merge_left) { vmg->start = prev->vm_start; vmg->target = prev; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);/* * If this merge would result in removal of the next VMA but we
@@ -1957,7 +1955,8 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,VM_BUG_ON_VMA (faulted_in_anon_vma, new_vma); *vmap = vma = new_vma; } - *need_rmap_locks = (new_vma->vm_pgoff <= vma->vm_pgoff); + *need_rmap_locks = + (vma_start_pgoff (new_vma) <=vma_start_pgoff (vma)); }else { new_vma =vm_area_dup (vma);if (!new_vma)
mm/vma_exec.c +2 -2
@@ -37,7 +37,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)unsigned long new_end = old_end - shift;VMA_ITERATOR (vmi, mm, new_start);VMG_STATE (vmg, mm, &vmi, new_start, old_end,EMPTY_VMA_FLAGS , - vma->vm_pgoff); +vma_start_pgoff (vma));struct vm_area_struct *next;struct mmu_gather tlb;PAGETABLE_MOVE (pmc, vma, vma, old_start, new_start, length);
@@ -89,7 +89,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)vma_prev (&vmi);/* Shrink the vma to just the new range */ -return vma_shrink (&vmi, vma, new_start, new_end, vma->vm_pgoff); +return vma_shrink (&vmi, vma, new_start, new_end,vma_start_pgoff (vma)); }/*
mm/vma_init.c +1 -1
@@ -46,7 +46,7 @@ static void vm_area_init_from(const struct vm_area_struct *src, dest->vm_start = src->vm_start; dest->vm_end = src->vm_end; dest->anon_vma = src->anon_vma; - dest->vm_pgoff = src->vm_pgoff; + dest->vm_pgoff =vma_start_pgoff (src); dest->vm_file = src->vm_file; dest->vm_private_data = src->vm_private_data;vm_flags_init (dest, src->vm_flags);
Signature · 1 line
2.54.0
Re: [PATCH 16/30] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
Quoted 15 lines · On Mon, 29 Jun 2026 13:23:27 +0100 Lorenzo Stoakes <[email protected]> wrote:
There are many instances in which
linear_page_index()(as well aslinear_page_delta()) is open-coded, which is confusing and inconsistent.Additionally,
vma->vm_pgoffdoesn't necessarily make it clear that this is the page offset of the start of the VMA range.Doing so also aids greppability.
So use
vma_start_pgoff()in favour of directly accessingvma->vm_pgoff, andlinear_page_index()where we can.This also lays the ground for future changes which will add an anonymous page offset in order to be able to index MAP_PRIVATE-file backed anon folios in terms of their virtual page offset.
Quoted 8 lines
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- include/linux/huge_mm.h | 1 + include/linux/hugetlb.h | 3 +-- include/linux/pagemap.h | 2 +- mm/damon/vaddr.c | 5 +++--
I quickly looked only the DAMON part. Looks nice and cleaner, thank you!
Thanks,
SJ
[...]
Re: [PATCH 16/30] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
In reply to Lorenzo Stoakessmall nit: perhaps the subject should simply have mm: as it hits mm in general
Quoted 50 lines · Lorenzo Stoakes
There are many instances in which
linear_page_index()(as well aslinear_page_delta()) is open-coded, which is confusing and inconsistent.Additionally,
vma->vm_pgoffdoesn't necessarily make it clear that this is the page offset of the start of the VMA range.Doing so also aids greppability.
So use
vma_start_pgoff()in favour of directly accessingvma->vm_pgoff, andlinear_page_index()where we can.This also lays the ground for future changes which will add an anonymous page offset in order to be able to index MAP_PRIVATE-file backed anon folios in terms of their virtual page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- include/linux/huge_mm.h | 1 + include/linux/hugetlb.h | 3 +-- include/linux/pagemap.h | 2 +- mm/damon/vaddr.c | 5 +++-- mm/debug.c | 2 +- mm/filemap.c | 7 ++++--- mm/huge_memory.c | 2 +- mm/hugetlb.c | 11 ++++------- mm/internal.h | 24 ++++++++++++++---------- mm/khugepaged.c | 3 ++- mm/madvise.c | 6 +++--- mm/mapping_dirty_helpers.c | 2 +- mm/memory.c | 25 +++++++++++++------------ mm/mempolicy.c | 13 +++++++------ mm/mremap.c | 12 ++++-------- mm/msync.c | 4 ++-- mm/nommu.c | 7 ++++--- mm/pagewalk.c | 2 +- mm/shmem.c | 9 +++++---- mm/userfaultfd.c | 4 ++-- mm/util.c | 4 ++-- mm/vma.c | 15 +++++++-------- mm/vma_exec.c | 4 ++-- mm/vma_init.c | 2 +- 24 files changed, 86 insertions(+), 83 deletions(-)include/linux/huge_mm.h +0
-0
include/linux/huge_mm.h static inline bool thp_vma_suitable_order(struct vm_area_struct *vma, · line 233
Quoted 4 lines
include/linux/huge_mm.h +1
-0@@ -230,6 +230,7 @@ static inline bool thp_vma_suitable_order(struct vm_area_struct *vma,/* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous (vma)) { +/* vma_start_pgoff() in mm.h so not available. */
Yay for gigaheaders...
Quoted 8 lines
include/linux/huge_mm.h +0
-0if (!IS_ALIGNED ((vma->vm_start >>PAGE_SHIFT ) - vma->vm_pgoff, hpage_size >>PAGE_SHIFT ))return false;include/linux/hugetlb.h +0
-0
include/linux/hugetlb.h static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma, · line 797
Quoted 8 lines
include/linux/hugetlb.h +1
-2@@ -792,8 +792,7 @@ static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma, {struct hstate *h =hstate_vma (vma); -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }
Anyway, nothing jumped out at me.
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 16/30] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
Quoted 54 lines · Pedro Falcato
small nit: perhaps the subject should simply have mm: as it hits mm in general
Quoted 51 lines · Lorenzo Stoakes
There are many instances in which
linear_page_index()(as well aslinear_page_delta()) is open-coded, which is confusing and inconsistent.Additionally,
vma->vm_pgoffdoesn't necessarily make it clear that this is the page offset of the start of the VMA range.Doing so also aids greppability.
So use
vma_start_pgoff()in favour of directly accessingvma->vm_pgoff, andlinear_page_index()where we can.This also lays the ground for future changes which will add an anonymous page offset in order to be able to index MAP_PRIVATE-file backed anon folios in terms of their virtual page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- include/linux/huge_mm.h | 1 + include/linux/hugetlb.h | 3 +-- include/linux/pagemap.h | 2 +- mm/damon/vaddr.c | 5 +++-- mm/debug.c | 2 +- mm/filemap.c | 7 ++++--- mm/huge_memory.c | 2 +- mm/hugetlb.c | 11 ++++------- mm/internal.h | 24 ++++++++++++++---------- mm/khugepaged.c | 3 ++- mm/madvise.c | 6 +++--- mm/mapping_dirty_helpers.c | 2 +- mm/memory.c | 25 +++++++++++++------------ mm/mempolicy.c | 13 +++++++------ mm/mremap.c | 12 ++++-------- mm/msync.c | 4 ++-- mm/nommu.c | 7 ++++--- mm/pagewalk.c | 2 +- mm/shmem.c | 9 +++++---- mm/userfaultfd.c | 4 ++-- mm/util.c | 4 ++-- mm/vma.c | 15 +++++++-------- mm/vma_exec.c | 4 ++-- mm/vma_init.c | 2 +- 24 files changed, 86 insertions(+), 83 deletions(-)include/linux/huge_mm.h +0
-0@@ -230,6 +230,7 @@ static inline bool thp_vma_suitable_order(struct vm_area_struct *vma,
Quoted 5 lines
include/linux/huge_mm.h static inline bool thp_vma_suitable_order(struct vm_area_struct *vma, · line 233
Quoted 3 lines
include/linux/huge_mm.h +1
-0/* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous (vma)) { +/* vma_start_pgoff() in mm.h so not available. */ Yay for gigaheaders...
Yup C headers are a constant pain :)
Quoted 20 lines
Quoted 9 lines
include/linux/huge_mm.h +0
-0if (!IS_ALIGNED ((vma->vm_start >>PAGE_SHIFT ) - vma->vm_pgoff, hpage_size >>PAGE_SHIFT ))return false;include/linux/hugetlb.h +0
-0include/linux/hugetlb.h static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma, · line 797
Quoted 8 lines
include/linux/hugetlb.h +1
-2@@ -792,8 +792,7 @@ static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma, {struct hstate *h =hstate_vma (vma); -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }Anyway, nothing jumped out at me.
Quoted 1 line
Reviewed-by Pedro Falcato <[email protected]>
Thanks!
Quoted 4 lines
--
Pedro
Cheers, Lorenzo
Re: [PATCH 16/30] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
In reply to Lorenzo StoakesQuoted 11 lines · Lorenzo Stoakes
There are many instances in which
linear_page_index()(as well aslinear_page_delta()) is open-coded, which is confusing and inconsistent.Additionally,
vma->vm_pgoffdoesn't necessarily make it clear that this is the page offset of the start of the VMA range.Doing so also aids greppability.
So use
vma_start_pgoff()in favour of directly accessingvma->vm_pgoff, andlinear_page_index()where we can.
Quoted 7 lines
This also lays the ground for future changes which will add an anonymous page offset in order to be able to index MAP_PRIVATE-file backed anon folios in terms of their virtual page offset.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
such a nice readability improvement, thank you.
This made me wonder though whether we can make it painful for people to open-code this in the future, otherwise I fear another one of these patches creeping in 5-10 years down the line.
Not sure it's feasible though
Quoted 718 lines
--- include/linux/huge_mm.h | 1 + include/linux/hugetlb.h | 3 +-- include/linux/pagemap.h | 2 +- mm/damon/vaddr.c | 5 +++-- mm/debug.c | 2 +- mm/filemap.c | 7 ++++--- mm/huge_memory.c | 2 +- mm/hugetlb.c | 11 ++++------- mm/internal.h | 24 ++++++++++++++---------- mm/khugepaged.c | 3 ++- mm/madvise.c | 6 +++--- mm/mapping_dirty_helpers.c | 2 +- mm/memory.c | 25 +++++++++++++------------ mm/mempolicy.c | 13 +++++++------ mm/mremap.c | 12 ++++-------- mm/msync.c | 4 ++-- mm/nommu.c | 7 ++++--- mm/pagewalk.c | 2 +- mm/shmem.c | 9 +++++---- mm/userfaultfd.c | 4 ++-- mm/util.c | 4 ++-- mm/vma.c | 15 +++++++-------- mm/vma_exec.c | 4 ++-- mm/vma_init.c | 2 +- 24 files changed, 86 insertions(+), 83 deletions(-)include/linux/huge_mm.h +1
-0@@ -230,6 +230,7 @@ static inline bool thp_vma_suitable_order(struct vm_area_struct *vma,/* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous (vma)) { +/* vma_start_pgoff() in mm.h so not available. */ if (!IS_ALIGNED ((vma->vm_start >>PAGE_SHIFT ) - vma->vm_pgoff, hpage_size >>PAGE_SHIFT ))return false;include/linux/hugetlb.h +1
-2@@ -792,8 +792,7 @@ static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma, {struct hstate *h =hstate_vma (vma); -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }static inline bool order_is_gigantic (unsigned int order)include/linux/pagemap.h +1
-1@@ -1101,7 +1101,7 @@ static inline pgoff_t linear_page_index(const struct vm_area_struct *vma,pgoff_t pgoff; pgoff =linear_page_delta (vma, address); - pgoff += vma->vm_pgoff; + pgoff +=vma_start_pgoff (vma);return pgoff; }mm/damon/vaddr.c +3
-2@@ -12,6 +12,7 @@#include <linux/mman.h> #include <linux/mmu_notifier.h> #include <linux/page_idle.h> +#include <linux/pagemap.h> #include <linux/pagewalk.h> #include <linux/sched/mm.h> @@ -627,8 +628,8 @@ static void damos_va_migrate_dests_add(struct folio *folio, } order =folio_order (folio); - ilx = vma->vm_pgoff >> order; - ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order); + ilx =vma_start_pgoff (vma) >> order; + ilx +=linear_page_delta (vma, addr) >> order;for (i =0 ; i < dests->nr_dests; i++) weight_total += dests->weight_arr[i];mm/debug.c +1
-1@@ -163,7 +163,7 @@ void dump_vma(const struct vm_area_struct *vma)"flags: %#lx(%pGv)\n" , vma, (void *)vma->vm_start, (void *)vma->vm_end, vma->vm_mm, (unsigned long )pgprot_val (vma->vm_page_prot), - vma->anon_vma, vma->vm_ops, vma->vm_pgoff, + vma->anon_vma, vma->vm_ops,vma_start_pgoff (vma), vma->vm_file, vma->vm_private_data, #ifdef CONFIG_PER_VMA_LOCK refcount_read(&vma->vm_refcnt),mm/filemap.c +4
-3@@ -3402,8 +3402,8 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)* of memory. */ struct vm_area_struct *vma = vmf->vma; -unsigned long start = vma->vm_pgoff; -unsigned long end = start +vma_pages (vma); +const unsigned long start =vma_start_pgoff (vma); +const unsigned long end =vma_end_pgoff (vma);unsigned long ra_end; ra->order =exec_folio_order ();@@ -3921,7 +3921,8 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, goto out; } - addr = vma->vm_start + ((start_pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + + ((start_pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT ); vmf->pte =pte_offset_map_lock (vma->vm_mm, vmf->pmd, addr, &vmf->ptl);if (!vmf->pte) {folio_unlock (folio);mm/huge_memory.c +1
-1@@ -180,7 +180,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,*/ if (!in_pf &&shmem_file (vma->vm_file))return orders &shmem_allowable_huge_orders (file_inode (vma->vm_file), - vma, vma->vm_pgoff,0 , + vma,vma_start_pgoff (vma),0 , forced_collapse);if (!vma_is_anonymous (vma)) {mm/hugetlb.c +4
-7@@ -1011,8 +1011,7 @@ static long region_count(struct resv_map *resv, long f, long t)static pgoff_t vma_hugecache_offset (struct hstate *h,struct vm_area_struct *vma,unsigned long address) { -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }/* @@ -5372,8 +5371,7 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,* from page cache lookup which is in HPAGE_SIZE units. */ address = address &huge_page_mask (h); - pgoff = ((address - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff; + pgoff =linear_page_index (vma, address); mapping = vma->vm_file->f_mapping;/* @@ -6771,7 +6769,7 @@ static unsigned long page_table_shareable(struct vm_area_struct *svma,struct vm_area_struct *vma, unsigned long addr, pgoff_t idx) { - unsigned long saddr = ((idx - svma->vm_pgoff) << PAGE_SHIFT) + + unsigned long saddr = ((idx - vma_start_pgoff(svma)) << PAGE_SHIFT) + svma->vm_start;unsigned long sbase = saddr &PUD_MASK ;unsigned long s_end = sbase +PUD_SIZE ;@@ -6856,8 +6854,7 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, pud_t *pud) { struct address_space *mapping = vma->vm_file->f_mapping; -pgoff_t idx = ((addr - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff; +const pgoff_t idx =linear_page_index (vma, addr);struct vm_area_struct *svma;unsigned long saddr;pte_t *spte =NULL ;mm/internal.h +14
-10@@ -1143,26 +1143,28 @@ static inline bool folio_within_range(struct folio *folio, struct vm_area_struct *vma, unsigned long start, unsigned long end) { - pgoff_t pgoff, addr; -unsigned long vma_pglen =vma_pages (vma); + const unsigned long vma_pglen = vma_pages(vma); +pgoff_t pgoff_folio, pgoff_vma_start; +unsigned long addr;VM_WARN_ON_FOLIO (folio_test_ksm (folio), folio);if (start > end)return false; + pgoff_folio =folio_pgoff (folio); + pgoff_vma_start =vma_start_pgoff (vma); +if (start < vma->vm_start) start = vma->vm_start;if (end > vma->vm_end) end = vma->vm_end; - pgoff =folio_pgoff (folio); -/* if folio start address is not in vma range */ -if (!in_range (pgoff, vma->vm_pgoff, vma_pglen)) +if (!in_range (pgoff_folio, pgoff_vma_start, vma_pglen))return false; - addr = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + ((pgoff_folio - pgoff_vma_start) <<PAGE_SHIFT );return !(addr < start || end - addr <folio_size (folio)); }@@ -1234,15 +1236,16 @@ extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);static inline unsigned long vma_address (const struct vm_area_struct *vma,pgoff_t pgoff,unsigned long nr_pages) { +const pgoff_t pgoff_start =vma_start_pgoff (vma);unsigned long address; -if (pgoff >= vma->vm_pgoff) { +if (pgoff >= pgoff_start) { address = vma->vm_start + - ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + ((pgoff - pgoff_start) <<PAGE_SHIFT );/* Check for address beyond vma (or wrapped through 0?) */ if (address < vma->vm_start || address >= vma->vm_end) address = -EFAULT ; - }else if (pgoff + nr_pages -1 >= vma->vm_pgoff) { + }else if (pgoff + nr_pages -1 >= pgoff_start) {/* Test above avoids possibility of wrap to 0 on 32-bit */ address = vma->vm_start; }else {@@ -1266,7 +1269,8 @@ static inline unsigned long vma_address_end(struct page_vma_mapped_walk *pvmw)return pvmw->address +PAGE_SIZE ; pgoff = pvmw->pgoff + pvmw->nr_pages; - address = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + address = vma->vm_start + + ((pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT );/* Check for address beyond vma (or wrapped through 0?) */ if (address < vma->vm_start || address > vma->vm_end) address = vma->vm_end;mm/khugepaged.c +2
-1@@ -2145,7 +2145,8 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)spinlock_t *ptl;bool success = false; - addr = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + + ((pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT );if (addr & ~HPAGE_PMD_MASK || vma->vm_end < addr +HPAGE_PMD_SIZE )continue ;mm/madvise.c +3
-3@@ -253,7 +253,7 @@ static void shmem_swapin_range(struct vm_area_struct *vma,continue ; addr = vma->vm_start + - ((xas.xa_index - vma->vm_pgoff) <<PAGE_SHIFT ); + ((xas.xa_index -vma_start_pgoff (vma)) <<PAGE_SHIFT );xas_pause (&xas);rcu_read_unlock ();@@ -318,7 +318,7 @@ static long madvise_willneed(struct madvise_behavior *madv_behavior)mark_mmap_lock_dropped (madv_behavior);get_file (file); offset = (loff_t)(start - vma->vm_start) - + ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT );mmap_read_unlock (mm);vfs_fadvise (file, offset, end - start,POSIX_FADV_WILLNEED );fput (file);@@ -1023,7 +1023,7 @@ static long madvise_remove(struct madvise_behavior *madv_behavior)return -EACCES ; offset = (loff_t)(start - vma->vm_start) - + ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT );/* * Filesystem's fallocate may need to take i_rwsem. We need to mm/mapping_dirty_helpers.c +1
-1@@ -95,7 +95,7 @@ static int clean_record_pte(pte_t *pte, unsigned long addr,if (pte_dirty (ptent)) {pgoff_t pgoff = ((addr - walk->vma->vm_start) >>PAGE_SHIFT ) + - walk->vma->vm_pgoff - cwalk->bitmap_pgoff; +vma_start_pgoff (walk->vma) - cwalk->bitmap_pgoff;pte_t old_pte =ptep_modify_prot_start (walk->vma, addr, pte); ptent =pte_mkclean (old_pte);mm/memory.c +13
-12@@ -725,10 +725,10 @@ static inline struct page *__vm_normal_page(struct vm_area_struct *vma,if (!pfn_valid (pfn))return NULL ; }else { -unsigned long off = (addr - vma->vm_start) >>PAGE_SHIFT ; +const pgoff_t index =linear_page_index (vma, addr);/* Only CoW'ed anon folios are "normal". */ -if (pfn == vma->vm_pgoff + off) +if (pfn == index)return NULL ;if (!is_cow_mapping (vma->vm_flags))return NULL ;@@ -2643,7 +2643,7 @@ static int __vm_map_pages(struct vm_area_struct *vma, struct page **pages,int vm_map_pages (struct vm_area_struct *vma,struct page **pages,unsigned long num) { -return __vm_map_pages (vma, pages, num, vma->vm_pgoff); +return __vm_map_pages (vma, pages, num,vma_start_pgoff (vma)); }EXPORT_SYMBOL (vm_map_pages);@@ -3298,7 +3298,8 @@ int vm_iomap_memory(struct vm_area_struct *vma, phys_addr_t start, unsigned longunsigned long pfn;int err; - err =__simple_ioremap_prep (vm_len, vma->vm_pgoff, start, len, &pfn); + err =__simple_ioremap_prep (vm_len,vma_start_pgoff (vma), start, len, + &pfn);if (err)return err;@@ -4342,15 +4343,15 @@ static inline void unmap_mapping_range_tree(struct address_space *mapping, struct zap_details *details) {struct vm_area_struct *vma; -unsigned long start, size;struct mmu_gather tlb; mapping_interval_tree_foreach(vma, mapping, first_index, last_index) { -const pgoff_t start_idx =max (first_index, vma->vm_pgoff); +const pgoff_t start_idx =max (first_index,vma_start_pgoff (vma));const pgoff_t end_idx =min (last_index,vma_last_pgoff (vma)) +1 ; - - start = vma->vm_start + ((start_idx - vma->vm_pgoff) <<PAGE_SHIFT ); - size = (end_idx - start_idx) <<PAGE_SHIFT ; +const pgoff_t offset = start_idx -vma_start_pgoff (vma); +const unsigned long offset_bytes = offset <<PAGE_SHIFT ; +const unsigned long start = vma->vm_start + offset_bytes; +const unsigned long size = (end_idx - start_idx) <<PAGE_SHIFT ;tlb_gather_mmu (&tlb, vma->vm_mm);zap_vma_range_batched (&tlb, vma, start, size, details);@@ -5684,7 +5685,7 @@ vm_fault_t finish_fault(struct vm_fault *vmf) }else if (nr_pages >1 ) {pgoff_t idx =folio_page_idx (folio, page);/* The page offset of vmf->address within the VMA. */ -pgoff_t vma_off = vmf->pgoff - vmf->vma->vm_pgoff; +pgoff_t vma_off = vmf->pgoff -vma_start_pgoff (vmf->vma);/* The index of the entry in the pagetable for fault page. */ pgoff_t pte_off =pte_index (vmf->address);@@ -5796,7 +5797,7 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)pgoff_t nr_pages =READ_ONCE (fault_around_pages);pgoff_t pte_off =pte_index (vmf->address);/* The page offset of vmf->address within the VMA. */ -pgoff_t vma_off = vmf->pgoff - vmf->vma->vm_pgoff; +pgoff_t vma_off = vmf->pgoff -vma_start_pgoff (vmf->vma);pgoff_t from_pte, to_pte;vm_fault_t ret;@@ -7274,7 +7275,7 @@ void print_vma_addr(char *prefix, unsigned long ip)if (vma && vma->vm_file) {struct file *f = vma->vm_file; ip -= vma->vm_start; - ip += vma->vm_pgoff <<PAGE_SHIFT ; + ip +=vma_start_pgoff (vma) <<PAGE_SHIFT ;printk ("%s%pD[%lx,%lx+%lx]" , prefix, f, ip, vma->vm_start, vma->vm_end - vma->vm_start);mm/mempolicy.c +7
-6@@ -2048,8 +2048,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struct *vma, pol =get_task_policy (current);if (pol->mode ==MPOL_INTERLEAVE || pol->mode ==MPOL_WEIGHTED_INTERLEAVE ) { - *ilx += vma->vm_pgoff >> order; - *ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order); + *ilx +=vma_start_pgoff (vma) >> order; + *ilx +=linear_page_delta (vma, addr) >> order; }return pol; }@@ -3250,16 +3250,17 @@ EXPORT_SYMBOL_FOR_MODULES(mpol_shared_policy_init, "kvm");int mpol_set_shared_policy (struct shared_policy *sp,struct vm_area_struct *vma,struct mempolicy *pol) { -int err; +const pgoff_t pgoff =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);struct sp_node *new =NULL ; -unsigned long sz =vma_pages (vma); +int err;if (pol) { - new =sp_alloc (vma->vm_pgoff, vma->vm_pgoff + sz, pol); + new =sp_alloc (pgoff, pgoff_end, pol);if (!new)return -ENOMEM ; } - err =shared_policy_replace (sp, vma->vm_pgoff, vma->vm_pgoff + sz, new); + err =shared_policy_replace (sp, pgoff, pgoff_end, new);if (err && new)sp_free (new);return err;mm/mremap.c +4
-8@@ -948,8 +948,7 @@ static unsigned long vrm_set_new_addr(struct vma_remap_struct *vrm)struct vm_area_struct *vma = vrm->vma;unsigned long map_flags =0 ;/* Page Offset _into_ the VMA. */ -pgoff_t internal_pgoff = (vrm->addr - vma->vm_start) >>PAGE_SHIFT ; -pgoff_t pgoff = vma->vm_pgoff + internal_pgoff; +const pgoff_t pgoff =linear_page_index (vma, vrm->addr);unsigned long new_addr =vrm_implies_new_addr (vrm) ? vrm->new_addr :0 ;unsigned long res;@@ -1255,12 +1254,10 @@ static void unmap_source_vma(struct vma_remap_struct *vrm)static int copy_vma_and_data (struct vma_remap_struct *vrm,struct vm_area_struct **new_vma_ptr) { -unsigned long internal_offset = vrm->addr - vrm->vma->vm_start; -unsigned long internal_pgoff = internal_offset >>PAGE_SHIFT ; -unsigned long new_pgoff = vrm->vma->vm_pgoff + internal_pgoff; -unsigned long moved_len; +const unsigned long new_pgoff =linear_page_index (vrm->vma, vrm->addr);struct vm_area_struct *vma = vrm->vma;struct vm_area_struct *new_vma; +unsigned long moved_len;int err =0 ;PAGETABLE_MOVE (pmc,NULL ,NULL , vrm->addr, vrm->new_addr, vrm->old_len);@@ -1802,8 +1799,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm) vrm->populate_expand = true;/* Need to be careful about a growing mapping */ - pgoff = (addr - vma->vm_start) >>PAGE_SHIFT ; - pgoff += vma->vm_pgoff; + pgoff =linear_page_index (vma, addr);if (pgoff + (new_len >>PAGE_SHIFT ) < pgoff)return -EINVAL ;mm/msync.c +2
-2@@ -12,6 +12,7 @@#include <linux/mm.h> #include <linux/mman.h> #include <linux/file.h> +#include <linux/pagemap.h> #include <linux/syscalls.h> #include <linux/sched.h> @@ -85,8 +86,7 @@ SYSCALL_DEFINE3(msync, unsigned long, start, size_t, len, int, flags) goto out_unlock; } file = vma->vm_file; - fstart = (start - vma->vm_start) + - ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + fstart = (loff_t )linear_page_index (vma, start) <<PAGE_SHIFT ; fend = fstart + (min (end, vma->vm_end) - start) -1 ; start = vma->vm_end; if ((flags & MS_SYNC) && file &&mm/nommu.c +4
-3@@ -975,7 +975,7 @@ static int do_mmap_private(struct vm_area_struct *vma,/* read the contents of a file into the copy */ loff_t fpos; - fpos = vma->vm_pgoff; + fpos =vma_start_pgoff (vma); fpos <<=PAGE_SHIFT ; ret =kernel_read (vma->vm_file, base, len, &fpos);@@ -1355,7 +1355,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,delete_nommu_region (vma->vm_region);if (new_below) { vma->vm_region->vm_start = vma->vm_start = addr; - vma->vm_region->vm_pgoff = vma->vm_pgoff += npages; + vma->vm_pgoff += npages; + vma->vm_region->vm_pgoff =vma_start_pgoff (vma); }else { vma->vm_region->vm_end = vma->vm_end = addr; vma->vm_region->vm_top = addr;@@ -1603,7 +1604,7 @@ int vm_iomap_memory(struct vm_area_struct *vma, phys_addr_t start, unsigned longunsigned long pfn = start >>PAGE_SHIFT ;unsigned long vm_len = vma->vm_end - vma->vm_start; - pfn += vma->vm_pgoff; + pfn +=vma_start_pgoff (vma);return io_remap_pfn_range (vma, vma->vm_start, pfn, vm_len, vma->vm_page_prot); }EXPORT_SYMBOL (vm_iomap_memory);mm/pagewalk.c +1
-1@@ -813,7 +813,7 @@ int walk_page_mapping(struct address_space *mapping, pgoff_t first_index,mapping_interval_tree_foreach (vma, mapping, first_index, first_index + nr - 1) {/* Clip to the vma */ - vba = vma->vm_pgoff; + vba =vma_start_pgoff (vma); vea = vba +vma_pages (vma); cba = first_index; cba =max (cba, vba);mm/shmem.c +5
-4@@ -1032,6 +1032,8 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma)struct inode *inode =file_inode (vma->vm_file);struct shmem_inode_info *info =SHMEM_I (inode);struct address_space *mapping = inode->i_mapping; +const pgoff_t pgoff =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);unsigned long swapped;/* Be careful as we don't hold info->lock */ @@ -1045,12 +1047,11 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma)if (!swapped)return 0 ; -if (!vma->vm_pgoff && vma->vm_end - vma->vm_start >= inode->i_size) +if (!pgoff && vma->vm_end - vma->vm_start >= inode->i_size)return swapped <<PAGE_SHIFT ;/* Here comes the more involved part */ -return shmem_partial_swap_usage (mapping, vma->vm_pgoff, - vma->vm_pgoff +vma_pages (vma)); +return shmem_partial_swap_usage (mapping, pgoff, pgoff_end); }/* @@ -2839,7 +2840,7 @@ static struct mempolicy *shmem_get_policy(struct vm_area_struct *vma,* by page order, as in shmem_get_pgoff_policy() and get_vma_policy()). */ *ilx = inode->i_ino; - index = ((addr - vma->vm_start) >>PAGE_SHIFT ) + vma->vm_pgoff; + index =linear_page_index (vma, addr);return mpol_shared_policy_lookup (&SHMEM_I (inode)->policy, index); }mm/userfaultfd.c +2
-2@@ -481,7 +481,7 @@ static void mfill_retry_state_save(struct mfill_retry_state *s, { s->flags =vma_flags_and_mask (&vma->flags,MFILL_RETRY_STATE_VMA_FLAGS ); s->ops =vma_uffd_ops (vma); - s->pgoff = vma->vm_pgoff; + s->pgoff =vma_start_pgoff (vma);if (vma->vm_file) s->file =get_file (vma->vm_file);@@ -507,7 +507,7 @@ static bool mfill_retry_state_changed(struct mfill_retry_state *state,/* VMA was file backed, but file, inode or offset has changed */ if (!vma->vm_file || vma->vm_file->f_inode != state->file->f_inode || - state->file != vma->vm_file || vma->vm_pgoff != state->pgoff) + state->file != vma->vm_file ||vma_start_pgoff (vma) != state->pgoff)return true;return false;mm/util.c +2
-2@@ -1188,7 +1188,7 @@ void compat_set_desc_from_vma(struct vm_area_desc *desc, desc->start = vma->vm_start; desc->end = vma->vm_end; - desc->pgoff = vma->vm_pgoff; + desc->pgoff =vma_start_pgoff (vma); desc->vm_file = vma->vm_file; desc->vma_flags = vma->flags; desc->page_prot = vma->vm_page_prot;@@ -1379,7 +1379,7 @@ static int call_vma_mapped(struct vm_area_struct *vma)if (!vm_ops || !vm_ops->mapped)return 0 ; - err = vm_ops->mapped (vma->vm_start, vma->vm_end, vma->vm_pgoff, + err = vm_ops->mapped (vma->vm_start, vma->vm_end,vma_start_pgoff (vma), vma->vm_file, &vm_private_data);if (err)return err;mm/vma.c +7
-8@@ -967,10 +967,9 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(* prev middle next * extend delete delete */ - vmg->start = prev->vm_start; vmg->end = next->vm_end; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);/* * We already ensured anon_vma compatibility above, so now it's @@ -987,9 +986,8 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(* prev middle * extend shrink/delete */ - vmg->start = prev->vm_start; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);if (!vmg->__remove_middle) vmg->__adjust_middle_start = true;@@ -1011,13 +1009,13 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(if (vmg->__remove_middle) { vmg->end = next->vm_end; - vmg->pgoff = next->vm_pgoff - pglen; + vmg->pgoff =vma_start_pgoff (next) - pglen; }else {/* We shrink middle and expand next. */ vmg->__adjust_next_start = true; vmg->start = middle->vm_start; vmg->end = start; - vmg->pgoff = middle->vm_pgoff; + vmg->pgoff =vma_start_pgoff (middle); } err =dup_anon_vma (next, middle, &anon_dup);@@ -1126,7 +1124,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)if (can_merge_left) { vmg->start = prev->vm_start; vmg->target = prev; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);/* * If this merge would result in removal of the next VMA but we @@ -1957,7 +1955,8 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,VM_BUG_ON_VMA (faulted_in_anon_vma, new_vma); *vmap = vma = new_vma; } - *need_rmap_locks = (new_vma->vm_pgoff <= vma->vm_pgoff); + *need_rmap_locks = + (vma_start_pgoff (new_vma) <=vma_start_pgoff (vma)); }else { new_vma =vm_area_dup (vma);if (!new_vma)mm/vma_exec.c +2
-2@@ -37,7 +37,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)unsigned long new_end = old_end - shift;VMA_ITERATOR (vmi, mm, new_start);VMG_STATE (vmg, mm, &vmi, new_start, old_end,EMPTY_VMA_FLAGS , - vma->vm_pgoff); +vma_start_pgoff (vma));struct vm_area_struct *next;struct mmu_gather tlb;PAGETABLE_MOVE (pmc, vma, vma, old_start, new_start, length);@@ -89,7 +89,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)vma_prev (&vmi);/* Shrink the vma to just the new range */ -return vma_shrink (&vmi, vma, new_start, new_end, vma->vm_pgoff); +return vma_shrink (&vmi, vma, new_start, new_end,vma_start_pgoff (vma)); }/* mm/vma_init.c +1
-1@@ -46,7 +46,7 @@ static void vm_area_init_from(const struct vm_area_struct *src, dest->vm_start = src->vm_start; dest->vm_end = src->vm_end; dest->anon_vma = src->anon_vma; - dest->vm_pgoff = src->vm_pgoff; + dest->vm_pgoff =vma_start_pgoff (src); dest->vm_file = src->vm_file; dest->vm_private_data = src->vm_private_data;vm_flags_init (dest, src->vm_flags);Signature · 1 line
2.54.0
Lorenzo Stoakes
maintainerRe: [PATCH 16/30] mm/vma: use vma_start_pgoff(), linear_page_index() in mm code
Quoted 18 lines · Gregory Price
Quoted 17 lines · Lorenzo Stoakes
There are many instances in which
linear_page_index()(as well aslinear_page_delta()) is open-coded, which is confusing and inconsistent.Additionally,
vma->vm_pgoffdoesn't necessarily make it clear that this is the page offset of the start of the VMA range.Doing so also aids greppability.
So use
vma_start_pgoff()in favour of directly accessingvma->vm_pgoff, andlinear_page_index()where we can.This also lays the ground for future changes which will add an anonymous page offset in order to be able to index MAP_PRIVATE-file backed anon folios in terms of their virtual page offset.
No functional change intended.
Quoted 7 lines
Quoted 1 line
Signed-off-by Lorenzo Stoakes <[email protected]>such a nice readability improvement, thank you.
This made me wonder though whether we can make it painful for people to open-code this in the future, otherwise I fear another one of these patches creeping in 5-10 years down the line.
Yeah for sure, I wonder if we could somehow make some VMA fields private honestly which would address a lot of this. I think there clever ways of making certain things a gap in the struct outwardly that can do it.
But one for the future :)
Quoted 4 lines
Not sure it's feasible though
Reviewed-by Gregory Price <[email protected]>
Thanks! :)
Cheers, Lorenzo
Quoted 719 lines
Quoted 719 lines
--- include/linux/huge_mm.h | 1 + include/linux/hugetlb.h | 3 +-- include/linux/pagemap.h | 2 +- mm/damon/vaddr.c | 5 +++-- mm/debug.c | 2 +- mm/filemap.c | 7 ++++--- mm/huge_memory.c | 2 +- mm/hugetlb.c | 11 ++++------- mm/internal.h | 24 ++++++++++++++---------- mm/khugepaged.c | 3 ++- mm/madvise.c | 6 +++--- mm/mapping_dirty_helpers.c | 2 +- mm/memory.c | 25 +++++++++++++------------ mm/mempolicy.c | 13 +++++++------ mm/mremap.c | 12 ++++-------- mm/msync.c | 4 ++-- mm/nommu.c | 7 ++++--- mm/pagewalk.c | 2 +- mm/shmem.c | 9 +++++---- mm/userfaultfd.c | 4 ++-- mm/util.c | 4 ++-- mm/vma.c | 15 +++++++-------- mm/vma_exec.c | 4 ++-- mm/vma_init.c | 2 +- 24 files changed, 86 insertions(+), 83 deletions(-)include/linux/huge_mm.h +1
-0@@ -230,6 +230,7 @@ static inline bool thp_vma_suitable_order(struct vm_area_struct *vma,/* Don't have to check pgoff for anonymous vma */ if (!vma_is_anonymous (vma)) { +/* vma_start_pgoff() in mm.h so not available. */ if (!IS_ALIGNED ((vma->vm_start >>PAGE_SHIFT ) - vma->vm_pgoff, hpage_size >>PAGE_SHIFT ))return false;include/linux/hugetlb.h +1
-2@@ -792,8 +792,7 @@ static inline pgoff_t hugetlb_linear_page_index(struct vm_area_struct *vma, {struct hstate *h =hstate_vma (vma); -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }static inline bool order_is_gigantic (unsigned int order)include/linux/pagemap.h +1
-1@@ -1101,7 +1101,7 @@ static inline pgoff_t linear_page_index(const struct vm_area_struct *vma,pgoff_t pgoff; pgoff =linear_page_delta (vma, address); - pgoff += vma->vm_pgoff; + pgoff +=vma_start_pgoff (vma);return pgoff; }mm/damon/vaddr.c +3
-2@@ -12,6 +12,7 @@#include <linux/mman.h> #include <linux/mmu_notifier.h> #include <linux/page_idle.h> +#include <linux/pagemap.h> #include <linux/pagewalk.h> #include <linux/sched/mm.h> @@ -627,8 +628,8 @@ static void damos_va_migrate_dests_add(struct folio *folio, } order =folio_order (folio); - ilx = vma->vm_pgoff >> order; - ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order); + ilx =vma_start_pgoff (vma) >> order; + ilx +=linear_page_delta (vma, addr) >> order;for (i =0 ; i < dests->nr_dests; i++) weight_total += dests->weight_arr[i];mm/debug.c +1
-1@@ -163,7 +163,7 @@ void dump_vma(const struct vm_area_struct *vma)"flags: %#lx(%pGv)\n" , vma, (void *)vma->vm_start, (void *)vma->vm_end, vma->vm_mm, (unsigned long )pgprot_val (vma->vm_page_prot), - vma->anon_vma, vma->vm_ops, vma->vm_pgoff, + vma->anon_vma, vma->vm_ops,vma_start_pgoff (vma), vma->vm_file, vma->vm_private_data, #ifdef CONFIG_PER_VMA_LOCK refcount_read(&vma->vm_refcnt),mm/filemap.c +4
-3@@ -3402,8 +3402,8 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)* of memory. */ struct vm_area_struct *vma = vmf->vma; -unsigned long start = vma->vm_pgoff; -unsigned long end = start +vma_pages (vma); +const unsigned long start =vma_start_pgoff (vma); +const unsigned long end =vma_end_pgoff (vma);unsigned long ra_end; ra->order =exec_folio_order ();@@ -3921,7 +3921,8 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf, goto out; } - addr = vma->vm_start + ((start_pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + + ((start_pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT ); vmf->pte =pte_offset_map_lock (vma->vm_mm, vmf->pmd, addr, &vmf->ptl);if (!vmf->pte) {folio_unlock (folio);mm/huge_memory.c +1
-1@@ -180,7 +180,7 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma,*/ if (!in_pf &&shmem_file (vma->vm_file))return orders &shmem_allowable_huge_orders (file_inode (vma->vm_file), - vma, vma->vm_pgoff,0 , + vma,vma_start_pgoff (vma),0 , forced_collapse);if (!vma_is_anonymous (vma)) {mm/hugetlb.c +4
-7@@ -1011,8 +1011,7 @@ static long region_count(struct resv_map *resv, long f, long t)static pgoff_t vma_hugecache_offset (struct hstate *h,struct vm_area_struct *vma,unsigned long address) { -return ((address - vma->vm_start) >>huge_page_shift (h)) + - (vma->vm_pgoff >>huge_page_order (h)); +return linear_page_index (vma, address) >>huge_page_order (h); }/* @@ -5372,8 +5371,7 @@ static void unmap_ref_private(struct mm_struct *mm, struct vm_area_struct *vma,* from page cache lookup which is in HPAGE_SIZE units. */ address = address &huge_page_mask (h); - pgoff = ((address - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff; + pgoff =linear_page_index (vma, address); mapping = vma->vm_file->f_mapping;/* @@ -6771,7 +6769,7 @@ static unsigned long page_table_shareable(struct vm_area_struct *svma,struct vm_area_struct *vma, unsigned long addr, pgoff_t idx) { - unsigned long saddr = ((idx - svma->vm_pgoff) << PAGE_SHIFT) + + unsigned long saddr = ((idx - vma_start_pgoff(svma)) << PAGE_SHIFT) + svma->vm_start;unsigned long sbase = saddr &PUD_MASK ;unsigned long s_end = sbase +PUD_SIZE ;@@ -6856,8 +6854,7 @@ pte_t *huge_pmd_share(struct mm_struct *mm, struct vm_area_struct *vma, unsigned long addr, pud_t *pud) { struct address_space *mapping = vma->vm_file->f_mapping; -pgoff_t idx = ((addr - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff; +const pgoff_t idx =linear_page_index (vma, addr);struct vm_area_struct *svma;unsigned long saddr;pte_t *spte =NULL ;mm/internal.h +14
-10@@ -1143,26 +1143,28 @@ static inline bool folio_within_range(struct folio *folio, struct vm_area_struct *vma, unsigned long start, unsigned long end) { - pgoff_t pgoff, addr; -unsigned long vma_pglen =vma_pages (vma); + const unsigned long vma_pglen = vma_pages(vma); +pgoff_t pgoff_folio, pgoff_vma_start; +unsigned long addr;VM_WARN_ON_FOLIO (folio_test_ksm (folio), folio);if (start > end)return false; + pgoff_folio =folio_pgoff (folio); + pgoff_vma_start =vma_start_pgoff (vma); +if (start < vma->vm_start) start = vma->vm_start;if (end > vma->vm_end) end = vma->vm_end; - pgoff =folio_pgoff (folio); -/* if folio start address is not in vma range */ -if (!in_range (pgoff, vma->vm_pgoff, vma_pglen)) +if (!in_range (pgoff_folio, pgoff_vma_start, vma_pglen))return false; - addr = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + ((pgoff_folio - pgoff_vma_start) <<PAGE_SHIFT );return !(addr < start || end - addr <folio_size (folio)); }@@ -1234,15 +1236,16 @@ extern pmd_t maybe_pmd_mkwrite(pmd_t pmd, struct vm_area_struct *vma);static inline unsigned long vma_address (const struct vm_area_struct *vma,pgoff_t pgoff,unsigned long nr_pages) { +const pgoff_t pgoff_start =vma_start_pgoff (vma);unsigned long address; -if (pgoff >= vma->vm_pgoff) { +if (pgoff >= pgoff_start) { address = vma->vm_start + - ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + ((pgoff - pgoff_start) <<PAGE_SHIFT );/* Check for address beyond vma (or wrapped through 0?) */ if (address < vma->vm_start || address >= vma->vm_end) address = -EFAULT ; - }else if (pgoff + nr_pages -1 >= vma->vm_pgoff) { + }else if (pgoff + nr_pages -1 >= pgoff_start) {/* Test above avoids possibility of wrap to 0 on 32-bit */ address = vma->vm_start; }else {@@ -1266,7 +1269,8 @@ static inline unsigned long vma_address_end(struct page_vma_mapped_walk *pvmw)return pvmw->address +PAGE_SIZE ; pgoff = pvmw->pgoff + pvmw->nr_pages; - address = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + address = vma->vm_start + + ((pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT );/* Check for address beyond vma (or wrapped through 0?) */ if (address < vma->vm_start || address > vma->vm_end) address = vma->vm_end;mm/khugepaged.c +2
-1@@ -2145,7 +2145,8 @@ static void retract_page_tables(struct address_space *mapping, pgoff_t pgoff)spinlock_t *ptl;bool success = false; - addr = vma->vm_start + ((pgoff - vma->vm_pgoff) <<PAGE_SHIFT ); + addr = vma->vm_start + + ((pgoff -vma_start_pgoff (vma)) <<PAGE_SHIFT );if (addr & ~HPAGE_PMD_MASK || vma->vm_end < addr +HPAGE_PMD_SIZE )continue ;mm/madvise.c +3
-3@@ -253,7 +253,7 @@ static void shmem_swapin_range(struct vm_area_struct *vma,continue ; addr = vma->vm_start + - ((xas.xa_index - vma->vm_pgoff) <<PAGE_SHIFT ); + ((xas.xa_index -vma_start_pgoff (vma)) <<PAGE_SHIFT );xas_pause (&xas);rcu_read_unlock ();@@ -318,7 +318,7 @@ static long madvise_willneed(struct madvise_behavior *madv_behavior)mark_mmap_lock_dropped (madv_behavior);get_file (file); offset = (loff_t)(start - vma->vm_start) - + ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT );mmap_read_unlock (mm);vfs_fadvise (file, offset, end - start,POSIX_FADV_WILLNEED );fput (file);@@ -1023,7 +1023,7 @@ static long madvise_remove(struct madvise_behavior *madv_behavior)return -EACCES ; offset = (loff_t)(start - vma->vm_start) - + ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT );/* * Filesystem's fallocate may need to take i_rwsem. We need to mm/mapping_dirty_helpers.c +1
-1@@ -95,7 +95,7 @@ static int clean_record_pte(pte_t *pte, unsigned long addr,if (pte_dirty (ptent)) {pgoff_t pgoff = ((addr - walk->vma->vm_start) >>PAGE_SHIFT ) + - walk->vma->vm_pgoff - cwalk->bitmap_pgoff; +vma_start_pgoff (walk->vma) - cwalk->bitmap_pgoff;pte_t old_pte =ptep_modify_prot_start (walk->vma, addr, pte); ptent =pte_mkclean (old_pte);mm/memory.c +13
-12@@ -725,10 +725,10 @@ static inline struct page *__vm_normal_page(struct vm_area_struct *vma,if (!pfn_valid (pfn))return NULL ; }else { -unsigned long off = (addr - vma->vm_start) >>PAGE_SHIFT ; +const pgoff_t index =linear_page_index (vma, addr);/* Only CoW'ed anon folios are "normal". */ -if (pfn == vma->vm_pgoff + off) +if (pfn == index)return NULL ;if (!is_cow_mapping (vma->vm_flags))return NULL ;@@ -2643,7 +2643,7 @@ static int __vm_map_pages(struct vm_area_struct *vma, struct page **pages,int vm_map_pages (struct vm_area_struct *vma,struct page **pages,unsigned long num) { -return __vm_map_pages (vma, pages, num, vma->vm_pgoff); +return __vm_map_pages (vma, pages, num,vma_start_pgoff (vma)); }EXPORT_SYMBOL (vm_map_pages);@@ -3298,7 +3298,8 @@ int vm_iomap_memory(struct vm_area_struct *vma, phys_addr_t start, unsigned longunsigned long pfn;int err; - err =__simple_ioremap_prep (vm_len, vma->vm_pgoff, start, len, &pfn); + err =__simple_ioremap_prep (vm_len,vma_start_pgoff (vma), start, len, + &pfn);if (err)return err;@@ -4342,15 +4343,15 @@ static inline void unmap_mapping_range_tree(struct address_space *mapping, struct zap_details *details) {struct vm_area_struct *vma; -unsigned long start, size;struct mmu_gather tlb; mapping_interval_tree_foreach(vma, mapping, first_index, last_index) { -const pgoff_t start_idx =max (first_index, vma->vm_pgoff); +const pgoff_t start_idx =max (first_index,vma_start_pgoff (vma));const pgoff_t end_idx =min (last_index,vma_last_pgoff (vma)) +1 ; - - start = vma->vm_start + ((start_idx - vma->vm_pgoff) <<PAGE_SHIFT ); - size = (end_idx - start_idx) <<PAGE_SHIFT ; +const pgoff_t offset = start_idx -vma_start_pgoff (vma); +const unsigned long offset_bytes = offset <<PAGE_SHIFT ; +const unsigned long start = vma->vm_start + offset_bytes; +const unsigned long size = (end_idx - start_idx) <<PAGE_SHIFT ;tlb_gather_mmu (&tlb, vma->vm_mm);zap_vma_range_batched (&tlb, vma, start, size, details);@@ -5684,7 +5685,7 @@ vm_fault_t finish_fault(struct vm_fault *vmf) }else if (nr_pages >1 ) {pgoff_t idx =folio_page_idx (folio, page);/* The page offset of vmf->address within the VMA. */ -pgoff_t vma_off = vmf->pgoff - vmf->vma->vm_pgoff; +pgoff_t vma_off = vmf->pgoff -vma_start_pgoff (vmf->vma);/* The index of the entry in the pagetable for fault page. */ pgoff_t pte_off =pte_index (vmf->address);@@ -5796,7 +5797,7 @@ static vm_fault_t do_fault_around(struct vm_fault *vmf)pgoff_t nr_pages =READ_ONCE (fault_around_pages);pgoff_t pte_off =pte_index (vmf->address);/* The page offset of vmf->address within the VMA. */ -pgoff_t vma_off = vmf->pgoff - vmf->vma->vm_pgoff; +pgoff_t vma_off = vmf->pgoff -vma_start_pgoff (vmf->vma);pgoff_t from_pte, to_pte;vm_fault_t ret;@@ -7274,7 +7275,7 @@ void print_vma_addr(char *prefix, unsigned long ip)if (vma && vma->vm_file) {struct file *f = vma->vm_file; ip -= vma->vm_start; - ip += vma->vm_pgoff <<PAGE_SHIFT ; + ip +=vma_start_pgoff (vma) <<PAGE_SHIFT ;printk ("%s%pD[%lx,%lx+%lx]" , prefix, f, ip, vma->vm_start, vma->vm_end - vma->vm_start);mm/mempolicy.c +7
-6@@ -2048,8 +2048,8 @@ struct mempolicy *get_vma_policy(struct vm_area_struct *vma, pol =get_task_policy (current);if (pol->mode ==MPOL_INTERLEAVE || pol->mode ==MPOL_WEIGHTED_INTERLEAVE ) { - *ilx += vma->vm_pgoff >> order; - *ilx += (addr - vma->vm_start) >> (PAGE_SHIFT + order); + *ilx +=vma_start_pgoff (vma) >> order; + *ilx +=linear_page_delta (vma, addr) >> order; }return pol; }@@ -3250,16 +3250,17 @@ EXPORT_SYMBOL_FOR_MODULES(mpol_shared_policy_init, "kvm");int mpol_set_shared_policy (struct shared_policy *sp,struct vm_area_struct *vma,struct mempolicy *pol) { -int err; +const pgoff_t pgoff =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);struct sp_node *new =NULL ; -unsigned long sz =vma_pages (vma); +int err;if (pol) { - new =sp_alloc (vma->vm_pgoff, vma->vm_pgoff + sz, pol); + new =sp_alloc (pgoff, pgoff_end, pol);if (!new)return -ENOMEM ; } - err =shared_policy_replace (sp, vma->vm_pgoff, vma->vm_pgoff + sz, new); + err =shared_policy_replace (sp, pgoff, pgoff_end, new);if (err && new)sp_free (new);return err;mm/mremap.c +4
-8@@ -948,8 +948,7 @@ static unsigned long vrm_set_new_addr(struct vma_remap_struct *vrm)struct vm_area_struct *vma = vrm->vma;unsigned long map_flags =0 ;/* Page Offset _into_ the VMA. */ -pgoff_t internal_pgoff = (vrm->addr - vma->vm_start) >>PAGE_SHIFT ; -pgoff_t pgoff = vma->vm_pgoff + internal_pgoff; +const pgoff_t pgoff =linear_page_index (vma, vrm->addr);unsigned long new_addr =vrm_implies_new_addr (vrm) ? vrm->new_addr :0 ;unsigned long res;@@ -1255,12 +1254,10 @@ static void unmap_source_vma(struct vma_remap_struct *vrm)static int copy_vma_and_data (struct vma_remap_struct *vrm,struct vm_area_struct **new_vma_ptr) { -unsigned long internal_offset = vrm->addr - vrm->vma->vm_start; -unsigned long internal_pgoff = internal_offset >>PAGE_SHIFT ; -unsigned long new_pgoff = vrm->vma->vm_pgoff + internal_pgoff; -unsigned long moved_len; +const unsigned long new_pgoff =linear_page_index (vrm->vma, vrm->addr);struct vm_area_struct *vma = vrm->vma;struct vm_area_struct *new_vma; +unsigned long moved_len;int err =0 ;PAGETABLE_MOVE (pmc,NULL ,NULL , vrm->addr, vrm->new_addr, vrm->old_len);@@ -1802,8 +1799,7 @@ static int check_prep_vma(struct vma_remap_struct *vrm) vrm->populate_expand = true;/* Need to be careful about a growing mapping */ - pgoff = (addr - vma->vm_start) >>PAGE_SHIFT ; - pgoff += vma->vm_pgoff; + pgoff =linear_page_index (vma, addr);if (pgoff + (new_len >>PAGE_SHIFT ) < pgoff)return -EINVAL ;mm/msync.c +2
-2@@ -12,6 +12,7 @@#include <linux/mm.h> #include <linux/mman.h> #include <linux/file.h> +#include <linux/pagemap.h> #include <linux/syscalls.h> #include <linux/sched.h> @@ -85,8 +86,7 @@ SYSCALL_DEFINE3(msync, unsigned long, start, size_t, len, int, flags) goto out_unlock; } file = vma->vm_file; - fstart = (start - vma->vm_start) + - ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); + fstart = (loff_t )linear_page_index (vma, start) <<PAGE_SHIFT ; fend = fstart + (min (end, vma->vm_end) - start) -1 ; start = vma->vm_end; if ((flags & MS_SYNC) && file &&mm/nommu.c +4
-3@@ -975,7 +975,7 @@ static int do_mmap_private(struct vm_area_struct *vma,/* read the contents of a file into the copy */ loff_t fpos; - fpos = vma->vm_pgoff; + fpos =vma_start_pgoff (vma); fpos <<=PAGE_SHIFT ; ret =kernel_read (vma->vm_file, base, len, &fpos);@@ -1355,7 +1355,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,delete_nommu_region (vma->vm_region);if (new_below) { vma->vm_region->vm_start = vma->vm_start = addr; - vma->vm_region->vm_pgoff = vma->vm_pgoff += npages; + vma->vm_pgoff += npages; + vma->vm_region->vm_pgoff =vma_start_pgoff (vma); }else { vma->vm_region->vm_end = vma->vm_end = addr; vma->vm_region->vm_top = addr;@@ -1603,7 +1604,7 @@ int vm_iomap_memory(struct vm_area_struct *vma, phys_addr_t start, unsigned longunsigned long pfn = start >>PAGE_SHIFT ;unsigned long vm_len = vma->vm_end - vma->vm_start; - pfn += vma->vm_pgoff; + pfn +=vma_start_pgoff (vma);return io_remap_pfn_range (vma, vma->vm_start, pfn, vm_len, vma->vm_page_prot); }EXPORT_SYMBOL (vm_iomap_memory);mm/pagewalk.c +1
-1@@ -813,7 +813,7 @@ int walk_page_mapping(struct address_space *mapping, pgoff_t first_index,mapping_interval_tree_foreach (vma, mapping, first_index, first_index + nr - 1) {/* Clip to the vma */ - vba = vma->vm_pgoff; + vba =vma_start_pgoff (vma); vea = vba +vma_pages (vma); cba = first_index; cba =max (cba, vba);mm/shmem.c +5
-4@@ -1032,6 +1032,8 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma)struct inode *inode =file_inode (vma->vm_file);struct shmem_inode_info *info =SHMEM_I (inode);struct address_space *mapping = inode->i_mapping; +const pgoff_t pgoff =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);unsigned long swapped;/* Be careful as we don't hold info->lock */ @@ -1045,12 +1047,11 @@ unsigned long shmem_swap_usage(struct vm_area_struct *vma)if (!swapped)return 0 ; -if (!vma->vm_pgoff && vma->vm_end - vma->vm_start >= inode->i_size) +if (!pgoff && vma->vm_end - vma->vm_start >= inode->i_size)return swapped <<PAGE_SHIFT ;/* Here comes the more involved part */ -return shmem_partial_swap_usage (mapping, vma->vm_pgoff, - vma->vm_pgoff +vma_pages (vma)); +return shmem_partial_swap_usage (mapping, pgoff, pgoff_end); }/* @@ -2839,7 +2840,7 @@ static struct mempolicy *shmem_get_policy(struct vm_area_struct *vma,* by page order, as in shmem_get_pgoff_policy() and get_vma_policy()). */ *ilx = inode->i_ino; - index = ((addr - vma->vm_start) >>PAGE_SHIFT ) + vma->vm_pgoff; + index =linear_page_index (vma, addr);return mpol_shared_policy_lookup (&SHMEM_I (inode)->policy, index); }mm/userfaultfd.c +2
-2@@ -481,7 +481,7 @@ static void mfill_retry_state_save(struct mfill_retry_state *s, { s->flags =vma_flags_and_mask (&vma->flags,MFILL_RETRY_STATE_VMA_FLAGS ); s->ops =vma_uffd_ops (vma); - s->pgoff = vma->vm_pgoff; + s->pgoff =vma_start_pgoff (vma);if (vma->vm_file) s->file =get_file (vma->vm_file);@@ -507,7 +507,7 @@ static bool mfill_retry_state_changed(struct mfill_retry_state *state,/* VMA was file backed, but file, inode or offset has changed */ if (!vma->vm_file || vma->vm_file->f_inode != state->file->f_inode || - state->file != vma->vm_file || vma->vm_pgoff != state->pgoff) + state->file != vma->vm_file ||vma_start_pgoff (vma) != state->pgoff)return true;return false;mm/util.c +2
-2@@ -1188,7 +1188,7 @@ void compat_set_desc_from_vma(struct vm_area_desc *desc, desc->start = vma->vm_start; desc->end = vma->vm_end; - desc->pgoff = vma->vm_pgoff; + desc->pgoff =vma_start_pgoff (vma); desc->vm_file = vma->vm_file; desc->vma_flags = vma->flags; desc->page_prot = vma->vm_page_prot;@@ -1379,7 +1379,7 @@ static int call_vma_mapped(struct vm_area_struct *vma)if (!vm_ops || !vm_ops->mapped)return 0 ; - err = vm_ops->mapped (vma->vm_start, vma->vm_end, vma->vm_pgoff, + err = vm_ops->mapped (vma->vm_start, vma->vm_end,vma_start_pgoff (vma), vma->vm_file, &vm_private_data);if (err)return err;mm/vma.c +7
-8@@ -967,10 +967,9 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(* prev middle next * extend delete delete */ - vmg->start = prev->vm_start; vmg->end = next->vm_end; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);/* * We already ensured anon_vma compatibility above, so now it's @@ -987,9 +986,8 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(* prev middle * extend shrink/delete */ - vmg->start = prev->vm_start; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);if (!vmg->__remove_middle) vmg->__adjust_middle_start = true;@@ -1011,13 +1009,13 @@ static __must_check struct vm_area_struct *vma_merge_existing_range(if (vmg->__remove_middle) { vmg->end = next->vm_end; - vmg->pgoff = next->vm_pgoff - pglen; + vmg->pgoff =vma_start_pgoff (next) - pglen; }else {/* We shrink middle and expand next. */ vmg->__adjust_next_start = true; vmg->start = middle->vm_start; vmg->end = start; - vmg->pgoff = middle->vm_pgoff; + vmg->pgoff =vma_start_pgoff (middle); } err =dup_anon_vma (next, middle, &anon_dup);@@ -1126,7 +1124,7 @@ struct vm_area_struct *vma_merge_new_range(struct vma_merge_struct *vmg)if (can_merge_left) { vmg->start = prev->vm_start; vmg->target = prev; - vmg->pgoff = prev->vm_pgoff; + vmg->pgoff =vma_start_pgoff (prev);/* * If this merge would result in removal of the next VMA but we @@ -1957,7 +1955,8 @@ struct vm_area_struct *copy_vma(struct vm_area_struct **vmap,VM_BUG_ON_VMA (faulted_in_anon_vma, new_vma); *vmap = vma = new_vma; } - *need_rmap_locks = (new_vma->vm_pgoff <= vma->vm_pgoff); + *need_rmap_locks = + (vma_start_pgoff (new_vma) <=vma_start_pgoff (vma)); }else { new_vma =vm_area_dup (vma);if (!new_vma)mm/vma_exec.c +2
-2@@ -37,7 +37,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)unsigned long new_end = old_end - shift;VMA_ITERATOR (vmi, mm, new_start);VMG_STATE (vmg, mm, &vmi, new_start, old_end,EMPTY_VMA_FLAGS , - vma->vm_pgoff); +vma_start_pgoff (vma));struct vm_area_struct *next;struct mmu_gather tlb;PAGETABLE_MOVE (pmc, vma, vma, old_start, new_start, length);@@ -89,7 +89,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)vma_prev (&vmi);/* Shrink the vma to just the new range */ -return vma_shrink (&vmi, vma, new_start, new_end, vma->vm_pgoff); +return vma_shrink (&vmi, vma, new_start, new_end,vma_start_pgoff (vma)); }/* mm/vma_init.c +1
-2@@ -46,7 +46,7 @@ static void vm_area_init_from(const struct vm_area_struct *src, dest->vm_start = src->vm_start; dest->vm_end = src->vm_end; dest->anon_vma = src->anon_vma; - dest->vm_pgoff = src->vm_pgoff; + dest->vm_pgoff =vma_start_pgoff (src); dest->vm_file = src->vm_file; dest->vm_private_data = src->vm_private_data;vm_flags_init (dest, src->vm_flags); --2.54.0
Lorenzo Stoakes
maintainerPATCH 17/30 mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
In reply to Lorenzo StoakesBe consistent in using vma_start_pgoff() and vma_end_pgoff(), which clearly indicates which part of the VMA the page offset refers to and aids greppability.
This is part of a broader series laying the ground to provide a virtual page offset for MAP_PRIVATE-file backed anon folios.
No functional change intended.
--- kernel/dma/coherent.c | 7 ++++--- kernel/dma/direct.c | 6 ++++-- kernel/dma/mapping.c | 8 +++++--- kernel/dma/ops_helpers.c | 4 ++-- kernel/events/core.c | 20 +++++++++++--------- kernel/events/uprobes.c | 11 +++++++---- kernel/kcov.c | 2 +- kernel/trace/ring_buffer.c | 3 ++- 8 files changed, 36 insertions(+), 25 deletions(-)
kernel/dma/coherent.c +4 -3
@@ -236,14 +236,15 @@ static int __dma_mmap_from_coherent(struct dma_coherent_mem *mem, {if (mem && vaddr >= mem->virt_base && vaddr + size <= (mem->virt_base + ((dma_addr_t )mem->size <<PAGE_SHIFT ))) { -unsigned long off = vma->vm_pgoff; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int start = (vaddr - mem->virt_base) >>PAGE_SHIFT ;unsigned long user_count =vma_pages (vma);int count =PAGE_ALIGN (size) >>PAGE_SHIFT ; *ret = -ENXIO ; -if (off < count && user_count <= count - off) { -unsigned long pfn = mem->pfn_base + start + off; +if (pgoff_start < count && pgoff_end <= count) { +unsigned long pfn = mem->pfn_base + start + pgoff_start; *ret =remap_pfn_range (vma, vma->vm_start, pfn, user_count <<PAGE_SHIFT , vma->vm_page_prot);
kernel/dma/direct.c +4 -2
@@ -534,6 +534,8 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ;unsigned long pfn =PHYS_PFN (dma_to_phys (dev, dma_addr)); +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int ret = -ENXIO ; vma->vm_page_prot =dma_pgprot (dev, vma->vm_page_prot, attrs);
@@ -545,9 +547,9 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,if (dma_mmap_from_global_coherent (vma, cpu_addr, size, &ret))return ret; -if (vma->vm_pgoff >= count || user_count > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ; -return remap_pfn_range (vma, vma->vm_start, pfn + vma->vm_pgoff, +return remap_pfn_range (vma, vma->vm_start, pfn + pgoff_start, user_count <<PAGE_SHIFT , vma->vm_page_prot); }
kernel/dma/mapping.c +5 -3
@@ -761,12 +761,14 @@ EXPORT_SYMBOL_GPL(dma_free_pages);int dma_mmap_pages (struct device *dev,struct vm_area_struct *vma,size_t size,struct page *page) { -unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma); +const unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -if (vma->vm_pgoff >= count ||vma_pages (vma) > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) + pgoff_start,vma_pages (vma) <<PAGE_SHIFT , vma->vm_page_prot); }EXPORT_SYMBOL_GPL (dma_mmap_pages);
kernel/dma/ops_helpers.c +2 -2
@@ -39,7 +39,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,#ifdef CONFIG_MMU unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; +unsigned long off =vma_start_pgoff (vma);struct page *page =dma_common_vaddr_to_page (cpu_addr);int ret = -ENXIO ;
@@ -52,7 +52,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) +vma_start_pgoff (vma), user_count <<PAGE_SHIFT , vma->vm_page_prot);#else return -ENXIO ;
kernel/events/core.c +11 -9
@@ -6998,7 +6998,7 @@ static void perf_mmap_open(struct vm_area_struct *vma)refcount_inc (&event->mmap_count);refcount_inc (&event->rb->mmap_count); -if (vma->vm_pgoff) +if (vma_start_pgoff (vma))refcount_inc (&event->rb->aux_mmap_count);if (mapped)
@@ -7032,7 +7032,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)* The AUX buffer is strictly a sub-buffer, serialize using aux_mutex * to avoid complications. */ -if (rb_has_aux (rb) && vma->vm_pgoff == rb->aux_pgoff && +if (rb_has_aux (rb) &&vma_start_pgoff (vma) == rb->aux_pgoff &&refcount_dec_and_mutex_lock (&rb->aux_mmap_count, &rb->aux_mutex)) {/* * Stop all AUX events that are writing to this buffer,
@@ -7190,7 +7190,8 @@ static int map_range(struct perf_buffer *rb, struct vm_area_struct *vma)*/ for (pagenum =0 ; pagenum < nr_pages; pagenum++) {unsigned long va = vma->vm_start +PAGE_SIZE * pagenum; -struct page *page =perf_mmap_to_page (rb, vma->vm_pgoff + pagenum); +struct page *page =perf_mmap_to_page (rb, +vma_start_pgoff (vma) + pagenum);if (page ==NULL ) { err = -EINVAL ;
@@ -7348,6 +7349,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,u64 aux_offset, aux_size;struct perf_buffer *rb;int ret, rb_flags =0 ; +const pgoff_t pgoff_start =vma_start_pgoff (vma); rb = event->rb;if (!rb)
@@ -7366,11 +7368,11 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (aux_offset <perf_data_size (rb) +PAGE_SIZE )return -EINVAL ; -if (aux_offset != vma->vm_pgoff <<PAGE_SHIFT ) +if (aux_offset != pgoff_start <<PAGE_SHIFT )return -EINVAL ;/* already mapped with a different offset */ -if (rb_has_aux (rb) && rb->aux_pgoff != vma->vm_pgoff) +if (rb_has_aux (rb) && rb->aux_pgoff != pgoff_start)return -EINVAL ;if (aux_size != nr_pages *PAGE_SIZE )
@@ -7400,7 +7402,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (vma->vm_flags &VM_WRITE ) rb_flags |=RING_BUFFER_WRITABLE ; - ret =rb_alloc_aux (rb, event, vma->vm_pgoff, nr_pages, + ret =rb_alloc_aux (rb, event, pgoff_start, nr_pages, event->attr.aux_watermark, rb_flags);if (ret) {refcount_dec (&rb->mmap_count);
@@ -7457,7 +7459,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)if (event->state <=PERF_EVENT_STATE_REVOKED )return -ENODEV ; -if (vma->vm_pgoff ==0 ) +if (!vma_start_pgoff (vma)) ret =perf_mmap_rb (vma, event, nr_pages);else ret =perf_mmap_aux (vma, event, nr_pages);
@@ -9884,7 +9886,7 @@ static bool perf_addr_filter_vma_adjust(struct perf_addr_filter *filter, struct perf_addr_filter_range *fr) {unsigned long vma_size = vma->vm_end - vma->vm_start; -unsigned long off = vma->vm_pgoff <<PAGE_SHIFT ; +unsigned long off =vma_start_pgoff (vma) <<PAGE_SHIFT ;struct file *file = vma->vm_file;if (!perf_addr_filter_match (filter, file, off, vma_size))
@@ -9974,7 +9976,7 @@ void perf_event_mmap(struct vm_area_struct *vma)/* .tid */ .start = vma->vm_start, .len = vma->vm_end - vma->vm_start, - .pgoff = (u64)vma->vm_pgoff << PAGE_SHIFT, + .pgoff = (u64)vma_start_pgoff(vma) << PAGE_SHIFT, },/* .maj (attr_mmap2 only) */ /* .min (attr_mmap2 only) */
kernel/events/uprobes.c +7 -4
@@ -144,12 +144,14 @@ static bool valid_vma(struct vm_area_struct *vma, bool is_register)static unsigned long offset_to_vaddr (struct vm_area_struct *vma,loff_t offset) { -return vma->vm_start + offset - ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); +return vma->vm_start + offset - + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ); }static loff_t vaddr_to_offset (struct vm_area_struct *vma,unsigned long vaddr) { -return ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ) + (vaddr - vma->vm_start); +return ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (vaddr - vma->vm_start); }/**
@@ -1482,7 +1484,7 @@ static int unapply_uprobe(struct uprobe *uprobe, struct mm_struct *mm) file_inode(vma->vm_file) != uprobe->inode)continue ; - offset = (loff_t )vma->vm_pgoff <<PAGE_SHIFT ; + offset = (loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ;if (uprobe->offset < offset || uprobe->offset >= offset + vma->vm_end - vma->vm_start)continue ;
@@ -2453,7 +2455,8 @@ static struct uprobe *find_active_uprobe_speculative(unsigned long bp_vaddr)if (!vm_file)return NULL ; - offset = (loff_t)(vma->vm_pgoff <<PAGE_SHIFT ) + (bp_vaddr - vma->vm_start); + offset = (loff_t)(vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (bp_vaddr - vma->vm_start); uprobe =find_uprobe_rcu (vm_file->f_inode, offset);if (!uprobe)return NULL ;
kernel/kcov.c +1 -1
@@ -512,7 +512,7 @@ static int kcov_mmap(struct file *filep, struct vm_area_struct *vma)spin_lock_irqsave (&kcov->lock, flags); size = kcov->size *sizeof (unsigned long ); -if (kcov->area ==NULL || vma->vm_pgoff !=0 || +if (kcov->area ==NULL ||vma_start_pgoff (vma) || vma->vm_end - vma->vm_start != size) { res = -EINVAL ; goto exit;
kernel/trace/ring_buffer.c +2 -1
@@ -7613,7 +7613,8 @@ static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer,static int __rb_map_vma (struct ring_buffer_per_cpu *cpu_buffer,struct vm_area_struct *vma) { -unsigned long nr_subbufs, nr_pages, nr_vma_pages, pgoff = vma->vm_pgoff; +unsigned long nr_subbufs, nr_pages, nr_vma_pages; +pgoff_t pgoff =vma_start_pgoff (vma);unsigned int subbuf_pages, subbuf_order; struct page **pages __free(kfree) = NULL;int p =0 , s =0 ;
Signature · 1 line
2.54.0
Re: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
Quoted 146 lines · Lorenzo Stoakes
Be consistent in using
vma_start_pgoff()andvma_end_pgoff(), which clearly indicates which part of the VMA the page offset refers to and aids greppability.This is part of a broader series laying the ground to provide a virtual page offset for MAP_PRIVATE-file backed anon folios.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- kernel/dma/coherent.c | 7 ++++--- kernel/dma/direct.c | 6 ++++-- kernel/dma/mapping.c | 8 +++++--- kernel/dma/ops_helpers.c | 4 ++-- kernel/events/core.c | 20 +++++++++++--------- kernel/events/uprobes.c | 11 +++++++---- kernel/kcov.c | 2 +- kernel/trace/ring_buffer.c | 3 ++- 8 files changed, 36 insertions(+), 25 deletions(-)kernel/dma/coherent.c +4
-3@@ -236,14 +236,15 @@ static int __dma_mmap_from_coherent(struct dma_coherent_mem *mem, {if (mem && vaddr >= mem->virt_base && vaddr + size <= (mem->virt_base + ((dma_addr_t )mem->size <<PAGE_SHIFT ))) { -unsigned long off = vma->vm_pgoff; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int start = (vaddr - mem->virt_base) >>PAGE_SHIFT ;unsigned long user_count =vma_pages (vma);int count =PAGE_ALIGN (size) >>PAGE_SHIFT ; *ret = -ENXIO ; -if (off < count && user_count <= count - off) { -unsigned long pfn = mem->pfn_base + start + off; +if (pgoff_start < count && pgoff_end <= count) { +unsigned long pfn = mem->pfn_base + start + pgoff_start; *ret =remap_pfn_range (vma, vma->vm_start, pfn, user_count <<PAGE_SHIFT , vma->vm_page_prot);kernel/dma/direct.c +4
-2@@ -534,6 +534,8 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ;unsigned long pfn =PHYS_PFN (dma_to_phys (dev, dma_addr)); +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int ret = -ENXIO ; vma->vm_page_prot =dma_pgprot (dev, vma->vm_page_prot, attrs);@@ -545,9 +547,9 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,if (dma_mmap_from_global_coherent (vma, cpu_addr, size, &ret))return ret; -if (vma->vm_pgoff >= count || user_count > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ; -return remap_pfn_range (vma, vma->vm_start, pfn + vma->vm_pgoff, +return remap_pfn_range (vma, vma->vm_start, pfn + pgoff_start, user_count <<PAGE_SHIFT , vma->vm_page_prot); }kernel/dma/mapping.c +5
-3@@ -761,12 +761,14 @@ EXPORT_SYMBOL_GPL(dma_free_pages);int dma_mmap_pages (struct device *dev,struct vm_area_struct *vma,size_t size,struct page *page) { -unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma); +const unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -if (vma->vm_pgoff >= count ||vma_pages (vma) > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) + pgoff_start,vma_pages (vma) <<PAGE_SHIFT , vma->vm_page_prot); }EXPORT_SYMBOL_GPL (dma_mmap_pages);kernel/dma/ops_helpers.c +2
-2@@ -39,7 +39,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,#ifdef CONFIG_MMU unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; +unsigned long off =vma_start_pgoff (vma);struct page *page =dma_common_vaddr_to_page (cpu_addr);int ret = -ENXIO ;@@ -52,7 +52,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) +vma_start_pgoff (vma), user_count <<PAGE_SHIFT , vma->vm_page_prot);#else return -ENXIO ;kernel/events/core.c +4
-3@@ -6998,7 +6998,7 @@ static void perf_mmap_open(struct vm_area_struct *vma)refcount_inc (&event->mmap_count);refcount_inc (&event->rb->mmap_count); -if (vma->vm_pgoff) +if (vma_start_pgoff (vma))refcount_inc (&event->rb->aux_mmap_count);if (mapped)@@ -7032,7 +7032,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)* The AUX buffer is strictly a sub-buffer, serialize using aux_mutex * to avoid complications. */ -if (rb_has_aux (rb) && vma->vm_pgoff == rb->aux_pgoff && +if (rb_has_aux (rb) &&vma_start_pgoff (vma) == rb->aux_pgoff &&refcount_dec_and_mutex_lock (&rb->aux_mmap_count, &rb->aux_mutex)) {/* * Stop all AUX events that are writing to this buffer, @@ -7190,7 +7190,8 @@ static int map_range(struct perf_buffer *rb, struct vm_area_struct *vma)*/ for (pagenum =0 ; pagenum < nr_pages; pagenum++) {unsigned long va = vma->vm_start +PAGE_SIZE * pagenum; -struct page *page =perf_mmap_to_page (rb, vma->vm_pgoff + pagenum); +struct page *page =perf_mmap_to_page (rb, +vma_start_pgoff (vma) + pagenum);if (page ==NULL ) { err = -EINVAL ;
kernel/events/core.c static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event, · line 7352
Quoted 4 lines
kernel/events/core.c +1
-0@@ -7348,6 +7349,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,u64 aux_offset, aux_size;struct perf_buffer *rb;int ret, rb_flags =0 ; +const pgoff_t pgoff_start =vma_start_pgoff (vma);
Variable decs here seem to be in reverse christmas tree order, so perhaps move this to the top.
Quoted 62 lines
kernel/events/core.c +6
-6rb = event->rb;if (!rb)@@ -7366,11 +7368,11 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (aux_offset <perf_data_size (rb) +PAGE_SIZE )return -EINVAL ; -if (aux_offset != vma->vm_pgoff <<PAGE_SHIFT ) +if (aux_offset != pgoff_start <<PAGE_SHIFT )return -EINVAL ;/* already mapped with a different offset */ -if (rb_has_aux (rb) && rb->aux_pgoff != vma->vm_pgoff) +if (rb_has_aux (rb) && rb->aux_pgoff != pgoff_start)return -EINVAL ;if (aux_size != nr_pages *PAGE_SIZE )@@ -7400,7 +7402,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (vma->vm_flags &VM_WRITE ) rb_flags |=RING_BUFFER_WRITABLE ; - ret =rb_alloc_aux (rb, event, vma->vm_pgoff, nr_pages, + ret =rb_alloc_aux (rb, event, pgoff_start, nr_pages, event->attr.aux_watermark, rb_flags);if (ret) {refcount_dec (&rb->mmap_count);@@ -7457,7 +7459,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)if (event->state <=PERF_EVENT_STATE_REVOKED )return -ENODEV ; -if (vma->vm_pgoff ==0 ) +if (!vma_start_pgoff (vma)) ret =perf_mmap_rb (vma, event, nr_pages);else ret =perf_mmap_aux (vma, event, nr_pages);@@ -9884,7 +9886,7 @@ static bool perf_addr_filter_vma_adjust(struct perf_addr_filter *filter, struct perf_addr_filter_range *fr) {unsigned long vma_size = vma->vm_end - vma->vm_start; -unsigned long off = vma->vm_pgoff <<PAGE_SHIFT ; +unsigned long off =vma_start_pgoff (vma) <<PAGE_SHIFT ;struct file *file = vma->vm_file;if (!perf_addr_filter_match (filter, file, off, vma_size))@@ -9974,7 +9976,7 @@ void perf_event_mmap(struct vm_area_struct *vma)/* .tid */ .start = vma->vm_start, .len = vma->vm_end - vma->vm_start, - .pgoff = (u64)vma->vm_pgoff << PAGE_SHIFT, + .pgoff = (u64)vma_start_pgoff(vma) << PAGE_SHIFT, },/* .maj (attr_mmap2 only) */ /* .min (attr_mmap2 only) */ kernel/events/uprobes.c +0
-1@@ -144,12 +144,14 @@ static bool valid_vma(struct vm_area_struct *vma, bool is_register)static unsigned long offset_to_vaddr (struct vm_area_struct *vma,loff_t offset) { -return vma->vm_start + offset - ((loff_t )vma->vm_pgoff <<PAGE_SHIFT );
kernel/events/uprobes.c static bool valid_vma(struct vm_area_struct *vma, bool is_register) · line 155
Quoted 10 lines
kernel/events/uprobes.c +4
-1+return vma->vm_start + offset - + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ); }static loff_t vaddr_to_offset (struct vm_area_struct *vma,unsigned long vaddr) { -return ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ) + (vaddr - vma->vm_start); +return ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (vaddr - vma->vm_start); }
Something we've seen in this series is that perhaps something like
static inline loff_t vma_start_off(vma)
{
return ((loff_t) vma_start_pgoff(vma)) << PAGE_SHIFT;
}could be worth it.
Quoted 12 lines
kernel/events/uprobes.c +1
-1/** @@ -1482,7 +1484,7 @@ static int unapply_uprobe(struct uprobe *uprobe, struct mm_struct *mm) file_inode(vma->vm_file) != uprobe->inode)continue ; - offset = (loff_t )vma->vm_pgoff <<PAGE_SHIFT ; + offset = (loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ;if (uprobe->offset < offset || uprobe->offset >= offset + vma->vm_end - vma->vm_start)continue ;
kernel/events/uprobes.c static struct uprobe *find_active_uprobe_speculative(unsigned long bp_vaddr) · line 2459
Quoted 6 lines
kernel/events/uprobes.c +2
-1@@ -2453,7 +2455,8 @@ static struct uprobe *find_active_uprobe_speculative(unsigned long bp_vaddr)if (!vm_file)return NULL ; - offset = (loff_t)(vma->vm_pgoff <<PAGE_SHIFT ) + (bp_vaddr - vma->vm_start); + offset = (loff_t)(vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (bp_vaddr - vma->vm_start);
This is more extremely contrived logic that could be better expressed as
loff_t vma_linear_off(vma, bp_vaddr);
kernel/kcov.c static int kcov_mmap(struct file *filep, struct vm_area_struct *vma) · line 515
Quoted 13 lines
kernel/events/uprobes.c +0
-0uprobe =find_uprobe_rcu (vm_file->f_inode, offset);if (!uprobe)return NULL ;kernel/kcov.c +1
-1@@ -512,7 +512,7 @@ static int kcov_mmap(struct file *filep, struct vm_area_struct *vma)spin_lock_irqsave (&kcov->lock, flags); size = kcov->size *sizeof (unsigned long ); - if (kcov->area == NULL || vma->vm_pgoff != 0 || + if (kcov->area == NULL || vma_start_pgoff(vma) ||
as a nit, perhaps vma_start_pgoff(vma) > 0
would be a little more idiomatic.
Quoted 8 lines
kernel/kcov.c +0
-0vma->vm_end - vma->vm_start != size) { res = -EINVAL ; goto exit;kernel/trace/ring_buffer.c +0
-0
kernel/trace/ring_buffer.c static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer, · line 7620
Quoted 9 lines
kernel/trace/ring_buffer.c +2
-1@@ -7613,7 +7613,8 @@ static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer,static int __rb_map_vma (struct ring_buffer_per_cpu *cpu_buffer,struct vm_area_struct *vma) { -unsigned long nr_subbufs, nr_pages, nr_vma_pages, pgoff = vma->vm_pgoff; +unsigned long nr_subbufs, nr_pages, nr_vma_pages; +pgoff_t pgoff =vma_start_pgoff (vma);unsigned int subbuf_pages, subbuf_order; struct page **pages __free(kfree) = NULL;int p =0 , s =0 ;
Anyway, in general:
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
Quoted 144 lines · Pedro Falcato
Quoted 143 lines · Lorenzo Stoakes
Be consistent in using
vma_start_pgoff()andvma_end_pgoff(), which clearly indicates which part of the VMA the page offset refers to and aids greppability.This is part of a broader series laying the ground to provide a virtual page offset for MAP_PRIVATE-file backed anon folios.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- kernel/dma/coherent.c | 7 ++++--- kernel/dma/direct.c | 6 ++++-- kernel/dma/mapping.c | 8 +++++--- kernel/dma/ops_helpers.c | 4 ++-- kernel/events/core.c | 20 +++++++++++--------- kernel/events/uprobes.c | 11 +++++++---- kernel/kcov.c | 2 +- kernel/trace/ring_buffer.c | 3 ++- 8 files changed, 36 insertions(+), 25 deletions(-)kernel/dma/coherent.c +4
-3@@ -236,14 +236,15 @@ static int __dma_mmap_from_coherent(struct dma_coherent_mem *mem, {if (mem && vaddr >= mem->virt_base && vaddr + size <= (mem->virt_base + ((dma_addr_t )mem->size <<PAGE_SHIFT ))) { -unsigned long off = vma->vm_pgoff; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int start = (vaddr - mem->virt_base) >>PAGE_SHIFT ;unsigned long user_count =vma_pages (vma);int count =PAGE_ALIGN (size) >>PAGE_SHIFT ; *ret = -ENXIO ; -if (off < count && user_count <= count - off) { -unsigned long pfn = mem->pfn_base + start + off; +if (pgoff_start < count && pgoff_end <= count) { +unsigned long pfn = mem->pfn_base + start + pgoff_start; *ret =remap_pfn_range (vma, vma->vm_start, pfn, user_count <<PAGE_SHIFT , vma->vm_page_prot);kernel/dma/direct.c +4
-2@@ -534,6 +534,8 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ;unsigned long pfn =PHYS_PFN (dma_to_phys (dev, dma_addr)); +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int ret = -ENXIO ; vma->vm_page_prot =dma_pgprot (dev, vma->vm_page_prot, attrs);@@ -545,9 +547,9 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,if (dma_mmap_from_global_coherent (vma, cpu_addr, size, &ret))return ret; -if (vma->vm_pgoff >= count || user_count > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ; -return remap_pfn_range (vma, vma->vm_start, pfn + vma->vm_pgoff, +return remap_pfn_range (vma, vma->vm_start, pfn + pgoff_start, user_count <<PAGE_SHIFT , vma->vm_page_prot); }kernel/dma/mapping.c +5
-3@@ -761,12 +761,14 @@ EXPORT_SYMBOL_GPL(dma_free_pages);int dma_mmap_pages (struct device *dev,struct vm_area_struct *vma,size_t size,struct page *page) { -unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma); +const unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -if (vma->vm_pgoff >= count ||vma_pages (vma) > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) + pgoff_start,vma_pages (vma) <<PAGE_SHIFT , vma->vm_page_prot); }EXPORT_SYMBOL_GPL (dma_mmap_pages);kernel/dma/ops_helpers.c +2
-2@@ -39,7 +39,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,#ifdef CONFIG_MMU unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; +unsigned long off =vma_start_pgoff (vma);struct page *page =dma_common_vaddr_to_page (cpu_addr);int ret = -ENXIO ;@@ -52,7 +52,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) +vma_start_pgoff (vma), user_count <<PAGE_SHIFT , vma->vm_page_prot);#else return -ENXIO ;kernel/events/core.c +4
-3@@ -6998,7 +6998,7 @@ static void perf_mmap_open(struct vm_area_struct *vma)refcount_inc (&event->mmap_count);refcount_inc (&event->rb->mmap_count); -if (vma->vm_pgoff) +if (vma_start_pgoff (vma))refcount_inc (&event->rb->aux_mmap_count);if (mapped)@@ -7032,7 +7032,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)* The AUX buffer is strictly a sub-buffer, serialize using aux_mutex * to avoid complications. */ -if (rb_has_aux (rb) && vma->vm_pgoff == rb->aux_pgoff && +if (rb_has_aux (rb) &&vma_start_pgoff (vma) == rb->aux_pgoff &&refcount_dec_and_mutex_lock (&rb->aux_mmap_count, &rb->aux_mutex)) {/* * Stop all AUX events that are writing to this buffer, @@ -7190,7 +7190,8 @@ static int map_range(struct perf_buffer *rb, struct vm_area_struct *vma)*/ for (pagenum =0 ; pagenum < nr_pages; pagenum++) {unsigned long va = vma->vm_start +PAGE_SIZE * pagenum; -struct page *page =perf_mmap_to_page (rb, vma->vm_pgoff + pagenum); +struct page *page =perf_mmap_to_page (rb, +vma_start_pgoff (vma) + pagenum);
Quoted 10 lines
kernel/events/core.c static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event, · line 7352
Quoted 7 lines
kernel/events/core.c +1
-0if (page ==NULL ) { err = -EINVAL ;@@ -7348,6 +7349,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,u64 aux_offset, aux_size;struct perf_buffer *rb;int ret, rb_flags =0 ; +const pgoff_t pgoff_start =vma_start_pgoff (vma);Variable decs here seem to be in reverse christmas tree order, so perhaps move this to the top.
Ack will change on respin.
Quoted 76 lines
Quoted 63 lines
kernel/events/core.c +6
-6rb = event->rb;if (!rb)@@ -7366,11 +7368,11 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (aux_offset <perf_data_size (rb) +PAGE_SIZE )return -EINVAL ; -if (aux_offset != vma->vm_pgoff <<PAGE_SHIFT ) +if (aux_offset != pgoff_start <<PAGE_SHIFT )return -EINVAL ;/* already mapped with a different offset */ -if (rb_has_aux (rb) && rb->aux_pgoff != vma->vm_pgoff) +if (rb_has_aux (rb) && rb->aux_pgoff != pgoff_start)return -EINVAL ;if (aux_size != nr_pages *PAGE_SIZE )@@ -7400,7 +7402,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (vma->vm_flags &VM_WRITE ) rb_flags |=RING_BUFFER_WRITABLE ; - ret =rb_alloc_aux (rb, event, vma->vm_pgoff, nr_pages, + ret =rb_alloc_aux (rb, event, pgoff_start, nr_pages, event->attr.aux_watermark, rb_flags);if (ret) {refcount_dec (&rb->mmap_count);@@ -7457,7 +7459,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)if (event->state <=PERF_EVENT_STATE_REVOKED )return -ENODEV ; -if (vma->vm_pgoff ==0 ) +if (!vma_start_pgoff (vma)) ret =perf_mmap_rb (vma, event, nr_pages);else ret =perf_mmap_aux (vma, event, nr_pages);@@ -9884,7 +9886,7 @@ static bool perf_addr_filter_vma_adjust(struct perf_addr_filter *filter, struct perf_addr_filter_range *fr) {unsigned long vma_size = vma->vm_end - vma->vm_start; -unsigned long off = vma->vm_pgoff <<PAGE_SHIFT ; +unsigned long off =vma_start_pgoff (vma) <<PAGE_SHIFT ;struct file *file = vma->vm_file;if (!perf_addr_filter_match (filter, file, off, vma_size))@@ -9974,7 +9976,7 @@ void perf_event_mmap(struct vm_area_struct *vma)/* .tid */ .start = vma->vm_start, .len = vma->vm_end - vma->vm_start, - .pgoff = (u64)vma->vm_pgoff << PAGE_SHIFT, + .pgoff = (u64)vma_start_pgoff(vma) << PAGE_SHIFT, },/* .maj (attr_mmap2 only) */ /* .min (attr_mmap2 only) */ kernel/events/uprobes.c +0
-1@@ -144,12 +144,14 @@ static bool valid_vma(struct vm_area_struct *vma, bool is_register)static unsigned long offset_to_vaddr (struct vm_area_struct *vma,loff_t offset) { -return vma->vm_start + offset - ((loff_t )vma->vm_pgoff <<PAGE_SHIFT );kernel/events/uprobes.c static bool valid_vma(struct vm_area_struct *vma, bool is_register) · line 155
Quoted 10 lines
kernel/events/uprobes.c +4
-1+return vma->vm_start + offset - + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ); }static loff_t vaddr_to_offset (struct vm_area_struct *vma,unsigned long vaddr) { -return ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ) + (vaddr - vma->vm_start); +return ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (vaddr - vma->vm_start); }Something we've seen in this series is that perhaps something like
Quoted 6 lines
static inline
loff_tvma_start_off(vma){ return ((loff_t) vma_start_pgoff(vma)) << PAGE_SHIFT; }could be worth it.
Yeah I already thought about this kind of thing, but in the end decided against it at least for now, maybe I could revisit with something like that added though...
But given this series is being held off on already for acceptance, I'm not sure adding _yet more_ changes will be welcomed.
Plus I worry people will get confused.
There's 2 forms I noticed:
1. effectively: linear_page_index(vma, address) << PAGE_SHIFT
I wanted to rewrite this as such, but you can't, because of course vaddr might have non-page aligned bits (or tags) that you lose by doing that. So ugh.
You'd then need to write a slightly nuanced version like:
static inline unsigned long vma_offset(const struct vm_area_struct *vma,
const unsigned long address)
{
/* Retains page offset and tags. */
return address - vma->vm_start;
}static inline unsigned long linear_page_offset(const struct vm_area_struct *vma,
const unsigned long address)
{
const unsigned long addr = vma_start_pgoff(vma) << PAGE_SHIFT;
addr += linear_delta(vma, address);
return addr;
}(Could also do:)
static inline pgoff_t linear_page_delta(const struct vm_area_struct *vma,
const unsigned long address)
{
return vma_offset(vma, address) >> PAGE_SHIFT;
}BUT.
I think this will confuse people. I already in a previous version of this series named linear_page_delta() as linear_page_offset() and then changed it to avoid confusion.
And I'm not sure it's really all that useful. Perhaps retaining vma_offset() would be though.
2.
This is a much more useful form I noticed, effectively drivers doing the inverse of a linear_page_index() to get the address:
static inline unsigned long linear_page_address(const struct vm_area_struct *vma,
const pgoff_t pgoff)
{
const pgoff_t page_delta = pgoff - vma_start_pgoff(vma);
const unsigned long offset = page_delta << PAGE_SHIFT;
return vma->vm_start + offset;
}This is one that I think makes more sense.
But in general, I'd rather hold off from yet more churn here.
I'm making these changes to establish a basis for virtual page offsets introduced in [0], rather than just cleaning up in general.
These changes are really to make it such that we more consistently use these forms, so when I introduce the virt pgoff versions, it fits canonically into that.
And also beacuse I may as well improve kernel code as I go :)
But I think adding yet more doesn't really serve the same purpose.
But it's food for a follow up perhaps?
Quoted 16 lines
Quoted 16 lines
kernel/events/uprobes.c +1
-1/** @@ -1482,7 +1484,7 @@ static int unapply_uprobe(struct uprobe *uprobe, struct mm_struct *mm) file_inode(vma->vm_file) != uprobe->inode)continue ; - offset = (loff_t )vma->vm_pgoff <<PAGE_SHIFT ; + offset = (loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ;if (uprobe->offset < offset || uprobe->offset >= offset + vma->vm_end - vma->vm_start)continue ;@@ -2453,7 +2455,8 @@ static struct uprobe *find_active_uprobe_speculative(unsigned long bp_vaddr)if (!vm_file)return NULL ;
Quoted 7 lines
kernel/events/uprobes.c static struct uprobe *find_active_uprobe_speculative(unsigned long bp_vaddr) · line 2459
Quoted 3 lines
kernel/events/uprobes.c +2
-1- offset = (loff_t)(vma->vm_pgoff <<PAGE_SHIFT ) + (bp_vaddr - vma->vm_start); + offset = (loff_t)(vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (bp_vaddr - vma->vm_start);This is more extremely contrived logic that could be better expressed as
loff_tvma_linear_off(vma, bp_vaddr);
See above.
Quoted 10 lines
Quoted 10 lines
kernel/events/uprobes.c +0
-0uprobe =find_uprobe_rcu (vm_file->f_inode, offset);if (!uprobe)return NULL ;kernel/kcov.c +0
-0@@ -512,7 +512,7 @@ static int kcov_mmap(struct file *filep, struct vm_area_struct *vma)
Quoted 7 lines
kernel/kcov.c static int kcov_mmap(struct file *filep, struct vm_area_struct *vma) · line 515
Quoted 4 lines
kernel/kcov.c +1
-1spin_lock_irqsave (&kcov->lock, flags); size = kcov->size *sizeof (unsigned long ); - if (kcov->area == NULL || vma->vm_pgoff != 0 || + if (kcov->area == NULL || vma_start_pgoff(vma) ||as a nit, perhaps
vma_start_pgoff(vma)> 0
would be a little more idiomatic.
I felt the if (<val>) form was more idiomatic?
Quoted 21 lines
Quoted 9 lines
kernel/kcov.c +0
-0vma->vm_end - vma->vm_start != size) { res = -EINVAL ; goto exit;kernel/trace/ring_buffer.c +0
-0kernel/trace/ring_buffer.c static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer, · line 7620
Quoted 9 lines
kernel/trace/ring_buffer.c +2
-1@@ -7613,7 +7613,8 @@ static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer,static int __rb_map_vma (struct ring_buffer_per_cpu *cpu_buffer,struct vm_area_struct *vma) { -unsigned long nr_subbufs, nr_pages, nr_vma_pages, pgoff = vma->vm_pgoff; +unsigned long nr_subbufs, nr_pages, nr_vma_pages; +pgoff_t pgoff =vma_start_pgoff (vma);unsigned int subbuf_pages, subbuf_order; struct page **pages __free(kfree) = NULL;int p =0 , s =0 ;Anyway, in general:
Quoted 1 line
Acked-by Pedro Falcato <[email protected]>
Thanks!
Quoted 3 lines
--
Pedro
Gregory Price
maintainerRe: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
Quoted 1 line · Lorenzo Stoakes
...
Quoted 7 lines
static inline unsigned long vma_offset(const struct
vm_area_struct*vma,
const unsigned long address){ /* Retains page offset and tags. */ return address - vma->vm_start; }
...
Quoted 3 lines
And I'm not sure it's really all that useful. Perhaps retaining
vma_offset()would be though.
Silly question:
What's the purpose of retaining tags in a non-address value?
That sounds like there's fragility just waiting to be broken.
(I presume you are talking about things like ARM MTE and such, right?)
Quoted 7 lines
This is one that I think makes more sense.
But in general, I'd rather hold off from yet more churn here.
I'm making these changes to establish a basis for virtual page offsets introduced in [0], rather than just cleaning up in general.
I agree with this. If the refactors here suddenly have to think about corner cases on things like tags, that's better resolved separately.
~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
Quoted 10 lines · Gregory Price
Quoted 1 line · Lorenzo Stoakes
...
Quoted 7 lines
static inline unsigned long vma_offset(const struct
vm_area_struct*vma,
const unsigned long address){ /* Retains page offset and tags. */ return address - vma->vm_start; }
Quoted 6 lines
...
Quoted 3 lines
And I'm not sure it's really all that useful. Perhaps retaining
vma_offset()would be though.Silly question:
There's no such thing :)
Quoted 2 lines
What's the purpose of retaining tags in a non-address value?
Well if you want to reconstitute an address from it later then that'd be the intent, but I'm being hand wavey here for sure.
Main thing is to retain stuff under the page mask
Quoted 7 lines
That sounds like there's fragility just waiting to be broken.
(I presume you are talking about things like ARM MTE and such, right?)
Quoted 2 lines
This is one that I think makes more sense.
Quoted 8 lines
Quoted 5 lines
But in general, I'd rather hold off from yet more churn here.
I'm making these changes to establish a basis for virtual page offsets introduced in [0], rather than just cleaning up in general.
I agree with this. If the refactors here suddenly have to think about corner cases on things like tags, that's better resolved separately.
Yeah this series is huge enough as it is :P
Quoted 2 lines
~Gregory
Cheers, Lorenzo
Gregory Price
maintainerRe: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
Quoted 6 lines · Lorenzo Stoakes
Quoted 2 lines · Gregory Price
Silly question:
There's no such thing :)
All my questions are silly. That's my secret :]
Quoted 8 lines
Quoted 2 lines
What's the purpose of retaining tags in a non-address value?
Well if you want to reconstitute an address from it later then that'd be the intent, but I'm being hand wavey here for sure.
Main thing is to retain stuff under the page mask
Right, which ...
vvvvvv
Quoted 1 line
Quoted 1 line
That sounds like there's fragility just waiting to be broken.
If that's an actual issue, then our relationship with tags is painful :[
~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
Quoted 9 lines · Gregory Price
Quoted 6 lines · Lorenzo Stoakes
Quoted 2 lines · Gregory Price
Silly question:
There's no such thing :)
All my questions are silly. That's my secret :]
Mine too ;)
Quoted 10 lines
Quoted 9 lines
Quoted 3 lines
What's the purpose of retaining tags in a non-address value?
Well if you want to reconstitute an address from it later then that'd be the intent, but I'm being hand wavey here for sure.
Main thing is to retain stuff under the page mask
Quoted 6 lines
Right, which ...
vvvvvv
Quoted 1 line
Quoted 1 line
That sounds like there's fragility just waiting to be broken.
If that's an actual issue, then our relationship with tags is painful :[
Yeah, I'm just not sure tbh.
But all that is stuf for future masochism^Wfun anyway ;) focusing on the 'respin the monster series' fun^Wmasochism atm...
Quoted 2 lines
~Gregory
Cheers, Lorenzo
Re: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
In reply to Lorenzo StoakesQuoted 7 lines · Lorenzo Stoakes
Be consistent in using
vma_start_pgoff()andvma_end_pgoff(), which clearly indicates which part of the VMA the page offset refers to and aids greppability.This is part of a broader series laying the ground to provide a virtual page offset for MAP_PRIVATE-file backed anon folios.
Quoted 8 lines
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- kernel/dma/coherent.c | 7 ++++--- kernel/dma/direct.c | 6 ++++-- kernel/dma/mapping.c | 8 +++++--- kernel/dma/ops_helpers.c | 4 ++--
Quoted 246 lines
kernel/events/core.c | 20 +++++++++++--------- kernel/events/uprobes.c | 11 +++++++---- kernel/kcov.c | 2 +- kernel/trace/ring_buffer.c | 3 ++- 8 files changed, 36 insertions(+), 25 deletions(-)kernel/dma/coherent.c +4
-3@@ -236,14 +236,15 @@ static int __dma_mmap_from_coherent(struct dma_coherent_mem *mem, {if (mem && vaddr >= mem->virt_base && vaddr + size <= (mem->virt_base + ((dma_addr_t )mem->size <<PAGE_SHIFT ))) { -unsigned long off = vma->vm_pgoff; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int start = (vaddr - mem->virt_base) >>PAGE_SHIFT ;unsigned long user_count =vma_pages (vma);int count =PAGE_ALIGN (size) >>PAGE_SHIFT ; *ret = -ENXIO ; -if (off < count && user_count <= count - off) { -unsigned long pfn = mem->pfn_base + start + off; +if (pgoff_start < count && pgoff_end <= count) { +unsigned long pfn = mem->pfn_base + start + pgoff_start; *ret =remap_pfn_range (vma, vma->vm_start, pfn, user_count <<PAGE_SHIFT , vma->vm_page_prot);kernel/dma/direct.c +4
-2@@ -534,6 +534,8 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ;unsigned long pfn =PHYS_PFN (dma_to_phys (dev, dma_addr)); +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma);int ret = -ENXIO ; vma->vm_page_prot =dma_pgprot (dev, vma->vm_page_prot, attrs);@@ -545,9 +547,9 @@ int dma_direct_mmap(struct device *dev, struct vm_area_struct *vma,if (dma_mmap_from_global_coherent (vma, cpu_addr, size, &ret))return ret; -if (vma->vm_pgoff >= count || user_count > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ; -return remap_pfn_range (vma, vma->vm_start, pfn + vma->vm_pgoff, +return remap_pfn_range (vma, vma->vm_start, pfn + pgoff_start, user_count <<PAGE_SHIFT , vma->vm_page_prot); }kernel/dma/mapping.c +5
-3@@ -761,12 +761,14 @@ EXPORT_SYMBOL_GPL(dma_free_pages);int dma_mmap_pages (struct device *dev,struct vm_area_struct *vma,size_t size,struct page *page) { -unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; +const pgoff_t pgoff_start =vma_start_pgoff (vma); +const pgoff_t pgoff_end =vma_end_pgoff (vma); +const unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -if (vma->vm_pgoff >= count ||vma_pages (vma) > count - vma->vm_pgoff) +if (pgoff_start >= count || pgoff_end > count)return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) + pgoff_start,vma_pages (vma) <<PAGE_SHIFT , vma->vm_page_prot); }EXPORT_SYMBOL_GPL (dma_mmap_pages);kernel/dma/ops_helpers.c +2
-2@@ -39,7 +39,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,#ifdef CONFIG_MMU unsigned long user_count =vma_pages (vma);unsigned long count =PAGE_ALIGN (size) >>PAGE_SHIFT ; -unsigned long off = vma->vm_pgoff; +unsigned long off =vma_start_pgoff (vma);struct page *page =dma_common_vaddr_to_page (cpu_addr);int ret = -ENXIO ;@@ -52,7 +52,7 @@ int dma_common_mmap(struct device *dev, struct vm_area_struct *vma,return -ENXIO ;return remap_pfn_range (vma, vma->vm_start, -page_to_pfn (page) + vma->vm_pgoff, +page_to_pfn (page) +vma_start_pgoff (vma), user_count <<PAGE_SHIFT , vma->vm_page_prot);#else return -ENXIO ;kernel/events/core.c +11
-9@@ -6998,7 +6998,7 @@ static void perf_mmap_open(struct vm_area_struct *vma)refcount_inc (&event->mmap_count);refcount_inc (&event->rb->mmap_count); -if (vma->vm_pgoff) +if (vma_start_pgoff (vma))refcount_inc (&event->rb->aux_mmap_count);if (mapped)@@ -7032,7 +7032,7 @@ static void perf_mmap_close(struct vm_area_struct *vma)* The AUX buffer is strictly a sub-buffer, serialize using aux_mutex * to avoid complications. */ -if (rb_has_aux (rb) && vma->vm_pgoff == rb->aux_pgoff && +if (rb_has_aux (rb) &&vma_start_pgoff (vma) == rb->aux_pgoff &&refcount_dec_and_mutex_lock (&rb->aux_mmap_count, &rb->aux_mutex)) {/* * Stop all AUX events that are writing to this buffer, @@ -7190,7 +7190,8 @@ static int map_range(struct perf_buffer *rb, struct vm_area_struct *vma)*/ for (pagenum =0 ; pagenum < nr_pages; pagenum++) {unsigned long va = vma->vm_start +PAGE_SIZE * pagenum; -struct page *page =perf_mmap_to_page (rb, vma->vm_pgoff + pagenum); +struct page *page =perf_mmap_to_page (rb, +vma_start_pgoff (vma) + pagenum);if (page ==NULL ) { err = -EINVAL ;@@ -7348,6 +7349,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,u64 aux_offset, aux_size;struct perf_buffer *rb;int ret, rb_flags =0 ; +const pgoff_t pgoff_start =vma_start_pgoff (vma); rb = event->rb;if (!rb)@@ -7366,11 +7368,11 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (aux_offset <perf_data_size (rb) +PAGE_SIZE )return -EINVAL ; -if (aux_offset != vma->vm_pgoff <<PAGE_SHIFT ) +if (aux_offset != pgoff_start <<PAGE_SHIFT )return -EINVAL ;/* already mapped with a different offset */ -if (rb_has_aux (rb) && rb->aux_pgoff != vma->vm_pgoff) +if (rb_has_aux (rb) && rb->aux_pgoff != pgoff_start)return -EINVAL ;if (aux_size != nr_pages *PAGE_SIZE )@@ -7400,7 +7402,7 @@ static int perf_mmap_aux(struct vm_area_struct *vma, struct perf_event *event,if (vma->vm_flags &VM_WRITE ) rb_flags |=RING_BUFFER_WRITABLE ; - ret =rb_alloc_aux (rb, event, vma->vm_pgoff, nr_pages, + ret =rb_alloc_aux (rb, event, pgoff_start, nr_pages, event->attr.aux_watermark, rb_flags);if (ret) {refcount_dec (&rb->mmap_count);@@ -7457,7 +7459,7 @@ static int perf_mmap(struct file *file, struct vm_area_struct *vma)if (event->state <=PERF_EVENT_STATE_REVOKED )return -ENODEV ; -if (vma->vm_pgoff ==0 ) +if (!vma_start_pgoff (vma)) ret =perf_mmap_rb (vma, event, nr_pages);else ret =perf_mmap_aux (vma, event, nr_pages);@@ -9884,7 +9886,7 @@ static bool perf_addr_filter_vma_adjust(struct perf_addr_filter *filter, struct perf_addr_filter_range *fr) {unsigned long vma_size = vma->vm_end - vma->vm_start; -unsigned long off = vma->vm_pgoff <<PAGE_SHIFT ; +unsigned long off =vma_start_pgoff (vma) <<PAGE_SHIFT ;struct file *file = vma->vm_file;if (!perf_addr_filter_match (filter, file, off, vma_size))@@ -9974,7 +9976,7 @@ void perf_event_mmap(struct vm_area_struct *vma)/* .tid */ .start = vma->vm_start, .len = vma->vm_end - vma->vm_start, - .pgoff = (u64)vma->vm_pgoff << PAGE_SHIFT, + .pgoff = (u64)vma_start_pgoff(vma) << PAGE_SHIFT, },/* .maj (attr_mmap2 only) */ /* .min (attr_mmap2 only) */ kernel/events/uprobes.c +7
-4@@ -144,12 +144,14 @@ static bool valid_vma(struct vm_area_struct *vma, bool is_register)static unsigned long offset_to_vaddr (struct vm_area_struct *vma,loff_t offset) { -return vma->vm_start + offset - ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ); +return vma->vm_start + offset - + ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ); }static loff_t vaddr_to_offset (struct vm_area_struct *vma,unsigned long vaddr) { -return ((loff_t )vma->vm_pgoff <<PAGE_SHIFT ) + (vaddr - vma->vm_start); +return ((loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (vaddr - vma->vm_start); }/** @@ -1482,7 +1484,7 @@ static int unapply_uprobe(struct uprobe *uprobe, struct mm_struct *mm) file_inode(vma->vm_file) != uprobe->inode)continue ; - offset = (loff_t )vma->vm_pgoff <<PAGE_SHIFT ; + offset = (loff_t )vma_start_pgoff (vma) <<PAGE_SHIFT ;if (uprobe->offset < offset || uprobe->offset >= offset + vma->vm_end - vma->vm_start)continue ;@@ -2453,7 +2455,8 @@ static struct uprobe *find_active_uprobe_speculative(unsigned long bp_vaddr)if (!vm_file)return NULL ; - offset = (loff_t)(vma->vm_pgoff <<PAGE_SHIFT ) + (bp_vaddr - vma->vm_start); + offset = (loff_t)(vma_start_pgoff (vma) <<PAGE_SHIFT ) + + (bp_vaddr - vma->vm_start); uprobe =find_uprobe_rcu (vm_file->f_inode, offset);if (!uprobe)return NULL ;kernel/kcov.c +1
-1@@ -512,7 +512,7 @@ static int kcov_mmap(struct file *filep, struct vm_area_struct *vma)spin_lock_irqsave (&kcov->lock, flags); size = kcov->size *sizeof (unsigned long ); -if (kcov->area ==NULL || vma->vm_pgoff !=0 || +if (kcov->area ==NULL ||vma_start_pgoff (vma) || vma->vm_end - vma->vm_start != size) { res = -EINVAL ; goto exit;kernel/trace/ring_buffer.c +0
-0
kernel/trace/ring_buffer.c static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer, · line 7620
Quoted 9 lines
kernel/trace/ring_buffer.c +2
-1@@ -7613,7 +7613,8 @@ static int __rb_inc_dec_mapped(struct ring_buffer_per_cpu *cpu_buffer,static int __rb_map_vma (struct ring_buffer_per_cpu *cpu_buffer,struct vm_area_struct *vma) { -unsigned long nr_subbufs, nr_pages, nr_vma_pages, pgoff = vma->vm_pgoff; +unsigned long nr_subbufs, nr_pages, nr_vma_pages; +pgoff_t pgoff =vma_start_pgoff (vma);unsigned int subbuf_pages, subbuf_order; struct page **pages __free(kfree) = NULL;int p =0 , s =0 ;
Best regards
Signature · 2 lines
Marek Szyprowski, PhD Samsung R&D Institute Poland
Re: [PATCH 17/30] mm: prefer vma_[start,end]_pgoff() to vma->vm_pgoff in kernel/
In reply to Lorenzo StoakesQuoted 10 lines · Lorenzo Stoakes
Be consistent in using
vma_start_pgoff()andvma_end_pgoff(), which clearly indicates which part of the VMA the page offset refers to and aids greppability.This is part of a broader series laying the ground to provide a virtual page offset for MAP_PRIVATE-file backed anon folios.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 18/30 mm/vma: remove duplicative vma_pgoff_offset() helper
In reply to Lorenzo StoakesThis is doing what linear_page_index() does, so eliminate it and replace it with linear_page_index().
Update the VMA userland tests to reflect this change.
No functional change intended.
--- mm/vma.h | 9 +-------- tools/testing/vma/include/dup.h | 16 ++++++++++++++++ 2 files changed, 17 insertions(+), 8 deletions(-)
mm/vma.h +1 -8
@@ -247,13 +247,6 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)return vmg_start_pgoff (vmg) +vmg_pages (vmg); } -/* Assumes addr >= vma->vm_start. */ -static inline pgoff_t vma_pgoff_offset (struct vm_area_struct *vma, -unsigned long addr) -{ -return vma->vm_pgoff +PHYS_PFN (addr - vma->vm_start); -} -#define VMG_STATE (name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \ struct vma_merge_struct name = { \ .mm = mm_, \
@@ -275,7 +268,7 @@ static inline pgoff_t vma_pgoff_offset(struct vm_area_struct *vma, .start = start_, \ .end = end_, \ .vm_flags = vma_->vm_flags, \ - .pgoff = vma_pgoff_offset(vma_, start_), \ + .pgoff = linear_page_index(vma_, start_), \ .file = vma_->vm_file, \ .anon_vma = vma_->anon_vma, \ .policy = vma_policy(vma_), \
tools/testing/vma/include/dup.h +16 -0
@@ -1548,3 +1548,19 @@ static inline pgprot_t vma_get_page_prot(vma_flags_t vma_flags)return vm_get_page_prot (vm_flags); } + +static inline pgoff_t linear_page_delta (const struct vm_area_struct *vma, +const unsigned long address) +{ +return (address - vma->vm_start) >>PAGE_SHIFT ; +} + +static inline pgoff_t linear_page_index (const struct vm_area_struct *vma, +const unsigned long address) +{ +pgoff_t pgoff; + + pgoff =linear_page_delta (vma, address); + pgoff +=vma_start_pgoff (vma); +return pgoff; +}
Signature · 1 line
2.54.0
Re: [PATCH 18/30] mm/vma: remove duplicative vma_pgoff_offset() helper
Quoted 8 lines · Lorenzo Stoakes
This is doing what
linear_page_index()does, so eliminate it and replace it withlinear_page_index().Update the VMA userland tests to reflect this change.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 59 lines
--- mm/vma.h | 9 +-------- tools/testing/vma/include/dup.h | 16 ++++++++++++++++ 2 files changed, 17 insertions(+), 8 deletions(-)mm/vma.h +1
-8@@ -247,13 +247,6 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)return vmg_start_pgoff (vmg) +vmg_pages (vmg); } -/* Assumes addr >= vma->vm_start. */ -static inline pgoff_t vma_pgoff_offset (struct vm_area_struct *vma, -unsigned long addr) -{ -return vma->vm_pgoff +PHYS_PFN (addr - vma->vm_start); -} -#define VMG_STATE (name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \ struct vma_merge_struct name = { \ .mm = mm_, \@@ -275,7 +268,7 @@ static inline pgoff_t vma_pgoff_offset(struct vm_area_struct *vma, .start = start_, \ .end = end_, \ .vm_flags = vma_->vm_flags, \ - .pgoff = vma_pgoff_offset(vma_, start_), \ + .pgoff = linear_page_index(vma_, start_), \ .file = vma_->vm_file, \ .anon_vma = vma_->anon_vma, \ .policy = vma_policy(vma_), \tools/testing/vma/include/dup.h +16
-0@@ -1548,3 +1548,19 @@ static inline pgprot_t vma_get_page_prot(vma_flags_t vma_flags)return vm_get_page_prot (vm_flags); } + +static inline pgoff_t linear_page_delta (const struct vm_area_struct *vma, +const unsigned long address) +{ +return (address - vma->vm_start) >>PAGE_SHIFT ; +} + +static inline pgoff_t linear_page_index (const struct vm_area_struct *vma, +const unsigned long address) +{ +pgoff_t pgoff; + + pgoff =linear_page_delta (vma, address); + pgoff +=vma_start_pgoff (vma); +return pgoff; +}Signature · 1 line
2.54.0
Signature · 1 line
Pedro
Re: [PATCH 18/30] mm/vma: remove duplicative vma_pgoff_offset() helper
In reply to Lorenzo StoakesQuoted 8 lines · Lorenzo Stoakes
This is doing what
linear_page_index()does, so eliminate it and replace it withlinear_page_index().Update the VMA userland tests to reflect this change.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 19/30 mm: use linear_page_[index, delta]() consistently
In reply to Lorenzo StoakesThere are a number of places where we open code what linear_page_index() and linear_page_delta() calculate.
Replace this code with the appropriate functions for consistency.
No functional change intended.
--- arch/arm/mm/fault-armv.c | 2 +- arch/x86/kernel/cpu/sgx/virt.c | 3 ++- drivers/comedi/comedi_fops.c | 3 ++- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- drivers/gpu/drm/gma500/gem.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 3 ++- drivers/gpu/drm/omapdrm/omap_gem.c | 5 +++-- drivers/gpu/drm/tegra/gem.c | 3 ++- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 ++++--- drivers/vfio/pci/nvgrace-gpu/main.c | 3 ++- drivers/vfio/pci/vfio_pci_core.c | 3 ++- mm/nommu.c | 2 +- mm/vma.c | 2 +- virt/kvm/guest_memfd.c | 2 +- 14 files changed, 26 insertions(+), 17 deletions(-)
arch/arm/mm/fault-armv.c +1 -1
@@ -132,7 +132,7 @@ make_coherent(struct address_space *mapping, struct vm_area_struct *vma,pgoff_t pgoff;int aliases =0 ; - pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >>PAGE_SHIFT ); + pgoff =linear_page_index (vma, addr);/* * If we have any shared mappings that are in the same mm
arch/x86/kernel/cpu/sgx/virt.c +2 -1
@@ -9,6 +9,7 @@#include <linux/miscdevice.h> #include <linux/mm.h> #include <linux/mman.h> +#include <linux/pagemap.h> #include <linux/sched/mm.h> #include <linux/sched/signal.h> #include <linux/slab.h>
@@ -41,7 +42,7 @@ static int __sgx_vepc_fault(struct sgx_vepc *vepc,WARN_ON (!mutex_is_locked (&vepc->lock));/* Calculate index of EPC page in virtual EPC's page_array */ - index = vma->vm_pgoff +PFN_DOWN (addr - vma->vm_start); + index =linear_page_index (vma, addr); epc_page =xa_load (&vepc->page_array, index);if (epc_page)
drivers/comedi/comedi_fops.c +2 -1
@@ -25,6 +25,7 @@#include <linux/fs.h> #include <linux/comedi/comedidev.h> #include <linux/cdev.h> +#include <linux/pagemap.h> #include <linux/io.h> #include <linux/uaccess.h>
@@ -2462,7 +2463,7 @@ static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, {struct comedi_buf_map *bm = vma->vm_private_data;unsigned long offset = - addr - vma->vm_start + (vma->vm_pgoff <<PAGE_SHIFT ); + addr - vma->vm_start + (vma_start_pgoff (vma) <<PAGE_SHIFT );if (len <0 )return -EINVAL ;
drivers/gpu/drm/etnaviv/etnaviv_gem.c +2 -1
@@ -6,6 +6,7 @@#include <drm/drm_prime.h> #include <drm/drm_print.h> #include <linux/dma-mapping.h> +#include <linux/pagemap.h> #include <linux/shmem_fs.h> #include <linux/spinlock.h> #include <linux/vmalloc.h>
@@ -188,7 +189,7 @@ static vm_fault_t etnaviv_gem_fault(struct vm_fault *vmf) }/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address); pfn =page_to_pfn (pages[pgoff]);
drivers/gpu/drm/gma500/gem.c +1 -1
@@ -288,7 +288,7 @@ static vm_fault_t psb_gem_fault(struct vm_fault *vmf)/* Page relative to the VMA start - we must calculate this ourselves because vmf->pgoff is the fake GEM offset */ - page_offset = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + page_offset =linear_page_delta (vma, vmf->address);/* CPU view of the page, don't go via the GART for CPU writes */ if (pobj->stolen)
drivers/gpu/drm/msm/msm_gem.c +2 -1
@@ -9,6 +9,7 @@#include <linux/spinlock.h> #include <linux/shmem_fs.h> #include <linux/dma-buf.h> +#include <linux/pagemap.h> #include <drm/drm_dumb_buffers.h> #include <drm/drm_prime.h>
@@ -360,7 +361,7 @@ static vm_fault_t msm_gem_fault(struct vm_fault *vmf) }/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address); pfn =page_to_pfn (pages[pgoff]);
drivers/gpu/drm/omapdrm/omap_gem.c +3 -2
@@ -5,6 +5,7 @@*/ #include <linux/dma-mapping.h> +#include <linux/pagemap.h> #include <linux/seq_file.h> #include <linux/shmem_fs.h> #include <linux/spinlock.h>
@@ -359,7 +360,7 @@ static vm_fault_t omap_gem_fault_1d(struct drm_gem_object *obj,pgoff_t pgoff;/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address);if (omap_obj->pages) {omap_gem_cpu_sync_page (obj, pgoff);
@@ -407,7 +408,7 @@ static vm_fault_t omap_gem_fault_2d(struct drm_gem_object *obj,const int m =DIV_ROUND_UP (omap_obj->width << fmt,PAGE_SIZE );/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address);/* * Actual address we start mapping at is rounded down to previous slot
drivers/gpu/drm/tegra/gem.c +2 -1
@@ -13,6 +13,7 @@#include <linux/dma-buf.h> #include <linux/iommu.h> #include <linux/module.h> +#include <linux/pagemap.h> #include <linux/vmalloc.h> #include <drm/drm_drv.h>
@@ -564,7 +565,7 @@ static vm_fault_t tegra_bo_fault(struct vm_fault *vmf)if (!bo->pages)return VM_FAULT_SIGBUS ; - offset = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + offset =linear_page_delta (vma, vmf->address); page = bo->pages[offset];return vmf_insert_page (vma, vmf->address, page);
drivers/gpu/drm/ttm/ttm_bo_vm.c +4 -3
@@ -32,6 +32,7 @@#define pr_fmt (fmt) "[TTM] " fmt#include <linux/export.h> +#include <linux/pagemap.h> #include <drm/ttm/ttm_bo.h> #include <drm/ttm/ttm_placement.h>
@@ -208,9 +209,9 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf,if (unlikely (err !=0 ))return VM_FAULT_SIGBUS ; - page_offset = ((address - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff -drm_vma_node_start (&bo->base.vma_node); - page_last =vma_pages (vma) + vma->vm_pgoff - + page_offset =linear_page_index (vma, address) - +drm_vma_node_start (&bo->base.vma_node); + page_last =vma_end_pgoff (vma) -drm_vma_node_start (&bo->base.vma_node);if (unlikely (page_offset >=PFN_UP (bo->base.size)))
drivers/vfio/pci/nvgrace-gpu/main.c +2 -1
@@ -11,6 +11,7 @@#include <linux/jiffies.h> #include <linux/sched.h> #include <linux/pci-p2pdma.h> +#include <linux/pagemap.h> #include <linux/pm_runtime.h> #include <linux/memory-failure.h>
@@ -385,7 +386,7 @@ static unsigned long addr_to_pgoff(struct vm_area_struct *vma,u64 pgoff = vma->vm_pgoff & ((1U << (VFIO_PCI_OFFSET_SHIFT -PAGE_SHIFT )) -1 ); -return ((addr - vma->vm_start) >>PAGE_SHIFT ) + pgoff; +return linear_page_delta (vma, addr) + pgoff; } static vm_fault_t nvgrace_gpu_vfio_pci_huge_fault(struct vm_fault *vmf,
drivers/vfio/pci/vfio_pci_core.c +2 -1
@@ -19,6 +19,7 @@#include <linux/module.h> #include <linux/mutex.h> #include <linux/notifier.h> +#include <linux/pagemap.h> #include <linux/pci.h> #include <linux/pm_runtime.h> #include <linux/slab.h>
@@ -1727,7 +1728,7 @@ static vm_fault_t vfio_pci_mmap_huge_fault(struct vm_fault *vmf,struct vm_area_struct *vma = vmf->vma;struct vfio_pci_core_device *vdev = vma->vm_private_data;unsigned long addr = vmf->address & ~((PAGE_SIZE << order) -1 ); -unsigned long pgoff = (addr - vma->vm_start) >>PAGE_SHIFT ; +unsigned long pgoff =linear_page_delta (vma, addr);unsigned long pfn =vma_to_pfn (vma) + pgoff;vm_fault_t ret =VM_FAULT_FALLBACK ;
mm/nommu.c +1 -1
@@ -1332,7 +1332,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, *region = *vma->vm_region; new->vm_region = region; - npages = (addr - vma->vm_start) >>PAGE_SHIFT ; + npages =linear_page_delta (vma, addr);if (new_below) { region->vm_top = region->vm_end = new->vm_end = addr;
mm/vma.c +1 -1
@@ -517,7 +517,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, new->vm_end = addr; }else { new->vm_start = addr; - new->vm_pgoff += ((addr - vma->vm_start) >>PAGE_SHIFT ); + new->vm_pgoff +=linear_page_delta (vma, addr); } err = -ENOMEM ;
virt/kvm/guest_memfd.c +1 -1
@@ -440,7 +440,7 @@ static int kvm_gmem_set_policy(struct vm_area_struct *vma, struct mempolicy *mpostatic struct mempolicy *kvm_gmem_get_policy (struct vm_area_struct *vma,unsigned long addr,pgoff_t *ilx) { -pgoff_t pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >>PAGE_SHIFT ); +pgoff_t pgoff =linear_page_index (vma, addr);struct inode *inode =file_inode (vma->vm_file); *ilx = inode->i_ino;
Signature · 1 line
2.54.0
Re: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
Hi
Am 29.06.26 um 14:23 schrieb Lorenzo Stoakes:
Quoted 8 lines
There are a number of places where we open code what
linear_page_index()andlinear_page_delta()calculate.Replace this code with the appropriate functions for consistency.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
For the DRM changes:
See below for two additional comments.
Quoted 17 lines
--- arch/arm/mm/fault-armv.c | 2 +- arch/x86/kernel/cpu/sgx/virt.c | 3 ++- drivers/comedi/comedi_fops.c | 3 ++- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- drivers/gpu/drm/gma500/gem.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 3 ++- drivers/gpu/drm/omapdrm/omap_gem.c | 5 +++-- drivers/gpu/drm/tegra/gem.c | 3 ++- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 ++++--- drivers/vfio/pci/nvgrace-gpu/main.c | 3 ++- drivers/vfio/pci/vfio_pci_core.c | 3 ++- mm/nommu.c | 2 +- mm/vma.c | 2 +- virt/kvm/guest_memfd.c | 2 +- 14 files changed, 26 insertions(+), 17 deletions(-)
[...]
drivers/comedi/comedi_fops.c static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, · line 2466
Quoted 9 lines
#include <linux/io.h> #include <linux/uaccess.h>drivers/comedi/comedi_fops.c +1
-1@@ -2462,7 +2463,7 @@ static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, {struct comedi_buf_map *bm = vma->vm_private_data;unsigned long offset = - addr - vma->vm_start + (vma->vm_pgoff <<PAGE_SHIFT ); + addr - vma->vm_start + (vma_start_pgoff (vma) <<PAGE_SHIFT );
This doesn't seem to belong here.
Quoted 122 lines
drivers/comedi/comedi_fops.c +0
-0if (len <0 )return -EINVAL ;drivers/gpu/drm/etnaviv/etnaviv_gem.c +2
-1@@ -6,6 +6,7 @@#include <drm/drm_prime.h> #include <drm/drm_print.h> #include <linux/dma-mapping.h> +#include <linux/pagemap.h> #include <linux/shmem_fs.h> #include <linux/spinlock.h> #include <linux/vmalloc.h> @@ -188,7 +189,7 @@ static vm_fault_t etnaviv_gem_fault(struct vm_fault *vmf) }/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address); pfn =page_to_pfn (pages[pgoff]);drivers/gpu/drm/gma500/gem.c +1
-1@@ -288,7 +288,7 @@ static vm_fault_t psb_gem_fault(struct vm_fault *vmf)/* Page relative to the VMA start - we must calculate this ourselves because vmf->pgoff is the fake GEM offset */ - page_offset = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + page_offset =linear_page_delta (vma, vmf->address);/* CPU view of the page, don't go via the GART for CPU writes */ if (pobj->stolen)drivers/gpu/drm/msm/msm_gem.c +2
-1@@ -9,6 +9,7 @@#include <linux/spinlock.h> #include <linux/shmem_fs.h> #include <linux/dma-buf.h> +#include <linux/pagemap.h> #include <drm/drm_dumb_buffers.h> #include <drm/drm_prime.h> @@ -360,7 +361,7 @@ static vm_fault_t msm_gem_fault(struct vm_fault *vmf) }/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address); pfn =page_to_pfn (pages[pgoff]);drivers/gpu/drm/omapdrm/omap_gem.c +3
-2@@ -5,6 +5,7 @@*/ #include <linux/dma-mapping.h> +#include <linux/pagemap.h> #include <linux/seq_file.h> #include <linux/shmem_fs.h> #include <linux/spinlock.h> @@ -359,7 +360,7 @@ static vm_fault_t omap_gem_fault_1d(struct drm_gem_object *obj,pgoff_t pgoff;/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address);if (omap_obj->pages) {omap_gem_cpu_sync_page (obj, pgoff);@@ -407,7 +408,7 @@ static vm_fault_t omap_gem_fault_2d(struct drm_gem_object *obj,const int m =DIV_ROUND_UP (omap_obj->width << fmt,PAGE_SIZE );/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address);/* * Actual address we start mapping at is rounded down to previous slot drivers/gpu/drm/tegra/gem.c +2
-1@@ -13,6 +13,7 @@#include <linux/dma-buf.h> #include <linux/iommu.h> #include <linux/module.h> +#include <linux/pagemap.h> #include <linux/vmalloc.h> #include <drm/drm_drv.h> @@ -564,7 +565,7 @@ static vm_fault_t tegra_bo_fault(struct vm_fault *vmf)if (!bo->pages)return VM_FAULT_SIGBUS ; - offset = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + offset =linear_page_delta (vma, vmf->address); page = bo->pages[offset];return vmf_insert_page (vma, vmf->address, page);drivers/gpu/drm/ttm/ttm_bo_vm.c +1
-0@@ -32,6 +32,7 @@#define pr_fmt (fmt) "[TTM] " fmt#include <linux/export.h> +#include <linux/pagemap.h> #include <drm/ttm/ttm_bo.h> #include <drm/ttm/ttm_placement.h>
drivers/gpu/drm/ttm/ttm_bo_vm.c vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, · line 215
Quoted 10 lines
drivers/gpu/drm/ttm/ttm_bo_vm.c +3
-3@@ -208,9 +209,9 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf,if (unlikely (err !=0 ))return VM_FAULT_SIGBUS ; - page_offset = ((address - vma->vm_start) >>PAGE_SHIFT ) + - vma->vm_pgoff -drm_vma_node_start (&bo->base.vma_node); - page_last =vma_pages (vma) + vma->vm_pgoff - + page_offset =linear_page_index (vma, address) - +drm_vma_node_start (&bo->base.vma_node); + page_last =vma_end_pgoff (vma) -drm_vma_node_start (&bo->base.vma_node);
Not your fault, but page_last seems misnamed here.
Best regards
Thomas
Quoted 83 lines
drivers/gpu/drm/ttm/ttm_bo_vm.c +0
-0if (unlikely (page_offset >=PFN_UP (bo->base.size)))drivers/vfio/pci/nvgrace-gpu/main.c +2
-1@@ -11,6 +11,7 @@#include <linux/jiffies.h> #include <linux/sched.h> #include <linux/pci-p2pdma.h> +#include <linux/pagemap.h> #include <linux/pm_runtime.h> #include <linux/memory-failure.h> @@ -385,7 +386,7 @@ static unsigned long addr_to_pgoff(struct vm_area_struct *vma,u64 pgoff = vma->vm_pgoff & ((1U << (VFIO_PCI_OFFSET_SHIFT -PAGE_SHIFT )) -1 ); -return ((addr - vma->vm_start) >>PAGE_SHIFT ) + pgoff; +return linear_page_delta (vma, addr) + pgoff; } static vm_fault_t nvgrace_gpu_vfio_pci_huge_fault(struct vm_fault *vmf,drivers/vfio/pci/vfio_pci_core.c +2
-1@@ -19,6 +19,7 @@#include <linux/module.h> #include <linux/mutex.h> #include <linux/notifier.h> +#include <linux/pagemap.h> #include <linux/pci.h> #include <linux/pm_runtime.h> #include <linux/slab.h> @@ -1727,7 +1728,7 @@ static vm_fault_t vfio_pci_mmap_huge_fault(struct vm_fault *vmf,struct vm_area_struct *vma = vmf->vma;struct vfio_pci_core_device *vdev = vma->vm_private_data;unsigned long addr = vmf->address & ~((PAGE_SIZE << order) -1 ); -unsigned long pgoff = (addr - vma->vm_start) >>PAGE_SHIFT ; +unsigned long pgoff =linear_page_delta (vma, addr);unsigned long pfn =vma_to_pfn (vma) + pgoff;vm_fault_t ret =VM_FAULT_FALLBACK ;mm/nommu.c +1
-1@@ -1332,7 +1332,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, *region = *vma->vm_region; new->vm_region = region; - npages = (addr - vma->vm_start) >>PAGE_SHIFT ; + npages =linear_page_delta (vma, addr);if (new_below) { region->vm_top = region->vm_end = new->vm_end = addr;mm/vma.c +1
-1@@ -517,7 +517,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, new->vm_end = addr; }else { new->vm_start = addr; - new->vm_pgoff += ((addr - vma->vm_start) >>PAGE_SHIFT ); + new->vm_pgoff +=linear_page_delta (vma, addr); } err = -ENOMEM ;virt/kvm/guest_memfd.c +1
-1@@ -440,7 +440,7 @@ static int kvm_gmem_set_policy(struct vm_area_struct *vma, struct mempolicy *mpostatic struct mempolicy *kvm_gmem_get_policy (struct vm_area_struct *vma,unsigned long addr,pgoff_t *ilx) { -pgoff_t pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >>PAGE_SHIFT ); +pgoff_t pgoff =linear_page_index (vma, addr);struct inode *inode =file_inode (vma->vm_file); *ilx = inode->i_ino;
Signature · 6 lines
-- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)
Lorenzo Stoakes
maintainerRe: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
Quoted 8 lines · Thomas Zimmermann
Hi
Am 29.06.26 um 14:23 schrieb Lorenzo Stoakes:
Quoted 5 lines
There are a number of places where we open code what
linear_page_index()andlinear_page_delta()calculate.Replace this code with the appropriate functions for consistency.
Quoted 7 lines
Quoted 3 lines
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>For the DRM changes:
Reviewed-by Thomas Zimmermann <[email protected]>
Thanks!
Quoted 24 lines
See below for two additional comments.
Quoted 17 lines
--- arch/arm/mm/fault-armv.c | 2 +- arch/x86/kernel/cpu/sgx/virt.c | 3 ++- drivers/comedi/comedi_fops.c | 3 ++- drivers/gpu/drm/etnaviv/etnaviv_gem.c | 3 ++- drivers/gpu/drm/gma500/gem.c | 2 +- drivers/gpu/drm/msm/msm_gem.c | 3 ++- drivers/gpu/drm/omapdrm/omap_gem.c | 5 +++-- drivers/gpu/drm/tegra/gem.c | 3 ++- drivers/gpu/drm/ttm/ttm_bo_vm.c | 7 ++++--- drivers/vfio/pci/nvgrace-gpu/main.c | 3 ++- drivers/vfio/pci/vfio_pci_core.c | 3 ++- mm/nommu.c | 2 +- mm/vma.c | 2 +- virt/kvm/guest_memfd.c | 2 +- 14 files changed, 26 insertions(+), 17 deletions(-)[...]
Quoted 10 lines
drivers/comedi/comedi_fops.c static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, · line 2466
Quoted 8 lines
#include <linux/io.h> #include <linux/uaccess.h>drivers/comedi/comedi_fops.c +1
-1@@ -2462,7 +2463,7 @@ static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, {struct comedi_buf_map *bm = vma->vm_private_data;unsigned long offset = - addr - vma->vm_start + (vma->vm_pgoff <<PAGE_SHIFT ); + addr - vma->vm_start + (vma_start_pgoff (vma) <<PAGE_SHIFT );This doesn't seem to belong here.
Ah yeah, I'll move that on a respin thanks!
Quoted 106 lines
Quoted 106 lines
drivers/comedi/comedi_fops.c +0
-0if (len <0 )return -EINVAL ;drivers/gpu/drm/etnaviv/etnaviv_gem.c +2
-1@@ -6,6 +6,7 @@#include <drm/drm_prime.h> #include <drm/drm_print.h> #include <linux/dma-mapping.h> +#include <linux/pagemap.h> #include <linux/shmem_fs.h> #include <linux/spinlock.h> #include <linux/vmalloc.h> @@ -188,7 +189,7 @@ static vm_fault_t etnaviv_gem_fault(struct vm_fault *vmf) }/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address); pfn =page_to_pfn (pages[pgoff]);drivers/gpu/drm/gma500/gem.c +1
-1@@ -288,7 +288,7 @@ static vm_fault_t psb_gem_fault(struct vm_fault *vmf)/* Page relative to the VMA start - we must calculate this ourselves because vmf->pgoff is the fake GEM offset */ - page_offset = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + page_offset =linear_page_delta (vma, vmf->address);/* CPU view of the page, don't go via the GART for CPU writes */ if (pobj->stolen)drivers/gpu/drm/msm/msm_gem.c +2
-1@@ -9,6 +9,7 @@#include <linux/spinlock.h> #include <linux/shmem_fs.h> #include <linux/dma-buf.h> +#include <linux/pagemap.h> #include <drm/drm_dumb_buffers.h> #include <drm/drm_prime.h> @@ -360,7 +361,7 @@ static vm_fault_t msm_gem_fault(struct vm_fault *vmf) }/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address); pfn =page_to_pfn (pages[pgoff]);drivers/gpu/drm/omapdrm/omap_gem.c +3
-2@@ -5,6 +5,7 @@*/ #include <linux/dma-mapping.h> +#include <linux/pagemap.h> #include <linux/seq_file.h> #include <linux/shmem_fs.h> #include <linux/spinlock.h> @@ -359,7 +360,7 @@ static vm_fault_t omap_gem_fault_1d(struct drm_gem_object *obj,pgoff_t pgoff;/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address);if (omap_obj->pages) {omap_gem_cpu_sync_page (obj, pgoff);@@ -407,7 +408,7 @@ static vm_fault_t omap_gem_fault_2d(struct drm_gem_object *obj,const int m =DIV_ROUND_UP (omap_obj->width << fmt,PAGE_SIZE );/* We don't use vmf->pgoff since that has the fake offset: */ - pgoff = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + pgoff =linear_page_delta (vma, vmf->address);/* * Actual address we start mapping at is rounded down to previous slot drivers/gpu/drm/tegra/gem.c +2
-1@@ -13,6 +13,7 @@#include <linux/dma-buf.h> #include <linux/iommu.h> #include <linux/module.h> +#include <linux/pagemap.h> #include <linux/vmalloc.h> #include <drm/drm_drv.h> @@ -564,7 +565,7 @@ static vm_fault_t tegra_bo_fault(struct vm_fault *vmf)if (!bo->pages)return VM_FAULT_SIGBUS ; - offset = (vmf->address - vma->vm_start) >>PAGE_SHIFT ; + offset =linear_page_delta (vma, vmf->address); page = bo->pages[offset];return vmf_insert_page (vma, vmf->address, page);drivers/gpu/drm/ttm/ttm_bo_vm.c +1
-1@@ -32,6 +32,7 @@#define pr_fmt (fmt) "[TTM] " fmt#include <linux/export.h> +#include <linux/pagemap.h> #include <drm/ttm/ttm_bo.h> #include <drm/ttm/ttm_placement.h> @@ -208,9 +209,9 @@ vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf,if (unlikely (err !=0 ))return VM_FAULT_SIGBUS ; - page_offset = ((address - vma->vm_start) >> PAGE_SHIFT) +
Quoted 8 lines
drivers/gpu/drm/ttm/ttm_bo_vm.c vm_fault_t ttm_bo_vm_fault_reserved(struct vm_fault *vmf, · line 215
Quoted 6 lines
drivers/gpu/drm/ttm/ttm_bo_vm.c +3
-2- vma->vm_pgoff -drm_vma_node_start (&bo->base.vma_node); - page_last =vma_pages (vma) + vma->vm_pgoff - + page_offset =linear_page_index (vma, address) - +drm_vma_node_start (&bo->base.vma_node); + page_last =vma_end_pgoff (vma) -drm_vma_node_start (&bo->base.vma_node);Not your fault, but
page_lastseems misnamed here.
Yeah :)
Quoted 79 lines
Best regards
ThomasQuoted 67 lines
drivers/gpu/drm/ttm/ttm_bo_vm.c +0
-0if (unlikely (page_offset >=PFN_UP (bo->base.size)))drivers/vfio/pci/nvgrace-gpu/main.c +2
-1@@ -11,6 +11,7 @@#include <linux/jiffies.h> #include <linux/sched.h> #include <linux/pci-p2pdma.h> +#include <linux/pagemap.h> #include <linux/pm_runtime.h> #include <linux/memory-failure.h> @@ -385,7 +386,7 @@ static unsigned long addr_to_pgoff(struct vm_area_struct *vma,u64 pgoff = vma->vm_pgoff & ((1U << (VFIO_PCI_OFFSET_SHIFT -PAGE_SHIFT )) -1 ); -return ((addr - vma->vm_start) >>PAGE_SHIFT ) + pgoff; +return linear_page_delta (vma, addr) + pgoff; } static vm_fault_t nvgrace_gpu_vfio_pci_huge_fault(struct vm_fault *vmf,drivers/vfio/pci/vfio_pci_core.c +2
-1@@ -19,6 +19,7 @@#include <linux/module.h> #include <linux/mutex.h> #include <linux/notifier.h> +#include <linux/pagemap.h> #include <linux/pci.h> #include <linux/pm_runtime.h> #include <linux/slab.h> @@ -1727,7 +1728,7 @@ static vm_fault_t vfio_pci_mmap_huge_fault(struct vm_fault *vmf,struct vm_area_struct *vma = vmf->vma;struct vfio_pci_core_device *vdev = vma->vm_private_data;unsigned long addr = vmf->address & ~((PAGE_SIZE << order) -1 ); -unsigned long pgoff = (addr - vma->vm_start) >>PAGE_SHIFT ; +unsigned long pgoff =linear_page_delta (vma, addr);unsigned long pfn =vma_to_pfn (vma) + pgoff;vm_fault_t ret =VM_FAULT_FALLBACK ;mm/nommu.c +1
-1@@ -1332,7 +1332,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, *region = *vma->vm_region; new->vm_region = region; - npages = (addr - vma->vm_start) >>PAGE_SHIFT ; + npages =linear_page_delta (vma, addr);if (new_below) { region->vm_top = region->vm_end = new->vm_end = addr;mm/vma.c +1
-1@@ -517,7 +517,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, new->vm_end = addr; }else { new->vm_start = addr; - new->vm_pgoff += ((addr - vma->vm_start) >>PAGE_SHIFT ); + new->vm_pgoff +=linear_page_delta (vma, addr); } err = -ENOMEM ;virt/kvm/guest_memfd.c +0
-0virt/kvm/guest_memfd.c static int kvm_gmem_set_policy(struct vm_area_struct *vma, struct mempolicy *mpo · line 445
Quoted 7 lines
virt/kvm/guest_memfd.c +1
-1@@ -440,7 +440,7 @@ static int kvm_gmem_set_policy(struct vm_area_struct *vma, struct mempolicy *mpostatic struct mempolicy *kvm_gmem_get_policy (struct vm_area_struct *vma,unsigned long addr,pgoff_t *ilx) { -pgoff_t pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >>PAGE_SHIFT ); +pgoff_t pgoff =linear_page_index (vma, addr);struct inode *inode =file_inode (vma->vm_file); *ilx = inode->i_ino;
Quoted 9 lines
--
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)
Cheers, Lorenzo
Re: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
In reply to Lorenzo StoakesQuoted 8 lines · Lorenzo Stoakes
There are a number of places where we open code what
linear_page_index()andlinear_page_delta()calculate.Replace this code with the appropriate functions for consistency.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
In reply to Lorenzo StoakesQuoted 14 lines · Lorenzo Stoakes
virt/kvm/guest_memfd.c +1
-2@@ -440,7 +440,7 @@ static int kvm_gmem_set_policy(struct vm_area_struct *vma, struct mempolicy *mpostatic struct mempolicy *kvm_gmem_get_policy (struct vm_area_struct *vma,unsigned long addr,pgoff_t *ilx) { -pgoff_t pgoff = vma->vm_pgoff + ((addr - vma->vm_start) >>PAGE_SHIFT ); +pgoff_t pgoff =linear_page_index (vma, addr);struct inode *inode =file_inode (vma->vm_file); *ilx = inode->i_ino; --
Quoted 1 line
2.54.0
For the guest_memfd change:
Thank you!
Re: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
In reply to Lorenzo StoakesQuoted 16 lines · Lorenzo Stoakes
arch/x86/kernel/cpu/sgx/virt.c +1
-0@@ -9,6 +9,7 @@ #include <linux/miscdevice.h> #include <linux/mm.h> #include <linux/mman.h> +#include <linux/pagemap.h> #include <linux/sched/mm.h> #include <linux/sched/signal.h> #include <linux/slab.h>@@ -41,7 +42,7 @@ static int __sgx_vepc_fault(struct sgx_vepc *vepc, WARN_ON(!mutex_is_locked(&vepc->lock));
/* Calculate index of EPC page in virtual EPC'spage_array*/
arch/x86/kernel/cpu/sgx/virt.c static int __sgx_vepc_fault(struct sgx_vepc *vepc, · line 45
Quoted 2 lines
arch/x86/kernel/cpu/sgx/virt.c +1
-1- index = vma->vm_pgoff +PFN_DOWN (addr - vma->vm_start); + index =linear_page_index (vma, addr);
For SGX:
Re: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
In reply to Lorenzo StoakesQuoted 8 lines · Lorenzo Stoakes
There are a number of places where we open code what
linear_page_index()andlinear_page_delta()calculate.Replace this code with the appropriate functions for consistency.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
one nit
...
Quoted 13 lines
drivers/comedi/comedi_fops.c +1
-0@@ -25,6 +25,7 @@#include <linux/fs.h> #include <linux/comedi/comedidev.h> #include <linux/cdev.h> +#include <linux/pagemap.h> #include <linux/io.h> #include <linux/uaccess.h>
drivers/comedi/comedi_fops.c static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, · line 2467
Quoted 6 lines
drivers/comedi/comedi_fops.c +1
-1@@ -2462,7 +2463,7 @@ static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, {struct comedi_buf_map *bm = vma->vm_private_data;unsigned long offset = - addr - vma->vm_start + (vma->vm_pgoff <<PAGE_SHIFT ); + addr - vma->vm_start + (vma_start_pgoff (vma) <<PAGE_SHIFT );
Obviously correct, but was this intended for a different patch?
~Gregory
Gregory Price
maintainerRe: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
Quoted 11 lines · Gregory Price
drivers/comedi/comedi_fops.c static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, · line 2467
Quoted 7 lines
drivers/comedi/comedi_fops.c +1
-1@@ -2462,7 +2463,7 @@ static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, {struct comedi_buf_map *bm = vma->vm_private_data;unsigned long offset = - addr - vma->vm_start + (vma->vm_pgoff <<PAGE_SHIFT ); + addr - vma->vm_start + (vma_start_pgoff (vma) <<PAGE_SHIFT );Obviously correct, but was this intended for a different patch?
~Gregory
bleh already caught, sorry for the noise :]
Lorenzo Stoakes
maintainerRe: [PATCH 19/30] mm: use linear_page_[index, delta]() consistently
Quoted 11 lines · Gregory Price
Quoted 10 lines · Gregory Price
drivers/comedi/comedi_fops.c static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, · line 2467
Quoted 7 lines
drivers/comedi/comedi_fops.c +1
-1@@ -2462,7 +2463,7 @@ static int comedi_vm_access(struct vm_area_struct *vma, unsigned long addr, {struct comedi_buf_map *bm = vma->vm_private_data;unsigned long offset = - addr - vma->vm_start + (vma->vm_pgoff <<PAGE_SHIFT ); + addr - vma->vm_start + (vma_start_pgoff (vma) <<PAGE_SHIFT );Obviously correct, but was this intended for a different patch?
Quoted 3 lines
Quoted 1 line
~Gregory
bleh already caught, sorry for the noise :]
Haha no thanks for picking it up, many eyes bugs etc. etc. ;)
Cheers, Lorenzo
Lorenzo Stoakes
maintainerPATCH 20/30 mm/vma: introduce vma_assert_can_modify()
In reply to Lorenzo Stoakesvma_assert_write_locked() and vma_assert_attached() are useful for their own purposes, however VMA code absolutely does allow the modification of non-write locked VMAs if they are at that point detached (i.e. unreachable from anywhere).
It's therefore useful to be able to assert that a VMA is either detached (modification doesn't matter) or write locked (you're explicitly locked for modification).
Therefore introduce vma_assert_can_modify() for this purpose.
While we're here, make vma_is_attached() available generally - if !CONFIG_PER_VMA_LOCKS, then there's no sense in which a VMA is detached (vma_mark_detached() is a noop), so have this default to true in this case.
--- include/linux/mmap_lock.h | 8 ++++++++ 1 file changed, 8 insertions(+)
include/linux/mmap_lock.h +8 -0
@@ -506,6 +506,8 @@ static inline __must_checkint vma_start_write_killable (struct vm_area_struct *vma) {return 0 ; }static inline void vma_assert_write_locked (struct vm_area_struct *vma) {mmap_assert_write_locked (vma->vm_mm); } +static inline bool vma_is_attached (struct vm_area_struct *vma) + {return true; }static inline void vma_assert_attached (struct vm_area_struct *vma) {}static inline void vma_assert_detached (struct vm_area_struct *vma) {}static inline void vma_mark_attached (struct vm_area_struct *vma) {}
@@ -530,6 +532,12 @@ static inline void vma_assert_stabilised(struct vm_area_struct *vma)#endif /* CONFIG_PER_VMA_LOCK */ +static inline void vma_assert_can_modify (struct vm_area_struct *vma) +{ +if (vma_is_attached (vma)) +vma_assert_write_locked (vma); +} +static inline void mmap_write_lock (struct mm_struct *mm) {__mmap_lock_trace_start_locking (mm, true);
Signature · 1 line
2.54.0
Pedro Falcato
reviewerRe: [PATCH 20/30] mm/vma: introduce vma_assert_can_modify()
Quoted 8 lines · Lorenzo Stoakes
vma_assert_write_locked()andvma_assert_attached()are useful for their own purposes, however VMA code absolutely does allow the modification of non-write locked VMAs if they are at that point detached (i.e. unreachable from anywhere).It's therefore useful to be able to assert that a VMA is either detached (modification doesn't matter) or write locked (you're explicitly locked for modification).
Hmm, I was wondering why detached does not imply write_locked, and then realized that new VMAs aren't write-locked. Could we do it by default? Like a simple:
vma->vm_lock_seq = __vma_raw_mm_seqnum(vma);
might do the trick. I don't see why it wouldn't work? Is there some other case I am not considering?
Quoted 42 lines
Therefore introduce
vma_assert_can_modify()for this purpose.While we're here, make
vma_is_attached()available generally - if !CONFIG_PER_VMA_LOCKS, then there's no sense in which a VMA is detached (vma_mark_detached()is a noop), so have this default to true in this case.Signed-off-by Lorenzo Stoakes <[email protected]>--- include/linux/mmap_lock.h | 8 ++++++++ 1 file changed, 8 insertions(+)include/linux/mmap_lock.h +8
-0@@ -506,6 +506,8 @@ static inline __must_checkint vma_start_write_killable (struct vm_area_struct *vma) {return 0 ; }static inline void vma_assert_write_locked (struct vm_area_struct *vma) {mmap_assert_write_locked (vma->vm_mm); } +static inline bool vma_is_attached (struct vm_area_struct *vma) + {return true; }static inline void vma_assert_attached (struct vm_area_struct *vma) {}static inline void vma_assert_detached (struct vm_area_struct *vma) {}static inline void vma_mark_attached (struct vm_area_struct *vma) {}@@ -530,6 +532,12 @@ static inline void vma_assert_stabilised(struct vm_area_struct *vma)#endif /* CONFIG_PER_VMA_LOCK */ +static inline void vma_assert_can_modify (struct vm_area_struct *vma) +{ +if (vma_is_attached (vma)) +vma_assert_write_locked (vma); +} +static inline void mmap_write_lock (struct mm_struct *mm) {__mmap_lock_trace_start_locking (mm, true);Signature · 1 line
2.54.0
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 20/30] mm/vma: introduce vma_assert_can_modify()
Quoted 11 lines · Pedro Falcato
Quoted 8 lines · Lorenzo Stoakes
vma_assert_write_locked()andvma_assert_attached()are useful for their own purposes, however VMA code absolutely does allow the modification of non-write locked VMAs if they are at that point detached (i.e. unreachable from anywhere).It's therefore useful to be able to assert that a VMA is either detached (modification doesn't matter) or write locked (you're explicitly locked for modification).
Hmm, I was wondering why detached does not imply
write_locked, and then
For one obviously when detaching they are also write locked (see vma_mark_detached()) but yeah the point of this is when you have a VMA allocated which doesn't hold the appropriate lock.
Quoted 7 lines
realized that new VMAs aren't write-locked. Could we do it by default? Like a simple:
vma->vm_lock_seq = __vma_raw_mm_seqnum(vma);might do the trick. I don't see why it wouldn't work? Is there some other case I am not considering?
Firstly, I'd rather not rework the VMA lock logic as part of this series. It's subtle and such changes are tricky.
I'm trying to achieve the minimum changes while adding extra validation as I can in preparation for the scalable CoW work.
And I'm not sure wanting to alter the fundamentals of VMA locks is a good reason not to ack a patch :)
But since you bring it up...
There's subtleties here too.
- At what point do you do this assignment? If it's the VMA allocation logic, it's not obvious then that you even have the mmap write lock necessarily. Now you're making assumptions that might be broken, or broken in future.
Now everybody who allocates a VMA has to 'just know' that they need to assign an mm and mark it write-locked (but in a special new VMA way) before setting the pgoff.
- Detached means it is not currently in any tree, nor belongs necessarily to any mm. So the concept of it being write locked is meaningless.
- It's broken to perform actions on a new VMA that is not yet linked into any tree that would require the VMA write lock. Currently the code _explicitly_ asserts that a detached VMA is not attempted to be write locked.
- So we have a good way of catching people doing stupid or broken stuff to VMAs that are not in the correct state (we currently _don't_ do that for detached VMAs that _were_ in the tree, probably we should change that...!)
- You'd have to create a new function to do this since we explicitly disallow doing this right now, and that's more complexity and then you're then creating a whole new meaning as to what VMA write lock acquisition is, which is even more added complexity.
- create_init_stack_vma() would break, so would hugetlb (lol) and static gate VMAs explicitly do not belong to an mm, so there's simply no concept of them being VMA write locked anyway.
- I'm not sure violating the invariant of seqnum = 0 = no write locked VMAs is safe.
- Things get quite horrendous on fork (prior to us taking tmp's VMA write lock)
- you'd have to assign this field after duplicating the VMA midway through attaching it to the new mm, where that mm has a new seqnum. But the new mm has a seqnum of 0... so now you're having to duplicate it but alter what vma_lock_init() does to reset the seqnum to 0 but then what if the 'duplication' logic asserts VMA write locked?
It's very chicken and egg - on VMA duplication you want to be able to write the very fields that you need to do anything with the VMA prior to it being linked in anywhere.
And we probably don't want to bake in the assumption that to change fundamental fields requires that you always hold the write lock as a consequence.
Quoted 31 lines
Quoted 31 lines
Therefore introduce
vma_assert_can_modify()for this purpose.While we're here, make
vma_is_attached()available generally - if !CONFIG_PER_VMA_LOCKS, then there's no sense in which a VMA is detached (vma_mark_detached()is a noop), so have this default to true in this case.Signed-off-by Lorenzo Stoakes <[email protected]>--- include/linux/mmap_lock.h | 8 ++++++++ 1 file changed, 8 insertions(+)include/linux/mmap_lock.h +2
-0@@ -506,6 +506,8 @@ static inline __must_checkint vma_start_write_killable (struct vm_area_struct *vma) {return 0 ; }static inline void vma_assert_write_locked (struct vm_area_struct *vma) {mmap_assert_write_locked (vma->vm_mm); } +static inline bool vma_is_attached (struct vm_area_struct *vma) + {return true; }static inline void vma_assert_attached (struct vm_area_struct *vma) {}static inline void vma_assert_detached (struct vm_area_struct *vma) {}static inline void vma_mark_attached (struct vm_area_struct *vma) {}@@ -530,6 +532,12 @@ static inline void vma_assert_stabilised(struct vm_area_struct *vma)#endif /* CONFIG_PER_VMA_LOCK */
Quoted 15 lines
Quoted 10 lines
include/linux/mmap_lock.h +6
-1+static inline void vma_assert_can_modify (struct vm_area_struct *vma) +{ +if (vma_is_attached (vma)) +vma_assert_write_locked (vma); +} +static inline void mmap_write_lock (struct mm_struct *mm) {__mmap_lock_trace_start_locking (mm, true); --Quoted 2 lines
2.54.0
--
Pedro
Thanks, Lorenzo
Re: [PATCH 20/30] mm/vma: introduce vma_assert_can_modify()
In reply to Lorenzo StoakesQuoted 5 lines · Lorenzo Stoakes
vma_assert_write_locked()andvma_assert_attached()are useful for their own purposes, however VMA code absolutely does allow the modification of non-write locked VMAs if they are at that point detached (i.e. unreachable from anywhere).
curiosity: I presume this happens mostly during init and/or teardown of a vma?
Quoted 4 lines
It's therefore useful to be able to assert that a VMA is either detached (modification doesn't matter) or write locked (you're explicitly locked for modification).
Quoted 8 lines
Therefore introduce
vma_assert_can_modify()for this purpose.While we're here, make
vma_is_attached()available generally - if !CONFIG_PER_VMA_LOCKS, then there's no sense in which a VMA is detached (vma_mark_detached()is a noop), so have this default to true in this case.Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 21/30 mm/vma: add and use vma_[add/sub]_pgoff()
In reply to Lorenzo StoakesAdd helpers for adding or subtracting to a VMA's page offset, exposed internally for VMA users within mm in mm/vma.h.
This is to lay the foundations for tracking anonymous page offset for MAP_PRIVATE file-backed mappings, where adding and subtracting from this value must be reflected in both the file and anonymous offsets.
These are used on VMA split and downward stack expansion.
No functional change intended.
--- mm/nommu.c | 6 ++++-- mm/vma.c | 6 +++--- mm/vma.h | 12 ++++++++++++ tools/testing/vma/include/dup.h | 13 ++++++++++++- 4 files changed, 31 insertions(+), 6 deletions(-)
mm/nommu.c +4 -2
@@ -41,6 +41,7 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include "internal.h" +#include "vma.h" unsigned long highest_memmap_pfn;int heap_stack_gap =0 ;
@@ -1338,7 +1339,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, region->vm_top = region->vm_end = new->vm_end = addr; }else { region->vm_start = new->vm_start = addr; - region->vm_pgoff = new->vm_pgoff += npages; +vma_add_pgoff (new, npages); + region->vm_pgoff =vma_start_pgoff (new); }vma_iter_config (vmi, new->vm_start, new->vm_end);
@@ -1355,7 +1357,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,delete_nommu_region (vma->vm_region);if (new_below) { vma->vm_region->vm_start = vma->vm_start = addr; - vma->vm_pgoff += npages; +vma_add_pgoff (vma, npages); vma->vm_region->vm_pgoff =vma_start_pgoff (vma); }else { vma->vm_region->vm_end = vma->vm_end = addr;
mm/vma.c +3 -3
@@ -517,7 +517,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, new->vm_end = addr; }else { new->vm_start = addr; - new->vm_pgoff +=linear_page_delta (vma, addr); +vma_add_pgoff (new,linear_page_delta (vma, addr)); } err = -ENOMEM ;
@@ -556,7 +556,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,if (new_below) { vma->vm_start = addr; - vma->vm_pgoff += (addr - new->vm_start) >>PAGE_SHIFT ; +vma_add_pgoff (vma, (addr - new->vm_start) >>PAGE_SHIFT ); }else { vma->vm_end = addr; }
@@ -3305,7 +3305,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address)vm_stat_account (mm, vma->vm_flags, grow);anon_vma_interval_tree_pre_update_vma (vma); vma->vm_start = address; - vma->vm_pgoff -= grow; +vma_sub_pgoff (vma, grow);/* Overwrite old entry in mtree. */ vma_iter_store_overwrite (&vmi, vma);anon_vma_interval_tree_post_update_vma (vma);
mm/vma.h +12 -0
@@ -247,6 +247,18 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)return vmg_start_pgoff (vmg) +vmg_pages (vmg); } +static inline void vma_add_pgoff (struct vm_area_struct *vma,pgoff_t delta) +{ +vma_assert_can_modify (vma); + vma->vm_pgoff += delta; +} + +static inline void vma_sub_pgoff (struct vm_area_struct *vma,pgoff_t delta) +{ +vma_assert_can_modify (vma); + vma->vm_pgoff -= delta; +} +#define VMG_STATE (name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \ struct vma_merge_struct name = { \ .mm = mm_, \
tools/testing/vma/include/dup.h +12 -1
@@ -1163,6 +1163,11 @@ static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi)return mas_find (&vmi->mas,ULONG_MAX ); } +static inline bool vma_is_attached (struct vm_area_struct *vma) +{ +return refcount_read (&vma->vm_refcnt); +} +/* * WARNING: to avoid racing with vma_mark_attached()/vma_mark_detached(), these * assertions should be made either under mmap_write_lock or when the object
@@ -1170,7 +1175,13 @@ static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi)*/ static inline void vma_assert_attached (struct vm_area_struct *vma) { -WARN_ON_ONCE (!refcount_read (&vma->vm_refcnt)); +WARN_ON_ONCE (!vma_is_attached (vma)); +} + +static inline void vma_assert_can_modify (struct vm_area_struct *vma) +{ +if (vma_is_attached (vma)) +vma_assert_write_locked (vma); }static inline void vma_assert_detached (struct vm_area_struct *vma)
Signature · 1 line
2.54.0
Re: [PATCH 21/30] mm/vma: add and use vma_[add/sub]_pgoff()
Quoted 123 lines · Lorenzo Stoakes
Add helpers for adding or subtracting to a VMA's page offset, exposed internally for VMA users within mm in
mm/vma.h.This is to lay the foundations for tracking anonymous page offset for
MAP_PRIVATEfile-backed mappings, where adding and subtracting from this value must be reflected in both the file and anonymous offsets.These are used on VMA split and downward stack expansion.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/nommu.c | 6 ++++-- mm/vma.c | 6 +++--- mm/vma.h | 12 ++++++++++++ tools/testing/vma/include/dup.h | 13 ++++++++++++- 4 files changed, 31 insertions(+), 6 deletions(-)mm/nommu.c +4
-2@@ -41,6 +41,7 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include "internal.h" +#include "vma.h" unsigned long highest_memmap_pfn;int heap_stack_gap =0 ;@@ -1338,7 +1339,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, region->vm_top = region->vm_end = new->vm_end = addr; }else { region->vm_start = new->vm_start = addr; - region->vm_pgoff = new->vm_pgoff += npages; +vma_add_pgoff (new, npages); + region->vm_pgoff =vma_start_pgoff (new); }vma_iter_config (vmi, new->vm_start, new->vm_end);@@ -1355,7 +1357,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,delete_nommu_region (vma->vm_region);if (new_below) { vma->vm_region->vm_start = vma->vm_start = addr; - vma->vm_pgoff += npages; +vma_add_pgoff (vma, npages); vma->vm_region->vm_pgoff =vma_start_pgoff (vma); }else { vma->vm_region->vm_end = vma->vm_end = addr;mm/vma.c +3
-3@@ -517,7 +517,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, new->vm_end = addr; }else { new->vm_start = addr; - new->vm_pgoff +=linear_page_delta (vma, addr); +vma_add_pgoff (new,linear_page_delta (vma, addr)); } err = -ENOMEM ;@@ -556,7 +556,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,if (new_below) { vma->vm_start = addr; - vma->vm_pgoff += (addr - new->vm_start) >>PAGE_SHIFT ; +vma_add_pgoff (vma, (addr - new->vm_start) >>PAGE_SHIFT ); }else { vma->vm_end = addr; }@@ -3305,7 +3305,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address)vm_stat_account (mm, vma->vm_flags, grow);anon_vma_interval_tree_pre_update_vma (vma); vma->vm_start = address; - vma->vm_pgoff -= grow; +vma_sub_pgoff (vma, grow);/* Overwrite old entry in mtree. */ vma_iter_store_overwrite (&vmi, vma);anon_vma_interval_tree_post_update_vma (vma);mm/vma.h +12
-0@@ -247,6 +247,18 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)return vmg_start_pgoff (vmg) +vmg_pages (vmg); } +static inline void vma_add_pgoff (struct vm_area_struct *vma,pgoff_t delta) +{ +vma_assert_can_modify (vma); + vma->vm_pgoff += delta; +} + +static inline void vma_sub_pgoff (struct vm_area_struct *vma,pgoff_t delta) +{ +vma_assert_can_modify (vma); + vma->vm_pgoff -= delta; +} +#define VMG_STATE (name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \ struct vma_merge_struct name = { \ .mm = mm_, \tools/testing/vma/include/dup.h +5
-0@@ -1163,6 +1163,11 @@ static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi)return mas_find (&vmi->mas,ULONG_MAX ); } +static inline bool vma_is_attached (struct vm_area_struct *vma) +{ +return refcount_read (&vma->vm_refcnt); +} +/* * WARNING: to avoid racing with vma_mark_attached()/vma_mark_detached(), these * assertions should be made either under mmap_write_lock or when the object @@ -1170,7 +1175,13 @@ static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi)*/ static inline void vma_assert_attached (struct vm_area_struct *vma)
tools/testing/vma/include/dup.h static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi) · line 1185
Quoted 10 lines
tools/testing/vma/include/dup.h +7
-1{ -WARN_ON_ONCE (!refcount_read (&vma->vm_refcnt)); +WARN_ON_ONCE (!vma_is_attached (vma)); +} + +static inline void vma_assert_can_modify (struct vm_area_struct *vma) +{ +if (vma_is_attached (vma)) +vma_assert_write_locked (vma); }
These hunks in dup.h look lost. Should perhaps be on the previous patch (adding the helpers).
Anyway, Obviously Correct(tm).
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 21/30] mm/vma: add and use vma_[add/sub]_pgoff()
Quoted 129 lines · Pedro Falcato
Quoted 128 lines · Lorenzo Stoakes
Add helpers for adding or subtracting to a VMA's page offset, exposed internally for VMA users within mm in
mm/vma.h.This is to lay the foundations for tracking anonymous page offset for
MAP_PRIVATEfile-backed mappings, where adding and subtracting from this value must be reflected in both the file and anonymous offsets.These are used on VMA split and downward stack expansion.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/nommu.c | 6 ++++-- mm/vma.c | 6 +++--- mm/vma.h | 12 ++++++++++++ tools/testing/vma/include/dup.h | 13 ++++++++++++- 4 files changed, 31 insertions(+), 6 deletions(-)mm/nommu.c +4
-2@@ -41,6 +41,7 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include "internal.h" +#include "vma.h" unsigned long highest_memmap_pfn;int heap_stack_gap =0 ;@@ -1338,7 +1339,8 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, region->vm_top = region->vm_end = new->vm_end = addr; }else { region->vm_start = new->vm_start = addr; - region->vm_pgoff = new->vm_pgoff += npages; +vma_add_pgoff (new, npages); + region->vm_pgoff =vma_start_pgoff (new); }vma_iter_config (vmi, new->vm_start, new->vm_end);@@ -1355,7 +1357,7 @@ static int split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,delete_nommu_region (vma->vm_region);if (new_below) { vma->vm_region->vm_start = vma->vm_start = addr; - vma->vm_pgoff += npages; +vma_add_pgoff (vma, npages); vma->vm_region->vm_pgoff =vma_start_pgoff (vma); }else { vma->vm_region->vm_end = vma->vm_end = addr;mm/vma.c +3
-3@@ -517,7 +517,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma, new->vm_end = addr; }else { new->vm_start = addr; - new->vm_pgoff +=linear_page_delta (vma, addr); +vma_add_pgoff (new,linear_page_delta (vma, addr)); } err = -ENOMEM ;@@ -556,7 +556,7 @@ __split_vma(struct vma_iterator *vmi, struct vm_area_struct *vma,if (new_below) { vma->vm_start = addr; - vma->vm_pgoff += (addr - new->vm_start) >>PAGE_SHIFT ; +vma_add_pgoff (vma, (addr - new->vm_start) >>PAGE_SHIFT ); }else { vma->vm_end = addr; }@@ -3305,7 +3305,7 @@ int expand_downwards(struct vm_area_struct *vma, unsigned long address)vm_stat_account (mm, vma->vm_flags, grow);anon_vma_interval_tree_pre_update_vma (vma); vma->vm_start = address; - vma->vm_pgoff -= grow; +vma_sub_pgoff (vma, grow);/* Overwrite old entry in mtree. */ vma_iter_store_overwrite (&vmi, vma);anon_vma_interval_tree_post_update_vma (vma);mm/vma.h +12
-0@@ -247,6 +247,18 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)return vmg_start_pgoff (vmg) +vmg_pages (vmg); } +static inline void vma_add_pgoff (struct vm_area_struct *vma,pgoff_t delta) +{ +vma_assert_can_modify (vma); + vma->vm_pgoff += delta; +} + +static inline void vma_sub_pgoff (struct vm_area_struct *vma,pgoff_t delta) +{ +vma_assert_can_modify (vma); + vma->vm_pgoff -= delta; +} +#define VMG_STATE (name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \ struct vma_merge_struct name = { \ .mm = mm_, \tools/testing/vma/include/dup.h +8
-1@@ -1163,6 +1163,11 @@ static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi)return mas_find (&vmi->mas,ULONG_MAX ); } +static inline bool vma_is_attached (struct vm_area_struct *vma) +{ +return refcount_read (&vma->vm_refcnt); +} +/* * WARNING: to avoid racing with vma_mark_attached()/vma_mark_detached(), these * assertions should be made either under mmap_write_lock or when the object @@ -1170,7 +1175,13 @@ static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi)*/ static inline void vma_assert_attached (struct vm_area_struct *vma) { -WARN_ON_ONCE (!refcount_read (&vma->vm_refcnt)); +WARN_ON_ONCE (!vma_is_attached (vma)); +} +
Quoted 8 lines
tools/testing/vma/include/dup.h static inline struct vm_area_struct *vma_next(struct vma_iterator *vmi) · line 1185
Quoted 5 lines
tools/testing/vma/include/dup.h +4
-0+static inline void vma_assert_can_modify (struct vm_area_struct *vma) +{ +if (vma_is_attached (vma)) +vma_assert_write_locked (vma); }These hunks in
dup.hlook lost. Should perhaps be on the previous patch (adding the helpers).
Yeah, it's because the VMA code starts actually using them in a way that otherwise breaks the tests at this point, but you're right, functionally it'd be nicer to add them at the point they're introduced.
Will fix that up on respin!
Quoted 4 lines
Anyway, Obviously Correct(tm).
Reviewed-by Pedro Falcato <[email protected]>
Thanks!
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Re: [PATCH 21/30] mm/vma: add and use vma_[add/sub]_pgoff()
In reply to Lorenzo StoakesQuoted 7 lines · Lorenzo Stoakes
Add helpers for adding or subtracting to a VMA's page offset, exposed internally for VMA users within mm in
mm/vma.h.This is to lay the foundations for tracking anonymous page offset for
MAP_PRIVATEfile-backed mappings, where adding and subtracting from this value must be reflected in both the file and anonymous offsets.
Quoted 5 lines
These are used on VMA split and downward stack expansion.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 22/30 mm/vma: move __install_special_mapping() to vma.c
In reply to Lorenzo StoakesThis function is operating on VMAs and rightly belongs in vma.c, where it can be subject to VMA userland testing and allows us to isolate it from the rest of mm.
The _install_special_mapping() function will remain in mmap.c as a wrapper, since this is used by architecture-specific code.
Doing so allows us to isolate more functions in vma.c for the same reasons.
This forms part of work to allow for tracking MAP_PRIVATE file-backed mappings by their anonymous virtual page offset, as doing so allows us to isolate and keep code that interacts with this together.
No functional change intended.
--- mm/mmap.c | 38 -------------------------------------- mm/vma.c | 38 ++++++++++++++++++++++++++++++++++++++ mm/vma.h | 5 +++++ 3 files changed, 43 insertions(+), 38 deletions(-)
mm/mmap.c +0 -38
@@ -1447,44 +1447,6 @@ static vm_fault_t special_mapping_fault(struct vm_fault *vmf)return VM_FAULT_SIGBUS ; } -static struct vm_area_struct *__install_special_mapping ( -struct mm_struct *mm, -unsigned long addr,unsigned long len, -vm_flags_t vm_flags,void *priv, -const struct vm_operations_struct *ops) -{ -int ret; -struct vm_area_struct *vma; - - vma =vm_area_alloc (mm); -if (unlikely (vma ==NULL )) -return ERR_PTR (-ENOMEM ); - -vma_set_range (vma, addr, addr + len,0 ); - vm_flags |= mm->def_flags |VM_DONTEXPAND ; -if (pgtable_supports_soft_dirty ()) - vm_flags |=VM_SOFTDIRTY ; -vm_flags_init (vma, vm_flags & ~VM_LOCKED_MASK ); - vma->vm_page_prot =vm_get_page_prot (vma->vm_flags); - - vma->vm_ops = ops; - vma->vm_private_data = priv; - - ret =insert_vm_struct (mm, vma); -if (ret) - goto out; - -vm_stat_account (mm, vma->vm_flags, len >>PAGE_SHIFT ); - -perf_event_mmap (vma); - -return vma; - -out: -vm_area_free (vma); -return ERR_PTR (ret); -} -bool vma_is_special_mapping (const struct vm_area_struct *vma,const struct vm_special_mapping *sm) {
mm/vma.c +38 -0
@@ -3399,3 +3399,41 @@ __weak unsigned long vma_mmu_pagesize(struct vm_area_struct *vma) {return vma_kernel_pagesize (vma); } + +struct vm_area_struct *__install_special_mapping ( +struct mm_struct *mm, +unsigned long addr,unsigned long len, +vm_flags_t vm_flags,void *priv, +const struct vm_operations_struct *ops) +{ +int ret; +struct vm_area_struct *vma; + + vma =vm_area_alloc (mm); +if (unlikely (vma ==NULL )) +return ERR_PTR (-ENOMEM ); + +vma_set_range (vma, addr, addr + len,0 ); + vm_flags |= mm->def_flags |VM_DONTEXPAND ; +if (pgtable_supports_soft_dirty ()) + vm_flags |=VM_SOFTDIRTY ; +vm_flags_init (vma, vm_flags & ~VM_LOCKED_MASK ); + vma->vm_page_prot =vm_get_page_prot (vma->vm_flags); + + vma->vm_ops = ops; + vma->vm_private_data = priv; + + ret =insert_vm_struct (mm, vma); +if (ret) + goto out; + +vm_stat_account (mm, vma->vm_flags, len >>PAGE_SHIFT ); + +perf_event_mmap (vma); + +return vma; + +out: +vm_area_free (vma); +return ERR_PTR (ret); +}
mm/vma.h +5 -0
@@ -775,4 +775,9 @@ static inline bool map_deny_write_exec(const vma_flags_t *old, }#endif +struct vm_area_struct *__install_special_mapping (struct mm_struct *mm, +unsigned long addr,unsigned long len, +vm_flags_t vm_flags,void *priv, +const struct vm_operations_struct *ops); +#endif /* __MM_VMA_H */
Signature · 1 line
2.54.0
Re: [PATCH 22/30] mm/vma: move __install_special_mapping() to vma.c
Quoted 122 lines · Lorenzo Stoakes
This function is operating on VMAs and rightly belongs in
vma.c, where it can be subject to VMA userland testing and allows us to isolate it from the rest of mm.The
_install_special_mapping()function will remain inmmap.cas a wrapper, since this is used by architecture-specific code.Doing so allows us to isolate more functions in
vma.cfor the same reasons.This forms part of work to allow for tracking
MAP_PRIVATEfile-backed mappings by their anonymous virtual page offset, as doing so allows us to isolate and keep code that interacts with this together.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/mmap.c | 38 -------------------------------------- mm/vma.c | 38 ++++++++++++++++++++++++++++++++++++++ mm/vma.h | 5 +++++ 3 files changed, 43 insertions(+), 38 deletions(-)mm/mmap.c +0
-38@@ -1447,44 +1447,6 @@ static vm_fault_t special_mapping_fault(struct vm_fault *vmf)return VM_FAULT_SIGBUS ; } -static struct vm_area_struct *__install_special_mapping ( -struct mm_struct *mm, -unsigned long addr,unsigned long len, -vm_flags_t vm_flags,void *priv, -const struct vm_operations_struct *ops) -{ -int ret; -struct vm_area_struct *vma; - - vma =vm_area_alloc (mm); -if (unlikely (vma ==NULL )) -return ERR_PTR (-ENOMEM ); - -vma_set_range (vma, addr, addr + len,0 ); - vm_flags |= mm->def_flags |VM_DONTEXPAND ; -if (pgtable_supports_soft_dirty ()) - vm_flags |=VM_SOFTDIRTY ; -vm_flags_init (vma, vm_flags & ~VM_LOCKED_MASK ); - vma->vm_page_prot =vm_get_page_prot (vma->vm_flags); - - vma->vm_ops = ops; - vma->vm_private_data = priv; - - ret =insert_vm_struct (mm, vma); -if (ret) - goto out; - -vm_stat_account (mm, vma->vm_flags, len >>PAGE_SHIFT ); - -perf_event_mmap (vma); - -return vma; - -out: -vm_area_free (vma); -return ERR_PTR (ret); -} -bool vma_is_special_mapping (const struct vm_area_struct *vma,const struct vm_special_mapping *sm) {mm/vma.c +38
-0@@ -3399,3 +3399,41 @@ __weak unsigned long vma_mmu_pagesize(struct vm_area_struct *vma) {return vma_kernel_pagesize (vma); } + +struct vm_area_struct *__install_special_mapping ( +struct mm_struct *mm, +unsigned long addr,unsigned long len, +vm_flags_t vm_flags,void *priv, +const struct vm_operations_struct *ops) +{ +int ret; +struct vm_area_struct *vma; + + vma =vm_area_alloc (mm); +if (unlikely (vma ==NULL )) +return ERR_PTR (-ENOMEM ); + +vma_set_range (vma, addr, addr + len,0 ); + vm_flags |= mm->def_flags |VM_DONTEXPAND ; +if (pgtable_supports_soft_dirty ()) + vm_flags |=VM_SOFTDIRTY ; +vm_flags_init (vma, vm_flags & ~VM_LOCKED_MASK ); + vma->vm_page_prot =vm_get_page_prot (vma->vm_flags); + + vma->vm_ops = ops; + vma->vm_private_data = priv; + + ret =insert_vm_struct (mm, vma); +if (ret) + goto out; + +vm_stat_account (mm, vma->vm_flags, len >>PAGE_SHIFT ); + +perf_event_mmap (vma); + +return vma; + +out: +vm_area_free (vma); +return ERR_PTR (ret); +}mm/vma.h +0
-0
mm/vma.h static inline bool map_deny_write_exec(const vma_flags_t *old, · line 783
Quoted 9 lines
mm/vma.h +5
-0@@ -775,4 +775,9 @@ static inline bool map_deny_write_exec(const vma_flags_t *old, }#endif +struct vm_area_struct *__install_special_mapping (struct mm_struct *mm, +unsigned long addr,unsigned long len, +vm_flags_t vm_flags,void *priv, +const struct vm_operations_struct *ops); +#endif /* __MM_VMA_H */
I'm really annoyed that _install_special_mapping has a leading underscore. That's it.
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 22/30] mm/vma: move __install_special_mapping() to vma.c
Quoted 126 lines · Pedro Falcato
Quoted 125 lines · Lorenzo Stoakes
This function is operating on VMAs and rightly belongs in
vma.c, where it can be subject to VMA userland testing and allows us to isolate it from the rest of mm.The
_install_special_mapping()function will remain inmmap.cas a wrapper, since this is used by architecture-specific code.Doing so allows us to isolate more functions in
vma.cfor the same reasons.This forms part of work to allow for tracking
MAP_PRIVATEfile-backed mappings by their anonymous virtual page offset, as doing so allows us to isolate and keep code that interacts with this together.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/mmap.c | 38 -------------------------------------- mm/vma.c | 38 ++++++++++++++++++++++++++++++++++++++ mm/vma.h | 5 +++++ 3 files changed, 43 insertions(+), 38 deletions(-)mm/mmap.c +0
-38@@ -1447,44 +1447,6 @@ static vm_fault_t special_mapping_fault(struct vm_fault *vmf)return VM_FAULT_SIGBUS ; } -static struct vm_area_struct *__install_special_mapping ( -struct mm_struct *mm, -unsigned long addr,unsigned long len, -vm_flags_t vm_flags,void *priv, -const struct vm_operations_struct *ops) -{ -int ret; -struct vm_area_struct *vma; - - vma =vm_area_alloc (mm); -if (unlikely (vma ==NULL )) -return ERR_PTR (-ENOMEM ); - -vma_set_range (vma, addr, addr + len,0 ); - vm_flags |= mm->def_flags |VM_DONTEXPAND ; -if (pgtable_supports_soft_dirty ()) - vm_flags |=VM_SOFTDIRTY ; -vm_flags_init (vma, vm_flags & ~VM_LOCKED_MASK ); - vma->vm_page_prot =vm_get_page_prot (vma->vm_flags); - - vma->vm_ops = ops; - vma->vm_private_data = priv; - - ret =insert_vm_struct (mm, vma); -if (ret) - goto out; - -vm_stat_account (mm, vma->vm_flags, len >>PAGE_SHIFT ); - -perf_event_mmap (vma); - -return vma; - -out: -vm_area_free (vma); -return ERR_PTR (ret); -} -bool vma_is_special_mapping (const struct vm_area_struct *vma,const struct vm_special_mapping *sm) {mm/vma.c +38
-0@@ -3399,3 +3399,41 @@ __weak unsigned long vma_mmu_pagesize(struct vm_area_struct *vma) {return vma_kernel_pagesize (vma); } + +struct vm_area_struct *__install_special_mapping ( +struct mm_struct *mm, +unsigned long addr,unsigned long len, +vm_flags_t vm_flags,void *priv, +const struct vm_operations_struct *ops) +{ +int ret; +struct vm_area_struct *vma; + + vma =vm_area_alloc (mm); +if (unlikely (vma ==NULL )) +return ERR_PTR (-ENOMEM ); + +vma_set_range (vma, addr, addr + len,0 ); + vm_flags |= mm->def_flags |VM_DONTEXPAND ; +if (pgtable_supports_soft_dirty ()) + vm_flags |=VM_SOFTDIRTY ; +vm_flags_init (vma, vm_flags & ~VM_LOCKED_MASK ); + vma->vm_page_prot =vm_get_page_prot (vma->vm_flags); + + vma->vm_ops = ops; + vma->vm_private_data = priv; + + ret =insert_vm_struct (mm, vma); +if (ret) + goto out; + +vm_stat_account (mm, vma->vm_flags, len >>PAGE_SHIFT ); + +perf_event_mmap (vma); + +return vma; + +out: +vm_area_free (vma); +return ERR_PTR (ret); +}mm/vma.h +0
-0@@ -775,4 +775,9 @@ static inline bool map_deny_write_exec(const vma_flags_t *old, }#endif
Quoted 9 lines
mm/vma.h static inline bool map_deny_write_exec(const vma_flags_t *old, · line 783
Quoted 6 lines
mm/vma.h +5
-0+struct vm_area_struct *__install_special_mapping (struct mm_struct *mm, +unsigned long addr,unsigned long len, +vm_flags_t vm_flags,void *priv, +const struct vm_operations_struct *ops); +#endif /* __MM_VMA_H */ I'm really annoyed that
_install_special_mappinghas a leading underscore. That's it.
Yeah it's horrible :)
Quoted 2 lines
Reviewed-by Pedro Falcato <[email protected]>
Thanks!
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Re: [PATCH 22/30] mm/vma: move __install_special_mapping() to vma.c
In reply to Lorenzo StoakesQuoted 9 lines · Lorenzo Stoakes
This function is operating on VMAs and rightly belongs in
vma.c, where it can be subject to VMA userland testing and allows us to isolate it from the rest of mm.The
_install_special_mapping()function will remain inmmap.cas a wrapper, since this is used by architecture-specific code.Doing so allows us to isolate more functions in
vma.cfor the same reasons.
Quoted 7 lines
This forms part of work to allow for tracking
MAP_PRIVATEfile-backed mappings by their anonymous virtual page offset, as doing so allows us to isolate and keep code that interacts with this together.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 23/30 mm/vma: make vma_set_range() static, drop insert_vm_struct() decl
In reply to Lorenzo StoakesWith __install_special_mapping() moved to vma.c, vma_set_range() can be made into a static function there and is now completely isolated from the rest of mm.
While we're here, we can also remove the insert_vm_struct() declaration from mm.h - the function is implemented in vma.c and already declared in vma.h, and has no users outside of mm.
Also update the VMA userland tests to reflect this change.
No functional change intended.
--- include/linux/mm.h | 1 - mm/internal.h | 9 --------- mm/vma.c | 8 ++++++++ tools/testing/vma/shared.c | 9 --------- tools/testing/vma/shared.h | 5 ----- 5 files changed, 8 insertions(+), 24 deletions(-)
include/linux/mm.h +0 -1
@@ -4103,7 +4103,6 @@ void anon_vma_interval_tree_verify(struct anon_vma_chain *avc);/* mmap.c */ extern int __vm_enough_memory (const struct mm_struct *mm,long pages,int cap_sys_admin); -extern int insert_vm_struct (struct mm_struct *,struct vm_area_struct *);extern void exit_mmap (struct mm_struct *);bool mmap_read_lock_maybe_expand (struct mm_struct *mm,struct vm_area_struct *vma,unsigned long addr,bool write);
mm/internal.h +0 -9
@@ -1720,15 +1720,6 @@ extern bool mirrored_kernelcore;bool memblock_has_mirror (void );void memblock_free_all (void ); -static __always_inline void vma_set_range(struct vm_area_struct *vma, -unsigned long start,unsigned long end, -pgoff_t pgoff) -{ - vma->vm_start = start; - vma->vm_end = end; - vma->vm_pgoff = pgoff; -} -static inline bool vma_soft_dirty_enabled (struct vm_area_struct *vma) {/*
mm/vma.c +8 -0
@@ -70,6 +70,14 @@ struct mmap_state { .state = VMA_MERGE_START, \ } +static void vma_set_range (struct vm_area_struct *vma,unsigned long start, +unsigned long end,pgoff_t pgoff) +{ + vma->vm_start = start; + vma->vm_end = end; + vma->vm_pgoff = pgoff; +} +/* Was this VMA ever forked from a parent, i.e. maybe contains CoW mappings? */ static bool vma_is_fork_child (struct vm_area_struct *vma) {
tools/testing/vma/shared.c +0 -9
@@ -120,12 +120,3 @@ unsigned long rlimit(unsigned int limit) {return (unsigned long )-1 ; } - -void vma_set_range (struct vm_area_struct *vma, -unsigned long start,unsigned long end, -pgoff_t pgoff) -{ - vma->vm_start = start; - vma->vm_end = end; - vma->vm_pgoff = pgoff; -}
tools/testing/vma/shared.h +0 -5
@@ -125,8 +125,3 @@ void __vma_set_dummy_anon_vma(struct vm_area_struct *vma,/* Provide a simple dummy VMA/anon_vma dummy setup for testing. */ void vma_set_dummy_anon_vma (struct vm_area_struct *vma,struct anon_vma_chain *avc); - -/* Helper function to specify a VMA's range. */ -void vma_set_range (struct vm_area_struct *vma, -unsigned long start,unsigned long end, -pgoff_t pgoff);
Signature · 1 line
2.54.0
Re: [PATCH 23/30] mm/vma: make vma_set_range() static, drop insert_vm_struct() decl
Quoted 8 lines · Lorenzo Stoakes
With
__install_special_mapping()moved tovma.c,vma_set_range()can be made into a static function there and is now completely isolated from the rest of mm.While we're here, we can also remove the
insert_vm_struct()declaration frommm.h- the function is implemented invma.cand already declared invma.h, and has no users outside of mm.
Quoted 5 lines
Also update the VMA userland tests to reflect this change.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 23/30] mm/vma: make vma_set_range() static, drop insert_vm_struct() decl
In reply to Lorenzo StoakesQuoted 8 lines · Lorenzo Stoakes
With
__install_special_mapping()moved tovma.c,vma_set_range()can be made into a static function there and is now completely isolated from the rest of mm.While we're here, we can also remove the
insert_vm_struct()declaration frommm.h- the function is implemented invma.cand already declared invma.h, and has no users outside of mm.
Quoted 7 lines
Also update the VMA userland tests to reflect this change.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>---
5 files changed, 8 insertions(+), 24 deletions(-)
Net negative lines of code, the best kind of commit
Lorenzo Stoakes
maintainerPATCH 24/30 mm/vma: update vma_shrink() to not pass unnecessary pgoff parameter
In reply to Lorenzo Stoakesvma_shrink() does not need to adjust vma->vm_pgoff, we were passing this parameter solely to satisfy vma_set_range()'s requirement for pgoff being specified.
Since vma_set_range() is now isolated to vma.c, we can simply introduce __vma_set_range() which sets only vma->vm_[start, end], and invoke this instead, removing pgoff from vma_shrink() altogether.
No functional change intended.
--- mm/vma.c | 14 ++++++++++---- mm/vma.h | 2 +- mm/vma_exec.c | 2 +- tools/testing/vma/tests/merge.c | 2 +- 4 files changed, 13 insertions(+), 7 deletions(-)
mm/vma.c +10 -4
@@ -70,11 +70,17 @@ struct mmap_state { .state = VMA_MERGE_START, \ } -static void vma_set_range (struct vm_area_struct *vma,unsigned long start, -unsigned long end,pgoff_t pgoff) +static void __vma_set_range (struct vm_area_struct *vma,unsigned long start, +unsigned long end) { vma->vm_start = start; vma->vm_end = end; +} + +static void vma_set_range (struct vm_area_struct *vma,unsigned long start, +unsigned long end,pgoff_t pgoff) +{ +__vma_set_range (vma, start, end); vma->vm_pgoff = pgoff; }
@@ -1289,7 +1295,7 @@ int vma_expand(struct vma_merge_struct *vmg)* Returns: 0 on success, -ENOMEM otherwise */ int vma_shrink (struct vma_iterator *vmi,struct vm_area_struct *vma, -unsigned long start,unsigned long end,pgoff_t pgoff) +unsigned long start,unsigned long end) {struct vma_prepare vp;
@@ -1310,7 +1316,7 @@ int vma_shrink(struct vma_iterator *vmi, struct vm_area_struct *vma,vma_adjust_trans_huge (vma, start, end,NULL );vma_iter_clear (vmi); -vma_set_range (vma, start, end, pgoff); +__vma_set_range (vma, start, end);vma_complete (&vp, vmi, vma->vm_mm);validate_mm (vma->vm_mm);return 0 ;
mm/vma.h +1 -1
@@ -298,7 +298,7 @@ void validate_mm(struct mm_struct *mm);__must_check int vma_expand (struct vma_merge_struct *vmg);__must_check int vma_shrink (struct vma_iterator *vmi,struct vm_area_struct *vma, -unsigned long start,unsigned long end,pgoff_t pgoff); +unsigned long start,unsigned long end);static inline int vma_iter_store_gfp (struct vma_iterator *vmi,struct vm_area_struct *vma,gfp_t gfp)
mm/vma_exec.c +1 -1
@@ -89,7 +89,7 @@ int relocate_vma_down(struct vm_area_struct *vma, unsigned long shift)vma_prev (&vmi);/* Shrink the vma to just the new range */ -return vma_shrink (&vmi, vma, new_start, new_end,vma_start_pgoff (vma)); +return vma_shrink (&vmi, vma, new_start, new_end); }/*
tools/testing/vma/tests/merge.c +1 -1
@@ -227,7 +227,7 @@ static bool test_simple_shrink(void)ASSERT_FALSE (attach_vma (&mm, vma)); -ASSERT_FALSE (vma_shrink (&vmi, vma,0 ,0x1000 ,0 )); +ASSERT_FALSE (vma_shrink (&vmi, vma,0 ,0x1000 ));ASSERT_EQ (vma->vm_start,0 );ASSERT_EQ (vma->vm_end,0x1000 );
Signature · 1 line
2.54.0
Re: [PATCH 24/30] mm/vma: update vma_shrink() to not pass unnecessary pgoff parameter
Quoted 11 lines · Lorenzo Stoakes
vma_shrink()does not need to adjustvma->vm_pgoff, we were passing this parameter solely to satisfy vma_set_range()'s requirement for pgoff being specified.Since
vma_set_range()is now isolated tovma.c, we can simply introduce__vma_set_range()which sets only vma->vm_[start, end], and invoke this instead, removing pgoff fromvma_shrink()altogether.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 24/30] mm/vma: update vma_shrink() to not pass unnecessary pgoff parameter
In reply to Lorenzo StoakesQuoted 11 lines · Lorenzo Stoakes
vma_shrink()does not need to adjustvma->vm_pgoff, we were passing this parameter solely to satisfy vma_set_range()'s requirement for pgoff being specified.Since
vma_set_range()is now isolated tovma.c, we can simply introduce__vma_set_range()which sets only vma->vm_[start, end], and invoke this instead, removing pgoff fromvma_shrink()altogether.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 25/30 mm/vma: update vmg_adjust_set_range() to offset pgoff instead
In reply to Lorenzo StoakesWe are calculating the pgoff as an offset, since we have vma_add_pgoff() and vma_sub_pgoff() available, just offset this value directly and use __vma_set_range() for vma->vm_[start, end] values.
We take care to update the range before offsetting the page offset, so the adjusted VMA's vm_start and vm_pgoff are mutually consistent at the point the page offset helpers operate - this matters once vma_set_pgoff() comes to assert invariants which relate the two.
Doing so lays the foundation for future work which allows for use of virtual page offsets for MAP_PRIVATE-file backed mappings.
No functional change intended.
--- mm/vma.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)
mm/vma.c +4 -11
@@ -714,9 +714,6 @@ void validate_mm(struct mm_struct *mm)*/ static void vmg_adjust_set_range (struct vma_merge_struct *vmg) { -struct vm_area_struct *adjust; -pgoff_t pgoff; -if (vmg->__adjust_middle_start) {/* * vmg->start vmg->end
@@ -735,8 +732,8 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)struct vm_area_struct *middle = vmg->middle;const unsigned long delta = vmg->end - middle->vm_start; - pgoff =vma_start_pgoff (middle) + (delta >>PAGE_SHIFT ); - adjust = middle; +__vma_set_range (middle, vmg->end, middle->vm_end); +vma_add_pgoff (middle, delta >>PAGE_SHIFT ); }else if (vmg->__adjust_next_start) {/* * Originally:
@@ -764,13 +761,9 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)struct vm_area_struct *next = vmg->next;const unsigned long delta = next->vm_start - vmg->end; - pgoff =vma_start_pgoff (next) - (delta >>PAGE_SHIFT ); - adjust = next; - }else { -return ; +__vma_set_range (next, vmg->end, next->vm_end); +vma_sub_pgoff (next, delta >>PAGE_SHIFT ); } - -vma_set_range (adjust, vmg->end, adjust->vm_end, pgoff); }/*
Signature · 1 line
2.54.0
Re: [PATCH 25/30] mm/vma: update vmg_adjust_set_range() to offset pgoff instead
Quoted 48 lines · Lorenzo Stoakes
We are calculating the pgoff as an offset, since we have
vma_add_pgoff()andvma_sub_pgoff()available, just offset this value directly and use__vma_set_range()for vma->vm_[start, end] values.We take care to update the range before offsetting the page offset, so the adjusted VMA's
vm_startandvm_pgoffare mutually consistent at the point the page offset helpers operate - this matters oncevma_set_pgoff()comes to assert invariants which relate the two.Doing so lays the foundation for future work which allows for use of virtual page offsets for MAP_PRIVATE-file backed mappings.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/vma.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)mm/vma.c +2
-5@@ -714,9 +714,6 @@ void validate_mm(struct mm_struct *mm)*/ static void vmg_adjust_set_range (struct vma_merge_struct *vmg) { -struct vm_area_struct *adjust; -pgoff_t pgoff; -if (vmg->__adjust_middle_start) {/* * vmg->start vmg->end @@ -735,8 +732,8 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)struct vm_area_struct *middle = vmg->middle;const unsigned long delta = vmg->end - middle->vm_start; - pgoff =vma_start_pgoff (middle) + (delta >>PAGE_SHIFT ); - adjust = middle; +__vma_set_range (middle, vmg->end, middle->vm_end); +vma_add_pgoff (middle, delta >>PAGE_SHIFT ); }else if (vmg->__adjust_next_start) {/* * Originally: @@ -764,13 +761,9 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)struct vm_area_struct *next = vmg->next;const unsigned long delta = next->vm_start - vmg->end;
mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 767
Quoted 10 lines
mm/vma.c +2
-6- pgoff =vma_start_pgoff (next) - (delta >>PAGE_SHIFT ); - adjust = next; - }else { -return ; +__vma_set_range (next, vmg->end, next->vm_end); +vma_sub_pgoff (next, delta >>PAGE_SHIFT ); } - -vma_set_range (adjust, vmg->end, adjust->vm_end, pgoff); }
Maybe this should be squashed with That Other Patch that touches this.
Anyway,
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 25/30] mm/vma: update vmg_adjust_set_range() to offset pgoff instead
Quoted 49 lines · Pedro Falcato
Quoted 48 lines · Lorenzo Stoakes
We are calculating the pgoff as an offset, since we have
vma_add_pgoff()andvma_sub_pgoff()available, just offset this value directly and use__vma_set_range()for vma->vm_[start, end] values.We take care to update the range before offsetting the page offset, so the adjusted VMA's
vm_startandvm_pgoffare mutually consistent at the point the page offset helpers operate - this matters oncevma_set_pgoff()comes to assert invariants which relate the two.Doing so lays the foundation for future work which allows for use of virtual page offsets for MAP_PRIVATE-file backed mappings.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/vma.c | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-)mm/vma.c +2
-5@@ -714,9 +714,6 @@ void validate_mm(struct mm_struct *mm)*/ static void vmg_adjust_set_range (struct vma_merge_struct *vmg) { -struct vm_area_struct *adjust; -pgoff_t pgoff; -if (vmg->__adjust_middle_start) {/* * vmg->start vmg->end @@ -735,8 +732,8 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)struct vm_area_struct *middle = vmg->middle;const unsigned long delta = vmg->end - middle->vm_start; - pgoff =vma_start_pgoff (middle) + (delta >>PAGE_SHIFT ); - adjust = middle; +__vma_set_range (middle, vmg->end, middle->vm_end); +vma_add_pgoff (middle, delta >>PAGE_SHIFT ); }else if (vmg->__adjust_next_start) {/* * Originally: @@ -764,13 +761,9 @@ static void vmg_adjust_set_range(struct vma_merge_struct *vmg)struct vm_area_struct *next = vmg->next;const unsigned long delta = next->vm_start - vmg->end;
Quoted 12 lines
mm/vma.c static void vmg_adjust_set_range(struct vma_merge_struct *vmg) · line 767
Quoted 10 lines
mm/vma.c +2
-6- pgoff =vma_start_pgoff (next) - (delta >>PAGE_SHIFT ); - adjust = next; - }else { -return ; +__vma_set_range (next, vmg->end, next->vm_end); +vma_sub_pgoff (next, delta >>PAGE_SHIFT ); } - -vma_set_range (adjust, vmg->end, adjust->vm_end, pgoff); }Maybe this should be squashed with That Other Patch that touches this.
Ah this separation makes more sense from the point of view of the virt pgoff stuff in the RFC (see [0]).
And would rather keep as vaguely bitesized as possible :>)
Quoted 4 lines
Anyway,
Reviewed-by Pedro Falcato <[email protected]>
Thanks!
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Re: [PATCH 25/30] mm/vma: update vmg_adjust_set_range() to offset pgoff instead
In reply to Lorenzo StoakesQuoted 9 lines · Lorenzo Stoakes
We are calculating the pgoff as an offset, since we have
vma_add_pgoff()andvma_sub_pgoff()available, just offset this value directly and use__vma_set_range()for vma->vm_[start, end] values.We take care to update the range before offsetting the page offset, so the adjusted VMA's
vm_startandvm_pgoffare mutually consistent at the point the page offset helpers operate - this matters oncevma_set_pgoff()comes to assert invariants which relate the two.
Given the prior code just did it in 3 lines of code with no special dance, there's a presumption here that this is done in a critical section and no external actor can see the inconsistent state?
(i.e. range is updated but offset is not)
Just think about this from a bisection/debugging standpoint.
Quoted 6 lines
Doing so lays the foundation for future work which allows for use of virtual page offsets for MAP_PRIVATE-file backed mappings.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 26/30 mm/vma: introduce and use vma_set_pgoff()
In reply to Lorenzo StoakesIn order to lay the foundation for work that permits us to track the virtual page offset of MAP_PRIVATE file-backed mappings, we abstract the assignment of vma->vm_pgoff to vma_set_pgoff().
We additionally add a lock check here using the newly introduced vma_assert_can_modify(). This asserts the VMA write lock if the VMA is attached.
We also assert that, if this is an anonymous VMA and unfaulted, that its (virtual) page offset is equal to the page offset of the VMA's address.
In order to maintain correctness given this assert, we also update __install_special_mapping() to invoke vma_set_range() after it's set vma->vm_ops (which determine whether the VMA is anonymous or not).
We do not use vma_set_pgoff() in vm_area_init_from(), as at the point of forking, we don't necessarily have correct locking state.
Updating vma_set_range() covers most cases, but in addition to this we also update insert_vm_struct(), compat_set_vma_from_desc() and nommu callers.
We also update vma_add_pgoff() and vma_sub_pgoff() to use vma_set_pgoff().
While we're here, we drop a BUG_ON() and update insert_vm_struct()'s comment to reflect the fact anonymous mappings can be added here.
No functional change intended.
--- mm/nommu.c | 2 +- mm/vma.c | 14 +++++++------- mm/vma.h | 15 ++++++++++++--- tools/testing/vma/include/dup.h | 2 +- 4 files changed, 21 insertions(+), 12 deletions(-)
mm/nommu.c +1 -1
@@ -1059,7 +1059,7 @@ unsigned long do_mmap(struct file *file, region->vm_pgoff = pgoff;vm_flags_init (vma, vm_flags); - vma->vm_pgoff = pgoff; +vma_set_pgoff (vma, pgoff);if (file) { region->vm_file =get_file (file);
mm/vma.c +7 -7
@@ -81,7 +81,7 @@ static void vma_set_range(struct vm_area_struct *vma, unsigned long start, unsigned long end, pgoff_t pgoff) {__vma_set_range (vma ,start ,end ); - vma->vm_pgoff = pgoff; +vma_set_pgoff (vma, pgoff); }/* Was this VMA ever forked from a parent, i.e. maybe contains CoW mappings? */
@@ -3345,9 +3345,9 @@ int __vm_munmap(unsigned long start, size_t len, bool unlock)return ret; } -/* Insert vm structure into process list sorted by address -* and into the inode's i_mmap tree. If vm_file is non-NULL -* then i_mmap_rwsem is taken here. +/* +* Insert vm structure into process list sorted by address +* and into the inode's i_mmap tree if file-backed. */ int insert_vm_struct (struct mm_struct *mm,struct vm_area_struct *vma) {
@@ -3373,8 +3373,8 @@ int insert_vm_struct(struct mm_struct *mm, struct vm_area_struct *vma)* Similarly in do_mmap and in do_brk_flags. */ if (vma_is_anonymous (vma)) { -BUG_ON (vma->anon_vma); - vma->vm_pgoff = vma->vm_start >>PAGE_SHIFT ; +WARN_ON_ONCE (vma->anon_vma); +vma_set_pgoff (vma, vma->vm_start >>PAGE_SHIFT ); }if (vma_link (mm, vma)) {
@@ -3420,7 +3420,6 @@ struct vm_area_struct *__install_special_mapping(if (unlikely (vma ==NULL ))return ERR_PTR (-ENOMEM ); -vma_set_range (vma, addr, addr + len,0 ); vm_flags |= mm->def_flags |VM_DONTEXPAND ;if (pgtable_supports_soft_dirty ()) vm_flags |=VM_SOFTDIRTY ;
@@ -3429,6 +3428,7 @@ struct vm_area_struct *__install_special_mapping( vma->vm_ops = ops; vma->vm_private_data = priv; +vma_set_range (vma, addr, addr + len,0 ); ret =insert_vm_struct (mm, vma);if (ret)
mm/vma.h +12 -3
@@ -247,16 +247,25 @@ static inline pgoff_t vmg_end_pgoff(const struct vma_merge_struct *vmg)return vmg_start_pgoff (vmg) +vmg_pages (vmg); } +static inline void vma_set_pgoff (struct vm_area_struct *vma,pgoff_t pgoff) +{ +vma_assert_can_modify (vma); + +VM_WARN_ON_ONCE (vma_is_anonymous (vma) && !vma->anon_vma && + pgoff != vma->vm_start >>PAGE_SHIFT ); + vma->vm_pgoff = pgoff; +} +static inline void vma_add_pgoff (struct vm_area_struct *vma,pgoff_t delta) {vma_assert_can_modify (vma); - vma->vm_pgoff += delta; +vma_set_pgoff (vma,vma_start_pgoff (vma) + delta); }static inline void vma_sub_pgoff (struct vm_area_struct *vma,pgoff_t delta) {vma_assert_can_modify (vma); - vma->vm_pgoff -= delta; +vma_set_pgoff (vma,vma_start_pgoff (vma) - delta); }#define VMG_STATE (name, mm_, vmi_, start_, end_, vma_flags_, pgoff_) \
@@ -332,7 +341,7 @@ static inline void compat_set_vma_from_desc(struct vm_area_struct *vma,*/ /* Mutable fields. Populated with initial state. */ - vma->vm_pgoff = desc->pgoff; +vma_set_pgoff (vma, desc->pgoff);if (desc->vm_file != vma->vm_file)vma_set_file (vma, desc->vm_file); vma->flags = desc->vma_flags;
tools/testing/vma/include/dup.h +1 -1
@@ -1186,7 +1186,7 @@ static inline void vma_assert_can_modify(struct vm_area_struct *vma)static inline void vma_assert_detached (struct vm_area_struct *vma) { -WARN_ON_ONCE (refcount_read (&vma->vm_refcnt)); +WARN_ON_ONCE (vma_is_attached (vma)); }static inline void vma_assert_write_locked (struct vm_area_struct *);
Signature · 1 line
2.54.0
Re: [PATCH 26/30] mm/vma: introduce and use vma_set_pgoff()
Quoted 21 lines · Lorenzo Stoakes
In order to lay the foundation for work that permits us to track the virtual page offset of
MAP_PRIVATEfile-backed mappings, we abstract the assignment ofvma->vm_pgofftovma_set_pgoff().We additionally add a lock check here using the newly introduced
vma_assert_can_modify(). This asserts the VMA write lock if the VMA is attached.We also assert that, if this is an anonymous VMA and unfaulted, that its (virtual) page offset is equal to the page offset of the VMA's address.
In order to maintain correctness given this assert, we also update
__install_special_mapping()to invokevma_set_range()after it's setvma->vm_ops(which determine whether the VMA is anonymous or not).We do not use
vma_set_pgoff()invm_area_init_from(), as at the point of forking, we don't necessarily have correct locking state.Updating
vma_set_range()covers most cases, but in addition to this we also updateinsert_vm_struct(),compat_set_vma_from_desc()and nommu callers.
Quoted 8 lines
We also update
vma_add_pgoff()andvma_sub_pgoff()to usevma_set_pgoff().While we're here, we drop a
BUG_ON()and update insert_vm_struct()'s comment to reflect the fact anonymous mappings can be added here.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Re: [PATCH 26/30] mm/vma: introduce and use vma_set_pgoff()
In reply to Lorenzo StoakesQuoted 21 lines · Lorenzo Stoakes
In order to lay the foundation for work that permits us to track the virtual page offset of
MAP_PRIVATEfile-backed mappings, we abstract the assignment ofvma->vm_pgofftovma_set_pgoff().We additionally add a lock check here using the newly introduced
vma_assert_can_modify(). This asserts the VMA write lock if the VMA is attached.We also assert that, if this is an anonymous VMA and unfaulted, that its (virtual) page offset is equal to the page offset of the VMA's address.
In order to maintain correctness given this assert, we also update
__install_special_mapping()to invokevma_set_range()after it's setvma->vm_ops(which determine whether the VMA is anonymous or not).We do not use
vma_set_pgoff()invm_area_init_from(), as at the point of forking, we don't necessarily have correct locking state.Updating
vma_set_range()covers most cases, but in addition to this we also updateinsert_vm_struct(),compat_set_vma_from_desc()and nommu callers.
Quoted 8 lines
We also update
vma_add_pgoff()andvma_sub_pgoff()to usevma_set_pgoff().While we're here, we drop a
BUG_ON()and update insert_vm_struct()'s comment to reflect the fact anonymous mappings can be added here.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 27/30 mm/vma: correct incorrect vma.h inclusion
In reply to Lorenzo StoakesThe only files which should be including vma.h are the implementation files for the core VMA logic - vma.c, vma_init.c, and vma_exec.c.
This is in order to allow for userland testing of core VMA logic. In this cases, vma_internal.h and vma.h are included, providing both the dependencies upon which the core VMA logic requires and its declarations.
Userland testable VMA logic is achieved by having separate vma_internal.h implementations for userland and kernel.
Callers other than the core VMA implementation should include internal.h instead. This header does not need to include vma_internal.h as it only contains the vma.h declarations, for which the includes already present suffice.
Update code to reflect this, update comments to reflect the fact there are 3 VMA implementation files and document things more clearly.
While we're here, slightly improve the language of the comment describing vma_exec.c.
No functional change intended.
--- mm/mmu_notifier.c | 2 +- mm/nommu.c | 1 - mm/vma.c | 4 ++++ mm/vma.h | 9 ++++++++- mm/vma_exec.c | 8 ++++++-- mm/vma_init.c | 4 ++++ mm/vma_internal.h | 4 ++-- 7 files changed, 25 insertions(+), 7 deletions(-)
mm/mmu_notifier.c +1 -1
@@ -19,7 +19,7 @@#include <linux/sched/mm.h> #include <linux/slab.h> -#include "vma.h" +#include "internal.h" /* global SRCU for all MMs */ DEFINE_STATIC_SRCU (srcu);
mm/nommu.c +0 -1
@@ -41,7 +41,6 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include "internal.h" -#include "vma.h" unsigned long highest_memmap_pfn;int heap_stack_gap =0 ;
mm/vma.c +4 -0
@@ -4,6 +4,10 @@* VMA-specific functions. */ +/* +* To allow for userland testing we place internal dependencies in +* vma_internal.h and external VMA API declarations in vma.h. +*/ #include "vma_internal.h" #include "vma.h"
mm/vma.h +8 -1
@@ -2,7 +2,14 @@/* * vma.h * -* Core VMA manipulation API implemented in vma.c. +* Core VMA manipulation API implemented in vma.c, vma_init.c and vma_exec.c. +* +* Note that, in order for VMA logic to be userland testable, this header +* intentionally includes no dependencies. +* +* This is specifically scoped to mm-only. Users of this functionality (other +* than the core VMA implementation itself) should not include this header +* directly, but rather include internal.h. */ #ifndef __MM_VMA_H#define __MM_VMA_H
mm/vma_exec.c +6 -2
@@ -1,10 +1,14 @@// SPDX-License-Identifier: GPL-2.0-only /* -* Functions explicitly implemented for exec functionality which however are -* explicitly VMA-only logic. +* Functions provided for exec functionality which however are +* specifically VMA-only logic. */ +/* +* To allow for userland testing we place internal dependencies in +* vma_internal.h and external VMA API declarations in vma.h. +*/ #include "vma_internal.h" #include "vma.h"
mm/vma_init.c +4 -0
@@ -5,6 +5,10 @@* between CONFIG_MMU and non-CONFIG_MMU kernel configurations. */ +/* +* To allow for userland testing we place internal dependencies in +* vma_internal.h and external VMA API declarations in vma.h. +*/ #include "vma_internal.h" #include "vma.h"
mm/vma_internal.h +2 -2
@@ -2,8 +2,8 @@/* * vma_internal.h * -* Headers required by vma.c, which can be substituted accordingly when testing -* VMA functionality. +* Headers required by vma.c, vma_init.c and vma_exec.c, which can be +* substituted accordingly when testing VMA functionality. */ #ifndef __MM_VMA_INTERNAL_H
Signature · 1 line
2.54.0
Re: [PATCH 27/30] mm/vma: correct incorrect vma.h inclusion
Quoted 15 lines · Lorenzo Stoakes
The only files which should be including
vma.hare the implementation files for the core VMA logic -vma.c,vma_init.c, andvma_exec.c.This is in order to allow for userland testing of core VMA logic. In this cases,
vma_internal.handvma.hare included, providing both the dependencies upon which the core VMA logic requires and its declarations.Userland testable VMA logic is achieved by having separate
vma_internal.himplementations for userland and kernel.Callers other than the core VMA implementation should include
internal.hinstead. This header does not need to includevma_internal.has it only contains thevma.hdeclarations, for which the includes already present suffice.
Quoted 5 lines
Update code to reflect this, update comments to reflect the fact there are 3 VMA implementation files and document things more clearly.
While we're here, slightly improve the language of the comment describing
vma_exec.c.
Two random thoughts:
1) perhaps vma.h -> vma_private.h
2) https://lore.kernel.org/all/CAHk-=wghMm2c+AYEcwYY7drSVXB27DYqc-ZXpFiq=XFs-w59wA@mail.gmail.com/
mm/vma/whatever.c :) would PROBABLY solve the issue of people snooping vma.h
Quoted 4 lines
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 124 lines
--- mm/mmu_notifier.c | 2 +- mm/nommu.c | 1 - mm/vma.c | 4 ++++ mm/vma.h | 9 ++++++++- mm/vma_exec.c | 8 ++++++-- mm/vma_init.c | 4 ++++ mm/vma_internal.h | 4 ++-- 7 files changed, 25 insertions(+), 7 deletions(-)mm/mmu_notifier.c +1
-1@@ -19,7 +19,7 @@#include <linux/sched/mm.h> #include <linux/slab.h> -#include "vma.h" +#include "internal.h" /* global SRCU for all MMs */ DEFINE_STATIC_SRCU (srcu);mm/nommu.c +0
-1@@ -41,7 +41,6 @@#include <asm/tlbflush.h> #include <asm/mmu_context.h> #include "internal.h" -#include "vma.h" unsigned long highest_memmap_pfn;int heap_stack_gap =0 ;mm/vma.c +4
-0@@ -4,6 +4,10 @@* VMA-specific functions. */ +/* +* To allow for userland testing we place internal dependencies in +* vma_internal.h and external VMA API declarations in vma.h. +*/ #include "vma_internal.h" #include "vma.h" mm/vma.h +8
-1@@ -2,7 +2,14 @@/* * vma.h * -* Core VMA manipulation API implemented in vma.c. +* Core VMA manipulation API implemented in vma.c, vma_init.c and vma_exec.c. +* +* Note that, in order for VMA logic to be userland testable, this header +* intentionally includes no dependencies. +* +* This is specifically scoped to mm-only. Users of this functionality (other +* than the core VMA implementation itself) should not include this header +* directly, but rather include internal.h. */ #ifndef __MM_VMA_H#define __MM_VMA_Hmm/vma_exec.c +6
-2@@ -1,10 +1,14 @@// SPDX-License-Identifier: GPL-2.0-only /* -* Functions explicitly implemented for exec functionality which however are -* explicitly VMA-only logic. +* Functions provided for exec functionality which however are +* specifically VMA-only logic. */ +/* +* To allow for userland testing we place internal dependencies in +* vma_internal.h and external VMA API declarations in vma.h. +*/ #include "vma_internal.h" #include "vma.h" mm/vma_init.c +4
-0@@ -5,6 +5,10 @@* between CONFIG_MMU and non-CONFIG_MMU kernel configurations. */ +/* +* To allow for userland testing we place internal dependencies in +* vma_internal.h and external VMA API declarations in vma.h. +*/ #include "vma_internal.h" #include "vma.h" mm/vma_internal.h +2
-2@@ -2,8 +2,8 @@/* * vma_internal.h * -* Headers required by vma.c, which can be substituted accordingly when testing -* VMA functionality. +* Headers required by vma.c, vma_init.c and vma_exec.c, which can be +* substituted accordingly when testing VMA functionality. */ #ifndef __MM_VMA_INTERNAL_HSignature · 1 line
2.54.0
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 27/30] mm/vma: correct incorrect vma.h inclusion
Quoted 16 lines · Pedro Falcato
Quoted 15 lines · Lorenzo Stoakes
The only files which should be including
vma.hare the implementation files for the core VMA logic -vma.c,vma_init.c, andvma_exec.c.This is in order to allow for userland testing of core VMA logic. In this cases,
vma_internal.handvma.hare included, providing both the dependencies upon which the core VMA logic requires and its declarations.Userland testable VMA logic is achieved by having separate
vma_internal.himplementations for userland and kernel.Callers other than the core VMA implementation should include
internal.hinstead. This header does not need to includevma_internal.has it only contains thevma.hdeclarations, for which the includes already present suffice.
Quoted 8 lines
Quoted 5 lines
Update code to reflect this, update comments to reflect the fact there are 3 VMA implementation files and document things more clearly.
While we're here, slightly improve the language of the comment describing
vma_exec.c.Two random thoughts:
1) perhapsvma.h->vma_private.h
Not a bad idea thanks!
Quoted 2 lines
2) https://lore.kernel.org/all/CAHk-=wghMm2c+AYEcwYY7drSVXB27DYqc-ZXpFiq=XFs-w59wA@mail.gmail.com/
mm/vma/whatever.c:) would PROBABLY solve the issue of people snoopingvma.h
I think a vma/ subdir would probably confuse things further, I think renaming to vma_private.h neatly solves it actually, along with a comment maybe in the header itself?
Then again mm/vma/{vma.c,init.c, exec.c, private.h} isn't too crazy either. Though vma.h is the actual 'shared' bit, and vma_internal.h is the private bit that the userland changes. But could switch things around vma.h -> mm/vma/internal.h that mm/internal.h imports, and mm/vma_internal.h becomes mm/vma/private.h.
I think that could work but definitely a follow-up!
Quoted 7 lines
Quoted 5 lines
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>Reviewed-by Pedro Falcato <[email protected]>
Thanks!
Cheers, Lorenzo
Re: [PATCH 27/30] mm/vma: correct incorrect vma.h inclusion
In reply to Lorenzo StoakesQuoted 18 lines · Lorenzo Stoakes
The only files which should be including
vma.hare the implementation files for the core VMA logic -vma.c,vma_init.c, andvma_exec.c.This is in order to allow for userland testing of core VMA logic. In this cases,
vma_internal.handvma.hare included, providing both the dependencies upon which the core VMA logic requires and its declarations.Userland testable VMA logic is achieved by having separate
vma_internal.himplementations for userland and kernel.Callers other than the core VMA implementation should include
internal.hinstead. This header does not need to includevma_internal.has it only contains thevma.hdeclarations, for which the includes already present suffice.Update code to reflect this, update comments to reflect the fact there are 3 VMA implementation files and document things more clearly.
Quoted 6 lines
While we're here, slightly improve the language of the comment describing
vma_exec.c.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Quoted 4 lines
+/*
+ * To allow for userland testing we place internal dependencies in
+ *vma_internal.hand external VMA API declarations invma.h.
+ */
...
Quoted 4 lines
+/*
+ * To allow for userland testing we place internal dependencies in
+ *vma_internal.hand external VMA API declarations invma.h.
+ */
...
Quoted 4 lines
+/*
+ * To allow for userland testing we place internal dependencies in
+ *vma_internal.hand external VMA API declarations invma.h.
+ */
Do you actually need 3 copies of this comment or just one copy in vma_internal.h?
Lorenzo Stoakes
maintainerRe: [PATCH 27/30] mm/vma: correct incorrect vma.h inclusion
Quoted 37 lines · Gregory Price
Quoted 18 lines · Lorenzo Stoakes
The only files which should be including
vma.hare the implementation files for the core VMA logic -vma.c,vma_init.c, andvma_exec.c.This is in order to allow for userland testing of core VMA logic. In this cases,
vma_internal.handvma.hare included, providing both the dependencies upon which the core VMA logic requires and its declarations.Userland testable VMA logic is achieved by having separate
vma_internal.himplementations for userland and kernel.Callers other than the core VMA implementation should include
internal.hinstead. This header does not need to includevma_internal.has it only contains thevma.hdeclarations, for which the includes already present suffice.Update code to reflect this, update comments to reflect the fact there are 3 VMA implementation files and document things more clearly.
Quoted 6 lines
While we're here, slightly improve the language of the comment describing
vma_exec.c.No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>Reviewed-by Gregory Price <[email protected]>Quoted 4 lines
+/*
+ * To allow for userland testing we place internal dependencies in
+ *vma_internal.hand external VMA API declarations invma.h.
+ */...
Quoted 4 lines
+/*
+ * To allow for userland testing we place internal dependencies in
+ *vma_internal.hand external VMA API declarations invma.h.
+ */
Quoted 8 lines
...
Quoted 4 lines
+/*
+ * To allow for userland testing we place internal dependencies in
+ *vma_internal.hand external VMA API declarations invma.h.
+ */Do you actually need 3 copies of this comment or just one copy in
vma_internal.h?
I'd rather have it at a glance, it's a bit silly but C headers are silly :P
BTW you're kinda racing against time here as I'm on the verge of sending v2 :P
Gregory Price
maintainerRe: [PATCH 27/30] mm/vma: correct incorrect vma.h inclusion
Quoted 8 lines · Lorenzo Stoakes
Quoted 3 lines
Do you actually need 3 copies of this comment or just one copy in
vma_internal.h?I'd rather have it at a glance, it's a bit silly but C headers are silly :P
BTW you're kinda racing against time here as I'm on the verge of sending v2 :P
:[ i don't refresh my inbox when i get hyperfocused and now i am sad
Lorenzo Stoakes
maintainerRe: [PATCH 27/30] mm/vma: correct incorrect vma.h inclusion
Quoted 11 lines · Gregory Price
Quoted 8 lines · Lorenzo Stoakes
Quoted 3 lines
Do you actually need 3 copies of this comment or just one copy in
vma_internal.h?I'd rather have it at a glance, it's a bit silly but C headers are silly :P
BTW you're kinda racing against time here as I'm on the verge of sending v2 :P
:[ i don't refresh my inbox when i get hyperfocused and now i am sad
Oh no! But that hyperfocus sounds familiar ;)
Apologies for that, I did hold off some hours. But you can at least see that I did all you asked in v1 :)
Cheers, Lorenzo
Lorenzo Stoakes
maintainerPATCH 28/30 mm/vma: use guard clauses in can_vma_merge_[before, after]()
In reply to Lorenzo StoakesRather than combining a bunch of conditionals in a single expression, simplify by inverting the mergeability requirements into guard clauses.
that is - instead of checking what must be true for the conditions to be met, instead check the inverse of the requirements and return false if any are true, defaulting to true.
No functional change intended.
--- mm/vma.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-)
mm/vma.c +14 -13
@@ -215,13 +215,13 @@ static void init_multi_vma_prep(struct vma_prepare *vp,*/ static bool can_vma_merge_before (struct vma_merge_struct *vmg) { -if (is_mergeable_vma (vmg,/* merge_next = */ true) && -is_mergeable_anon_vma (vmg,/* merge_next = */ true)) { -if (vmg_end_pgoff (vmg) ==vma_start_pgoff (vmg->next)) -return true; - } - -return false; +if (!is_mergeable_vma (vmg,/* merge_next = */ true)) +return false; +if (!is_mergeable_anon_vma (vmg,/* merge_next = */ true)) +return false; +if (vmg_end_pgoff (vmg) !=vma_start_pgoff (vmg->next)) +return false; +return true; }/*
@@ -235,12 +235,13 @@ static bool can_vma_merge_before(struct vma_merge_struct *vmg)*/ static bool can_vma_merge_after (struct vma_merge_struct *vmg) { -if (is_mergeable_vma (vmg,/* merge_next = */ false) && -is_mergeable_anon_vma (vmg,/* merge_next = */ false)) { -if (vma_end_pgoff (vmg->prev) ==vmg_start_pgoff (vmg)) -return true; - } -return false; +if (!is_mergeable_vma (vmg,/* merge_next = */ false)) +return false; +if (!is_mergeable_anon_vma (vmg,/* merge_next = */ false)) +return false; +if (vma_end_pgoff (vmg->prev) !=vmg_start_pgoff (vmg)) +return false; +return true; } static void __vma_link_file(struct vm_area_struct *vma,
Signature · 1 line
2.54.0
Re: [PATCH 28/30] mm/vma: use guard clauses in can_vma_merge_[before, after]()
Quoted 49 lines · Lorenzo Stoakes
Rather than combining a bunch of conditionals in a single expression, simplify by inverting the mergeability requirements into guard clauses.
that is - instead of checking what must be true for the conditions to be met, instead check the inverse of the requirements and return false if any are true, defaulting to true.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>--- mm/vma.c | 27 ++++++++++++++------------- 1 file changed, 14 insertions(+), 13 deletions(-)mm/vma.c +7
-13@@ -215,13 +215,13 @@ static void init_multi_vma_prep(struct vma_prepare *vp,*/ static bool can_vma_merge_before (struct vma_merge_struct *vmg) { -if (is_mergeable_vma (vmg,/* merge_next = */ true) && -is_mergeable_anon_vma (vmg,/* merge_next = */ true)) { -if (vmg_end_pgoff (vmg) ==vma_start_pgoff (vmg->next)) -return true; - } - -return false; +if (!is_mergeable_vma (vmg,/* merge_next = */ true)) +return false; +if (!is_mergeable_anon_vma (vmg,/* merge_next = */ true)) +return false; +if (vmg_end_pgoff (vmg) !=vma_start_pgoff (vmg->next)) +return false; +return true; }/* @@ -235,12 +235,13 @@ static bool can_vma_merge_before(struct vma_merge_struct *vmg)*/ static bool can_vma_merge_after (struct vma_merge_struct *vmg) { -if (is_mergeable_vma (vmg,/* merge_next = */ false) && -is_mergeable_anon_vma (vmg,/* merge_next = */ false)) { -if (vma_end_pgoff (vmg->prev) ==vmg_start_pgoff (vmg)) -return true; - } -return false;
mm/vma.c static bool can_vma_merge_before(struct vma_merge_struct *vmg) · line 247
Quoted 10 lines
mm/vma.c +7
-0+if (!is_mergeable_vma (vmg,/* merge_next = */ false)) +return false; +if (!is_mergeable_anon_vma (vmg,/* merge_next = */ false)) +return false; +if (vma_end_pgoff (vmg->prev) !=vmg_start_pgoff (vmg)) +return false; +return true; } static void __vma_link_file(struct vm_area_struct *vma,
Looks nicer, thanks.
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 28/30] mm/vma: use guard clauses in can_vma_merge_[before, after]()
Quoted 1 line · Pedro Falcato
Looks nicer, thanks.
Thanks!
Quoted 2 lines
Reviewed-by Pedro Falcato <[email protected]>
Also thanks! :)
Quoted 3 lines
--
Pedro
Cheers, Lorenzo
Re: [PATCH 28/30] mm/vma: use guard clauses in can_vma_merge_[before, after]()
In reply to Lorenzo StoakesQuoted 10 lines · Lorenzo Stoakes
Rather than combining a bunch of conditionals in a single expression, simplify by inverting the mergeability requirements into guard clauses.
that is - instead of checking what must be true for the conditions to be met, instead check the inverse of the requirements and return false if any are true, defaulting to true.
No functional change intended.
Signed-off-by Lorenzo Stoakes <[email protected]>
Lorenzo Stoakes
maintainerPATCH 29/30 tools/testing/vma: default VMA flag bits to 64-bit
In reply to Lorenzo StoakesWith all of the sanitisers turned on, setting the VMA flag bits depth to 128 by default results in overly long build times.
Reduce this to 64 - we can always manipulate these later for testing of larger bitmaps as needed.
--- tools/testing/vma/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tools/testing/vma/Makefile +1 -1
@@ -10,7 +10,7 @@ OFILES = $(SHARED_OFILES) main.o shared.o maple-shim.o TARGETS = vma # These can be varied to test different sizes. -CFLAGS += -DNUM_VMA_FLAG_BITS=128 -DNUM_MM_FLAG_BITS=128 +CFLAGS += -DNUM_VMA_FLAG_BITS=64 -DNUM_MM_FLAG_BITS=64 main.o: main.c shared.c shared.h vma_internal.h tests/merge.c tests/mmap.c tests/vma.c ../../../mm/vma.c ../../../mm/vma_init.c ../../../mm/vma_exec.c ../../../mm/vma.h include/custom.h include/dup.h include/stubs.h
Signature · 1 line
2.54.0
Pedro Falcato
reviewerRe: [PATCH 29/30] tools/testing/vma: default VMA flag bits to 64-bit
Quoted 6 lines · Lorenzo Stoakes
With all of the sanitisers turned on, setting the VMA flag bits depth to 128 by default results in overly long build times.
Reduce this to 64 - we can always manipulate these later for testing of larger bitmaps as needed.
Hmm, what's the problem with the sanitizers? Shouldn't this just result in slightly different codegen?
Quoted 21 lines
Signed-off-by Lorenzo Stoakes <[email protected]>--- tools/testing/vma/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/vma/Makefile +1
-1@@ -10,7 +10,7 @@ OFILES = $(SHARED_OFILES) main.o shared.o maple-shim.o TARGETS = vma # These can be varied to test different sizes. -CFLAGS += -DNUM_VMA_FLAG_BITS=128 -DNUM_MM_FLAG_BITS=128 +CFLAGS += -DNUM_VMA_FLAG_BITS=64 -DNUM_MM_FLAG_BITS=64 main.o: main.c shared.c shared.h vma_internal.h tests/merge.c tests/mmap.c tests/vma.c ../../../mm/vma.c ../../../mm/vma_init.c ../../../mm/vma_exec.c ../../../mm/vma.h include/custom.h include/dup.h include/stubs.hSignature · 1 line
2.54.0
Signature · 1 line
Pedro
Lorenzo Stoakes
maintainerRe: [PATCH 29/30] tools/testing/vma: default VMA flag bits to 64-bit
Quoted 10 lines · Pedro Falcato
Quoted 6 lines · Lorenzo Stoakes
With all of the sanitisers turned on, setting the VMA flag bits depth to 128 by default results in overly long build times.
Reduce this to 64 - we can always manipulate these later for testing of larger bitmaps as needed.
Hmm, what's the problem with the sanitizers? Shouldn't this just result in slightly different codegen?
I'm not sure but it results in vastly longer build times. It was a fun idea but it partly defeats the point of the VMA userland tests.
Rather than spend too long investigating I'd rather we move to a sane default.
Quoted 17 lines
Quoted 17 lines
Signed-off-by Lorenzo Stoakes <[email protected]>--- tools/testing/vma/Makefile | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)tools/testing/vma/Makefile +1
-1@@ -10,7 +10,7 @@ OFILES = $(SHARED_OFILES) main.o shared.o maple-shim.o TARGETS = vma # These can be varied to test different sizes. -CFLAGS += -DNUM_VMA_FLAG_BITS=128 -DNUM_MM_FLAG_BITS=128 +CFLAGS += -DNUM_VMA_FLAG_BITS=64 -DNUM_MM_FLAG_BITS=64
Quoted 8 lines
Quoted 5 lines
tools/testing/vma/Makefile +0
-1main.o: main.c shared.c shared.h vma_internal.h tests/merge.c tests/mmap.c tests/vma.c ../../../mm/vma.c ../../../mm/vma_init.c ../../../mm/vma_exec.c ../../../mm/vma.h include/custom.h include/dup.h include/stubs.h --2.54.0
--
Pedro
Thanks, Lorenzo
Re: [PATCH 29/30] tools/testing/vma: default VMA flag bits to 64-bit
In reply to Lorenzo StoakesQuoted 7 lines · Lorenzo Stoakes
With all of the sanitisers turned on, setting the VMA flag bits depth to 128 by default results in overly long build times.
Reduce this to 64 - we can always manipulate these later for testing of larger bitmaps as needed.
Signed-off-by Lorenzo Stoakes <[email protected]>
Seems like this causes instrumented inlining to go crazy.
static inline void bitmap_or(...) {
if (small_const_nbits(nbits))
*dst = *src1 | *src2;
else
__bitmap_or(dst, src1, src2, nbits);
^^^^ this branch is being hit ^^^^
}tl;dr: __bitmap_or() gets emitted and then the sanitizers have to instrument a ton of inlined functions instead of the compiler simply injecting a couple 'or' instructions.
if you wanted to keep the coverage you could do something like
# These can be varied to test different sizes.NUM_VMA_FLAG_BITS ?= 64NUM_MM_FLAG_BITS ?= 64
CFLAGS += -DNUM_VMA_FLAG_BITS=$(NUM_VMA_FLAG_BITS) -DNUM_MM_FLAG_BITS=$(NUM_MM_FLAG_BITS)
w/
make NUM_VMA_FLAG_BITS=128
would let you still test the branch but not by default.
anyway:
~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 29/30] tools/testing/vma: default VMA flag bits to 64-bit
Quoted 30 lines · Gregory Price
Quoted 7 lines · Lorenzo Stoakes
With all of the sanitisers turned on, setting the VMA flag bits depth to 128 by default results in overly long build times.
Reduce this to 64 - we can always manipulate these later for testing of larger bitmaps as needed.
Signed-off-by Lorenzo Stoakes <[email protected]>Seems like this causes instrumented inlining to go crazy.
static inline void bitmap_or(...) { if (small_const_nbits(nbits)) *dst = *src1 | *src2; else __bitmap_or(dst, src1, src2, nbits); ^^^^ this branch is being hit ^^^^ }tl;dr:
__bitmap_or()gets emitted and then the sanitizers have to instrument a ton of inlined functions instead of the compiler simply injecting a couple 'or' instructions.if you wanted to keep the coverage you could do something like
# These can be varied to test different sizes.
NUM_VMA_FLAG_BITS?= 64NUM_MM_FLAG_BITS?= 64
CFLAGS += -DNUM_VMA_FLAG_BITS=$(NUM_VMA_FLAG_BITS) -DNUM_MM_FLAG_BITS=$(NUM_MM_FLAG_BITS)
Quoted 5 lines
w/
make NUM_VMA_FLAG_BITS=128
would let you still test the branch but not by default.
Yeah I need to make it easier to set.
And that's a pity re: the sanitisers, the generated code should still be efficient (will investigate some more over time) but yeah puts a lot more pressure on the sanitisers for sure.
Is there a way maybe to configure the compiler to not instrument like that or somehow derive things? Or maybe it's inherent... I am kinda fine with assuming the bitmap stuff is OK!
Quoted 4 lines
anyway:
Reviewed-by Gregory Price <[email protected]>
Sorry but I'm sending out the v2 I thought maybe you were done :|| and I'm so tired with this heatwave + UK houses being like ovens in the summer (cold in the winter!) that I just needed to get it sending.
Could you pop it over on the relevant patch on the v2? Once the sending stops, b4 went crazy and added half the kernel so it's taking a while ;)
Quoted 2 lines
~Gregory
Cheers, Lorenzo
Lorenzo Stoakes
maintainerPATCH 30/30 tools/testing/vma: output compared expression on ASSERT_[EQ, NE]()
In reply to Lorenzo StoakesUpdate the macros to output the compared values at hex for easier debugging when test asserts fail.
Also remove unused IS_SET() macro.
--- tools/testing/vma/shared.h | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-)
tools/testing/vma/shared.h +19 -12
@@ -21,19 +21,28 @@ } \ } while (0) -#define ASSERT_TRUE (_expr) \ - do { \ - if (!(_expr)) { \ - fprintf(stderr, \ - "Assert FAILED at %s:%d:%s(): %s is FALSE.\n", \ - __FILE__, __LINE__, __FUNCTION__, #_expr); \ - return false; \ - } \ +#define __ASSERT_TRUE (_expr, _fmt, ...) \ + do { \ + if (!(_expr)) { \ + fprintf(stderr, \ + "Assert FAILED at %s:%d:%s(): %s is FALSE" \ + _fmt ".\n", \ + __FILE__, __LINE__, __FUNCTION__, #_expr \ + __VA_OPT__(,) __VA_ARGS__); \ + return false; \ + } \ } while (0) +#define __TO_SCALAR (x) ((unsigned long long)(uintptr_t)(x)) + +#define ASSERT_TRUE (_expr) __ASSERT_TRUE(_expr, "")#define ASSERT_FALSE (_expr) ASSERT_TRUE(!(_expr)) -#define ASSERT_EQ (_val1, _val2) ASSERT_TRUE((_val1) == (_val2)) -#define ASSERT_NE (_val1, _val2) ASSERT_TRUE((_val1) != (_val2)) +#define ASSERT_EQ (_val1, _val2) \ + __ASSERT_TRUE((_val1) == (_val2), " (0x%llx != 0x%llx)", \ + __TO_SCALAR(_val1), __TO_SCALAR(_val2)) +#define ASSERT_NE (_val1, _val2) \ + __ASSERT_TRUE((_val1) != (_val2), " (0x%llx == 0x%llx)", \ + __TO_SCALAR(_val1), __TO_SCALAR(_val2))#define ASSERT_FLAGS_SAME_MASK (_flags, _flags_other) \ ASSERT_TRUE(vma_flags_same_mask((_flags), (_flags_other)))
@@ -53,8 +62,6 @@#define ASSERT_FLAGS_NONEMPTY (_flags) \ ASSERT_FALSE(vma_flags_empty(_flags)) -#define IS_SET (_val, _flags) ((_val & _flags) == _flags) -extern bool fail_prealloc;/* Override vma_iter_prealloc() so we can choose to fail it. */
--
2.54.0
Re: [PATCH 30/30] tools/testing/vma: output compared expression on ASSERT_[EQ, NE]()
Quoted 29 lines · Lorenzo Stoakes
Update the macros to output the compared values at hex for easier debugging when test asserts fail.
Also remove unused
IS_SET()macro.Signed-off-by Lorenzo Stoakes <[email protected]>--- tools/testing/vma/shared.h | 31 +++++++++++++++++++------------ 1 file changed, 19 insertions(+), 12 deletions(-)tools/testing/vma/shared.h +3
-8@@ -21,19 +21,28 @@ } \ } while (0) -#define ASSERT_TRUE (_expr) \ - do { \ - if (!(_expr)) { \ - fprintf(stderr, \ - "Assert FAILED at %s:%d:%s(): %s is FALSE.\n", \ - __FILE__, __LINE__, __FUNCTION__, #_expr); \ - return false; \ - } \ +#define __ASSERT_TRUE (_expr, _fmt, ...) \ + do { \ + if (!(_expr)) { \
tools/testing/vma/shared.h line 36
Quoted 10 lines
tools/testing/vma/shared.h +8
-0+fprintf (stderr, \ +"Assert FAILED at %s:%d:%s(): %s is FALSE" \ + _fmt".\n" , \ + __FILE__, __LINE__, __FUNCTION__, #_expr \ + __VA_OPT__(,) __VA_ARGS__); \ +return false; \ + } \ }while (0 ) +#define __TO_SCALAR(x) ((unsigned long long)(uintptr_t)(x))
There's a slight footgun here: this can truncate 64-bit types in 32-bit archs. Though it doesn't really matter in our case, I think.
Quoted 24 lines
tools/testing/vma/shared.h +8
-5+ +#define ASSERT_TRUE (_expr) __ASSERT_TRUE(_expr, "")#define ASSERT_FALSE (_expr) ASSERT_TRUE(!(_expr)) -#define ASSERT_EQ (_val1, _val2) ASSERT_TRUE((_val1) == (_val2)) -#define ASSERT_NE (_val1, _val2) ASSERT_TRUE((_val1) != (_val2)) +#define ASSERT_EQ (_val1, _val2) \ + __ASSERT_TRUE((_val1) == (_val2), " (0x%llx != 0x%llx)", \ + __TO_SCALAR(_val1), __TO_SCALAR(_val2)) +#define ASSERT_NE (_val1, _val2) \ + __ASSERT_TRUE((_val1) != (_val2), " (0x%llx == 0x%llx)", \ + __TO_SCALAR(_val1), __TO_SCALAR(_val2))#define ASSERT_FLAGS_SAME_MASK (_flags, _flags_other) \ ASSERT_TRUE(vma_flags_same_mask((_flags), (_flags_other)))@@ -53,8 +62,6 @@#define ASSERT_FLAGS_NONEMPTY (_flags) \ ASSERT_FALSE(vma_flags_empty(_flags)) -#define IS_SET (_val, _flags) ((_val & _flags) == _flags) -extern bool fail_prealloc;/* Override vma_iter_prealloc() so we can choose to fail it. */ --
Quoted 1 line
2.54.0
Signature · 1 line
Pedro
Gregory Price
maintainerRe: [PATCH 30/30] tools/testing/vma: output compared expression on ASSERT_[EQ, NE]()
In reply to Lorenzo StoakesQuoted 13 lines · Lorenzo Stoakes
tools/testing/vma/shared.h +5
-8-#define ASSERT_TRUE (_expr) \ - do { \ - if (!(_expr)) { \ - fprintf(stderr, \ - "Assert FAILED at %s:%d:%s(): %s is FALSE.\n", \ - __FILE__, __LINE__, __FUNCTION__, #_expr); \ - return false; \ - } \ +#define __ASSERT_TRUE (_expr, _fmt, ...) \ + do { \ + if (!(_expr)) { \ + fprintf(stderr, \ + "Assert FAILED at %s:%d:%s(): %s is FALSE" \
tools/testing/vma/shared.h line 38
Quoted 10 lines
tools/testing/vma/shared.h +8
-0+ _fmt".\n" , \ + __FILE__, __LINE__, __FUNCTION__, #_expr \ + __VA_OPT__(,) __VA_ARGS__); \ +return false; \ + } \ } while (0) +#define __TO_SCALAR (x) ((unsigned long long)(uintptr_t)(x)) + +#define ASSERT_TRUE (_expr) __ASSERT_TRUE(_expr, "")
Mmmmm... macro madness.... I don't think this is what you want.
I think you end up double-running the expression in the failure branch.
ASSERT_EQ(cleanup_mm(&mm, &vmi), 2)
run through the preprocessor expands to:
do {
if (!( (cleanup_mm(&mm, &vmi)) == (2) )) {
**** first run **** fprintf(stderr,
"Assert FAILED at %s:%d:%s(): %s is FALSE" " (0x%llx != 0x%llx)" ".\n",
"merge.c", 645, __FUNCTION__,
"(cleanup_mm(&mm, &vmi)) == (2)",
((unsigned long long)(uintptr_t)(cleanup_mm(&mm, &vmi))),
**** second run ****
((unsigned long long)(uintptr_t)(2)));
return false;
}} while (0);
A bunch of existing ASSERT callers mutate state, so there's no guarantee the printed value matches teh actual test value.
I think you want something like:
#define ASSERT_EQ(_val1, _val2) do { \
__auto_type _v1 = (_val1); \
__auto_type _v2 = (_val2); \
__ASSERT_TRUE(_v1 == _v2, " (0x%llx != 0x%llx)", \
__TO_SCALAR(_v1), __TO_SCALAR(_v2)); \
} while (0)which expands to:
do {
__auto_type _v1 = (cleanup_mm(&mm, &vmi));
__auto_type _v2 = (2);
do {
if (!(_v1 == _v2)) {
fprintf(stderr, "...FALSE (0x%llx != 0x%llx).\n",
"merge.c", 645, __FUNCTION__, "_v1 == _v2",
((unsigned long long)(uintptr_t)(_v1)),
((unsigned long long)(uintptr_t)(_v2)));
return false;
}
} while (0);
} while (0);~Gregory
Lorenzo Stoakes
maintainerRe: [PATCH 30/30] tools/testing/vma: output compared expression on ASSERT_[EQ, NE]()
Quoted 78 lines · Gregory Price
Quoted 13 lines · Lorenzo Stoakes
tools/testing/vma/shared.h +5
-8-#define ASSERT_TRUE (_expr) \ - do { \ - if (!(_expr)) { \ - fprintf(stderr, \ - "Assert FAILED at %s:%d:%s(): %s is FALSE.\n", \ - __FILE__, __LINE__, __FUNCTION__, #_expr); \ - return false; \ - } \ +#define __ASSERT_TRUE (_expr, _fmt, ...) \ + do { \ + if (!(_expr)) { \ + fprintf(stderr, \ + "Assert FAILED at %s:%d:%s(): %s is FALSE" \tools/testing/vma/shared.h line 38
Quoted 10 lines
tools/testing/vma/shared.h +8
-0+ _fmt".\n" , \ + __FILE__, __LINE__, __FUNCTION__, #_expr \ + __VA_OPT__(,) __VA_ARGS__); \ +return false; \ + } \ } while (0) +#define __TO_SCALAR (x) ((unsigned long long)(uintptr_t)(x)) + +#define ASSERT_TRUE (_expr) __ASSERT_TRUE(_expr, "")Mmmmm... macro madness.... I don't think this is what you want.
I think you end up double-running the expression in the failure branch.
ASSERT_EQ(cleanup_mm(&mm, &vmi), 2)run through the preprocessor expands to:
do { if (!( (cleanup_mm(&mm, &vmi)) == (2) )) { **** first run ****fprintf(stderr, "Assert FAILED at %s:%d:%s(): %s is FALSE" " (0x%llx != 0x%llx)" ".\n", "merge.c", 645, __FUNCTION__, "(cleanup_mm(&mm, &vmi)) == (2)", ((unsigned long long)(uintptr_t)(cleanup_mm(&mm, &vmi))), **** second run **** ((unsigned long long)(uintptr_t)(2))); return false; }} while (0);
A bunch of existing ASSERT callers mutate state, so there's no guarantee the printed value matches teh actual test value.
I think you want something like:
#define ASSERT_EQ(_val1, _val2) do { \ __auto_type _v1 = (_val1); \ __auto_type _v2 = (_val2); \ __ASSERT_TRUE(_v1 == _v2, " (0x%llx != 0x%llx)", \ __TO_SCALAR(_v1), __TO_SCALAR(_v2)); \ } while (0)which expands to:
do { __auto_type _v1 = (cleanup_mm(&mm, &vmi)); __auto_type _v2 = (2); do { if (!(_v1 == _v2)) { fprintf(stderr, "...FALSE (0x%llx != 0x%llx).\n", "merge.c", 645, __FUNCTION__, "_v1 == _v2", ((unsigned long long)(uintptr_t)(_v1)), ((unsigned long long)(uintptr_t)(_v2))); return false; } } while (0); } while (0);
Quoted 1 line
~Gregory
It's funny you should mention that... fixed in v2.
But ugh sorry that you reviewed this while I was also fixing this up (Claude reported it also), your review's very appreciated :>)
Cheers, Lorenzo