mm: make mmap_miss accounting symmetric for VM_SEQ_READ
Subsystems (3)
PATCH 1/1 mm: make mmap_miss accounting symmetric for VM_SEQ_READ
do_sync_mmap_readahead() skips both the mmap_miss increment and the MMAP_LOTSAMISS check for VM_SEQ_READ mappings, since sequential access is non-speculative and should always read ahead. The two decrement sites in do_async_mmap_readahead() and filemap_map_pages() do not mirror this skip, so concurrent faults on a VM_SEQ_READ mapping can still drive ra->mmap_miss down to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.
Skip the decrement for VM_SEQ_READ in both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change for VM_SEQ_READ users, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops a VM_SEQ_READ mapping from biasing the counter for other mappings of the same file.
--- mm/filemap.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)
mm/filemap.c +12 -2
@@ -3433,8 +3433,13 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,* Don't touch the mmap_miss counter to avoid decreasing it multiple * times for a single folio and break the balance with mmap_miss * increase in do_sync_mmap_readahead(). +* +* VM_SEQ_READ mappings skip the mmap_miss increment in +* do_sync_mmap_readahead(), so skip the decrement here as well to +* keep the counter symmetric. */ -if (likely (!folio_test_locked (folio))) { +if (likely (!folio_test_locked (folio)) && + !(vmf->vma->vm_flags &VM_SEQ_READ )) { mmap_miss =READ_ONCE (ra->mmap_miss);if (mmap_miss)WRITE_ONCE (ra->mmap_miss, --mmap_miss);
@@ -3935,10 +3940,15 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,* In such situation, read-ahead is only a waste of IO. * Don't decrease mmap_miss in this scenario to make sure * we can stop read-ahead. +* +* VM_SEQ_READ mappings skip the mmap_miss increment in +* do_sync_mmap_readahead(), so skip the decrement here as +* well to keep the counter symmetric. */ if ((map_ret &VM_FAULT_NOPAGE ) && !(vmf->flags &FAULT_FLAG_TRIED ) && - !folio_test_workingset (folio)) { + !folio_test_workingset (folio) && + !(vma->vm_flags &VM_SEQ_READ )) {unsigned short mmap_miss; mmap_miss =READ_ONCE (file->f_ra.mmap_miss);
Signature · 1 line
2.52.0
Quoted 10 lines · Usama Arif
do_sync_mmap_readahead()skips both themmap_missincrement and theMMAP_LOTSAMISScheck forVM_SEQ_READmappings, since sequential access is non-speculative and should always read ahead. The two decrement sites indo_async_mmap_readahead()andfilemap_map_pages()do not mirror this skip, so concurrent faults on aVM_SEQ_READmapping can still drivera->mmap_missdown to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.
Quoted 8 lines
Skip the decrement for
VM_SEQ_READin both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change forVM_SEQ_READusers, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops aVM_SEQ_READmapping from biasing the counter for other mappings of the same file.Signed-off-by Usama Arif <[email protected]>
The patch is a result of review from sashiko on another one of my patches:
https://lore.kernel.org/all/[email protected]/
Looks good to me.
Quoted 64 lines · Usama Arif
do_sync_mmap_readahead()skips both themmap_missincrement and theMMAP_LOTSAMISScheck forVM_SEQ_READmappings, since sequential access is non-speculative and should always read ahead. The two decrement sites indo_async_mmap_readahead()andfilemap_map_pages()do not mirror this skip, so concurrent faults on aVM_SEQ_READmapping can still drivera->mmap_missdown to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.Skip the decrement for
VM_SEQ_READin both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change forVM_SEQ_READusers, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops aVM_SEQ_READmapping from biasing the counter for other mappings of the same file.Signed-off-by Usama Arif <[email protected]>---
mm/filemap.c| 14 ++++++++++++--1 file changed, 12 insertions(+), 2 deletions(-)mm/filemap.c +10
-1@@ -3433,8 +3433,13 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,* Don't touch the mmap_miss counter to avoid decreasing it multiple * times for a single folio and break the balance with mmap_miss * increase in do_sync_mmap_readahead(). +* +* VM_SEQ_READ mappings skip the mmap_miss increment in +* do_sync_mmap_readahead(), so skip the decrement here as well to +* keep the counter symmetric. */ -if (likely (!folio_test_locked (folio))) { +if (likely (!folio_test_locked (folio)) && + !(vmf->vma->vm_flags &VM_SEQ_READ )) { mmap_miss =READ_ONCE (ra->mmap_miss);if (mmap_miss)WRITE_ONCE (ra->mmap_miss, --mmap_miss);@@ -3935,10 +3940,15 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,* In such situation, read-ahead is only a waste of IO. * Don't decrease mmap_miss in this scenario to make sure * we can stop read-ahead. +* +* VM_SEQ_READ mappings skip the mmap_miss increment in +* do_sync_mmap_readahead(), so skip the decrement here as +* well to keep the counter symmetric. */ if ((
map_ret&VM_FAULT_NOPAGE) &&
!(vmf->flags &FAULT_FLAG_TRIED) &&mm/filemap.c +2
-1- !folio_test_workingset(folio)) { + !folio_test_workingset(folio) && + !(vma->vm_flags & VM_SEQ_READ)) {unsigned short
mmap_miss;
mmap_miss=READ_ONCE(file->f_ra.mmap_miss);Signature · 1 line
2.52.0
Quoted 10 lines · Usama Arif
do_sync_mmap_readahead()skips both themmap_missincrement and theMMAP_LOTSAMISScheck forVM_SEQ_READmappings, since sequential access is non-speculative and should always read ahead. The two decrement sites indo_async_mmap_readahead()andfilemap_map_pages()do not mirror this skip, so concurrent faults on aVM_SEQ_READmapping can still drivera->mmap_missdown to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.
Quoted 8 lines
Skip the decrement for
VM_SEQ_READin both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change forVM_SEQ_READusers, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops aVM_SEQ_READmapping from biasing the counter for other mappings of the same file.Signed-off-by Usama Arif <[email protected]>
Looks good. Feel free to add:
Honza
Quoted 43 lines
--- mm/filemap.c | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-)mm/filemap.c +12
-2@@ -3433,8 +3433,13 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,* Don't touch the mmap_miss counter to avoid decreasing it multiple * times for a single folio and break the balance with mmap_miss * increase in do_sync_mmap_readahead(). +* +* VM_SEQ_READ mappings skip the mmap_miss increment in +* do_sync_mmap_readahead(), so skip the decrement here as well to +* keep the counter symmetric. */ -if (likely (!folio_test_locked (folio))) { +if (likely (!folio_test_locked (folio)) && + !(vmf->vma->vm_flags &VM_SEQ_READ )) { mmap_miss =READ_ONCE (ra->mmap_miss);if (mmap_miss)WRITE_ONCE (ra->mmap_miss, --mmap_miss);@@ -3935,10 +3940,15 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,* In such situation, read-ahead is only a waste of IO. * Don't decrease mmap_miss in this scenario to make sure * we can stop read-ahead. +* +* VM_SEQ_READ mappings skip the mmap_miss increment in +* do_sync_mmap_readahead(), so skip the decrement here as +* well to keep the counter symmetric. */ if ((map_ret &VM_FAULT_NOPAGE ) && !(vmf->flags &FAULT_FLAG_TRIED ) && - !folio_test_workingset (folio)) { + !folio_test_workingset (folio) && + !(vma->vm_flags &VM_SEQ_READ )) {unsigned short mmap_miss; mmap_miss =READ_ONCE (file->f_ra.mmap_miss);Signature · 1 line
2.52.0
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR