Skip to contentSkip to conversation

mm: make mmap_miss accounting symmetric for VM_SEQ_READ

PATCH Applied Usama Arif 3 participants 4 messages May 25 – Jun 2

Subsystems (3)
lore
Share & tools
lore

Usama Arif

author
#0 lore raw

PATCH 1/1 mm: make mmap_miss accounting symmetric for VM_SEQ_READ

do_sync_mmap_readahead() skips both the mmap_miss increment and the MMAP_LOTSAMISS check for VM_SEQ_READ mappings, since sequential access is non-speculative and should always read ahead. The two decrement sites in do_async_mmap_readahead() and filemap_map_pages() do not mirror this skip, so concurrent faults on a VM_SEQ_READ mapping can still drive ra->mmap_miss down to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.

Skip the decrement for VM_SEQ_READ in both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change for VM_SEQ_READ users, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops a VM_SEQ_READ mapping from biasing the counter for other mappings of the same file.

---
 mm/filemap.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
mm/filemap.c +12 -2
index 5aaba0d3e81d..cca20e350c95 100644
@@ -3433,8 +3433,13 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,
 	 * Don't touch the mmap_miss counter to avoid decreasing it multiple
 	 * times for a single folio and break the balance with mmap_miss
 	 * increase in do_sync_mmap_readahead().
+	 *
+	 * VM_SEQ_READ mappings skip the mmap_miss increment in
+	 * do_sync_mmap_readahead(), so skip the decrement here as well to
+	 * keep the counter symmetric.
 	 */
-	if (likely(!folio_test_locked(folio))) {
+	if (likely(!folio_test_locked(folio)) &&
+	    !(vmf->vma->vm_flags & VM_SEQ_READ)) {
 		mmap_miss = READ_ONCE(ra->mmap_miss);
 		if (mmap_miss)
 			WRITE_ONCE(ra->mmap_miss, --mmap_miss);
@@ -3935,10 +3940,15 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,
 		 * In such situation, read-ahead is only a waste of IO.
 		 * Don't decrease mmap_miss in this scenario to make sure
 		 * we can stop read-ahead.
+		 *
+		 * VM_SEQ_READ mappings skip the mmap_miss increment in
+		 * do_sync_mmap_readahead(), so skip the decrement here as
+		 * well to keep the counter symmetric.
 		 */
 		if ((map_ret & VM_FAULT_NOPAGE) &&
 		    !(vmf->flags & FAULT_FLAG_TRIED) &&
-		    !folio_test_workingset(folio)) {
+		    !folio_test_workingset(folio) &&
+		    !(vma->vm_flags & VM_SEQ_READ)) {
 			unsigned short mmap_miss;
 
 			mmap_miss = READ_ONCE(file->f_ra.mmap_miss);
Signature · 1 line
2.52.0
author
#1 lore raw
Quoted 10 lines · Usama Arif

do_sync_mmap_readahead() skips both the mmap_miss increment and the MMAP_LOTSAMISS check for VM_SEQ_READ mappings, since sequential access is non-speculative and should always read ahead. The two decrement sites in do_async_mmap_readahead() and filemap_map_pages() do not mirror this skip, so concurrent faults on a VM_SEQ_READ mapping can still drive ra->mmap_miss down to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.

Quoted 8 lines

Skip the decrement for VM_SEQ_READ in both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change for VM_SEQ_READ users, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops a VM_SEQ_READ mapping from biasing the counter for other mappings of the same file.

The patch is a result of review from sashiko on another one of my patches:
https://lore.kernel.org/all/[email protected]/

Reviewed-by on v1 1/1
#2 lore raw
In reply to Usama Arif

Looks good to me.

Quoted 64 lines · Usama Arif

do_sync_mmap_readahead() skips both the mmap_miss increment and the MMAP_LOTSAMISS check for VM_SEQ_READ mappings, since sequential access is non-speculative and should always read ahead. The two decrement sites in do_async_mmap_readahead() and filemap_map_pages() do not mirror this skip, so concurrent faults on a VM_SEQ_READ mapping can still drive ra->mmap_miss down to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.

Skip the decrement for VM_SEQ_READ in both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change for VM_SEQ_READ users, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops a VM_SEQ_READ mapping from biasing the counter for other mappings of the same file.

---
mm/filemap.c | 14 ++++++++++++--

1 file changed, 12 insertions(+), 2 deletions(-)
mm/filemap.c +10 -1
index 5aaba0d3e81d..cca20e350c95 100644
@@ -3433,8 +3433,13 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,
 * Don't touch the mmap_miss counter to avoid decreasing it multiple
 * times for a single folio and break the balance with mmap_miss
 * increase in do_sync_mmap_readahead().
+ *
+ * VM_SEQ_READ mappings skip the mmap_miss increment in
+ * do_sync_mmap_readahead(), so skip the decrement here as well to
+ * keep the counter symmetric.
 */
- if (likely(!folio_test_locked(folio))) {
+ if (likely(!folio_test_locked(folio)) &&
+    !(vmf->vma->vm_flags & VM_SEQ_READ)) {
 mmap_miss = READ_ONCE(ra->mmap_miss);
 if (mmap_miss)
 WRITE_ONCE(ra->mmap_miss, --mmap_miss);
@@ -3935,10 +3940,15 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,
 * In such situation, read-ahead is only a waste of IO.
 * Don't decrease mmap_miss in this scenario to make sure
 * we can stop read-ahead.
+ *
+ * VM_SEQ_READ mappings skip the mmap_miss increment in
+ * do_sync_mmap_readahead(), so skip the decrement here as
+ * well to keep the counter symmetric.
 */

if ((map_ret & VM_FAULT_NOPAGE) &&
!(vmf->flags & FAULT_FLAG_TRIED) &&

mm/filemap.c +2 -1
-    !folio_test_workingset(folio)) {
+    !folio_test_workingset(folio) &&
+    !(vma->vm_flags & VM_SEQ_READ)) {

unsigned short mmap_miss;

mmap_miss = READ_ONCE(file->f_ra.mmap_miss);

Signature · 1 line
2.52.0
maintainer Reviewed-by on v1 1/1
#3 lore raw
In reply to Usama Arif
Quoted 10 lines · Usama Arif

do_sync_mmap_readahead() skips both the mmap_miss increment and the MMAP_LOTSAMISS check for VM_SEQ_READ mappings, since sequential access is non-speculative and should always read ahead. The two decrement sites in do_async_mmap_readahead() and filemap_map_pages() do not mirror this skip, so concurrent faults on a VM_SEQ_READ mapping can still drive ra->mmap_miss down to zero through the decrement paths even though nothing in the sync path ever increments it. The counter itself is per-file (file->f_ra.mmap_miss), so it can be moved by any VMA mapping the file, not just the one currently faulting.

Quoted 8 lines

Skip the decrement for VM_SEQ_READ in both decrement sites so the counter only moves for mappings that also participate in the increment side. No functional change for VM_SEQ_READ users, since the increment-side gate already prevents the counter from being consulted on their behalf, but it stops a VM_SEQ_READ mapping from biasing the counter for other mappings of the same file.

Looks good. Feel free to add:

Honza

Quoted 43 lines
---
 mm/filemap.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
mm/filemap.c +12 -2
index 5aaba0d3e81d..cca20e350c95 100644
@@ -3433,8 +3433,13 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,
 	 * Don't touch the mmap_miss counter to avoid decreasing it multiple
 	 * times for a single folio and break the balance with mmap_miss
 	 * increase in do_sync_mmap_readahead().
+	 *
+	 * VM_SEQ_READ mappings skip the mmap_miss increment in
+	 * do_sync_mmap_readahead(), so skip the decrement here as well to
+	 * keep the counter symmetric.
 	 */
-	if (likely(!folio_test_locked(folio))) {
+	if (likely(!folio_test_locked(folio)) &&
+	    !(vmf->vma->vm_flags & VM_SEQ_READ)) {
 		mmap_miss = READ_ONCE(ra->mmap_miss);
 		if (mmap_miss)
 			WRITE_ONCE(ra->mmap_miss, --mmap_miss);
@@ -3935,10 +3940,15 @@ vm_fault_t filemap_map_pages(struct vm_fault *vmf,
 		 * In such situation, read-ahead is only a waste of IO.
 		 * Don't decrease mmap_miss in this scenario to make sure
 		 * we can stop read-ahead.
+		 *
+		 * VM_SEQ_READ mappings skip the mmap_miss increment in
+		 * do_sync_mmap_readahead(), so skip the decrement here as
+		 * well to keep the counter symmetric.
 		 */
 		if ((map_ret & VM_FAULT_NOPAGE) &&
 		    !(vmf->flags & FAULT_FLAG_TRIED) &&
-		    !folio_test_workingset(folio)) {
+		    !folio_test_workingset(folio) &&
+		    !(vma->vm_flags & VM_SEQ_READ)) {
 			unsigned short mmap_miss;
 
 			mmap_miss = READ_ONCE(file->f_ra.mmap_miss);
Signature · 1 line
2.52.0
Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR