Skip to contentSkip to conversation

mm/readahead: Skip fully overlapped range

PATCH In review Aubrey Li 3 participants 9 messages Sep 23, 2025 – Nov 7, 2025

Subsystems (3)
lore
Share & tools
lore

Aubrey Li

author
#0 lore raw

PATCH 1/1 mm/readahead: Skip fully overlapped range

RocksDB sequential read benchmark under high concurrency shows severe lock contention. Multiple threads may issue readahead on the same file simultaneously, which leads to heavy contention on the xas spinlock in filemap_add_folio(). Perf profiling indicates 30%~60% of CPU time spent there.

To mitigate this issue, a readahead request will be skipped if its range is fully covered by an ongoing readahead. This avoids redundant work and significantly reduces lock contention. In one-second sampling, contention on xas spinlock dropped from 138,314 times to 2,144 times, resulting in a large performance improvement in the benchmark.

w/o patch w/ patch
RocksDB-readseq (ops/sec)
(32-threads) 1.2M 2.4M

---
 mm/readahead.c | 19 ++++++++++++++++++-
 1 file changed, 18 insertions(+), 1 deletion(-)
mm/readahead.c +18 -1
index 20d36d6b055e..57ae1a137730 100644
@@ -337,7 +337,7 @@ void force_page_cache_ra(struct readahead_control *ractl,
 	struct address_space *mapping = ractl->mapping;
 	struct file_ra_state *ra = ractl->ra;
 	struct backing_dev_info *bdi = inode_to_bdi(mapping->host);
-	unsigned long max_pages;
+	unsigned long max_pages, index;
 
 	if (unlikely(!mapping->a_ops->read_folio && !mapping->a_ops->readahead))
 		return;
@@ -348,6 +348,19 @@ void force_page_cache_ra(struct readahead_control *ractl,
 	 */
 	max_pages = max_t(unsigned long, bdi->io_pages, ra->ra_pages);
 	nr_to_read = min_t(unsigned long, nr_to_read, max_pages);
+
+	index = readahead_index(ractl);
+	/*
+	 * Skip this readahead if the requested range is fully covered
+	 * by the ongoing readahead range. This typically occurs in
+	 * concurrent scenarios.
+	 */
+	if (index >= ra->start && index + nr_to_read  <= ra->start + ra->size)
+		return;
+
+	ra->start = index;
+	ra->size = nr_to_read;
+
 	while (nr_to_read) {
 		unsigned long this_chunk = (2 * 1024 * 1024) / PAGE_SIZE;
 
@@ -357,6 +370,10 @@ void force_page_cache_ra(struct readahead_control *ractl,
 
 		nr_to_read -= this_chunk;
 	}
+
+	/* Reset readahead state to allow the next readahead */
+	ra->start = 0;
+	ra->size = 0;
 }
 
 /*
Signature · 1 line
2.43.0
maintainer
#1 lore raw
Quoted 12 lines · On Tue, 23 Sep 2025 11:59:46 +0800 Aubrey Li <[email protected]> wrote:

RocksDB sequential read benchmark under high concurrency shows severe lock contention. Multiple threads may issue readahead on the same file simultaneously, which leads to heavy contention on the xas spinlock in filemap_add_folio(). Perf profiling indicates 30%~60% of CPU time spent there.

To mitigate this issue, a readahead request will be skipped if its range is fully covered by an ongoing readahead. This avoids redundant work and significantly reduces lock contention. In one-second sampling, contention on xas spinlock dropped from 138,314 times to 2,144 times, resulting in a large performance improvement in the benchmark.

Quoted 3 lines

w/o patch w/ patch
RocksDB-readseq (ops/sec)
(32-threads) 1.2M 2.4M

On which kernel version? In recent times we've made a few readahead changes to address issues with high concurrency and a quick retest on mm.git's current mm-stable branch would be interesting please.

author
#2 lore raw
Quoted 14 lines · Andrew Morton
Quoted 12 lines · On Tue, 23 Sep 2025 11:59:46 +0800 Aubrey Li <[email protected]> wrote:

RocksDB sequential read benchmark under high concurrency shows severe lock contention. Multiple threads may issue readahead on the same file simultaneously, which leads to heavy contention on the xas spinlock in filemap_add_folio(). Perf profiling indicates 30%~60% of CPU time spent there.

To mitigate this issue, a readahead request will be skipped if its range is fully covered by an ongoing readahead. This avoids redundant work and significantly reduces lock contention. In one-second sampling, contention on xas spinlock dropped from 138,314 times to 2,144 times, resulting in a large performance improvement in the benchmark.

Quoted 8 lines
Quoted 3 lines

w/o patch w/ patch
RocksDB-readseq (ops/sec)
(32-threads) 1.2M 2.4M

On which kernel version? In recent times we've made a few readahead changes to address issues with high concurrency and a quick retest on mm.git's current mm-stable branch would be interesting please.

I'm on v6.16.7. Thanks Andrew for the information, let me check with mm.git.

Thanks,
-Aubrey

maintainer
#3 lore raw
Quoted 19 lines · Aubrey Li
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Tue, 23 Sep 2025 11:59:46 +0800 Aubrey Li <[email protected]> wrote:

RocksDB sequential read benchmark under high concurrency shows severe lock contention. Multiple threads may issue readahead on the same file simultaneously, which leads to heavy contention on the xas spinlock in filemap_add_folio(). Perf profiling indicates 30%~60% of CPU time spent there.

To mitigate this issue, a readahead request will be skipped if its range is fully covered by an ongoing readahead. This avoids redundant work and significantly reduces lock contention. In one-second sampling, contention on xas spinlock dropped from 138,314 times to 2,144 times, resulting in a large performance improvement in the benchmark.

w/o patch w/ patch
RocksDB-readseq (ops/sec)
(32-threads) 1.2M 2.4M

Quoted 5 lines
Quoted 3 lines

On which kernel version? In recent times we've made a few readahead changes to address issues with high concurrency and a quick retest on mm.git's current mm-stable branch would be interesting please.

I'm on v6.16.7. Thanks Andrew for the information, let me check with mm.git.

I don't expect much of a change for this load but getting test result with mm.git as a confirmation would be nice. Also, based on the fact that the patch you propose helps, this looks like there are many threads sharing one struct file which race to read the same content. That is actually rather problematic for current readahead code because there's *no synchronization* on updating file's readhead state. So threads can race and corrupt the state in interesting ways under one another's hands. On rare occasions I've observed this with heavy NFS workload where the NFS server is multithreaded. Since the practical outcome is "just" reduced read throughput / reading too much, it was never high enough on my priority list to fix properly (I do have some preliminary patch for that laying around but there are some open questions that require deeper thinking - like how to handle a situation where one threads does readahead, filesystem requests some alignment of the request size after the fact, so we'd like to update readahead state but another thread has modified the shared readahead state in the mean time). But if we're going to work on improving behavior of readahead for multiple threads sharing readahead state, fixing the code so that readahead state is at least consistent is IMO the first necessary step. And then we can pile more complex logic on top of that.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#4 lore raw
Quoted 20 lines · Jan Kara
Quoted 19 lines · Aubrey Li
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Tue, 23 Sep 2025 11:59:46 +0800 Aubrey Li <[email protected]> wrote:

RocksDB sequential read benchmark under high concurrency shows severe lock contention. Multiple threads may issue readahead on the same file simultaneously, which leads to heavy contention on the xas spinlock in filemap_add_folio(). Perf profiling indicates 30%~60% of CPU time spent there.

To mitigate this issue, a readahead request will be skipped if its range is fully covered by an ongoing readahead. This avoids redundant work and significantly reduces lock contention. In one-second sampling, contention on xas spinlock dropped from 138,314 times to 2,144 times, resulting in a large performance improvement in the benchmark.

w/o patch w/ patch
RocksDB-readseq (ops/sec)
(32-threads) 1.2M 2.4M

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

On which kernel version? In recent times we've made a few readahead changes to address issues with high concurrency and a quick retest on mm.git's current mm-stable branch would be interesting please.

I'm on v6.16.7. Thanks Andrew for the information, let me check with mm.git.

I don't expect much of a change for this load but getting test result with mm.git as a confirmation would be nice.

Yes, the hotspot remains on mm.git:mm-stable branch.

- 88.68% clone3
- 88.68% start_thread
- 88.68% reader_thread
- 88.27% syscall entry_SYSCALL_64_after_hwframe do_syscall_64 ksys_readahead generic_fadvise force_page_cache_ra page_cache_ra_unbounded filemap_add_folio __filemap_add_folio _raw_spin_lock_irq
- do_raw_spin_lock native_queued_spin_lock_slowpath

Quoted 10 lines

Also, based on the fact that the
patch you propose helps, this looks like there are many threads sharing one struct file which race to read the same content. That is actually rather problematic for current readahead code because there's *no synchronization* on updating file's readhead state. So threads can race and corrupt the state in interesting ways under one another's hands. On rare occasions I've observed this with heavy NFS workload where the NFS server is multithreaded. Since the practical outcome is "just" reduced read throughput / reading too much, it was never high enough on my priority list to fix properly (I do have some preliminary patch for that laying around

Quoted 8 lines

but there are some open questions that require deeper thinking - like how to handle a situation where one threads does readahead, filesystem requests some alignment of the request size after the fact, so we'd like to update readahead state but another thread has modified the shared readahead state in the mean time). But if we're going to work on improving behavior of readahead for multiple threads sharing readahead state, fixing the code so that readahead state is at least consistent is IMO the first necessary step. And then we can pile more complex logic on top of that.

This makes sense. I actually had a version using atomic operations to update ra in my patch, but I found that ra is also updated in other paths without synchronization, so I dropped the atomic operations before sending the patch. Let me check what I can do for this.

Have you put your preliminary patch somewhere?

Thanks,
-Aubrey

Quoted 2 lines

Honza

author
#5 lore raw
In reply to Jan Kara
Quoted 37 lines · Jan Kara
Quoted 19 lines · Aubrey Li
Quoted 18 lines · Andrew Morton
Quoted 15 lines · On Tue, 23 Sep 2025 11:59:46 +0800 Aubrey Li <[email protected]> wrote:

RocksDB sequential read benchmark under high concurrency shows severe lock contention. Multiple threads may issue readahead on the same file simultaneously, which leads to heavy contention on the xas spinlock in filemap_add_folio(). Perf profiling indicates 30%~60% of CPU time spent there.

To mitigate this issue, a readahead request will be skipped if its range is fully covered by an ongoing readahead. This avoids redundant work and significantly reduces lock contention. In one-second sampling, contention on xas spinlock dropped from 138,314 times to 2,144 times, resulting in a large performance improvement in the benchmark.

w/o patch w/ patch
RocksDB-readseq (ops/sec)
(32-threads) 1.2M 2.4M

Quoted 5 lines
Quoted 3 lines

On which kernel version? In recent times we've made a few readahead changes to address issues with high concurrency and a quick retest on mm.git's current mm-stable branch would be interesting please.

I'm on v6.16.7. Thanks Andrew for the information, let me check with mm.git.

I don't expect much of a change for this load but getting test result with mm.git as a confirmation would be nice. Also, based on the fact that the patch you propose helps, this looks like there are many threads sharing one struct file which race to read the same content. That is actually rather problematic for current readahead code because there's *no synchronization* on updating file's readhead state. So threads can race and corrupt the state in interesting ways under one another's hands. On rare occasions I've observed this with heavy NFS workload where the NFS server is multithreaded. Since the practical outcome is "just" reduced read throughput / reading too much, it was never high enough on my priority list to fix properly (I do have some preliminary patch for that laying around

Quoted 9 lines

but there are some open questions that require deeper thinking - like how to handle a situation where one threads does readahead, filesystem requests some alignment of the request size after the fact, so we'd like to update readahead state but another thread has modified the shared readahead state in the mean time). But if we're going to work on improving behavior of readahead for multiple threads sharing readahead state, fixing the code so that readahead state is at least consistent is IMO the first necessary step. And then we can pile more complex logic on top of that.

If I understand this article correctly, especially the following passage:
- https://lwn.net/Articles/888715/

"""
A core idea in readahead is to take a risk and read more than was requested. If that risk brings rewards and the extra data is accessed, then that justifies a further risk of reading even more data that hasn't been requested. When performing a single sequential read through a file, the details of past behavior can easily be stored in the struct file_ra_state. However if an application reads from two, three, or more, sections of the file and interleaves these sequential reads, then file_ra_state cannot keep track of all that state. Instead we rely on the content already in the page cache. Specifically we have a flag, PG_readahead, which can be set on a page. That name should be read in the past tense: the page was read ahead.A risk was taken when reading that page so, if it pays off and the page is accessed, then that is justification for taking another risk and reading some more. """

file_ra_state is considered a performance hint, not a critical correctness field. The race conditions on file's readahead state don't affect the correctness of file I/O because later the page cache mechanisms ensure data consistency, it won't cause wrong data to be read. I think that's why we do not lock file_ra_state today, to avoid performance penalties on this hot path.

That said, this patch didn't make things worse, and it does take a risk but brings the rewards of RocksDB's readseq benchmark.

Thanks,
-Aubrey

maintainer
#6 lore raw
Quoted 8 lines · On Tue, 30 Sep 2025 13:35:43 +0800 Aubrey Li <[email protected]> wrote:

file_ra_state is considered a performance hint, not a critical correctness field. The race conditions on file's readahead state don't affect the correctness of file I/O because later the page cache mechanisms ensure data consistency, it won't cause wrong data to be read. I think that's why we do not lock file_ra_state today, to avoid performance penalties on this hot path.

That said, this patch didn't make things worse, and it does take a risk but brings the rewards of RocksDB's readseq benchmark.

So if I may summarize:

- you've identifed and addressed an issue with concurrent readahead against an fd

- Jan points out that we don't properly handle concurrent access to a file's ra_state. This is somewhat offtopic, but we should address this sometime anyway. Then we can address the RocksDB issue later.

Alternatively, we could fix this issue right now and let the concurrency fixes come later. Not as pretty, but it's practical.

Another practicality: improving a benchmark is nice, but do we have any reasons to believe that this change will improve any real-world workload? If so, which and by how much?

maintainer
#7 lore raw

Sorry for not replying earlier. I wanted make up my mind about this and other stuff was keeping preempting me...

Quoted 8 lines · Andrew Morton
Quoted 6 lines · On Tue, 30 Sep 2025 13:35:43 +0800 Aubrey Li <[email protected]> wrote:

file_ra_state is considered a performance hint, not a critical correctness field. The race conditions on file's readahead state don't affect the correctness of file I/O because later the page cache mechanisms ensure data consistency, it won't cause wrong data to be read. I think that's why we do not lock file_ra_state today, to avoid performance penalties on this hot path.

Quoted 7 lines
Quoted 2 lines

That said, this patch didn't make things worse, and it does take a risk but brings the rewards of RocksDB's readseq benchmark.

So if I may summarize:

- you've identifed and addressed an issue with concurrent readahead against an fd

Right but let me also note that the patch modifies only force_page_cache_ra() which is a pretty peculiar function. It's used at two places:
1) When page_cache_sync_ra() decides it isn't worth to do a proper readahead and just wants to read that one one.

2) From POSIX_FADV_WILLNEED - I suppose this is Aubrey's case.

As such it seems to be fixing mostly a "don't do it when it hurts" kind of load from the benchmark than a widely used practical case since I'm not sure many programs call POSIX_FADV_WILLNEED from many threads in parallel for the same range.

Quoted 3 lines

- Jan points out that we don't properly handle concurrent access to a file's ra_state. This is somewhat offtopic, but we should address this sometime anyway. Then we can address the RocksDB issue later.

The problem I had with the patch is that it adds more racy updates & checks for the shared ra state so it's kind of difficult to say whether some workload will not now more often clobber the ra state resulting in poor readahead behavior. Also as I looked into the patch now another objection I have is that force_page_cache_ra() previously didn't touch the ra state at all, it just read the requested pages. After the patch force_page_cache_ra() will destroy the readahead state completely. This is definitely something we don't want to do.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
author
#8 lore raw

Really sorry for the late, too. Thunderbird collapsed this thread, but didn't highlight it as unread, I thought no one response, :(

Quoted 26 lines · Jan Kara

Sorry for not replying earlier. I wanted make up my mind about this and other stuff was keeping preempting me...

Quoted 8 lines · Andrew Morton
Quoted 6 lines · On Tue, 30 Sep 2025 13:35:43 +0800 Aubrey Li <[email protected]> wrote:

file_ra_state is considered a performance hint, not a critical correctness field. The race conditions on file's readahead state don't affect the correctness of file I/O because later the page cache mechanisms ensure data consistency, it won't cause wrong data to be read. I think that's why we do not lock file_ra_state today, to avoid performance penalties on this hot path.

Quoted 7 lines
Quoted 2 lines

That said, this patch didn't make things worse, and it does take a risk but brings the rewards of RocksDB's readseq benchmark.

So if I may summarize:

- you've identifed and addressed an issue with concurrent readahead against an fd

Right but let me also note that the patch modifies only force_page_cache_ra() which is a pretty peculiar function. It's used at two places:
1) When page_cache_sync_ra() decides it isn't worth to do a proper readahead and just wants to read that one one.

Quoted 14 lines

2) From POSIX_FADV_WILLNEED - I suppose this is Aubrey's case.

As such it seems to be fixing mostly a "don't do it when it hurts" kind of load from the benchmark than a widely used practical case since I'm not sure many programs call POSIX_FADV_WILLNEED from many threads in parallel for the same range.

Quoted 7 lines

- Jan points out that we don't properly handle concurrent access to a file's ra_state. This is somewhat offtopic, but we should address this sometime anyway. Then we can address the RocksDB issue later.

Another practicality: improving a benchmark is nice, but do we have any reasons to believe that this change will improve any real-world workload? If so, which and by how much?

I only have RocksDB on my side, but this isn't a lab case but a real case. It's an issue reported by a customer. They use this case to stress test the system under high-concurrency data workloads, it could have business impact.

Quoted 9 lines

The problem I had with the patch is that it adds more racy updates & checks for the shared ra state so it's kind of difficult to say whether some workload will not now more often clobber the ra state resulting in poor readahead behavior. Also as I looked into the patch now another objection I have is that force_page_cache_ra() previously didn't touch the ra state at all, it just read the requested pages. After the patch force_page_cache_ra() will destroy the readahead state completely. This is definitely something we don't want to do.

This is also something I worried about, so I added two trace points at the entry and exit of force_page_cache_ra(), and I got all ZEROs.

test-9858    [018] .....   554.352691: force_page_cache_ra: force_page_cache_ra entry: ra->start = 0, ra->size = 0
test-9858    [018] .....   554.352695: force_page_cache_ra: force_page_cache_ra exit: ra->start = 0, ra->size = 0
test-9855    [009] .....   554.352701: force_page_cache_ra: force_page_cache_ra entry: ra->start = 0, ra->size = 0
test-9855    [009] .....   554.352705: force_page_cache_ra: force_page_cache_ra exit: ra->start = 0, ra->size = 0

I think for this code path, my patch doesn't break anything. Do we have any other code paths I can check?

Anyway, thanks Andrew and Jan for the detailed feedback and discussion. if we later plan to make file_ra_state concurrency-safe first, I'd be happy to help test or rebase this optimization on top of that work.

Thanks,
-Aubrey