Skip to contentSkip to conversation

Revert "mm: limit filemap_fault readahead to VMA boundaries"

PATCH Applied Lorenzo Stoakes 8 participants 22 messages Jun 19 – Jun 24

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Revert "mm: limit filemap_fault readahead to VMA boundaries"

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

Fixes 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries")
---
 include/linux/pagemap.h | 2 --
 mm/filemap.c            | 4 ----
 mm/readahead.c          | 6 +-----
 3 files changed, 1 insertion(+), 11 deletions(-)
include/linux/pagemap.h +0 -2
index 627771e82eb1..2c3718d592d6 100644
@@ -1348,7 +1348,6 @@ struct readahead_control {
 	struct file_ra_state *ra;
 /* private: use the readahead_* accessors instead */
 	pgoff_t _index;
-	pgoff_t _max_index; /* limit readahead to _max_index, inclusive */
 	unsigned int _nr_pages;
 	unsigned int _batch_count;
 	bool dropbehind;
@@ -1362,7 +1361,6 @@ struct readahead_control {
 		.mapping = m,						\
 		.ra = r,						\
 		._index = i,						\
-		._max_index = ULONG_MAX,				\
 	}
 
 #define VM_READAHEAD_PAGES	(SZ_128K / PAGE_SIZE)
mm/filemap.c +0 -4
index dc3a0e960b9f..17a64837597c 100644
@@ -3312,8 +3312,6 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
 	unsigned int thp_order = 0;
 	unsigned short mmap_miss;
 
-	ractl._max_index = vmf->vma->vm_pgoff + vma_pages(vmf->vma) - 1;
-
 	/* Use the readahead code, even if readahead is disabled */
 	if (IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE) && (vm_flags & VM_HUGEPAGE)) {
 		/*
@@ -3409,7 +3407,6 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)
 		 * mmap read-around
 		 */
 		ra->start = max_t(long, 0, vmf->pgoff - ra->ra_pages / 2);
-		ra->start = max(ra->start, vmf->vma->vm_pgoff);
 		ra->size = ra->ra_pages;
 		ra->async_size = ra->ra_pages / 4;
 		ra->order = 0;
@@ -3457,7 +3454,6 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf,
 	}
 
 	if (folio_test_readahead(folio)) {
-		ractl._max_index = vmf->vma->vm_pgoff + vma_pages(vmf->vma) - 1;
 		fpin = maybe_unlock_mmap_for_io(vmf, fpin);
 		page_cache_async_ra(&ractl, folio, ra->ra_pages);
 	}
mm/readahead.c +1 -5
index 38ce16e3fcbd..558c92957518 100644
@@ -335,8 +335,6 @@ static void do_page_cache_ra(struct readahead_control *ractl,
 		return;
 
 	end_index = (isize - 1) >> PAGE_SHIFT;
-	if (end_index > ractl->_max_index)
-		end_index = ractl->_max_index;
 	if (index > end_index)
 		return;
 	/* Don't read past the page containing the last byte of the file */
@@ -487,7 +485,7 @@ void page_cache_ra_order(struct readahead_control *ractl,
 	pgoff_t start = readahead_index(ractl);
 	pgoff_t index = start;
 	unsigned int min_order = mapping_min_folio_order(mapping);
-	pgoff_t limit;
+	pgoff_t limit = (i_size_read(mapping->host) - 1) >> PAGE_SHIFT;
 	pgoff_t mark;
 	unsigned int nofs;
 	int err = 0;
@@ -500,8 +498,6 @@ void page_cache_ra_order(struct readahead_control *ractl,
 		goto fallback;
 	}
 
-	limit = (i_size_read(mapping->host) - 1) >> PAGE_SHIFT;
-	limit = min(limit, ractl->_max_index);
 	if (limit > index + ra->size - 1) {
 		limit = index + ra->size - 1;
 		mark = index + ra->size - ra->async_size;

--
2.54.0

Acked-by on v1 1/1
#1 lore raw
Quoted 11 lines · Lorenzo Stoakes

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

Quoted 5 lines
Fixes 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries")

---

Thanks!

Signature · 3 lines
Cheers,

David
Reviewed-by on v1 1/1
#2 lore raw
In reply to Lorenzo Stoakes
Quoted 11 lines · Lorenzo Stoakes

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

Quoted 4 lines
Fixes 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries")
Signature · 1 line
Pedro
maintainer Reviewed-by on v1 1/1
#3 lore raw
In reply to Lorenzo Stoakes
Quoted 11 lines · Lorenzo Stoakes

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

Quoted 4 lines
Fixes 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries")
maintainer
#4 lore raw
In reply to Lorenzo Stoakes
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

I'm a little conflicted here.

7b32f64bc512 avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.

IOW, the revert might make the kernel worse, overall.

If someone plans to get down and analyse that test case then come up with a new version of 7b32f64bc512 then OK. Is there such a person?

I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.

maintainer
#5 lore raw
Quoted 20 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

I'm a little conflicted here.

7b32f64bc512 avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.

Quoted 8 lines

IOW, the revert might make the kernel worse, overall.

If someone plans to get down and analyse that test case then come up with a new version of 7b32f64bc512 then OK. Is there such a person?

I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.

Suren did that work.
https://lore.kernel.org/linux-mm/CAJuCfpH_Fp=J-m-kVZ3VVmqH0cAfNX6CQKWt3XmAJq1Rp9sFwQ@mail.gmail.com/

#6 lore raw

+cc Suren, Pedro, David - I messed up the cc on this patch.

Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

Quoted 4 lines
Quoted 4 lines

I'm a little conflicted here.

7b32f64bc512 avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care

I wouldn't say it's a bug fix? The kernel has always done this with no reported bugs.

Quoted 11 lines
Quoted 11 lines

about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.

IOW, the revert might make the kernel worse, overall.

If someone plans to get down and analyse that test case then come up with a new version of 7b32f64bc512 then OK. Is there such a person?

I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.

As Willy points out [0], we've identified that it's a real usecase that we're regressing.

There's likely others too in the real world.

This approach has also not been without debate in the past (see [1]), I think this patch slipped under the radar on that front a bit.

So yeah, this does not to be reverted, and as a hotfix sooner rather than later.

Quoted 3 lines

Suren did that work.
https://lore.kernel.org/linux-mm/CAJuCfpH_Fp=J-m-kVZ3VVmqH0cAfNX6CQKWt3XmAJq1Rp9sFwQ@mail.gmail.com/

Thanks for highlighting that Willy (copied -> [0]).

Overall, if a patch causes a major regression in a core mm subsystem, people can wait a cycle to re-evaluate.

Thanks, Lorenzo

[0]:https://lore.kernel.org/linux-mm/CAJuCfpH_Fp=J-m-kVZ3VVmqH0cAfNX6CQKWt3XmAJq1Rp9sFwQ@mail.gmail.com/ [1]:https://lore.kernel.org/linux-mm/CAC_TJvfG8GcwG_2w1o6GOTZS8tfEx2h9A91qsenYfYsX8Te=Bg@mail.gmail.com/

#7 lore raw
Quoted 1 line · Lorenzo Stoakes

So yeah, this does not to be reverted, and as a hotfix sooner rather than later.

Does _need_ to be reverted :)

What an unfortunate typo... :P

#8 lore raw
In reply to Lorenzo Stoakes
Quoted 33 lines · Lorenzo Stoakes

+cc Suren, Pedro, David - I messed up the cc on this patch.

Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

Quoted 4 lines
Quoted 4 lines

I'm a little conflicted here.

7b32f64bc512 avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care

I wouldn't say it's a bug fix? The kernel has always done this with no reported bugs.

Quoted 8 lines
Quoted 8 lines

about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.

IOW, the revert might make the kernel worse, overall.

If someone plans to get down and analyse that test case then come up with a new version of 7b32f64bc512 then OK. Is there such a person?

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.

As Willy points out [0], we've identified that it's a real usecase that we're regressing.

There's likely others too in the real world.

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Quoted 3 lines

This approach has also not been without debate in the past (see [1]), I think this patch slipped under the radar on that front a bit.

Yeah, sadly it seems like the patch _may_ need a rather substantial rework :| So I would err on the side of caution for now and revert it as well.

Signature · 1 line
Pedro
#9 lore raw
Quoted 39 lines · Pedro Falcato
Quoted 37 lines · Lorenzo Stoakes

+cc Suren, Pedro, David - I messed up the cc on this patch.

Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:

This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.

This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.

There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.

Quoted 4 lines
Quoted 4 lines

I'm a little conflicted here.

7b32f64bc512 avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care

I wouldn't say it's a bug fix? The kernel has always done this with no reported bugs.

Quoted 11 lines
Quoted 11 lines

about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.

IOW, the revert might make the kernel worse, overall.

If someone plans to get down and analyse that test case then come up with a new version of 7b32f64bc512 then OK. Is there such a person?

I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.

Quoted 8 lines
Quoted 4 lines

As Willy points out [0], we've identified that it's a real usecase that we're regressing.

There's likely others too in the real world.

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.

Quoted 9 lines
Quoted 3 lines

This approach has also not been without debate in the past (see [1]), I think this patch slipped under the radar on that front a bit.

Yeah, sadly it seems like the patch _may_ need a rather substantial rework :| So I would err on the side of caution for now and revert it as well.

--
Pedro

maintainer
#10 lore raw
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.

If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling read() for a single byte at a time.

Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.

#11 lore raw
Quoted 10 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.

Quoted 8 lines

If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling read() for a single byte at a time.

Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.

That what I meant in the last part of my reply :)

But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.

#12 lore raw
Quoted 20 lines · Suren Baghdasaryan
Quoted 10 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.

Quoted 8 lines

If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling read() for a single byte at a time.

Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.

Quoted 5 lines

That what I meant in the last part of my reply :)

But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.

To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.

(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)

Signature · 1 line
Pedro
#13 lore raw
Quoted 22 lines · Pedro Falcato
Quoted 20 lines · Suren Baghdasaryan
Quoted 18 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.

If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling read() for a single byte at a time.

Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.

Quoted 8 lines
Quoted 5 lines

That what I meant in the last part of my reply :)

But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.

To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.

Ack.

Quoted 3 lines

(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)

For such streaminig workload I think read() would be more appropriate, and the change limiting read-ahead by VMA size would not affect it.

Quoted 3 lines

--
Pedro

maintainer
#14 lore raw
Quoted 32 lines · Suren Baghdasaryan
Quoted 22 lines · Pedro Falcato
Quoted 20 lines · Suren Baghdasaryan
Quoted 18 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.

If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling read() for a single byte at a time.

Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.

Quoted 8 lines
Quoted 5 lines

That what I meant in the last part of my reply :)

But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.

To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.

Quoted 7 lines

Ack.

Quoted 2 lines

(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)

For such streaminig workload I think read() would be more appropriate, and the change limiting read-ahead by VMA size would not affect it.

In fact if you look at that app, it has three different methods of reading the input frames (mmap, normal read, fully buffered file in memory) and one is chosen based on config option. So the benchmark apparently configures the mmap method of accessing the input file.

Anyway I agree with you using mmap-frame-by-frame approach for this usecase is kind of odd but not completely insane and it does demostrate people do odd things like this. For others the usecase of Android doing tons of small sparse mmaps into executable might look odd as well :) So this seems to be one of the cases where it's difficult to please everybody with the same behavior and we usually stick to the legacy behavior in cases like that (as much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.

Honza

Signature · 2 lines
Jan Kara <[email protected]>
SUSE Labs, CR
#15 lore raw
Quoted 46 lines · Jan Kara
Quoted 32 lines · Suren Baghdasaryan
Quoted 22 lines · Pedro Falcato
Quoted 20 lines · Suren Baghdasaryan
Quoted 18 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines

Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.

Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.

If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling read() for a single byte at a time.

Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.

Quoted 8 lines
Quoted 5 lines

That what I meant in the last part of my reply :)

But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.

To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.

Quoted 7 lines

Ack.

Quoted 2 lines

(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)

For such streaminig workload I think read() would be more appropriate, and the change limiting read-ahead by VMA size would not affect it.

In fact if you look at that app, it has three different methods of reading the input frames (mmap, normal read, fully buffered file in memory) and one is chosen based on config option. So the benchmark apparently configures the mmap method of accessing the input file.

Quoted 9 lines

Anyway I agree with you using mmap-frame-by-frame approach for this usecase is kind of odd but not completely insane and it does demostrate people do odd things like this. For others the usecase of Android doing tons of small sparse mmaps into executable might look odd as well :) So this seems to be one of the cases where it's difficult to please everybody with the same behavior and we usually stick to the legacy behavior in cases like that (as much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.

Reading the thread, I think we're all in agreement that this is a workload that exists and we are regressing it here, thus the revert is the right course of action.

Andrew - could we move this to the hotfixes branch please?

Quoted 5 lines

Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR

Thanks, Lorenzo

maintainer
#16 lore raw
Quoted 9 lines · On Mon, 22 Jun 2026 14:55:14 +0100 Lorenzo Stoakes <[email protected]> wrote:
Quoted 9 lines

Anyway I agree with you using mmap-frame-by-frame approach for this usecase is kind of odd but not completely insane and it does demostrate people do odd things like this. For others the usecase of Android doing tons of small sparse mmaps into executable might look odd as well :) So this seems to be one of the cases where it's difficult to please everybody with the same behavior and we usually stick to the legacy behavior in cases like that (as much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.

Would a helpful heuristic be to do what 7b32f64bc512 is doing, but only if PROT_EXEC?

Quoted 5 lines

Reading the thread, I think we're all in agreement that this is a workload that exists and we are regressing it here, thus the revert is the right course of action.

Andrew - could we move this to the hotfixes branch please?

No need? I'll send Linus the mm-stable batch #2 this week.

maintainer
#17 lore raw
Quoted 6 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines

much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.

Would a helpful heuristic be to do what 7b32f64bc512 is doing, but only if PROT_EXEC?

I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.

#18 lore raw
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines

much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.

Quoted 6 lines
Quoted 2 lines

Would a helpful heuristic be to do what 7b32f64bc512 is doing, but only if PROT_EXEC?

I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.

Thanks for the suggestion! That sounds sensible to me.

Quoted 1 line
#19 lore raw
Quoted 7 lines · Suren Baghdasaryan
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines

much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Would a helpful heuristic be to do what 7b32f64bc512 is doing, but only if PROT_EXEC?

I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.

Thanks for the suggestion! That sounds sensible to me.

I don't think this works. Here's an example readelf -a from a random, trivial ELF I have:

pfalcato@pedro-suse:~/linux> cc -g main.c
pfalcato@pedro-suse:~/linux> readelf -a a.out
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00

  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x401040
  Start of program headers:          64 (bytes into file)
  Start of section headers:          18448 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         14
  Size of section headers:           64 (bytes)
  Number of section headers:         38

Section header string table index: 37

Section Headers:

  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .note.gnu.pr[...] NOTE             0000000000400350  00000350
       0000000000000040  0000000000000000   A       0     0     8
  [ 2] .note.gnu.bu[...] NOTE             0000000000400390  00000390
       0000000000000024  0000000000000000   A       0     0     4
  [ 3] .interp           PROGBITS         00000000004003b4  000003b4
       000000000000001c  0000000000000000   A       0     0     1
  [ 4] .hash             HASH             00000000004003d0  000003d0
       0000000000000024  0000000000000004   A       6     0     8
  [ 5] .gnu.hash         GNU_HASH         00000000004003f8  000003f8
       000000000000001c  0000000000000000   A       6     0     8
  [ 6] .dynsym           DYNSYM           0000000000400418  00000418
       0000000000000060  0000000000000018   A       7     1     8
  [ 7] .dynstr           STRTAB           0000000000400478  00000478
       000000000000004a  0000000000000000   A       0     0     1
  [ 8] .gnu.version      VERSYM           00000000004004c2  000004c2
       0000000000000008  0000000000000002   A       6     0     2
  [ 9] .gnu.version_r    VERNEED          00000000004004d0  000004d0
       0000000000000030  0000000000000000   A       7     1     8
  [10] .rela.dyn         RELA             0000000000400500  00000500
       0000000000000030  0000000000000018   A       6     0     8
  [11] .rela.plt         RELA             0000000000400530  00000530
       0000000000000018  0000000000000018  AI       6    24     8
  [12] .init             PROGBITS         0000000000401000  00001000
       000000000000001b  0000000000000000  AX       0     0     4
  [13] .plt              PROGBITS         0000000000401020  00001020
       0000000000000020  0000000000000010  AX       0     0     16
  [14] .text             PROGBITS         0000000000401040  00001040
       000000000000011b  0000000000000000  AX       0     0     16
  [15] .fini             PROGBITS         000000000040115c  0000115c
       000000000000000d  0000000000000000  AX       0     0     4
  [16] .rodata           PROGBITS         0000000000402000  00002000
       0000000000000004  0000000000000004  AM       0     0     4
  [17] .eh_frame_hdr     PROGBITS         0000000000402004  00002004
       000000000000002c  0000000000000000   A       0     0     4
  [18] .eh_frame         PROGBITS         0000000000402030  00002030
       0000000000000088  0000000000000000   A       0     0     8
  [19] .note.ABI-tag     NOTE             00000000004020b8  000020b8
       0000000000000020  0000000000000000   A       0     0     4
  [20] .init_array       INIT_ARRAY       0000000000403de8  00002de8
       0000000000000008  0000000000000008  WA       0     0     8
  [21] .fini_array       FINI_ARRAY       0000000000403df0  00002df0
       0000000000000008  0000000000000008  WA       0     0     8
  [22] .dynamic          DYNAMIC          0000000000403df8  00002df8
       00000000000001e0  0000000000000010  WA       7     0     8
  [23] .got              PROGBITS         0000000000403fd8  00002fd8
       0000000000000010  0000000000000008  WA       0     0     8
  [24] .got.plt          PROGBITS         0000000000403fe8  00002fe8
       0000000000000020  0000000000000008  WA       0     0     8
  [25] .data             PROGBITS         0000000000404008  00003008
       0000000000000010  0000000000000000  WA       0     0     8
  [26] .bss              NOBITS           0000000000404018  00003018
       0000000000000008  0000000000000000  WA       0     0     1
  [27] .comment          PROGBITS         0000000000000000  00003018
       0000000000000019  0000000000000001  MS       0     0     1
  [28] .debug_aranges    PROGBITS         0000000000000000  00003040
       0000000000000150  0000000000000000           0     0     16
  [29] .debug_info       PROGBITS         0000000000000000  00003190
       0000000000000444  0000000000000000           0     0     1
  [30] .debug_abbrev     PROGBITS         0000000000000000  000035d4
       0000000000000245  0000000000000000           0     0     1
  [31] .debug_line       PROGBITS         0000000000000000  00003819
       0000000000000274  0000000000000000           0     0     1
  [32] .debug_str        PROGBITS         0000000000000000  00003a8d
       0000000000000540  0000000000000001  MS       0     0     1
  [33] .debug_line_str   PROGBITS         0000000000000000  00003fcd
       0000000000000163  0000000000000001  MS       0     0     1
  [34] .debug_rnglists   PROGBITS         0000000000000000  00004130
       0000000000000042  0000000000000000           0     0     1
  [35] .symtab           SYMTAB           0000000000000000  00004178
       0000000000000360  0000000000000018          36    20     8
  [36] .strtab           STRTAB           0000000000000000  000044d8
       00000000000001bc  0000000000000000           0     0     1
  [37] .shstrtab         STRTAB           0000000000000000  00004694
       0000000000000176  0000000000000000           0     0     1

Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), D (mbind), l (large), p (processor specific)

Notice the section header table, and how it starts after program text and program data, and how all the other ELF gunk (debug info, symtab, strtab(s)) also goes after .data. So (mostly) the real problematic readahead would be on the RW VMA that covers .data.

(This also matches my understanding of linkers, where they generally do (to put it simply) ELF headers - program headers - .text - .data - .bss, with stripable gunk after it.)

It's also the case that synchronous RA on VM_EXEC is already pretty conservative and limited, see the big if (vm_flags & VM_EXEC) in do_sync_mmap_readahead(). (I think the underlying logic behind also implies that async RA will not be started against these pages, but I am not sure).

Signature · 1 line
Pedro
#20 lore raw
Quoted 138 lines · Pedro Falcato
Quoted 7 lines · Suren Baghdasaryan
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines

much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.

Quoted 8 lines
Quoted 6 lines
Quoted 2 lines

Would a helpful heuristic be to do what 7b32f64bc512 is doing, but only if PROT_EXEC?

I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.

Thanks for the suggestion! That sounds sensible to me.

I don't think this works. Here's an example readelf -a from a random, trivial ELF I have:

pfalcato@pedro-suse:~/linux> cc -g main.c
pfalcato@pedro-suse:~/linux> readelf -a a.out
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00

  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x401040
  Start of program headers:          64 (bytes into file)
  Start of section headers:          18448 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         14
  Size of section headers:           64 (bytes)
  Number of section headers:         38

Section header string table index: 37

Section Headers:

  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .note.gnu.pr[...] NOTE             0000000000400350  00000350
       0000000000000040  0000000000000000   A       0     0     8
  [ 2] .note.gnu.bu[...] NOTE             0000000000400390  00000390
       0000000000000024  0000000000000000   A       0     0     4
  [ 3] .interp           PROGBITS         00000000004003b4  000003b4
       000000000000001c  0000000000000000   A       0     0     1
  [ 4] .hash             HASH             00000000004003d0  000003d0
       0000000000000024  0000000000000004   A       6     0     8
  [ 5] .gnu.hash         GNU_HASH         00000000004003f8  000003f8
       000000000000001c  0000000000000000   A       6     0     8
  [ 6] .dynsym           DYNSYM           0000000000400418  00000418
       0000000000000060  0000000000000018   A       7     1     8
  [ 7] .dynstr           STRTAB           0000000000400478  00000478
       000000000000004a  0000000000000000   A       0     0     1
  [ 8] .gnu.version      VERSYM           00000000004004c2  000004c2
       0000000000000008  0000000000000002   A       6     0     2
  [ 9] .gnu.version_r    VERNEED          00000000004004d0  000004d0
       0000000000000030  0000000000000000   A       7     1     8
  [10] .rela.dyn         RELA             0000000000400500  00000500
       0000000000000030  0000000000000018   A       6     0     8
  [11] .rela.plt         RELA             0000000000400530  00000530
       0000000000000018  0000000000000018  AI       6    24     8
  [12] .init             PROGBITS         0000000000401000  00001000
       000000000000001b  0000000000000000  AX       0     0     4
  [13] .plt              PROGBITS         0000000000401020  00001020
       0000000000000020  0000000000000010  AX       0     0     16
  [14] .text             PROGBITS         0000000000401040  00001040
       000000000000011b  0000000000000000  AX       0     0     16
  [15] .fini             PROGBITS         000000000040115c  0000115c
       000000000000000d  0000000000000000  AX       0     0     4
  [16] .rodata           PROGBITS         0000000000402000  00002000
       0000000000000004  0000000000000004  AM       0     0     4
  [17] .eh_frame_hdr     PROGBITS         0000000000402004  00002004
       000000000000002c  0000000000000000   A       0     0     4
  [18] .eh_frame         PROGBITS         0000000000402030  00002030
       0000000000000088  0000000000000000   A       0     0     8
  [19] .note.ABI-tag     NOTE             00000000004020b8  000020b8
       0000000000000020  0000000000000000   A       0     0     4
  [20] .init_array       INIT_ARRAY       0000000000403de8  00002de8
       0000000000000008  0000000000000008  WA       0     0     8
  [21] .fini_array       FINI_ARRAY       0000000000403df0  00002df0
       0000000000000008  0000000000000008  WA       0     0     8
  [22] .dynamic          DYNAMIC          0000000000403df8  00002df8
       00000000000001e0  0000000000000010  WA       7     0     8
  [23] .got              PROGBITS         0000000000403fd8  00002fd8
       0000000000000010  0000000000000008  WA       0     0     8
  [24] .got.plt          PROGBITS         0000000000403fe8  00002fe8
       0000000000000020  0000000000000008  WA       0     0     8
  [25] .data             PROGBITS         0000000000404008  00003008
       0000000000000010  0000000000000000  WA       0     0     8
  [26] .bss              NOBITS           0000000000404018  00003018
       0000000000000008  0000000000000000  WA       0     0     1
  [27] .comment          PROGBITS         0000000000000000  00003018
       0000000000000019  0000000000000001  MS       0     0     1
  [28] .debug_aranges    PROGBITS         0000000000000000  00003040
       0000000000000150  0000000000000000           0     0     16
  [29] .debug_info       PROGBITS         0000000000000000  00003190
       0000000000000444  0000000000000000           0     0     1
  [30] .debug_abbrev     PROGBITS         0000000000000000  000035d4
       0000000000000245  0000000000000000           0     0     1
  [31] .debug_line       PROGBITS         0000000000000000  00003819
       0000000000000274  0000000000000000           0     0     1
  [32] .debug_str        PROGBITS         0000000000000000  00003a8d
       0000000000000540  0000000000000001  MS       0     0     1
  [33] .debug_line_str   PROGBITS         0000000000000000  00003fcd
       0000000000000163  0000000000000001  MS       0     0     1
  [34] .debug_rnglists   PROGBITS         0000000000000000  00004130
       0000000000000042  0000000000000000           0     0     1
  [35] .symtab           SYMTAB           0000000000000000  00004178
       0000000000000360  0000000000000018          36    20     8
  [36] .strtab           STRTAB           0000000000000000  000044d8
       00000000000001bc  0000000000000000           0     0     1
  [37] .shstrtab         STRTAB           0000000000000000  00004694
       0000000000000176  0000000000000000           0     0     1

Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), D (mbind), l (large), p (processor specific)

Notice the section header table, and how it starts after program text and program data, and how all the other ELF gunk (debug info, symtab, strtab(s)) also goes after .data. So (mostly) the real problematic readahead would be on the RW VMA that covers .data.

(This also matches my understanding of linkers, where they generally do (to put it simply) ELF headers - program headers - .text - .data - .bss, with stripable gunk after it.)

Quoted 8 lines

It's also the case that synchronous RA on VM_EXEC is already pretty conservative and limited, see the big if (vm_flags & VM_EXEC) in do_sync_mmap_readahead(). (I think the underlying logic behind also implies that async RA will not be started against these pages, but I am not sure).

--
Pedro

Yes, I think readahead of VM_EXEC is already restricted to the VMA. Maybe there is an edge case where someone does buffered reads on an ELF file, leaving a PG_readahead flag inside the VM_EXEC range, then it could trigger async readahead beyond the end, but that sounds minor.

For next steps: Suppose we show the mmap usage in this video encoder is significantly inefficient compared to buffered reads or a big mmap and that project accepts a contribution to move away from the small mmaps. Would we be comfortable attempting this again as is? Probably there would be a lag before all users of the encoder update and they may see this bad perf.

If it isn't OK as is for the foreseeable future, we could consider applying the limit conditionally. Maybe a CONFIG or sysfs option. Or, use some heuristic to turn it on automatically, e.g. only for ELF files, or files with VM_EXEC mappings.

We are planning to measure how this change affects memory usage of Android devices in the field, but we don't have the data yet.

#21 lore raw
Quoted 141 lines · Frederick Mayle
Quoted 125 lines · Pedro Falcato
Quoted 2 lines · Suren Baghdasaryan

Thanks for the suggestion! That sounds sensible to me.

I don't think this works. Here's an example readelf -a from a random, trivial ELF I have:

pfalcato@pedro-suse:~/linux> cc -g main.c
pfalcato@pedro-suse:~/linux> readelf -a a.out
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00

  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              EXEC (Executable file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x401040
  Start of program headers:          64 (bytes into file)
  Start of section headers:          18448 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         14
  Size of section headers:           64 (bytes)
  Number of section headers:         38

Section header string table index: 37

Section Headers:

  [Nr] Name              Type             Address           Offset
       Size              EntSize          Flags  Link  Info  Align
  [ 0]                   NULL             0000000000000000  00000000
       0000000000000000  0000000000000000           0     0     0
  [ 1] .note.gnu.pr[...] NOTE             0000000000400350  00000350
       0000000000000040  0000000000000000   A       0     0     8
  [ 2] .note.gnu.bu[...] NOTE             0000000000400390  00000390
       0000000000000024  0000000000000000   A       0     0     4
  [ 3] .interp           PROGBITS         00000000004003b4  000003b4
       000000000000001c  0000000000000000   A       0     0     1
  [ 4] .hash             HASH             00000000004003d0  000003d0
       0000000000000024  0000000000000004   A       6     0     8
  [ 5] .gnu.hash         GNU_HASH         00000000004003f8  000003f8
       000000000000001c  0000000000000000   A       6     0     8
  [ 6] .dynsym           DYNSYM           0000000000400418  00000418
       0000000000000060  0000000000000018   A       7     1     8
  [ 7] .dynstr           STRTAB           0000000000400478  00000478
       000000000000004a  0000000000000000   A       0     0     1
  [ 8] .gnu.version      VERSYM           00000000004004c2  000004c2
       0000000000000008  0000000000000002   A       6     0     2
  [ 9] .gnu.version_r    VERNEED          00000000004004d0  000004d0
       0000000000000030  0000000000000000   A       7     1     8
  [10] .rela.dyn         RELA             0000000000400500  00000500
       0000000000000030  0000000000000018   A       6     0     8
  [11] .rela.plt         RELA             0000000000400530  00000530
       0000000000000018  0000000000000018  AI       6    24     8
  [12] .init             PROGBITS         0000000000401000  00001000
       000000000000001b  0000000000000000  AX       0     0     4
  [13] .plt              PROGBITS         0000000000401020  00001020
       0000000000000020  0000000000000010  AX       0     0     16
  [14] .text             PROGBITS         0000000000401040  00001040
       000000000000011b  0000000000000000  AX       0     0     16
  [15] .fini             PROGBITS         000000000040115c  0000115c
       000000000000000d  0000000000000000  AX       0     0     4
  [16] .rodata           PROGBITS         0000000000402000  00002000
       0000000000000004  0000000000000004  AM       0     0     4
  [17] .eh_frame_hdr     PROGBITS         0000000000402004  00002004
       000000000000002c  0000000000000000   A       0     0     4
  [18] .eh_frame         PROGBITS         0000000000402030  00002030
       0000000000000088  0000000000000000   A       0     0     8
  [19] .note.ABI-tag     NOTE             00000000004020b8  000020b8
       0000000000000020  0000000000000000   A       0     0     4
  [20] .init_array       INIT_ARRAY       0000000000403de8  00002de8
       0000000000000008  0000000000000008  WA       0     0     8
  [21] .fini_array       FINI_ARRAY       0000000000403df0  00002df0
       0000000000000008  0000000000000008  WA       0     0     8
  [22] .dynamic          DYNAMIC          0000000000403df8  00002df8
       00000000000001e0  0000000000000010  WA       7     0     8
  [23] .got              PROGBITS         0000000000403fd8  00002fd8
       0000000000000010  0000000000000008  WA       0     0     8
  [24] .got.plt          PROGBITS         0000000000403fe8  00002fe8
       0000000000000020  0000000000000008  WA       0     0     8
  [25] .data             PROGBITS         0000000000404008  00003008
       0000000000000010  0000000000000000  WA       0     0     8
  [26] .bss              NOBITS           0000000000404018  00003018
       0000000000000008  0000000000000000  WA       0     0     1
  [27] .comment          PROGBITS         0000000000000000  00003018
       0000000000000019  0000000000000001  MS       0     0     1
  [28] .debug_aranges    PROGBITS         0000000000000000  00003040
       0000000000000150  0000000000000000           0     0     16
  [29] .debug_info       PROGBITS         0000000000000000  00003190
       0000000000000444  0000000000000000           0     0     1
  [30] .debug_abbrev     PROGBITS         0000000000000000  000035d4
       0000000000000245  0000000000000000           0     0     1
  [31] .debug_line       PROGBITS         0000000000000000  00003819
       0000000000000274  0000000000000000           0     0     1
  [32] .debug_str        PROGBITS         0000000000000000  00003a8d
       0000000000000540  0000000000000001  MS       0     0     1
  [33] .debug_line_str   PROGBITS         0000000000000000  00003fcd
       0000000000000163  0000000000000001  MS       0     0     1
  [34] .debug_rnglists   PROGBITS         0000000000000000  00004130
       0000000000000042  0000000000000000           0     0     1
  [35] .symtab           SYMTAB           0000000000000000  00004178
       0000000000000360  0000000000000018          36    20     8
  [36] .strtab           STRTAB           0000000000000000  000044d8
       00000000000001bc  0000000000000000           0     0     1
  [37] .shstrtab         STRTAB           0000000000000000  00004694
       0000000000000176  0000000000000000           0     0     1

Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), D (mbind), l (large), p (processor specific)

Notice the section header table, and how it starts after program text and program data, and how all the other ELF gunk (debug info, symtab, strtab(s)) also goes after .data. So (mostly) the real problematic readahead would be on the RW VMA that covers .data.

(This also matches my understanding of linkers, where they generally do (to put it simply) ELF headers - program headers - .text - .data - .bss, with stripable gunk after it.)

Quoted 8 lines

It's also the case that synchronous RA on VM_EXEC is already pretty conservative and limited, see the big if (vm_flags & VM_EXEC) in do_sync_mmap_readahead(). (I think the underlying logic behind also implies that async RA will not be started against these pages, but I am not sure).

--
Pedro

Yes, I think readahead of VM_EXEC is already restricted to the VMA. Maybe there is an edge case where someone does buffered reads on an ELF file, leaving a PG_readahead flag inside the VM_EXEC range, then it could trigger async readahead beyond the end, but that sounds minor.

Quoted 6 lines

For next steps: Suppose we show the mmap usage in this video encoder is significantly inefficient compared to buffered reads or a big mmap and that project accepts a contribution to move away from the small mmaps. Would we be comfortable attempting this again as is? Probably there would be a lag before all users of the encoder update and they may see this bad perf.

How could you be sure that there isn't some other proprietary (or even just another open-source) software out there that relies on similar things, but doesn't provide easy benchmarks that can easily catch this?

Signature · 3 lines
Cheers,

David