Revert "mm: limit filemap_fault readahead to VMA boundaries"
Subsystems (3)
PATCH 1/1 Revert "mm: limit filemap_fault readahead to VMA boundaries"
This reverts commit 7b32f64bc512b40b268776c5ac4d354b325b3197.
This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit de97ae6222c1 ("mm/readahead: no PG_readahead on EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries") only.
--- include/linux/pagemap.h | 2 -- mm/filemap.c | 4 ---- mm/readahead.c | 6 +----- 3 files changed, 1 insertion(+), 11 deletions(-)
include/linux/pagemap.h +0 -2
@@ -1348,7 +1348,6 @@ struct readahead_control {struct file_ra_state *ra;/* private: use the readahead_* accessors instead */ pgoff_t _index; -pgoff_t _max_index;/* limit readahead to _max_index, inclusive */ unsigned int _nr_pages;unsigned int _batch_count;bool dropbehind;
@@ -1362,7 +1361,6 @@ struct readahead_control { .mapping = m, \ .ra = r, \ ._index = i, \ - ._max_index = ULONG_MAX, \ }#define VM_READAHEAD_PAGES (SZ_128K / PAGE_SIZE)
mm/filemap.c +0 -4
@@ -3312,8 +3312,6 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)unsigned int thp_order =0 ;unsigned short mmap_miss; - ractl._max_index = vmf->vma->vm_pgoff +vma_pages (vmf->vma) -1 ; -/* Use the readahead code, even if readahead is disabled */ if (IS_ENABLED (CONFIG_TRANSPARENT_HUGEPAGE ) && (vm_flags &VM_HUGEPAGE )) {/*
@@ -3409,7 +3407,6 @@ static struct file *do_sync_mmap_readahead(struct vm_fault *vmf)* mmap read-around */ ra->start =max_t (long,0 , vmf->pgoff - ra->ra_pages /2 ); - ra->start =max (ra->start, vmf->vma->vm_pgoff); ra->size = ra->ra_pages; ra->async_size = ra->ra_pages /4 ; ra->order =0 ;
@@ -3457,7 +3454,6 @@ static struct file *do_async_mmap_readahead(struct vm_fault *vmf, }if (folio_test_readahead (folio)) { - ractl._max_index = vmf->vma->vm_pgoff +vma_pages (vmf->vma) -1 ; fpin =maybe_unlock_mmap_for_io (vmf, fpin);page_cache_async_ra (&ractl, folio, ra->ra_pages); }
mm/readahead.c +1 -5
@@ -335,8 +335,6 @@ static void do_page_cache_ra(struct readahead_control *ractl,return ; end_index = (isize -1 ) >>PAGE_SHIFT ; -if (end_index > ractl->_max_index) - end_index = ractl->_max_index;if (index > end_index)return ;/* Don't read past the page containing the last byte of the file */
@@ -487,7 +485,7 @@ void page_cache_ra_order(struct readahead_control *ractl,pgoff_t start =readahead_index (ractl);pgoff_t index = start;unsigned int min_order =mapping_min_folio_order (mapping); -pgoff_t limit; +pgoff_t limit = (i_size_read (mapping->host) -1 ) >>PAGE_SHIFT ;pgoff_t mark;unsigned int nofs;int err =0 ;
@@ -500,8 +498,6 @@ void page_cache_ra_order(struct readahead_control *ractl, goto fallback; } - limit = (i_size_read (mapping->host) -1 ) >>PAGE_SHIFT ; - limit =min (limit, ractl->_max_index);if (limit > index + ra->size -1 ) { limit = index + ra->size -1 ; mark = index + ra->size - ra->async_size;
--
2.54.0
Quoted 11 lines · Lorenzo Stoakes
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.
Quoted 5 lines
Fixes 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries")Reported-by kernel test robot <[email protected]>Signed-off-by Lorenzo Stoakes <[email protected]>---
Thanks!
Signature · 3 lines
Cheers, David
Quoted 11 lines · Lorenzo Stoakes
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.
Quoted 4 lines
Fixes 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries")Reported-by kernel test robot <[email protected]>Signed-off-by Lorenzo Stoakes <[email protected]>
Signature · 1 line
Pedro
Quoted 11 lines · Lorenzo Stoakes
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.
Quoted 4 lines
Fixes 7b32f64bc512 ("mm: limit filemap_fault readahead to VMA boundaries")Reported-by kernel test robot <[email protected]>Signed-off-by Lorenzo Stoakes <[email protected]>
Andrew Morton
maintainerQuoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.
I'm a little conflicted here.
7b32f64bc512 avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.
IOW, the revert might make the kernel worse, overall.
If someone plans to get down and analyse that test case then come up with a new version of 7b32f64bc512 then OK. Is there such a person?
I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.
Matthew Wilcox
maintainerQuoted 20 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.I'm a little conflicted here.
7b32f64bc512avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.
Quoted 8 lines
IOW, the revert might make the kernel worse, overall.
If someone plans to get down and analyse that test case then come up with a new version of
7b32f64bc512then OK. Is there such a person?I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.
Suren did that work.
https://lore.kernel.org/linux-mm/CAJuCfpH_Fp=J-m-kVZ3VVmqH0cAfNX6CQKWt3XmAJq1Rp9sFwQ@mail.gmail.com/
+cc Suren, Pedro, David - I messed up the cc on this patch.
Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.
Quoted 4 lines
Quoted 4 lines
I'm a little conflicted here.
7b32f64bc512avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we care
I wouldn't say it's a bug fix? The kernel has always done this with no reported bugs.
Quoted 11 lines
Quoted 11 lines
about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.
IOW, the revert might make the kernel worse, overall.
If someone plans to get down and analyse that test case then come up with a new version of
7b32f64bc512then OK. Is there such a person?I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.
As Willy points out [0], we've identified that it's a real usecase that we're regressing.
There's likely others too in the real world.
This approach has also not been without debate in the past (see [1]), I think this patch slipped under the radar on that front a bit.
So yeah, this does not to be reverted, and as a hotfix sooner rather than later.
Quoted 3 lines
Suren did that work.
https://lore.kernel.org/linux-mm/CAJuCfpH_Fp=J-m-kVZ3VVmqH0cAfNX6CQKWt3XmAJq1Rp9sFwQ@mail.gmail.com/
Thanks for highlighting that Willy (copied -> [0]).
Overall, if a patch causes a major regression in a core mm subsystem, people can wait a cycle to re-evaluate.
Thanks, Lorenzo
[0]:https://lore.kernel.org/linux-mm/CAJuCfpH_Fp=J-m-kVZ3VVmqH0cAfNX6CQKWt3XmAJq1Rp9sFwQ@mail.gmail.com/ [1]:https://lore.kernel.org/linux-mm/CAC_TJvfG8GcwG_2w1o6GOTZS8tfEx2h9A91qsenYfYsX8Te=Bg@mail.gmail.com/
Quoted 1 line · Lorenzo Stoakes
So yeah, this does not to be reverted, and as a hotfix sooner rather than later.
Does _need_ to be reverted :)
What an unfortunate typo... :P
Quoted 33 lines · Lorenzo Stoakes
+cc Suren, Pedro, David - I messed up the cc on this patch.
Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.Quoted 4 lines
Quoted 4 lines
I'm a little conflicted here.
7b32f64bc512avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we careI wouldn't say it's a bug fix? The kernel has always done this with no reported bugs.
Quoted 8 lines
Quoted 8 lines
about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.
IOW, the revert might make the kernel worse, overall.
If someone plans to get down and analyse that test case then come up with a new version of
7b32f64bc512then OK. Is there such a person?
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.
As Willy points out [0], we've identified that it's a real usecase that we're regressing.
There's likely others too in the real world.
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Quoted 3 lines
This approach has also not been without debate in the past (see [1]), I think this patch slipped under the radar on that front a bit.
Yeah, sadly it seems like the patch _may_ need a rather substantial rework :| So I would err on the side of caution for now and revert it as well.
Signature · 1 line
Pedro
Quoted 39 lines · Pedro Falcato
Quoted 37 lines · Lorenzo Stoakes
+cc Suren, Pedro, David - I messed up the cc on this patch.
Quoted 14 lines · Matthew Wilcox
Quoted 13 lines · Andrew Morton
Quoted 10 lines · On Fri, 19 Jun 2026 12:28:51 +0100 Lorenzo Stoakes <[email protected]> wrote:
This reverts commit
7b32f64bc512b40b268776c5ac4d354b325b3197.This patch caused a significant performance regression, so revert it, and we can determine whether the approach is sensible or not moving forwards, and if so how to avoid this.
There was a merge conflict with commit
de97ae6222c1("mm/readahead: noPG_readaheadon EOF"), care was taken to ensure that the revert retained the behaviour of this patch and cleanly reverts commit7b32f64bc512("mm: limitfilemap_faultreadahead to VMA boundaries") only.Quoted 4 lines
Quoted 4 lines
I'm a little conflicted here.
7b32f64bc512avoided readahead of "file pages outside the mapped region", which is clearly desirable (arguably a bug fix?) and we careI wouldn't say it's a bug fix? The kernel has always done this with no reported bugs.
Quoted 11 lines
Quoted 11 lines
about performance of executable mappings. Whereas it isn't clear that we care about whatever the heck that test case was doing.
IOW, the revert might make the kernel worse, overall.
If someone plans to get down and analyse that test case then come up with a new version of
7b32f64bc512then OK. Is there such a person?I'll park the revert in mm-unstable for now, but would prefer not to rush it in until we better understand what's going on with that test case and what can be done to address it.
Quoted 8 lines
Quoted 4 lines
As Willy points out [0], we've identified that it's a real usecase that we're regressing.
There's likely others too in the real world.
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.
Quoted 9 lines
Quoted 3 lines
This approach has also not been without debate in the past (see [1]), I think this patch slipped under the radar on that front a bit.
Yeah, sadly it seems like the patch _may_ need a rather substantial rework :| So I would err on the side of caution for now and revert it as well.
--
Pedro
Matthew Wilcox
maintainerQuoted 7 lines · Suren Baghdasaryan
Quoted 3 lines
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.
If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling read() for a single byte at a time.
Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.
Quoted 10 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.
Quoted 8 lines
If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling
read()for a single byte at a time.Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.
That what I meant in the last part of my reply :)
But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.
Quoted 20 lines · Suren Baghdasaryan
Quoted 10 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.
Quoted 8 lines
If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling
read()for a single byte at a time.Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.
Quoted 5 lines
That what I meant in the last part of my reply :)
But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.
To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.
(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)
Signature · 1 line
Pedro
Quoted 22 lines · Pedro Falcato
Quoted 20 lines · Suren Baghdasaryan
Quoted 18 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.
If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling
read()for a single byte at a time.Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.
Quoted 8 lines
Quoted 5 lines
That what I meant in the last part of my reply :)
But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.
To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.
Ack.
Quoted 3 lines
(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)
For such streaminig workload I think read() would be more appropriate, and the change limiting read-ahead by VMA size would not affect it.
Quoted 3 lines
--
Pedro
Jan Kara
maintainerQuoted 32 lines · Suren Baghdasaryan
Quoted 22 lines · Pedro Falcato
Quoted 20 lines · Suren Baghdasaryan
Quoted 18 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.
If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling
read()for a single byte at a time.Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.
Quoted 8 lines
Quoted 5 lines
That what I meant in the last part of my reply :)
But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.
To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.
Quoted 7 lines
Ack.
Quoted 2 lines
(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)
For such streaminig workload I think
read()would be more appropriate, and the change limiting read-ahead by VMA size would not affect it.
In fact if you look at that app, it has three different methods of reading the input frames (mmap, normal read, fully buffered file in memory) and one is chosen based on config option. So the benchmark apparently configures the mmap method of accessing the input file.
Anyway I agree with you using mmap-frame-by-frame approach for this usecase is kind of odd but not completely insane and it does demostrate people do odd things like this. For others the usecase of Android doing tons of small sparse mmaps into executable might look odd as well :) So this seems to be one of the cases where it's difficult to please everybody with the same behavior and we usually stick to the legacy behavior in cases like that (as much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.
Honza
Signature · 2 lines
Jan Kara <[email protected]> SUSE Labs, CR
Quoted 46 lines · Jan Kara
Quoted 32 lines · Suren Baghdasaryan
Quoted 22 lines · Pedro Falcato
Quoted 20 lines · Suren Baghdasaryan
Quoted 18 lines · Matthew Wilcox
Quoted 7 lines · Suren Baghdasaryan
Quoted 3 lines
Yeah, the mmap usage on SVT-AV1 is a little suspicious, but I guess the pattern may come up time and time again for people doing chunked mmap reads. I think we need to take a closer look at this change.
Frame-by-frame mmapping for a streaming workoad is quite questionable IMHO and I don't think we should be optimizing or encouraging such usage, but I understand the reasoning for the revert.
If userspace is properly tuned, it doesn't need automatic readahead at all; it can disable it and issue manual readaheads. The point of automatic readahead is to cope with userspace which is doing stupid things like calling
read()for a single byte at a time.Could it have better defaults? Maybe! It's worth a try. But those attempts need to have a very low bar for reversion when it turns out they affect other workloads.
Quoted 8 lines
Quoted 5 lines
That what I meant in the last part of my reply :)
But in general, I think that benchmark measuring "how many frames per second a CPU can encode" should be revised so it doesn't depend on file prefetching mechanisms. Otherwise it's measuring something else.
To be clear, that's not (necessarily) a benchmark, the code we were looking at is their encoder app.
Quoted 7 lines
Ack.
Quoted 2 lines
(it's also generally not possible to prefetch the whole thing into RAM because, well, raw video files are quite large)
For such streaminig workload I think
read()would be more appropriate, and the change limiting read-ahead by VMA size would not affect it.In fact if you look at that app, it has three different methods of reading the input frames (mmap, normal read, fully buffered file in memory) and one is chosen based on config option. So the benchmark apparently configures the mmap method of accessing the input file.
Quoted 9 lines
Anyway I agree with you using mmap-frame-by-frame approach for this usecase is kind of odd but not completely insane and it does demostrate people do odd things like this. For others the usecase of Android doing tons of small sparse mmaps into executable might look odd as well :) So this seems to be one of the cases where it's difficult to please everybody with the same behavior and we usually stick to the legacy behavior in cases like that (as much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.
Reading the thread, I think we're all in agreement that this is a workload that exists and we are regressing it here, thus the revert is the right course of action.
Andrew - could we move this to the hotfixes branch please?
Quoted 5 lines
Honza
--
Jan Kara <[email protected]>
SUSE Labs, CR
Thanks, Lorenzo
Andrew Morton
maintainerQuoted 9 lines · On Mon, 22 Jun 2026 14:55:14 +0100 Lorenzo Stoakes <[email protected]> wrote:
Quoted 9 lines
Anyway I agree with you using mmap-frame-by-frame approach for this usecase is kind of odd but not completely insane and it does demostrate people do odd things like this. For others the usecase of Android doing tons of small sparse mmaps into executable might look odd as well :) So this seems to be one of the cases where it's difficult to please everybody with the same behavior and we usually stick to the legacy behavior in cases like that (as much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.
Would a helpful heuristic be to do what 7b32f64bc512 is doing, but only if PROT_EXEC?
Quoted 5 lines
Reading the thread, I think we're all in agreement that this is a workload that exists and we are regressing it here, thus the revert is the right course of action.
Andrew - could we move this to the hotfixes branch please?
No need? I'll send Linus the mm-stable batch #2 this week.
Matthew Wilcox
maintainerQuoted 6 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines
much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.
Would a helpful heuristic be to do what
7b32f64bc512is doing, but only ifPROT_EXEC?
I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines
much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.
Quoted 6 lines
Quoted 2 lines
Would a helpful heuristic be to do what
7b32f64bc512is doing, but only ifPROT_EXEC?I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.
Thanks for the suggestion! That sounds sensible to me.
Quoted 1 line
Quoted 7 lines · Suren Baghdasaryan
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines
much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Would a helpful heuristic be to do what
7b32f64bc512is doing, but only ifPROT_EXEC?I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.
Thanks for the suggestion! That sounds sensible to me.
I don't think this works. Here's an example readelf -a from a random, trivial ELF I have:
pfalcato@pedro-suse:~/linux> cc -g main.c
pfalcato@pedro-suse:~/linux> readelf -a a.out
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: EXEC (Executable file) Machine: Advanced Micro Devices X86-64 Version: 0x1 Entry point address: 0x401040 Start of program headers: 64 (bytes into file) Start of section headers: 18448 (bytes into file) Flags: 0x0 Size of this header: 64 (bytes) Size of program headers: 56 (bytes) Number of program headers: 14 Size of section headers: 64 (bytes) Number of section headers: 38
Section header string table index: 37
Section Headers:
[Nr] Name Type Address Offset
Size EntSize Flags Link Info Align
[ 0] NULL 0000000000000000 00000000
0000000000000000 0000000000000000 0 0 0
[ 1] .note.gnu.pr[...] NOTE 0000000000400350 00000350
0000000000000040 0000000000000000 A 0 0 8
[ 2] .note.gnu.bu[...] NOTE 0000000000400390 00000390
0000000000000024 0000000000000000 A 0 0 4
[ 3] .interp PROGBITS 00000000004003b4 000003b4
000000000000001c 0000000000000000 A 0 0 1
[ 4] .hash HASH 00000000004003d0 000003d0
0000000000000024 0000000000000004 A 6 0 8
[ 5] .gnu.hash GNU_HASH 00000000004003f8 000003f8
000000000000001c 0000000000000000 A 6 0 8
[ 6] .dynsym DYNSYM 0000000000400418 00000418
0000000000000060 0000000000000018 A 7 1 8
[ 7] .dynstr STRTAB 0000000000400478 00000478
000000000000004a 0000000000000000 A 0 0 1
[ 8] .gnu.version VERSYM 00000000004004c2 000004c2
0000000000000008 0000000000000002 A 6 0 2
[ 9] .gnu.version_r VERNEED 00000000004004d0 000004d0
0000000000000030 0000000000000000 A 7 1 8
[10] .rela.dyn RELA 0000000000400500 00000500
0000000000000030 0000000000000018 A 6 0 8
[11] .rela.plt RELA 0000000000400530 00000530
0000000000000018 0000000000000018 AI 6 24 8
[12] .init PROGBITS 0000000000401000 00001000
000000000000001b 0000000000000000 AX 0 0 4
[13] .plt PROGBITS 0000000000401020 00001020
0000000000000020 0000000000000010 AX 0 0 16
[14] .text PROGBITS 0000000000401040 00001040
000000000000011b 0000000000000000 AX 0 0 16
[15] .fini PROGBITS 000000000040115c 0000115c
000000000000000d 0000000000000000 AX 0 0 4
[16] .rodata PROGBITS 0000000000402000 00002000
0000000000000004 0000000000000004 AM 0 0 4
[17] .eh_frame_hdr PROGBITS 0000000000402004 00002004
000000000000002c 0000000000000000 A 0 0 4
[18] .eh_frame PROGBITS 0000000000402030 00002030
0000000000000088 0000000000000000 A 0 0 8
[19] .note.ABI-tag NOTE 00000000004020b8 000020b8
0000000000000020 0000000000000000 A 0 0 4
[20] .init_array INIT_ARRAY 0000000000403de8 00002de8
0000000000000008 0000000000000008 WA 0 0 8
[21] .fini_array FINI_ARRAY 0000000000403df0 00002df0
0000000000000008 0000000000000008 WA 0 0 8
[22] .dynamic DYNAMIC 0000000000403df8 00002df8
00000000000001e0 0000000000000010 WA 7 0 8
[23] .got PROGBITS 0000000000403fd8 00002fd8
0000000000000010 0000000000000008 WA 0 0 8
[24] .got.plt PROGBITS 0000000000403fe8 00002fe8
0000000000000020 0000000000000008 WA 0 0 8
[25] .data PROGBITS 0000000000404008 00003008
0000000000000010 0000000000000000 WA 0 0 8
[26] .bss NOBITS 0000000000404018 00003018
0000000000000008 0000000000000000 WA 0 0 1
[27] .comment PROGBITS 0000000000000000 00003018
0000000000000019 0000000000000001 MS 0 0 1
[28] .debug_aranges PROGBITS 0000000000000000 00003040
0000000000000150 0000000000000000 0 0 16
[29] .debug_info PROGBITS 0000000000000000 00003190
0000000000000444 0000000000000000 0 0 1
[30] .debug_abbrev PROGBITS 0000000000000000 000035d4
0000000000000245 0000000000000000 0 0 1
[31] .debug_line PROGBITS 0000000000000000 00003819
0000000000000274 0000000000000000 0 0 1
[32] .debug_str PROGBITS 0000000000000000 00003a8d
0000000000000540 0000000000000001 MS 0 0 1
[33] .debug_line_str PROGBITS 0000000000000000 00003fcd
0000000000000163 0000000000000001 MS 0 0 1
[34] .debug_rnglists PROGBITS 0000000000000000 00004130
0000000000000042 0000000000000000 0 0 1
[35] .symtab SYMTAB 0000000000000000 00004178
0000000000000360 0000000000000018 36 20 8
[36] .strtab STRTAB 0000000000000000 000044d8
00000000000001bc 0000000000000000 0 0 1
[37] .shstrtab STRTAB 0000000000000000 00004694
0000000000000176 0000000000000000 0 0 1Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), D (mbind), l (large), p (processor specific)
Notice the section header table, and how it starts after program text and program data, and how all the other ELF gunk (debug info, symtab, strtab(s)) also goes after .data. So (mostly) the real problematic readahead would be on the RW VMA that covers .data.
(This also matches my understanding of linkers, where they generally do (to put it simply) ELF headers - program headers - .text - .data - .bss, with stripable gunk after it.)
It's also the case that synchronous RA on VM_EXEC is already pretty conservative and limited, see the big if (vm_flags & VM_EXEC) in do_sync_mmap_readahead(). (I think the underlying logic behind also implies that async RA will not be started against these pages, but I am not sure).
Signature · 1 line
Pedro
Quoted 138 lines · Pedro Falcato
Quoted 7 lines · Suren Baghdasaryan
Quoted 6 lines · Matthew Wilcox
Quoted 4 lines · Andrew Morton
Quoted 3 lines
Quoted 3 lines
much as I personally find restricting mmap readahead to a VMA a sensible thing to do). We just need to figure out how to improve the Android usecase.
Quoted 8 lines
Quoted 6 lines
Quoted 2 lines
Would a helpful heuristic be to do what
7b32f64bc512is doing, but only ifPROT_EXEC?I was wondering the same thing. But I think it's right to back this out for now and try that after -rc1 so it gets some time soaking and bot-teesting.
Thanks for the suggestion! That sounds sensible to me.
I don't think this works. Here's an example readelf -a from a random, trivial ELF I have:
pfalcato@pedro-suse:~/linux> cc -g
main.c
pfalcato@pedro-suse:~/linux> readelf -a a.out
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: EXEC (Executable file) Machine: Advanced Micro Devices X86-64 Version: 0x1 Entry point address: 0x401040 Start of program headers: 64 (bytes into file) Start of section headers: 18448 (bytes into file) Flags: 0x0 Size of this header: 64 (bytes) Size of program headers: 56 (bytes) Number of program headers: 14 Size of section headers: 64 (bytes) Number of section headers: 38Section header string table index: 37
Section Headers:
[Nr] Name Type Address Offset Size EntSize Flags Link Info Align [ 0] NULL 0000000000000000 00000000 0000000000000000 0000000000000000 0 0 0 [ 1] .note.gnu.pr[...] NOTE 0000000000400350 00000350 0000000000000040 0000000000000000 A 0 0 8 [ 2] .note.gnu.bu[...] NOTE 0000000000400390 00000390 0000000000000024 0000000000000000 A 0 0 4 [ 3] .interp PROGBITS 00000000004003b4 000003b4 000000000000001c 0000000000000000 A 0 0 1 [ 4] .hash HASH 00000000004003d0 000003d0 0000000000000024 0000000000000004 A 6 0 8 [ 5] .gnu.hash GNU_HASH 00000000004003f8 000003f8 000000000000001c 0000000000000000 A 6 0 8 [ 6] .dynsym DYNSYM 0000000000400418 00000418 0000000000000060 0000000000000018 A 7 1 8 [ 7] .dynstr STRTAB 0000000000400478 00000478 000000000000004a 0000000000000000 A 0 0 1 [ 8] .gnu.version VERSYM 00000000004004c2 000004c2 0000000000000008 0000000000000002 A 6 0 2 [ 9] .gnu.version_r VERNEED 00000000004004d0 000004d0 0000000000000030 0000000000000000 A 7 1 8 [10] .rela.dyn RELA 0000000000400500 00000500 0000000000000030 0000000000000018 A 6 0 8 [11] .rela.plt RELA 0000000000400530 00000530 0000000000000018 0000000000000018 AI 6 24 8 [12] .init PROGBITS 0000000000401000 00001000 000000000000001b 0000000000000000 AX 0 0 4 [13] .plt PROGBITS 0000000000401020 00001020 0000000000000020 0000000000000010 AX 0 0 16 [14] .text PROGBITS 0000000000401040 00001040 000000000000011b 0000000000000000 AX 0 0 16 [15] .fini PROGBITS 000000000040115c 0000115c 000000000000000d 0000000000000000 AX 0 0 4 [16] .rodata PROGBITS 0000000000402000 00002000 0000000000000004 0000000000000004 AM 0 0 4 [17] .eh_frame_hdr PROGBITS 0000000000402004 00002004 000000000000002c 0000000000000000 A 0 0 4 [18] .eh_frame PROGBITS 0000000000402030 00002030 0000000000000088 0000000000000000 A 0 0 8 [19] .note.ABI-tag NOTE 00000000004020b8 000020b8 0000000000000020 0000000000000000 A 0 0 4 [20] .init_array INIT_ARRAY 0000000000403de8 00002de8 0000000000000008 0000000000000008 WA 0 0 8 [21] .fini_array FINI_ARRAY 0000000000403df0 00002df0 0000000000000008 0000000000000008 WA 0 0 8 [22] .dynamic DYNAMIC 0000000000403df8 00002df8 00000000000001e0 0000000000000010 WA 7 0 8 [23] .got PROGBITS 0000000000403fd8 00002fd8 0000000000000010 0000000000000008 WA 0 0 8 [24] .got.plt PROGBITS 0000000000403fe8 00002fe8 0000000000000020 0000000000000008 WA 0 0 8 [25] .data PROGBITS 0000000000404008 00003008 0000000000000010 0000000000000000 WA 0 0 8 [26] .bss NOBITS 0000000000404018 00003018 0000000000000008 0000000000000000 WA 0 0 1 [27] .comment PROGBITS 0000000000000000 00003018 0000000000000019 0000000000000001 MS 0 0 1 [28] .debug_aranges PROGBITS 0000000000000000 00003040 0000000000000150 0000000000000000 0 0 16 [29] .debug_info PROGBITS 0000000000000000 00003190 0000000000000444 0000000000000000 0 0 1 [30] .debug_abbrev PROGBITS 0000000000000000 000035d4 0000000000000245 0000000000000000 0 0 1 [31] .debug_line PROGBITS 0000000000000000 00003819 0000000000000274 0000000000000000 0 0 1 [32] .debug_str PROGBITS 0000000000000000 00003a8d 0000000000000540 0000000000000001 MS 0 0 1 [33] .debug_line_str PROGBITS 0000000000000000 00003fcd 0000000000000163 0000000000000001 MS 0 0 1 [34] .debug_rnglists PROGBITS 0000000000000000 00004130 0000000000000042 0000000000000000 0 0 1 [35] .symtab SYMTAB 0000000000000000 00004178 0000000000000360 0000000000000018 36 20 8 [36] .strtab STRTAB 0000000000000000 000044d8 00000000000001bc 0000000000000000 0 0 1 [37] .shstrtab STRTAB 0000000000000000 00004694 0000000000000176 0000000000000000 0 0 1Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), D (mbind), l (large), p (processor specific)Notice the section header table, and how it starts after program text and program data, and how all the other ELF gunk (debug info, symtab, strtab(s)) also goes after .data. So (mostly) the real problematic readahead would be on the RW VMA that covers .data.
(This also matches my understanding of linkers, where they generally do (to put it simply) ELF headers - program headers - .text - .data - .bss, with stripable gunk after it.)
Quoted 8 lines
It's also the case that synchronous RA on
VM_EXECis already pretty conservative and limited, see the big if (vm_flags&VM_EXEC) indo_sync_mmap_readahead(). (I think the underlying logic behind also implies that async RA will not be started against these pages, but I am not sure).--
Pedro
Yes, I think readahead of VM_EXEC is already restricted to the VMA. Maybe there is an edge case where someone does buffered reads on an ELF file, leaving a PG_readahead flag inside the VM_EXEC range, then it could trigger async readahead beyond the end, but that sounds minor.
For next steps: Suppose we show the mmap usage in this video encoder is significantly inefficient compared to buffered reads or a big mmap and that project accepts a contribution to move away from the small mmaps. Would we be comfortable attempting this again as is? Probably there would be a lag before all users of the encoder update and they may see this bad perf.
If it isn't OK as is for the foreseeable future, we could consider applying the limit conditionally. Maybe a CONFIG or sysfs option. Or, use some heuristic to turn it on automatically, e.g. only for ELF files, or files with VM_EXEC mappings.
We are planning to measure how this change affects memory usage of Android devices in the field, but we don't have the data yet.
Quoted 141 lines · Frederick Mayle
Quoted 125 lines · Pedro Falcato
Quoted 2 lines · Suren Baghdasaryan
Thanks for the suggestion! That sounds sensible to me.
I don't think this works. Here's an example readelf -a from a random, trivial ELF I have:
pfalcato@pedro-suse:~/linux> cc -g
main.c
pfalcato@pedro-suse:~/linux> readelf -a a.out
ELF Header:
Magic: 7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00Class: ELF64 Data: 2's complement, little endian Version: 1 (current) OS/ABI: UNIX - System V ABI Version: 0 Type: EXEC (Executable file) Machine: Advanced Micro Devices X86-64 Version: 0x1 Entry point address: 0x401040 Start of program headers: 64 (bytes into file) Start of section headers: 18448 (bytes into file) Flags: 0x0 Size of this header: 64 (bytes) Size of program headers: 56 (bytes) Number of program headers: 14 Size of section headers: 64 (bytes) Number of section headers: 38Section header string table index: 37
Section Headers:
[Nr] Name Type Address Offset Size EntSize Flags Link Info Align [ 0] NULL 0000000000000000 00000000 0000000000000000 0000000000000000 0 0 0 [ 1] .note.gnu.pr[...] NOTE 0000000000400350 00000350 0000000000000040 0000000000000000 A 0 0 8 [ 2] .note.gnu.bu[...] NOTE 0000000000400390 00000390 0000000000000024 0000000000000000 A 0 0 4 [ 3] .interp PROGBITS 00000000004003b4 000003b4 000000000000001c 0000000000000000 A 0 0 1 [ 4] .hash HASH 00000000004003d0 000003d0 0000000000000024 0000000000000004 A 6 0 8 [ 5] .gnu.hash GNU_HASH 00000000004003f8 000003f8 000000000000001c 0000000000000000 A 6 0 8 [ 6] .dynsym DYNSYM 0000000000400418 00000418 0000000000000060 0000000000000018 A 7 1 8 [ 7] .dynstr STRTAB 0000000000400478 00000478 000000000000004a 0000000000000000 A 0 0 1 [ 8] .gnu.version VERSYM 00000000004004c2 000004c2 0000000000000008 0000000000000002 A 6 0 2 [ 9] .gnu.version_r VERNEED 00000000004004d0 000004d0 0000000000000030 0000000000000000 A 7 1 8 [10] .rela.dyn RELA 0000000000400500 00000500 0000000000000030 0000000000000018 A 6 0 8 [11] .rela.plt RELA 0000000000400530 00000530 0000000000000018 0000000000000018 AI 6 24 8 [12] .init PROGBITS 0000000000401000 00001000 000000000000001b 0000000000000000 AX 0 0 4 [13] .plt PROGBITS 0000000000401020 00001020 0000000000000020 0000000000000010 AX 0 0 16 [14] .text PROGBITS 0000000000401040 00001040 000000000000011b 0000000000000000 AX 0 0 16 [15] .fini PROGBITS 000000000040115c 0000115c 000000000000000d 0000000000000000 AX 0 0 4 [16] .rodata PROGBITS 0000000000402000 00002000 0000000000000004 0000000000000004 AM 0 0 4 [17] .eh_frame_hdr PROGBITS 0000000000402004 00002004 000000000000002c 0000000000000000 A 0 0 4 [18] .eh_frame PROGBITS 0000000000402030 00002030 0000000000000088 0000000000000000 A 0 0 8 [19] .note.ABI-tag NOTE 00000000004020b8 000020b8 0000000000000020 0000000000000000 A 0 0 4 [20] .init_array INIT_ARRAY 0000000000403de8 00002de8 0000000000000008 0000000000000008 WA 0 0 8 [21] .fini_array FINI_ARRAY 0000000000403df0 00002df0 0000000000000008 0000000000000008 WA 0 0 8 [22] .dynamic DYNAMIC 0000000000403df8 00002df8 00000000000001e0 0000000000000010 WA 7 0 8 [23] .got PROGBITS 0000000000403fd8 00002fd8 0000000000000010 0000000000000008 WA 0 0 8 [24] .got.plt PROGBITS 0000000000403fe8 00002fe8 0000000000000020 0000000000000008 WA 0 0 8 [25] .data PROGBITS 0000000000404008 00003008 0000000000000010 0000000000000000 WA 0 0 8 [26] .bss NOBITS 0000000000404018 00003018 0000000000000008 0000000000000000 WA 0 0 1 [27] .comment PROGBITS 0000000000000000 00003018 0000000000000019 0000000000000001 MS 0 0 1 [28] .debug_aranges PROGBITS 0000000000000000 00003040 0000000000000150 0000000000000000 0 0 16 [29] .debug_info PROGBITS 0000000000000000 00003190 0000000000000444 0000000000000000 0 0 1 [30] .debug_abbrev PROGBITS 0000000000000000 000035d4 0000000000000245 0000000000000000 0 0 1 [31] .debug_line PROGBITS 0000000000000000 00003819 0000000000000274 0000000000000000 0 0 1 [32] .debug_str PROGBITS 0000000000000000 00003a8d 0000000000000540 0000000000000001 MS 0 0 1 [33] .debug_line_str PROGBITS 0000000000000000 00003fcd 0000000000000163 0000000000000001 MS 0 0 1 [34] .debug_rnglists PROGBITS 0000000000000000 00004130 0000000000000042 0000000000000000 0 0 1 [35] .symtab SYMTAB 0000000000000000 00004178 0000000000000360 0000000000000018 36 20 8 [36] .strtab STRTAB 0000000000000000 000044d8 00000000000001bc 0000000000000000 0 0 1 [37] .shstrtab STRTAB 0000000000000000 00004694 0000000000000176 0000000000000000 0 0 1Key to Flags:
W (write), A (alloc), X (execute), M (merge), S (strings), I (info), L (link order), O (extra OS processing required), G (group), T (TLS), C (compressed), x (unknown), o (OS specific), E (exclude), D (mbind), l (large), p (processor specific)Notice the section header table, and how it starts after program text and program data, and how all the other ELF gunk (debug info, symtab, strtab(s)) also goes after .data. So (mostly) the real problematic readahead would be on the RW VMA that covers .data.
(This also matches my understanding of linkers, where they generally do (to put it simply) ELF headers - program headers - .text - .data - .bss, with stripable gunk after it.)
Quoted 8 lines
It's also the case that synchronous RA on
VM_EXECis already pretty conservative and limited, see the big if (vm_flags&VM_EXEC) indo_sync_mmap_readahead(). (I think the underlying logic behind also implies that async RA will not be started against these pages, but I am not sure).--
PedroYes, I think readahead of
VM_EXECis already restricted to the VMA. Maybe there is an edge case where someone does buffered reads on an ELF file, leaving aPG_readaheadflag inside theVM_EXECrange, then it could trigger async readahead beyond the end, but that sounds minor.
Quoted 6 lines
For next steps: Suppose we show the mmap usage in this video encoder is significantly inefficient compared to buffered reads or a big mmap and that project accepts a contribution to move away from the small mmaps. Would we be comfortable attempting this again as is? Probably there would be a lag before all users of the encoder update and they may see this bad perf.
How could you be sure that there isn't some other proprietary (or even just another open-source) software out there that relies on similar things, but doesn't provide easy benchmarks that can easily catch this?
Signature · 3 lines
Cheers, David