mm: vmalloc: Actually use the in-place vrealloc region
Subsystems (3)
cover mm: vmalloc: Actually use the in-place vrealloc region
Hi,
This fixes a performance regression[1] with vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.
Thanks!
-Kees
[1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/
Kees Cook (2):
mm: vmalloc: Actually use the in-place vrealloc region
mm: vmalloc: Only zero-init on vrealloc shrink
mm/vmalloc.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
Signature · 1 line
2.34.1
PATCH 1/2 mm: vmalloc: Actually use the in-place vrealloc region
The refactoring to not build a new vmalloc region only actually worked when shrinking. Actually return the resized area when it grows. Ugh.
---
Cc: Andrew Morton <[email protected]>
Cc: Uladzislau Rezki <[email protected]>
Cc: <[email protected]>
--- mm/vmalloc.c | 1 + 1 file changed, 1 insertion(+)
mm/vmalloc.c +1 -0
@@ -4111,6 +4111,7 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)if (want_init_on_alloc (flags))memset ((void *)p + old_size,0 , size - old_size); vm->requested_size = size; +return (void *)p; }/* TODO: Grow the vm_area, i.e. allocate and map additional pages. */
Signature · 1 line
2.34.1
PATCH 2/2 mm: vmalloc: Only zero-init on vrealloc shrink
In reply to Kees CookThe common case is to grow reallocations, and since init_on_alloc will have already zeroed the whole allocation, we only need to zero when shrinking the allocation.
---
Cc: Andrew Morton <[email protected]>
Cc: Uladzislau Rezki <[email protected]>
Cc: <[email protected]>
--- mm/vmalloc.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
mm/vmalloc.c +7 -5
@@ -4093,8 +4093,8 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)* would be a good heuristic for when to shrink the vm_area? */ if (size <= old_size) { -/* Zero out "freed" memory. */ -if (want_init_on_free ()) +/* Zero out "freed" memory, potentially for future realloc. */ +if (want_init_on_free () ||want_init_on_alloc (flags))memset ((void *)p + size,0 , old_size - size); vm->requested_size = size;kasan_poison_vmalloc (p + size, old_size - size);
@@ -4107,9 +4107,11 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)if (size <= alloced_size) {kasan_unpoison_vmalloc (p + old_size, size - old_size,KASAN_VMALLOC_PROT_NORMAL ); -/* Zero out "alloced" memory. */ -if (want_init_on_alloc (flags)) -memset ((void *)p + old_size,0 , size - old_size); +/* +* No need to zero memory here, as unused memory will have +* already been zeroed at initial allocation time or during +* realloc shrink time. +*/ vm->requested_size = size;return (void *)p; }
Signature · 1 line
2.34.1
Quoted 8 lines · Kees Cook
Hi,
This fixes a performance regression[1] with
vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.Thanks!
Quoted 7 lines
-Kees
[1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/
Kees Cook (2):
mm: vmalloc: Actually use the in-place vrealloc region
mm: vmalloc: Only zero-init on vrealloc shrink
Thank you for the prompt fix! I'll remember to include a more thorough note on reproducing the issue next time.
With the patchset applied, BPF selftests on both 6.15-rc6 and 6.14.7-rc2 passes successfully.
Quoted 10 lines · Kees Cook
Hi,
This fixes a performance regression[1] with
vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.Thanks!
-Kees
Quoted 9 lines
[1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/
Kees Cook (2):
mm: vmalloc: Actually use the in-place vrealloc region
mm: vmalloc: Only zero-init on vrealloc shrinkmm/vmalloc.c | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
Looks good to me both.
--
Uladzislau Rezki
Quoted 5 lines · Kees Cook
Hi,
This fixes a performance regression[1] with
vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.
Quoted 3 lines · Kees Cook
This fixes a performance regression[1] with
vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.
Andrew, can you get these to Linus this week? I can also send them his way if you'd rather?
-Kees
Signature · 1 line
Kees Cook
Andrew Morton
maintainerQuoted 4 lines · On Mon, 19 May 2025 12:18:42 -0700 Kees Cook <[email protected]> wrote:
Quoted 3 lines · Kees Cook
This fixes a performance regression[1] with
vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.
No -stable backporting will be needed?
Quoted 1 line
Andrew, can you get these to Linus this week?
Sure.
Quoted 8 lines · Andrew Morton
Quoted 4 lines · On Mon, 19 May 2025 12:18:42 -0700 Kees Cook <[email protected]> wrote:
Quoted 3 lines · Kees Cook
This fixes a performance regression[1] with
vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.No -stable backporting will be needed?
I think it will, since the vrealloc patches were backported automatically, e.g. in v6.14.y: 0b391a520b4e ("mm: vmalloc: support more granular vrealloc() sizing")
Quoted 3 lines
Quoted 1 line
Andrew, can you get these to Linus this week?
Sure.
Thanks!
Signature · 1 line
Kees Cook