Skip to contentSkip to conversation

mm: vmalloc: Actually use the in-place vrealloc region

PATCH Applied Kees Cook 5 participants 9 messages May 15, 2025 – May 20, 2025

Subsystems (3)
lore
Share & tools
lore

Kees Cook

author
#0 lore raw

cover mm: vmalloc: Actually use the in-place vrealloc region

Hi,

This fixes a performance regression[1] with vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.

Thanks!

-Kees

[1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/

Kees Cook (2):
mm: vmalloc: Actually use the in-place vrealloc region
mm: vmalloc: Only zero-init on vrealloc shrink

 mm/vmalloc.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH 1/2 mm: vmalloc: Actually use the in-place vrealloc region

The refactoring to not build a new vmalloc region only actually worked when shrinking. Actually return the resized area when it grows. Ugh.

---
Cc: Andrew Morton <[email protected]>
Cc: Uladzislau Rezki <[email protected]>
Cc: <[email protected]>

---
 mm/vmalloc.c | 1 +
 1 file changed, 1 insertion(+)
mm/vmalloc.c +1 -0
index 2d7511654831..74bd00fd734d 100644
@@ -4111,6 +4111,7 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 		if (want_init_on_alloc(flags))
 			memset((void *)p + old_size, 0, size - old_size);
 		vm->requested_size = size;
+		return (void *)p;
 	}
 
 	/* TODO: Grow the vm_area, i.e. allocate and map additional pages. */
Signature · 1 line
2.34.1
author
#2 lore raw

PATCH 2/2 mm: vmalloc: Only zero-init on vrealloc shrink

In reply to Kees Cook

The common case is to grow reallocations, and since init_on_alloc will have already zeroed the whole allocation, we only need to zero when shrinking the allocation.

Fixes a0309faf1cb0 ("mm: vmalloc: support more granular vrealloc() sizing")

---
Cc: Andrew Morton <[email protected]>
Cc: Uladzislau Rezki <[email protected]>
Cc: <[email protected]>

---
 mm/vmalloc.c | 12 +++++++-----
 1 file changed, 7 insertions(+), 5 deletions(-)
mm/vmalloc.c +7 -5
index 74bd00fd734d..00cf1b575c89 100644
@@ -4093,8 +4093,8 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 	 * would be a good heuristic for when to shrink the vm_area?
 	 */
 	if (size <= old_size) {
-		/* Zero out "freed" memory. */
-		if (want_init_on_free())
+		/* Zero out "freed" memory, potentially for future realloc. */
+		if (want_init_on_free() || want_init_on_alloc(flags))
 			memset((void *)p + size, 0, old_size - size);
 		vm->requested_size = size;
 		kasan_poison_vmalloc(p + size, old_size - size);
@@ -4107,9 +4107,11 @@ void *vrealloc_noprof(const void *p, size_t size, gfp_t flags)
 	if (size <= alloced_size) {
 		kasan_unpoison_vmalloc(p + old_size, size - old_size,
 				       KASAN_VMALLOC_PROT_NORMAL);
-		/* Zero out "alloced" memory. */
-		if (want_init_on_alloc(flags))
-			memset((void *)p + old_size, 0, size - old_size);
+		/*
+		 * No need to zero memory here, as unused memory will have
+		 * already been zeroed at initial allocation time or during
+		 * realloc shrink time.
+		 */
 		vm->requested_size = size;
 		return (void *)p;
 	}
Signature · 1 line
2.34.1
Tested-by on the v1 series
#3 lore raw
In reply to Kees Cook
Quoted 8 lines · Kees Cook

Hi,

This fixes a performance regression[1] with vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.

Thanks!

Quoted 7 lines

-Kees

[1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/

Kees Cook (2):
mm: vmalloc: Actually use the in-place vrealloc region
mm: vmalloc: Only zero-init on vrealloc shrink

Thank you for the prompt fix! I'll remember to include a more thorough note on reproducing the issue next time.

With the patchset applied, BPF selftests on both 6.15-rc6 and 6.14.7-rc2 passes successfully.

maintainer Reviewed-by on the v1 series
#4 lore raw
In reply to Kees Cook
Quoted 10 lines · Kees Cook

Hi,

This fixes a performance regression[1] with vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.

Thanks!

-Kees

Quoted 9 lines

[1] https://lore.kernel.org/lkml/20250515-bpf-verifier-slowdown-vwo2meju4cgp2su5ckj@6gi6ssxbnfqg/

Kees Cook (2):
mm: vmalloc: Actually use the in-place vrealloc region
mm: vmalloc: Only zero-init on vrealloc shrink

 mm/vmalloc.c | 13 ++++++++-----
 1 file changed, 8 insertions(+), 5 deletions(-)

Looks good to me both.

--
Uladzislau Rezki

author
#6 lore raw
In reply to Kees Cook
Quoted 3 lines · Kees Cook

This fixes a performance regression[1] with vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.

Andrew, can you get these to Linus this week? I can also send them his way if you'd rather?

-Kees

Signature · 1 line
Kees Cook
maintainer
#7 lore raw
Quoted 4 lines · On Mon, 19 May 2025 12:18:42 -0700 Kees Cook <[email protected]> wrote:
Quoted 3 lines · Kees Cook

This fixes a performance regression[1] with vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.

No -stable backporting will be needed?

Quoted 1 line

Andrew, can you get these to Linus this week?

Sure.

author
#8 lore raw
Quoted 8 lines · Andrew Morton
Quoted 4 lines · On Mon, 19 May 2025 12:18:42 -0700 Kees Cook <[email protected]> wrote:
Quoted 3 lines · Kees Cook

This fixes a performance regression[1] with vrealloc(). This needs to get into v6.15, which is where the regression originates, and then it'll get backport to the -stable releases as well.

No -stable backporting will be needed?

I think it will, since the vrealloc patches were backported automatically, e.g. in v6.14.y: 0b391a520b4e ("mm: vmalloc: support more granular vrealloc() sizing")

Quoted 3 lines
Quoted 1 line

Andrew, can you get these to Linus this week?

Sure.

Thanks!

Signature · 1 line
Kees Cook