Skip to contentSkip to conversation

slub: fix data loss and overflow in krealloc()

PATCH In review Marco Elver 3 participants 5 messages Apr 16 – Apr 17

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 slub: fix data loss and overflow in krealloc()

Commit 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc") introduced the ability to force a reallocation if the original object does not satisfy new alignment or NUMA node, even when the object is being shrunk.

This introduced two bugs in the reallocation fallback path:

1. Data loss during NUMA migration: The jump to 'alloc_new' happens before 'ks' and 'orig_size' are initialized. As a result, the memcpy() in the 'alloc_new' block would copy 0 bytes into the new allocation.

2. Buffer overflow during shrinking: When shrinking an object while forcing a new alignment, 'new_size' is smaller than the old size. However, the memcpy() used the old size ('orig_size ?: ks'), leading to an out-of-bounds write.

The same overflow bug exists in the kvrealloc() fallback path, where the old bucket size ksize(p) is copied into the new buffer without being bounded by the new size.

A simple reproducer:

	// e.g. add to lkdtm as KREALLOC_SHRINK_OVERFLOW
	while (1) {
		void *p = kmalloc(128, GFP_KERNEL);
		p = krealloc_node_align(p, 64, 256, GFP_KERNEL, NUMA_NO_NODE);
		kfree(p);
	}

demonstrates the issue:

==================================================================

  BUG: KFENCE: out-of-bounds write in memcpy_orig+0x68/0x130

Out-of-bounds write at 0xffff8883ad757038 (120B right of kfence-#47):

   memcpy_orig+0x68/0x130
   krealloc_node_align_noprof+0x1c8/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...

kfence-#47: 0xffff8883ad756fc0-0xffff8883ad756fff, size=64, cache=kmalloc-64

allocated by task 316 on cpu 7 at 97.680481s (0.021813s ago):

   krealloc_node_align_noprof+0x19c/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...
  ==================================================================

Fix it by moving the old size calculation to the top of __do_krealloc() and bounding all copy lengths by the new allocation size.

Fixes 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc")
---
 mm/slub.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
mm/slub.c +12 -12
index 92362eeb13e5..161079ac5ba1 100644
@@ -6645,16 +6645,6 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
 	if (!kasan_check_byte(p))
 		return NULL;
 
-	/*
-	 * If reallocation is not necessary (e. g. the new size is less
-	 * than the current allocated size), the current allocation will be
-	 * preserved unless __GFP_THISNODE is set. In the latter case a new
-	 * allocation on the requested node will be attempted.
-	 */
-	if (unlikely(flags & __GFP_THISNODE) && nid != NUMA_NO_NODE &&
-		     nid != page_to_nid(virt_to_page(p)))
-		goto alloc_new;
-
 	if (is_kfence_address(p)) {
 		ks = orig_size = kfence_ksize(p);
 	} else {
@@ -6673,6 +6663,16 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
 		}
 	}
 
+	/*
+	 * If reallocation is not necessary (e. g. the new size is less
+	 * than the current allocated size), the current allocation will be
+	 * preserved unless __GFP_THISNODE is set. In the latter case a new
+	 * allocation on the requested node will be attempted.
+	 */
+	if (unlikely(flags & __GFP_THISNODE) && nid != NUMA_NO_NODE &&
+		     nid != page_to_nid(virt_to_page(p)))
+		goto alloc_new;
+
 	/* If the old object doesn't fit, allocate a bigger one */
 	if (new_size > ks)
 		goto alloc_new;
@@ -6707,7 +6707,7 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
 	if (ret && p) {
 		/* Disable KASAN checks as the object's redzone is accessed. */
 		kasan_disable_current();
-		memcpy(ret, kasan_reset_tag(p), orig_size ?: ks);
+		memcpy(ret, kasan_reset_tag(p), min(new_size, (size_t)(orig_size ?: ks)));
 		kasan_enable_current();
 	}
 
@@ -6941,7 +6941,7 @@ void *kvrealloc_node_align_noprof(const void *p, size_t size, unsigned long alig
 		if (p) {
 			/* We already know that `p` is not a vmalloc address. */
 			kasan_disable_current();
-			memcpy(n, kasan_reset_tag(p), ksize(p));
+			memcpy(n, kasan_reset_tag(p), min(size, ksize(p)));
 			kasan_enable_current();
 
 			kfree(p);
Signature · 1 line
2.54.0.rc1.513.gad8abe7a5a-goog
#1 lore raw

[+Cc relevant folks]

Quoted 11 lines · Marco Elver

Commit 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc") introduced the ability to force a reallocation if the original object does not satisfy new alignment or NUMA node, even when the object is being shrunk.

This introduced two bugs in the reallocation fallback path:

1. Data loss during NUMA migration: The jump to 'alloc_new' happens before 'ks' and 'orig_size' are initialized. As a result, the memcpy() in the 'alloc_new' block would copy 0 bytes into the new allocation.

Ouch.

Quoted 4 lines

2. Buffer overflow during shrinking: When shrinking an object while forcing a new alignment, 'new_size' is smaller than the old size. However, the memcpy() used the old size ('orig_size ?: ks'), leading to an out-of-bounds write.

Right. before the commit we didn't reallocate when the size is smaller.

Quoted 34 lines

The same overflow bug exists in the kvrealloc() fallback path, where the old bucket size ksize(p) is copied into the new buffer without being bounded by the new size.

A simple reproducer:

	// e.g. add to lkdtm as KREALLOC_SHRINK_OVERFLOW
	while (1) {
		void *p = kmalloc(128, GFP_KERNEL);
		p = krealloc_node_align(p, 64, 256, GFP_KERNEL, NUMA_NO_NODE);
		kfree(p);
	}

demonstrates the issue:

==================================================================

  BUG: KFENCE: out-of-bounds write in memcpy_orig+0x68/0x130

Out-of-bounds write at 0xffff8883ad757038 (120B right of kfence-#47):

   memcpy_orig+0x68/0x130
   krealloc_node_align_noprof+0x1c8/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...

kfence-#47: 0xffff8883ad756fc0-0xffff8883ad756fff, size=64, cache=kmalloc-64

allocated by task 316 on cpu 7 at 97.680481s (0.021813s ago):

   krealloc_node_align_noprof+0x19c/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...
  ==================================================================
Quoted 8 lines

Fix it by moving the old size calculation to the top of __do_krealloc() and bounding all copy lengths by the new allocation size.

Fixes 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc")

---

Looks good to me, but I think we still have a similar issue in vrealloc_node_align_noprof()? (goto need_realloc; due to NUMA mismatch but the new size is smaller)

Signature · 2 lines
Cheers,
Harry / Hyeonggon
#2 lore raw
Quoted 68 lines · Harry Yoo (Oracle)

[+Cc relevant folks]

Quoted 11 lines · Marco Elver

Commit 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc") introduced the ability to force a reallocation if the original object does not satisfy new alignment or NUMA node, even when the object is being shrunk.

This introduced two bugs in the reallocation fallback path:

1. Data loss during NUMA migration: The jump to 'alloc_new' happens before 'ks' and 'orig_size' are initialized. As a result, the memcpy() in the 'alloc_new' block would copy 0 bytes into the new allocation.

Ouch.

Quoted 4 lines

2. Buffer overflow during shrinking: When shrinking an object while forcing a new alignment, 'new_size' is smaller than the old size. However, the memcpy() used the old size ('orig_size ?: ks'), leading to an out-of-bounds write.

Right. before the commit we didn't reallocate when the size is smaller.

Quoted 34 lines

The same overflow bug exists in the kvrealloc() fallback path, where the old bucket size ksize(p) is copied into the new buffer without being bounded by the new size.

A simple reproducer:

      // e.g. add to lkdtm as KREALLOC_SHRINK_OVERFLOW
      while (1) {
              void *p = kmalloc(128, GFP_KERNEL);
              p = krealloc_node_align(p, 64, 256, GFP_KERNEL, NUMA_NO_NODE);
              kfree(p);
      }

demonstrates the issue:

==================================================================

  BUG: KFENCE: out-of-bounds write in memcpy_orig+0x68/0x130

Out-of-bounds write at 0xffff8883ad757038 (120B right of kfence-#47):

   memcpy_orig+0x68/0x130
   krealloc_node_align_noprof+0x1c8/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...

kfence-#47: 0xffff8883ad756fc0-0xffff8883ad756fff, size=64, cache=kmalloc-64

allocated by task 316 on cpu 7 at 97.680481s (0.021813s ago):

   krealloc_node_align_noprof+0x19c/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...
  ==================================================================
Quoted 8 lines

Fix it by moving the old size calculation to the top of __do_krealloc() and bounding all copy lengths by the new allocation size.

Fixes 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc")

---

Quoted 3 lines

Looks good to me, but I think we still have a similar issue in vrealloc_node_align_noprof()? (goto need_realloc; due to NUMA mismatch but the new size is smaller)

Good find.
That's a separate patch, though, since it's in the vmalloc subsystem (it's also not confidence-inspiring that vrealloc_node_align_noprof has a bunch of TODOs sprinkled all over...).
Since you found that, do you want to claim it?

Also by the looks of it slub and vmalloc patches go through different trees these days per MAINTAINERS.

Thanks,
-- Marco

maintainer Reviewed-by
#3 lore raw
Quoted 68 lines · Marco Elver
Quoted 67 lines · Harry Yoo (Oracle)

[+Cc relevant folks]

Quoted 11 lines · Marco Elver

Commit 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc") introduced the ability to force a reallocation if the original object does not satisfy new alignment or NUMA node, even when the object is being shrunk.

This introduced two bugs in the reallocation fallback path:

1. Data loss during NUMA migration: The jump to 'alloc_new' happens before 'ks' and 'orig_size' are initialized. As a result, the memcpy() in the 'alloc_new' block would copy 0 bytes into the new allocation.

Ouch.

Quoted 4 lines

2. Buffer overflow during shrinking: When shrinking an object while forcing a new alignment, 'new_size' is smaller than the old size. However, the memcpy() used the old size ('orig_size ?: ks'), leading to an out-of-bounds write.

Right. before the commit we didn't reallocate when the size is smaller.

Quoted 42 lines

The same overflow bug exists in the kvrealloc() fallback path, where the old bucket size ksize(p) is copied into the new buffer without being bounded by the new size.

A simple reproducer:

      // e.g. add to lkdtm as KREALLOC_SHRINK_OVERFLOW
      while (1) {
              void *p = kmalloc(128, GFP_KERNEL);
              p = krealloc_node_align(p, 64, 256, GFP_KERNEL, NUMA_NO_NODE);
              kfree(p);
      }

demonstrates the issue:

==================================================================

  BUG: KFENCE: out-of-bounds write in memcpy_orig+0x68/0x130

Out-of-bounds write at 0xffff8883ad757038 (120B right of kfence-#47):

   memcpy_orig+0x68/0x130
   krealloc_node_align_noprof+0x1c8/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...

kfence-#47: 0xffff8883ad756fc0-0xffff8883ad756fff, size=64, cache=kmalloc-64

allocated by task 316 on cpu 7 at 97.680481s (0.021813s ago):

   krealloc_node_align_noprof+0x19c/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...
  ==================================================================

Fix it by moving the old size calculation to the top of __do_krealloc() and bounding all copy lengths by the new allocation size.

Fixes 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc")

---

Quoted 7 lines
Quoted 3 lines

Looks good to me, but I think we still have a similar issue in vrealloc_node_align_noprof()? (goto need_realloc; due to NUMA mismatch but the new size is smaller)

Good find.

That's a separate patch, though, since it's in the vmalloc subsystem

You're right. for this patch:

Looks good to me,

Quoted 2 lines

(it's also not confidence-inspiring that vrealloc_node_align_noprof has a bunch of TODOs sprinkled all over...).

;)

looks like one of the TODOs is going to be tacked though. (shrinking)
https://lore.kernel.org/linux-mm/[email protected]

Quoted 1 line

Since you found that, do you want to claim it?

I have many stuffs going on my plate now (including re-reviewing the typed kmalloc caches patch) so it'd be nice if somebody could claim :)

Quoted 2 lines

Also by the looks of it slub and vmalloc patches go through different trees these days per MAINTAINERS.

Right. slab patches go through the slab tree.

Signature · 2 lines
Cheers,
Harry / Hyeonggon
#4 lore raw
In reply to Marco Elver
Quoted 51 lines · Marco Elver

Commit 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc") introduced the ability to force a reallocation if the original object does not satisfy new alignment or NUMA node, even when the object is being shrunk.

This introduced two bugs in the reallocation fallback path:

1. Data loss during NUMA migration: The jump to 'alloc_new' happens before 'ks' and 'orig_size' are initialized. As a result, the memcpy() in the 'alloc_new' block would copy 0 bytes into the new allocation.

2. Buffer overflow during shrinking: When shrinking an object while forcing a new alignment, 'new_size' is smaller than the old size. However, the memcpy() used the old size ('orig_size ?: ks'), leading to an out-of-bounds write.

The same overflow bug exists in the kvrealloc() fallback path, where the old bucket size ksize(p) is copied into the new buffer without being bounded by the new size.

A simple reproducer:

	// e.g. add to lkdtm as KREALLOC_SHRINK_OVERFLOW
	while (1) {
		void *p = kmalloc(128, GFP_KERNEL);
		p = krealloc_node_align(p, 64, 256, GFP_KERNEL, NUMA_NO_NODE);
		kfree(p);
	}

demonstrates the issue:

==================================================================

  BUG: KFENCE: out-of-bounds write in memcpy_orig+0x68/0x130

Out-of-bounds write at 0xffff8883ad757038 (120B right of kfence-#47):

   memcpy_orig+0x68/0x130
   krealloc_node_align_noprof+0x1c8/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...

kfence-#47: 0xffff8883ad756fc0-0xffff8883ad756fff, size=64, cache=kmalloc-64

allocated by task 316 on cpu 7 at 97.680481s (0.021813s ago):

   krealloc_node_align_noprof+0x19c/0x340
   lkdtm_KREALLOC_SHRINK_OVERFLOW+0x8c/0xc0 [lkdtm]
   lkdtm_do_action+0x3a/0x60 [lkdtm]
   ...
  ==================================================================
Quoted 7 lines

Fix it by moving the old size calculation to the top of __do_krealloc() and bounding all copy lengths by the new allocation size.

Fixes 2cd8231796b5 ("mm/slub: allow to set node and align in k[v]realloc")

Ouch, thanks. Added to slab/for-next-fixes
Indeed the vrealloc would be separate patch with different Fixes: commit and handled in the mm tree.

Quoted 60 lines
---
 mm/slub.c | 24 ++++++++++++------------
 1 file changed, 12 insertions(+), 12 deletions(-)
mm/slub.c +12 -12
index 92362eeb13e5..161079ac5ba1 100644
@@ -6645,16 +6645,6 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
 	if (!kasan_check_byte(p))
 		return NULL;
 
-	/*
-	 * If reallocation is not necessary (e. g. the new size is less
-	 * than the current allocated size), the current allocation will be
-	 * preserved unless __GFP_THISNODE is set. In the latter case a new
-	 * allocation on the requested node will be attempted.
-	 */
-	if (unlikely(flags & __GFP_THISNODE) && nid != NUMA_NO_NODE &&
-		     nid != page_to_nid(virt_to_page(p)))
-		goto alloc_new;
-
 	if (is_kfence_address(p)) {
 		ks = orig_size = kfence_ksize(p);
 	} else {
@@ -6673,6 +6663,16 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
 		}
 	}
 
+	/*
+	 * If reallocation is not necessary (e. g. the new size is less
+	 * than the current allocated size), the current allocation will be
+	 * preserved unless __GFP_THISNODE is set. In the latter case a new
+	 * allocation on the requested node will be attempted.
+	 */
+	if (unlikely(flags & __GFP_THISNODE) && nid != NUMA_NO_NODE &&
+		     nid != page_to_nid(virt_to_page(p)))
+		goto alloc_new;
+
 	/* If the old object doesn't fit, allocate a bigger one */
 	if (new_size > ks)
 		goto alloc_new;
@@ -6707,7 +6707,7 @@ __do_krealloc(const void *p, size_t new_size, unsigned long align, gfp_t flags,
 	if (ret && p) {
 		/* Disable KASAN checks as the object's redzone is accessed. */
 		kasan_disable_current();
-		memcpy(ret, kasan_reset_tag(p), orig_size ?: ks);
+		memcpy(ret, kasan_reset_tag(p), min(new_size, (size_t)(orig_size ?: ks)));
 		kasan_enable_current();
 	}
 
@@ -6941,7 +6941,7 @@ void *kvrealloc_node_align_noprof(const void *p, size_t size, unsigned long alig
 		if (p) {
 			/* We already know that `p` is not a vmalloc address. */
 			kasan_disable_current();
-			memcpy(n, kasan_reset_tag(p), ksize(p));
+			memcpy(n, kasan_reset_tag(p), min(size, ksize(p)));
 			kasan_enable_current();
 
 			kfree(p);